2024-05-21T15:13:34.762 INFO:root:teuthology version: 0.0.1.dev287+gb9e3da8 2024-05-21T15:13:34.763 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589 --owner scheduled_pdonnell@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} --verbose -- /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589/orig.config.yaml 2024-05-21T15:13:34.816 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-21T15:13:34.909 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589 branch: wip-pdonnell-testing-20240517.203521-squid-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} email: pdonnell@redhat.com first_in_suite: false job_id: '7718589' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240517.203521-squid-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240517.203521-squid-debug sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9749 sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d sleep_before_teardown: 0 subset: 272/512 suite: fs suite_branch: wip-pdonnell-testing-20240517.203521-squid-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOhNZTqVwp9QmXsoa4nAR+2NaUzKRbiUs/b5twV1a1Hmv+j/14JAMeVyskID0bEYkvLHRlacZxetQIvZ5pVbStU= smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG7EZNey4KK4kwIvecJGdUP3N3nBW11K0551N+dnJjIMxfpgHmNT1xygv8zW8wtvRM5nPyJwkTedb/W1kocGG0U= smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLuvWL6QyS0Qzv8z2qIlPQfoHorsKMd/e6u4VIrwDLZ90mvs4NXwM7sQ1JkSxanBTXP/6qDd99srfQVrD61Csm4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a timestamp: 2024-05-21_13:18:14 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-21T15:13:34.909 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa; will attempt to use it 2024-05-21T15:13:34.910 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa/tasks 2024-05-21T15:13:34.910 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-21T15:13:34.912 INFO:teuthology.task.internal:Checking packages... 2024-05-21T15:13:34.938 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2b507c724ef6c8295ee0a35d185cdab167fd61d' 2024-05-21T15:13:34.938 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-21T15:13:34.938 INFO:teuthology.packaging:ref: None 2024-05-21T15:13:34.938 INFO:teuthology.packaging:tag: None 2024-05-21T15:13:34.938 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240517.203521-squid-debug 2024-05-21T15:13:34.938 INFO:teuthology.packaging:sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:13:34.938 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240517.203521-squid-debug 2024-05-21T15:13:35.075 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2744.gd2b507c7 2024-05-21T15:13:35.076 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-21T15:13:35.083 INFO:teuthology.task.internal:no buildpackages task found 2024-05-21T15:13:35.083 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-21T15:13:35.095 INFO:teuthology.task.internal:Saving configuration 2024-05-21T15:13:35.114 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-21T15:13:35.122 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-21T15:13:35.147 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 15:08:49.839837', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOhNZTqVwp9QmXsoa4nAR+2NaUzKRbiUs/b5twV1a1Hmv+j/14JAMeVyskID0bEYkvLHRlacZxetQIvZ5pVbStU='} 2024-05-21T15:13:35.167 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 15:08:49.838617', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG7EZNey4KK4kwIvecJGdUP3N3nBW11K0551N+dnJjIMxfpgHmNT1xygv8zW8wtvRM5nPyJwkTedb/W1kocGG0U='} 2024-05-21T15:13:35.188 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 15:08:49.837131', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLuvWL6QyS0Qzv8z2qIlPQfoHorsKMd/e6u4VIrwDLZ90mvs4NXwM7sQ1JkSxanBTXP/6qDd99srfQVrD61Csm4='} 2024-05-21T15:13:35.189 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-21T15:13:35.195 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-21T15:13:35.195 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-21T15:13:35.195 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-21T15:13:35.195 INFO:teuthology.run_tasks:Running task console_log... 2024-05-21T15:13:35.279 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6ace381000>, signals=[15]) 2024-05-21T15:13:35.279 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-21T15:13:35.285 INFO:teuthology.task.internal:Opening connections... 2024-05-21T15:13:35.285 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2024-05-21T15:13:35.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T15:13:35.364 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-05-21T15:13:35.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T15:13:35.440 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-05-21T15:13:35.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T15:13:35.518 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-21T15:13:35.526 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2024-05-21T15:13:35.545 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2024-05-21T15:13:35.545 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-05-21T15:13:35.603 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream" 2024-05-21T15:13:35.604 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9" 2024-05-21T15:13:35.604 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos" 2024-05-21T15:13:35.604 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora" 2024-05-21T15:13:35.604 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9" 2024-05-21T15:13:35.604 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9" 2024-05-21T15:13:35.604 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T15:13:35.604 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2024-05-21T15:13:35.604 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon" 2024-05-21T15:13:35.604 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T15:13:35.604 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/" 2024-05-21T15:13:35.604 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T15:13:35.605 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T15:13:35.605 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T15:13:35.605 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-05-21T15:13:35.629 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-05-21T15:13:35.648 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-05-21T15:13:35.648 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-05-21T15:13:35.706 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-05-21T15:13:35.706 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-05-21T15:13:35.706 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-05-21T15:13:35.706 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-05-21T15:13:35.706 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-05-21T15:13:35.706 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-05-21T15:13:35.706 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T15:13:35.706 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-05-21T15:13:35.707 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-05-21T15:13:35.707 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T15:13:35.707 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-05-21T15:13:35.707 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T15:13:35.707 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T15:13:35.707 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T15:13:35.707 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-05-21T15:13:35.737 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-05-21T15:13:35.756 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-05-21T15:13:35.756 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-05-21T15:13:35.814 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2024-05-21T15:13:35.814 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2024-05-21T15:13:35.814 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2024-05-21T15:13:35.814 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2024-05-21T15:13:35.814 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2024-05-21T15:13:35.814 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2024-05-21T15:13:35.814 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T15:13:35.814 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-05-21T15:13:35.814 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2024-05-21T15:13:35.814 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T15:13:35.814 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2024-05-21T15:13:35.815 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T15:13:35.815 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T15:13:35.815 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T15:13:35.815 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-05-21T15:13:35.837 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-21T15:13:35.848 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-21T15:13:35.854 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-21T15:13:35.855 DEBUG:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2024-05-21T15:13:35.857 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-05-21T15:13:35.859 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-05-21T15:13:35.875 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-21T15:13:35.883 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-21T15:13:35.883 DEBUG:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2024-05-21T15:13:35.915 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-05-21T15:13:35.917 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-05-21T15:13:35.952 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-21T15:13:36.026 INFO:teuthology.run_tasks:Running task kernel... 2024-05-21T15:13:36.041 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-21T15:13:36.041 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-21T15:13:36.041 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-21T15:13:36.041 DEBUG:teuthology.orchestra.run.smithi038:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T15:13:36.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:13:36.059 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-05-21T15:13:36.117 INFO:teuthology.orchestra.run.smithi038.stdout:5.14.0-437.el9.x86_64 2024-05-21T15:13:36.117 INFO:teuthology.task.kernel:Running kernel on smithi038: 5.14.0-437.el9.x86_64 2024-05-21T15:13:36.117 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum install -y kernel 2024-05-21T15:13:38.630 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.1 MB 00:00 2024-05-21T15:13:40.448 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-05-21T15:13:44.657 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-21T15:13:46.116 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 16 kB 00:00 2024-05-21T15:13:47.243 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-05-21T15:13:52.576 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-21T15:13:54.533 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-21T15:13:54.533 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-21T15:13:54.625 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-05-21T15:13:54.636 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-05-21T15:13:54.636 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-05-21T15:13:54.637 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-05-21T15:13:54.637 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-05-21T15:13:54.637 INFO:teuthology.orchestra.run.smithi038.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-21T15:13:54.637 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-05-21T15:13:54.637 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-21T15:13:54.637 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-21T15:13:54.637 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-21T15:13:54.637 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:13:54.637 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-05-21T15:13:54.637 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-05-21T15:13:54.637 INFO:teuthology.orchestra.run.smithi038.stdout:Install 4 Packages 2024-05-21T15:13:54.637 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:13:54.638 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 78 M 2024-05-21T15:13:54.638 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 123 M 2024-05-21T15:13:54.638 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-05-21T15:13:55.012 INFO:teuthology.orchestra.run.smithi038.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 5.5 MB/s | 576 kB 00:00 2024-05-21T15:13:55.529 INFO:teuthology.orchestra.run.smithi038.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-05-21T15:13:56.054 INFO:teuthology.orchestra.run.smithi038.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 30 MB/s | 34 MB 00:01 2024-05-21T15:13:56.413 INFO:teuthology.orchestra.run.smithi038.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 20 MB/s | 28 MB 00:01 2024-05-21T15:13:56.413 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-05-21T15:13:56.413 INFO:teuthology.orchestra.run.smithi038.stdout:Total 44 MB/s | 78 MB 00:01 2024-05-21T15:13:57.125 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-05-21T15:13:57.550 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-05-21T15:13:57.551 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-05-21T15:13:58.534 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-05-21T15:13:58.534 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-05-21T15:14:01.080 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-05-21T15:14:01.741 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-21T15:14:01.822 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T15:14:04.598 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T15:14:04.751 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T15:14:09.836 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T15:14:09.886 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T15:14:15.009 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T15:14:46.236 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T15:14:46.276 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-21T15:14:46.659 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T15:14:46.659 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-21T15:14:46.659 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T15:14:46.659 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T15:14:47.439 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T15:14:47.440 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:14:47.440 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-05-21T15:14:47.440 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-21T15:14:47.440 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-21T15:14:47.440 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-21T15:14:47.440 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-21T15:14:47.441 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:14:47.441 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-05-21T15:14:47.830 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T15:14:47.917 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T15:14:47.917 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T15:14:47.917 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-21T15:14:47.917 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-21T15:14:47.917 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-21T15:14:47.917 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-21T15:14:47.918 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi038.front.sepia.ceph.com, path=None, version=distro) 2024-05-21T15:14:47.918 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T15:14:47.970 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T15:14:47.970 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T15:14:47.971 DEBUG:teuthology.orchestra.run.smithi038:> sudo rpm -qi grub2-tools 2024-05-21T15:14:48.011 INFO:teuthology.orchestra.run.smithi038.stdout:Name : grub2-tools 2024-05-21T15:14:48.011 INFO:teuthology.orchestra.run.smithi038.stdout:Epoch : 1 2024-05-21T15:14:48.011 INFO:teuthology.orchestra.run.smithi038.stdout:Version : 2.06 2024-05-21T15:14:48.011 INFO:teuthology.orchestra.run.smithi038.stdout:Release : 68.el9 2024-05-21T15:14:48.012 INFO:teuthology.orchestra.run.smithi038.stdout:Architecture: x86_64 2024-05-21T15:14:48.012 INFO:teuthology.orchestra.run.smithi038.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-21T15:14:48.012 INFO:teuthology.orchestra.run.smithi038.stdout:Group : Unspecified 2024-05-21T15:14:48.012 INFO:teuthology.orchestra.run.smithi038.stdout:Size : 8264254 2024-05-21T15:14:48.012 INFO:teuthology.orchestra.run.smithi038.stdout:License : GPLv3+ 2024-05-21T15:14:48.012 INFO:teuthology.orchestra.run.smithi038.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-21T15:14:48.012 INFO:teuthology.orchestra.run.smithi038.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-21T15:14:48.012 INFO:teuthology.orchestra.run.smithi038.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-21T15:14:48.012 INFO:teuthology.orchestra.run.smithi038.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-21T15:14:48.012 INFO:teuthology.orchestra.run.smithi038.stdout:Packager : builder@centos.org 2024-05-21T15:14:48.012 INFO:teuthology.orchestra.run.smithi038.stdout:Vendor : CentOS 2024-05-21T15:14:48.012 INFO:teuthology.orchestra.run.smithi038.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-21T15:14:48.012 INFO:teuthology.orchestra.run.smithi038.stdout:Summary : Support tools for GRUB. 2024-05-21T15:14:48.013 INFO:teuthology.orchestra.run.smithi038.stdout:Description : 2024-05-21T15:14:48.013 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:14:48.013 INFO:teuthology.orchestra.run.smithi038.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-21T15:14:48.013 INFO:teuthology.orchestra.run.smithi038.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-21T15:14:48.013 INFO:teuthology.orchestra.run.smithi038.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-21T15:14:48.013 INFO:teuthology.orchestra.run.smithi038.stdout:hardware devices. 2024-05-21T15:14:48.013 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:14:48.013 INFO:teuthology.orchestra.run.smithi038.stdout:This subpackage provides tools for support of all platforms. 2024-05-21T15:14:48.014 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-21T15:14:48.014 INFO:teuthology.task.kernel:Updating grub on smithi038 to boot 5.14.0-447.el9.x86_64 2024-05-21T15:14:48.014 DEBUG:teuthology.orchestra.run.smithi038:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-21T15:14:48.644 INFO:teuthology.orchestra.run.smithi038.stderr:Generating grub configuration file ... 2024-05-21T15:14:50.133 INFO:teuthology.orchestra.run.smithi038.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-21T15:14:50.170 INFO:teuthology.orchestra.run.smithi038.stderr:done 2024-05-21T15:14:50.173 DEBUG:teuthology.orchestra.run.smithi038:> mktemp 2024-05-21T15:14:50.189 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.kpdXXRkndO 2024-05-21T15:14:50.189 DEBUG:teuthology.orchestra.run.smithi038:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kpdXXRkndO 2024-05-21T15:14:50.255 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.kpdXXRkndO 2024-05-21T15:14:50.398 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.kpdXXRkndO is 6KB 2024-05-21T15:14:50.449 DEBUG:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.kpdXXRkndO 2024-05-21T15:14:50.467 DEBUG:teuthology.orchestra.run.smithi038:> sudo /bin/ls /boot/loader/entries || true 2024-05-21T15:14:50.533 INFO:teuthology.orchestra.run.smithi038.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-21T15:14:50.533 INFO:teuthology.orchestra.run.smithi038.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-21T15:14:50.533 INFO:teuthology.orchestra.run.smithi038.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-21T15:14:50.533 INFO:teuthology.orchestra.run.smithi038.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-21T15:14:50.535 DEBUG:teuthology.orchestra.run.smithi038:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-21T15:14:50.676 DEBUG:teuthology.orchestra.run.smithi038:> sudo shutdown -r now 2024-05-21T15:15:20.681 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-21T15:15:20.682 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-05-21T15:15:20.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T15:15:55.295 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.38 2024-05-21T15:16:04.296 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-05-21T15:16:04.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T15:16:22.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.38 2024-05-21T15:16:34.687 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-05-21T15:16:34.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T15:16:35.107 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-05-21T15:16:35.518 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-05-21T15:16:35.518 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-447.el9.x86_64"... 2024-05-21T15:16:35.518 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-05-21T15:16:35.533 INFO:teuthology.orchestra.run.smithi038.stdout:5.14.0-447.el9.x86_64 2024-05-21T15:16:35.534 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-21T15:16:35.534 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-21T15:16:35.534 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-21T15:16:36.535 DEBUG:teuthology.parallel:result is None 2024-05-21T15:16:36.536 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-21T15:16:36.543 INFO:teuthology.task.internal:Creating test directory... 2024-05-21T15:16:36.543 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T15:16:36.545 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T15:16:36.548 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T15:16:36.566 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-21T15:16:36.573 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-21T15:16:36.584 INFO:teuthology.task.internal:Creating archive directory... 2024-05-21T15:16:36.584 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T15:16:36.606 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T15:16:36.609 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T15:16:36.645 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-21T15:16:36.654 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-21T15:16:36.654 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T15:16:36.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:16:36.688 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T15:16:36.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:16:36.705 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T15:16:36.708 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T15:16:36.731 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T15:16:36.764 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T15:16:36.783 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T15:16:36.786 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T15:16:36.792 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T15:16:36.802 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T15:16:36.821 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T15:16:36.822 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-21T15:16:36.830 INFO:teuthology.task.internal:Configuring sudo... 2024-05-21T15:16:36.830 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T15:16:36.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T15:16:36.834 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T15:16:36.872 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-21T15:16:36.881 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-21T15:16:36.882 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T15:16:36.904 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T15:16:36.906 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T15:16:36.930 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T15:16:37.005 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T15:16:37.049 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T15:16:37.131 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T15:16:37.173 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:16:37.174 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T15:16:37.239 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T15:16:37.286 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T15:16:37.338 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T15:16:37.423 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T15:16:37.464 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T15:16:37.465 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T15:16:37.531 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T15:16:37.576 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T15:16:37.628 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T15:16:37.713 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T15:16:37.758 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-21T15:16:37.758 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T15:16:37.827 DEBUG:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2024-05-21T15:16:37.829 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-05-21T15:16:37.832 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-05-21T15:16:37.874 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T15:16:37.875 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T15:16:37.916 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T15:16:38.305 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-21T15:16:38.314 INFO:teuthology.task.internal:Starting timer... 2024-05-21T15:16:38.314 INFO:teuthology.run_tasks:Running task pcp... 2024-05-21T15:16:38.328 INFO:teuthology.run_tasks:Running task selinux... 2024-05-21T15:16:38.337 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-21T15:16:38.337 DEBUG:teuthology.orchestra.run.smithi038:> sudo service auditd rotate 2024-05-21T15:16:38.483 INFO:teuthology.orchestra.run.smithi038.stdout:Rotating logs: 2024-05-21T15:16:38.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-05-21T15:16:38.555 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-05-21T15:16:38.557 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-05-21T15:16:38.617 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-05-21T15:16:38.619 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-21T15:16:38.619 DEBUG:teuthology.orchestra.run.smithi038:> /usr/sbin/getenforce 2024-05-21T15:16:38.641 INFO:teuthology.orchestra.run.smithi038.stdout:Permissive 2024-05-21T15:16:38.641 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-05-21T15:16:38.670 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-05-21T15:16:38.671 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-05-21T15:16:38.699 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-05-21T15:16:38.699 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi038.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi053.front.sepia.ceph.com': 'permissive'} 2024-05-21T15:16:38.699 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T15:16:38.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:16:38.727 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T15:16:38.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:16:38.755 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T15:16:38.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:16:38.782 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-21T15:16:38.782 DEBUG:teuthology.orchestra.run.smithi038:> sudo /usr/sbin/setenforce permissive 2024-05-21T15:16:38.826 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-05-21T15:16:38.852 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-05-21T15:16:38.877 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-21T15:16:38.887 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-21T15:16:38.891 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-21T15:16:39.017 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-21T15:16:39.028 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-21T15:16:39.029 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi038.front.sepia.ceph.com,smithi045.front.sepia.ceph.com,smithi053.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-21T15:21:30.457 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi038.front.sepia.ceph.com'), Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi053.front.sepia.ceph.com')] 2024-05-21T15:21:30.458 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-05-21T15:21:30.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T15:21:30.539 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-05-21T15:21:30.625 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-05-21T15:21:30.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-21T15:21:30.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T15:21:30.700 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-05-21T15:21:30.781 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-21T15:21:30.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-05-21T15:21:30.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T15:21:30.861 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-05-21T15:21:30.943 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-05-21T15:21:30.944 INFO:teuthology.run_tasks:Running task clock... 2024-05-21T15:21:30.958 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-21T15:21:30.958 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T15:21:30.959 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T15:21:30.961 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T15:21:30.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T15:21:30.964 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T15:21:30.964 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T15:21:30.990 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T15:21:30.991 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T15:21:31.005 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T15:21:31.007 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T15:21:31.016 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T15:21:31.030 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-05-21T15:21:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: ntpd: command not found 2024-05-21T15:21:31.032 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T15:21:31.041 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-05-21T15:21:31.043 INFO:teuthology.orchestra.run.smithi038.stdout:506 Cannot talk to daemon 2024-05-21T15:21:31.054 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T15:21:31.057 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-05-21T15:21:31.058 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T15:21:31.068 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T15:21:31.069 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-05-21T15:21:31.074 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T15:21:31.083 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T15:21:31.097 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T15:21:31.134 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-05-21T15:21:31.136 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-05-21T15:21:31.138 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T15:21:31.138 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-05-21T15:21:31.138 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:21:31.138 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:21:31.138 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:21:31.138 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:21:31.140 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T15:21:31.140 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-05-21T15:21:31.140 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:21:31.140 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:21:31.140 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:21:31.141 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:21:31.156 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-05-21T15:21:31.160 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T15:21:31.160 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-05-21T15:21:31.160 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:21:31.160 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:21:31.160 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:21:31.161 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:21:31.161 INFO:teuthology.run_tasks:Running task install... 2024-05-21T15:21:31.173 DEBUG:teuthology.task.install:project ceph 2024-05-21T15:21:31.173 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'}} 2024-05-21T15:21:31.173 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'} 2024-05-21T15:21:31.174 INFO:teuthology.task.install:Using flavor: default 2024-05-21T15:21:31.179 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-21T15:21:31.179 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-21T15:21:31.179 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'tag': None, 'wait_for_package': False} 2024-05-21T15:21:31.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:21:31.182 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'tag': None, 'wait_for_package': False} 2024-05-21T15:21:31.182 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:21:31.183 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'tag': None, 'wait_for_package': False} 2024-05-21T15:21:31.183 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:21:31.333 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/ 2024-05-21T15:21:31.333 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2744.gd2b507c7 2024-05-21T15:21:31.408 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/ 2024-05-21T15:21:31.408 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2744.gd2b507c7 2024-05-21T15:21:31.411 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/ 2024-05-21T15:21:31.411 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2744.gd2b507c7 2024-05-21T15:21:31.514 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T15:21:31.515 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-21T15:21:31.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T15:21:31.542 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-21T15:21:31.542 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2b507c724ef6c8295ee0a35d185cdab167fd61d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T15:21:31.592 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T15:21:31.593 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:21:31.593 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T15:21:31.595 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T15:21:31.595 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T15:21:31.595 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T15:21:31.609 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T15:21:31.622 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-21T15:21:31.622 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2b507c724ef6c8295ee0a35d185cdab167fd61d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T15:21:31.623 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-21T15:21:31.623 DEBUG:teuthology.orchestra.run.smithi038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2b507c724ef6c8295ee0a35d185cdab167fd61d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T15:21:31.685 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T15:21:31.688 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T15:21:31.691 DEBUG:teuthology.orchestra.run.smithi038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T15:21:31.750 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-05-21T15:21:31.751 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-05-21T15:21:31.764 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T15:21:31.769 DEBUG:teuthology.orchestra.run.smithi038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T15:21:31.829 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-05-21T15:21:31.831 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-05-21T15:21:31.833 INFO:teuthology.orchestra.run.smithi038.stdout:check_obsoletes = 1 2024-05-21T15:21:31.835 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum clean all 2024-05-21T15:21:31.987 INFO:teuthology.orchestra.run.smithi053.stdout:45 files removed 2024-05-21T15:21:32.014 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-21T15:21:32.065 INFO:teuthology.orchestra.run.smithi045.stdout:45 files removed 2024-05-21T15:21:32.074 INFO:teuthology.orchestra.run.smithi038.stdout:45 files removed 2024-05-21T15:21:32.093 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-21T15:21:32.100 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-21T15:21:32.683 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 263 kB/s | 83 kB 00:00 2024-05-21T15:21:32.698 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 203 kB/s | 83 kB 00:00 2024-05-21T15:21:32.813 INFO:teuthology.orchestra.run.smithi038.stdout:ceph packages for x86_64 197 kB/s | 83 kB 00:00 2024-05-21T15:21:33.242 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 22 kB/s | 11 kB 00:00 2024-05-21T15:21:33.257 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 22 kB/s | 11 kB 00:00 2024-05-21T15:21:33.405 INFO:teuthology.orchestra.run.smithi038.stdout:ceph noarch packages 21 kB/s | 11 kB 00:00 2024-05-21T15:21:33.618 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-05-21T15:21:33.674 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-05-21T15:21:33.889 INFO:teuthology.orchestra.run.smithi038.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-05-21T15:21:34.134 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.1 MB 00:00 2024-05-21T15:21:34.274 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-05-21T15:21:35.231 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.1 MB 00:01 2024-05-21T15:21:36.966 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-05-21T15:21:37.752 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 19 MB 00:02 2024-05-21T15:21:37.942 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 19 MB 00:02 2024-05-21T15:21:42.101 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-21T15:21:42.567 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.2 MB 00:01 2024-05-21T15:21:42.961 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.2 MB 00:01 2024-05-21T15:21:43.635 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 16 kB 00:00 2024-05-21T15:21:43.976 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 87 kB/s | 16 kB 00:00 2024-05-21T15:21:44.345 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 82 kB/s | 16 kB 00:00 2024-05-21T15:21:46.029 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 9.3 MB/s | 21 MB 00:02 2024-05-21T15:21:46.190 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-21T15:21:46.289 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 21 MB 00:02 2024-05-21T15:21:51.445 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-21T15:21:51.446 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-05-21T15:21:51.653 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-05-21T15:21:53.657 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T15:21:53.658 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T15:21:53.664 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T15:21:53.664 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T15:21:53.668 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-21T15:21:53.669 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-21T15:21:53.670 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-21T15:21:53.671 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-21T15:21:53.672 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-21T15:21:53.675 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-21T15:21:53.676 INFO:teuthology.orchestra.run.smithi053.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-21T15:21:53.676 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-21T15:21:53.676 INFO:teuthology.orchestra.run.smithi053.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-21T15:21:53.677 INFO:teuthology.orchestra.run.smithi053.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-21T15:21:53.678 INFO:teuthology.orchestra.run.smithi053.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-21T15:21:53.680 INFO:teuthology.orchestra.run.smithi053.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-21T15:21:53.682 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-21T15:21:53.683 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-21T15:21:53.743 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T15:21:53.749 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-05-21T15:21:53.762 INFO:teuthology.orchestra.run.smithi053.stdout:======================================================================================= 2024-05-21T15:21:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-05-21T15:21:53.763 INFO:teuthology.orchestra.run.smithi053.stdout:======================================================================================= 2024-05-21T15:21:53.763 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-05-21T15:21:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-21T15:21:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 6.6 k 2024-05-21T15:21:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.4 M 2024-05-21T15:21:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 863 k 2024-05-21T15:21:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 147 k 2024-05-21T15:21:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 1.5 M 2024-05-21T15:21:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 142 k 2024-05-21T15:21:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 3.5 M 2024-05-21T15:21:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 7.4 M 2024-05-21T15:21:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 49 k 2024-05-21T15:21:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 9.8 M 2024-05-21T15:21:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 46 M 2024-05-21T15:21:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 264 k 2024-05-21T15:21:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 763 k 2024-05-21T15:21:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 221 k 2024-05-21T15:21:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 27 k 2024-05-21T15:21:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 18 k 2024-05-21T15:21:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-21T15:21:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-21T15:21:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-21T15:21:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 32 k 2024-05-21T15:21:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 756 k 2024-05-21T15:21:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 127 k 2024-05-21T15:21:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-21T15:21:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-21T15:21:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-21T15:21:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-21T15:21:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-21T15:21:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-21T15:21:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 161 k 2024-05-21T15:21:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 320 k 2024-05-21T15:21:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 299 k 2024-05-21T15:21:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 99 k 2024-05-21T15:21:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 86 k 2024-05-21T15:21:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.0 M 2024-05-21T15:21:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 169 k 2024-05-21T15:21:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-21T15:21:53.766 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-05-21T15:21:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-21T15:21:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-21T15:21:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-21T15:21:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-21T15:21:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-21T15:21:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-21T15:21:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-21T15:21:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.3 M 2024-05-21T15:21:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.1 M 2024-05-21T15:21:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-21T15:21:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-21T15:21:53.767 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-05-21T15:21:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T15:21:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-21T15:21:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T15:21:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 19 M 2024-05-21T15:21:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 32 k 2024-05-21T15:21:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 2.3 M 2024-05-21T15:21:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 249 k 2024-05-21T15:21:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 4.9 M 2024-05-21T15:21:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 18 M 2024-05-21T15:21:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 16 k 2024-05-21T15:21:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 25 k 2024-05-21T15:21:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T15:21:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T15:21:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T15:21:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T15:21:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T15:21:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T15:21:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T15:21:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T15:21:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 158 k 2024-05-21T15:21:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T15:21:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T15:21:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T15:21:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T15:21:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T15:21:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 497 k 2024-05-21T15:21:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T15:21:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.0 M 2024-05-21T15:21:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T15:21:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-21T15:21:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T15:21:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T15:21:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T15:21:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T15:21:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T15:21:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T15:21:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T15:21:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-21T15:21:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T15:21:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T15:21:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T15:21:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-21T15:21:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T15:21:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T15:21:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T15:21:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 45 k 2024-05-21T15:21:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 135 k 2024-05-21T15:21:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T15:21:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T15:21:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T15:21:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T15:21:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-21T15:21:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-21T15:21:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T15:21:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T15:21:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T15:21:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T15:21:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T15:21:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T15:21:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T15:21:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-21T15:21:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T15:21:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T15:21:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T15:21:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T15:21:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T15:21:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T15:21:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T15:21:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T15:21:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T15:21:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T15:21:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T15:21:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T15:21:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T15:21:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T15:21:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T15:21:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T15:21:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T15:21:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T15:21:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T15:21:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T15:21:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T15:21:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T15:21:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T15:21:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T15:21:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-21T15:21:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T15:21:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T15:21:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T15:21:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T15:21:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T15:21:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T15:21:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T15:21:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T15:21:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T15:21:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T15:21:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T15:21:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-21T15:21:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T15:21:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T15:21:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T15:21:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T15:21:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T15:21:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T15:21:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T15:21:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T15:21:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T15:21:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-21T15:21:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T15:21:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-21T15:21:53.776 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2024-05-21T15:21:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T15:21:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:21:53.776 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-05-21T15:21:53.776 INFO:teuthology.orchestra.run.smithi053.stdout:======================================================================================= 2024-05-21T15:21:53.776 INFO:teuthology.orchestra.run.smithi053.stdout:Install 148 Packages 2024-05-21T15:21:53.776 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 11 Packages 2024-05-21T15:21:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:21:53.777 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 222 M 2024-05-21T15:21:53.777 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-05-21T15:21:53.778 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-05-21T15:21:53.778 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-05-21T15:21:53.778 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-05-21T15:21:53.778 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T15:21:53.778 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-21T15:21:53.778 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 6.6 k 2024-05-21T15:21:53.778 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.4 M 2024-05-21T15:21:53.778 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 863 k 2024-05-21T15:21:53.778 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 147 k 2024-05-21T15:21:53.778 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 1.5 M 2024-05-21T15:21:53.778 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 142 k 2024-05-21T15:21:53.778 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 3.5 M 2024-05-21T15:21:53.779 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 7.4 M 2024-05-21T15:21:53.779 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 49 k 2024-05-21T15:21:53.779 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 9.8 M 2024-05-21T15:21:53.779 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 46 M 2024-05-21T15:21:53.779 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 264 k 2024-05-21T15:21:53.779 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 763 k 2024-05-21T15:21:53.779 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 221 k 2024-05-21T15:21:53.779 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 27 k 2024-05-21T15:21:53.779 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 18 k 2024-05-21T15:21:53.779 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-21T15:21:53.779 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-21T15:21:53.779 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-21T15:21:53.779 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 32 k 2024-05-21T15:21:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 756 k 2024-05-21T15:21:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 127 k 2024-05-21T15:21:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-21T15:21:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-21T15:21:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-21T15:21:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-21T15:21:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-21T15:21:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-21T15:21:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 161 k 2024-05-21T15:21:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 320 k 2024-05-21T15:21:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 299 k 2024-05-21T15:21:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 99 k 2024-05-21T15:21:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 86 k 2024-05-21T15:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.0 M 2024-05-21T15:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 169 k 2024-05-21T15:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-21T15:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-05-21T15:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-21T15:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-21T15:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-21T15:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-21T15:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-21T15:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-21T15:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-21T15:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.3 M 2024-05-21T15:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.1 M 2024-05-21T15:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-21T15:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-21T15:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-05-21T15:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T15:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-21T15:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T15:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 19 M 2024-05-21T15:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 32 k 2024-05-21T15:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 2.3 M 2024-05-21T15:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 249 k 2024-05-21T15:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 4.9 M 2024-05-21T15:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 18 M 2024-05-21T15:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 16 k 2024-05-21T15:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 25 k 2024-05-21T15:21:53.783 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T15:21:53.783 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T15:21:53.783 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T15:21:53.783 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T15:21:53.783 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T15:21:53.783 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T15:21:53.783 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T15:21:53.783 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T15:21:53.783 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 158 k 2024-05-21T15:21:53.783 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T15:21:53.783 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T15:21:53.783 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T15:21:53.783 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T15:21:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T15:21:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 497 k 2024-05-21T15:21:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T15:21:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.0 M 2024-05-21T15:21:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T15:21:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-21T15:21:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T15:21:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T15:21:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T15:21:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T15:21:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T15:21:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T15:21:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T15:21:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-21T15:21:53.785 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T15:21:53.785 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T15:21:53.785 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T15:21:53.785 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-21T15:21:53.785 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T15:21:53.785 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T15:21:53.785 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T15:21:53.785 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 45 k 2024-05-21T15:21:53.785 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 135 k 2024-05-21T15:21:53.785 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T15:21:53.785 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T15:21:53.785 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T15:21:53.785 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T15:21:53.786 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-21T15:21:53.786 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-21T15:21:53.786 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T15:21:53.786 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T15:21:53.786 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T15:21:53.786 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T15:21:53.786 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T15:21:53.786 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T15:21:53.786 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T15:21:53.786 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-21T15:21:53.786 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T15:21:53.786 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T15:21:53.786 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T15:21:53.787 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T15:21:53.787 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T15:21:53.787 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T15:21:53.787 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T15:21:53.787 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T15:21:53.787 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T15:21:53.787 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T15:21:53.787 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T15:21:53.787 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T15:21:53.787 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T15:21:53.787 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T15:21:53.787 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T15:21:53.787 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T15:21:53.788 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T15:21:53.788 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T15:21:53.788 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T15:21:53.788 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T15:21:53.788 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T15:21:53.788 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T15:21:53.788 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T15:21:53.788 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T15:21:53.788 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-21T15:21:53.788 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T15:21:53.789 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T15:21:53.789 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T15:21:53.789 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T15:21:53.789 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T15:21:53.789 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T15:21:53.789 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T15:21:53.789 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T15:21:53.789 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T15:21:53.789 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T15:21:53.789 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T15:21:53.789 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-21T15:21:53.789 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T15:21:53.789 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T15:21:53.790 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T15:21:53.790 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T15:21:53.790 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T15:21:53.790 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T15:21:53.790 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T15:21:53.790 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T15:21:53.790 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T15:21:53.790 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-21T15:21:53.790 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T15:21:53.790 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-21T15:21:53.790 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-05-21T15:21:53.791 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T15:21:53.791 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:21:53.791 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T15:21:53.791 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-05-21T15:21:53.791 INFO:teuthology.orchestra.run.smithi045.stdout:Install 148 Packages 2024-05-21T15:21:53.791 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 11 Packages 2024-05-21T15:21:53.791 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:21:53.791 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 222 M 2024-05-21T15:21:53.791 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T15:21:53.964 INFO:teuthology.orchestra.run.smithi038.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T15:21:53.965 INFO:teuthology.orchestra.run.smithi038.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T15:21:53.976 INFO:teuthology.orchestra.run.smithi038.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-21T15:21:53.977 INFO:teuthology.orchestra.run.smithi038.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-21T15:21:53.978 INFO:teuthology.orchestra.run.smithi038.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-21T15:21:53.979 INFO:teuthology.orchestra.run.smithi038.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-21T15:21:53.981 INFO:teuthology.orchestra.run.smithi038.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-21T15:21:53.983 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-21T15:21:53.984 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-21T15:21:54.053 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-05-21T15:21:54.077 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-05-21T15:21:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-05-21T15:21:54.077 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-05-21T15:21:54.077 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-05-21T15:21:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-21T15:21:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: ceph x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 6.6 k 2024-05-21T15:21:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.4 M 2024-05-21T15:21:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 863 k 2024-05-21T15:21:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 147 k 2024-05-21T15:21:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 1.5 M 2024-05-21T15:21:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 142 k 2024-05-21T15:21:54.077 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 3.5 M 2024-05-21T15:21:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 7.4 M 2024-05-21T15:21:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 49 k 2024-05-21T15:21:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 9.8 M 2024-05-21T15:21:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 46 M 2024-05-21T15:21:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 264 k 2024-05-21T15:21:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 763 k 2024-05-21T15:21:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 221 k 2024-05-21T15:21:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 27 k 2024-05-21T15:21:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 18 k 2024-05-21T15:21:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-21T15:21:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-21T15:21:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-21T15:21:54.078 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 32 k 2024-05-21T15:21:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 756 k 2024-05-21T15:21:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 127 k 2024-05-21T15:21:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-21T15:21:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-21T15:21:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-21T15:21:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-21T15:21:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-21T15:21:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-21T15:21:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 161 k 2024-05-21T15:21:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 320 k 2024-05-21T15:21:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 299 k 2024-05-21T15:21:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 99 k 2024-05-21T15:21:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 86 k 2024-05-21T15:21:54.079 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.0 M 2024-05-21T15:21:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 169 k 2024-05-21T15:21:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-21T15:21:54.080 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrading: 2024-05-21T15:21:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-21T15:21:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-21T15:21:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-21T15:21:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-21T15:21:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-21T15:21:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-21T15:21:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-21T15:21:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.3 M 2024-05-21T15:21:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.1 M 2024-05-21T15:21:54.080 INFO:teuthology.orchestra.run.smithi038.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-21T15:21:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-21T15:21:54.081 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-05-21T15:21:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T15:21:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-21T15:21:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T15:21:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 19 M 2024-05-21T15:21:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 32 k 2024-05-21T15:21:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 2.3 M 2024-05-21T15:21:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 249 k 2024-05-21T15:21:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 4.9 M 2024-05-21T15:21:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 18 M 2024-05-21T15:21:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 16 k 2024-05-21T15:21:54.081 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 25 k 2024-05-21T15:21:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T15:21:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T15:21:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T15:21:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T15:21:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T15:21:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T15:21:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T15:21:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T15:21:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 158 k 2024-05-21T15:21:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T15:21:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T15:21:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T15:21:54.082 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T15:21:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T15:21:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 497 k 2024-05-21T15:21:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T15:21:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.0 M 2024-05-21T15:21:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T15:21:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-21T15:21:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T15:21:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T15:21:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T15:21:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T15:21:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T15:21:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T15:21:54.083 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T15:21:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-21T15:21:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T15:21:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T15:21:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T15:21:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-21T15:21:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T15:21:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T15:21:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T15:21:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 45 k 2024-05-21T15:21:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 135 k 2024-05-21T15:21:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T15:21:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T15:21:54.084 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T15:21:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T15:21:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-21T15:21:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-21T15:21:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T15:21:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T15:21:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T15:21:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T15:21:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T15:21:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T15:21:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T15:21:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-21T15:21:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T15:21:54.085 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T15:21:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T15:21:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T15:21:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T15:21:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T15:21:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T15:21:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T15:21:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T15:21:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T15:21:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T15:21:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T15:21:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T15:21:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T15:21:54.086 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T15:21:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T15:21:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T15:21:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T15:21:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T15:21:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T15:21:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T15:21:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T15:21:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T15:21:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T15:21:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-21T15:21:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T15:21:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T15:21:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T15:21:54.087 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T15:21:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T15:21:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T15:21:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T15:21:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T15:21:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T15:21:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T15:21:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T15:21:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-21T15:21:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T15:21:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T15:21:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T15:21:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T15:21:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T15:21:54.088 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T15:21:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T15:21:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T15:21:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T15:21:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-21T15:21:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T15:21:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-21T15:21:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:Installing weak dependencies: 2024-05-21T15:21:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T15:21:54.089 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:21:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-05-21T15:21:54.089 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-05-21T15:21:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:Install 148 Packages 2024-05-21T15:21:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrade 11 Packages 2024-05-21T15:21:54.090 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:21:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 222 M 2024-05-21T15:21:54.090 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-05-21T15:21:54.397 INFO:teuthology.orchestra.run.smithi045.stdout:(1/159): ceph-19.0.0-2744.gd2b507c7.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-05-21T15:21:54.572 INFO:teuthology.orchestra.run.smithi053.stdout:(1/159): ceph-19.0.0-2744.gd2b507c7.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-05-21T15:21:54.606 INFO:teuthology.orchestra.run.smithi045.stdout:(2/159): ceph-fuse-19.0.0-2744.gd2b507c7.el9.x8 4.0 MB/s | 863 kB 00:00 2024-05-21T15:21:54.665 INFO:teuthology.orchestra.run.smithi045.stdout:(3/159): ceph-immutable-object-cache-19.0.0-274 2.5 MB/s | 147 kB 00:00 2024-05-21T15:21:54.915 INFO:teuthology.orchestra.run.smithi045.stdout:(4/159): ceph-base-19.0.0-2744.gd2b507c7.el9.x8 7.8 MB/s | 5.4 MB 00:00 2024-05-21T15:21:54.965 INFO:teuthology.orchestra.run.smithi045.stdout:(5/159): ceph-mds-19.0.0-2744.gd2b507c7.el9.x86 7.5 MB/s | 2.3 MB 00:00 2024-05-21T15:21:55.005 INFO:teuthology.orchestra.run.smithi053.stdout:(2/159): ceph-fuse-19.0.0-2744.gd2b507c7.el9.x8 1.9 MB/s | 863 kB 00:00 2024-05-21T15:21:55.082 INFO:teuthology.orchestra.run.smithi045.stdout:(6/159): ceph-mgr-19.0.0-2744.gd2b507c7.el9.x86 8.9 MB/s | 1.5 MB 00:00 2024-05-21T15:21:55.122 INFO:teuthology.orchestra.run.smithi053.stdout:(3/159): ceph-immutable-object-cache-19.0.0-274 1.2 MB/s | 147 kB 00:00 2024-05-21T15:21:55.230 INFO:teuthology.orchestra.run.smithi038.stdout:(1/159): ceph-19.0.0-2744.gd2b507c7.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-05-21T15:21:55.657 INFO:teuthology.orchestra.run.smithi045.stdout:(7/159): ceph-mon-19.0.0-2744.gd2b507c7.el9.x86 7.0 MB/s | 4.9 MB 00:00 2024-05-21T15:21:56.014 INFO:teuthology.orchestra.run.smithi053.stdout:(4/159): ceph-mds-19.0.0-2744.gd2b507c7.el9.x86 2.5 MB/s | 2.3 MB 00:00 2024-05-21T15:21:56.355 INFO:teuthology.orchestra.run.smithi038.stdout:(2/159): ceph-fuse-19.0.0-2744.gd2b507c7.el9.x8 767 kB/s | 863 kB 00:01 2024-05-21T15:21:56.547 INFO:teuthology.orchestra.run.smithi038.stdout:(3/159): ceph-immutable-object-cache-19.0.0-274 767 kB/s | 147 kB 00:00 2024-05-21T15:21:56.547 INFO:teuthology.orchestra.run.smithi053.stdout:(5/159): ceph-mgr-19.0.0-2744.gd2b507c7.el9.x86 2.8 MB/s | 1.5 MB 00:00 2024-05-21T15:21:56.806 INFO:teuthology.orchestra.run.smithi053.stdout:(6/159): ceph-base-19.0.0-2744.gd2b507c7.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-05-21T15:21:57.299 INFO:teuthology.orchestra.run.smithi045.stdout:(8/159): ceph-osd-19.0.0-2744.gd2b507c7.el9.x86 8.0 MB/s | 18 MB 00:02 2024-05-21T15:21:57.441 INFO:teuthology.orchestra.run.smithi045.stdout:(9/159): ceph-radosgw-19.0.0-2744.gd2b507c7.el9 5.5 MB/s | 9.8 MB 00:01 2024-05-21T15:21:57.466 INFO:teuthology.orchestra.run.smithi045.stdout:(10/159): ceph-selinux-19.0.0-2744.gd2b507c7.el 151 kB/s | 25 kB 00:00 2024-05-21T15:21:57.566 INFO:teuthology.orchestra.run.smithi045.stdout:(11/159): cephfs-mirror-19.0.0-2744.gd2b507c7.e 2.2 MB/s | 221 kB 00:00 2024-05-21T15:21:57.658 INFO:teuthology.orchestra.run.smithi045.stdout:(12/159): cephfs-shell-19.0.0-2744.gd2b507c7.el 295 kB/s | 27 kB 00:00 2024-05-21T15:21:57.742 INFO:teuthology.orchestra.run.smithi045.stdout:(13/159): libcephfs-devel-19.0.0-2744.gd2b507c7 380 kB/s | 32 kB 00:00 2024-05-21T15:21:57.805 INFO:teuthology.orchestra.run.smithi038.stdout:(4/159): ceph-mds-19.0.0-2744.gd2b507c7.el9.x86 1.8 MB/s | 2.3 MB 00:01 2024-05-21T15:21:57.918 INFO:teuthology.orchestra.run.smithi045.stdout:(14/159): libcephfs2-19.0.0-2744.gd2b507c7.el9. 4.2 MB/s | 756 kB 00:00 2024-05-21T15:21:58.001 INFO:teuthology.orchestra.run.smithi045.stdout:(15/159): libcephsqlite-19.0.0-2744.gd2b507c7.e 1.9 MB/s | 158 kB 00:00 2024-05-21T15:21:58.056 INFO:teuthology.orchestra.run.smithi053.stdout:(7/159): ceph-mon-19.0.0-2744.gd2b507c7.el9.x86 3.2 MB/s | 4.9 MB 00:01 2024-05-21T15:21:58.068 INFO:teuthology.orchestra.run.smithi045.stdout:(16/159): librados-devel-19.0.0-2744.gd2b507c7. 1.9 MB/s | 127 kB 00:00 2024-05-21T15:21:58.168 INFO:teuthology.orchestra.run.smithi045.stdout:(17/159): libradosstriper1-19.0.0-2744.gd2b507c 4.9 MB/s | 497 kB 00:00 2024-05-21T15:21:58.927 INFO:teuthology.orchestra.run.smithi045.stdout:(18/159): librgw2-19.0.0-2744.gd2b507c7.el9.x86 6.6 MB/s | 5.0 MB 00:00 2024-05-21T15:21:58.997 INFO:teuthology.orchestra.run.smithi038.stdout:(5/159): ceph-mgr-19.0.0-2744.gd2b507c7.el9.x86 1.2 MB/s | 1.5 MB 00:01 2024-05-21T15:21:59.002 INFO:teuthology.orchestra.run.smithi045.stdout:(19/159): python3-ceph-argparse-19.0.0-2744.gd2 605 kB/s | 45 kB 00:00 2024-05-21T15:21:59.086 INFO:teuthology.orchestra.run.smithi045.stdout:(20/159): python3-ceph-common-19.0.0-2744.gd2b5 1.6 MB/s | 135 kB 00:00 2024-05-21T15:21:59.153 INFO:teuthology.orchestra.run.smithi045.stdout:(21/159): python3-cephfs-19.0.0-2744.gd2b507c7. 2.4 MB/s | 161 kB 00:00 2024-05-21T15:21:59.228 INFO:teuthology.orchestra.run.smithi045.stdout:(22/159): python3-rados-19.0.0-2744.gd2b507c7.e 4.2 MB/s | 320 kB 00:00 2024-05-21T15:21:59.281 INFO:teuthology.orchestra.run.smithi053.stdout:(8/159): ceph-osd-19.0.0-2744.gd2b507c7.el9.x86 7.2 MB/s | 18 MB 00:02 2024-05-21T15:21:59.295 INFO:teuthology.orchestra.run.smithi045.stdout:(23/159): python3-rbd-19.0.0-2744.gd2b507c7.el9 4.4 MB/s | 299 kB 00:00 2024-05-21T15:21:59.348 INFO:teuthology.orchestra.run.smithi053.stdout:(9/159): ceph-selinux-19.0.0-2744.gd2b507c7.el9 378 kB/s | 25 kB 00:00 2024-05-21T15:21:59.362 INFO:teuthology.orchestra.run.smithi045.stdout:(24/159): python3-rgw-19.0.0-2744.gd2b507c7.el9 1.5 MB/s | 99 kB 00:00 2024-05-21T15:21:59.571 INFO:teuthology.orchestra.run.smithi045.stdout:(25/159): ceph-common-19.0.0-2744.gd2b507c7.el9 3.6 MB/s | 19 MB 00:05 2024-05-21T15:21:59.596 INFO:teuthology.orchestra.run.smithi045.stdout:(26/159): rbd-fuse-19.0.0-2744.gd2b507c7.el9.x8 367 kB/s | 86 kB 00:00 2024-05-21T15:21:59.687 INFO:teuthology.orchestra.run.smithi045.stdout:(27/159): rbd-nbd-19.0.0-2744.gd2b507c7.el9.x86 1.8 MB/s | 169 kB 00:00 2024-05-21T15:21:59.771 INFO:teuthology.orchestra.run.smithi045.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2744.g 381 kB/s | 32 kB 00:00 2024-05-21T15:21:59.855 INFO:teuthology.orchestra.run.smithi045.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2744.gd2b507c 1.7 MB/s | 142 kB 00:00 2024-05-21T15:22:00.497 INFO:teuthology.orchestra.run.smithi045.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2744.gd2b50 5.5 MB/s | 3.5 MB 00:00 2024-05-21T15:22:01.039 INFO:teuthology.orchestra.run.smithi045.stdout:(31/159): rbd-mirror-19.0.0-2744.gd2b507c7.el9. 2.1 MB/s | 3.0 MB 00:01 2024-05-21T15:22:01.097 INFO:teuthology.orchestra.run.smithi038.stdout:(6/159): ceph-mon-19.0.0-2744.gd2b507c7.el9.x86 2.3 MB/s | 4.9 MB 00:02 2024-05-21T15:22:01.214 INFO:teuthology.orchestra.run.smithi045.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2744.gd2 1.4 MB/s | 249 kB 00:00 2024-05-21T15:22:01.314 INFO:teuthology.orchestra.run.smithi045.stdout:(33/159): ceph-mgr-rook-19.0.0-2744.gd2b507c7.e 493 kB/s | 49 kB 00:00 2024-05-21T15:22:01.406 INFO:teuthology.orchestra.run.smithi045.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2744.gd 180 kB/s | 16 kB 00:00 2024-05-21T15:22:01.423 INFO:teuthology.orchestra.run.smithi038.stdout:(7/159): ceph-base-19.0.0-2744.gd2b507c7.el9.x8 838 kB/s | 5.4 MB 00:06 2024-05-21T15:22:01.449 INFO:teuthology.orchestra.run.smithi053.stdout:(10/159): ceph-radosgw-19.0.0-2744.gd2b507c7.el 2.9 MB/s | 9.8 MB 00:03 2024-05-21T15:22:01.573 INFO:teuthology.orchestra.run.smithi045.stdout:(35/159): ceph-volume-19.0.0-2744.gd2b507c7.el9 1.5 MB/s | 264 kB 00:00 2024-05-21T15:22:01.590 INFO:teuthology.orchestra.run.smithi053.stdout:(11/159): cephfs-mirror-19.0.0-2744.gd2b507c7.e 1.5 MB/s | 221 kB 00:00 2024-05-21T15:22:01.674 INFO:teuthology.orchestra.run.smithi053.stdout:(12/159): cephfs-shell-19.0.0-2744.gd2b507c7.el 324 kB/s | 27 kB 00:00 2024-05-21T15:22:01.723 INFO:teuthology.orchestra.run.smithi045.stdout:(36/159): ceph-mgr-diskprediction-local-19.0.0- 6.0 MB/s | 7.4 MB 00:01 2024-05-21T15:22:01.733 INFO:teuthology.orchestra.run.smithi053.stdout:(13/159): libcephfs-devel-19.0.0-2744.gd2b507c7 544 kB/s | 32 kB 00:00 2024-05-21T15:22:01.782 INFO:teuthology.orchestra.run.smithi045.stdout:(37/159): cephfs-top-19.0.0-2744.gd2b507c7.el9. 314 kB/s | 18 kB 00:00 2024-05-21T15:22:01.840 INFO:teuthology.orchestra.run.smithi045.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 80 kB 00:00 2024-05-21T15:22:01.932 INFO:teuthology.orchestra.run.smithi045.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-05-21T15:22:01.957 INFO:teuthology.orchestra.run.smithi045.stdout:(40/159): cephadm-19.0.0-2744.gd2b507c7.el9.noa 1.9 MB/s | 763 kB 00:00 2024-05-21T15:22:02.000 INFO:teuthology.orchestra.run.smithi053.stdout:(14/159): libcephfs2-19.0.0-2744.gd2b507c7.el9. 2.8 MB/s | 756 kB 00:00 2024-05-21T15:22:02.016 INFO:teuthology.orchestra.run.smithi045.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-05-21T15:22:02.049 INFO:teuthology.orchestra.run.smithi045.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-21T15:22:02.067 INFO:teuthology.orchestra.run.smithi053.stdout:(15/159): libcephsqlite-19.0.0-2744.gd2b507c7.e 2.3 MB/s | 158 kB 00:00 2024-05-21T15:22:02.117 INFO:teuthology.orchestra.run.smithi045.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-21T15:22:02.142 INFO:teuthology.orchestra.run.smithi053.stdout:(16/159): librados-devel-19.0.0-2744.gd2b507c7. 1.7 MB/s | 127 kB 00:00 2024-05-21T15:22:02.151 INFO:teuthology.orchestra.run.smithi045.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 9.1 MB/s | 1.2 MB 00:00 2024-05-21T15:22:02.217 INFO:teuthology.orchestra.run.smithi045.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-05-21T15:22:02.251 INFO:teuthology.orchestra.run.smithi045.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-05-21T15:22:02.302 INFO:teuthology.orchestra.run.smithi045.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-21T15:22:02.375 INFO:teuthology.orchestra.run.smithi053.stdout:(17/159): libradosstriper1-19.0.0-2744.gd2b507c 2.1 MB/s | 497 kB 00:00 2024-05-21T15:22:02.427 INFO:teuthology.orchestra.run.smithi045.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-05-21T15:22:02.468 INFO:teuthology.orchestra.run.smithi045.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-05-21T15:22:02.510 INFO:teuthology.orchestra.run.smithi045.stdout:(50/159): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-05-21T15:22:02.536 INFO:teuthology.orchestra.run.smithi045.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 324 kB/s | 22 kB 00:00 2024-05-21T15:22:02.561 INFO:teuthology.orchestra.run.smithi045.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-21T15:22:02.586 INFO:teuthology.orchestra.run.smithi045.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-05-21T15:22:02.628 INFO:teuthology.orchestra.run.smithi045.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-05-21T15:22:02.678 INFO:teuthology.orchestra.run.smithi045.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-05-21T15:22:02.704 INFO:teuthology.orchestra.run.smithi045.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-05-21T15:22:02.729 INFO:teuthology.orchestra.run.smithi045.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-21T15:22:02.754 INFO:teuthology.orchestra.run.smithi045.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-05-21T15:22:02.788 INFO:teuthology.orchestra.run.smithi045.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-21T15:22:02.821 INFO:teuthology.orchestra.run.smithi045.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-05-21T15:22:02.847 INFO:teuthology.orchestra.run.smithi045.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-05-21T15:22:02.872 INFO:teuthology.orchestra.run.smithi045.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-21T15:22:02.897 INFO:teuthology.orchestra.run.smithi045.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-21T15:22:02.923 INFO:teuthology.orchestra.run.smithi045.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-21T15:22:02.948 INFO:teuthology.orchestra.run.smithi045.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 705 kB/s | 35 kB 00:00 2024-05-21T15:22:03.032 INFO:teuthology.orchestra.run.smithi045.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 44 MB/s | 4.7 MB 00:00 2024-05-21T15:22:03.099 INFO:teuthology.orchestra.run.smithi045.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-05-21T15:22:03.157 INFO:teuthology.orchestra.run.smithi045.stdout:(68/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-05-21T15:22:03.191 INFO:teuthology.orchestra.run.smithi045.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 9.7 MB/s | 911 kB 00:00 2024-05-21T15:22:03.216 INFO:teuthology.orchestra.run.smithi045.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-21T15:22:03.283 INFO:teuthology.orchestra.run.smithi045.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-21T15:22:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-05-21T15:22:03.392 INFO:teuthology.orchestra.run.smithi045.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 804 kB/s | 87 kB 00:00 2024-05-21T15:22:03.442 INFO:teuthology.orchestra.run.smithi045.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-05-21T15:22:03.526 INFO:teuthology.orchestra.run.smithi045.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-05-21T15:22:03.551 INFO:teuthology.orchestra.run.smithi045.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-05-21T15:22:03.576 INFO:teuthology.orchestra.run.smithi045.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-05-21T15:22:03.601 INFO:teuthology.orchestra.run.smithi045.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-05-21T15:22:03.627 INFO:teuthology.orchestra.run.smithi045.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-05-21T15:22:03.669 INFO:teuthology.orchestra.run.smithi045.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 837 kB/s | 35 kB 00:00 2024-05-21T15:22:03.694 INFO:teuthology.orchestra.run.smithi045.stdout:(81/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-05-21T15:22:03.745 INFO:teuthology.orchestra.run.smithi045.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-21T15:22:03.811 INFO:teuthology.orchestra.run.smithi045.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-05-21T15:22:03.836 INFO:teuthology.orchestra.run.smithi045.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-05-21T15:22:03.862 INFO:teuthology.orchestra.run.smithi045.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-21T15:22:03.887 INFO:teuthology.orchestra.run.smithi045.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-05-21T15:22:03.912 INFO:teuthology.orchestra.run.smithi045.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-05-21T15:22:03.929 INFO:teuthology.orchestra.run.smithi045.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-05-21T15:22:03.963 INFO:teuthology.orchestra.run.smithi045.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-21T15:22:03.988 INFO:teuthology.orchestra.run.smithi045.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-05-21T15:22:04.013 INFO:teuthology.orchestra.run.smithi045.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-21T15:22:04.055 INFO:teuthology.orchestra.run.smithi045.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-21T15:22:04.122 INFO:teuthology.orchestra.run.smithi045.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 665 kB/s | 44 kB 00:00 2024-05-21T15:22:04.126 INFO:teuthology.orchestra.run.smithi053.stdout:(18/159): ceph-common-19.0.0-2744.gd2b507c7.el9 2.0 MB/s | 19 MB 00:09 2024-05-21T15:22:04.173 INFO:teuthology.orchestra.run.smithi045.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-21T15:22:04.193 INFO:teuthology.orchestra.run.smithi053.stdout:(19/159): librgw2-19.0.0-2744.gd2b507c7.el9.x86 2.8 MB/s | 5.0 MB 00:01 2024-05-21T15:22:04.214 INFO:teuthology.orchestra.run.smithi045.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 868 kB/s | 36 kB 00:00 2024-05-21T15:22:04.218 INFO:teuthology.orchestra.run.smithi053.stdout:(20/159): python3-ceph-argparse-19.0.0-2744.gd2 492 kB/s | 45 kB 00:00 2024-05-21T15:22:04.257 INFO:teuthology.orchestra.run.smithi045.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-05-21T15:22:04.307 INFO:teuthology.orchestra.run.smithi045.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-05-21T15:22:04.318 INFO:teuthology.orchestra.run.smithi053.stdout:(21/159): python3-ceph-common-19.0.0-2744.gd2b5 1.1 MB/s | 135 kB 00:00 2024-05-21T15:22:04.360 INFO:teuthology.orchestra.run.smithi053.stdout:(22/159): python3-cephfs-19.0.0-2744.gd2b507c7. 1.1 MB/s | 161 kB 00:00 2024-05-21T15:22:04.452 INFO:teuthology.orchestra.run.smithi053.stdout:(23/159): python3-rados-19.0.0-2744.gd2b507c7.e 2.3 MB/s | 320 kB 00:00 2024-05-21T15:22:04.482 INFO:teuthology.orchestra.run.smithi045.stdout:(98/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-05-21T15:22:04.502 INFO:teuthology.orchestra.run.smithi053.stdout:(24/159): python3-rbd-19.0.0-2744.gd2b507c7.el9 2.1 MB/s | 299 kB 00:00 2024-05-21T15:22:04.507 INFO:teuthology.orchestra.run.smithi045.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 288 kB/s | 58 kB 00:00 2024-05-21T15:22:04.528 INFO:teuthology.orchestra.run.smithi053.stdout:(25/159): python3-rgw-19.0.0-2744.gd2b507c7.el9 1.3 MB/s | 99 kB 00:00 2024-05-21T15:22:04.533 INFO:teuthology.orchestra.run.smithi045.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-05-21T15:22:04.558 INFO:teuthology.orchestra.run.smithi045.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-21T15:22:04.578 INFO:teuthology.orchestra.run.smithi053.stdout:(26/159): rbd-fuse-19.0.0-2744.gd2b507c7.el9.x8 1.1 MB/s | 86 kB 00:00 2024-05-21T15:22:04.686 INFO:teuthology.orchestra.run.smithi053.stdout:(27/159): rbd-nbd-19.0.0-2744.gd2b507c7.el9.x86 1.5 MB/s | 169 kB 00:00 2024-05-21T15:22:04.762 INFO:teuthology.orchestra.run.smithi053.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2744.g 423 kB/s | 32 kB 00:00 2024-05-21T15:22:04.837 INFO:teuthology.orchestra.run.smithi053.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2744.gd2b507c 1.9 MB/s | 142 kB 00:00 2024-05-21T15:22:04.983 INFO:teuthology.orchestra.run.smithi045.stdout:(102/159): ceph-test-19.0.0-2744.gd2b507c7.el9. 6.2 MB/s | 46 MB 00:07 2024-05-21T15:22:05.008 INFO:teuthology.orchestra.run.smithi045.stdout:(103/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 94 kB/s | 45 kB 00:00 2024-05-21T15:22:05.176 INFO:teuthology.orchestra.run.smithi045.stdout:(104/159): lua-devel-5.4.4-4.el9.x86_64.rpm 36 kB/s | 22 kB 00:00 2024-05-21T15:22:05.371 INFO:teuthology.orchestra.run.smithi053.stdout:(30/159): rbd-mirror-19.0.0-2744.gd2b507c7.el9. 3.6 MB/s | 3.0 MB 00:00 2024-05-21T15:22:05.984 INFO:teuthology.orchestra.run.smithi045.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 863 kB/s | 863 kB 00:01 2024-05-21T15:22:06.096 INFO:teuthology.orchestra.run.smithi053.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2744.gd2b50 2.8 MB/s | 3.5 MB 00:01 2024-05-21T15:22:06.188 INFO:teuthology.orchestra.run.smithi053.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2744.gd2 2.7 MB/s | 249 kB 00:00 2024-05-21T15:22:06.247 INFO:teuthology.orchestra.run.smithi038.stdout:(8/159): ceph-common-19.0.0-2744.gd2b507c7.el9. 1.7 MB/s | 19 MB 00:11 2024-05-21T15:22:06.255 INFO:teuthology.orchestra.run.smithi053.stdout:(33/159): ceph-mgr-rook-19.0.0-2744.gd2b507c7.e 742 kB/s | 49 kB 00:00 2024-05-21T15:22:06.356 INFO:teuthology.orchestra.run.smithi038.stdout:(9/159): ceph-radosgw-19.0.0-2744.gd2b507c7.el9 2.0 MB/s | 9.8 MB 00:04 2024-05-21T15:22:06.381 INFO:teuthology.orchestra.run.smithi038.stdout:(10/159): ceph-selinux-19.0.0-2744.gd2b507c7.el 189 kB/s | 25 kB 00:00 2024-05-21T15:22:06.440 INFO:teuthology.orchestra.run.smithi038.stdout:(11/159): cephfs-mirror-19.0.0-2744.gd2b507c7.e 3.7 MB/s | 221 kB 00:00 2024-05-21T15:22:06.490 INFO:teuthology.orchestra.run.smithi038.stdout:(12/159): cephfs-shell-19.0.0-2744.gd2b507c7.el 542 kB/s | 27 kB 00:00 2024-05-21T15:22:06.540 INFO:teuthology.orchestra.run.smithi038.stdout:(13/159): libcephfs-devel-19.0.0-2744.gd2b507c7 635 kB/s | 32 kB 00:00 2024-05-21T15:22:06.664 INFO:teuthology.orchestra.run.smithi053.stdout:(34/159): ceph-test-19.0.0-2744.gd2b507c7.el9.x 6.4 MB/s | 46 MB 00:07 2024-05-21T15:22:06.689 INFO:teuthology.orchestra.run.smithi053.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2744.gd 38 kB/s | 16 kB 00:00 2024-05-21T15:22:06.798 INFO:teuthology.orchestra.run.smithi053.stdout:(36/159): ceph-volume-19.0.0-2744.gd2b507c7.el9 1.9 MB/s | 264 kB 00:00 2024-05-21T15:22:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:(14/159): libcephfs2-19.0.0-2744.gd2b507c7.el9. 2.4 MB/s | 756 kB 00:00 2024-05-21T15:22:06.856 INFO:teuthology.orchestra.run.smithi053.stdout:(37/159): cephfs-top-19.0.0-2744.gd2b507c7.el9. 314 kB/s | 18 kB 00:00 2024-05-21T15:22:06.916 INFO:teuthology.orchestra.run.smithi038.stdout:(15/159): libcephsqlite-19.0.0-2744.gd2b507c7.e 2.3 MB/s | 158 kB 00:00 2024-05-21T15:22:06.940 INFO:teuthology.orchestra.run.smithi053.stdout:(38/159): cephadm-19.0.0-2744.gd2b507c7.el9.noa 3.0 MB/s | 763 kB 00:00 2024-05-21T15:22:06.975 INFO:teuthology.orchestra.run.smithi038.stdout:(16/159): librados-devel-19.0.0-2744.gd2b507c7. 2.1 MB/s | 127 kB 00:00 2024-05-21T15:22:07.062 INFO:teuthology.orchestra.run.smithi045.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 18 kB/s | 19 kB 00:01 2024-05-21T15:22:07.117 INFO:teuthology.orchestra.run.smithi038.stdout:(17/159): libradosstriper1-19.0.0-2744.gd2b507c 3.4 MB/s | 497 kB 00:00 2024-05-21T15:22:07.249 INFO:teuthology.orchestra.run.smithi053.stdout:(39/159): ceph-mgr-diskprediction-local-19.0.0- 3.9 MB/s | 7.4 MB 00:01 2024-05-21T15:22:07.332 INFO:teuthology.orchestra.run.smithi053.stdout:(40/159): fuse-2.9.9-15.el9.x86_64.rpm 169 kB/s | 80 kB 00:00 2024-05-21T15:22:07.343 INFO:teuthology.orchestra.run.smithi045.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 142 kB/s | 309 kB 00:02 2024-05-21T15:22:07.376 INFO:teuthology.orchestra.run.smithi045.stdout:(108/159): abseil-cpp-20211102.0-3.el9.x86_64.r 231 kB/s | 548 kB 00:02 2024-05-21T15:22:07.674 INFO:teuthology.orchestra.run.smithi053.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 741 kB/s | 253 kB 00:00 2024-05-21T15:22:07.799 INFO:teuthology.orchestra.run.smithi053.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 935 kB/s | 803 kB 00:00 2024-05-21T15:22:07.833 INFO:teuthology.orchestra.run.smithi053.stdout:(43/159): libquadmath-11.4.1-3.el9.x86_64.rpm 334 kB/s | 195 kB 00:00 2024-05-21T15:22:07.908 INFO:teuthology.orchestra.run.smithi053.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 981 kB/s | 106 kB 00:00 2024-05-21T15:22:07.942 INFO:teuthology.orchestra.run.smithi038.stdout:(18/159): librgw2-19.0.0-2744.gd2b507c7.el9.x86 6.1 MB/s | 5.0 MB 00:00 2024-05-21T15:22:08.001 INFO:teuthology.orchestra.run.smithi038.stdout:(19/159): python3-ceph-argparse-19.0.0-2744.gd2 775 kB/s | 45 kB 00:00 2024-05-21T15:22:08.034 INFO:teuthology.orchestra.run.smithi053.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 999 kB/s | 125 kB 00:00 2024-05-21T15:22:08.050 INFO:teuthology.orchestra.run.smithi053.stdout:(46/159): python3-pycparser-2.20-6.el9.noarch.r 623 kB/s | 135 kB 00:00 2024-05-21T15:22:08.059 INFO:teuthology.orchestra.run.smithi038.stdout:(20/159): python3-ceph-common-19.0.0-2744.gd2b5 2.3 MB/s | 135 kB 00:00 2024-05-21T15:22:08.084 INFO:teuthology.orchestra.run.smithi053.stdout:(47/159): python3-cryptography-36.0.1-4.el9.x86 3.0 MB/s | 1.2 MB 00:00 2024-05-21T15:22:08.126 INFO:teuthology.orchestra.run.smithi038.stdout:(21/159): python3-cephfs-19.0.0-2744.gd2b507c7. 2.4 MB/s | 161 kB 00:00 2024-05-21T15:22:08.143 INFO:teuthology.orchestra.run.smithi053.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-21T15:22:08.227 INFO:teuthology.orchestra.run.smithi045.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 29 kB/s | 26 kB 00:00 2024-05-21T15:22:08.235 INFO:teuthology.orchestra.run.smithi038.stdout:(22/159): python3-rados-19.0.0-2744.gd2b507c7.e 2.9 MB/s | 320 kB 00:00 2024-05-21T15:22:08.260 INFO:teuthology.orchestra.run.smithi045.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 55 kB/s | 49 kB 00:00 2024-05-21T15:22:08.278 INFO:teuthology.orchestra.run.smithi053.stdout:(49/159): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-05-21T15:22:08.310 INFO:teuthology.orchestra.run.smithi038.stdout:(23/159): python3-rbd-19.0.0-2744.gd2b507c7.el9 3.9 MB/s | 299 kB 00:00 2024-05-21T15:22:08.386 INFO:teuthology.orchestra.run.smithi038.stdout:(24/159): python3-rgw-19.0.0-2744.gd2b507c7.el9 1.3 MB/s | 99 kB 00:00 2024-05-21T15:22:08.405 INFO:teuthology.orchestra.run.smithi053.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 408 kB/s | 107 kB 00:00 2024-05-21T15:22:08.427 INFO:teuthology.orchestra.run.smithi053.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 146 kB/s | 22 kB 00:00 2024-05-21T15:22:08.453 INFO:teuthology.orchestra.run.smithi038.stdout:(25/159): rbd-fuse-19.0.0-2744.gd2b507c7.el9.x8 1.3 MB/s | 86 kB 00:00 2024-05-21T15:22:08.469 INFO:teuthology.orchestra.run.smithi053.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 787 kB/s | 33 kB 00:00 2024-05-21T15:22:08.469 INFO:teuthology.orchestra.run.smithi045.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 3.1 MB/s | 4.4 MB 00:01 2024-05-21T15:22:08.494 INFO:teuthology.orchestra.run.smithi045.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 252 kB/s | 67 kB 00:00 2024-05-21T15:22:08.502 INFO:teuthology.orchestra.run.smithi053.stdout:(53/159): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 943 kB 00:00 2024-05-21T15:22:08.519 INFO:teuthology.orchestra.run.smithi045.stdout:(113/159): luarocks-3.9.2-5.el9.noarch.rpm 591 kB/s | 151 kB 00:00 2024-05-21T15:22:08.529 INFO:teuthology.orchestra.run.smithi053.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 2.5 MB/s | 313 kB 00:00 2024-05-21T15:22:08.678 INFO:teuthology.orchestra.run.smithi053.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 104 kB/s | 18 kB 00:00 2024-05-21T15:22:08.703 INFO:teuthology.orchestra.run.smithi053.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 64 kB/s | 11 kB 00:00 2024-05-21T15:22:08.731 INFO:teuthology.orchestra.run.smithi053.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-21T15:22:08.754 INFO:teuthology.orchestra.run.smithi053.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-05-21T15:22:08.761 INFO:teuthology.orchestra.run.smithi045.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.8 MB/s | 839 kB 00:00 2024-05-21T15:22:08.786 INFO:teuthology.orchestra.run.smithi045.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-05-21T15:22:08.787 INFO:teuthology.orchestra.run.smithi053.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-21T15:22:08.803 INFO:teuthology.orchestra.run.smithi045.stdout:(116/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 153 kB/s | 43 kB 00:00 2024-05-21T15:22:08.821 INFO:teuthology.orchestra.run.smithi053.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-05-21T15:22:08.828 INFO:teuthology.orchestra.run.smithi038.stdout:(26/159): rbd-mirror-19.0.0-2744.gd2b507c7.el9. 8.1 MB/s | 3.0 MB 00:00 2024-05-21T15:22:08.846 INFO:teuthology.orchestra.run.smithi053.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-05-21T15:22:08.871 INFO:teuthology.orchestra.run.smithi053.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-05-21T15:22:08.887 INFO:teuthology.orchestra.run.smithi038.stdout:(27/159): rbd-nbd-19.0.0-2744.gd2b507c7.el9.x86 2.8 MB/s | 169 kB 00:00 2024-05-21T15:22:08.897 INFO:teuthology.orchestra.run.smithi053.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-21T15:22:08.945 INFO:teuthology.orchestra.run.smithi038.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2744.g 545 kB/s | 32 kB 00:00 2024-05-21T15:22:08.947 INFO:teuthology.orchestra.run.smithi053.stdout:(64/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.2 MB/s | 3.0 MB 00:00 2024-05-21T15:22:08.973 INFO:teuthology.orchestra.run.smithi053.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-05-21T15:22:08.978 INFO:teuthology.orchestra.run.smithi045.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 149 kB/s | 32 kB 00:00 2024-05-21T15:22:08.998 INFO:teuthology.orchestra.run.smithi053.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 350 kB/s | 35 kB 00:00 2024-05-21T15:22:09.012 INFO:teuthology.orchestra.run.smithi038.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2744.gd2b507c 2.1 MB/s | 142 kB 00:00 2024-05-21T15:22:09.029 INFO:teuthology.orchestra.run.smithi045.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 67 kB/s | 15 kB 00:00 2024-05-21T15:22:09.054 INFO:teuthology.orchestra.run.smithi045.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 686 kB/s | 172 kB 00:00 2024-05-21T15:22:09.090 INFO:teuthology.orchestra.run.smithi053.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-05-21T15:22:09.132 INFO:teuthology.orchestra.run.smithi053.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-21T15:22:09.199 INFO:teuthology.orchestra.run.smithi053.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-05-21T15:22:09.221 INFO:teuthology.orchestra.run.smithi045.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-05-21T15:22:09.232 INFO:teuthology.orchestra.run.smithi053.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-05-21T15:22:09.258 INFO:teuthology.orchestra.run.smithi053.stdout:(71/159): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-05-21T15:22:09.271 INFO:teuthology.orchestra.run.smithi045.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.2 MB/s | 303 kB 00:00 2024-05-21T15:22:09.296 INFO:teuthology.orchestra.run.smithi045.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 849 kB/s | 206 kB 00:00 2024-05-21T15:22:09.300 INFO:teuthology.orchestra.run.smithi053.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-21T15:22:09.316 INFO:teuthology.orchestra.run.smithi053.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-21T15:22:09.359 INFO:teuthology.orchestra.run.smithi053.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-05-21T15:22:09.434 INFO:teuthology.orchestra.run.smithi053.stdout:(75/159): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-05-21T15:22:09.459 INFO:teuthology.orchestra.run.smithi053.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-21T15:22:09.493 INFO:teuthology.orchestra.run.smithi053.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-05-21T15:22:09.513 INFO:teuthology.orchestra.run.smithi038.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2744.gd2b50 7.0 MB/s | 3.5 MB 00:00 2024-05-21T15:22:09.518 INFO:teuthology.orchestra.run.smithi053.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-05-21T15:22:09.563 INFO:teuthology.orchestra.run.smithi045.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 6.0 MB/s | 2.0 MB 00:00 2024-05-21T15:22:09.568 INFO:teuthology.orchestra.run.smithi053.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-05-21T15:22:09.588 INFO:teuthology.orchestra.run.smithi045.stdout:(124/159): python3-grpcio-tools-1.46.7-10.el9.x 456 kB/s | 144 kB 00:00 2024-05-21T15:22:09.593 INFO:teuthology.orchestra.run.smithi053.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-05-21T15:22:09.614 INFO:teuthology.orchestra.run.smithi045.stdout:(125/159): python3-jaraco-8.2.1-3.el9.noarch.rp 34 kB/s | 11 kB 00:00 2024-05-21T15:22:09.627 INFO:teuthology.orchestra.run.smithi053.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 591 kB/s | 35 kB 00:00 2024-05-21T15:22:09.719 INFO:teuthology.orchestra.run.smithi053.stdout:(82/159): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-05-21T15:22:09.744 INFO:teuthology.orchestra.run.smithi053.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-05-21T15:22:09.770 INFO:teuthology.orchestra.run.smithi053.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-21T15:22:09.789 INFO:teuthology.orchestra.run.smithi045.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 78 kB/s | 18 kB 00:00 2024-05-21T15:22:09.795 INFO:teuthology.orchestra.run.smithi053.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-21T15:22:09.796 INFO:teuthology.orchestra.run.smithi038.stdout:(31/159): ceph-osd-19.0.0-2744.gd2b507c7.el9.x8 2.0 MB/s | 18 MB 00:08 2024-05-21T15:22:09.820 INFO:teuthology.orchestra.run.smithi053.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-05-21T15:22:09.839 INFO:teuthology.orchestra.run.smithi045.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 103 kB/s | 23 kB 00:00 2024-05-21T15:22:09.845 INFO:teuthology.orchestra.run.smithi053.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-21T15:22:09.864 INFO:teuthology.orchestra.run.smithi045.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 78 kB/s | 19 kB 00:00 2024-05-21T15:22:09.871 INFO:teuthology.orchestra.run.smithi053.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-21T15:22:09.896 INFO:teuthology.orchestra.run.smithi053.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-21T15:22:09.921 INFO:teuthology.orchestra.run.smithi053.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-21T15:22:09.922 INFO:teuthology.orchestra.run.smithi038.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2744.gd2 1.9 MB/s | 249 kB 00:00 2024-05-21T15:22:09.947 INFO:teuthology.orchestra.run.smithi053.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-21T15:22:10.013 INFO:teuthology.orchestra.run.smithi038.stdout:(33/159): ceph-mgr-rook-19.0.0-2744.gd2b507c7.e 539 kB/s | 49 kB 00:00 2024-05-21T15:22:10.023 INFO:teuthology.orchestra.run.smithi045.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 84 kB/s | 20 kB 00:00 2024-05-21T15:22:10.030 INFO:teuthology.orchestra.run.smithi053.stdout:(92/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-05-21T15:22:10.056 INFO:teuthology.orchestra.run.smithi053.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-05-21T15:22:10.081 INFO:teuthology.orchestra.run.smithi053.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 876 kB/s | 44 kB 00:00 2024-05-21T15:22:10.089 INFO:teuthology.orchestra.run.smithi038.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2744.gd 219 kB/s | 16 kB 00:00 2024-05-21T15:22:10.106 INFO:teuthology.orchestra.run.smithi053.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-21T15:22:10.131 INFO:teuthology.orchestra.run.smithi045.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-05-21T15:22:10.132 INFO:teuthology.orchestra.run.smithi053.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-05-21T15:22:10.157 INFO:teuthology.orchestra.run.smithi045.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 158 kB/s | 46 kB 00:00 2024-05-21T15:22:10.157 INFO:teuthology.orchestra.run.smithi053.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-21T15:22:10.183 INFO:teuthology.orchestra.run.smithi053.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-21T15:22:10.207 INFO:teuthology.orchestra.run.smithi053.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-21T15:22:10.225 INFO:teuthology.orchestra.run.smithi053.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-05-21T15:22:10.231 INFO:teuthology.orchestra.run.smithi038.stdout:(35/159): ceph-volume-19.0.0-2744.gd2b507c7.el9 1.8 MB/s | 264 kB 00:00 2024-05-21T15:22:10.250 INFO:teuthology.orchestra.run.smithi053.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-05-21T15:22:10.257 INFO:teuthology.orchestra.run.smithi045.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 337 kB/s | 79 kB 00:00 2024-05-21T15:22:10.267 INFO:teuthology.orchestra.run.smithi053.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-05-21T15:22:10.342 INFO:teuthology.orchestra.run.smithi053.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-05-21T15:22:10.357 INFO:teuthology.orchestra.run.smithi045.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 256 kB/s | 58 kB 00:00 2024-05-21T15:22:10.390 INFO:teuthology.orchestra.run.smithi038.stdout:(36/159): ceph-mgr-diskprediction-local-19.0.0- 8.4 MB/s | 7.4 MB 00:00 2024-05-21T15:22:10.399 INFO:teuthology.orchestra.run.smithi045.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2024-05-21T15:22:10.440 INFO:teuthology.orchestra.run.smithi038.stdout:(37/159): cephfs-top-19.0.0-2744.gd2b507c7.el9. 366 kB/s | 18 kB 00:00 2024-05-21T15:22:10.483 INFO:teuthology.orchestra.run.smithi045.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 73 kB/s | 16 kB 00:00 2024-05-21T15:22:10.507 INFO:teuthology.orchestra.run.smithi038.stdout:(38/159): cephadm-19.0.0-2744.gd2b507c7.el9.noa 2.7 MB/s | 763 kB 00:00 2024-05-21T15:22:10.526 INFO:teuthology.orchestra.run.smithi053.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-05-21T15:22:10.582 INFO:teuthology.orchestra.run.smithi038.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 566 kB/s | 80 kB 00:00 2024-05-21T15:22:10.592 INFO:teuthology.orchestra.run.smithi045.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 384 kB/s | 90 kB 00:00 2024-05-21T15:22:10.633 INFO:teuthology.orchestra.run.smithi045.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 120 kB/s | 28 kB 00:00 2024-05-21T15:22:10.674 INFO:teuthology.orchestra.run.smithi038.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-21T15:22:10.684 INFO:teuthology.orchestra.run.smithi053.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-05-21T15:22:10.717 INFO:teuthology.orchestra.run.smithi045.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 132 kB/s | 31 kB 00:00 2024-05-21T15:22:10.783 INFO:teuthology.orchestra.run.smithi038.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-05-21T15:22:10.825 INFO:teuthology.orchestra.run.smithi038.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-05-21T15:22:10.834 INFO:teuthology.orchestra.run.smithi045.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 779 kB/s | 188 kB 00:00 2024-05-21T15:22:10.859 INFO:teuthology.orchestra.run.smithi045.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 262 kB/s | 59 kB 00:00 2024-05-21T15:22:10.867 INFO:teuthology.orchestra.run.smithi038.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-21T15:22:10.910 INFO:teuthology.orchestra.run.smithi053.stdout:(106/159): abseil-cpp-20211102.0-3.el9.x86_64.r 966 kB/s | 548 kB 00:00 2024-05-21T15:22:10.917 INFO:teuthology.orchestra.run.smithi038.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-21T15:22:10.935 INFO:teuthology.orchestra.run.smithi053.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 812 kB/s | 309 kB 00:00 2024-05-21T15:22:10.951 INFO:teuthology.orchestra.run.smithi045.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 153 kB/s | 36 kB 00:00 2024-05-21T15:22:10.976 INFO:teuthology.orchestra.run.smithi038.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-21T15:22:10.977 INFO:teuthology.orchestra.run.smithi053.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 626 kB/s | 26 kB 00:00 2024-05-21T15:22:11.002 INFO:teuthology.orchestra.run.smithi053.stdout:(109/159): grpc-data-1.46.7-10.el9.noarch.rpm 61 kB/s | 19 kB 00:00 2024-05-21T15:22:11.027 INFO:teuthology.orchestra.run.smithi053.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-05-21T15:22:11.059 INFO:teuthology.orchestra.run.smithi038.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-21T15:22:11.068 INFO:teuthology.orchestra.run.smithi045.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 218 kB/s | 51 kB 00:00 2024-05-21T15:22:11.069 INFO:teuthology.orchestra.run.smithi053.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-21T15:22:11.095 INFO:teuthology.orchestra.run.smithi053.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-05-21T15:22:11.102 INFO:teuthology.orchestra.run.smithi045.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 948 kB/s | 230 kB 00:00 2024-05-21T15:22:11.160 INFO:teuthology.orchestra.run.smithi038.stdout:(47/159): python3-cryptography-36.0.1-4.el9.x86 3.3 MB/s | 1.2 MB 00:00 2024-05-21T15:22:11.185 INFO:teuthology.orchestra.run.smithi045.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 383 kB/s | 90 kB 00:00 2024-05-21T15:22:11.187 INFO:teuthology.orchestra.run.smithi053.stdout:(113/159): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-05-21T15:22:11.194 INFO:teuthology.orchestra.run.smithi038.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 5.0 MB/s | 689 kB 00:00 2024-05-21T15:22:11.220 INFO:teuthology.orchestra.run.smithi053.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-05-21T15:22:11.261 INFO:teuthology.orchestra.run.smithi038.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-21T15:22:11.262 INFO:teuthology.orchestra.run.smithi053.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 580 kB/s | 43 kB 00:00 2024-05-21T15:22:11.304 INFO:teuthology.orchestra.run.smithi053.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 385 kB/s | 32 kB 00:00 2024-05-21T15:22:11.311 INFO:teuthology.orchestra.run.smithi038.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-05-21T15:22:11.346 INFO:teuthology.orchestra.run.smithi053.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-05-21T15:22:11.379 INFO:teuthology.orchestra.run.smithi053.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-21T15:22:11.405 INFO:teuthology.orchestra.run.smithi053.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-05-21T15:22:11.489 INFO:teuthology.orchestra.run.smithi053.stdout:(120/159): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-21T15:22:11.570 INFO:teuthology.orchestra.run.smithi038.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 1.2 MB/s | 313 kB 00:00 2024-05-21T15:22:11.610 INFO:teuthology.orchestra.run.smithi045.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 788 kB/s | 427 kB 00:00 2024-05-21T15:22:11.620 INFO:teuthology.orchestra.run.smithi038.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-05-21T15:22:11.636 INFO:teuthology.orchestra.run.smithi045.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 37 kB/s | 20 kB 00:00 2024-05-21T15:22:11.661 INFO:teuthology.orchestra.run.smithi045.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 403 kB/s | 192 kB 00:00 2024-05-21T15:22:11.680 INFO:teuthology.orchestra.run.smithi053.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.0 MB/s | 303 kB 00:00 2024-05-21T15:22:11.731 INFO:teuthology.orchestra.run.smithi053.stdout:(122/159): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-05-21T15:22:11.765 INFO:teuthology.orchestra.run.smithi053.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-05-21T15:22:11.790 INFO:teuthology.orchestra.run.smithi053.stdout:(124/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.1 MB/s | 839 kB 00:00 2024-05-21T15:22:11.807 INFO:teuthology.orchestra.run.smithi053.stdout:(125/159): python3-jaraco-8.2.1-3.el9.noarch.rp 141 kB/s | 11 kB 00:00 2024-05-21T15:22:11.824 INFO:teuthology.orchestra.run.smithi053.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 300 kB/s | 18 kB 00:00 2024-05-21T15:22:11.841 INFO:teuthology.orchestra.run.smithi053.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-05-21T15:22:11.858 INFO:teuthology.orchestra.run.smithi053.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-05-21T15:22:11.862 INFO:teuthology.orchestra.run.smithi038.stdout:(53/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-05-21T15:22:11.874 INFO:teuthology.orchestra.run.smithi053.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-05-21T15:22:11.904 INFO:teuthology.orchestra.run.smithi038.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 440 kB/s | 18 kB 00:00 2024-05-21T15:22:11.917 INFO:teuthology.orchestra.run.smithi053.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 789 kB/s | 46 kB 00:00 2024-05-21T15:22:11.950 INFO:teuthology.orchestra.run.smithi053.stdout:(131/159): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-05-21T15:22:12.009 INFO:teuthology.orchestra.run.smithi053.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 589 kB/s | 79 kB 00:00 2024-05-21T15:22:12.021 INFO:teuthology.orchestra.run.smithi038.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-05-21T15:22:12.034 INFO:teuthology.orchestra.run.smithi053.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-21T15:22:12.063 INFO:teuthology.orchestra.run.smithi038.stdout:(56/159): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-05-21T15:22:12.076 INFO:teuthology.orchestra.run.smithi053.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-05-21T15:22:12.095 INFO:teuthology.orchestra.run.smithi045.stdout:(148/159): librados2-19.0.0-2744.gd2b507c7.el9. 7.6 MB/s | 3.3 MB 00:00 2024-05-21T15:22:12.101 INFO:teuthology.orchestra.run.smithi053.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-21T15:22:12.122 INFO:teuthology.orchestra.run.smithi038.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 774 kB/s | 45 kB 00:00 2024-05-21T15:22:12.143 INFO:teuthology.orchestra.run.smithi053.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-05-21T15:22:12.153 INFO:teuthology.orchestra.run.smithi045.stdout:(149/159): librbd1-19.0.0-2744.gd2b507c7.el9.x8 6.3 MB/s | 3.1 MB 00:00 2024-05-21T15:22:12.156 INFO:teuthology.orchestra.run.smithi038.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-21T15:22:12.169 INFO:teuthology.orchestra.run.smithi053.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 458 kB/s | 31 kB 00:00 2024-05-21T15:22:12.210 INFO:teuthology.orchestra.run.smithi053.stdout:(138/159): python3-natsort-7.1.1-5.el9.noarch.r 197 kB/s | 58 kB 00:00 2024-05-21T15:22:12.236 INFO:teuthology.orchestra.run.smithi053.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 885 kB/s | 59 kB 00:00 2024-05-21T15:22:12.269 INFO:teuthology.orchestra.run.smithi053.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-05-21T15:22:12.311 INFO:teuthology.orchestra.run.smithi053.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-05-21T15:22:12.322 INFO:teuthology.orchestra.run.smithi038.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-05-21T15:22:12.353 INFO:teuthology.orchestra.run.smithi053.stdout:(142/159): python3-routes-2.5.1-5.el9.noarch.rp 899 kB/s | 188 kB 00:00 2024-05-21T15:22:12.356 INFO:teuthology.orchestra.run.smithi038.stdout:(60/159): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-05-21T15:22:12.378 INFO:teuthology.orchestra.run.smithi053.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-05-21T15:22:12.404 INFO:teuthology.orchestra.run.smithi053.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-05-21T15:22:12.437 INFO:teuthology.orchestra.run.smithi053.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-05-21T15:22:12.445 INFO:teuthology.orchestra.run.smithi045.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 6.2 MB/s | 1.8 MB 00:00 2024-05-21T15:22:12.479 INFO:teuthology.orchestra.run.smithi053.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-21T15:22:12.487 INFO:teuthology.orchestra.run.smithi045.stdout:(151/159): thrift-0.15.0-2.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-05-21T15:22:12.563 INFO:teuthology.orchestra.run.smithi053.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-21T15:22:12.613 INFO:teuthology.orchestra.run.smithi053.stdout:(148/159): python3-werkzeug-2.0.3-3.el9.1.noarc 1.6 MB/s | 427 kB 00:00 2024-05-21T15:22:12.620 INFO:teuthology.orchestra.run.smithi045.stdout:(152/159): NetworkManager-1.47.90-1.el9.x86_64. 4.4 MB/s | 2.3 MB 00:00 2024-05-21T15:22:12.746 INFO:teuthology.orchestra.run.smithi045.stdout:(153/159): NetworkManager-team-1.47.90-1.el9.x8 138 kB/s | 41 kB 00:00 2024-05-21T15:22:12.782 INFO:teuthology.orchestra.run.smithi038.stdout:(61/159): ceph-test-19.0.0-2744.gd2b507c7.el9.x 7.2 MB/s | 46 MB 00:06 2024-05-21T15:22:12.787 INFO:teuthology.orchestra.run.smithi045.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 220 kB/s | 37 kB 00:00 2024-05-21T15:22:12.807 INFO:teuthology.orchestra.run.smithi038.stdout:(62/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 53 kB/s | 25 kB 00:00 2024-05-21T15:22:12.822 INFO:teuthology.orchestra.run.smithi045.stdout:(155/159): NetworkManager-tui-1.47.90-1.el9.x86 742 kB/s | 248 kB 00:00 2024-05-21T15:22:12.832 INFO:teuthology.orchestra.run.smithi038.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 100 kB/s | 47 kB 00:00 2024-05-21T15:22:12.863 INFO:teuthology.orchestra.run.smithi045.stdout:(156/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.7 MB/s | 207 kB 00:00 2024-05-21T15:22:12.882 INFO:teuthology.orchestra.run.smithi038.stdout:(64/159): openblas-0.3.21-2.el9.x86_64.rpm 706 kB/s | 35 kB 00:00 2024-05-21T15:22:12.908 INFO:teuthology.orchestra.run.smithi038.stdout:(65/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-05-21T15:22:12.913 INFO:teuthology.orchestra.run.smithi045.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.0 MB/s | 260 kB 00:00 2024-05-21T15:22:13.008 INFO:teuthology.orchestra.run.smithi038.stdout:(66/159): lua-5.4.4-4.el9.x86_64.rpm 939 kB/s | 188 kB 00:00 2024-05-21T15:22:13.013 INFO:teuthology.orchestra.run.smithi045.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 7.0 MB/s | 1.3 MB 00:00 2024-05-21T15:22:13.114 INFO:teuthology.orchestra.run.smithi053.stdout:(149/159): librados2-19.0.0-2744.gd2b507c7.el9. 5.2 MB/s | 3.3 MB 00:00 2024-05-21T15:22:13.155 INFO:teuthology.orchestra.run.smithi045.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.2 MB/s | 2.4 MB 00:00 2024-05-21T15:22:13.161 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T15:22:13.162 INFO:teuthology.orchestra.run.smithi045.stdout:Total 11 MB/s | 222 MB 00:19 2024-05-21T15:22:13.181 INFO:teuthology.orchestra.run.smithi053.stdout:(150/159): librbd1-19.0.0-2744.gd2b507c7.el9.x8 5.0 MB/s | 3.1 MB 00:00 2024-05-21T15:22:13.200 INFO:teuthology.orchestra.run.smithi038.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-05-21T15:22:13.375 INFO:teuthology.orchestra.run.smithi038.stdout:(68/159): openblas-openmp-0.3.21-2.el9.x86_64.r 9.6 MB/s | 4.7 MB 00:00 2024-05-21T15:22:13.442 INFO:teuthology.orchestra.run.smithi038.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.3 MB/s | 4.4 MB 00:00 2024-05-21T15:22:13.467 INFO:teuthology.orchestra.run.smithi038.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 3.3 MB/s | 911 kB 00:00 2024-05-21T15:22:13.510 INFO:teuthology.orchestra.run.smithi038.stdout:(71/159): postgresql-private-libs-13.14-1.el9.x 994 kB/s | 133 kB 00:00 2024-05-21T15:22:13.531 INFO:teuthology.orchestra.run.smithi053.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 118 kB/s | 41 kB 00:00 2024-05-21T15:22:13.569 INFO:teuthology.orchestra.run.smithi038.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-21T15:22:13.573 INFO:teuthology.orchestra.run.smithi053.stdout:(152/159): NetworkManager-1.47.90-1.el9.x86_64. 2.5 MB/s | 2.3 MB 00:00 2024-05-21T15:22:13.636 INFO:teuthology.orchestra.run.smithi038.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 6.0 MB/s | 1.0 MB 00:00 2024-05-21T15:22:13.681 INFO:teuthology.orchestra.run.smithi053.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 339 kB/s | 37 kB 00:00 2024-05-21T15:22:13.694 INFO:teuthology.orchestra.run.smithi038.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-05-21T15:22:13.778 INFO:teuthology.orchestra.run.smithi038.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-21T15:22:13.799 INFO:teuthology.orchestra.run.smithi053.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.7 MB/s | 207 kB 00:00 2024-05-21T15:22:13.861 INFO:teuthology.orchestra.run.smithi038.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-21T15:22:13.899 INFO:teuthology.orchestra.run.smithi053.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.5 MB/s | 260 kB 00:00 2024-05-21T15:22:13.953 INFO:teuthology.orchestra.run.smithi038.stdout:(77/159): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-05-21T15:22:13.974 INFO:teuthology.orchestra.run.smithi053.stdout:(156/159): NetworkManager-tui-1.47.90-1.el9.x86 560 kB/s | 248 kB 00:00 2024-05-21T15:22:13.978 INFO:teuthology.orchestra.run.smithi038.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 219 kB/s | 26 kB 00:00 2024-05-21T15:22:14.054 INFO:teuthology.orchestra.run.smithi038.stdout:(79/159): postgresql-server-13.14-1.el9.x86_64. 9.5 MB/s | 5.8 MB 00:00 2024-05-21T15:22:14.079 INFO:teuthology.orchestra.run.smithi038.stdout:(80/159): python3-jsonpointer-2.0-4.el9.noarch. 152 kB/s | 19 kB 00:00 2024-05-21T15:22:14.108 INFO:teuthology.orchestra.run.smithi053.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 1.3 MB 00:00 2024-05-21T15:22:14.121 INFO:teuthology.orchestra.run.smithi038.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 518 kB/s | 35 kB 00:00 2024-05-21T15:22:14.147 INFO:teuthology.orchestra.run.smithi038.stdout:(82/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-05-21T15:22:14.150 INFO:teuthology.orchestra.run.smithi053.stdout:(158/159): NetworkManager-libnm-1.47.90-1.el9.x 1.7 MB/s | 1.8 MB 00:01 2024-05-21T15:22:14.214 INFO:teuthology.orchestra.run.smithi038.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-05-21T15:22:14.247 INFO:teuthology.orchestra.run.smithi038.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-21T15:22:14.306 INFO:teuthology.orchestra.run.smithi038.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 841 kB/s | 77 kB 00:00 2024-05-21T15:22:14.323 INFO:teuthology.orchestra.run.smithi038.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 561 kB/s | 42 kB 00:00 2024-05-21T15:22:14.392 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T15:22:14.432 INFO:teuthology.orchestra.run.smithi038.stdout:(87/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-05-21T15:22:14.457 INFO:teuthology.orchestra.run.smithi038.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-05-21T15:22:14.458 INFO:teuthology.orchestra.run.smithi053.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 2.4 MB 00:00 2024-05-21T15:22:14.466 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-05-21T15:22:14.466 INFO:teuthology.orchestra.run.smithi053.stdout:Total 11 MB/s | 222 MB 00:20 2024-05-21T15:22:14.482 INFO:teuthology.orchestra.run.smithi038.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-05-21T15:22:14.541 INFO:teuthology.orchestra.run.smithi038.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 879 kB/s | 51 kB 00:00 2024-05-21T15:22:14.566 INFO:teuthology.orchestra.run.smithi038.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 645 kB/s | 54 kB 00:00 2024-05-21T15:22:14.569 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T15:22:14.569 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T15:22:14.591 INFO:teuthology.orchestra.run.smithi038.stdout:(92/159): python3-pyasn1-modules-0.4.8-6.el9.no 1.7 MB/s | 279 kB 00:00 2024-05-21T15:22:14.642 INFO:teuthology.orchestra.run.smithi038.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-05-21T15:22:14.667 INFO:teuthology.orchestra.run.smithi038.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 586 kB/s | 44 kB 00:00 2024-05-21T15:22:14.717 INFO:teuthology.orchestra.run.smithi038.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 722 kB/s | 36 kB 00:00 2024-05-21T15:22:14.742 INFO:teuthology.orchestra.run.smithi038.stdout:(96/159): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-05-21T15:22:14.793 INFO:teuthology.orchestra.run.smithi038.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-05-21T15:22:14.844 INFO:teuthology.orchestra.run.smithi038.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-21T15:22:14.869 INFO:teuthology.orchestra.run.smithi038.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 767 kB/s | 58 kB 00:00 2024-05-21T15:22:14.927 INFO:teuthology.orchestra.run.smithi038.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-05-21T15:22:14.953 INFO:teuthology.orchestra.run.smithi038.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 762 kB/s | 64 kB 00:00 2024-05-21T15:22:15.003 INFO:teuthology.orchestra.run.smithi038.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-05-21T15:22:15.253 INFO:teuthology.orchestra.run.smithi038.stdout:(103/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-05-21T15:22:15.278 INFO:teuthology.orchestra.run.smithi038.stdout:(104/159): lua-devel-5.4.4-4.el9.x86_64.rpm 68 kB/s | 22 kB 00:00 2024-05-21T15:22:15.554 INFO:teuthology.orchestra.run.smithi038.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-05-21T15:22:15.596 INFO:teuthology.orchestra.run.smithi038.stdout:(106/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-05-21T15:22:15.621 INFO:teuthology.orchestra.run.smithi038.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 290 kB/s | 19 kB 00:00 2024-05-21T15:22:15.666 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-05-21T15:22:15.688 INFO:teuthology.orchestra.run.smithi038.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-05-21T15:22:15.755 INFO:teuthology.orchestra.run.smithi038.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 731 kB/s | 49 kB 00:00 2024-05-21T15:22:15.830 INFO:teuthology.orchestra.run.smithi038.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 897 kB/s | 67 kB 00:00 2024-05-21T15:22:15.844 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-05-21T15:22:15.844 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-05-21T15:22:15.889 INFO:teuthology.orchestra.run.smithi038.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-21T15:22:15.948 INFO:teuthology.orchestra.run.smithi038.stdout:(112/159): protobuf-compiler-3.14.0-13.el9.x86_ 914 kB/s | 863 kB 00:00 2024-05-21T15:22:16.073 INFO:teuthology.orchestra.run.smithi038.stdout:(113/159): libarrow-9.0.0-11.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-05-21T15:22:16.110 INFO:teuthology.orchestra.run.smithi038.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 839 kB 00:00 2024-05-21T15:22:16.140 INFO:teuthology.orchestra.run.smithi038.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 649 kB/s | 43 kB 00:00 2024-05-21T15:22:16.174 INFO:teuthology.orchestra.run.smithi038.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-05-21T15:22:16.208 INFO:teuthology.orchestra.run.smithi038.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-05-21T15:22:16.241 INFO:teuthology.orchestra.run.smithi038.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-21T15:22:16.283 INFO:teuthology.orchestra.run.smithi038.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-05-21T15:22:16.308 INFO:teuthology.orchestra.run.smithi038.stdout:(120/159): python3-asyncssh-2.13.2-5.el9.noarch 1.5 MB/s | 548 kB 00:00 2024-05-21T15:22:16.334 INFO:teuthology.orchestra.run.smithi038.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-21T15:22:16.367 INFO:teuthology.orchestra.run.smithi038.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-21T15:22:16.409 INFO:teuthology.orchestra.run.smithi038.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-21T15:22:16.435 INFO:teuthology.orchestra.run.smithi038.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 159 kB/s | 11 kB 00:00 2024-05-21T15:22:16.457 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T15:22:16.458 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T15:22:16.502 INFO:teuthology.orchestra.run.smithi038.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-05-21T15:22:16.527 INFO:teuthology.orchestra.run.smithi038.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-05-21T15:22:16.586 INFO:teuthology.orchestra.run.smithi038.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-05-21T15:22:16.611 INFO:teuthology.orchestra.run.smithi038.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-05-21T15:22:16.662 INFO:teuthology.orchestra.run.smithi038.stdout:(129/159): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-05-21T15:22:16.687 INFO:teuthology.orchestra.run.smithi038.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-05-21T15:22:16.729 INFO:teuthology.orchestra.run.smithi038.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-05-21T15:22:16.763 INFO:teuthology.orchestra.run.smithi038.stdout:(132/159): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-05-21T15:22:16.788 INFO:teuthology.orchestra.run.smithi038.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 573 kB/s | 58 kB 00:00 2024-05-21T15:22:16.813 INFO:teuthology.orchestra.run.smithi038.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-21T15:22:16.838 INFO:teuthology.orchestra.run.smithi038.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-05-21T15:22:16.855 INFO:teuthology.orchestra.run.smithi038.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-21T15:22:16.881 INFO:teuthology.orchestra.run.smithi038.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 419 kB/s | 28 kB 00:00 2024-05-21T15:22:16.906 INFO:teuthology.orchestra.run.smithi038.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 457 kB/s | 31 kB 00:00 2024-05-21T15:22:16.934 INFO:teuthology.orchestra.run.smithi038.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-21T15:22:16.956 INFO:teuthology.orchestra.run.smithi038.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 783 kB/s | 59 kB 00:00 2024-05-21T15:22:16.982 INFO:teuthology.orchestra.run.smithi038.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-05-21T15:22:17.007 INFO:teuthology.orchestra.run.smithi038.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-05-21T15:22:17.033 INFO:teuthology.orchestra.run.smithi038.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-21T15:22:17.058 INFO:teuthology.orchestra.run.smithi038.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-21T15:22:17.083 INFO:teuthology.orchestra.run.smithi038.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-05-21T15:22:17.108 INFO:teuthology.orchestra.run.smithi038.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 264 kB/s | 20 kB 00:00 2024-05-21T15:22:17.134 INFO:teuthology.orchestra.run.smithi038.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-21T15:22:17.276 INFO:teuthology.orchestra.run.smithi038.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-05-21T15:22:17.459 INFO:teuthology.orchestra.run.smithi038.stdout:(149/159): librbd1-19.0.0-2744.gd2b507c7.el9.x8 9.5 MB/s | 3.1 MB 00:00 2024-05-21T15:22:17.743 INFO:teuthology.orchestra.run.smithi038.stdout:(150/159): NetworkManager-1.47.90-1.el9.x86_64. 4.9 MB/s | 2.3 MB 00:00 2024-05-21T15:22:17.746 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-05-21T15:22:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-05-21T15:22:17.810 INFO:teuthology.orchestra.run.smithi038.stdout:(151/159): NetworkManager-libnm-1.47.90-1.el9.x 5.1 MB/s | 1.8 MB 00:00 2024-05-21T15:22:17.836 INFO:teuthology.orchestra.run.smithi038.stdout:(152/159): NetworkManager-team-1.47.90-1.el9.x8 451 kB/s | 41 kB 00:00 2024-05-21T15:22:17.869 INFO:teuthology.orchestra.run.smithi038.stdout:(153/159): NetworkManager-tui-1.47.90-1.el9.x86 4.1 MB/s | 248 kB 00:00 2024-05-21T15:22:17.886 INFO:teuthology.orchestra.run.smithi038.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 729 kB/s | 37 kB 00:00 2024-05-21T15:22:17.979 INFO:teuthology.orchestra.run.smithi038.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.8 MB/s | 260 kB 00:00 2024-05-21T15:22:18.179 INFO:teuthology.orchestra.run.smithi038.stdout:(156/159): elfutils-libelf-0.191-4.el9.x86_64.r 669 kB/s | 207 kB 00:00 2024-05-21T15:22:18.229 INFO:teuthology.orchestra.run.smithi038.stdout:(157/159): librados2-19.0.0-2744.gd2b507c7.el9. 3.0 MB/s | 3.3 MB 00:01 2024-05-21T15:22:18.296 INFO:teuthology.orchestra.run.smithi038.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 4.2 MB/s | 1.3 MB 00:00 2024-05-21T15:22:18.663 INFO:teuthology.orchestra.run.smithi038.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 2.4 MB 00:00 2024-05-21T15:22:18.665 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-05-21T15:22:18.666 INFO:teuthology.orchestra.run.smithi038.stdout:Total 9.0 MB/s | 222 MB 00:24 2024-05-21T15:22:19.941 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-05-21T15:22:20.116 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-05-21T15:22:20.116 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-05-21T15:22:22.341 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-05-21T15:22:22.341 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-05-21T15:22:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T15:22:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-21T15:22:23.133 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-21T15:22:23.188 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-21T15:22:23.228 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-21T15:22:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-21T15:22:23.297 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-21T15:22:23.647 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-21T15:22:23.762 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:22:23.906 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:22:23.929 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:22:24.022 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:22:24.050 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-21T15:22:24.083 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-21T15:22:24.248 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-05-21T15:22:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-21T15:22:24.324 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-21T15:22:24.338 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-21T15:22:24.351 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:22:24.376 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-21T15:22:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-21T15:22:24.449 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:22:24.464 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-21T15:22:24.477 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-21T15:22:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-21T15:22:24.532 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-21T15:22:24.540 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-21T15:22:24.562 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T15:22:24.588 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T15:22:24.746 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T15:22:24.871 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T15:22:24.883 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-21T15:22:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:22:25.044 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T15:22:25.096 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T15:22:25.135 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:22:25.166 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:22:25.197 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:22:25.258 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:22:25.272 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:22:25.287 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-21T15:22:25.320 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-21T15:22:25.480 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:22:25.506 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-21T15:22:25.561 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-21T15:22:25.588 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:22:25.590 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-21T15:22:25.686 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-21T15:22:25.695 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:22:25.722 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-21T15:22:25.728 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-21T15:22:25.772 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-21T15:22:25.785 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-21T15:22:25.807 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T15:22:25.833 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T15:22:25.848 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-21T15:22:25.889 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-21T15:22:25.933 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-21T15:22:25.965 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-21T15:22:25.993 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-21T15:22:26.007 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T15:22:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-21T15:22:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-21T15:22:26.142 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T15:22:26.146 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-21T15:22:26.182 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-21T15:22:26.262 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-21T15:22:26.293 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 34/170 2024-05-21T15:22:26.304 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T15:22:26.337 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 35/170 2024-05-21T15:22:26.365 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 36/170 2024-05-21T15:22:26.374 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T15:22:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-21T15:22:26.488 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:22:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-21T15:22:26.558 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:22:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-21T15:22:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-21T15:22:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-21T15:22:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:22:26.909 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-21T15:22:27.004 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-21T15:22:27.038 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-21T15:22:27.073 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-21T15:22:27.148 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-21T15:22:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-21T15:22:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-21T15:22:27.276 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-21T15:22:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-21T15:22:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-21T15:22:27.375 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-21T15:22:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-21T15:22:27.428 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-21T15:22:27.457 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-21T15:22:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-21T15:22:27.492 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-21T15:22:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-21T15:22:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-21T15:22:27.571 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-21T15:22:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 47/170 2024-05-21T15:22:27.602 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 34/170 2024-05-21T15:22:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-21T15:22:27.654 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 35/170 2024-05-21T15:22:27.683 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 36/170 2024-05-21T15:22:27.748 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-21T15:22:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-21T15:22:27.798 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-21T15:22:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-21T15:22:27.876 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-21T15:22:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-21T15:22:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-21T15:22:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-21T15:22:28.013 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-21T15:22:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-21T15:22:28.077 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-21T15:22:28.085 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-21T15:22:28.120 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-21T15:22:28.164 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-21T15:22:28.206 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-21T15:22:28.350 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-21T15:22:28.390 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-21T15:22:28.466 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-21T15:22:28.565 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-21T15:22:28.598 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-21T15:22:28.636 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-21T15:22:28.639 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-21T15:22:28.664 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-21T15:22:28.688 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-21T15:22:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-21T15:22:28.766 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-21T15:22:28.767 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-21T15:22:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-21T15:22:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-21T15:22:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 47/170 2024-05-21T15:22:28.897 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-21T15:22:28.933 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-21T15:22:28.937 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-21T15:22:28.966 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-21T15:22:29.000 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-21T15:22:29.003 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-05-21T15:22:29.040 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-21T15:22:29.079 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-21T15:22:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-21T15:22:29.092 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-21T15:22:29.109 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-21T15:22:29.135 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-21T15:22:29.135 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-21T15:22:29.144 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-21T15:22:29.169 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-21T15:22:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-21T15:22:29.210 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-21T15:22:29.226 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-21T15:22:29.244 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-21T15:22:29.269 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-21T15:22:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-21T15:22:29.294 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-21T15:22:29.311 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-21T15:22:29.336 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-21T15:22:29.344 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-21T15:22:29.356 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-21T15:22:29.384 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-21T15:22:29.404 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-21T15:22:29.414 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-21T15:22:29.439 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-21T15:22:29.470 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-21T15:22:29.482 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-21T15:22:29.524 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-21T15:22:29.582 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-21T15:22:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-21T15:22:29.669 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-21T15:22:29.691 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-21T15:22:29.718 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-21T15:22:29.760 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-21T15:22:29.820 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:22:29.859 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-21T15:22:29.892 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-21T15:22:29.924 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-21T15:22:29.949 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:22:29.949 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-21T15:22:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-21T15:22:29.988 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:22:30.054 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-21T15:22:30.081 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:22:30.115 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-21T15:22:30.162 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-21T15:22:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-21T15:22:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-21T15:22:30.260 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-21T15:22:30.294 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-21T15:22:30.334 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-21T15:22:30.374 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-21T15:22:30.403 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-21T15:22:30.406 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-21T15:22:30.412 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-21T15:22:30.429 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-21T15:22:30.447 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-21T15:22:30.454 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-21T15:22:30.477 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-21T15:22:30.497 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-21T15:22:30.504 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:22:30.514 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-21T15:22:30.531 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-21T15:22:30.611 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:22:30.617 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-21T15:22:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-21T15:22:30.639 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-21T15:22:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-21T15:22:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-21T15:22:30.702 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-21T15:22:30.732 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T15:22:30.736 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-21T15:22:30.749 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-21T15:22:30.752 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T15:22:30.781 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-21T15:22:30.860 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-21T15:22:30.872 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-21T15:22:30.920 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-21T15:22:30.935 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T15:22:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-21T15:22:31.106 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T15:22:31.209 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-21T15:22:31.239 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-21T15:22:31.271 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T15:22:31.319 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-21T15:22:31.331 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T15:22:31.435 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:22:31.508 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:22:31.701 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-21T15:22:31.722 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:22:31.750 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-21T15:22:31.783 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-21T15:22:31.833 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-21T15:22:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-21T15:22:31.931 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-21T15:22:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-21T15:22:31.972 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-21T15:22:31.993 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-21T15:22:32.003 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-21T15:22:32.008 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-21T15:22:32.084 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-21T15:22:32.084 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-21T15:22:32.119 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-21T15:22:32.124 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-21T15:22:32.125 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-21T15:22:32.167 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-21T15:22:32.200 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-21T15:22:32.227 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-21T15:22:32.299 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-21T15:22:32.304 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-21T15:22:32.333 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-21T15:22:32.352 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-21T15:22:32.389 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-21T15:22:32.414 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-21T15:22:32.433 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-21T15:22:32.512 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-21T15:22:32.544 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 34/170 2024-05-21T15:22:32.587 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 35/170 2024-05-21T15:22:32.616 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 36/170 2024-05-21T15:22:32.692 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-21T15:22:32.740 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-21T15:22:32.817 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-21T15:22:32.955 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-21T15:22:33.027 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-21T15:22:33.107 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-21T15:22:33.229 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-21T15:22:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-21T15:22:33.518 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-21T15:22:33.546 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-21T15:22:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-21T15:22:33.584 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-21T15:22:33.668 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-21T15:22:33.702 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-21T15:22:33.739 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-21T15:22:33.779 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-21T15:22:33.809 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 47/170 2024-05-21T15:22:33.852 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-21T15:22:33.924 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-21T15:22:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-21T15:22:34.002 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-21T15:22:34.054 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-21T15:22:34.058 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-21T15:22:34.088 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 104/170 2024-05-21T15:22:34.118 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-21T15:22:34.125 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-21T15:22:34.193 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-21T15:22:34.229 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-21T15:22:34.266 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-21T15:22:34.314 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-21T15:22:34.357 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-21T15:22:34.401 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-21T15:22:34.442 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-21T15:22:34.529 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-21T15:22:34.587 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-21T15:22:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-21T15:22:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-21T15:22:34.627 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-21T15:22:34.627 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-21T15:22:34.658 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-21T15:22:34.661 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-21T15:22:34.670 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-21T15:22:34.716 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-21T15:22:34.746 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/170 2024-05-21T15:22:34.767 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-21T15:22:34.803 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-21T15:22:34.843 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-21T15:22:34.885 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-21T15:22:34.909 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-21T15:22:34.995 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-21T15:22:35.010 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-21T15:22:35.045 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-21T15:22:35.137 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-21T15:22:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-21T15:22:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 104/170 2024-05-21T15:22:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-21T15:22:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-21T15:22:35.204 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-21T15:22:35.238 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-21T15:22:35.278 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-21T15:22:35.317 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-21T15:22:35.347 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-21T15:22:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-21T15:22:35.415 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-21T15:22:35.449 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-21T15:22:35.513 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-21T15:22:35.607 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-21T15:22:35.664 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-21T15:22:35.694 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-21T15:22:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-21T15:22:35.740 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-21T15:22:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-21T15:22:35.795 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-21T15:22:35.825 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/170 2024-05-21T15:22:35.853 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-21T15:22:35.906 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-21T15:22:36.679 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-21T15:22:36.735 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-21T15:22:36.769 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-21T15:22:36.879 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-21T15:22:36.933 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-21T15:22:36.979 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-21T15:22:37.017 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:22:37.051 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:22:37.077 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-21T15:22:37.113 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-21T15:22:37.297 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-21T15:22:37.327 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-21T15:22:37.409 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-21T15:22:37.636 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:22:37.671 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-21T15:22:37.762 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-21T15:22:37.762 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T15:22:37.763 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T15:22:37.763 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:22:37.789 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-21T15:22:38.103 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-21T15:22:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:22:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:22:38.222 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-21T15:22:38.808 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:22:38.851 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-21T15:22:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-21T15:22:38.926 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T15:22:38.926 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T15:22:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:22:38.952 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-21T15:22:39.560 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-21T15:22:39.640 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-21T15:22:39.668 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-21T15:22:39.701 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-21T15:22:40.039 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-21T15:22:40.089 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-21T15:22:40.181 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-21T15:22:40.211 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 104/170 2024-05-21T15:22:40.257 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-21T15:22:40.759 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-21T15:22:40.784 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-21T15:22:40.856 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-21T15:22:40.886 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/170 2024-05-21T15:22:43.217 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:22:43.257 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:22:43.849 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:22:43.886 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-21T15:22:43.977 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-21T15:22:43.977 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T15:22:43.977 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T15:22:43.977 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:22:44.003 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-21T15:22:49.673 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-21T15:22:49.673 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-05-21T15:22:49.673 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-05-21T15:22:49.673 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-05-21T15:22:49.674 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-05-21T15:22:49.674 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-05-21T15:22:49.674 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-05-21T15:22:49.674 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-05-21T15:22:49.674 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:22:49.840 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-21T15:22:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-21T15:22:49.857 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:49.857 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T15:22:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:22:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-21T15:22:50.977 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-05-21T15:22:50.977 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-05-21T15:22:50.977 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-05-21T15:22:50.977 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-05-21T15:22:50.977 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-05-21T15:22:50.977 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /root 2024-05-21T15:22:50.977 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-05-21T15:22:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:22:51.154 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-21T15:22:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-21T15:22:51.170 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:51.171 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T15:22:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:22:51.493 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-21T15:22:51.520 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-21T15:22:51.520 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:51.520 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T15:22:51.521 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T15:22:51.521 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T15:22:51.521 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:22:51.752 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-21T15:22:51.776 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-21T15:22:51.776 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:51.776 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T15:22:51.776 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T15:22:51.776 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T15:22:51.777 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:22:52.269 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-21T15:22:52.293 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-21T15:22:52.294 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:52.294 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T15:22:52.294 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T15:22:52.294 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T15:22:52.294 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:22:52.409 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-21T15:22:52.467 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-21T15:22:52.530 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-21T15:22:52.726 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-21T15:22:52.779 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-21T15:22:52.807 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-21T15:22:52.833 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-21T15:22:52.833 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:52.833 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T15:22:52.834 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T15:22:52.834 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T15:22:52.834 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:22:52.859 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-21T15:22:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-21T15:22:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-21T15:22:53.089 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-21T15:22:53.089 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:53.089 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T15:22:53.089 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T15:22:53.090 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T15:22:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:22:53.100 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-21T15:22:53.146 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-21T15:22:53.178 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-21T15:22:53.205 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-21T15:22:53.575 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-21T15:22:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-21T15:22:53.599 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:53.599 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T15:22:53.599 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T15:22:53.599 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T15:22:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:22:53.660 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-21T15:22:53.688 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-21T15:22:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-21T15:22:53.720 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 128/170 2024-05-21T15:22:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-21T15:22:53.784 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 129/170 2024-05-21T15:22:53.811 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-21T15:22:53.836 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-21T15:22:54.042 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-21T15:22:54.093 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-21T15:22:54.174 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-21T15:22:54.264 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-21T15:22:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-21T15:22:54.484 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-21T15:22:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-21T15:22:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-21T15:22:54.900 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-21T15:22:54.922 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-21T15:22:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-21T15:22:54.950 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-21T15:22:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 128/170 2024-05-21T15:22:55.047 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 129/170 2024-05-21T15:22:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-21T15:22:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-21T15:22:55.980 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-21T15:22:56.087 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-21T15:22:56.175 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-21T15:22:56.208 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-21T15:22:56.229 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-21T15:22:56.229 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /sys 2024-05-21T15:22:56.229 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /proc 2024-05-21T15:22:56.229 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /mnt 2024-05-21T15:22:56.229 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /var/tmp 2024-05-21T15:22:56.229 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /home 2024-05-21T15:22:56.230 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /root 2024-05-21T15:22:56.230 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /tmp 2024-05-21T15:22:56.230 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:22:56.261 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 133/170 2024-05-21T15:22:56.290 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-21T15:22:56.315 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-21T15:22:56.315 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:56.315 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T15:22:56.315 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T15:22:56.315 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T15:22:56.315 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:22:56.366 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-21T15:22:56.377 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-21T15:22:56.402 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-21T15:22:56.419 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-21T15:22:56.419 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:56.419 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T15:22:56.419 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:22:56.480 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 136/170 2024-05-21T15:22:56.532 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/170 2024-05-21T15:22:57.232 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-21T15:22:57.258 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-21T15:22:57.366 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-21T15:22:57.495 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-21T15:22:57.528 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-21T15:22:57.539 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 133/170 2024-05-21T15:22:57.552 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-21T15:22:57.552 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:57.552 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T15:22:57.552 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T15:22:57.552 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T15:22:57.553 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:22:57.568 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-21T15:22:57.593 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-21T15:22:57.593 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:57.593 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T15:22:57.593 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T15:22:57.593 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T15:22:57.593 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:22:57.594 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-21T15:22:57.618 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-21T15:22:57.618 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:57.618 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T15:22:57.618 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:22:57.644 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-21T15:22:57.655 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-21T15:22:57.670 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-21T15:22:57.694 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-21T15:22:57.695 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:57.695 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-21T15:22:57.695 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T15:22:57.695 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T15:22:57.695 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:22:57.758 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 136/170 2024-05-21T15:22:57.811 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/170 2024-05-21T15:22:58.046 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-21T15:22:58.071 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-21T15:22:58.071 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:58.071 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T15:22:58.071 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T15:22:58.071 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T15:22:58.071 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:22:58.072 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-21T15:22:58.101 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-21T15:22:58.101 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:58.102 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T15:22:58.102 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T15:22:58.102 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T15:22:58.102 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:22:58.341 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-21T15:22:58.366 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-21T15:22:58.366 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:58.366 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T15:22:58.367 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T15:22:58.367 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T15:22:58.367 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:22:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-21T15:22:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-21T15:22:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-21T15:22:58.829 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:58.829 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T15:22:58.829 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T15:22:58.829 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T15:22:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:22:58.858 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-21T15:22:58.872 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-21T15:22:58.883 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-21T15:22:58.883 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:58.883 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T15:22:58.883 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T15:22:58.883 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T15:22:58.883 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:22:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-21T15:22:58.896 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:58.896 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T15:22:58.896 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:22:58.948 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-21T15:22:58.972 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-21T15:22:58.972 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:58.972 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-21T15:22:58.972 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T15:22:58.972 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T15:22:58.973 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:22:58.989 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-21T15:22:59.047 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-21T15:22:59.111 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-21T15:22:59.298 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-21T15:22:59.306 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-21T15:22:59.322 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-21T15:22:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:22:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T15:22:59.322 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T15:22:59.323 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T15:22:59.323 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:22:59.360 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-21T15:22:59.441 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-21T15:22:59.538 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-21T15:22:59.691 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-21T15:22:59.735 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-21T15:22:59.771 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-21T15:22:59.802 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-21T15:23:00.277 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-21T15:23:00.310 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-21T15:23:00.342 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 128/170 2024-05-21T15:23:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 129/170 2024-05-21T15:23:00.435 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-21T15:23:01.528 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-21T15:23:01.572 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-21T15:23:02.594 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-21T15:23:02.622 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-21T15:23:02.731 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-21T15:23:02.867 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 143/170 2024-05-21T15:23:02.903 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 133/170 2024-05-21T15:23:02.909 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 144/170 2024-05-21T15:23:02.949 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-21T15:23:02.949 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 145/170 2024-05-21T15:23:02.975 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-21T15:23:02.975 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:23:02.975 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T15:23:02.975 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T15:23:02.975 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T15:23:02.975 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:23:02.981 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-21T15:23:03.009 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-21T15:23:03.033 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-21T15:23:03.045 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-21T15:23:03.147 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 136/170 2024-05-21T15:23:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 148/170 2024-05-21T15:23:03.205 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/170 2024-05-21T15:23:03.246 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-21T15:23:03.279 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 150/170 2024-05-21T15:23:03.320 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-21T15:23:03.349 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-21T15:23:03.387 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T15:23:04.016 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:23:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:23:04.081 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:23:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 143/170 2024-05-21T15:23:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 144/170 2024-05-21T15:23:04.176 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-21T15:23:04.202 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-21T15:23:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 145/170 2024-05-21T15:23:04.227 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-21T15:23:04.227 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:23:04.227 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T15:23:04.227 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T15:23:04.227 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T15:23:04.227 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:23:04.241 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-21T15:23:04.268 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-21T15:23:04.270 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-21T15:23:04.294 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-21T15:23:04.294 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:23:04.294 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T15:23:04.294 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:23:04.345 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-21T15:23:04.370 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-21T15:23:04.370 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:23:04.370 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-21T15:23:04.370 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T15:23:04.370 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T15:23:04.370 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:23:04.463 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 148/170 2024-05-21T15:23:04.514 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-21T15:23:04.547 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 150/170 2024-05-21T15:23:04.587 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-21T15:23:04.617 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-21T15:23:04.671 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T15:23:04.696 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-21T15:23:04.720 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-21T15:23:04.720 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:23:04.721 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T15:23:04.721 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T15:23:04.721 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T15:23:04.721 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:23:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 155/170 2024-05-21T15:23:05.091 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-21T15:23:05.134 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-21T15:23:05.218 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-21T15:23:05.253 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-21T15:23:05.289 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:23:05.299 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T15:23:05.327 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:23:05.328 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-21T15:23:05.347 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:23:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-21T15:23:05.355 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T15:23:05.440 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T15:23:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-21T15:23:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T15:23:05.557 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T15:23:05.582 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-21T15:23:05.588 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:23:05.588 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:23:05.776 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:23:05.776 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T15:23:05.852 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T15:23:05.877 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-21T15:23:05.943 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T15:23:06.258 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 155/170 2024-05-21T15:23:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-21T15:23:06.398 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-21T15:23:06.485 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-21T15:23:06.517 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-21T15:23:06.542 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T15:23:06.576 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-21T15:23:06.602 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-21T15:23:06.603 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T15:23:06.687 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T15:23:06.720 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-21T15:23:06.720 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T15:23:06.813 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T15:23:06.837 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-21T15:23:06.842 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:23:06.843 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:23:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:23:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T15:23:07.115 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T15:23:07.148 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-21T15:23:07.215 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T15:23:09.559 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 143/170 2024-05-21T15:23:09.600 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 144/170 2024-05-21T15:23:09.641 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 145/170 2024-05-21T15:23:09.689 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-21T15:23:09.716 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-21T15:23:09.914 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 148/170 2024-05-21T15:23:09.963 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-21T15:23:09.998 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 150/170 2024-05-21T15:23:10.044 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-21T15:23:10.074 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-21T15:23:10.119 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T15:23:10.731 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:23:10.767 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:23:10.789 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:23:11.581 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T15:23:11.581 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 1/170 2024-05-21T15:23:11.582 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2/170 2024-05-21T15:23:11.582 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 3/170 2024-05-21T15:23:11.582 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 4/170 2024-05-21T15:23:11.582 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 5/170 2024-05-21T15:23:11.582 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 6/170 2024-05-21T15:23:11.582 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 7/170 2024-05-21T15:23:11.582 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:23:11.582 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:23:11.582 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-21T15:23:11.582 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 11/170 2024-05-21T15:23:11.582 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 12/170 2024-05-21T15:23:11.582 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/170 2024-05-21T15:23:11.582 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:23:11.583 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 15/170 2024-05-21T15:23:11.583 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 16/170 2024-05-21T15:23:11.583 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 17/170 2024-05-21T15:23:11.583 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 18/170 2024-05-21T15:23:11.583 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 19/170 2024-05-21T15:23:11.583 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 20/170 2024-05-21T15:23:11.583 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 21/170 2024-05-21T15:23:11.583 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 22/170 2024-05-21T15:23:11.583 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 23/170 2024-05-21T15:23:11.583 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 24/170 2024-05-21T15:23:11.583 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 25/170 2024-05-21T15:23:11.583 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 26/170 2024-05-21T15:23:11.583 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 27/170 2024-05-21T15:23:11.584 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 28/170 2024-05-21T15:23:11.584 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 29/170 2024-05-21T15:23:11.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 30/170 2024-05-21T15:23:11.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 31/170 2024-05-21T15:23:11.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 32/170 2024-05-21T15:23:11.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 33/170 2024-05-21T15:23:11.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 34/170 2024-05-21T15:23:11.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 35/170 2024-05-21T15:23:11.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 36/170 2024-05-21T15:23:11.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 37/170 2024-05-21T15:23:11.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 38/170 2024-05-21T15:23:11.599 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 39/170 2024-05-21T15:23:11.599 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-21T15:23:11.599 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-21T15:23:11.599 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-21T15:23:11.599 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-21T15:23:11.599 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-21T15:23:11.599 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-21T15:23:11.599 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-21T15:23:11.599 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-21T15:23:11.599 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-21T15:23:11.599 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-21T15:23:11.599 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-21T15:23:11.599 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-21T15:23:11.600 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-21T15:23:11.600 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-21T15:23:11.600 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-21T15:23:11.600 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-21T15:23:11.600 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-21T15:23:11.600 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-21T15:23:11.600 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-21T15:23:11.600 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-21T15:23:11.600 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-21T15:23:11.600 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-21T15:23:11.600 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-21T15:23:11.601 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-21T15:23:11.601 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-21T15:23:11.601 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-21T15:23:11.601 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-21T15:23:11.601 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-21T15:23:11.601 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-21T15:23:11.601 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-21T15:23:11.601 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-21T15:23:11.601 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-21T15:23:11.601 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-21T15:23:11.601 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-21T15:23:11.602 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-21T15:23:11.602 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-21T15:23:11.602 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-21T15:23:11.602 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-21T15:23:11.602 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-21T15:23:11.602 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-21T15:23:11.602 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-21T15:23:11.602 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-21T15:23:11.602 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-21T15:23:11.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-21T15:23:11.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-21T15:23:11.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-21T15:23:11.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-21T15:23:11.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-21T15:23:11.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-21T15:23:11.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-21T15:23:11.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-21T15:23:11.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-21T15:23:11.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-21T15:23:11.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-21T15:23:11.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-21T15:23:11.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-21T15:23:11.603 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-21T15:23:11.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-21T15:23:11.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-21T15:23:11.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-21T15:23:11.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-21T15:23:11.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-21T15:23:11.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-21T15:23:11.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-21T15:23:11.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-21T15:23:11.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-21T15:23:11.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-21T15:23:11.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-21T15:23:11.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-21T15:23:11.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-21T15:23:11.604 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-21T15:23:11.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-21T15:23:11.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-21T15:23:11.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-21T15:23:11.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-21T15:23:11.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-21T15:23:11.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-21T15:23:11.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-21T15:23:11.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-21T15:23:11.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-21T15:23:11.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-21T15:23:11.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-21T15:23:11.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-21T15:23:11.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-21T15:23:11.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-21T15:23:11.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-21T15:23:11.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-21T15:23:11.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-21T15:23:11.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-21T15:23:11.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-21T15:23:11.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-21T15:23:11.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-21T15:23:11.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-21T15:23:11.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-21T15:23:11.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-21T15:23:11.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-21T15:23:11.606 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-21T15:23:11.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-21T15:23:11.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-21T15:23:11.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-21T15:23:11.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-21T15:23:11.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-21T15:23:11.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-21T15:23:11.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-21T15:23:11.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-21T15:23:11.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-21T15:23:11.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-21T15:23:11.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-21T15:23:11.607 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-21T15:23:11.608 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 149/170 2024-05-21T15:23:11.608 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-21T15:23:11.608 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 151/170 2024-05-21T15:23:11.608 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-21T15:23:11.608 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T15:23:11.608 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-21T15:23:11.608 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-21T15:23:11.608 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-21T15:23:11.608 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-21T15:23:11.608 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-21T15:23:11.608 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-21T15:23:11.608 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T15:23:11.609 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-21T15:23:11.609 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-21T15:23:11.609 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-21T15:23:11.609 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-21T15:23:11.609 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-21T15:23:11.609 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-21T15:23:11.609 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-21T15:23:11.609 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-21T15:23:11.609 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-21T15:23:11.723 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 155/170 2024-05-21T15:23:11.822 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-21T15:23:11.864 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-21T15:23:11.965 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-21T15:23:12.000 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-21T15:23:12.025 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T15:23:12.050 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-21T15:23:12.077 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-21T15:23:12.077 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T15:23:12.161 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T15:23:12.203 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-21T15:23:12.203 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T15:23:12.287 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T15:23:12.312 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-21T15:23:12.318 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:23:12.318 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:23:12.506 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:23:12.507 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T15:23:12.590 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T15:23:12.623 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-21T15:23:12.692 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T15:23:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T15:23:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 1/170 2024-05-21T15:23:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2/170 2024-05-21T15:23:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 3/170 2024-05-21T15:23:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 4/170 2024-05-21T15:23:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 5/170 2024-05-21T15:23:12.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 6/170 2024-05-21T15:23:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 7/170 2024-05-21T15:23:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:23:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:23:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-21T15:23:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 11/170 2024-05-21T15:23:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 12/170 2024-05-21T15:23:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/170 2024-05-21T15:23:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:23:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 15/170 2024-05-21T15:23:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 16/170 2024-05-21T15:23:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 17/170 2024-05-21T15:23:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 18/170 2024-05-21T15:23:12.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 19/170 2024-05-21T15:23:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 20/170 2024-05-21T15:23:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 21/170 2024-05-21T15:23:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 22/170 2024-05-21T15:23:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 23/170 2024-05-21T15:23:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 24/170 2024-05-21T15:23:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 25/170 2024-05-21T15:23:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 26/170 2024-05-21T15:23:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 27/170 2024-05-21T15:23:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 28/170 2024-05-21T15:23:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 29/170 2024-05-21T15:23:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 30/170 2024-05-21T15:23:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 31/170 2024-05-21T15:23:12.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 32/170 2024-05-21T15:23:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 33/170 2024-05-21T15:23:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 34/170 2024-05-21T15:23:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 35/170 2024-05-21T15:23:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 36/170 2024-05-21T15:23:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 37/170 2024-05-21T15:23:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 38/170 2024-05-21T15:23:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 39/170 2024-05-21T15:23:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-21T15:23:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-21T15:23:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-21T15:23:12.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-21T15:23:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-21T15:23:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-21T15:23:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-21T15:23:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-21T15:23:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-21T15:23:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-21T15:23:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-21T15:23:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-21T15:23:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-21T15:23:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-21T15:23:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-21T15:23:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-21T15:23:12.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-21T15:23:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-21T15:23:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-21T15:23:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-21T15:23:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-21T15:23:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-21T15:23:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-21T15:23:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-21T15:23:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-21T15:23:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-21T15:23:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-21T15:23:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-21T15:23:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-21T15:23:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-21T15:23:12.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-21T15:23:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-21T15:23:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-21T15:23:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-21T15:23:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-21T15:23:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-21T15:23:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-21T15:23:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-21T15:23:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-21T15:23:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-21T15:23:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-21T15:23:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-21T15:23:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-21T15:23:12.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-21T15:23:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-21T15:23:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-21T15:23:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-21T15:23:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-21T15:23:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-21T15:23:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-21T15:23:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-21T15:23:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-21T15:23:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-21T15:23:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-21T15:23:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-21T15:23:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-21T15:23:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-21T15:23:12.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-21T15:23:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-21T15:23:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-21T15:23:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-21T15:23:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-21T15:23:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-21T15:23:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-21T15:23:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-21T15:23:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-21T15:23:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-21T15:23:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-21T15:23:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-21T15:23:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-21T15:23:12.773 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-21T15:23:12.774 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-21T15:23:12.774 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-21T15:23:12.774 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-21T15:23:12.774 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-21T15:23:12.774 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-21T15:23:12.774 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-21T15:23:12.774 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-21T15:23:12.774 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-21T15:23:12.774 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-21T15:23:12.774 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-21T15:23:12.774 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-21T15:23:12.774 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-21T15:23:12.774 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-21T15:23:12.774 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-21T15:23:12.775 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-21T15:23:12.775 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-21T15:23:12.775 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-21T15:23:12.775 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-21T15:23:12.775 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-21T15:23:12.775 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-21T15:23:12.775 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-21T15:23:12.775 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-21T15:23:12.775 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-21T15:23:12.775 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-21T15:23:12.775 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-21T15:23:12.775 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-21T15:23:12.775 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-21T15:23:12.776 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-21T15:23:12.776 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-21T15:23:12.776 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-21T15:23:12.776 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-21T15:23:12.776 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-21T15:23:12.776 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-21T15:23:12.776 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-21T15:23:12.776 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-21T15:23:12.776 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-21T15:23:12.776 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-21T15:23:12.776 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-21T15:23:12.776 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 149/170 2024-05-21T15:23:12.776 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-21T15:23:12.776 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 151/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-21T15:23:12.777 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-21T15:23:12.778 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-21T15:23:12.778 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-21T15:23:12.778 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-21T15:23:14.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-21T15:23:14.057 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:23:14.057 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-05-21T15:23:14.057 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-21T15:23:14.057 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-21T15:23:14.057 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-21T15:23:14.058 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-21T15:23:14.058 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-21T15:23:14.058 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-21T15:23:14.058 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-21T15:23:14.058 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.058 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.058 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-21T15:23:14.058 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-21T15:23:14.058 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T15:23:14.058 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T15:23:14.058 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.2-8.el9.noarch 2024-05-21T15:23:14.058 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.7.4-5.el9.x86_64 2024-05-21T15:23:14.058 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T15:23:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:14.059 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:14.060 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-21T15:23:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.4-9.el9.x86_64 2024-05-21T15:23:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T15:23:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T15:23:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T15:23:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-21T15:23:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T15:23:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T15:23:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-21T15:23:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T15:23:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T15:23:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.061 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T15:23:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T15:23:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T15:23:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T15:23:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T15:23:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T15:23:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-21T15:23:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T15:23:14.062 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-21T15:23:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-21T15:23:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T15:23:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T15:23:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T15:23:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T15:23:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T15:23:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T15:23:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-21T15:23:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T15:23:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-13.14-1.el9.x86_64 2024-05-21T15:23:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-21T15:23:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-21T15:23:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-21T15:23:14.063 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T15:23:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T15:23:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T15:23:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-21T15:23:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T15:23:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T15:23:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T15:23:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T15:23:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T15:23:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T15:23:14.064 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T15:23:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-21T15:23:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-21T15:23:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T15:23:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T15:23:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T15:23:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T15:23:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T15:23:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T15:23:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T15:23:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T15:23:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T15:23:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T15:23:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T15:23:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T15:23:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T15:23:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T15:23:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T15:23:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T15:23:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T15:23:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T15:23:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T15:23:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T15:23:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T15:23:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T15:23:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T15:23:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T15:23:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T15:23:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T15:23:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T15:23:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T15:23:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T15:23:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T15:23:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T15:23:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T15:23:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-21T15:23:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T15:23:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T15:23:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T15:23:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T15:23:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T15:23:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T15:23:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T15:23:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T15:23:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T15:23:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T15:23:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T15:23:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-21T15:23:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T15:23:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T15:23:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T15:23:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T15:23:14.068 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T15:23:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T15:23:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T15:23:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T15:23:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T15:23:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T15:23:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-21T15:23:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-21T15:23:14.069 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T15:23:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-21T15:23:14.070 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T15:23:14.070 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T15:23:14.494 DEBUG:teuthology.parallel:result is None 2024-05-21T15:23:15.315 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-21T15:23:15.315 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:23:15.315 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-05-21T15:23:15.315 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-21T15:23:15.315 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-21T15:23:15.316 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-21T15:23:15.316 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-21T15:23:15.316 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-21T15:23:15.316 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-21T15:23:15.316 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-21T15:23:15.316 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.316 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.316 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-21T15:23:15.316 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-21T15:23:15.316 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-05-21T15:23:15.316 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T15:23:15.316 INFO:teuthology.orchestra.run.smithi053.stdout: automake-1.16.2-8.el9.noarch 2024-05-21T15:23:15.316 INFO:teuthology.orchestra.run.smithi053.stdout: bison-3.7.4-5.el9.x86_64 2024-05-21T15:23:15.317 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T15:23:15.317 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.317 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.317 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.317 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.317 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:15.317 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.317 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.317 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.317 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:15.317 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:15.317 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:15.317 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:15.318 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:15.318 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.318 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.318 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:15.318 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.318 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.318 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.318 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:15.318 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:15.318 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.318 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.318 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:15.319 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-21T15:23:15.319 INFO:teuthology.orchestra.run.smithi053.stdout: flex-2.6.4-9.el9.x86_64 2024-05-21T15:23:15.319 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T15:23:15.319 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T15:23:15.319 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T15:23:15.319 INFO:teuthology.orchestra.run.smithi053.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-21T15:23:15.319 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T15:23:15.319 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T15:23:15.319 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-21T15:23:15.319 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T15:23:15.319 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T15:23:15.319 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.319 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.320 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.320 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T15:23:15.320 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T15:23:15.320 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T15:23:15.320 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T15:23:15.320 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T15:23:15.320 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.320 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.320 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T15:23:15.321 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.321 INFO:teuthology.orchestra.run.smithi053.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-21T15:23:15.321 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T15:23:15.321 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-21T15:23:15.321 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-21T15:23:15.321 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T15:23:15.321 INFO:teuthology.orchestra.run.smithi053.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T15:23:15.321 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T15:23:15.321 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T15:23:15.321 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T15:23:15.321 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T15:23:15.321 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-21T15:23:15.321 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T15:23:15.322 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-13.14-1.el9.x86_64 2024-05-21T15:23:15.322 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-21T15:23:15.322 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-21T15:23:15.322 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-21T15:23:15.322 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T15:23:15.322 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T15:23:15.322 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T15:23:15.322 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-21T15:23:15.322 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T15:23:15.322 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T15:23:15.322 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T15:23:15.322 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.322 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.323 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.323 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T15:23:15.323 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T15:23:15.323 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T15:23:15.323 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T15:23:15.323 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-21T15:23:15.323 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-21T15:23:15.323 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T15:23:15.323 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T15:23:15.323 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T15:23:15.323 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T15:23:15.323 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T15:23:15.323 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T15:23:15.323 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T15:23:15.324 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T15:23:15.324 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T15:23:15.324 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T15:23:15.324 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T15:23:15.324 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T15:23:15.324 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T15:23:15.324 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T15:23:15.324 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T15:23:15.324 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T15:23:15.324 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T15:23:15.324 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T15:23:15.324 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T15:23:15.324 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T15:23:15.325 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T15:23:15.325 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T15:23:15.325 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T15:23:15.325 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T15:23:15.325 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T15:23:15.325 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T15:23:15.325 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T15:23:15.325 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T15:23:15.325 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T15:23:15.325 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T15:23:15.325 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T15:23:15.325 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T15:23:15.325 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-21T15:23:15.325 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T15:23:15.326 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.326 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.326 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T15:23:15.326 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T15:23:15.326 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T15:23:15.326 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.326 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T15:23:15.326 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T15:23:15.326 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T15:23:15.326 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T15:23:15.326 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T15:23:15.326 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T15:23:15.327 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T15:23:15.327 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-21T15:23:15.327 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T15:23:15.327 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T15:23:15.327 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T15:23:15.327 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T15:23:15.327 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T15:23:15.327 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T15:23:15.327 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T15:23:15.327 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.327 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.327 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:15.327 INFO:teuthology.orchestra.run.smithi053.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T15:23:15.327 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T15:23:15.328 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T15:23:15.328 INFO:teuthology.orchestra.run.smithi053.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-21T15:23:15.328 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-21T15:23:15.328 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T15:23:15.328 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-21T15:23:15.328 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:23:15.328 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-05-21T15:23:15.725 DEBUG:teuthology.parallel:result is None 2024-05-21T15:23:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T15:23:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 1/170 2024-05-21T15:23:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2/170 2024-05-21T15:23:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 3/170 2024-05-21T15:23:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 4/170 2024-05-21T15:23:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 5/170 2024-05-21T15:23:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 6/170 2024-05-21T15:23:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 7/170 2024-05-21T15:23:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:23:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:23:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-21T15:23:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 11/170 2024-05-21T15:23:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 12/170 2024-05-21T15:23:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/170 2024-05-21T15:23:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:23:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 15/170 2024-05-21T15:23:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 16/170 2024-05-21T15:23:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 17/170 2024-05-21T15:23:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 18/170 2024-05-21T15:23:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 19/170 2024-05-21T15:23:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 20/170 2024-05-21T15:23:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 21/170 2024-05-21T15:23:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 22/170 2024-05-21T15:23:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 23/170 2024-05-21T15:23:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 24/170 2024-05-21T15:23:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 25/170 2024-05-21T15:23:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 26/170 2024-05-21T15:23:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 27/170 2024-05-21T15:23:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 28/170 2024-05-21T15:23:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 29/170 2024-05-21T15:23:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 30/170 2024-05-21T15:23:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 31/170 2024-05-21T15:23:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 32/170 2024-05-21T15:23:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 33/170 2024-05-21T15:23:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 34/170 2024-05-21T15:23:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 35/170 2024-05-21T15:23:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 36/170 2024-05-21T15:23:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 37/170 2024-05-21T15:23:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 38/170 2024-05-21T15:23:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 39/170 2024-05-21T15:23:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-21T15:23:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-21T15:23:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-21T15:23:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-21T15:23:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-21T15:23:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-21T15:23:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-21T15:23:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-21T15:23:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-21T15:23:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-21T15:23:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-21T15:23:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-21T15:23:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-21T15:23:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-21T15:23:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-21T15:23:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-21T15:23:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-21T15:23:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-21T15:23:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-21T15:23:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-21T15:23:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-21T15:23:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-21T15:23:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-21T15:23:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-21T15:23:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-21T15:23:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-21T15:23:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-21T15:23:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-21T15:23:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-21T15:23:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-21T15:23:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-21T15:23:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-21T15:23:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-21T15:23:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-21T15:23:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-21T15:23:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-21T15:23:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-21T15:23:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-21T15:23:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-21T15:23:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-21T15:23:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-21T15:23:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-21T15:23:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-21T15:23:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-21T15:23:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-21T15:23:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-21T15:23:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-21T15:23:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-21T15:23:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-21T15:23:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-21T15:23:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-21T15:23:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-21T15:23:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-21T15:23:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-21T15:23:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-21T15:23:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-21T15:23:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-21T15:23:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-21T15:23:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-21T15:23:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-21T15:23:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-21T15:23:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-21T15:23:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-21T15:23:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-21T15:23:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-21T15:23:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-21T15:23:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-21T15:23:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-21T15:23:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-21T15:23:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-21T15:23:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-21T15:23:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-21T15:23:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-21T15:23:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-21T15:23:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-21T15:23:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-21T15:23:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-21T15:23:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-21T15:23:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-21T15:23:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-21T15:23:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-21T15:23:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-21T15:23:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-21T15:23:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-21T15:23:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-21T15:23:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-21T15:23:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-21T15:23:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-21T15:23:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-21T15:23:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-21T15:23:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-21T15:23:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-21T15:23:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-21T15:23:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-21T15:23:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-21T15:23:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-21T15:23:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-21T15:23:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-21T15:23:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-21T15:23:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-21T15:23:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-21T15:23:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-21T15:23:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-21T15:23:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-21T15:23:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-21T15:23:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-21T15:23:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-21T15:23:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-21T15:23:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-21T15:23:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 149/170 2024-05-21T15:23:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-21T15:23:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 151/170 2024-05-21T15:23:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-21T15:23:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T15:23:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-21T15:23:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-21T15:23:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-21T15:23:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-21T15:23:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-21T15:23:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-21T15:23:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T15:23:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-21T15:23:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-21T15:23:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-21T15:23:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-21T15:23:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-21T15:23:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-21T15:23:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-21T15:23:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-21T15:23:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-21T15:23:21.488 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-21T15:23:21.489 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:23:21.489 INFO:teuthology.orchestra.run.smithi038.stdout:Upgraded: 2024-05-21T15:23:21.489 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-21T15:23:21.489 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-21T15:23:21.489 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-21T15:23:21.489 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-21T15:23:21.489 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-21T15:23:21.489 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-21T15:23:21.489 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-21T15:23:21.489 INFO:teuthology.orchestra.run.smithi038.stdout: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.489 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.490 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-21T15:23:21.490 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-21T15:23:21.490 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-05-21T15:23:21.490 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T15:23:21.490 INFO:teuthology.orchestra.run.smithi038.stdout: automake-1.16.2-8.el9.noarch 2024-05-21T15:23:21.490 INFO:teuthology.orchestra.run.smithi038.stdout: bison-3.7.4-5.el9.x86_64 2024-05-21T15:23:21.490 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T15:23:21.490 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.490 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.490 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.490 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.490 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:21.491 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.491 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.491 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.491 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:21.491 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:21.491 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:21.491 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:21.491 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:21.491 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.491 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.491 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:21.491 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.491 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.492 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.492 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:21.492 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:21.492 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.492 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.492 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:23:21.492 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-21T15:23:21.492 INFO:teuthology.orchestra.run.smithi038.stdout: flex-2.6.4-9.el9.x86_64 2024-05-21T15:23:21.492 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T15:23:21.492 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T15:23:21.492 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T15:23:21.492 INFO:teuthology.orchestra.run.smithi038.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-21T15:23:21.492 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T15:23:21.492 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T15:23:21.493 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-21T15:23:21.493 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T15:23:21.493 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T15:23:21.493 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.493 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.493 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.493 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T15:23:21.493 INFO:teuthology.orchestra.run.smithi038.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T15:23:21.493 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T15:23:21.493 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T15:23:21.493 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T15:23:21.493 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.493 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.494 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T15:23:21.494 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.494 INFO:teuthology.orchestra.run.smithi038.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-21T15:23:21.494 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T15:23:21.494 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-21T15:23:21.494 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-21T15:23:21.494 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T15:23:21.494 INFO:teuthology.orchestra.run.smithi038.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T15:23:21.494 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T15:23:21.494 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T15:23:21.494 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T15:23:21.494 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T15:23:21.494 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-21T15:23:21.495 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T15:23:21.495 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-13.14-1.el9.x86_64 2024-05-21T15:23:21.495 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-21T15:23:21.495 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-21T15:23:21.495 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-21T15:23:21.495 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T15:23:21.495 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T15:23:21.495 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T15:23:21.495 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-21T15:23:21.495 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T15:23:21.495 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T15:23:21.495 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T15:23:21.495 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.495 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T15:23:21.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T15:23:21.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T15:23:21.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T15:23:21.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-21T15:23:21.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-21T15:23:21.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T15:23:21.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T15:23:21.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T15:23:21.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T15:23:21.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T15:23:21.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T15:23:21.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T15:23:21.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T15:23:21.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T15:23:21.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T15:23:21.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T15:23:21.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T15:23:21.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T15:23:21.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T15:23:21.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T15:23:21.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T15:23:21.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T15:23:21.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T15:23:21.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T15:23:21.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T15:23:21.498 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T15:23:21.498 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T15:23:21.498 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T15:23:21.498 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T15:23:21.498 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T15:23:21.498 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T15:23:21.498 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T15:23:21.498 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T15:23:21.498 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T15:23:21.498 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T15:23:21.498 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T15:23:21.498 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T15:23:21.498 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-21T15:23:21.499 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T15:23:21.499 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.499 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.499 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T15:23:21.499 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T15:23:21.499 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T15:23:21.499 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.499 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T15:23:21.499 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T15:23:21.499 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T15:23:21.499 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T15:23:21.499 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T15:23:21.499 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T15:23:21.499 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T15:23:21.500 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-21T15:23:21.500 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T15:23:21.500 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T15:23:21.500 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T15:23:21.500 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T15:23:21.500 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T15:23:21.500 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T15:23:21.500 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T15:23:21.500 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.500 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.500 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:23:21.500 INFO:teuthology.orchestra.run.smithi038.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T15:23:21.500 INFO:teuthology.orchestra.run.smithi038.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T15:23:21.501 INFO:teuthology.orchestra.run.smithi038.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T15:23:21.501 INFO:teuthology.orchestra.run.smithi038.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-21T15:23:21.501 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-21T15:23:21.501 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T15:23:21.501 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-21T15:23:21.501 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:23:21.501 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-05-21T15:23:21.927 DEBUG:teuthology.parallel:result is None 2024-05-21T15:23:21.928 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:23:22.081 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T15:23:22.106 INFO:teuthology.orchestra.run.smithi038.stdout:19.0.0-2744.gd2b507c7.el9 2024-05-21T15:23:22.107 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2744.gd2b507c7.el9 2024-05-21T15:23:22.107 INFO:teuthology.task.install:The correct ceph version 19.0.0-2744.gd2b507c7 is installed. 2024-05-21T15:23:22.109 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:23:22.245 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T15:23:22.272 INFO:teuthology.orchestra.run.smithi045.stdout:19.0.0-2744.gd2b507c7.el9 2024-05-21T15:23:22.272 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2744.gd2b507c7.el9 2024-05-21T15:23:22.272 INFO:teuthology.task.install:The correct ceph version 19.0.0-2744.gd2b507c7 is installed. 2024-05-21T15:23:22.275 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:23:22.517 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T15:23:22.544 INFO:teuthology.orchestra.run.smithi053.stdout:19.0.0-2744.gd2b507c7.el9 2024-05-21T15:23:22.544 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2744.gd2b507c7.el9 2024-05-21T15:23:22.544 INFO:teuthology.task.install:The correct ceph version 19.0.0-2744.gd2b507c7 is installed. 2024-05-21T15:23:22.546 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-21T15:23:22.546 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:23:22.546 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T15:23:22.580 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T15:23:22.580 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T15:23:22.612 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-21T15:23:22.612 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T15:23:22.643 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-21T15:23:22.644 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:23:22.644 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T15:23:22.674 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T15:23:22.744 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T15:23:22.744 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T15:23:22.775 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T15:23:22.843 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-21T15:23:22.843 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T15:23:22.873 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T15:23:22.942 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-21T15:23:22.943 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:23:22.943 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T15:23:22.972 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T15:23:23.041 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T15:23:23.041 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T15:23:23.071 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T15:23:23.140 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-21T15:23:23.140 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T15:23:23.169 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T15:23:23.237 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-21T15:23:23.238 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:23:23.238 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T15:23:23.268 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T15:23:23.337 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T15:23:23.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T15:23:23.366 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T15:23:23.436 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-21T15:23:23.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T15:23:23.465 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T15:23:23.534 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-21T15:23:23.632 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'subvols': {'create': 2}} 2024-05-21T15:23:23.632 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:23:23.633 INFO:tasks.cephadm:Cluster fsid is 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:23:23.633 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-21T15:23:23.633 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.38', 'mon.b': '172.21.15.45', 'mon.c': '172.21.15.53'} 2024-05-21T15:23:23.633 INFO:tasks.cephadm:First mon is mon.a on smithi038 2024-05-21T15:23:23.633 INFO:tasks.cephadm:First mgr is x 2024-05-21T15:23:23.633 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-21T15:23:23.634 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname $(hostname -s) 2024-05-21T15:23:23.665 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2024-05-21T15:23:23.694 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname $(hostname -s) 2024-05-21T15:23:23.724 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-21T15:23:23.724 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:23:23.864 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79503/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2744-gd2b507c7', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2744.gd2b507c7'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-17 22:05:47.367530', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240517.203521-squid-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-21T15:23:24.080 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240517.203521-squid-debug, sha1 d2b507c724ef6c8295ee0a35d185cdab167fd61d from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:23:24.082 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/x86_64/flavors/default/cephadm 2024-05-21T15:23:24.082 DEBUG:teuthology.orchestra.run.smithi038:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T15:23:24.540 INFO:teuthology.orchestra.run.smithi038.stdout:-rw-r--r--. 1 ubuntu ubuntu 782070 May 21 15:23 /home/ubuntu/cephtest/cephadm 2024-05-21T15:23:24.541 DEBUG:teuthology.orchestra.run.smithi045:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T15:23:25.156 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r--. 1 ubuntu ubuntu 782070 May 21 15:23 /home/ubuntu/cephtest/cephadm 2024-05-21T15:23:25.156 DEBUG:teuthology.orchestra.run.smithi053:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T15:23:25.817 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-r--r--. 1 ubuntu ubuntu 782070 May 21 15:23 /home/ubuntu/cephtest/cephadm 2024-05-21T15:23:25.817 DEBUG:teuthology.orchestra.run.smithi038:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T15:23:25.837 DEBUG:teuthology.orchestra.run.smithi045:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T15:23:25.857 DEBUG:teuthology.orchestra.run.smithi053:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T15:23:25.892 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d on all hosts... 2024-05-21T15:23:25.892 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d pull 2024-05-21T15:23:25.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d pull 2024-05-21T15:23:25.900 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d pull 2024-05-21T15:23:26.721 INFO:teuthology.orchestra.run.smithi053.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-21T15:23:26.826 INFO:teuthology.orchestra.run.smithi038.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-21T15:23:26.857 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-21T15:23:56.202 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:23:56.202 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-21T15:23:56.202 INFO:teuthology.orchestra.run.smithi038.stdout: "image_id": "cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3", 2024-05-21T15:23:56.202 INFO:teuthology.orchestra.run.smithi038.stdout: "repo_digests": [ 2024-05-21T15:23:56.202 INFO:teuthology.orchestra.run.smithi038.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708" 2024-05-21T15:23:56.202 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-05-21T15:23:56.202 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:23:57.860 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-21T15:23:57.860 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-21T15:23:57.860 INFO:teuthology.orchestra.run.smithi053.stdout: "image_id": "cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3", 2024-05-21T15:23:57.861 INFO:teuthology.orchestra.run.smithi053.stdout: "repo_digests": [ 2024-05-21T15:23:57.861 INFO:teuthology.orchestra.run.smithi053.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708" 2024-05-21T15:23:57.861 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-05-21T15:23:57.861 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-21T15:24:03.844 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-21T15:24:03.844 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-21T15:24:03.844 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3", 2024-05-21T15:24:03.845 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2024-05-21T15:24:03.845 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708" 2024-05-21T15:24:03.845 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-05-21T15:24:03.845 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-21T15:24:03.870 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph 2024-05-21T15:24:03.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2024-05-21T15:24:03.934 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph 2024-05-21T15:24:03.968 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 777 /etc/ceph 2024-05-21T15:24:03.998 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2024-05-21T15:24:04.023 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /etc/ceph 2024-05-21T15:24:04.052 INFO:tasks.cephadm:Writing seed config... 2024-05-21T15:24:04.053 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-21T15:24:04.053 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-21T15:24:04.053 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-21T15:24:04.053 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-21T15:24:04.053 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-21T15:24:04.053 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-21T15:24:04.053 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-21T15:24:04.053 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-21T15:24:04.054 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-21T15:24:04.054 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-21T15:24:04.054 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-21T15:24:04.054 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-21T15:24:04.054 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-21T15:24:04.054 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-21T15:24:04.054 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-21T15:24:04.054 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-21T15:24:04.054 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-21T15:24:04.054 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-21T15:24:04.054 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-21T15:24:04.054 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-21T15:24:04.054 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-21T15:24:04.055 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-21T15:24:04.055 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-21T15:24:04.055 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-21T15:24:04.055 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-21T15:24:04.055 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-21T15:24:04.055 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-21T15:24:04.055 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-21T15:24:04.055 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-21T15:24:04.055 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-21T15:24:04.055 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-21T15:24:04.055 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-21T15:24:04.055 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-21T15:24:04.055 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-21T15:24:04.055 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-21T15:24:04.056 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-21T15:24:04.056 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-21T15:24:04.056 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-05-21T15:24:04.056 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-21T15:24:04.056 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-21T15:24:04.056 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-21T15:24:04.056 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-21T15:24:04.056 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-05-21T15:24:04.056 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-21T15:24:04.056 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-21T15:24:04.056 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-21T15:24:04.056 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-21T15:24:04.057 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:24:04.057 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-21T15:24:04.075 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 102ec4fa-1786-11ef-bc9a-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-21T15:24:04.076 DEBUG:teuthology.orchestra.run.smithi038:mon.a> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.a.service 2024-05-21T15:24:04.119 DEBUG:teuthology.orchestra.run.smithi038:mgr.x> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.x.service 2024-05-21T15:24:04.162 INFO:tasks.cephadm:Bootstrapping... 2024-05-21T15:24:04.162 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d -v bootstrap --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.38 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:24:04.364 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-05-21T15:24:04.364 INFO:teuthology.orchestra.run.smithi038.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d', '-v', 'bootstrap', '--fsid', '102ec4fa-1786-11ef-bc9a-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.38', '--skip-admin-label'] 2024-05-21T15:24:04.364 INFO:teuthology.orchestra.run.smithi038.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-21T15:24:04.364 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying podman|docker is present... 2024-05-21T15:24:04.391 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout 5.0.2 2024-05-21T15:24:04.391 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying lvm2 is present... 2024-05-21T15:24:04.391 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying time synchronization is in place... 2024-05-21T15:24:04.399 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-21T15:24:04.399 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-21T15:24:04.405 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-21T15:24:04.405 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-05-21T15:24:04.412 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout enabled 2024-05-21T15:24:04.419 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout active 2024-05-21T15:24:04.419 INFO:teuthology.orchestra.run.smithi038.stdout:Unit chronyd.service is enabled and running 2024-05-21T15:24:04.419 INFO:teuthology.orchestra.run.smithi038.stdout:Repeating the final host check... 2024-05-21T15:24:04.444 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout 5.0.2 2024-05-21T15:24:04.444 INFO:teuthology.orchestra.run.smithi038.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-21T15:24:04.444 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl is present 2024-05-21T15:24:04.444 INFO:teuthology.orchestra.run.smithi038.stdout:lvcreate is present 2024-05-21T15:24:04.450 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-21T15:24:04.450 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-21T15:24:04.456 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-21T15:24:04.457 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-05-21T15:24:04.463 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout enabled 2024-05-21T15:24:04.470 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout active 2024-05-21T15:24:04.470 INFO:teuthology.orchestra.run.smithi038.stdout:Unit chronyd.service is enabled and running 2024-05-21T15:24:04.470 INFO:teuthology.orchestra.run.smithi038.stdout:Host looks OK 2024-05-21T15:24:04.470 INFO:teuthology.orchestra.run.smithi038.stdout:Cluster fsid: 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:24:04.470 INFO:teuthology.orchestra.run.smithi038.stdout:Acquiring lock 140134671644032 on /run/cephadm/102ec4fa-1786-11ef-bc9a-c7b262605968.lock 2024-05-21T15:24:04.471 INFO:teuthology.orchestra.run.smithi038.stdout:Lock 140134671644032 acquired on /run/cephadm/102ec4fa-1786-11ef-bc9a-c7b262605968.lock 2024-05-21T15:24:04.471 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying IP 172.21.15.38 port 3300 ... 2024-05-21T15:24:04.471 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying IP 172.21.15.38 port 6789 ... 2024-05-21T15:24:04.471 INFO:teuthology.orchestra.run.smithi038.stdout:Base mon IP(s) is [172.21.15.38:3300, 172.21.15.38:6789], mon addrv is [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-05-21T15:24:04.475 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.38 metric 100 2024-05-21T15:24:04.475 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.38 metric 100 2024-05-21T15:24:04.477 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-21T15:24:04.478 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-21T15:24:04.478 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-21T15:24:04.480 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-21T15:24:04.480 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-21T15:24:04.480 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-21T15:24:04.480 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-21T15:24:04.481 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-05-21T15:24:04.481 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-21T15:24:04.481 INFO:teuthology.orchestra.run.smithi038.stdout:Mon IP `172.21.15.38` is in CIDR network `172.21.0.0/20` 2024-05-21T15:24:04.481 INFO:teuthology.orchestra.run.smithi038.stdout:Mon IP `172.21.15.38` is in CIDR network `172.21.0.0/20` 2024-05-21T15:24:04.481 INFO:teuthology.orchestra.run.smithi038.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-21T15:24:04.482 INFO:teuthology.orchestra.run.smithi038.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-21T15:24:04.483 INFO:teuthology.orchestra.run.smithi038.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-21T15:24:04.928 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 2024-05-21T15:24:04.928 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-21T15:24:04.928 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Getting image source signatures 2024-05-21T15:24:04.928 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-21T15:24:04.928 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying blob sha256:506d33f63dccdaac2c87f708e3d3ef98b8f2fc00666d50b72896243431ca9bae 2024-05-21T15:24:04.928 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying config sha256:cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 2024-05-21T15:24:04.928 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-21T15:24:05.834 INFO:teuthology.orchestra.run.smithi038.stdout:ceph: stdout ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev) 2024-05-21T15:24:05.834 INFO:teuthology.orchestra.run.smithi038.stdout:Ceph version: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev) 2024-05-21T15:24:05.834 INFO:teuthology.orchestra.run.smithi038.stdout:Extracting ceph user uid/gid from container image... 2024-05-21T15:24:06.806 INFO:teuthology.orchestra.run.smithi038.stdout:stat: stdout 167 167 2024-05-21T15:24:06.807 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial keys... 2024-05-21T15:24:07.662 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQCXvExm2y24CBAAPd9/YBAoFKCx7FJuSSz7rQ== 2024-05-21T15:24:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQCYvExmoqOMABAA/2qf+YK0Gq/80hmEehanzQ== 2024-05-21T15:24:14.611 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQCZvExmrjddBBAAABbN8xOibA/DPpY0JntPRg== 2024-05-21T15:24:14.612 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial monmap... 2024-05-21T15:24:21.659 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-21T15:24:21.659 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-21T15:24:21.659 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:24:21.660 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-21T15:24:21.660 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool for a [v2:172.21.15.38:3300,v1:172.21.15.38:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-21T15:24:21.660 INFO:teuthology.orchestra.run.smithi038.stdout:setting min_mon_release = quincy 2024-05-21T15:24:21.660 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: set fsid to 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:24:21.660 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-21T15:24:21.660 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:24:21.660 INFO:teuthology.orchestra.run.smithi038.stdout:Creating mon... 2024-05-21T15:24:22.640 INFO:teuthology.orchestra.run.smithi038.stdout:create mon.a on 2024-05-21T15:24:22.863 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-21T15:24:23.010 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-21T15:24:23.216 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-102ec4fa-1786-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-102ec4fa-1786-11ef-bc9a-c7b262605968.target. 2024-05-21T15:24:23.216 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-102ec4fa-1786-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-102ec4fa-1786-11ef-bc9a-c7b262605968.target. 2024-05-21T15:24:23.442 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.a 2024-05-21T15:24:23.442 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to reset failed state of unit ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.a.service: Unit ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.a.service not loaded. 2024-05-21T15:24:23.612 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-102ec4fa-1786-11ef-bc9a-c7b262605968.target.wants/ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.a.service → /etc/systemd/system/ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@.service. 2024-05-21T15:24:24.113 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T15:24:24.114 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-05-21T15:24:24.121 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T15:24:24.121 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-05-21T15:24:24.121 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-05-21T15:24:24.121 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to enable service . firewalld.service is not available 2024-05-21T15:24:24.121 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mon to start... 2024-05-21T15:24:24.121 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mon... 2024-05-21T15:24:25.279 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:25 smithi038 ceph-mon[31626]: from='client.? 172.21.15.38:0/1094936694' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-21T15:24:25.314 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout cluster: 2024-05-21T15:24:25.315 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout id: 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:24:25.315 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-21T15:24:25.315 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-05-21T15:24:25.315 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout services: 2024-05-21T15:24:25.315 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.588778s) 2024-05-21T15:24:25.315 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-21T15:24:25.315 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-21T15:24:25.315 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-05-21T15:24:25.315 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout data: 2024-05-21T15:24:25.315 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-21T15:24:25.315 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-21T15:24:25.315 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-21T15:24:25.315 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout pgs: 2024-05-21T15:24:25.316 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-05-21T15:24:25.316 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.587+0000 7fb8767d8700 1 Processor -- start 2024-05-21T15:24:25.316 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.587+0000 7fb8767d8700 1 -- start start 2024-05-21T15:24:25.316 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.588+0000 7fb8767d8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb870107af0 0x7fb870107f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:25.316 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.588+0000 7fb8767d8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701084f0 con 0x7fb870107af0 2024-05-21T15:24:25.316 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.588+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb870107af0 0x7fb870107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:25.316 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.588+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb870107af0 0x7fb870107f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57912/0 (socket says 172.21.15.38:57912) 2024-05-21T15:24:25.316 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.588+0000 7fb86ffff700 1 -- 172.21.15.38:0/1404759403 learned_addr learned my addr 172.21.15.38:0/1404759403 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:25.316 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.588+0000 7fb86ffff700 1 -- 172.21.15.38:0/1404759403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb870108c20 con 0x7fb870107af0 2024-05-21T15:24:25.316 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.589+0000 7fb86ffff700 1 --2- 172.21.15.38:0/1404759403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb870107af0 0x7fb870107f00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb86000bb40 tx=0x7fb86000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb92d92cfdc3444 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:25.316 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.589+0000 7fb86f7fe700 1 -- 172.21.15.38:0/1404759403 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb86000dbe0 con 0x7fb870107af0 2024-05-21T15:24:25.316 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.589+0000 7fb86f7fe700 1 -- 172.21.15.38:0/1404759403 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb86000dd80 con 0x7fb870107af0 2024-05-21T15:24:25.317 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.589+0000 7fb86f7fe700 1 -- 172.21.15.38:0/1404759403 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb860004180 con 0x7fb870107af0 2024-05-21T15:24:25.317 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.590+0000 7fb8767d8700 1 -- 172.21.15.38:0/1404759403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb870107af0 msgr2=0x7fb870107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:25.317 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.590+0000 7fb8767d8700 1 --2- 172.21.15.38:0/1404759403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb870107af0 0x7fb870107f00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb86000bb40 tx=0x7fb86000be50 comp rx=0 tx=0).stop 2024-05-21T15:24:25.317 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.590+0000 7fb8767d8700 1 -- 172.21.15.38:0/1404759403 shutdown_connections 2024-05-21T15:24:25.317 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.590+0000 7fb8767d8700 1 --2- 172.21.15.38:0/1404759403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb870107af0 0x7fb870107f00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:25.317 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.590+0000 7fb8767d8700 1 -- 172.21.15.38:0/1404759403 >> 172.21.15.38:0/1404759403 conn(0x7fb8701031b0 msgr2=0x7fb870105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:25.317 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.590+0000 7fb8767d8700 1 -- 172.21.15.38:0/1404759403 shutdown_connections 2024-05-21T15:24:25.317 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.590+0000 7fb8767d8700 1 -- 172.21.15.38:0/1404759403 wait complete. 2024-05-21T15:24:25.317 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.590+0000 7fb8767d8700 1 Processor -- start 2024-05-21T15:24:25.317 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.590+0000 7fb8767d8700 1 -- start start 2024-05-21T15:24:25.317 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.591+0000 7fb8767d8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb870107af0 0x7fb87019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:25.317 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.591+0000 7fb8767d8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701090f0 con 0x7fb870107af0 2024-05-21T15:24:25.318 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.591+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb870107af0 0x7fb87019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:25.318 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.591+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb870107af0 0x7fb87019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57922/0 (socket says 172.21.15.38:57922) 2024-05-21T15:24:25.318 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.591+0000 7fb86ffff700 1 -- 172.21.15.38:0/1094936694 learned_addr learned my addr 172.21.15.38:0/1094936694 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:25.318 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.591+0000 7fb86ffff700 1 -- 172.21.15.38:0/1094936694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87019aa00 con 0x7fb870107af0 2024-05-21T15:24:25.318 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.591+0000 7fb86ffff700 1 --2- 172.21.15.38:0/1094936694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb870107af0 0x7fb87019a4c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb86000b340 tx=0x7fb860004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:25.318 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.591+0000 7fb86dffb700 1 -- 172.21.15.38:0/1094936694 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb860029070 con 0x7fb870107af0 2024-05-21T15:24:25.318 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.591+0000 7fb86dffb700 1 -- 172.21.15.38:0/1094936694 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8600044e0 con 0x7fb870107af0 2024-05-21T15:24:25.318 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.591+0000 7fb8767d8700 1 -- 172.21.15.38:0/1094936694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87019ac90 con 0x7fb870107af0 2024-05-21T15:24:25.318 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.591+0000 7fb86dffb700 1 -- 172.21.15.38:0/1094936694 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8600105a0 con 0x7fb870107af0 2024-05-21T15:24:25.318 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.592+0000 7fb8767d8700 1 -- 172.21.15.38:0/1094936694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87019d8a0 con 0x7fb870107af0 2024-05-21T15:24:25.318 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.592+0000 7fb86dffb700 1 -- 172.21.15.38:0/1094936694 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb86002c070 con 0x7fb870107af0 2024-05-21T15:24:25.318 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.592+0000 7fb86dffb700 1 -- 172.21.15.38:0/1094936694 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb860021070 con 0x7fb870107af0 2024-05-21T15:24:25.319 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.592+0000 7fb8767d8700 1 -- 172.21.15.38:0/1094936694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb870063ca0 con 0x7fb870107af0 2024-05-21T15:24:25.319 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.595+0000 7fb86dffb700 1 -- 172.21.15.38:0/1094936694 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb860015070 con 0x7fb870107af0 2024-05-21T15:24:25.319 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.668+0000 7fb8767d8700 1 -- 172.21.15.38:0/1094936694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb87002dc60 con 0x7fb870107af0 2024-05-21T15:24:25.319 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.669+0000 7fb86dffb700 1 -- 172.21.15.38:0/1094936694 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fb87002dc60 con 0x7fb870107af0 2024-05-21T15:24:25.319 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.669+0000 7fb8767d8700 1 -- 172.21.15.38:0/1094936694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb870107af0 msgr2=0x7fb87019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:25.319 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.669+0000 7fb8767d8700 1 --2- 172.21.15.38:0/1094936694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb870107af0 0x7fb87019a4c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb86000b340 tx=0x7fb860004750 comp rx=0 tx=0).stop 2024-05-21T15:24:25.319 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.669+0000 7fb8767d8700 1 -- 172.21.15.38:0/1094936694 shutdown_connections 2024-05-21T15:24:25.319 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.669+0000 7fb8767d8700 1 --2- 172.21.15.38:0/1094936694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb870107af0 0x7fb87019a4c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:25.319 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.669+0000 7fb8767d8700 1 -- 172.21.15.38:0/1094936694 >> 172.21.15.38:0/1094936694 conn(0x7fb8701031b0 msgr2=0x7fb870190c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:25.319 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.670+0000 7fb8767d8700 1 -- 172.21.15.38:0/1094936694 shutdown_connections 2024-05-21T15:24:25.320 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:24.670+0000 7fb8767d8700 1 -- 172.21.15.38:0/1094936694 wait complete. 2024-05-21T15:24:25.320 INFO:teuthology.orchestra.run.smithi038.stdout:mon is available 2024-05-21T15:24:25.320 INFO:teuthology.orchestra.run.smithi038.stdout:Assimilating anything we can from ceph.conf... 2024-05-21T15:24:26.323 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-05-21T15:24:26.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [global] 2024-05-21T15:24:26.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout fsid = 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:24:26.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-05-21T15:24:26.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-21T15:24:26.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-21T15:24:26.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-21T15:24:26.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-21T15:24:26.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-05-21T15:24:26.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-21T15:24:26.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-21T15:24:26.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-05-21T15:24:26.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [osd] 2024-05-21T15:24:26.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-05-21T15:24:26.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-21T15:24:26.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-21T15:24:26.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-21T15:24:26.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-21T15:24:26.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-21T15:24:26.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.746+0000 7f4b9128b700 1 Processor -- start 2024-05-21T15:24:26.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.746+0000 7f4b9128b700 1 -- start start 2024-05-21T15:24:26.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.746+0000 7f4b9128b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b8c107b80 0x7f4b8c107f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.746+0000 7f4b9128b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c108580 con 0x7f4b8c107b80 2024-05-21T15:24:26.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.747+0000 7f4b8affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b8c107b80 0x7f4b8c107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.747+0000 7f4b8affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b8c107b80 0x7f4b8c107f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54026/0 (socket says 172.21.15.38:54026) 2024-05-21T15:24:26.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.747+0000 7f4b8affd700 1 -- 172.21.15.38:0/3595107789 learned_addr learned my addr 172.21.15.38:0/3595107789 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:26.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.747+0000 7f4b8affd700 1 -- 172.21.15.38:0/3595107789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c108cb0 con 0x7f4b8c107b80 2024-05-21T15:24:26.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.747+0000 7f4b8affd700 1 --2- 172.21.15.38:0/3595107789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b8c107b80 0x7f4b8c107f90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009e80 tx=0x7f4b7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34523862b29fae96 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:26.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.747+0000 7f4b8a7fc700 1 -- 172.21.15.38:0/3595107789 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b7c017070 con 0x7f4b8c107b80 2024-05-21T15:24:26.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.747+0000 7f4b8a7fc700 1 -- 172.21.15.38:0/3595107789 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b7c003a20 con 0x7f4b8c107b80 2024-05-21T15:24:26.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.747+0000 7f4b8a7fc700 1 -- 172.21.15.38:0/3595107789 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b7c003d20 con 0x7f4b8c107b80 2024-05-21T15:24:26.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.748+0000 7f4b9128b700 1 -- 172.21.15.38:0/3595107789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b8c107b80 msgr2=0x7f4b8c107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.748+0000 7f4b9128b700 1 --2- 172.21.15.38:0/3595107789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b8c107b80 0x7f4b8c107f90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009e80 tx=0x7f4b7c00b330 comp rx=0 tx=0).stop 2024-05-21T15:24:26.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.748+0000 7f4b9128b700 1 -- 172.21.15.38:0/3595107789 shutdown_connections 2024-05-21T15:24:26.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.748+0000 7f4b9128b700 1 --2- 172.21.15.38:0/3595107789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b8c107b80 0x7f4b8c107f90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.748+0000 7f4b9128b700 1 -- 172.21.15.38:0/3595107789 >> 172.21.15.38:0/3595107789 conn(0x7f4b8c103220 msgr2=0x7f4b8c105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:26.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.748+0000 7f4b9128b700 1 -- 172.21.15.38:0/3595107789 shutdown_connections 2024-05-21T15:24:26.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.748+0000 7f4b9128b700 1 -- 172.21.15.38:0/3595107789 wait complete. 2024-05-21T15:24:26.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.749+0000 7f4b9128b700 1 Processor -- start 2024-05-21T15:24:26.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.749+0000 7f4b9128b700 1 -- start start 2024-05-21T15:24:26.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.749+0000 7f4b9128b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b8c107b80 0x7f4b8c07c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.749+0000 7f4b9128b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c1091e0 con 0x7f4b8c107b80 2024-05-21T15:24:26.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.750+0000 7f4b8affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b8c107b80 0x7f4b8c07c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.750+0000 7f4b8affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b8c107b80 0x7f4b8c07c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54028/0 (socket says 172.21.15.38:54028) 2024-05-21T15:24:26.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.750+0000 7f4b8affd700 1 -- 172.21.15.38:0/1502350792 learned_addr learned my addr 172.21.15.38:0/1502350792 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:26.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.750+0000 7f4b8affd700 1 -- 172.21.15.38:0/1502350792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c07c730 con 0x7f4b8c107b80 2024-05-21T15:24:26.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.750+0000 7f4b8affd700 1 --2- 172.21.15.38:0/1502350792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b8c107b80 0x7f4b8c07c1f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c004000 tx=0x7f4b7c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:26.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.750+0000 7f4b88ff9700 1 -- 172.21.15.38:0/1502350792 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b7c022020 con 0x7f4b8c107b80 2024-05-21T15:24:26.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.750+0000 7f4b88ff9700 1 -- 172.21.15.38:0/1502350792 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b7c018040 con 0x7f4b8c107b80 2024-05-21T15:24:26.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.750+0000 7f4b9128b700 1 -- 172.21.15.38:0/1502350792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8c19f540 con 0x7f4b8c107b80 2024-05-21T15:24:26.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.750+0000 7f4b88ff9700 1 -- 172.21.15.38:0/1502350792 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b7c017070 con 0x7f4b8c107b80 2024-05-21T15:24:26.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.750+0000 7f4b9128b700 1 -- 172.21.15.38:0/1502350792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b8c19fa20 con 0x7f4b8c107b80 2024-05-21T15:24:26.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.751+0000 7f4b88ff9700 1 -- 172.21.15.38:0/1502350792 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4b7c025020 con 0x7f4b8c107b80 2024-05-21T15:24:26.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.751+0000 7f4b88ff9700 1 -- 172.21.15.38:0/1502350792 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4b7c00f050 con 0x7f4b8c107b80 2024-05-21T15:24:26.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.752+0000 7f4b9128b700 1 -- 172.21.15.38:0/1502350792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b8c063ca0 con 0x7f4b8c107b80 2024-05-21T15:24:26.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.754+0000 7f4b88ff9700 1 -- 172.21.15.38:0/1502350792 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4b7c02c030 con 0x7f4b8c107b80 2024-05-21T15:24:26.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.824+0000 7f4b9128b700 1 -- 172.21.15.38:0/1502350792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f4b8c02dfb0 con 0x7f4b8c107b80 2024-05-21T15:24:26.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.829+0000 7f4b88ff9700 1 -- 172.21.15.38:0/1502350792 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+454 (secure 0 0 0) 0x7f4b7c01ed90 con 0x7f4b8c107b80 2024-05-21T15:24:26.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.830+0000 7f4b88ff9700 1 -- 172.21.15.38:0/1502350792 <== mon.0 v2:172.21.15.38:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4b7c035070 con 0x7f4b8c107b80 2024-05-21T15:24:26.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.830+0000 7f4b9128b700 1 -- 172.21.15.38:0/1502350792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b8c107b80 msgr2=0x7f4b8c07c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.830+0000 7f4b9128b700 1 --2- 172.21.15.38:0/1502350792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b8c107b80 0x7f4b8c07c1f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c004000 tx=0x7f4b7c004ad0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.830+0000 7f4b9128b700 1 -- 172.21.15.38:0/1502350792 shutdown_connections 2024-05-21T15:24:26.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.830+0000 7f4b9128b700 1 --2- 172.21.15.38:0/1502350792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b8c107b80 0x7f4b8c07c1f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.830+0000 7f4b9128b700 1 -- 172.21.15.38:0/1502350792 >> 172.21.15.38:0/1502350792 conn(0x7f4b8c103220 msgr2=0x7f4b8c191ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:26.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.830+0000 7f4b9128b700 1 -- 172.21.15.38:0/1502350792 shutdown_connections 2024-05-21T15:24:26.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:25.830+0000 7f4b9128b700 1 -- 172.21.15.38:0/1502350792 wait complete. 2024-05-21T15:24:26.335 INFO:teuthology.orchestra.run.smithi038.stdout:Generating new minimal ceph.conf... 2024-05-21T15:24:26.446 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:26 smithi038 ceph-mon[31626]: from='client.? 172.21.15.38:0/1502350792' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-21T15:24:26.446 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:26 smithi038 ceph-mon[31626]: from='client.? 172.21.15.38:0/1502350792' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-21T15:24:28.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:27 smithi038 ceph-mon[31626]: from='client.? 172.21.15.38:0/3351649349' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:24:30.162 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.759+0000 7fb08cc44700 1 Processor -- start 2024-05-21T15:24:30.162 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.759+0000 7fb08cc44700 1 -- start start 2024-05-21T15:24:30.162 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.759+0000 7fb08cc44700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb088107b80 0x7fb088107f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:30.162 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.759+0000 7fb08cc44700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb088108580 con 0x7fb088107b80 2024-05-21T15:24:30.163 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.759+0000 7fb08659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb088107b80 0x7fb088107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:30.163 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.759+0000 7fb08659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb088107b80 0x7fb088107f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54038/0 (socket says 172.21.15.38:54038) 2024-05-21T15:24:30.163 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.759+0000 7fb08659c700 1 -- 172.21.15.38:0/2593307465 learned_addr learned my addr 172.21.15.38:0/2593307465 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:30.163 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.760+0000 7fb08659c700 1 -- 172.21.15.38:0/2593307465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb088108cb0 con 0x7fb088107b80 2024-05-21T15:24:30.163 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.760+0000 7fb08659c700 1 --2- 172.21.15.38:0/2593307465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb088107b80 0x7fb088107f90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb0780098a0 tx=0x7fb078009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80124b2461fc3a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:30.163 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.760+0000 7fb08559a700 1 -- 172.21.15.38:0/2593307465 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb078004160 con 0x7fb088107b80 2024-05-21T15:24:30.163 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.760+0000 7fb08559a700 1 -- 172.21.15.38:0/2593307465 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb078004300 con 0x7fb088107b80 2024-05-21T15:24:30.163 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.760+0000 7fb08559a700 1 -- 172.21.15.38:0/2593307465 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb078011b50 con 0x7fb088107b80 2024-05-21T15:24:30.163 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.761+0000 7fb08cc44700 1 -- 172.21.15.38:0/2593307465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb088107b80 msgr2=0x7fb088107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:30.163 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.761+0000 7fb08cc44700 1 --2- 172.21.15.38:0/2593307465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb088107b80 0x7fb088107f90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb0780098a0 tx=0x7fb078009bb0 comp rx=0 tx=0).stop 2024-05-21T15:24:30.163 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.761+0000 7fb08cc44700 1 -- 172.21.15.38:0/2593307465 shutdown_connections 2024-05-21T15:24:30.163 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.761+0000 7fb08cc44700 1 --2- 172.21.15.38:0/2593307465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb088107b80 0x7fb088107f90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:30.164 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.761+0000 7fb08cc44700 1 -- 172.21.15.38:0/2593307465 >> 172.21.15.38:0/2593307465 conn(0x7fb088103220 msgr2=0x7fb088105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:30.164 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.761+0000 7fb08cc44700 1 -- 172.21.15.38:0/2593307465 shutdown_connections 2024-05-21T15:24:30.164 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.761+0000 7fb08cc44700 1 -- 172.21.15.38:0/2593307465 wait complete. 2024-05-21T15:24:30.164 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.762+0000 7fb08cc44700 1 Processor -- start 2024-05-21T15:24:30.164 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.762+0000 7fb08cc44700 1 -- start start 2024-05-21T15:24:30.164 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.762+0000 7fb08cc44700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb088107b80 0x7fb08819d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:30.164 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.762+0000 7fb08cc44700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb08810c060 con 0x7fb088107b80 2024-05-21T15:24:30.164 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.762+0000 7fb08659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb088107b80 0x7fb08819d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:30.164 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.762+0000 7fb08659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb088107b80 0x7fb08819d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54040/0 (socket says 172.21.15.38:54040) 2024-05-21T15:24:30.165 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.762+0000 7fb08659c700 1 -- 172.21.15.38:0/3351649349 learned_addr learned my addr 172.21.15.38:0/3351649349 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:30.165 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.762+0000 7fb08659c700 1 -- 172.21.15.38:0/3351649349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08819d610 con 0x7fb088107b80 2024-05-21T15:24:30.165 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.763+0000 7fb08659c700 1 --2- 172.21.15.38:0/3351649349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb088107b80 0x7fb08819d0d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb078011710 tx=0x7fb078011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:30.165 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.763+0000 7fb0777fe700 1 -- 172.21.15.38:0/3351649349 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0780038b0 con 0x7fb088107b80 2024-05-21T15:24:30.165 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.763+0000 7fb0777fe700 1 -- 172.21.15.38:0/3351649349 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb078011e50 con 0x7fb088107b80 2024-05-21T15:24:30.165 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.763+0000 7fb08cc44700 1 -- 172.21.15.38:0/3351649349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08819d8a0 con 0x7fb088107b80 2024-05-21T15:24:30.165 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.763+0000 7fb0777fe700 1 -- 172.21.15.38:0/3351649349 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0780193c0 con 0x7fb088107b80 2024-05-21T15:24:30.165 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.763+0000 7fb08cc44700 1 -- 172.21.15.38:0/3351649349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0881a04b0 con 0x7fb088107b80 2024-05-21T15:24:30.165 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.763+0000 7fb0777fe700 1 -- 172.21.15.38:0/3351649349 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb078019a10 con 0x7fb088107b80 2024-05-21T15:24:30.165 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.763+0000 7fb0777fe700 1 -- 172.21.15.38:0/3351649349 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb0780243f0 con 0x7fb088107b80 2024-05-21T15:24:30.165 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.764+0000 7fb08cc44700 1 -- 172.21.15.38:0/3351649349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb088063ca0 con 0x7fb088107b80 2024-05-21T15:24:30.165 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.766+0000 7fb0777fe700 1 -- 172.21.15.38:0/3351649349 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb07801a430 con 0x7fb088107b80 2024-05-21T15:24:30.166 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.837+0000 7fb08cc44700 1 -- 172.21.15.38:0/3351649349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb0881a0680 con 0x7fb088107b80 2024-05-21T15:24:30.166 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.837+0000 7fb0777fe700 1 -- 172.21.15.38:0/3351649349 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fb078032030 con 0x7fb088107b80 2024-05-21T15:24:30.166 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.838+0000 7fb08cc44700 1 -- 172.21.15.38:0/3351649349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb088107b80 msgr2=0x7fb08819d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:30.166 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.838+0000 7fb08cc44700 1 --2- 172.21.15.38:0/3351649349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb088107b80 0x7fb08819d0d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb078011710 tx=0x7fb078011c10 comp rx=0 tx=0).stop 2024-05-21T15:24:30.166 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.838+0000 7fb08cc44700 1 -- 172.21.15.38:0/3351649349 shutdown_connections 2024-05-21T15:24:30.166 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.838+0000 7fb08cc44700 1 --2- 172.21.15.38:0/3351649349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb088107b80 0x7fb08819d0d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:30.166 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.838+0000 7fb08cc44700 1 -- 172.21.15.38:0/3351649349 >> 172.21.15.38:0/3351649349 conn(0x7fb088103220 msgr2=0x7fb088103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:30.166 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.838+0000 7fb08cc44700 1 -- 172.21.15.38:0/3351649349 shutdown_connections 2024-05-21T15:24:30.166 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:27.838+0000 7fb08cc44700 1 -- 172.21.15.38:0/3351649349 wait complete. 2024-05-21T15:24:30.166 INFO:teuthology.orchestra.run.smithi038.stdout:Restarting the monitor... 2024-05-21T15:24:30.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:30 smithi038 systemd[1]: Stopping Ceph mon.a for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:24:30.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:30 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a[31622]: 2024-05-21T15:24:30.429+0000 7feb62056700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:24:30.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:30 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a[31622]: 2024-05-21T15:24:30.429+0000 7feb62056700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-21T15:24:30.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:30 smithi038 podman[31873]: 2024-05-21 15:24:30.614579147 +0000 UTC m=+0.279372837 container died 3e13f646c45806abf32f962ffd3e2903e7ca7019584110b82997342dad6b213d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:24:31.125 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:30 smithi038 podman[31873]: 2024-05-21 15:24:30.937798023 +0000 UTC m=+0.602591707 container cleanup 3e13f646c45806abf32f962ffd3e2903e7ca7019584110b82997342dad6b213d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-21T15:24:31.125 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:30 smithi038 bash[31873]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a 2024-05-21T15:24:31.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 podman[31887]: 2024-05-21 15:24:31.122234062 +0000 UTC m=+0.504069622 container remove 3e13f646c45806abf32f962ffd3e2903e7ca7019584110b82997342dad6b213d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-21T15:24:31.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 systemd[1]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-21T15:24:31.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 systemd[1]: Stopped Ceph mon.a for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:24:31.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 systemd[1]: Starting Ceph mon.a for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:24:31.818 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 podman[31976]: 2024-05-21 15:24:31.564857976 +0000 UTC m=+0.114006057 container create df34e728985d64b8cb94871454f459374e35a1ecad3d31ae2cfdcc627ae65131 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2) 2024-05-21T15:24:31.818 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 podman[31976]: 2024-05-21 15:24:31.468538727 +0000 UTC m=+0.017686781 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:24:31.819 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 podman[31976]: 2024-05-21 15:24:31.76721075 +0000 UTC m=+0.316358789 container init df34e728985d64b8cb94871454f459374e35a1ecad3d31ae2cfdcc627ae65131 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:24:31.819 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 podman[31976]: 2024-05-21 15:24:31.770404259 +0000 UTC m=+0.319552298 container start df34e728985d64b8cb94871454f459374e35a1ecad3d31ae2cfdcc627ae65131 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-21T15:24:31.819 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: set uid:gid to 167:167 (ceph:ceph) 2024-05-21T15:24:31.819 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev), process ceph-mon, pid 2 2024-05-21T15:24:31.819 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: pidfile_write: ignore empty --pid-file 2024-05-21T15:24:31.819 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: load: jerasure load: lrc 2024-05-21T15:24:31.819 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: RocksDB version: 7.9.2 2024-05-21T15:24:31.819 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Git sha 0 2024-05-21T15:24:31.819 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Compile date 2024-05-17 21:21:31 2024-05-21T15:24:31.819 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: DB SUMMARY 2024-05-21T15:24:31.819 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: DB Session ID: IHOF1SK7J2YXVBP769DZ 2024-05-21T15:24:31.819 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: CURRENT file: CURRENT 2024-05-21T15:24:31.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: IDENTITY file: IDENTITY 2024-05-21T15:24:31.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-21T15:24:31.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-21T15:24:31.820 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104726 ; 2024-05-21T15:24:31.821 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.error_if_exists: 0 2024-05-21T15:24:31.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.create_if_missing: 0 2024-05-21T15:24:31.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.paranoid_checks: 1 2024-05-21T15:24:31.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-21T15:24:31.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-21T15:24:31.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-21T15:24:31.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.env: 0x555a1491b4a0 2024-05-21T15:24:31.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.fs: PosixFileSystem 2024-05-21T15:24:31.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.info_log: 0x555a15b95d40 2024-05-21T15:24:31.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_file_opening_threads: 16 2024-05-21T15:24:31.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.statistics: (nil) 2024-05-21T15:24:31.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.use_fsync: 0 2024-05-21T15:24:31.822 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_log_file_size: 0 2024-05-21T15:24:31.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-21T15:24:31.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-21T15:24:31.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.keep_log_file_num: 1000 2024-05-21T15:24:31.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.recycle_log_file_num: 0 2024-05-21T15:24:31.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.allow_fallocate: 1 2024-05-21T15:24:31.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.allow_mmap_reads: 0 2024-05-21T15:24:31.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.allow_mmap_writes: 0 2024-05-21T15:24:31.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.use_direct_reads: 0 2024-05-21T15:24:31.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-21T15:24:31.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.create_missing_column_families: 0 2024-05-21T15:24:31.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.db_log_dir: 2024-05-21T15:24:31.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.wal_dir: 2024-05-21T15:24:31.823 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-21T15:24:31.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-21T15:24:31.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-21T15:24:31.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-21T15:24:31.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-21T15:24:31.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-21T15:24:31.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.advise_random_on_open: 1 2024-05-21T15:24:31.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.db_write_buffer_size: 0 2024-05-21T15:24:31.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.write_buffer_manager: 0x555a15b692c0 2024-05-21T15:24:31.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-21T15:24:31.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-21T15:24:31.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-21T15:24:31.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.rate_limiter: (nil) 2024-05-21T15:24:31.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-21T15:24:31.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.wal_recovery_mode: 2 2024-05-21T15:24:31.825 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.enable_thread_tracking: 0 2024-05-21T15:24:31.825 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.enable_pipelined_write: 0 2024-05-21T15:24:31.825 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.unordered_write: 0 2024-05-21T15:24:31.825 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-21T15:24:31.825 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-21T15:24:31.825 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-21T15:24:31.825 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-21T15:24:31.825 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.row_cache: None 2024-05-21T15:24:31.825 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.wal_filter: None 2024-05-21T15:24:31.825 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-21T15:24:31.825 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.allow_ingest_behind: 0 2024-05-21T15:24:31.826 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.two_write_queues: 0 2024-05-21T15:24:31.826 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.manual_wal_flush: 0 2024-05-21T15:24:31.826 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.wal_compression: 0 2024-05-21T15:24:31.826 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.atomic_flush: 0 2024-05-21T15:24:31.826 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-21T15:24:31.826 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-21T15:24:31.826 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-21T15:24:31.826 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.log_readahead_size: 0 2024-05-21T15:24:31.826 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-21T15:24:31.826 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.best_efforts_recovery: 0 2024-05-21T15:24:31.826 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-21T15:24:31.826 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-21T15:24:31.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.allow_data_in_errors: 0 2024-05-21T15:24:31.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.db_host_id: __hostname__ 2024-05-21T15:24:31.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-21T15:24:31.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_background_jobs: 2 2024-05-21T15:24:31.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_background_compactions: -1 2024-05-21T15:24:31.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_subcompactions: 1 2024-05-21T15:24:31.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-21T15:24:31.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-21T15:24:31.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-21T15:24:31.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_total_wal_size: 0 2024-05-21T15:24:31.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-21T15:24:31.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-21T15:24:31.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-21T15:24:31.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-21T15:24:31.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_open_files: -1 2024-05-21T15:24:31.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.bytes_per_sync: 0 2024-05-21T15:24:31.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-21T15:24:31.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-21T15:24:31.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compaction_readahead_size: 0 2024-05-21T15:24:31.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_background_flushes: -1 2024-05-21T15:24:31.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Compression algorithms supported: 2024-05-21T15:24:31.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-21T15:24:31.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: kZSTD supported: 0 2024-05-21T15:24:31.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: kXpressCompression supported: 0 2024-05-21T15:24:31.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: kLZ4HCCompression supported: 1 2024-05-21T15:24:31.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: kZlibCompression supported: 1 2024-05-21T15:24:31.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: kSnappyCompression supported: 1 2024-05-21T15:24:31.829 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: kLZ4Compression supported: 1 2024-05-21T15:24:31.829 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: kBZip2Compression supported: 0 2024-05-21T15:24:31.829 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-21T15:24:31.829 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-21T15:24:31.829 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-21T15:24:31.829 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-21T15:24:31.829 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-21T15:24:31.829 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.merge_operator: 2024-05-21T15:24:31.829 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compaction_filter: None 2024-05-21T15:24:31.829 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compaction_filter_factory: None 2024-05-21T15:24:31.829 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.sst_partitioner_factory: None 2024-05-21T15:24:31.829 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-21T15:24:31.830 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-21T15:24:31.830 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555a15b95e40) 2024-05-21T15:24:31.830 INFO:journalctl@ceph.mon.a.smithi038.stdout: cache_index_and_filter_blocks: 1 2024-05-21T15:24:31.830 INFO:journalctl@ceph.mon.a.smithi038.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-21T15:24:31.830 INFO:journalctl@ceph.mon.a.smithi038.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-21T15:24:31.830 INFO:journalctl@ceph.mon.a.smithi038.stdout: pin_top_level_index_and_filter: 1 2024-05-21T15:24:31.830 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_type: 0 2024-05-21T15:24:31.830 INFO:journalctl@ceph.mon.a.smithi038.stdout: data_block_index_type: 0 2024-05-21T15:24:31.830 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_shortening: 1 2024-05-21T15:24:31.830 INFO:journalctl@ceph.mon.a.smithi038.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-21T15:24:31.830 INFO:journalctl@ceph.mon.a.smithi038.stdout: checksum: 4 2024-05-21T15:24:31.830 INFO:journalctl@ceph.mon.a.smithi038.stdout: no_block_cache: 0 2024-05-21T15:24:31.831 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache: 0x555a15b99090 2024-05-21T15:24:31.831 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_name: BinnedLRUCache 2024-05-21T15:24:31.831 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_options: 2024-05-21T15:24:31.831 INFO:journalctl@ceph.mon.a.smithi038.stdout: capacity : 536870912 2024-05-21T15:24:31.831 INFO:journalctl@ceph.mon.a.smithi038.stdout: num_shard_bits : 4 2024-05-21T15:24:31.831 INFO:journalctl@ceph.mon.a.smithi038.stdout: strict_capacity_limit : 0 2024-05-21T15:24:31.831 INFO:journalctl@ceph.mon.a.smithi038.stdout: high_pri_pool_ratio: 0.000 2024-05-21T15:24:31.831 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_compressed: (nil) 2024-05-21T15:24:31.831 INFO:journalctl@ceph.mon.a.smithi038.stdout: persistent_cache: (nil) 2024-05-21T15:24:31.831 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_size: 4096 2024-05-21T15:24:31.831 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_size_deviation: 10 2024-05-21T15:24:31.832 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_restart_interval: 16 2024-05-21T15:24:31.832 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_block_restart_interval: 1 2024-05-21T15:24:31.832 INFO:journalctl@ceph.mon.a.smithi038.stdout: metadata_block_size: 4096 2024-05-21T15:24:31.832 INFO:journalctl@ceph.mon.a.smithi038.stdout: partition_filters: 0 2024-05-21T15:24:31.832 INFO:journalctl@ceph.mon.a.smithi038.stdout: use_delta_encoding: 1 2024-05-21T15:24:31.832 INFO:journalctl@ceph.mon.a.smithi038.stdout: filter_policy: bloomfilter 2024-05-21T15:24:31.832 INFO:journalctl@ceph.mon.a.smithi038.stdout: whole_key_filtering: 1 2024-05-21T15:24:31.832 INFO:journalctl@ceph.mon.a.smithi038.stdout: verify_compression: 0 2024-05-21T15:24:31.832 INFO:journalctl@ceph.mon.a.smithi038.stdout: read_amp_bytes_per_bit: 0 2024-05-21T15:24:31.832 INFO:journalctl@ceph.mon.a.smithi038.stdout: format_version: 5 2024-05-21T15:24:31.832 INFO:journalctl@ceph.mon.a.smithi038.stdout: enable_index_compression: 1 2024-05-21T15:24:31.832 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_align: 0 2024-05-21T15:24:31.832 INFO:journalctl@ceph.mon.a.smithi038.stdout: max_auto_readahead_size: 262144 2024-05-21T15:24:31.832 INFO:journalctl@ceph.mon.a.smithi038.stdout: prepopulate_block_cache: 0 2024-05-21T15:24:31.833 INFO:journalctl@ceph.mon.a.smithi038.stdout: initial_auto_readahead_size: 8192 2024-05-21T15:24:31.833 INFO:journalctl@ceph.mon.a.smithi038.stdout: num_file_reads_for_auto_readahead: 2 2024-05-21T15:24:31.833 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.write_buffer_size: 33554432 2024-05-21T15:24:31.833 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_write_buffer_number: 2 2024-05-21T15:24:31.833 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compression: NoCompression 2024-05-21T15:24:31.833 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.bottommost_compression: Disabled 2024-05-21T15:24:31.833 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.prefix_extractor: nullptr 2024-05-21T15:24:31.833 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-21T15:24:31.833 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.num_levels: 7 2024-05-21T15:24:31.833 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-21T15:24:31.833 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-21T15:24:31.833 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-21T15:24:31.833 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-21T15:24:31.833 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compression_opts.level: 32767 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compression_opts.strategy: 0 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-21T15:24:31.834 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compression_opts.enabled: false 2024-05-21T15:24:31.835 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-21T15:24:31.835 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-21T15:24:31.835 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-21T15:24:31.835 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-21T15:24:31.835 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.target_file_size_base: 67108864 2024-05-21T15:24:31.835 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-21T15:24:31.835 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-21T15:24:31.835 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-21T15:24:31.835 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-21T15:24:31.835 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-21T15:24:31.835 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-21T15:24:31.835 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-21T15:24:31.835 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-21T15:24:31.835 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-21T15:24:31.836 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-21T15:24:31.836 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-21T15:24:31.836 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-21T15:24:31.836 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-21T15:24:31.836 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-21T15:24:31.836 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.arena_block_size: 1048576 2024-05-21T15:24:31.836 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-21T15:24:31.836 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-21T15:24:31.836 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.disable_auto_compactions: 0 2024-05-21T15:24:31.836 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-21T15:24:31.836 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-21T15:24:31.836 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-21T15:24:31.836 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-21T15:24:31.836 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-21T15:24:31.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-21T15:24:31.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-21T15:24:31.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-21T15:24:31.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-21T15:24:31.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-21T15:24:31.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.table_properties_collectors: 2024-05-21T15:24:31.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.inplace_update_support: 0 2024-05-21T15:24:31.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-21T15:24:31.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-21T15:24:31.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-21T15:24:31.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-21T15:24:31.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.bloom_locality: 0 2024-05-21T15:24:31.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.max_successive_merges: 0 2024-05-21T15:24:31.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-21T15:24:31.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.paranoid_file_checks: 0 2024-05-21T15:24:31.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.force_consistency_checks: 1 2024-05-21T15:24:31.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.report_bg_io_stats: 0 2024-05-21T15:24:31.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.ttl: 2592000 2024-05-21T15:24:31.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-21T15:24:31.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-21T15:24:31.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-21T15:24:31.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.enable_blob_files: false 2024-05-21T15:24:31.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.min_blob_size: 0 2024-05-21T15:24:31.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.blob_file_size: 268435456 2024-05-21T15:24:31.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-21T15:24:31.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-21T15:24:31.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-21T15:24:31.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-21T15:24:31.839 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-21T15:24:31.839 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.blob_file_starting_level: 0 2024-05-21T15:24:31.839 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-21T15:24:31.839 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-21T15:24:31.839 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-21T15:24:31.839 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 168fdf5e-5a20-461e-a773-b923f79dfe12 2024-05-21T15:24:31.839 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716305071810652, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-21T15:24:31.839 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-21T15:24:31.839 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716305071811959, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99819, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97733, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14171, "raw_average_key_size": 49, "raw_value_size": 90060, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716305071, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "168fdf5e-5a20-461e-a773-b923f79dfe12", "db_session_id": "IHOF1SK7J2YXVBP769DZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-21T15:24:31.839 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716305071812044, "job": 1, "event": "recovery_finished"} 2024-05-21T15:24:31.839 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-21T15:24:31.839 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-21T15:24:31.839 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555a15b7fc00 2024-05-21T15:24:31.839 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: DB pointer 0x555a1a96c000 2024-05-21T15:24:31.840 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-21T15:24:31.840 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-21T15:24:31.840 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** DB Stats ** 2024-05-21T15:24:31.840 INFO:journalctl@ceph.mon.a.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T15:24:31.840 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-21T15:24:31.840 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T15:24:31.840 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T15:24:31.840 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-21T15:24:31.840 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T15:24:31.840 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T15:24:31.840 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-05-21T15:24:31.840 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** Compaction Stats [default] ** 2024-05-21T15:24:31.840 INFO:journalctl@ceph.mon.a.smithi038.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T15:24:31.840 INFO:journalctl@ceph.mon.a.smithi038.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-21T15:24:31.841 INFO:journalctl@ceph.mon.a.smithi038.stdout: L0 2/0 99.31 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T15:24:31.841 INFO:journalctl@ceph.mon.a.smithi038.stdout: Sum 2/0 99.31 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T15:24:31.841 INFO:journalctl@ceph.mon.a.smithi038.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T15:24:31.841 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-05-21T15:24:31.841 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** Compaction Stats [default] ** 2024-05-21T15:24:31.841 INFO:journalctl@ceph.mon.a.smithi038.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T15:24:31.841 INFO:journalctl@ceph.mon.a.smithi038.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-21T15:24:31.841 INFO:journalctl@ceph.mon.a.smithi038.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 130.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T15:24:31.841 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-05-21T15:24:31.841 INFO:journalctl@ceph.mon.a.smithi038.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-21T15:24:31.841 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-05-21T15:24:31.842 INFO:journalctl@ceph.mon.a.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T15:24:31.842 INFO:journalctl@ceph.mon.a.smithi038.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-21T15:24:31.842 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-21T15:24:31.842 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-21T15:24:31.842 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-21T15:24:31.842 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-21T15:24:31.842 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative compaction: 0.00 GB write, 16.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T15:24:31.842 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval compaction: 0.00 GB write, 16.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T15:24:31.842 INFO:journalctl@ceph.mon.a.smithi038.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-21T15:24:31.842 INFO:journalctl@ceph.mon.a.smithi038.stdout: Block cache BinnedLRUCache@0x555a15b99090#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.7e-05 secs_since: 0 2024-05-21T15:24:31.842 INFO:journalctl@ceph.mon.a.smithi038.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-21T15:24:31.842 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-05-21T15:24:31.842 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-21T15:24:31.843 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: starting mon.a rank 0 at public addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] at bind addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:24:31.843 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: mon.a@-1(???) e1 preinit fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:24:31.843 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: mon.a@-1(???).mds e1 new map 2024-05-21T15:24:31.843 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: mon.a@-1(???).mds e1 print_map 2024-05-21T15:24:31.843 INFO:journalctl@ceph.mon.a.smithi038.stdout: e1 2024-05-21T15:24:31.843 INFO:journalctl@ceph.mon.a.smithi038.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-21T15:24:31.843 INFO:journalctl@ceph.mon.a.smithi038.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T15:24:31.843 INFO:journalctl@ceph.mon.a.smithi038.stdout: legacy client fscid: -1 2024-05-21T15:24:31.843 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-05-21T15:24:31.843 INFO:journalctl@ceph.mon.a.smithi038.stdout: No filesystems configured 2024-05-21T15:24:31.843 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-21T15:24:31.843 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T15:24:31.843 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T15:24:31.843 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T15:24:31.844 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-21T15:24:31.844 INFO:teuthology.orchestra.run.smithi038.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-21T15:24:32.073 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 bash[31976]: df34e728985d64b8cb94871454f459374e35a1ecad3d31ae2cfdcc627ae65131 2024-05-21T15:24:32.073 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 systemd[1]: Started Ceph mon.a for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:24:32.073 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-21T15:24:32.073 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: monmap epoch 1 2024-05-21T15:24:32.073 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:24:32.073 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: last_changed 2024-05-21T15:24:17.361696+0000 2024-05-21T15:24:32.073 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: created 2024-05-21T15:24:17.361696+0000 2024-05-21T15:24:32.073 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: min_mon_release 19 (squid) 2024-05-21T15:24:32.073 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: election_strategy: 1 2024-05-21T15:24:32.073 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-05-21T15:24:32.074 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: fsmap 2024-05-21T15:24:32.074 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: osdmap e1: 0 total, 0 up, 0 in 2024-05-21T15:24:32.074 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:31 smithi038 ceph-mon[31995]: mgrmap e1: no daemons active 2024-05-21T15:24:32.962 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.270+0000 7fa79d662700 1 Processor -- start 2024-05-21T15:24:32.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.270+0000 7fa79d662700 1 -- start start 2024-05-21T15:24:32.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.270+0000 7fa79d662700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa798108c70 0x7fa798109080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:32.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.270+0000 7fa79d662700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa798109670 con 0x7fa798108c70 2024-05-21T15:24:32.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.271+0000 7fa796ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa798108c70 0x7fa798109080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:32.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.271+0000 7fa796ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa798108c70 0x7fa798109080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54052/0 (socket says 172.21.15.38:54052) 2024-05-21T15:24:32.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.271+0000 7fa796ffd700 1 -- 172.21.15.38:0/2069793809 learned_addr learned my addr 172.21.15.38:0/2069793809 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:32.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.271+0000 7fa796ffd700 1 -- 172.21.15.38:0/2069793809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa798109da0 con 0x7fa798108c70 2024-05-21T15:24:32.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.271+0000 7fa796ffd700 1 --2- 172.21.15.38:0/2069793809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa798108c70 0x7fa798109080 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa7880098a0 tx=0x7fa788009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a837d35e314eece9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:32.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.272+0000 7fa7967fc700 1 -- 172.21.15.38:0/2069793809 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa788004160 con 0x7fa798108c70 2024-05-21T15:24:32.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.272+0000 7fa7967fc700 1 -- 172.21.15.38:0/2069793809 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa788004300 con 0x7fa798108c70 2024-05-21T15:24:32.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.272+0000 7fa7967fc700 1 -- 172.21.15.38:0/2069793809 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa788011b50 con 0x7fa798108c70 2024-05-21T15:24:32.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.272+0000 7fa79d662700 1 -- 172.21.15.38:0/2069793809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa798108c70 msgr2=0x7fa798109080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:32.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.272+0000 7fa79d662700 1 --2- 172.21.15.38:0/2069793809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa798108c70 0x7fa798109080 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa7880098a0 tx=0x7fa788009bb0 comp rx=0 tx=0).stop 2024-05-21T15:24:32.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.272+0000 7fa79d662700 1 -- 172.21.15.38:0/2069793809 shutdown_connections 2024-05-21T15:24:32.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.272+0000 7fa79d662700 1 --2- 172.21.15.38:0/2069793809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa798108c70 0x7fa798109080 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:32.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.272+0000 7fa79d662700 1 -- 172.21.15.38:0/2069793809 >> 172.21.15.38:0/2069793809 conn(0x7fa7981042d0 msgr2=0x7fa798106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:32.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.273+0000 7fa79d662700 1 -- 172.21.15.38:0/2069793809 shutdown_connections 2024-05-21T15:24:32.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.273+0000 7fa79d662700 1 -- 172.21.15.38:0/2069793809 wait complete. 2024-05-21T15:24:32.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.273+0000 7fa79d662700 1 Processor -- start 2024-05-21T15:24:32.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.273+0000 7fa79d662700 1 -- start start 2024-05-21T15:24:32.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.274+0000 7fa79d662700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa798108c70 0x7fa79819d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:32.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.274+0000 7fa79d662700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa79810d150 con 0x7fa798108c70 2024-05-21T15:24:32.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.274+0000 7fa796ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa798108c70 0x7fa79819d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.274+0000 7fa796ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa798108c70 0x7fa79819d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54058/0 (socket says 172.21.15.38:54058) 2024-05-21T15:24:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.274+0000 7fa796ffd700 1 -- 172.21.15.38:0/3516489040 learned_addr learned my addr 172.21.15.38:0/3516489040 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.274+0000 7fa796ffd700 1 -- 172.21.15.38:0/3516489040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79819d630 con 0x7fa798108c70 2024-05-21T15:24:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.274+0000 7fa796ffd700 1 --2- 172.21.15.38:0/3516489040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa798108c70 0x7fa79819d0f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa788011710 tx=0x7fa788011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.274+0000 7fa794ff9700 1 -- 172.21.15.38:0/3516489040 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7880038b0 con 0x7fa798108c70 2024-05-21T15:24:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.274+0000 7fa794ff9700 1 -- 172.21.15.38:0/3516489040 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa788011e50 con 0x7fa798108c70 2024-05-21T15:24:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.274+0000 7fa79d662700 1 -- 172.21.15.38:0/3516489040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79819d8c0 con 0x7fa798108c70 2024-05-21T15:24:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.274+0000 7fa794ff9700 1 -- 172.21.15.38:0/3516489040 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7880193c0 con 0x7fa798108c70 2024-05-21T15:24:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.274+0000 7fa79d662700 1 -- 172.21.15.38:0/3516489040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7981a04d0 con 0x7fa798108c70 2024-05-21T15:24:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.275+0000 7fa794ff9700 1 -- 172.21.15.38:0/3516489040 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa788019a10 con 0x7fa798108c70 2024-05-21T15:24:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.275+0000 7fa794ff9700 1 -- 172.21.15.38:0/3516489040 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa7880243f0 con 0x7fa798108c70 2024-05-21T15:24:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.275+0000 7fa79d662700 1 -- 172.21.15.38:0/3516489040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa798063ca0 con 0x7fa798108c70 2024-05-21T15:24:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.278+0000 7fa794ff9700 1 -- 172.21.15.38:0/3516489040 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa788043dd0 con 0x7fa798108c70 2024-05-21T15:24:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.348+0000 7fa79d662700 1 -- 172.21.15.38:0/3516489040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fa7981a06a0 con 0x7fa798108c70 2024-05-21T15:24:32.970 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.350+0000 7fa794ff9700 1 -- 172.21.15.38:0/3516489040 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fa788032030 con 0x7fa798108c70 2024-05-21T15:24:32.970 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.351+0000 7fa79d662700 1 -- 172.21.15.38:0/3516489040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa798108c70 msgr2=0x7fa79819d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:32.970 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.351+0000 7fa79d662700 1 --2- 172.21.15.38:0/3516489040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa798108c70 0x7fa79819d0f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa788011710 tx=0x7fa788011c10 comp rx=0 tx=0).stop 2024-05-21T15:24:32.970 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.351+0000 7fa79d662700 1 -- 172.21.15.38:0/3516489040 shutdown_connections 2024-05-21T15:24:32.970 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.351+0000 7fa79d662700 1 --2- 172.21.15.38:0/3516489040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa798108c70 0x7fa79819d0f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:32.970 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.351+0000 7fa79d662700 1 -- 172.21.15.38:0/3516489040 >> 172.21.15.38:0/3516489040 conn(0x7fa7981042d0 msgr2=0x7fa798104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:32.970 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.351+0000 7fa79d662700 1 -- 172.21.15.38:0/3516489040 shutdown_connections 2024-05-21T15:24:32.970 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:32.351+0000 7fa79d662700 1 -- 172.21.15.38:0/3516489040 wait complete. 2024-05-21T15:24:32.970 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-21T15:24:32.993 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:24:32.993 INFO:teuthology.orchestra.run.smithi038.stdout:Creating mgr... 2024-05-21T15:24:32.993 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-21T15:24:32.994 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-21T15:24:33.252 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.x 2024-05-21T15:24:33.252 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to reset failed state of unit ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.x.service: Unit ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.x.service not loaded. 2024-05-21T15:24:33.404 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-102ec4fa-1786-11ef-bc9a-c7b262605968.target.wants/ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@.service. 2024-05-21T15:24:33.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:33 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3516489040' entity='client.admin' 2024-05-21T15:24:33.528 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:33 smithi038 systemd[1]: Starting Ceph mgr.x for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:24:33.915 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:33 smithi038 podman[32212]: 2024-05-21 15:24:33.54013677 +0000 UTC m=+0.017197415 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:24:33.915 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:33 smithi038 podman[32212]: 2024-05-21 15:24:33.64291715 +0000 UTC m=+0.119977779 container create f79809031e3a1490d0609b03ea3f23db78d1d82421e4d8a88070f25a7564a839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-21T15:24:33.915 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:33 smithi038 podman[32212]: 2024-05-21 15:24:33.845165542 +0000 UTC m=+0.322226180 container init f79809031e3a1490d0609b03ea3f23db78d1d82421e4d8a88070f25a7564a839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:24:33.915 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:33 smithi038 podman[32212]: 2024-05-21 15:24:33.847954517 +0000 UTC m=+0.325015155 container start f79809031e3a1490d0609b03ea3f23db78d1d82421e4d8a88070f25a7564a839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-21T15:24:33.915 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:33 smithi038 bash[32212]: f79809031e3a1490d0609b03ea3f23db78d1d82421e4d8a88070f25a7564a839 2024-05-21T15:24:33.929 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T15:24:33.929 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-05-21T15:24:33.936 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T15:24:33.936 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-05-21T15:24:33.937 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-05-21T15:24:33.937 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to enable service . firewalld.service is not available 2024-05-21T15:24:33.945 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T15:24:33.946 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-05-21T15:24:33.953 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T15:24:33.953 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-05-21T15:24:33.953 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-05-21T15:24:33.953 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-21T15:24:33.954 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr to start... 2024-05-21T15:24:33.954 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr... 2024-05-21T15:24:34.189 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:33 smithi038 systemd[1]: Started Ceph mgr.x for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:24:34.189 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:34 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:34.023+0000 7f709efcf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T15:24:34.497 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:34 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:34.219+0000 7f709efcf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:24:34.497 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:34 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:34.328+0000 7f709efcf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T15:24:34.756 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:34 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3123366157' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T15:24:34.757 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:34 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:34.662+0000 7f709efcf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T15:24:34.757 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:34 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:34.753+0000 7f709efcf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T15:24:35.465 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-05-21T15:24:35.468 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:24:35.468 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "102ec4fa-1786-11ef-bc9a-c7b262605968", 2024-05-21T15:24:35.468 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T15:24:35.468 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T15:24:35.468 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T15:24:35.468 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T15:24:35.468 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:35.469 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T15:24:35.469 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T15:24:35.469 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-05-21T15:24:35.469 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:24:35.469 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T15:24:35.469 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T15:24:35.469 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:24:35.469 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-21T15:24:35.469 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T15:24:35.469 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:35.469 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T15:24:35.469 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T15:24:35.469 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:35.469 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T15:24:35.470 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:35.470 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T15:24:35.470 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T15:24:35.470 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T15:24:35.479 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T15:24:35.479 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T15:24:35.479 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T15:24:35.479 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:35.479 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T15:24:35.479 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T15:24:35.480 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T15:24:35.481 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T15:24:35.481 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T15:24:35.481 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T15:24:35.481 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T15:24:35.481 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T15:24:35.481 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:24:35.481 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:24:35.481 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:35.481 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T15:24:35.481 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:35.481 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T15:24:24.082698+0000", 2024-05-21T15:24:35.481 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:24:35.481 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:35.482 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T15:24:35.482 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:24:35.482 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.411+0000 7f05f2eee700 1 Processor -- start 2024-05-21T15:24:35.482 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.412+0000 7f05f2eee700 1 -- start start 2024-05-21T15:24:35.482 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.412+0000 7f05f2eee700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec107af0 0x7f05ec107f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:35.482 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.412+0000 7f05f2eee700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1084f0 con 0x7f05ec107af0 2024-05-21T15:24:35.482 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.412+0000 7f05f0c8a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec107af0 0x7f05ec107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:35.482 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.412+0000 7f05f0c8a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec107af0 0x7f05ec107f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54096/0 (socket says 172.21.15.38:54096) 2024-05-21T15:24:35.482 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.412+0000 7f05f0c8a700 1 -- 172.21.15.38:0/2929062754 learned_addr learned my addr 172.21.15.38:0/2929062754 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:35.482 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.413+0000 7f05f0c8a700 1 -- 172.21.15.38:0/2929062754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec108c20 con 0x7f05ec107af0 2024-05-21T15:24:35.482 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.413+0000 7f05f0c8a700 1 --2- 172.21.15.38:0/2929062754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec107af0 0x7f05ec107f00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f05e00098a0 tx=0x7f05e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36a23064c9d8aab server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:35.482 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.413+0000 7f05eb7fe700 1 -- 172.21.15.38:0/2929062754 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e0004160 con 0x7f05ec107af0 2024-05-21T15:24:35.482 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.413+0000 7f05eb7fe700 1 -- 172.21.15.38:0/2929062754 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f05e0004300 con 0x7f05ec107af0 2024-05-21T15:24:35.482 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.413+0000 7f05eb7fe700 1 -- 172.21.15.38:0/2929062754 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e0011b50 con 0x7f05ec107af0 2024-05-21T15:24:35.483 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.414+0000 7f05f2eee700 1 -- 172.21.15.38:0/2929062754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec107af0 msgr2=0x7f05ec107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:35.483 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.414+0000 7f05f2eee700 1 --2- 172.21.15.38:0/2929062754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec107af0 0x7f05ec107f00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f05e00098a0 tx=0x7f05e0009bb0 comp rx=0 tx=0).stop 2024-05-21T15:24:35.483 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.414+0000 7f05f2eee700 1 -- 172.21.15.38:0/2929062754 shutdown_connections 2024-05-21T15:24:35.483 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.414+0000 7f05f2eee700 1 --2- 172.21.15.38:0/2929062754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec107af0 0x7f05ec107f00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:35.483 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.414+0000 7f05f2eee700 1 -- 172.21.15.38:0/2929062754 >> 172.21.15.38:0/2929062754 conn(0x7f05ec1031b0 msgr2=0x7f05ec105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:35.483 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.414+0000 7f05f2eee700 1 -- 172.21.15.38:0/2929062754 shutdown_connections 2024-05-21T15:24:35.483 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.414+0000 7f05f2eee700 1 -- 172.21.15.38:0/2929062754 wait complete. 2024-05-21T15:24:35.483 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.415+0000 7f05f2eee700 1 Processor -- start 2024-05-21T15:24:35.483 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.415+0000 7f05f2eee700 1 -- start start 2024-05-21T15:24:35.483 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.415+0000 7f05f2eee700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec107af0 0x7f05ec19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:35.483 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.415+0000 7f05f2eee700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec07b140 con 0x7f05ec107af0 2024-05-21T15:24:35.483 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.415+0000 7f05f0c8a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec107af0 0x7f05ec19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:35.483 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.415+0000 7f05f0c8a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec107af0 0x7f05ec19ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54104/0 (socket says 172.21.15.38:54104) 2024-05-21T15:24:35.483 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.415+0000 7f05f0c8a700 1 -- 172.21.15.38:0/3123366157 learned_addr learned my addr 172.21.15.38:0/3123366157 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.415+0000 7f05f0c8a700 1 -- 172.21.15.38:0/3123366157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec19b370 con 0x7f05ec107af0 2024-05-21T15:24:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.415+0000 7f05f0c8a700 1 --2- 172.21.15.38:0/3123366157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec107af0 0x7f05ec19ae30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009fb0 tx=0x7f05e0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.416+0000 7f05e9ffb700 1 -- 172.21.15.38:0/3123366157 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e00038b0 con 0x7f05ec107af0 2024-05-21T15:24:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.416+0000 7f05f2eee700 1 -- 172.21.15.38:0/3123366157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ec19b600 con 0x7f05ec107af0 2024-05-21T15:24:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.416+0000 7f05e9ffb700 1 -- 172.21.15.38:0/3123366157 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f05e0011e50 con 0x7f05ec107af0 2024-05-21T15:24:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.416+0000 7f05e9ffb700 1 -- 172.21.15.38:0/3123366157 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e001a5c0 con 0x7f05ec107af0 2024-05-21T15:24:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.416+0000 7f05f2eee700 1 -- 172.21.15.38:0/3123366157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ec19e210 con 0x7f05ec107af0 2024-05-21T15:24:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.416+0000 7f05e9ffb700 1 -- 172.21.15.38:0/3123366157 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f05e001ac50 con 0x7f05ec107af0 2024-05-21T15:24:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.416+0000 7f05e9ffb700 1 -- 172.21.15.38:0/3123366157 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f05e0019700 con 0x7f05ec107af0 2024-05-21T15:24:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.417+0000 7f05f2eee700 1 -- 172.21.15.38:0/3123366157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ec063ca0 con 0x7f05ec107af0 2024-05-21T15:24:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.420+0000 7f05e9ffb700 1 -- 172.21.15.38:0/3123366157 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f05e0003a50 con 0x7f05ec107af0 2024-05-21T15:24:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.493+0000 7f05f2eee700 1 -- 172.21.15.38:0/3123366157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f05ec194040 con 0x7f05ec107af0 2024-05-21T15:24:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.494+0000 7f05e9ffb700 1 -- 172.21.15.38:0/3123366157 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f05e0019a20 con 0x7f05ec107af0 2024-05-21T15:24:35.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.495+0000 7f05f2eee700 1 -- 172.21.15.38:0/3123366157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec107af0 msgr2=0x7f05ec19ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:35.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.495+0000 7f05f2eee700 1 --2- 172.21.15.38:0/3123366157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec107af0 0x7f05ec19ae30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009fb0 tx=0x7f05e0011c10 comp rx=0 tx=0).stop 2024-05-21T15:24:35.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.495+0000 7f05f2eee700 1 -- 172.21.15.38:0/3123366157 shutdown_connections 2024-05-21T15:24:35.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.495+0000 7f05f2eee700 1 --2- 172.21.15.38:0/3123366157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec107af0 0x7f05ec19ae30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:35.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.495+0000 7f05f2eee700 1 -- 172.21.15.38:0/3123366157 >> 172.21.15.38:0/3123366157 conn(0x7f05ec1031b0 msgr2=0x7f05ec191100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:35.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.495+0000 7f05f2eee700 1 -- 172.21.15.38:0/3123366157 shutdown_connections 2024-05-21T15:24:35.485 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:34.495+0000 7f05f2eee700 1 -- 172.21.15.38:0/3123366157 wait complete. 2024-05-21T15:24:35.485 INFO:teuthology.orchestra.run.smithi038.stdout:mgr not available, waiting (1/15)... 2024-05-21T15:24:36.392 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:36 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:36.035+0000 7f709efcf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T15:24:36.392 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:36 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:36.212+0000 7f709efcf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T15:24:36.392 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:36 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:36.388+0000 7f709efcf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T15:24:37.001 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:36 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:36.689+0000 7f709efcf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T15:24:37.001 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:36 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:36.787+0000 7f709efcf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T15:24:37.001 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:36 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:36.998+0000 7f709efcf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T15:24:37.282 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:37 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:37.198+0000 7f709efcf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:24:38.101 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:37 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:37.858+0000 7f709efcf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T15:24:38.425 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:38 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/592963076' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T15:24:38.425 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:38 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:38.317+0000 7f709efcf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T15:24:38.425 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:38 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:38.421+0000 7f709efcf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T15:24:38.771 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:38 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:38.635+0000 7f709efcf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T15:24:38.773 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-05-21T15:24:38.784 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:24:38.785 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "102ec4fa-1786-11ef-bc9a-c7b262605968", 2024-05-21T15:24:38.785 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T15:24:38.785 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T15:24:38.785 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T15:24:38.785 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T15:24:38.785 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:38.785 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T15:24:38.785 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T15:24:38.785 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-05-21T15:24:38.785 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:24:38.785 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T15:24:38.785 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T15:24:38.785 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:24:38.786 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-05-21T15:24:38.786 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T15:24:38.786 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:38.786 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T15:24:38.786 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T15:24:38.786 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:38.786 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T15:24:38.786 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:38.786 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T15:24:38.786 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T15:24:38.787 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T15:24:38.787 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T15:24:38.787 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T15:24:38.787 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T15:24:38.787 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:38.787 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T15:24:38.787 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T15:24:38.787 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T15:24:38.787 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T15:24:38.787 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T15:24:38.787 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T15:24:38.788 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T15:24:38.788 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T15:24:38.788 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T15:24:38.788 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:38.788 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T15:24:38.788 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:38.788 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T15:24:38.788 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T15:24:38.788 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:38.788 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T15:24:38.789 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T15:24:38.789 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T15:24:38.789 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T15:24:38.789 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T15:24:38.789 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T15:24:38.789 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T15:24:38.789 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:24:38.789 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:24:38.789 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:38.789 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T15:24:38.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:38.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T15:24:24.082698+0000", 2024-05-21T15:24:38.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:24:38.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:38.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T15:24:38.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:24:38.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.961+0000 7f0298725700 1 Processor -- start 2024-05-21T15:24:38.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.961+0000 7f0298725700 1 -- start start 2024-05-21T15:24:38.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.961+0000 7f0298725700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0290109750 0x7f029010bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:38.790 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.961+0000 7f0298725700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f029007c660 con 0x7f0290109750 2024-05-21T15:24:38.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.962+0000 7f02964c1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0290109750 0x7f029010bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:38.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.962+0000 7f02964c1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0290109750 0x7f029010bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42970/0 (socket says 172.21.15.38:42970) 2024-05-21T15:24:38.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.962+0000 7f02964c1700 1 -- 172.21.15.38:0/1628489845 learned_addr learned my addr 172.21.15.38:0/1628489845 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:38.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.962+0000 7f02964c1700 1 -- 172.21.15.38:0/1628489845 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029010c0b0 con 0x7f0290109750 2024-05-21T15:24:38.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.962+0000 7f02964c1700 1 --2- 172.21.15.38:0/1628489845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0290109750 0x7f029010bb70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f028c0098a0 tx=0x7f028c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510bb15b692470ae server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:38.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.962+0000 7f02954bf700 1 -- 172.21.15.38:0/1628489845 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f028c004160 con 0x7f0290109750 2024-05-21T15:24:38.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.962+0000 7f02954bf700 1 -- 172.21.15.38:0/1628489845 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f028c004300 con 0x7f0290109750 2024-05-21T15:24:38.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.963+0000 7f02954bf700 1 -- 172.21.15.38:0/1628489845 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f028c011b50 con 0x7f0290109750 2024-05-21T15:24:38.791 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.963+0000 7f0298725700 1 -- 172.21.15.38:0/1628489845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0290109750 msgr2=0x7f029010bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:38.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.963+0000 7f0298725700 1 --2- 172.21.15.38:0/1628489845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0290109750 0x7f029010bb70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f028c0098a0 tx=0x7f028c009bb0 comp rx=0 tx=0).stop 2024-05-21T15:24:38.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.963+0000 7f0298725700 1 -- 172.21.15.38:0/1628489845 shutdown_connections 2024-05-21T15:24:38.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.963+0000 7f0298725700 1 --2- 172.21.15.38:0/1628489845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0290109750 0x7f029010bb70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:38.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.964+0000 7f0298725700 1 -- 172.21.15.38:0/1628489845 >> 172.21.15.38:0/1628489845 conn(0x7f0290103230 msgr2=0x7f0290105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:38.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.964+0000 7f0298725700 1 -- 172.21.15.38:0/1628489845 shutdown_connections 2024-05-21T15:24:38.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.964+0000 7f0298725700 1 -- 172.21.15.38:0/1628489845 wait complete. 2024-05-21T15:24:38.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.964+0000 7f0298725700 1 Processor -- start 2024-05-21T15:24:38.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.964+0000 7f0298725700 1 -- start start 2024-05-21T15:24:38.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.964+0000 7f0298725700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0290109750 0x7f029019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:38.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.964+0000 7f0298725700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f029010d010 con 0x7f0290109750 2024-05-21T15:24:38.792 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.965+0000 7f02964c1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0290109750 0x7f029019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:38.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.965+0000 7f02964c1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0290109750 0x7f029019d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42982/0 (socket says 172.21.15.38:42982) 2024-05-21T15:24:38.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.965+0000 7f02964c1700 1 -- 172.21.15.38:0/592963076 learned_addr learned my addr 172.21.15.38:0/592963076 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:38.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.965+0000 7f02964c1700 1 -- 172.21.15.38:0/592963076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029019d5d0 con 0x7f0290109750 2024-05-21T15:24:38.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.965+0000 7f02964c1700 1 --2- 172.21.15.38:0/592963076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0290109750 0x7f029019d090 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f028c009fb0 tx=0x7f028c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:38.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.965+0000 7f02877fe700 1 -- 172.21.15.38:0/592963076 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f028c0038b0 con 0x7f0290109750 2024-05-21T15:24:38.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.965+0000 7f02877fe700 1 -- 172.21.15.38:0/592963076 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f028c011e50 con 0x7f0290109750 2024-05-21T15:24:38.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.965+0000 7f0298725700 1 -- 172.21.15.38:0/592963076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029019d860 con 0x7f0290109750 2024-05-21T15:24:38.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.965+0000 7f02877fe700 1 -- 172.21.15.38:0/592963076 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f028c01a430 con 0x7f0290109750 2024-05-21T15:24:38.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.966+0000 7f0298725700 1 -- 172.21.15.38:0/592963076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02901a0470 con 0x7f0290109750 2024-05-21T15:24:38.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.966+0000 7f02877fe700 1 -- 172.21.15.38:0/592963076 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f028c01aa90 con 0x7f0290109750 2024-05-21T15:24:38.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.966+0000 7f02877fe700 1 -- 172.21.15.38:0/592963076 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f028c019700 con 0x7f0290109750 2024-05-21T15:24:38.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.966+0000 7f0298725700 1 -- 172.21.15.38:0/592963076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0290063ca0 con 0x7f0290109750 2024-05-21T15:24:38.793 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:37.970+0000 7f02877fe700 1 -- 172.21.15.38:0/592963076 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f028c043dd0 con 0x7f0290109750 2024-05-21T15:24:38.794 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:38.046+0000 7f0298725700 1 -- 172.21.15.38:0/592963076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f02901a0760 con 0x7f0290109750 2024-05-21T15:24:38.794 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:38.047+0000 7f02877fe700 1 -- 172.21.15.38:0/592963076 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f028c032020 con 0x7f0290109750 2024-05-21T15:24:38.794 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:38.048+0000 7f0298725700 1 -- 172.21.15.38:0/592963076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0290109750 msgr2=0x7f029019d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:38.794 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:38.048+0000 7f0298725700 1 --2- 172.21.15.38:0/592963076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0290109750 0x7f029019d090 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f028c009fb0 tx=0x7f028c011b60 comp rx=0 tx=0).stop 2024-05-21T15:24:38.794 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:38.048+0000 7f0298725700 1 -- 172.21.15.38:0/592963076 shutdown_connections 2024-05-21T15:24:38.794 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:38.048+0000 7f0298725700 1 --2- 172.21.15.38:0/592963076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0290109750 0x7f029019d090 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:38.794 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:38.048+0000 7f0298725700 1 -- 172.21.15.38:0/592963076 >> 172.21.15.38:0/592963076 conn(0x7f0290103230 msgr2=0x7f0290104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:38.794 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:38.048+0000 7f0298725700 1 -- 172.21.15.38:0/592963076 shutdown_connections 2024-05-21T15:24:38.794 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:38.048+0000 7f0298725700 1 -- 172.21.15.38:0/592963076 wait complete. 2024-05-21T15:24:38.794 INFO:teuthology.orchestra.run.smithi038.stdout:mgr not available, waiting (2/15)... 2024-05-21T15:24:39.528 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:39 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:39.185+0000 7f709efcf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T15:24:39.528 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:39 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:39.281+0000 7f709efcf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T15:24:39.898 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:39 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:39.534+0000 7f709efcf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T15:24:39.898 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:39 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:39.674+0000 7f709efcf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T15:24:39.899 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:39 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:39.773+0000 7f709efcf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T15:24:39.899 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:39 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:39.895+0000 7f709efcf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T15:24:40.528 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:40 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:40.206+0000 7f709efcf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T15:24:41.124 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:40 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:40.838+0000 7f709efcf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T15:24:41.124 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:41 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:41.067+0000 7f709efcf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T15:24:41.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:41 smithi038 ceph-mon[31995]: Activating manager daemon x 2024-05-21T15:24:41.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:41 smithi038 ceph-mon[31995]: mgrmap e2: x(active, starting, since 0.0083495s) 2024-05-21T15:24:41.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:41 smithi038 ceph-mon[31995]: from='mgr.14100 172.21.15.38:0/1648815432' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T15:24:41.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:41 smithi038 ceph-mon[31995]: from='mgr.14100 172.21.15.38:0/1648815432' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T15:24:41.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:41 smithi038 ceph-mon[31995]: from='mgr.14100 172.21.15.38:0/1648815432' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T15:24:41.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:41 smithi038 ceph-mon[31995]: from='mgr.14100 172.21.15.38:0/1648815432' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:24:41.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:41 smithi038 ceph-mon[31995]: from='mgr.14100 172.21.15.38:0/1648815432' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T15:24:41.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:41 smithi038 ceph-mon[31995]: Manager daemon x is now available 2024-05-21T15:24:41.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:41 smithi038 ceph-mon[31995]: from='mgr.14100 172.21.15.38:0/1648815432' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T15:24:41.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:41 smithi038 ceph-mon[31995]: from='mgr.14100 172.21.15.38:0/1648815432' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T15:24:41.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:41 smithi038 ceph-mon[31995]: from='mgr.14100 172.21.15.38:0/1648815432' entity='mgr.x' 2024-05-21T15:24:41.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:41 smithi038 ceph-mon[31995]: from='mgr.14100 172.21.15.38:0/1648815432' entity='mgr.x' 2024-05-21T15:24:41.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:41 smithi038 ceph-mon[31995]: from='mgr.14100 172.21.15.38:0/1648815432' entity='mgr.x' 2024-05-21T15:24:41.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-05-21T15:24:41.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:24:41.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "102ec4fa-1786-11ef-bc9a-c7b262605968", 2024-05-21T15:24:41.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T15:24:41.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T15:24:41.833 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T15:24:41.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T15:24:41.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:41.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T15:24:41.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T15:24:41.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-05-21T15:24:41.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:24:41.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T15:24:41.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T15:24:41.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:24:41.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-05-21T15:24:41.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T15:24:41.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:41.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T15:24:41.834 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T15:24:41.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:41.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T15:24:41.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:41.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T15:24:41.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T15:24:41.835 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T15:24:41.842 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T15:24:41.842 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T15:24:41.842 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T15:24:41.842 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:41.843 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T15:24:41.843 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T15:24:41.843 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T15:24:41.843 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T15:24:41.843 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T15:24:41.843 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T15:24:41.843 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T15:24:41.843 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T15:24:41.843 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T15:24:41.843 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:41.843 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T15:24:41.843 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:41.843 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T15:24:41.844 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T15:24:41.844 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:41.844 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T15:24:41.844 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T15:24:41.844 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T15:24:41.844 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T15:24:41.844 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T15:24:41.844 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T15:24:41.844 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T15:24:41.844 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:24:41.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:24:41.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:41.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T15:24:41.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:41.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T15:24:24.082698+0000", 2024-05-21T15:24:41.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:24:41.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:41.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T15:24:41.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:24:41.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.177+0000 7fe96c105700 1 Processor -- start 2024-05-21T15:24:41.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.177+0000 7fe96c105700 1 -- start start 2024-05-21T15:24:41.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.178+0000 7fe96c105700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe96407bca0 0x7fe96410b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:41.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.178+0000 7fe96c105700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe96410bd30 con 0x7fe96407bca0 2024-05-21T15:24:41.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.178+0000 7fe969ea1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe96407bca0 0x7fe96410b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:41.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.178+0000 7fe969ea1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe96407bca0 0x7fe96410b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43028/0 (socket says 172.21.15.38:43028) 2024-05-21T15:24:41.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.178+0000 7fe969ea1700 1 -- 172.21.15.38:0/587590243 learned_addr learned my addr 172.21.15.38:0/587590243 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:41.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.178+0000 7fe969ea1700 1 -- 172.21.15.38:0/587590243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96410beb0 con 0x7fe96407bca0 2024-05-21T15:24:41.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.178+0000 7fe969ea1700 1 --2- 172.21.15.38:0/587590243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe96407bca0 0x7fe96410b740 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe9600098a0 tx=0x7fe960009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3527bea5340c862a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:41.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.178+0000 7fe968e9f700 1 -- 172.21.15.38:0/587590243 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe960004160 con 0x7fe96407bca0 2024-05-21T15:24:41.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.179+0000 7fe968e9f700 1 -- 172.21.15.38:0/587590243 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe960004300 con 0x7fe96407bca0 2024-05-21T15:24:41.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.179+0000 7fe968e9f700 1 -- 172.21.15.38:0/587590243 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe960011b50 con 0x7fe96407bca0 2024-05-21T15:24:41.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.179+0000 7fe96c105700 1 -- 172.21.15.38:0/587590243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe96407bca0 msgr2=0x7fe96410b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:41.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.179+0000 7fe96c105700 1 --2- 172.21.15.38:0/587590243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe96407bca0 0x7fe96410b740 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe9600098a0 tx=0x7fe960009bb0 comp rx=0 tx=0).stop 2024-05-21T15:24:41.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.179+0000 7fe96c105700 1 -- 172.21.15.38:0/587590243 shutdown_connections 2024-05-21T15:24:41.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.179+0000 7fe96c105700 1 --2- 172.21.15.38:0/587590243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe96407bca0 0x7fe96410b740 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:41.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.179+0000 7fe96c105700 1 -- 172.21.15.38:0/587590243 >> 172.21.15.38:0/587590243 conn(0x7fe9641031b0 msgr2=0x7fe964105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:41.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.179+0000 7fe96c105700 1 -- 172.21.15.38:0/587590243 shutdown_connections 2024-05-21T15:24:41.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.179+0000 7fe96c105700 1 -- 172.21.15.38:0/587590243 wait complete. 2024-05-21T15:24:41.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.180+0000 7fe96c105700 1 Processor -- start 2024-05-21T15:24:41.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.180+0000 7fe96c105700 1 -- start start 2024-05-21T15:24:41.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.180+0000 7fe96c105700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe96407bca0 0x7fe96419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:41.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.180+0000 7fe96c105700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe96410cf90 con 0x7fe96407bca0 2024-05-21T15:24:41.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.180+0000 7fe969ea1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe96407bca0 0x7fe96419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:41.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.181+0000 7fe969ea1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe96407bca0 0x7fe96419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43030/0 (socket says 172.21.15.38:43030) 2024-05-21T15:24:41.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.181+0000 7fe969ea1700 1 -- 172.21.15.38:0/1173799604 learned_addr learned my addr 172.21.15.38:0/1173799604 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:41.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.181+0000 7fe969ea1700 1 -- 172.21.15.38:0/1173799604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96419d550 con 0x7fe96407bca0 2024-05-21T15:24:41.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.181+0000 7fe969ea1700 1 --2- 172.21.15.38:0/1173799604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe96407bca0 0x7fe96419d010 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe960011850 tx=0x7fe960011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:41.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.181+0000 7fe95affd700 1 -- 172.21.15.38:0/1173799604 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe960003af0 con 0x7fe96407bca0 2024-05-21T15:24:41.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.181+0000 7fe95affd700 1 -- 172.21.15.38:0/1173799604 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe960020070 con 0x7fe96407bca0 2024-05-21T15:24:41.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.181+0000 7fe96c105700 1 -- 172.21.15.38:0/1173799604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96419d7e0 con 0x7fe96407bca0 2024-05-21T15:24:41.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.181+0000 7fe95affd700 1 -- 172.21.15.38:0/1173799604 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9600195a0 con 0x7fe96407bca0 2024-05-21T15:24:41.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.181+0000 7fe96c105700 1 -- 172.21.15.38:0/1173799604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9641a03f0 con 0x7fe96407bca0 2024-05-21T15:24:41.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.182+0000 7fe95affd700 1 -- 172.21.15.38:0/1173799604 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 2) ==== 46043+0+0 (secure 0 0 0) 0x7fe9600198a0 con 0x7fe96407bca0 2024-05-21T15:24:41.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.182+0000 7fe96c105700 1 -- 172.21.15.38:0/1173799604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe964063ca0 con 0x7fe96407bca0 2024-05-21T15:24:41.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.182+0000 7fe95affd700 1 -- 172.21.15.38:0/1173799604 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe960011e00 con 0x7fe96407bca0 2024-05-21T15:24:41.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.185+0000 7fe95affd700 1 -- 172.21.15.38:0/1173799604 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe960003c90 con 0x7fe96407bca0 2024-05-21T15:24:41.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.257+0000 7fe96c105700 1 -- 172.21.15.38:0/1173799604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe9641a0700 con 0x7fe96407bca0 2024-05-21T15:24:41.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.258+0000 7fe95affd700 1 -- 172.21.15.38:0/1173799604 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe960023530 con 0x7fe96407bca0 2024-05-21T15:24:41.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.258+0000 7fe96c105700 1 -- 172.21.15.38:0/1173799604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe96407bca0 msgr2=0x7fe96419d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:41.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.258+0000 7fe96c105700 1 --2- 172.21.15.38:0/1173799604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe96407bca0 0x7fe96419d010 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe960011850 tx=0x7fe960011c10 comp rx=0 tx=0).stop 2024-05-21T15:24:41.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.258+0000 7fe96c105700 1 -- 172.21.15.38:0/1173799604 shutdown_connections 2024-05-21T15:24:41.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.258+0000 7fe96c105700 1 --2- 172.21.15.38:0/1173799604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe96407bca0 0x7fe96419d010 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:41.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.258+0000 7fe96c105700 1 -- 172.21.15.38:0/1173799604 >> 172.21.15.38:0/1173799604 conn(0x7fe9641031b0 msgr2=0x7fe964104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:41.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.258+0000 7fe96c105700 1 -- 172.21.15.38:0/1173799604 shutdown_connections 2024-05-21T15:24:41.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:41.259+0000 7fe96c105700 1 -- 172.21.15.38:0/1173799604 wait complete. 2024-05-21T15:24:41.849 INFO:teuthology.orchestra.run.smithi038.stdout:mgr not available, waiting (3/15)... 2024-05-21T15:24:42.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:42 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1173799604' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T15:24:42.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:42 smithi038 ceph-mon[31995]: mgrmap e3: x(active, since 1.01271s) 2024-05-21T15:24:44.239 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:44 smithi038 ceph-mon[31995]: mgrmap e4: x(active, since 2s) 2024-05-21T15:24:45.184 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:45 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1979808210' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T15:24:45.196 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-05-21T15:24:45.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:24:45.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "102ec4fa-1786-11ef-bc9a-c7b262605968", 2024-05-21T15:24:45.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T15:24:45.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T15:24:45.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T15:24:45.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T15:24:45.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:45.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T15:24:45.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T15:24:45.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-05-21T15:24:45.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:24:45.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T15:24:45.197 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T15:24:45.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:24:45.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-21T15:24:45.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T15:24:45.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:45.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T15:24:45.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T15:24:45.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:45.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T15:24:45.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:45.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T15:24:45.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T15:24:45.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T15:24:45.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T15:24:45.198 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:45.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T15:24:45.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T15:24:45.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:45.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T15:24:45.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T15:24:45.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T15:24:45.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T15:24:45.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T15:24:45.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T15:24:45.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T15:24:45.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:24:45.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:24:45.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:45.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T15:24:24.082698+0000", 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.299+0000 7eff21e55700 1 Processor -- start 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.299+0000 7eff21e55700 1 -- start start 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.299+0000 7eff21e55700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff1c107b70 0x7eff1c107f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.299+0000 7eff21e55700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c108570 con 0x7eff1c107b70 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.299+0000 7eff1b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff1c107b70 0x7eff1c107f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.300+0000 7eff1b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff1c107b70 0x7eff1c107f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43042/0 (socket says 172.21.15.38:43042) 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.300+0000 7eff1b7fe700 1 -- 172.21.15.38:0/1391308414 learned_addr learned my addr 172.21.15.38:0/1391308414 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.300+0000 7eff1b7fe700 1 -- 172.21.15.38:0/1391308414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c108ca0 con 0x7eff1c107b70 2024-05-21T15:24:45.201 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.300+0000 7eff1b7fe700 1 --2- 172.21.15.38:0/1391308414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff1c107b70 0x7eff1c107f80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0098a0 tx=0x7eff0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa4124788c93254 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:45.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.300+0000 7eff1affd700 1 -- 172.21.15.38:0/1391308414 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff0c004160 con 0x7eff1c107b70 2024-05-21T15:24:45.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.300+0000 7eff1affd700 1 -- 172.21.15.38:0/1391308414 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7eff0c004300 con 0x7eff1c107b70 2024-05-21T15:24:45.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.300+0000 7eff1affd700 1 -- 172.21.15.38:0/1391308414 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff0c011b50 con 0x7eff1c107b70 2024-05-21T15:24:45.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.301+0000 7eff21e55700 1 -- 172.21.15.38:0/1391308414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff1c107b70 msgr2=0x7eff1c107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:45.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.301+0000 7eff21e55700 1 --2- 172.21.15.38:0/1391308414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff1c107b70 0x7eff1c107f80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0098a0 tx=0x7eff0c009bb0 comp rx=0 tx=0).stop 2024-05-21T15:24:45.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.301+0000 7eff21e55700 1 -- 172.21.15.38:0/1391308414 shutdown_connections 2024-05-21T15:24:45.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.301+0000 7eff21e55700 1 --2- 172.21.15.38:0/1391308414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff1c107b70 0x7eff1c107f80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:45.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.301+0000 7eff21e55700 1 -- 172.21.15.38:0/1391308414 >> 172.21.15.38:0/1391308414 conn(0x7eff1c103230 msgr2=0x7eff1c105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:45.202 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.301+0000 7eff21e55700 1 -- 172.21.15.38:0/1391308414 shutdown_connections 2024-05-21T15:24:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.301+0000 7eff21e55700 1 -- 172.21.15.38:0/1391308414 wait complete. 2024-05-21T15:24:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.302+0000 7eff21e55700 1 Processor -- start 2024-05-21T15:24:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.302+0000 7eff21e55700 1 -- start start 2024-05-21T15:24:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.302+0000 7eff21e55700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff1c107b70 0x7eff1c106530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.302+0000 7eff21e55700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c10c050 con 0x7eff1c107b70 2024-05-21T15:24:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.302+0000 7eff1b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff1c107b70 0x7eff1c106530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.302+0000 7eff1b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff1c107b70 0x7eff1c106530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43048/0 (socket says 172.21.15.38:43048) 2024-05-21T15:24:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.302+0000 7eff1b7fe700 1 -- 172.21.15.38:0/1979808210 learned_addr learned my addr 172.21.15.38:0/1979808210 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.303+0000 7eff1b7fe700 1 -- 172.21.15.38:0/1979808210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c106a70 con 0x7eff1c107b70 2024-05-21T15:24:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.303+0000 7eff1b7fe700 1 --2- 172.21.15.38:0/1979808210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff1c107b70 0x7eff1c106530 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7eff0c011710 tx=0x7eff0c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.303+0000 7eff197fa700 1 -- 172.21.15.38:0/1979808210 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff0c0038b0 con 0x7eff1c107b70 2024-05-21T15:24:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.303+0000 7eff197fa700 1 -- 172.21.15.38:0/1979808210 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7eff0c011e50 con 0x7eff1c107b70 2024-05-21T15:24:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.303+0000 7eff21e55700 1 -- 172.21.15.38:0/1979808210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1c104cc0 con 0x7eff1c107b70 2024-05-21T15:24:45.203 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.303+0000 7eff197fa700 1 -- 172.21.15.38:0/1979808210 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff0c0193c0 con 0x7eff1c107b70 2024-05-21T15:24:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.303+0000 7eff21e55700 1 -- 172.21.15.38:0/1979808210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1c1051a0 con 0x7eff1c107b70 2024-05-21T15:24:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.304+0000 7eff197fa700 1 -- 172.21.15.38:0/1979808210 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7eff0c019630 con 0x7eff1c107b70 2024-05-21T15:24:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.304+0000 7eff21e55700 1 -- 172.21.15.38:0/1979808210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff1c063ca0 con 0x7eff1c107b70 2024-05-21T15:24:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.304+0000 7eff197fa700 1 --2- 172.21.15.38:0/1979808210 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7eff08039000 0x7eff0803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.304+0000 7eff197fa700 1 -- 172.21.15.38:0/1979808210 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7eff0c04cff0 con 0x7eff1c107b70 2024-05-21T15:24:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.304+0000 7eff12ffd700 1 --2- 172.21.15.38:0/1979808210 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7eff08039000 0x7eff0803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.305+0000 7eff12ffd700 1 --2- 172.21.15.38:0/1979808210 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7eff08039000 0x7eff0803b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7eff04006fd0 tx=0x7eff04006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.310+0000 7eff197fa700 1 -- 172.21.15.38:0/1979808210 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7eff0c00f2e0 con 0x7eff1c107b70 2024-05-21T15:24:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.602+0000 7eff21e55700 1 -- 172.21.15.38:0/1979808210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7eff1c196380 con 0x7eff1c107b70 2024-05-21T15:24:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.602+0000 7eff197fa700 1 -- 172.21.15.38:0/1979808210 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7eff0c01a430 con 0x7eff1c107b70 2024-05-21T15:24:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.604+0000 7eff21e55700 1 -- 172.21.15.38:0/1979808210 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7eff08039000 msgr2=0x7eff0803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.604+0000 7eff21e55700 1 --2- 172.21.15.38:0/1979808210 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7eff08039000 0x7eff0803b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7eff04006fd0 tx=0x7eff04006e40 comp rx=0 tx=0).stop 2024-05-21T15:24:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.604+0000 7eff21e55700 1 -- 172.21.15.38:0/1979808210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff1c107b70 msgr2=0x7eff1c106530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:45.204 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.604+0000 7eff21e55700 1 --2- 172.21.15.38:0/1979808210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff1c107b70 0x7eff1c106530 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7eff0c011710 tx=0x7eff0c011c10 comp rx=0 tx=0).stop 2024-05-21T15:24:45.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.604+0000 7eff21e55700 1 -- 172.21.15.38:0/1979808210 shutdown_connections 2024-05-21T15:24:45.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.604+0000 7eff21e55700 1 --2- 172.21.15.38:0/1979808210 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7eff08039000 0x7eff0803b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:45.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.604+0000 7eff21e55700 1 --2- 172.21.15.38:0/1979808210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff1c107b70 0x7eff1c106530 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:45.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.604+0000 7eff21e55700 1 -- 172.21.15.38:0/1979808210 >> 172.21.15.38:0/1979808210 conn(0x7eff1c103230 msgr2=0x7eff1c1936f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:45.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.604+0000 7eff21e55700 1 -- 172.21.15.38:0/1979808210 shutdown_connections 2024-05-21T15:24:45.205 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:44.604+0000 7eff21e55700 1 -- 172.21.15.38:0/1979808210 wait complete. 2024-05-21T15:24:45.205 INFO:teuthology.orchestra.run.smithi038.stdout:mgr is available 2024-05-21T15:24:46.376 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-05-21T15:24:46.376 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [global] 2024-05-21T15:24:46.376 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout fsid = 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:24:46.376 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-21T15:24:46.376 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-21T15:24:46.376 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-21T15:24:46.376 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-21T15:24:46.376 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-05-21T15:24:46.376 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-21T15:24:46.376 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-21T15:24:46.376 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-05-21T15:24:46.376 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [osd] 2024-05-21T15:24:46.377 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-05-21T15:24:46.377 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-21T15:24:46.377 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-21T15:24:46.377 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-21T15:24:46.377 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-21T15:24:46.377 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-21T15:24:46.377 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.588+0000 7f836f9eb700 1 Processor -- start 2024-05-21T15:24:46.377 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.588+0000 7f836f9eb700 1 -- start start 2024-05-21T15:24:46.377 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.589+0000 7f836f9eb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f836807bca0 0x7f836810b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:46.377 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.589+0000 7f836f9eb700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f836810bd30 con 0x7f836807bca0 2024-05-21T15:24:46.377 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.589+0000 7f836d787700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f836807bca0 0x7f836810b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:46.377 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.589+0000 7f836d787700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f836807bca0 0x7f836810b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48484/0 (socket says 172.21.15.38:48484) 2024-05-21T15:24:46.377 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.589+0000 7f836d787700 1 -- 172.21.15.38:0/3544326423 learned_addr learned my addr 172.21.15.38:0/3544326423 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:46.377 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.589+0000 7f836d787700 1 -- 172.21.15.38:0/3544326423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836810beb0 con 0x7f836807bca0 2024-05-21T15:24:46.378 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.589+0000 7f836d787700 1 --2- 172.21.15.38:0/3544326423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f836807bca0 0x7f836810b740 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8358009e80 tx=0x7f835800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e9f210768122ac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:46.378 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.589+0000 7f8367fff700 1 -- 172.21.15.38:0/3544326423 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8358017070 con 0x7f836807bca0 2024-05-21T15:24:46.378 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.589+0000 7f8367fff700 1 -- 172.21.15.38:0/3544326423 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f835800b8c0 con 0x7f836807bca0 2024-05-21T15:24:46.378 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.589+0000 7f8367fff700 1 -- 172.21.15.38:0/3544326423 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83580045c0 con 0x7f836807bca0 2024-05-21T15:24:46.378 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.590+0000 7f836f9eb700 1 -- 172.21.15.38:0/3544326423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f836807bca0 msgr2=0x7f836810b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:46.378 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.590+0000 7f836f9eb700 1 --2- 172.21.15.38:0/3544326423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f836807bca0 0x7f836810b740 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8358009e80 tx=0x7f835800b330 comp rx=0 tx=0).stop 2024-05-21T15:24:46.378 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.590+0000 7f836f9eb700 1 -- 172.21.15.38:0/3544326423 shutdown_connections 2024-05-21T15:24:46.378 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.590+0000 7f836f9eb700 1 --2- 172.21.15.38:0/3544326423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f836807bca0 0x7f836810b740 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:46.378 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.590+0000 7f836f9eb700 1 -- 172.21.15.38:0/3544326423 >> 172.21.15.38:0/3544326423 conn(0x7f83681031b0 msgr2=0x7f8368105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:46.378 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.590+0000 7f836f9eb700 1 -- 172.21.15.38:0/3544326423 shutdown_connections 2024-05-21T15:24:46.378 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.590+0000 7f836f9eb700 1 -- 172.21.15.38:0/3544326423 wait complete. 2024-05-21T15:24:46.378 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.591+0000 7f836f9eb700 1 Processor -- start 2024-05-21T15:24:46.378 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.591+0000 7f836f9eb700 1 -- start start 2024-05-21T15:24:46.379 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.591+0000 7f836f9eb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f836807bca0 0x7f836819cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:46.379 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.591+0000 7f836f9eb700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f836810cf90 con 0x7f836807bca0 2024-05-21T15:24:46.379 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.591+0000 7f836d787700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f836807bca0 0x7f836819cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:46.379 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.591+0000 7f836d787700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f836807bca0 0x7f836819cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48486/0 (socket says 172.21.15.38:48486) 2024-05-21T15:24:46.379 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.591+0000 7f836d787700 1 -- 172.21.15.38:0/3449991785 learned_addr learned my addr 172.21.15.38:0/3449991785 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:46.379 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.592+0000 7f836d787700 1 -- 172.21.15.38:0/3449991785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836819d4f0 con 0x7f836807bca0 2024-05-21T15:24:46.379 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.592+0000 7f836d787700 1 --2- 172.21.15.38:0/3449991785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f836807bca0 0x7f836819cfb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8358009e50 tx=0x7f835800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:46.379 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.592+0000 7f83667fc700 1 -- 172.21.15.38:0/3449991785 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8358020020 con 0x7f836807bca0 2024-05-21T15:24:46.379 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.592+0000 7f83667fc700 1 -- 172.21.15.38:0/3449991785 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8358003b50 con 0x7f836807bca0 2024-05-21T15:24:46.379 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.592+0000 7f83667fc700 1 -- 172.21.15.38:0/3449991785 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8358017040 con 0x7f836807bca0 2024-05-21T15:24:46.379 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.592+0000 7f836f9eb700 1 -- 172.21.15.38:0/3449991785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836819d780 con 0x7f836807bca0 2024-05-21T15:24:46.379 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.592+0000 7f836f9eb700 1 -- 172.21.15.38:0/3449991785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83681a0390 con 0x7f836807bca0 2024-05-21T15:24:46.379 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.593+0000 7f83667fc700 1 -- 172.21.15.38:0/3449991785 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f8358023020 con 0x7f836807bca0 2024-05-21T15:24:46.380 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.593+0000 7f836f9eb700 1 -- 172.21.15.38:0/3449991785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8368063ca0 con 0x7f836807bca0 2024-05-21T15:24:46.380 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.593+0000 7f83667fc700 1 --2- 172.21.15.38:0/3449991785 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f8350038fb0 0x7f835003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:46.380 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.593+0000 7f83667fc700 1 -- 172.21.15.38:0/3449991785 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f835800f050 con 0x7f836807bca0 2024-05-21T15:24:46.380 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.593+0000 7f836cf86700 1 --2- 172.21.15.38:0/3449991785 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f8350038fb0 0x7f835003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:46.380 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.594+0000 7f836cf86700 1 --2- 172.21.15.38:0/3449991785 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f8350038fb0 0x7f835003b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f835c006fd0 tx=0x7f835c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:46.380 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.599+0000 7f83667fc700 1 -- 172.21.15.38:0/3449991785 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f835804ee20 con 0x7f836807bca0 2024-05-21T15:24:46.380 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.805+0000 7f836f9eb700 1 -- 172.21.15.38:0/3449991785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f83681a1b90 con 0x7f836807bca0 2024-05-21T15:24:46.380 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.806+0000 7f83667fc700 1 -- 172.21.15.38:0/3449991785 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+398 (secure 0 0 0) 0x7f835802a020 con 0x7f836807bca0 2024-05-21T15:24:46.380 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.809+0000 7f836f9eb700 1 -- 172.21.15.38:0/3449991785 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f8350038fb0 msgr2=0x7f835003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:46.380 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.809+0000 7f836f9eb700 1 --2- 172.21.15.38:0/3449991785 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f8350038fb0 0x7f835003b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f835c006fd0 tx=0x7f835c006e40 comp rx=0 tx=0).stop 2024-05-21T15:24:46.380 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.809+0000 7f836f9eb700 1 -- 172.21.15.38:0/3449991785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f836807bca0 msgr2=0x7f836819cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:46.380 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.809+0000 7f836f9eb700 1 --2- 172.21.15.38:0/3449991785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f836807bca0 0x7f836819cfb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8358009e50 tx=0x7f835800bd30 comp rx=0 tx=0).stop 2024-05-21T15:24:46.380 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.809+0000 7f836f9eb700 1 -- 172.21.15.38:0/3449991785 shutdown_connections 2024-05-21T15:24:46.381 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.809+0000 7f836f9eb700 1 --2- 172.21.15.38:0/3449991785 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f8350038fb0 0x7f835003b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:46.381 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.809+0000 7f836f9eb700 1 --2- 172.21.15.38:0/3449991785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f836807bca0 0x7f836819cfb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:46.381 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.809+0000 7f836f9eb700 1 -- 172.21.15.38:0/3449991785 >> 172.21.15.38:0/3449991785 conn(0x7f83681031b0 msgr2=0x7f8368104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:46.381 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.809+0000 7f836f9eb700 1 -- 172.21.15.38:0/3449991785 shutdown_connections 2024-05-21T15:24:46.381 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:45.809+0000 7f836f9eb700 1 -- 172.21.15.38:0/3449991785 wait complete. 2024-05-21T15:24:46.381 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling cephadm module... 2024-05-21T15:24:46.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:46 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3449991785' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-21T15:24:47.353 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:47 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2072877223' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-21T15:24:47.353 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:47 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: ignoring --setuser ceph since I am not root 2024-05-21T15:24:47.353 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:47 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: ignoring --setgroup ceph since I am not root 2024-05-21T15:24:47.353 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:47 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:47.349+0000 7fe2fbdef200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T15:24:47.656 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:47 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:47.544+0000 7fe2fbdef200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:24:47.656 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:47 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:47.652+0000 7fe2fbdef200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T15:24:47.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.794+0000 7f0073d14700 1 Processor -- start 2024-05-21T15:24:47.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.794+0000 7f0073d14700 1 -- start start 2024-05-21T15:24:47.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.795+0000 7f0073d14700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f006c07bc80 0x7f006c10db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:47.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.795+0000 7f0073d14700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f006c10e0a0 con 0x7f006c07bc80 2024-05-21T15:24:47.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.795+0000 7f0071ab0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f006c07bc80 0x7f006c10db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:47.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.795+0000 7f0071ab0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f006c07bc80 0x7f006c10db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48494/0 (socket says 172.21.15.38:48494) 2024-05-21T15:24:47.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.795+0000 7f0071ab0700 1 -- 172.21.15.38:0/1793333618 learned_addr learned my addr 172.21.15.38:0/1793333618 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:47.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.795+0000 7f0071ab0700 1 -- 172.21.15.38:0/1793333618 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c10e220 con 0x7f006c07bc80 2024-05-21T15:24:47.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.796+0000 7f0071ab0700 1 --2- 172.21.15.38:0/1793333618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f006c07bc80 0x7f006c10db60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0068009e80 tx=0x7f006800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355c2a38163f9638 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:47.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.796+0000 7f0070aae700 1 -- 172.21.15.38:0/1793333618 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0068017070 con 0x7f006c07bc80 2024-05-21T15:24:47.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.796+0000 7f0070aae700 1 -- 172.21.15.38:0/1793333618 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f006800b8c0 con 0x7f006c07bc80 2024-05-21T15:24:47.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.796+0000 7f0070aae700 1 -- 172.21.15.38:0/1793333618 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00680045c0 con 0x7f006c07bc80 2024-05-21T15:24:47.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.796+0000 7f0073d14700 1 -- 172.21.15.38:0/1793333618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f006c07bc80 msgr2=0x7f006c10db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:47.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.796+0000 7f0073d14700 1 --2- 172.21.15.38:0/1793333618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f006c07bc80 0x7f006c10db60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0068009e80 tx=0x7f006800b330 comp rx=0 tx=0).stop 2024-05-21T15:24:47.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.797+0000 7f0073d14700 1 -- 172.21.15.38:0/1793333618 shutdown_connections 2024-05-21T15:24:47.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.797+0000 7f0073d14700 1 --2- 172.21.15.38:0/1793333618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f006c07bc80 0x7f006c10db60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:47.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.797+0000 7f0073d14700 1 -- 172.21.15.38:0/1793333618 >> 172.21.15.38:0/1793333618 conn(0x7f006c1031b0 msgr2=0x7f006c105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:47.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.797+0000 7f0073d14700 1 -- 172.21.15.38:0/1793333618 shutdown_connections 2024-05-21T15:24:47.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.797+0000 7f0073d14700 1 -- 172.21.15.38:0/1793333618 wait complete. 2024-05-21T15:24:47.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.797+0000 7f0073d14700 1 Processor -- start 2024-05-21T15:24:47.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.797+0000 7f0073d14700 1 -- start start 2024-05-21T15:24:47.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.798+0000 7f0073d14700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f006c07bc80 0x7f006c19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:47.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.798+0000 7f0073d14700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f006c10f190 con 0x7f006c07bc80 2024-05-21T15:24:47.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.798+0000 7f0071ab0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f006c07bc80 0x7f006c19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:47.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.798+0000 7f0071ab0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f006c07bc80 0x7f006c19f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48496/0 (socket says 172.21.15.38:48496) 2024-05-21T15:24:47.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.798+0000 7f0071ab0700 1 -- 172.21.15.38:0/2072877223 learned_addr learned my addr 172.21.15.38:0/2072877223 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:47.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.798+0000 7f0071ab0700 1 -- 172.21.15.38:0/2072877223 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c19f6b0 con 0x7f006c07bc80 2024-05-21T15:24:47.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.798+0000 7f0071ab0700 1 --2- 172.21.15.38:0/2072877223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f006c07bc80 0x7f006c19f170 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0068009530 tx=0x7f006800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:47.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.798+0000 7f0062ffd700 1 -- 172.21.15.38:0/2072877223 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0068020020 con 0x7f006c07bc80 2024-05-21T15:24:47.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.798+0000 7f0062ffd700 1 -- 172.21.15.38:0/2072877223 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0068003b50 con 0x7f006c07bc80 2024-05-21T15:24:47.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.798+0000 7f0073d14700 1 -- 172.21.15.38:0/2072877223 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006c19f940 con 0x7f006c07bc80 2024-05-21T15:24:47.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.798+0000 7f0062ffd700 1 -- 172.21.15.38:0/2072877223 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0068017040 con 0x7f006c07bc80 2024-05-21T15:24:47.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.798+0000 7f0073d14700 1 -- 172.21.15.38:0/2072877223 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006c1a2550 con 0x7f006c07bc80 2024-05-21T15:24:47.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.799+0000 7f0062ffd700 1 -- 172.21.15.38:0/2072877223 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f0068023020 con 0x7f006c07bc80 2024-05-21T15:24:47.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.800+0000 7f0073d14700 1 -- 172.21.15.38:0/2072877223 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f006c063ca0 con 0x7f006c07bc80 2024-05-21T15:24:47.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.800+0000 7f0062ffd700 1 --2- 172.21.15.38:0/2072877223 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f0058039050 0x7f005803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:47.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.800+0000 7f0062ffd700 1 -- 172.21.15.38:0/2072877223 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f006800f050 con 0x7f006c07bc80 2024-05-21T15:24:47.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.800+0000 7f00712af700 1 --2- 172.21.15.38:0/2072877223 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f0058039050 0x7f005803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:47.684 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.800+0000 7f00712af700 1 --2- 172.21.15.38:0/2072877223 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f0058039050 0x7f005803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f005c006fd0 tx=0x7f005c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:47.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:46.805+0000 7f0062ffd700 1 -- 172.21.15.38:0/2072877223 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f006802a020 con 0x7f006c07bc80 2024-05-21T15:24:47.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:47.072+0000 7f0073d14700 1 -- 172.21.15.38:0/2072877223 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f006c1a2c20 con 0x7f006c07bc80 2024-05-21T15:24:47.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:47.134+0000 7f0062ffd700 1 -- 172.21.15.38:0/2072877223 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f006804caa0 con 0x7f006c07bc80 2024-05-21T15:24:47.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:47.136+0000 7f0062ffd700 1 -- 172.21.15.38:0/2072877223 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f0068025400 con 0x7f006c07bc80 2024-05-21T15:24:47.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:47.137+0000 7f0073d14700 1 -- 172.21.15.38:0/2072877223 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f0058039050 msgr2=0x7f005803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:47.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:47.137+0000 7f0073d14700 1 --2- 172.21.15.38:0/2072877223 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f0058039050 0x7f005803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f005c006fd0 tx=0x7f005c006e40 comp rx=0 tx=0).stop 2024-05-21T15:24:47.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:47.137+0000 7f0073d14700 1 -- 172.21.15.38:0/2072877223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f006c07bc80 msgr2=0x7f006c19f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:47.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:47.137+0000 7f0073d14700 1 --2- 172.21.15.38:0/2072877223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f006c07bc80 0x7f006c19f170 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0068009530 tx=0x7f006800bd50 comp rx=0 tx=0).stop 2024-05-21T15:24:47.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:47.137+0000 7f0073d14700 1 -- 172.21.15.38:0/2072877223 shutdown_connections 2024-05-21T15:24:47.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:47.137+0000 7f0073d14700 1 --2- 172.21.15.38:0/2072877223 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f0058039050 0x7f005803b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:47.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:47.137+0000 7f0073d14700 1 --2- 172.21.15.38:0/2072877223 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f006c07bc80 0x7f006c19f170 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:47.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:47.137+0000 7f0073d14700 1 -- 172.21.15.38:0/2072877223 >> 172.21.15.38:0/2072877223 conn(0x7f006c1031b0 msgr2=0x7f006c104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:47.685 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:47.137+0000 7f0073d14700 1 -- 172.21.15.38:0/2072877223 shutdown_connections 2024-05-21T15:24:47.686 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:47.137+0000 7f0073d14700 1 -- 172.21.15.38:0/2072877223 wait complete. 2024-05-21T15:24:48.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:48 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2072877223' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-21T15:24:48.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:48 smithi038 ceph-mon[31995]: mgrmap e5: x(active, since 6s) 2024-05-21T15:24:48.274 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:47 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:47.984+0000 7fe2fbdef200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T15:24:48.274 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:48 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:48.082+0000 7fe2fbdef200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T15:24:48.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:24:48.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-21T15:24:48.921 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T15:24:48.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-21T15:24:48.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-21T15:24:48.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:24:48.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.063+0000 7f75b2c33700 1 Processor -- start 2024-05-21T15:24:48.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.063+0000 7f75b2c33700 1 -- start start 2024-05-21T15:24:48.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.063+0000 7f75b2c33700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75ac07c550 0x7f75ac10b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:48.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.063+0000 7f75b2c33700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac10bdc0 con 0x7f75ac07c550 2024-05-21T15:24:48.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.064+0000 7f75b09cf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75ac07c550 0x7f75ac10b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:48.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.064+0000 7f75b09cf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75ac07c550 0x7f75ac10b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48520/0 (socket says 172.21.15.38:48520) 2024-05-21T15:24:48.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.064+0000 7f75b09cf700 1 -- 172.21.15.38:0/2642127921 learned_addr learned my addr 172.21.15.38:0/2642127921 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:48.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.064+0000 7f75b09cf700 1 -- 172.21.15.38:0/2642127921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ac10bf40 con 0x7f75ac07c550 2024-05-21T15:24:48.922 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.064+0000 7f75b09cf700 1 --2- 172.21.15.38:0/2642127921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75ac07c550 0x7f75ac10b7b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f75a00098a0 tx=0x7f75a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4325a6c5ad66ee4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.064+0000 7f75ab7fe700 1 -- 172.21.15.38:0/2642127921 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75a0004160 con 0x7f75ac07c550 2024-05-21T15:24:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.064+0000 7f75ab7fe700 1 -- 172.21.15.38:0/2642127921 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f75a0004300 con 0x7f75ac07c550 2024-05-21T15:24:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.064+0000 7f75ab7fe700 1 -- 172.21.15.38:0/2642127921 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75a0011b50 con 0x7f75ac07c550 2024-05-21T15:24:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.065+0000 7f75b2c33700 1 -- 172.21.15.38:0/2642127921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75ac07c550 msgr2=0x7f75ac10b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.065+0000 7f75b2c33700 1 --2- 172.21.15.38:0/2642127921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75ac07c550 0x7f75ac10b7b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f75a00098a0 tx=0x7f75a0009bb0 comp rx=0 tx=0).stop 2024-05-21T15:24:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.065+0000 7f75b2c33700 1 -- 172.21.15.38:0/2642127921 shutdown_connections 2024-05-21T15:24:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.065+0000 7f75b2c33700 1 --2- 172.21.15.38:0/2642127921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75ac07c550 0x7f75ac10b7b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.065+0000 7f75b2c33700 1 -- 172.21.15.38:0/2642127921 >> 172.21.15.38:0/2642127921 conn(0x7f75ac103220 msgr2=0x7f75ac105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.065+0000 7f75b2c33700 1 -- 172.21.15.38:0/2642127921 shutdown_connections 2024-05-21T15:24:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.065+0000 7f75b2c33700 1 -- 172.21.15.38:0/2642127921 wait complete. 2024-05-21T15:24:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.066+0000 7f75b2c33700 1 Processor -- start 2024-05-21T15:24:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.066+0000 7f75b2c33700 1 -- start start 2024-05-21T15:24:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.066+0000 7f75b2c33700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75ac07c550 0x7f75ac19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:48.923 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.066+0000 7f75b2c33700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac10d020 con 0x7f75ac07c550 2024-05-21T15:24:48.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.066+0000 7f75b09cf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75ac07c550 0x7f75ac19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:48.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.066+0000 7f75b09cf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75ac07c550 0x7f75ac19f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48534/0 (socket says 172.21.15.38:48534) 2024-05-21T15:24:48.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.066+0000 7f75b09cf700 1 -- 172.21.15.38:0/3685167323 learned_addr learned my addr 172.21.15.38:0/3685167323 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:48.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.067+0000 7f75b09cf700 1 -- 172.21.15.38:0/3685167323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ac19f7e0 con 0x7f75ac07c550 2024-05-21T15:24:48.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.067+0000 7f75b09cf700 1 --2- 172.21.15.38:0/3685167323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75ac07c550 0x7f75ac19f2a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009fb0 tx=0x7f75a0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:48.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.067+0000 7f75a9ffb700 1 -- 172.21.15.38:0/3685167323 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75a00038b0 con 0x7f75ac07c550 2024-05-21T15:24:48.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.067+0000 7f75a9ffb700 1 -- 172.21.15.38:0/3685167323 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f75a0011e50 con 0x7f75ac07c550 2024-05-21T15:24:48.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.067+0000 7f75b2c33700 1 -- 172.21.15.38:0/3685167323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ac19fa70 con 0x7f75ac07c550 2024-05-21T15:24:48.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.067+0000 7f75a9ffb700 1 -- 172.21.15.38:0/3685167323 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75a001a430 con 0x7f75ac07c550 2024-05-21T15:24:48.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.067+0000 7f75b2c33700 1 -- 172.21.15.38:0/3685167323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ac1a2680 con 0x7f75ac07c550 2024-05-21T15:24:48.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.068+0000 7f75b2c33700 1 -- 172.21.15.38:0/3685167323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ac063ca0 con 0x7f75ac07c550 2024-05-21T15:24:48.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.068+0000 7f75a9ffb700 1 -- 172.21.15.38:0/3685167323 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f75a001a6e0 con 0x7f75ac07c550 2024-05-21T15:24:48.924 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.069+0000 7f75a9ffb700 1 --2- 172.21.15.38:0/3685167323 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f759c039050 0x7f759c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:48.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.069+0000 7f75a9ffb700 1 -- 172.21.15.38:0/3685167323 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f75a004d030 con 0x7f75ac07c550 2024-05-21T15:24:48.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.069+0000 7f75abfff700 1 -- 172.21.15.38:0/3685167323 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f759c039050 msgr2=0x7f759c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3523378851 2024-05-21T15:24:48.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.069+0000 7f75abfff700 1 --2- 172.21.15.38:0/3685167323 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f759c039050 0x7f759c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:24:48.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.074+0000 7f75a9ffb700 1 -- 172.21.15.38:0/3685167323 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f75a00172b0 con 0x7f75ac07c550 2024-05-21T15:24:48.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.270+0000 7f75abfff700 1 -- 172.21.15.38:0/3685167323 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f759c039050 msgr2=0x7f759c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3523378851 2024-05-21T15:24:48.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.270+0000 7f75abfff700 1 --2- 172.21.15.38:0/3685167323 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f759c039050 0x7f759c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T15:24:48.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.344+0000 7f75b2c33700 1 -- 172.21.15.38:0/3685167323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f75ac1a2970 con 0x7f75ac07c550 2024-05-21T15:24:48.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.345+0000 7f75a9ffb700 1 -- 172.21.15.38:0/3685167323 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f75a000fdd0 con 0x7f75ac07c550 2024-05-21T15:24:48.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.347+0000 7f75b2c33700 1 -- 172.21.15.38:0/3685167323 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f759c039050 msgr2=0x7f759c03b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:24:48.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.347+0000 7f75b2c33700 1 --2- 172.21.15.38:0/3685167323 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f759c039050 0x7f759c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:48.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.347+0000 7f75b2c33700 1 -- 172.21.15.38:0/3685167323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75ac07c550 msgr2=0x7f75ac19f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:48.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.347+0000 7f75b2c33700 1 --2- 172.21.15.38:0/3685167323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75ac07c550 0x7f75ac19f2a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009fb0 tx=0x7f75a0011b60 comp rx=0 tx=0).stop 2024-05-21T15:24:48.925 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.347+0000 7f75b2c33700 1 -- 172.21.15.38:0/3685167323 shutdown_connections 2024-05-21T15:24:48.926 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.347+0000 7f75b2c33700 1 --2- 172.21.15.38:0/3685167323 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f759c039050 0x7f759c03b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:48.926 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.347+0000 7f75b2c33700 1 --2- 172.21.15.38:0/3685167323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75ac07c550 0x7f75ac19f2a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:48.926 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.347+0000 7f75b2c33700 1 -- 172.21.15.38:0/3685167323 >> 172.21.15.38:0/3685167323 conn(0x7f75ac103220 msgr2=0x7f75ac107650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:48.926 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.347+0000 7f75b2c33700 1 -- 172.21.15.38:0/3685167323 shutdown_connections 2024-05-21T15:24:48.926 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:48.347+0000 7f75b2c33700 1 -- 172.21.15.38:0/3685167323 wait complete. 2024-05-21T15:24:48.926 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for the mgr to restart... 2024-05-21T15:24:48.926 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr epoch 5... 2024-05-21T15:24:49.224 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:49 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3685167323' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-21T15:24:49.528 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:49 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:49.366+0000 7fe2fbdef200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T15:24:49.946 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:49 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:49.548+0000 7fe2fbdef200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T15:24:49.946 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:49 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:49.725+0000 7fe2fbdef200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T15:24:50.278 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:50 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:50.032+0000 7fe2fbdef200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T15:24:50.278 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:50 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:50.130+0000 7fe2fbdef200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T15:24:50.748 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:50 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:50.341+0000 7fe2fbdef200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T15:24:50.748 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:50 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:50.546+0000 7fe2fbdef200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:24:51.528 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:51 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:51.182+0000 7fe2fbdef200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T15:24:51.951 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:51 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:51.630+0000 7fe2fbdef200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T15:24:51.951 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:51 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:51.733+0000 7fe2fbdef200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T15:24:51.951 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:51 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:51.947+0000 7fe2fbdef200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T15:24:52.602 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:52 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:52.501+0000 7fe2fbdef200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T15:24:52.859 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:52 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:52.599+0000 7fe2fbdef200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T15:24:53.224 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:52 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:52.855+0000 7fe2fbdef200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T15:24:53.225 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:52 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:52.996+0000 7fe2fbdef200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T15:24:53.225 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:53 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:53.097+0000 7fe2fbdef200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T15:24:53.225 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:53 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:53.221+0000 7fe2fbdef200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T15:24:54.028 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:53 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:53.535+0000 7fe2fbdef200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T15:24:54.458 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:54 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:54.170+0000 7fe2fbdef200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T15:24:54.458 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:54 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:24:54.401+0000 7fe2fbdef200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T15:24:54.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:54 smithi038 ceph-mon[31995]: Active manager daemon x restarted 2024-05-21T15:24:54.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:54 smithi038 ceph-mon[31995]: Activating manager daemon x 2024-05-21T15:24:54.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:54 smithi038 ceph-mon[31995]: osdmap e2: 0 total, 0 up, 0 in 2024-05-21T15:24:54.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:54 smithi038 ceph-mon[31995]: mgrmap e6: x(active, starting, since 0.00624095s) 2024-05-21T15:24:54.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:54 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:24:54.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:54 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T15:24:54.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:54 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T15:24:54.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:54 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T15:24:54.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:54 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T15:24:54.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:54 smithi038 ceph-mon[31995]: Manager daemon x is now available 2024-05-21T15:24:54.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:54 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' 2024-05-21T15:24:54.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:54 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' 2024-05-21T15:24:54.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:54 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:24:54.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:54 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:24:55.713 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:55 smithi038 ceph-mon[31995]: Found migration_current of "None". Setting to last migration. 2024-05-21T15:24:55.713 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:55 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T15:24:55.713 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:55 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T15:24:55.713 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:55 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' 2024-05-21T15:24:55.713 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:55 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' 2024-05-21T15:24:55.713 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:55 smithi038 ceph-mon[31995]: mgrmap e7: x(active, since 1.00646s) 2024-05-21T15:24:55.937 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:24:55.938 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-21T15:24:55.938 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-21T15:24:55.938 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:24:55.938 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.335+0000 7f2ad2b23700 1 Processor -- start 2024-05-21T15:24:55.938 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.335+0000 7f2ad2b23700 1 -- start start 2024-05-21T15:24:55.938 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.335+0000 7f2ad2b23700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2acc07c6f0 0x7f2acc10b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:55.938 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.335+0000 7f2ad2b23700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc10bd90 con 0x7f2acc07c6f0 2024-05-21T15:24:55.938 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.336+0000 7f2ad08bf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2acc07c6f0 0x7f2acc10b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:55.938 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.336+0000 7f2ad08bf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2acc07c6f0 0x7f2acc10b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48546/0 (socket says 172.21.15.38:48546) 2024-05-21T15:24:55.938 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.336+0000 7f2ad08bf700 1 -- 172.21.15.38:0/57471743 learned_addr learned my addr 172.21.15.38:0/57471743 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:55.938 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.336+0000 7f2ad08bf700 1 -- 172.21.15.38:0/57471743 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc10bf10 con 0x7f2acc07c6f0 2024-05-21T15:24:55.938 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.336+0000 7f2ad08bf700 1 --2- 172.21.15.38:0/57471743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2acc07c6f0 0x7f2acc10b780 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00098a0 tx=0x7f2ac0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda8a7f820b78711 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:55.939 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.336+0000 7f2acb7fe700 1 -- 172.21.15.38:0/57471743 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0004160 con 0x7f2acc07c6f0 2024-05-21T15:24:55.939 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.336+0000 7f2acb7fe700 1 -- 172.21.15.38:0/57471743 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2ac0004300 con 0x7f2acc07c6f0 2024-05-21T15:24:55.939 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.336+0000 7f2acb7fe700 1 -- 172.21.15.38:0/57471743 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0011b50 con 0x7f2acc07c6f0 2024-05-21T15:24:55.939 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.337+0000 7f2ad2b23700 1 -- 172.21.15.38:0/57471743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2acc07c6f0 msgr2=0x7f2acc10b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:55.939 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.337+0000 7f2ad2b23700 1 --2- 172.21.15.38:0/57471743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2acc07c6f0 0x7f2acc10b780 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00098a0 tx=0x7f2ac0009bb0 comp rx=0 tx=0).stop 2024-05-21T15:24:55.939 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.337+0000 7f2ad2b23700 1 -- 172.21.15.38:0/57471743 shutdown_connections 2024-05-21T15:24:55.939 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.337+0000 7f2ad2b23700 1 --2- 172.21.15.38:0/57471743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2acc07c6f0 0x7f2acc10b780 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:55.939 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.337+0000 7f2ad2b23700 1 -- 172.21.15.38:0/57471743 >> 172.21.15.38:0/57471743 conn(0x7f2acc103230 msgr2=0x7f2acc105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:55.939 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.337+0000 7f2ad2b23700 1 -- 172.21.15.38:0/57471743 shutdown_connections 2024-05-21T15:24:55.939 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.337+0000 7f2ad2b23700 1 -- 172.21.15.38:0/57471743 wait complete. 2024-05-21T15:24:55.939 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.338+0000 7f2ad2b23700 1 Processor -- start 2024-05-21T15:24:55.939 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.338+0000 7f2ad2b23700 1 -- start start 2024-05-21T15:24:55.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.338+0000 7f2ad2b23700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2acc07c6f0 0x7f2acc19d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:55.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.338+0000 7f2ad2b23700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc10cff0 con 0x7f2acc07c6f0 2024-05-21T15:24:55.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.338+0000 7f2ad08bf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2acc07c6f0 0x7f2acc19d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:55.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.338+0000 7f2ad08bf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2acc07c6f0 0x7f2acc19d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48558/0 (socket says 172.21.15.38:48558) 2024-05-21T15:24:55.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.338+0000 7f2ad08bf700 1 -- 172.21.15.38:0/2809232106 learned_addr learned my addr 172.21.15.38:0/2809232106 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:55.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.339+0000 7f2ad08bf700 1 -- 172.21.15.38:0/2809232106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc19d580 con 0x7f2acc07c6f0 2024-05-21T15:24:55.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.339+0000 7f2ad08bf700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2acc07c6f0 0x7f2acc19d040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009fb0 tx=0x7f2ac0011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:55.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.339+0000 7f2ac9ffb700 1 -- 172.21.15.38:0/2809232106 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac00038b0 con 0x7f2acc07c6f0 2024-05-21T15:24:55.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.339+0000 7f2ad2b23700 1 -- 172.21.15.38:0/2809232106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2acc19d810 con 0x7f2acc07c6f0 2024-05-21T15:24:55.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.339+0000 7f2ac9ffb700 1 -- 172.21.15.38:0/2809232106 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2ac0011e50 con 0x7f2acc07c6f0 2024-05-21T15:24:55.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.339+0000 7f2ac9ffb700 1 -- 172.21.15.38:0/2809232106 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ac0019550 con 0x7f2acc07c6f0 2024-05-21T15:24:55.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.339+0000 7f2ad2b23700 1 -- 172.21.15.38:0/2809232106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2acc1a0420 con 0x7f2acc07c6f0 2024-05-21T15:24:55.940 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.340+0000 7f2ac9ffb700 1 -- 172.21.15.38:0/2809232106 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f2ac0019850 con 0x7f2acc07c6f0 2024-05-21T15:24:55.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.341+0000 7f2ac9ffb700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f2abc039000 0x7f2abc03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:55.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.341+0000 7f2ac9ffb700 1 -- 172.21.15.38:0/2809232106 --> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2abc03bc20 con 0x7f2abc039000 2024-05-21T15:24:55.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.341+0000 7f2ac9ffb700 1 -- 172.21.15.38:0/2809232106 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2ac004e270 con 0x7f2acc07c6f0 2024-05-21T15:24:55.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.341+0000 7f2acbfff700 1 -- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f2abc039000 msgr2=0x7f2abc03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3523378851 2024-05-21T15:24:55.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.341+0000 7f2acbfff700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f2abc039000 0x7f2abc03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:24:55.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.542+0000 7f2acbfff700 1 -- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f2abc039000 msgr2=0x7f2abc03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3523378851 2024-05-21T15:24:55.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.542+0000 7f2acbfff700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f2abc039000 0x7f2abc03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T15:24:55.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.943+0000 7f2acbfff700 1 -- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f2abc039000 msgr2=0x7f2abc03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3523378851 2024-05-21T15:24:55.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:49.943+0000 7f2acbfff700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f2abc039000 0x7f2abc03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-21T15:24:55.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:50.744+0000 7f2acbfff700 1 -- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f2abc039000 msgr2=0x7f2abc03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3523378851 2024-05-21T15:24:55.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:50.744+0000 7f2acbfff700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f2abc039000 0x7f2abc03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-21T15:24:55.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:52.345+0000 7f2acbfff700 1 -- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f2abc039000 msgr2=0x7f2abc03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3523378851 2024-05-21T15:24:55.941 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:52.345+0000 7f2acbfff700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f2abc039000 0x7f2abc03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-21T15:24:55.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:54.408+0000 7f2ac9ffb700 1 -- 172.21.15.38:0/2809232106 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f2ac0029d00 con 0x7f2acc07c6f0 2024-05-21T15:24:55.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:54.408+0000 7f2ac9ffb700 1 -- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f2abc039000 msgr2=0x7f2abc03b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:24:55.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:54.408+0000 7f2ac9ffb700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f2abc039000 0x7f2abc03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:55.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.407+0000 7f2ac9ffb700 1 -- 172.21.15.38:0/2809232106 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f2ac0018350 con 0x7f2acc07c6f0 2024-05-21T15:24:55.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.407+0000 7f2ac9ffb700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f2abc03c980 0x7f2abc03eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:55.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.407+0000 7f2ac9ffb700 1 -- 172.21.15.38:0/2809232106 --> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2ac0003bf0 con 0x7f2abc03c980 2024-05-21T15:24:55.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.408+0000 7f2acbfff700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f2abc03c980 0x7f2abc03eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:55.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.409+0000 7f2acbfff700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f2abc03c980 0x7f2abc03eda0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8003a10 tx=0x7f2ab8009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:55.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.409+0000 7f2ac9ffb700 1 -- 172.21.15.38:0/2809232106 <== mgr.14120 v2:172.21.15.38:6800/1481949523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f2ac0003bf0 con 0x7f2abc03c980 2024-05-21T15:24:55.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.416+0000 7f2ad2b23700 1 -- 172.21.15.38:0/2809232106 --> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f2acc063ca0 con 0x7f2abc03c980 2024-05-21T15:24:55.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.416+0000 7f2ac9ffb700 1 -- 172.21.15.38:0/2809232106 <== mgr.14120 v2:172.21.15.38:6800/1481949523 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f2acc063ca0 con 0x7f2abc03c980 2024-05-21T15:24:55.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.417+0000 7f2ad2b23700 1 -- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f2abc03c980 msgr2=0x7f2abc03eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:55.942 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.417+0000 7f2ad2b23700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f2abc03c980 0x7f2abc03eda0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8003a10 tx=0x7f2ab8009290 comp rx=0 tx=0).stop 2024-05-21T15:24:55.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.417+0000 7f2ad2b23700 1 -- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2acc07c6f0 msgr2=0x7f2acc19d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:55.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.417+0000 7f2ad2b23700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2acc07c6f0 0x7f2acc19d040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009fb0 tx=0x7f2ac0011bf0 comp rx=0 tx=0).stop 2024-05-21T15:24:55.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.417+0000 7f2ad2b23700 1 -- 172.21.15.38:0/2809232106 shutdown_connections 2024-05-21T15:24:55.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.417+0000 7f2ad2b23700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2acc07c6f0 0x7f2acc19d040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:55.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.417+0000 7f2ad2b23700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f2abc03c980 0x7f2abc03eda0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:55.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.417+0000 7f2ad2b23700 1 --2- 172.21.15.38:0/2809232106 >> [v2:172.21.15.38:6800/3523378851,v1:172.21.15.38:6801/3523378851] conn(0x7f2abc039000 0x7f2abc03b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:55.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.417+0000 7f2ad2b23700 1 -- 172.21.15.38:0/2809232106 >> 172.21.15.38:0/2809232106 conn(0x7f2acc103230 msgr2=0x7f2acc104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:55.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.417+0000 7f2ad2b23700 1 -- 172.21.15.38:0/2809232106 shutdown_connections 2024-05-21T15:24:55.943 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:55.417+0000 7f2ad2b23700 1 -- 172.21.15.38:0/2809232106 wait complete. 2024-05-21T15:24:55.943 INFO:teuthology.orchestra.run.smithi038.stdout:mgr epoch 5 is available 2024-05-21T15:24:55.943 INFO:teuthology.orchestra.run.smithi038.stdout:Setting orchestrator backend to cephadm... 2024-05-21T15:24:56.610 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:56 smithi038 ceph-mon[31995]: [21/May/2024:15:24:55] ENGINE Bus STARTING 2024-05-21T15:24:56.610 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:56 smithi038 ceph-mon[31995]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-21T15:24:56.610 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:56 smithi038 ceph-mon[31995]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-21T15:24:56.610 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:56 smithi038 ceph-mon[31995]: [21/May/2024:15:24:55] ENGINE Serving on https://172.21.15.38:7150 2024-05-21T15:24:56.610 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:56 smithi038 ceph-mon[31995]: [21/May/2024:15:24:55] ENGINE Serving on http://172.21.15.38:8765 2024-05-21T15:24:56.610 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:56 smithi038 ceph-mon[31995]: [21/May/2024:15:24:55] ENGINE Bus STARTED 2024-05-21T15:24:56.611 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:56 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:24:57.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.372+0000 7fb7bdfa2700 1 Processor -- start 2024-05-21T15:24:57.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.372+0000 7fb7bdfa2700 1 -- start start 2024-05-21T15:24:57.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.372+0000 7fb7bdfa2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7b807bc80 0x7fb7b810b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:57.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.373+0000 7fb7bdfa2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b810bd30 con 0x7fb7b807bc80 2024-05-21T15:24:57.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.373+0000 7fb7b77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7b807bc80 0x7fb7b810b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:57.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.373+0000 7fb7b77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7b807bc80 0x7fb7b810b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41424/0 (socket says 172.21.15.38:41424) 2024-05-21T15:24:57.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.373+0000 7fb7b77fe700 1 -- 172.21.15.38:0/1949017688 learned_addr learned my addr 172.21.15.38:0/1949017688 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:57.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.373+0000 7fb7b77fe700 1 -- 172.21.15.38:0/1949017688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b810beb0 con 0x7fb7b807bc80 2024-05-21T15:24:57.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.373+0000 7fb7b77fe700 1 --2- 172.21.15.38:0/1949017688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7b807bc80 0x7fb7b810b760 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009e80 tx=0x7fb7a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c594517682928ae9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:57.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.373+0000 7fb7b67fc700 1 -- 172.21.15.38:0/1949017688 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a8017070 con 0x7fb7b807bc80 2024-05-21T15:24:57.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.373+0000 7fb7b67fc700 1 -- 172.21.15.38:0/1949017688 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb7a800b8c0 con 0x7fb7b807bc80 2024-05-21T15:24:57.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.373+0000 7fb7b67fc700 1 -- 172.21.15.38:0/1949017688 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a80045c0 con 0x7fb7b807bc80 2024-05-21T15:24:57.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.374+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1949017688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7b807bc80 msgr2=0x7fb7b810b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:57.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.374+0000 7fb7bdfa2700 1 --2- 172.21.15.38:0/1949017688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7b807bc80 0x7fb7b810b760 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009e80 tx=0x7fb7a800b330 comp rx=0 tx=0).stop 2024-05-21T15:24:57.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.374+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1949017688 shutdown_connections 2024-05-21T15:24:57.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.374+0000 7fb7bdfa2700 1 --2- 172.21.15.38:0/1949017688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7b807bc80 0x7fb7b810b760 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:57.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.374+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1949017688 >> 172.21.15.38:0/1949017688 conn(0x7fb7b81031b0 msgr2=0x7fb7b8105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:57.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.374+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1949017688 shutdown_connections 2024-05-21T15:24:57.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.374+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1949017688 wait complete. 2024-05-21T15:24:57.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.375+0000 7fb7bdfa2700 1 Processor -- start 2024-05-21T15:24:57.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.375+0000 7fb7bdfa2700 1 -- start start 2024-05-21T15:24:57.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.375+0000 7fb7bdfa2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7b807bc80 0x7fb7b819cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:57.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.375+0000 7fb7bdfa2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b810cf90 con 0x7fb7b807bc80 2024-05-21T15:24:57.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.376+0000 7fb7b77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7b807bc80 0x7fb7b819cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:57.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.376+0000 7fb7b77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7b807bc80 0x7fb7b819cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41432/0 (socket says 172.21.15.38:41432) 2024-05-21T15:24:57.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.376+0000 7fb7b77fe700 1 -- 172.21.15.38:0/1130831035 learned_addr learned my addr 172.21.15.38:0/1130831035 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:57.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.376+0000 7fb7b77fe700 1 -- 172.21.15.38:0/1130831035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b819d4f0 con 0x7fb7b807bc80 2024-05-21T15:24:57.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.376+0000 7fb7b77fe700 1 --2- 172.21.15.38:0/1130831035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7b807bc80 0x7fb7b819cfb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009e50 tx=0x7fb7a800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:57.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.376+0000 7fb7b4ff9700 1 -- 172.21.15.38:0/1130831035 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a8020020 con 0x7fb7b807bc80 2024-05-21T15:24:57.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.376+0000 7fb7b4ff9700 1 -- 172.21.15.38:0/1130831035 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb7a8003b50 con 0x7fb7b807bc80 2024-05-21T15:24:57.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.376+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1130831035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b819d780 con 0x7fb7b807bc80 2024-05-21T15:24:57.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.376+0000 7fb7b4ff9700 1 -- 172.21.15.38:0/1130831035 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a8017040 con 0x7fb7b807bc80 2024-05-21T15:24:57.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.376+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1130831035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b81a0390 con 0x7fb7b807bc80 2024-05-21T15:24:57.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.377+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1130831035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b8063ca0 con 0x7fb7b807bc80 2024-05-21T15:24:57.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.377+0000 7fb7b4ff9700 1 -- 172.21.15.38:0/1130831035 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fb7a8023020 con 0x7fb7b807bc80 2024-05-21T15:24:57.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.377+0000 7fb7b4ff9700 1 --2- 172.21.15.38:0/1130831035 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fb7a4038ed0 0x7fb7a403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:57.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.378+0000 7fb7b4ff9700 1 -- 172.21.15.38:0/1130831035 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb7a800f050 con 0x7fb7b807bc80 2024-05-21T15:24:57.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.378+0000 7fb7b6ffd700 1 --2- 172.21.15.38:0/1130831035 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fb7a4038ed0 0x7fb7a403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:57.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.378+0000 7fb7b6ffd700 1 --2- 172.21.15.38:0/1130831035 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fb7a4038ed0 0x7fb7a403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0006fd0 tx=0x7fb7a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.383+0000 7fb7b4ff9700 1 -- 172.21.15.38:0/1130831035 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb7a8052020 con 0x7fb7b807bc80 2024-05-21T15:24:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.465+0000 7fb7b4ff9700 1 -- 172.21.15.38:0/1130831035 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb7a8025400 con 0x7fb7b807bc80 2024-05-21T15:24:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.606+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1130831035 --> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fb7b8105200 con 0x7fb7a4038ed0 2024-05-21T15:24:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.614+0000 7fb7b4ff9700 1 -- 172.21.15.38:0/1130831035 <== mgr.14120 v2:172.21.15.38:6800/1481949523 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb7b8105200 con 0x7fb7a4038ed0 2024-05-21T15:24:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.617+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1130831035 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fb7a4038ed0 msgr2=0x7fb7a403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.617+0000 7fb7bdfa2700 1 --2- 172.21.15.38:0/1130831035 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fb7a4038ed0 0x7fb7a403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0006fd0 tx=0x7fb7a0006e40 comp rx=0 tx=0).stop 2024-05-21T15:24:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.617+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1130831035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7b807bc80 msgr2=0x7fb7b819cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.617+0000 7fb7bdfa2700 1 --2- 172.21.15.38:0/1130831035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7b807bc80 0x7fb7b819cfb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009e50 tx=0x7fb7a800bd30 comp rx=0 tx=0).stop 2024-05-21T15:24:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.617+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1130831035 shutdown_connections 2024-05-21T15:24:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.617+0000 7fb7bdfa2700 1 --2- 172.21.15.38:0/1130831035 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fb7a4038ed0 0x7fb7a403b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.617+0000 7fb7bdfa2700 1 --2- 172.21.15.38:0/1130831035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7b807bc80 0x7fb7b819cfb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.617+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1130831035 >> 172.21.15.38:0/1130831035 conn(0x7fb7b81031b0 msgr2=0x7fb7b8104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.617+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1130831035 shutdown_connections 2024-05-21T15:24:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:56.617+0000 7fb7bdfa2700 1 -- 172.21.15.38:0/1130831035 wait complete. 2024-05-21T15:24:57.512 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:57 smithi038 ceph-mon[31995]: mgrmap e8: x(active, since 2s) 2024-05-21T15:24:57.512 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:57 smithi038 ceph-mon[31995]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:24:57.512 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:57 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' 2024-05-21T15:24:57.512 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:57 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:24:58.282 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-21T15:24:58.282 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.558+0000 7f93968ae700 1 Processor -- start 2024-05-21T15:24:58.282 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.558+0000 7f93968ae700 1 -- start start 2024-05-21T15:24:58.282 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.559+0000 7f93968ae700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9390107b00 0x7f9390107f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:58.282 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.559+0000 7f93968ae700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9390108500 con 0x7f9390107b00 2024-05-21T15:24:58.282 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.559+0000 7f938ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9390107b00 0x7f9390107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:58.282 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.559+0000 7f938ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9390107b00 0x7f9390107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41446/0 (socket says 172.21.15.38:41446) 2024-05-21T15:24:58.282 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.559+0000 7f938ffff700 1 -- 172.21.15.38:0/2475965884 learned_addr learned my addr 172.21.15.38:0/2475965884 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:58.283 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.559+0000 7f938ffff700 1 -- 172.21.15.38:0/2475965884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9390108c30 con 0x7f9390107b00 2024-05-21T15:24:58.283 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.559+0000 7f938ffff700 1 --2- 172.21.15.38:0/2475965884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9390107b00 0x7f9390107f10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f938000bb40 tx=0x7f938000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437862219b04cdfd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:58.283 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.560+0000 7f938effd700 1 -- 172.21.15.38:0/2475965884 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f938000dbe0 con 0x7f9390107b00 2024-05-21T15:24:58.283 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.560+0000 7f938effd700 1 -- 172.21.15.38:0/2475965884 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f93800044f0 con 0x7f9390107b00 2024-05-21T15:24:58.283 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.560+0000 7f938effd700 1 -- 172.21.15.38:0/2475965884 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9380010470 con 0x7f9390107b00 2024-05-21T15:24:58.283 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.560+0000 7f93968ae700 1 -- 172.21.15.38:0/2475965884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9390107b00 msgr2=0x7f9390107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:58.283 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.560+0000 7f93968ae700 1 --2- 172.21.15.38:0/2475965884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9390107b00 0x7f9390107f10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f938000bb40 tx=0x7f938000be50 comp rx=0 tx=0).stop 2024-05-21T15:24:58.283 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.560+0000 7f93968ae700 1 -- 172.21.15.38:0/2475965884 shutdown_connections 2024-05-21T15:24:58.283 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.560+0000 7f93968ae700 1 --2- 172.21.15.38:0/2475965884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9390107b00 0x7f9390107f10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:58.283 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.560+0000 7f93968ae700 1 -- 172.21.15.38:0/2475965884 >> 172.21.15.38:0/2475965884 conn(0x7f93901031e0 msgr2=0x7f9390105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:58.283 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.560+0000 7f93968ae700 1 -- 172.21.15.38:0/2475965884 shutdown_connections 2024-05-21T15:24:58.283 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.561+0000 7f93968ae700 1 -- 172.21.15.38:0/2475965884 wait complete. 2024-05-21T15:24:58.283 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.561+0000 7f93968ae700 1 Processor -- start 2024-05-21T15:24:58.283 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.561+0000 7f93968ae700 1 -- start start 2024-05-21T15:24:58.284 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.562+0000 7f93968ae700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9390107b00 0x7f939019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:58.284 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.562+0000 7f93968ae700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f939007b140 con 0x7f9390107b00 2024-05-21T15:24:58.284 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.562+0000 7f938ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9390107b00 0x7f939019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:58.284 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.562+0000 7f938ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9390107b00 0x7f939019f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41450/0 (socket says 172.21.15.38:41450) 2024-05-21T15:24:58.284 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.562+0000 7f938ffff700 1 -- 172.21.15.38:0/26885690 learned_addr learned my addr 172.21.15.38:0/26885690 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:24:58.284 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.562+0000 7f938ffff700 1 -- 172.21.15.38:0/26885690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939019f840 con 0x7f9390107b00 2024-05-21T15:24:58.284 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.562+0000 7f938ffff700 1 --2- 172.21.15.38:0/26885690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9390107b00 0x7f939019f300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9380003c50 tx=0x7f93800040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:58.284 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.562+0000 7f938d7fa700 1 -- 172.21.15.38:0/26885690 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f938000de20 con 0x7f9390107b00 2024-05-21T15:24:58.284 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.562+0000 7f938d7fa700 1 -- 172.21.15.38:0/26885690 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f938001d070 con 0x7f9390107b00 2024-05-21T15:24:58.284 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.562+0000 7f93968ae700 1 -- 172.21.15.38:0/26885690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939019fad0 con 0x7f9390107b00 2024-05-21T15:24:58.284 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.562+0000 7f938d7fa700 1 -- 172.21.15.38:0/26885690 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9380019bb0 con 0x7f9390107b00 2024-05-21T15:24:58.284 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.562+0000 7f93968ae700 1 -- 172.21.15.38:0/26885690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93901a26e0 con 0x7f9390107b00 2024-05-21T15:24:58.284 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.563+0000 7f938d7fa700 1 -- 172.21.15.38:0/26885690 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f938001f030 con 0x7f9390107b00 2024-05-21T15:24:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.563+0000 7f93968ae700 1 -- 172.21.15.38:0/26885690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9390063ca0 con 0x7f9390107b00 2024-05-21T15:24:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.564+0000 7f938d7fa700 1 --2- 172.21.15.38:0/26885690 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f93780391f0 0x7f937803b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.564+0000 7f938d7fa700 1 -- 172.21.15.38:0/26885690 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f938004d510 con 0x7f9390107b00 2024-05-21T15:24:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.564+0000 7f938f7fe700 1 --2- 172.21.15.38:0/26885690 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f93780391f0 0x7f937803b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.564+0000 7f938f7fe700 1 --2- 172.21.15.38:0/26885690 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f93780391f0 0x7f937803b700 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9384006fd0 tx=0x7f9384006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.569+0000 7f938d7fa700 1 -- 172.21.15.38:0/26885690 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9380015070 con 0x7f9390107b00 2024-05-21T15:24:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.777+0000 7f93968ae700 1 -- 172.21.15.38:0/26885690 --> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9390062a80 con 0x7f93780391f0 2024-05-21T15:24:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.778+0000 7f938d7fa700 1 -- 172.21.15.38:0/26885690 <== mgr.14120 v2:172.21.15.38:6800/1481949523 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9390062a80 con 0x7f93780391f0 2024-05-21T15:24:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.780+0000 7f93968ae700 1 -- 172.21.15.38:0/26885690 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f93780391f0 msgr2=0x7f937803b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.780+0000 7f93968ae700 1 --2- 172.21.15.38:0/26885690 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f93780391f0 0x7f937803b700 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9384006fd0 tx=0x7f9384006e40 comp rx=0 tx=0).stop 2024-05-21T15:24:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.780+0000 7f93968ae700 1 -- 172.21.15.38:0/26885690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9390107b00 msgr2=0x7f939019f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.780+0000 7f93968ae700 1 --2- 172.21.15.38:0/26885690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9390107b00 0x7f939019f300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9380003c50 tx=0x7f93800040b0 comp rx=0 tx=0).stop 2024-05-21T15:24:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.780+0000 7f93968ae700 1 -- 172.21.15.38:0/26885690 shutdown_connections 2024-05-21T15:24:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.780+0000 7f93968ae700 1 --2- 172.21.15.38:0/26885690 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f93780391f0 0x7f937803b700 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:58.286 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.780+0000 7f93968ae700 1 --2- 172.21.15.38:0/26885690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9390107b00 0x7f939019f300 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:58.286 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.780+0000 7f93968ae700 1 -- 172.21.15.38:0/26885690 >> 172.21.15.38:0/26885690 conn(0x7f93901031e0 msgr2=0x7f939018f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:58.286 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.780+0000 7f93968ae700 1 -- 172.21.15.38:0/26885690 shutdown_connections 2024-05-21T15:24:58.286 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:57.780+0000 7f93968ae700 1 -- 172.21.15.38:0/26885690 wait complete. 2024-05-21T15:24:58.286 INFO:teuthology.orchestra.run.smithi038.stdout:Generating ssh key... 2024-05-21T15:24:58.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:58 smithi038 ceph-mon[31995]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:24:59.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:59 smithi038 ceph-mon[31995]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:24:59.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:24:59 smithi038 ceph-mon[31995]: Generating ssh key... 2024-05-21T15:24:59.779 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: Generating public/private rsa key pair. 2024-05-21T15:24:59.779 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: Your identification has been saved in /tmp/tmpgaca5lny/key. 2024-05-21T15:24:59.779 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: Your public key has been saved in /tmp/tmpgaca5lny/key.pub. 2024-05-21T15:24:59.779 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: The key fingerprint is: 2024-05-21T15:24:59.779 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: SHA256:8bqbXJycx3C9jbGxvb9xeCnyKWndawEPRxkI2hO8Wcw ceph-102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:24:59.779 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: The key's randomart image is: 2024-05-21T15:24:59.779 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: +---[RSA 3072]----+ 2024-05-21T15:24:59.779 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: | .o+ ..o| 2024-05-21T15:24:59.779 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: | o..E o | 2024-05-21T15:24:59.779 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: | .. o+ . | 2024-05-21T15:24:59.779 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: | o o.+ . | 2024-05-21T15:24:59.780 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: | S o . O | 2024-05-21T15:24:59.780 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: | + * &.| 2024-05-21T15:24:59.780 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: | . *.=.Bo*| 2024-05-21T15:24:59.780 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: | . + =o.o++| 2024-05-21T15:24:59.780 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: | =.. .o.+=| 2024-05-21T15:24:59.780 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:24:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: +----[SHA256]-----+ 2024-05-21T15:25:00.130 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.688+0000 7f768fcaa700 1 Processor -- start 2024-05-21T15:25:00.130 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.688+0000 7f768fcaa700 1 -- start start 2024-05-21T15:25:00.130 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.688+0000 7f768fcaa700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7688108c70 0x7f7688109080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.130 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.688+0000 7f768fcaa700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7688109670 con 0x7f7688108c70 2024-05-21T15:25:00.131 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.688+0000 7f768da46700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7688108c70 0x7f7688109080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.131 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.689+0000 7f768da46700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7688108c70 0x7f7688109080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41456/0 (socket says 172.21.15.38:41456) 2024-05-21T15:25:00.131 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.689+0000 7f768da46700 1 -- 172.21.15.38:0/2562041231 learned_addr learned my addr 172.21.15.38:0/2562041231 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:00.131 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.689+0000 7f768da46700 1 -- 172.21.15.38:0/2562041231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7688109da0 con 0x7f7688108c70 2024-05-21T15:25:00.131 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.689+0000 7f768da46700 1 --2- 172.21.15.38:0/2562041231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7688108c70 0x7f7688109080 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f76840098a0 tx=0x7f7684009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287897d3e88e0c18 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.131 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.689+0000 7f768ca44700 1 -- 172.21.15.38:0/2562041231 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7684004160 con 0x7f7688108c70 2024-05-21T15:25:00.131 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.689+0000 7f768ca44700 1 -- 172.21.15.38:0/2562041231 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7684004300 con 0x7f7688108c70 2024-05-21T15:25:00.131 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.689+0000 7f768ca44700 1 -- 172.21.15.38:0/2562041231 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7684011b50 con 0x7f7688108c70 2024-05-21T15:25:00.131 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.690+0000 7f768fcaa700 1 -- 172.21.15.38:0/2562041231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7688108c70 msgr2=0x7f7688109080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.131 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.690+0000 7f768fcaa700 1 --2- 172.21.15.38:0/2562041231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7688108c70 0x7f7688109080 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f76840098a0 tx=0x7f7684009bb0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.131 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.690+0000 7f768fcaa700 1 -- 172.21.15.38:0/2562041231 shutdown_connections 2024-05-21T15:25:00.131 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.690+0000 7f768fcaa700 1 --2- 172.21.15.38:0/2562041231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7688108c70 0x7f7688109080 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.131 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.690+0000 7f768fcaa700 1 -- 172.21.15.38:0/2562041231 >> 172.21.15.38:0/2562041231 conn(0x7f76881042d0 msgr2=0x7f7688106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.131 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.690+0000 7f768fcaa700 1 -- 172.21.15.38:0/2562041231 shutdown_connections 2024-05-21T15:25:00.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.690+0000 7f768fcaa700 1 -- 172.21.15.38:0/2562041231 wait complete. 2024-05-21T15:25:00.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.691+0000 7f768fcaa700 1 Processor -- start 2024-05-21T15:25:00.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.691+0000 7f768fcaa700 1 -- start start 2024-05-21T15:25:00.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.691+0000 7f768fcaa700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7688108c70 0x7f768819d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.691+0000 7f768fcaa700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f768810d150 con 0x7f7688108c70 2024-05-21T15:25:00.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.691+0000 7f768da46700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7688108c70 0x7f768819d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.691+0000 7f768da46700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7688108c70 0x7f768819d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41466/0 (socket says 172.21.15.38:41466) 2024-05-21T15:25:00.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.691+0000 7f768da46700 1 -- 172.21.15.38:0/3253558010 learned_addr learned my addr 172.21.15.38:0/3253558010 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:00.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.692+0000 7f768da46700 1 -- 172.21.15.38:0/3253558010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768819d630 con 0x7f7688108c70 2024-05-21T15:25:00.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.692+0000 7f768da46700 1 --2- 172.21.15.38:0/3253558010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7688108c70 0x7f768819d0f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7684011710 tx=0x7f7684011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.692+0000 7f767effd700 1 -- 172.21.15.38:0/3253558010 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76840038b0 con 0x7f7688108c70 2024-05-21T15:25:00.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.692+0000 7f768fcaa700 1 -- 172.21.15.38:0/3253558010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768819d8c0 con 0x7f7688108c70 2024-05-21T15:25:00.132 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.692+0000 7f767effd700 1 -- 172.21.15.38:0/3253558010 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7684011e50 con 0x7f7688108c70 2024-05-21T15:25:00.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.692+0000 7f767effd700 1 -- 172.21.15.38:0/3253558010 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76840193c0 con 0x7f7688108c70 2024-05-21T15:25:00.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.692+0000 7f768fcaa700 1 -- 172.21.15.38:0/3253558010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768807c960 con 0x7f7688108c70 2024-05-21T15:25:00.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.693+0000 7f767effd700 1 -- 172.21.15.38:0/3253558010 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7684019680 con 0x7f7688108c70 2024-05-21T15:25:00.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.693+0000 7f768fcaa700 1 -- 172.21.15.38:0/3253558010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7688063ca0 con 0x7f7688108c70 2024-05-21T15:25:00.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.693+0000 7f767effd700 1 --2- 172.21.15.38:0/3253558010 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f767403d410 0x7f767403f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.693+0000 7f767effd700 1 -- 172.21.15.38:0/3253558010 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f768404d1c0 con 0x7f7688108c70 2024-05-21T15:25:00.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.693+0000 7f768d245700 1 --2- 172.21.15.38:0/3253558010 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f767403d410 0x7f767403f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.694+0000 7f768d245700 1 --2- 172.21.15.38:0/3253558010 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f767403d410 0x7f767403f920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7678006fd0 tx=0x7f7678006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.698+0000 7f767effd700 1 -- 172.21.15.38:0/3253558010 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7684020240 con 0x7f7688108c70 2024-05-21T15:25:00.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:58.905+0000 7f768fcaa700 1 -- 172.21.15.38:0/3253558010 --> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f7688105490 con 0x7f767403d410 2024-05-21T15:25:00.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:59.494+0000 7f767effd700 1 -- 172.21.15.38:0/3253558010 <== mgr.14120 v2:172.21.15.38:6800/1481949523 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7688105490 con 0x7f767403d410 2024-05-21T15:25:00.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:59.496+0000 7f768fcaa700 1 -- 172.21.15.38:0/3253558010 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f767403d410 msgr2=0x7f767403f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:59.496+0000 7f768fcaa700 1 --2- 172.21.15.38:0/3253558010 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f767403d410 0x7f767403f920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7678006fd0 tx=0x7f7678006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:00.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:59.496+0000 7f768fcaa700 1 -- 172.21.15.38:0/3253558010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7688108c70 msgr2=0x7f768819d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:59.496+0000 7f768fcaa700 1 --2- 172.21.15.38:0/3253558010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7688108c70 0x7f768819d0f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7684011710 tx=0x7f7684011c10 comp rx=0 tx=0).stop 2024-05-21T15:25:00.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:59.496+0000 7f768fcaa700 1 -- 172.21.15.38:0/3253558010 shutdown_connections 2024-05-21T15:25:00.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:59.496+0000 7f768fcaa700 1 --2- 172.21.15.38:0/3253558010 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f767403d410 0x7f767403f920 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:59.496+0000 7f768fcaa700 1 --2- 172.21.15.38:0/3253558010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7688108c70 0x7f768819d0f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:59.496+0000 7f768fcaa700 1 -- 172.21.15.38:0/3253558010 >> 172.21.15.38:0/3253558010 conn(0x7f76881042d0 msgr2=0x7f7688104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:59.497+0000 7f768fcaa700 1 -- 172.21.15.38:0/3253558010 shutdown_connections 2024-05-21T15:25:00.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:24:59.497+0000 7f768fcaa700 1 -- 172.21.15.38:0/3253558010 wait complete. 2024-05-21T15:25:00.766 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:00 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' 2024-05-21T15:25:00.766 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:00 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' 2024-05-21T15:25:01.338 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcR17PAkEDw2SCQ3e4KouhkPPSB7TBcTjeVLnBiYvGKYaJ7pDQ95chFcKonzFxfnDpoElb1OfAVEzg+2pNzxs0j8ayHtUUlbJJo6qRi6bUEBfANJudXawfmpjXDVOHTpfvK4duLXQQB2jRtraffoLhUypuXlRMXyDJJ3VshkeyDgpTSh5gPrXOJgUJyPe6peKtyLPvAK31OVdoQiD4/2Tyn4x5JBdde97EqLnpQpwDuj7S9ycR2LDytnrKaKZ8+/0DHmV0328K6PKJg+OksPFZb1XOEa9rjPLP8AyprOLbeRjKEC+f/OL/hPot8zSDxdoTo5McYUK43w86CmHcBveE2LbsZzTSuxDI1qE1qY7BNNjuHGp7n+5BRAKmH2WPSCoIMS/tUCEvyubG7lL6SGX8dIOAQ3BUWshf+eS8I0GoVtq5+8+UjTdLMNRFRJlKyzA40JmaUQ60kZB8gtxOej63JVaE70o9KDPaYxYPkqo32Q1Y42TGygacYGDCyfGNTuU= ceph-102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:25:01.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.544+0000 7f0bc339a700 1 Processor -- start 2024-05-21T15:25:01.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.544+0000 7f0bc339a700 1 -- start start 2024-05-21T15:25:01.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.544+0000 7f0bc339a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bbc07c550 0x7f0bbc10b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.544+0000 7f0bc339a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc10bdc0 con 0x7f0bbc07c550 2024-05-21T15:25:01.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.544+0000 7f0bc1136700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bbc07c550 0x7f0bbc10b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.544+0000 7f0bc1136700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bbc07c550 0x7f0bbc10b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41474/0 (socket says 172.21.15.38:41474) 2024-05-21T15:25:01.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.544+0000 7f0bc1136700 1 -- 172.21.15.38:0/3018751829 learned_addr learned my addr 172.21.15.38:0/3018751829 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:01.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.545+0000 7f0bc1136700 1 -- 172.21.15.38:0/3018751829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc10bf40 con 0x7f0bbc07c550 2024-05-21T15:25:01.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.545+0000 7f0bc1136700 1 --2- 172.21.15.38:0/3018751829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bbc07c550 0x7f0bbc10b7b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80098a0 tx=0x7f0bb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cae833497be2af server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.545+0000 7f0bb3fff700 1 -- 172.21.15.38:0/3018751829 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bb8004160 con 0x7f0bbc07c550 2024-05-21T15:25:01.339 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.545+0000 7f0bb3fff700 1 -- 172.21.15.38:0/3018751829 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0bb8004300 con 0x7f0bbc07c550 2024-05-21T15:25:01.340 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.545+0000 7f0bb3fff700 1 -- 172.21.15.38:0/3018751829 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bb8011b50 con 0x7f0bbc07c550 2024-05-21T15:25:01.340 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.546+0000 7f0bc339a700 1 -- 172.21.15.38:0/3018751829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bbc07c550 msgr2=0x7f0bbc10b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.340 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.546+0000 7f0bc339a700 1 --2- 172.21.15.38:0/3018751829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bbc07c550 0x7f0bbc10b7b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80098a0 tx=0x7f0bb8009bb0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.340 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.546+0000 7f0bc339a700 1 -- 172.21.15.38:0/3018751829 shutdown_connections 2024-05-21T15:25:01.340 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.546+0000 7f0bc339a700 1 --2- 172.21.15.38:0/3018751829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bbc07c550 0x7f0bbc10b7b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.340 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.546+0000 7f0bc339a700 1 -- 172.21.15.38:0/3018751829 >> 172.21.15.38:0/3018751829 conn(0x7f0bbc103220 msgr2=0x7f0bbc105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:01.340 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.546+0000 7f0bc339a700 1 -- 172.21.15.38:0/3018751829 shutdown_connections 2024-05-21T15:25:01.340 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.546+0000 7f0bc339a700 1 -- 172.21.15.38:0/3018751829 wait complete. 2024-05-21T15:25:01.340 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.547+0000 7f0bc339a700 1 Processor -- start 2024-05-21T15:25:01.340 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.547+0000 7f0bc339a700 1 -- start start 2024-05-21T15:25:01.340 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.547+0000 7f0bc339a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bbc07c550 0x7f0bbc19d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.340 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.547+0000 7f0bc339a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc10d020 con 0x7f0bbc07c550 2024-05-21T15:25:01.340 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.547+0000 7f0bc1136700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bbc07c550 0x7f0bbc19d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.340 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.547+0000 7f0bc1136700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bbc07c550 0x7f0bbc19d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41478/0 (socket says 172.21.15.38:41478) 2024-05-21T15:25:01.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.547+0000 7f0bc1136700 1 -- 172.21.15.38:0/191455899 learned_addr learned my addr 172.21.15.38:0/191455899 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:01.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.547+0000 7f0bc1136700 1 -- 172.21.15.38:0/191455899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc19d5e0 con 0x7f0bbc07c550 2024-05-21T15:25:01.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.548+0000 7f0bc1136700 1 --2- 172.21.15.38:0/191455899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bbc07c550 0x7f0bbc19d0a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009fb0 tx=0x7f0bb8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.548+0000 7f0bb27fc700 1 -- 172.21.15.38:0/191455899 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bb80038b0 con 0x7f0bbc07c550 2024-05-21T15:25:01.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.548+0000 7f0bb27fc700 1 -- 172.21.15.38:0/191455899 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0bb8011e50 con 0x7f0bbc07c550 2024-05-21T15:25:01.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.548+0000 7f0bc339a700 1 -- 172.21.15.38:0/191455899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bbc19d870 con 0x7f0bbc07c550 2024-05-21T15:25:01.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.548+0000 7f0bb27fc700 1 -- 172.21.15.38:0/191455899 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bb801a430 con 0x7f0bbc07c550 2024-05-21T15:25:01.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.548+0000 7f0bc339a700 1 -- 172.21.15.38:0/191455899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bbc1a0480 con 0x7f0bbc07c550 2024-05-21T15:25:01.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.549+0000 7f0bc339a700 1 -- 172.21.15.38:0/191455899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bbc063ca0 con 0x7f0bbc07c550 2024-05-21T15:25:01.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.549+0000 7f0bb27fc700 1 -- 172.21.15.38:0/191455899 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0bb801a6e0 con 0x7f0bbc07c550 2024-05-21T15:25:01.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.549+0000 7f0bb27fc700 1 --2- 172.21.15.38:0/191455899 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f0ba8039050 0x7f0ba803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.549+0000 7f0bb27fc700 1 -- 172.21.15.38:0/191455899 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0bb804d3c0 con 0x7f0bbc07c550 2024-05-21T15:25:01.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.549+0000 7f0bc0935700 1 --2- 172.21.15.38:0/191455899 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f0ba8039050 0x7f0ba803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.550+0000 7f0bc0935700 1 --2- 172.21.15.38:0/191455899 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f0ba8039050 0x7f0ba803b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0bac006fd0 tx=0x7f0bac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.554+0000 7f0bb27fc700 1 -- 172.21.15.38:0/191455899 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0bb804cd50 con 0x7f0bbc07c550 2024-05-21T15:25:01.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.762+0000 7f0bc339a700 1 -- 172.21.15.38:0/191455899 --> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f0bbc104330 con 0x7f0ba8039050 2024-05-21T15:25:01.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.763+0000 7f0bb27fc700 1 -- 172.21.15.38:0/191455899 <== mgr.14120 v2:172.21.15.38:6800/1481949523 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f0bbc104330 con 0x7f0ba8039050 2024-05-21T15:25:01.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.765+0000 7f0bc339a700 1 -- 172.21.15.38:0/191455899 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f0ba8039050 msgr2=0x7f0ba803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.765+0000 7f0bc339a700 1 --2- 172.21.15.38:0/191455899 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f0ba8039050 0x7f0ba803b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0bac006fd0 tx=0x7f0bac006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:01.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.765+0000 7f0bc339a700 1 -- 172.21.15.38:0/191455899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bbc07c550 msgr2=0x7f0bbc19d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.765+0000 7f0bc339a700 1 --2- 172.21.15.38:0/191455899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bbc07c550 0x7f0bbc19d0a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009fb0 tx=0x7f0bb8011b60 comp rx=0 tx=0).stop 2024-05-21T15:25:01.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.766+0000 7f0bc339a700 1 -- 172.21.15.38:0/191455899 shutdown_connections 2024-05-21T15:25:01.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.766+0000 7f0bc339a700 1 --2- 172.21.15.38:0/191455899 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f0ba8039050 0x7f0ba803b560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.766+0000 7f0bc339a700 1 --2- 172.21.15.38:0/191455899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bbc07c550 0x7f0bbc19d0a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.766+0000 7f0bc339a700 1 -- 172.21.15.38:0/191455899 >> 172.21.15.38:0/191455899 conn(0x7f0bbc103220 msgr2=0x7f0bbc103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:01.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.766+0000 7f0bc339a700 1 -- 172.21.15.38:0/191455899 shutdown_connections 2024-05-21T15:25:01.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:00.766+0000 7f0bc339a700 1 -- 172.21.15.38:0/191455899 wait complete. 2024-05-21T15:25:01.343 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-21T15:25:01.343 INFO:teuthology.orchestra.run.smithi038.stdout:Adding key to root@localhost authorized_keys... 2024-05-21T15:25:01.343 INFO:teuthology.orchestra.run.smithi038.stdout:Adding host smithi038... 2024-05-21T15:25:01.592 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:01 smithi038 ceph-mon[31995]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:25:02.653 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:02 smithi038 ceph-mon[31995]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi038", "addr": "172.21.15.38", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:25:03.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:03 smithi038 ceph-mon[31995]: Deploying cephadm binary to smithi038 2024-05-21T15:25:06.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Added host 'smithi038' with addr '172.21.15.38' 2024-05-21T15:25:06.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.816+0000 7f117f66e700 1 Processor -- start 2024-05-21T15:25:06.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.816+0000 7f117f66e700 1 -- start start 2024-05-21T15:25:06.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.816+0000 7f117f66e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1178108c60 0x7f1178109070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:06.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.816+0000 7f117f66e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1178109660 con 0x7f1178108c60 2024-05-21T15:25:06.845 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.817+0000 7f117d40a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1178108c60 0x7f1178109070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:06.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.817+0000 7f117d40a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1178108c60 0x7f1178109070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41482/0 (socket says 172.21.15.38:41482) 2024-05-21T15:25:06.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.817+0000 7f117d40a700 1 -- 172.21.15.38:0/3019372265 learned_addr learned my addr 172.21.15.38:0/3019372265 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:06.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.817+0000 7f117d40a700 1 -- 172.21.15.38:0/3019372265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1178109d90 con 0x7f1178108c60 2024-05-21T15:25:06.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.817+0000 7f117d40a700 1 --2- 172.21.15.38:0/3019372265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1178108c60 0x7f1178109070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f11740098a0 tx=0x7f1174009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fab946e748fe2c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:06.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.817+0000 7f116ffff700 1 -- 172.21.15.38:0/3019372265 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1174004160 con 0x7f1178108c60 2024-05-21T15:25:06.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.817+0000 7f116ffff700 1 -- 172.21.15.38:0/3019372265 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1174004300 con 0x7f1178108c60 2024-05-21T15:25:06.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.817+0000 7f116ffff700 1 -- 172.21.15.38:0/3019372265 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1174011b50 con 0x7f1178108c60 2024-05-21T15:25:06.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.818+0000 7f117f66e700 1 -- 172.21.15.38:0/3019372265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1178108c60 msgr2=0x7f1178109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:06.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.818+0000 7f117f66e700 1 --2- 172.21.15.38:0/3019372265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1178108c60 0x7f1178109070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f11740098a0 tx=0x7f1174009bb0 comp rx=0 tx=0).stop 2024-05-21T15:25:06.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.818+0000 7f117f66e700 1 -- 172.21.15.38:0/3019372265 shutdown_connections 2024-05-21T15:25:06.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.818+0000 7f117f66e700 1 --2- 172.21.15.38:0/3019372265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1178108c60 0x7f1178109070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:06.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.818+0000 7f117f66e700 1 -- 172.21.15.38:0/3019372265 >> 172.21.15.38:0/3019372265 conn(0x7f1178104340 msgr2=0x7f1178106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:06.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.818+0000 7f117f66e700 1 -- 172.21.15.38:0/3019372265 shutdown_connections 2024-05-21T15:25:06.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.818+0000 7f117f66e700 1 -- 172.21.15.38:0/3019372265 wait complete. 2024-05-21T15:25:06.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.819+0000 7f117f66e700 1 Processor -- start 2024-05-21T15:25:06.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.819+0000 7f117f66e700 1 -- start start 2024-05-21T15:25:06.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.819+0000 7f117f66e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1178108c60 0x7f117819d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:06.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.819+0000 7f117f66e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f117810d140 con 0x7f1178108c60 2024-05-21T15:25:06.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.819+0000 7f117d40a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1178108c60 0x7f117819d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:06.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.819+0000 7f117d40a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1178108c60 0x7f117819d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41486/0 (socket says 172.21.15.38:41486) 2024-05-21T15:25:06.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.819+0000 7f117d40a700 1 -- 172.21.15.38:0/3935983183 learned_addr learned my addr 172.21.15.38:0/3935983183 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:06.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.820+0000 7f117d40a700 1 -- 172.21.15.38:0/3935983183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117819d640 con 0x7f1178108c60 2024-05-21T15:25:06.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.820+0000 7f117d40a700 1 --2- 172.21.15.38:0/3935983183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1178108c60 0x7f117819d100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f11740046e0 tx=0x7f1174011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:06.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.820+0000 7f116e7fc700 1 -- 172.21.15.38:0/3935983183 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11740039e0 con 0x7f1178108c60 2024-05-21T15:25:06.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.820+0000 7f116e7fc700 1 -- 172.21.15.38:0/3935983183 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1174003b80 con 0x7f1178108c60 2024-05-21T15:25:06.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.820+0000 7f117f66e700 1 -- 172.21.15.38:0/3935983183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117819d8d0 con 0x7f1178108c60 2024-05-21T15:25:06.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.820+0000 7f116e7fc700 1 -- 172.21.15.38:0/3935983183 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1174024650 con 0x7f1178108c60 2024-05-21T15:25:06.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.820+0000 7f117f66e700 1 -- 172.21.15.38:0/3935983183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11781a04e0 con 0x7f1178108c60 2024-05-21T15:25:06.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.821+0000 7f116e7fc700 1 -- 172.21.15.38:0/3935983183 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1174022020 con 0x7f1178108c60 2024-05-21T15:25:06.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.821+0000 7f117f66e700 1 -- 172.21.15.38:0/3935983183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1178063ca0 con 0x7f1178108c60 2024-05-21T15:25:06.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.821+0000 7f116e7fc700 1 --2- 172.21.15.38:0/3935983183 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f11640390a0 0x7f116403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:06.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.821+0000 7f116e7fc700 1 -- 172.21.15.38:0/3935983183 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1174011e00 con 0x7f1178108c60 2024-05-21T15:25:06.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.822+0000 7f117cc09700 1 --2- 172.21.15.38:0/3935983183 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f11640390a0 0x7f116403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:06.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.822+0000 7f117cc09700 1 --2- 172.21.15.38:0/3935983183 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f11640390a0 0x7f116403b5b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1168006fd0 tx=0x7f1168006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:06.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:01.827+0000 7f116e7fc700 1 -- 172.21.15.38:0/3935983183 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1174003d30 con 0x7f1178108c60 2024-05-21T15:25:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:02.037+0000 7f117f66e700 1 -- 172.21.15.38:0/3935983183 --> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi038", "addr": "172.21.15.38", "target": ["mon-mgr", ""]}) -- 0x7f11781065e0 con 0x7f11640390a0 2024-05-21T15:25:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:06.293+0000 7f116e7fc700 1 -- 172.21.15.38:0/3935983183 <== mgr.14120 v2:172.21.15.38:6800/1481949523 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f11781065e0 con 0x7f11640390a0 2024-05-21T15:25:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:06.296+0000 7f117f66e700 1 -- 172.21.15.38:0/3935983183 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f11640390a0 msgr2=0x7f116403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:06.296+0000 7f117f66e700 1 --2- 172.21.15.38:0/3935983183 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f11640390a0 0x7f116403b5b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1168006fd0 tx=0x7f1168006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:06.296+0000 7f117f66e700 1 -- 172.21.15.38:0/3935983183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1178108c60 msgr2=0x7f117819d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:06.296+0000 7f117f66e700 1 --2- 172.21.15.38:0/3935983183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1178108c60 0x7f117819d100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f11740046e0 tx=0x7f1174011c10 comp rx=0 tx=0).stop 2024-05-21T15:25:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:06.296+0000 7f117f66e700 1 -- 172.21.15.38:0/3935983183 shutdown_connections 2024-05-21T15:25:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:06.296+0000 7f117f66e700 1 --2- 172.21.15.38:0/3935983183 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f11640390a0 0x7f116403b5b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:06.296+0000 7f117f66e700 1 --2- 172.21.15.38:0/3935983183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1178108c60 0x7f117819d100 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:06.296+0000 7f117f66e700 1 -- 172.21.15.38:0/3935983183 >> 172.21.15.38:0/3935983183 conn(0x7f1178104340 msgr2=0x7f1178105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:06.296+0000 7f117f66e700 1 -- 172.21.15.38:0/3935983183 shutdown_connections 2024-05-21T15:25:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:06.296+0000 7f117f66e700 1 -- 172.21.15.38:0/3935983183 wait complete. 2024-05-21T15:25:06.849 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying unmanaged mon service... 2024-05-21T15:25:07.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:07 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' 2024-05-21T15:25:07.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:07 smithi038 ceph-mon[31995]: Added host smithi038 2024-05-21T15:25:07.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:07 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:25:08.235 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-21T15:25:08.236 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.322+0000 7f8485fd9700 1 Processor -- start 2024-05-21T15:25:08.236 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.322+0000 7f8485fd9700 1 -- start start 2024-05-21T15:25:08.236 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.322+0000 7f8485fd9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f848007c550 0x7f848010b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:08.241 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.322+0000 7f8485fd9700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f848010bdd0 con 0x7f848007c550 2024-05-21T15:25:08.241 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.322+0000 7f847f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f848007c550 0x7f848010b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:08.241 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.323+0000 7f847f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f848007c550 0x7f848010b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50834/0 (socket says 172.21.15.38:50834) 2024-05-21T15:25:08.241 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.323+0000 7f847f7fe700 1 -- 172.21.15.38:0/3947122012 learned_addr learned my addr 172.21.15.38:0/3947122012 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:08.241 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.323+0000 7f847f7fe700 1 -- 172.21.15.38:0/3947122012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848010bf50 con 0x7f848007c550 2024-05-21T15:25:08.241 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.323+0000 7f847f7fe700 1 --2- 172.21.15.38:0/3947122012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f848007c550 0x7f848010b7c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f84700098a0 tx=0x7f8470009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31eb6af665c9c542 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:08.241 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.323+0000 7f847e7fc700 1 -- 172.21.15.38:0/3947122012 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8470004160 con 0x7f848007c550 2024-05-21T15:25:08.241 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.323+0000 7f847e7fc700 1 -- 172.21.15.38:0/3947122012 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8470004300 con 0x7f848007c550 2024-05-21T15:25:08.241 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.323+0000 7f847e7fc700 1 -- 172.21.15.38:0/3947122012 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8470011b50 con 0x7f848007c550 2024-05-21T15:25:08.242 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.324+0000 7f8485fd9700 1 -- 172.21.15.38:0/3947122012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f848007c550 msgr2=0x7f848010b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:08.242 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.324+0000 7f8485fd9700 1 --2- 172.21.15.38:0/3947122012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f848007c550 0x7f848010b7c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f84700098a0 tx=0x7f8470009bb0 comp rx=0 tx=0).stop 2024-05-21T15:25:08.242 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.324+0000 7f8485fd9700 1 -- 172.21.15.38:0/3947122012 shutdown_connections 2024-05-21T15:25:08.242 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.324+0000 7f8485fd9700 1 --2- 172.21.15.38:0/3947122012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f848007c550 0x7f848010b7c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:08.242 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.324+0000 7f8485fd9700 1 -- 172.21.15.38:0/3947122012 >> 172.21.15.38:0/3947122012 conn(0x7f8480103270 msgr2=0x7f84801056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:08.242 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.324+0000 7f8485fd9700 1 -- 172.21.15.38:0/3947122012 shutdown_connections 2024-05-21T15:25:08.242 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.324+0000 7f8485fd9700 1 -- 172.21.15.38:0/3947122012 wait complete. 2024-05-21T15:25:08.242 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.325+0000 7f8485fd9700 1 Processor -- start 2024-05-21T15:25:08.242 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.325+0000 7f8485fd9700 1 -- start start 2024-05-21T15:25:08.242 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.325+0000 7f8485fd9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f848007c550 0x7f848019d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:08.242 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.325+0000 7f8485fd9700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f848010d030 con 0x7f848007c550 2024-05-21T15:25:08.242 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.325+0000 7f847f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f848007c550 0x7f848019d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:08.242 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.325+0000 7f847f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f848007c550 0x7f848019d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50840/0 (socket says 172.21.15.38:50840) 2024-05-21T15:25:08.242 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.325+0000 7f847f7fe700 1 -- 172.21.15.38:0/1154760479 learned_addr learned my addr 172.21.15.38:0/1154760479 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:08.243 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.325+0000 7f847f7fe700 1 -- 172.21.15.38:0/1154760479 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848019d5c0 con 0x7f848007c550 2024-05-21T15:25:08.243 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.326+0000 7f847f7fe700 1 --2- 172.21.15.38:0/1154760479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f848007c550 0x7f848019d080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8470009fb0 tx=0x7f8470011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:08.243 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.326+0000 7f847cff9700 1 -- 172.21.15.38:0/1154760479 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84700038b0 con 0x7f848007c550 2024-05-21T15:25:08.243 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.326+0000 7f847cff9700 1 -- 172.21.15.38:0/1154760479 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8470011e50 con 0x7f848007c550 2024-05-21T15:25:08.243 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.326+0000 7f8485fd9700 1 -- 172.21.15.38:0/1154760479 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848019d850 con 0x7f848007c550 2024-05-21T15:25:08.243 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.326+0000 7f847cff9700 1 -- 172.21.15.38:0/1154760479 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f847001a430 con 0x7f848007c550 2024-05-21T15:25:08.243 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.326+0000 7f8485fd9700 1 -- 172.21.15.38:0/1154760479 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84801a0460 con 0x7f848007c550 2024-05-21T15:25:08.243 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.327+0000 7f847cff9700 1 -- 172.21.15.38:0/1154760479 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f847001a730 con 0x7f848007c550 2024-05-21T15:25:08.243 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.327+0000 7f847cff9700 1 --2- 172.21.15.38:0/1154760479 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8468039000 0x7f846803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:08.243 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.327+0000 7f847cff9700 1 -- 172.21.15.38:0/1154760479 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f847004e220 con 0x7f848007c550 2024-05-21T15:25:08.243 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.327+0000 7f847effd700 1 --2- 172.21.15.38:0/1154760479 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8468039000 0x7f846803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:08.243 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.327+0000 7f8485fd9700 1 -- 172.21.15.38:0/1154760479 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8480063ca0 con 0x7f848007c550 2024-05-21T15:25:08.243 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.327+0000 7f847effd700 1 --2- 172.21.15.38:0/1154760479 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8468039000 0x7f846803b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8474006fd0 tx=0x7f8474006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:08.243 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.332+0000 7f847cff9700 1 -- 172.21.15.38:0/1154760479 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f84700172b0 con 0x7f848007c550 2024-05-21T15:25:08.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.565+0000 7f8485fd9700 1 -- 172.21.15.38:0/1154760479 --> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8480105400 con 0x7f8468039000 2024-05-21T15:25:08.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.570+0000 7f847cff9700 1 -- 172.21.15.38:0/1154760479 <== mgr.14120 v2:172.21.15.38:6800/1481949523 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8480105400 con 0x7f8468039000 2024-05-21T15:25:08.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.573+0000 7f8485fd9700 1 -- 172.21.15.38:0/1154760479 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8468039000 msgr2=0x7f846803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:08.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.573+0000 7f8485fd9700 1 --2- 172.21.15.38:0/1154760479 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8468039000 0x7f846803b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8474006fd0 tx=0x7f8474006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:08.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.573+0000 7f8485fd9700 1 -- 172.21.15.38:0/1154760479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f848007c550 msgr2=0x7f848019d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:08.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.573+0000 7f8485fd9700 1 --2- 172.21.15.38:0/1154760479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f848007c550 0x7f848019d080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8470009fb0 tx=0x7f8470011b60 comp rx=0 tx=0).stop 2024-05-21T15:25:08.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.573+0000 7f8485fd9700 1 -- 172.21.15.38:0/1154760479 shutdown_connections 2024-05-21T15:25:08.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.573+0000 7f8485fd9700 1 --2- 172.21.15.38:0/1154760479 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8468039000 0x7f846803b510 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:08.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.573+0000 7f8485fd9700 1 --2- 172.21.15.38:0/1154760479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f848007c550 0x7f848019d080 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:08.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.573+0000 7f8485fd9700 1 -- 172.21.15.38:0/1154760479 >> 172.21.15.38:0/1154760479 conn(0x7f8480103270 msgr2=0x7f8480104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:08.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.573+0000 7f8485fd9700 1 -- 172.21.15.38:0/1154760479 shutdown_connections 2024-05-21T15:25:08.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:07.573+0000 7f8485fd9700 1 -- 172.21.15.38:0/1154760479 wait complete. 2024-05-21T15:25:08.244 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying unmanaged mgr service... 2024-05-21T15:25:08.878 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:08 smithi038 ceph-mon[31995]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:25:08.879 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:08 smithi038 ceph-mon[31995]: Saving service mon spec with placement count:5 2024-05-21T15:25:08.879 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:08 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' 2024-05-21T15:25:09.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-21T15:25:09.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.654+0000 7fcaba6f0700 1 Processor -- start 2024-05-21T15:25:09.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.654+0000 7fcaba6f0700 1 -- start start 2024-05-21T15:25:09.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.654+0000 7fcaba6f0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcab41096d0 0x7fcab410baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:09.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.654+0000 7fcaba6f0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcab407bdd0 con 0x7fcab41096d0 2024-05-21T15:25:09.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.654+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcab41096d0 0x7fcab410baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:09.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.654+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcab41096d0 0x7fcab410baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50856/0 (socket says 172.21.15.38:50856) 2024-05-21T15:25:09.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.654+0000 7fcab3fff700 1 -- 172.21.15.38:0/1021977038 learned_addr learned my addr 172.21.15.38:0/1021977038 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:09.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.654+0000 7fcab3fff700 1 -- 172.21.15.38:0/1021977038 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab410c030 con 0x7fcab41096d0 2024-05-21T15:25:09.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.655+0000 7fcab3fff700 1 --2- 172.21.15.38:0/1021977038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcab41096d0 0x7fcab410baf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40098a0 tx=0x7fcaa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6294321254a022b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:09.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.655+0000 7fcab2ffd700 1 -- 172.21.15.38:0/1021977038 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaa4004160 con 0x7fcab41096d0 2024-05-21T15:25:09.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.655+0000 7fcab2ffd700 1 -- 172.21.15.38:0/1021977038 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcaa4004300 con 0x7fcab41096d0 2024-05-21T15:25:09.951 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.655+0000 7fcab2ffd700 1 -- 172.21.15.38:0/1021977038 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaa4011b50 con 0x7fcab41096d0 2024-05-21T15:25:09.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.656+0000 7fcaba6f0700 1 -- 172.21.15.38:0/1021977038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcab41096d0 msgr2=0x7fcab410baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:09.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.656+0000 7fcaba6f0700 1 --2- 172.21.15.38:0/1021977038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcab41096d0 0x7fcab410baf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40098a0 tx=0x7fcaa4009bb0 comp rx=0 tx=0).stop 2024-05-21T15:25:09.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.656+0000 7fcaba6f0700 1 -- 172.21.15.38:0/1021977038 shutdown_connections 2024-05-21T15:25:09.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.656+0000 7fcaba6f0700 1 --2- 172.21.15.38:0/1021977038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcab41096d0 0x7fcab410baf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:09.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.656+0000 7fcaba6f0700 1 -- 172.21.15.38:0/1021977038 >> 172.21.15.38:0/1021977038 conn(0x7fcab41031b0 msgr2=0x7fcab4105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:09.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.656+0000 7fcaba6f0700 1 -- 172.21.15.38:0/1021977038 shutdown_connections 2024-05-21T15:25:09.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.656+0000 7fcaba6f0700 1 -- 172.21.15.38:0/1021977038 wait complete. 2024-05-21T15:25:09.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.657+0000 7fcaba6f0700 1 Processor -- start 2024-05-21T15:25:09.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.657+0000 7fcaba6f0700 1 -- start start 2024-05-21T15:25:09.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.657+0000 7fcaba6f0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcab41096d0 0x7fcab419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:09.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.657+0000 7fcaba6f0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcab410cf90 con 0x7fcab41096d0 2024-05-21T15:25:09.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.657+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcab41096d0 0x7fcab419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:09.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.657+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcab41096d0 0x7fcab419f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50864/0 (socket says 172.21.15.38:50864) 2024-05-21T15:25:09.952 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.657+0000 7fcab3fff700 1 -- 172.21.15.38:0/3259584896 learned_addr learned my addr 172.21.15.38:0/3259584896 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.657+0000 7fcab3fff700 1 -- 172.21.15.38:0/3259584896 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab419f760 con 0x7fcab41096d0 2024-05-21T15:25:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.658+0000 7fcab3fff700 1 --2- 172.21.15.38:0/3259584896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcab41096d0 0x7fcab419f220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4011850 tx=0x7fcaa4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.658+0000 7fcab17fa700 1 -- 172.21.15.38:0/3259584896 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaa4003af0 con 0x7fcab41096d0 2024-05-21T15:25:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.658+0000 7fcaba6f0700 1 -- 172.21.15.38:0/3259584896 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab419f9f0 con 0x7fcab41096d0 2024-05-21T15:25:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.658+0000 7fcab17fa700 1 -- 172.21.15.38:0/3259584896 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcaa4020070 con 0x7fcab41096d0 2024-05-21T15:25:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.658+0000 7fcab17fa700 1 -- 172.21.15.38:0/3259584896 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaa40195a0 con 0x7fcab41096d0 2024-05-21T15:25:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.658+0000 7fcaba6f0700 1 -- 172.21.15.38:0/3259584896 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab41a2600 con 0x7fcab41096d0 2024-05-21T15:25:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.659+0000 7fcaba6f0700 1 -- 172.21.15.38:0/3259584896 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab4063ca0 con 0x7fcab41096d0 2024-05-21T15:25:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.659+0000 7fcab17fa700 1 -- 172.21.15.38:0/3259584896 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcaa40198c0 con 0x7fcab41096d0 2024-05-21T15:25:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.659+0000 7fcab17fa700 1 --2- 172.21.15.38:0/3259584896 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fca9c039050 0x7fca9c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.660+0000 7fcab17fa700 1 -- 172.21.15.38:0/3259584896 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcaa4011e00 con 0x7fcab41096d0 2024-05-21T15:25:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.660+0000 7fcab37fe700 1 --2- 172.21.15.38:0/3259584896 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fca9c039050 0x7fca9c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:09.953 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.660+0000 7fcab37fe700 1 --2- 172.21.15.38:0/3259584896 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fca9c039050 0x7fca9c03b560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8006fd0 tx=0x7fcaa8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:09.954 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.669+0000 7fcab17fa700 1 -- 172.21.15.38:0/3259584896 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcaa4003c90 con 0x7fcab41096d0 2024-05-21T15:25:09.954 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.896+0000 7fcaba6f0700 1 -- 172.21.15.38:0/3259584896 --> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcab4107cc0 con 0x7fca9c039050 2024-05-21T15:25:09.954 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.905+0000 7fcab17fa700 1 -- 172.21.15.38:0/3259584896 <== mgr.14120 v2:172.21.15.38:6800/1481949523 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcab4107cc0 con 0x7fca9c039050 2024-05-21T15:25:09.954 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.908+0000 7fcaba6f0700 1 -- 172.21.15.38:0/3259584896 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fca9c039050 msgr2=0x7fca9c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:09.954 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.908+0000 7fcaba6f0700 1 --2- 172.21.15.38:0/3259584896 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fca9c039050 0x7fca9c03b560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8006fd0 tx=0x7fcaa8006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:09.954 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.908+0000 7fcaba6f0700 1 -- 172.21.15.38:0/3259584896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcab41096d0 msgr2=0x7fcab419f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:09.954 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.908+0000 7fcaba6f0700 1 --2- 172.21.15.38:0/3259584896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcab41096d0 0x7fcab419f220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4011850 tx=0x7fcaa4011c10 comp rx=0 tx=0).stop 2024-05-21T15:25:09.954 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.909+0000 7fcaba6f0700 1 -- 172.21.15.38:0/3259584896 shutdown_connections 2024-05-21T15:25:09.954 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.909+0000 7fcaba6f0700 1 --2- 172.21.15.38:0/3259584896 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fca9c039050 0x7fca9c03b560 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:09.954 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.909+0000 7fcaba6f0700 1 --2- 172.21.15.38:0/3259584896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcab41096d0 0x7fcab419f220 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:09.954 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.909+0000 7fcaba6f0700 1 -- 172.21.15.38:0/3259584896 >> 172.21.15.38:0/3259584896 conn(0x7fcab41031b0 msgr2=0x7fcab41075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:09.954 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.909+0000 7fcaba6f0700 1 -- 172.21.15.38:0/3259584896 shutdown_connections 2024-05-21T15:25:09.954 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:08.909+0000 7fcaba6f0700 1 -- 172.21.15.38:0/3259584896 wait complete. 2024-05-21T15:25:10.171 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:09 smithi038 ceph-mon[31995]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:25:10.171 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:09 smithi038 ceph-mon[31995]: Saving service mgr spec with placement count:2 2024-05-21T15:25:10.172 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:09 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' 2024-05-21T15:25:10.172 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:09 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' 2024-05-21T15:25:11.175 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.414+0000 7f59c1c13700 1 Processor -- start 2024-05-21T15:25:11.175 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.414+0000 7f59c1c13700 1 -- start start 2024-05-21T15:25:11.175 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.415+0000 7f59c1c13700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59bc07bcc0 0x7f59bc10b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:11.175 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.415+0000 7f59c1c13700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc10bd30 con 0x7f59bc07bcc0 2024-05-21T15:25:11.175 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.415+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59bc07bcc0 0x7f59bc10b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:11.175 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.415+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59bc07bcc0 0x7f59bc10b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50870/0 (socket says 172.21.15.38:50870) 2024-05-21T15:25:11.176 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.415+0000 7f59bb7fe700 1 -- 172.21.15.38:0/1184838937 learned_addr learned my addr 172.21.15.38:0/1184838937 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:11.176 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.415+0000 7f59bb7fe700 1 -- 172.21.15.38:0/1184838937 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc10beb0 con 0x7f59bc07bcc0 2024-05-21T15:25:11.176 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.416+0000 7f59bb7fe700 1 --2- 172.21.15.38:0/1184838937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59bc07bcc0 0x7f59bc10b720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0098a0 tx=0x7f59ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b53d71fcd5bed4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:11.176 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.416+0000 7f59ba7fc700 1 -- 172.21.15.38:0/1184838937 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59ac004160 con 0x7f59bc07bcc0 2024-05-21T15:25:11.176 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.416+0000 7f59ba7fc700 1 -- 172.21.15.38:0/1184838937 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59ac004300 con 0x7f59bc07bcc0 2024-05-21T15:25:11.176 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.416+0000 7f59ba7fc700 1 -- 172.21.15.38:0/1184838937 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59ac011b20 con 0x7f59bc07bcc0 2024-05-21T15:25:11.176 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.417+0000 7f59c1c13700 1 -- 172.21.15.38:0/1184838937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59bc07bcc0 msgr2=0x7f59bc10b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:11.176 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.417+0000 7f59c1c13700 1 --2- 172.21.15.38:0/1184838937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59bc07bcc0 0x7f59bc10b720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0098a0 tx=0x7f59ac009bb0 comp rx=0 tx=0).stop 2024-05-21T15:25:11.176 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.417+0000 7f59c1c13700 1 -- 172.21.15.38:0/1184838937 shutdown_connections 2024-05-21T15:25:11.176 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.417+0000 7f59c1c13700 1 --2- 172.21.15.38:0/1184838937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59bc07bcc0 0x7f59bc10b720 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:11.176 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.417+0000 7f59c1c13700 1 -- 172.21.15.38:0/1184838937 >> 172.21.15.38:0/1184838937 conn(0x7f59bc1031b0 msgr2=0x7f59bc105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:11.176 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.417+0000 7f59c1c13700 1 -- 172.21.15.38:0/1184838937 shutdown_connections 2024-05-21T15:25:11.176 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.417+0000 7f59c1c13700 1 -- 172.21.15.38:0/1184838937 wait complete. 2024-05-21T15:25:11.177 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.418+0000 7f59c1c13700 1 Processor -- start 2024-05-21T15:25:11.177 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.418+0000 7f59c1c13700 1 -- start start 2024-05-21T15:25:11.177 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.418+0000 7f59c1c13700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59bc07bcc0 0x7f59bc19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:11.177 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.418+0000 7f59c1c13700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc10cfd0 con 0x7f59bc07bcc0 2024-05-21T15:25:11.177 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.418+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59bc07bcc0 0x7f59bc19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:11.177 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.418+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59bc07bcc0 0x7f59bc19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50886/0 (socket says 172.21.15.38:50886) 2024-05-21T15:25:11.177 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.418+0000 7f59bb7fe700 1 -- 172.21.15.38:0/2248968971 learned_addr learned my addr 172.21.15.38:0/2248968971 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:11.177 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.419+0000 7f59bb7fe700 1 -- 172.21.15.38:0/2248968971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc19d4f0 con 0x7f59bc07bcc0 2024-05-21T15:25:11.177 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.419+0000 7f59bb7fe700 1 --2- 172.21.15.38:0/2248968971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59bc07bcc0 0x7f59bc19cfb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59ac011740 tx=0x7f59ac011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:11.177 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.419+0000 7f59b8ff9700 1 -- 172.21.15.38:0/2248968971 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59ac003a40 con 0x7f59bc07bcc0 2024-05-21T15:25:11.177 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.419+0000 7f59c1c13700 1 -- 172.21.15.38:0/2248968971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59bc19d780 con 0x7f59bc07bcc0 2024-05-21T15:25:11.177 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.419+0000 7f59b8ff9700 1 -- 172.21.15.38:0/2248968971 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59ac003be0 con 0x7f59bc07bcc0 2024-05-21T15:25:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.419+0000 7f59c1c13700 1 -- 172.21.15.38:0/2248968971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59bc1a0390 con 0x7f59bc07bcc0 2024-05-21T15:25:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.419+0000 7f59b8ff9700 1 -- 172.21.15.38:0/2248968971 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59ac01a720 con 0x7f59bc07bcc0 2024-05-21T15:25:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.421+0000 7f59c1c13700 1 -- 172.21.15.38:0/2248968971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f599c005360 con 0x7f59bc07bcc0 2024-05-21T15:25:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.421+0000 7f59b8ff9700 1 -- 172.21.15.38:0/2248968971 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f59ac024aa0 con 0x7f59bc07bcc0 2024-05-21T15:25:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.421+0000 7f59b8ff9700 1 --2- 172.21.15.38:0/2248968971 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f59a8039000 0x7f59a803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.421+0000 7f59b8ff9700 1 -- 172.21.15.38:0/2248968971 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f59ac04d640 con 0x7f59bc07bcc0 2024-05-21T15:25:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.421+0000 7f59baffd700 1 --2- 172.21.15.38:0/2248968971 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f59a8039000 0x7f59a803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.422+0000 7f59baffd700 1 --2- 172.21.15.38:0/2248968971 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f59a8039000 0x7f59a803b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f59a4006fd0 tx=0x7f59a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.426+0000 7f59b8ff9700 1 -- 172.21.15.38:0/2248968971 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f59ac011e20 con 0x7f59bc07bcc0 2024-05-21T15:25:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.632+0000 7f59c1c13700 1 -- 172.21.15.38:0/2248968971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f599c005650 con 0x7f59bc07bcc0 2024-05-21T15:25:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.640+0000 7f59b8ff9700 1 -- 172.21.15.38:0/2248968971 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f59ac02d770 con 0x7f59bc07bcc0 2024-05-21T15:25:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.642+0000 7f59c1c13700 1 -- 172.21.15.38:0/2248968971 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f59a8039000 msgr2=0x7f59a803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:11.178 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.642+0000 7f59c1c13700 1 --2- 172.21.15.38:0/2248968971 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f59a8039000 0x7f59a803b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f59a4006fd0 tx=0x7f59a4006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:11.179 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.642+0000 7f59c1c13700 1 -- 172.21.15.38:0/2248968971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59bc07bcc0 msgr2=0x7f59bc19cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:11.179 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.642+0000 7f59c1c13700 1 --2- 172.21.15.38:0/2248968971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59bc07bcc0 0x7f59bc19cfb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59ac011740 tx=0x7f59ac011ba0 comp rx=0 tx=0).stop 2024-05-21T15:25:11.179 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.642+0000 7f59c1c13700 1 -- 172.21.15.38:0/2248968971 shutdown_connections 2024-05-21T15:25:11.179 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.642+0000 7f59c1c13700 1 --2- 172.21.15.38:0/2248968971 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f59a8039000 0x7f59a803b510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:11.179 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.642+0000 7f59c1c13700 1 --2- 172.21.15.38:0/2248968971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59bc07bcc0 0x7f59bc19cfb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:11.179 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.642+0000 7f59c1c13700 1 -- 172.21.15.38:0/2248968971 >> 172.21.15.38:0/2248968971 conn(0x7f59bc1031b0 msgr2=0x7f59bc104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:11.179 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.642+0000 7f59c1c13700 1 -- 172.21.15.38:0/2248968971 shutdown_connections 2024-05-21T15:25:11.179 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:10.642+0000 7f59c1c13700 1 -- 172.21.15.38:0/2248968971 wait complete. 2024-05-21T15:25:11.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:11 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' 2024-05-21T15:25:11.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:11 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2248968971' entity='client.admin' 2024-05-21T15:25:13.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:12 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/778760728' entity='client.admin' 2024-05-21T15:25:13.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:12 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' 2024-05-21T15:25:13.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.576+0000 7f8ea2b82700 1 Processor -- start 2024-05-21T15:25:13.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.576+0000 7f8ea2b82700 1 -- start start 2024-05-21T15:25:13.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.576+0000 7f8ea2b82700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e9c108060 0x7f8e9c108470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:13.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.576+0000 7f8ea2b82700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c108a40 con 0x7f8e9c108060 2024-05-21T15:25:13.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.576+0000 7f8ea091e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e9c108060 0x7f8e9c108470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:13.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.576+0000 7f8ea091e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e9c108060 0x7f8e9c108470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50902/0 (socket says 172.21.15.38:50902) 2024-05-21T15:25:13.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.576+0000 7f8ea091e700 1 -- 172.21.15.38:0/3938563641 learned_addr learned my addr 172.21.15.38:0/3938563641 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:13.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.577+0000 7f8ea091e700 1 -- 172.21.15.38:0/3938563641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9c108bc0 con 0x7f8e9c108060 2024-05-21T15:25:13.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.577+0000 7f8ea091e700 1 --2- 172.21.15.38:0/3938563641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e9c108060 0x7f8e9c108470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e900098a0 tx=0x7f8e90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f040656067445d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:13.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.577+0000 7f8e9b7fe700 1 -- 172.21.15.38:0/3938563641 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e90004160 con 0x7f8e9c108060 2024-05-21T15:25:13.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.577+0000 7f8e9b7fe700 1 -- 172.21.15.38:0/3938563641 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e90004300 con 0x7f8e9c108060 2024-05-21T15:25:13.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.577+0000 7f8e9b7fe700 1 -- 172.21.15.38:0/3938563641 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e90011b20 con 0x7f8e9c108060 2024-05-21T15:25:13.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.578+0000 7f8ea2b82700 1 -- 172.21.15.38:0/3938563641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e9c108060 msgr2=0x7f8e9c108470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:13.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.578+0000 7f8ea2b82700 1 --2- 172.21.15.38:0/3938563641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e9c108060 0x7f8e9c108470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e900098a0 tx=0x7f8e90009bb0 comp rx=0 tx=0).stop 2024-05-21T15:25:13.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.578+0000 7f8ea2b82700 1 -- 172.21.15.38:0/3938563641 shutdown_connections 2024-05-21T15:25:13.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.578+0000 7f8ea2b82700 1 --2- 172.21.15.38:0/3938563641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e9c108060 0x7f8e9c108470 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:13.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.578+0000 7f8ea2b82700 1 -- 172.21.15.38:0/3938563641 >> 172.21.15.38:0/3938563641 conn(0x7f8e9c1028f0 msgr2=0x7f8e9c104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:13.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.578+0000 7f8ea2b82700 1 -- 172.21.15.38:0/3938563641 shutdown_connections 2024-05-21T15:25:13.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.578+0000 7f8ea2b82700 1 -- 172.21.15.38:0/3938563641 wait complete. 2024-05-21T15:25:13.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.579+0000 7f8ea2b82700 1 Processor -- start 2024-05-21T15:25:13.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.579+0000 7f8ea2b82700 1 -- start start 2024-05-21T15:25:13.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.579+0000 7f8ea2b82700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e9c108060 0x7f8e9c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:13.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.579+0000 7f8ea2b82700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c10c010 con 0x7f8e9c108060 2024-05-21T15:25:13.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.579+0000 7f8ea091e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e9c108060 0x7f8e9c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:13.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.579+0000 7f8ea091e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e9c108060 0x7f8e9c19cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50906/0 (socket says 172.21.15.38:50906) 2024-05-21T15:25:13.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.579+0000 7f8ea091e700 1 -- 172.21.15.38:0/778760728 learned_addr learned my addr 172.21.15.38:0/778760728 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:13.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.580+0000 7f8ea091e700 1 -- 172.21.15.38:0/778760728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9c19d500 con 0x7f8e9c108060 2024-05-21T15:25:13.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.580+0000 7f8ea091e700 1 --2- 172.21.15.38:0/778760728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e9c108060 0x7f8e9c19cfc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8e90011740 tx=0x7f8e90011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:13.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.580+0000 7f8e99ffb700 1 -- 172.21.15.38:0/778760728 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e90003a40 con 0x7f8e9c108060 2024-05-21T15:25:13.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.580+0000 7f8ea2b82700 1 -- 172.21.15.38:0/778760728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9c19d790 con 0x7f8e9c108060 2024-05-21T15:25:13.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.580+0000 7f8e99ffb700 1 -- 172.21.15.38:0/778760728 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e90003be0 con 0x7f8e9c108060 2024-05-21T15:25:13.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.580+0000 7f8e99ffb700 1 -- 172.21.15.38:0/778760728 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e9001a720 con 0x7f8e9c108060 2024-05-21T15:25:13.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.580+0000 7f8ea2b82700 1 -- 172.21.15.38:0/778760728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9c1a03a0 con 0x7f8e9c108060 2024-05-21T15:25:13.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.581+0000 7f8e99ffb700 1 -- 172.21.15.38:0/778760728 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8e9001aa20 con 0x7f8e9c108060 2024-05-21T15:25:13.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.581+0000 7f8ea2b82700 1 -- 172.21.15.38:0/778760728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e9c063ca0 con 0x7f8e9c108060 2024-05-21T15:25:13.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.582+0000 7f8e99ffb700 1 --2- 172.21.15.38:0/778760728 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8e8c039000 0x7f8e8c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:13.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.582+0000 7f8e99ffb700 1 -- 172.21.15.38:0/778760728 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8e9004d730 con 0x7f8e9c108060 2024-05-21T15:25:13.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.582+0000 7f8e9bfff700 1 --2- 172.21.15.38:0/778760728 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8e8c039000 0x7f8e8c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:13.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.582+0000 7f8e9bfff700 1 --2- 172.21.15.38:0/778760728 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8e8c039000 0x7f8e8c03b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8e88006fd0 tx=0x7f8e88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:13.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.589+0000 7f8e99ffb700 1 -- 172.21.15.38:0/778760728 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8e90011e20 con 0x7f8e9c108060 2024-05-21T15:25:13.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.809+0000 7f8ea2b82700 1 -- 172.21.15.38:0/778760728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f8e9c02df10 con 0x7f8e9c108060 2024-05-21T15:25:13.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.812+0000 7f8e99ffb700 1 -- 172.21.15.38:0/778760728 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f8e90019c70 con 0x7f8e9c108060 2024-05-21T15:25:13.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.814+0000 7f8ea2b82700 1 -- 172.21.15.38:0/778760728 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8e8c039000 msgr2=0x7f8e8c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:13.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.814+0000 7f8ea2b82700 1 --2- 172.21.15.38:0/778760728 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8e8c039000 0x7f8e8c03b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8e88006fd0 tx=0x7f8e88006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:13.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.814+0000 7f8ea2b82700 1 -- 172.21.15.38:0/778760728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e9c108060 msgr2=0x7f8e9c19cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:13.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.814+0000 7f8ea2b82700 1 --2- 172.21.15.38:0/778760728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e9c108060 0x7f8e9c19cfc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8e90011740 tx=0x7f8e90011ba0 comp rx=0 tx=0).stop 2024-05-21T15:25:13.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.814+0000 7f8ea2b82700 1 -- 172.21.15.38:0/778760728 shutdown_connections 2024-05-21T15:25:13.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.814+0000 7f8ea2b82700 1 --2- 172.21.15.38:0/778760728 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8e8c039000 0x7f8e8c03b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:13.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.814+0000 7f8ea2b82700 1 --2- 172.21.15.38:0/778760728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e9c108060 0x7f8e9c19cfc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:13.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.814+0000 7f8ea2b82700 1 -- 172.21.15.38:0/778760728 >> 172.21.15.38:0/778760728 conn(0x7f8e9c1028f0 msgr2=0x7f8e9c193570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:13.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.814+0000 7f8ea2b82700 1 -- 172.21.15.38:0/778760728 shutdown_connections 2024-05-21T15:25:13.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:11.814+0000 7f8ea2b82700 1 -- 172.21.15.38:0/778760728 wait complete. 2024-05-21T15:25:13.081 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling the dashboard module... 2024-05-21T15:25:15.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:14 smithi038 ceph-mon[31995]: from='mgr.14120 172.21.15.38:0/3887460964' entity='mgr.x' 2024-05-21T15:25:15.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:14 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3020029365' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-21T15:25:15.029 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:14 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: ignoring --setuser ceph since I am not root 2024-05-21T15:25:15.029 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:14 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: ignoring --setgroup ceph since I am not root 2024-05-21T15:25:15.345 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:15 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:15.037+0000 7f48f0e04200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T15:25:15.345 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:15 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:15.232+0000 7f48f0e04200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:25:15.345 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:15 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:15.341+0000 7f48f0e04200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T15:25:16.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:15 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3020029365' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-21T15:25:16.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:15 smithi038 ceph-mon[31995]: mgrmap e9: x(active, since 20s) 2024-05-21T15:25:16.029 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:15 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:15.669+0000 7f48f0e04200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T15:25:16.029 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:15 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:15.760+0000 7f48f0e04200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T15:25:16.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.537+0000 7f8e6780a700 1 Processor -- start 2024-05-21T15:25:16.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.537+0000 7f8e6780a700 1 -- start start 2024-05-21T15:25:16.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.538+0000 7f8e6780a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e60106f30 0x7f8e60107340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:16.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.538+0000 7f8e6780a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e60107930 con 0x7f8e60106f30 2024-05-21T15:25:16.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.538+0000 7f8e655a6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e60106f30 0x7f8e60107340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:16.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.538+0000 7f8e655a6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e60106f30 0x7f8e60107340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50910/0 (socket says 172.21.15.38:50910) 2024-05-21T15:25:16.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.538+0000 7f8e655a6700 1 -- 172.21.15.38:0/3043073330 learned_addr learned my addr 172.21.15.38:0/3043073330 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:16.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.538+0000 7f8e655a6700 1 -- 172.21.15.38:0/3043073330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e60107ab0 con 0x7f8e60106f30 2024-05-21T15:25:16.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.539+0000 7f8e655a6700 1 --2- 172.21.15.38:0/3043073330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e60106f30 0x7f8e60107340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009e80 tx=0x7f8e5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe78ad28cfb4a67 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:16.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.539+0000 7f8e5ffff700 1 -- 172.21.15.38:0/3043073330 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e50017070 con 0x7f8e60106f30 2024-05-21T15:25:16.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.539+0000 7f8e5ffff700 1 -- 172.21.15.38:0/3043073330 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e5000b8c0 con 0x7f8e60106f30 2024-05-21T15:25:16.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.539+0000 7f8e5ffff700 1 -- 172.21.15.38:0/3043073330 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e500045c0 con 0x7f8e60106f30 2024-05-21T15:25:16.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.540+0000 7f8e6780a700 1 -- 172.21.15.38:0/3043073330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e60106f30 msgr2=0x7f8e60107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:16.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.540+0000 7f8e6780a700 1 --2- 172.21.15.38:0/3043073330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e60106f30 0x7f8e60107340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009e80 tx=0x7f8e5000b330 comp rx=0 tx=0).stop 2024-05-21T15:25:16.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.540+0000 7f8e6780a700 1 -- 172.21.15.38:0/3043073330 shutdown_connections 2024-05-21T15:25:16.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.540+0000 7f8e6780a700 1 --2- 172.21.15.38:0/3043073330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e60106f30 0x7f8e60107340 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:16.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.540+0000 7f8e6780a700 1 -- 172.21.15.38:0/3043073330 >> 172.21.15.38:0/3043073330 conn(0x7f8e601028f0 msgr2=0x7f8e60104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:16.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.540+0000 7f8e6780a700 1 -- 172.21.15.38:0/3043073330 shutdown_connections 2024-05-21T15:25:16.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.540+0000 7f8e6780a700 1 -- 172.21.15.38:0/3043073330 wait complete. 2024-05-21T15:25:16.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.541+0000 7f8e6780a700 1 Processor -- start 2024-05-21T15:25:16.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.541+0000 7f8e6780a700 1 -- start start 2024-05-21T15:25:16.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.541+0000 7f8e6780a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e60106f30 0x7f8e6019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:16.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.541+0000 7f8e6780a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6010d000 con 0x7f8e60106f30 2024-05-21T15:25:16.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.541+0000 7f8e655a6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e60106f30 0x7f8e6019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:16.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.541+0000 7f8e655a6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e60106f30 0x7f8e6019cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50914/0 (socket says 172.21.15.38:50914) 2024-05-21T15:25:16.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.541+0000 7f8e655a6700 1 -- 172.21.15.38:0/3020029365 learned_addr learned my addr 172.21.15.38:0/3020029365 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:16.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.541+0000 7f8e655a6700 1 -- 172.21.15.38:0/3020029365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6019d510 con 0x7f8e60106f30 2024-05-21T15:25:16.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.542+0000 7f8e655a6700 1 --2- 172.21.15.38:0/3020029365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e60106f30 0x7f8e6019cfd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009530 tx=0x7f8e50003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:16.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.542+0000 7f8e5e7fc700 1 -- 172.21.15.38:0/3020029365 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e50020020 con 0x7f8e60106f30 2024-05-21T15:25:16.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.542+0000 7f8e6780a700 1 -- 172.21.15.38:0/3020029365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6019d7a0 con 0x7f8e60106f30 2024-05-21T15:25:16.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.542+0000 7f8e5e7fc700 1 -- 172.21.15.38:0/3020029365 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e5000bba0 con 0x7f8e60106f30 2024-05-21T15:25:16.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.542+0000 7f8e5e7fc700 1 -- 172.21.15.38:0/3020029365 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e50017040 con 0x7f8e60106f30 2024-05-21T15:25:16.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.542+0000 7f8e6780a700 1 -- 172.21.15.38:0/3020029365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e601a03b0 con 0x7f8e60106f30 2024-05-21T15:25:16.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.543+0000 7f8e5e7fc700 1 -- 172.21.15.38:0/3020029365 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8e50023020 con 0x7f8e60106f30 2024-05-21T15:25:16.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.543+0000 7f8e5e7fc700 1 --2- 172.21.15.38:0/3020029365 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8e48039050 0x7f8e4803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:16.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.543+0000 7f8e5e7fc700 1 -- 172.21.15.38:0/3020029365 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8e5000f050 con 0x7f8e60106f30 2024-05-21T15:25:16.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.543+0000 7f8e64da5700 1 --2- 172.21.15.38:0/3020029365 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8e48039050 0x7f8e4803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:16.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.543+0000 7f8e6780a700 1 -- 172.21.15.38:0/3020029365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e60063ca0 con 0x7f8e60106f30 2024-05-21T15:25:16.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.544+0000 7f8e64da5700 1 --2- 172.21.15.38:0/3020029365 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8e48039050 0x7f8e4803b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8e54006fd0 tx=0x7f8e54006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:16.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.549+0000 7f8e5e7fc700 1 -- 172.21.15.38:0/3020029365 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8e5002a020 con 0x7f8e60106f30 2024-05-21T15:25:16.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:13.820+0000 7f8e6780a700 1 -- 172.21.15.38:0/3020029365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f8e601a0580 con 0x7f8e60106f30 2024-05-21T15:25:16.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:14.821+0000 7f8e5e7fc700 1 -- 172.21.15.38:0/3020029365 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f8e5002b260 con 0x7f8e60106f30 2024-05-21T15:25:16.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:14.822+0000 7f8e5e7fc700 1 -- 172.21.15.38:0/3020029365 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f8e50025400 con 0x7f8e60106f30 2024-05-21T15:25:16.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:14.824+0000 7f8e6780a700 1 -- 172.21.15.38:0/3020029365 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8e48039050 msgr2=0x7f8e4803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:16.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:14.824+0000 7f8e6780a700 1 --2- 172.21.15.38:0/3020029365 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8e48039050 0x7f8e4803b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8e54006fd0 tx=0x7f8e54006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:16.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:14.824+0000 7f8e6780a700 1 -- 172.21.15.38:0/3020029365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e60106f30 msgr2=0x7f8e6019cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:16.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:14.824+0000 7f8e6780a700 1 --2- 172.21.15.38:0/3020029365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e60106f30 0x7f8e6019cfd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009530 tx=0x7f8e50003d00 comp rx=0 tx=0).stop 2024-05-21T15:25:16.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:14.824+0000 7f8e6780a700 1 -- 172.21.15.38:0/3020029365 shutdown_connections 2024-05-21T15:25:16.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:14.824+0000 7f8e6780a700 1 --2- 172.21.15.38:0/3020029365 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7f8e48039050 0x7f8e4803b560 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:16.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:14.824+0000 7f8e6780a700 1 --2- 172.21.15.38:0/3020029365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e60106f30 0x7f8e6019cfd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:16.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:14.824+0000 7f8e6780a700 1 -- 172.21.15.38:0/3020029365 >> 172.21.15.38:0/3020029365 conn(0x7f8e601028f0 msgr2=0x7f8e601031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:16.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:14.824+0000 7f8e6780a700 1 -- 172.21.15.38:0/3020029365 shutdown_connections 2024-05-21T15:25:16.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:14.824+0000 7f8e6780a700 1 -- 172.21.15.38:0/3020029365 wait complete. 2024-05-21T15:25:17.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:16 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2059849965' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-21T15:25:17.391 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:17 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:17.042+0000 7f48f0e04200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T15:25:17.391 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:17 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:17.225+0000 7f48f0e04200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T15:25:17.401 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:25:17.401 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-21T15:25:17.401 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T15:25:17.401 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-21T15:25:17.401 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-21T15:25:17.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:25:17.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.531+0000 7fef9e9f0700 1 Processor -- start 2024-05-21T15:25:17.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.531+0000 7fef9e9f0700 1 -- start start 2024-05-21T15:25:17.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.532+0000 7fef9e9f0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef9807bc80 0x7fef9810b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:17.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.532+0000 7fef9e9f0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fef9810bd30 con 0x7fef9807bc80 2024-05-21T15:25:17.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.532+0000 7fef97fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef9807bc80 0x7fef9810b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:17.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.532+0000 7fef97fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef9807bc80 0x7fef9810b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34292/0 (socket says 172.21.15.38:34292) 2024-05-21T15:25:17.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.532+0000 7fef97fff700 1 -- 172.21.15.38:0/3062284855 learned_addr learned my addr 172.21.15.38:0/3062284855 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:17.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.532+0000 7fef97fff700 1 -- 172.21.15.38:0/3062284855 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9810beb0 con 0x7fef9807bc80 2024-05-21T15:25:17.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.532+0000 7fef97fff700 1 --2- 172.21.15.38:0/3062284855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef9807bc80 0x7fef9810b760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef880098a0 tx=0x7fef88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67909eb8f1083ef4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:17.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.533+0000 7fef96ffd700 1 -- 172.21.15.38:0/3062284855 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef88004160 con 0x7fef9807bc80 2024-05-21T15:25:17.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.533+0000 7fef96ffd700 1 -- 172.21.15.38:0/3062284855 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef88004300 con 0x7fef9807bc80 2024-05-21T15:25:17.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.533+0000 7fef96ffd700 1 -- 172.21.15.38:0/3062284855 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef88011b20 con 0x7fef9807bc80 2024-05-21T15:25:17.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.533+0000 7fef9e9f0700 1 -- 172.21.15.38:0/3062284855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef9807bc80 msgr2=0x7fef9810b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:17.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.533+0000 7fef9e9f0700 1 --2- 172.21.15.38:0/3062284855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef9807bc80 0x7fef9810b760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef880098a0 tx=0x7fef88009bb0 comp rx=0 tx=0).stop 2024-05-21T15:25:17.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.533+0000 7fef9e9f0700 1 -- 172.21.15.38:0/3062284855 shutdown_connections 2024-05-21T15:25:17.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.534+0000 7fef9e9f0700 1 --2- 172.21.15.38:0/3062284855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef9807bc80 0x7fef9810b760 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:17.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.534+0000 7fef9e9f0700 1 -- 172.21.15.38:0/3062284855 >> 172.21.15.38:0/3062284855 conn(0x7fef981031b0 msgr2=0x7fef98105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:17.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.534+0000 7fef9e9f0700 1 -- 172.21.15.38:0/3062284855 shutdown_connections 2024-05-21T15:25:17.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.534+0000 7fef9e9f0700 1 -- 172.21.15.38:0/3062284855 wait complete. 2024-05-21T15:25:17.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.534+0000 7fef9e9f0700 1 Processor -- start 2024-05-21T15:25:17.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.534+0000 7fef9e9f0700 1 -- start start 2024-05-21T15:25:17.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.535+0000 7fef9e9f0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef9807bc80 0x7fef9819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:17.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.535+0000 7fef9e9f0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fef9810cfd0 con 0x7fef9807bc80 2024-05-21T15:25:17.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.535+0000 7fef97fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef9807bc80 0x7fef9819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.535+0000 7fef97fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef9807bc80 0x7fef9819cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34308/0 (socket says 172.21.15.38:34308) 2024-05-21T15:25:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.535+0000 7fef97fff700 1 -- 172.21.15.38:0/2059849965 learned_addr learned my addr 172.21.15.38:0/2059849965 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.535+0000 7fef97fff700 1 -- 172.21.15.38:0/2059849965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9819d4b0 con 0x7fef9807bc80 2024-05-21T15:25:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.535+0000 7fef97fff700 1 --2- 172.21.15.38:0/2059849965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef9807bc80 0x7fef9819cf70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef88011740 tx=0x7fef88011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.535+0000 7fef957fa700 1 -- 172.21.15.38:0/2059849965 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef88003a40 con 0x7fef9807bc80 2024-05-21T15:25:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.536+0000 7fef9e9f0700 1 -- 172.21.15.38:0/2059849965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9819d740 con 0x7fef9807bc80 2024-05-21T15:25:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.536+0000 7fef957fa700 1 -- 172.21.15.38:0/2059849965 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef88003be0 con 0x7fef9807bc80 2024-05-21T15:25:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.536+0000 7fef957fa700 1 -- 172.21.15.38:0/2059849965 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef8801a750 con 0x7fef9807bc80 2024-05-21T15:25:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.536+0000 7fef9e9f0700 1 -- 172.21.15.38:0/2059849965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef981a0350 con 0x7fef9807bc80 2024-05-21T15:25:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.536+0000 7fef957fa700 1 -- 172.21.15.38:0/2059849965 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fef8801aa50 con 0x7fef9807bc80 2024-05-21T15:25:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.537+0000 7fef957fa700 1 --2- 172.21.15.38:0/2059849965 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fef80039050 0x7fef8003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.537+0000 7fef957fa700 1 -- 172.21.15.38:0/2059849965 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fef8804e770 con 0x7fef9807bc80 2024-05-21T15:25:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.537+0000 7fef977fe700 1 -- 172.21.15.38:0/2059849965 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fef80039050 msgr2=0x7fef8003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1481949523 2024-05-21T15:25:17.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.537+0000 7fef9e9f0700 1 -- 172.21.15.38:0/2059849965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef9819dba0 con 0x7fef9807bc80 2024-05-21T15:25:17.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.537+0000 7fef977fe700 1 --2- 172.21.15.38:0/2059849965 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fef80039050 0x7fef8003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:25:17.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.542+0000 7fef957fa700 1 -- 172.21.15.38:0/2059849965 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fef88011e20 con 0x7fef9807bc80 2024-05-21T15:25:17.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.738+0000 7fef977fe700 1 -- 172.21.15.38:0/2059849965 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fef80039050 msgr2=0x7fef8003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1481949523 2024-05-21T15:25:17.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.738+0000 7fef977fe700 1 --2- 172.21.15.38:0/2059849965 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fef80039050 0x7fef8003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T15:25:17.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.809+0000 7fef9e9f0700 1 -- 172.21.15.38:0/2059849965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fef981a07d0 con 0x7fef9807bc80 2024-05-21T15:25:17.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.810+0000 7fef957fa700 1 -- 172.21.15.38:0/2059849965 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fef8804e080 con 0x7fef9807bc80 2024-05-21T15:25:17.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.812+0000 7fef9e9f0700 1 -- 172.21.15.38:0/2059849965 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fef80039050 msgr2=0x7fef8003b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:25:17.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.812+0000 7fef9e9f0700 1 --2- 172.21.15.38:0/2059849965 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fef80039050 0x7fef8003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:17.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.812+0000 7fef9e9f0700 1 -- 172.21.15.38:0/2059849965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef9807bc80 msgr2=0x7fef9819cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:17.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.812+0000 7fef9e9f0700 1 --2- 172.21.15.38:0/2059849965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef9807bc80 0x7fef9819cf70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef88011740 tx=0x7fef88011ba0 comp rx=0 tx=0).stop 2024-05-21T15:25:17.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.812+0000 7fef9e9f0700 1 -- 172.21.15.38:0/2059849965 shutdown_connections 2024-05-21T15:25:17.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.812+0000 7fef9e9f0700 1 --2- 172.21.15.38:0/2059849965 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7fef80039050 0x7fef8003b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:17.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.812+0000 7fef9e9f0700 1 --2- 172.21.15.38:0/2059849965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef9807bc80 0x7fef9819cf70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:17.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.812+0000 7fef9e9f0700 1 -- 172.21.15.38:0/2059849965 >> 172.21.15.38:0/2059849965 conn(0x7fef981031b0 msgr2=0x7fef98104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:17.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.813+0000 7fef9e9f0700 1 -- 172.21.15.38:0/2059849965 shutdown_connections 2024-05-21T15:25:17.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:16.813+0000 7fef9e9f0700 1 -- 172.21.15.38:0/2059849965 wait complete. 2024-05-21T15:25:17.407 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for the mgr to restart... 2024-05-21T15:25:17.407 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr epoch 9... 2024-05-21T15:25:17.721 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:17 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:17.405+0000 7f48f0e04200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T15:25:18.029 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:17 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:17.717+0000 7f48f0e04200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T15:25:18.029 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:17 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:17.820+0000 7f48f0e04200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T15:25:18.450 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:18 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:18.036+0000 7f48f0e04200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T15:25:18.451 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:18 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:18.242+0000 7f48f0e04200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:25:19.214 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:18 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:18.877+0000 7f48f0e04200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T15:25:19.528 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:19 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:19.328+0000 7f48f0e04200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T15:25:19.528 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:19 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:19.431+0000 7f48f0e04200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T15:25:20.028 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:19 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:19.646+0000 7f48f0e04200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T15:25:20.528 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:20 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:20.201+0000 7f48f0e04200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T15:25:20.528 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:20 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:20.300+0000 7f48f0e04200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T15:25:20.853 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:20 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:20.560+0000 7f48f0e04200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T15:25:20.853 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:20 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:20.701+0000 7f48f0e04200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T15:25:20.853 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:20 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:20.804+0000 7f48f0e04200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T15:25:21.247 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:20 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:20.929+0000 7f48f0e04200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T15:25:21.247 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:21 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:21.243+0000 7f48f0e04200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T15:25:22.172 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:21 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:21.881+0000 7f48f0e04200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T15:25:22.172 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:25:22 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:25:22.115+0000 7f48f0e04200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T15:25:22.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:22 smithi038 ceph-mon[31995]: Active manager daemon x restarted 2024-05-21T15:25:22.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:22 smithi038 ceph-mon[31995]: Activating manager daemon x 2024-05-21T15:25:22.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:22 smithi038 ceph-mon[31995]: osdmap e3: 0 total, 0 up, 0 in 2024-05-21T15:25:22.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:22 smithi038 ceph-mon[31995]: mgrmap e10: x(active, starting, since 0.00731021s) 2024-05-21T15:25:22.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:25:22.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T15:25:22.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T15:25:22.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T15:25:22.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T15:25:22.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:22 smithi038 ceph-mon[31995]: Manager daemon x is now available 2024-05-21T15:25:22.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:25:23.374 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T15:25:23.374 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T15:25:23.374 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:23 smithi038 ceph-mon[31995]: mgrmap e11: x(active, since 1.00886s) 2024-05-21T15:25:23.659 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:25:23.660 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-21T15:25:23.660 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-21T15:25:23.660 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:25:23.660 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.839+0000 7ff00c89a700 1 Processor -- start 2024-05-21T15:25:23.660 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.839+0000 7ff00c89a700 1 -- start start 2024-05-21T15:25:23.660 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.840+0000 7ff00c89a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00807bc80 0x7ff00810b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:23.660 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.840+0000 7ff00c89a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff00810bd40 con 0x7ff00807bc80 2024-05-21T15:25:23.660 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.840+0000 7ff00659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00807bc80 0x7ff00810b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:23.660 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.840+0000 7ff00659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00807bc80 0x7ff00810b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34322/0 (socket says 172.21.15.38:34322) 2024-05-21T15:25:23.660 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.840+0000 7ff00659c700 1 -- 172.21.15.38:0/3975705607 learned_addr learned my addr 172.21.15.38:0/3975705607 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:23.660 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.840+0000 7ff00659c700 1 -- 172.21.15.38:0/3975705607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00810bec0 con 0x7ff00807bc80 2024-05-21T15:25:23.661 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.840+0000 7ff00659c700 1 --2- 172.21.15.38:0/3975705607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00807bc80 0x7ff00810b770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feff80098a0 tx=0x7feff8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e05dfadcf2964ce server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:23.661 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.841+0000 7ff00559a700 1 -- 172.21.15.38:0/3975705607 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feff8004160 con 0x7ff00807bc80 2024-05-21T15:25:23.661 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.841+0000 7ff00559a700 1 -- 172.21.15.38:0/3975705607 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff8004300 con 0x7ff00807bc80 2024-05-21T15:25:23.661 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.841+0000 7ff00559a700 1 -- 172.21.15.38:0/3975705607 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feff8011b20 con 0x7ff00807bc80 2024-05-21T15:25:23.661 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.841+0000 7ff00c89a700 1 -- 172.21.15.38:0/3975705607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00807bc80 msgr2=0x7ff00810b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:23.661 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.842+0000 7ff00c89a700 1 --2- 172.21.15.38:0/3975705607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00807bc80 0x7ff00810b770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feff80098a0 tx=0x7feff8009bb0 comp rx=0 tx=0).stop 2024-05-21T15:25:23.661 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.842+0000 7ff00c89a700 1 -- 172.21.15.38:0/3975705607 shutdown_connections 2024-05-21T15:25:23.661 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.842+0000 7ff00c89a700 1 --2- 172.21.15.38:0/3975705607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00807bc80 0x7ff00810b770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:23.661 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.842+0000 7ff00c89a700 1 -- 172.21.15.38:0/3975705607 >> 172.21.15.38:0/3975705607 conn(0x7ff0081031e0 msgr2=0x7ff008105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:23.661 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.842+0000 7ff00c89a700 1 -- 172.21.15.38:0/3975705607 shutdown_connections 2024-05-21T15:25:23.661 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.842+0000 7ff00c89a700 1 -- 172.21.15.38:0/3975705607 wait complete. 2024-05-21T15:25:23.661 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.843+0000 7ff00c89a700 1 Processor -- start 2024-05-21T15:25:23.661 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.843+0000 7ff00c89a700 1 -- start start 2024-05-21T15:25:23.661 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.843+0000 7ff00c89a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00807bc80 0x7ff008108ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:23.662 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.843+0000 7ff00c89a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff00810cfe0 con 0x7ff00807bc80 2024-05-21T15:25:23.662 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.843+0000 7ff00659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00807bc80 0x7ff008108ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:23.662 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.843+0000 7ff00659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00807bc80 0x7ff008108ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34328/0 (socket says 172.21.15.38:34328) 2024-05-21T15:25:23.662 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.843+0000 7ff00659c700 1 -- 172.21.15.38:0/182520170 learned_addr learned my addr 172.21.15.38:0/182520170 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:23.662 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.843+0000 7ff00659c700 1 -- 172.21.15.38:0/182520170 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff008109420 con 0x7ff00807bc80 2024-05-21T15:25:23.662 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.843+0000 7ff00659c700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00807bc80 0x7ff008108ee0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feff8009fb0 tx=0x7feff8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:23.662 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.844+0000 7feff77fe700 1 -- 172.21.15.38:0/182520170 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feff8003930 con 0x7ff00807bc80 2024-05-21T15:25:23.662 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.844+0000 7ff00c89a700 1 -- 172.21.15.38:0/182520170 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff008107670 con 0x7ff00807bc80 2024-05-21T15:25:23.662 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.844+0000 7feff77fe700 1 -- 172.21.15.38:0/182520170 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff8003ad0 con 0x7ff00807bc80 2024-05-21T15:25:23.662 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.844+0000 7feff77fe700 1 -- 172.21.15.38:0/182520170 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feff8019690 con 0x7ff00807bc80 2024-05-21T15:25:23.662 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.844+0000 7ff00c89a700 1 -- 172.21.15.38:0/182520170 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff008107ad0 con 0x7ff00807bc80 2024-05-21T15:25:23.662 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.845+0000 7feff77fe700 1 -- 172.21.15.38:0/182520170 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7feff8019990 con 0x7ff00807bc80 2024-05-21T15:25:23.663 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.845+0000 7feff77fe700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7feff0039050 0x7feff003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:23.663 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.845+0000 7feff77fe700 1 -- 172.21.15.38:0/182520170 --> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feff003bc70 con 0x7feff0039050 2024-05-21T15:25:23.663 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.845+0000 7feff77fe700 1 -- 172.21.15.38:0/182520170 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feff804d700 con 0x7ff00807bc80 2024-05-21T15:25:23.663 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.845+0000 7ff005d9b700 1 -- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7feff0039050 msgr2=0x7feff003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1481949523 2024-05-21T15:25:23.663 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:17.845+0000 7ff005d9b700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7feff0039050 0x7feff003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:25:23.663 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:18.046+0000 7ff005d9b700 1 -- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7feff0039050 msgr2=0x7feff003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1481949523 2024-05-21T15:25:23.663 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:18.046+0000 7ff005d9b700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7feff0039050 0x7feff003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T15:25:23.663 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:18.447+0000 7ff005d9b700 1 -- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7feff0039050 msgr2=0x7feff003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1481949523 2024-05-21T15:25:23.663 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:18.447+0000 7ff005d9b700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7feff0039050 0x7feff003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-21T15:25:23.663 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:19.248+0000 7ff005d9b700 1 -- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7feff0039050 msgr2=0x7feff003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1481949523 2024-05-21T15:25:23.663 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:19.248+0000 7ff005d9b700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7feff0039050 0x7feff003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-21T15:25:23.663 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:20.849+0000 7ff005d9b700 1 -- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7feff0039050 msgr2=0x7feff003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1481949523 2024-05-21T15:25:23.664 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:20.849+0000 7ff005d9b700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7feff0039050 0x7feff003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-21T15:25:23.664 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:22.123+0000 7feff77fe700 1 -- 172.21.15.38:0/182520170 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7feff802d960 con 0x7ff00807bc80 2024-05-21T15:25:23.664 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:22.123+0000 7feff77fe700 1 -- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7feff0039050 msgr2=0x7feff003b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:25:23.664 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:22.123+0000 7feff77fe700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7feff0039050 0x7feff003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:23.664 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.123+0000 7feff77fe700 1 -- 172.21.15.38:0/182520170 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7feff804f500 con 0x7ff00807bc80 2024-05-21T15:25:23.664 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.123+0000 7feff77fe700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7feff804d290 0x7feff003eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:23.664 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.123+0000 7feff77fe700 1 -- 172.21.15.38:0/182520170 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feff003bc70 con 0x7feff804d290 2024-05-21T15:25:23.664 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.123+0000 7ff005d9b700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7feff804d290 0x7feff003eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:23.664 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.124+0000 7ff005d9b700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7feff804d290 0x7feff003eb70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7feffc003a10 tx=0x7feffc0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:23.664 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.125+0000 7feff77fe700 1 -- 172.21.15.38:0/182520170 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7feff003bc70 con 0x7feff804d290 2024-05-21T15:25:23.664 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.133+0000 7ff00c89a700 1 -- 172.21.15.38:0/182520170 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff008063ca0 con 0x7feff804d290 2024-05-21T15:25:23.664 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.133+0000 7feff77fe700 1 -- 172.21.15.38:0/182520170 <== mgr.14152 v2:172.21.15.38:6800/1295226559 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ff008063ca0 con 0x7feff804d290 2024-05-21T15:25:23.664 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.133+0000 7ff00c89a700 1 -- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7feff804d290 msgr2=0x7feff003eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:23.665 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.134+0000 7ff00c89a700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7feff804d290 0x7feff003eb70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7feffc003a10 tx=0x7feffc0092b0 comp rx=0 tx=0).stop 2024-05-21T15:25:23.665 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.134+0000 7ff00c89a700 1 -- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00807bc80 msgr2=0x7ff008108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:23.665 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.134+0000 7ff00c89a700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00807bc80 0x7ff008108ee0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feff8009fb0 tx=0x7feff8011c10 comp rx=0 tx=0).stop 2024-05-21T15:25:23.665 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.134+0000 7ff00c89a700 1 -- 172.21.15.38:0/182520170 shutdown_connections 2024-05-21T15:25:23.665 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.134+0000 7ff00c89a700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7feff804d290 0x7feff003eb70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:23.665 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.134+0000 7ff00c89a700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff00807bc80 0x7ff008108ee0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:23.665 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.134+0000 7ff00c89a700 1 --2- 172.21.15.38:0/182520170 >> [v2:172.21.15.38:6800/1481949523,v1:172.21.15.38:6801/1481949523] conn(0x7feff0039050 0x7feff003b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:23.665 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.134+0000 7ff00c89a700 1 -- 172.21.15.38:0/182520170 >> 172.21.15.38:0/182520170 conn(0x7ff0081031e0 msgr2=0x7ff008104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:23.665 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.134+0000 7ff00c89a700 1 -- 172.21.15.38:0/182520170 shutdown_connections 2024-05-21T15:25:23.665 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:23.134+0000 7ff00c89a700 1 -- 172.21.15.38:0/182520170 wait complete. 2024-05-21T15:25:23.665 INFO:teuthology.orchestra.run.smithi038.stdout:mgr epoch 9 is available 2024-05-21T15:25:23.666 INFO:teuthology.orchestra.run.smithi038.stdout:Generating a dashboard self-signed certificate... 2024-05-21T15:25:24.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:24 smithi038 ceph-mon[31995]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-21T15:25:24.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:24 smithi038 ceph-mon[31995]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-21T15:25:25.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-21T15:25:25.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.063+0000 7eff13f6c700 1 Processor -- start 2024-05-21T15:25:25.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.063+0000 7eff13f6c700 1 -- start start 2024-05-21T15:25:25.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.063+0000 7eff13f6c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c107c00 0x7eff0c108010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:25.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.063+0000 7eff13f6c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c108600 con 0x7eff0c107c00 2024-05-21T15:25:25.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.063+0000 7eff11d08700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c107c00 0x7eff0c108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:25.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.063+0000 7eff11d08700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c107c00 0x7eff0c108010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34402/0 (socket says 172.21.15.38:34402) 2024-05-21T15:25:25.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.063+0000 7eff11d08700 1 -- 172.21.15.38:0/2112245856 learned_addr learned my addr 172.21.15.38:0/2112245856 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:25.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.064+0000 7eff11d08700 1 -- 172.21.15.38:0/2112245856 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0c108d30 con 0x7eff0c107c00 2024-05-21T15:25:25.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.064+0000 7eff11d08700 1 --2- 172.21.15.38:0/2112245856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c107c00 0x7eff0c108010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7eff080098a0 tx=0x7eff08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95bb12f702ca4587 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:25.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.064+0000 7eff10d06700 1 -- 172.21.15.38:0/2112245856 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff08004160 con 0x7eff0c107c00 2024-05-21T15:25:25.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.064+0000 7eff10d06700 1 -- 172.21.15.38:0/2112245856 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff08004300 con 0x7eff0c107c00 2024-05-21T15:25:25.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.064+0000 7eff10d06700 1 -- 172.21.15.38:0/2112245856 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff08011b20 con 0x7eff0c107c00 2024-05-21T15:25:25.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.065+0000 7eff13f6c700 1 -- 172.21.15.38:0/2112245856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c107c00 msgr2=0x7eff0c108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:25.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.065+0000 7eff13f6c700 1 --2- 172.21.15.38:0/2112245856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c107c00 0x7eff0c108010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7eff080098a0 tx=0x7eff08009bb0 comp rx=0 tx=0).stop 2024-05-21T15:25:25.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.065+0000 7eff13f6c700 1 -- 172.21.15.38:0/2112245856 shutdown_connections 2024-05-21T15:25:25.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.065+0000 7eff13f6c700 1 --2- 172.21.15.38:0/2112245856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c107c00 0x7eff0c108010 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:25.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.065+0000 7eff13f6c700 1 -- 172.21.15.38:0/2112245856 >> 172.21.15.38:0/2112245856 conn(0x7eff0c1032c0 msgr2=0x7eff0c105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:25.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.065+0000 7eff13f6c700 1 -- 172.21.15.38:0/2112245856 shutdown_connections 2024-05-21T15:25:25.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.065+0000 7eff13f6c700 1 -- 172.21.15.38:0/2112245856 wait complete. 2024-05-21T15:25:25.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.066+0000 7eff13f6c700 1 Processor -- start 2024-05-21T15:25:25.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.066+0000 7eff13f6c700 1 -- start start 2024-05-21T15:25:25.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.066+0000 7eff13f6c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c107c00 0x7eff0c07c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:25.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.066+0000 7eff13f6c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c10c0e0 con 0x7eff0c107c00 2024-05-21T15:25:25.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.066+0000 7eff11d08700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c107c00 0x7eff0c07c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:25.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.066+0000 7eff11d08700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c107c00 0x7eff0c07c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34408/0 (socket says 172.21.15.38:34408) 2024-05-21T15:25:25.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.066+0000 7eff11d08700 1 -- 172.21.15.38:0/2827906149 learned_addr learned my addr 172.21.15.38:0/2827906149 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:25.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.066+0000 7eff11d08700 1 -- 172.21.15.38:0/2827906149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0c07c910 con 0x7eff0c107c00 2024-05-21T15:25:25.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.066+0000 7eff11d08700 1 --2- 172.21.15.38:0/2827906149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c107c00 0x7eff0c07c3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7eff08009fb0 tx=0x7eff08011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:25.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.067+0000 7eff02ffd700 1 -- 172.21.15.38:0/2827906149 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff08003930 con 0x7eff0c107c00 2024-05-21T15:25:25.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.067+0000 7eff02ffd700 1 -- 172.21.15.38:0/2827906149 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff08003ad0 con 0x7eff0c107c00 2024-05-21T15:25:25.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.067+0000 7eff13f6c700 1 -- 172.21.15.38:0/2827906149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0c19fe60 con 0x7eff0c107c00 2024-05-21T15:25:25.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.067+0000 7eff02ffd700 1 -- 172.21.15.38:0/2827906149 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff08019670 con 0x7eff0c107c00 2024-05-21T15:25:25.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.067+0000 7eff13f6c700 1 -- 172.21.15.38:0/2827906149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0c1a02c0 con 0x7eff0c107c00 2024-05-21T15:25:25.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.068+0000 7eff13f6c700 1 -- 172.21.15.38:0/2827906149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0c063ca0 con 0x7eff0c107c00 2024-05-21T15:25:25.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.068+0000 7eff02ffd700 1 -- 172.21.15.38:0/2827906149 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7eff0801a430 con 0x7eff0c107c00 2024-05-21T15:25:25.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.068+0000 7eff02ffd700 1 --2- 172.21.15.38:0/2827906149 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efef8038f20 0x7efef803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:25.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.068+0000 7eff02ffd700 1 -- 172.21.15.38:0/2827906149 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7eff0804d580 con 0x7eff0c107c00 2024-05-21T15:25:25.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.068+0000 7eff11507700 1 --2- 172.21.15.38:0/2827906149 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efef8038f20 0x7efef803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:25.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.069+0000 7eff11507700 1 --2- 172.21.15.38:0/2827906149 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efef8038f20 0x7efef803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efefc006fd0 tx=0x7efefc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:25.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.077+0000 7eff02ffd700 1 -- 172.21.15.38:0/2827906149 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7eff08011e20 con 0x7eff0c107c00 2024-05-21T15:25:25.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.176+0000 7eff02ffd700 1 -- 172.21.15.38:0/2827906149 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7eff08019cf0 con 0x7eff0c107c00 2024-05-21T15:25:25.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.293+0000 7eff13f6c700 1 -- 172.21.15.38:0/2827906149 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7eff0c1052b0 con 0x7efef8038f20 2024-05-21T15:25:25.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.449+0000 7eff02ffd700 1 -- 172.21.15.38:0/2827906149 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7eff0c1052b0 con 0x7efef8038f20 2024-05-21T15:25:25.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.452+0000 7eff13f6c700 1 -- 172.21.15.38:0/2827906149 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efef8038f20 msgr2=0x7efef803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:25.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.452+0000 7eff13f6c700 1 --2- 172.21.15.38:0/2827906149 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efef8038f20 0x7efef803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efefc006fd0 tx=0x7efefc006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:25.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.452+0000 7eff13f6c700 1 -- 172.21.15.38:0/2827906149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c107c00 msgr2=0x7eff0c07c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:25.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.452+0000 7eff13f6c700 1 --2- 172.21.15.38:0/2827906149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c107c00 0x7eff0c07c3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7eff08009fb0 tx=0x7eff08011c10 comp rx=0 tx=0).stop 2024-05-21T15:25:25.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.452+0000 7eff13f6c700 1 -- 172.21.15.38:0/2827906149 shutdown_connections 2024-05-21T15:25:25.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.452+0000 7eff13f6c700 1 --2- 172.21.15.38:0/2827906149 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efef8038f20 0x7efef803b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:25.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.452+0000 7eff13f6c700 1 --2- 172.21.15.38:0/2827906149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff0c107c00 0x7eff0c07c3d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:25.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.452+0000 7eff13f6c700 1 -- 172.21.15.38:0/2827906149 >> 172.21.15.38:0/2827906149 conn(0x7eff0c1032c0 msgr2=0x7eff0c104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:25.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.452+0000 7eff13f6c700 1 -- 172.21.15.38:0/2827906149 shutdown_connections 2024-05-21T15:25:25.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:24.452+0000 7eff13f6c700 1 -- 172.21.15.38:0/2827906149 wait complete. 2024-05-21T15:25:25.011 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial admin user... 2024-05-21T15:25:25.428 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:25 smithi038 ceph-mon[31995]: mgrmap e12: x(active, since 2s) 2024-05-21T15:25:25.428 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:25 smithi038 ceph-mon[31995]: [21/May/2024:15:25:24] ENGINE Bus STARTING 2024-05-21T15:25:25.428 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:25.429 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:25 smithi038 ceph-mon[31995]: [21/May/2024:15:25:24] ENGINE Serving on http://172.21.15.38:8765 2024-05-21T15:25:25.429 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:25.429 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:25.429 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:25 smithi038 ceph-mon[31995]: [21/May/2024:15:25:24] ENGINE Serving on https://172.21.15.38:7150 2024-05-21T15:25:25.429 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:25 smithi038 ceph-mon[31995]: [21/May/2024:15:25:24] ENGINE Bus STARTED 2024-05-21T15:25:26.497 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$DQDDjainIiWDFwrilhLXle6RsJYTpxTruAYd1wBZ.jdhvcA/nP9A2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716305125, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-21T15:25:26.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.468+0000 7f4ea22dd700 1 Processor -- start 2024-05-21T15:25:26.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.469+0000 7f4ea22dd700 1 -- start start 2024-05-21T15:25:26.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.469+0000 7f4ea22dd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e9c107af0 0x7f4e9c107f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:26.498 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.469+0000 7f4ea22dd700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c1084f0 con 0x7f4e9c107af0 2024-05-21T15:25:26.499 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.469+0000 7f4e9bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e9c107af0 0x7f4e9c107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:26.499 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.469+0000 7f4e9bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e9c107af0 0x7f4e9c107f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54336/0 (socket says 172.21.15.38:54336) 2024-05-21T15:25:26.499 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.469+0000 7f4e9bfff700 1 -- 172.21.15.38:0/1961982114 learned_addr learned my addr 172.21.15.38:0/1961982114 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:26.499 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.469+0000 7f4e9bfff700 1 -- 172.21.15.38:0/1961982114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c108c20 con 0x7f4e9c107af0 2024-05-21T15:25:26.499 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.470+0000 7f4e9bfff700 1 --2- 172.21.15.38:0/1961982114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e9c107af0 0x7f4e9c107f00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0098a0 tx=0x7f4e8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ce09487c5296ba server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:26.499 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.470+0000 7f4e9affd700 1 -- 172.21.15.38:0/1961982114 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e8c004160 con 0x7f4e9c107af0 2024-05-21T15:25:26.499 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.470+0000 7f4e9affd700 1 -- 172.21.15.38:0/1961982114 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e8c004300 con 0x7f4e9c107af0 2024-05-21T15:25:26.499 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.470+0000 7f4e9affd700 1 -- 172.21.15.38:0/1961982114 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e8c011b20 con 0x7f4e9c107af0 2024-05-21T15:25:26.499 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.471+0000 7f4ea22dd700 1 -- 172.21.15.38:0/1961982114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e9c107af0 msgr2=0x7f4e9c107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:26.499 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.471+0000 7f4ea22dd700 1 --2- 172.21.15.38:0/1961982114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e9c107af0 0x7f4e9c107f00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0098a0 tx=0x7f4e8c009bb0 comp rx=0 tx=0).stop 2024-05-21T15:25:26.499 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.471+0000 7f4ea22dd700 1 -- 172.21.15.38:0/1961982114 shutdown_connections 2024-05-21T15:25:26.499 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.471+0000 7f4ea22dd700 1 --2- 172.21.15.38:0/1961982114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e9c107af0 0x7f4e9c107f00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:26.499 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.471+0000 7f4ea22dd700 1 -- 172.21.15.38:0/1961982114 >> 172.21.15.38:0/1961982114 conn(0x7f4e9c1031b0 msgr2=0x7f4e9c105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:26.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.471+0000 7f4ea22dd700 1 -- 172.21.15.38:0/1961982114 shutdown_connections 2024-05-21T15:25:26.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.471+0000 7f4ea22dd700 1 -- 172.21.15.38:0/1961982114 wait complete. 2024-05-21T15:25:26.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.472+0000 7f4ea22dd700 1 Processor -- start 2024-05-21T15:25:26.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.472+0000 7f4ea22dd700 1 -- start start 2024-05-21T15:25:26.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.472+0000 7f4ea22dd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e9c107af0 0x7f4e9c106330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:26.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.472+0000 7f4ea22dd700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c10bfd0 con 0x7f4e9c107af0 2024-05-21T15:25:26.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.472+0000 7f4e9bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e9c107af0 0x7f4e9c106330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:26.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.472+0000 7f4e9bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e9c107af0 0x7f4e9c106330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54350/0 (socket says 172.21.15.38:54350) 2024-05-21T15:25:26.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.472+0000 7f4e9bfff700 1 -- 172.21.15.38:0/2267275833 learned_addr learned my addr 172.21.15.38:0/2267275833 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:26.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.472+0000 7f4e9bfff700 1 -- 172.21.15.38:0/2267275833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c106870 con 0x7f4e9c107af0 2024-05-21T15:25:26.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.473+0000 7f4e9bfff700 1 --2- 172.21.15.38:0/2267275833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e9c107af0 0x7f4e9c106330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c011ad0 tx=0x7f4e8c011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:26.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.473+0000 7f4e997fa700 1 -- 172.21.15.38:0/2267275833 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e8c003a40 con 0x7f4e9c107af0 2024-05-21T15:25:26.500 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.473+0000 7f4e997fa700 1 -- 172.21.15.38:0/2267275833 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e8c003be0 con 0x7f4e9c107af0 2024-05-21T15:25:26.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.473+0000 7f4ea22dd700 1 -- 172.21.15.38:0/2267275833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9c104ac0 con 0x7f4e9c107af0 2024-05-21T15:25:26.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.473+0000 7f4e997fa700 1 -- 172.21.15.38:0/2267275833 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e8c024740 con 0x7f4e9c107af0 2024-05-21T15:25:26.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.473+0000 7f4ea22dd700 1 -- 172.21.15.38:0/2267275833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e9c104f20 con 0x7f4e9c107af0 2024-05-21T15:25:26.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.474+0000 7f4e997fa700 1 -- 172.21.15.38:0/2267275833 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f4e8c0248e0 con 0x7f4e9c107af0 2024-05-21T15:25:26.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.474+0000 7f4e997fa700 1 --2- 172.21.15.38:0/2267275833 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4e84039100 0x7f4e8403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:26.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.474+0000 7f4e997fa700 1 -- 172.21.15.38:0/2267275833 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4e8c04e3d0 con 0x7f4e9c107af0 2024-05-21T15:25:26.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.474+0000 7f4e9b7fe700 1 --2- 172.21.15.38:0/2267275833 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4e84039100 0x7f4e8403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:26.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.474+0000 7f4ea22dd700 1 -- 172.21.15.38:0/2267275833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e9c063ca0 con 0x7f4e9c107af0 2024-05-21T15:25:26.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.475+0000 7f4e9b7fe700 1 --2- 172.21.15.38:0/2267275833 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4e84039100 0x7f4e8403b610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e90006fd0 tx=0x7f4e90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:26.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.480+0000 7f4e997fa700 1 -- 172.21.15.38:0/2267275833 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4e8c0172b0 con 0x7f4e9c107af0 2024-05-21T15:25:26.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.710+0000 7f4ea22dd700 1 -- 172.21.15.38:0/2267275833 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f4e9c195f30 con 0x7f4e84039100 2024-05-21T15:25:26.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.960+0000 7f4e997fa700 1 -- 172.21.15.38:0/2267275833 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f4e9c195f30 con 0x7f4e84039100 2024-05-21T15:25:26.501 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.962+0000 7f4ea22dd700 1 -- 172.21.15.38:0/2267275833 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4e84039100 msgr2=0x7f4e8403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:26.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.962+0000 7f4ea22dd700 1 --2- 172.21.15.38:0/2267275833 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4e84039100 0x7f4e8403b610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e90006fd0 tx=0x7f4e90006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:26.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.962+0000 7f4ea22dd700 1 -- 172.21.15.38:0/2267275833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e9c107af0 msgr2=0x7f4e9c106330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:26.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.962+0000 7f4ea22dd700 1 --2- 172.21.15.38:0/2267275833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e9c107af0 0x7f4e9c106330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c011ad0 tx=0x7f4e8c011bb0 comp rx=0 tx=0).stop 2024-05-21T15:25:26.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.963+0000 7f4ea22dd700 1 -- 172.21.15.38:0/2267275833 shutdown_connections 2024-05-21T15:25:26.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.963+0000 7f4ea22dd700 1 --2- 172.21.15.38:0/2267275833 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4e84039100 0x7f4e8403b610 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:26.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.963+0000 7f4ea22dd700 1 --2- 172.21.15.38:0/2267275833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e9c107af0 0x7f4e9c106330 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:26.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.963+0000 7f4ea22dd700 1 -- 172.21.15.38:0/2267275833 >> 172.21.15.38:0/2267275833 conn(0x7f4e9c1031b0 msgr2=0x7f4e9c1935f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:26.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.963+0000 7f4ea22dd700 1 -- 172.21.15.38:0/2267275833 shutdown_connections 2024-05-21T15:25:26.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:25.963+0000 7f4ea22dd700 1 -- 172.21.15.38:0/2267275833 wait complete. 2024-05-21T15:25:26.502 INFO:teuthology.orchestra.run.smithi038.stdout:Fetching dashboard port number... 2024-05-21T15:25:26.926 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:26 smithi038 ceph-mon[31995]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:25:26.926 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:26.927 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:27.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 8443 2024-05-21T15:25:27.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.938+0000 7f129b44a700 1 Processor -- start 2024-05-21T15:25:27.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.938+0000 7f129b44a700 1 -- start start 2024-05-21T15:25:27.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.938+0000 7f129b44a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12941097e0 0x7f129410bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:27.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.938+0000 7f129b44a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f129407c800 con 0x7f12941097e0 2024-05-21T15:25:27.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.938+0000 7f12991e6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12941097e0 0x7f129410bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:27.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.939+0000 7f12991e6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12941097e0 0x7f129410bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54358/0 (socket says 172.21.15.38:54358) 2024-05-21T15:25:27.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.939+0000 7f12991e6700 1 -- 172.21.15.38:0/2783204182 learned_addr learned my addr 172.21.15.38:0/2783204182 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:27.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.939+0000 7f12991e6700 1 -- 172.21.15.38:0/2783204182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129410c140 con 0x7f12941097e0 2024-05-21T15:25:27.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.939+0000 7f12991e6700 1 --2- 172.21.15.38:0/2783204182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12941097e0 0x7f129410bc00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f12900098a0 tx=0x7f1290009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df1dd1d8c11bc0c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:27.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.939+0000 7f128bfff700 1 -- 172.21.15.38:0/2783204182 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1290004160 con 0x7f12941097e0 2024-05-21T15:25:27.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.939+0000 7f128bfff700 1 -- 172.21.15.38:0/2783204182 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1290004300 con 0x7f12941097e0 2024-05-21T15:25:27.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.939+0000 7f128bfff700 1 -- 172.21.15.38:0/2783204182 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1290011b20 con 0x7f12941097e0 2024-05-21T15:25:27.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.940+0000 7f129b44a700 1 -- 172.21.15.38:0/2783204182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12941097e0 msgr2=0x7f129410bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:27.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.940+0000 7f129b44a700 1 --2- 172.21.15.38:0/2783204182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12941097e0 0x7f129410bc00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f12900098a0 tx=0x7f1290009bb0 comp rx=0 tx=0).stop 2024-05-21T15:25:27.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.940+0000 7f129b44a700 1 -- 172.21.15.38:0/2783204182 shutdown_connections 2024-05-21T15:25:27.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.940+0000 7f129b44a700 1 --2- 172.21.15.38:0/2783204182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12941097e0 0x7f129410bc00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:27.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.940+0000 7f129b44a700 1 -- 172.21.15.38:0/2783204182 >> 172.21.15.38:0/2783204182 conn(0x7f12941032c0 msgr2=0x7f1294105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:27.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.940+0000 7f129b44a700 1 -- 172.21.15.38:0/2783204182 shutdown_connections 2024-05-21T15:25:27.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.940+0000 7f129b44a700 1 -- 172.21.15.38:0/2783204182 wait complete. 2024-05-21T15:25:27.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.941+0000 7f129b44a700 1 Processor -- start 2024-05-21T15:25:27.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.941+0000 7f129b44a700 1 -- start start 2024-05-21T15:25:27.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.941+0000 7f129b44a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12941097e0 0x7f129419d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:27.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.941+0000 7f129b44a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f129410d0e0 con 0x7f12941097e0 2024-05-21T15:25:27.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.941+0000 7f12991e6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12941097e0 0x7f129419d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:27.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.941+0000 7f12991e6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12941097e0 0x7f129419d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54370/0 (socket says 172.21.15.38:54370) 2024-05-21T15:25:27.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.941+0000 7f12991e6700 1 -- 172.21.15.38:0/875868239 learned_addr learned my addr 172.21.15.38:0/875868239 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:27.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.941+0000 7f12991e6700 1 -- 172.21.15.38:0/875868239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129419d5c0 con 0x7f12941097e0 2024-05-21T15:25:27.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.942+0000 7f12991e6700 1 --2- 172.21.15.38:0/875868239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12941097e0 0x7f129419d080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1290000c00 tx=0x7f1290011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:27.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.942+0000 7f128a7fc700 1 -- 172.21.15.38:0/875868239 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1290003b60 con 0x7f12941097e0 2024-05-21T15:25:27.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.942+0000 7f128a7fc700 1 -- 172.21.15.38:0/875868239 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f129001a910 con 0x7f12941097e0 2024-05-21T15:25:27.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.942+0000 7f128a7fc700 1 -- 172.21.15.38:0/875868239 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1290024930 con 0x7f12941097e0 2024-05-21T15:25:27.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.942+0000 7f129b44a700 1 -- 172.21.15.38:0/875868239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129419d850 con 0x7f12941097e0 2024-05-21T15:25:27.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.942+0000 7f129b44a700 1 -- 172.21.15.38:0/875868239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12941a0460 con 0x7f12941097e0 2024-05-21T15:25:27.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.943+0000 7f128a7fc700 1 -- 172.21.15.38:0/875868239 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f1290022020 con 0x7f12941097e0 2024-05-21T15:25:27.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.943+0000 7f129b44a700 1 -- 172.21.15.38:0/875868239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1294063ca0 con 0x7f12941097e0 2024-05-21T15:25:27.736 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.943+0000 7f128a7fc700 1 --2- 172.21.15.38:0/875868239 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12800390a0 0x7f128003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:27.736 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.943+0000 7f128a7fc700 1 -- 172.21.15.38:0/875868239 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f129004d540 con 0x7f12941097e0 2024-05-21T15:25:27.736 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.943+0000 7f12989e5700 1 --2- 172.21.15.38:0/875868239 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12800390a0 0x7f128003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:27.736 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.944+0000 7f12989e5700 1 --2- 172.21.15.38:0/875868239 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12800390a0 0x7f128003b5b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1284006fd0 tx=0x7f1284006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:27.736 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:26.949+0000 7f128a7fc700 1 -- 172.21.15.38:0/875868239 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f129000b5a0 con 0x7f12941097e0 2024-05-21T15:25:27.736 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:27.163+0000 7f129b44a700 1 -- 172.21.15.38:0/875868239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f12941a0630 con 0x7f12941097e0 2024-05-21T15:25:27.736 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:27.164+0000 7f128a7fc700 1 -- 172.21.15.38:0/875868239 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f129001ab30 con 0x7f12941097e0 2024-05-21T15:25:27.736 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:27.167+0000 7f129b44a700 1 -- 172.21.15.38:0/875868239 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12800390a0 msgr2=0x7f128003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:27.737 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:27.167+0000 7f129b44a700 1 --2- 172.21.15.38:0/875868239 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12800390a0 0x7f128003b5b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1284006fd0 tx=0x7f1284006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:27.737 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:27.167+0000 7f129b44a700 1 -- 172.21.15.38:0/875868239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12941097e0 msgr2=0x7f129419d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:27.737 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:27.167+0000 7f129b44a700 1 --2- 172.21.15.38:0/875868239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12941097e0 0x7f129419d080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1290000c00 tx=0x7f1290011c10 comp rx=0 tx=0).stop 2024-05-21T15:25:27.737 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:27.167+0000 7f129b44a700 1 -- 172.21.15.38:0/875868239 shutdown_connections 2024-05-21T15:25:27.737 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:27.167+0000 7f129b44a700 1 --2- 172.21.15.38:0/875868239 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12800390a0 0x7f128003b5b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:27.737 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:27.168+0000 7f129b44a700 1 --2- 172.21.15.38:0/875868239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12941097e0 0x7f129419d080 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:27.737 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:27.168+0000 7f129b44a700 1 -- 172.21.15.38:0/875868239 >> 172.21.15.38:0/875868239 conn(0x7f12941032c0 msgr2=0x7f1294104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:27.737 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:27.168+0000 7f129b44a700 1 -- 172.21.15.38:0/875868239 shutdown_connections 2024-05-21T15:25:27.738 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:27.168+0000 7f129b44a700 1 -- 172.21.15.38:0/875868239 wait complete. 2024-05-21T15:25:27.739 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T15:25:27.739 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-05-21T15:25:27.747 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T15:25:27.747 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-05-21T15:25:27.747 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-05-21T15:25:27.747 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-21T15:25:27.749 INFO:teuthology.orchestra.run.smithi038.stdout:Ceph Dashboard is now available at: 2024-05-21T15:25:27.749 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:25:27.749 INFO:teuthology.orchestra.run.smithi038.stdout: URL: https://smithi038.front.sepia.ceph.com:8443/ 2024-05-21T15:25:27.749 INFO:teuthology.orchestra.run.smithi038.stdout: User: admin 2024-05-21T15:25:27.749 INFO:teuthology.orchestra.run.smithi038.stdout: Password: vdjj5lreoc 2024-05-21T15:25:27.749 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:25:27.750 INFO:teuthology.orchestra.run.smithi038.stdout:Saving cluster configuration to /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config directory 2024-05-21T15:25:27.750 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling autotune for osd_memory_target 2024-05-21T15:25:28.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:27 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/875868239' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-21T15:25:29.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.037+0000 7f25975d0700 1 Processor -- start 2024-05-21T15:25:29.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.037+0000 7f25975d0700 1 -- start start 2024-05-21T15:25:29.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.037+0000 7f25975d0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2590109760 0x7f259010bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:29.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.037+0000 7f25975d0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f259007c660 con 0x7f2590109760 2024-05-21T15:25:29.693 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.037+0000 7f259536c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2590109760 0x7f259010bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:29.694 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.038+0000 7f259536c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2590109760 0x7f259010bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54380/0 (socket says 172.21.15.38:54380) 2024-05-21T15:25:29.694 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.038+0000 7f259536c700 1 -- 172.21.15.38:0/1110430555 learned_addr learned my addr 172.21.15.38:0/1110430555 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:29.694 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.038+0000 7f259536c700 1 -- 172.21.15.38:0/1110430555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259010c0c0 con 0x7f2590109760 2024-05-21T15:25:29.694 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.038+0000 7f259536c700 1 --2- 172.21.15.38:0/1110430555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2590109760 0x7f259010bb80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f258c0098a0 tx=0x7f258c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06985387daf2dfa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:29.694 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.038+0000 7f2587fff700 1 -- 172.21.15.38:0/1110430555 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f258c004160 con 0x7f2590109760 2024-05-21T15:25:29.694 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.038+0000 7f2587fff700 1 -- 172.21.15.38:0/1110430555 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258c004300 con 0x7f2590109760 2024-05-21T15:25:29.694 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.038+0000 7f2587fff700 1 -- 172.21.15.38:0/1110430555 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f258c011b20 con 0x7f2590109760 2024-05-21T15:25:29.694 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.039+0000 7f25975d0700 1 -- 172.21.15.38:0/1110430555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2590109760 msgr2=0x7f259010bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:29.694 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.039+0000 7f25975d0700 1 --2- 172.21.15.38:0/1110430555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2590109760 0x7f259010bb80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f258c0098a0 tx=0x7f258c009bb0 comp rx=0 tx=0).stop 2024-05-21T15:25:29.694 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.039+0000 7f25975d0700 1 -- 172.21.15.38:0/1110430555 shutdown_connections 2024-05-21T15:25:29.694 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.039+0000 7f25975d0700 1 --2- 172.21.15.38:0/1110430555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2590109760 0x7f259010bb80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:29.694 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.039+0000 7f25975d0700 1 -- 172.21.15.38:0/1110430555 >> 172.21.15.38:0/1110430555 conn(0x7f2590103220 msgr2=0x7f2590105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:29.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.039+0000 7f25975d0700 1 -- 172.21.15.38:0/1110430555 shutdown_connections 2024-05-21T15:25:29.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.039+0000 7f25975d0700 1 -- 172.21.15.38:0/1110430555 wait complete. 2024-05-21T15:25:29.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.040+0000 7f25975d0700 1 Processor -- start 2024-05-21T15:25:29.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.040+0000 7f25975d0700 1 -- start start 2024-05-21T15:25:29.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.040+0000 7f25975d0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2590109760 0x7f259019d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:29.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.040+0000 7f25975d0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f259010d060 con 0x7f2590109760 2024-05-21T15:25:29.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.040+0000 7f259536c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2590109760 0x7f259019d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:29.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.040+0000 7f259536c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2590109760 0x7f259019d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54396/0 (socket says 172.21.15.38:54396) 2024-05-21T15:25:29.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.040+0000 7f259536c700 1 -- 172.21.15.38:0/1519366276 learned_addr learned my addr 172.21.15.38:0/1519366276 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:29.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.040+0000 7f259536c700 1 -- 172.21.15.38:0/1519366276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259019d5e0 con 0x7f2590109760 2024-05-21T15:25:29.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.041+0000 7f259536c700 1 --2- 172.21.15.38:0/1519366276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2590109760 0x7f259019d0a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f258c009fb0 tx=0x7f258c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:29.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.041+0000 7f25867fc700 1 -- 172.21.15.38:0/1519366276 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f258c003930 con 0x7f2590109760 2024-05-21T15:25:29.695 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.041+0000 7f25975d0700 1 -- 172.21.15.38:0/1519366276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259019d870 con 0x7f2590109760 2024-05-21T15:25:29.696 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.041+0000 7f25867fc700 1 -- 172.21.15.38:0/1519366276 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258c003ad0 con 0x7f2590109760 2024-05-21T15:25:29.696 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.041+0000 7f25867fc700 1 -- 172.21.15.38:0/1519366276 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f258c019670 con 0x7f2590109760 2024-05-21T15:25:29.696 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.041+0000 7f25975d0700 1 -- 172.21.15.38:0/1519366276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25901a0480 con 0x7f2590109760 2024-05-21T15:25:29.696 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.042+0000 7f25975d0700 1 -- 172.21.15.38:0/1519366276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259019dcd0 con 0x7f2590109760 2024-05-21T15:25:29.696 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.042+0000 7f25867fc700 1 -- 172.21.15.38:0/1519366276 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f258c019970 con 0x7f2590109760 2024-05-21T15:25:29.696 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.042+0000 7f25867fc700 1 --2- 172.21.15.38:0/1519366276 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f257c0390f0 0x7f257c03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:29.696 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.042+0000 7f25867fc700 1 -- 172.21.15.38:0/1519366276 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f258c04d7b0 con 0x7f2590109760 2024-05-21T15:25:29.696 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.042+0000 7f2594b6b700 1 --2- 172.21.15.38:0/1519366276 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f257c0390f0 0x7f257c03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:29.696 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.043+0000 7f2594b6b700 1 --2- 172.21.15.38:0/1519366276 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f257c0390f0 0x7f257c03b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2580006fd0 tx=0x7f2580006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:29.696 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.047+0000 7f25867fc700 1 -- 172.21.15.38:0/1519366276 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f258c011e20 con 0x7f2590109760 2024-05-21T15:25:29.696 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.253+0000 7f25975d0700 1 -- 172.21.15.38:0/1519366276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f25901a0650 con 0x7f2590109760 2024-05-21T15:25:29.696 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.254+0000 7f25867fc700 1 -- 172.21.15.38:0/1519366276 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f258c019db0 con 0x7f2590109760 2024-05-21T15:25:29.696 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.256+0000 7f25975d0700 1 -- 172.21.15.38:0/1519366276 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f257c0390f0 msgr2=0x7f257c03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:29.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.256+0000 7f25975d0700 1 --2- 172.21.15.38:0/1519366276 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f257c0390f0 0x7f257c03b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2580006fd0 tx=0x7f2580006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:29.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.256+0000 7f25975d0700 1 -- 172.21.15.38:0/1519366276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2590109760 msgr2=0x7f259019d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:29.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.256+0000 7f25975d0700 1 --2- 172.21.15.38:0/1519366276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2590109760 0x7f259019d0a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f258c009fb0 tx=0x7f258c011c10 comp rx=0 tx=0).stop 2024-05-21T15:25:29.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.256+0000 7f25975d0700 1 -- 172.21.15.38:0/1519366276 shutdown_connections 2024-05-21T15:25:29.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.256+0000 7f25975d0700 1 --2- 172.21.15.38:0/1519366276 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f257c0390f0 0x7f257c03b600 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:29.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.256+0000 7f25975d0700 1 --2- 172.21.15.38:0/1519366276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2590109760 0x7f259019d0a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:29.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.256+0000 7f25975d0700 1 -- 172.21.15.38:0/1519366276 >> 172.21.15.38:0/1519366276 conn(0x7f2590103220 msgr2=0x7f2590103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:29.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.256+0000 7f25975d0700 1 -- 172.21.15.38:0/1519366276 shutdown_connections 2024-05-21T15:25:29.697 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:29.257+0000 7f25975d0700 1 -- 172.21.15.38:0/1519366276 wait complete. 2024-05-21T15:25:30.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:29 smithi038 ceph-mon[31995]: mgrmap e13: x(active, since 6s) 2024-05-21T15:25:31.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.118+0000 7f9cd44b2700 1 Processor -- start 2024-05-21T15:25:31.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.118+0000 7f9cd44b2700 1 -- start start 2024-05-21T15:25:31.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.119+0000 7f9cd44b2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ccc10a1f0 0x7f9ccc10a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.119+0000 7f9cd44b2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc10abd0 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.119+0000 7f9cd224e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ccc10a1f0 0x7f9ccc10a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.119+0000 7f9cd224e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ccc10a1f0 0x7f9ccc10a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54412/0 (socket says 172.21.15.38:54412) 2024-05-21T15:25:31.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.119+0000 7f9cd224e700 1 -- 172.21.15.38:0/1622458492 learned_addr learned my addr 172.21.15.38:0/1622458492 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:31.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.119+0000 7f9cd224e700 1 -- 172.21.15.38:0/1622458492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc10ad50 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.119+0000 7f9cd224e700 1 --2- 172.21.15.38:0/1622458492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ccc10a1f0 0x7f9ccc10a600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009e80 tx=0x7f9cc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c0e7cf9e66436c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:31.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.120+0000 7f9cd124c700 1 -- 172.21.15.38:0/1622458492 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cc8017070 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.120+0000 7f9cd124c700 1 -- 172.21.15.38:0/1622458492 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cc800b8c0 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.120+0000 7f9cd124c700 1 -- 172.21.15.38:0/1622458492 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cc80045c0 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.120+0000 7f9cd44b2700 1 -- 172.21.15.38:0/1622458492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ccc10a1f0 msgr2=0x7f9ccc10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:31.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.120+0000 7f9cd44b2700 1 --2- 172.21.15.38:0/1622458492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ccc10a1f0 0x7f9ccc10a600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009e80 tx=0x7f9cc800b330 comp rx=0 tx=0).stop 2024-05-21T15:25:31.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.120+0000 7f9cd44b2700 1 -- 172.21.15.38:0/1622458492 shutdown_connections 2024-05-21T15:25:31.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.121+0000 7f9cd44b2700 1 --2- 172.21.15.38:0/1622458492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ccc10a1f0 0x7f9ccc10a600 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.121+0000 7f9cd44b2700 1 -- 172.21.15.38:0/1622458492 >> 172.21.15.38:0/1622458492 conn(0x7f9ccc1029b0 msgr2=0x7f9ccc104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:31.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.121+0000 7f9cd44b2700 1 -- 172.21.15.38:0/1622458492 shutdown_connections 2024-05-21T15:25:31.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.121+0000 7f9cd44b2700 1 -- 172.21.15.38:0/1622458492 wait complete. 2024-05-21T15:25:31.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.121+0000 7f9cd44b2700 1 Processor -- start 2024-05-21T15:25:31.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.121+0000 7f9cd44b2700 1 -- start start 2024-05-21T15:25:31.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.122+0000 7f9cd44b2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ccc10a1f0 0x7f9ccc19d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.122+0000 7f9cd44b2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc10d190 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.122+0000 7f9cd224e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ccc10a1f0 0x7f9ccc19d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.122+0000 7f9cd224e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ccc10a1f0 0x7f9ccc19d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54424/0 (socket says 172.21.15.38:54424) 2024-05-21T15:25:31.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.122+0000 7f9cd224e700 1 -- 172.21.15.38:0/2328982734 learned_addr learned my addr 172.21.15.38:0/2328982734 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:31.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.122+0000 7f9cd224e700 1 -- 172.21.15.38:0/2328982734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc19d5b0 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.122+0000 7f9cd224e700 1 --2- 172.21.15.38:0/2328982734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ccc10a1f0 0x7f9ccc19d070 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009fd0 tx=0x7f9cc8003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:31.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.122+0000 7f9cc37fe700 1 -- 172.21.15.38:0/2328982734 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cc8020020 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.122+0000 7f9cd44b2700 1 -- 172.21.15.38:0/2328982734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ccc19d840 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.122+0000 7f9cc37fe700 1 -- 172.21.15.38:0/2328982734 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cc8012430 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.122+0000 7f9cc37fe700 1 -- 172.21.15.38:0/2328982734 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cc8017040 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.122+0000 7f9cd44b2700 1 -- 172.21.15.38:0/2328982734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ccc1a0450 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.123+0000 7f9cc37fe700 1 -- 172.21.15.38:0/2328982734 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9cc8023020 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.123+0000 7f9cc37fe700 1 --2- 172.21.15.38:0/2328982734 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9cb8039140 0x7f9cb803b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.123+0000 7f9cc37fe700 1 -- 172.21.15.38:0/2328982734 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9cc800f050 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.123+0000 7f9cd44b2700 1 -- 172.21.15.38:0/2328982734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ccc063ca0 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.124+0000 7f9cd1a4d700 1 --2- 172.21.15.38:0/2328982734 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9cb8039140 0x7f9cb803b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.124+0000 7f9cd1a4d700 1 --2- 172.21.15.38:0/2328982734 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9cb8039140 0x7f9cb803b650 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc006fd0 tx=0x7f9cbc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:31.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.129+0000 7f9cc37fe700 1 -- 172.21.15.38:0/2328982734 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9cc802a020 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.453+0000 7f9cd44b2700 1 -- 172.21.15.38:0/2328982734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f9ccc1a0a00 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.458+0000 7f9cc37fe700 1 -- 172.21.15.38:0/2328982734 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f9cc8011390 con 0x7f9ccc10a1f0 2024-05-21T15:25:31.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.460+0000 7f9cd44b2700 1 -- 172.21.15.38:0/2328982734 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9cb8039140 msgr2=0x7f9cb803b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:31.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.460+0000 7f9cd44b2700 1 --2- 172.21.15.38:0/2328982734 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9cb8039140 0x7f9cb803b650 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc006fd0 tx=0x7f9cbc006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:31.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.460+0000 7f9cd44b2700 1 -- 172.21.15.38:0/2328982734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ccc10a1f0 msgr2=0x7f9ccc19d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:31.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.460+0000 7f9cd44b2700 1 --2- 172.21.15.38:0/2328982734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ccc10a1f0 0x7f9ccc19d070 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009fd0 tx=0x7f9cc8003d00 comp rx=0 tx=0).stop 2024-05-21T15:25:31.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.460+0000 7f9cd44b2700 1 -- 172.21.15.38:0/2328982734 shutdown_connections 2024-05-21T15:25:31.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.460+0000 7f9cd44b2700 1 --2- 172.21.15.38:0/2328982734 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9cb8039140 0x7f9cb803b650 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.460+0000 7f9cd44b2700 1 --2- 172.21.15.38:0/2328982734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ccc10a1f0 0x7f9ccc19d070 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.031 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.460+0000 7f9cd44b2700 1 -- 172.21.15.38:0/2328982734 >> 172.21.15.38:0/2328982734 conn(0x7f9ccc1029b0 msgr2=0x7f9ccc104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:31.031 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.460+0000 7f9cd44b2700 1 -- 172.21.15.38:0/2328982734 shutdown_connections 2024-05-21T15:25:31.031 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-05-21T15:25:30.460+0000 7f9cd44b2700 1 -- 172.21.15.38:0/2328982734 wait complete. 2024-05-21T15:25:31.031 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-21T15:25:31.031 INFO:teuthology.orchestra.run.smithi038.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-21T15:25:31.031 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:25:31.031 INFO:teuthology.orchestra.run.smithi038.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:25:31.031 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:25:31.031 INFO:teuthology.orchestra.run.smithi038.stdout:Or, if you are only running a single cluster on this host: 2024-05-21T15:25:31.031 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:25:31.031 INFO:teuthology.orchestra.run.smithi038.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-21T15:25:31.032 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:25:31.032 INFO:teuthology.orchestra.run.smithi038.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-21T15:25:31.032 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:25:31.032 INFO:teuthology.orchestra.run.smithi038.stdout: ceph telemetry on 2024-05-21T15:25:31.032 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:25:31.032 INFO:teuthology.orchestra.run.smithi038.stdout:For more information see: 2024-05-21T15:25:31.032 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:25:31.032 INFO:teuthology.orchestra.run.smithi038.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-21T15:25:31.033 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:25:31.033 INFO:teuthology.orchestra.run.smithi038.stdout:Bootstrap complete. 2024-05-21T15:25:31.066 INFO:tasks.cephadm:Fetching config... 2024-05-21T15:25:31.066 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:25:31.066 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-21T15:25:31.083 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-21T15:25:31.083 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:25:31.083 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-21T15:25:31.143 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-21T15:25:31.144 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:25:31.144 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-21T15:25:31.210 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-21T15:25:31.210 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:25:31.210 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-21T15:25:31.266 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-21T15:25:31.267 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcR17PAkEDw2SCQ3e4KouhkPPSB7TBcTjeVLnBiYvGKYaJ7pDQ95chFcKonzFxfnDpoElb1OfAVEzg+2pNzxs0j8ayHtUUlbJJo6qRi6bUEBfANJudXawfmpjXDVOHTpfvK4duLXQQB2jRtraffoLhUypuXlRMXyDJJ3VshkeyDgpTSh5gPrXOJgUJyPe6peKtyLPvAK31OVdoQiD4/2Tyn4x5JBdde97EqLnpQpwDuj7S9ycR2LDytnrKaKZ8+/0DHmV0328K6PKJg+OksPFZb1XOEa9rjPLP8AyprOLbeRjKEC+f/OL/hPot8zSDxdoTo5McYUK43w86CmHcBveE2LbsZzTSuxDI1qE1qY7BNNjuHGp7n+5BRAKmH2WPSCoIMS/tUCEvyubG7lL6SGX8dIOAQ3BUWshf+eS8I0GoVtq5+8+UjTdLMNRFRJlKyzA40JmaUQ60kZB8gtxOej63JVaE70o9KDPaYxYPkqo32Q1Y42TGygacYGDCyfGNTuU= ceph-102ec4fa-1786-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T15:25:31.341 INFO:teuthology.orchestra.run.smithi038.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcR17PAkEDw2SCQ3e4KouhkPPSB7TBcTjeVLnBiYvGKYaJ7pDQ95chFcKonzFxfnDpoElb1OfAVEzg+2pNzxs0j8ayHtUUlbJJo6qRi6bUEBfANJudXawfmpjXDVOHTpfvK4duLXQQB2jRtraffoLhUypuXlRMXyDJJ3VshkeyDgpTSh5gPrXOJgUJyPe6peKtyLPvAK31OVdoQiD4/2Tyn4x5JBdde97EqLnpQpwDuj7S9ycR2LDytnrKaKZ8+/0DHmV0328K6PKJg+OksPFZb1XOEa9rjPLP8AyprOLbeRjKEC+f/OL/hPot8zSDxdoTo5McYUK43w86CmHcBveE2LbsZzTSuxDI1qE1qY7BNNjuHGp7n+5BRAKmH2WPSCoIMS/tUCEvyubG7lL6SGX8dIOAQ3BUWshf+eS8I0GoVtq5+8+UjTdLMNRFRJlKyzA40JmaUQ60kZB8gtxOej63JVaE70o9KDPaYxYPkqo32Q1Y42TGygacYGDCyfGNTuU= ceph-102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:25:31.353 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcR17PAkEDw2SCQ3e4KouhkPPSB7TBcTjeVLnBiYvGKYaJ7pDQ95chFcKonzFxfnDpoElb1OfAVEzg+2pNzxs0j8ayHtUUlbJJo6qRi6bUEBfANJudXawfmpjXDVOHTpfvK4duLXQQB2jRtraffoLhUypuXlRMXyDJJ3VshkeyDgpTSh5gPrXOJgUJyPe6peKtyLPvAK31OVdoQiD4/2Tyn4x5JBdde97EqLnpQpwDuj7S9ycR2LDytnrKaKZ8+/0DHmV0328K6PKJg+OksPFZb1XOEa9rjPLP8AyprOLbeRjKEC+f/OL/hPot8zSDxdoTo5McYUK43w86CmHcBveE2LbsZzTSuxDI1qE1qY7BNNjuHGp7n+5BRAKmH2WPSCoIMS/tUCEvyubG7lL6SGX8dIOAQ3BUWshf+eS8I0GoVtq5+8+UjTdLMNRFRJlKyzA40JmaUQ60kZB8gtxOej63JVaE70o9KDPaYxYPkqo32Q1Y42TGygacYGDCyfGNTuU= ceph-102ec4fa-1786-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T15:25:31.394 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcR17PAkEDw2SCQ3e4KouhkPPSB7TBcTjeVLnBiYvGKYaJ7pDQ95chFcKonzFxfnDpoElb1OfAVEzg+2pNzxs0j8ayHtUUlbJJo6qRi6bUEBfANJudXawfmpjXDVOHTpfvK4duLXQQB2jRtraffoLhUypuXlRMXyDJJ3VshkeyDgpTSh5gPrXOJgUJyPe6peKtyLPvAK31OVdoQiD4/2Tyn4x5JBdde97EqLnpQpwDuj7S9ycR2LDytnrKaKZ8+/0DHmV0328K6PKJg+OksPFZb1XOEa9rjPLP8AyprOLbeRjKEC+f/OL/hPot8zSDxdoTo5McYUK43w86CmHcBveE2LbsZzTSuxDI1qE1qY7BNNjuHGp7n+5BRAKmH2WPSCoIMS/tUCEvyubG7lL6SGX8dIOAQ3BUWshf+eS8I0GoVtq5+8+UjTdLMNRFRJlKyzA40JmaUQ60kZB8gtxOej63JVaE70o9KDPaYxYPkqo32Q1Y42TGygacYGDCyfGNTuU= ceph-102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:25:31.406 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcR17PAkEDw2SCQ3e4KouhkPPSB7TBcTjeVLnBiYvGKYaJ7pDQ95chFcKonzFxfnDpoElb1OfAVEzg+2pNzxs0j8ayHtUUlbJJo6qRi6bUEBfANJudXawfmpjXDVOHTpfvK4duLXQQB2jRtraffoLhUypuXlRMXyDJJ3VshkeyDgpTSh5gPrXOJgUJyPe6peKtyLPvAK31OVdoQiD4/2Tyn4x5JBdde97EqLnpQpwDuj7S9ycR2LDytnrKaKZ8+/0DHmV0328K6PKJg+OksPFZb1XOEa9rjPLP8AyprOLbeRjKEC+f/OL/hPot8zSDxdoTo5McYUK43w86CmHcBveE2LbsZzTSuxDI1qE1qY7BNNjuHGp7n+5BRAKmH2WPSCoIMS/tUCEvyubG7lL6SGX8dIOAQ3BUWshf+eS8I0GoVtq5+8+UjTdLMNRFRJlKyzA40JmaUQ60kZB8gtxOej63JVaE70o9KDPaYxYPkqo32Q1Y42TGygacYGDCyfGNTuU= ceph-102ec4fa-1786-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T15:25:31.446 INFO:teuthology.orchestra.run.smithi053.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcR17PAkEDw2SCQ3e4KouhkPPSB7TBcTjeVLnBiYvGKYaJ7pDQ95chFcKonzFxfnDpoElb1OfAVEzg+2pNzxs0j8ayHtUUlbJJo6qRi6bUEBfANJudXawfmpjXDVOHTpfvK4duLXQQB2jRtraffoLhUypuXlRMXyDJJ3VshkeyDgpTSh5gPrXOJgUJyPe6peKtyLPvAK31OVdoQiD4/2Tyn4x5JBdde97EqLnpQpwDuj7S9ycR2LDytnrKaKZ8+/0DHmV0328K6PKJg+OksPFZb1XOEa9rjPLP8AyprOLbeRjKEC+f/OL/hPot8zSDxdoTo5McYUK43w86CmHcBveE2LbsZzTSuxDI1qE1qY7BNNjuHGp7n+5BRAKmH2WPSCoIMS/tUCEvyubG7lL6SGX8dIOAQ3BUWshf+eS8I0GoVtq5+8+UjTdLMNRFRJlKyzA40JmaUQ60kZB8gtxOej63JVaE70o9KDPaYxYPkqo32Q1Y42TGygacYGDCyfGNTuU= ceph-102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:25:31.459 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-21T15:25:31.707 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:25:31.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:31 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2328982734' entity='client.admin' 2024-05-21T15:25:34.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.135+0000 7f967b6da700 1 -- 172.21.15.38:0/29623905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9674103590 msgr2=0x7f96741059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.135+0000 7f967b6da700 1 --2- 172.21.15.38:0/29623905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9674103590 0x7f96741059b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9670009910 tx=0x7f9670009c20 comp rx=0 tx=0).stop 2024-05-21T15:25:34.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.135+0000 7f967b6da700 1 -- 172.21.15.38:0/29623905 shutdown_connections 2024-05-21T15:25:34.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.135+0000 7f967b6da700 1 --2- 172.21.15.38:0/29623905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9674103590 0x7f96741059b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.135+0000 7f967b6da700 1 -- 172.21.15.38:0/29623905 >> 172.21.15.38:0/29623905 conn(0x7f96740fcfb0 msgr2=0x7f96740ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:34.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.136+0000 7f967b6da700 1 -- 172.21.15.38:0/29623905 shutdown_connections 2024-05-21T15:25:34.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.136+0000 7f967b6da700 1 -- 172.21.15.38:0/29623905 wait complete. 2024-05-21T15:25:34.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.136+0000 7f967b6da700 1 Processor -- start 2024-05-21T15:25:34.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.136+0000 7f967b6da700 1 -- start start 2024-05-21T15:25:34.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.137+0000 7f967b6da700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9674103590 0x7f9674102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.137+0000 7f967b6da700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9674108ba0 con 0x7f9674103590 2024-05-21T15:25:34.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.137+0000 7f9679476700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9674103590 0x7f9674102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.137+0000 7f9679476700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9674103590 0x7f9674102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54436/0 (socket says 172.21.15.38:54436) 2024-05-21T15:25:34.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.137+0000 7f9679476700 1 -- 172.21.15.38:0/3182038536 learned_addr learned my addr 172.21.15.38:0/3182038536 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:34.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.137+0000 7f9679476700 1 -- 172.21.15.38:0/3182038536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9674103250 con 0x7f9674103590 2024-05-21T15:25:34.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.137+0000 7f9679476700 1 --2- 172.21.15.38:0/3182038536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9674103590 0x7f9674102d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9670004fb0 tx=0x7f9670005090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:34.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.137+0000 7f966a7fc700 1 -- 172.21.15.38:0/3182038536 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9670027070 con 0x7f9674103590 2024-05-21T15:25:34.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.137+0000 7f966a7fc700 1 -- 172.21.15.38:0/3182038536 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9670022470 con 0x7f9674103590 2024-05-21T15:25:34.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.137+0000 7f967b6da700 1 -- 172.21.15.38:0/3182038536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96741014a0 con 0x7f9674103590 2024-05-21T15:25:34.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.137+0000 7f966a7fc700 1 -- 172.21.15.38:0/3182038536 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f967000f5b0 con 0x7f9674103590 2024-05-21T15:25:34.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.138+0000 7f967b6da700 1 -- 172.21.15.38:0/3182038536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9674101980 con 0x7f9674103590 2024-05-21T15:25:34.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.138+0000 7f967b6da700 1 -- 172.21.15.38:0/3182038536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9674109320 con 0x7f9674103590 2024-05-21T15:25:34.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.138+0000 7f966a7fc700 1 -- 172.21.15.38:0/3182038536 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f967002a070 con 0x7f9674103590 2024-05-21T15:25:34.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.139+0000 7f966a7fc700 1 --2- 172.21.15.38:0/3182038536 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9660039150 0x7f966003b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.139+0000 7f966a7fc700 1 -- 172.21.15.38:0/3182038536 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f967001e030 con 0x7f9674103590 2024-05-21T15:25:34.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.139+0000 7f9678c75700 1 --2- 172.21.15.38:0/3182038536 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9660039150 0x7f966003b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.139+0000 7f9678c75700 1 --2- 172.21.15.38:0/3182038536 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9660039150 0x7f966003b660 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9664006fd0 tx=0x7f9664006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:34.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.144+0000 7f966a7fc700 1 -- 172.21.15.38:0/3182038536 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9670013070 con 0x7f9674103590 2024-05-21T15:25:34.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.350+0000 7f967b6da700 1 -- 172.21.15.38:0/3182038536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f9674191540 con 0x7f9674103590 2024-05-21T15:25:34.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.354+0000 7f966a7fc700 1 -- 172.21.15.38:0/3182038536 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f96700292c0 con 0x7f9674103590 2024-05-21T15:25:34.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.357+0000 7f967b6da700 1 -- 172.21.15.38:0/3182038536 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9660039150 msgr2=0x7f966003b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.357+0000 7f967b6da700 1 --2- 172.21.15.38:0/3182038536 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9660039150 0x7f966003b660 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9664006fd0 tx=0x7f9664006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:34.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.357+0000 7f967b6da700 1 -- 172.21.15.38:0/3182038536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9674103590 msgr2=0x7f9674102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.357+0000 7f967b6da700 1 --2- 172.21.15.38:0/3182038536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9674103590 0x7f9674102d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9670004fb0 tx=0x7f9670005090 comp rx=0 tx=0).stop 2024-05-21T15:25:34.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.357+0000 7f967b6da700 1 -- 172.21.15.38:0/3182038536 shutdown_connections 2024-05-21T15:25:34.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.357+0000 7f967b6da700 1 --2- 172.21.15.38:0/3182038536 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9660039150 0x7f966003b660 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.357+0000 7f967b6da700 1 --2- 172.21.15.38:0/3182038536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9674103590 0x7f9674102d10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.357+0000 7f967b6da700 1 -- 172.21.15.38:0/3182038536 >> 172.21.15.38:0/3182038536 conn(0x7f96740fcfb0 msgr2=0x7f96740fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:34.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.357+0000 7f967b6da700 1 -- 172.21.15.38:0/3182038536 shutdown_connections 2024-05-21T15:25:34.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:34.357+0000 7f967b6da700 1 -- 172.21.15.38:0/3182038536 wait complete. 2024-05-21T15:25:34.963 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-21T15:25:34.963 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-21T15:25:35.640 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:25:35.764 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:35 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3182038536' entity='client.admin' 2024-05-21T15:25:36.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:36.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:36.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:25:36.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:36.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:25:36.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:37.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.167+0000 7f508c54f700 1 -- 172.21.15.38:0/1015041659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50841010f0 msgr2=0x7f5084101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:37.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.167+0000 7f508c54f700 1 --2- 172.21.15.38:0/1015041659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50841010f0 0x7f5084101500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f508000aa20 tx=0x7f508000ad30 comp rx=0 tx=0).stop 2024-05-21T15:25:37.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.167+0000 7f508c54f700 1 -- 172.21.15.38:0/1015041659 shutdown_connections 2024-05-21T15:25:37.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.167+0000 7f508c54f700 1 --2- 172.21.15.38:0/1015041659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50841010f0 0x7f5084101500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:37.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.167+0000 7f508c54f700 1 -- 172.21.15.38:0/1015041659 >> 172.21.15.38:0/1015041659 conn(0x7f50840fc730 msgr2=0x7f50840febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:37.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.167+0000 7f508c54f700 1 -- 172.21.15.38:0/1015041659 shutdown_connections 2024-05-21T15:25:37.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.167+0000 7f508c54f700 1 -- 172.21.15.38:0/1015041659 wait complete. 2024-05-21T15:25:37.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.168+0000 7f508c54f700 1 Processor -- start 2024-05-21T15:25:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.168+0000 7f508c54f700 1 -- start start 2024-05-21T15:25:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.168+0000 7f508c54f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50841010f0 0x7f5084198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.168+0000 7f508c54f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5084106970 con 0x7f50841010f0 2024-05-21T15:25:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.168+0000 7f508a2eb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50841010f0 0x7f5084198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.168+0000 7f508a2eb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50841010f0 0x7f5084198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53836/0 (socket says 172.21.15.38:53836) 2024-05-21T15:25:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.168+0000 7f508a2eb700 1 -- 172.21.15.38:0/2650043851 learned_addr learned my addr 172.21.15.38:0/2650043851 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.168+0000 7f508a2eb700 1 -- 172.21.15.38:0/2650043851 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5084198bb0 con 0x7f50841010f0 2024-05-21T15:25:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.169+0000 7f508a2eb700 1 --2- 172.21.15.38:0/2650043851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50841010f0 0x7f5084198670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5080000c00 tx=0x7f5080005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.169+0000 7f507b7fe700 1 -- 172.21.15.38:0/2650043851 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5080005410 con 0x7f50841010f0 2024-05-21T15:25:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.169+0000 7f507b7fe700 1 -- 172.21.15.38:0/2650043851 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50800055b0 con 0x7f50841010f0 2024-05-21T15:25:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.169+0000 7f508c54f700 1 -- 172.21.15.38:0/2650043851 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5084198e40 con 0x7f50841010f0 2024-05-21T15:25:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.169+0000 7f507b7fe700 1 -- 172.21.15.38:0/2650043851 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f508000e870 con 0x7f50841010f0 2024-05-21T15:25:37.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.169+0000 7f508c54f700 1 -- 172.21.15.38:0/2650043851 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508419ba50 con 0x7f50841010f0 2024-05-21T15:25:37.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.170+0000 7f507b7fe700 1 -- 172.21.15.38:0/2650043851 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f508000eb20 con 0x7f50841010f0 2024-05-21T15:25:37.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.170+0000 7f508c54f700 1 -- 172.21.15.38:0/2650043851 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50841070f0 con 0x7f50841010f0 2024-05-21T15:25:37.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.170+0000 7f507b7fe700 1 --2- 172.21.15.38:0/2650043851 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5070039150 0x7f507003b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:37.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.170+0000 7f507b7fe700 1 -- 172.21.15.38:0/2650043851 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f508001e070 con 0x7f50841010f0 2024-05-21T15:25:37.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.170+0000 7f5089aea700 1 --2- 172.21.15.38:0/2650043851 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5070039150 0x7f507003b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:37.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.171+0000 7f5089aea700 1 --2- 172.21.15.38:0/2650043851 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5070039150 0x7f507003b660 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5074006fd0 tx=0x7f5074006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:37.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.175+0000 7f507b7fe700 1 -- 172.21.15.38:0/2650043851 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f508000ee10 con 0x7f50841010f0 2024-05-21T15:25:37.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.385+0000 7f508c54f700 1 -- 172.21.15.38:0/2650043851 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f508402dd70 con 0x7f5070039150 2024-05-21T15:25:37.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.391+0000 7f507b7fe700 1 -- 172.21.15.38:0/2650043851 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f508402dd70 con 0x7f5070039150 2024-05-21T15:25:37.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.393+0000 7f508c54f700 1 -- 172.21.15.38:0/2650043851 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5070039150 msgr2=0x7f507003b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:37.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.393+0000 7f508c54f700 1 --2- 172.21.15.38:0/2650043851 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5070039150 0x7f507003b660 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5074006fd0 tx=0x7f5074006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:37.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.393+0000 7f508c54f700 1 -- 172.21.15.38:0/2650043851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50841010f0 msgr2=0x7f5084198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:37.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.393+0000 7f508c54f700 1 --2- 172.21.15.38:0/2650043851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50841010f0 0x7f5084198670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5080000c00 tx=0x7f5080005ea0 comp rx=0 tx=0).stop 2024-05-21T15:25:37.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.393+0000 7f508c54f700 1 -- 172.21.15.38:0/2650043851 shutdown_connections 2024-05-21T15:25:37.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.393+0000 7f508c54f700 1 --2- 172.21.15.38:0/2650043851 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5070039150 0x7f507003b660 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:37.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.393+0000 7f508c54f700 1 --2- 172.21.15.38:0/2650043851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50841010f0 0x7f5084198670 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:37.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.393+0000 7f508c54f700 1 -- 172.21.15.38:0/2650043851 >> 172.21.15.38:0/2650043851 conn(0x7f50840fc730 msgr2=0x7f50840fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:37.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.394+0000 7f508c54f700 1 -- 172.21.15.38:0/2650043851 shutdown_connections 2024-05-21T15:25:37.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:37.394+0000 7f508c54f700 1 -- 172.21.15.38:0/2650043851 wait complete. 2024-05-21T15:25:38.094 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi045 2024-05-21T15:25:38.094 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T15:25:38.094 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/etc/ceph/ceph.conf 2024-05-21T15:25:38.114 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T15:25:38.114 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:25:38.173 INFO:tasks.cephadm:Adding host smithi045 to orchestrator... 2024-05-21T15:25:38.173 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch host add smithi045 2024-05-21T15:25:38.423 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:25:38.669 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:38 smithi038 ceph-mon[31995]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:25:38.670 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:38 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:38.670 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:38 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:25:38.670 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:38 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:25:38.670 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:38 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:25:38.670 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:38 smithi038 ceph-mon[31995]: Updating smithi038:/etc/ceph/ceph.conf 2024-05-21T15:25:38.670 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:38 smithi038 ceph-mon[31995]: Updating smithi038:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:25:39.581 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:39 smithi038 ceph-mon[31995]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:25:39.581 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:39 smithi038 ceph-mon[31995]: Updating smithi038:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:25:39.581 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:39 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:39.581 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:39 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:39.581 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:39 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:39.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.783+0000 7fe1f204d700 1 -- 172.21.15.38:0/1296773603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ec107910 msgr2=0x7fe1ec107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:39.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.783+0000 7fe1f204d700 1 --2- 172.21.15.38:0/1296773603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ec107910 0x7fe1ec107d20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009910 tx=0x7fe1dc009c20 comp rx=0 tx=0).stop 2024-05-21T15:25:39.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.783+0000 7fe1f204d700 1 -- 172.21.15.38:0/1296773603 shutdown_connections 2024-05-21T15:25:39.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.783+0000 7fe1f204d700 1 --2- 172.21.15.38:0/1296773603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ec107910 0x7fe1ec107d20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:39.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.783+0000 7fe1f204d700 1 -- 172.21.15.38:0/1296773603 >> 172.21.15.38:0/1296773603 conn(0x7fe1ec077430 msgr2=0x7fe1ec077890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:39.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.783+0000 7fe1f204d700 1 -- 172.21.15.38:0/1296773603 shutdown_connections 2024-05-21T15:25:39.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.783+0000 7fe1f204d700 1 -- 172.21.15.38:0/1296773603 wait complete. 2024-05-21T15:25:39.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.784+0000 7fe1f204d700 1 Processor -- start 2024-05-21T15:25:39.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.784+0000 7fe1f204d700 1 -- start start 2024-05-21T15:25:39.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.784+0000 7fe1f204d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ec107910 0x7fe1ec1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:39.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.784+0000 7fe1f204d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec10bfb0 con 0x7fe1ec107910 2024-05-21T15:25:39.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.785+0000 7fe1eb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ec107910 0x7fe1ec1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:39.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.785+0000 7fe1eb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ec107910 0x7fe1ec1977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53848/0 (socket says 172.21.15.38:53848) 2024-05-21T15:25:39.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.785+0000 7fe1eb7fe700 1 -- 172.21.15.38:0/1711910749 learned_addr learned my addr 172.21.15.38:0/1711910749 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:39.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.785+0000 7fe1eb7fe700 1 -- 172.21.15.38:0/1711910749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec197ce0 con 0x7fe1ec107910 2024-05-21T15:25:39.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.785+0000 7fe1eb7fe700 1 --2- 172.21.15.38:0/1711910749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ec107910 0x7fe1ec1977a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc004f80 tx=0x7fe1dc005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:39.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.785+0000 7fe1e8ff9700 1 -- 172.21.15.38:0/1711910749 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1dc026070 con 0x7fe1ec107910 2024-05-21T15:25:39.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.785+0000 7fe1e8ff9700 1 -- 172.21.15.38:0/1711910749 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1dc00b6f0 con 0x7fe1ec107910 2024-05-21T15:25:39.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.785+0000 7fe1f204d700 1 -- 172.21.15.38:0/1711910749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ec197fd0 con 0x7fe1ec107910 2024-05-21T15:25:39.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.785+0000 7fe1e8ff9700 1 -- 172.21.15.38:0/1711910749 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1dc003910 con 0x7fe1ec107910 2024-05-21T15:25:39.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.785+0000 7fe1f204d700 1 -- 172.21.15.38:0/1711910749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ec19bfd0 con 0x7fe1ec107910 2024-05-21T15:25:39.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.786+0000 7fe1e8ff9700 1 -- 172.21.15.38:0/1711910749 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe1dc029070 con 0x7fe1ec107910 2024-05-21T15:25:39.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.786+0000 7fe1f204d700 1 -- 172.21.15.38:0/1711910749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ec103020 con 0x7fe1ec107910 2024-05-21T15:25:39.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.787+0000 7fe1e8ff9700 1 --2- 172.21.15.38:0/1711910749 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1d40390f0 0x7fe1d403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:39.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.787+0000 7fe1e8ff9700 1 -- 172.21.15.38:0/1711910749 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe1dc01e070 con 0x7fe1ec107910 2024-05-21T15:25:39.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.787+0000 7fe1eaffd700 1 --2- 172.21.15.38:0/1711910749 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1d40390f0 0x7fe1d403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:39.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.787+0000 7fe1eaffd700 1 --2- 172.21.15.38:0/1711910749 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1d40390f0 0x7fe1d403b600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0006fd0 tx=0x7fe1e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:39.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:39.792+0000 7fe1e8ff9700 1 -- 172.21.15.38:0/1711910749 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe1dc013070 con 0x7fe1ec107910 2024-05-21T15:25:40.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:40.002+0000 7fe1f204d700 1 -- 172.21.15.38:0/1711910749 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi045", "target": ["mon-mgr", ""]}) -- 0x7fe1ec198b00 con 0x7fe1d40390f0 2024-05-21T15:25:41.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:40 smithi038 ceph-mon[31995]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:25:41.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:40 smithi038 ceph-mon[31995]: Deploying cephadm binary to smithi045 2024-05-21T15:25:43.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:43 smithi038 ceph-mon[31995]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:25:44.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:44.305+0000 7fe1e8ff9700 1 -- 172.21.15.38:0/1711910749 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fe1ec198b00 con 0x7fe1d40390f0 2024-05-21T15:25:44.309 INFO:teuthology.orchestra.run.smithi038.stdout:Added host 'smithi045' with addr '172.21.15.45' 2024-05-21T15:25:44.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:44.307+0000 7fe1f204d700 1 -- 172.21.15.38:0/1711910749 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1d40390f0 msgr2=0x7fe1d403b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:44.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:44.307+0000 7fe1f204d700 1 --2- 172.21.15.38:0/1711910749 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1d40390f0 0x7fe1d403b600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0006fd0 tx=0x7fe1e0006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:44.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:44.308+0000 7fe1f204d700 1 -- 172.21.15.38:0/1711910749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ec107910 msgr2=0x7fe1ec1977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:44.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:44.308+0000 7fe1f204d700 1 --2- 172.21.15.38:0/1711910749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ec107910 0x7fe1ec1977a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc004f80 tx=0x7fe1dc005060 comp rx=0 tx=0).stop 2024-05-21T15:25:44.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:44.308+0000 7fe1f204d700 1 -- 172.21.15.38:0/1711910749 shutdown_connections 2024-05-21T15:25:44.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:44.308+0000 7fe1f204d700 1 --2- 172.21.15.38:0/1711910749 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1d40390f0 0x7fe1d403b600 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:44.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:44.308+0000 7fe1f204d700 1 --2- 172.21.15.38:0/1711910749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1ec107910 0x7fe1ec1977a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:44.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:44.308+0000 7fe1f204d700 1 -- 172.21.15.38:0/1711910749 >> 172.21.15.38:0/1711910749 conn(0x7fe1ec077430 msgr2=0x7fe1ec100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:44.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:44.308+0000 7fe1f204d700 1 -- 172.21.15.38:0/1711910749 shutdown_connections 2024-05-21T15:25:44.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:44.308+0000 7fe1f204d700 1 -- 172.21.15.38:0/1711910749 wait complete. 2024-05-21T15:25:44.946 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-21T15:25:45.198 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:25:45.546 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:45 smithi038 ceph-mon[31995]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:25:45.546 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:45.546 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:45 smithi038 ceph-mon[31995]: Added host smithi045 2024-05-21T15:25:45.546 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:25:45.546 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:46.509 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:46 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:46.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.750+0000 7f44b50bb700 1 -- 172.21.15.38:0/2466324610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44b01089e0 msgr2=0x7f44b0108df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:46.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.750+0000 7f44b50bb700 1 --2- 172.21.15.38:0/2466324610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44b01089e0 0x7f44b0108df0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f44a000b0a0 tx=0x7f44a000b3b0 comp rx=0 tx=0).stop 2024-05-21T15:25:46.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.750+0000 7f44b50bb700 1 -- 172.21.15.38:0/2466324610 shutdown_connections 2024-05-21T15:25:46.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.750+0000 7f44b50bb700 1 --2- 172.21.15.38:0/2466324610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44b01089e0 0x7f44b0108df0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:46.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.750+0000 7f44b50bb700 1 -- 172.21.15.38:0/2466324610 >> 172.21.15.38:0/2466324610 conn(0x7f44b00fe4f0 msgr2=0x7f44b0100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:46.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.750+0000 7f44b50bb700 1 -- 172.21.15.38:0/2466324610 shutdown_connections 2024-05-21T15:25:46.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.750+0000 7f44b50bb700 1 -- 172.21.15.38:0/2466324610 wait complete. 2024-05-21T15:25:46.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.751+0000 7f44b50bb700 1 Processor -- start 2024-05-21T15:25:46.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.751+0000 7f44b50bb700 1 -- start start 2024-05-21T15:25:46.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.752+0000 7f44b50bb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44b01089e0 0x7f44b019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:46.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.752+0000 7f44b50bb700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f44b010d080 con 0x7f44b01089e0 2024-05-21T15:25:46.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.752+0000 7f44aed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44b01089e0 0x7f44b019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:46.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.752+0000 7f44aed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44b01089e0 0x7f44b019e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58728/0 (socket says 172.21.15.38:58728) 2024-05-21T15:25:46.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.752+0000 7f44aed9d700 1 -- 172.21.15.38:0/2239734690 learned_addr learned my addr 172.21.15.38:0/2239734690 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:46.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.752+0000 7f44aed9d700 1 -- 172.21.15.38:0/2239734690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b019ea30 con 0x7f44b01089e0 2024-05-21T15:25:46.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.752+0000 7f44aed9d700 1 --2- 172.21.15.38:0/2239734690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44b01089e0 0x7f44b019e4f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f44a00062a0 tx=0x7f44a0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:46.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.752+0000 7f4497fff700 1 -- 172.21.15.38:0/2239734690 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44a0017040 con 0x7f44b01089e0 2024-05-21T15:25:46.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.752+0000 7f4497fff700 1 -- 172.21.15.38:0/2239734690 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a0022070 con 0x7f44b01089e0 2024-05-21T15:25:46.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.752+0000 7f44b50bb700 1 -- 172.21.15.38:0/2239734690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b019ecc0 con 0x7f44b01089e0 2024-05-21T15:25:46.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.753+0000 7f4497fff700 1 -- 172.21.15.38:0/2239734690 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44a001d440 con 0x7f44b01089e0 2024-05-21T15:25:46.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.753+0000 7f44b50bb700 1 -- 172.21.15.38:0/2239734690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b0198680 con 0x7f44b01089e0 2024-05-21T15:25:46.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.753+0000 7f4497fff700 1 -- 172.21.15.38:0/2239734690 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f44a001d740 con 0x7f44b01089e0 2024-05-21T15:25:46.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.753+0000 7f44b50bb700 1 -- 172.21.15.38:0/2239734690 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b01040f0 con 0x7f44b01089e0 2024-05-21T15:25:46.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.754+0000 7f4497fff700 1 --2- 172.21.15.38:0/2239734690 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44980390f0 0x7f449803b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:46.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.754+0000 7f4497fff700 1 -- 172.21.15.38:0/2239734690 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f44a000f050 con 0x7f44b01089e0 2024-05-21T15:25:46.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.754+0000 7f44ae59c700 1 --2- 172.21.15.38:0/2239734690 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44980390f0 0x7f449803b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:46.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.754+0000 7f44ae59c700 1 --2- 172.21.15.38:0/2239734690 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44980390f0 0x7f449803b600 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f44a4006fd0 tx=0x7f44a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:46.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.759+0000 7f4497fff700 1 -- 172.21.15.38:0/2239734690 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f44a0007e20 con 0x7f44b01089e0 2024-05-21T15:25:46.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.973+0000 7f44b50bb700 1 -- 172.21.15.38:0/2239734690 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44b0199b90 con 0x7f44980390f0 2024-05-21T15:25:46.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.975+0000 7f4497fff700 1 -- 172.21.15.38:0/2239734690 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f44b0199b90 con 0x7f44980390f0 2024-05-21T15:25:46.979 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:25:46.979 INFO:teuthology.orchestra.run.smithi038.stdout:[{"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}, {"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}] 2024-05-21T15:25:46.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.977+0000 7f44b50bb700 1 -- 172.21.15.38:0/2239734690 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44980390f0 msgr2=0x7f449803b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:46.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.977+0000 7f44b50bb700 1 --2- 172.21.15.38:0/2239734690 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44980390f0 0x7f449803b600 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f44a4006fd0 tx=0x7f44a4006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:46.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.977+0000 7f44b50bb700 1 -- 172.21.15.38:0/2239734690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44b01089e0 msgr2=0x7f44b019e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:46.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.977+0000 7f44b50bb700 1 --2- 172.21.15.38:0/2239734690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44b01089e0 0x7f44b019e4f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f44a00062a0 tx=0x7f44a0003ce0 comp rx=0 tx=0).stop 2024-05-21T15:25:46.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.977+0000 7f44b50bb700 1 -- 172.21.15.38:0/2239734690 shutdown_connections 2024-05-21T15:25:46.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.977+0000 7f44b50bb700 1 --2- 172.21.15.38:0/2239734690 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44980390f0 0x7f449803b600 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:46.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.977+0000 7f44b50bb700 1 --2- 172.21.15.38:0/2239734690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44b01089e0 0x7f44b019e4f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:46.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.977+0000 7f44b50bb700 1 -- 172.21.15.38:0/2239734690 >> 172.21.15.38:0/2239734690 conn(0x7f44b00fe4f0 msgr2=0x7f44b00ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:46.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.977+0000 7f44b50bb700 1 -- 172.21.15.38:0/2239734690 shutdown_connections 2024-05-21T15:25:46.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:46.977+0000 7f44b50bb700 1 -- 172.21.15.38:0/2239734690 wait complete. 2024-05-21T15:25:47.580 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi053 2024-05-21T15:25:47.580 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-21T15:25:47.580 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.conf 2024-05-21T15:25:47.600 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-21T15:25:47.600 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:25:47.658 INFO:tasks.cephadm:Adding host smithi053 to orchestrator... 2024-05-21T15:25:47.659 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch host add smithi053 2024-05-21T15:25:47.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:47 smithi038 ceph-mon[31995]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:25:47.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:47 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:47.906 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:25:48.732 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:48 smithi038 ceph-mon[31995]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:25:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.210+0000 7ff76517a700 1 -- 172.21.15.38:0/3655216687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff760107940 msgr2=0x7ff760107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.210+0000 7ff76517a700 1 --2- 172.21.15.38:0/3655216687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff760107940 0x7ff760107d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff750009910 tx=0x7ff750009c20 comp rx=0 tx=0).stop 2024-05-21T15:25:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.210+0000 7ff76517a700 1 -- 172.21.15.38:0/3655216687 shutdown_connections 2024-05-21T15:25:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.210+0000 7ff76517a700 1 --2- 172.21.15.38:0/3655216687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff760107940 0x7ff760107d50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.210+0000 7ff76517a700 1 -- 172.21.15.38:0/3655216687 >> 172.21.15.38:0/3655216687 conn(0x7ff7600fd430 msgr2=0x7ff7600ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.210+0000 7ff76517a700 1 -- 172.21.15.38:0/3655216687 shutdown_connections 2024-05-21T15:25:49.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.210+0000 7ff76517a700 1 -- 172.21.15.38:0/3655216687 wait complete. 2024-05-21T15:25:49.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.211+0000 7ff76517a700 1 Processor -- start 2024-05-21T15:25:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.211+0000 7ff76517a700 1 -- start start 2024-05-21T15:25:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.211+0000 7ff76517a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff760107940 0x7ff7600fe350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.211+0000 7ff76517a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff76010bfe0 con 0x7ff760107940 2024-05-21T15:25:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.211+0000 7ff75ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff760107940 0x7ff7600fe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.211+0000 7ff75ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff760107940 0x7ff7600fe350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58756/0 (socket says 172.21.15.38:58756) 2024-05-21T15:25:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.211+0000 7ff75ed9d700 1 -- 172.21.15.38:0/395265135 learned_addr learned my addr 172.21.15.38:0/395265135 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.212+0000 7ff75ed9d700 1 -- 172.21.15.38:0/395265135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff760100150 con 0x7ff760107940 2024-05-21T15:25:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.212+0000 7ff75ed9d700 1 --2- 172.21.15.38:0/395265135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff760107940 0x7ff7600fe350 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff750004f80 tx=0x7ff750005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.212+0000 7ff747fff700 1 -- 172.21.15.38:0/395265135 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff750026070 con 0x7ff760107940 2024-05-21T15:25:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.212+0000 7ff76517a700 1 -- 172.21.15.38:0/395265135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7600fe890 con 0x7ff760107940 2024-05-21T15:25:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.212+0000 7ff747fff700 1 -- 172.21.15.38:0/395265135 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff75000b6f0 con 0x7ff760107940 2024-05-21T15:25:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.212+0000 7ff747fff700 1 -- 172.21.15.38:0/395265135 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff750003910 con 0x7ff760107940 2024-05-21T15:25:49.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.212+0000 7ff76517a700 1 -- 172.21.15.38:0/395265135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7600fed50 con 0x7ff760107940 2024-05-21T15:25:49.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.213+0000 7ff76517a700 1 -- 172.21.15.38:0/395265135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff760103050 con 0x7ff760107940 2024-05-21T15:25:49.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.213+0000 7ff747fff700 1 -- 172.21.15.38:0/395265135 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff750029070 con 0x7ff760107940 2024-05-21T15:25:49.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.213+0000 7ff747fff700 1 --2- 172.21.15.38:0/395265135 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff7480390f0 0x7ff74803b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:49.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.214+0000 7ff747fff700 1 -- 172.21.15.38:0/395265135 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff75001e070 con 0x7ff760107940 2024-05-21T15:25:49.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.214+0000 7ff75e59c700 1 --2- 172.21.15.38:0/395265135 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff7480390f0 0x7ff74803b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:49.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.214+0000 7ff75e59c700 1 --2- 172.21.15.38:0/395265135 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff7480390f0 0x7ff74803b600 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff754006fd0 tx=0x7ff754006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:49.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.219+0000 7ff747fff700 1 -- 172.21.15.38:0/395265135 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff750013070 con 0x7ff760107940 2024-05-21T15:25:49.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:49.429+0000 7ff76517a700 1 -- 172.21.15.38:0/395265135 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}) -- 0x7ff7600ff830 con 0x7ff7480390f0 2024-05-21T15:25:49.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:49 smithi038 ceph-mon[31995]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:25:50.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:50 smithi038 ceph-mon[31995]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:25:50.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:50 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:50.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:50 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:50.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:50 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:50.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:50 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:50.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:50 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:25:50.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:50 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:25:50.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:50 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:25:51.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:51 smithi038 ceph-mon[31995]: Deploying cephadm binary to smithi053 2024-05-21T15:25:51.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:51 smithi038 ceph-mon[31995]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:25:51.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:51 smithi038 ceph-mon[31995]: Updating smithi045:/etc/ceph/ceph.conf 2024-05-21T15:25:51.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:51 smithi038 ceph-mon[31995]: Updating smithi045:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:25:53.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:53.742+0000 7ff747fff700 1 -- 172.21.15.38:0/395265135 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7ff7600ff830 con 0x7ff7480390f0 2024-05-21T15:25:53.746 INFO:teuthology.orchestra.run.smithi038.stdout:Added host 'smithi053' with addr '172.21.15.53' 2024-05-21T15:25:53.746 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:53 smithi038 ceph-mon[31995]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:25:53.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:53.744+0000 7ff76517a700 1 -- 172.21.15.38:0/395265135 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff7480390f0 msgr2=0x7ff74803b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:53.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:53.744+0000 7ff76517a700 1 --2- 172.21.15.38:0/395265135 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff7480390f0 0x7ff74803b600 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff754006fd0 tx=0x7ff754006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:53.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:53.744+0000 7ff76517a700 1 -- 172.21.15.38:0/395265135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff760107940 msgr2=0x7ff7600fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:53.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:53.744+0000 7ff76517a700 1 --2- 172.21.15.38:0/395265135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff760107940 0x7ff7600fe350 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff750004f80 tx=0x7ff750005060 comp rx=0 tx=0).stop 2024-05-21T15:25:53.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:53.744+0000 7ff76517a700 1 -- 172.21.15.38:0/395265135 shutdown_connections 2024-05-21T15:25:53.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:53.744+0000 7ff76517a700 1 --2- 172.21.15.38:0/395265135 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff7480390f0 0x7ff74803b600 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:53.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:53.744+0000 7ff76517a700 1 --2- 172.21.15.38:0/395265135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff760107940 0x7ff7600fe350 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:53.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:53.744+0000 7ff76517a700 1 -- 172.21.15.38:0/395265135 >> 172.21.15.38:0/395265135 conn(0x7ff7600fd430 msgr2=0x7ff760105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:53.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:53.744+0000 7ff76517a700 1 -- 172.21.15.38:0/395265135 shutdown_connections 2024-05-21T15:25:53.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:53.745+0000 7ff76517a700 1 -- 172.21.15.38:0/395265135 wait complete. 2024-05-21T15:25:54.362 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-21T15:25:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:25:54.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:54 smithi038 ceph-mon[31995]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:25:54.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:54 smithi038 ceph-mon[31995]: Updating smithi045:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:25:54.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:54.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:54.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:54.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:54.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:54 smithi038 ceph-mon[31995]: Added host smithi053 2024-05-21T15:25:54.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:25:54.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.089+0000 7f402a587700 1 -- 172.21.15.38:0/414865446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40241078f0 msgr2=0x7f4024107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.089+0000 7f402a587700 1 --2- 172.21.15.38:0/414865446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40241078f0 0x7f4024107d00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4014009910 tx=0x7f4014009c20 comp rx=0 tx=0).stop 2024-05-21T15:25:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.089+0000 7f402a587700 1 -- 172.21.15.38:0/414865446 shutdown_connections 2024-05-21T15:25:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.089+0000 7f402a587700 1 --2- 172.21.15.38:0/414865446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40241078f0 0x7f4024107d00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.089+0000 7f402a587700 1 -- 172.21.15.38:0/414865446 >> 172.21.15.38:0/414865446 conn(0x7f40240fd440 msgr2=0x7f40240ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.089+0000 7f402a587700 1 -- 172.21.15.38:0/414865446 shutdown_connections 2024-05-21T15:25:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.089+0000 7f402a587700 1 -- 172.21.15.38:0/414865446 wait complete. 2024-05-21T15:25:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.090+0000 7f402a587700 1 Processor -- start 2024-05-21T15:25:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.090+0000 7f402a587700 1 -- start start 2024-05-21T15:25:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.090+0000 7f402a587700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40241078f0 0x7f40241996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.090+0000 7f402a587700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f402410bf90 con 0x7f40241078f0 2024-05-21T15:25:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.090+0000 7f4023fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40241078f0 0x7f40241996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.090+0000 7f4023fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40241078f0 0x7f40241996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48082/0 (socket says 172.21.15.38:48082) 2024-05-21T15:25:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.090+0000 7f4023fff700 1 -- 172.21.15.38:0/3180864624 learned_addr learned my addr 172.21.15.38:0/3180864624 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.091+0000 7f4023fff700 1 -- 172.21.15.38:0/3180864624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402406b8f0 con 0x7f40241078f0 2024-05-21T15:25:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.091+0000 7f4023fff700 1 --2- 172.21.15.38:0/3180864624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40241078f0 0x7f40241996e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f40140092e0 tx=0x7f4014005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.091+0000 7f40217fa700 1 -- 172.21.15.38:0/3180864624 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4014026070 con 0x7f40241078f0 2024-05-21T15:25:56.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.091+0000 7f40217fa700 1 -- 172.21.15.38:0/3180864624 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f401400b750 con 0x7f40241078f0 2024-05-21T15:25:56.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.091+0000 7f402a587700 1 -- 172.21.15.38:0/3180864624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4024199ce0 con 0x7f40241078f0 2024-05-21T15:25:56.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.091+0000 7f40217fa700 1 -- 172.21.15.38:0/3180864624 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4014003970 con 0x7f40241078f0 2024-05-21T15:25:56.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.091+0000 7f402a587700 1 -- 172.21.15.38:0/3180864624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402419a1a0 con 0x7f40241078f0 2024-05-21T15:25:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.092+0000 7f402a587700 1 -- 172.21.15.38:0/3180864624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4024103000 con 0x7f40241078f0 2024-05-21T15:25:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.092+0000 7f40217fa700 1 -- 172.21.15.38:0/3180864624 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4014029070 con 0x7f40241078f0 2024-05-21T15:25:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.092+0000 7f40217fa700 1 --2- 172.21.15.38:0/3180864624 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f400c0390f0 0x7f400c03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.092+0000 7f40217fa700 1 -- 172.21.15.38:0/3180864624 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f401401e070 con 0x7f40241078f0 2024-05-21T15:25:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.093+0000 7f40237fe700 1 --2- 172.21.15.38:0/3180864624 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f400c0390f0 0x7f400c03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.093+0000 7f40237fe700 1 --2- 172.21.15.38:0/3180864624 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f400c0390f0 0x7f400c03b600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4018006fd0 tx=0x7f4018006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:56.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.098+0000 7f40217fa700 1 -- 172.21.15.38:0/3180864624 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4014013070 con 0x7f40241078f0 2024-05-21T15:25:56.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:55 smithi038 ceph-mon[31995]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:25:56.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:55 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:56.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.311+0000 7f402a587700 1 -- 172.21.15.38:0/3180864624 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f402406b040 con 0x7f400c0390f0 2024-05-21T15:25:56.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.312+0000 7f40217fa700 1 -- 172.21.15.38:0/3180864624 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f402406b040 con 0x7f400c0390f0 2024-05-21T15:25:56.316 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:25:56.316 INFO:teuthology.orchestra.run.smithi038.stdout:[{"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}, {"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}] 2024-05-21T15:25:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.314+0000 7f402a587700 1 -- 172.21.15.38:0/3180864624 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f400c0390f0 msgr2=0x7f400c03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.314+0000 7f402a587700 1 --2- 172.21.15.38:0/3180864624 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f400c0390f0 0x7f400c03b600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4018006fd0 tx=0x7f4018006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.315+0000 7f402a587700 1 -- 172.21.15.38:0/3180864624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40241078f0 msgr2=0x7f40241996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.315+0000 7f402a587700 1 --2- 172.21.15.38:0/3180864624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40241078f0 0x7f40241996e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f40140092e0 tx=0x7f4014005030 comp rx=0 tx=0).stop 2024-05-21T15:25:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.315+0000 7f402a587700 1 -- 172.21.15.38:0/3180864624 shutdown_connections 2024-05-21T15:25:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.315+0000 7f402a587700 1 --2- 172.21.15.38:0/3180864624 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f400c0390f0 0x7f400c03b600 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.315+0000 7f402a587700 1 --2- 172.21.15.38:0/3180864624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40241078f0 0x7f40241996e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.315+0000 7f402a587700 1 -- 172.21.15.38:0/3180864624 >> 172.21.15.38:0/3180864624 conn(0x7f40240fd440 msgr2=0x7f40240fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.315+0000 7f402a587700 1 -- 172.21.15.38:0/3180864624 shutdown_connections 2024-05-21T15:25:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:56.315+0000 7f402a587700 1 -- 172.21.15.38:0/3180864624 wait complete. 2024-05-21T15:25:56.965 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-21T15:25:56.965 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd crush tunables default 2024-05-21T15:25:57.209 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:25:57.565 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:57 smithi038 ceph-mon[31995]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:25:57.565 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:57 smithi038 ceph-mon[31995]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:25:57.566 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:25:58.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.650+0000 7fc65bf90700 1 -- 172.21.15.38:0/2001241391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc654107940 msgr2=0x7fc654107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:58.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.650+0000 7fc65bf90700 1 --2- 172.21.15.38:0/2001241391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc654107940 0x7fc654107d50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc648009910 tx=0x7fc648009c20 comp rx=0 tx=0).stop 2024-05-21T15:25:58.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.650+0000 7fc65bf90700 1 -- 172.21.15.38:0/2001241391 shutdown_connections 2024-05-21T15:25:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.650+0000 7fc65bf90700 1 --2- 172.21.15.38:0/2001241391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc654107940 0x7fc654107d50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.650+0000 7fc65bf90700 1 -- 172.21.15.38:0/2001241391 >> 172.21.15.38:0/2001241391 conn(0x7fc6540fd430 msgr2=0x7fc6540ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.650+0000 7fc65bf90700 1 -- 172.21.15.38:0/2001241391 shutdown_connections 2024-05-21T15:25:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.650+0000 7fc65bf90700 1 -- 172.21.15.38:0/2001241391 wait complete. 2024-05-21T15:25:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.650+0000 7fc65bf90700 1 Processor -- start 2024-05-21T15:25:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.651+0000 7fc65bf90700 1 -- start start 2024-05-21T15:25:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.651+0000 7fc65bf90700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc654107940 0x7fc65419e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.651+0000 7fc65bf90700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc65410bfe0 con 0x7fc654107940 2024-05-21T15:25:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.651+0000 7fc659d2c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc654107940 0x7fc65419e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.651+0000 7fc659d2c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc654107940 0x7fc65419e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48102/0 (socket says 172.21.15.38:48102) 2024-05-21T15:25:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.651+0000 7fc659d2c700 1 -- 172.21.15.38:0/456173409 learned_addr learned my addr 172.21.15.38:0/456173409 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:25:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.651+0000 7fc659d2c700 1 -- 172.21.15.38:0/456173409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65419eaa0 con 0x7fc654107940 2024-05-21T15:25:58.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.651+0000 7fc659d2c700 1 --2- 172.21.15.38:0/456173409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc654107940 0x7fc65419e560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc648004f80 tx=0x7fc648005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.652+0000 7fc646ffd700 1 -- 172.21.15.38:0/456173409 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc648026070 con 0x7fc654107940 2024-05-21T15:25:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.652+0000 7fc646ffd700 1 -- 172.21.15.38:0/456173409 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc64800b6f0 con 0x7fc654107940 2024-05-21T15:25:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.652+0000 7fc65bf90700 1 -- 172.21.15.38:0/456173409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65419ed90 con 0x7fc654107940 2024-05-21T15:25:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.652+0000 7fc646ffd700 1 -- 172.21.15.38:0/456173409 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc648003910 con 0x7fc654107940 2024-05-21T15:25:58.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.652+0000 7fc65bf90700 1 -- 172.21.15.38:0/456173409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6541988c0 con 0x7fc654107940 2024-05-21T15:25:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.653+0000 7fc646ffd700 1 -- 172.21.15.38:0/456173409 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc648029070 con 0x7fc654107940 2024-05-21T15:25:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.653+0000 7fc646ffd700 1 --2- 172.21.15.38:0/456173409 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc640039150 0x7fc64003b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.653+0000 7fc65bf90700 1 -- 172.21.15.38:0/456173409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc654103050 con 0x7fc654107940 2024-05-21T15:25:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.653+0000 7fc646ffd700 1 -- 172.21.15.38:0/456173409 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc64801e070 con 0x7fc654107940 2024-05-21T15:25:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.653+0000 7fc65952b700 1 --2- 172.21.15.38:0/456173409 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc640039150 0x7fc64003b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:58.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.653+0000 7fc65952b700 1 --2- 172.21.15.38:0/456173409 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc640039150 0x7fc64003b660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc650006fd0 tx=0x7fc650006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:58.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.658+0000 7fc646ffd700 1 -- 172.21.15.38:0/456173409 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc648013070 con 0x7fc654107940 2024-05-21T15:25:58.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:58.872+0000 7fc65bf90700 1 -- 172.21.15.38:0/456173409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc65419efc0 con 0x7fc654107940 2024-05-21T15:25:59.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:59.471+0000 7fc646ffd700 1 -- 172.21.15.38:0/456173409 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fc648027d40 con 0x7fc654107940 2024-05-21T15:25:59.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:59.474+0000 7fc65bf90700 1 -- 172.21.15.38:0/456173409 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc640039150 msgr2=0x7fc64003b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:59.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:59.474+0000 7fc65bf90700 1 --2- 172.21.15.38:0/456173409 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc640039150 0x7fc64003b660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc650006fd0 tx=0x7fc650006e40 comp rx=0 tx=0).stop 2024-05-21T15:25:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:59.474+0000 7fc65bf90700 1 -- 172.21.15.38:0/456173409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc654107940 msgr2=0x7fc65419e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:59.474+0000 7fc65bf90700 1 --2- 172.21.15.38:0/456173409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc654107940 0x7fc65419e560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc648004f80 tx=0x7fc648005060 comp rx=0 tx=0).stop 2024-05-21T15:25:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:59.474+0000 7fc65bf90700 1 -- 172.21.15.38:0/456173409 shutdown_connections 2024-05-21T15:25:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:59.474+0000 7fc65bf90700 1 --2- 172.21.15.38:0/456173409 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc640039150 0x7fc64003b660 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:59.474+0000 7fc65bf90700 1 --2- 172.21.15.38:0/456173409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc654107940 0x7fc65419e560 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:59.474+0000 7fc65bf90700 1 -- 172.21.15.38:0/456173409 >> 172.21.15.38:0/456173409 conn(0x7fc6540fd430 msgr2=0x7fc654105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:59.474+0000 7fc65bf90700 1 -- 172.21.15.38:0/456173409 shutdown_connections 2024-05-21T15:25:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:25:59.474+0000 7fc65bf90700 1 -- 172.21.15.38:0/456173409 wait complete. 2024-05-21T15:25:59.478 INFO:teuthology.orchestra.run.smithi038.stderr:adjusted tunables profile to default 2024-05-21T15:25:59.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:59 smithi038 ceph-mon[31995]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:25:59.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:25:59 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/456173409' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-21T15:26:00.045 INFO:tasks.cephadm:Adding mon.a on smithi038 2024-05-21T15:26:00.045 INFO:tasks.cephadm:Adding mon.b on smithi045 2024-05-21T15:26:00.045 INFO:tasks.cephadm:Adding mon.c on smithi053 2024-05-21T15:26:00.046 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch apply mon '3;smithi038:172.21.15.38=a;smithi045:172.21.15.45=b;smithi053:172.21.15.53=c' 2024-05-21T15:26:00.282 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:26:00.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:00 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/456173409' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-21T15:26:00.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:00 smithi038 ceph-mon[31995]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:26:00.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:00.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:00.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:00.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:00.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:00.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:00.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:00.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:00 smithi038 ceph-mon[31995]: Updating smithi053:/etc/ceph/ceph.conf 2024-05-21T15:26:01.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.829+0000 7ff33d05c700 1 -- 172.21.15.53:0/1109357869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3381078f0 msgr2=0x7ff338107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:01.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.829+0000 7ff33d05c700 1 --2- 172.21.15.53:0/1109357869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3381078f0 0x7ff338107d00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff328009910 tx=0x7ff328009c20 comp rx=0 tx=0).stop 2024-05-21T15:26:01.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.829+0000 7ff33d05c700 1 -- 172.21.15.53:0/1109357869 shutdown_connections 2024-05-21T15:26:01.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.829+0000 7ff33d05c700 1 --2- 172.21.15.53:0/1109357869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3381078f0 0x7ff338107d00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:01.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.829+0000 7ff33d05c700 1 -- 172.21.15.53:0/1109357869 >> 172.21.15.53:0/1109357869 conn(0x7ff3380fd420 msgr2=0x7ff3380ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:01.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.829+0000 7ff33d05c700 1 -- 172.21.15.53:0/1109357869 shutdown_connections 2024-05-21T15:26:01.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.829+0000 7ff33d05c700 1 -- 172.21.15.53:0/1109357869 wait complete. 2024-05-21T15:26:01.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.830+0000 7ff33d05c700 1 Processor -- start 2024-05-21T15:26:01.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.830+0000 7ff33d05c700 1 -- start start 2024-05-21T15:26:01.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.830+0000 7ff33d05c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3381078f0 0x7ff338199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:01.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.830+0000 7ff33d05c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff33810bf90 con 0x7ff3381078f0 2024-05-21T15:26:01.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.831+0000 7ff336d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3381078f0 0x7ff338199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:01.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.831+0000 7ff336d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3381078f0 0x7ff338199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.53:46754/0 (socket says 172.21.15.53:46754) 2024-05-21T15:26:01.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.831+0000 7ff336d9d700 1 -- 172.21.15.53:0/1043582432 learned_addr learned my addr 172.21.15.53:0/1043582432 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-21T15:26:01.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.831+0000 7ff336d9d700 1 -- 172.21.15.53:0/1043582432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff338199bd0 con 0x7ff3381078f0 2024-05-21T15:26:01.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.831+0000 7ff336d9d700 1 --2- 172.21.15.53:0/1043582432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3381078f0 0x7ff338199690 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff328004f80 tx=0x7ff328005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:01.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.831+0000 7ff31ffff700 1 -- 172.21.15.53:0/1043582432 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff328026070 con 0x7ff3381078f0 2024-05-21T15:26:01.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.832+0000 7ff33d05c700 1 -- 172.21.15.53:0/1043582432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff338199ec0 con 0x7ff3381078f0 2024-05-21T15:26:01.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.832+0000 7ff33d05c700 1 -- 172.21.15.53:0/1043582432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33819a380 con 0x7ff3381078f0 2024-05-21T15:26:01.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.832+0000 7ff31ffff700 1 -- 172.21.15.53:0/1043582432 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff32800b6f0 con 0x7ff3381078f0 2024-05-21T15:26:01.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.832+0000 7ff31ffff700 1 -- 172.21.15.53:0/1043582432 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff328030060 con 0x7ff3381078f0 2024-05-21T15:26:01.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.832+0000 7ff33d05c700 1 -- 172.21.15.53:0/1043582432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff338103000 con 0x7ff3381078f0 2024-05-21T15:26:01.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.833+0000 7ff31ffff700 1 -- 172.21.15.53:0/1043582432 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff328029070 con 0x7ff3381078f0 2024-05-21T15:26:01.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.833+0000 7ff31ffff700 1 --2- 172.21.15.53:0/1043582432 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3200390f0 0x7ff32003b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:01.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.833+0000 7ff31ffff700 1 -- 172.21.15.53:0/1043582432 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff32801e070 con 0x7ff3381078f0 2024-05-21T15:26:01.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.833+0000 7ff33659c700 1 --2- 172.21.15.53:0/1043582432 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3200390f0 0x7ff32003b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:01.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.834+0000 7ff33659c700 1 --2- 172.21.15.53:0/1043582432 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3200390f0 0x7ff32003b600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff32c006fd0 tx=0x7ff32c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:01.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:01.838+0000 7ff31ffff700 1 -- 172.21.15.53:0/1043582432 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff328013070 con 0x7ff3381078f0 2024-05-21T15:26:02.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:01 smithi038 ceph-mon[31995]: Updating smithi053:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:26:02.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:01 smithi038 ceph-mon[31995]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:02.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:01 smithi038 ceph-mon[31995]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:26:02.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:01 smithi038 ceph-mon[31995]: Updating smithi053:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:26:02.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:01 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:02.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:01 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:02.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:01 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:02.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:02.058+0000 7ff33d05c700 1 -- 172.21.15.53:0/1043582432 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi038:172.21.15.38=a;smithi045:172.21.15.45=b;smithi053:172.21.15.53=c", "target": ["mon-mgr", ""]}) -- 0x7ff33806b050 con 0x7ff3200390f0 2024-05-21T15:26:02.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:02.068+0000 7ff31ffff700 1 -- 172.21.15.53:0/1043582432 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff33806b050 con 0x7ff3200390f0 2024-05-21T15:26:02.071 INFO:teuthology.orchestra.run.smithi053.stdout:Scheduled mon update... 2024-05-21T15:26:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:02.070+0000 7ff33d05c700 1 -- 172.21.15.53:0/1043582432 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3200390f0 msgr2=0x7ff32003b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:02.070+0000 7ff33d05c700 1 --2- 172.21.15.53:0/1043582432 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3200390f0 0x7ff32003b600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff32c006fd0 tx=0x7ff32c006e40 comp rx=0 tx=0).stop 2024-05-21T15:26:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:02.070+0000 7ff33d05c700 1 -- 172.21.15.53:0/1043582432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3381078f0 msgr2=0x7ff338199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:02.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:02.070+0000 7ff33d05c700 1 --2- 172.21.15.53:0/1043582432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3381078f0 0x7ff338199690 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff328004f80 tx=0x7ff328005060 comp rx=0 tx=0).stop 2024-05-21T15:26:02.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:02.070+0000 7ff33d05c700 1 -- 172.21.15.53:0/1043582432 shutdown_connections 2024-05-21T15:26:02.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:02.070+0000 7ff33d05c700 1 --2- 172.21.15.53:0/1043582432 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3200390f0 0x7ff32003b600 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:02.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:02.070+0000 7ff33d05c700 1 --2- 172.21.15.53:0/1043582432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3381078f0 0x7ff338199690 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:02.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:02.070+0000 7ff33d05c700 1 -- 172.21.15.53:0/1043582432 >> 172.21.15.53:0/1043582432 conn(0x7ff3380fd420 msgr2=0x7ff3380fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:02.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:02.070+0000 7ff33d05c700 1 -- 172.21.15.53:0/1043582432 shutdown_connections 2024-05-21T15:26:02.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:02.070+0000 7ff33d05c700 1 -- 172.21.15.53:0/1043582432 wait complete. 2024-05-21T15:26:02.661 DEBUG:teuthology.orchestra.run.smithi045:mon.b> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.b.service 2024-05-21T15:26:02.664 DEBUG:teuthology.orchestra.run.smithi053:mon.c> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.c.service 2024-05-21T15:26:02.667 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T15:26:02.667 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T15:26:03.337 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:26:03.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:03 smithi038 ceph-mon[31995]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi038:172.21.15.38=a;smithi045:172.21.15.45=b;smithi053:172.21.15.53=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:26:03.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:03 smithi038 ceph-mon[31995]: Saving service mon spec with placement smithi038:172.21.15.38=a;smithi045:172.21.15.45=b;smithi053:172.21.15.53=c;count:3 2024-05-21T15:26:03.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:03.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:03.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:03.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:03.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:03.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:26:03.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:03.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:03 smithi038 ceph-mon[31995]: Deploying daemon mon.c on smithi053 2024-05-21T15:26:03.529 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:03 smithi038 ceph-mon[31995]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:05.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:05 smithi038 ceph-mon[31995]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:06.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.695+0000 7fe43ff69700 1 -- 172.21.15.53:0/3619105878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe438103b60 msgr2=0x7fe438103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:06.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.695+0000 7fe43ff69700 1 --2- 172.21.15.53:0/3619105878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe438103b60 0x7fe438103f70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe434009910 tx=0x7fe434009c20 comp rx=0 tx=0).stop 2024-05-21T15:26:06.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.695+0000 7fe43ff69700 1 -- 172.21.15.53:0/3619105878 shutdown_connections 2024-05-21T15:26:06.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.695+0000 7fe43ff69700 1 --2- 172.21.15.53:0/3619105878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe438103b60 0x7fe438103f70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.701 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.695+0000 7fe43ff69700 1 -- 172.21.15.53:0/3619105878 >> 172.21.15.53:0/3619105878 conn(0x7fe4380fd3f0 msgr2=0x7fe4380ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:06.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.695+0000 7fe43ff69700 1 -- 172.21.15.53:0/3619105878 shutdown_connections 2024-05-21T15:26:06.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.695+0000 7fe43ff69700 1 -- 172.21.15.53:0/3619105878 wait complete. 2024-05-21T15:26:06.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.696+0000 7fe43ff69700 1 Processor -- start 2024-05-21T15:26:06.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.696+0000 7fe43ff69700 1 -- start start 2024-05-21T15:26:06.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.696+0000 7fe43ff69700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe438103b60 0x7fe438101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.696+0000 7fe43ff69700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe43810d0e0 con 0x7fe438103b60 2024-05-21T15:26:06.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.697+0000 7fe43dd05700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe438103b60 0x7fe438101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.697+0000 7fe43dd05700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe438103b60 0x7fe438101850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.53:57424/0 (socket says 172.21.15.53:57424) 2024-05-21T15:26:06.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.697+0000 7fe43dd05700 1 -- 172.21.15.53:0/1186721089 learned_addr learned my addr 172.21.15.53:0/1186721089 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-21T15:26:06.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.697+0000 7fe43dd05700 1 -- 172.21.15.53:0/1186721089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4381036a0 con 0x7fe438103b60 2024-05-21T15:26:06.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.697+0000 7fe43dd05700 1 --2- 172.21.15.53:0/1186721089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe438103b60 0x7fe438101850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe434006010 tx=0x7fe434005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:06.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.697+0000 7fe42effd700 1 -- 172.21.15.53:0/1186721089 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe434026070 con 0x7fe438103b60 2024-05-21T15:26:06.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.697+0000 7fe42effd700 1 -- 172.21.15.53:0/1186721089 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe43400bc30 con 0x7fe438103b60 2024-05-21T15:26:06.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.697+0000 7fe43ff69700 1 -- 172.21.15.53:0/1186721089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe438101d90 con 0x7fe438103b60 2024-05-21T15:26:06.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.697+0000 7fe42effd700 1 -- 172.21.15.53:0/1186721089 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe434003ab0 con 0x7fe438103b60 2024-05-21T15:26:06.703 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.697+0000 7fe43ff69700 1 -- 172.21.15.53:0/1186721089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe438102190 con 0x7fe438103b60 2024-05-21T15:26:06.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.698+0000 7fe43ff69700 1 -- 172.21.15.53:0/1186721089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe43806b220 con 0x7fe438103b60 2024-05-21T15:26:06.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.698+0000 7fe42effd700 1 -- 172.21.15.53:0/1186721089 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe434029070 con 0x7fe438103b60 2024-05-21T15:26:06.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.699+0000 7fe42effd700 1 --2- 172.21.15.53:0/1186721089 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe4240390f0 0x7fe42403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.699+0000 7fe42effd700 1 -- 172.21.15.53:0/1186721089 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe43401e070 con 0x7fe438103b60 2024-05-21T15:26:06.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.699+0000 7fe43d504700 1 --2- 172.21.15.53:0/1186721089 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe4240390f0 0x7fe42403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.704 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.699+0000 7fe43d504700 1 --2- 172.21.15.53:0/1186721089 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe4240390f0 0x7fe42403b600 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe428006fd0 tx=0x7fe428006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:06.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.704+0000 7fe42effd700 1 -- 172.21.15.53:0/1186721089 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe434013070 con 0x7fe438103b60 2024-05-21T15:26:07.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.999+0000 7fe43ff69700 1 -- 172.21.15.53:0/1186721089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe438102410 con 0x7fe438103b60 2024-05-21T15:26:07.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:06.999+0000 7fe42effd700 1 -- 172.21.15.53:0/1186721089 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fe434030090 con 0x7fe438103b60 2024-05-21T15:26:07.003 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:26:07.003 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","modified":"2024-05-21T15:24:17.361696Z","created":"2024-05-21T15:24:17.361696Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-21T15:26:07.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:07.002+0000 7fe43ff69700 1 -- 172.21.15.53:0/1186721089 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe4240390f0 msgr2=0x7fe42403b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:07.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:07.002+0000 7fe43ff69700 1 --2- 172.21.15.53:0/1186721089 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe4240390f0 0x7fe42403b600 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe428006fd0 tx=0x7fe428006e40 comp rx=0 tx=0).stop 2024-05-21T15:26:07.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:07.002+0000 7fe43ff69700 1 -- 172.21.15.53:0/1186721089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe438103b60 msgr2=0x7fe438101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:07.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:07.002+0000 7fe43ff69700 1 --2- 172.21.15.53:0/1186721089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe438103b60 0x7fe438101850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe434006010 tx=0x7fe434005050 comp rx=0 tx=0).stop 2024-05-21T15:26:07.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:07.002+0000 7fe43ff69700 1 -- 172.21.15.53:0/1186721089 shutdown_connections 2024-05-21T15:26:07.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:07.002+0000 7fe43ff69700 1 --2- 172.21.15.53:0/1186721089 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe4240390f0 0x7fe42403b600 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:07.002+0000 7fe43ff69700 1 --2- 172.21.15.53:0/1186721089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe438103b60 0x7fe438101850 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:07.002+0000 7fe43ff69700 1 -- 172.21.15.53:0/1186721089 >> 172.21.15.53:0/1186721089 conn(0x7fe4380fd3f0 msgr2=0x7fe438104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:07.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:07.002+0000 7fe43ff69700 1 -- 172.21.15.53:0/1186721089 shutdown_connections 2024-05-21T15:26:07.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:07.002+0000 7fe43ff69700 1 -- 172.21.15.53:0/1186721089 wait complete. 2024-05-21T15:26:07.006 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-05-21T15:26:07.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:07 smithi038 ceph-mon[31995]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:07.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:07 smithi038 ceph-mon[31995]: from='client.? 172.21.15.53:0/1186721089' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T15:26:08.952 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T15:26:08.953 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T15:26:09.207 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:26:12.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:26:12.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:26:12.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: mon.a calling monitor election 2024-05-21T15:26:12.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:12.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:26:12.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:26:12.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: mon.c calling monitor election 2024-05-21T15:26:12.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:12.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:26:12.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:26:12.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:26:12.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T15:26:12.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: monmap epoch 2 2024-05-21T15:26:12.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:26:12.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: last_changed 2024-05-21T15:26:07.391245+0000 2024-05-21T15:26:12.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: created 2024-05-21T15:24:17.361696+0000 2024-05-21T15:26:12.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: min_mon_release 19 (squid) 2024-05-21T15:26:12.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: election_strategy: 1 2024-05-21T15:26:12.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-05-21T15:26:12.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: 1: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.c 2024-05-21T15:26:12.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: fsmap 2024-05-21T15:26:12.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:26:12.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: mgrmap e13: x(active, since 50s) 2024-05-21T15:26:12.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: overall HEALTH_OK 2024-05-21T15:26:12.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:12.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:12.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:12.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:26:12.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:12 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:12.883 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:12 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:13.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.572+0000 7fb4feba6700 1 -- 172.21.15.53:0/211936666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80035c0 msgr2=0x7fb4e8005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:13.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.572+0000 7fb4feba6700 1 --2- 172.21.15.53:0/211936666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80035c0 0x7fb4e8005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb4f806abc0 tx=0x7fb4ec00feb0 comp rx=0 tx=0).stop 2024-05-21T15:26:13.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.572+0000 7fb4feba6700 1 -- 172.21.15.53:0/211936666 shutdown_connections 2024-05-21T15:26:13.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.572+0000 7fb4feba6700 1 --2- 172.21.15.53:0/211936666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80035c0 0x7fb4e8005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:13.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.572+0000 7fb4feba6700 1 -- 172.21.15.53:0/211936666 >> 172.21.15.53:0/211936666 conn(0x7fb4f80fd4d0 msgr2=0x7fb4f80ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:13.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.572+0000 7fb4feba6700 1 -- 172.21.15.53:0/211936666 shutdown_connections 2024-05-21T15:26:13.576 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.572+0000 7fb4feba6700 1 -- 172.21.15.53:0/211936666 wait complete. 2024-05-21T15:26:13.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.573+0000 7fb4feba6700 1 Processor -- start 2024-05-21T15:26:13.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.573+0000 7fb4feba6700 1 -- start start 2024-05-21T15:26:13.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.573+0000 7fb4feba6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80035c0 0x7fb4f8101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:13.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.573+0000 7fb4feba6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4f8101e90 0x7fb4f81022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:13.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.573+0000 7fb4feba6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f810abd0 con 0x7fb4e80035c0 2024-05-21T15:26:13.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.573+0000 7fb4feba6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f810aa50 con 0x7fb4f8101e90 2024-05-21T15:26:13.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.573+0000 7fb4f7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4f8101e90 0x7fb4f81022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:13.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.573+0000 7fb4f7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4f8101e90 0x7fb4f81022a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37740/0 (socket says 172.21.15.53:37740) 2024-05-21T15:26:13.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.573+0000 7fb4f7fff700 1 -- 172.21.15.53:0/2763082384 learned_addr learned my addr 172.21.15.53:0/2763082384 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-21T15:26:13.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.574+0000 7fb4f7fff700 1 -- 172.21.15.53:0/2763082384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4f8101e90 msgr2=0x7fb4f81022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-21T15:26:13.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.574+0000 7fb4f7fff700 1 -- 172.21.15.53:0/2763082384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4f8101e90 msgr2=0x7fb4f81022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-21T15:26:13.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.574+0000 7fb4f7fff700 1 --2- 172.21.15.53:0/2763082384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4f8101e90 0x7fb4f81022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T15:26:13.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.574+0000 7fb4fc942700 1 --2- 172.21.15.53:0/2763082384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80035c0 0x7fb4f8101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:13.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.574+0000 7fb4f7fff700 1 --2- 172.21.15.53:0/2763082384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4f8101e90 0x7fb4f81022a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:26:13.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.574+0000 7fb4fc942700 1 -- 172.21.15.53:0/2763082384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4f8101e90 msgr2=0x7fb4f81022a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:26:13.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.574+0000 7fb4fc942700 1 --2- 172.21.15.53:0/2763082384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4f8101e90 0x7fb4f81022a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:13.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.574+0000 7fb4fc942700 1 -- 172.21.15.53:0/2763082384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f8103880 con 0x7fb4e80035c0 2024-05-21T15:26:13.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.574+0000 7fb4fc942700 1 --2- 172.21.15.53:0/2763082384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80035c0 0x7fb4f8101950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0059d0 tx=0x7fb4ec0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:13.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.574+0000 7fb4f5ffb700 1 -- 172.21.15.53:0/2763082384 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb4ec004b20 con 0x7fb4e80035c0 2024-05-21T15:26:13.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.574+0000 7fb4feba6700 1 -- 172.21.15.53:0/2763082384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f81a4d60 con 0x7fb4e80035c0 2024-05-21T15:26:13.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.574+0000 7fb4f5ffb700 1 -- 172.21.15.53:0/2763082384 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4ec013e40 con 0x7fb4e80035c0 2024-05-21T15:26:13.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.574+0000 7fb4f5ffb700 1 -- 172.21.15.53:0/2763082384 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb4ec011cd0 con 0x7fb4e80035c0 2024-05-21T15:26:13.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.575+0000 7fb4feba6700 1 -- 172.21.15.53:0/2763082384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f81a5260 con 0x7fb4e80035c0 2024-05-21T15:26:13.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.576+0000 7fb4feba6700 1 -- 172.21.15.53:0/2763082384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f806b3c0 con 0x7fb4e80035c0 2024-05-21T15:26:13.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.576+0000 7fb4f5ffb700 1 -- 172.21.15.53:0/2763082384 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb4ec023070 con 0x7fb4e80035c0 2024-05-21T15:26:13.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.576+0000 7fb4f5ffb700 1 --2- 172.21.15.53:0/2763082384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb4dc038e90 0x7fb4dc03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:13.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.576+0000 7fb4f5ffb700 1 -- 172.21.15.53:0/2763082384 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb4ec020050 con 0x7fb4e80035c0 2024-05-21T15:26:13.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.576+0000 7fb4f7fff700 1 --2- 172.21.15.53:0/2763082384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb4dc038e90 0x7fb4dc03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:13.581 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.577+0000 7fb4f7fff700 1 --2- 172.21.15.53:0/2763082384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb4dc038e90 0x7fb4dc03b3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8108910 tx=0x7fb4e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:13.585 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.582+0000 7fb4f5ffb700 1 -- 172.21.15.53:0/2763082384 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb4ec031020 con 0x7fb4e80035c0 2024-05-21T15:26:13.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:13 smithi038 ceph-mon[31995]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:13.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:13 smithi038 ceph-mon[31995]: Deploying daemon mon.b on smithi045 2024-05-21T15:26:13.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:13 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:26:13.878 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.874+0000 7fb4feba6700 1 -- 172.21.15.53:0/2763082384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb4f802dec0 con 0x7fb4e80035c0 2024-05-21T15:26:13.878 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:13 smithi053 ceph-mon[36496]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:13.878 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:13 smithi053 ceph-mon[36496]: Deploying daemon mon.b on smithi045 2024-05-21T15:26:13.879 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:13 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:26:13.879 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.875+0000 7fb4f5ffb700 1 -- 172.21.15.53:0/2763082384 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fb4ec028030 con 0x7fb4e80035c0 2024-05-21T15:26:13.879 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:26:13.879 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":2,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","modified":"2024-05-21T15:26:07.391245Z","created":"2024-05-21T15:24:17.361696Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-21T15:26:13.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.877+0000 7fb4feba6700 1 -- 172.21.15.53:0/2763082384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb4dc038e90 msgr2=0x7fb4dc03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:13.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.877+0000 7fb4feba6700 1 --2- 172.21.15.53:0/2763082384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb4dc038e90 0x7fb4dc03b3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8108910 tx=0x7fb4e4008040 comp rx=0 tx=0).stop 2024-05-21T15:26:13.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.877+0000 7fb4feba6700 1 -- 172.21.15.53:0/2763082384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80035c0 msgr2=0x7fb4f8101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:13.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.877+0000 7fb4feba6700 1 --2- 172.21.15.53:0/2763082384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80035c0 0x7fb4f8101950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0059d0 tx=0x7fb4ec0057d0 comp rx=0 tx=0).stop 2024-05-21T15:26:13.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.877+0000 7fb4feba6700 1 -- 172.21.15.53:0/2763082384 shutdown_connections 2024-05-21T15:26:13.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.877+0000 7fb4feba6700 1 --2- 172.21.15.53:0/2763082384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb4dc038e90 0x7fb4dc03b3a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:13.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.877+0000 7fb4feba6700 1 --2- 172.21.15.53:0/2763082384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80035c0 0x7fb4f8101950 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:13.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.877+0000 7fb4feba6700 1 --2- 172.21.15.53:0/2763082384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4f8101e90 0x7fb4f81022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:13.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.877+0000 7fb4feba6700 1 -- 172.21.15.53:0/2763082384 >> 172.21.15.53:0/2763082384 conn(0x7fb4f80fd4d0 msgr2=0x7fb4f81002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:13.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.877+0000 7fb4feba6700 1 -- 172.21.15.53:0/2763082384 shutdown_connections 2024-05-21T15:26:13.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:13.877+0000 7fb4feba6700 1 -- 172.21.15.53:0/2763082384 wait complete. 2024-05-21T15:26:13.882 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 2 2024-05-21T15:26:14.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:14 smithi053 ceph-mon[36496]: from='client.? 172.21.15.53:0/2763082384' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T15:26:14.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:14 smithi038 ceph-mon[31995]: from='client.? 172.21.15.53:0/2763082384' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T15:26:14.779 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:26:14 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:26:14.394+0000 7f48ae0c0700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-21T15:26:15.466 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T15:26:15.466 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T15:26:15.720 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:26:15.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:15 smithi038 ceph-mon[31995]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:15.819 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:15 smithi053 ceph-mon[36496]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:21.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:21.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: mon.c calling monitor election 2024-05-21T15:26:21.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:26:21.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: mon.a calling monitor election 2024-05-21T15:26:21.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:21.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:26:21.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:21.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:21.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:21.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:21.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:21.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:21.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:21.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T15:26:21.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: monmap epoch 3 2024-05-21T15:26:21.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:26:21.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: last_changed 2024-05-21T15:26:16.454926+0000 2024-05-21T15:26:21.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: created 2024-05-21T15:24:17.361696+0000 2024-05-21T15:26:21.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: min_mon_release 19 (squid) 2024-05-21T15:26:21.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: election_strategy: 1 2024-05-21T15:26:21.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-05-21T15:26:21.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: 1: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.c 2024-05-21T15:26:21.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: 2: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-05-21T15:26:21.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: fsmap 2024-05-21T15:26:21.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:26:21.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: mgrmap e13: x(active, since 59s) 2024-05-21T15:26:21.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: overall HEALTH_OK 2024-05-21T15:26:21.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:21.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:21.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:21.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: mon.c calling monitor election 2024-05-21T15:26:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:26:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: mon.a calling monitor election 2024-05-21T15:26:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:26:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: monmap epoch 3 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: last_changed 2024-05-21T15:26:16.454926+0000 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: created 2024-05-21T15:24:17.361696+0000 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: min_mon_release 19 (squid) 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: election_strategy: 1 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: 1: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.c 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: 2: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: fsmap 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:26:21.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: mgrmap e13: x(active, since 59s) 2024-05-21T15:26:21.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: overall HEALTH_OK 2024-05-21T15:26:21.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:21.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:21.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:21.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:22.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:22.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:22.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:23.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.003+0000 7fd07b4e1700 1 -- 172.21.15.53:0/628295838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd060008500 msgr2=0x7fd06000a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:23.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.003+0000 7fd07b4e1700 1 --2- 172.21.15.53:0/628295838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd060008500 0x7fd06000a9c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd074102a70 tx=0x7fd07000fe90 comp rx=0 tx=0).stop 2024-05-21T15:26:23.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.003+0000 7fd07b4e1700 1 -- 172.21.15.53:0/628295838 shutdown_connections 2024-05-21T15:26:23.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.003+0000 7fd07b4e1700 1 --2- 172.21.15.53:0/628295838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd060008500 0x7fd06000a9c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:23.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.003+0000 7fd07b4e1700 1 -- 172.21.15.53:0/628295838 >> 172.21.15.53:0/628295838 conn(0x7fd0740fd4d0 msgr2=0x7fd0740ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:23.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.003+0000 7fd07b4e1700 1 -- 172.21.15.53:0/628295838 shutdown_connections 2024-05-21T15:26:23.008 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.003+0000 7fd07b4e1700 1 -- 172.21.15.53:0/628295838 wait complete. 2024-05-21T15:26:23.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.004+0000 7fd07b4e1700 1 Processor -- start 2024-05-21T15:26:23.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.004+0000 7fd07b4e1700 1 -- start start 2024-05-21T15:26:23.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.004+0000 7fd07b4e1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd060008500 0x7fd07419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:23.010 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.004+0000 7fd07b4e1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd07419cb80 0x7fd074196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:23.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.004+0000 7fd07b4e1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd074196da0 0x7fd0741971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:23.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.004+0000 7fd07b4e1700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd0741089d0 con 0x7fd060008500 2024-05-21T15:26:23.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.004+0000 7fd07b4e1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd074108850 con 0x7fd07419cb80 2024-05-21T15:26:23.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.004+0000 7fd07b4e1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd074108b50 con 0x7fd074196da0 2024-05-21T15:26:23.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.004+0000 7fd07927d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd060008500 0x7fd07419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:23.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.004+0000 7fd079a7e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd074196da0 0x7fd0741971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:23.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd07927d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd060008500 0x7fd07419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.53:42386/0 (socket says 172.21.15.53:42386) 2024-05-21T15:26:23.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd07927d700 1 -- 172.21.15.53:0/1228696664 learned_addr learned my addr 172.21.15.53:0/1228696664 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-21T15:26:23.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd078a7c700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd07419cb80 0x7fd074196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:23.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd078a7c700 1 -- 172.21.15.53:0/1228696664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd07419cb80 msgr2=0x7fd074196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-21T15:26:23.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd078a7c700 1 -- 172.21.15.53:0/1228696664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd07419cb80 msgr2=0x7fd074196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-21T15:26:23.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd078a7c700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd07419cb80 0x7fd074196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T15:26:23.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd078a7c700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd07419cb80 0x7fd074196860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:26:23.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd079a7e700 1 -- 172.21.15.53:0/1228696664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd07419cb80 msgr2=0x7fd074196860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:26:23.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd079a7e700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd07419cb80 0x7fd074196860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:23.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd079a7e700 1 -- 172.21.15.53:0/1228696664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd060008500 msgr2=0x7fd07419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:23.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd079a7e700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd060008500 0x7fd07419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:23.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd079a7e700 1 -- 172.21.15.53:0/1228696664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0741a34d0 con 0x7fd074196da0 2024-05-21T15:26:23.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd07927d700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd060008500 0x7fd07419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:26:23.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd079a7e700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd074196da0 0x7fd0741971b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00bd50 tx=0x7fd06c00bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:23.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd06a7fc700 1 -- 172.21.15.53:0/1228696664 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c019070 con 0x7fd074196da0 2024-05-21T15:26:23.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd06a7fc700 1 -- 172.21.15.53:0/1228696664 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd06c00cc90 con 0x7fd074196da0 2024-05-21T15:26:23.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd07b4e1700 1 -- 172.21.15.53:0/1228696664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0741a3760 con 0x7fd074196da0 2024-05-21T15:26:23.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.005+0000 7fd06a7fc700 1 -- 172.21.15.53:0/1228696664 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c012740 con 0x7fd074196da0 2024-05-21T15:26:23.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.006+0000 7fd07b4e1700 1 -- 172.21.15.53:0/1228696664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0741a3c20 con 0x7fd074196da0 2024-05-21T15:26:23.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.007+0000 7fd06a7fc700 1 -- 172.21.15.53:0/1228696664 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd06c01c070 con 0x7fd074196da0 2024-05-21T15:26:23.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.007+0000 7fd06a7fc700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd058039580 0x7fd05803ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:23.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.007+0000 7fd07b4e1700 1 -- 172.21.15.53:0/1228696664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd074063ca0 con 0x7fd074196da0 2024-05-21T15:26:23.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.007+0000 7fd06a7fc700 1 -- 172.21.15.53:0/1228696664 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd06c04ba30 con 0x7fd074196da0 2024-05-21T15:26:23.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.007+0000 7fd07927d700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd058039580 0x7fd05803ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:23.014 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.008+0000 7fd07927d700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd058039580 0x7fd05803ba90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd070011890 tx=0x7fd070029040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:23.017 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.013+0000 7fd06a7fc700 1 -- 172.21.15.53:0/1228696664 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd06c007290 con 0x7fd074196da0 2024-05-21T15:26:23.030 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:23.030 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:23.030 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:23.313 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.310+0000 7fd07b4e1700 1 -- 172.21.15.53:0/1228696664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd07402dc60 con 0x7fd074196da0 2024-05-21T15:26:23.314 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.310+0000 7fd06a7fc700 1 -- 172.21.15.53:0/1228696664 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7fd06c018170 con 0x7fd074196da0 2024-05-21T15:26:23.314 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-21T15:26:23.314 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":3,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","modified":"2024-05-21T15:26:16.454926Z","created":"2024-05-21T15:24:17.361696Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-21T15:26:23.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.314+0000 7fd07b4e1700 1 -- 172.21.15.53:0/1228696664 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd058039580 msgr2=0x7fd05803ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:23.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.314+0000 7fd07b4e1700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd058039580 0x7fd05803ba90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd070011890 tx=0x7fd070029040 comp rx=0 tx=0).stop 2024-05-21T15:26:23.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.314+0000 7fd07b4e1700 1 -- 172.21.15.53:0/1228696664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd074196da0 msgr2=0x7fd0741971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:23.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.314+0000 7fd07b4e1700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd074196da0 0x7fd0741971b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00bd50 tx=0x7fd06c00bd80 comp rx=0 tx=0).stop 2024-05-21T15:26:23.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.314+0000 7fd07b4e1700 1 -- 172.21.15.53:0/1228696664 shutdown_connections 2024-05-21T15:26:23.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.314+0000 7fd07b4e1700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd058039580 0x7fd05803ba90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:23.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.314+0000 7fd07b4e1700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd074196da0 0x7fd0741971b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:23.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.314+0000 7fd07b4e1700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd060008500 0x7fd07419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:23.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.314+0000 7fd07b4e1700 1 --2- 172.21.15.53:0/1228696664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd07419cb80 0x7fd074196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:23.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.314+0000 7fd07b4e1700 1 -- 172.21.15.53:0/1228696664 >> 172.21.15.53:0/1228696664 conn(0x7fd0740fd4d0 msgr2=0x7fd074105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:23.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.314+0000 7fd07b4e1700 1 -- 172.21.15.53:0/1228696664 shutdown_connections 2024-05-21T15:26:23.318 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:23.314+0000 7fd07b4e1700 1 -- 172.21.15.53:0/1228696664 wait complete. 2024-05-21T15:26:23.319 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 3 2024-05-21T15:26:23.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:24.002 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-21T15:26:24.002 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph config generate-minimal-conf 2024-05-21T15:26:24.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: mon.b calling monitor election 2024-05-21T15:26:24.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: Updating smithi038:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:26:24.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: Updating smithi045:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:26:24.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: Updating smithi053:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:26:24.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: mon.b calling monitor election 2024-05-21T15:26:24.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: mon.c calling monitor election 2024-05-21T15:26:24.766 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.766 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: mon.a calling monitor election 2024-05-21T15:26:24.766 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T15:26:24.766 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: monmap epoch 3 2024-05-21T15:26:24.766 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:26:24.766 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: last_changed 2024-05-21T15:26:16.454926+0000 2024-05-21T15:26:24.766 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: created 2024-05-21T15:24:17.361696+0000 2024-05-21T15:26:24.766 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: min_mon_release 19 (squid) 2024-05-21T15:26:24.766 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: election_strategy: 1 2024-05-21T15:26:24.766 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-05-21T15:26:24.766 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: 1: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.c 2024-05-21T15:26:24.766 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: 2: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-05-21T15:26:24.767 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: fsmap 2024-05-21T15:26:24.767 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:26:24.767 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: mgrmap e13: x(active, since 61s) 2024-05-21T15:26:24.767 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: overall HEALTH_OK 2024-05-21T15:26:24.767 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.767 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.767 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.767 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.767 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.767 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.767 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.767 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.767 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.768 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T15:26:24.768 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:26:24.768 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:26:24.768 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:24.768 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: Reconfiguring daemon mon.a on smithi038 2024-05-21T15:26:24.768 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:24.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: mon.b calling monitor election 2024-05-21T15:26:24.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: Updating smithi038:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:26:24.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: Updating smithi045:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:26:24.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: Updating smithi053:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:26:24.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: mon.b calling monitor election 2024-05-21T15:26:24.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: mon.c calling monitor election 2024-05-21T15:26:24.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: mon.a calling monitor election 2024-05-21T15:26:24.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T15:26:24.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: monmap epoch 3 2024-05-21T15:26:24.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:26:24.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: last_changed 2024-05-21T15:26:16.454926+0000 2024-05-21T15:26:24.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: created 2024-05-21T15:24:17.361696+0000 2024-05-21T15:26:24.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: min_mon_release 19 (squid) 2024-05-21T15:26:24.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: election_strategy: 1 2024-05-21T15:26:24.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-05-21T15:26:24.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: 1: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.c 2024-05-21T15:26:24.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: 2: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-05-21T15:26:24.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: fsmap 2024-05-21T15:26:24.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:26:24.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: mgrmap e13: x(active, since 61s) 2024-05-21T15:26:24.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: overall HEALTH_OK 2024-05-21T15:26:24.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T15:26:24.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:26:24.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:26:24.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:24.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: Reconfiguring daemon mon.a on smithi038 2024-05-21T15:26:24.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:24.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: mon.b calling monitor election 2024-05-21T15:26:24.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: Updating smithi038:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:26:24.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: Updating smithi045:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:26:24.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: Updating smithi053:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:26:24.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: mon.b calling monitor election 2024-05-21T15:26:24.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: mon.c calling monitor election 2024-05-21T15:26:24.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: mon.a calling monitor election 2024-05-21T15:26:24.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T15:26:24.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: monmap epoch 3 2024-05-21T15:26:24.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:26:24.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: last_changed 2024-05-21T15:26:16.454926+0000 2024-05-21T15:26:24.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: created 2024-05-21T15:24:17.361696+0000 2024-05-21T15:26:24.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: min_mon_release 19 (squid) 2024-05-21T15:26:24.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: election_strategy: 1 2024-05-21T15:26:24.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-05-21T15:26:24.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: 1: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.c 2024-05-21T15:26:24.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: 2: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-05-21T15:26:24.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: fsmap 2024-05-21T15:26:24.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:26:24.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: mgrmap e13: x(active, since 61s) 2024-05-21T15:26:24.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: overall HEALTH_OK 2024-05-21T15:26:24.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.962 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.963 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:24.963 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T15:26:24.963 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:26:24.963 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:26:24.963 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:24.963 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: Reconfiguring daemon mon.a on smithi038 2024-05-21T15:26:24.963 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:26:25.265 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:26:25.914 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:25 smithi045 ceph-mon[36409]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:25.914 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:25.914 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:25.914 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:26:25.914 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:26:25.914 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:25.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:25 smithi053 ceph-mon[36496]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:25.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:25.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:25.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:26:25.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:26:25.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:26.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:25 smithi038 ceph-mon[31995]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:26.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:26.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:26.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:26:26.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:26:26.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:26.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.793+0000 7f8171013700 1 -- 172.21.15.38:0/2836760974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816c107d90 msgr2=0x7f816c1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:26.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.793+0000 7f8171013700 1 --2- 172.21.15.38:0/2836760974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816c107d90 0x7f816c1081a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f81600099c0 tx=0x7f8160009cd0 comp rx=0 tx=0).stop 2024-05-21T15:26:26.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.793+0000 7f8171013700 1 -- 172.21.15.38:0/2836760974 shutdown_connections 2024-05-21T15:26:26.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.793+0000 7f8171013700 1 --2- 172.21.15.38:0/2836760974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f816c1086e0 0x7f816c110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:26.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.793+0000 7f8171013700 1 --2- 172.21.15.38:0/2836760974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816c103460 0x7f816c103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:26.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.793+0000 7f8171013700 1 --2- 172.21.15.38:0/2836760974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816c107d90 0x7f816c1081a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:26.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.793+0000 7f8171013700 1 -- 172.21.15.38:0/2836760974 >> 172.21.15.38:0/2836760974 conn(0x7f816c0fed70 msgr2=0x7f816c1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:26.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.793+0000 7f8171013700 1 -- 172.21.15.38:0/2836760974 shutdown_connections 2024-05-21T15:26:26.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.793+0000 7f8171013700 1 -- 172.21.15.38:0/2836760974 wait complete. 2024-05-21T15:26:26.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.794+0000 7f8171013700 1 Processor -- start 2024-05-21T15:26:26.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.794+0000 7f8171013700 1 -- start start 2024-05-21T15:26:26.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.794+0000 7f8171013700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f816c103460 0x7f816c075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.794+0000 7f8171013700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816c107d90 0x7f816c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f8171013700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816c1086e0 0x7f816c0764d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f8171013700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f816c113440 con 0x7f816c1086e0 2024-05-21T15:26:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f8171013700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f816c1132c0 con 0x7f816c103460 2024-05-21T15:26:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f8171013700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f816c1135c0 con 0x7f816c107d90 2024-05-21T15:26:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f816b59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816c1086e0 0x7f816c0764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f816b59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816c1086e0 0x7f816c0764d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33008/0 (socket says 172.21.15.38:33008) 2024-05-21T15:26:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f816b59e700 1 -- 172.21.15.38:0/2611199123 learned_addr learned my addr 172.21.15.38:0/2611199123 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:26:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f816ad9d700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f816c103460 0x7f816c075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f816a59c700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816c107d90 0x7f816c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f816b59e700 1 -- 172.21.15.38:0/2611199123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816c107d90 msgr2=0x7f816c075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f816b59e700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816c107d90 0x7f816c075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:26.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f816b59e700 1 -- 172.21.15.38:0/2611199123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f816c103460 msgr2=0x7f816c075a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f816b59e700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f816c103460 0x7f816c075a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f816b59e700 1 -- 172.21.15.38:0/2611199123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c1a8170 con 0x7f816c1086e0 2024-05-21T15:26:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f816ad9d700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f816c103460 0x7f816c075a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:26:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f816a59c700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816c107d90 0x7f816c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:26:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f816b59e700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816c1086e0 0x7f816c0764d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8164007ee0 tx=0x7f816400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f8153fff700 1 -- 172.21.15.38:0/2611199123 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164004bb0 con 0x7f816c1086e0 2024-05-21T15:26:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f8153fff700 1 -- 172.21.15.38:0/2611199123 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8164004d50 con 0x7f816c1086e0 2024-05-21T15:26:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f8171013700 1 -- 172.21.15.38:0/2611199123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c1a8460 con 0x7f816c1086e0 2024-05-21T15:26:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.795+0000 7f8153fff700 1 -- 172.21.15.38:0/2611199123 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164005960 con 0x7f816c1086e0 2024-05-21T15:26:26.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.796+0000 7f8171013700 1 -- 172.21.15.38:0/2611199123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c1a8920 con 0x7f816c1086e0 2024-05-21T15:26:26.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.796+0000 7f8153fff700 1 -- 172.21.15.38:0/2611199123 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f816401a020 con 0x7f816c1086e0 2024-05-21T15:26:26.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.797+0000 7f8171013700 1 -- 172.21.15.38:0/2611199123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816c050170 con 0x7f816c1086e0 2024-05-21T15:26:26.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.797+0000 7f8153fff700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8154039330 0x7f815403b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:26.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.797+0000 7f8153fff700 1 -- 172.21.15.38:0/2611199123 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f816404ba00 con 0x7f816c1086e0 2024-05-21T15:26:26.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.797+0000 7f816ad9d700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8154039330 0x7f815403b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:26.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.797+0000 7f816ad9d700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8154039330 0x7f815403b840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f815c011490 tx=0x7f815c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:26.802+0000 7f8153fff700 1 -- 172.21.15.38:0/2611199123 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8164010070 con 0x7f816c1086e0 2024-05-21T15:26:26.816 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:26 smithi045 ceph-mon[36409]: Reconfiguring mon.b (monmap changed)... 2024-05-21T15:26:26.816 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:26 smithi045 ceph-mon[36409]: Reconfiguring daemon mon.b on smithi045 2024-05-21T15:26:26.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:26 smithi053 ceph-mon[36496]: Reconfiguring mon.b (monmap changed)... 2024-05-21T15:26:26.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:26 smithi053 ceph-mon[36496]: Reconfiguring daemon mon.b on smithi045 2024-05-21T15:26:27.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:27.008+0000 7f8171013700 1 -- 172.21.15.38:0/2611199123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f816c02dc60 con 0x7f816c1086e0 2024-05-21T15:26:27.012 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:26 smithi038 ceph-mon[31995]: Reconfiguring mon.b (monmap changed)... 2024-05-21T15:26:27.012 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:26 smithi038 ceph-mon[31995]: Reconfiguring daemon mon.b on smithi045 2024-05-21T15:26:27.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:27.008+0000 7f8153fff700 1 -- 172.21.15.38:0/2611199123 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7f8164019520 con 0x7f816c1086e0 2024-05-21T15:26:27.012 INFO:teuthology.orchestra.run.smithi038.stdout:# minimal ceph.conf for 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:26:27.012 INFO:teuthology.orchestra.run.smithi038.stdout:[global] 2024-05-21T15:26:27.012 INFO:teuthology.orchestra.run.smithi038.stdout: fsid = 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:26:27.013 INFO:teuthology.orchestra.run.smithi038.stdout: mon_host = [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] 2024-05-21T15:26:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:27.011+0000 7f8171013700 1 -- 172.21.15.38:0/2611199123 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8154039330 msgr2=0x7f815403b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:27.011+0000 7f8171013700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8154039330 0x7f815403b840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f815c011490 tx=0x7f815c009260 comp rx=0 tx=0).stop 2024-05-21T15:26:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:27.011+0000 7f8171013700 1 -- 172.21.15.38:0/2611199123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816c1086e0 msgr2=0x7f816c0764d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:27.011+0000 7f8171013700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816c1086e0 0x7f816c0764d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8164007ee0 tx=0x7f816400d3b0 comp rx=0 tx=0).stop 2024-05-21T15:26:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:27.011+0000 7f8171013700 1 -- 172.21.15.38:0/2611199123 shutdown_connections 2024-05-21T15:26:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:27.011+0000 7f8171013700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8154039330 0x7f815403b840 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:27.011+0000 7f8171013700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f816c1086e0 0x7f816c0764d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:27.011+0000 7f8171013700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f816c103460 0x7f816c075a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:27.011+0000 7f8171013700 1 --2- 172.21.15.38:0/2611199123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816c107d90 0x7f816c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:27.011+0000 7f8171013700 1 -- 172.21.15.38:0/2611199123 >> 172.21.15.38:0/2611199123 conn(0x7f816c0fed70 msgr2=0x7f816c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:27.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:27.011+0000 7f8171013700 1 -- 172.21.15.38:0/2611199123 shutdown_connections 2024-05-21T15:26:27.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:27.011+0000 7f8171013700 1 -- 172.21.15.38:0/2611199123 wait complete. 2024-05-21T15:26:27.578 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-21T15:26:27.578 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:26:27.578 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T15:26:27.606 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:26:27.606 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:26:27.672 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T15:26:27.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T15:26:27.706 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T15:26:27.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:26:27.775 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-21T15:26:27.775 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T15:26:27.804 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-21T15:26:27.804 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:26:27.870 INFO:tasks.cephadm:Adding mgr.x on smithi038 2024-05-21T15:26:27.870 INFO:tasks.cephadm:Adding mgr.y on smithi045 2024-05-21T15:26:27.870 INFO:tasks.cephadm:Adding mgr.z on smithi053 2024-05-21T15:26:27.870 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch apply mgr '3;smithi038=x;smithi045=y;smithi053=z' 2024-05-21T15:26:27.932 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:27 smithi053 ceph-mon[36496]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:27.932 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:27.932 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:27.932 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:27 smithi053 ceph-mon[36496]: Reconfiguring mon.c (monmap changed)... 2024-05-21T15:26:27.932 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:26:27.933 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:26:27.933 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:27.933 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:27 smithi053 ceph-mon[36496]: Reconfiguring daemon mon.c on smithi053 2024-05-21T15:26:27.933 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:27 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2611199123' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:28.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:27 smithi045 ceph-mon[36409]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:28.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:28.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:28.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:27 smithi045 ceph-mon[36409]: Reconfiguring mon.c (monmap changed)... 2024-05-21T15:26:28.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:26:28.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:26:28.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:28.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:27 smithi045 ceph-mon[36409]: Reconfiguring daemon mon.c on smithi053 2024-05-21T15:26:28.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:27 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2611199123' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:28.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:27 smithi038 ceph-mon[31995]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:28.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:28.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:28.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:27 smithi038 ceph-mon[31995]: Reconfiguring mon.c (monmap changed)... 2024-05-21T15:26:28.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:26:28.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:26:28.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:28.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:27 smithi038 ceph-mon[31995]: Reconfiguring daemon mon.c on smithi053 2024-05-21T15:26:28.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:27 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2611199123' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:28.480 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:26:29.571 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:29 smithi053 ceph-mon[36496]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:29.571 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:29 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:29.572 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:29 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:29.572 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:29 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:29.572 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:29 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:29.572 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:29 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:29.572 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:29 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:29.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:29 smithi045 ceph-mon[36409]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:29.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:29 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:29.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:29 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:29.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:29 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:29.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:29 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:29.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:29 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:29.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:29 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:29.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:29 smithi038 ceph-mon[31995]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:29.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:29 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:29.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:29 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:29.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:29 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:29.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:29 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:29.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:29 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:29.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:29 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:29.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.779+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/1283717418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f9c103b90 msgr2=0x7f8f9c10a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:29.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.779+0000 7f8fa1ea8700 1 --2- 172.21.15.53:0/1283717418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f9c103b90 0x7f8f9c10a440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400b0a0 tx=0x7f8f9400b3b0 comp rx=0 tx=0).stop 2024-05-21T15:26:29.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.779+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/1283717418 shutdown_connections 2024-05-21T15:26:29.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.779+0000 7f8fa1ea8700 1 --2- 172.21.15.53:0/1283717418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f9c103b90 0x7f8f9c10a440 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:29.785 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.779+0000 7f8fa1ea8700 1 --2- 172.21.15.53:0/1283717418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f9c1020c0 0x7f8f9c1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:29.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.779+0000 7f8fa1ea8700 1 --2- 172.21.15.53:0/1283717418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f9c1031f0 0x7f8f9c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:29.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.779+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/1283717418 >> 172.21.15.53:0/1283717418 conn(0x7f8f9c0fd700 msgr2=0x7f8f9c0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:29.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.779+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/1283717418 shutdown_connections 2024-05-21T15:26:29.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.779+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/1283717418 wait complete. 2024-05-21T15:26:29.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.780+0000 7f8fa1ea8700 1 Processor -- start 2024-05-21T15:26:29.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.780+0000 7f8fa1ea8700 1 -- start start 2024-05-21T15:26:29.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.780+0000 7f8fa1ea8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f9c1020c0 0x7f8f9c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:29.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.780+0000 7f8fa1ea8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f9c1031f0 0x7f8f9c199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:29.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.780+0000 7f8fa1ea8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f9c103b90 0x7f8f9c1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:29.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.780+0000 7f8fa1ea8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c10dc90 con 0x7f8f9c1020c0 2024-05-21T15:26:29.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.780+0000 7f8fa1ea8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c10db10 con 0x7f8f9c103b90 2024-05-21T15:26:29.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.780+0000 7f8fa1ea8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c10de10 con 0x7f8f9c1031f0 2024-05-21T15:26:29.786 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8f9affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f9c1031f0 0x7f8f9c199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:29.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8f9b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f9c1020c0 0x7f8f9c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:29.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8f9affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f9c1031f0 0x7f8f9c199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33266/0 (socket says 172.21.15.53:33266) 2024-05-21T15:26:29.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8f9affd700 1 -- 172.21.15.53:0/933765085 learned_addr learned my addr 172.21.15.53:0/933765085 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-21T15:26:29.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8f9bfff700 1 --2- 172.21.15.53:0/933765085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f9c103b90 0x7f8f9c1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:29.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8f9affd700 1 -- 172.21.15.53:0/933765085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f9c103b90 msgr2=0x7f8f9c1a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:29.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8f9affd700 1 --2- 172.21.15.53:0/933765085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f9c103b90 0x7f8f9c1a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:29.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8f9affd700 1 -- 172.21.15.53:0/933765085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f9c1020c0 msgr2=0x7f8f9c199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:29.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8f9affd700 1 --2- 172.21.15.53:0/933765085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f9c1020c0 0x7f8f9c199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:29.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8f9affd700 1 -- 172.21.15.53:0/933765085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9c1a1380 con 0x7f8f9c1031f0 2024-05-21T15:26:29.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8f9affd700 1 --2- 172.21.15.53:0/933765085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f9c1031f0 0x7f8f9c199d60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000d950 tx=0x7f8f9000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:29.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8f98ff9700 1 -- 172.21.15.53:0/933765085 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90016070 con 0x7f8f9c1031f0 2024-05-21T15:26:29.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8f98ff9700 1 -- 172.21.15.53:0/933765085 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f9000fd70 con 0x7f8f9c1031f0 2024-05-21T15:26:29.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/933765085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9c1a1670 con 0x7f8f9c1031f0 2024-05-21T15:26:29.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.781+0000 7f8f98ff9700 1 -- 172.21.15.53:0/933765085 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9000e610 con 0x7f8f9c1031f0 2024-05-21T15:26:29.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.782+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/933765085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9c1a1bb0 con 0x7f8f9c1031f0 2024-05-21T15:26:29.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.782+0000 7f8f98ff9700 1 -- 172.21.15.53:0/933765085 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8f90019070 con 0x7f8f9c1031f0 2024-05-21T15:26:29.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.782+0000 7f8f98ff9700 1 --2- 172.21.15.53:0/933765085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f84039330 0x7f8f8403b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:29.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.783+0000 7f8f98ff9700 1 -- 172.21.15.53:0/933765085 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8f9004ce00 con 0x7f8f9c1031f0 2024-05-21T15:26:29.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.783+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/933765085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f9c050170 con 0x7f8f9c1031f0 2024-05-21T15:26:29.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.783+0000 7f8f9b7fe700 1 --2- 172.21.15.53:0/933765085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f84039330 0x7f8f8403b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:29.789 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.783+0000 7f8f9b7fe700 1 --2- 172.21.15.53:0/933765085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f84039330 0x7f8f8403b840 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c1030f0 tx=0x7f8f8c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:29.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:29.788+0000 7f8f98ff9700 1 -- 172.21.15.53:0/933765085 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8f90010e10 con 0x7f8f9c1031f0 2024-05-21T15:26:30.013 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:30.009+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/933765085 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi045=y;smithi053=z", "target": ["mon-mgr", ""]}) -- 0x7f8f9c0ff0b0 con 0x7f8f84039330 2024-05-21T15:26:30.020 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:30.017+0000 7f8f98ff9700 1 -- 172.21.15.53:0/933765085 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8f9c0ff0b0 con 0x7f8f84039330 2024-05-21T15:26:30.020 INFO:teuthology.orchestra.run.smithi053.stdout:Scheduled mgr update... 2024-05-21T15:26:30.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:30.019+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/933765085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f84039330 msgr2=0x7f8f8403b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:30.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:30.019+0000 7f8fa1ea8700 1 --2- 172.21.15.53:0/933765085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f84039330 0x7f8f8403b840 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c1030f0 tx=0x7f8f8c007400 comp rx=0 tx=0).stop 2024-05-21T15:26:30.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:30.019+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/933765085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f9c1031f0 msgr2=0x7f8f9c199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:30.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:30.019+0000 7f8fa1ea8700 1 --2- 172.21.15.53:0/933765085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f9c1031f0 0x7f8f9c199d60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000d950 tx=0x7f8f9000dc60 comp rx=0 tx=0).stop 2024-05-21T15:26:30.023 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:30.019+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/933765085 shutdown_connections 2024-05-21T15:26:30.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:30.019+0000 7f8fa1ea8700 1 --2- 172.21.15.53:0/933765085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f84039330 0x7f8f8403b840 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:30.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:30.019+0000 7f8fa1ea8700 1 --2- 172.21.15.53:0/933765085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f9c103b90 0x7f8f9c1a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:30.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:30.019+0000 7f8fa1ea8700 1 --2- 172.21.15.53:0/933765085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f9c1020c0 0x7f8f9c199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:30.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:30.019+0000 7f8fa1ea8700 1 --2- 172.21.15.53:0/933765085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f9c1031f0 0x7f8f9c199d60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:30.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:30.019+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/933765085 >> 172.21.15.53:0/933765085 conn(0x7f8f9c0fd700 msgr2=0x7f8f9c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:30.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:30.019+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/933765085 shutdown_connections 2024-05-21T15:26:30.024 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:26:30.019+0000 7f8fa1ea8700 1 -- 172.21.15.53:0/933765085 wait complete. 2024-05-21T15:26:30.581 DEBUG:teuthology.orchestra.run.smithi045:mgr.y> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.y.service 2024-05-21T15:26:30.584 DEBUG:teuthology.orchestra.run.smithi053:mgr.z> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.z.service 2024-05-21T15:26:30.627 INFO:tasks.cephadm:Deploying OSDs... 2024-05-21T15:26:30.627 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:26:30.627 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T15:26:30.648 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T15:26:30.648 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 2024-05-21T15:26:30.707 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T15:26:30.708 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:26:30.708 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-21T15:26:30.708 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:26:30.708 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:26:30.708 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-05-21 15:25:34.432454497 +0000 2024-05-21T15:26:30.708 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-05-21 15:20:28.571347461 +0000 2024-05-21T15:26:30.708 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-05-21 15:20:28.571347461 +0000 2024-05-21T15:26:30.708 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-05-21 15:20:28.571347461 +0000 2024-05-21T15:26:30.708 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T15:26:30.778 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-05-21T15:26:30.778 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-05-21T15:26:30.778 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000323037 s, 1.6 MB/s 2024-05-21T15:26:30.779 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T15:26:30.838 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 2024-05-21T15:26:30.896 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T15:26:30.896 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:26:30.896 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-21T15:26:30.896 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:26:30.896 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:26:30.896 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-05-21 15:25:34.467453899 +0000 2024-05-21T15:26:30.896 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-05-21 15:20:28.914341476 +0000 2024-05-21T15:26:30.897 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-05-21 15:20:28.914341476 +0000 2024-05-21T15:26:30.897 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-05-21 15:20:28.914341476 +0000 2024-05-21T15:26:30.897 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T15:26:30.963 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-05-21T15:26:30.964 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-05-21T15:26:30.964 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000292216 s, 1.8 MB/s 2024-05-21T15:26:30.965 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T15:26:31.024 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 2024-05-21T15:26:31.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:31 smithi038 ceph-mon[31995]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi045=y;smithi053=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:26:31.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:31 smithi038 ceph-mon[31995]: Saving service mgr spec with placement smithi038=x;smithi045=y;smithi053=z;count:3 2024-05-21T15:26:31.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:31.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:31.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:31.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:31.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:31.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:26:31.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T15:26:31.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:26:31.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:31.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:31 smithi038 ceph-mon[31995]: Deploying daemon mgr.z on smithi053 2024-05-21T15:26:31.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:31 smithi038 ceph-mon[31995]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:31.045 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T15:26:31.045 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:26:31.045 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-21T15:26:31.045 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:26:31.045 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:26:31.045 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-05-21 15:25:34.501453319 +0000 2024-05-21T15:26:31.045 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-05-21 15:20:29.275335177 +0000 2024-05-21T15:26:31.046 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-05-21 15:20:29.275335177 +0000 2024-05-21T15:26:31.046 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-05-21 15:20:29.275335177 +0000 2024-05-21T15:26:31.046 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T15:26:31.112 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-05-21T15:26:31.112 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-05-21T15:26:31.112 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000290742 s, 1.8 MB/s 2024-05-21T15:26:31.113 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T15:26:31.170 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 2024-05-21T15:26:31.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:31 smithi053 ceph-mon[36496]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi045=y;smithi053=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:26:31.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:31 smithi053 ceph-mon[36496]: Saving service mgr spec with placement smithi038=x;smithi045=y;smithi053=z;count:3 2024-05-21T15:26:31.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:31.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:31.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:31.211 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:31.211 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:31.211 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:26:31.211 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T15:26:31.211 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:26:31.211 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:31.211 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:31 smithi053 ceph-mon[36496]: Deploying daemon mgr.z on smithi053 2024-05-21T15:26:31.211 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:31 smithi053 ceph-mon[36496]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:31.227 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T15:26:31.228 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:26:31.228 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-21T15:26:31.228 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:26:31.228 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:26:31.228 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-05-21 15:25:34.535452739 +0000 2024-05-21T15:26:31.228 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-05-21 15:20:29.623329105 +0000 2024-05-21T15:26:31.228 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-05-21 15:20:29.623329105 +0000 2024-05-21T15:26:31.228 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-05-21 15:20:29.623329105 +0000 2024-05-21T15:26:31.228 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T15:26:31.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-05-21T15:26:31.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-05-21T15:26:31.293 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000288161 s, 1.8 MB/s 2024-05-21T15:26:31.294 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T15:26:31.351 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T15:26:31.351 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T15:26:31.371 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T15:26:31.371 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-05-21T15:26:31.429 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T15:26:31.429 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:26:31.429 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-21T15:26:31.429 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:26:31.429 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:26:31.429 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-21 15:26:15.912562404 +0000 2024-05-21T15:26:31.429 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-21 15:20:42.039584183 +0000 2024-05-21T15:26:31.430 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-21 15:20:42.039584183 +0000 2024-05-21T15:26:31.430 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-21 15:20:42.039584183 +0000 2024-05-21T15:26:31.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T15:26:31.494 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:31 smithi045 ceph-mon[36409]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi045=y;smithi053=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:26:31.495 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:31 smithi045 ceph-mon[36409]: Saving service mgr spec with placement smithi038=x;smithi045=y;smithi053=z;count:3 2024-05-21T15:26:31.495 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:31.495 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:31.495 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:31.495 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:31.495 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:31.495 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:26:31.495 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T15:26:31.495 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:26:31.495 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:31.495 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:31 smithi045 ceph-mon[36409]: Deploying daemon mgr.z on smithi053 2024-05-21T15:26:31.495 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:31 smithi045 ceph-mon[36409]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:31.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-21T15:26:31.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-21T15:26:31.497 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000121261 s, 4.2 MB/s 2024-05-21T15:26:31.499 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T15:26:31.558 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-05-21T15:26:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T15:26:31.616 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:26:31.616 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-21T15:26:31.616 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:26:31.616 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:26:31.616 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-21 15:26:15.912562404 +0000 2024-05-21T15:26:31.616 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-21 15:20:42.371578823 +0000 2024-05-21T15:26:31.617 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-21 15:20:42.371578823 +0000 2024-05-21T15:26:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-21 15:20:42.371578823 +0000 2024-05-21T15:26:31.617 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T15:26:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-21T15:26:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-21T15:26:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000113807 s, 4.5 MB/s 2024-05-21T15:26:31.683 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T15:26:31.741 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-05-21T15:26:31.797 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T15:26:31.797 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:26:31.797 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-21T15:26:31.797 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:26:31.798 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:26:31.798 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-21 15:26:15.912562404 +0000 2024-05-21T15:26:31.798 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-21 15:20:42.704573447 +0000 2024-05-21T15:26:31.798 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-21 15:20:42.704573447 +0000 2024-05-21T15:26:31.798 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-21 15:20:42.704573447 +0000 2024-05-21T15:26:31.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T15:26:31.862 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-21T15:26:31.862 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-21T15:26:31.862 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000115119 s, 4.4 MB/s 2024-05-21T15:26:31.863 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T15:26:31.922 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-05-21T15:26:31.980 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T15:26:31.980 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:26:31.980 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-21T15:26:31.981 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:26:31.981 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:26:31.981 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-21 15:26:15.913562389 +0000 2024-05-21T15:26:31.981 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-21 15:20:43.036568088 +0000 2024-05-21T15:26:31.981 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-21 15:20:43.036568088 +0000 2024-05-21T15:26:31.982 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-21 15:20:43.036568088 +0000 2024-05-21T15:26:31.982 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T15:26:32.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-21T15:26:32.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-21T15:26:32.047 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.00011729 s, 4.4 MB/s 2024-05-21T15:26:32.049 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T15:26:32.107 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-21T15:26:32.107 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T15:26:32.125 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T15:26:32.125 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-05-21T15:26:32.182 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T15:26:32.182 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:26:32.182 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-21T15:26:32.182 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:26:32.182 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:26:32.182 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-21 15:26:32.075761827 +0000 2024-05-21T15:26:32.182 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-21 15:20:50.078971069 +0000 2024-05-21T15:26:32.182 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-21 15:20:50.078971069 +0000 2024-05-21T15:26:32.182 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-05-21 15:20:50.078971069 +0000 2024-05-21T15:26:32.183 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T15:26:32.250 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:32 smithi053 systemd[1]: Starting Ceph mgr.z for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:26:32.254 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-21T15:26:32.254 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-21T15:26:32.254 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000111539 s, 4.6 MB/s 2024-05-21T15:26:32.256 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T15:26:32.312 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-05-21T15:26:32.367 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T15:26:32.368 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:26:32.368 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-21T15:26:32.368 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:26:32.368 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:26:32.368 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-21 15:26:32.075761827 +0000 2024-05-21T15:26:32.368 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-21 15:20:50.400964296 +0000 2024-05-21T15:26:32.368 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-21 15:20:50.400964296 +0000 2024-05-21T15:26:32.368 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-05-21 15:20:50.400964296 +0000 2024-05-21T15:26:32.368 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T15:26:32.435 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-21T15:26:32.435 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-21T15:26:32.435 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000104563 s, 4.9 MB/s 2024-05-21T15:26:32.435 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T15:26:32.492 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-05-21T15:26:32.527 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:32 smithi053 podman[38171]: 2024-05-21 15:26:32.274314702 +0000 UTC m=+0.018472410 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:26:32.527 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:32 smithi053 podman[38171]: 2024-05-21 15:26:32.384717538 +0000 UTC m=+0.128875223 container create 75e9738b9c6d42b7585a5490bf14a295007d5ee13c00a4e84ce2d9684f02884a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-21T15:26:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T15:26:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:26:32.544 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-21T15:26:32.544 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:26:32.544 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:26:32.544 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-21 15:26:32.076761808 +0000 2024-05-21T15:26:32.544 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-21 15:20:50.717957628 +0000 2024-05-21T15:26:32.544 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-21 15:20:50.717957628 +0000 2024-05-21T15:26:32.544 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-05-21 15:20:50.717957628 +0000 2024-05-21T15:26:32.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T15:26:32.608 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-21T15:26:32.608 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-21T15:26:32.608 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.00010118 s, 5.1 MB/s 2024-05-21T15:26:32.609 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T15:26:32.665 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-05-21T15:26:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T15:26:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:26:32.721 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-21T15:26:32.721 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:26:32.721 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:26:32.721 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-21 15:26:32.076761808 +0000 2024-05-21T15:26:32.721 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-21 15:20:51.060950412 +0000 2024-05-21T15:26:32.721 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-21 15:20:51.060950412 +0000 2024-05-21T15:26:32.721 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-05-21 15:20:51.060950412 +0000 2024-05-21T15:26:32.722 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T15:26:32.784 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:32 smithi053 podman[38171]: 2024-05-21 15:26:32.671190531 +0000 UTC m=+0.415348223 container init 75e9738b9c6d42b7585a5490bf14a295007d5ee13c00a4e84ce2d9684f02884a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:26:32.784 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:32 smithi053 podman[38171]: 2024-05-21 15:26:32.67466727 +0000 UTC m=+0.418824970 container start 75e9738b9c6d42b7585a5490bf14a295007d5ee13c00a4e84ce2d9684f02884a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-21T15:26:32.785 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:32 smithi053 ceph-mgr[38324]: -- 172.21.15.53:0/2951806941 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b832de5a0 con 0x555b8341c000 2024-05-21T15:26:32.785 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:32 smithi053 bash[38171]: 75e9738b9c6d42b7585a5490bf14a295007d5ee13c00a4e84ce2d9684f02884a 2024-05-21T15:26:32.785 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:32 smithi053 systemd[1]: Started Ceph mgr.z for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:26:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-21T15:26:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-21T15:26:32.788 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000119683 s, 4.3 MB/s 2024-05-21T15:26:32.789 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T15:26:32.847 INFO:tasks.cephadm:Deploying osd.0 on smithi038 with /dev/vg_nvme/lv_4... 2024-05-21T15:26:32.847 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T15:26:33.062 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:32 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:32.866+0000 7fc402e61200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T15:26:33.063 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:33 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:33.059+0000 7fc402e61200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:26:33.095 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:26:33.407 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:33 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:33.166+0000 7fc402e61200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T15:26:33.709 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:33 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:33.492+0000 7fc402e61200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T15:26:33.709 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:33 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:33.580+0000 7fc402e61200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T15:26:34.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:33 smithi045 ceph-mon[36409]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:34.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:34.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:34.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:34.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:26:34.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T15:26:34.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:26:34.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:34.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:33 smithi045 ceph-mon[36409]: Deploying daemon mgr.y on smithi045 2024-05-21T15:26:34.115 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:33 smithi038 ceph-mon[31995]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:34.115 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:34.115 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:34.115 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:34.115 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:26:34.115 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T15:26:34.115 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:26:34.116 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:34.116 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:33 smithi038 ceph-mon[31995]: Deploying daemon mgr.y on smithi045 2024-05-21T15:26:34.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:33 smithi053 ceph-mon[36496]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:34.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:34.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:34.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:34.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:26:34.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T15:26:34.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:26:34.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:34.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:33 smithi053 ceph-mon[36496]: Deploying daemon mgr.y on smithi045 2024-05-21T15:26:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T15:26:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T15:26:34.962 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-05-21T15:26:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-05-21T15:26:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0103817 s, 1.0 GB/s 2024-05-21T15:26:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-05-21T15:26:35.014 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:34 smithi045 systemd[1]: Starting Ceph mgr.y for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:26:35.195 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:34 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:34.838+0000 7fc402e61200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T15:26:35.195 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:35 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:35.017+0000 7fc402e61200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T15:26:35.195 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:35 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:35.191+0000 7fc402e61200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T15:26:35.341 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:35 smithi045 podman[37800]: 2024-05-21 15:26:34.984817893 +0000 UTC m=+0.017453913 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:26:35.342 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:35 smithi045 podman[37800]: 2024-05-21 15:26:35.098174551 +0000 UTC m=+0.130810557 container create 65f2b4e1696a869ba2ff4e8d0f47cc0b184e4f5d22f38158974c5bc048041316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-21T15:26:35.342 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:35 smithi045 podman[37800]: 2024-05-21 15:26:35.300377053 +0000 UTC m=+0.333013066 container init 65f2b4e1696a869ba2ff4e8d0f47cc0b184e4f5d22f38158974c5bc048041316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-21T15:26:35.342 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:35 smithi045 podman[37800]: 2024-05-21 15:26:35.30322234 +0000 UTC m=+0.335858356 container start 65f2b4e1696a869ba2ff4e8d0f47cc0b184e4f5d22f38158974c5bc048041316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:26:35.342 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:35 smithi045 ceph-mgr[37820]: -- 172.21.15.45:0/1386442073 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56344356a5a0 con 0x5634436a8480 2024-05-21T15:26:35.574 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_4 2024-05-21T15:26:35.662 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:35 smithi045 bash[37800]: 65f2b4e1696a869ba2ff4e8d0f47cc0b184e4f5d22f38158974c5bc048041316 2024-05-21T15:26:35.662 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:35 smithi045 systemd[1]: Started Ceph mgr.y for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:26:35.662 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:35 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:35.473+0000 7fc7fa5e2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T15:26:35.810 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:35 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:35.494+0000 7fc402e61200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T15:26:35.810 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:35 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:35.589+0000 7fc402e61200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T15:26:35.810 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:35 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:35.806+0000 7fc402e61200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T15:26:35.815 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:26:36.014 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:35 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:35.673+0000 7fc7fa5e2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:26:36.015 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:35 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:35.786+0000 7fc7fa5e2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T15:26:36.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:35 smithi045 ceph-mon[36409]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:36.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:35 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:36.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:35 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:36.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:35 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:36.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:35 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:36.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:35 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:36.125 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:35 smithi038 ceph-mon[31995]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:36.125 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:35 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:36.126 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:35 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:36.126 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:35 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:36.126 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:35 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:36.126 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:35 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:36.209 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:36 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:36.012+0000 7fc402e61200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:26:36.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:35 smithi053 ceph-mon[36496]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:36.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:35 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:36.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:35 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:36.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:35 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:36.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:35 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:36.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:35 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:36.492 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:36 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:36.124+0000 7fc7fa5e2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T15:26:36.492 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:36 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:36.221+0000 7fc7fa5e2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T15:26:36.959 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:36 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:36.674+0000 7fc402e61200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T15:26:37.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.363+0000 7fe1a70ca700 1 -- 172.21.15.38:0/949652503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a0104750 msgr2=0x7fe1a0104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.363+0000 7fe1a70ca700 1 --2- 172.21.15.38:0/949652503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a0104750 0x7fe1a0104b60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe194009990 tx=0x7fe194009ca0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.363+0000 7fe1a70ca700 1 -- 172.21.15.38:0/949652503 shutdown_connections 2024-05-21T15:26:37.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.363+0000 7fe1a70ca700 1 --2- 172.21.15.38:0/949652503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1a0101af0 0x7fe1a010b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.363+0000 7fe1a70ca700 1 --2- 172.21.15.38:0/949652503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a0104750 0x7fe1a0104b60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.363+0000 7fe1a70ca700 1 --2- 172.21.15.38:0/949652503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1a01011a0 0x7fe1a01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.363+0000 7fe1a70ca700 1 -- 172.21.15.38:0/949652503 >> 172.21.15.38:0/949652503 conn(0x7fe1a00fce90 msgr2=0x7fe1a00ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.363+0000 7fe1a70ca700 1 -- 172.21.15.38:0/949652503 shutdown_connections 2024-05-21T15:26:37.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.363+0000 7fe1a70ca700 1 -- 172.21.15.38:0/949652503 wait complete. 2024-05-21T15:26:37.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.363+0000 7fe1a70ca700 1 Processor -- start 2024-05-21T15:26:37.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.363+0000 7fe1a70ca700 1 -- start start 2024-05-21T15:26:37.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a70ca700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a01011a0 0x7fe1a0103d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a70ca700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1a0101af0 0x7fe1a01023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a70ca700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1a0104750 0x7fe1a0102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a70ca700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a010ed50 con 0x7fe1a01011a0 2024-05-21T15:26:37.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a70ca700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a010ebd0 con 0x7fe1a0104750 2024-05-21T15:26:37.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a70ca700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a010eed0 con 0x7fe1a0101af0 2024-05-21T15:26:37.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a4e66700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a01011a0 0x7fe1a0103d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a4e66700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a01011a0 0x7fe1a0103d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33816/0 (socket says 172.21.15.38:33816) 2024-05-21T15:26:37.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a4e66700 1 -- 172.21.15.38:0/506417013 learned_addr learned my addr 172.21.15.38:0/506417013 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:26:37.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe19ffff700 1 --2- 172.21.15.38:0/506417013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1a0101af0 0x7fe1a01023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a5667700 1 --2- 172.21.15.38:0/506417013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1a0104750 0x7fe1a0102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a4e66700 1 -- 172.21.15.38:0/506417013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1a0101af0 msgr2=0x7fe1a01023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a4e66700 1 --2- 172.21.15.38:0/506417013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1a0101af0 0x7fe1a01023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a4e66700 1 -- 172.21.15.38:0/506417013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1a0104750 msgr2=0x7fe1a0102910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a4e66700 1 --2- 172.21.15.38:0/506417013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1a0104750 0x7fe1a0102910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a4e66700 1 -- 172.21.15.38:0/506417013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a0103070 con 0x7fe1a01011a0 2024-05-21T15:26:37.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.364+0000 7fe1a4e66700 1 --2- 172.21.15.38:0/506417013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a01011a0 0x7fe1a0103d20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe194006010 tx=0x7fe1940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.365+0000 7fe19dffb700 1 -- 172.21.15.38:0/506417013 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194027070 con 0x7fe1a01011a0 2024-05-21T15:26:37.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.365+0000 7fe19dffb700 1 -- 172.21.15.38:0/506417013 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe194004150 con 0x7fe1a01011a0 2024-05-21T15:26:37.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.365+0000 7fe1a70ca700 1 -- 172.21.15.38:0/506417013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a01a3910 con 0x7fe1a01011a0 2024-05-21T15:26:37.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.365+0000 7fe19dffb700 1 -- 172.21.15.38:0/506417013 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194022e50 con 0x7fe1a01011a0 2024-05-21T15:26:37.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.365+0000 7fe1a70ca700 1 -- 172.21.15.38:0/506417013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a01a3cb0 con 0x7fe1a01011a0 2024-05-21T15:26:37.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.366+0000 7fe19dffb700 1 -- 172.21.15.38:0/506417013 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe19402a070 con 0x7fe1a01011a0 2024-05-21T15:26:37.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.366+0000 7fe1a70ca700 1 -- 172.21.15.38:0/506417013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a0050170 con 0x7fe1a01011a0 2024-05-21T15:26:37.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.366+0000 7fe19dffb700 1 --2- 172.21.15.38:0/506417013 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1900394d0 0x7fe19003b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.366+0000 7fe19dffb700 1 -- 172.21.15.38:0/506417013 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe19404d060 con 0x7fe1a01011a0 2024-05-21T15:26:37.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.366+0000 7fe19ffff700 1 --2- 172.21.15.38:0/506417013 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1900394d0 0x7fe19003b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.367+0000 7fe19ffff700 1 --2- 172.21.15.38:0/506417013 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1900394d0 0x7fe19003b9e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe1a01036d0 tx=0x7fe18c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.373+0000 7fe19dffb700 1 -- 172.21.15.38:0/506417013 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe194013070 con 0x7fe1a01011a0 2024-05-21T15:26:37.438 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:37 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:37.125+0000 7fc402e61200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T15:26:37.438 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:37 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:37.225+0000 7fc402e61200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T15:26:37.438 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:37 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:37.434+0000 7fc402e61200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T15:26:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:37.592+0000 7fe1a70ca700 1 -- 172.21.15.38:0/506417013 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe1a002e0c0 con 0x7fe1900394d0 2024-05-21T15:26:37.829 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:37 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:37.514+0000 7fc7fa5e2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T15:26:37.830 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:37 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:37.692+0000 7fc7fa5e2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T15:26:37.931 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:37.931 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:37.932 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:37.932 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:37.932 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:37.932 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:37.932 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:37.932 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:37.932 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:37.932 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:26:37.932 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:26:37.932 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:37.932 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:26:37.932 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:26:37.932 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:38.080 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:37 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:37.981+0000 7fc402e61200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T15:26:38.080 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:38 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:38.076+0000 7fc402e61200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T15:26:38.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:38.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:38.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:38.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:38.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:38.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:38.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:38.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:38.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:38.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:26:38.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:26:38.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:38.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:26:38.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:26:38.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:38.174 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:37 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:37.872+0000 7fc7fa5e2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T15:26:38.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:38.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:38.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:38.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:38.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:38.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:38.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:38.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:38.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:38.175 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:26:38.175 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:26:38.175 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:38.175 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:26:38.175 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:26:38.175 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:38.478 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:38 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:38.169+0000 7fc7fa5e2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T15:26:38.478 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:38 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:38.265+0000 7fc7fa5e2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T15:26:38.478 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:38 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:38.474+0000 7fc7fa5e2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T15:26:38.687 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:38 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:38.327+0000 7fc402e61200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T15:26:38.687 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:38 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:38.466+0000 7fc402e61200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T15:26:38.687 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:38 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:38.564+0000 7fc402e61200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T15:26:38.687 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:38 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:38.683+0000 7fc402e61200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T15:26:38.764 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:38 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:38.672+0000 7fc7fa5e2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:26:38.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:38 smithi053 ceph-mon[36496]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T15:26:38.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:38 smithi053 ceph-mon[36496]: Reconfiguring daemon mgr.x on smithi038 2024-05-21T15:26:38.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:38 smithi053 ceph-mon[36496]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:26:39.078 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:38 smithi038 ceph-mon[31995]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T15:26:39.078 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:38 smithi038 ceph-mon[31995]: Reconfiguring daemon mgr.x on smithi038 2024-05-21T15:26:39.078 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:38 smithi038 ceph-mon[31995]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:26:39.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:38 smithi045 ceph-mon[36409]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T15:26:39.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:38 smithi045 ceph-mon[36409]: Reconfiguring daemon mgr.x on smithi038 2024-05-21T15:26:39.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:38 smithi045 ceph-mon[36409]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:26:39.459 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:38 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:38.992+0000 7fc402e61200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T15:26:39.740 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:39 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:39.302+0000 7fc7fa5e2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T15:26:39.891 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:39 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:39.616+0000 7fc402e61200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T15:26:39.891 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:26:39 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z[38262]: 2024-05-21T15:26:39.838+0000 7fc402e61200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T15:26:39.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:39.932+0000 7fe19dffb700 1 -- 172.21.15.38:0/506417013 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7fe19400f980 con 0x7fe1a01011a0 2024-05-21T15:26:40.014 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:39 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:39.736+0000 7fc7fa5e2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T15:26:40.015 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:39 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:39.836+0000 7fc7fa5e2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T15:26:40.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:39 smithi045 ceph-mon[36409]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:40.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:39 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:40.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:39 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:40.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:39 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:40.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:39 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:40.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:39 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:40.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:39 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:40.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:39 smithi045 ceph-mon[36409]: Standby manager daemon z started 2024-05-21T15:26:40.016 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:39 smithi045 ceph-mon[36409]: from='mgr.? 172.21.15.53:0/3103166423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T15:26:40.016 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:39 smithi045 ceph-mon[36409]: from='mgr.? 172.21.15.53:0/3103166423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:26:40.016 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:39 smithi045 ceph-mon[36409]: from='mgr.? 172.21.15.53:0/3103166423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T15:26:40.016 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:39 smithi045 ceph-mon[36409]: from='mgr.? 172.21.15.53:0/3103166423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:26:40.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:39 smithi038 ceph-mon[31995]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:40.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:39 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:40.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:39 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:40.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:39 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:40.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:39 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:40.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:39 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:40.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:39 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:40.152 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:39 smithi038 ceph-mon[31995]: Standby manager daemon z started 2024-05-21T15:26:40.152 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:39 smithi038 ceph-mon[31995]: from='mgr.? 172.21.15.53:0/3103166423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T15:26:40.152 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:39 smithi038 ceph-mon[31995]: from='mgr.? 172.21.15.53:0/3103166423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:26:40.152 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:39 smithi038 ceph-mon[31995]: from='mgr.? 172.21.15.53:0/3103166423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T15:26:40.152 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:39 smithi038 ceph-mon[31995]: from='mgr.? 172.21.15.53:0/3103166423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:26:40.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:39 smithi053 ceph-mon[36496]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:40.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:39 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:40.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:39 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:40.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:39 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:40.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:39 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:40.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:39 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:40.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:39 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:40.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:39 smithi053 ceph-mon[36496]: Standby manager daemon z started 2024-05-21T15:26:40.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:39 smithi053 ceph-mon[36496]: from='mgr.? 172.21.15.53:0/3103166423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T15:26:40.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:39 smithi053 ceph-mon[36496]: from='mgr.? 172.21.15.53:0/3103166423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:26:40.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:39 smithi053 ceph-mon[36496]: from='mgr.? 172.21.15.53:0/3103166423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T15:26:40.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:39 smithi053 ceph-mon[36496]: from='mgr.? 172.21.15.53:0/3103166423' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:26:40.514 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:40 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:40.044+0000 7fc7fa5e2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T15:26:40.933 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:40 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:40.584+0000 7fc7fa5e2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T15:26:40.933 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:40 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:40.679+0000 7fc7fa5e2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T15:26:41.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:40 smithi053 ceph-mon[36496]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-21T15:26:41.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:40 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T15:26:41.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:40 smithi053 ceph-mon[36496]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:41.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:40 smithi045 ceph-mon[36409]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-21T15:26:41.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:40 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T15:26:41.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:40 smithi045 ceph-mon[36409]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:41.265 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:40 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:40.930+0000 7fc7fa5e2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T15:26:41.265 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:41 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:41.070+0000 7fc7fa5e2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T15:26:41.265 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:41 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:41.168+0000 7fc7fa5e2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T15:26:41.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:40 smithi038 ceph-mon[31995]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-21T15:26:41.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:40 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T15:26:41.279 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:40 smithi038 ceph-mon[31995]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:41.599 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:41 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:41.288+0000 7fc7fa5e2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T15:26:41.599 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:41 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:41.596+0000 7fc7fa5e2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T15:26:42.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:41 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4084862868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6fb6df5-af66-434f-9f35-f571243a0e39"}]: dispatch 2024-05-21T15:26:42.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:41 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4084862868' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6fb6df5-af66-434f-9f35-f571243a0e39"}]': finished 2024-05-21T15:26:42.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:41 smithi053 ceph-mon[36496]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T15:26:42.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:41 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:26:42.211 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:41 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3362480062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:26:42.223 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:41 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4084862868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6fb6df5-af66-434f-9f35-f571243a0e39"}]: dispatch 2024-05-21T15:26:42.223 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:41 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4084862868' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6fb6df5-af66-434f-9f35-f571243a0e39"}]': finished 2024-05-21T15:26:42.223 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:41 smithi045 ceph-mon[36409]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T15:26:42.223 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:41 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:26:42.223 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:41 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3362480062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:26:42.279 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:41 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4084862868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6fb6df5-af66-434f-9f35-f571243a0e39"}]: dispatch 2024-05-21T15:26:42.279 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:41 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4084862868' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6fb6df5-af66-434f-9f35-f571243a0e39"}]': finished 2024-05-21T15:26:42.279 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:41 smithi038 ceph-mon[31995]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T15:26:42.279 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:41 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:26:42.279 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:41 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3362480062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:26:42.514 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:42 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:42.220+0000 7fc7fa5e2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T15:26:42.514 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:26:42 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y[37815]: 2024-05-21T15:26:42.442+0000 7fc7fa5e2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T15:26:42.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:42.959+0000 7fe19dffb700 1 -- 172.21.15.38:0/506417013 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fe19401e8f0 con 0x7fe1a01011a0 2024-05-21T15:26:43.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:42 smithi053 ceph-mon[36496]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:43.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:42 smithi053 ceph-mon[36496]: Standby manager daemon y started 2024-05-21T15:26:43.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:42 smithi053 ceph-mon[36496]: from='mgr.? 172.21.15.45:0/3148016338' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T15:26:43.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:42 smithi053 ceph-mon[36496]: from='mgr.? 172.21.15.45:0/3148016338' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:26:43.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:42 smithi053 ceph-mon[36496]: from='mgr.? 172.21.15.45:0/3148016338' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T15:26:43.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:42 smithi053 ceph-mon[36496]: from='mgr.? 172.21.15.45:0/3148016338' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:26:43.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:42 smithi045 ceph-mon[36409]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:43.265 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:42 smithi045 ceph-mon[36409]: Standby manager daemon y started 2024-05-21T15:26:43.265 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:42 smithi045 ceph-mon[36409]: from='mgr.? 172.21.15.45:0/3148016338' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T15:26:43.265 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:42 smithi045 ceph-mon[36409]: from='mgr.? 172.21.15.45:0/3148016338' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:26:43.265 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:42 smithi045 ceph-mon[36409]: from='mgr.? 172.21.15.45:0/3148016338' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T15:26:43.265 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:42 smithi045 ceph-mon[36409]: from='mgr.? 172.21.15.45:0/3148016338' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:26:43.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:42 smithi038 ceph-mon[31995]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:43.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:42 smithi038 ceph-mon[31995]: Standby manager daemon y started 2024-05-21T15:26:43.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:42 smithi038 ceph-mon[31995]: from='mgr.? 172.21.15.45:0/3148016338' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T15:26:43.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:42 smithi038 ceph-mon[31995]: from='mgr.? 172.21.15.45:0/3148016338' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:26:43.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:42 smithi038 ceph-mon[31995]: from='mgr.? 172.21.15.45:0/3148016338' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T15:26:43.279 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:42 smithi038 ceph-mon[31995]: from='mgr.? 172.21.15.45:0/3148016338' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:26:44.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:43 smithi045 ceph-mon[36409]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-21T15:26:44.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:43 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T15:26:44.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:43 smithi038 ceph-mon[31995]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-21T15:26:44.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:43 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T15:26:44.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:43 smithi053 ceph-mon[36496]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-21T15:26:44.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:43 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T15:26:45.081 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:44 smithi038 ceph-mon[31995]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:45.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:44 smithi053 ceph-mon[36496]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:45.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:44 smithi045 ceph-mon[36409]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:47.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:47 smithi053 ceph-mon[36496]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:47.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:47 smithi045 ceph-mon[36409]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:47.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:47 smithi038 ceph-mon[31995]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:49.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:49 smithi053 ceph-mon[36496]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:49.460 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:49 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-21T15:26:49.460 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:49 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:49.460 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:49 smithi053 ceph-mon[36496]: Deploying daemon osd.0 on smithi038 2024-05-21T15:26:49.485 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:49 smithi038 ceph-mon[31995]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:49.485 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:49 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-21T15:26:49.485 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:49 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:49.485 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:49 smithi038 ceph-mon[31995]: Deploying daemon osd.0 on smithi038 2024-05-21T15:26:49.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:49 smithi045 ceph-mon[36409]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:49.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:49 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-21T15:26:49.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:49 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:49.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:49 smithi045 ceph-mon[36409]: Deploying daemon osd.0 on smithi038 2024-05-21T15:26:51.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:51 smithi053 ceph-mon[36496]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:51.489 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:51 smithi038 ceph-mon[31995]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:51.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:51 smithi045 ceph-mon[36409]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:53.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:53 smithi053 ceph-mon[36496]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:53.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:53 smithi038 ceph-mon[31995]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:53.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:53 smithi045 ceph-mon[36409]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:54.640 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:54.641 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:54.641 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:54.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:54.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:54.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:55.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:55.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:55.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:55.871 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:55 smithi038 ceph-mon[31995]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:55.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:55 smithi053 ceph-mon[36496]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:56.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:55 smithi045 ceph-mon[36409]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:57.146 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:57.146 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:57.146 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:57.146 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:57.146 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:57.146 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:56 smithi038 ceph-mon[31995]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:57.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:57.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:57.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:57.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:57.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:57.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:56 smithi053 ceph-mon[36496]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:57.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:57.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:57.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:57.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:57.265 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:57.265 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:56 smithi045 ceph-mon[36409]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:57.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:57.504+0000 7fe19dffb700 1 -- 172.21.15.38:0/506417013 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe18c00bdb0 con 0x7fe1900394d0 2024-05-21T15:26:57.508 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 0 on host 'smithi038' 2024-05-21T15:26:57.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:57.507+0000 7fe1a70ca700 1 -- 172.21.15.38:0/506417013 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1900394d0 msgr2=0x7fe19003b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:57.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:57.507+0000 7fe1a70ca700 1 --2- 172.21.15.38:0/506417013 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1900394d0 0x7fe19003b9e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe1a01036d0 tx=0x7fe18c00b410 comp rx=0 tx=0).stop 2024-05-21T15:26:57.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:57.507+0000 7fe1a70ca700 1 -- 172.21.15.38:0/506417013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a01011a0 msgr2=0x7fe1a0103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:57.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:57.507+0000 7fe1a70ca700 1 --2- 172.21.15.38:0/506417013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a01011a0 0x7fe1a0103d20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe194006010 tx=0x7fe1940047b0 comp rx=0 tx=0).stop 2024-05-21T15:26:57.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:57.507+0000 7fe1a70ca700 1 -- 172.21.15.38:0/506417013 shutdown_connections 2024-05-21T15:26:57.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:57.507+0000 7fe1a70ca700 1 --2- 172.21.15.38:0/506417013 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1900394d0 0x7fe19003b9e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:57.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:57.507+0000 7fe1a70ca700 1 --2- 172.21.15.38:0/506417013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1a0104750 0x7fe1a0102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:57.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:57.507+0000 7fe1a70ca700 1 --2- 172.21.15.38:0/506417013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1a01011a0 0x7fe1a0103d20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:57.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:57.507+0000 7fe1a70ca700 1 --2- 172.21.15.38:0/506417013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1a0101af0 0x7fe1a01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:57.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:57.507+0000 7fe1a70ca700 1 -- 172.21.15.38:0/506417013 >> 172.21.15.38:0/506417013 conn(0x7fe1a00fce90 msgr2=0x7fe1a01076f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:57.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:57.507+0000 7fe1a70ca700 1 -- 172.21.15.38:0/506417013 shutdown_connections 2024-05-21T15:26:57.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:26:57.507+0000 7fe1a70ca700 1 -- 172.21.15.38:0/506417013 wait complete. 2024-05-21T15:26:58.085 DEBUG:teuthology.orchestra.run.smithi038:osd.0> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.0.service 2024-05-21T15:26:58.088 INFO:tasks.cephadm:Deploying osd.1 on smithi045 with /dev/vg_nvme/lv_4... 2024-05-21T15:26:58.088 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T15:26:58.341 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:26:58.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:58 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:58.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:58 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:58.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:58 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:58.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:58 smithi053 ceph-mon[36496]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-21T15:26:58.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:58 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:58.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:58 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:58.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:58 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:58.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:58 smithi045 ceph-mon[36409]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-21T15:26:58.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:58 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:58.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:58 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:26:58.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:58 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:58.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:58 smithi038 ceph-mon[31995]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-21T15:26:59.682 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:59 smithi045 ceph-mon[36409]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:59.683 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:59 smithi045 ceph-mon[36409]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T15:26:59.683 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:59 smithi045 ceph-mon[36409]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T15:26:59.683 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:59 smithi045 ceph-mon[36409]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-05-21T15:26:59.683 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:26:59 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:26:59.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:59 smithi038 ceph-mon[31995]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:59.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:59 smithi038 ceph-mon[31995]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T15:26:59.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:59 smithi038 ceph-mon[31995]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T15:26:59.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:59 smithi038 ceph-mon[31995]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-05-21T15:26:59.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:26:59 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:26:59.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:59 smithi053 ceph-mon[36496]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:26:59.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:59 smithi053 ceph-mon[36496]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T15:26:59.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:59 smithi053 ceph-mon[36496]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T15:26:59.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:59 smithi053 ceph-mon[36496]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-05-21T15:26:59.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:26:59 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:27:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T15:27:00.254 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T15:27:00.254 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-05-21T15:27:00.254 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-05-21T15:27:00.254 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0084805 s, 1.2 GB/s 2024-05-21T15:27:00.254 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-05-21T15:27:00.278 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:26:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-0[46982]: 2024-05-21T15:26:59.922+0000 7fe72c325700 -1 osd.0 0 waiting for initial osdmap 2024-05-21T15:27:00.279 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:26:59 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-0[46982]: 2024-05-21T15:26:59.928+0000 7fe7240d2700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:27:00.652 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:00 smithi038 ceph-mon[31995]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-05-21T15:27:00.652 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:00 smithi038 ceph-mon[31995]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T15:27:00.652 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:27:00.652 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:27:00.652 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:00 smithi038 ceph-mon[31995]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' 2024-05-21T15:27:00.694 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:00 smithi045 ceph-mon[36409]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-05-21T15:27:00.694 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:00 smithi045 ceph-mon[36409]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T15:27:00.694 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:27:00.694 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:27:00.694 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:00 smithi045 ceph-mon[36409]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' 2024-05-21T15:27:00.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:00 smithi053 ceph-mon[36496]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-05-21T15:27:00.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:00 smithi053 ceph-mon[36496]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T15:27:00.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:27:00.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:27:00.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:00 smithi053 ceph-mon[36496]: from='osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282]' entity='osd.0' 2024-05-21T15:27:00.978 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_4 2024-05-21T15:27:01.230 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:27:01.596 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:01 smithi045 ceph-mon[36409]: purged_snaps scrub starts 2024-05-21T15:27:01.596 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:01 smithi045 ceph-mon[36409]: purged_snaps scrub ok 2024-05-21T15:27:01.596 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:01 smithi045 ceph-mon[36409]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:27:01.596 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:01 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:27:01.596 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:01 smithi045 ceph-mon[36409]: osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282] boot 2024-05-21T15:27:01.596 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:01 smithi045 ceph-mon[36409]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T15:27:01.597 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:01 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:27:01.597 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:01 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:01.597 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:01 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:01.597 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:01 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:27:01.597 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:01 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:01.597 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:01 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:01.597 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:01 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:01.597 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:01 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:01.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:01 smithi038 ceph-mon[31995]: purged_snaps scrub starts 2024-05-21T15:27:01.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:01 smithi038 ceph-mon[31995]: purged_snaps scrub ok 2024-05-21T15:27:01.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:01 smithi038 ceph-mon[31995]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:27:01.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:01 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:27:01.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:01 smithi038 ceph-mon[31995]: osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282] boot 2024-05-21T15:27:01.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:01 smithi038 ceph-mon[31995]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T15:27:01.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:01 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:27:01.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:01 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:01.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:01 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:01.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:01 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:27:01.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:01 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:01.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:01 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:01.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:01 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:01.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:01 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:01.861 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:01 smithi053 ceph-mon[36496]: purged_snaps scrub starts 2024-05-21T15:27:01.861 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:01 smithi053 ceph-mon[36496]: purged_snaps scrub ok 2024-05-21T15:27:01.861 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:01 smithi053 ceph-mon[36496]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:27:01.861 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:01 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:27:01.861 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:01 smithi053 ceph-mon[36496]: osd.0 [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282] boot 2024-05-21T15:27:01.861 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:01 smithi053 ceph-mon[36496]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T15:27:01.861 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:01 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:27:01.861 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:01 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:01.861 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:01 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:01.862 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:01 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:27:01.862 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:01 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:01.862 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:01 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:01.862 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:01 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:01.862 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:01 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:02.636 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:02 smithi045 ceph-mon[36409]: Detected new or changed devices on smithi038 2024-05-21T15:27:02.636 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:02 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi038 to 17093M 2024-05-21T15:27:02.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.668+0000 7febcbde5700 1 -- 172.21.15.45:0/2632165207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4103b90 msgr2=0x7febc410a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:02.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.668+0000 7febcbde5700 1 --2- 172.21.15.45:0/2632165207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4103b90 0x7febc410a620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7febbc00b0a0 tx=0x7febbc00b3b0 comp rx=0 tx=0).stop 2024-05-21T15:27:02.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.668+0000 7febcbde5700 1 -- 172.21.15.45:0/2632165207 shutdown_connections 2024-05-21T15:27:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.668+0000 7febcbde5700 1 --2- 172.21.15.45:0/2632165207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4103b90 0x7febc410a620 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.668+0000 7febcbde5700 1 --2- 172.21.15.45:0/2632165207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febc4102650 0x7febc4102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.668+0000 7febcbde5700 1 --2- 172.21.15.45:0/2632165207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febc4103030 0x7febc4103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.668+0000 7febcbde5700 1 -- 172.21.15.45:0/2632165207 >> 172.21.15.45:0/2632165207 conn(0x7febc40fcea0 msgr2=0x7febc40ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.669+0000 7febcbde5700 1 -- 172.21.15.45:0/2632165207 shutdown_connections 2024-05-21T15:27:02.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.669+0000 7febcbde5700 1 -- 172.21.15.45:0/2632165207 wait complete. 2024-05-21T15:27:02.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.669+0000 7febcbde5700 1 Processor -- start 2024-05-21T15:27:02.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.669+0000 7febcbde5700 1 -- start start 2024-05-21T15:27:02.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.669+0000 7febcbde5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febc4102650 0x7febc41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:02.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.669+0000 7febcbde5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4103030 0x7febc4199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:02.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.669+0000 7febcbde5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febc4103b90 0x7febc41a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:02.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febcbde5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7febc410dca0 con 0x7febc4102650 2024-05-21T15:27:02.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febcbde5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7febc410db20 con 0x7febc4103030 2024-05-21T15:27:02.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febcbde5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7febc410de20 con 0x7febc4103b90 2024-05-21T15:27:02.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febc9380700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4103030 0x7febc4199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:02.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febc9380700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4103030 0x7febc4199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46032/0 (socket says 172.21.15.45:46032) 2024-05-21T15:27:02.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febc9380700 1 -- 172.21.15.45:0/2086734554 learned_addr learned my addr 172.21.15.45:0/2086734554 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-21T15:27:02.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febca382700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febc4103b90 0x7febc41a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:02.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febc9b81700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febc4102650 0x7febc41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:02.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febc9380700 1 -- 172.21.15.45:0/2086734554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febc4103b90 msgr2=0x7febc41a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:02.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febc9380700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febc4103b90 0x7febc41a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:02.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febc9380700 1 -- 172.21.15.45:0/2086734554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febc4102650 msgr2=0x7febc41997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:02.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febc9380700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febc4102650 0x7febc41997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:02.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febc9380700 1 -- 172.21.15.45:0/2086734554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc41a1340 con 0x7febc4103030 2024-05-21T15:27:02.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febc9b81700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febc4102650 0x7febc41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:27:02.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febc9380700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4103030 0x7febc4199d20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7febb400e9a0 tx=0x7febb400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:02.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febbaffd700 1 -- 172.21.15.45:0/2086734554 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb400cbd0 con 0x7febc4103030 2024-05-21T15:27:02.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febcbde5700 1 -- 172.21.15.45:0/2086734554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc41a1630 con 0x7febc4103030 2024-05-21T15:27:02.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febbaffd700 1 -- 172.21.15.45:0/2086734554 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febb400cd70 con 0x7febc4103030 2024-05-21T15:27:02.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.670+0000 7febbaffd700 1 -- 172.21.15.45:0/2086734554 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4018750 con 0x7febc4103030 2024-05-21T15:27:02.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.671+0000 7febcbde5700 1 -- 172.21.15.45:0/2086734554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc41a1bc0 con 0x7febc4103030 2024-05-21T15:27:02.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.672+0000 7febcbde5700 1 -- 172.21.15.45:0/2086734554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc4050170 con 0x7febc4103030 2024-05-21T15:27:02.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.674+0000 7febbaffd700 1 -- 172.21.15.45:0/2086734554 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7febb4010430 con 0x7febc4103030 2024-05-21T15:27:02.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.674+0000 7febbaffd700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7febb00a2b90 0x7febb00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:02.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.674+0000 7febbaffd700 1 -- 172.21.15.45:0/2086734554 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7febb40cdd00 con 0x7febc4103030 2024-05-21T15:27:02.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.675+0000 7febc9b81700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7febb00a2b90 0x7febb00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:02.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.675+0000 7febc9b81700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7febb00a2b90 0x7febb00a50a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7febc406af40 tx=0x7febc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:02.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.678+0000 7febbaffd700 1 -- 172.21.15.45:0/2086734554 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7febb409c6d0 con 0x7febc4103030 2024-05-21T15:27:02.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:02 smithi038 ceph-mon[31995]: Detected new or changed devices on smithi038 2024-05-21T15:27:02.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:02 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi038 to 17093M 2024-05-21T15:27:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:02.892+0000 7febcbde5700 1 -- 172.21.15.45:0/2086734554 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7febc41a1ee0 con 0x7febb00a2b90 2024-05-21T15:27:02.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:02 smithi053 ceph-mon[36496]: Detected new or changed devices on smithi038 2024-05-21T15:27:02.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:02 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi038 to 17093M 2024-05-21T15:27:03.746 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:03 smithi045 ceph-mon[36409]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:03.746 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:03 smithi045 ceph-mon[36409]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T15:27:03.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:03 smithi045 ceph-mon[36409]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:27:03.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:27:03.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:27:03.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:03.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:03 smithi038 ceph-mon[31995]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:03.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:03 smithi038 ceph-mon[31995]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T15:27:03.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:03 smithi038 ceph-mon[31995]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:27:03.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:27:03.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:27:03.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:03.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:03 smithi053 ceph-mon[36496]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:03.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:03 smithi053 ceph-mon[36496]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T15:27:03.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:03 smithi053 ceph-mon[36496]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:27:03.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:27:03.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:27:03.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:05.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:05 smithi045 ceph-mon[36409]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:05.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:05 smithi038 ceph-mon[31995]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:05.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:05 smithi053 ceph-mon[36496]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:06.768 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:06 smithi045 ceph-mon[36409]: from='client.? 172.21.15.45:0/2370095789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61df021e-6d15-4875-baab-e7ff8c388a17"}]: dispatch 2024-05-21T15:27:06.768 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:06 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61df021e-6d15-4875-baab-e7ff8c388a17"}]: dispatch 2024-05-21T15:27:06.768 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:06 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61df021e-6d15-4875-baab-e7ff8c388a17"}]': finished 2024-05-21T15:27:06.768 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:06 smithi045 ceph-mon[36409]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T15:27:06.768 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:06 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:06.768 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:06 smithi045 ceph-mon[36409]: from='client.? 172.21.15.45:0/2849306370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:27:06.788 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:06 smithi038 ceph-mon[31995]: from='client.? 172.21.15.45:0/2370095789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61df021e-6d15-4875-baab-e7ff8c388a17"}]: dispatch 2024-05-21T15:27:06.788 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:06 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61df021e-6d15-4875-baab-e7ff8c388a17"}]: dispatch 2024-05-21T15:27:06.788 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:06 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61df021e-6d15-4875-baab-e7ff8c388a17"}]': finished 2024-05-21T15:27:06.788 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:06 smithi038 ceph-mon[31995]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T15:27:06.788 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:06 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:06.788 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:06 smithi038 ceph-mon[31995]: from='client.? 172.21.15.45:0/2849306370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:27:06.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:06 smithi053 ceph-mon[36496]: from='client.? 172.21.15.45:0/2370095789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61df021e-6d15-4875-baab-e7ff8c388a17"}]: dispatch 2024-05-21T15:27:06.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:06 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61df021e-6d15-4875-baab-e7ff8c388a17"}]: dispatch 2024-05-21T15:27:06.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:06 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61df021e-6d15-4875-baab-e7ff8c388a17"}]': finished 2024-05-21T15:27:06.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:06 smithi053 ceph-mon[36496]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T15:27:06.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:06 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:06.961 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:06 smithi053 ceph-mon[36496]: from='client.? 172.21.15.45:0/2849306370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:27:08.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:07 smithi045 ceph-mon[36409]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:08.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:07 smithi038 ceph-mon[31995]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:08.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:07 smithi053 ceph-mon[36496]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:10.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:09 smithi045 ceph-mon[36409]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:10.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:09 smithi038 ceph-mon[31995]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:10.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:09 smithi053 ceph-mon[36496]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:12.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:11 smithi045 ceph-mon[36409]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:12.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:11 smithi038 ceph-mon[31995]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:12.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:11 smithi053 ceph-mon[36496]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:13.788 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:13 smithi045 ceph-mon[36409]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:13.788 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:13 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-21T15:27:13.788 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:13 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:13.789 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:13 smithi045 ceph-mon[36409]: Deploying daemon osd.1 on smithi045 2024-05-21T15:27:14.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:13 smithi038 ceph-mon[31995]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:14.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:13 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-21T15:27:14.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:13 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:14.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:13 smithi038 ceph-mon[31995]: Deploying daemon osd.1 on smithi045 2024-05-21T15:27:14.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:13 smithi053 ceph-mon[36496]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:14.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:13 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-21T15:27:14.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:13 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:14.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:13 smithi053 ceph-mon[36496]: Deploying daemon osd.1 on smithi045 2024-05-21T15:27:15.949 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:15 smithi045 ceph-mon[36409]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:16.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:15 smithi038 ceph-mon[31995]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:16.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:15 smithi053 ceph-mon[36496]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:17.963 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:17 smithi045 ceph-mon[36409]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:18.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:17 smithi038 ceph-mon[31995]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:18.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:17 smithi053 ceph-mon[36496]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:19.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:19 smithi045 ceph-mon[36409]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:19.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:19.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:19.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:27:19.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:19 smithi053 ceph-mon[36496]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:19.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:19.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:19.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:27:19.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:19 smithi038 ceph-mon[31995]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:19.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:19.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:19.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:27:21.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:21.663+0000 7febbaffd700 1 -- 172.21.15.45:0/2086734554 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7febc41a1ee0 con 0x7febb00a2b90 2024-05-21T15:27:21.667 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 1 on host 'smithi045' 2024-05-21T15:27:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:21.665+0000 7febcbde5700 1 -- 172.21.15.45:0/2086734554 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7febb00a2b90 msgr2=0x7febb00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:21.665+0000 7febcbde5700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7febb00a2b90 0x7febb00a50a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7febc406af40 tx=0x7febc0009260 comp rx=0 tx=0).stop 2024-05-21T15:27:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:21.665+0000 7febcbde5700 1 -- 172.21.15.45:0/2086734554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4103030 msgr2=0x7febc4199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:21.665+0000 7febcbde5700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4103030 0x7febc4199d20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7febb400e9a0 tx=0x7febb400ecb0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:21.665+0000 7febcbde5700 1 -- 172.21.15.45:0/2086734554 shutdown_connections 2024-05-21T15:27:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:21.665+0000 7febcbde5700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7febb00a2b90 0x7febb00a50a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:21.665+0000 7febcbde5700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febc4103b90 0x7febc41a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:21.665+0000 7febcbde5700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febc4102650 0x7febc41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:21.665+0000 7febcbde5700 1 --2- 172.21.15.45:0/2086734554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4103030 0x7febc4199d20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:21.665+0000 7febcbde5700 1 -- 172.21.15.45:0/2086734554 >> 172.21.15.45:0/2086734554 conn(0x7febc40fcea0 msgr2=0x7febc41012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:21.665+0000 7febcbde5700 1 -- 172.21.15.45:0/2086734554 shutdown_connections 2024-05-21T15:27:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:27:21.665+0000 7febcbde5700 1 -- 172.21.15.45:0/2086734554 wait complete. 2024-05-21T15:27:21.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:21 smithi053 ceph-mon[36496]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:21.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:21.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:21.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:21.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:21.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:27:22.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:21 smithi045 ceph-mon[36409]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:22.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:21 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:22.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:21 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:22.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:21 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:22.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:21 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:22.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:21 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:22.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:21 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:22.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:21 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:22.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:21 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:27:22.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:21 smithi038 ceph-mon[31995]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:22.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:22.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:22.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:22.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:22.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:22.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:22.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:22.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:27:22.306 DEBUG:teuthology.orchestra.run.smithi045:osd.1> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.1.service 2024-05-21T15:27:22.309 INFO:tasks.cephadm:Deploying osd.2 on smithi053 with /dev/vg_nvme/lv_4... 2024-05-21T15:27:22.309 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T15:27:22.567 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:27:22.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:22 smithi053 ceph-mon[36496]: from='osd.1 [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T15:27:22.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:22 smithi053 ceph-mon[36496]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T15:27:23.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:22 smithi045 ceph-mon[36409]: from='osd.1 [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T15:27:23.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:22 smithi045 ceph-mon[36409]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T15:27:23.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:22 smithi038 ceph-mon[31995]: from='osd.1 [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T15:27:23.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:22 smithi038 ceph-mon[31995]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T15:27:23.752 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:23 smithi045 ceph-mon[36409]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:23.752 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:23 smithi045 ceph-mon[36409]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T15:27:23.752 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:23 smithi045 ceph-mon[36409]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T15:27:23.752 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:23 smithi045 ceph-mon[36409]: from='osd.1 [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:27:23.752 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:23.752 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:23 smithi045 ceph-mon[36409]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:27:23.752 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:27:23 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-1[41015]: 2024-05-21T15:27:23.708+0000 7f5a09b6a700 -1 osd.1 0 waiting for initial osdmap 2024-05-21T15:27:23.753 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:27:23 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-1[41015]: 2024-05-21T15:27:23.714+0000 7f5a0591f700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:27:24.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:23 smithi038 ceph-mon[31995]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:24.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:23 smithi038 ceph-mon[31995]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T15:27:24.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:23 smithi038 ceph-mon[31995]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T15:27:24.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:23 smithi038 ceph-mon[31995]: from='osd.1 [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:27:24.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:24.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:23 smithi038 ceph-mon[31995]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:27:24.034 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:23 smithi053 ceph-mon[36496]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:24.034 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:23 smithi053 ceph-mon[36496]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T15:27:24.034 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:23 smithi053 ceph-mon[36496]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T15:27:24.034 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:23 smithi053 ceph-mon[36496]: from='osd.1 [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:27:24.034 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:24.034 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:23 smithi053 ceph-mon[36496]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:27:24.435 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T15:27:24.435 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T15:27:24.436 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-05-21T15:27:24.436 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-05-21T15:27:24.436 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00827922 s, 1.3 GB/s 2024-05-21T15:27:24.436 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-05-21T15:27:24.843 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:24 smithi045 ceph-mon[36409]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T15:27:24.843 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:24 smithi045 ceph-mon[36409]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T15:27:24.843 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:24.843 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:24.972 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:24 smithi053 ceph-mon[36496]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T15:27:24.972 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:24 smithi053 ceph-mon[36496]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T15:27:24.972 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:24.972 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:24.973 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_4 2024-05-21T15:27:25.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:24 smithi038 ceph-mon[31995]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T15:27:25.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:24 smithi038 ceph-mon[31995]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T15:27:25.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:25.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:25.227 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:27:25.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:25 smithi053 ceph-mon[36496]: purged_snaps scrub starts 2024-05-21T15:27:25.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:25 smithi053 ceph-mon[36496]: purged_snaps scrub ok 2024-05-21T15:27:25.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:25 smithi053 ceph-mon[36496]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:25.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:25.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:25 smithi053 ceph-mon[36496]: osd.1 [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940] boot 2024-05-21T15:27:25.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:25 smithi053 ceph-mon[36496]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T15:27:25.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:25.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:25.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:25.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:27:25.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:25.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:25.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:25.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:26.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:25 smithi045 ceph-mon[36409]: purged_snaps scrub starts 2024-05-21T15:27:26.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:25 smithi045 ceph-mon[36409]: purged_snaps scrub ok 2024-05-21T15:27:26.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:25 smithi045 ceph-mon[36409]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:26.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:26.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:25 smithi045 ceph-mon[36409]: osd.1 [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940] boot 2024-05-21T15:27:26.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:25 smithi045 ceph-mon[36409]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T15:27:26.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:26.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:26.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:26.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:27:26.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:26.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:26.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:26.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:26.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:25 smithi038 ceph-mon[31995]: purged_snaps scrub starts 2024-05-21T15:27:26.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:25 smithi038 ceph-mon[31995]: purged_snaps scrub ok 2024-05-21T15:27:26.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:25 smithi038 ceph-mon[31995]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:27:26.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:26.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:25 smithi038 ceph-mon[31995]: osd.1 [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940] boot 2024-05-21T15:27:26.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:25 smithi038 ceph-mon[31995]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T15:27:26.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:27:26.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:26.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:26.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:27:26.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:26.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:26.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:26.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:26.778 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.775+0000 7fab7d40c700 1 -- 172.21.15.53:0/182171373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab78103ba0 msgr2=0x7fab7810a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:26.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.775+0000 7fab7d40c700 1 --2- 172.21.15.53:0/182171373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab78103ba0 0x7fab7810a450 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fab7000b0a0 tx=0x7fab7000b3b0 comp rx=0 tx=0).stop 2024-05-21T15:27:26.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.775+0000 7fab7d40c700 1 -- 172.21.15.53:0/182171373 shutdown_connections 2024-05-21T15:27:26.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.775+0000 7fab7d40c700 1 --2- 172.21.15.53:0/182171373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab78103ba0 0x7fab7810a450 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:26.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.775+0000 7fab7d40c700 1 --2- 172.21.15.53:0/182171373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab781020d0 0x7fab781024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:26.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.775+0000 7fab7d40c700 1 --2- 172.21.15.53:0/182171373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab78103200 0x7fab78103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:26.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.775+0000 7fab7d40c700 1 -- 172.21.15.53:0/182171373 >> 172.21.15.53:0/182171373 conn(0x7fab780fd710 msgr2=0x7fab780ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:26.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.775+0000 7fab7d40c700 1 -- 172.21.15.53:0/182171373 shutdown_connections 2024-05-21T15:27:26.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.775+0000 7fab7d40c700 1 -- 172.21.15.53:0/182171373 wait complete. 2024-05-21T15:27:26.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.775+0000 7fab7d40c700 1 Processor -- start 2024-05-21T15:27:26.779 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab7d40c700 1 -- start start 2024-05-21T15:27:26.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab7d40c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab781020d0 0x7fab780752f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:26.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab7d40c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab78103200 0x7fab7819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:26.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab7d40c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab78103ba0 0x7fab781a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:26.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab7d40c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fab7810dcc0 con 0x7fab78103ba0 2024-05-21T15:27:26.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab7d40c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab7810db40 con 0x7fab781020d0 2024-05-21T15:27:26.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab7d40c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fab7810de40 con 0x7fab78103200 2024-05-21T15:27:26.781 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab767fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab78103200 0x7fab7819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:26.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab767fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab78103200 0x7fab7819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57380/0 (socket says 172.21.15.53:57380) 2024-05-21T15:27:26.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab767fc700 1 -- 172.21.15.53:0/2001859375 learned_addr learned my addr 172.21.15.53:0/2001859375 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-21T15:27:26.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab76ffd700 1 --2- 172.21.15.53:0/2001859375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab781020d0 0x7fab780752f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:26.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab777fe700 1 --2- 172.21.15.53:0/2001859375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab78103ba0 0x7fab781a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:26.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab767fc700 1 -- 172.21.15.53:0/2001859375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab781020d0 msgr2=0x7fab780752f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:26.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab767fc700 1 --2- 172.21.15.53:0/2001859375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab781020d0 0x7fab780752f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:26.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab767fc700 1 -- 172.21.15.53:0/2001859375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab78103ba0 msgr2=0x7fab781a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:26.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab767fc700 1 --2- 172.21.15.53:0/2001859375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab78103ba0 0x7fab781a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:26.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab767fc700 1 -- 172.21.15.53:0/2001859375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab781a1360 con 0x7fab78103200 2024-05-21T15:27:26.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.776+0000 7fab767fc700 1 --2- 172.21.15.53:0/2001859375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab78103200 0x7fab7819e5a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00d950 tx=0x7fab6c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:26.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.777+0000 7fab5ffff700 1 -- 172.21.15.53:0/2001859375 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c016070 con 0x7fab78103200 2024-05-21T15:27:26.782 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.777+0000 7fab5ffff700 1 -- 172.21.15.53:0/2001859375 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab6c009e20 con 0x7fab78103200 2024-05-21T15:27:26.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.777+0000 7fab7d40c700 1 -- 172.21.15.53:0/2001859375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab781a1650 con 0x7fab78103200 2024-05-21T15:27:26.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.777+0000 7fab5ffff700 1 -- 172.21.15.53:0/2001859375 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c00f620 con 0x7fab78103200 2024-05-21T15:27:26.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.777+0000 7fab7d40c700 1 -- 172.21.15.53:0/2001859375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab781a1be0 con 0x7fab78103200 2024-05-21T15:27:26.783 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.778+0000 7fab7d40c700 1 -- 172.21.15.53:0/2001859375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab78050170 con 0x7fab78103200 2024-05-21T15:27:26.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.779+0000 7fab5ffff700 1 -- 172.21.15.53:0/2001859375 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fab6c019070 con 0x7fab78103200 2024-05-21T15:27:26.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.780+0000 7fab5ffff700 1 --2- 172.21.15.53:0/2001859375 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fab600a2c90 0x7fab600a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:26.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.780+0000 7fab5ffff700 1 -- 172.21.15.53:0/2001859375 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fab6c0cd3c0 con 0x7fab78103200 2024-05-21T15:27:26.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.780+0000 7fab76ffd700 1 --2- 172.21.15.53:0/2001859375 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fab600a2c90 0x7fab600a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:26.784 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.781+0000 7fab76ffd700 1 --2- 172.21.15.53:0/2001859375 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fab600a2c90 0x7fab600a51a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab78103100 tx=0x7fab6800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:26.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.784+0000 7fab5ffff700 1 -- 172.21.15.53:0/2001859375 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fab6c09bad0 con 0x7fab78103200 2024-05-21T15:27:27.000 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:26.997+0000 7fab7d40c700 1 -- 172.21.15.53:0/2001859375 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fab78062a80 con 0x7fab600a2c90 2024-05-21T15:27:27.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:26 smithi053 ceph-mon[36496]: Detected new or changed devices on smithi045 2024-05-21T15:27:27.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:26 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi045 to 17093M 2024-05-21T15:27:27.001 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:26 smithi053 ceph-mon[36496]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T15:27:27.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:26 smithi045 ceph-mon[36409]: Detected new or changed devices on smithi045 2024-05-21T15:27:27.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:26 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi045 to 17093M 2024-05-21T15:27:27.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:26 smithi045 ceph-mon[36409]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T15:27:27.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:26 smithi038 ceph-mon[31995]: Detected new or changed devices on smithi045 2024-05-21T15:27:27.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:26 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi045 to 17093M 2024-05-21T15:27:27.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:26 smithi038 ceph-mon[31995]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T15:27:27.908 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:27 smithi053 ceph-mon[36496]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:27.908 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:27:27.908 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:27:27.908 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:28.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:27 smithi045 ceph-mon[36409]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:28.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:27:28.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:27:28.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:28.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:27 smithi038 ceph-mon[31995]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:28.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:27:28.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:27:28.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:28.755 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:28 smithi053 ceph-mon[36496]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:27:29.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:28 smithi045 ceph-mon[36409]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:27:29.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:28 smithi038 ceph-mon[31995]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:27:30.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:29 smithi053 ceph-mon[36496]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:30.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:29 smithi053 ceph-mon[36496]: from='client.? 172.21.15.53:0/2795118833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dba2ab3-c388-47ea-b756-68611a94f77b"}]: dispatch 2024-05-21T15:27:30.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:29 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dba2ab3-c388-47ea-b756-68611a94f77b"}]: dispatch 2024-05-21T15:27:30.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:29 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7dba2ab3-c388-47ea-b756-68611a94f77b"}]': finished 2024-05-21T15:27:30.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:29 smithi053 ceph-mon[36496]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T15:27:30.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:29 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:30.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:29 smithi045 ceph-mon[36409]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:30.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:29 smithi045 ceph-mon[36409]: from='client.? 172.21.15.53:0/2795118833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dba2ab3-c388-47ea-b756-68611a94f77b"}]: dispatch 2024-05-21T15:27:30.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:29 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dba2ab3-c388-47ea-b756-68611a94f77b"}]: dispatch 2024-05-21T15:27:30.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:29 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7dba2ab3-c388-47ea-b756-68611a94f77b"}]': finished 2024-05-21T15:27:30.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:29 smithi045 ceph-mon[36409]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T15:27:30.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:29 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:30.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:29 smithi038 ceph-mon[31995]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:30.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:29 smithi038 ceph-mon[31995]: from='client.? 172.21.15.53:0/2795118833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dba2ab3-c388-47ea-b756-68611a94f77b"}]: dispatch 2024-05-21T15:27:30.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:29 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dba2ab3-c388-47ea-b756-68611a94f77b"}]: dispatch 2024-05-21T15:27:30.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:29 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7dba2ab3-c388-47ea-b756-68611a94f77b"}]': finished 2024-05-21T15:27:30.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:29 smithi038 ceph-mon[31995]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T15:27:30.029 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:29 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:31.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:30 smithi045 ceph-mon[36409]: from='client.? 172.21.15.53:0/3138604794' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:27:31.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:30 smithi038 ceph-mon[31995]: from='client.? 172.21.15.53:0/3138604794' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:27:31.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:30 smithi053 ceph-mon[36496]: from='client.? 172.21.15.53:0/3138604794' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:27:32.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:31 smithi045 ceph-mon[36409]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:32.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:31 smithi038 ceph-mon[31995]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:32.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:31 smithi053 ceph-mon[36496]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:33.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:33 smithi053 ceph-mon[36496]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:34.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:33 smithi045 ceph-mon[36409]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:34.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:33 smithi038 ceph-mon[31995]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:36.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:35 smithi038 ceph-mon[31995]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:36.105 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:35 smithi053 ceph-mon[36496]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:36.266 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:35 smithi045 ceph-mon[36409]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:36.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:36 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-21T15:27:36.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:36 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:37.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-21T15:27:37.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:37.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:36 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-21T15:27:37.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:36 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:37.889 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:37 smithi053 ceph-mon[36496]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:37.889 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:37 smithi053 ceph-mon[36496]: Deploying daemon osd.2 on smithi053 2024-05-21T15:27:38.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:37 smithi038 ceph-mon[31995]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:38.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:37 smithi038 ceph-mon[31995]: Deploying daemon osd.2 on smithi053 2024-05-21T15:27:38.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:37 smithi045 ceph-mon[36409]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:38.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:37 smithi045 ceph-mon[36409]: Deploying daemon osd.2 on smithi053 2024-05-21T15:27:39.913 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:39 smithi053 ceph-mon[36496]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:40.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:39 smithi045 ceph-mon[36409]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:40.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:39 smithi038 ceph-mon[31995]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:42.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:41 smithi053 ceph-mon[36496]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:42.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:41 smithi045 ceph-mon[36409]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:42.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:41 smithi038 ceph-mon[31995]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:42.992 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:42 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:42.992 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:42 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:42.993 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:42 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:27:42.993 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:42 smithi053 ceph-mon[36496]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:43.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:42 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:43.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:42 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:43.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:42 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:27:43.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:42 smithi045 ceph-mon[36409]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:43.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:42 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:43.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:42 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:43.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:42 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:27:43.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:42 smithi038 ceph-mon[31995]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:45.719 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:45.715+0000 7fab5ffff700 1 -- 172.21.15.53:0/2001859375 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fab78062a80 con 0x7fab600a2c90 2024-05-21T15:27:45.720 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 2 on host 'smithi053' 2024-05-21T15:27:45.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:45.718+0000 7fab7d40c700 1 -- 172.21.15.53:0/2001859375 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fab600a2c90 msgr2=0x7fab600a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:45.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:45.718+0000 7fab7d40c700 1 --2- 172.21.15.53:0/2001859375 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fab600a2c90 0x7fab600a51a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab78103100 tx=0x7fab6800b410 comp rx=0 tx=0).stop 2024-05-21T15:27:45.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:45.718+0000 7fab7d40c700 1 -- 172.21.15.53:0/2001859375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab78103200 msgr2=0x7fab7819e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:45.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:45.718+0000 7fab7d40c700 1 --2- 172.21.15.53:0/2001859375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab78103200 0x7fab7819e5a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00d950 tx=0x7fab6c00dc60 comp rx=0 tx=0).stop 2024-05-21T15:27:45.721 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:45.718+0000 7fab7d40c700 1 -- 172.21.15.53:0/2001859375 shutdown_connections 2024-05-21T15:27:45.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:45.718+0000 7fab7d40c700 1 --2- 172.21.15.53:0/2001859375 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fab600a2c90 0x7fab600a51a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:45.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:45.718+0000 7fab7d40c700 1 --2- 172.21.15.53:0/2001859375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab78103ba0 0x7fab781a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:45.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:45.718+0000 7fab7d40c700 1 --2- 172.21.15.53:0/2001859375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab781020d0 0x7fab780752f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:45.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:45.718+0000 7fab7d40c700 1 --2- 172.21.15.53:0/2001859375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab78103200 0x7fab7819e5a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:45.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:45.718+0000 7fab7d40c700 1 -- 172.21.15.53:0/2001859375 >> 172.21.15.53:0/2001859375 conn(0x7fab780fd710 msgr2=0x7fab78106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:45.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:45.718+0000 7fab7d40c700 1 -- 172.21.15.53:0/2001859375 shutdown_connections 2024-05-21T15:27:45.722 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:27:45.718+0000 7fab7d40c700 1 -- 172.21.15.53:0/2001859375 wait complete. 2024-05-21T15:27:45.924 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:45 smithi053 ceph-mon[36496]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:45.925 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:45 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:45.925 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:45 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:45.925 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:45 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:45.925 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:45 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:45.925 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:45 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:45.925 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:45 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:45.925 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:45 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:27:45.925 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:45 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:46.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:45 smithi045 ceph-mon[36409]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:46.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:45 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:46.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:45 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:46.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:45 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:46.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:45 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:46.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:45 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:46.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:45 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:46.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:45 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:27:46.015 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:45 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:46.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:45 smithi038 ceph-mon[31995]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:46.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:46.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:46.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:46.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:46.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:46.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:46.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:27:46.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:46.372 DEBUG:teuthology.orchestra.run.smithi053:osd.2> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.2.service 2024-05-21T15:27:46.374 INFO:tasks.cephadm:Deploying osd.3 on smithi038 with /dev/vg_nvme/lv_3... 2024-05-21T15:27:46.374 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T15:27:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:27:46.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:46 smithi053 ceph-mon[36496]: from='osd.2 [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T15:27:46.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:46 smithi053 ceph-mon[36496]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T15:27:47.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:46 smithi038 ceph-mon[31995]: from='osd.2 [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T15:27:47.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:46 smithi038 ceph-mon[31995]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T15:27:47.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:46 smithi045 ceph-mon[36409]: from='osd.2 [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T15:27:47.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:46 smithi045 ceph-mon[36409]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T15:27:47.897 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:47 smithi053 ceph-mon[36496]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:47.898 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:47 smithi053 ceph-mon[36496]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T15:27:47.898 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:47 smithi053 ceph-mon[36496]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T15:27:47.898 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:47 smithi053 ceph-mon[36496]: from='osd.2 [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:27:47.898 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:47 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:47.898 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:47 smithi053 ceph-mon[36496]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:27:47.898 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:27:47 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-2[41627]: 2024-05-21T15:27:47.771+0000 7fc708f71700 -1 osd.2 0 waiting for initial osdmap 2024-05-21T15:27:47.898 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:27:47 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-2[41627]: 2024-05-21T15:27:47.776+0000 7fc704d26700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:27:48.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:47 smithi038 ceph-mon[31995]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:48.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:47 smithi038 ceph-mon[31995]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T15:27:48.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:47 smithi038 ceph-mon[31995]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T15:27:48.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:47 smithi038 ceph-mon[31995]: from='osd.2 [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:27:48.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:47 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:48.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:47 smithi038 ceph-mon[31995]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:27:48.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:47 smithi045 ceph-mon[36409]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:48.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:47 smithi045 ceph-mon[36409]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T15:27:48.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:47 smithi045 ceph-mon[36409]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T15:27:48.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:47 smithi045 ceph-mon[36409]: from='osd.2 [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:27:48.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:47 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:48.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:47 smithi045 ceph-mon[36409]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:27:48.916 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:48 smithi053 ceph-mon[36496]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-21T15:27:48.916 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:48 smithi053 ceph-mon[36496]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T15:27:48.917 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:48 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:48.917 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:48 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:48.917 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:48 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:48.941 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T15:27:48.942 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T15:27:48.942 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-05-21T15:27:48.942 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-05-21T15:27:48.942 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0146556 s, 715 MB/s 2024-05-21T15:27:48.942 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-05-21T15:27:49.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:48 smithi045 ceph-mon[36409]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-21T15:27:49.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:48 smithi045 ceph-mon[36409]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T15:27:49.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:48 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:49.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:48 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:49.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:48 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:49.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:48 smithi038 ceph-mon[31995]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-21T15:27:49.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:48 smithi038 ceph-mon[31995]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T15:27:49.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:48 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:49.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:48 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:49.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:48 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:49.630 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_3 2024-05-21T15:27:49.885 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:27:50.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:49 smithi038 ceph-mon[31995]: purged_snaps scrub starts 2024-05-21T15:27:50.008 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:49 smithi038 ceph-mon[31995]: purged_snaps scrub ok 2024-05-21T15:27:50.008 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:49 smithi038 ceph-mon[31995]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:50.008 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:49 smithi038 ceph-mon[31995]: osd.2 [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925] boot 2024-05-21T15:27:50.008 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:49 smithi038 ceph-mon[31995]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T15:27:50.008 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:49 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:50.008 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:49 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:50.008 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:49 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:50.008 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:49 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:27:50.008 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:49 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:50.008 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:49 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:50.008 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:49 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:50.008 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:49 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:50.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:49 smithi053 ceph-mon[36496]: purged_snaps scrub starts 2024-05-21T15:27:50.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:49 smithi053 ceph-mon[36496]: purged_snaps scrub ok 2024-05-21T15:27:50.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:49 smithi053 ceph-mon[36496]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:50.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:49 smithi053 ceph-mon[36496]: osd.2 [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925] boot 2024-05-21T15:27:50.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:49 smithi053 ceph-mon[36496]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T15:27:50.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:49 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:50.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:49 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:50.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:49 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:50.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:49 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:27:50.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:49 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:50.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:49 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:50.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:49 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:50.210 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:49 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:50.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:49 smithi045 ceph-mon[36409]: purged_snaps scrub starts 2024-05-21T15:27:50.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:49 smithi045 ceph-mon[36409]: purged_snaps scrub ok 2024-05-21T15:27:50.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:49 smithi045 ceph-mon[36409]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:27:50.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:49 smithi045 ceph-mon[36409]: osd.2 [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925] boot 2024-05-21T15:27:50.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:49 smithi045 ceph-mon[36409]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T15:27:50.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:49 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:27:50.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:49 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:50.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:49 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:50.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:49 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:27:50.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:49 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:50.265 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:49 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:50.265 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:49 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:27:50.265 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:49 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:50.954 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:50 smithi038 ceph-mon[31995]: Detected new or changed devices on smithi053 2024-05-21T15:27:50.954 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:50 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi053 to 17093M 2024-05-21T15:27:50.954 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:50 smithi038 ceph-mon[31995]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T15:27:50.954 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:50 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:27:51.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:50 smithi053 ceph-mon[36496]: Detected new or changed devices on smithi053 2024-05-21T15:27:51.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:50 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi053 to 17093M 2024-05-21T15:27:51.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:50 smithi053 ceph-mon[36496]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T15:27:51.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:50 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:27:51.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:50 smithi045 ceph-mon[36409]: Detected new or changed devices on smithi053 2024-05-21T15:27:51.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:50 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi053 to 17093M 2024-05-21T15:27:51.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:50 smithi045 ceph-mon[36409]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T15:27:51.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:50 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:27:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.633+0000 7f5146579700 1 -- 172.21.15.38:0/4147862370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5140106630 msgr2=0x7f5140108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.633+0000 7f5146579700 1 --2- 172.21.15.38:0/4147862370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5140106630 0x7f5140108a50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5128009990 tx=0x7f5128009ca0 comp rx=0 tx=0).stop 2024-05-21T15:27:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.633+0000 7f5146579700 1 -- 172.21.15.38:0/4147862370 shutdown_connections 2024-05-21T15:27:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.633+0000 7f5146579700 1 --2- 172.21.15.38:0/4147862370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5140108f90 0x7f514010b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.633+0000 7f5146579700 1 --2- 172.21.15.38:0/4147862370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5140103cd0 0x7f51401060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.633+0000 7f5146579700 1 --2- 172.21.15.38:0/4147862370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5140106630 0x7f5140108a50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.633+0000 7f5146579700 1 -- 172.21.15.38:0/4147862370 >> 172.21.15.38:0/4147862370 conn(0x7f51400fd710 msgr2=0x7f51400ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.633+0000 7f5146579700 1 -- 172.21.15.38:0/4147862370 shutdown_connections 2024-05-21T15:27:51.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.633+0000 7f5146579700 1 -- 172.21.15.38:0/4147862370 wait complete. 2024-05-21T15:27:51.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.634+0000 7f5146579700 1 Processor -- start 2024-05-21T15:27:51.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.634+0000 7f5146579700 1 -- start start 2024-05-21T15:27:51.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.634+0000 7f5146579700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5140103cd0 0x7f5140103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:51.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.634+0000 7f5146579700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5140106630 0x7f5140101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:51.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.634+0000 7f5146579700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5140108f90 0x7f5140102120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:51.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.634+0000 7f5146579700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f514010ed90 con 0x7f5140108f90 2024-05-21T15:27:51.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.634+0000 7f5146579700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f514010ec10 con 0x7f5140106630 2024-05-21T15:27:51.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.634+0000 7f5146579700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f514010ef10 con 0x7f5140103cd0 2024-05-21T15:27:51.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f5144b16700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5140108f90 0x7f5140102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:51.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f5144b16700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5140108f90 0x7f5140102120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52842/0 (socket says 172.21.15.38:52842) 2024-05-21T15:27:51.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f5144b16700 1 -- 172.21.15.38:0/3516689256 learned_addr learned my addr 172.21.15.38:0/3516689256 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:27:51.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f5137fff700 1 --2- 172.21.15.38:0/3516689256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5140106630 0x7f5140101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:51.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f513ffff700 1 --2- 172.21.15.38:0/3516689256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5140103cd0 0x7f5140103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:51.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f5144b16700 1 -- 172.21.15.38:0/3516689256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5140103cd0 msgr2=0x7f5140103530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:51.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f5144b16700 1 --2- 172.21.15.38:0/3516689256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5140103cd0 0x7f5140103530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:51.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f5144b16700 1 -- 172.21.15.38:0/3516689256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5140106630 msgr2=0x7f5140101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:51.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f5144b16700 1 --2- 172.21.15.38:0/3516689256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5140106630 0x7f5140101be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:51.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f5144b16700 1 -- 172.21.15.38:0/3516689256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5140102880 con 0x7f5140108f90 2024-05-21T15:27:51.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f5144b16700 1 --2- 172.21.15.38:0/3516689256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5140108f90 0x7f5140102120 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f513800ada0 tx=0x7f513800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:51.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f513dffb700 1 -- 172.21.15.38:0/3516689256 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138017020 con 0x7f5140108f90 2024-05-21T15:27:51.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f5146579700 1 -- 172.21.15.38:0/3516689256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51401a3920 con 0x7f5140108f90 2024-05-21T15:27:51.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f513dffb700 1 -- 172.21.15.38:0/3516689256 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f513800cdb0 con 0x7f5140108f90 2024-05-21T15:27:51.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f513dffb700 1 -- 172.21.15.38:0/3516689256 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51380053e0 con 0x7f5140108f90 2024-05-21T15:27:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.635+0000 7f5146579700 1 -- 172.21.15.38:0/3516689256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51401a3e00 con 0x7f5140108f90 2024-05-21T15:27:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.636+0000 7f5146579700 1 -- 172.21.15.38:0/3516689256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f514002dc60 con 0x7f5140108f90 2024-05-21T15:27:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.638+0000 7f513dffb700 1 -- 172.21.15.38:0/3516689256 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f513800c8d0 con 0x7f5140108f90 2024-05-21T15:27:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.639+0000 7f513dffb700 1 --2- 172.21.15.38:0/3516689256 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f512c0a2b90 0x7f512c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.639+0000 7f513ffff700 1 --2- 172.21.15.38:0/3516689256 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f512c0a2b90 0x7f512c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:51.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.639+0000 7f513dffb700 1 -- 172.21.15.38:0/3516689256 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7f51380cd8b0 con 0x7f5140108f90 2024-05-21T15:27:51.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.640+0000 7f513ffff700 1 --2- 172.21.15.38:0/3516689256 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f512c0a2b90 0x7f512c0a50a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5130005950 tx=0x7f513000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:51.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.643+0000 7f513dffb700 1 -- 172.21.15.38:0/3516689256 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5138021020 con 0x7f5140108f90 2024-05-21T15:27:51.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:51.872+0000 7f5146579700 1 -- 172.21.15.38:0/3516689256 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5140062a80 con 0x7f512c0a2b90 2024-05-21T15:27:52.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:51 smithi053 ceph-mon[36496]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:27:52.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:51 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T15:27:52.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:51 smithi053 ceph-mon[36496]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T15:27:52.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:51 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:27:52.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:51 smithi045 ceph-mon[36409]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:27:52.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:51 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T15:27:52.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:51 smithi045 ceph-mon[36409]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T15:27:52.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:51 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:27:52.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:51 smithi038 ceph-mon[31995]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:27:52.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:51 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T15:27:52.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:51 smithi038 ceph-mon[31995]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T15:27:52.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:51 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:27:52.678 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:27:52 smithi038 sudo[51277]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T15:27:52.678 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:27:52 smithi038 sudo[51277]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:27:52.678 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:27:52 smithi038 sudo[51277]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:27:52.678 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:27:52 smithi038 sudo[51277]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:27:52.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:27:52.820+0000 7f513dffb700 1 -- 172.21.15.38:0/3516689256 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5138097cd0 con 0x7f5140108f90 2024-05-21T15:27:53.027 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:27:52 smithi038 sudo[51329]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T15:27:53.028 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:27:52 smithi038 sudo[51329]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:27:53.028 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:27:52 smithi038 sudo[51329]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:27:53.028 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:27:52 smithi038 sudo[51329]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:27:53.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:52 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T15:27:53.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:52 smithi038 ceph-mon[31995]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T15:27:53.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:52 smithi038 ceph-mon[31995]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:27:53.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:52 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:27:53.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:52 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:27:53.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:52 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:53.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:52 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T15:27:53.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:52 smithi053 ceph-mon[36496]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T15:27:53.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:52 smithi053 ceph-mon[36496]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:27:53.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:52 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:27:53.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:52 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:27:53.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:52 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:53.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:52 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T15:27:53.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:52 smithi045 ceph-mon[36409]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T15:27:53.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:52 smithi045 ceph-mon[36409]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:27:53.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:52 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:27:53.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:52 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:27:53.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:52 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:27:53.763 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:27:53 smithi045 sudo[44833]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T15:27:53.764 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:27:53 smithi045 sudo[44833]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:27:53.764 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:27:53 smithi045 sudo[44833]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:27:53.764 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:27:53 smithi045 sudo[44833]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:27:54.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:53 smithi053 ceph-mon[36496]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:27:54.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:53 smithi053 ceph-mon[36496]: osdmap e22: 3 total, 3 up, 3 in 2024-05-21T15:27:54.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:53 smithi053 ceph-mon[36496]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-21T15:27:54.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:54.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:54.263 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:27:53 smithi045 sudo[44836]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T15:27:54.264 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:27:53 smithi045 sudo[44836]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:27:54.264 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:27:53 smithi045 sudo[44836]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:27:54.264 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:27:53 smithi045 sudo[44836]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:27:54.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:53 smithi045 ceph-mon[36409]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:27:54.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:53 smithi045 ceph-mon[36409]: osdmap e22: 3 total, 3 up, 3 in 2024-05-21T15:27:54.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:53 smithi045 ceph-mon[36409]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-21T15:27:54.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:54.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:54.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:53 smithi038 ceph-mon[31995]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:27:54.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:53 smithi038 ceph-mon[31995]: osdmap e22: 3 total, 3 up, 3 in 2024-05-21T15:27:54.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:53 smithi038 ceph-mon[31995]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-21T15:27:54.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:54.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:54.703 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:27:54 smithi053 sudo[45368]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T15:27:54.703 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:27:54 smithi053 sudo[45368]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:27:54.703 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:27:54 smithi053 sudo[45368]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:27:54.703 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:27:54 smithi053 sudo[45368]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:27:54.959 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:27:54 smithi053 sudo[45371]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T15:27:54.959 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:27:54 smithi053 sudo[45371]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:27:54.959 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:27:54 smithi053 sudo[45371]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:27:54.959 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:27:54 smithi053 sudo[45371]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:27:55.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:55 smithi038 sudo[51438]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T15:27:55.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:55 smithi038 sudo[51438]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:27:55.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:55 smithi038 sudo[51438]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:27:55.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:55 smithi038 sudo[51438]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:27:56.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:55 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1517123529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3de4eed-679b-4828-ae1e-468585b7c1f3"}]: dispatch 2024-05-21T15:27:56.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:55 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1517123529' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3de4eed-679b-4828-ae1e-468585b7c1f3"}]': finished 2024-05-21T15:27:56.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:55 smithi045 ceph-mon[36409]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T15:27:56.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:55 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:27:56.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:55 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:56.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:55 smithi045 ceph-mon[36409]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:27:56.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:55 smithi038 sudo[51441]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T15:27:56.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:55 smithi038 sudo[51441]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:27:56.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:55 smithi038 sudo[51441]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:27:56.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:55 smithi038 sudo[51441]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:27:56.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:55 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1517123529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3de4eed-679b-4828-ae1e-468585b7c1f3"}]: dispatch 2024-05-21T15:27:56.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:55 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1517123529' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3de4eed-679b-4828-ae1e-468585b7c1f3"}]': finished 2024-05-21T15:27:56.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:55 smithi038 ceph-mon[31995]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T15:27:56.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:55 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:27:56.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:55 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:56.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:55 smithi038 ceph-mon[31995]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:27:56.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:56 smithi053 sudo[45374]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T15:27:56.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:56 smithi053 sudo[45374]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:27:56.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:56 smithi053 sudo[45374]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:27:56.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:56 smithi053 sudo[45374]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:27:56.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:56 smithi053 sudo[45377]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T15:27:56.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:56 smithi053 sudo[45377]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:27:56.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:56 smithi053 sudo[45377]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:27:56.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:56 smithi053 sudo[45377]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:27:56.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:56 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1517123529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3de4eed-679b-4828-ae1e-468585b7c1f3"}]: dispatch 2024-05-21T15:27:56.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:56 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1517123529' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3de4eed-679b-4828-ae1e-468585b7c1f3"}]': finished 2024-05-21T15:27:56.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:56 smithi053 ceph-mon[36496]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T15:27:56.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:27:56.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:27:56.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:56 smithi053 ceph-mon[36496]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:27:57.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:56 smithi045 sudo[44839]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T15:27:57.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:56 smithi045 sudo[44839]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:27:57.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:56 smithi045 sudo[44839]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:27:57.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 sudo[44839]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:27:57.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 sudo[44842]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T15:27:57.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 sudo[44842]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:27:57.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 sudo[44842]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:27:57.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 sudo[44842]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:27:57.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 ceph-mon[36409]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:27:57.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 ceph-mon[36409]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:27:57.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:27:57.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:27:57.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:27:57.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 ceph-mon[36409]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:27:57.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:27:57.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:27:57.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:27:57.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2084970031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:27:57.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 ceph-mon[36409]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:27:57.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:27:57.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:27:57.765 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:27:57.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:57 smithi038 ceph-mon[31995]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:27:57.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:57 smithi038 ceph-mon[31995]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:27:57.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:27:57.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:27:57.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:27:57.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:57 smithi038 ceph-mon[31995]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:27:57.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:27:57.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:27:57.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:27:57.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:57 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2084970031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:27:57.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:57 smithi038 ceph-mon[31995]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:27:57.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:27:57.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:27:57.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:27:57.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:57 smithi053 ceph-mon[36496]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:27:57.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:57 smithi053 ceph-mon[36496]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:27:57.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:27:57.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:27:57.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:27:57.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:57 smithi053 ceph-mon[36496]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:27:57.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:27:57.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:27:57.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:27:57.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:57 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2084970031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:27:57.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:57 smithi053 ceph-mon[36496]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:27:57.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:27:57.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:27:57.960 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:27:58.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:58 smithi045 ceph-mon[36409]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:27:58.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:58 smithi045 ceph-mon[36409]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:27:58.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:58 smithi045 ceph-mon[36409]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:27:58.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:58 smithi038 ceph-mon[31995]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:27:58.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:58 smithi038 ceph-mon[31995]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:27:58.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:58 smithi038 ceph-mon[31995]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:27:58.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:58 smithi053 ceph-mon[36496]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:27:58.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:58 smithi053 ceph-mon[36496]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:27:58.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:58 smithi053 ceph-mon[36496]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:27:59.675 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:27:59 smithi038 ceph-mon[31995]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:27:59.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:27:59 smithi045 ceph-mon[36409]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:27:59.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:27:59 smithi053 ceph-mon[36496]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:28:01.589 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:01 smithi038 ceph-mon[31995]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:28:01.761 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:01 smithi053 ceph-mon[36496]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:28:01.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:01 smithi045 ceph-mon[36409]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:28:03.634 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:03 smithi038 ceph-mon[31995]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-05-21T15:28:03.635 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-21T15:28:03.635 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:03.635 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:03 smithi038 ceph-mon[31995]: Deploying daemon osd.3 on smithi038 2024-05-21T15:28:03.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:03 smithi045 ceph-mon[36409]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-05-21T15:28:03.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-21T15:28:03.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:03.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:03 smithi045 ceph-mon[36409]: Deploying daemon osd.3 on smithi038 2024-05-21T15:28:03.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:03 smithi053 ceph-mon[36496]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-05-21T15:28:03.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-21T15:28:03.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:03.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:03 smithi053 ceph-mon[36496]: Deploying daemon osd.3 on smithi038 2024-05-21T15:28:05.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:05 smithi045 ceph-mon[36409]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-05-21T15:28:05.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:05 smithi038 ceph-mon[31995]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-05-21T15:28:05.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:05 smithi053 ceph-mon[36496]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-05-21T15:28:07.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:07 smithi038 ceph-mon[31995]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-21T15:28:07.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:07 smithi045 ceph-mon[36409]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-21T15:28:07.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:07 smithi053 ceph-mon[36496]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-21T15:28:09.871 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:09 smithi038 ceph-mon[31995]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-21T15:28:09.871 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:09 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:09.871 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:09 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:09.871 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:09 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:28:09.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:09 smithi053 ceph-mon[36496]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-21T15:28:09.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:09 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:09.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:09 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:09.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:09 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:28:10.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:09 smithi045 ceph-mon[36409]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-21T15:28:10.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:09 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:10.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:09 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:10.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:09 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:28:11.870 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:11 smithi038 ceph-mon[31995]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-21T15:28:11.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:11 smithi053 ceph-mon[36496]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-21T15:28:12.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:11 smithi045 ceph-mon[36409]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-21T15:28:12.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:28:12.235+0000 7f513dffb700 1 -- 172.21.15.38:0/3516689256 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5140062a80 con 0x7f512c0a2b90 2024-05-21T15:28:12.238 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 3 on host 'smithi038' 2024-05-21T15:28:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:28:12.239+0000 7f5146579700 1 -- 172.21.15.38:0/3516689256 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f512c0a2b90 msgr2=0x7f512c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:28:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:28:12.239+0000 7f5146579700 1 --2- 172.21.15.38:0/3516689256 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f512c0a2b90 0x7f512c0a50a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5130005950 tx=0x7f513000a300 comp rx=0 tx=0).stop 2024-05-21T15:28:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:28:12.239+0000 7f5146579700 1 -- 172.21.15.38:0/3516689256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5140108f90 msgr2=0x7f5140102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:28:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:28:12.239+0000 7f5146579700 1 --2- 172.21.15.38:0/3516689256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5140108f90 0x7f5140102120 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f513800ada0 tx=0x7f513800c330 comp rx=0 tx=0).stop 2024-05-21T15:28:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:28:12.239+0000 7f5146579700 1 -- 172.21.15.38:0/3516689256 shutdown_connections 2024-05-21T15:28:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:28:12.239+0000 7f5146579700 1 --2- 172.21.15.38:0/3516689256 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f512c0a2b90 0x7f512c0a50a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:28:12.239+0000 7f5146579700 1 --2- 172.21.15.38:0/3516689256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5140108f90 0x7f5140102120 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:28:12.239+0000 7f5146579700 1 --2- 172.21.15.38:0/3516689256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5140103cd0 0x7f5140103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:28:12.239+0000 7f5146579700 1 --2- 172.21.15.38:0/3516689256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5140106630 0x7f5140101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:12.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:28:12.239+0000 7f5146579700 1 -- 172.21.15.38:0/3516689256 >> 172.21.15.38:0/3516689256 conn(0x7f51400fd710 msgr2=0x7f51401047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:28:12.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:28:12.239+0000 7f5146579700 1 -- 172.21.15.38:0/3516689256 shutdown_connections 2024-05-21T15:28:12.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:28:12.239+0000 7f5146579700 1 -- 172.21.15.38:0/3516689256 wait complete. 2024-05-21T15:28:12.830 DEBUG:teuthology.orchestra.run.smithi038:osd.3> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.3.service 2024-05-21T15:28:12.832 INFO:tasks.cephadm:Deploying osd.4 on smithi045 with /dev/vg_nvme/lv_3... 2024-05-21T15:28:12.833 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T15:28:13.092 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:28:13.464 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:13 smithi045 ceph-mon[36409]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-21T15:28:13.464 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:13 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:13.464 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:13 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:13.465 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:13 smithi045 ceph-mon[36409]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T15:28:13.465 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:13 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:13.465 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:13 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:13.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:13 smithi053 ceph-mon[36496]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-21T15:28:13.508 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:13 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:13.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:13 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:13.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:13 smithi053 ceph-mon[36496]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T15:28:13.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:13 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:13.509 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:13 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:13.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:13 smithi038 ceph-mon[31995]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-21T15:28:13.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:13 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:13.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:13 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:13.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:13 smithi038 ceph-mon[31995]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T15:28:13.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:13 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:13.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:13 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:14.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:14 smithi053 ceph-mon[36496]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T15:28:14.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:14 smithi053 ceph-mon[36496]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T15:28:14.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:14 smithi053 ceph-mon[36496]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-05-21T15:28:14.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:14 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:28:14.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:14 smithi038 ceph-mon[31995]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T15:28:14.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:14 smithi038 ceph-mon[31995]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T15:28:14.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:14 smithi038 ceph-mon[31995]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-05-21T15:28:14.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:14 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:28:14.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:14 smithi045 ceph-mon[36409]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T15:28:14.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:14 smithi045 ceph-mon[36409]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T15:28:14.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:14 smithi045 ceph-mon[36409]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-05-21T15:28:14.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:14 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:28:14.778 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:28:14 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-3[53448]: 2024-05-21T15:28:14.695+0000 7ff92bb31700 -1 osd.3 0 waiting for initial osdmap 2024-05-21T15:28:14.778 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:28:14 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-3[53448]: 2024-05-21T15:28:14.701+0000 7ff9238de700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:28:15.134 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T15:28:15.134 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T15:28:15.134 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-05-21T15:28:15.134 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-05-21T15:28:15.134 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00854022 s, 1.2 GB/s 2024-05-21T15:28:15.134 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-05-21T15:28:15.449 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:15 smithi045 ceph-mon[36409]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:28:15.449 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:15 smithi045 ceph-mon[36409]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-05-21T15:28:15.450 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:15 smithi045 ceph-mon[36409]: osdmap e25: 4 total, 3 up, 4 in 2024-05-21T15:28:15.450 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:15 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:28:15.450 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:15 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:28:15.450 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:15 smithi045 ceph-mon[36409]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' 2024-05-21T15:28:15.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:15 smithi038 ceph-mon[31995]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:28:15.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:15 smithi038 ceph-mon[31995]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-05-21T15:28:15.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:15 smithi038 ceph-mon[31995]: osdmap e25: 4 total, 3 up, 4 in 2024-05-21T15:28:15.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:15 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:28:15.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:15 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:28:15.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:15 smithi038 ceph-mon[31995]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' 2024-05-21T15:28:15.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:15 smithi053 ceph-mon[36496]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:28:15.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:15 smithi053 ceph-mon[36496]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-05-21T15:28:15.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:15 smithi053 ceph-mon[36496]: osdmap e25: 4 total, 3 up, 4 in 2024-05-21T15:28:15.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:15 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:28:15.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:15 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:28:15.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:15 smithi053 ceph-mon[36496]: from='osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033]' entity='osd.3' 2024-05-21T15:28:15.773 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_3 2024-05-21T15:28:16.020 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:28:16.438 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:16 smithi045 ceph-mon[36409]: purged_snaps scrub starts 2024-05-21T15:28:16.438 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:16 smithi045 ceph-mon[36409]: purged_snaps scrub ok 2024-05-21T15:28:16.439 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:16 smithi045 ceph-mon[36409]: osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033] boot 2024-05-21T15:28:16.439 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:16 smithi045 ceph-mon[36409]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T15:28:16.439 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:16 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:28:16.439 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:16 smithi045 ceph-mon[36409]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T15:28:16.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:16 smithi038 ceph-mon[31995]: purged_snaps scrub starts 2024-05-21T15:28:16.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:16 smithi038 ceph-mon[31995]: purged_snaps scrub ok 2024-05-21T15:28:16.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:16 smithi038 ceph-mon[31995]: osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033] boot 2024-05-21T15:28:16.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:16 smithi038 ceph-mon[31995]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T15:28:16.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:16 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:28:16.528 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:16 smithi038 ceph-mon[31995]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T15:28:16.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:16 smithi053 ceph-mon[36496]: purged_snaps scrub starts 2024-05-21T15:28:16.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:16 smithi053 ceph-mon[36496]: purged_snaps scrub ok 2024-05-21T15:28:16.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:16 smithi053 ceph-mon[36496]: osd.3 [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033] boot 2024-05-21T15:28:16.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:16 smithi053 ceph-mon[36496]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T15:28:16.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:16 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:28:16.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:16 smithi053 ceph-mon[36496]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T15:28:17.411 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 149 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:17.411 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: Detected new or changed devices on smithi038 2024-05-21T15:28:17.411 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.411 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.411 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:28:17.411 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:28:17.411 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi038 to 8546M 2024-05-21T15:28:17.412 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.412 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:17.412 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:28:17.412 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.412 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:28:17.412 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:17.412 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:28:17.412 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.412 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:17 smithi045 ceph-mon[36409]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T15:28:17.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.654+0000 7f5256d09700 1 -- 172.21.15.45:0/1465653966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250103cd0 msgr2=0x7f52501060f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:28:17.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.654+0000 7f5256d09700 1 --2- 172.21.15.45:0/1465653966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250103cd0 0x7f52501060f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5244009990 tx=0x7f5244009ca0 comp rx=0 tx=0).stop 2024-05-21T15:28:17.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.654+0000 7f5256d09700 1 -- 172.21.15.45:0/1465653966 shutdown_connections 2024-05-21T15:28:17.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.654+0000 7f5256d09700 1 --2- 172.21.15.45:0/1465653966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5250108f90 0x7f525010b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:17.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.654+0000 7f5256d09700 1 --2- 172.21.15.45:0/1465653966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250103cd0 0x7f52501060f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:17.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.654+0000 7f5256d09700 1 --2- 172.21.15.45:0/1465653966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5250106630 0x7f5250108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:17.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.654+0000 7f5256d09700 1 -- 172.21.15.45:0/1465653966 >> 172.21.15.45:0/1465653966 conn(0x7f52500fd710 msgr2=0x7f52500ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:28:17.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.655+0000 7f5256d09700 1 -- 172.21.15.45:0/1465653966 shutdown_connections 2024-05-21T15:28:17.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.655+0000 7f5256d09700 1 -- 172.21.15.45:0/1465653966 wait complete. 2024-05-21T15:28:17.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.655+0000 7f5256d09700 1 Processor -- start 2024-05-21T15:28:17.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.655+0000 7f5256d09700 1 -- start start 2024-05-21T15:28:17.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.655+0000 7f5256d09700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5250103cd0 0x7f5250199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:28:17.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.655+0000 7f5256d09700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250106630 0x7f5250199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:28:17.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.655+0000 7f5256d09700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5250108f90 0x7f52501a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:28:17.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.655+0000 7f5256d09700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f525010edb0 con 0x7f5250108f90 2024-05-21T15:28:17.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.655+0000 7f5256d09700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f525010ec30 con 0x7f5250106630 2024-05-21T15:28:17.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.655+0000 7f5256d09700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f525010ef30 con 0x7f5250103cd0 2024-05-21T15:28:17.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f524ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250106630 0x7f5250199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:28:17.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f524ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250106630 0x7f5250199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50786/0 (socket says 172.21.15.45:50786) 2024-05-21T15:28:17.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f524ffff700 1 -- 172.21.15.45:0/3639228078 learned_addr learned my addr 172.21.15.45:0/3639228078 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-21T15:28:17.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f5254aa5700 1 --2- 172.21.15.45:0/3639228078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5250103cd0 0x7f5250199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:28:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f52552a6700 1 --2- 172.21.15.45:0/3639228078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5250108f90 0x7f52501a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:28:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f524ffff700 1 -- 172.21.15.45:0/3639228078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5250103cd0 msgr2=0x7f5250199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:28:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f524ffff700 1 --2- 172.21.15.45:0/3639228078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5250103cd0 0x7f5250199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f524ffff700 1 -- 172.21.15.45:0/3639228078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5250108f90 msgr2=0x7f52501a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:28:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f524ffff700 1 --2- 172.21.15.45:0/3639228078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5250108f90 0x7f52501a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f524ffff700 1 -- 172.21.15.45:0/3639228078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52501a1360 con 0x7f5250106630 2024-05-21T15:28:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f524ffff700 1 --2- 172.21.15.45:0/3639228078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250106630 0x7f5250199dd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f523c00c9b0 tx=0x7f523c00ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:28:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f524dffb700 1 -- 172.21.15.45:0/3639228078 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c007b00 con 0x7f5250106630 2024-05-21T15:28:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f524dffb700 1 -- 172.21.15.45:0/3639228078 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f523c007ca0 con 0x7f5250106630 2024-05-21T15:28:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f5256d09700 1 -- 172.21.15.45:0/3639228078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52501a1650 con 0x7f5250106630 2024-05-21T15:28:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.656+0000 7f524dffb700 1 -- 172.21.15.45:0/3639228078 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c018730 con 0x7f5250106630 2024-05-21T15:28:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.657+0000 7f5256d09700 1 -- 172.21.15.45:0/3639228078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52501a1b90 con 0x7f5250106630 2024-05-21T15:28:17.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.658+0000 7f5256d09700 1 -- 172.21.15.45:0/3639228078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5250050170 con 0x7f5250106630 2024-05-21T15:28:17.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.659+0000 7f524dffb700 1 -- 172.21.15.45:0/3639228078 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f523c0188d0 con 0x7f5250106630 2024-05-21T15:28:17.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.659+0000 7f524dffb700 1 --2- 172.21.15.45:0/3639228078 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f52400a2c20 0x7f52400a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:28:17.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.659+0000 7f524dffb700 1 -- 172.21.15.45:0/3639228078 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7f523c0cde50 con 0x7f5250106630 2024-05-21T15:28:17.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.660+0000 7f5254aa5700 1 --2- 172.21.15.45:0/3639228078 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f52400a2c20 0x7f52400a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:28:17.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.660+0000 7f5254aa5700 1 --2- 172.21.15.45:0/3639228078 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f52400a2c20 0x7f52400a5130 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f524400b5c0 tx=0x7f524401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:28:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.664+0000 7f524dffb700 1 -- 172.21.15.45:0/3639228078 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f523c014030 con 0x7f5250106630 2024-05-21T15:28:17.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 149 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:17.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: Detected new or changed devices on smithi038 2024-05-21T15:28:17.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:28:17.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:28:17.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi038 to 8546M 2024-05-21T15:28:17.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:17.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:28:17.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:28:17.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:17.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:28:17.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.710 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:17 smithi053 ceph-mon[36496]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T15:28:17.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 149 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:17.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: Detected new or changed devices on smithi038 2024-05-21T15:28:17.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:28:17.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:28:17.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi038 to 8546M 2024-05-21T15:28:17.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:17.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:28:17.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:28:17.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:17.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:28:17.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:17.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:17 smithi038 ceph-mon[31995]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T15:28:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:17.879+0000 7f5256d09700 1 -- 172.21.15.45:0/3639228078 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5250062a80 con 0x7f52400a2c20 2024-05-21T15:28:18.625 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:18 smithi045 ceph-mon[36409]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:28:18.625 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:18 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:28:18.625 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:18 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:28:18.625 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:18 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:18.625 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:18 smithi045 ceph-mon[36409]: osdmap e29: 4 total, 4 up, 4 in 2024-05-21T15:28:18.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:18 smithi053 ceph-mon[36496]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:28:18.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:18 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:28:18.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:18 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:28:18.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:18 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:18.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:18 smithi053 ceph-mon[36496]: osdmap e29: 4 total, 4 up, 4 in 2024-05-21T15:28:18.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:18 smithi038 ceph-mon[31995]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:28:18.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:18 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:28:18.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:18 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:28:18.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:18 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:18.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:18 smithi038 ceph-mon[31995]: osdmap e29: 4 total, 4 up, 4 in 2024-05-21T15:28:19.653 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:19 smithi045 ceph-mon[36409]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 149 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:19.653 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:19 smithi045 ceph-mon[36409]: osdmap e30: 4 total, 4 up, 4 in 2024-05-21T15:28:19.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:19 smithi053 ceph-mon[36496]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 149 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:19.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:19 smithi053 ceph-mon[36496]: osdmap e30: 4 total, 4 up, 4 in 2024-05-21T15:28:19.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:19 smithi038 ceph-mon[31995]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 149 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:19.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:19 smithi038 ceph-mon[31995]: osdmap e30: 4 total, 4 up, 4 in 2024-05-21T15:28:21.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:21 smithi053 ceph-mon[36496]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 149 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:21.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:21 smithi053 ceph-mon[36496]: from='client.? 172.21.15.45:0/1731152972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b728a85c-8f55-4a40-a5e5-9ce853372338"}]: dispatch 2024-05-21T15:28:21.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:21 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b728a85c-8f55-4a40-a5e5-9ce853372338"}]: dispatch 2024-05-21T15:28:21.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:21 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b728a85c-8f55-4a40-a5e5-9ce853372338"}]': finished 2024-05-21T15:28:21.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:21 smithi053 ceph-mon[36496]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T15:28:21.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:21.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:21 smithi045 ceph-mon[36409]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 149 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:21.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:21 smithi045 ceph-mon[36409]: from='client.? 172.21.15.45:0/1731152972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b728a85c-8f55-4a40-a5e5-9ce853372338"}]: dispatch 2024-05-21T15:28:21.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:21 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b728a85c-8f55-4a40-a5e5-9ce853372338"}]: dispatch 2024-05-21T15:28:21.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:21 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b728a85c-8f55-4a40-a5e5-9ce853372338"}]': finished 2024-05-21T15:28:21.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:21 smithi045 ceph-mon[36409]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T15:28:21.764 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:21 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:21.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:21 smithi038 ceph-mon[31995]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 149 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:21.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:21 smithi038 ceph-mon[31995]: from='client.? 172.21.15.45:0/1731152972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b728a85c-8f55-4a40-a5e5-9ce853372338"}]: dispatch 2024-05-21T15:28:21.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:21 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b728a85c-8f55-4a40-a5e5-9ce853372338"}]: dispatch 2024-05-21T15:28:21.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:21 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b728a85c-8f55-4a40-a5e5-9ce853372338"}]': finished 2024-05-21T15:28:21.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:21 smithi038 ceph-mon[31995]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T15:28:21.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:22.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:22 smithi053 ceph-mon[36496]: from='client.? 172.21.15.45:0/1851199978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:28:22.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:22 smithi045 ceph-mon[36409]: from='client.? 172.21.15.45:0/1851199978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:28:22.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:22 smithi038 ceph-mon[31995]: from='client.? 172.21.15.45:0/1851199978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:28:23.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:23 smithi053 ceph-mon[36496]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-21T15:28:23.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:23 smithi045 ceph-mon[36409]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-21T15:28:23.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:23 smithi038 ceph-mon[31995]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-21T15:28:25.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:25 smithi053 ceph-mon[36496]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:28:25.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:25 smithi045 ceph-mon[36409]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:28:25.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:25 smithi038 ceph-mon[31995]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:28:27.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:27 smithi045 ceph-mon[36409]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-21T15:28:27.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:27 smithi053 ceph-mon[36496]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-21T15:28:27.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:27 smithi038 ceph-mon[31995]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-21T15:28:28.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:28 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-21T15:28:28.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:28 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:28.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:28 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-21T15:28:28.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:28 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:28.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:28 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-21T15:28:28.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:28 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:29.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:29 smithi045 ceph-mon[36409]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-21T15:28:29.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:29 smithi045 ceph-mon[36409]: Deploying daemon osd.4 on smithi045 2024-05-21T15:28:29.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:29 smithi053 ceph-mon[36496]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-21T15:28:29.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:29 smithi053 ceph-mon[36496]: Deploying daemon osd.4 on smithi045 2024-05-21T15:28:29.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:29 smithi038 ceph-mon[31995]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-21T15:28:29.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:29 smithi038 ceph-mon[31995]: Deploying daemon osd.4 on smithi045 2024-05-21T15:28:31.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:31 smithi053 ceph-mon[36496]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:28:31.712 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:31 smithi045 ceph-mon[36409]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:28:31.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:31 smithi038 ceph-mon[31995]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:28:33.641 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:33 smithi045 ceph-mon[36409]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-21T15:28:33.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:33 smithi053 ceph-mon[36496]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-21T15:28:33.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:33 smithi038 ceph-mon[31995]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-21T15:28:34.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:28:34.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:34.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:34.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:28:34.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:34.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:34.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:28:34.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:34.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:35.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:35 smithi053 ceph-mon[36496]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:35.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:35 smithi045 ceph-mon[36409]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:35.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:35 smithi038 ceph-mon[31995]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:37.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:37 smithi045 ceph-mon[36409]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:37.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:37.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:37.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:37.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:28:37.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:37 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:37.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:37 smithi053 ceph-mon[36496]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:37.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:37.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:37.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:37.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:28:37.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:37 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:37.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:37 smithi038 ceph-mon[31995]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:37.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:37.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:37.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:37.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:28:37.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:37 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:38.008 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 4 on host 'smithi045' 2024-05-21T15:28:38.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:38.005+0000 7f524dffb700 1 -- 172.21.15.45:0/3639228078 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5250062a80 con 0x7f52400a2c20 2024-05-21T15:28:38.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:38.008+0000 7f5256d09700 1 -- 172.21.15.45:0/3639228078 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f52400a2c20 msgr2=0x7f52400a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:28:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:38.008+0000 7f5256d09700 1 --2- 172.21.15.45:0/3639228078 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f52400a2c20 0x7f52400a5130 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f524400b5c0 tx=0x7f524401a040 comp rx=0 tx=0).stop 2024-05-21T15:28:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:38.008+0000 7f5256d09700 1 -- 172.21.15.45:0/3639228078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250106630 msgr2=0x7f5250199dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:28:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:38.008+0000 7f5256d09700 1 --2- 172.21.15.45:0/3639228078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250106630 0x7f5250199dd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f523c00c9b0 tx=0x7f523c00ccc0 comp rx=0 tx=0).stop 2024-05-21T15:28:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:38.008+0000 7f5256d09700 1 -- 172.21.15.45:0/3639228078 shutdown_connections 2024-05-21T15:28:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:38.008+0000 7f5256d09700 1 --2- 172.21.15.45:0/3639228078 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f52400a2c20 0x7f52400a5130 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:38.008+0000 7f5256d09700 1 --2- 172.21.15.45:0/3639228078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5250108f90 0x7f52501a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:38.008+0000 7f5256d09700 1 --2- 172.21.15.45:0/3639228078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5250103cd0 0x7f5250199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:38.008+0000 7f5256d09700 1 --2- 172.21.15.45:0/3639228078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250106630 0x7f5250199dd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:38.008+0000 7f5256d09700 1 -- 172.21.15.45:0/3639228078 >> 172.21.15.45:0/3639228078 conn(0x7f52500fd710 msgr2=0x7f5250104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:28:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:38.008+0000 7f5256d09700 1 -- 172.21.15.45:0/3639228078 shutdown_connections 2024-05-21T15:28:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:28:38.008+0000 7f5256d09700 1 -- 172.21.15.45:0/3639228078 wait complete. 2024-05-21T15:28:38.677 DEBUG:teuthology.orchestra.run.smithi045:osd.4> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.4.service 2024-05-21T15:28:38.679 INFO:tasks.cephadm:Deploying osd.5 on smithi053 with /dev/vg_nvme/lv_3... 2024-05-21T15:28:38.679 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T15:28:38.935 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:28:38.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:38 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:38.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:38 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:38.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:38 smithi053 ceph-mon[36496]: from='osd.4 [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T15:28:38.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:38 smithi053 ceph-mon[36496]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T15:28:38.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:38 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:28:38.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:38 smithi053 ceph-mon[36496]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:39.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:38 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:39.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:38 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:39.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:38 smithi045 ceph-mon[36409]: from='osd.4 [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T15:28:39.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:38 smithi045 ceph-mon[36409]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T15:28:39.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:38 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:28:39.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:38 smithi045 ceph-mon[36409]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:39.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:38 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:39.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:38 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:39.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:38 smithi038 ceph-mon[31995]: from='osd.4 [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T15:28:39.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:38 smithi038 ceph-mon[31995]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T15:28:39.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:38 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:28:39.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:38 smithi038 ceph-mon[31995]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:40.244 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:39 smithi045 ceph-mon[36409]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T15:28:40.244 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:39 smithi045 ceph-mon[36409]: from='osd.4 [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:28:40.244 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:39 smithi045 ceph-mon[36409]: osdmap e32: 5 total, 4 up, 5 in 2024-05-21T15:28:40.244 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:39 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:40.244 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:39 smithi045 ceph-mon[36409]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:28:40.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:39 smithi038 ceph-mon[31995]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T15:28:40.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:39 smithi038 ceph-mon[31995]: from='osd.4 [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:28:40.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:39 smithi038 ceph-mon[31995]: osdmap e32: 5 total, 4 up, 5 in 2024-05-21T15:28:40.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:39 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:40.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:39 smithi038 ceph-mon[31995]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:28:40.321 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:39 smithi053 ceph-mon[36496]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T15:28:40.322 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:39 smithi053 ceph-mon[36496]: from='osd.4 [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:28:40.322 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:39 smithi053 ceph-mon[36496]: osdmap e32: 5 total, 4 up, 5 in 2024-05-21T15:28:40.322 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:39 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:40.322 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:39 smithi053 ceph-mon[36496]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:28:40.895 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:28:40 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-4[47644]: 2024-05-21T15:28:40.511+0000 7fd49877b700 -1 osd.4 0 waiting for initial osdmap 2024-05-21T15:28:40.895 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:28:40 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-4[47644]: 2024-05-21T15:28:40.519+0000 7fd48fd27700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:28:40.942 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T15:28:40.942 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T15:28:40.942 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-05-21T15:28:40.942 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-05-21T15:28:40.942 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00855726 s, 1.2 GB/s 2024-05-21T15:28:40.942 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-05-21T15:28:41.213 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:40 smithi045 ceph-mon[36409]: purged_snaps scrub starts 2024-05-21T15:28:41.214 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:40 smithi045 ceph-mon[36409]: purged_snaps scrub ok 2024-05-21T15:28:41.214 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:40 smithi045 ceph-mon[36409]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T15:28:41.214 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:40 smithi045 ceph-mon[36409]: osdmap e33: 5 total, 4 up, 5 in 2024-05-21T15:28:41.214 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:40 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:41.214 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:40 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:41.214 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:40 smithi045 ceph-mon[36409]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:41.214 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:40 smithi045 ceph-mon[36409]: from='osd.4 ' entity='osd.4' 2024-05-21T15:28:41.270 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:40 smithi053 ceph-mon[36496]: purged_snaps scrub starts 2024-05-21T15:28:41.270 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:40 smithi053 ceph-mon[36496]: purged_snaps scrub ok 2024-05-21T15:28:41.270 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:40 smithi053 ceph-mon[36496]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T15:28:41.270 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:40 smithi053 ceph-mon[36496]: osdmap e33: 5 total, 4 up, 5 in 2024-05-21T15:28:41.270 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:40 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:41.271 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:40 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:41.271 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:40 smithi053 ceph-mon[36496]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:41.271 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:40 smithi053 ceph-mon[36496]: from='osd.4 ' entity='osd.4' 2024-05-21T15:28:41.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:40 smithi038 ceph-mon[31995]: purged_snaps scrub starts 2024-05-21T15:28:41.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:40 smithi038 ceph-mon[31995]: purged_snaps scrub ok 2024-05-21T15:28:41.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:40 smithi038 ceph-mon[31995]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T15:28:41.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:40 smithi038 ceph-mon[31995]: osdmap e33: 5 total, 4 up, 5 in 2024-05-21T15:28:41.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:40 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:41.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:40 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:41.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:40 smithi038 ceph-mon[31995]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:28:41.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:40 smithi038 ceph-mon[31995]: from='osd.4 ' entity='osd.4' 2024-05-21T15:28:41.548 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_3 2024-05-21T15:28:41.812 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:28:42.183 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:41 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:42.183 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:41 smithi053 ceph-mon[36496]: osd.4 [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193] boot 2024-05-21T15:28:42.183 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:41 smithi053 ceph-mon[36496]: osdmap e34: 5 total, 5 up, 5 in 2024-05-21T15:28:42.183 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:41 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:42.183 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:41 smithi053 ceph-mon[36496]: Detected new or changed devices on smithi045 2024-05-21T15:28:42.183 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:41 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:42.184 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:41 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:42.184 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:41 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:28:42.184 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:41 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:28:42.184 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:41 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi045 to 8546M 2024-05-21T15:28:42.184 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:41 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:42.184 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:41 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:42.184 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:41 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:28:42.184 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:41 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:42.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:41 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:42.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:41 smithi045 ceph-mon[36409]: osd.4 [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193] boot 2024-05-21T15:28:42.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:41 smithi045 ceph-mon[36409]: osdmap e34: 5 total, 5 up, 5 in 2024-05-21T15:28:42.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:41 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:42.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:41 smithi045 ceph-mon[36409]: Detected new or changed devices on smithi045 2024-05-21T15:28:42.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:41 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:42.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:41 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:42.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:41 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:28:42.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:41 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:28:42.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:41 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi045 to 8546M 2024-05-21T15:28:42.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:41 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:42.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:41 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:42.265 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:41 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:28:42.265 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:41 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:42.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:41 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:42.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:41 smithi038 ceph-mon[31995]: osd.4 [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193] boot 2024-05-21T15:28:42.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:41 smithi038 ceph-mon[31995]: osdmap e34: 5 total, 5 up, 5 in 2024-05-21T15:28:42.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:41 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:28:42.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:41 smithi038 ceph-mon[31995]: Detected new or changed devices on smithi045 2024-05-21T15:28:42.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:41 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:42.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:41 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:42.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:41 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:28:42.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:41 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:28:42.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:41 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi045 to 8546M 2024-05-21T15:28:42.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:41 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:42.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:41 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:42.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:41 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:28:42.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:41 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:28:43.661 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.655+0000 7f50001ac700 1 -- 172.21.15.53:0/671155898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff81020c0 msgr2=0x7f4ff81024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:28:43.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.655+0000 7f50001ac700 1 --2- 172.21.15.53:0/671155898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff81020c0 0x7f4ff81024d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009990 tx=0x7f4ff4009ca0 comp rx=0 tx=0).stop 2024-05-21T15:28:43.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.655+0000 7f50001ac700 1 -- 172.21.15.53:0/671155898 shutdown_connections 2024-05-21T15:28:43.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.655+0000 7f50001ac700 1 --2- 172.21.15.53:0/671155898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ff8103b90 0x7f4ff810a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:43.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.655+0000 7f50001ac700 1 --2- 172.21.15.53:0/671155898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff81020c0 0x7f4ff81024d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:43.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.655+0000 7f50001ac700 1 --2- 172.21.15.53:0/671155898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ff81031f0 0x7f4ff8103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:43.662 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.655+0000 7f50001ac700 1 -- 172.21.15.53:0/671155898 >> 172.21.15.53:0/671155898 conn(0x7f4ff80fd700 msgr2=0x7f4ff80ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:28:43.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.655+0000 7f50001ac700 1 -- 172.21.15.53:0/671155898 shutdown_connections 2024-05-21T15:28:43.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.655+0000 7f50001ac700 1 -- 172.21.15.53:0/671155898 wait complete. 2024-05-21T15:28:43.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.656+0000 7f50001ac700 1 Processor -- start 2024-05-21T15:28:43.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.656+0000 7f50001ac700 1 -- start start 2024-05-21T15:28:43.663 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.656+0000 7f50001ac700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ff81020c0 0x7f4ff806af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:28:43.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.656+0000 7f50001ac700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff81031f0 0x7f4ff819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:28:43.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.656+0000 7f50001ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ff8103b90 0x7f4ff81a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:28:43.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.656+0000 7f50001ac700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff810dca0 con 0x7f4ff81020c0 2024-05-21T15:28:43.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.656+0000 7f50001ac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff810db20 con 0x7f4ff8103b90 2024-05-21T15:28:43.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.656+0000 7f50001ac700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff810de20 con 0x7f4ff81031f0 2024-05-21T15:28:43.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.656+0000 7f4ffd747700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff81031f0 0x7f4ff819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:28:43.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f4ffd747700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff81031f0 0x7f4ff819e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51314/0 (socket says 172.21.15.53:51314) 2024-05-21T15:28:43.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f4ffd747700 1 -- 172.21.15.53:0/2614146930 learned_addr learned my addr 172.21.15.53:0/2614146930 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-21T15:28:43.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f4ffdf48700 1 --2- 172.21.15.53:0/2614146930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ff81020c0 0x7f4ff806af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:28:43.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f4ffe749700 1 --2- 172.21.15.53:0/2614146930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ff8103b90 0x7f4ff81a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:28:43.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f4ffd747700 1 -- 172.21.15.53:0/2614146930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ff8103b90 msgr2=0x7f4ff81a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:28:43.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f4ffd747700 1 --2- 172.21.15.53:0/2614146930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ff8103b90 0x7f4ff81a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:43.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f4ffd747700 1 -- 172.21.15.53:0/2614146930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ff81020c0 msgr2=0x7f4ff806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:28:43.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f4ffd747700 1 --2- 172.21.15.53:0/2614146930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ff81020c0 0x7f4ff806af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:28:43.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f4ffd747700 1 -- 172.21.15.53:0/2614146930 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff81a1380 con 0x7f4ff81031f0 2024-05-21T15:28:43.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f4ffd747700 1 --2- 172.21.15.53:0/2614146930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff81031f0 0x7f4ff819e610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800c9b0 tx=0x7f4fe800ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:28:43.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f4feeffd700 1 -- 172.21.15.53:0/2614146930 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8007b00 con 0x7f4ff81031f0 2024-05-21T15:28:43.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f50001ac700 1 -- 172.21.15.53:0/2614146930 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff81a1670 con 0x7f4ff81031f0 2024-05-21T15:28:43.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f4feeffd700 1 -- 172.21.15.53:0/2614146930 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe800ce50 con 0x7f4ff81031f0 2024-05-21T15:28:43.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f4feeffd700 1 -- 172.21.15.53:0/2614146930 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8018660 con 0x7f4ff81031f0 2024-05-21T15:28:43.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.657+0000 7f50001ac700 1 -- 172.21.15.53:0/2614146930 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff81a1c00 con 0x7f4ff81031f0 2024-05-21T15:28:43.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.658+0000 7f50001ac700 1 -- 172.21.15.53:0/2614146930 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff8050170 con 0x7f4ff81031f0 2024-05-21T15:28:43.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.660+0000 7f4feeffd700 1 -- 172.21.15.53:0/2614146930 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4fe800f450 con 0x7f4ff81031f0 2024-05-21T15:28:43.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.660+0000 7f4feeffd700 1 --2- 172.21.15.53:0/2614146930 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4fe40a2c20 0x7f4fe40a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:28:43.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.660+0000 7f4feeffd700 1 -- 172.21.15.53:0/2614146930 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f4fe80ce170 con 0x7f4ff81031f0 2024-05-21T15:28:43.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.661+0000 7f4ffdf48700 1 --2- 172.21.15.53:0/2614146930 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4fe40a2c20 0x7f4fe40a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:28:43.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.661+0000 7f4ffdf48700 1 --2- 172.21.15.53:0/2614146930 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4fe40a2c20 0x7f4fe40a5130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400b8e0 tx=0x7f4ff4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:28:43.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.664+0000 7f4feeffd700 1 -- 172.21.15.53:0/2614146930 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4fe8014030 con 0x7f4ff81031f0 2024-05-21T15:28:43.881 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:28:43.879+0000 7f50001ac700 1 -- 172.21.15.53:0/2614146930 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4ff8062a80 con 0x7f4fe40a2c20 2024-05-21T15:28:43.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:43 smithi053 ceph-mon[36496]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:43.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:43 smithi053 ceph-mon[36496]: osdmap e35: 5 total, 5 up, 5 in 2024-05-21T15:28:44.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:43 smithi045 ceph-mon[36409]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:44.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:43 smithi045 ceph-mon[36409]: osdmap e35: 5 total, 5 up, 5 in 2024-05-21T15:28:44.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:43 smithi038 ceph-mon[31995]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:44.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:43 smithi038 ceph-mon[31995]: osdmap e35: 5 total, 5 up, 5 in 2024-05-21T15:28:44.818 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:44 smithi053 ceph-mon[36496]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:28:44.819 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:44 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:28:44.819 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:44 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:28:44.819 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:44 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:45.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:44 smithi045 ceph-mon[36409]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:28:45.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:44 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:28:45.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:44 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:28:45.014 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:44 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:45.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:44 smithi038 ceph-mon[31995]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:28:45.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:44 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:28:45.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:44 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:28:45.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:44 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:45.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:45 smithi045 ceph-mon[36409]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:45.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:45 smithi053 ceph-mon[36496]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:46.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:45 smithi038 ceph-mon[31995]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:46.916 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:46 smithi053 ceph-mon[36496]: from='client.? 172.21.15.53:0/165952614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b94853cc-2ef8-410f-b900-023c6d74ea56"}]: dispatch 2024-05-21T15:28:46.916 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:46 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b94853cc-2ef8-410f-b900-023c6d74ea56"}]: dispatch 2024-05-21T15:28:46.916 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:46 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b94853cc-2ef8-410f-b900-023c6d74ea56"}]': finished 2024-05-21T15:28:46.916 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:46 smithi053 ceph-mon[36496]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T15:28:46.916 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:46 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:28:47.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:46 smithi045 ceph-mon[36409]: from='client.? 172.21.15.53:0/165952614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b94853cc-2ef8-410f-b900-023c6d74ea56"}]: dispatch 2024-05-21T15:28:47.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:46 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b94853cc-2ef8-410f-b900-023c6d74ea56"}]: dispatch 2024-05-21T15:28:47.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:46 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b94853cc-2ef8-410f-b900-023c6d74ea56"}]': finished 2024-05-21T15:28:47.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:46 smithi045 ceph-mon[36409]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T15:28:47.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:46 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:28:47.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:46 smithi038 ceph-mon[31995]: from='client.? 172.21.15.53:0/165952614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b94853cc-2ef8-410f-b900-023c6d74ea56"}]: dispatch 2024-05-21T15:28:47.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:46 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b94853cc-2ef8-410f-b900-023c6d74ea56"}]: dispatch 2024-05-21T15:28:47.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:46 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b94853cc-2ef8-410f-b900-023c6d74ea56"}]': finished 2024-05-21T15:28:47.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:46 smithi038 ceph-mon[31995]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T15:28:47.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:46 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:28:47.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:47 smithi053 ceph-mon[36496]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:47.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:47 smithi053 ceph-mon[36496]: from='client.? 172.21.15.53:0/2494302323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:28:48.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:47 smithi045 ceph-mon[36409]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:48.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:47 smithi045 ceph-mon[36409]: from='client.? 172.21.15.53:0/2494302323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:28:48.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:47 smithi038 ceph-mon[31995]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:48.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:47 smithi038 ceph-mon[31995]: from='client.? 172.21.15.53:0/2494302323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:28:49.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:49 smithi053 ceph-mon[36496]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:50.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:49 smithi045 ceph-mon[36409]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:50.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:49 smithi038 ceph-mon[31995]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:51.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:51 smithi053 ceph-mon[36496]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:52.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:51 smithi045 ceph-mon[36409]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:52.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:51 smithi038 ceph-mon[31995]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:53.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:53 smithi053 ceph-mon[36496]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:54.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:53 smithi045 ceph-mon[36409]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:54.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:53 smithi038 ceph-mon[31995]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:54.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-21T15:28:54.949 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:55.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-21T15:28:55.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:55.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-21T15:28:55.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:28:55.870 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:55 smithi053 ceph-mon[36496]: Deploying daemon osd.5 on smithi053 2024-05-21T15:28:55.870 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:55 smithi053 ceph-mon[36496]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:56.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:55 smithi045 ceph-mon[36409]: Deploying daemon osd.5 on smithi053 2024-05-21T15:28:56.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:55 smithi045 ceph-mon[36409]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:56.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:55 smithi038 ceph-mon[31995]: Deploying daemon osd.5 on smithi053 2024-05-21T15:28:56.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:55 smithi038 ceph-mon[31995]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:57.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:57 smithi053 ceph-mon[36496]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:58.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:57 smithi045 ceph-mon[36409]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:58.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:57 smithi038 ceph-mon[31995]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:28:59.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:28:59 smithi053 ceph-mon[36496]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:00.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:28:59 smithi045 ceph-mon[36409]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:00.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:28:59 smithi038 ceph-mon[31995]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:01.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:01.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:01.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:01.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:00 smithi045 ceph-mon[36409]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:01.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:01.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:01.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:01.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:00 smithi038 ceph-mon[31995]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:01.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:01.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:01.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:01.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:00 smithi053 ceph-mon[36496]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:03.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:03 smithi045 ceph-mon[36409]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:03.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:03.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:03.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:03.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:03.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:03.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:03 smithi038 ceph-mon[31995]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:03.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:03.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:03.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:03.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:03.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:03.647 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:03 smithi053 ceph-mon[36496]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:03.647 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:03.647 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:03.647 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:03.647 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:03.647 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:04.003 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:29:04.001+0000 7f4feeffd700 1 -- 172.21.15.53:0/2614146930 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4ff8062a80 con 0x7f4fe40a2c20 2024-05-21T15:29:04.003 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 5 on host 'smithi053' 2024-05-21T15:29:04.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:29:04.003+0000 7f50001ac700 1 -- 172.21.15.53:0/2614146930 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4fe40a2c20 msgr2=0x7f4fe40a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:29:04.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:29:04.003+0000 7f50001ac700 1 --2- 172.21.15.53:0/2614146930 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4fe40a2c20 0x7f4fe40a5130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400b8e0 tx=0x7f4ff4005fd0 comp rx=0 tx=0).stop 2024-05-21T15:29:04.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:29:04.003+0000 7f50001ac700 1 -- 172.21.15.53:0/2614146930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff81031f0 msgr2=0x7f4ff819e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:29:04.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:29:04.003+0000 7f50001ac700 1 --2- 172.21.15.53:0/2614146930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff81031f0 0x7f4ff819e610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800c9b0 tx=0x7f4fe800ccc0 comp rx=0 tx=0).stop 2024-05-21T15:29:04.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:29:04.003+0000 7f50001ac700 1 -- 172.21.15.53:0/2614146930 shutdown_connections 2024-05-21T15:29:04.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:29:04.003+0000 7f50001ac700 1 --2- 172.21.15.53:0/2614146930 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4fe40a2c20 0x7f4fe40a5130 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:04.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:29:04.003+0000 7f50001ac700 1 --2- 172.21.15.53:0/2614146930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ff8103b90 0x7f4ff81a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:04.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:29:04.003+0000 7f50001ac700 1 --2- 172.21.15.53:0/2614146930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ff81020c0 0x7f4ff806af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:04.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:29:04.003+0000 7f50001ac700 1 --2- 172.21.15.53:0/2614146930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff81031f0 0x7f4ff819e610 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:04.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:29:04.003+0000 7f50001ac700 1 -- 172.21.15.53:0/2614146930 >> 172.21.15.53:0/2614146930 conn(0x7f4ff80fd700 msgr2=0x7f4ff8106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:29:04.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:29:04.003+0000 7f50001ac700 1 -- 172.21.15.53:0/2614146930 shutdown_connections 2024-05-21T15:29:04.006 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:29:04.003+0000 7f50001ac700 1 -- 172.21.15.53:0/2614146930 wait complete. 2024-05-21T15:29:04.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:04 smithi045 ceph-mon[36409]: from='osd.5 [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T15:29:04.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:04 smithi045 ceph-mon[36409]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T15:29:04.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:04 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:04.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:04 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:04.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:04 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:04.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:04 smithi038 ceph-mon[31995]: from='osd.5 [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T15:29:04.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:04 smithi038 ceph-mon[31995]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T15:29:04.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:04 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:04.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:04 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:04.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:04 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:04.669 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:04 smithi053 ceph-mon[36496]: from='osd.5 [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T15:29:04.669 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:04 smithi053 ceph-mon[36496]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T15:29:04.669 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:04 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:04.669 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:04 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:04.669 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:04 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:04.696 DEBUG:teuthology.orchestra.run.smithi053:osd.5> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.5.service 2024-05-21T15:29:04.698 INFO:tasks.cephadm:Deploying osd.6 on smithi038 with /dev/vg_nvme/lv_2... 2024-05-21T15:29:04.698 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-21T15:29:04.958 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:29:05.281 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:05 smithi053 ceph-mon[36496]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:05.281 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:05 smithi053 ceph-mon[36496]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T15:29:05.281 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:05 smithi053 ceph-mon[36496]: from='osd.5 [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:29:05.281 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:05 smithi053 ceph-mon[36496]: osdmap e37: 6 total, 5 up, 6 in 2024-05-21T15:29:05.282 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:05 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:29:05.282 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:05 smithi053 ceph-mon[36496]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:29:05.282 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:05 smithi053 ceph-mon[36496]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-21T15:29:05.334 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:05 smithi038 ceph-mon[31995]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:05.334 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:05 smithi038 ceph-mon[31995]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T15:29:05.334 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:05 smithi038 ceph-mon[31995]: from='osd.5 [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:29:05.335 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:05 smithi038 ceph-mon[31995]: osdmap e37: 6 total, 5 up, 6 in 2024-05-21T15:29:05.335 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:05 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:29:05.335 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:05 smithi038 ceph-mon[31995]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:29:05.335 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:05 smithi038 ceph-mon[31995]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-21T15:29:05.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:05 smithi045 ceph-mon[36409]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:05.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:05 smithi045 ceph-mon[36409]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T15:29:05.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:05 smithi045 ceph-mon[36409]: from='osd.5 [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:29:05.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:05 smithi045 ceph-mon[36409]: osdmap e37: 6 total, 5 up, 6 in 2024-05-21T15:29:05.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:05 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:29:05.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:05 smithi045 ceph-mon[36409]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:29:05.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:05 smithi045 ceph-mon[36409]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-21T15:29:05.934 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:29:05 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-5[48261]: 2024-05-21T15:29:05.726+0000 7f0d11424700 -1 osd.5 0 waiting for initial osdmap 2024-05-21T15:29:05.935 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:29:05 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-5[48261]: 2024-05-21T15:29:05.734+0000 7f0d091d1700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:29:06.420 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:06 smithi038 ceph-mon[31995]: osdmap e38: 6 total, 5 up, 6 in 2024-05-21T15:29:06.420 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:06 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:29:06.420 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:06 smithi038 ceph-mon[31995]: from='osd.5 ' entity='osd.5' 2024-05-21T15:29:06.420 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:06 smithi038 ceph-mon[31995]: osd.5 [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902] boot 2024-05-21T15:29:06.420 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:06 smithi038 ceph-mon[31995]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T15:29:06.421 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:06 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:29:06.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:06 smithi045 ceph-mon[36409]: osdmap e38: 6 total, 5 up, 6 in 2024-05-21T15:29:06.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:06 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:29:06.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:06 smithi045 ceph-mon[36409]: from='osd.5 ' entity='osd.5' 2024-05-21T15:29:06.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:06 smithi045 ceph-mon[36409]: osd.5 [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902] boot 2024-05-21T15:29:06.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:06 smithi045 ceph-mon[36409]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T15:29:06.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:06 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:29:06.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:06 smithi053 ceph-mon[36496]: osdmap e38: 6 total, 5 up, 6 in 2024-05-21T15:29:06.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:06 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:29:06.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:06 smithi053 ceph-mon[36496]: from='osd.5 ' entity='osd.5' 2024-05-21T15:29:06.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:06 smithi053 ceph-mon[36496]: osd.5 [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902] boot 2024-05-21T15:29:06.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:06 smithi053 ceph-mon[36496]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T15:29:06.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:06 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:29:07.007 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-21T15:29:07.008 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-21T15:29:07.008 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-05-21T15:29:07.008 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-05-21T15:29:07.008 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00844377 s, 1.2 GB/s 2024-05-21T15:29:07.008 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-05-21T15:29:07.795 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_2 2024-05-21T15:29:08.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:07 smithi038 ceph-mon[31995]: purged_snaps scrub starts 2024-05-21T15:29:08.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:07 smithi038 ceph-mon[31995]: purged_snaps scrub ok 2024-05-21T15:29:08.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:07 smithi038 ceph-mon[31995]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:08.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:07 smithi038 ceph-mon[31995]: osdmap e40: 6 total, 6 up, 6 in 2024-05-21T15:29:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:29:08.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:07 smithi053 ceph-mon[36496]: purged_snaps scrub starts 2024-05-21T15:29:08.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:07 smithi053 ceph-mon[36496]: purged_snaps scrub ok 2024-05-21T15:29:08.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:07 smithi053 ceph-mon[36496]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:08.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:07 smithi053 ceph-mon[36496]: osdmap e40: 6 total, 6 up, 6 in 2024-05-21T15:29:08.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:07 smithi045 ceph-mon[36409]: purged_snaps scrub starts 2024-05-21T15:29:08.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:07 smithi045 ceph-mon[36409]: purged_snaps scrub ok 2024-05-21T15:29:08.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:07 smithi045 ceph-mon[36409]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:29:08.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:07 smithi045 ceph-mon[36409]: osdmap e40: 6 total, 6 up, 6 in 2024-05-21T15:29:09.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:08 smithi053 ceph-mon[36496]: osdmap e41: 6 total, 6 up, 6 in 2024-05-21T15:29:09.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:08 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:09.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:08 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:09.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:08 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:09.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:08 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:09.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:08 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:09.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:08 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:09.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:08 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:09.209 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:08 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:09.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:08 smithi045 ceph-mon[36409]: osdmap e41: 6 total, 6 up, 6 in 2024-05-21T15:29:09.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:08 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:09.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:08 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:09.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:08 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:09.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:08 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:09.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:08 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:09.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:08 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:09.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:08 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:09.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:08 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:09.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:08 smithi038 ceph-mon[31995]: osdmap e41: 6 total, 6 up, 6 in 2024-05-21T15:29:09.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:08 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:09.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:08 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:09.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:08 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:09.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:08 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:09.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:08 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:09.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:08 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:09.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:08 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:09.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:08 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.867+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1320839219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7f0103860 msgr2=0x7fc7f010b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:29:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.867+0000 7fc7f6a92700 1 --2- 172.21.15.38:0/1320839219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7f0103860 0x7fc7f010b570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800b0a0 tx=0x7fc7e800b3b0 comp rx=0 tx=0).stop 2024-05-21T15:29:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.867+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1320839219 shutdown_connections 2024-05-21T15:29:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.867+0000 7fc7f6a92700 1 --2- 172.21.15.38:0/1320839219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7f0103860 0x7fc7f010b570 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.867+0000 7fc7f6a92700 1 --2- 172.21.15.38:0/1320839219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f0104830 0x7fc7f0104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.867+0000 7fc7f6a92700 1 --2- 172.21.15.38:0/1320839219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7f0102de0 0x7fc7f01031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.867+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1320839219 >> 172.21.15.38:0/1320839219 conn(0x7fc7f00fe850 msgr2=0x7fc7f0100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:29:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.867+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1320839219 shutdown_connections 2024-05-21T15:29:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.867+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1320839219 wait complete. 2024-05-21T15:29:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7f6a92700 1 Processor -- start 2024-05-21T15:29:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7f6a92700 1 -- start start 2024-05-21T15:29:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7f6a92700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f0102de0 0x7fc7f01998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:29:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7f6a92700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7f0103860 0x7fc7f0199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:29:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7f6a92700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7f0104830 0x7fc7f01a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:29:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7f6a92700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f010eec0 con 0x7fc7f0103860 2024-05-21T15:29:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7f6a92700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f010ed40 con 0x7fc7f0102de0 2024-05-21T15:29:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7f6a92700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f010f040 con 0x7fc7f0104830 2024-05-21T15:29:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7effff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7f0103860 0x7fc7f0199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:29:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7effff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7f0103860 0x7fc7f0199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54066/0 (socket says 172.21.15.38:54066) 2024-05-21T15:29:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7effff700 1 -- 172.21.15.38:0/1803635353 learned_addr learned my addr 172.21.15.38:0/1803635353 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:29:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7f502f700 1 --2- 172.21.15.38:0/1803635353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7f0104830 0x7fc7f01a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:29:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7f482e700 1 --2- 172.21.15.38:0/1803635353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f0102de0 0x7fc7f01998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:29:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7effff700 1 -- 172.21.15.38:0/1803635353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7f0104830 msgr2=0x7fc7f01a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:29:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7effff700 1 --2- 172.21.15.38:0/1803635353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7f0104830 0x7fc7f01a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7effff700 1 -- 172.21.15.38:0/1803635353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f0102de0 msgr2=0x7fc7f01998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:29:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7effff700 1 --2- 172.21.15.38:0/1803635353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f0102de0 0x7fc7f01998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.868+0000 7fc7effff700 1 -- 172.21.15.38:0/1803635353 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f01a1400 con 0x7fc7f0103860 2024-05-21T15:29:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.869+0000 7fc7effff700 1 --2- 172.21.15.38:0/1803635353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7f0103860 0x7fc7f0199e30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800d950 tx=0x7fc7d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:29:09.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.869+0000 7fc7edffb700 1 -- 172.21.15.38:0/1803635353 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8016070 con 0x7fc7f0103860 2024-05-21T15:29:09.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.869+0000 7fc7edffb700 1 -- 172.21.15.38:0/1803635353 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d800fd70 con 0x7fc7f0103860 2024-05-21T15:29:09.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.869+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1803635353 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f01a16f0 con 0x7fc7f0103860 2024-05-21T15:29:09.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.869+0000 7fc7edffb700 1 -- 172.21.15.38:0/1803635353 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d800e610 con 0x7fc7f0103860 2024-05-21T15:29:09.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.869+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1803635353 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f01a1c80 con 0x7fc7f0103860 2024-05-21T15:29:09.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.870+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1803635353 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f0050170 con 0x7fc7f0103860 2024-05-21T15:29:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.871+0000 7fc7edffb700 1 -- 172.21.15.38:0/1803635353 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc7d8019070 con 0x7fc7f0103860 2024-05-21T15:29:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.872+0000 7fc7edffb700 1 --2- 172.21.15.38:0/1803635353 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc7dc0a2ca0 0x7fc7dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:29:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.872+0000 7fc7edffb700 1 -- 172.21.15.38:0/1803635353 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7fc7d80cdbb0 con 0x7fc7f0103860 2024-05-21T15:29:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.872+0000 7fc7f482e700 1 --2- 172.21.15.38:0/1803635353 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc7dc0a2ca0 0x7fc7dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:29:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.873+0000 7fc7f482e700 1 --2- 172.21.15.38:0/1803635353 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc7dc0a2ca0 0x7fc7dc0a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0104220 tx=0x7fc7e0007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:29:09.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:09.877+0000 7fc7edffb700 1 -- 172.21.15.38:0/1803635353 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc7d809b9d0 con 0x7fc7f0103860 2024-05-21T15:29:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:10.094+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1803635353 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc7f002e010 con 0x7fc7dc0a2ca0 2024-05-21T15:29:10.112 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:09 smithi038 ceph-mon[31995]: Detected new or changed devices on smithi053 2024-05-21T15:29:10.112 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:09 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi053 to 8546M 2024-05-21T15:29:10.112 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:09 smithi038 ceph-mon[31995]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:10.112 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:09 smithi038 ceph-mon[31995]: osdmap e42: 6 total, 6 up, 6 in 2024-05-21T15:29:10.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:09 smithi053 ceph-mon[36496]: Detected new or changed devices on smithi053 2024-05-21T15:29:10.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:09 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi053 to 8546M 2024-05-21T15:29:10.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:09 smithi053 ceph-mon[36496]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:10.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:09 smithi053 ceph-mon[36496]: osdmap e42: 6 total, 6 up, 6 in 2024-05-21T15:29:10.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:09 smithi045 ceph-mon[36409]: Detected new or changed devices on smithi053 2024-05-21T15:29:10.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:09 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi053 to 8546M 2024-05-21T15:29:10.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:09 smithi045 ceph-mon[36409]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:10.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:09 smithi045 ceph-mon[36409]: osdmap e42: 6 total, 6 up, 6 in 2024-05-21T15:29:10.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:10 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:29:10.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:10 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:29:10.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:10 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:10.986 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:10 smithi038 ceph-mon[31995]: osdmap e43: 6 total, 6 up, 6 in 2024-05-21T15:29:11.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:10 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:29:11.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:10 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:29:11.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:10 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:11.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:10 smithi053 ceph-mon[36496]: osdmap e43: 6 total, 6 up, 6 in 2024-05-21T15:29:11.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:10 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:29:11.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:10 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:29:11.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:10 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:11.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:10 smithi045 ceph-mon[36409]: osdmap e43: 6 total, 6 up, 6 in 2024-05-21T15:29:12.100 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:11 smithi038 ceph-mon[31995]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:29:12.100 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:11 smithi038 ceph-mon[31995]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:12.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:11 smithi053 ceph-mon[36496]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:29:12.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:11 smithi053 ceph-mon[36496]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:12.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:11 smithi045 ceph-mon[36409]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:29:12.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:11 smithi045 ceph-mon[36409]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:14.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:13 smithi053 ceph-mon[36496]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-21T15:29:14.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:13 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1876591600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d609b60e-d3c8-482f-850d-f313e97c8c2e"}]: dispatch 2024-05-21T15:29:14.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:13 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1876591600' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d609b60e-d3c8-482f-850d-f313e97c8c2e"}]': finished 2024-05-21T15:29:14.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:13 smithi053 ceph-mon[36496]: osdmap e44: 7 total, 6 up, 7 in 2024-05-21T15:29:14.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:13 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:14.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:13 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2759778226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:29:14.251 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:13 smithi045 ceph-mon[36409]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-21T15:29:14.251 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:13 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1876591600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d609b60e-d3c8-482f-850d-f313e97c8c2e"}]: dispatch 2024-05-21T15:29:14.251 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:13 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1876591600' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d609b60e-d3c8-482f-850d-f313e97c8c2e"}]': finished 2024-05-21T15:29:14.251 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:13 smithi045 ceph-mon[36409]: osdmap e44: 7 total, 6 up, 7 in 2024-05-21T15:29:14.251 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:13 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:14.251 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:13 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2759778226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:29:14.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:13 smithi038 ceph-mon[31995]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-21T15:29:14.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:13 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1876591600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d609b60e-d3c8-482f-850d-f313e97c8c2e"}]: dispatch 2024-05-21T15:29:14.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:13 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1876591600' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d609b60e-d3c8-482f-850d-f313e97c8c2e"}]': finished 2024-05-21T15:29:14.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:13 smithi038 ceph-mon[31995]: osdmap e44: 7 total, 6 up, 7 in 2024-05-21T15:29:14.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:13 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:14.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:13 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2759778226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:29:16.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:15 smithi053 ceph-mon[36496]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:29:16.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:15 smithi045 ceph-mon[36409]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:29:16.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:15 smithi038 ceph-mon[31995]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:29:18.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:17 smithi053 ceph-mon[36496]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-21T15:29:18.249 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:17 smithi038 ceph-mon[31995]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-21T15:29:18.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:17 smithi045 ceph-mon[36409]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-21T15:29:20.181 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:19 smithi038 ceph-mon[31995]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:29:20.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:19 smithi053 ceph-mon[36496]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:29:20.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:19 smithi045 ceph-mon[36409]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:29:21.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:20 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-21T15:29:21.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:20 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:21.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:20 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-21T15:29:21.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:20 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:21.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:20 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-21T15:29:21.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:20 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:22.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:21 smithi053 ceph-mon[36496]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:29:22.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:21 smithi053 ceph-mon[36496]: Deploying daemon osd.6 on smithi038 2024-05-21T15:29:22.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:21 smithi045 ceph-mon[36409]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:29:22.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:21 smithi045 ceph-mon[36409]: Deploying daemon osd.6 on smithi038 2024-05-21T15:29:22.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:21 smithi038 ceph-mon[31995]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:29:22.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:21 smithi038 ceph-mon[31995]: Deploying daemon osd.6 on smithi038 2024-05-21T15:29:24.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:23 smithi053 ceph-mon[36496]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:24.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:23 smithi045 ceph-mon[36409]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:24.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:23 smithi038 ceph-mon[31995]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:26.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:25 smithi038 ceph-mon[31995]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:26.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:25 smithi053 ceph-mon[36496]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:26.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:25 smithi045 ceph-mon[36409]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:27.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:27 smithi053 ceph-mon[36496]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:27.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:27.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:27.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:27.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:27 smithi045 ceph-mon[36409]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:27.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:27.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:27.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:27.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:27 smithi038 ceph-mon[31995]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:27.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:27.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:27.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:29.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:29 smithi053 ceph-mon[36496]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:29.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:29 smithi045 ceph-mon[36409]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:29.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:29 smithi038 ceph-mon[31995]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:29.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:29.957+0000 7fc7edffb700 1 -- 172.21.15.38:0/1803635353 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc7f002e010 con 0x7fc7dc0a2ca0 2024-05-21T15:29:29.960 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 6 on host 'smithi038' 2024-05-21T15:29:29.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:29.959+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1803635353 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc7dc0a2ca0 msgr2=0x7fc7dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:29:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:29.959+0000 7fc7f6a92700 1 --2- 172.21.15.38:0/1803635353 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc7dc0a2ca0 0x7fc7dc0a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0104220 tx=0x7fc7e0007400 comp rx=0 tx=0).stop 2024-05-21T15:29:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:29.959+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1803635353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7f0103860 msgr2=0x7fc7f0199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:29:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:29.960+0000 7fc7f6a92700 1 --2- 172.21.15.38:0/1803635353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7f0103860 0x7fc7f0199e30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800d950 tx=0x7fc7d800dc60 comp rx=0 tx=0).stop 2024-05-21T15:29:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:29.960+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1803635353 shutdown_connections 2024-05-21T15:29:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:29.960+0000 7fc7f6a92700 1 --2- 172.21.15.38:0/1803635353 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc7dc0a2ca0 0x7fc7dc0a51b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:29.960+0000 7fc7f6a92700 1 --2- 172.21.15.38:0/1803635353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7f0104830 0x7fc7f01a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:29.960+0000 7fc7f6a92700 1 --2- 172.21.15.38:0/1803635353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f0102de0 0x7fc7f01998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:29.960+0000 7fc7f6a92700 1 --2- 172.21.15.38:0/1803635353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7f0103860 0x7fc7f0199e30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:29.960+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1803635353 >> 172.21.15.38:0/1803635353 conn(0x7fc7f00fe850 msgr2=0x7fc7f0107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:29:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:29.960+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1803635353 shutdown_connections 2024-05-21T15:29:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:29:29.960+0000 7fc7f6a92700 1 -- 172.21.15.38:0/1803635353 wait complete. 2024-05-21T15:29:30.676 DEBUG:teuthology.orchestra.run.smithi038:osd.6> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.6.service 2024-05-21T15:29:30.679 INFO:tasks.cephadm:Deploying osd.7 on smithi045 with /dev/vg_nvme/lv_2... 2024-05-21T15:29:30.679 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-21T15:29:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:29:31.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:30 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:31.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:30 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:31.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:30 smithi045 ceph-mon[36409]: from='osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-21T15:29:31.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:30 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:31.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:30 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:31.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:30 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:31.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:30 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:31.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:30 smithi053 ceph-mon[36496]: from='osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-21T15:29:31.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:30 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:31.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:30 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:31.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:30 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:31.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:30 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:31.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:30 smithi038 ceph-mon[31995]: from='osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-21T15:29:31.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:30 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:31.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:30 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:32.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:31 smithi053 ceph-mon[36496]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:32.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:31 smithi053 ceph-mon[36496]: from='osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T15:29:32.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:31 smithi053 ceph-mon[36496]: osdmap e45: 7 total, 6 up, 7 in 2024-05-21T15:29:32.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:31 smithi053 ceph-mon[36496]: from='osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-05-21T15:29:32.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:32.228 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:31 smithi045 ceph-mon[36409]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:32.228 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:31 smithi045 ceph-mon[36409]: from='osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T15:29:32.228 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:31 smithi045 ceph-mon[36409]: osdmap e45: 7 total, 6 up, 7 in 2024-05-21T15:29:32.228 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:31 smithi045 ceph-mon[36409]: from='osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-05-21T15:29:32.228 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:32.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:31 smithi038 ceph-mon[31995]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:32.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:31 smithi038 ceph-mon[31995]: from='osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T15:29:32.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:31 smithi038 ceph-mon[31995]: osdmap e45: 7 total, 6 up, 7 in 2024-05-21T15:29:32.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:31 smithi038 ceph-mon[31995]: from='osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-05-21T15:29:32.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:32.277 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:29:31 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-6[59961]: 2024-05-21T15:29:31.971+0000 7f6057c63700 -1 osd.6 0 waiting for initial osdmap 2024-05-21T15:29:32.277 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:29:31 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-6[59961]: 2024-05-21T15:29:31.983+0000 7f6053a18700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:29:33.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:32 smithi038 ceph-mon[31995]: from='osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-05-21T15:29:33.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:32 smithi038 ceph-mon[31995]: osdmap e46: 7 total, 6 up, 7 in 2024-05-21T15:29:33.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:33.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:33.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:32 smithi038 ceph-mon[31995]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:33.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:32 smithi045 ceph-mon[36409]: from='osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-05-21T15:29:33.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:32 smithi045 ceph-mon[36409]: osdmap e46: 7 total, 6 up, 7 in 2024-05-21T15:29:33.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:33.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:33.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:32 smithi045 ceph-mon[36409]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-21T15:29:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-21T15:29:33.288 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-05-21T15:29:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-05-21T15:29:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00853137 s, 1.2 GB/s 2024-05-21T15:29:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-05-21T15:29:33.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:32 smithi053 ceph-mon[36496]: from='osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-05-21T15:29:33.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:32 smithi053 ceph-mon[36496]: osdmap e46: 7 total, 6 up, 7 in 2024-05-21T15:29:33.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:33.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:33.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:32 smithi053 ceph-mon[36496]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:29:33.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_2 2024-05-21T15:29:34.239 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:29:34.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:33 smithi045 ceph-mon[36409]: purged_snaps scrub starts 2024-05-21T15:29:34.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:33 smithi045 ceph-mon[36409]: purged_snaps scrub ok 2024-05-21T15:29:34.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:33 smithi045 ceph-mon[36409]: osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500] boot 2024-05-21T15:29:34.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:33 smithi045 ceph-mon[36409]: osdmap e47: 7 total, 7 up, 7 in 2024-05-21T15:29:34.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:34.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:33 smithi038 ceph-mon[31995]: purged_snaps scrub starts 2024-05-21T15:29:34.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:33 smithi038 ceph-mon[31995]: purged_snaps scrub ok 2024-05-21T15:29:34.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:33 smithi038 ceph-mon[31995]: osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500] boot 2024-05-21T15:29:34.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:33 smithi038 ceph-mon[31995]: osdmap e47: 7 total, 7 up, 7 in 2024-05-21T15:29:34.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:34.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:33 smithi053 ceph-mon[36496]: purged_snaps scrub starts 2024-05-21T15:29:34.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:33 smithi053 ceph-mon[36496]: purged_snaps scrub ok 2024-05-21T15:29:34.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:33 smithi053 ceph-mon[36496]: osd.6 [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500] boot 2024-05-21T15:29:34.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:33 smithi053 ceph-mon[36496]: osdmap e47: 7 total, 7 up, 7 in 2024-05-21T15:29:34.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:29:35.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: osdmap e48: 7 total, 7 up, 7 in 2024-05-21T15:29:35.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:35.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: Detected new or changed devices on smithi038 2024-05-21T15:29:35.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:35.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:35.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:35.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi038 to 5697M 2024-05-21T15:29:35.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:35.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:35.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:35.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:35.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:35.264 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: osdmap e48: 7 total, 7 up, 7 in 2024-05-21T15:29:35.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:35.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: Detected new or changed devices on smithi038 2024-05-21T15:29:35.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:35.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:35.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:35.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi038 to 5697M 2024-05-21T15:29:35.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:35.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:35.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:35.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:35.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:35.278 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: osdmap e48: 7 total, 7 up, 7 in 2024-05-21T15:29:35.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:35.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: Detected new or changed devices on smithi038 2024-05-21T15:29:35.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:35.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:35.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:29:35.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi038 to 5697M 2024-05-21T15:29:35.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:35.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:35.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:35.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:35.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:35.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:35.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.929+0000 7f148dd88700 1 -- 172.21.15.45:0/4151284850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1488103860 msgr2=0x7f148810b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:29:35.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.929+0000 7f148dd88700 1 --2- 172.21.15.45:0/4151284850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1488103860 0x7f148810b570 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f148000b0a0 tx=0x7f148000b3b0 comp rx=0 tx=0).stop 2024-05-21T15:29:35.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.930+0000 7f148dd88700 1 -- 172.21.15.45:0/4151284850 shutdown_connections 2024-05-21T15:29:35.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.930+0000 7f148dd88700 1 --2- 172.21.15.45:0/4151284850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1488103860 0x7f148810b570 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:35.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.930+0000 7f148dd88700 1 --2- 172.21.15.45:0/4151284850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1488104830 0x7f1488104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:35.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.930+0000 7f148dd88700 1 --2- 172.21.15.45:0/4151284850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1488102de0 0x7f14881031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:35.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.930+0000 7f148dd88700 1 -- 172.21.15.45:0/4151284850 >> 172.21.15.45:0/4151284850 conn(0x7f14880fe850 msgr2=0x7f1488100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:29:35.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.930+0000 7f148dd88700 1 -- 172.21.15.45:0/4151284850 shutdown_connections 2024-05-21T15:29:35.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.930+0000 7f148dd88700 1 -- 172.21.15.45:0/4151284850 wait complete. 2024-05-21T15:29:35.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.930+0000 7f148dd88700 1 Processor -- start 2024-05-21T15:29:35.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f148dd88700 1 -- start start 2024-05-21T15:29:35.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f148dd88700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1488102de0 0x7f14880773f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:29:35.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f148dd88700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1488103860 0x7f1488075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:29:35.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f148dd88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1488104830 0x7f1488075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:29:35.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f148dd88700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f148810ef40 con 0x7f1488103860 2024-05-21T15:29:35.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f148dd88700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f148810edc0 con 0x7f1488104830 2024-05-21T15:29:35.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f148dd88700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f148810f0c0 con 0x7f1488102de0 2024-05-21T15:29:35.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f1487fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1488104830 0x7f1488075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:29:35.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f1487fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1488104830 0x7f1488075fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37178/0 (socket says 172.21.15.45:37178) 2024-05-21T15:29:35.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f1487fff700 1 -- 172.21.15.45:0/3307868525 learned_addr learned my addr 172.21.15.45:0/3307868525 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-21T15:29:35.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f1486ffd700 1 --2- 172.21.15.45:0/3307868525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1488103860 0x7f1488075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:29:35.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f14877fe700 1 --2- 172.21.15.45:0/3307868525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1488102de0 0x7f14880773f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:29:35.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f1487fff700 1 -- 172.21.15.45:0/3307868525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1488102de0 msgr2=0x7f14880773f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:29:35.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f1487fff700 1 --2- 172.21.15.45:0/3307868525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1488102de0 0x7f14880773f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:35.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f1487fff700 1 -- 172.21.15.45:0/3307868525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1488103860 msgr2=0x7f1488075aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:29:35.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f1487fff700 1 --2- 172.21.15.45:0/3307868525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1488103860 0x7f1488075aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:35.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.931+0000 7f1487fff700 1 -- 172.21.15.45:0/3307868525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1488076740 con 0x7f1488104830 2024-05-21T15:29:35.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.932+0000 7f1487fff700 1 --2- 172.21.15.45:0/3307868525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1488104830 0x7f1488075fe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1480009cd0 tx=0x7f1480004e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:29:35.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.932+0000 7f1484ff9700 1 -- 172.21.15.45:0/3307868525 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480021070 con 0x7f1488104830 2024-05-21T15:29:35.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.932+0000 7f1484ff9700 1 -- 172.21.15.45:0/3307868525 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1480004020 con 0x7f1488104830 2024-05-21T15:29:35.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.932+0000 7f1484ff9700 1 -- 172.21.15.45:0/3307868525 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148001d6b0 con 0x7f1488104830 2024-05-21T15:29:35.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.932+0000 7f148dd88700 1 -- 172.21.15.45:0/3307868525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14881a3a70 con 0x7f1488104830 2024-05-21T15:29:35.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.932+0000 7f148dd88700 1 -- 172.21.15.45:0/3307868525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14881a3ec0 con 0x7f1488104830 2024-05-21T15:29:35.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.933+0000 7f148dd88700 1 -- 172.21.15.45:0/3307868525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1488050170 con 0x7f1488104830 2024-05-21T15:29:35.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.934+0000 7f1484ff9700 1 -- 172.21.15.45:0/3307868525 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f14800041d0 con 0x7f1488104830 2024-05-21T15:29:35.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.935+0000 7f1484ff9700 1 --2- 172.21.15.45:0/3307868525 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f14700a2ca0 0x7f14700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:29:35.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.935+0000 7f1484ff9700 1 -- 172.21.15.45:0/3307868525 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f148000f020 con 0x7f1488104830 2024-05-21T15:29:35.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.935+0000 7f14877fe700 1 --2- 172.21.15.45:0/3307868525 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f14700a2ca0 0x7f14700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:29:35.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.936+0000 7f14877fe700 1 --2- 172.21.15.45:0/3307868525 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f14700a2ca0 0x7f14700a51b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1478006fd0 tx=0x7f1478008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:29:35.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:35.940+0000 7f1484ff9700 1 -- 172.21.15.45:0/3307868525 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f148002a020 con 0x7f1488104830 2024-05-21T15:29:36.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:36.158+0000 7f148dd88700 1 -- 172.21.15.45:0/3307868525 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f148802e080 con 0x7f14700a2ca0 2024-05-21T15:29:36.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:36 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:29:36.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:36 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:29:36.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:36 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:36.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:29:36.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:29:36.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:36.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:36 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:29:36.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:36 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:29:36.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:36 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:37.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:37 smithi045 ceph-mon[36409]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:29:37.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:37 smithi045 ceph-mon[36409]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:37.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:37 smithi038 ceph-mon[31995]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:29:37.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:37 smithi038 ceph-mon[31995]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:37.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:37 smithi053 ceph-mon[36496]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:29:37.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:37 smithi053 ceph-mon[36496]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:39 smithi045 ceph-mon[36409]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:39 smithi045 ceph-mon[36409]: from='client.? 172.21.15.45:0/839387835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7"}]: dispatch 2024-05-21T15:29:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:39 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7"}]: dispatch 2024-05-21T15:29:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:39 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7"}]': finished 2024-05-21T15:29:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:39 smithi045 ceph-mon[36409]: osdmap e49: 8 total, 7 up, 8 in 2024-05-21T15:29:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:39 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:39.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:39 smithi038 ceph-mon[31995]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:39.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:39 smithi038 ceph-mon[31995]: from='client.? 172.21.15.45:0/839387835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7"}]: dispatch 2024-05-21T15:29:39.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:39 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7"}]: dispatch 2024-05-21T15:29:39.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:39 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7"}]': finished 2024-05-21T15:29:39.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:39 smithi038 ceph-mon[31995]: osdmap e49: 8 total, 7 up, 8 in 2024-05-21T15:29:39.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:39 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:39.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:39 smithi053 ceph-mon[36496]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:39.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:39 smithi053 ceph-mon[36496]: from='client.? 172.21.15.45:0/839387835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7"}]: dispatch 2024-05-21T15:29:39.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:39 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7"}]: dispatch 2024-05-21T15:29:39.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:39 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7"}]': finished 2024-05-21T15:29:39.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:39 smithi053 ceph-mon[36496]: osdmap e49: 8 total, 7 up, 8 in 2024-05-21T15:29:39.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:39 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:40.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:40 smithi045 ceph-mon[36409]: from='client.? 172.21.15.45:0/1586840784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:29:40.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:40 smithi038 ceph-mon[31995]: from='client.? 172.21.15.45:0/1586840784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:29:40.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:40 smithi053 ceph-mon[36496]: from='client.? 172.21.15.45:0/1586840784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:29:41.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:41 smithi045 ceph-mon[36409]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:41.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:41 smithi038 ceph-mon[31995]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:41.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:41 smithi053 ceph-mon[36496]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:43.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:43 smithi045 ceph-mon[36409]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:43.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:43 smithi038 ceph-mon[31995]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:43.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:43 smithi053 ceph-mon[36496]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:45.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:45 smithi045 ceph-mon[36409]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:45.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:45 smithi038 ceph-mon[31995]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:45.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:45 smithi053 ceph-mon[36496]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:47.353 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:47 smithi045 ceph-mon[36409]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:47.353 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:47 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-21T15:29:47.353 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:47 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:47.353 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:47 smithi045 ceph-mon[36409]: Deploying daemon osd.7 on smithi045 2024-05-21T15:29:47.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:47 smithi038 ceph-mon[31995]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:47.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:47 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-21T15:29:47.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:47 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:47.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:47 smithi038 ceph-mon[31995]: Deploying daemon osd.7 on smithi045 2024-05-21T15:29:47.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:47 smithi053 ceph-mon[36496]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:47.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:47 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-21T15:29:47.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:47 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:47.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:47 smithi053 ceph-mon[36496]: Deploying daemon osd.7 on smithi045 2024-05-21T15:29:49.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:49 smithi038 ceph-mon[31995]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:49.599 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:49 smithi045 ceph-mon[36409]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:49.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:49 smithi053 ceph-mon[36496]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:51.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:51 smithi038 ceph-mon[31995]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:51.575 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:51 smithi045 ceph-mon[36409]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:51.709 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:51 smithi053 ceph-mon[36496]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:53.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:53 smithi045 ceph-mon[36409]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:53.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:53.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:53.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:53.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:53 smithi038 ceph-mon[31995]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:53.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:53.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:53.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:53.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:53 smithi053 ceph-mon[36496]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:53.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:53.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:53.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:55.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:55 smithi045 ceph-mon[36409]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:55.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:55 smithi038 ceph-mon[31995]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:55.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:55 smithi053 ceph-mon[36496]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:56.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:56.274+0000 7f1484ff9700 1 -- 172.21.15.45:0/3307868525 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f148802e080 con 0x7f14700a2ca0 2024-05-21T15:29:56.276 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 7 on host 'smithi045' 2024-05-21T15:29:56.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:56.276+0000 7f148dd88700 1 -- 172.21.15.45:0/3307868525 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f14700a2ca0 msgr2=0x7f14700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:29:56.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:56.276+0000 7f148dd88700 1 --2- 172.21.15.45:0/3307868525 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f14700a2ca0 0x7f14700a51b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1478006fd0 tx=0x7f1478008040 comp rx=0 tx=0).stop 2024-05-21T15:29:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:56.276+0000 7f148dd88700 1 -- 172.21.15.45:0/3307868525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1488104830 msgr2=0x7f1488075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:29:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:56.276+0000 7f148dd88700 1 --2- 172.21.15.45:0/3307868525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1488104830 0x7f1488075fe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1480009cd0 tx=0x7f1480004e70 comp rx=0 tx=0).stop 2024-05-21T15:29:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:56.276+0000 7f148dd88700 1 -- 172.21.15.45:0/3307868525 shutdown_connections 2024-05-21T15:29:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:56.276+0000 7f148dd88700 1 --2- 172.21.15.45:0/3307868525 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f14700a2ca0 0x7f14700a51b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:56.276+0000 7f148dd88700 1 --2- 172.21.15.45:0/3307868525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1488104830 0x7f1488075fe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:56.276+0000 7f148dd88700 1 --2- 172.21.15.45:0/3307868525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1488102de0 0x7f14880773f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:56.276+0000 7f148dd88700 1 --2- 172.21.15.45:0/3307868525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1488103860 0x7f1488075aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:29:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:56.276+0000 7f148dd88700 1 -- 172.21.15.45:0/3307868525 >> 172.21.15.45:0/3307868525 conn(0x7f14880fe850 msgr2=0x7f1488107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:29:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:56.277+0000 7f148dd88700 1 -- 172.21.15.45:0/3307868525 shutdown_connections 2024-05-21T15:29:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:29:56.277+0000 7f148dd88700 1 -- 172.21.15.45:0/3307868525 wait complete. 2024-05-21T15:29:56.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:56.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:56.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:56 smithi045 ceph-mon[36409]: from='osd.7 [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T15:29:56.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:56 smithi045 ceph-mon[36409]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T15:29:56.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:56.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:56.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:56.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:56 smithi053 ceph-mon[36496]: from='osd.7 [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T15:29:56.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:56 smithi053 ceph-mon[36496]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T15:29:56.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:56.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:29:56.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:29:56.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:56 smithi038 ceph-mon[31995]: from='osd.7 [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T15:29:56.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:56 smithi038 ceph-mon[31995]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T15:29:56.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:29:56.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:29:56.969 DEBUG:teuthology.orchestra.run.smithi045:osd.7> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.7.service 2024-05-21T15:29:56.973 INFO:tasks.cephadm:Deploying osd.8 on smithi053 with /dev/vg_nvme/lv_2... 2024-05-21T15:29:56.973 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-21T15:29:57.228 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:29:57.492 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:57 smithi045 ceph-mon[36409]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:57.492 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:57 smithi045 ceph-mon[36409]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T15:29:57.492 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:57 smithi045 ceph-mon[36409]: osdmap e50: 8 total, 7 up, 8 in 2024-05-21T15:29:57.492 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:57 smithi045 ceph-mon[36409]: from='osd.7 [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:29:57.492 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:57.492 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:57 smithi045 ceph-mon[36409]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:29:57.492 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:29:57 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-7[54512]: 2024-05-21T15:29:57.321+0000 7fedb9905700 -1 osd.7 0 waiting for initial osdmap 2024-05-21T15:29:57.493 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:29:57 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-7[54512]: 2024-05-21T15:29:57.333+0000 7fedb56ba700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:29:57.610 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:57 smithi053 ceph-mon[36496]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:57.611 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:57 smithi053 ceph-mon[36496]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T15:29:57.611 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:57 smithi053 ceph-mon[36496]: osdmap e50: 8 total, 7 up, 8 in 2024-05-21T15:29:57.611 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:57 smithi053 ceph-mon[36496]: from='osd.7 [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:29:57.611 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:57.611 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:57 smithi053 ceph-mon[36496]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:29:57.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:57 smithi038 ceph-mon[31995]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:57.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:57 smithi038 ceph-mon[31995]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T15:29:57.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:57 smithi038 ceph-mon[31995]: osdmap e50: 8 total, 7 up, 8 in 2024-05-21T15:29:57.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:57 smithi038 ceph-mon[31995]: from='osd.7 [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:29:57.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:57.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:57 smithi038 ceph-mon[31995]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:29:58.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:58 smithi045 ceph-mon[36409]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T15:29:58.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:58 smithi045 ceph-mon[36409]: osdmap e51: 8 total, 7 up, 8 in 2024-05-21T15:29:58.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:58 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:58.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:58 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:58.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:58 smithi045 ceph-mon[36409]: osd.7 [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746] boot 2024-05-21T15:29:58.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:58 smithi053 ceph-mon[36496]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T15:29:58.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:58 smithi053 ceph-mon[36496]: osdmap e51: 8 total, 7 up, 8 in 2024-05-21T15:29:58.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:58 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:58.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:58 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:58.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:58 smithi053 ceph-mon[36496]: osd.7 [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746] boot 2024-05-21T15:29:58.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:58 smithi038 ceph-mon[31995]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T15:29:58.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:58 smithi038 ceph-mon[31995]: osdmap e51: 8 total, 7 up, 8 in 2024-05-21T15:29:58.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:58 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:58.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:58 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:58.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:58 smithi038 ceph-mon[31995]: osd.7 [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746] boot 2024-05-21T15:29:59.307 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-21T15:29:59.307 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-21T15:29:59.307 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-05-21T15:29:59.307 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-05-21T15:29:59.307 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00855369 s, 1.2 GB/s 2024-05-21T15:29:59.307 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-05-21T15:29:59.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:59 smithi053 ceph-mon[36496]: purged_snaps scrub starts 2024-05-21T15:29:59.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:59 smithi053 ceph-mon[36496]: purged_snaps scrub ok 2024-05-21T15:29:59.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:59 smithi053 ceph-mon[36496]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:59.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:59 smithi053 ceph-mon[36496]: osdmap e52: 8 total, 8 up, 8 in 2024-05-21T15:29:59.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:59 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:59.616 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:29:59 smithi053 ceph-mon[36496]: osdmap e53: 8 total, 8 up, 8 in 2024-05-21T15:29:59.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:59 smithi045 ceph-mon[36409]: purged_snaps scrub starts 2024-05-21T15:29:59.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:59 smithi045 ceph-mon[36409]: purged_snaps scrub ok 2024-05-21T15:29:59.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:59 smithi045 ceph-mon[36409]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:59.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:59 smithi045 ceph-mon[36409]: osdmap e52: 8 total, 8 up, 8 in 2024-05-21T15:29:59.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:59 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:59.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:29:59 smithi045 ceph-mon[36409]: osdmap e53: 8 total, 8 up, 8 in 2024-05-21T15:29:59.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:59 smithi038 ceph-mon[31995]: purged_snaps scrub starts 2024-05-21T15:29:59.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:59 smithi038 ceph-mon[31995]: purged_snaps scrub ok 2024-05-21T15:29:59.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:59 smithi038 ceph-mon[31995]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:29:59.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:59 smithi038 ceph-mon[31995]: osdmap e52: 8 total, 8 up, 8 in 2024-05-21T15:29:59.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:59 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:29:59.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:29:59 smithi038 ceph-mon[31995]: osdmap e53: 8 total, 8 up, 8 in 2024-05-21T15:30:00.118 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_2 2024-05-21T15:30:00.364 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:30:00.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:00 smithi053 ceph-mon[36496]: overall HEALTH_OK 2024-05-21T15:30:00.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:00.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:00.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:00.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:00.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:00.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:00.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:00.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:00.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:00.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:00 smithi053 ceph-mon[36496]: osdmap e54: 8 total, 8 up, 8 in 2024-05-21T15:30:00.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:00 smithi045 ceph-mon[36409]: overall HEALTH_OK 2024-05-21T15:30:00.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:00.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:00.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:00.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:00.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:00.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:00.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:00.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:00.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:00.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:00 smithi045 ceph-mon[36409]: osdmap e54: 8 total, 8 up, 8 in 2024-05-21T15:30:00.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:00 smithi038 ceph-mon[31995]: overall HEALTH_OK 2024-05-21T15:30:00.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:00.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:00.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:00.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:00.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:00.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:00.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:00.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:00.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:00.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:00 smithi038 ceph-mon[31995]: osdmap e54: 8 total, 8 up, 8 in 2024-05-21T15:30:01.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:01 smithi053 ceph-mon[36496]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:01.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:01 smithi053 ceph-mon[36496]: Detected new or changed devices on smithi045 2024-05-21T15:30:01.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:01 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi045 to 5697M 2024-05-21T15:30:01.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:01 smithi045 ceph-mon[36409]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:01.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:01 smithi045 ceph-mon[36409]: Detected new or changed devices on smithi045 2024-05-21T15:30:01.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:01 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi045 to 5697M 2024-05-21T15:30:01.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:01 smithi038 ceph-mon[31995]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:01.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:01 smithi038 ceph-mon[31995]: Detected new or changed devices on smithi045 2024-05-21T15:30:01.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:01 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi045 to 5697M 2024-05-21T15:30:02.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.089+0000 7f49c5fe8700 1 -- 172.21.15.53:0/3133817573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0103200 msgr2=0x7f49c0103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:30:02.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.089+0000 7f49c5fe8700 1 --2- 172.21.15.53:0/3133817573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0103200 0x7f49c0103610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009990 tx=0x7f49b4009ca0 comp rx=0 tx=0).stop 2024-05-21T15:30:02.091 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.089+0000 7f49c5fe8700 1 -- 172.21.15.53:0/3133817573 shutdown_connections 2024-05-21T15:30:02.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.090+0000 7f49c5fe8700 1 --2- 172.21.15.53:0/3133817573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49c0103ba0 0x7f49c010a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:02.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.090+0000 7f49c5fe8700 1 --2- 172.21.15.53:0/3133817573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49c01020d0 0x7f49c01024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:02.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.090+0000 7f49c5fe8700 1 --2- 172.21.15.53:0/3133817573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0103200 0x7f49c0103610 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:02.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.090+0000 7f49c5fe8700 1 -- 172.21.15.53:0/3133817573 >> 172.21.15.53:0/3133817573 conn(0x7f49c00fd710 msgr2=0x7f49c00ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:30:02.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.090+0000 7f49c5fe8700 1 -- 172.21.15.53:0/3133817573 shutdown_connections 2024-05-21T15:30:02.092 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.090+0000 7f49c5fe8700 1 -- 172.21.15.53:0/3133817573 wait complete. 2024-05-21T15:30:02.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.090+0000 7f49c5fe8700 1 Processor -- start 2024-05-21T15:30:02.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.090+0000 7f49c5fe8700 1 -- start start 2024-05-21T15:30:02.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49c5fe8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49c01020d0 0x7f49c00752d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:30:02.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49c5fe8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0103200 0x7f49c019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:30:02.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49c5fe8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49c0103ba0 0x7f49c01a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:30:02.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49c5fe8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f49c010dca0 con 0x7f49c01020d0 2024-05-21T15:30:02.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49c5fe8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f49c010db20 con 0x7f49c0103ba0 2024-05-21T15:30:02.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49c5fe8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f49c010de20 con 0x7f49c0103200 2024-05-21T15:30:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49beffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0103200 0x7f49c019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:30:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49beffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0103200 0x7f49c019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60076/0 (socket says 172.21.15.53:60076) 2024-05-21T15:30:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49beffd700 1 -- 172.21.15.53:0/936112640 learned_addr learned my addr 172.21.15.53:0/936112640 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-21T15:30:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49bf7fe700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49c01020d0 0x7f49c00752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:30:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49bffff700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49c0103ba0 0x7f49c01a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:30:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49beffd700 1 -- 172.21.15.53:0/936112640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49c0103ba0 msgr2=0x7f49c01a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:30:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49beffd700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49c0103ba0 0x7f49c01a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49beffd700 1 -- 172.21.15.53:0/936112640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49c01020d0 msgr2=0x7f49c00752d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:30:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49beffd700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49c01020d0 0x7f49c00752d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49beffd700 1 -- 172.21.15.53:0/936112640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c01a1330 con 0x7f49c0103200 2024-05-21T15:30:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49bf7fe700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49c01020d0 0x7f49c00752d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:30:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49bffff700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49c0103ba0 0x7f49c01a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:30:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49beffd700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0103200 0x7f49c019e620 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f49b40098e0 tx=0x7f49b40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:30:02.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.091+0000 7f49bcff9700 1 -- 172.21.15.53:0/936112640 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b4003c70 con 0x7f49c0103200 2024-05-21T15:30:02.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.092+0000 7f49c5fe8700 1 -- 172.21.15.53:0/936112640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c01a15c0 con 0x7f49c0103200 2024-05-21T15:30:02.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.092+0000 7f49bcff9700 1 -- 172.21.15.53:0/936112640 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b4004830 con 0x7f49c0103200 2024-05-21T15:30:02.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.092+0000 7f49bcff9700 1 -- 172.21.15.53:0/936112640 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b40054e0 con 0x7f49c0103200 2024-05-21T15:30:02.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.092+0000 7f49c5fe8700 1 -- 172.21.15.53:0/936112640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c01a19b0 con 0x7f49c0103200 2024-05-21T15:30:02.096 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.093+0000 7f49c5fe8700 1 -- 172.21.15.53:0/936112640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c0050170 con 0x7f49c0103200 2024-05-21T15:30:02.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.094+0000 7f49bcff9700 1 -- 172.21.15.53:0/936112640 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f49b4003e10 con 0x7f49c0103200 2024-05-21T15:30:02.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.094+0000 7f49bcff9700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f49a80a2c20 0x7f49a80a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:30:02.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.095+0000 7f49bcff9700 1 -- 172.21.15.53:0/936112640 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7f49b40cfc00 con 0x7f49c0103200 2024-05-21T15:30:02.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.095+0000 7f49bf7fe700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f49a80a2c20 0x7f49a80a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:30:02.097 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.095+0000 7f49bf7fe700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f49a80a2c20 0x7f49a80a5130 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49c0103100 tx=0x7f49b000a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:30:02.099 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.098+0000 7f49bcff9700 1 -- 172.21.15.53:0/936112640 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f49b4030020 con 0x7f49c0103200 2024-05-21T15:30:02.317 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:02.316+0000 7f49c5fe8700 1 -- 172.21.15.53:0/936112640 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f49c0062a80 con 0x7f49a80a2c20 2024-05-21T15:30:02.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:02 smithi053 ceph-mon[36496]: osdmap e55: 8 total, 8 up, 8 in 2024-05-21T15:30:02.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:02 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:30:02.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:02 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:30:02.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:02 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:02.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:02 smithi045 ceph-mon[36409]: osdmap e55: 8 total, 8 up, 8 in 2024-05-21T15:30:02.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:02 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:30:02.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:02 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:30:02.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:02 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:02.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:02 smithi038 ceph-mon[31995]: osdmap e55: 8 total, 8 up, 8 in 2024-05-21T15:30:02.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:02 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:30:02.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:02 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:30:02.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:02 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:03.664 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:03 smithi045 ceph-mon[36409]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:03.664 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:03 smithi045 ceph-mon[36409]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:30:03.664 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:03 smithi045 ceph-mon[36409]: osdmap e56: 8 total, 8 up, 8 in 2024-05-21T15:30:03.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:03 smithi053 ceph-mon[36496]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:03.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:03 smithi053 ceph-mon[36496]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:30:03.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:03 smithi053 ceph-mon[36496]: osdmap e56: 8 total, 8 up, 8 in 2024-05-21T15:30:03.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:03 smithi038 ceph-mon[31995]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:03.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:03 smithi038 ceph-mon[31995]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:30:03.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:03 smithi038 ceph-mon[31995]: osdmap e56: 8 total, 8 up, 8 in 2024-05-21T15:30:05.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:05 smithi053 ceph-mon[36496]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:05.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:05 smithi053 ceph-mon[36496]: from='client.? 172.21.15.53:0/1190675319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc9538ac-44ea-4518-86b7-1a4dc9f10d57"}]: dispatch 2024-05-21T15:30:05.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:05 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc9538ac-44ea-4518-86b7-1a4dc9f10d57"}]: dispatch 2024-05-21T15:30:05.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:05 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc9538ac-44ea-4518-86b7-1a4dc9f10d57"}]': finished 2024-05-21T15:30:05.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:05 smithi053 ceph-mon[36496]: osdmap e57: 9 total, 8 up, 9 in 2024-05-21T15:30:05.708 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:05 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:05.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:05 smithi045 ceph-mon[36409]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:05.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:05 smithi045 ceph-mon[36409]: from='client.? 172.21.15.53:0/1190675319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc9538ac-44ea-4518-86b7-1a4dc9f10d57"}]: dispatch 2024-05-21T15:30:05.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:05 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc9538ac-44ea-4518-86b7-1a4dc9f10d57"}]: dispatch 2024-05-21T15:30:05.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:05 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc9538ac-44ea-4518-86b7-1a4dc9f10d57"}]': finished 2024-05-21T15:30:05.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:05 smithi045 ceph-mon[36409]: osdmap e57: 9 total, 8 up, 9 in 2024-05-21T15:30:05.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:05 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:05.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:05 smithi038 ceph-mon[31995]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:05.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:05 smithi038 ceph-mon[31995]: from='client.? 172.21.15.53:0/1190675319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc9538ac-44ea-4518-86b7-1a4dc9f10d57"}]: dispatch 2024-05-21T15:30:05.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:05 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc9538ac-44ea-4518-86b7-1a4dc9f10d57"}]: dispatch 2024-05-21T15:30:05.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:05 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc9538ac-44ea-4518-86b7-1a4dc9f10d57"}]': finished 2024-05-21T15:30:05.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:05 smithi038 ceph-mon[31995]: osdmap e57: 9 total, 8 up, 9 in 2024-05-21T15:30:05.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:05 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:06.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:06 smithi053 ceph-mon[36496]: from='client.? 172.21.15.53:0/1265453867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:30:06.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:06 smithi045 ceph-mon[36409]: from='client.? 172.21.15.53:0/1265453867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:30:06.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:06 smithi038 ceph-mon[31995]: from='client.? 172.21.15.53:0/1265453867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:30:07.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:07 smithi053 ceph-mon[36496]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:07.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:07 smithi045 ceph-mon[36409]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:07.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:07 smithi038 ceph-mon[31995]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:09.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:09 smithi053 ceph-mon[36496]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:09.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:09 smithi045 ceph-mon[36409]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:09.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:09 smithi038 ceph-mon[31995]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:11.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:11 smithi053 ceph-mon[36496]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:11.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:11 smithi045 ceph-mon[36409]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:11.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:11 smithi038 ceph-mon[31995]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:13.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:13 smithi053 ceph-mon[36496]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:13.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:13 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-21T15:30:13.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:13 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:13.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:13 smithi053 ceph-mon[36496]: Deploying daemon osd.8 on smithi053 2024-05-21T15:30:13.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:13 smithi045 ceph-mon[36409]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:13.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:13 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-21T15:30:13.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:13 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:13.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:13 smithi045 ceph-mon[36409]: Deploying daemon osd.8 on smithi053 2024-05-21T15:30:13.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:13 smithi038 ceph-mon[31995]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:13.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:13 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-21T15:30:13.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:13 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:13.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:13 smithi038 ceph-mon[31995]: Deploying daemon osd.8 on smithi053 2024-05-21T15:30:15.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:15 smithi053 ceph-mon[36496]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:15.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:15 smithi045 ceph-mon[36409]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:15.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:15 smithi038 ceph-mon[31995]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:17.578 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:17 smithi053 ceph-mon[36496]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:17.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:17 smithi045 ceph-mon[36409]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:17.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:17 smithi038 ceph-mon[31995]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:19.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:19.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:30:19.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:19 smithi053 ceph-mon[36496]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:19.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:19.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:19.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:30:19.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:19 smithi045 ceph-mon[36409]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:19.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:19.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:19.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:30:19.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:19 smithi038 ceph-mon[31995]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:19.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:21.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:21 smithi053 ceph-mon[36496]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:21.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:21 smithi045 ceph-mon[36409]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:21.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:21 smithi038 ceph-mon[31995]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:21.788 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:21.786+0000 7f49bcff9700 1 -- 172.21.15.53:0/936112640 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f49c0062a80 con 0x7f49a80a2c20 2024-05-21T15:30:21.788 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 8 on host 'smithi053' 2024-05-21T15:30:21.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:21.788+0000 7f49c5fe8700 1 -- 172.21.15.53:0/936112640 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f49a80a2c20 msgr2=0x7f49a80a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:30:21.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:21.788+0000 7f49c5fe8700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f49a80a2c20 0x7f49a80a5130 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49c0103100 tx=0x7f49b000a680 comp rx=0 tx=0).stop 2024-05-21T15:30:21.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:21.788+0000 7f49c5fe8700 1 -- 172.21.15.53:0/936112640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0103200 msgr2=0x7f49c019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:30:21.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:21.788+0000 7f49c5fe8700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0103200 0x7f49c019e620 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f49b40098e0 tx=0x7f49b40047b0 comp rx=0 tx=0).stop 2024-05-21T15:30:21.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:21.788+0000 7f49c5fe8700 1 -- 172.21.15.53:0/936112640 shutdown_connections 2024-05-21T15:30:21.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:21.788+0000 7f49c5fe8700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f49a80a2c20 0x7f49a80a5130 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:21.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:21.788+0000 7f49c5fe8700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49c0103ba0 0x7f49c01a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:21.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:21.788+0000 7f49c5fe8700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49c01020d0 0x7f49c00752d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:21.790 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:21.788+0000 7f49c5fe8700 1 --2- 172.21.15.53:0/936112640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0103200 0x7f49c019e620 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:21.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:21.788+0000 7f49c5fe8700 1 -- 172.21.15.53:0/936112640 >> 172.21.15.53:0/936112640 conn(0x7f49c00fd710 msgr2=0x7f49c0106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:30:21.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:21.790+0000 7f49c5fe8700 1 -- 172.21.15.53:0/936112640 shutdown_connections 2024-05-21T15:30:21.791 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:30:21.790+0000 7f49c5fe8700 1 -- 172.21.15.53:0/936112640 wait complete. 2024-05-21T15:30:22.506 DEBUG:teuthology.orchestra.run.smithi053:osd.8> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.8.service 2024-05-21T15:30:22.508 INFO:tasks.cephadm:Deploying osd.9 on smithi038 with /dev/vg_nvme/lv_1... 2024-05-21T15:30:22.509 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-21T15:30:22.771 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:30:22.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:22.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:22.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:22.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:22.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:22 smithi038 ceph-mon[31995]: from='osd.8 [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T15:30:22.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:22 smithi038 ceph-mon[31995]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T15:30:23.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:23.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:23.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:23.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:23.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:22 smithi053 ceph-mon[36496]: from='osd.8 [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T15:30:23.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:22 smithi053 ceph-mon[36496]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T15:30:23.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:23.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:23.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:23.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:23.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:22 smithi045 ceph-mon[36409]: from='osd.8 [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T15:30:23.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:22 smithi045 ceph-mon[36409]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T15:30:23.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:23 smithi053 ceph-mon[36496]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:23.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:23 smithi053 ceph-mon[36496]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-21T15:30:23.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:23 smithi053 ceph-mon[36496]: osdmap e58: 9 total, 8 up, 9 in 2024-05-21T15:30:23.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:23.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:23 smithi053 ceph-mon[36496]: from='osd.8 [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:30:23.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:23 smithi053 ceph-mon[36496]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:30:24.129 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:23 smithi038 ceph-mon[31995]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:24.129 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:23 smithi038 ceph-mon[31995]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-21T15:30:24.129 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:23 smithi038 ceph-mon[31995]: osdmap e58: 9 total, 8 up, 9 in 2024-05-21T15:30:24.129 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:24.130 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:23 smithi038 ceph-mon[31995]: from='osd.8 [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:30:24.130 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:23 smithi038 ceph-mon[31995]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:30:24.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:23 smithi045 ceph-mon[36409]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:24.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:23 smithi045 ceph-mon[36409]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-21T15:30:24.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:23 smithi045 ceph-mon[36409]: osdmap e58: 9 total, 8 up, 9 in 2024-05-21T15:30:24.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:24.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:23 smithi045 ceph-mon[36409]: from='osd.8 [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:30:24.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:23 smithi045 ceph-mon[36409]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:30:24.865 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:30:24 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-8[54931]: 2024-05-21T15:30:24.569+0000 7fb003acf700 -1 osd.8 0 waiting for initial osdmap 2024-05-21T15:30:24.865 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:30:24 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-8[54931]: 2024-05-21T15:30:24.585+0000 7faffb07b700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:30:24.891 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-21T15:30:24.891 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-21T15:30:24.892 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-05-21T15:30:24.892 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-05-21T15:30:24.892 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0166618 s, 629 MB/s 2024-05-21T15:30:24.892 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-05-21T15:30:25.195 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:25 smithi053 ceph-mon[36496]: purged_snaps scrub starts 2024-05-21T15:30:25.195 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:25 smithi053 ceph-mon[36496]: purged_snaps scrub ok 2024-05-21T15:30:25.195 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:25 smithi053 ceph-mon[36496]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-21T15:30:25.195 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:25 smithi053 ceph-mon[36496]: osdmap e59: 9 total, 8 up, 9 in 2024-05-21T15:30:25.195 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:25.195 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:25 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:25.195 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:25 smithi053 ceph-mon[36496]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:25.195 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:25 smithi053 ceph-mon[36496]: from='osd.8 ' entity='osd.8' 2024-05-21T15:30:25.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:25 smithi038 ceph-mon[31995]: purged_snaps scrub starts 2024-05-21T15:30:25.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:25 smithi038 ceph-mon[31995]: purged_snaps scrub ok 2024-05-21T15:30:25.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:25 smithi038 ceph-mon[31995]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-21T15:30:25.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:25 smithi038 ceph-mon[31995]: osdmap e59: 9 total, 8 up, 9 in 2024-05-21T15:30:25.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:25.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:25 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:25.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:25 smithi038 ceph-mon[31995]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:25.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:25 smithi038 ceph-mon[31995]: from='osd.8 ' entity='osd.8' 2024-05-21T15:30:25.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:25 smithi045 ceph-mon[36409]: purged_snaps scrub starts 2024-05-21T15:30:25.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:25 smithi045 ceph-mon[36409]: purged_snaps scrub ok 2024-05-21T15:30:25.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:25 smithi045 ceph-mon[36409]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-21T15:30:25.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:25 smithi045 ceph-mon[36409]: osdmap e59: 9 total, 8 up, 9 in 2024-05-21T15:30:25.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:25.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:25 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:25.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:25 smithi045 ceph-mon[36409]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:30:25.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:25 smithi045 ceph-mon[36409]: from='osd.8 ' entity='osd.8' 2024-05-21T15:30:25.643 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_1 2024-05-21T15:30:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:30:26.195 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: osd.8 [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173] boot 2024-05-21T15:30:26.195 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: osdmap e60: 9 total, 9 up, 9 in 2024-05-21T15:30:26.195 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:26.195 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: Detected new or changed devices on smithi053 2024-05-21T15:30:26.195 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:26.195 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:26.195 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:26.196 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:26.196 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:26.196 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi053 to 5697M 2024-05-21T15:30:26.196 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:26.196 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:26.196 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:26.196 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:26.196 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:30:26.196 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:26.196 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:26.197 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:26 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:26.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: osd.8 [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173] boot 2024-05-21T15:30:26.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: osdmap e60: 9 total, 9 up, 9 in 2024-05-21T15:30:26.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:26.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: Detected new or changed devices on smithi053 2024-05-21T15:30:26.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:26.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:26.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:26.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:26.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:26.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi053 to 5697M 2024-05-21T15:30:26.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:26.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:26.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:26.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:26.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:30:26.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:26.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:26.459 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:26 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:26.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: osd.8 [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173] boot 2024-05-21T15:30:26.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: osdmap e60: 9 total, 9 up, 9 in 2024-05-21T15:30:26.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:30:26.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: Detected new or changed devices on smithi053 2024-05-21T15:30:26.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:26.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:26.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:26.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:26.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:26.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi053 to 5697M 2024-05-21T15:30:26.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:26.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:26.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:26.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:26.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:30:26.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:26.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:26.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:26 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:27.198 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:27 smithi038 ceph-mon[31995]: osdmap e61: 9 total, 9 up, 9 in 2024-05-21T15:30:27.198 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:27 smithi038 ceph-mon[31995]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:27.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:27 smithi053 ceph-mon[36496]: osdmap e61: 9 total, 9 up, 9 in 2024-05-21T15:30:27.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:27 smithi053 ceph-mon[36496]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:27.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:27 smithi045 ceph-mon[36409]: osdmap e61: 9 total, 9 up, 9 in 2024-05-21T15:30:27.553 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:27 smithi045 ceph-mon[36409]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:27.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.570+0000 7fa9885b7700 1 -- 172.21.15.38:0/3782699641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9801020c0 msgr2=0x7fa9801024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:30:27.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.570+0000 7fa9885b7700 1 --2- 172.21.15.38:0/3782699641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9801020c0 0x7fa9801024d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009990 tx=0x7fa97c009ca0 comp rx=0 tx=0).stop 2024-05-21T15:30:27.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.570+0000 7fa9885b7700 1 -- 172.21.15.38:0/3782699641 shutdown_connections 2024-05-21T15:30:27.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.570+0000 7fa9885b7700 1 --2- 172.21.15.38:0/3782699641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa980103b90 0x7fa98010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:27.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.570+0000 7fa9885b7700 1 --2- 172.21.15.38:0/3782699641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9801020c0 0x7fa9801024d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:27.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.570+0000 7fa9885b7700 1 --2- 172.21.15.38:0/3782699641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9801031f0 0x7fa980103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:27.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.571+0000 7fa9885b7700 1 -- 172.21.15.38:0/3782699641 >> 172.21.15.38:0/3782699641 conn(0x7fa9800fd700 msgr2=0x7fa9800ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:30:27.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.571+0000 7fa9885b7700 1 -- 172.21.15.38:0/3782699641 shutdown_connections 2024-05-21T15:30:27.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.571+0000 7fa9885b7700 1 -- 172.21.15.38:0/3782699641 wait complete. 2024-05-21T15:30:27.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.571+0000 7fa9885b7700 1 Processor -- start 2024-05-21T15:30:27.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.571+0000 7fa9885b7700 1 -- start start 2024-05-21T15:30:27.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.571+0000 7fa9885b7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9801020c0 0x7fa98006aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:30:27.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.571+0000 7fa9885b7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9801031f0 0x7fa9801a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:30:27.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.571+0000 7fa9885b7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa980103b90 0x7fa9801a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:30:27.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa9885b7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa98010dc60 con 0x7fa9801020c0 2024-05-21T15:30:27.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa9885b7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa98010dae0 con 0x7fa980103b90 2024-05-21T15:30:27.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa9885b7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa98010dde0 con 0x7fa9801031f0 2024-05-21T15:30:27.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa986353700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9801020c0 0x7fa98006aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:30:27.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa986353700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9801020c0 0x7fa98006aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54732/0 (socket says 172.21.15.38:54732) 2024-05-21T15:30:27.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa986353700 1 -- 172.21.15.38:0/2109731384 learned_addr learned my addr 172.21.15.38:0/2109731384 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:30:27.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa985b52700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9801031f0 0x7fa9801a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:30:27.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa986b54700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa980103b90 0x7fa9801a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:30:27.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa986353700 1 -- 172.21.15.38:0/2109731384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9801031f0 msgr2=0x7fa9801a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:30:27.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa986353700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9801031f0 0x7fa9801a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:27.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa986353700 1 -- 172.21.15.38:0/2109731384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa980103b90 msgr2=0x7fa9801a2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:30:27.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa986353700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa980103b90 0x7fa9801a2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:27.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa986353700 1 -- 172.21.15.38:0/2109731384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9801a3630 con 0x7fa9801020c0 2024-05-21T15:30:27.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa985b52700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9801031f0 0x7fa9801a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:30:27.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa986353700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9801020c0 0x7fa98006aef0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa97c000c00 tx=0x7fa97c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:30:27.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa9777fe700 1 -- 172.21.15.38:0/2109731384 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c027070 con 0x7fa9801020c0 2024-05-21T15:30:27.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa9885b7700 1 -- 172.21.15.38:0/2109731384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9801a38c0 con 0x7fa9801020c0 2024-05-21T15:30:27.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa9777fe700 1 -- 172.21.15.38:0/2109731384 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa97c004150 con 0x7fa9801020c0 2024-05-21T15:30:27.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.572+0000 7fa9777fe700 1 -- 172.21.15.38:0/2109731384 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c00f460 con 0x7fa9801020c0 2024-05-21T15:30:27.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.573+0000 7fa9885b7700 1 -- 172.21.15.38:0/2109731384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9801a3d10 con 0x7fa9801020c0 2024-05-21T15:30:27.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.573+0000 7fa9885b7700 1 -- 172.21.15.38:0/2109731384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa980050170 con 0x7fa9801020c0 2024-05-21T15:30:27.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.575+0000 7fa9777fe700 1 -- 172.21.15.38:0/2109731384 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa97c02a070 con 0x7fa9801020c0 2024-05-21T15:30:27.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.576+0000 7fa9777fe700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa96c0a2c20 0x7fa96c0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:30:27.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.576+0000 7fa985b52700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa96c0a2c20 0x7fa96c0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:30:27.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.576+0000 7fa9777fe700 1 -- 172.21.15.38:0/2109731384 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7fa97c0cfe00 con 0x7fa9801020c0 2024-05-21T15:30:27.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.576+0000 7fa985b52700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa96c0a2c20 0x7fa96c0a5130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa970005950 tx=0x7fa9700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:30:27.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.579+0000 7fa9777fe700 1 -- 172.21.15.38:0/2109731384 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa97c031080 con 0x7fa9801020c0 2024-05-21T15:30:27.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:27.799+0000 7fa9885b7700 1 -- 172.21.15.38:0/2109731384 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa980062a80 con 0x7fa96c0a2c20 2024-05-21T15:30:28.443 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:28 smithi038 ceph-mon[31995]: osdmap e62: 9 total, 9 up, 9 in 2024-05-21T15:30:28.443 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:28 smithi038 ceph-mon[31995]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:30:28.443 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:28 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:30:28.443 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:28 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:30:28.443 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:28 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:28.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:28 smithi053 ceph-mon[36496]: osdmap e62: 9 total, 9 up, 9 in 2024-05-21T15:30:28.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:28 smithi053 ceph-mon[36496]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:30:28.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:28 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:30:28.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:28 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:30:28.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:28 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:28.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:28 smithi045 ceph-mon[36409]: osdmap e62: 9 total, 9 up, 9 in 2024-05-21T15:30:28.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:28 smithi045 ceph-mon[36409]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:30:28.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:28 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:30:28.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:28 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:30:28.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:28 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:29.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:29 smithi053 ceph-mon[36496]: osdmap e63: 9 total, 9 up, 9 in 2024-05-21T15:30:29.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:29 smithi053 ceph-mon[36496]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:29.503 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:29 smithi038 ceph-mon[31995]: osdmap e63: 9 total, 9 up, 9 in 2024-05-21T15:30:29.503 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:29 smithi038 ceph-mon[31995]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:29.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:29 smithi045 ceph-mon[36409]: osdmap e63: 9 total, 9 up, 9 in 2024-05-21T15:30:29.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:29 smithi045 ceph-mon[36409]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:30.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:30 smithi053 ceph-mon[36496]: osdmap e64: 9 total, 9 up, 9 in 2024-05-21T15:30:30.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:30 smithi045 ceph-mon[36409]: osdmap e64: 9 total, 9 up, 9 in 2024-05-21T15:30:30.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:30 smithi038 ceph-mon[31995]: osdmap e64: 9 total, 9 up, 9 in 2024-05-21T15:30:31.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:31 smithi053 ceph-mon[36496]: pgmap v208: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-05-21T15:30:31.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:31 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3020781664' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8885020-dc81-4f22-8592-45c5179c24ad"}]: dispatch 2024-05-21T15:30:31.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:31 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3020781664' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8885020-dc81-4f22-8592-45c5179c24ad"}]': finished 2024-05-21T15:30:31.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:31 smithi053 ceph-mon[36496]: osdmap e65: 10 total, 9 up, 10 in 2024-05-21T15:30:31.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:31.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:31 smithi045 ceph-mon[36409]: pgmap v208: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-05-21T15:30:31.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:31 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3020781664' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8885020-dc81-4f22-8592-45c5179c24ad"}]: dispatch 2024-05-21T15:30:31.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:31 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3020781664' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8885020-dc81-4f22-8592-45c5179c24ad"}]': finished 2024-05-21T15:30:31.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:31 smithi045 ceph-mon[36409]: osdmap e65: 10 total, 9 up, 10 in 2024-05-21T15:30:31.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:31.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:31 smithi038 ceph-mon[31995]: pgmap v208: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-05-21T15:30:31.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:31 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3020781664' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8885020-dc81-4f22-8592-45c5179c24ad"}]: dispatch 2024-05-21T15:30:31.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:31 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3020781664' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8885020-dc81-4f22-8592-45c5179c24ad"}]': finished 2024-05-21T15:30:31.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:31 smithi038 ceph-mon[31995]: osdmap e65: 10 total, 9 up, 10 in 2024-05-21T15:30:31.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:32.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:32 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2818559852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:30:32.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:32 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2818559852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:30:32.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:32 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2818559852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:30:33.413 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:33 smithi045 ceph-mon[36409]: pgmap v210: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-05-21T15:30:33.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:33 smithi053 ceph-mon[36496]: pgmap v210: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-05-21T15:30:33.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:33 smithi038 ceph-mon[31995]: pgmap v210: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-05-21T15:30:35.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:35 smithi045 ceph-mon[36409]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-05-21T15:30:35.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:35 smithi038 ceph-mon[31995]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-05-21T15:30:35.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:35 smithi053 ceph-mon[36496]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-05-21T15:30:37.476 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:37 smithi038 ceph-mon[31995]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:30:37.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:37 smithi045 ceph-mon[36409]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:30:37.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:37 smithi053 ceph-mon[36496]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:30:39.354 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:39 smithi038 ceph-mon[31995]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:39.354 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:39 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-21T15:30:39.354 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:39 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:39.354 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:39 smithi038 ceph-mon[31995]: Deploying daemon osd.9 on smithi038 2024-05-21T15:30:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:39 smithi045 ceph-mon[36409]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:39 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-21T15:30:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:39 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:39 smithi045 ceph-mon[36409]: Deploying daemon osd.9 on smithi038 2024-05-21T15:30:39.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:39 smithi053 ceph-mon[36496]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:39.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:39 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-21T15:30:39.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:39 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:39.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:39 smithi053 ceph-mon[36496]: Deploying daemon osd.9 on smithi038 2024-05-21T15:30:41.374 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:41 smithi038 ceph-mon[31995]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:41.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:41 smithi045 ceph-mon[36409]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:41.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:41 smithi053 ceph-mon[36496]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:43.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:43 smithi038 ceph-mon[31995]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:43.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:43 smithi053 ceph-mon[36496]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:43.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:43 smithi045 ceph-mon[36409]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:45.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:45 smithi053 ceph-mon[36496]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:45.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:45 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:45.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:45 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:45.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:45 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:30:45.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:45 smithi045 ceph-mon[36409]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:45.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:45 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:45.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:45 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:45.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:45 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:30:45.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:45 smithi038 ceph-mon[31995]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:45.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:45.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:45.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:30:47.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:47 smithi053 ceph-mon[36496]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:47.751 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:47 smithi038 ceph-mon[31995]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:47.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:47 smithi045 ceph-mon[36409]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:48.211 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 9 on host 'smithi038' 2024-05-21T15:30:48.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:48.209+0000 7fa9777fe700 1 -- 172.21.15.38:0/2109731384 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa980062a80 con 0x7fa96c0a2c20 2024-05-21T15:30:48.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:48.212+0000 7fa9885b7700 1 -- 172.21.15.38:0/2109731384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa96c0a2c20 msgr2=0x7fa96c0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:30:48.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:48.212+0000 7fa9885b7700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa96c0a2c20 0x7fa96c0a5130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa970005950 tx=0x7fa9700058e0 comp rx=0 tx=0).stop 2024-05-21T15:30:48.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:48.212+0000 7fa9885b7700 1 -- 172.21.15.38:0/2109731384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9801020c0 msgr2=0x7fa98006aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:30:48.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:48.212+0000 7fa9885b7700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9801020c0 0x7fa98006aef0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa97c000c00 tx=0x7fa97c009510 comp rx=0 tx=0).stop 2024-05-21T15:30:48.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:48.212+0000 7fa9885b7700 1 -- 172.21.15.38:0/2109731384 shutdown_connections 2024-05-21T15:30:48.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:48.212+0000 7fa9885b7700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa96c0a2c20 0x7fa96c0a5130 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:48.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:48.212+0000 7fa9885b7700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa980103b90 0x7fa9801a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:48.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:48.212+0000 7fa9885b7700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9801020c0 0x7fa98006aef0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:48.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:48.212+0000 7fa9885b7700 1 --2- 172.21.15.38:0/2109731384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9801031f0 0x7fa9801a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:48.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:48.212+0000 7fa9885b7700 1 -- 172.21.15.38:0/2109731384 >> 172.21.15.38:0/2109731384 conn(0x7fa9800fd700 msgr2=0x7fa980106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:30:48.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:48.212+0000 7fa9885b7700 1 -- 172.21.15.38:0/2109731384 shutdown_connections 2024-05-21T15:30:48.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:30:48.212+0000 7fa9885b7700 1 -- 172.21.15.38:0/2109731384 wait complete. 2024-05-21T15:30:48.981 DEBUG:teuthology.orchestra.run.smithi038:osd.9> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.9.service 2024-05-21T15:30:48.984 INFO:tasks.cephadm:Deploying osd.10 on smithi045 with /dev/vg_nvme/lv_1... 2024-05-21T15:30:48.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-21T15:30:49.245 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:30:49.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:49 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:49.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:49 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:49.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:49 smithi038 ceph-mon[31995]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:49.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:49 smithi038 ceph-mon[31995]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-21T15:30:49.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:49 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:49.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:49 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:49.367 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:49 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:49.367 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:49 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:49.367 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:49 smithi045 ceph-mon[36409]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:49.367 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:49 smithi045 ceph-mon[36409]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-21T15:30:49.367 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:49 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:49.367 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:49 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:49.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:49 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:49.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:49 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:49.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:49 smithi053 ceph-mon[36496]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:49.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:49 smithi053 ceph-mon[36496]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-21T15:30:49.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:49 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:49.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:49 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:50.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:50 smithi045 ceph-mon[36409]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-21T15:30:50.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:50 smithi045 ceph-mon[36409]: osdmap e66: 10 total, 9 up, 10 in 2024-05-21T15:30:50.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:50 smithi045 ceph-mon[36409]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-05-21T15:30:50.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:50 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:50.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:50 smithi053 ceph-mon[36496]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-21T15:30:50.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:50 smithi053 ceph-mon[36496]: osdmap e66: 10 total, 9 up, 10 in 2024-05-21T15:30:50.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:50 smithi053 ceph-mon[36496]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-05-21T15:30:50.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:50 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:50.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:50 smithi038 ceph-mon[31995]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-21T15:30:50.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:50 smithi038 ceph-mon[31995]: osdmap e66: 10 total, 9 up, 10 in 2024-05-21T15:30:50.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:50 smithi038 ceph-mon[31995]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-05-21T15:30:50.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:50 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:50.959 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:30:50 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:30:50.646+0000 7fd9643af700 -1 osd.9 0 waiting for initial osdmap 2024-05-21T15:30:50.959 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:30:50 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:30:50.663+0000 7fd95b95b700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:30:51.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:51 smithi038 ceph-mon[31995]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-05-21T15:30:51.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:51 smithi038 ceph-mon[31995]: osdmap e67: 10 total, 9 up, 10 in 2024-05-21T15:30:51.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:51 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:51.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:51 smithi038 ceph-mon[31995]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:51.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:51 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:51.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:51 smithi038 ceph-mon[31995]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' 2024-05-21T15:30:51.296 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:51 smithi045 ceph-mon[36409]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-05-21T15:30:51.296 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:51 smithi045 ceph-mon[36409]: osdmap e67: 10 total, 9 up, 10 in 2024-05-21T15:30:51.296 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:51 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:51.296 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:51 smithi045 ceph-mon[36409]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:51.297 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:51 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:51.297 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:51 smithi045 ceph-mon[36409]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' 2024-05-21T15:30:51.343 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-21T15:30:51.344 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-21T15:30:51.344 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-05-21T15:30:51.344 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-05-21T15:30:51.344 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00889038 s, 1.2 GB/s 2024-05-21T15:30:51.344 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 2024-05-21T15:30:51.344 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-05-21T15:30:51.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:51 smithi053 ceph-mon[36496]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-05-21T15:30:51.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:51 smithi053 ceph-mon[36496]: osdmap e67: 10 total, 9 up, 10 in 2024-05-21T15:30:51.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:51 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:51.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:51 smithi053 ceph-mon[36496]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:51.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:51 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:51.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:51 smithi053 ceph-mon[36496]: from='osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649]' entity='osd.9' 2024-05-21T15:30:52.019 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_1 2024-05-21T15:30:52.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:52 smithi045 ceph-mon[36409]: purged_snaps scrub starts 2024-05-21T15:30:52.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:52 smithi045 ceph-mon[36409]: purged_snaps scrub ok 2024-05-21T15:30:52.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:52 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:52.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:52 smithi045 ceph-mon[36409]: osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649] boot 2024-05-21T15:30:52.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:52 smithi045 ceph-mon[36409]: osdmap e68: 10 total, 10 up, 10 in 2024-05-21T15:30:52.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:52 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:52.269 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:30:52.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:52 smithi053 ceph-mon[36496]: purged_snaps scrub starts 2024-05-21T15:30:52.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:52 smithi053 ceph-mon[36496]: purged_snaps scrub ok 2024-05-21T15:30:52.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:52 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:52.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:52 smithi053 ceph-mon[36496]: osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649] boot 2024-05-21T15:30:52.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:52 smithi053 ceph-mon[36496]: osdmap e68: 10 total, 10 up, 10 in 2024-05-21T15:30:52.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:52 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:52.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:52 smithi038 ceph-mon[31995]: purged_snaps scrub starts 2024-05-21T15:30:52.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:52 smithi038 ceph-mon[31995]: purged_snaps scrub ok 2024-05-21T15:30:52.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:52 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:52.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:52 smithi038 ceph-mon[31995]: osd.9 [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649] boot 2024-05-21T15:30:52.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:52 smithi038 ceph-mon[31995]: osdmap e68: 10 total, 10 up, 10 in 2024-05-21T15:30:52.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:52 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:30:53.926 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:53.926 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: Detected new or changed devices on smithi038 2024-05-21T15:30:53.926 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:53.926 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:53.927 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:53.927 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:53.927 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:53.927 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:53.927 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi038 to 4273M 2024-05-21T15:30:53.927 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:53.927 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:53.927 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:53.927 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:53.927 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:30:53.927 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:53.927 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:53.927 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:53.928 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:53 smithi045 ceph-mon[36409]: osdmap e69: 10 total, 10 up, 10 in 2024-05-21T15:30:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.935+0000 7fdefb7f0700 1 -- 172.21.15.45:0/4180633646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef4103b90 msgr2=0x7fdef410a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:30:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.935+0000 7fdefb7f0700 1 --2- 172.21.15.45:0/4180633646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef4103b90 0x7fdef410a440 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009990 tx=0x7fdee4009ca0 comp rx=0 tx=0).stop 2024-05-21T15:30:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.935+0000 7fdefb7f0700 1 -- 172.21.15.45:0/4180633646 shutdown_connections 2024-05-21T15:30:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.935+0000 7fdefb7f0700 1 --2- 172.21.15.45:0/4180633646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef4103b90 0x7fdef410a440 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.935+0000 7fdefb7f0700 1 --2- 172.21.15.45:0/4180633646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdef41020c0 0x7fdef41024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:53.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.935+0000 7fdefb7f0700 1 --2- 172.21.15.45:0/4180633646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef41031f0 0x7fdef4103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.935+0000 7fdefb7f0700 1 -- 172.21.15.45:0/4180633646 >> 172.21.15.45:0/4180633646 conn(0x7fdef40fd700 msgr2=0x7fdef40ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:30:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.935+0000 7fdefb7f0700 1 -- 172.21.15.45:0/4180633646 shutdown_connections 2024-05-21T15:30:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.935+0000 7fdefb7f0700 1 -- 172.21.15.45:0/4180633646 wait complete. 2024-05-21T15:30:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.936+0000 7fdefb7f0700 1 Processor -- start 2024-05-21T15:30:53.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.936+0000 7fdefb7f0700 1 -- start start 2024-05-21T15:30:53.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.936+0000 7fdefb7f0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdef41020c0 0x7fdef4105e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:30:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.936+0000 7fdefb7f0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef41031f0 0x7fdef4104530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:30:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.936+0000 7fdefb7f0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef4103b90 0x7fdef4104a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:30:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.936+0000 7fdefb7f0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdef410dc90 con 0x7fdef41020c0 2024-05-21T15:30:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.936+0000 7fdefb7f0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdef410db10 con 0x7fdef41031f0 2024-05-21T15:30:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.936+0000 7fdefb7f0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdef410de10 con 0x7fdef4103b90 2024-05-21T15:30:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.936+0000 7fdef8d8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef41031f0 0x7fdef4104530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:30:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.936+0000 7fdef8d8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef41031f0 0x7fdef4104530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37836/0 (socket says 172.21.15.45:37836) 2024-05-21T15:30:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.936+0000 7fdef8d8b700 1 -- 172.21.15.45:0/1093441256 learned_addr learned my addr 172.21.15.45:0/1093441256 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-21T15:30:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.937+0000 7fdef9d8d700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef4103b90 0x7fdef4104a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:30:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.937+0000 7fdef958c700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdef41020c0 0x7fdef4105e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:30:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.937+0000 7fdef8d8b700 1 -- 172.21.15.45:0/1093441256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef4103b90 msgr2=0x7fdef4104a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:30:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.937+0000 7fdef8d8b700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef4103b90 0x7fdef4104a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.937+0000 7fdef8d8b700 1 -- 172.21.15.45:0/1093441256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdef41020c0 msgr2=0x7fdef4105e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:30:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.937+0000 7fdef8d8b700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdef41020c0 0x7fdef4105e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:30:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.937+0000 7fdef8d8b700 1 -- 172.21.15.45:0/1093441256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef41051d0 con 0x7fdef41031f0 2024-05-21T15:30:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.937+0000 7fdef958c700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdef41020c0 0x7fdef4105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:30:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.937+0000 7fdef8d8b700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef41031f0 0x7fdef4104530 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdee800d950 tx=0x7fdee800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:30:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.937+0000 7fdef27fc700 1 -- 172.21.15.45:0/1093441256 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8016070 con 0x7fdef41031f0 2024-05-21T15:30:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.937+0000 7fdef27fc700 1 -- 172.21.15.45:0/1093441256 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee8009e20 con 0x7fdef41031f0 2024-05-21T15:30:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.937+0000 7fdefb7f0700 1 -- 172.21.15.45:0/1093441256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef41a38b0 con 0x7fdef41031f0 2024-05-21T15:30:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.937+0000 7fdef27fc700 1 -- 172.21.15.45:0/1093441256 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee800f620 con 0x7fdef41031f0 2024-05-21T15:30:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.937+0000 7fdefb7f0700 1 -- 172.21.15.45:0/1093441256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef41a3d90 con 0x7fdef41031f0 2024-05-21T15:30:53.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.938+0000 7fdefb7f0700 1 -- 172.21.15.45:0/1093441256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef4050170 con 0x7fdef41031f0 2024-05-21T15:30:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.940+0000 7fdef27fc700 1 -- 172.21.15.45:0/1093441256 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdee8019070 con 0x7fdef41031f0 2024-05-21T15:30:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.940+0000 7fdef27fc700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdedc0a2c20 0x7fdedc0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:30:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.940+0000 7fdef27fc700 1 -- 172.21.15.45:0/1093441256 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fdee80ce440 con 0x7fdef41031f0 2024-05-21T15:30:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.941+0000 7fdef958c700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdedc0a2c20 0x7fdedc0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:30:53.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.941+0000 7fdef958c700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdedc0a2c20 0x7fdedc0a5130 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdef41030f0 tx=0x7fdeec007c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:30:53.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:53.944+0000 7fdef27fc700 1 -- 172.21.15.45:0/1093441256 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdee809bba0 con 0x7fdef41031f0 2024-05-21T15:30:53.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:53.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: Detected new or changed devices on smithi038 2024-05-21T15:30:53.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:53.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:53.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:53.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:53.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:53.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:53.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi038 to 4273M 2024-05-21T15:30:53.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:53.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:53.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:53.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:53.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:30:53.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:53.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:53.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:53.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:53 smithi053 ceph-mon[36496]: osdmap e69: 10 total, 10 up, 10 in 2024-05-21T15:30:54.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:30:54.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: Detected new or changed devices on smithi038 2024-05-21T15:30:54.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:54.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:54.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:54.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:54.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:54.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:30:54.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi038 to 4273M 2024-05-21T15:30:54.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:54.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:54.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:54.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:54.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:30:54.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:54.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:30:54.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:30:54.027 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:53 smithi038 ceph-mon[31995]: osdmap e69: 10 total, 10 up, 10 in 2024-05-21T15:30:54.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:30:54.160+0000 7fdefb7f0700 1 -- 172.21.15.45:0/1093441256 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdef40ff690 con 0x7fdedc0a2c20 2024-05-21T15:30:54.947 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:30:54.948 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:30:54.948 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:54.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:30:54.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:30:54.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:55.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:30:55.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:30:55.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:30:55.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:55 smithi053 ceph-mon[36496]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:30:55.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:55 smithi053 ceph-mon[36496]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:30:55.958 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:55 smithi045 ceph-mon[36409]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:30:55.959 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:55 smithi045 ceph-mon[36409]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:30:56.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:55 smithi038 ceph-mon[31995]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:30:56.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:55 smithi038 ceph-mon[31995]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:30:57.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:57 smithi053 ceph-mon[36496]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:30:57.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:57 smithi053 ceph-mon[36496]: from='client.? 172.21.15.45:0/3481205845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56587d67-8492-49d1-8e4f-1ee050b7f9cc"}]: dispatch 2024-05-21T15:30:57.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:57 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56587d67-8492-49d1-8e4f-1ee050b7f9cc"}]: dispatch 2024-05-21T15:30:57.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:57 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56587d67-8492-49d1-8e4f-1ee050b7f9cc"}]': finished 2024-05-21T15:30:57.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:57 smithi053 ceph-mon[36496]: osdmap e70: 11 total, 10 up, 11 in 2024-05-21T15:30:57.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:30:58.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:57 smithi045 ceph-mon[36409]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:30:58.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:57 smithi045 ceph-mon[36409]: from='client.? 172.21.15.45:0/3481205845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56587d67-8492-49d1-8e4f-1ee050b7f9cc"}]: dispatch 2024-05-21T15:30:58.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:57 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56587d67-8492-49d1-8e4f-1ee050b7f9cc"}]: dispatch 2024-05-21T15:30:58.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:57 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56587d67-8492-49d1-8e4f-1ee050b7f9cc"}]': finished 2024-05-21T15:30:58.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:57 smithi045 ceph-mon[36409]: osdmap e70: 11 total, 10 up, 11 in 2024-05-21T15:30:58.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:30:58.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:57 smithi038 ceph-mon[31995]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:30:58.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:57 smithi038 ceph-mon[31995]: from='client.? 172.21.15.45:0/3481205845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56587d67-8492-49d1-8e4f-1ee050b7f9cc"}]: dispatch 2024-05-21T15:30:58.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:57 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56587d67-8492-49d1-8e4f-1ee050b7f9cc"}]: dispatch 2024-05-21T15:30:58.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:57 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56587d67-8492-49d1-8e4f-1ee050b7f9cc"}]': finished 2024-05-21T15:30:58.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:57 smithi038 ceph-mon[31995]: osdmap e70: 11 total, 10 up, 11 in 2024-05-21T15:30:58.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:30:58.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:58 smithi053 ceph-mon[36496]: from='client.? 172.21.15.45:0/1561238107' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:30:59.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:58 smithi045 ceph-mon[36409]: from='client.? 172.21.15.45:0/1561238107' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:30:59.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:58 smithi038 ceph-mon[31995]: from='client.? 172.21.15.45:0/1561238107' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:30:59.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:30:59 smithi053 ceph-mon[36496]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:00.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:30:59 smithi045 ceph-mon[36409]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:00.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:30:59 smithi038 ceph-mon[31995]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:01.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:01 smithi053 ceph-mon[36496]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:02.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:01 smithi045 ceph-mon[36409]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:02.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:01 smithi038 ceph-mon[31995]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:03.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:03 smithi053 ceph-mon[36496]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:04.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:03 smithi045 ceph-mon[36409]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:04.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:03 smithi038 ceph-mon[31995]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:05.908 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:05 smithi045 ceph-mon[36409]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:05.908 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:05 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-21T15:31:05.908 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:05 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:05.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:05 smithi053 ceph-mon[36496]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:05.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:05 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-21T15:31:05.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:05 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:06.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:05 smithi038 ceph-mon[31995]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:06.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:05 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-21T15:31:06.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:05 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:06.858 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:06 smithi045 ceph-mon[36409]: Deploying daemon osd.10 on smithi045 2024-05-21T15:31:06.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:06 smithi053 ceph-mon[36496]: Deploying daemon osd.10 on smithi045 2024-05-21T15:31:07.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:06 smithi038 ceph-mon[31995]: Deploying daemon osd.10 on smithi045 2024-05-21T15:31:07.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:07 smithi053 ceph-mon[36496]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:08.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:07 smithi045 ceph-mon[36409]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:08.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:07 smithi038 ceph-mon[31995]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:10.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:09 smithi045 ceph-mon[36409]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:10.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:09 smithi038 ceph-mon[31995]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:10.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:09 smithi053 ceph-mon[36496]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:11.941 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:11 smithi045 ceph-mon[36409]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:11.941 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:11 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:11.941 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:11 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:11.941 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:11 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:12.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:11 smithi038 ceph-mon[31995]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:12.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:11 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:12.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:11 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:12.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:11 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:12.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:11 smithi053 ceph-mon[36496]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:12.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:11 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:12.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:11 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:12.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:11 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:13.800 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:13 smithi045 ceph-mon[36409]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:14.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:13 smithi038 ceph-mon[31995]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:14.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:13 smithi053 ceph-mon[36496]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:14.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:31:14.882+0000 7fdef27fc700 1 -- 172.21.15.45:0/1093441256 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdef40ff690 con 0x7fdedc0a2c20 2024-05-21T15:31:14.884 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 10 on host 'smithi045' 2024-05-21T15:31:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:31:14.885+0000 7fdefb7f0700 1 -- 172.21.15.45:0/1093441256 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdedc0a2c20 msgr2=0x7fdedc0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:31:14.885+0000 7fdefb7f0700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdedc0a2c20 0x7fdedc0a5130 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdef41030f0 tx=0x7fdeec007c20 comp rx=0 tx=0).stop 2024-05-21T15:31:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:31:14.885+0000 7fdefb7f0700 1 -- 172.21.15.45:0/1093441256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef41031f0 msgr2=0x7fdef4104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:31:14.885+0000 7fdefb7f0700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef41031f0 0x7fdef4104530 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdee800d950 tx=0x7fdee800dc60 comp rx=0 tx=0).stop 2024-05-21T15:31:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:31:14.885+0000 7fdefb7f0700 1 -- 172.21.15.45:0/1093441256 shutdown_connections 2024-05-21T15:31:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:31:14.885+0000 7fdefb7f0700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdedc0a2c20 0x7fdedc0a5130 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:31:14.885+0000 7fdefb7f0700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef4103b90 0x7fdef4104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:31:14.885+0000 7fdefb7f0700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdef41020c0 0x7fdef4105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:31:14.885+0000 7fdefb7f0700 1 --2- 172.21.15.45:0/1093441256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef41031f0 0x7fdef4104530 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:31:14.885+0000 7fdefb7f0700 1 -- 172.21.15.45:0/1093441256 >> 172.21.15.45:0/1093441256 conn(0x7fdef40fd700 msgr2=0x7fdef40fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:31:14.885+0000 7fdefb7f0700 1 -- 172.21.15.45:0/1093441256 shutdown_connections 2024-05-21T15:31:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-21T15:31:14.885+0000 7fdefb7f0700 1 -- 172.21.15.45:0/1093441256 wait complete. 2024-05-21T15:31:15.639 DEBUG:teuthology.orchestra.run.smithi045:osd.10> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.10.service 2024-05-21T15:31:15.641 INFO:tasks.cephadm:Deploying osd.11 on smithi053 with /dev/vg_nvme/lv_1... 2024-05-21T15:31:15.641 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-21T15:31:15.899 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:31:15.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:15 smithi053 ceph-mon[36496]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:15.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:15 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:15.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:15 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:15.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:15 smithi053 ceph-mon[36496]: from='osd.10 [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T15:31:15.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:15 smithi053 ceph-mon[36496]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T15:31:15.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:15 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:15.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:15 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:16.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:15 smithi045 ceph-mon[36409]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:16.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:15 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:16.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:15 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:16.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:15 smithi045 ceph-mon[36409]: from='osd.10 [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T15:31:16.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:15 smithi045 ceph-mon[36409]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T15:31:16.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:15 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:16.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:15 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:16.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:15 smithi038 ceph-mon[31995]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:16.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:15 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:16.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:15 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:16.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:15 smithi038 ceph-mon[31995]: from='osd.10 [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T15:31:16.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:15 smithi038 ceph-mon[31995]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T15:31:16.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:15 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:16.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:15 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:16.894 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:16 smithi045 ceph-mon[36409]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-21T15:31:16.894 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:16 smithi045 ceph-mon[36409]: osdmap e71: 11 total, 10 up, 11 in 2024-05-21T15:31:16.894 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:16 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:31:16.894 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:16 smithi045 ceph-mon[36409]: from='osd.10 [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:31:16.895 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:16 smithi045 ceph-mon[36409]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:31:16.895 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:31:16 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:31:16.877+0000 7f5339fdd700 -1 osd.10 0 waiting for initial osdmap 2024-05-21T15:31:16.895 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:31:16 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:31:16.893+0000 7f5335d92700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:31:17.130 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:16 smithi053 ceph-mon[36496]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-21T15:31:17.131 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:16 smithi053 ceph-mon[36496]: osdmap e71: 11 total, 10 up, 11 in 2024-05-21T15:31:17.131 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:16 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:31:17.131 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:16 smithi053 ceph-mon[36496]: from='osd.10 [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:31:17.131 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:16 smithi053 ceph-mon[36496]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:31:17.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:16 smithi038 ceph-mon[31995]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-21T15:31:17.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:16 smithi038 ceph-mon[31995]: osdmap e71: 11 total, 10 up, 11 in 2024-05-21T15:31:17.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:16 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:31:17.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:16 smithi038 ceph-mon[31995]: from='osd.10 [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:31:17.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:16 smithi038 ceph-mon[31995]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-21T15:31:18.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:17 smithi053 ceph-mon[36496]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:18.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:17 smithi053 ceph-mon[36496]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T15:31:18.027 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:17 smithi053 ceph-mon[36496]: osdmap e72: 11 total, 10 up, 11 in 2024-05-21T15:31:18.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:31:18.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:17 smithi053 ceph-mon[36496]: osd.10 [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766] boot 2024-05-21T15:31:18.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:17 smithi053 ceph-mon[36496]: osdmap e73: 11 total, 11 up, 11 in 2024-05-21T15:31:18.028 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:31:18.028 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-21T15:31:18.028 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-21T15:31:18.028 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-05-21T15:31:18.028 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-05-21T15:31:18.028 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00830842 s, 1.3 GB/s 2024-05-21T15:31:18.029 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 2024-05-21T15:31:18.029 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-05-21T15:31:18.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:17 smithi045 ceph-mon[36409]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:18.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:17 smithi045 ceph-mon[36409]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T15:31:18.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:17 smithi045 ceph-mon[36409]: osdmap e72: 11 total, 10 up, 11 in 2024-05-21T15:31:18.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:31:18.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:17 smithi045 ceph-mon[36409]: osd.10 [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766] boot 2024-05-21T15:31:18.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:17 smithi045 ceph-mon[36409]: osdmap e73: 11 total, 11 up, 11 in 2024-05-21T15:31:18.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:31:18.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:17 smithi038 ceph-mon[31995]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:31:18.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:17 smithi038 ceph-mon[31995]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T15:31:18.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:17 smithi038 ceph-mon[31995]: osdmap e72: 11 total, 10 up, 11 in 2024-05-21T15:31:18.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:31:18.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:17 smithi038 ceph-mon[31995]: osd.10 [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766] boot 2024-05-21T15:31:18.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:17 smithi038 ceph-mon[31995]: osdmap e73: 11 total, 11 up, 11 in 2024-05-21T15:31:18.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:31:18.714 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_1 2024-05-21T15:31:18.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:18 smithi053 ceph-mon[36496]: purged_snaps scrub starts 2024-05-21T15:31:18.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:18 smithi053 ceph-mon[36496]: purged_snaps scrub ok 2024-05-21T15:31:18.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:18 smithi053 ceph-mon[36496]: osdmap e74: 11 total, 11 up, 11 in 2024-05-21T15:31:18.967 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:31:19.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:18 smithi045 ceph-mon[36409]: purged_snaps scrub starts 2024-05-21T15:31:19.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:18 smithi045 ceph-mon[36409]: purged_snaps scrub ok 2024-05-21T15:31:19.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:18 smithi045 ceph-mon[36409]: osdmap e74: 11 total, 11 up, 11 in 2024-05-21T15:31:19.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:18 smithi038 ceph-mon[31995]: purged_snaps scrub starts 2024-05-21T15:31:19.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:18 smithi038 ceph-mon[31995]: purged_snaps scrub ok 2024-05-21T15:31:19.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:18 smithi038 ceph-mon[31995]: osdmap e74: 11 total, 11 up, 11 in 2024-05-21T15:31:20.098 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:19 smithi053 ceph-mon[36496]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:20.099 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:19 smithi053 ceph-mon[36496]: osdmap e75: 11 total, 11 up, 11 in 2024-05-21T15:31:20.099 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:20.099 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:20.099 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:20.099 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:20.099 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:20.099 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:20.099 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:20.099 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:20.099 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:20.100 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:20.100 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:19 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:20.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:19 smithi045 ceph-mon[36409]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:20.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:19 smithi045 ceph-mon[36409]: osdmap e75: 11 total, 11 up, 11 in 2024-05-21T15:31:20.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:20.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:20.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:20.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:20.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:20.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:20.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:20.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:20.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:20.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:20.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:19 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:20.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:19 smithi038 ceph-mon[31995]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:20.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:19 smithi038 ceph-mon[31995]: osdmap e75: 11 total, 11 up, 11 in 2024-05-21T15:31:20.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:20.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:20.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:20.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:20.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:20.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:20.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:20.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:20.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:20.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:20.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:19 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:20.664 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.662+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/2858022075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f38104780 msgr2=0x7f8f38104b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:20.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.662+0000 7f8f3dd7e700 1 --2- 172.21.15.53:0/2858022075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f38104780 0x7f8f38104b90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009990 tx=0x7f8f28009ca0 comp rx=0 tx=0).stop 2024-05-21T15:31:20.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.662+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/2858022075 shutdown_connections 2024-05-21T15:31:20.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.662+0000 7f8f3dd7e700 1 --2- 172.21.15.53:0/2858022075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38101b20 0x7f8f3810b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:20.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.662+0000 7f8f3dd7e700 1 --2- 172.21.15.53:0/2858022075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f38104780 0x7f8f38104b90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:20.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.662+0000 7f8f3dd7e700 1 --2- 172.21.15.53:0/2858022075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f381011d0 0x7f8f381015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:20.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.662+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/2858022075 >> 172.21.15.53:0/2858022075 conn(0x7f8f380fcea0 msgr2=0x7f8f380ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:20.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.662+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/2858022075 shutdown_connections 2024-05-21T15:31:20.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.662+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/2858022075 wait complete. 2024-05-21T15:31:20.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.662+0000 7f8f3dd7e700 1 Processor -- start 2024-05-21T15:31:20.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.663+0000 7f8f3dd7e700 1 -- start start 2024-05-21T15:31:20.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.663+0000 7f8f3dd7e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f381011d0 0x7f8f38103d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:20.665 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.663+0000 7f8f3dd7e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38101b20 0x7f8f38102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:20.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.663+0000 7f8f3dd7e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f38104780 0x7f8f38102940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:20.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.663+0000 7f8f3dd7e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3810ed80 con 0x7f8f38101b20 2024-05-21T15:31:20.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.663+0000 7f8f3dd7e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3810ec00 con 0x7f8f38104780 2024-05-21T15:31:20.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.663+0000 7f8f3dd7e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3810ef00 con 0x7f8f381011d0 2024-05-21T15:31:20.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.663+0000 7f8f377fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f381011d0 0x7f8f38103d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:20.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.663+0000 7f8f377fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f381011d0 0x7f8f38103d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57176/0 (socket says 172.21.15.53:57176) 2024-05-21T15:31:20.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.663+0000 7f8f377fe700 1 -- 172.21.15.53:0/1045422076 learned_addr learned my addr 172.21.15.53:0/1045422076 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-21T15:31:20.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.663+0000 7f8f37fff700 1 --2- 172.21.15.53:0/1045422076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f38104780 0x7f8f38102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:20.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.663+0000 7f8f36ffd700 1 --2- 172.21.15.53:0/1045422076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38101b20 0x7f8f38102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:20.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.664+0000 7f8f377fe700 1 -- 172.21.15.53:0/1045422076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f38104780 msgr2=0x7f8f38102940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:20.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.664+0000 7f8f377fe700 1 --2- 172.21.15.53:0/1045422076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f38104780 0x7f8f38102940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:20.666 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.664+0000 7f8f377fe700 1 -- 172.21.15.53:0/1045422076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38101b20 msgr2=0x7f8f38102400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:20.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.664+0000 7f8f377fe700 1 --2- 172.21.15.53:0/1045422076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38101b20 0x7f8f38102400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:20.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.664+0000 7f8f377fe700 1 -- 172.21.15.53:0/1045422076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f381030a0 con 0x7f8f381011d0 2024-05-21T15:31:20.667 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.664+0000 7f8f377fe700 1 --2- 172.21.15.53:0/1045422076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f381011d0 0x7f8f38103d50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800b5c0 tx=0x7f8f280047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:20.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.664+0000 7f8f34ff9700 1 -- 172.21.15.53:0/1045422076 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28004830 con 0x7f8f381011d0 2024-05-21T15:31:20.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.664+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/1045422076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f381a3930 con 0x7f8f381011d0 2024-05-21T15:31:20.668 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.664+0000 7f8f34ff9700 1 -- 172.21.15.53:0/1045422076 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f28005020 con 0x7f8f381011d0 2024-05-21T15:31:20.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.664+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/1045422076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f381a3cd0 con 0x7f8f381011d0 2024-05-21T15:31:20.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.664+0000 7f8f34ff9700 1 -- 172.21.15.53:0/1045422076 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28003da0 con 0x7f8f381011d0 2024-05-21T15:31:20.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.665+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/1045422076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f24005360 con 0x7f8f381011d0 2024-05-21T15:31:20.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.666+0000 7f8f34ff9700 1 -- 172.21.15.53:0/1045422076 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8f280049d0 con 0x7f8f381011d0 2024-05-21T15:31:20.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.667+0000 7f8f34ff9700 1 --2- 172.21.15.53:0/1045422076 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f200a2ca0 0x7f8f200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:20.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.667+0000 7f8f34ff9700 1 -- 172.21.15.53:0/1045422076 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f8f280cfab0 con 0x7f8f381011d0 2024-05-21T15:31:20.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.667+0000 7f8f36ffd700 1 --2- 172.21.15.53:0/1045422076 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f200a2ca0 0x7f8f200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:20.669 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.668+0000 7f8f36ffd700 1 --2- 172.21.15.53:0/1045422076 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f200a2ca0 0x7f8f200a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f38103700 tx=0x7f8f2c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:20.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.672+0000 7f8f34ff9700 1 -- 172.21.15.53:0/1045422076 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8f2805d850 con 0x7f8f381011d0 2024-05-21T15:31:20.888 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:20.887+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/1045422076 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8f24000bf0 con 0x7f8f200a2ca0 2024-05-21T15:31:21.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:20 smithi053 ceph-mon[36496]: Detected new or changed devices on smithi045 2024-05-21T15:31:21.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:20 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi045 to 4273M 2024-05-21T15:31:21.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:20 smithi053 ceph-mon[36496]: osdmap e76: 11 total, 11 up, 11 in 2024-05-21T15:31:21.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:20 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:21.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:20 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:21.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:20 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:21.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:20 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:31:21.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:20 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:31:21.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:20 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:21.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:20 smithi045 ceph-mon[36409]: Detected new or changed devices on smithi045 2024-05-21T15:31:21.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:20 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi045 to 4273M 2024-05-21T15:31:21.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:20 smithi045 ceph-mon[36409]: osdmap e76: 11 total, 11 up, 11 in 2024-05-21T15:31:21.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:20 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:21.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:20 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:21.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:20 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:21.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:20 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:31:21.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:20 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:31:21.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:20 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:21.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:20 smithi038 ceph-mon[31995]: Detected new or changed devices on smithi045 2024-05-21T15:31:21.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:20 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi045 to 4273M 2024-05-21T15:31:21.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:20 smithi038 ceph-mon[31995]: osdmap e76: 11 total, 11 up, 11 in 2024-05-21T15:31:21.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:20 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:21.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:20 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:21.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:20 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:21.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:20 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:31:21.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:20 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:31:21.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:20 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:22.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:21 smithi053 ceph-mon[36496]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:22.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:21 smithi053 ceph-mon[36496]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:31:22.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:21 smithi053 ceph-mon[36496]: osdmap e77: 11 total, 11 up, 11 in 2024-05-21T15:31:22.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:21 smithi045 ceph-mon[36409]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:22.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:21 smithi045 ceph-mon[36409]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:31:22.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:21 smithi045 ceph-mon[36409]: osdmap e77: 11 total, 11 up, 11 in 2024-05-21T15:31:22.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:21 smithi038 ceph-mon[31995]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:22.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:21 smithi038 ceph-mon[31995]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:31:22.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:21 smithi038 ceph-mon[31995]: osdmap e77: 11 total, 11 up, 11 in 2024-05-21T15:31:24.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:23 smithi053 ceph-mon[36496]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:24.296 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:23 smithi053 ceph-mon[36496]: from='client.? 172.21.15.53:0/2377049028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b27f85e0-a264-4ee8-9e24-251bc01305c2"}]: dispatch 2024-05-21T15:31:24.297 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:23 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b27f85e0-a264-4ee8-9e24-251bc01305c2"}]: dispatch 2024-05-21T15:31:24.297 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:23 smithi053 ceph-mon[36496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b27f85e0-a264-4ee8-9e24-251bc01305c2"}]': finished 2024-05-21T15:31:24.297 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:23 smithi053 ceph-mon[36496]: osdmap e78: 12 total, 11 up, 12 in 2024-05-21T15:31:24.297 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:24.298 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:23 smithi038 ceph-mon[31995]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:24.298 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:23 smithi038 ceph-mon[31995]: from='client.? 172.21.15.53:0/2377049028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b27f85e0-a264-4ee8-9e24-251bc01305c2"}]: dispatch 2024-05-21T15:31:24.298 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:23 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b27f85e0-a264-4ee8-9e24-251bc01305c2"}]: dispatch 2024-05-21T15:31:24.299 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:23 smithi038 ceph-mon[31995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b27f85e0-a264-4ee8-9e24-251bc01305c2"}]': finished 2024-05-21T15:31:24.299 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:23 smithi038 ceph-mon[31995]: osdmap e78: 12 total, 11 up, 12 in 2024-05-21T15:31:24.299 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:24.299 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:23 smithi045 ceph-mon[36409]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:24.299 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:23 smithi045 ceph-mon[36409]: from='client.? 172.21.15.53:0/2377049028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b27f85e0-a264-4ee8-9e24-251bc01305c2"}]: dispatch 2024-05-21T15:31:24.299 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:23 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b27f85e0-a264-4ee8-9e24-251bc01305c2"}]: dispatch 2024-05-21T15:31:24.299 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:23 smithi045 ceph-mon[36409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b27f85e0-a264-4ee8-9e24-251bc01305c2"}]': finished 2024-05-21T15:31:24.299 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:23 smithi045 ceph-mon[36409]: osdmap e78: 12 total, 11 up, 12 in 2024-05-21T15:31:24.299 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:25.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:24 smithi053 ceph-mon[36496]: from='client.? 172.21.15.53:0/2376208973' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:31:25.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:24 smithi045 ceph-mon[36409]: from='client.? 172.21.15.53:0/2376208973' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:31:25.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:24 smithi038 ceph-mon[31995]: from='client.? 172.21.15.53:0/2376208973' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:31:26.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:25 smithi053 ceph-mon[36496]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-21T15:31:26.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:25 smithi045 ceph-mon[36409]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-21T15:31:26.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:25 smithi038 ceph-mon[31995]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-21T15:31:28.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:27 smithi053 ceph-mon[36496]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-21T15:31:28.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:27 smithi045 ceph-mon[36409]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-21T15:31:28.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:27 smithi038 ceph-mon[31995]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-21T15:31:30.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:29 smithi053 ceph-mon[36496]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:31:30.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:29 smithi045 ceph-mon[36409]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:31:30.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:29 smithi038 ceph-mon[31995]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:31:32.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:31 smithi053 ceph-mon[36496]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-21T15:31:32.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-21T15:31:32.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:32.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:31 smithi045 ceph-mon[36409]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-21T15:31:32.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-21T15:31:32.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:32.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:31 smithi038 ceph-mon[31995]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-21T15:31:32.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-21T15:31:32.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:33.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:32 smithi045 ceph-mon[36409]: Deploying daemon osd.11 on smithi053 2024-05-21T15:31:33.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:32 smithi038 ceph-mon[31995]: Deploying daemon osd.11 on smithi053 2024-05-21T15:31:33.413 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:32 smithi053 ceph-mon[36496]: Deploying daemon osd.11 on smithi053 2024-05-21T15:31:34.134 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:33 smithi053 ceph-mon[36496]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:31:34.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:33 smithi045 ceph-mon[36409]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:31:34.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:33 smithi038 ceph-mon[31995]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:31:36.077 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:35 smithi053 ceph-mon[36496]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:36.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:35 smithi045 ceph-mon[36409]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:36.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:35 smithi038 ceph-mon[31995]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:37.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:37 smithi038 ceph-mon[31995]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:37.280 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:37 smithi053 ceph-mon[36496]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:37.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:37 smithi045 ceph-mon[36409]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:38.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:38 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:38.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:38 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:38.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:38 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:38.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:38 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:38.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:38 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:38.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:38 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:38.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:38 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:38.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:38 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:38.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:38 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:39.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:39 smithi053 ceph-mon[36496]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:39.426 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:39 smithi038 ceph-mon[31995]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:39 smithi045 ceph-mon[36409]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:41.423 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:41 smithi053 ceph-mon[36496]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:41.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:41 smithi038 ceph-mon[31995]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:41.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:41 smithi045 ceph-mon[36409]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:41.809 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:41.808+0000 7f8f34ff9700 1 -- 172.21.15.53:0/1045422076 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8f24000bf0 con 0x7f8f200a2ca0 2024-05-21T15:31:41.810 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 11 on host 'smithi053' 2024-05-21T15:31:41.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:41.810+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/1045422076 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f200a2ca0 msgr2=0x7f8f200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:41.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:41.810+0000 7f8f3dd7e700 1 --2- 172.21.15.53:0/1045422076 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f200a2ca0 0x7f8f200a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f38103700 tx=0x7f8f2c00b410 comp rx=0 tx=0).stop 2024-05-21T15:31:41.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:41.810+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/1045422076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f381011d0 msgr2=0x7f8f38103d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:41.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:41.810+0000 7f8f3dd7e700 1 --2- 172.21.15.53:0/1045422076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f381011d0 0x7f8f38103d50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800b5c0 tx=0x7f8f280047b0 comp rx=0 tx=0).stop 2024-05-21T15:31:41.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:41.810+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/1045422076 shutdown_connections 2024-05-21T15:31:41.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:41.810+0000 7f8f3dd7e700 1 --2- 172.21.15.53:0/1045422076 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f200a2ca0 0x7f8f200a51b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:41.811 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:41.810+0000 7f8f3dd7e700 1 --2- 172.21.15.53:0/1045422076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f38104780 0x7f8f38102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:41.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:41.810+0000 7f8f3dd7e700 1 --2- 172.21.15.53:0/1045422076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f381011d0 0x7f8f38103d50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:41.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:41.810+0000 7f8f3dd7e700 1 --2- 172.21.15.53:0/1045422076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38101b20 0x7f8f38102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:41.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:41.810+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/1045422076 >> 172.21.15.53:0/1045422076 conn(0x7f8f380fcea0 msgr2=0x7f8f38107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:41.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:41.811+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/1045422076 shutdown_connections 2024-05-21T15:31:41.812 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:41.811+0000 7f8f3dd7e700 1 -- 172.21.15.53:0/1045422076 wait complete. 2024-05-21T15:31:42.479 DEBUG:teuthology.orchestra.run.smithi053:osd.11> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.11.service 2024-05-21T15:31:42.481 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-21T15:31:42.482 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-21T15:31:42.756 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:31:43.114 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:42 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:43.114 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:42 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:43.114 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:42 smithi038 ceph-mon[31995]: from='osd.11 [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T15:31:43.114 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:42 smithi038 ceph-mon[31995]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T15:31:43.114 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:42 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:43.114 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:42 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:43.196 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:42 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:43.196 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:42 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:43.196 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:42 smithi053 ceph-mon[36496]: from='osd.11 [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T15:31:43.196 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:42 smithi053 ceph-mon[36496]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T15:31:43.196 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:42 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:43.196 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:42 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:43.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:42 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:43.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:42 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:43.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:42 smithi045 ceph-mon[36409]: from='osd.11 [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T15:31:43.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:42 smithi045 ceph-mon[36409]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T15:31:43.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:42 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:43.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:42 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:44.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:43 smithi038 ceph-mon[31995]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:44.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:43 smithi038 ceph-mon[31995]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-21T15:31:44.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:43 smithi038 ceph-mon[31995]: from='osd.11 [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:31:44.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:43 smithi038 ceph-mon[31995]: osdmap e79: 12 total, 11 up, 12 in 2024-05-21T15:31:44.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:43 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:44.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:43 smithi038 ceph-mon[31995]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:31:44.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:43 smithi053 ceph-mon[36496]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:44.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:43 smithi053 ceph-mon[36496]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-21T15:31:44.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:43 smithi053 ceph-mon[36496]: from='osd.11 [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:31:44.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:43 smithi053 ceph-mon[36496]: osdmap e79: 12 total, 11 up, 12 in 2024-05-21T15:31:44.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:43 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:44.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:43 smithi053 ceph-mon[36496]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:31:44.207 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:31:43 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:31:43.809+0000 7ff24d686700 -1 osd.11 0 waiting for initial osdmap 2024-05-21T15:31:44.207 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:31:43 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:31:43.831+0000 7ff24943b700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:31:44.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:43 smithi045 ceph-mon[36409]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:44.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:43 smithi045 ceph-mon[36409]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-21T15:31:44.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:43 smithi045 ceph-mon[36409]: from='osd.11 [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:31:44.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:43 smithi045 ceph-mon[36409]: osdmap e79: 12 total, 11 up, 12 in 2024-05-21T15:31:44.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:43 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:44.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:43 smithi045 ceph-mon[36409]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-21T15:31:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.744+0000 7ff2bda66700 1 -- 172.21.15.38:0/3119318885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b81031e0 msgr2=0x7ff2b81035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.744+0000 7ff2bda66700 1 --2- 172.21.15.38:0/3119318885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b81031e0 0x7ff2b81035f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009990 tx=0x7ff2a0009ca0 comp rx=0 tx=0).stop 2024-05-21T15:31:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.744+0000 7ff2bda66700 1 -- 172.21.15.38:0/3119318885 shutdown_connections 2024-05-21T15:31:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.744+0000 7ff2bda66700 1 --2- 172.21.15.38:0/3119318885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2b8103b80 0x7ff2b810a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.744+0000 7ff2bda66700 1 --2- 172.21.15.38:0/3119318885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2b81020b0 0x7ff2b81024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.744+0000 7ff2bda66700 1 --2- 172.21.15.38:0/3119318885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b81031e0 0x7ff2b81035f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.744+0000 7ff2bda66700 1 -- 172.21.15.38:0/3119318885 >> 172.21.15.38:0/3119318885 conn(0x7ff2b80fd6f0 msgr2=0x7ff2b80ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.745+0000 7ff2bda66700 1 -- 172.21.15.38:0/3119318885 shutdown_connections 2024-05-21T15:31:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.745+0000 7ff2bda66700 1 -- 172.21.15.38:0/3119318885 wait complete. 2024-05-21T15:31:44.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.746+0000 7ff2bda66700 1 Processor -- start 2024-05-21T15:31:44.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.746+0000 7ff2bda66700 1 -- start start 2024-05-21T15:31:44.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.746+0000 7ff2bda66700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2b81020b0 0x7ff2b81a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:44.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.746+0000 7ff2bda66700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2b81031e0 0x7ff2b81a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:44.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.746+0000 7ff2bda66700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b8103b80 0x7ff2b81a7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:44.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.746+0000 7ff2bda66700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b810dc70 con 0x7ff2b8103b80 2024-05-21T15:31:44.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.746+0000 7ff2bda66700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b810daf0 con 0x7ff2b81031e0 2024-05-21T15:31:44.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.746+0000 7ff2bda66700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b810ddf0 con 0x7ff2b81020b0 2024-05-21T15:31:44.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.746+0000 7ff2b7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b8103b80 0x7ff2b81a7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:44.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.746+0000 7ff2b77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2b81020b0 0x7ff2b81a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:44.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.746+0000 7ff2b7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b8103b80 0x7ff2b81a7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34726/0 (socket says 172.21.15.38:34726) 2024-05-21T15:31:44.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.746+0000 7ff2b7fff700 1 -- 172.21.15.38:0/1804400685 learned_addr learned my addr 172.21.15.38:0/1804400685 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:31:44.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.747+0000 7ff2aeffd700 1 --2- 172.21.15.38:0/1804400685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2b81031e0 0x7ff2b81a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:44.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.747+0000 7ff2b7fff700 1 -- 172.21.15.38:0/1804400685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2b81020b0 msgr2=0x7ff2b81a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:44.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.747+0000 7ff2b7fff700 1 --2- 172.21.15.38:0/1804400685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2b81020b0 0x7ff2b81a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:44.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.747+0000 7ff2b7fff700 1 -- 172.21.15.38:0/1804400685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2b81031e0 msgr2=0x7ff2b81a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:44.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.747+0000 7ff2b7fff700 1 --2- 172.21.15.38:0/1804400685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2b81031e0 0x7ff2b81a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:44.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.747+0000 7ff2b7fff700 1 -- 172.21.15.38:0/1804400685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b81a7a00 con 0x7ff2b8103b80 2024-05-21T15:31:44.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.747+0000 7ff2aeffd700 1 --2- 172.21.15.38:0/1804400685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2b81031e0 0x7ff2b81a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:31:44.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.747+0000 7ff2b7fff700 1 --2- 172.21.15.38:0/1804400685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b8103b80 0x7ff2b81a7380 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000ef30 tx=0x7ff2b000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:44.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.747+0000 7ff2b57fa700 1 -- 172.21.15.38:0/1804400685 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b00092e0 con 0x7ff2b8103b80 2024-05-21T15:31:44.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.747+0000 7ff2bda66700 1 -- 172.21.15.38:0/1804400685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b81a7c90 con 0x7ff2b8103b80 2024-05-21T15:31:44.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.747+0000 7ff2b57fa700 1 -- 172.21.15.38:0/1804400685 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b0004bb0 con 0x7ff2b8103b80 2024-05-21T15:31:44.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.747+0000 7ff2b57fa700 1 -- 172.21.15.38:0/1804400685 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0004280 con 0x7ff2b8103b80 2024-05-21T15:31:44.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.747+0000 7ff2bda66700 1 -- 172.21.15.38:0/1804400685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b81a8180 con 0x7ff2b8103b80 2024-05-21T15:31:44.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.748+0000 7ff2bda66700 1 -- 172.21.15.38:0/1804400685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b8050170 con 0x7ff2b8103b80 2024-05-21T15:31:44.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.750+0000 7ff2b57fa700 1 -- 172.21.15.38:0/1804400685 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff2b001b030 con 0x7ff2b8103b80 2024-05-21T15:31:44.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.750+0000 7ff2b57fa700 1 --2- 172.21.15.38:0/1804400685 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff2a40a2c20 0x7ff2a40a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:44.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.750+0000 7ff2b77fe700 1 --2- 172.21.15.38:0/1804400685 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff2a40a2c20 0x7ff2a40a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:44.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.751+0000 7ff2b57fa700 1 -- 172.21.15.38:0/1804400685 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7ff2b00cefb0 con 0x7ff2b8103b80 2024-05-21T15:31:44.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.751+0000 7ff2b77fe700 1 --2- 172.21.15.38:0/1804400685 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff2a40a2c20 0x7ff2a40a5130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff2b81030e0 tx=0x7ff2a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:44.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.754+0000 7ff2b57fa700 1 -- 172.21.15.38:0/1804400685 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff2b0022020 con 0x7ff2b8103b80 2024-05-21T15:31:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.966+0000 7ff2bda66700 1 -- 172.21.15.38:0/1804400685 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff2b81a85b0 con 0x7ff2b8103b80 2024-05-21T15:31:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.966+0000 7ff2b57fa700 1 -- 172.21.15.38:0/1804400685 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7ff2b81a85b0 con 0x7ff2b8103b80 2024-05-21T15:31:44.967 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:31:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.969+0000 7ff2bda66700 1 -- 172.21.15.38:0/1804400685 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff2a40a2c20 msgr2=0x7ff2a40a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.969+0000 7ff2bda66700 1 --2- 172.21.15.38:0/1804400685 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff2a40a2c20 0x7ff2a40a5130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff2b81030e0 tx=0x7ff2a8009260 comp rx=0 tx=0).stop 2024-05-21T15:31:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.969+0000 7ff2bda66700 1 -- 172.21.15.38:0/1804400685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b8103b80 msgr2=0x7ff2b81a7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.969+0000 7ff2bda66700 1 --2- 172.21.15.38:0/1804400685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b8103b80 0x7ff2b81a7380 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000ef30 tx=0x7ff2b000c3a0 comp rx=0 tx=0).stop 2024-05-21T15:31:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.969+0000 7ff2bda66700 1 -- 172.21.15.38:0/1804400685 shutdown_connections 2024-05-21T15:31:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.969+0000 7ff2bda66700 1 --2- 172.21.15.38:0/1804400685 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff2a40a2c20 0x7ff2a40a5130 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.969+0000 7ff2bda66700 1 --2- 172.21.15.38:0/1804400685 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b8103b80 0x7ff2b81a7380 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.969+0000 7ff2bda66700 1 --2- 172.21.15.38:0/1804400685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2b81020b0 0x7ff2b81a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.969+0000 7ff2bda66700 1 --2- 172.21.15.38:0/1804400685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2b81031e0 0x7ff2b81a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.969+0000 7ff2bda66700 1 -- 172.21.15.38:0/1804400685 >> 172.21.15.38:0/1804400685 conn(0x7ff2b80fd6f0 msgr2=0x7ff2b80fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.969+0000 7ff2bda66700 1 -- 172.21.15.38:0/1804400685 shutdown_connections 2024-05-21T15:31:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:44.969+0000 7ff2bda66700 1 -- 172.21.15.38:0/1804400685 wait complete. 2024-05-21T15:31:45.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:44 smithi038 ceph-mon[31995]: purged_snaps scrub starts 2024-05-21T15:31:45.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:44 smithi038 ceph-mon[31995]: purged_snaps scrub ok 2024-05-21T15:31:45.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:44 smithi038 ceph-mon[31995]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-21T15:31:45.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:44 smithi038 ceph-mon[31995]: osdmap e80: 12 total, 11 up, 12 in 2024-05-21T15:31:45.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:44 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:45.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:44 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:45.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:44 smithi053 ceph-mon[36496]: purged_snaps scrub starts 2024-05-21T15:31:45.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:44 smithi053 ceph-mon[36496]: purged_snaps scrub ok 2024-05-21T15:31:45.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:44 smithi053 ceph-mon[36496]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-21T15:31:45.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:44 smithi053 ceph-mon[36496]: osdmap e80: 12 total, 11 up, 12 in 2024-05-21T15:31:45.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:44 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:45.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:44 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:45.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:44 smithi045 ceph-mon[36409]: purged_snaps scrub starts 2024-05-21T15:31:45.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:44 smithi045 ceph-mon[36409]: purged_snaps scrub ok 2024-05-21T15:31:45.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:44 smithi045 ceph-mon[36409]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-21T15:31:45.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:44 smithi045 ceph-mon[36409]: osdmap e80: 12 total, 11 up, 12 in 2024-05-21T15:31:45.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:44 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:45.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:44 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:45.732 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1716305504,"num_in_osds":12,"osd_in_since":1716305483,"num_remapped_pgs":0} 2024-05-21T15:31:45.732 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T15:31:45.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:45 smithi053 ceph-mon[36496]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:45.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:45 smithi053 ceph-mon[36496]: osd.11 [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423] boot 2024-05-21T15:31:45.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:45 smithi053 ceph-mon[36496]: osdmap e81: 12 total, 12 up, 12 in 2024-05-21T15:31:45.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:45 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:45.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:45 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1804400685' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T15:31:45.995 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:31:46.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:45 smithi038 ceph-mon[31995]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:46.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:45 smithi038 ceph-mon[31995]: osd.11 [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423] boot 2024-05-21T15:31:46.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:45 smithi038 ceph-mon[31995]: osdmap e81: 12 total, 12 up, 12 in 2024-05-21T15:31:46.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:45 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:46.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:45 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1804400685' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T15:31:46.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:45 smithi045 ceph-mon[36409]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:31:46.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:45 smithi045 ceph-mon[36409]: osd.11 [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423] boot 2024-05-21T15:31:46.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:45 smithi045 ceph-mon[36409]: osdmap e81: 12 total, 12 up, 12 in 2024-05-21T15:31:46.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:45 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:31:46.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:45 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1804400685' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T15:31:47.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:46 smithi053 ceph-mon[36496]: osdmap e82: 12 total, 12 up, 12 in 2024-05-21T15:31:47.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:46 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:47.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:46 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:47.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:46 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:47.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:46 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:47.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:46 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:47.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:46 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:47.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:46 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:47.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:46 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:47.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:46 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:47.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:46 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:47.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:46 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:47.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:46 smithi045 ceph-mon[36409]: osdmap e82: 12 total, 12 up, 12 in 2024-05-21T15:31:47.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:46 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:47.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:46 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:47.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:46 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:47.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:46 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:47.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:46 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:47.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:46 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:47.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:46 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:47.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:46 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:47.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:46 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:47.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:46 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:47.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:46 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:47.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:46 smithi038 ceph-mon[31995]: osdmap e82: 12 total, 12 up, 12 in 2024-05-21T15:31:47.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:46 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:47.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:46 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:47.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:46 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:47.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:46 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:47.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:46 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:47.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:46 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:31:47.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:46 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:47.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:46 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:47.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:46 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:47.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:46 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:47.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:46 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:47.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.859+0000 7fd025bac700 1 -- 172.21.15.38:0/3526275251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd02010d770 msgr2=0x7fd02010fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:47.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.859+0000 7fd025bac700 1 --2- 172.21.15.38:0/3526275251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd02010d770 0x7fd02010fc50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd01800b0a0 tx=0x7fd01800b3b0 comp rx=0 tx=0).stop 2024-05-21T15:31:47.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.859+0000 7fd025bac700 1 -- 172.21.15.38:0/3526275251 shutdown_connections 2024-05-21T15:31:47.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.859+0000 7fd025bac700 1 --2- 172.21.15.38:0/3526275251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd02010d770 0x7fd02010fc50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:47.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.859+0000 7fd025bac700 1 --2- 172.21.15.38:0/3526275251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd020104300 0x7fd020104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:47.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.859+0000 7fd025bac700 1 --2- 172.21.15.38:0/3526275251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd020104c50 0x7fd02010d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:47.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.859+0000 7fd025bac700 1 -- 172.21.15.38:0/3526275251 >> 172.21.15.38:0/3526275251 conn(0x7fd0200fddb0 msgr2=0x7fd0201001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:47.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.859+0000 7fd025bac700 1 -- 172.21.15.38:0/3526275251 shutdown_connections 2024-05-21T15:31:47.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.859+0000 7fd025bac700 1 -- 172.21.15.38:0/3526275251 wait complete. 2024-05-21T15:31:47.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.860+0000 7fd025bac700 1 Processor -- start 2024-05-21T15:31:47.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.860+0000 7fd025bac700 1 -- start start 2024-05-21T15:31:47.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd025bac700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd020104300 0x7fd020101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:47.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd025bac700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd020104c50 0x7fd020102530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:47.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd025bac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd02010d770 0x7fd020102a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:47.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd025bac700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd020113440 con 0x7fd020104300 2024-05-21T15:31:47.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd025bac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0201132c0 con 0x7fd02010d770 2024-05-21T15:31:47.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd025bac700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd0201135c0 con 0x7fd020104c50 2024-05-21T15:31:47.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd01f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd020104300 0x7fd020101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:47.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd01f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd020104300 0x7fd020101ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34740/0 (socket says 172.21.15.38:34740) 2024-05-21T15:31:47.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd01f7fe700 1 -- 172.21.15.38:0/1340402271 learned_addr learned my addr 172.21.15.38:0/1340402271 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:31:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd01effd700 1 --2- 172.21.15.38:0/1340402271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd020104c50 0x7fd020102530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd01ffff700 1 --2- 172.21.15.38:0/1340402271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd02010d770 0x7fd020102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd01f7fe700 1 -- 172.21.15.38:0/1340402271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd020104c50 msgr2=0x7fd020102530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd01f7fe700 1 --2- 172.21.15.38:0/1340402271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd020104c50 0x7fd020102530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd01f7fe700 1 -- 172.21.15.38:0/1340402271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd02010d770 msgr2=0x7fd020102a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd01f7fe700 1 --2- 172.21.15.38:0/1340402271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd02010d770 0x7fd020102a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.861+0000 7fd01f7fe700 1 -- 172.21.15.38:0/1340402271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0201a80b0 con 0x7fd020104300 2024-05-21T15:31:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.862+0000 7fd01f7fe700 1 --2- 172.21.15.38:0/1340402271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd020104300 0x7fd020101ff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd01000d980 tx=0x7fd01000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.862+0000 7fd01cff9700 1 -- 172.21.15.38:0/1340402271 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd010017070 con 0x7fd020104300 2024-05-21T15:31:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.862+0000 7fd01cff9700 1 -- 172.21.15.38:0/1340402271 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd010005020 con 0x7fd020104300 2024-05-21T15:31:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.862+0000 7fd025bac700 1 -- 172.21.15.38:0/1340402271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0201a83a0 con 0x7fd020104300 2024-05-21T15:31:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.862+0000 7fd01cff9700 1 -- 172.21.15.38:0/1340402271 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01000fbf0 con 0x7fd020104300 2024-05-21T15:31:47.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.862+0000 7fd025bac700 1 -- 172.21.15.38:0/1340402271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0201a8890 con 0x7fd020104300 2024-05-21T15:31:47.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.863+0000 7fd025bac700 1 -- 172.21.15.38:0/1340402271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd020050170 con 0x7fd020104300 2024-05-21T15:31:47.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.864+0000 7fd01cff9700 1 -- 172.21.15.38:0/1340402271 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd010004a30 con 0x7fd020104300 2024-05-21T15:31:47.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.865+0000 7fd01cff9700 1 --2- 172.21.15.38:0/1340402271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd0080a2c20 0x7fd0080a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:47.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.865+0000 7fd01effd700 1 --2- 172.21.15.38:0/1340402271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd0080a2c20 0x7fd0080a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:47.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.865+0000 7fd01cff9700 1 -- 172.21.15.38:0/1340402271 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd0100cea40 con 0x7fd020104300 2024-05-21T15:31:47.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.866+0000 7fd01effd700 1 --2- 172.21.15.38:0/1340402271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd0080a2c20 0x7fd0080a5130 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd020100bc0 tx=0x7fd01400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:47.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:47.870+0000 7fd01cff9700 1 -- 172.21.15.38:0/1340402271 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd010020020 con 0x7fd020104300 2024-05-21T15:31:48.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:47 smithi038 ceph-mon[31995]: Detected new or changed devices on smithi053 2024-05-21T15:31:48.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:47 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi053 to 4273M 2024-05-21T15:31:48.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:47 smithi038 ceph-mon[31995]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:48.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:47 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:48.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:47 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:48.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:47 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:48.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:48.076+0000 7fd025bac700 1 -- 172.21.15.38:0/1340402271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd02002dc60 con 0x7fd020104300 2024-05-21T15:31:48.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:48.077+0000 7fd01cff9700 1 -- 172.21.15.38:0/1340402271 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18349 (secure 0 0 0) 0x7fd01009be40 con 0x7fd020104300 2024-05-21T15:31:48.078 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:31:48.078 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":82,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:31:45.813596+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:31:48.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:48.079+0000 7fd025bac700 1 -- 172.21.15.38:0/1340402271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd0080a2c20 msgr2=0x7fd0080a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:48.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:48.079+0000 7fd025bac700 1 --2- 172.21.15.38:0/1340402271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd0080a2c20 0x7fd0080a5130 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd020100bc0 tx=0x7fd01400b410 comp rx=0 tx=0).stop 2024-05-21T15:31:48.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:48.079+0000 7fd025bac700 1 -- 172.21.15.38:0/1340402271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd020104300 msgr2=0x7fd020101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:48.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:48.079+0000 7fd025bac700 1 --2- 172.21.15.38:0/1340402271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd020104300 0x7fd020101ff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd01000d980 tx=0x7fd01000dc90 comp rx=0 tx=0).stop 2024-05-21T15:31:48.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:48.079+0000 7fd025bac700 1 -- 172.21.15.38:0/1340402271 shutdown_connections 2024-05-21T15:31:48.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:48.079+0000 7fd025bac700 1 --2- 172.21.15.38:0/1340402271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd0080a2c20 0x7fd0080a5130 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:48.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:48.079+0000 7fd025bac700 1 --2- 172.21.15.38:0/1340402271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd02010d770 0x7fd020102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:48.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:48.079+0000 7fd025bac700 1 --2- 172.21.15.38:0/1340402271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd020104300 0x7fd020101ff0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:48.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:48.079+0000 7fd025bac700 1 --2- 172.21.15.38:0/1340402271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd020104c50 0x7fd020102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:48.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:48.079+0000 7fd025bac700 1 -- 172.21.15.38:0/1340402271 >> 172.21.15.38:0/1340402271 conn(0x7fd0200fddb0 msgr2=0x7fd0200fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:48.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:48.079+0000 7fd025bac700 1 -- 172.21.15.38:0/1340402271 shutdown_connections 2024-05-21T15:31:48.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:48.080+0000 7fd025bac700 1 -- 172.21.15.38:0/1340402271 wait complete. 2024-05-21T15:31:48.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:47 smithi053 ceph-mon[36496]: Detected new or changed devices on smithi053 2024-05-21T15:31:48.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:47 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi053 to 4273M 2024-05-21T15:31:48.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:47 smithi053 ceph-mon[36496]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:48.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:47 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:48.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:47 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:48.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:47 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:48.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:47 smithi045 ceph-mon[36409]: Detected new or changed devices on smithi053 2024-05-21T15:31:48.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:47 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi053 to 4273M 2024-05-21T15:31:48.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:47 smithi045 ceph-mon[36409]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:48.267 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:47 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:48.267 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:47 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:48.267 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:47 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:48.737 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:27:50.659367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:31:48.737 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-21T15:31:48.986 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:31:49.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:48 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1340402271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:31:49.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:48 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1340402271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:31:49.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:48 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1340402271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:31:50.127 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:49 smithi038 ceph-mon[31995]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:50.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:49 smithi053 ceph-mon[36496]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:50.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:49 smithi045 ceph-mon[36409]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:50.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.781+0000 7f9000868700 1 -- 172.21.15.38:0/3644211076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ffc10d760 msgr2=0x7f8ffc10fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:50.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.781+0000 7f9000868700 1 --2- 172.21.15.38:0/3644211076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ffc10d760 0x7f8ffc10fc40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400b0a0 tx=0x7f8ff400b3b0 comp rx=0 tx=0).stop 2024-05-21T15:31:50.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.781+0000 7f9000868700 1 -- 172.21.15.38:0/3644211076 shutdown_connections 2024-05-21T15:31:50.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.781+0000 7f9000868700 1 --2- 172.21.15.38:0/3644211076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ffc10d760 0x7f8ffc10fc40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:50.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.781+0000 7f9000868700 1 --2- 172.21.15.38:0/3644211076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ffc104380 0x7f8ffc104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:50.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.781+0000 7f9000868700 1 --2- 172.21.15.38:0/3644211076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ffc104cd0 0x7f8ffc10d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:50.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.781+0000 7f9000868700 1 -- 172.21.15.38:0/3644211076 >> 172.21.15.38:0/3644211076 conn(0x7f8ffc0fdc40 msgr2=0x7f8ffc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:50.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.782+0000 7f9000868700 1 -- 172.21.15.38:0/3644211076 shutdown_connections 2024-05-21T15:31:50.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.782+0000 7f9000868700 1 -- 172.21.15.38:0/3644211076 wait complete. 2024-05-21T15:31:50.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.782+0000 7f9000868700 1 Processor -- start 2024-05-21T15:31:50.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.782+0000 7f9000868700 1 -- start start 2024-05-21T15:31:50.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f9000868700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ffc104380 0x7f8ffc102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:50.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f9000868700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ffc104cd0 0x7f8ffc1025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f9000868700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ffc10d760 0x7f8ffc102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f9000868700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc113410 con 0x7f8ffc10d760 2024-05-21T15:31:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f9000868700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc113290 con 0x7f8ffc104cd0 2024-05-21T15:31:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f9000868700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc113590 con 0x7f8ffc104380 2024-05-21T15:31:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f8ffad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ffc10d760 0x7f8ffc102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f8ffad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ffc10d760 0x7f8ffc102af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34770/0 (socket says 172.21.15.38:34770) 2024-05-21T15:31:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f8ffa59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ffc104380 0x7f8ffc102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f8ffad9d700 1 -- 172.21.15.38:0/4114643479 learned_addr learned my addr 172.21.15.38:0/4114643479 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:31:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f8ff9d9b700 1 --2- 172.21.15.38:0/4114643479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ffc104cd0 0x7f8ffc1025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f8ffad9d700 1 -- 172.21.15.38:0/4114643479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ffc104380 msgr2=0x7f8ffc102070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f8ffad9d700 1 --2- 172.21.15.38:0/4114643479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ffc104380 0x7f8ffc102070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:50.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f8ffad9d700 1 -- 172.21.15.38:0/4114643479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ffc104cd0 msgr2=0x7f8ffc1025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:50.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f8ffad9d700 1 --2- 172.21.15.38:0/4114643479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ffc104cd0 0x7f8ffc1025b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:50.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f8ffad9d700 1 -- 172.21.15.38:0/4114643479 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ffc1a80a0 con 0x7f8ffc10d760 2024-05-21T15:31:50.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.783+0000 7f8ffad9d700 1 --2- 172.21.15.38:0/4114643479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ffc10d760 0x7f8ffc102af0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400b780 tx=0x7f8ff4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:50.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.784+0000 7f8feb7fe700 1 -- 172.21.15.38:0/4114643479 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4021070 con 0x7f8ffc10d760 2024-05-21T15:31:50.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.784+0000 7f8feb7fe700 1 -- 172.21.15.38:0/4114643479 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ff4007b80 con 0x7f8ffc10d760 2024-05-21T15:31:50.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.784+0000 7f9000868700 1 -- 172.21.15.38:0/4114643479 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ffc1a8330 con 0x7f8ffc10d760 2024-05-21T15:31:50.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.784+0000 7f8feb7fe700 1 -- 172.21.15.38:0/4114643479 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40052a0 con 0x7f8ffc10d760 2024-05-21T15:31:50.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.784+0000 7f9000868700 1 -- 172.21.15.38:0/4114643479 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ffc1a8870 con 0x7f8ffc10d760 2024-05-21T15:31:50.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.785+0000 7f9000868700 1 -- 172.21.15.38:0/4114643479 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ffc050170 con 0x7f8ffc10d760 2024-05-21T15:31:50.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.786+0000 7f8feb7fe700 1 -- 172.21.15.38:0/4114643479 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8ff4024070 con 0x7f8ffc10d760 2024-05-21T15:31:50.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.787+0000 7f8feb7fe700 1 --2- 172.21.15.38:0/4114643479 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8fe40a2c20 0x7f8fe40a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:50.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.787+0000 7f8ffa59c700 1 --2- 172.21.15.38:0/4114643479 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8fe40a2c20 0x7f8fe40a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:50.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.787+0000 7f8feb7fe700 1 -- 172.21.15.38:0/4114643479 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8ff400f020 con 0x7f8ffc10d760 2024-05-21T15:31:50.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.787+0000 7f8ffa59c700 1 --2- 172.21.15.38:0/4114643479 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8fe40a2c20 0x7f8fe40a5130 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009c60 tx=0x7f8fec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:50.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.792+0000 7f8feb7fe700 1 -- 172.21.15.38:0/4114643479 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8ff402b020 con 0x7f8ffc10d760 2024-05-21T15:31:51.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.998+0000 7f9000868700 1 -- 172.21.15.38:0/4114643479 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8ffc1a8a40 con 0x7f8ffc10d760 2024-05-21T15:31:51.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:50.998+0000 7f8feb7fe700 1 -- 172.21.15.38:0/4114643479 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f8ff409d540 con 0x7f8ffc10d760 2024-05-21T15:31:51.000 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-05-21T15:31:51.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:51.000+0000 7f9000868700 1 -- 172.21.15.38:0/4114643479 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8fe40a2c20 msgr2=0x7f8fe40a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:51.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:51.000+0000 7f9000868700 1 --2- 172.21.15.38:0/4114643479 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8fe40a2c20 0x7f8fe40a5130 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009c60 tx=0x7f8fec009260 comp rx=0 tx=0).stop 2024-05-21T15:31:51.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:51.000+0000 7f9000868700 1 -- 172.21.15.38:0/4114643479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ffc10d760 msgr2=0x7f8ffc102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:51.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:51.000+0000 7f9000868700 1 --2- 172.21.15.38:0/4114643479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ffc10d760 0x7f8ffc102af0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400b780 tx=0x7f8ff4004d00 comp rx=0 tx=0).stop 2024-05-21T15:31:51.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:51.000+0000 7f9000868700 1 -- 172.21.15.38:0/4114643479 shutdown_connections 2024-05-21T15:31:51.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:51.000+0000 7f9000868700 1 --2- 172.21.15.38:0/4114643479 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8fe40a2c20 0x7f8fe40a5130 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:51.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:51.000+0000 7f9000868700 1 --2- 172.21.15.38:0/4114643479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ffc10d760 0x7f8ffc102af0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:51.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:51.000+0000 7f9000868700 1 --2- 172.21.15.38:0/4114643479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ffc104380 0x7f8ffc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:51.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:51.000+0000 7f9000868700 1 --2- 172.21.15.38:0/4114643479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ffc104cd0 0x7f8ffc1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:51.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:51.000+0000 7f9000868700 1 -- 172.21.15.38:0/4114643479 >> 172.21.15.38:0/4114643479 conn(0x7f8ffc0fdc40 msgr2=0x7f8ffc107270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:51.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:51.001+0000 7f9000868700 1 -- 172.21.15.38:0/4114643479 shutdown_connections 2024-05-21T15:31:51.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:51.001+0000 7f9000868700 1 -- 172.21.15.38:0/4114643479 wait complete. 2024-05-21T15:31:51.726 INFO:tasks.cephadm:Adding mds.a on smithi038 2024-05-21T15:31:51.726 INFO:tasks.cephadm:Adding mds.d on smithi038 2024-05-21T15:31:51.726 INFO:tasks.cephadm:Adding mds.g on smithi038 2024-05-21T15:31:51.726 INFO:tasks.cephadm:Adding mds.j on smithi038 2024-05-21T15:31:51.727 INFO:tasks.cephadm:Adding mds.b on smithi045 2024-05-21T15:31:51.727 INFO:tasks.cephadm:Adding mds.e on smithi045 2024-05-21T15:31:51.727 INFO:tasks.cephadm:Adding mds.h on smithi045 2024-05-21T15:31:51.727 INFO:tasks.cephadm:Adding mds.k on smithi045 2024-05-21T15:31:51.727 INFO:tasks.cephadm:Adding mds.c on smithi053 2024-05-21T15:31:51.727 INFO:tasks.cephadm:Adding mds.f on smithi053 2024-05-21T15:31:51.727 INFO:tasks.cephadm:Adding mds.i on smithi053 2024-05-21T15:31:51.727 INFO:tasks.cephadm:Adding mds.l on smithi053 2024-05-21T15:31:51.727 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph orch apply mds all '12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l' 2024-05-21T15:31:51.987 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:31:52.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:51 smithi038 ceph-mon[31995]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:52.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:51 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4114643479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:31:52.109 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:51 smithi053 ceph-mon[36496]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:52.110 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:51 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4114643479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:31:52.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:51 smithi045 ceph-mon[36409]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:52.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:51 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4114643479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:31:53.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.668+0000 7f12d1905700 1 -- 172.21.15.53:0/1181452784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12cc103b80 msgr2=0x7f12cc10a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:53.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.668+0000 7f12d1905700 1 --2- 172.21.15.53:0/1181452784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12cc103b80 0x7f12cc10a430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f12c400b0a0 tx=0x7f12c400b3b0 comp rx=0 tx=0).stop 2024-05-21T15:31:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.668+0000 7f12d1905700 1 -- 172.21.15.53:0/1181452784 shutdown_connections 2024-05-21T15:31:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.668+0000 7f12d1905700 1 --2- 172.21.15.53:0/1181452784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12cc103b80 0x7f12cc10a430 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.668+0000 7f12d1905700 1 --2- 172.21.15.53:0/1181452784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12cc1020b0 0x7f12cc1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.668+0000 7f12d1905700 1 --2- 172.21.15.53:0/1181452784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc1031e0 0x7f12cc1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.668+0000 7f12d1905700 1 -- 172.21.15.53:0/1181452784 >> 172.21.15.53:0/1181452784 conn(0x7f12cc0fd6f0 msgr2=0x7f12cc0ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.668+0000 7f12d1905700 1 -- 172.21.15.53:0/1181452784 shutdown_connections 2024-05-21T15:31:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.668+0000 7f12d1905700 1 -- 172.21.15.53:0/1181452784 wait complete. 2024-05-21T15:31:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.669+0000 7f12d1905700 1 Processor -- start 2024-05-21T15:31:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.669+0000 7f12d1905700 1 -- start start 2024-05-21T15:31:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.669+0000 7f12d1905700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12cc1020b0 0x7f12cc199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.669+0000 7f12d1905700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc1031e0 0x7f12cc199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.669+0000 7f12d1905700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12cc103b80 0x7f12cc1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.669+0000 7f12d1905700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc10dc70 con 0x7f12cc1020b0 2024-05-21T15:31:53.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.669+0000 7f12d1905700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc10daf0 con 0x7f12cc1031e0 2024-05-21T15:31:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.669+0000 7f12d1905700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc10ddf0 con 0x7f12cc103b80 2024-05-21T15:31:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12cb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12cc103b80 0x7f12cc1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12caffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12cc1020b0 0x7f12cc199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12cb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12cc103b80 0x7f12cc1a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46544/0 (socket says 172.21.15.53:46544) 2024-05-21T15:31:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12cb7fe700 1 -- 172.21.15.53:0/2578871141 learned_addr learned my addr 172.21.15.53:0/2578871141 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-21T15:31:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12ca7fc700 1 --2- 172.21.15.53:0/2578871141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc1031e0 0x7f12cc199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12cb7fe700 1 -- 172.21.15.53:0/2578871141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc1031e0 msgr2=0x7f12cc199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12cb7fe700 1 --2- 172.21.15.53:0/2578871141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc1031e0 0x7f12cc199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12cb7fe700 1 -- 172.21.15.53:0/2578871141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12cc1020b0 msgr2=0x7f12cc199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:53.673 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12cb7fe700 1 --2- 172.21.15.53:0/2578871141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12cc1020b0 0x7f12cc199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:53.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12cb7fe700 1 -- 172.21.15.53:0/2578871141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc1a12f0 con 0x7f12cc103b80 2024-05-21T15:31:53.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12cb7fe700 1 --2- 172.21.15.53:0/2578871141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12cc103b80 0x7f12cc1a0d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f12c40062a0 tx=0x7f12c4004e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:53.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12d0903700 1 -- 172.21.15.53:0/2578871141 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4021070 con 0x7f12cc103b80 2024-05-21T15:31:53.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12d0903700 1 -- 172.21.15.53:0/2578871141 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12c4004020 con 0x7f12cc103b80 2024-05-21T15:31:53.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12d1905700 1 -- 172.21.15.53:0/2578871141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12cc1a1580 con 0x7f12cc103b80 2024-05-21T15:31:53.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12d0903700 1 -- 172.21.15.53:0/2578871141 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c401d6b0 con 0x7f12cc103b80 2024-05-21T15:31:53.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.670+0000 7f12d1905700 1 -- 172.21.15.53:0/2578871141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12cc1a1a30 con 0x7f12cc103b80 2024-05-21T15:31:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.671+0000 7f12d1905700 1 -- 172.21.15.53:0/2578871141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12cc050170 con 0x7f12cc103b80 2024-05-21T15:31:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.673+0000 7f12d0903700 1 -- 172.21.15.53:0/2578871141 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f12c4004540 con 0x7f12cc103b80 2024-05-21T15:31:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.673+0000 7f12d0903700 1 --2- 172.21.15.53:0/2578871141 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12b40a2c20 0x7f12b40a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.674+0000 7f12d0903700 1 -- 172.21.15.53:0/2578871141 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f12c400f020 con 0x7f12cc103b80 2024-05-21T15:31:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.674+0000 7f12caffd700 1 --2- 172.21.15.53:0/2578871141 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12b40a2c20 0x7f12b40a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:53.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.674+0000 7f12caffd700 1 --2- 172.21.15.53:0/2578871141 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12b40a2c20 0x7f12b40a5130 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f12cc1030e0 tx=0x7f12bc00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:53.678 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.678+0000 7f12d0903700 1 -- 172.21.15.53:0/2578871141 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f12c402a020 con 0x7f12cc103b80 2024-05-21T15:31:53.896 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.895+0000 7f12d1905700 1 -- 172.21.15.53:0/2578871141 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l", "target": ["mon-mgr", ""]}) -- 0x7f12cc0ff770 con 0x7f12b40a2c20 2024-05-21T15:31:53.905 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.904+0000 7f12d0903700 1 -- 172.21.15.53:0/2578871141 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f12cc0ff770 con 0x7f12b40a2c20 2024-05-21T15:31:53.905 INFO:teuthology.orchestra.run.smithi053.stdout:Scheduled mds.all update... 2024-05-21T15:31:53.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.907+0000 7f12d1905700 1 -- 172.21.15.53:0/2578871141 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12b40a2c20 msgr2=0x7f12b40a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:53.907 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.907+0000 7f12d1905700 1 --2- 172.21.15.53:0/2578871141 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12b40a2c20 0x7f12b40a5130 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f12cc1030e0 tx=0x7f12bc00a3b0 comp rx=0 tx=0).stop 2024-05-21T15:31:53.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.907+0000 7f12d1905700 1 -- 172.21.15.53:0/2578871141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12cc103b80 msgr2=0x7f12cc1a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:53.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.907+0000 7f12d1905700 1 --2- 172.21.15.53:0/2578871141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12cc103b80 0x7f12cc1a0d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f12c40062a0 tx=0x7f12c4004e40 comp rx=0 tx=0).stop 2024-05-21T15:31:53.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.907+0000 7f12d1905700 1 -- 172.21.15.53:0/2578871141 shutdown_connections 2024-05-21T15:31:53.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.907+0000 7f12d1905700 1 --2- 172.21.15.53:0/2578871141 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12b40a2c20 0x7f12b40a5130 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:53.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.907+0000 7f12d1905700 1 --2- 172.21.15.53:0/2578871141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12cc103b80 0x7f12cc1a0d00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:53.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.907+0000 7f12d1905700 1 --2- 172.21.15.53:0/2578871141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12cc1020b0 0x7f12cc199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:53.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.907+0000 7f12d1905700 1 --2- 172.21.15.53:0/2578871141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc1031e0 0x7f12cc199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:53.908 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.907+0000 7f12d1905700 1 -- 172.21.15.53:0/2578871141 >> 172.21.15.53:0/2578871141 conn(0x7f12cc0fd6f0 msgr2=0x7f12cc0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:53.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.907+0000 7f12d1905700 1 -- 172.21.15.53:0/2578871141 shutdown_connections 2024-05-21T15:31:53.909 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-21T15:31:53.907+0000 7f12d1905700 1 -- 172.21.15.53:0/2578871141 wait complete. 2024-05-21T15:31:53.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:53 smithi053 ceph-mon[36496]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:54.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:53 smithi045 ceph-mon[36409]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:54.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:53 smithi038 ceph-mon[31995]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:54.617 DEBUG:teuthology.orchestra.run.smithi038:mds.a> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.a.service 2024-05-21T15:31:54.620 DEBUG:teuthology.orchestra.run.smithi038:mds.d> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.d.service 2024-05-21T15:31:54.663 DEBUG:teuthology.orchestra.run.smithi038:mds.g> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.g.service 2024-05-21T15:31:54.706 DEBUG:teuthology.orchestra.run.smithi038:mds.j> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.j.service 2024-05-21T15:31:54.749 DEBUG:teuthology.orchestra.run.smithi045:mds.b> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.b.service 2024-05-21T15:31:54.752 DEBUG:teuthology.orchestra.run.smithi045:mds.e> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.e.service 2024-05-21T15:31:54.797 DEBUG:teuthology.orchestra.run.smithi045:mds.h> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.h.service 2024-05-21T15:31:54.840 DEBUG:teuthology.orchestra.run.smithi045:mds.k> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.k.service 2024-05-21T15:31:55.151 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:54 smithi053 ceph-mon[36496]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:31:55.152 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:54 smithi053 ceph-mon[36496]: Saving service mds.all spec with placement smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l;count:12 2024-05-21T15:31:55.152 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:55.152 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:55.152 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:55.152 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:55.152 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:55.152 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:31:55.152 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:31:55.152 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:55.152 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:54 smithi053 ceph-mon[36496]: Deploying daemon mds.l on smithi053 2024-05-21T15:31:55.167 DEBUG:teuthology.orchestra.run.smithi053:mds.c> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.c.service 2024-05-21T15:31:55.170 DEBUG:teuthology.orchestra.run.smithi053:mds.f> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.f.service 2024-05-21T15:31:55.190 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:54 smithi045 ceph-mon[36409]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:31:55.190 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:54 smithi045 ceph-mon[36409]: Saving service mds.all spec with placement smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l;count:12 2024-05-21T15:31:55.190 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:55.191 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:55.191 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:55.191 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:55.191 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:55.191 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:31:55.191 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:31:55.191 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:55.191 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:54 smithi045 ceph-mon[36409]: Deploying daemon mds.l on smithi053 2024-05-21T15:31:55.213 DEBUG:teuthology.orchestra.run.smithi053:mds.i> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.i.service 2024-05-21T15:31:55.256 DEBUG:teuthology.orchestra.run.smithi053:mds.l> sudo journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.l.service 2024-05-21T15:31:55.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:54 smithi038 ceph-mon[31995]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:31:55.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:54 smithi038 ceph-mon[31995]: Saving service mds.all spec with placement smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l;count:12 2024-05-21T15:31:55.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:55.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:31:55.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:55.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:31:55.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:55.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:31:55.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:31:55.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:55.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:54 smithi038 ceph-mon[31995]: Deploying daemon mds.l on smithi053 2024-05-21T15:31:55.299 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-21T15:31:55.300 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:31:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.436+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4068120164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfac100e20 msgr2=0x7fdfac105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.436+0000 7fdfb22a2640 1 --2- 172.21.15.38:0/4068120164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfac100e20 0x7fdfac105ae0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009900 tx=0x7fdf9c02a410 comp rx=0 tx=0).stop 2024-05-21T15:31:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.436+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4068120164 shutdown_connections 2024-05-21T15:31:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.436+0000 7fdfb22a2640 1 --2- 172.21.15.38:0/4068120164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac108740 0x7fdfac10ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.436+0000 7fdfb22a2640 1 --2- 172.21.15.38:0/4068120164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfac101320 0x7fdfac1080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.436+0000 7fdfb22a2640 1 --2- 172.21.15.38:0/4068120164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfac100e20 0x7fdfac105ae0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.436+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4068120164 >> 172.21.15.38:0/4068120164 conn(0x7fdfac0fc330 msgr2=0x7fdfac0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:55.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.436+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4068120164 shutdown_connections 2024-05-21T15:31:55.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.436+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4068120164 wait complete. 2024-05-21T15:31:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.437+0000 7fdfb22a2640 1 Processor -- start 2024-05-21T15:31:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.437+0000 7fdfb22a2640 1 -- start start 2024-05-21T15:31:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.437+0000 7fdfb22a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfac100e20 0x7fdfac1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.437+0000 7fdfb22a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac101320 0x7fdfac1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.437+0000 7fdfb22a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfac108740 0x7fdfac1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.437+0000 7fdfb22a2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac10e720 con 0x7fdfac108740 2024-05-21T15:31:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.437+0000 7fdfb22a2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac10e5a0 con 0x7fdfac101320 2024-05-21T15:31:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.437+0000 7fdfb22a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac10e8a0 con 0x7fdfac100e20 2024-05-21T15:31:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfb083f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfac108740 0x7fdfac1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfabfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfac100e20 0x7fdfac1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfabfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfac100e20 0x7fdfac1991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:60244/0 (socket says 172.21.15.38:60244) 2024-05-21T15:31:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfabfff640 1 -- 172.21.15.38:0/4253124536 learned_addr learned my addr 172.21.15.38:0/4253124536 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:31:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfb083f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfac108740 0x7fdfac1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46426/0 (socket says 172.21.15.38:46426) 2024-05-21T15:31:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfab7fe640 1 --2- 172.21.15.38:0/4253124536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac101320 0x7fdfac1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfabfff640 1 -- 172.21.15.38:0/4253124536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac101320 msgr2=0x7fdfac1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfabfff640 1 --2- 172.21.15.38:0/4253124536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac101320 0x7fdfac1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfabfff640 1 -- 172.21.15.38:0/4253124536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfac108740 msgr2=0x7fdfac1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfabfff640 1 --2- 172.21.15.38:0/4253124536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfac108740 0x7fdfac1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfabfff640 1 -- 172.21.15.38:0/4253124536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfac1a0ce0 con 0x7fdfac100e20 2024-05-21T15:31:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfabfff640 1 --2- 172.21.15.38:0/4253124536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfac100e20 0x7fdfac1991a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c031440 tx=0x7fdf9c032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfa97fa640 1 -- 172.21.15.38:0/4253124536 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c03f030 con 0x7fdfac100e20 2024-05-21T15:31:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4253124536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfac1a0f70 con 0x7fdfac100e20 2024-05-21T15:31:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfa97fa640 1 -- 172.21.15.38:0/4253124536 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf9c003ce0 con 0x7fdfac100e20 2024-05-21T15:31:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.438+0000 7fdfa97fa640 1 -- 172.21.15.38:0/4253124536 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c035070 con 0x7fdfac100e20 2024-05-21T15:31:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.439+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4253124536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfac1a1380 con 0x7fdfac100e20 2024-05-21T15:31:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.439+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4253124536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfac0fde50 con 0x7fdfac100e20 2024-05-21T15:31:55.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.441+0000 7fdfa97fa640 1 -- 172.21.15.38:0/4253124536 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdf9c042030 con 0x7fdfac100e20 2024-05-21T15:31:55.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.442+0000 7fdfa97fa640 1 --2- 172.21.15.38:0/4253124536 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdf940a2c30 0x7fdf940a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:55.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.442+0000 7fdfab7fe640 1 --2- 172.21.15.38:0/4253124536 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdf940a2c30 0x7fdf940a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:55.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.442+0000 7fdfa97fa640 1 -- 172.21.15.38:0/4253124536 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdf9c0ef080 con 0x7fdfac100e20 2024-05-21T15:31:55.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.442+0000 7fdfab7fe640 1 --2- 172.21.15.38:0/4253124536 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdf940a2c30 0x7fdf940a5140 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdfac19a240 tx=0x7fdfa0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:55.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.445+0000 7fdfa97fa640 1 -- 172.21.15.38:0/4253124536 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdf9c0bc480 con 0x7fdfac100e20 2024-05-21T15:31:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.631+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4253124536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfac02d4f0 con 0x7fdfac100e20 2024-05-21T15:31:55.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.632+0000 7fdfa97fa640 1 -- 172.21.15.38:0/4253124536 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18349 (secure 0 0 0) 0x7fdf9c0bbd80 con 0x7fdfac100e20 2024-05-21T15:31:55.635 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:31:55.636 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":82,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:31:45.813596+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:31:55.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.634+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4253124536 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdf940a2c30 msgr2=0x7fdf940a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:55.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.634+0000 7fdfb22a2640 1 --2- 172.21.15.38:0/4253124536 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdf940a2c30 0x7fdf940a5140 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdfac19a240 tx=0x7fdfa0005f90 comp rx=0 tx=0).stop 2024-05-21T15:31:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.634+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4253124536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfac100e20 msgr2=0x7fdfac1991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.634+0000 7fdfb22a2640 1 --2- 172.21.15.38:0/4253124536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfac100e20 0x7fdfac1991a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c031440 tx=0x7fdf9c032ce0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.634+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4253124536 shutdown_connections 2024-05-21T15:31:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.634+0000 7fdfb22a2640 1 --2- 172.21.15.38:0/4253124536 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdf940a2c30 0x7fdf940a5140 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.634+0000 7fdfb22a2640 1 --2- 172.21.15.38:0/4253124536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfac108740 0x7fdfac1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.634+0000 7fdfb22a2640 1 --2- 172.21.15.38:0/4253124536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfac101320 0x7fdfac1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.634+0000 7fdfb22a2640 1 --2- 172.21.15.38:0/4253124536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfac100e20 0x7fdfac1991a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.634+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4253124536 >> 172.21.15.38:0/4253124536 conn(0x7fdfac0fc330 msgr2=0x7fdfac1044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.634+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4253124536 shutdown_connections 2024-05-21T15:31:55.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.634+0000 7fdfb22a2640 1 -- 172.21.15.38:0/4253124536 wait complete. 2024-05-21T15:31:55.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:27:50.659367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:31:55.646 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:31:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.767+0000 7f512a185640 1 -- 172.21.15.38:0/3546349383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5124102ab0 msgr2=0x7f51241037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.767+0000 7f512a185640 1 --2- 172.21.15.38:0/3546349383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5124102ab0 0x7f51241037b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5114009860 tx=0x7f511402a340 comp rx=0 tx=0).stop 2024-05-21T15:31:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.767+0000 7f512a185640 1 -- 172.21.15.38:0/3546349383 shutdown_connections 2024-05-21T15:31:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.767+0000 7f512a185640 1 --2- 172.21.15.38:0/3546349383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51241089f0 0x7f512410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.767+0000 7f512a185640 1 --2- 172.21.15.38:0/3546349383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5124103cf0 0x7f5124104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.767+0000 7f512a185640 1 --2- 172.21.15.38:0/3546349383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5124102ab0 0x7f51241037b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.767+0000 7f512a185640 1 -- 172.21.15.38:0/3546349383 >> 172.21.15.38:0/3546349383 conn(0x7f51240fdca0 msgr2=0x7f5124100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.767+0000 7f512a185640 1 -- 172.21.15.38:0/3546349383 shutdown_connections 2024-05-21T15:31:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.767+0000 7f512a185640 1 -- 172.21.15.38:0/3546349383 wait complete. 2024-05-21T15:31:55.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.768+0000 7f512a185640 1 Processor -- start 2024-05-21T15:31:55.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.768+0000 7f512a185640 1 -- start start 2024-05-21T15:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.768+0000 7f512a185640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5124102ab0 0x7f5124199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.768+0000 7f512a185640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5124103cf0 0x7f51241997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.768+0000 7f512a185640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51241089f0 0x7f51241a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.768+0000 7f512a185640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f512410e6e0 con 0x7f5124102ab0 2024-05-21T15:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.768+0000 7f512a185640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f512410e560 con 0x7f51241089f0 2024-05-21T15:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.768+0000 7f512a185640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f512410e860 con 0x7f5124103cf0 2024-05-21T15:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.768+0000 7f51237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5124102ab0 0x7f5124199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.768+0000 7f5122ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5124103cf0 0x7f51241997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.768+0000 7f5122ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5124103cf0 0x7f51241997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:60252/0 (socket says 172.21.15.38:60252) 2024-05-21T15:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.768+0000 7f5122ffd640 1 -- 172.21.15.38:0/2438362474 learned_addr learned my addr 172.21.15.38:0/2438362474 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.768+0000 7f51237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5124102ab0 0x7f5124199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46452/0 (socket says 172.21.15.38:46452) 2024-05-21T15:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.769+0000 7f5123fff640 1 --2- 172.21.15.38:0/2438362474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51241089f0 0x7f51241a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.769+0000 7f5122ffd640 1 -- 172.21.15.38:0/2438362474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51241089f0 msgr2=0x7f51241a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.769+0000 7f5122ffd640 1 --2- 172.21.15.38:0/2438362474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51241089f0 0x7f51241a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.769+0000 7f5122ffd640 1 -- 172.21.15.38:0/2438362474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5124102ab0 msgr2=0x7f5124199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.769+0000 7f5122ffd640 1 --2- 172.21.15.38:0/2438362474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5124102ab0 0x7f5124199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.769+0000 7f5122ffd640 1 -- 172.21.15.38:0/2438362474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51241a0dc0 con 0x7f5124103cf0 2024-05-21T15:31:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.769+0000 7f5122ffd640 1 --2- 172.21.15.38:0/2438362474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5124103cf0 0x7f51241997c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f511800e9c0 tx=0x7f511800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.769+0000 7f5120ff9640 1 -- 172.21.15.38:0/2438362474 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511800cde0 con 0x7f5124103cf0 2024-05-21T15:31:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.769+0000 7f5120ff9640 1 -- 172.21.15.38:0/2438362474 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51180035f0 con 0x7f5124103cf0 2024-05-21T15:31:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.769+0000 7f512a185640 1 -- 172.21.15.38:0/2438362474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51241a10b0 con 0x7f5124103cf0 2024-05-21T15:31:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.769+0000 7f5120ff9640 1 -- 172.21.15.38:0/2438362474 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118010690 con 0x7f5124103cf0 2024-05-21T15:31:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.769+0000 7f512a185640 1 -- 172.21.15.38:0/2438362474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51241a1650 con 0x7f5124103cf0 2024-05-21T15:31:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.771+0000 7f512a185640 1 -- 172.21.15.38:0/2438362474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5124107800 con 0x7f5124103cf0 2024-05-21T15:31:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.772+0000 7f5120ff9640 1 -- 172.21.15.38:0/2438362474 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5118010830 con 0x7f5124103cf0 2024-05-21T15:31:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.772+0000 7f5120ff9640 1 --2- 172.21.15.38:0/2438362474 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f510c0a2c10 0x7f510c0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.773+0000 7f5120ff9640 1 -- 172.21.15.38:0/2438362474 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f51180cee80 con 0x7f5124103cf0 2024-05-21T15:31:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.773+0000 7f51237fe640 1 --2- 172.21.15.38:0/2438362474 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f510c0a2c10 0x7f510c0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:55.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.773+0000 7f51237fe640 1 --2- 172.21.15.38:0/2438362474 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f510c0a2c10 0x7f510c0a5120 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5114009830 tx=0x7f5114030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:55.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.777+0000 7f5120ff9640 1 -- 172.21.15.38:0/2438362474 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f511809c260 con 0x7f5124103cf0 2024-05-21T15:31:55.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.964+0000 7f512a185640 1 -- 172.21.15.38:0/2438362474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5124062750 con 0x7f5124103cf0 2024-05-21T15:31:55.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.965+0000 7f5120ff9640 1 -- 172.21.15.38:0/2438362474 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f511809bb60 con 0x7f5124103cf0 2024-05-21T15:31:55.966 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-05-21T15:31:55.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.967+0000 7f512a185640 1 -- 172.21.15.38:0/2438362474 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f510c0a2c10 msgr2=0x7f510c0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:55.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.967+0000 7f512a185640 1 --2- 172.21.15.38:0/2438362474 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f510c0a2c10 0x7f510c0a5120 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5114009830 tx=0x7f5114030550 comp rx=0 tx=0).stop 2024-05-21T15:31:55.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.967+0000 7f512a185640 1 -- 172.21.15.38:0/2438362474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5124103cf0 msgr2=0x7f51241997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:55.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.967+0000 7f512a185640 1 --2- 172.21.15.38:0/2438362474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5124103cf0 0x7f51241997c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f511800e9c0 tx=0x7f511800ee80 comp rx=0 tx=0).stop 2024-05-21T15:31:55.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.967+0000 7f512a185640 1 -- 172.21.15.38:0/2438362474 shutdown_connections 2024-05-21T15:31:55.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.967+0000 7f512a185640 1 --2- 172.21.15.38:0/2438362474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51241089f0 0x7f51241a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.967+0000 7f512a185640 1 --2- 172.21.15.38:0/2438362474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5124103cf0 0x7f51241997c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.967+0000 7f512a185640 1 --2- 172.21.15.38:0/2438362474 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f510c0a2c10 0x7f510c0a5120 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.967+0000 7f512a185640 1 --2- 172.21.15.38:0/2438362474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5124102ab0 0x7f5124199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:55.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.967+0000 7f512a185640 1 -- 172.21.15.38:0/2438362474 >> 172.21.15.38:0/2438362474 conn(0x7f51240fdca0 msgr2=0x7f51241049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:55.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.967+0000 7f512a185640 1 -- 172.21.15.38:0/2438362474 shutdown_connections 2024-05-21T15:31:55.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:55.967+0000 7f512a185640 1 -- 172.21.15.38:0/2438362474 wait complete. 2024-05-21T15:31:55.979 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:31:56.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.093+0000 7f618441b640 1 -- 172.21.15.38:0/643753750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f617c105f00 msgr2=0x7f617c108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:56.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.093+0000 7f618441b640 1 --2- 172.21.15.38:0/643753750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f617c105f00 0x7f617c108320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f616c009900 tx=0x7f616c02a410 comp rx=0 tx=0).stop 2024-05-21T15:31:56.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.093+0000 7f618441b640 1 -- 172.21.15.38:0/643753750 shutdown_connections 2024-05-21T15:31:56.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.093+0000 7f618441b640 1 --2- 172.21.15.38:0/643753750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f617c108860 0x7f617c10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.093+0000 7f618441b640 1 --2- 172.21.15.38:0/643753750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f617c105f00 0x7f617c108320 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.093+0000 7f618441b640 1 --2- 172.21.15.38:0/643753750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c1035a0 0x7f617c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.093+0000 7f618441b640 1 -- 172.21.15.38:0/643753750 >> 172.21.15.38:0/643753750 conn(0x7f617c0fcba0 msgr2=0x7f617c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.093+0000 7f618441b640 1 -- 172.21.15.38:0/643753750 shutdown_connections 2024-05-21T15:31:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.093+0000 7f618441b640 1 -- 172.21.15.38:0/643753750 wait complete. 2024-05-21T15:31:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.094+0000 7f618441b640 1 Processor -- start 2024-05-21T15:31:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.094+0000 7f618441b640 1 -- start start 2024-05-21T15:31:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.094+0000 7f618441b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f617c1035a0 0x7f617c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.094+0000 7f618441b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c105f00 0x7f617c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.094+0000 7f618441b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f617c108860 0x7f617c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.094+0000 7f618441b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f617c10e730 con 0x7f617c108860 2024-05-21T15:31:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.094+0000 7f618441b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f617c10e5b0 con 0x7f617c105f00 2024-05-21T15:31:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.094+0000 7f618441b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f617c10e8b0 con 0x7f617c1035a0 2024-05-21T15:31:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.094+0000 7f61829b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f617c108860 0x7f617c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.094+0000 7f61829b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f617c108860 0x7f617c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46456/0 (socket says 172.21.15.38:46456) 2024-05-21T15:31:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.094+0000 7f61829b8640 1 -- 172.21.15.38:0/4090922551 learned_addr learned my addr 172.21.15.38:0/4090922551 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:31:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.094+0000 7f61821b7640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f617c1035a0 0x7f617c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.094+0000 7f61819b6640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c105f00 0x7f617c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.095+0000 7f61829b8640 1 -- 172.21.15.38:0/4090922551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f617c1035a0 msgr2=0x7f617c1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.095+0000 7f61829b8640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f617c1035a0 0x7f617c1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:56.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.095+0000 7f61829b8640 1 -- 172.21.15.38:0/4090922551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c105f00 msgr2=0x7f617c199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:56.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.095+0000 7f61829b8640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c105f00 0x7f617c199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:56.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.095+0000 7f61829b8640 1 -- 172.21.15.38:0/4090922551 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617c1a0d20 con 0x7f617c108860 2024-05-21T15:31:56.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.095+0000 7f61821b7640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f617c1035a0 0x7f617c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:31:56.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.095+0000 7f61829b8640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f617c108860 0x7f617c1a06b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f617400eb40 tx=0x7f617400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:56.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.095+0000 7f61737fe640 1 -- 172.21.15.38:0/4090922551 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174004b20 con 0x7f617c108860 2024-05-21T15:31:56.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.095+0000 7f61737fe640 1 -- 172.21.15.38:0/4090922551 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6174004cc0 con 0x7f617c108860 2024-05-21T15:31:56.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.095+0000 7f618441b640 1 -- 172.21.15.38:0/4090922551 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617c1a0fb0 con 0x7f617c108860 2024-05-21T15:31:56.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.095+0000 7f61737fe640 1 -- 172.21.15.38:0/4090922551 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174010850 con 0x7f617c108860 2024-05-21T15:31:56.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.095+0000 7f618441b640 1 -- 172.21.15.38:0/4090922551 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f617c1a14c0 con 0x7f617c108860 2024-05-21T15:31:56.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.096+0000 7f618441b640 1 -- 172.21.15.38:0/4090922551 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f617c0fe6c0 con 0x7f617c108860 2024-05-21T15:31:56.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.097+0000 7f61737fe640 1 -- 172.21.15.38:0/4090922551 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6174005000 con 0x7f617c108860 2024-05-21T15:31:56.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.098+0000 7f61737fe640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f61680a2cb0 0x7f61680a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:56.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.098+0000 7f61821b7640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f61680a2cb0 0x7f61680a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:56.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.098+0000 7f61737fe640 1 -- 172.21.15.38:0/4090922551 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f61740cf220 con 0x7f617c108860 2024-05-21T15:31:56.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.099+0000 7f61821b7640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f61680a2cb0 0x7f61680a51c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f617c0699d0 tx=0x7f6178005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.102+0000 7f61737fe640 1 -- 172.21.15.38:0/4090922551 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f617409c620 con 0x7f617c108860 2024-05-21T15:31:56.110 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:55 smithi053 ceph-mon[36496]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:56.111 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:55 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4253124536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:31:56.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:55 smithi045 ceph-mon[36409]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:56.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:55 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4253124536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:31:56.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:55 smithi038 ceph-mon[31995]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:56.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:55 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4253124536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:31:56.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.324+0000 7f618441b640 1 -- 172.21.15.38:0/4090922551 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f617c02d2a0 con 0x7f617c108860 2024-05-21T15:31:56.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.325+0000 7f61737fe640 1 -- 172.21.15.38:0/4090922551 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f617409bf20 con 0x7f617c108860 2024-05-21T15:31:56.325 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:31:56.326 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-21T15:31:56.326 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 1 2024-05-21T15:31:56.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.327+0000 7f618441b640 1 -- 172.21.15.38:0/4090922551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f61680a2cb0 msgr2=0x7f61680a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:56.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.327+0000 7f618441b640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f61680a2cb0 0x7f61680a51c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f617c0699d0 tx=0x7f6178005f90 comp rx=0 tx=0).stop 2024-05-21T15:31:56.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.327+0000 7f618441b640 1 -- 172.21.15.38:0/4090922551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f617c108860 msgr2=0x7f617c1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:56.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.327+0000 7f618441b640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f617c108860 0x7f617c1a06b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f617400eb40 tx=0x7f617400c2d0 comp rx=0 tx=0).stop 2024-05-21T15:31:56.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.327+0000 7f618441b640 1 -- 172.21.15.38:0/4090922551 shutdown_connections 2024-05-21T15:31:56.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.327+0000 7f618441b640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f61680a2cb0 0x7f61680a51c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:56.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.327+0000 7f618441b640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f617c108860 0x7f617c1a06b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:56.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.327+0000 7f618441b640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c105f00 0x7f617c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:56.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.327+0000 7f618441b640 1 --2- 172.21.15.38:0/4090922551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f617c1035a0 0x7f617c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:56.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.327+0000 7f618441b640 1 -- 172.21.15.38:0/4090922551 >> 172.21.15.38:0/4090922551 conn(0x7f617c0fcba0 msgr2=0x7f617c104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:56.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.327+0000 7f618441b640 1 -- 172.21.15.38:0/4090922551 shutdown_connections 2024-05-21T15:31:56.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:56.327+0000 7f618441b640 1 -- 172.21.15.38:0/4090922551 wait complete. 2024-05-21T15:31:56.456 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:31:56 smithi053 systemd[1]: Starting Ceph mds.l for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:31:56.765 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:31:56 smithi053 podman[65988]: 2024-05-21 15:31:56.416385002 +0000 UTC m=+0.017779809 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:31:56.766 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:31:56 smithi053 podman[65988]: 2024-05-21 15:31:56.517745895 +0000 UTC m=+0.119140682 container create ea5eb1669df59a23d73271dbae0febc4d2d363775ce71ecf32bdd0893998337a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-l, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-21T15:31:56.766 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:31:56 smithi053 podman[65988]: 2024-05-21 15:31:56.762922347 +0000 UTC m=+0.364317134 container init ea5eb1669df59a23d73271dbae0febc4d2d363775ce71ecf32bdd0893998337a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-21T15:31:56.766 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:31:56 smithi053 podman[65988]: 2024-05-21 15:31:56.765550482 +0000 UTC m=+0.366945282 container start ea5eb1669df59a23d73271dbae0febc4d2d363775ce71ecf32bdd0893998337a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-l, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:31:57.206 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:31:56 smithi053 ceph-mds[66007]: -- 172.21.15.53:0/3091069850 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ba088fba40 con 0x55ba08973600 2024-05-21T15:31:57.207 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:31:56 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-l[66003]: starting mds.l at 2024-05-21T15:31:57.207 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:31:56 smithi053 bash[65988]: ea5eb1669df59a23d73271dbae0febc4d2d363775ce71ecf32bdd0893998337a 2024-05-21T15:31:57.207 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:31:56 smithi053 systemd[1]: Started Ceph mds.l for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:31:57.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:56 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2438362474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:31:57.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:56 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4090922551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:31:57.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:56 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.53:6832/1390207425,v1:172.21.15.53:6833/1390207425] up:boot 2024-05-21T15:31:57.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:56 smithi053 ceph-mon[36496]: fsmap 1 up:standby 2024-05-21T15:31:57.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:31:57.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:57.208 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:57.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:56 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2438362474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:31:57.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:56 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4090922551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:31:57.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:56 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.53:6832/1390207425,v1:172.21.15.53:6833/1390207425] up:boot 2024-05-21T15:31:57.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:56 smithi045 ceph-mon[36409]: fsmap 1 up:standby 2024-05-21T15:31:57.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:31:57.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:57.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:57.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:56 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2438362474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:31:57.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:56 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4090922551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:31:57.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:56 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.53:6832/1390207425,v1:172.21.15.53:6833/1390207425] up:boot 2024-05-21T15:31:57.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:56 smithi038 ceph-mon[31995]: fsmap 1 up:standby 2024-05-21T15:31:57.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:31:57.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:57.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:58.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:57 smithi045 ceph-mon[36409]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:58.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:58.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:31:58.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:31:58.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:58.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:57 smithi045 ceph-mon[36409]: Deploying daemon mds.k on smithi045 2024-05-21T15:31:58.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:57 smithi038 ceph-mon[31995]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:58.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:58.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:31:58.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:31:58.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:58.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:57 smithi038 ceph-mon[31995]: Deploying daemon mds.k on smithi045 2024-05-21T15:31:58.340 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:31:58.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:57 smithi053 ceph-mon[36496]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:31:58.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:31:58.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:31:58.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:31:58.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:31:58.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:57 smithi053 ceph-mon[36496]: Deploying daemon mds.k on smithi045 2024-05-21T15:31:58.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.470+0000 7f882c184640 1 -- 172.21.15.38:0/3921136230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8824102bf0 msgr2=0x7f8824103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:58.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.470+0000 7f882c184640 1 --2- 172.21.15.38:0/3921136230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8824102bf0 0x7f8824103000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8814009900 tx=0x7f881402a410 comp rx=0 tx=0).stop 2024-05-21T15:31:58.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.470+0000 7f882c184640 1 -- 172.21.15.38:0/3921136230 shutdown_connections 2024-05-21T15:31:58.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.470+0000 7f882c184640 1 --2- 172.21.15.38:0/3921136230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88241078f0 0x7f8824109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:58.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.470+0000 7f882c184640 1 --2- 172.21.15.38:0/3921136230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8824102bf0 0x7f8824103000 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:58.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.470+0000 7f882c184640 1 --2- 172.21.15.38:0/3921136230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88241019b0 0x7f88241026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:58.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.470+0000 7f882c184640 1 -- 172.21.15.38:0/3921136230 >> 172.21.15.38:0/3921136230 conn(0x7f88240fcba0 msgr2=0x7f88240ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:58.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.470+0000 7f882c184640 1 -- 172.21.15.38:0/3921136230 shutdown_connections 2024-05-21T15:31:58.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.470+0000 7f882c184640 1 -- 172.21.15.38:0/3921136230 wait complete. 2024-05-21T15:31:58.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882c184640 1 Processor -- start 2024-05-21T15:31:58.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882c184640 1 -- start start 2024-05-21T15:31:58.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882c184640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88241019b0 0x7f8824069a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:58.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882c184640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8824102bf0 0x7f882419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:58.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882c184640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88241078f0 0x7f88241a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:58.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882c184640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f882410d650 con 0x7f8824102bf0 2024-05-21T15:31:58.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882c184640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f882410d4d0 con 0x7f88241019b0 2024-05-21T15:31:58.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882c184640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f882410d7d0 con 0x7f88241078f0 2024-05-21T15:31:58.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882971f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8824102bf0 0x7f882419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:58.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882971f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8824102bf0 0x7f882419df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46468/0 (socket says 172.21.15.38:46468) 2024-05-21T15:31:58.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882971f640 1 -- 172.21.15.38:0/1074883442 learned_addr learned my addr 172.21.15.38:0/1074883442 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:31:58.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f8829f20640 1 --2- 172.21.15.38:0/1074883442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88241019b0 0x7f8824069a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:58.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882a721640 1 --2- 172.21.15.38:0/1074883442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88241078f0 0x7f88241a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:58.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882971f640 1 -- 172.21.15.38:0/1074883442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88241078f0 msgr2=0x7f88241a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:58.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882971f640 1 --2- 172.21.15.38:0/1074883442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88241078f0 0x7f88241a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:58.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882971f640 1 -- 172.21.15.38:0/1074883442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88241019b0 msgr2=0x7f8824069a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:58.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882971f640 1 --2- 172.21.15.38:0/1074883442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88241019b0 0x7f8824069a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:58.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.471+0000 7f882971f640 1 -- 172.21.15.38:0/1074883442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88241a0ce0 con 0x7f8824102bf0 2024-05-21T15:31:58.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.472+0000 7f882971f640 1 --2- 172.21.15.38:0/1074883442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8824102bf0 0x7f882419df70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8814031440 tx=0x7f8814032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:58.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.472+0000 7f881affd640 1 -- 172.21.15.38:0/1074883442 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881403f030 con 0x7f8824102bf0 2024-05-21T15:31:58.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.472+0000 7f881affd640 1 -- 172.21.15.38:0/1074883442 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8814003ce0 con 0x7f8824102bf0 2024-05-21T15:31:58.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.472+0000 7f881affd640 1 -- 172.21.15.38:0/1074883442 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814035070 con 0x7f8824102bf0 2024-05-21T15:31:58.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.472+0000 7f882c184640 1 -- 172.21.15.38:0/1074883442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88241a0f70 con 0x7f8824102bf0 2024-05-21T15:31:58.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.472+0000 7f882c184640 1 -- 172.21.15.38:0/1074883442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88241a1320 con 0x7f8824102bf0 2024-05-21T15:31:58.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.473+0000 7f882c184640 1 -- 172.21.15.38:0/1074883442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8824105ff0 con 0x7f8824102bf0 2024-05-21T15:31:58.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.474+0000 7f881affd640 1 -- 172.21.15.38:0/1074883442 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8814042030 con 0x7f8824102bf0 2024-05-21T15:31:58.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.475+0000 7f881affd640 1 --2- 172.21.15.38:0/1074883442 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f88100a2c30 0x7f88100a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:31:58.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.475+0000 7f8829f20640 1 --2- 172.21.15.38:0/1074883442 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f88100a2c30 0x7f88100a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:31:58.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.475+0000 7f881affd640 1 -- 172.21.15.38:0/1074883442 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f88140ef530 con 0x7f8824102bf0 2024-05-21T15:31:58.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.476+0000 7f8829f20640 1 --2- 172.21.15.38:0/1074883442 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f88100a2c30 0x7f88100a5140 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f88241025b0 tx=0x7f8820005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:31:58.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.479+0000 7f881affd640 1 -- 172.21.15.38:0/1074883442 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f88140bc890 con 0x7f8824102bf0 2024-05-21T15:31:58.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.703+0000 7f882c184640 1 -- 172.21.15.38:0/1074883442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f882402d2a0 con 0x7f8824102bf0 2024-05-21T15:31:58.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.704+0000 7f881affd640 1 -- 172.21.15.38:0/1074883442 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f88140bc190 con 0x7f8824102bf0 2024-05-21T15:31:58.704 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:31:58.705 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-21T15:31:58.705 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 2 2024-05-21T15:31:58.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.706+0000 7f882c184640 1 -- 172.21.15.38:0/1074883442 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f88100a2c30 msgr2=0x7f88100a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:58.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.706+0000 7f882c184640 1 --2- 172.21.15.38:0/1074883442 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f88100a2c30 0x7f88100a5140 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f88241025b0 tx=0x7f8820005f90 comp rx=0 tx=0).stop 2024-05-21T15:31:58.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.706+0000 7f882c184640 1 -- 172.21.15.38:0/1074883442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8824102bf0 msgr2=0x7f882419df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:31:58.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.706+0000 7f882c184640 1 --2- 172.21.15.38:0/1074883442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8824102bf0 0x7f882419df70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8814031440 tx=0x7f8814032ce0 comp rx=0 tx=0).stop 2024-05-21T15:31:58.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.706+0000 7f882c184640 1 -- 172.21.15.38:0/1074883442 shutdown_connections 2024-05-21T15:31:58.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.706+0000 7f882c184640 1 --2- 172.21.15.38:0/1074883442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88241078f0 0x7f88241a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:58.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.706+0000 7f882c184640 1 --2- 172.21.15.38:0/1074883442 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f88100a2c30 0x7f88100a5140 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:58.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.706+0000 7f882c184640 1 --2- 172.21.15.38:0/1074883442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8824102bf0 0x7f882419df70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:58.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.706+0000 7f882c184640 1 --2- 172.21.15.38:0/1074883442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88241019b0 0x7f8824069a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:31:58.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.706+0000 7f882c184640 1 -- 172.21.15.38:0/1074883442 >> 172.21.15.38:0/1074883442 conn(0x7f88240fcba0 msgr2=0x7f88240fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:31:58.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.706+0000 7f882c184640 1 -- 172.21.15.38:0/1074883442 shutdown_connections 2024-05-21T15:31:58.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:31:58.706+0000 7f882c184640 1 -- 172.21.15.38:0/1074883442 wait complete. 2024-05-21T15:31:59.233 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:58 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1074883442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:31:59.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:58 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1074883442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:31:59.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:58 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1074883442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:31:59.492 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:31:59 smithi045 podman[65498]: 2024-05-21 15:31:59.355423276 +0000 UTC m=+0.018014435 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:31:59.493 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:31:59 smithi045 podman[65498]: 2024-05-21 15:31:59.492842673 +0000 UTC m=+0.155433874 container create cf789efac309d22cc12bc417bf204dfa7edcf129f7f4cf98d7c0c878460963dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:31:59.761 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:31:59 smithi045 podman[65498]: 2024-05-21 15:31:59.737032948 +0000 UTC m=+0.399624091 container init cf789efac309d22cc12bc417bf204dfa7edcf129f7f4cf98d7c0c878460963dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-k, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-21T15:31:59.761 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:31:59 smithi045 podman[65498]: 2024-05-21 15:31:59.739866379 +0000 UTC m=+0.402457531 container start cf789efac309d22cc12bc417bf204dfa7edcf129f7f4cf98d7c0c878460963dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-k, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-21T15:32:00.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:59 smithi045 ceph-mon[36409]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:00.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:59 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:00.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:59 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:00.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:59 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:00.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:59 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:00.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:59 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:00.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:31:59 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:00.262 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:31:59 smithi045 ceph-mds[65517]: -- 172.21.15.45:0/1074696574 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d78357a40 con 0x556d783cf180 2024-05-21T15:32:00.262 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:31:59 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-k[65513]: starting mds.k at 2024-05-21T15:32:00.263 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:31:59 smithi045 bash[65498]: cf789efac309d22cc12bc417bf204dfa7edcf129f7f4cf98d7c0c878460963dd 2024-05-21T15:32:00.263 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:31:59 smithi045 systemd[1]: Started Ceph mds.k for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:32:00.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:59 smithi038 ceph-mon[31995]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:00.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:59 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:00.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:59 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:00.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:59 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:00.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:59 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:00.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:59 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:00.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:31:59 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:00.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:59 smithi053 ceph-mon[36496]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:00.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:59 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:00.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:59 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:00.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:59 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:00.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:59 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:00.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:59 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:00.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:31:59 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:00.718 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:32:00.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.838+0000 7fdde6a5f640 1 -- 172.21.15.38:0/2095717712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde0103d10 msgr2=0x7fdde0104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:00.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.838+0000 7fdde6a5f640 1 --2- 172.21.15.38:0/2095717712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde0103d10 0x7fdde0104120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009900 tx=0x7fddd402a410 comp rx=0 tx=0).stop 2024-05-21T15:32:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.838+0000 7fdde6a5f640 1 -- 172.21.15.38:0/2095717712 shutdown_connections 2024-05-21T15:32:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.838+0000 7fdde6a5f640 1 --2- 172.21.15.38:0/2095717712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdde0108a10 0x7fdde010ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.838+0000 7fdde6a5f640 1 --2- 172.21.15.38:0/2095717712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde0103d10 0x7fdde0104120 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.838+0000 7fdde6a5f640 1 --2- 172.21.15.38:0/2095717712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde0102ad0 0x7fdde01037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.838+0000 7fdde6a5f640 1 -- 172.21.15.38:0/2095717712 >> 172.21.15.38:0/2095717712 conn(0x7fdde00fdce0 msgr2=0x7fdde0100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.838+0000 7fdde6a5f640 1 -- 172.21.15.38:0/2095717712 shutdown_connections 2024-05-21T15:32:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.838+0000 7fdde6a5f640 1 -- 172.21.15.38:0/2095717712 wait complete. 2024-05-21T15:32:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.839+0000 7fdde6a5f640 1 Processor -- start 2024-05-21T15:32:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.839+0000 7fdde6a5f640 1 -- start start 2024-05-21T15:32:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.839+0000 7fdde6a5f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdde0102ad0 0x7fdde0075800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.839+0000 7fdde6a5f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde0103d10 0x7fdde0073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.839+0000 7fdde6a5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde0108a10 0x7fdde00743e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.839+0000 7fdde6a5f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdde010e740 con 0x7fdde0108a10 2024-05-21T15:32:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.839+0000 7fdde6a5f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdde010e5c0 con 0x7fdde0102ad0 2024-05-21T15:32:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.839+0000 7fdde6a5f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdde010e8c0 con 0x7fdde0103d10 2024-05-21T15:32:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdde4ffc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde0108a10 0x7fdde00743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdde4ffc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde0108a10 0x7fdde00743e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46490/0 (socket says 172.21.15.38:46490) 2024-05-21T15:32:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdde4ffc640 1 -- 172.21.15.38:0/1490223326 learned_addr learned my addr 172.21.15.38:0/1490223326 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdddf7fe640 1 --2- 172.21.15.38:0/1490223326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde0103d10 0x7fdde0073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdddffff640 1 --2- 172.21.15.38:0/1490223326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdde0102ad0 0x7fdde0075800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdde4ffc640 1 -- 172.21.15.38:0/1490223326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde0103d10 msgr2=0x7fdde0073ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdde4ffc640 1 --2- 172.21.15.38:0/1490223326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde0103d10 0x7fdde0073ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdde4ffc640 1 -- 172.21.15.38:0/1490223326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdde0102ad0 msgr2=0x7fdde0075800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdde4ffc640 1 --2- 172.21.15.38:0/1490223326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdde0102ad0 0x7fdde0075800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdde4ffc640 1 -- 172.21.15.38:0/1490223326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde0074a80 con 0x7fdde0108a10 2024-05-21T15:32:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdde4ffc640 1 --2- 172.21.15.38:0/1490223326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde0108a10 0x7fdde00743e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fddd800cce0 tx=0x7fddd800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:00.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdddd7fa640 1 -- 172.21.15.38:0/1490223326 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd800f450 con 0x7fdde0108a10 2024-05-21T15:32:00.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdddd7fa640 1 -- 172.21.15.38:0/1490223326 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddd800fad0 con 0x7fdde0108a10 2024-05-21T15:32:00.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdde6a5f640 1 -- 172.21.15.38:0/1490223326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde01a3270 con 0x7fdde0108a10 2024-05-21T15:32:00.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdddd7fa640 1 -- 172.21.15.38:0/1490223326 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8018c10 con 0x7fdde0108a10 2024-05-21T15:32:00.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.840+0000 7fdde6a5f640 1 -- 172.21.15.38:0/1490223326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde01a3680 con 0x7fdde0108a10 2024-05-21T15:32:00.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.841+0000 7fdde6a5f640 1 -- 172.21.15.38:0/1490223326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde0107110 con 0x7fdde0108a10 2024-05-21T15:32:00.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.843+0000 7fdddd7fa640 1 -- 172.21.15.38:0/1490223326 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fddd800f5f0 con 0x7fdde0108a10 2024-05-21T15:32:00.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.843+0000 7fdddd7fa640 1 --2- 172.21.15.38:0/1490223326 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fddc80a2c30 0x7fddc80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:00.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.843+0000 7fdddffff640 1 --2- 172.21.15.38:0/1490223326 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fddc80a2c30 0x7fddc80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:00.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.844+0000 7fdddd7fa640 1 -- 172.21.15.38:0/1490223326 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fddd80cf2e0 con 0x7fdde0108a10 2024-05-21T15:32:00.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.844+0000 7fdddffff640 1 --2- 172.21.15.38:0/1490223326 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fddc80a2c30 0x7fddc80a5140 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fddd0005fd0 tx=0x7fddd000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:00.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:00.847+0000 7fdddd7fa640 1 -- 172.21.15.38:0/1490223326 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fddd8014030 con 0x7fdde0108a10 2024-05-21T15:32:01.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:01.075+0000 7fdde6a5f640 1 -- 172.21.15.38:0/1490223326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdde002d2a0 con 0x7fdde0108a10 2024-05-21T15:32:01.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:01.075+0000 7fdddd7fa640 1 -- 172.21.15.38:0/1490223326 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7fddd809c6d0 con 0x7fdde0108a10 2024-05-21T15:32:01.076 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:01.076 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-21T15:32:01.076 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 3 2024-05-21T15:32:01.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:01.077+0000 7fdde6a5f640 1 -- 172.21.15.38:0/1490223326 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fddc80a2c30 msgr2=0x7fddc80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:01.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:01.077+0000 7fdde6a5f640 1 --2- 172.21.15.38:0/1490223326 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fddc80a2c30 0x7fddc80a5140 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fddd0005fd0 tx=0x7fddd000b390 comp rx=0 tx=0).stop 2024-05-21T15:32:01.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:01.077+0000 7fdde6a5f640 1 -- 172.21.15.38:0/1490223326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde0108a10 msgr2=0x7fdde00743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:01.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:01.077+0000 7fdde6a5f640 1 --2- 172.21.15.38:0/1490223326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde0108a10 0x7fdde00743e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fddd800cce0 tx=0x7fddd800a620 comp rx=0 tx=0).stop 2024-05-21T15:32:01.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:01.077+0000 7fdde6a5f640 1 -- 172.21.15.38:0/1490223326 shutdown_connections 2024-05-21T15:32:01.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:01.077+0000 7fdde6a5f640 1 --2- 172.21.15.38:0/1490223326 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fddc80a2c30 0x7fddc80a5140 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:01.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:01.077+0000 7fdde6a5f640 1 --2- 172.21.15.38:0/1490223326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdde0108a10 0x7fdde00743e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:01.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:01.077+0000 7fdde6a5f640 1 --2- 172.21.15.38:0/1490223326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde0103d10 0x7fdde0073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:01.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:01.077+0000 7fdde6a5f640 1 --2- 172.21.15.38:0/1490223326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdde0102ad0 0x7fdde0075800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:01.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:01.077+0000 7fdde6a5f640 1 -- 172.21.15.38:0/1490223326 >> 172.21.15.38:0/1490223326 conn(0x7fdde00fdce0 msgr2=0x7fdde0100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:01.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:01.077+0000 7fdde6a5f640 1 -- 172.21.15.38:0/1490223326 shutdown_connections 2024-05-21T15:32:01.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:01.077+0000 7fdde6a5f640 1 -- 172.21.15.38:0/1490223326 wait complete. 2024-05-21T15:32:01.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:00 smithi045 ceph-mon[36409]: Deploying daemon mds.j on smithi038 2024-05-21T15:32:01.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:00 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.45:6832/2551213098,v1:172.21.15.45:6833/2551213098] up:boot 2024-05-21T15:32:01.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:00 smithi045 ceph-mon[36409]: fsmap 2 up:standby 2024-05-21T15:32:01.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:32:01.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:00 smithi038 ceph-mon[31995]: Deploying daemon mds.j on smithi038 2024-05-21T15:32:01.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:00 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.45:6832/2551213098,v1:172.21.15.45:6833/2551213098] up:boot 2024-05-21T15:32:01.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:00 smithi038 ceph-mon[31995]: fsmap 2 up:standby 2024-05-21T15:32:01.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:32:01.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:00 smithi053 ceph-mon[36496]: Deploying daemon mds.j on smithi038 2024-05-21T15:32:01.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:00 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.45:6832/2551213098,v1:172.21.15.45:6833/2551213098] up:boot 2024-05-21T15:32:01.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:00 smithi053 ceph-mon[36496]: fsmap 2 up:standby 2024-05-21T15:32:01.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:32:02.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:01 smithi038 ceph-mon[31995]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:02.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:01 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1490223326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:02.201 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:01 smithi045 ceph-mon[36409]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:02.201 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:01 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1490223326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:02.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:01 smithi053 ceph-mon[36496]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:02.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:01 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1490223326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:02.476 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:32:02 smithi038 systemd[1]: Starting Ceph mds.j for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:32:02.744 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:32:02 smithi038 podman[71536]: 2024-05-21 15:32:02.379450951 +0000 UTC m=+0.017671975 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:32:02.744 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:32:02 smithi038 podman[71536]: 2024-05-21 15:32:02.478791815 +0000 UTC m=+0.117012837 container create e998518fd25d28828fe2bfc5d64c192f6f6fd8e9762176e9b18688b39014dfaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-21T15:32:02.744 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:32:02 smithi038 podman[71536]: 2024-05-21 15:32:02.706668071 +0000 UTC m=+0.344889075 container init e998518fd25d28828fe2bfc5d64c192f6f6fd8e9762176e9b18688b39014dfaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, CEPH_POINT_RELEASE=) 2024-05-21T15:32:02.744 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:32:02 smithi038 podman[71536]: 2024-05-21 15:32:02.709971517 +0000 UTC m=+0.348192520 container start e998518fd25d28828fe2bfc5d64c192f6f6fd8e9762176e9b18688b39014dfaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-21T15:32:02.744 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:32:02 smithi038 ceph-mds[71555]: -- 172.21.15.38:0/1737710491 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5639b1b75a40 con 0x5639b1beda80 2024-05-21T15:32:03.025 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:32:02 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-j[71551]: starting mds.j at 2024-05-21T15:32:03.025 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:32:02 smithi038 bash[71536]: e998518fd25d28828fe2bfc5d64c192f6f6fd8e9762176e9b18688b39014dfaf 2024-05-21T15:32:03.025 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:32:02 smithi038 systemd[1]: Started Ceph mds.j for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:32:03.090 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:32:03.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.209+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2443935484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f0102bf0 msgr2=0x7f39f0103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:03.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.209+0000 7f39f5fd5640 1 --2- 172.21.15.38:0/2443935484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f0102bf0 0x7f39f0103000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009900 tx=0x7f39e402a410 comp rx=0 tx=0).stop 2024-05-21T15:32:03.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.209+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2443935484 shutdown_connections 2024-05-21T15:32:03.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.209+0000 7f39f5fd5640 1 --2- 172.21.15.38:0/2443935484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39f01078f0 0x7f39f0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:03.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.209+0000 7f39f5fd5640 1 --2- 172.21.15.38:0/2443935484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f0102bf0 0x7f39f0103000 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:03.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.209+0000 7f39f5fd5640 1 --2- 172.21.15.38:0/2443935484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39f01019b0 0x7f39f01026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:03.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.209+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2443935484 >> 172.21.15.38:0/2443935484 conn(0x7f39f00fcba0 msgr2=0x7f39f00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:03.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.209+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2443935484 shutdown_connections 2024-05-21T15:32:03.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.209+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2443935484 wait complete. 2024-05-21T15:32:03.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.210+0000 7f39f5fd5640 1 Processor -- start 2024-05-21T15:32:03.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.210+0000 7f39f5fd5640 1 -- start start 2024-05-21T15:32:03.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.210+0000 7f39f5fd5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39f01019b0 0x7f39f01991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:03.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.210+0000 7f39f5fd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f0102bf0 0x7f39f0199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:03.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39f5fd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39f01078f0 0x7f39f01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:03.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39f5fd5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f39f010e730 con 0x7f39f0102bf0 2024-05-21T15:32:03.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39f5fd5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f39f010e5b0 con 0x7f39f01078f0 2024-05-21T15:32:03.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39f5fd5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f39f010e8b0 con 0x7f39f01019b0 2024-05-21T15:32:03.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f0102bf0 0x7f39f0199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:03.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f0102bf0 0x7f39f0199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46538/0 (socket says 172.21.15.38:46538) 2024-05-21T15:32:03.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39eeffd640 1 -- 172.21.15.38:0/2696469066 learned_addr learned my addr 172.21.15.38:0/2696469066 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:03.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39ef7fe640 1 --2- 172.21.15.38:0/2696469066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39f01019b0 0x7f39f01991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:03.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39effff640 1 --2- 172.21.15.38:0/2696469066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39f01078f0 0x7f39f01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:03.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39eeffd640 1 -- 172.21.15.38:0/2696469066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39f01019b0 msgr2=0x7f39f01991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:03.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39eeffd640 1 --2- 172.21.15.38:0/2696469066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39f01019b0 0x7f39f01991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:03.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39eeffd640 1 -- 172.21.15.38:0/2696469066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39f01078f0 msgr2=0x7f39f01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:03.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39eeffd640 1 --2- 172.21.15.38:0/2696469066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39f01078f0 0x7f39f01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:03.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39eeffd640 1 -- 172.21.15.38:0/2696469066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f01a0d20 con 0x7f39f0102bf0 2024-05-21T15:32:03.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.211+0000 7f39eeffd640 1 --2- 172.21.15.38:0/2696469066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f0102bf0 0x7f39f0199720 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f39e4006010 tx=0x7f39e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:03.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.212+0000 7f39ecff9640 1 -- 172.21.15.38:0/2696469066 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e403f030 con 0x7f39f0102bf0 2024-05-21T15:32:03.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.212+0000 7f39ecff9640 1 -- 172.21.15.38:0/2696469066 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e4003ce0 con 0x7f39f0102bf0 2024-05-21T15:32:03.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.212+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2696469066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f01a0fb0 con 0x7f39f0102bf0 2024-05-21T15:32:03.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.212+0000 7f39ecff9640 1 -- 172.21.15.38:0/2696469066 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4035070 con 0x7f39f0102bf0 2024-05-21T15:32:03.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.212+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2696469066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f01a13c0 con 0x7f39f0102bf0 2024-05-21T15:32:03.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.213+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2696469066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f0106700 con 0x7f39f0102bf0 2024-05-21T15:32:03.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.215+0000 7f39ecff9640 1 -- 172.21.15.38:0/2696469066 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f39e4042030 con 0x7f39f0102bf0 2024-05-21T15:32:03.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.216+0000 7f39ecff9640 1 --2- 172.21.15.38:0/2696469066 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f39d80a2c30 0x7f39d80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:03.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.216+0000 7f39ecff9640 1 -- 172.21.15.38:0/2696469066 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f39e40ef580 con 0x7f39f0102bf0 2024-05-21T15:32:03.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.216+0000 7f39ef7fe640 1 --2- 172.21.15.38:0/2696469066 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f39d80a2c30 0x7f39d80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:03.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.216+0000 7f39ef7fe640 1 --2- 172.21.15.38:0/2696469066 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f39d80a2c30 0x7f39d80a5140 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f39f01025b0 tx=0x7f39e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:03.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.218+0000 7f39ecff9640 1 -- 172.21.15.38:0/2696469066 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f39e40bc330 con 0x7f39f0102bf0 2024-05-21T15:32:03.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.456+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2696469066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39f002d2a0 con 0x7f39f0102bf0 2024-05-21T15:32:03.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.456+0000 7f39ecff9640 1 -- 172.21.15.38:0/2696469066 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f39e40c0630 con 0x7f39f0102bf0 2024-05-21T15:32:03.457 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:03.457 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-21T15:32:03.457 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 4 2024-05-21T15:32:03.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.458+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2696469066 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f39d80a2c30 msgr2=0x7f39d80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:03.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.458+0000 7f39f5fd5640 1 --2- 172.21.15.38:0/2696469066 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f39d80a2c30 0x7f39d80a5140 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f39f01025b0 tx=0x7f39e0005f90 comp rx=0 tx=0).stop 2024-05-21T15:32:03.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.458+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2696469066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f0102bf0 msgr2=0x7f39f0199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:03.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.458+0000 7f39f5fd5640 1 --2- 172.21.15.38:0/2696469066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f0102bf0 0x7f39f0199720 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f39e4006010 tx=0x7f39e40042f0 comp rx=0 tx=0).stop 2024-05-21T15:32:03.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.458+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2696469066 shutdown_connections 2024-05-21T15:32:03.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.458+0000 7f39f5fd5640 1 --2- 172.21.15.38:0/2696469066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39f01078f0 0x7f39f01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:03.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.458+0000 7f39f5fd5640 1 --2- 172.21.15.38:0/2696469066 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f39d80a2c30 0x7f39d80a5140 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:03.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.458+0000 7f39f5fd5640 1 --2- 172.21.15.38:0/2696469066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39f0102bf0 0x7f39f0199720 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:03.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.458+0000 7f39f5fd5640 1 --2- 172.21.15.38:0/2696469066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39f01019b0 0x7f39f01991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:03.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.458+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2696469066 >> 172.21.15.38:0/2696469066 conn(0x7f39f00fcba0 msgr2=0x7f39f01038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:03.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.459+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2696469066 shutdown_connections 2024-05-21T15:32:03.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:03.459+0000 7f39f5fd5640 1 -- 172.21.15.38:0/2696469066 wait complete. 2024-05-21T15:32:04.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:03 smithi053 ceph-mon[36496]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:04.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:04.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:04.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:04.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:04.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:04.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:04.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:03 smithi053 ceph-mon[36496]: Deploying daemon mds.d on smithi038 2024-05-21T15:32:04.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:03 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.38:6834/3555626860,v1:172.21.15.38:6835/3555626860] up:boot 2024-05-21T15:32:04.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:03 smithi053 ceph-mon[36496]: fsmap 3 up:standby 2024-05-21T15:32:04.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:03 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T15:32:04.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:03 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2696469066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:04.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:03 smithi045 ceph-mon[36409]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:04.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:04.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:04.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:04.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:04.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:04.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:04.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:03 smithi045 ceph-mon[36409]: Deploying daemon mds.d on smithi038 2024-05-21T15:32:04.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:03 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.38:6834/3555626860,v1:172.21.15.38:6835/3555626860] up:boot 2024-05-21T15:32:04.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:03 smithi045 ceph-mon[36409]: fsmap 3 up:standby 2024-05-21T15:32:04.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:03 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T15:32:04.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:03 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2696469066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:04.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:03 smithi038 ceph-mon[31995]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:04.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:04.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:04.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:04.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:04.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:04.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:04.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:03 smithi038 ceph-mon[31995]: Deploying daemon mds.d on smithi038 2024-05-21T15:32:04.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:03 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.38:6834/3555626860,v1:172.21.15.38:6835/3555626860] up:boot 2024-05-21T15:32:04.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:03 smithi038 ceph-mon[31995]: fsmap 3 up:standby 2024-05-21T15:32:04.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:03 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T15:32:04.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:03 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2696469066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:05.472 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:32:05.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.594+0000 7f4771938640 1 -- 172.21.15.38:0/430128131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c101140 msgr2=0x7f476c101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:05.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.594+0000 7f4771938640 1 --2- 172.21.15.38:0/430128131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c101140 0x7f476c101e40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f475c009900 tx=0x7f475c02a3f0 comp rx=0 tx=0).stop 2024-05-21T15:32:05.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.594+0000 7f4771938640 1 -- 172.21.15.38:0/430128131 shutdown_connections 2024-05-21T15:32:05.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.594+0000 7f4771938640 1 --2- 172.21.15.38:0/430128131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f476c107080 0x7f476c1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:05.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.594+0000 7f4771938640 1 --2- 172.21.15.38:0/430128131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f476c102380 0x7f476c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:05.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.594+0000 7f4771938640 1 --2- 172.21.15.38:0/430128131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c101140 0x7f476c101e40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:05.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.594+0000 7f4771938640 1 -- 172.21.15.38:0/430128131 >> 172.21.15.38:0/430128131 conn(0x7f476c0fc330 msgr2=0x7f476c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:05.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.594+0000 7f4771938640 1 -- 172.21.15.38:0/430128131 shutdown_connections 2024-05-21T15:32:05.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.594+0000 7f4771938640 1 -- 172.21.15.38:0/430128131 wait complete. 2024-05-21T15:32:05.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.595+0000 7f4771938640 1 Processor -- start 2024-05-21T15:32:05.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.595+0000 7f4771938640 1 -- start start 2024-05-21T15:32:05.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.595+0000 7f4771938640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f476c101140 0x7f476c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:05.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.595+0000 7f4771938640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c102380 0x7f476c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:05.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.595+0000 7f4771938640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f476c107080 0x7f476c1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:05.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.595+0000 7f4771938640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f476c10e780 con 0x7f476c102380 2024-05-21T15:32:05.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.595+0000 7f4771938640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f476c10e600 con 0x7f476c101140 2024-05-21T15:32:05.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.595+0000 7f4771938640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f476c10e900 con 0x7f476c107080 2024-05-21T15:32:05.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.595+0000 7f476a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c102380 0x7f476c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:05.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.595+0000 7f476a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c102380 0x7f476c1997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38942/0 (socket says 172.21.15.38:38942) 2024-05-21T15:32:05.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.595+0000 7f476a7fc640 1 -- 172.21.15.38:0/2596210150 learned_addr learned my addr 172.21.15.38:0/2596210150 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:05.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.595+0000 7f476b7fe640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f476c107080 0x7f476c1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:05.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.595+0000 7f476affd640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f476c101140 0x7f476c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:05.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.596+0000 7f476a7fc640 1 -- 172.21.15.38:0/2596210150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f476c107080 msgr2=0x7f476c1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:05.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.596+0000 7f476a7fc640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f476c107080 0x7f476c1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:05.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.596+0000 7f476a7fc640 1 -- 172.21.15.38:0/2596210150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f476c101140 msgr2=0x7f476c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:05.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.596+0000 7f476a7fc640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f476c101140 0x7f476c1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:05.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.596+0000 7f476a7fc640 1 -- 172.21.15.38:0/2596210150 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476c1a0d30 con 0x7f476c102380 2024-05-21T15:32:05.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.596+0000 7f476affd640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f476c101140 0x7f476c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:32:05.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.596+0000 7f476a7fc640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c102380 0x7f476c1997e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f476000d770 tx=0x7f476000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:05.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.596+0000 7f4770936640 1 -- 172.21.15.38:0/2596210150 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760017070 con 0x7f476c102380 2024-05-21T15:32:05.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.596+0000 7f4770936640 1 -- 172.21.15.38:0/2596210150 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4760005500 con 0x7f476c102380 2024-05-21T15:32:05.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.596+0000 7f4771938640 1 -- 172.21.15.38:0/2596210150 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476c1a0fc0 con 0x7f476c102380 2024-05-21T15:32:05.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.596+0000 7f4770936640 1 -- 172.21.15.38:0/2596210150 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760004e00 con 0x7f476c102380 2024-05-21T15:32:05.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.596+0000 7f4771938640 1 -- 172.21.15.38:0/2596210150 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476c1a1430 con 0x7f476c102380 2024-05-21T15:32:05.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.597+0000 7f4771938640 1 -- 172.21.15.38:0/2596210150 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f476c105780 con 0x7f476c102380 2024-05-21T15:32:05.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.599+0000 7f4770936640 1 -- 172.21.15.38:0/2596210150 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4760004fa0 con 0x7f476c102380 2024-05-21T15:32:05.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.599+0000 7f4770936640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f47540a2c30 0x7f47540a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:05.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.599+0000 7f4770936640 1 -- 172.21.15.38:0/2596210150 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f47600ce9d0 con 0x7f476c102380 2024-05-21T15:32:05.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.599+0000 7f476affd640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f47540a2c30 0x7f47540a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:05.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.600+0000 7f476affd640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f47540a2c30 0x7f47540a5140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f475c02a9e0 tx=0x7f475c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:05.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.604+0000 7f4770936640 1 -- 172.21.15.38:0/2596210150 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f476009bdd0 con 0x7f476c102380 2024-05-21T15:32:05.775 INFO:journalctl@ceph.mds.d.smithi038.stdout:May 21 15:32:05 smithi038 systemd[1]: Started Ceph mds.d for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:32:05.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.831+0000 7f4771938640 1 -- 172.21.15.38:0/2596210150 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f476c02d2a0 con 0x7f476c102380 2024-05-21T15:32:05.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.832+0000 7f4770936640 1 -- 172.21.15.38:0/2596210150 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f476009b6d0 con 0x7f476c102380 2024-05-21T15:32:05.833 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:05.834 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-21T15:32:05.835 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 4 2024-05-21T15:32:05.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.834+0000 7f4771938640 1 -- 172.21.15.38:0/2596210150 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f47540a2c30 msgr2=0x7f47540a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:05.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.834+0000 7f4771938640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f47540a2c30 0x7f47540a5140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f475c02a9e0 tx=0x7f475c031400 comp rx=0 tx=0).stop 2024-05-21T15:32:05.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.834+0000 7f4771938640 1 -- 172.21.15.38:0/2596210150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c102380 msgr2=0x7f476c1997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:05.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.834+0000 7f4771938640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c102380 0x7f476c1997e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f476000d770 tx=0x7f476000dc30 comp rx=0 tx=0).stop 2024-05-21T15:32:05.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.834+0000 7f4771938640 1 -- 172.21.15.38:0/2596210150 shutdown_connections 2024-05-21T15:32:05.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.834+0000 7f4771938640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f476c107080 0x7f476c1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:05.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.834+0000 7f4771938640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f47540a2c30 0x7f47540a5140 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:05.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.834+0000 7f4771938640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f476c102380 0x7f476c1997e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:05.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.834+0000 7f4771938640 1 --2- 172.21.15.38:0/2596210150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f476c101140 0x7f476c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:05.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.834+0000 7f4771938640 1 -- 172.21.15.38:0/2596210150 >> 172.21.15.38:0/2596210150 conn(0x7f476c0fc330 msgr2=0x7f476c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:05.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.834+0000 7f4771938640 1 -- 172.21.15.38:0/2596210150 shutdown_connections 2024-05-21T15:32:05.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:05.834+0000 7f4771938640 1 -- 172.21.15.38:0/2596210150 wait complete. 2024-05-21T15:32:06.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:05 smithi053 ceph-mon[36496]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:06.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:05 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:06.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:05 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:06.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:05 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:06.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:05 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:06.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:05 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:06.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:05 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:06.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:05 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2596210150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:06.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:05 smithi045 ceph-mon[36409]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:06.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:05 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:06.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:05 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:06.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:05 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:06.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:05 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:06.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:05 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:06.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:05 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:06.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:05 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2596210150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:06.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:05 smithi038 ceph-mon[31995]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:06.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:05 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:06.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:05 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:06.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:05 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:06.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:05 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:06.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:05 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:06.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:05 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:06.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:05 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2596210150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:07.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:06 smithi045 ceph-mon[36409]: Deploying daemon mds.e on smithi045 2024-05-21T15:32:07.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:06 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.38:6836/2818680145,v1:172.21.15.38:6837/2818680145] up:boot 2024-05-21T15:32:07.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:06 smithi045 ceph-mon[36409]: fsmap 4 up:standby 2024-05-21T15:32:07.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:06 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T15:32:07.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:06 smithi053 ceph-mon[36496]: Deploying daemon mds.e on smithi045 2024-05-21T15:32:07.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:06 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.38:6836/2818680145,v1:172.21.15.38:6837/2818680145] up:boot 2024-05-21T15:32:07.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:06 smithi053 ceph-mon[36496]: fsmap 4 up:standby 2024-05-21T15:32:07.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:06 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T15:32:07.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:06 smithi038 ceph-mon[31995]: Deploying daemon mds.e on smithi045 2024-05-21T15:32:07.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:06 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.38:6836/2818680145,v1:172.21.15.38:6837/2818680145] up:boot 2024-05-21T15:32:07.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:06 smithi038 ceph-mon[31995]: fsmap 4 up:standby 2024-05-21T15:32:07.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:06 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T15:32:07.847 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:32:07.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.977+0000 7f4f21b38640 1 -- 172.21.15.38:0/259396614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f1c1078e0 msgr2=0x7f4f1c109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:07.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.977+0000 7f4f21b38640 1 --2- 172.21.15.38:0/259396614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f1c1078e0 0x7f4f1c109d00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009ca0 tx=0x7f4f1402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:32:07.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.977+0000 7f4f21b38640 1 -- 172.21.15.38:0/259396614 shutdown_connections 2024-05-21T15:32:07.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.977+0000 7f4f21b38640 1 --2- 172.21.15.38:0/259396614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f1c1078e0 0x7f4f1c109d00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:07.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.977+0000 7f4f21b38640 1 --2- 172.21.15.38:0/259396614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f1c102be0 0x7f4f1c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:07.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.977+0000 7f4f21b38640 1 --2- 172.21.15.38:0/259396614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f1c1019a0 0x7f4f1c1026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:07.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.977+0000 7f4f21b38640 1 -- 172.21.15.38:0/259396614 >> 172.21.15.38:0/259396614 conn(0x7f4f1c0fcbb0 msgr2=0x7f4f1c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:07.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.977+0000 7f4f21b38640 1 -- 172.21.15.38:0/259396614 shutdown_connections 2024-05-21T15:32:07.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.977+0000 7f4f21b38640 1 -- 172.21.15.38:0/259396614 wait complete. 2024-05-21T15:32:07.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.978+0000 7f4f21b38640 1 Processor -- start 2024-05-21T15:32:07.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.978+0000 7f4f21b38640 1 -- start start 2024-05-21T15:32:07.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.978+0000 7f4f21b38640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f1c1019a0 0x7f4f1c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:07.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.978+0000 7f4f21b38640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f1c102be0 0x7f4f1c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:07.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.978+0000 7f4f21b38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f1c1078e0 0x7f4f1c19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:07.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.978+0000 7f4f21b38640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c10d600 con 0x7f4f1c1078e0 2024-05-21T15:32:07.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.978+0000 7f4f21b38640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c10d480 con 0x7f4f1c1019a0 2024-05-21T15:32:07.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.978+0000 7f4f21b38640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c10d780 con 0x7f4f1c102be0 2024-05-21T15:32:07.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.978+0000 7f4f1bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f1c1078e0 0x7f4f1c19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:07.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.978+0000 7f4f1bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f1c1078e0 0x7f4f1c19f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38960/0 (socket says 172.21.15.38:38960) 2024-05-21T15:32:07.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.978+0000 7f4f1bfff640 1 -- 172.21.15.38:0/3407111710 learned_addr learned my addr 172.21.15.38:0/3407111710 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:07.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.978+0000 7f4f1b7fe640 1 --2- 172.21.15.38:0/3407111710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f1c1019a0 0x7f4f1c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:07.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.978+0000 7f4f1affd640 1 --2- 172.21.15.38:0/3407111710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f1c102be0 0x7f4f1c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:07.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.979+0000 7f4f1bfff640 1 -- 172.21.15.38:0/3407111710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f1c102be0 msgr2=0x7f4f1c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:07.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.979+0000 7f4f1bfff640 1 --2- 172.21.15.38:0/3407111710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f1c102be0 0x7f4f1c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:07.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.979+0000 7f4f1bfff640 1 -- 172.21.15.38:0/3407111710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f1c1019a0 msgr2=0x7f4f1c198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:07.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.979+0000 7f4f1bfff640 1 --2- 172.21.15.38:0/3407111710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f1c1019a0 0x7f4f1c198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:07.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.979+0000 7f4f1bfff640 1 -- 172.21.15.38:0/3407111710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c19fbb0 con 0x7f4f1c1078e0 2024-05-21T15:32:07.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.979+0000 7f4f1bfff640 1 --2- 172.21.15.38:0/3407111710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f1c1078e0 0x7f4f1c19f540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4f14005cc0 tx=0x7f4f14009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:07.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.979+0000 7f4f18ff9640 1 -- 172.21.15.38:0/3407111710 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14003c30 con 0x7f4f1c1078e0 2024-05-21T15:32:07.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.979+0000 7f4f21b38640 1 -- 172.21.15.38:0/3407111710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1c19fe40 con 0x7f4f1c1078e0 2024-05-21T15:32:07.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.979+0000 7f4f18ff9640 1 -- 172.21.15.38:0/3407111710 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f14003dd0 con 0x7f4f1c1078e0 2024-05-21T15:32:07.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.979+0000 7f4f18ff9640 1 -- 172.21.15.38:0/3407111710 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1402fc50 con 0x7f4f1c1078e0 2024-05-21T15:32:07.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.979+0000 7f4f21b38640 1 -- 172.21.15.38:0/3407111710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1c1a0300 con 0x7f4f1c1078e0 2024-05-21T15:32:07.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.980+0000 7f4f21b38640 1 -- 172.21.15.38:0/3407111710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f1c105fe0 con 0x7f4f1c1078e0 2024-05-21T15:32:07.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.982+0000 7f4f18ff9640 1 -- 172.21.15.38:0/3407111710 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4f14004050 con 0x7f4f1c1078e0 2024-05-21T15:32:07.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.983+0000 7f4f18ff9640 1 --2- 172.21.15.38:0/3407111710 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f040a2c30 0x7f4f040a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:07.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.983+0000 7f4f1b7fe640 1 --2- 172.21.15.38:0/3407111710 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f040a2c30 0x7f4f040a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:07.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.983+0000 7f4f18ff9640 1 -- 172.21.15.38:0/3407111710 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4f140f0200 con 0x7f4f1c1078e0 2024-05-21T15:32:07.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.983+0000 7f4f1b7fe640 1 --2- 172.21.15.38:0/3407111710 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f040a2c30 0x7f4f040a5140 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c005fd0 tx=0x7f4f0c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:07.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:07.986+0000 7f4f18ff9640 1 -- 172.21.15.38:0/3407111710 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4f140bd600 con 0x7f4f1c1078e0 2024-05-21T15:32:08.176 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:07 smithi045 ceph-mon[36409]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:08.177 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:32:07 smithi045 systemd[1]: Starting Ceph mds.e for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:32:08.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:07 smithi053 ceph-mon[36496]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:08.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:08.208+0000 7f4f21b38640 1 -- 172.21.15.38:0/3407111710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f1c02d2a0 con 0x7f4f1c1078e0 2024-05-21T15:32:08.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:08.208+0000 7f4f18ff9640 1 -- 172.21.15.38:0/3407111710 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7f4f140bcf00 con 0x7f4f1c1078e0 2024-05-21T15:32:08.209 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:08.209 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-21T15:32:08.209 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 5 2024-05-21T15:32:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:08.210+0000 7f4f21b38640 1 -- 172.21.15.38:0/3407111710 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f040a2c30 msgr2=0x7f4f040a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:08.210+0000 7f4f21b38640 1 --2- 172.21.15.38:0/3407111710 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f040a2c30 0x7f4f040a5140 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c005fd0 tx=0x7f4f0c00b410 comp rx=0 tx=0).stop 2024-05-21T15:32:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:08.210+0000 7f4f21b38640 1 -- 172.21.15.38:0/3407111710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f1c1078e0 msgr2=0x7f4f1c19f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:08.210+0000 7f4f21b38640 1 --2- 172.21.15.38:0/3407111710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f1c1078e0 0x7f4f1c19f540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4f14005cc0 tx=0x7f4f14009890 comp rx=0 tx=0).stop 2024-05-21T15:32:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:08.210+0000 7f4f21b38640 1 -- 172.21.15.38:0/3407111710 shutdown_connections 2024-05-21T15:32:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:08.210+0000 7f4f21b38640 1 --2- 172.21.15.38:0/3407111710 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f040a2c30 0x7f4f040a5140 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:08.210+0000 7f4f21b38640 1 --2- 172.21.15.38:0/3407111710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f1c1078e0 0x7f4f1c19f540 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:08.210+0000 7f4f21b38640 1 --2- 172.21.15.38:0/3407111710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f1c102be0 0x7f4f1c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:08.210+0000 7f4f21b38640 1 --2- 172.21.15.38:0/3407111710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f1c1019a0 0x7f4f1c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:08.210+0000 7f4f21b38640 1 -- 172.21.15.38:0/3407111710 >> 172.21.15.38:0/3407111710 conn(0x7f4f1c0fcbb0 msgr2=0x7f4f1c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:08.210+0000 7f4f21b38640 1 -- 172.21.15.38:0/3407111710 shutdown_connections 2024-05-21T15:32:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:08.210+0000 7f4f21b38640 1 -- 172.21.15.38:0/3407111710 wait complete. 2024-05-21T15:32:08.221 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:07 smithi038 ceph-mon[31995]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:08.435 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:32:08 smithi045 podman[65813]: 2024-05-21 15:32:08.079869135 +0000 UTC m=+0.017946345 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:32:08.435 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:32:08 smithi045 podman[65813]: 2024-05-21 15:32:08.190065714 +0000 UTC m=+0.128142910 container create d12932c6e8e9f7c44604d22e80e721922690038fba14e212d0079c2a3d8c00e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:32:08.435 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:32:08 smithi045 podman[65813]: 2024-05-21 15:32:08.435168504 +0000 UTC m=+0.373245723 container init d12932c6e8e9f7c44604d22e80e721922690038fba14e212d0079c2a3d8c00e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-e, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2) 2024-05-21T15:32:08.761 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:32:08 smithi045 podman[65813]: 2024-05-21 15:32:08.437832812 +0000 UTC m=+0.375910017 container start d12932c6e8e9f7c44604d22e80e721922690038fba14e212d0079c2a3d8c00e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-21T15:32:08.761 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:32:08 smithi045 ceph-mds[65832]: -- 172.21.15.45:0/2440552753 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55571dc11a40 con 0x55571dc89a80 2024-05-21T15:32:08.762 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:32:08 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-e[65828]: starting mds.e at 2024-05-21T15:32:08.762 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:32:08 smithi045 bash[65813]: d12932c6e8e9f7c44604d22e80e721922690038fba14e212d0079c2a3d8c00e8 2024-05-21T15:32:08.762 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:32:08 smithi045 systemd[1]: Started Ceph mds.e for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:32:09.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:08 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3407111710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:09.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:08 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:09.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:08 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:09.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:08 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:09.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:08 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:09.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:08 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:09.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:08 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:09.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:08 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3407111710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:09.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:08 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:09.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:08 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:09.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:08 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:09.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:08 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:09.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:08 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:09.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:08 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:09.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:08 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3407111710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:09.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:08 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:09.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:08 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:09.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:08 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:09.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:08 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:09.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:08 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:09.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:08 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:09.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:09 smithi053 ceph-mon[36496]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:09.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:09 smithi053 ceph-mon[36496]: Deploying daemon mds.f on smithi053 2024-05-21T15:32:09.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:09 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] up:boot 2024-05-21T15:32:09.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:09 smithi053 ceph-mon[36496]: fsmap 5 up:standby 2024-05-21T15:32:09.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:09 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:32:10.224 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:32:10.248 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:09 smithi038 ceph-mon[31995]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:10.248 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:09 smithi038 ceph-mon[31995]: Deploying daemon mds.f on smithi053 2024-05-21T15:32:10.248 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:09 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] up:boot 2024-05-21T15:32:10.248 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:09 smithi038 ceph-mon[31995]: fsmap 5 up:standby 2024-05-21T15:32:10.248 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:09 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:32:10.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:09 smithi045 ceph-mon[36409]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:10.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:09 smithi045 ceph-mon[36409]: Deploying daemon mds.f on smithi053 2024-05-21T15:32:10.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:09 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] up:boot 2024-05-21T15:32:10.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:09 smithi045 ceph-mon[36409]: fsmap 5 up:standby 2024-05-21T15:32:10.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:09 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:32:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.341+0000 7f2029ce9640 1 -- 172.21.15.38:0/3727142650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2024102bf0 msgr2=0x7f2024103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.341+0000 7f2029ce9640 1 --2- 172.21.15.38:0/3727142650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2024102bf0 0x7f2024103000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f200c009900 tx=0x7f200c02a410 comp rx=0 tx=0).stop 2024-05-21T15:32:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.341+0000 7f2029ce9640 1 -- 172.21.15.38:0/3727142650 shutdown_connections 2024-05-21T15:32:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.341+0000 7f2029ce9640 1 --2- 172.21.15.38:0/3727142650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20241078f0 0x7f2024109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.341+0000 7f2029ce9640 1 --2- 172.21.15.38:0/3727142650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2024102bf0 0x7f2024103000 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.341+0000 7f2029ce9640 1 --2- 172.21.15.38:0/3727142650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20241019b0 0x7f20241026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.341+0000 7f2029ce9640 1 -- 172.21.15.38:0/3727142650 >> 172.21.15.38:0/3727142650 conn(0x7f20240fcba0 msgr2=0x7f20240ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.341+0000 7f2029ce9640 1 -- 172.21.15.38:0/3727142650 shutdown_connections 2024-05-21T15:32:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.341+0000 7f2029ce9640 1 -- 172.21.15.38:0/3727142650 wait complete. 2024-05-21T15:32:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.342+0000 7f2029ce9640 1 Processor -- start 2024-05-21T15:32:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.342+0000 7f2029ce9640 1 -- start start 2024-05-21T15:32:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.342+0000 7f2029ce9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20241019b0 0x7f20240fe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:10.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.342+0000 7f2029ce9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2024102bf0 0x7f20240fd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:10.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.342+0000 7f2029ce9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20241078f0 0x7f20240fd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:10.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.342+0000 7f2029ce9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f202410d620 con 0x7f2024102bf0 2024-05-21T15:32:10.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.342+0000 7f2029ce9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f202410d4a0 con 0x7f20241078f0 2024-05-21T15:32:10.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.342+0000 7f2029ce9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f202410d7a0 con 0x7f20241019b0 2024-05-21T15:32:10.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.342+0000 7f201affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2024102bf0 0x7f20240fd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:10.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.342+0000 7f201affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2024102bf0 0x7f20240fd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38966/0 (socket says 172.21.15.38:38966) 2024-05-21T15:32:10.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.342+0000 7f201affd640 1 -- 172.21.15.38:0/2041188765 learned_addr learned my addr 172.21.15.38:0/2041188765 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:10.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.342+0000 7f20237fe640 1 --2- 172.21.15.38:0/2041188765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20241019b0 0x7f20240fe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:10.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.342+0000 7f2023fff640 1 --2- 172.21.15.38:0/2041188765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20241078f0 0x7f20240fd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:10.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.343+0000 7f201affd640 1 -- 172.21.15.38:0/2041188765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20241019b0 msgr2=0x7f20240fe9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:10.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.343+0000 7f201affd640 1 --2- 172.21.15.38:0/2041188765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20241019b0 0x7f20240fe9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:10.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.343+0000 7f201affd640 1 -- 172.21.15.38:0/2041188765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20241078f0 msgr2=0x7f20240fd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:10.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.343+0000 7f201affd640 1 --2- 172.21.15.38:0/2041188765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20241078f0 0x7f20240fd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:10.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.343+0000 7f201affd640 1 -- 172.21.15.38:0/2041188765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20240fdcf0 con 0x7f2024102bf0 2024-05-21T15:32:10.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.343+0000 7f201affd640 1 --2- 172.21.15.38:0/2041188765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2024102bf0 0x7f20240fd0d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f200c006010 tx=0x7f200c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:10.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.343+0000 7f20217fa640 1 -- 172.21.15.38:0/2041188765 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c03f030 con 0x7f2024102bf0 2024-05-21T15:32:10.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.343+0000 7f20217fa640 1 -- 172.21.15.38:0/2041188765 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f200c003ad0 con 0x7f2024102bf0 2024-05-21T15:32:10.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.343+0000 7f2029ce9640 1 -- 172.21.15.38:0/2041188765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20241a32e0 con 0x7f2024102bf0 2024-05-21T15:32:10.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.343+0000 7f20217fa640 1 -- 172.21.15.38:0/2041188765 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c035070 con 0x7f2024102bf0 2024-05-21T15:32:10.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.343+0000 7f2029ce9640 1 -- 172.21.15.38:0/2041188765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20241a3690 con 0x7f2024102bf0 2024-05-21T15:32:10.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.344+0000 7f2029ce9640 1 -- 172.21.15.38:0/2041188765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20241a38f0 con 0x7f2024102bf0 2024-05-21T15:32:10.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.345+0000 7f20217fa640 1 -- 172.21.15.38:0/2041188765 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f200c042030 con 0x7f2024102bf0 2024-05-21T15:32:10.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.346+0000 7f20217fa640 1 --2- 172.21.15.38:0/2041188765 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f20100a2c30 0x7f20100a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:10.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.346+0000 7f20237fe640 1 --2- 172.21.15.38:0/2041188765 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f20100a2c30 0x7f20100a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:10.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.346+0000 7f20217fa640 1 -- 172.21.15.38:0/2041188765 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f200c0ef490 con 0x7f2024102bf0 2024-05-21T15:32:10.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.347+0000 7f20237fe640 1 --2- 172.21.15.38:0/2041188765 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f20100a2c30 0x7f20100a5140 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f20241025b0 tx=0x7f2014005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:10.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.351+0000 7f20217fa640 1 -- 172.21.15.38:0/2041188765 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f200c0bc240 con 0x7f2024102bf0 2024-05-21T15:32:10.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.574+0000 7f2029ce9640 1 -- 172.21.15.38:0/2041188765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20241a3ac0 con 0x7f2024102bf0 2024-05-21T15:32:10.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.575+0000 7f20217fa640 1 -- 172.21.15.38:0/2041188765 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f200c0c0540 con 0x7f2024102bf0 2024-05-21T15:32:10.576 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:10.576 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-21T15:32:10.576 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 6 2024-05-21T15:32:10.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.577+0000 7f2029ce9640 1 -- 172.21.15.38:0/2041188765 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f20100a2c30 msgr2=0x7f20100a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:10.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.577+0000 7f2029ce9640 1 --2- 172.21.15.38:0/2041188765 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f20100a2c30 0x7f20100a5140 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f20241025b0 tx=0x7f2014005f90 comp rx=0 tx=0).stop 2024-05-21T15:32:10.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.577+0000 7f2029ce9640 1 -- 172.21.15.38:0/2041188765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2024102bf0 msgr2=0x7f20240fd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:10.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.577+0000 7f2029ce9640 1 --2- 172.21.15.38:0/2041188765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2024102bf0 0x7f20240fd0d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f200c006010 tx=0x7f200c031b60 comp rx=0 tx=0).stop 2024-05-21T15:32:10.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.577+0000 7f2029ce9640 1 -- 172.21.15.38:0/2041188765 shutdown_connections 2024-05-21T15:32:10.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.577+0000 7f2029ce9640 1 --2- 172.21.15.38:0/2041188765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20241078f0 0x7f20240fd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:10.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.577+0000 7f2029ce9640 1 --2- 172.21.15.38:0/2041188765 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f20100a2c30 0x7f20100a5140 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:10.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.577+0000 7f2029ce9640 1 --2- 172.21.15.38:0/2041188765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2024102bf0 0x7f20240fd0d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:10.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.577+0000 7f2029ce9640 1 --2- 172.21.15.38:0/2041188765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20241019b0 0x7f20240fe9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:10.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.577+0000 7f2029ce9640 1 -- 172.21.15.38:0/2041188765 >> 172.21.15.38:0/2041188765 conn(0x7f20240fcba0 msgr2=0x7f2024108c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:10.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.577+0000 7f2029ce9640 1 -- 172.21.15.38:0/2041188765 shutdown_connections 2024-05-21T15:32:10.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:10.577+0000 7f2029ce9640 1 -- 172.21.15.38:0/2041188765 wait complete. 2024-05-21T15:32:10.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:10 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2041188765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:11.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:10 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2041188765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:11.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:10 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2041188765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:11.706 INFO:journalctl@ceph.mds.f.smithi053.stdout:May 21 15:32:11 smithi053 systemd[1]: Started Ceph mds.f for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:32:12.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:11 smithi053 ceph-mon[36496]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:12.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:11 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:12.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:11 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:12.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:11 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:12.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:11 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:12.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:11 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:12.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:11 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:12.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:11 smithi045 ceph-mon[36409]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:12.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:11 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:12.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:11 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:12.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:11 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:12.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:11 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:12.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:11 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:12.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:11 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:12.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:11 smithi038 ceph-mon[31995]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:12.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:11 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:12.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:11 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:12.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:11 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:12.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:11 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:12.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:11 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:12.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:11 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:12.589 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:32:12.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.719+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1573035453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5481078f0 msgr2=0x7fa548109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:12.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.719+0000 7fa54d0d3640 1 --2- 172.21.15.38:0/1573035453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5481078f0 0x7fa548109d10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa540009ca0 tx=0x7fa54002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:32:12.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.719+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1573035453 shutdown_connections 2024-05-21T15:32:12.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.719+0000 7fa54d0d3640 1 --2- 172.21.15.38:0/1573035453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5481078f0 0x7fa548109d10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:12.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.719+0000 7fa54d0d3640 1 --2- 172.21.15.38:0/1573035453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa548102bf0 0x7fa548103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:12.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.719+0000 7fa54d0d3640 1 --2- 172.21.15.38:0/1573035453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5481019b0 0x7fa5481026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:12.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.719+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1573035453 >> 172.21.15.38:0/1573035453 conn(0x7fa5480fcba0 msgr2=0x7fa5480ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:12.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.719+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1573035453 shutdown_connections 2024-05-21T15:32:12.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.719+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1573035453 wait complete. 2024-05-21T15:32:12.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.720+0000 7fa54d0d3640 1 Processor -- start 2024-05-21T15:32:12.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.720+0000 7fa54d0d3640 1 -- start start 2024-05-21T15:32:12.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.720+0000 7fa54d0d3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5481019b0 0x7fa5481991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:12.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.720+0000 7fa54d0d3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa548102bf0 0x7fa5481996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:12.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.720+0000 7fa54d0d3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5481078f0 0x7fa5481a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:12.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.720+0000 7fa54d0d3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa54810d630 con 0x7fa5481078f0 2024-05-21T15:32:12.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.720+0000 7fa54d0d3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa54810d4b0 con 0x7fa5481019b0 2024-05-21T15:32:12.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.720+0000 7fa54d0d3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa54810d7b0 con 0x7fa548102bf0 2024-05-21T15:32:12.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa54759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5481078f0 0x7fa5481a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:12.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa54759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5481078f0 0x7fa5481a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38990/0 (socket says 172.21.15.38:38990) 2024-05-21T15:32:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa54759e640 1 -- 172.21.15.38:0/1620583748 learned_addr learned my addr 172.21.15.38:0/1620583748 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa54659c640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa548102bf0 0x7fa5481996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa546d9d640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5481019b0 0x7fa5481991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa54759e640 1 -- 172.21.15.38:0/1620583748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa548102bf0 msgr2=0x7fa5481996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa54759e640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa548102bf0 0x7fa5481996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa54759e640 1 -- 172.21.15.38:0/1620583748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5481019b0 msgr2=0x7fa5481991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa54759e640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5481019b0 0x7fa5481991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa54759e640 1 -- 172.21.15.38:0/1620583748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5481a0cf0 con 0x7fa5481078f0 2024-05-21T15:32:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa546d9d640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5481019b0 0x7fa5481991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:32:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa54759e640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5481078f0 0x7fa5481a0680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa540005cc0 tx=0x7fa540004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa52ffff640 1 -- 172.21.15.38:0/1620583748 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540003b70 con 0x7fa5481078f0 2024-05-21T15:32:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1620583748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5481a0f80 con 0x7fa5481078f0 2024-05-21T15:32:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.721+0000 7fa52ffff640 1 -- 172.21.15.38:0/1620583748 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa540003d10 con 0x7fa5481078f0 2024-05-21T15:32:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.722+0000 7fa52ffff640 1 -- 172.21.15.38:0/1620583748 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54002fbd0 con 0x7fa5481078f0 2024-05-21T15:32:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.722+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1620583748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5481a13e0 con 0x7fa5481078f0 2024-05-21T15:32:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.723+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1620583748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa548068d70 con 0x7fa5481078f0 2024-05-21T15:32:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.724+0000 7fa52ffff640 1 -- 172.21.15.38:0/1620583748 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa54002fd70 con 0x7fa5481078f0 2024-05-21T15:32:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.725+0000 7fa52ffff640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa5300a2c30 0x7fa5300a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.725+0000 7fa546d9d640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa5300a2c30 0x7fa5300a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.725+0000 7fa52ffff640 1 -- 172.21.15.38:0/1620583748 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa5400f00c0 con 0x7fa5481078f0 2024-05-21T15:32:12.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.725+0000 7fa546d9d640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa5300a2c30 0x7fa5300a5140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa5481025b0 tx=0x7fa538005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:12.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.728+0000 7fa52ffff640 1 -- 172.21.15.38:0/1620583748 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa5400bd420 con 0x7fa5481078f0 2024-05-21T15:32:12.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.952+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1620583748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa548107040 con 0x7fa5481078f0 2024-05-21T15:32:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.953+0000 7fa52ffff640 1 -- 172.21.15.38:0/1620583748 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7fa548107040 con 0x7fa5481078f0 2024-05-21T15:32:12.954 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:12.954 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-21T15:32:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-05-21T15:32:12.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.955+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1620583748 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa5300a2c30 msgr2=0x7fa5300a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:12.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.955+0000 7fa54d0d3640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa5300a2c30 0x7fa5300a5140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa5481025b0 tx=0x7fa538005f90 comp rx=0 tx=0).stop 2024-05-21T15:32:12.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.955+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1620583748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5481078f0 msgr2=0x7fa5481a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:12.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.955+0000 7fa54d0d3640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5481078f0 0x7fa5481a0680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa540005cc0 tx=0x7fa540004ab0 comp rx=0 tx=0).stop 2024-05-21T15:32:12.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.955+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1620583748 shutdown_connections 2024-05-21T15:32:12.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.955+0000 7fa54d0d3640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa5300a2c30 0x7fa5300a5140 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:12.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.955+0000 7fa54d0d3640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa5481078f0 0x7fa5481a0680 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:12.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.955+0000 7fa54d0d3640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa548102bf0 0x7fa5481996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:12.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.955+0000 7fa54d0d3640 1 --2- 172.21.15.38:0/1620583748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5481019b0 0x7fa5481991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:12.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.955+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1620583748 >> 172.21.15.38:0/1620583748 conn(0x7fa5480fcba0 msgr2=0x7fa5480ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:12.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.955+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1620583748 shutdown_connections 2024-05-21T15:32:12.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:12.955+0000 7fa54d0d3640 1 -- 172.21.15.38:0/1620583748 wait complete. 2024-05-21T15:32:13.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:12 smithi053 ceph-mon[36496]: Deploying daemon mds.c on smithi053 2024-05-21T15:32:13.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:12 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.53:6834/1330413091,v1:172.21.15.53:6835/1330413091] up:boot 2024-05-21T15:32:13.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:12 smithi053 ceph-mon[36496]: fsmap 6 up:standby 2024-05-21T15:32:13.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:12 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:32:13.239 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:12 smithi045 ceph-mon[36409]: Deploying daemon mds.c on smithi053 2024-05-21T15:32:13.239 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:12 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.53:6834/1330413091,v1:172.21.15.53:6835/1330413091] up:boot 2024-05-21T15:32:13.239 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:12 smithi045 ceph-mon[36409]: fsmap 6 up:standby 2024-05-21T15:32:13.239 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:12 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:32:13.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:12 smithi038 ceph-mon[31995]: Deploying daemon mds.c on smithi053 2024-05-21T15:32:13.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:12 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.53:6834/1330413091,v1:172.21.15.53:6835/1330413091] up:boot 2024-05-21T15:32:13.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:12 smithi038 ceph-mon[31995]: fsmap 6 up:standby 2024-05-21T15:32:13.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:12 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:32:13.932 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:32:13 smithi053 systemd[1]: Starting Ceph mds.c for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:32:14.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:13 smithi053 ceph-mon[36496]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:14.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:13 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1620583748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:14.207 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:32:14 smithi053 podman[66593]: 2024-05-21 15:32:13.952721092 +0000 UTC m=+0.018705131 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:32:14.207 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:32:14 smithi053 podman[66593]: 2024-05-21 15:32:14.08370729 +0000 UTC m=+0.149691312 container create d992e7fc917d7d4065e0e5860333cd05393d63a04404507c36832d8a9a9cf084 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-c, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True) 2024-05-21T15:32:14.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:13 smithi045 ceph-mon[36409]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:14.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:13 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1620583748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:14.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:13 smithi038 ceph-mon[31995]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:14.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:13 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1620583748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:14.706 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:32:14 smithi053 podman[66593]: 2024-05-21 15:32:14.3618704 +0000 UTC m=+0.427854419 container init d992e7fc917d7d4065e0e5860333cd05393d63a04404507c36832d8a9a9cf084 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-21T15:32:14.707 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:32:14 smithi053 podman[66593]: 2024-05-21 15:32:14.364434741 +0000 UTC m=+0.430418768 container start d992e7fc917d7d4065e0e5860333cd05393d63a04404507c36832d8a9a9cf084 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-c, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-21T15:32:14.707 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:32:14 smithi053 ceph-mds[66612]: -- 172.21.15.53:0/1543871920 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5601da375a40 con 0x5601da3eda80 2024-05-21T15:32:14.707 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:32:14 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-c[66608]: starting mds.c at 2024-05-21T15:32:14.707 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:32:14 smithi053 bash[66593]: d992e7fc917d7d4065e0e5860333cd05393d63a04404507c36832d8a9a9cf084 2024-05-21T15:32:14.707 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:32:14 smithi053 systemd[1]: Started Ceph mds.c for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:32:14.968 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:32:15.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.098+0000 7f73ccb2c640 1 -- 172.21.15.38:0/3741800323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c8102bf0 msgr2=0x7f73c8103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:15.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.099+0000 7f73ccb2c640 1 --2- 172.21.15.38:0/3741800323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c8102bf0 0x7f73c8103000 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f73bc009ca0 tx=0x7f73bc02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:32:15.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.099+0000 7f73ccb2c640 1 -- 172.21.15.38:0/3741800323 shutdown_connections 2024-05-21T15:32:15.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.099+0000 7f73ccb2c640 1 --2- 172.21.15.38:0/3741800323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c81078f0 0x7f73c8109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:15.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.099+0000 7f73ccb2c640 1 --2- 172.21.15.38:0/3741800323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c8102bf0 0x7f73c8103000 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:15.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.099+0000 7f73ccb2c640 1 --2- 172.21.15.38:0/3741800323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73c81019b0 0x7f73c81026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:15.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.099+0000 7f73ccb2c640 1 -- 172.21.15.38:0/3741800323 >> 172.21.15.38:0/3741800323 conn(0x7f73c80fcba0 msgr2=0x7f73c80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:15.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.099+0000 7f73ccb2c640 1 -- 172.21.15.38:0/3741800323 shutdown_connections 2024-05-21T15:32:15.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.099+0000 7f73ccb2c640 1 -- 172.21.15.38:0/3741800323 wait complete. 2024-05-21T15:32:15.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73ccb2c640 1 Processor -- start 2024-05-21T15:32:15.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73ccb2c640 1 -- start start 2024-05-21T15:32:15.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73ccb2c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73c81019b0 0x7f73c81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73ccb2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c8102bf0 0x7f73c8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73ccb2c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c81078f0 0x7f73c819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73ccb2c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f73c810d620 con 0x7f73c8102bf0 2024-05-21T15:32:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73ccb2c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73c810d4a0 con 0x7f73c81078f0 2024-05-21T15:32:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73ccb2c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f73c810d7a0 con 0x7f73c81019b0 2024-05-21T15:32:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73c5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c8102bf0 0x7f73c8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73c5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c8102bf0 0x7f73c8198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50840/0 (socket says 172.21.15.38:50840) 2024-05-21T15:32:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73c5d9b640 1 -- 172.21.15.38:0/678206621 learned_addr learned my addr 172.21.15.38:0/678206621 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73c6d9d640 1 --2- 172.21.15.38:0/678206621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c81078f0 0x7f73c819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73c659c640 1 --2- 172.21.15.38:0/678206621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73c81019b0 0x7f73c81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73c5d9b640 1 -- 172.21.15.38:0/678206621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73c81019b0 msgr2=0x7f73c81980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73c5d9b640 1 --2- 172.21.15.38:0/678206621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73c81019b0 0x7f73c81980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73c5d9b640 1 -- 172.21.15.38:0/678206621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c81078f0 msgr2=0x7f73c819f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73c5d9b640 1 --2- 172.21.15.38:0/678206621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c81078f0 0x7f73c819f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.100+0000 7f73c5d9b640 1 -- 172.21.15.38:0/678206621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c819fc10 con 0x7f73c8102bf0 2024-05-21T15:32:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.101+0000 7f73c5d9b640 1 --2- 172.21.15.38:0/678206621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c8102bf0 0x7f73c8198610 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73bc02ade0 tx=0x7f73bc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.101+0000 7f73b77fe640 1 -- 172.21.15.38:0/678206621 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc02fa20 con 0x7f73c8102bf0 2024-05-21T15:32:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.101+0000 7f73b77fe640 1 -- 172.21.15.38:0/678206621 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73bc02fbc0 con 0x7f73c8102bf0 2024-05-21T15:32:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.101+0000 7f73ccb2c640 1 -- 172.21.15.38:0/678206621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c819fea0 con 0x7f73c8102bf0 2024-05-21T15:32:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.101+0000 7f73b77fe640 1 -- 172.21.15.38:0/678206621 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc0466e0 con 0x7f73c8102bf0 2024-05-21T15:32:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.101+0000 7f73ccb2c640 1 -- 172.21.15.38:0/678206621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c81a0310 con 0x7f73c8102bf0 2024-05-21T15:32:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.102+0000 7f73ccb2c640 1 -- 172.21.15.38:0/678206621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c8106700 con 0x7f73c8102bf0 2024-05-21T15:32:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.103+0000 7f73b77fe640 1 -- 172.21.15.38:0/678206621 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f73bc046880 con 0x7f73c8102bf0 2024-05-21T15:32:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.104+0000 7f73b77fe640 1 --2- 172.21.15.38:0/678206621 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73b00a2c30 0x7f73b00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.105+0000 7f73c659c640 1 --2- 172.21.15.38:0/678206621 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73b00a2c30 0x7f73b00a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.105+0000 7f73b77fe640 1 -- 172.21.15.38:0/678206621 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f73bc0f0260 con 0x7f73c8102bf0 2024-05-21T15:32:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.105+0000 7f73c659c640 1 --2- 172.21.15.38:0/678206621 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73b00a2c30 0x7f73b00a5140 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f73c81025b0 tx=0x7f73b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:15.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.108+0000 7f73b77fe640 1 -- 172.21.15.38:0/678206621 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f73bc081850 con 0x7f73c8102bf0 2024-05-21T15:32:15.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.332+0000 7f73ccb2c640 1 -- 172.21.15.38:0/678206621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73c802d2a0 con 0x7f73c8102bf0 2024-05-21T15:32:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.332+0000 7f73b77fe640 1 -- 172.21.15.38:0/678206621 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6343 (secure 0 0 0) 0x7f73bc0bd600 con 0x7f73c8102bf0 2024-05-21T15:32:15.333 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:15.333 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-21T15:32:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 8 2024-05-21T15:32:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.335+0000 7f73ccb2c640 1 -- 172.21.15.38:0/678206621 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73b00a2c30 msgr2=0x7f73b00a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.335+0000 7f73ccb2c640 1 --2- 172.21.15.38:0/678206621 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73b00a2c30 0x7f73b00a5140 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f73c81025b0 tx=0x7f73b8005f90 comp rx=0 tx=0).stop 2024-05-21T15:32:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.335+0000 7f73ccb2c640 1 -- 172.21.15.38:0/678206621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c8102bf0 msgr2=0x7f73c8198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.335+0000 7f73ccb2c640 1 --2- 172.21.15.38:0/678206621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c8102bf0 0x7f73c8198610 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73bc02ade0 tx=0x7f73bc031ee0 comp rx=0 tx=0).stop 2024-05-21T15:32:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.335+0000 7f73ccb2c640 1 -- 172.21.15.38:0/678206621 shutdown_connections 2024-05-21T15:32:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.335+0000 7f73ccb2c640 1 --2- 172.21.15.38:0/678206621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c81078f0 0x7f73c819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.335+0000 7f73ccb2c640 1 --2- 172.21.15.38:0/678206621 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73b00a2c30 0x7f73b00a5140 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.335+0000 7f73ccb2c640 1 --2- 172.21.15.38:0/678206621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73c8102bf0 0x7f73c8198610 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.335+0000 7f73ccb2c640 1 --2- 172.21.15.38:0/678206621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73c81019b0 0x7f73c81980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.335+0000 7f73ccb2c640 1 -- 172.21.15.38:0/678206621 >> 172.21.15.38:0/678206621 conn(0x7f73c80fcba0 msgr2=0x7f73c81038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.335+0000 7f73ccb2c640 1 -- 172.21.15.38:0/678206621 shutdown_connections 2024-05-21T15:32:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:15.335+0000 7f73ccb2c640 1 -- 172.21.15.38:0/678206621 wait complete. 2024-05-21T15:32:15.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:15 smithi045 ceph-mon[36409]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:15.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:15 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:15.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:15 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:15.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:15 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:15.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:15 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:15.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:15 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:15.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:15 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:15.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:15 smithi045 ceph-mon[36409]: Deploying daemon mds.h on smithi045 2024-05-21T15:32:15.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:15 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] up:boot 2024-05-21T15:32:15.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:15 smithi045 ceph-mon[36409]: fsmap 7 up:standby 2024-05-21T15:32:15.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:15 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:32:15.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:15 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/678206621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:15.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:15 smithi038 ceph-mon[31995]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:15.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:15 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:15.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:15 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:15.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:15 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:15.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:15 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:15.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:15 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:15.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:15 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:15.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:15 smithi038 ceph-mon[31995]: Deploying daemon mds.h on smithi045 2024-05-21T15:32:15.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:15 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] up:boot 2024-05-21T15:32:15.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:15 smithi038 ceph-mon[31995]: fsmap 7 up:standby 2024-05-21T15:32:15.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:15 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:32:15.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:15 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/678206621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:15.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:15 smithi053 ceph-mon[36496]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:15.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:15 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:15.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:15 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:15.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:15 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:15.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:15 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:15.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:15 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:15.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:15 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:15.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:15 smithi053 ceph-mon[36496]: Deploying daemon mds.h on smithi045 2024-05-21T15:32:15.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:15 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] up:boot 2024-05-21T15:32:15.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:15 smithi053 ceph-mon[36496]: fsmap 7 up:standby 2024-05-21T15:32:15.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:15 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:32:15.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:15 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/678206621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:17.005 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:32:16 smithi045 systemd[1]: Starting Ceph mds.h for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:32:17.261 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:32:17 smithi045 podman[66122]: 2024-05-21 15:32:16.90782487 +0000 UTC m=+0.017810359 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:32:17.261 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:32:17 smithi045 podman[66122]: 2024-05-21 15:32:17.03952487 +0000 UTC m=+0.149510361 container create d464e15f31c2e862e20ad622f6f453aea6893fb62cab5d1646e2cac7850e9cf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-h, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:32:17.347 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:32:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.477+0000 7f20b85f5640 1 -- 172.21.15.38:0/3479525591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b01019b0 msgr2=0x7f20b01026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.477+0000 7f20b85f5640 1 --2- 172.21.15.38:0/3479525591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b01019b0 0x7f20b01026b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009900 tx=0x7f20ac02a410 comp rx=0 tx=0).stop 2024-05-21T15:32:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.477+0000 7f20b85f5640 1 -- 172.21.15.38:0/3479525591 shutdown_connections 2024-05-21T15:32:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.477+0000 7f20b85f5640 1 --2- 172.21.15.38:0/3479525591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b01078f0 0x7f20b0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.477+0000 7f20b85f5640 1 --2- 172.21.15.38:0/3479525591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20b0102bf0 0x7f20b0103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.477+0000 7f20b85f5640 1 --2- 172.21.15.38:0/3479525591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b01019b0 0x7f20b01026b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.477+0000 7f20b85f5640 1 -- 172.21.15.38:0/3479525591 >> 172.21.15.38:0/3479525591 conn(0x7f20b00fcba0 msgr2=0x7f20b00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.477+0000 7f20b85f5640 1 -- 172.21.15.38:0/3479525591 shutdown_connections 2024-05-21T15:32:17.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.477+0000 7f20b85f5640 1 -- 172.21.15.38:0/3479525591 wait complete. 2024-05-21T15:32:17.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.478+0000 7f20b85f5640 1 Processor -- start 2024-05-21T15:32:17.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.478+0000 7f20b85f5640 1 -- start start 2024-05-21T15:32:17.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.478+0000 7f20b85f5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b01019b0 0x7f20b01991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:17.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.478+0000 7f20b85f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b0102bf0 0x7f20b0199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:17.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.478+0000 7f20b85f5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20b01078f0 0x7f20b01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:17.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.478+0000 7f20b85f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20b010e730 con 0x7f20b0102bf0 2024-05-21T15:32:17.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.478+0000 7f20b85f5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20b010e5b0 con 0x7f20b01019b0 2024-05-21T15:32:17.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.478+0000 7f20b85f5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f20b010e8b0 con 0x7f20b01078f0 2024-05-21T15:32:17.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.478+0000 7f20b5b90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b0102bf0 0x7f20b0199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:17.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.478+0000 7f20b5b90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b0102bf0 0x7f20b0199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50856/0 (socket says 172.21.15.38:50856) 2024-05-21T15:32:17.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.478+0000 7f20b5b90640 1 -- 172.21.15.38:0/2113512939 learned_addr learned my addr 172.21.15.38:0/2113512939 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:17.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.479+0000 7f20b6391640 1 --2- 172.21.15.38:0/2113512939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b01019b0 0x7f20b01991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:17.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.479+0000 7f20b6b92640 1 --2- 172.21.15.38:0/2113512939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20b01078f0 0x7f20b01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:17.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.479+0000 7f20b5b90640 1 -- 172.21.15.38:0/2113512939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20b01078f0 msgr2=0x7f20b01a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:17.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.479+0000 7f20b5b90640 1 --2- 172.21.15.38:0/2113512939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20b01078f0 0x7f20b01a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:17.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.479+0000 7f20b5b90640 1 -- 172.21.15.38:0/2113512939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b01019b0 msgr2=0x7f20b01991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:17.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.479+0000 7f20b5b90640 1 --2- 172.21.15.38:0/2113512939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b01019b0 0x7f20b01991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:17.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.479+0000 7f20b5b90640 1 -- 172.21.15.38:0/2113512939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b01a0d10 con 0x7f20b0102bf0 2024-05-21T15:32:17.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.479+0000 7f20b5b90640 1 --2- 172.21.15.38:0/2113512939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b0102bf0 0x7f20b0199710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f20a000d950 tx=0x7f20a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:17.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.479+0000 7f20a77fe640 1 -- 172.21.15.38:0/2113512939 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0017070 con 0x7f20b0102bf0 2024-05-21T15:32:17.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.479+0000 7f20a77fe640 1 -- 172.21.15.38:0/2113512939 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a0009c30 con 0x7f20b0102bf0 2024-05-21T15:32:17.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.479+0000 7f20b85f5640 1 -- 172.21.15.38:0/2113512939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b01a0fa0 con 0x7f20b0102bf0 2024-05-21T15:32:17.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.479+0000 7f20a77fe640 1 -- 172.21.15.38:0/2113512939 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a000f6b0 con 0x7f20b0102bf0 2024-05-21T15:32:17.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.479+0000 7f20b85f5640 1 -- 172.21.15.38:0/2113512939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b01a1460 con 0x7f20b0102bf0 2024-05-21T15:32:17.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.480+0000 7f20b85f5640 1 -- 172.21.15.38:0/2113512939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b0106700 con 0x7f20b0102bf0 2024-05-21T15:32:17.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.482+0000 7f20a77fe640 1 -- 172.21.15.38:0/2113512939 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f20a00109d0 con 0x7f20b0102bf0 2024-05-21T15:32:17.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.483+0000 7f20a77fe640 1 --2- 172.21.15.38:0/2113512939 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f209c0a2c30 0x7f209c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:17.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.483+0000 7f20b6391640 1 --2- 172.21.15.38:0/2113512939 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f209c0a2c30 0x7f209c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:17.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.483+0000 7f20a77fe640 1 -- 172.21.15.38:0/2113512939 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f20a00ce7e0 con 0x7f20b0102bf0 2024-05-21T15:32:17.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.483+0000 7f20b6391640 1 --2- 172.21.15.38:0/2113512939 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f209c0a2c30 0x7f209c0a5140 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f20ac005ab0 tx=0x7f20ac031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:17.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.487+0000 7f20a77fe640 1 -- 172.21.15.38:0/2113512939 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f20a0009de0 con 0x7f20b0102bf0 2024-05-21T15:32:17.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.716+0000 7f20b85f5640 1 -- 172.21.15.38:0/2113512939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20b002d2a0 con 0x7f20b0102bf0 2024-05-21T15:32:17.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.716+0000 7f20a77fe640 1 -- 172.21.15.38:0/2113512939 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7165 (secure 0 0 0) 0x7f20a009bbc0 con 0x7f20b0102bf0 2024-05-21T15:32:17.717 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:17.717 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24551,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-21T15:32:17.717 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-05-21T15:32:17.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.718+0000 7f20b85f5640 1 -- 172.21.15.38:0/2113512939 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f209c0a2c30 msgr2=0x7f209c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:17.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.718+0000 7f20b85f5640 1 --2- 172.21.15.38:0/2113512939 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f209c0a2c30 0x7f209c0a5140 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f20ac005ab0 tx=0x7f20ac031400 comp rx=0 tx=0).stop 2024-05-21T15:32:17.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.718+0000 7f20b85f5640 1 -- 172.21.15.38:0/2113512939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b0102bf0 msgr2=0x7f20b0199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:17.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.718+0000 7f20b85f5640 1 --2- 172.21.15.38:0/2113512939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b0102bf0 0x7f20b0199710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f20a000d950 tx=0x7f20a000de10 comp rx=0 tx=0).stop 2024-05-21T15:32:17.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.718+0000 7f20b85f5640 1 -- 172.21.15.38:0/2113512939 shutdown_connections 2024-05-21T15:32:17.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.718+0000 7f20b85f5640 1 --2- 172.21.15.38:0/2113512939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20b01078f0 0x7f20b01a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:17.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.718+0000 7f20b85f5640 1 --2- 172.21.15.38:0/2113512939 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f209c0a2c30 0x7f209c0a5140 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:17.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.718+0000 7f20b85f5640 1 --2- 172.21.15.38:0/2113512939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b0102bf0 0x7f20b0199710 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:17.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.718+0000 7f20b85f5640 1 --2- 172.21.15.38:0/2113512939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b01019b0 0x7f20b01991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:17.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.718+0000 7f20b85f5640 1 -- 172.21.15.38:0/2113512939 >> 172.21.15.38:0/2113512939 conn(0x7f20b00fcba0 msgr2=0x7f20b01038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:17.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.718+0000 7f20b85f5640 1 -- 172.21.15.38:0/2113512939 shutdown_connections 2024-05-21T15:32:17.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:17.718+0000 7f20b85f5640 1 -- 172.21.15.38:0/2113512939 wait complete. 2024-05-21T15:32:17.729 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:17 smithi038 ceph-mon[31995]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:17.729 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:17.729 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:17 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:17.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:17 smithi045 ceph-mon[36409]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:17.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:17.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:17 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:17.762 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:32:17 smithi045 podman[66122]: 2024-05-21 15:32:17.349892304 +0000 UTC m=+0.459877793 container init d464e15f31c2e862e20ad622f6f453aea6893fb62cab5d1646e2cac7850e9cf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-h, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:32:17.762 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:32:17 smithi045 podman[66122]: 2024-05-21 15:32:17.352724723 +0000 UTC m=+0.462710220 container start d464e15f31c2e862e20ad622f6f453aea6893fb62cab5d1646e2cac7850e9cf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-h, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-21T15:32:17.762 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:32:17 smithi045 ceph-mds[66141]: -- 172.21.15.45:0/348583240 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5582c2041a40 con 0x5582c20b9180 2024-05-21T15:32:17.762 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:32:17 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-h[66137]: starting mds.h at 2024-05-21T15:32:17.762 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:32:17 smithi045 bash[66122]: d464e15f31c2e862e20ad622f6f453aea6893fb62cab5d1646e2cac7850e9cf4 2024-05-21T15:32:17.762 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:32:17 smithi045 systemd[1]: Started Ceph mds.h for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:32:17.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:17 smithi053 ceph-mon[36496]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:17.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:17.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:17 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:18.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:18 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] up:boot 2024-05-21T15:32:18.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:18 smithi053 ceph-mon[36496]: fsmap 8 up:standby 2024-05-21T15:32:18.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:18 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:32:18.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:18 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:18.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:18 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:18.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:18 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:18.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:18 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:18.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:18 smithi053 ceph-mon[36496]: Deploying daemon mds.i on smithi053 2024-05-21T15:32:18.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:18 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2113512939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:19.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:18 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] up:boot 2024-05-21T15:32:19.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:18 smithi045 ceph-mon[36409]: fsmap 8 up:standby 2024-05-21T15:32:19.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:18 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:32:19.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:18 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:19.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:18 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:19.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:18 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:19.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:18 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:19.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:18 smithi045 ceph-mon[36409]: Deploying daemon mds.i on smithi053 2024-05-21T15:32:19.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:18 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2113512939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:19.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:18 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] up:boot 2024-05-21T15:32:19.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:18 smithi038 ceph-mon[31995]: fsmap 8 up:standby 2024-05-21T15:32:19.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:18 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:32:19.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:18 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:19.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:18 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:19.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:18 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:19.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:18 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:19.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:18 smithi038 ceph-mon[31995]: Deploying daemon mds.i on smithi053 2024-05-21T15:32:19.026 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:18 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2113512939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:19.730 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:32:19.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.860+0000 7fd985e4a640 1 -- 172.21.15.38:0/1830303902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9801035b0 msgr2=0x7fd9801059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:19.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.860+0000 7fd985e4a640 1 --2- 172.21.15.38:0/1830303902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9801035b0 0x7fd9801059d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd970009900 tx=0x7fd97002a410 comp rx=0 tx=0).stop 2024-05-21T15:32:19.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.860+0000 7fd985e4a640 1 -- 172.21.15.38:0/1830303902 shutdown_connections 2024-05-21T15:32:19.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.860+0000 7fd985e4a640 1 --2- 172.21.15.38:0/1830303902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd980108870 0x7fd98010ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:19.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.860+0000 7fd985e4a640 1 --2- 172.21.15.38:0/1830303902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd980105f10 0x7fd980108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:19.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.860+0000 7fd985e4a640 1 --2- 172.21.15.38:0/1830303902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9801035b0 0x7fd9801059d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:19.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.860+0000 7fd985e4a640 1 -- 172.21.15.38:0/1830303902 >> 172.21.15.38:0/1830303902 conn(0x7fd9800fcbd0 msgr2=0x7fd9800ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:19.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.860+0000 7fd985e4a640 1 -- 172.21.15.38:0/1830303902 shutdown_connections 2024-05-21T15:32:19.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.860+0000 7fd985e4a640 1 -- 172.21.15.38:0/1830303902 wait complete. 2024-05-21T15:32:19.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.860+0000 7fd985e4a640 1 Processor -- start 2024-05-21T15:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd985e4a640 1 -- start start 2024-05-21T15:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd985e4a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9801035b0 0x7fd980199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd985e4a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd980105f10 0x7fd980199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd985e4a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd980108870 0x7fd9801a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd985e4a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd98010e710 con 0x7fd9801035b0 2024-05-21T15:32:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd985e4a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd98010e590 con 0x7fd980108870 2024-05-21T15:32:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd985e4a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd98010e890 con 0x7fd980105f10 2024-05-21T15:32:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd97f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9801035b0 0x7fd980199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd97f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9801035b0 0x7fd980199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50880/0 (socket says 172.21.15.38:50880) 2024-05-21T15:32:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd97f7fe640 1 -- 172.21.15.38:0/2783781481 learned_addr learned my addr 172.21.15.38:0/2783781481 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd97ffff640 1 --2- 172.21.15.38:0/2783781481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd980108870 0x7fd9801a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd97effd640 1 --2- 172.21.15.38:0/2783781481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd980105f10 0x7fd980199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd97f7fe640 1 -- 172.21.15.38:0/2783781481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd980105f10 msgr2=0x7fd980199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd97f7fe640 1 --2- 172.21.15.38:0/2783781481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd980105f10 0x7fd980199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd97f7fe640 1 -- 172.21.15.38:0/2783781481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd980108870 msgr2=0x7fd9801a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd97f7fe640 1 --2- 172.21.15.38:0/2783781481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd980108870 0x7fd9801a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.861+0000 7fd97f7fe640 1 -- 172.21.15.38:0/2783781481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9801a0d40 con 0x7fd9801035b0 2024-05-21T15:32:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.862+0000 7fd97f7fe640 1 --2- 172.21.15.38:0/2783781481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9801035b0 0x7fd980199200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd970006010 tx=0x7fd9700043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:19.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.862+0000 7fd97cff9640 1 -- 172.21.15.38:0/2783781481 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97003f030 con 0x7fd9801035b0 2024-05-21T15:32:19.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.862+0000 7fd985e4a640 1 -- 172.21.15.38:0/2783781481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9801a0fd0 con 0x7fd9801035b0 2024-05-21T15:32:19.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.862+0000 7fd97cff9640 1 -- 172.21.15.38:0/2783781481 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd970003ce0 con 0x7fd9801035b0 2024-05-21T15:32:19.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.862+0000 7fd97cff9640 1 -- 172.21.15.38:0/2783781481 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970035070 con 0x7fd9801035b0 2024-05-21T15:32:19.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.862+0000 7fd985e4a640 1 -- 172.21.15.38:0/2783781481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9801a1430 con 0x7fd9801035b0 2024-05-21T15:32:19.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.863+0000 7fd985e4a640 1 -- 172.21.15.38:0/2783781481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9800fe6f0 con 0x7fd9801035b0 2024-05-21T15:32:19.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.864+0000 7fd97cff9640 1 -- 172.21.15.38:0/2783781481 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd970042030 con 0x7fd9801035b0 2024-05-21T15:32:19.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.865+0000 7fd97cff9640 1 --2- 172.21.15.38:0/2783781481 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd9680a2c30 0x7fd9680a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:19.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.865+0000 7fd97effd640 1 --2- 172.21.15.38:0/2783781481 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd9680a2c30 0x7fd9680a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:19.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.865+0000 7fd97cff9640 1 -- 172.21.15.38:0/2783781481 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd9700ef380 con 0x7fd9801035b0 2024-05-21T15:32:19.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.865+0000 7fd97effd640 1 --2- 172.21.15.38:0/2783781481 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd9680a2c30 0x7fd9680a5140 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd98019a2a0 tx=0x7fd97400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:19.868 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:19 smithi053 ceph-mon[36496]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:19.868 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:32:19 smithi053 systemd[1]: Starting Ceph mds.i for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:19.868+0000 7fd97cff9640 1 -- 172.21.15.38:0/2783781481 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd9700bc090 con 0x7fd9801035b0 2024-05-21T15:32:20.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:19 smithi045 ceph-mon[36409]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:20.019 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:19 smithi038 ceph-mon[31995]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:20.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:20.094+0000 7fd985e4a640 1 -- 172.21.15.38:0/2783781481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd98002d2a0 con 0x7fd9801035b0 2024-05-21T15:32:20.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:20.094+0000 7fd97cff9640 1 -- 172.21.15.38:0/2783781481 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7165 (secure 0 0 0) 0x7fd9700c0390 con 0x7fd9801035b0 2024-05-21T15:32:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:20.095 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24551,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-21T15:32:20.095 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-05-21T15:32:20.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:20.096+0000 7fd985e4a640 1 -- 172.21.15.38:0/2783781481 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd9680a2c30 msgr2=0x7fd9680a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:20.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:20.096+0000 7fd985e4a640 1 --2- 172.21.15.38:0/2783781481 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd9680a2c30 0x7fd9680a5140 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd98019a2a0 tx=0x7fd97400b390 comp rx=0 tx=0).stop 2024-05-21T15:32:20.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:20.096+0000 7fd985e4a640 1 -- 172.21.15.38:0/2783781481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9801035b0 msgr2=0x7fd980199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:20.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:20.096+0000 7fd985e4a640 1 --2- 172.21.15.38:0/2783781481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9801035b0 0x7fd980199200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd970006010 tx=0x7fd9700043b0 comp rx=0 tx=0).stop 2024-05-21T15:32:20.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:20.096+0000 7fd985e4a640 1 -- 172.21.15.38:0/2783781481 shutdown_connections 2024-05-21T15:32:20.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:20.096+0000 7fd985e4a640 1 --2- 172.21.15.38:0/2783781481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd980108870 0x7fd9801a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:20.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:20.096+0000 7fd985e4a640 1 --2- 172.21.15.38:0/2783781481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd980105f10 0x7fd980199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:20.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:20.096+0000 7fd985e4a640 1 --2- 172.21.15.38:0/2783781481 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd9680a2c30 0x7fd9680a5140 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:20.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:20.096+0000 7fd985e4a640 1 --2- 172.21.15.38:0/2783781481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9801035b0 0x7fd980199200 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:20.096+0000 7fd985e4a640 1 -- 172.21.15.38:0/2783781481 >> 172.21.15.38:0/2783781481 conn(0x7fd9800fcbd0 msgr2=0x7fd980104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:20.097+0000 7fd985e4a640 1 -- 172.21.15.38:0/2783781481 shutdown_connections 2024-05-21T15:32:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:20.097+0000 7fd985e4a640 1 -- 172.21.15.38:0/2783781481 wait complete. 2024-05-21T15:32:20.161 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:32:19 smithi053 podman[66893]: 2024-05-21 15:32:19.771338804 +0000 UTC m=+0.018110432 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:32:20.161 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:32:19 smithi053 podman[66893]: 2024-05-21 15:32:19.883566902 +0000 UTC m=+0.130338538 container create 80998c82a6867763fcbc759064d96f920d2f6d4f9843065e5049910690fc5da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-i, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-21T15:32:20.161 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:32:20 smithi053 podman[66893]: 2024-05-21 15:32:20.161623822 +0000 UTC m=+0.408395427 container init 80998c82a6867763fcbc759064d96f920d2f6d4f9843065e5049910690fc5da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514) 2024-05-21T15:32:20.456 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:32:20 smithi053 podman[66893]: 2024-05-21 15:32:20.164301876 +0000 UTC m=+0.411073490 container start 80998c82a6867763fcbc759064d96f920d2f6d4f9843065e5049910690fc5da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-i, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-21T15:32:20.456 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:32:20 smithi053 ceph-mds[66912]: -- 172.21.15.53:0/2827018895 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be661fda40 con 0x55be66275600 2024-05-21T15:32:20.456 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:32:20 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-i[66908]: starting mds.i at 2024-05-21T15:32:20.456 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:32:20 smithi053 bash[66893]: 80998c82a6867763fcbc759064d96f920d2f6d4f9843065e5049910690fc5da5 2024-05-21T15:32:20.456 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:32:20 smithi053 systemd[1]: Started Ceph mds.i for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:32:20.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:20 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2783781481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:20.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:20 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:20.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:20 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:20.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:20 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:20.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:20 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:20.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:20 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:20.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:20 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:21.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:20 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2783781481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:21.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:20 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:21.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:20 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:21.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:20 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:21.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:20 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:21.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:20 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:21.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:20 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:21.021 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:20 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2783781481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:21.021 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:20 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:21.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:20 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:21.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:20 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:21.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:20 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:21.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:20 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:21.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:20 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:21.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:21 smithi053 ceph-mon[36496]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:21.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:21 smithi053 ceph-mon[36496]: Deploying daemon mds.g on smithi038 2024-05-21T15:32:21.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:21 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] up:boot 2024-05-21T15:32:21.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:21 smithi053 ceph-mon[36496]: fsmap 9 up:standby 2024-05-21T15:32:21.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:21 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:32:22.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:21 smithi045 ceph-mon[36409]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:22.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:21 smithi045 ceph-mon[36409]: Deploying daemon mds.g on smithi038 2024-05-21T15:32:22.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:21 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] up:boot 2024-05-21T15:32:22.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:21 smithi045 ceph-mon[36409]: fsmap 9 up:standby 2024-05-21T15:32:22.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:21 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:32:22.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:21 smithi038 ceph-mon[31995]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:22.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:21 smithi038 ceph-mon[31995]: Deploying daemon mds.g on smithi038 2024-05-21T15:32:22.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:21 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] up:boot 2024-05-21T15:32:22.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:21 smithi038 ceph-mon[31995]: fsmap 9 up:standby 2024-05-21T15:32:22.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:21 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:32:22.109 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:32:22.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.411+0000 7fb940996640 1 -- 172.21.15.38:0/3482346567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c103d00 msgr2=0x7fb93c104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:22.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.411+0000 7fb940996640 1 --2- 172.21.15.38:0/3482346567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c103d00 0x7fb93c104110 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb924009900 tx=0x7fb92402a3f0 comp rx=0 tx=0).stop 2024-05-21T15:32:22.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.411+0000 7fb940996640 1 -- 172.21.15.38:0/3482346567 shutdown_connections 2024-05-21T15:32:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.411+0000 7fb940996640 1 --2- 172.21.15.38:0/3482346567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb93c108a00 0x7fb93c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.411+0000 7fb940996640 1 --2- 172.21.15.38:0/3482346567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c103d00 0x7fb93c104110 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.411+0000 7fb940996640 1 --2- 172.21.15.38:0/3482346567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb93c102ac0 0x7fb93c1037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.411+0000 7fb940996640 1 -- 172.21.15.38:0/3482346567 >> 172.21.15.38:0/3482346567 conn(0x7fb93c0fdcb0 msgr2=0x7fb93c100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.411+0000 7fb940996640 1 -- 172.21.15.38:0/3482346567 shutdown_connections 2024-05-21T15:32:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.411+0000 7fb940996640 1 -- 172.21.15.38:0/3482346567 wait complete. 2024-05-21T15:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb940996640 1 Processor -- start 2024-05-21T15:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb940996640 1 -- start start 2024-05-21T15:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb940996640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb93c102ac0 0x7fb93c075800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb940996640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb93c103d00 0x7fb93c073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb940996640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c108a00 0x7fb93c0743e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb940996640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c10e710 con 0x7fb93c108a00 2024-05-21T15:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb940996640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c10e590 con 0x7fb93c102ac0 2024-05-21T15:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb940996640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c10e890 con 0x7fb93c103d00 2024-05-21T15:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb93ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c108a00 0x7fb93c0743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb93ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c108a00 0x7fb93c0743e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50884/0 (socket says 172.21.15.38:50884) 2024-05-21T15:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb93ad9d640 1 -- 172.21.15.38:0/1472183790 learned_addr learned my addr 172.21.15.38:0/1472183790 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb93a59c640 1 --2- 172.21.15.38:0/1472183790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb93c102ac0 0x7fb93c075800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb931d9b640 1 --2- 172.21.15.38:0/1472183790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb93c103d00 0x7fb93c073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb93ad9d640 1 -- 172.21.15.38:0/1472183790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb93c103d00 msgr2=0x7fb93c073ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.412+0000 7fb93ad9d640 1 --2- 172.21.15.38:0/1472183790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb93c103d00 0x7fb93c073ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.413+0000 7fb93ad9d640 1 -- 172.21.15.38:0/1472183790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb93c102ac0 msgr2=0x7fb93c075800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.413+0000 7fb93ad9d640 1 --2- 172.21.15.38:0/1472183790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb93c102ac0 0x7fb93c075800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.413+0000 7fb93ad9d640 1 -- 172.21.15.38:0/1472183790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c074a80 con 0x7fb93c108a00 2024-05-21T15:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.413+0000 7fb93ad9d640 1 --2- 172.21.15.38:0/1472183790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c108a00 0x7fb93c0743e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb93400cce0 tx=0x7fb93400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.413+0000 7fb933fff640 1 -- 172.21.15.38:0/1472183790 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93400f450 con 0x7fb93c108a00 2024-05-21T15:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.413+0000 7fb940996640 1 -- 172.21.15.38:0/1472183790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93c1a3240 con 0x7fb93c108a00 2024-05-21T15:32:22.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.413+0000 7fb933fff640 1 -- 172.21.15.38:0/1472183790 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb93400fad0 con 0x7fb93c108a00 2024-05-21T15:32:22.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.413+0000 7fb933fff640 1 -- 172.21.15.38:0/1472183790 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934018c10 con 0x7fb93c108a00 2024-05-21T15:32:22.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.413+0000 7fb940996640 1 -- 172.21.15.38:0/1472183790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93c1a36a0 con 0x7fb93c108a00 2024-05-21T15:32:22.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.414+0000 7fb940996640 1 -- 172.21.15.38:0/1472183790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb93c107100 con 0x7fb93c108a00 2024-05-21T15:32:22.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.416+0000 7fb933fff640 1 -- 172.21.15.38:0/1472183790 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb93400f5f0 con 0x7fb93c108a00 2024-05-21T15:32:22.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.416+0000 7fb933fff640 1 --2- 172.21.15.38:0/1472183790 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9280a2c30 0x7fb9280a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:22.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.416+0000 7fb93a59c640 1 --2- 172.21.15.38:0/1472183790 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9280a2c30 0x7fb9280a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:22.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.416+0000 7fb933fff640 1 -- 172.21.15.38:0/1472183790 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb9340cf160 con 0x7fb93c108a00 2024-05-21T15:32:22.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.417+0000 7fb93a59c640 1 --2- 172.21.15.38:0/1472183790 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9280a2c30 0x7fb9280a5140 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb93c1036c0 tx=0x7fb92c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:22.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.424+0000 7fb933fff640 1 -- 172.21.15.38:0/1472183790 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb934014030 con 0x7fb93c108a00 2024-05-21T15:32:22.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.657+0000 7fb940996640 1 -- 172.21.15.38:0/1472183790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb93c02d2a0 con 0x7fb93c108a00 2024-05-21T15:32:22.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.658+0000 7fb933fff640 1 -- 172.21.15.38:0/1472183790 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7992 (secure 0 0 0) 0x7fb93409c560 con 0x7fb93c108a00 2024-05-21T15:32:22.658 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:22.659 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24551,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-21T15:32:22.659 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 10 2024-05-21T15:32:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.660+0000 7fb940996640 1 -- 172.21.15.38:0/1472183790 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9280a2c30 msgr2=0x7fb9280a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.660+0000 7fb940996640 1 --2- 172.21.15.38:0/1472183790 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9280a2c30 0x7fb9280a5140 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb93c1036c0 tx=0x7fb92c005f90 comp rx=0 tx=0).stop 2024-05-21T15:32:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.660+0000 7fb940996640 1 -- 172.21.15.38:0/1472183790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c108a00 msgr2=0x7fb93c0743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.660+0000 7fb940996640 1 --2- 172.21.15.38:0/1472183790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c108a00 0x7fb93c0743e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb93400cce0 tx=0x7fb93400a620 comp rx=0 tx=0).stop 2024-05-21T15:32:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.660+0000 7fb940996640 1 -- 172.21.15.38:0/1472183790 shutdown_connections 2024-05-21T15:32:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.660+0000 7fb940996640 1 --2- 172.21.15.38:0/1472183790 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9280a2c30 0x7fb9280a5140 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.660+0000 7fb940996640 1 --2- 172.21.15.38:0/1472183790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb93c108a00 0x7fb93c0743e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.660+0000 7fb940996640 1 --2- 172.21.15.38:0/1472183790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb93c103d00 0x7fb93c073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.660+0000 7fb940996640 1 --2- 172.21.15.38:0/1472183790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb93c102ac0 0x7fb93c075800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.660+0000 7fb940996640 1 -- 172.21.15.38:0/1472183790 >> 172.21.15.38:0/1472183790 conn(0x7fb93c0fdcb0 msgr2=0x7fb93c1000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.660+0000 7fb940996640 1 -- 172.21.15.38:0/1472183790 shutdown_connections 2024-05-21T15:32:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:22.660+0000 7fb940996640 1 -- 172.21.15.38:0/1472183790 wait complete. 2024-05-21T15:32:23.275 INFO:journalctl@ceph.mds.g.smithi038.stdout:May 21 15:32:23 smithi038 systemd[1]: Started Ceph mds.g for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:32:23.860 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:23 smithi038 ceph-mon[31995]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:23.860 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:23 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1472183790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:23.860 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:23.860 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:23.860 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:23.860 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:23.860 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:23.860 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:23.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:23 smithi053 ceph-mon[36496]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:23.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:23 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1472183790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:23.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:23.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:23.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:23.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:23.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:23.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:24.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:23 smithi045 ceph-mon[36409]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:24.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:23 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1472183790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:24.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:24.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:24.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:24.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:24.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:24.012 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:24.674 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:32:24.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.797+0000 7f22995a3640 1 -- 172.21.15.38:0/3733591910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2294107910 msgr2=0x7f2294109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:24.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.797+0000 7f22995a3640 1 --2- 172.21.15.38:0/3733591910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2294107910 0x7f2294109d30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f228c009900 tx=0x7f228c02a410 comp rx=0 tx=0).stop 2024-05-21T15:32:24.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.797+0000 7f22995a3640 1 -- 172.21.15.38:0/3733591910 shutdown_connections 2024-05-21T15:32:24.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.797+0000 7f22995a3640 1 --2- 172.21.15.38:0/3733591910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2294107910 0x7f2294109d30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:24.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.797+0000 7f22995a3640 1 --2- 172.21.15.38:0/3733591910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2294102c10 0x7f2294103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:24.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.797+0000 7f22995a3640 1 --2- 172.21.15.38:0/3733591910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22941019d0 0x7f22941026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:24.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.797+0000 7f22995a3640 1 -- 172.21.15.38:0/3733591910 >> 172.21.15.38:0/3733591910 conn(0x7f22940fcbe0 msgr2=0x7f22940ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:24.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.797+0000 7f22995a3640 1 -- 172.21.15.38:0/3733591910 shutdown_connections 2024-05-21T15:32:24.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.797+0000 7f22995a3640 1 -- 172.21.15.38:0/3733591910 wait complete. 2024-05-21T15:32:24.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.797+0000 7f22995a3640 1 Processor -- start 2024-05-21T15:32:24.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22995a3640 1 -- start start 2024-05-21T15:32:24.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22995a3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22941019d0 0x7f22940747f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:24.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22995a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2294102c10 0x7f229419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:24.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22995a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2294107910 0x7f22941a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:24.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22995a3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f229410d6a0 con 0x7f2294102c10 2024-05-21T15:32:24.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22995a3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f229410d520 con 0x7f22941019d0 2024-05-21T15:32:24.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22995a3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f229410d820 con 0x7f2294107910 2024-05-21T15:32:24.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22927fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2294102c10 0x7f229419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:24.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22927fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2294102c10 0x7f229419dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37946/0 (socket says 172.21.15.38:37946) 2024-05-21T15:32:24.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f2292ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22941019d0 0x7f22940747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:24.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22927fc640 1 -- 172.21.15.38:0/1693636511 learned_addr learned my addr 172.21.15.38:0/1693636511 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:24.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22937fe640 1 --2- 172.21.15.38:0/1693636511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2294107910 0x7f22941a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:24.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22927fc640 1 -- 172.21.15.38:0/1693636511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2294107910 msgr2=0x7f22941a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:24.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22927fc640 1 --2- 172.21.15.38:0/1693636511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2294107910 0x7f22941a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:24.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22927fc640 1 -- 172.21.15.38:0/1693636511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22941019d0 msgr2=0x7f22940747f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:24.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22927fc640 1 --2- 172.21.15.38:0/1693636511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22941019d0 0x7f22940747f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:24.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.798+0000 7f22927fc640 1 -- 172.21.15.38:0/1693636511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22941a0d30 con 0x7f2294102c10 2024-05-21T15:32:24.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.799+0000 7f22927fc640 1 --2- 172.21.15.38:0/1693636511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2294102c10 0x7f229419dfc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f228800eb90 tx=0x7f228800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:24.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.799+0000 7f227bfff640 1 -- 172.21.15.38:0/1693636511 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288004d20 con 0x7f2294102c10 2024-05-21T15:32:24.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.799+0000 7f227bfff640 1 -- 172.21.15.38:0/1693636511 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f228800ee90 con 0x7f2294102c10 2024-05-21T15:32:24.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.799+0000 7f227bfff640 1 -- 172.21.15.38:0/1693636511 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288010830 con 0x7f2294102c10 2024-05-21T15:32:24.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.799+0000 7f22995a3640 1 -- 172.21.15.38:0/1693636511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22941a0fc0 con 0x7f2294102c10 2024-05-21T15:32:24.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.799+0000 7f22995a3640 1 -- 172.21.15.38:0/1693636511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22941a1460 con 0x7f2294102c10 2024-05-21T15:32:24.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.800+0000 7f22995a3640 1 -- 172.21.15.38:0/1693636511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22940691c0 con 0x7f2294102c10 2024-05-21T15:32:24.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.801+0000 7f227bfff640 1 -- 172.21.15.38:0/1693636511 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2288005020 con 0x7f2294102c10 2024-05-21T15:32:24.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.802+0000 7f227bfff640 1 --2- 172.21.15.38:0/1693636511 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f227c0a2c30 0x7f227c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:24.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.802+0000 7f2292ffd640 1 --2- 172.21.15.38:0/1693636511 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f227c0a2c30 0x7f227c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:24.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.802+0000 7f227bfff640 1 -- 172.21.15.38:0/1693636511 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f22880cf190 con 0x7f2294102c10 2024-05-21T15:32:24.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.802+0000 7f2292ffd640 1 --2- 172.21.15.38:0/1693636511 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f227c0a2c30 0x7f227c0a5140 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2284005fd0 tx=0x7f228400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:24.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:24.807+0000 7f227bfff640 1 -- 172.21.15.38:0/1693636511 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f228809c590 con 0x7f2294102c10 2024-05-21T15:32:24.865 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:24 smithi038 ceph-mon[31995]: Deploying daemon mds.a on smithi038 2024-05-21T15:32:24.865 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:24 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.38:6838/2313367718,v1:172.21.15.38:6839/2313367718] up:boot 2024-05-21T15:32:24.865 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:24 smithi038 ceph-mon[31995]: fsmap 10 up:standby 2024-05-21T15:32:24.865 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:32:24.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:24 smithi053 ceph-mon[36496]: Deploying daemon mds.a on smithi038 2024-05-21T15:32:24.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:24 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.38:6838/2313367718,v1:172.21.15.38:6839/2313367718] up:boot 2024-05-21T15:32:24.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:24 smithi053 ceph-mon[36496]: fsmap 10 up:standby 2024-05-21T15:32:24.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:32:25.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:24 smithi045 ceph-mon[36409]: Deploying daemon mds.a on smithi038 2024-05-21T15:32:25.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:24 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.38:6838/2313367718,v1:172.21.15.38:6839/2313367718] up:boot 2024-05-21T15:32:25.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:24 smithi045 ceph-mon[36409]: fsmap 10 up:standby 2024-05-21T15:32:25.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:32:25.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:25.035+0000 7f22995a3640 1 -- 172.21.15.38:0/1693636511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2294106010 con 0x7f2294102c10 2024-05-21T15:32:25.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:25.036+0000 7f227bfff640 1 -- 172.21.15.38:0/1693636511 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8818 (secure 0 0 0) 0x7f228809be90 con 0x7f2294102c10 2024-05-21T15:32:25.036 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:25.037 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24551,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-21T15:32:25.037 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 11 2024-05-21T15:32:25.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:25.038+0000 7f22995a3640 1 -- 172.21.15.38:0/1693636511 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f227c0a2c30 msgr2=0x7f227c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:25.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:25.038+0000 7f22995a3640 1 --2- 172.21.15.38:0/1693636511 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f227c0a2c30 0x7f227c0a5140 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2284005fd0 tx=0x7f228400b410 comp rx=0 tx=0).stop 2024-05-21T15:32:25.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:25.038+0000 7f22995a3640 1 -- 172.21.15.38:0/1693636511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2294102c10 msgr2=0x7f229419dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:25.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:25.038+0000 7f22995a3640 1 --2- 172.21.15.38:0/1693636511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2294102c10 0x7f229419dfc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f228800eb90 tx=0x7f228800c6a0 comp rx=0 tx=0).stop 2024-05-21T15:32:25.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:25.038+0000 7f22995a3640 1 -- 172.21.15.38:0/1693636511 shutdown_connections 2024-05-21T15:32:25.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:25.038+0000 7f22995a3640 1 --2- 172.21.15.38:0/1693636511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2294107910 0x7f22941a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:25.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:25.038+0000 7f22995a3640 1 --2- 172.21.15.38:0/1693636511 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f227c0a2c30 0x7f227c0a5140 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:25.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:25.038+0000 7f22995a3640 1 --2- 172.21.15.38:0/1693636511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2294102c10 0x7f229419dfc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:25.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:25.038+0000 7f22995a3640 1 --2- 172.21.15.38:0/1693636511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22941019d0 0x7f22940747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:25.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:25.038+0000 7f22995a3640 1 -- 172.21.15.38:0/1693636511 >> 172.21.15.38:0/1693636511 conn(0x7f22940fcbe0 msgr2=0x7f22940ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:25.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:25.038+0000 7f22995a3640 1 -- 172.21.15.38:0/1693636511 shutdown_connections 2024-05-21T15:32:25.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:25.038+0000 7f22995a3640 1 -- 172.21.15.38:0/1693636511 wait complete. 2024-05-21T15:32:25.578 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:32:25 smithi038 systemd[1]: Starting Ceph mds.a for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:32:25.845 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:25 smithi038 ceph-mon[31995]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:25.845 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:25 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1693636511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:25.846 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:32:25 smithi038 podman[72879]: 2024-05-21 15:32:25.605870622 +0000 UTC m=+0.017825729 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:32:25.846 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:32:25 smithi038 podman[72879]: 2024-05-21 15:32:25.705766739 +0000 UTC m=+0.117721836 container create 0ddaa288b363cd79565a5a425fa0ca50890e2427213e605aae5247ee41170274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-05-21T15:32:25.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:25 smithi053 ceph-mon[36496]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:25.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:25 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1693636511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:26.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:25 smithi045 ceph-mon[36409]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:26.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:25 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1693636511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:26.276 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:32:25 smithi038 podman[72879]: 2024-05-21 15:32:25.975389122 +0000 UTC m=+0.387344217 container init 0ddaa288b363cd79565a5a425fa0ca50890e2427213e605aae5247ee41170274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514) 2024-05-21T15:32:26.398 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:32:25 smithi038 podman[72879]: 2024-05-21 15:32:25.978387246 +0000 UTC m=+0.390342346 container start 0ddaa288b363cd79565a5a425fa0ca50890e2427213e605aae5247ee41170274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-a, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-21T15:32:26.398 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:32:26 smithi038 ceph-mds[72898]: -- 172.21.15.38:0/1666243038 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c8e9347a40 con 0x55c8e93bf180 2024-05-21T15:32:26.398 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:32:26 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-a[72894]: starting mds.a at 2024-05-21T15:32:26.398 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:32:26 smithi038 bash[72879]: 0ddaa288b363cd79565a5a425fa0ca50890e2427213e605aae5247ee41170274 2024-05-21T15:32:26.398 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:32:26 smithi038 systemd[1]: Started Ceph mds.a for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:32:27.051 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:32:27.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.184+0000 7ff3fe88b640 1 -- 172.21.15.38:0/1973344667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3f8102bf0 msgr2=0x7ff3f8103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:27.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.184+0000 7ff3fe88b640 1 --2- 172.21.15.38:0/1973344667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3f8102bf0 0x7ff3f8103000 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009900 tx=0x7ff3e002a410 comp rx=0 tx=0).stop 2024-05-21T15:32:27.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.185+0000 7ff3fe88b640 1 -- 172.21.15.38:0/1973344667 shutdown_connections 2024-05-21T15:32:27.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.185+0000 7ff3fe88b640 1 --2- 172.21.15.38:0/1973344667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3f81078f0 0x7ff3f8109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:27.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.185+0000 7ff3fe88b640 1 --2- 172.21.15.38:0/1973344667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3f8102bf0 0x7ff3f8103000 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:27.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.185+0000 7ff3fe88b640 1 --2- 172.21.15.38:0/1973344667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3f81019b0 0x7ff3f81026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:27.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.185+0000 7ff3fe88b640 1 -- 172.21.15.38:0/1973344667 >> 172.21.15.38:0/1973344667 conn(0x7ff3f80fcba0 msgr2=0x7ff3f80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:27.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.185+0000 7ff3fe88b640 1 -- 172.21.15.38:0/1973344667 shutdown_connections 2024-05-21T15:32:27.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.185+0000 7ff3fe88b640 1 -- 172.21.15.38:0/1973344667 wait complete. 2024-05-21T15:32:27.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.185+0000 7ff3fe88b640 1 Processor -- start 2024-05-21T15:32:27.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.185+0000 7ff3fe88b640 1 -- start start 2024-05-21T15:32:27.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3fe88b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3f81019b0 0x7ff3f81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:27.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3fe88b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3f8102bf0 0x7ff3f8199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:27.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3fe88b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3f81078f0 0x7ff3f81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:27.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3fe88b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f810e790 con 0x7ff3f81019b0 2024-05-21T15:32:27.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3fe88b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f810e610 con 0x7ff3f8102bf0 2024-05-21T15:32:27.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3fe88b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f810e910 con 0x7ff3f81078f0 2024-05-21T15:32:27.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3f81019b0 0x7ff3f81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:27.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3f81019b0 0x7ff3f81991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37974/0 (socket says 172.21.15.38:37974) 2024-05-21T15:32:27.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3f7fff640 1 -- 172.21.15.38:0/217594427 learned_addr learned my addr 172.21.15.38:0/217594427 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:27.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3effff640 1 --2- 172.21.15.38:0/217594427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3f8102bf0 0x7ff3f8199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:27.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3fce28640 1 --2- 172.21.15.38:0/217594427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3f81078f0 0x7ff3f81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:27.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3f7fff640 1 -- 172.21.15.38:0/217594427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3f81078f0 msgr2=0x7ff3f81a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:27.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3f7fff640 1 --2- 172.21.15.38:0/217594427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3f81078f0 0x7ff3f81a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:27.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3f7fff640 1 -- 172.21.15.38:0/217594427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3f8102bf0 msgr2=0x7ff3f8199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:27.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3f7fff640 1 --2- 172.21.15.38:0/217594427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3f8102bf0 0x7ff3f8199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:27.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3f7fff640 1 -- 172.21.15.38:0/217594427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f81a0c20 con 0x7ff3f81019b0 2024-05-21T15:32:27.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3f7fff640 1 --2- 172.21.15.38:0/217594427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3f81019b0 0x7ff3f81991e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800c970 tx=0x7ff3e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:27.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3f5ffb640 1 -- 172.21.15.38:0/217594427 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8007c70 con 0x7ff3f81019b0 2024-05-21T15:32:27.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3f5ffb640 1 -- 172.21.15.38:0/217594427 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e8007e10 con 0x7ff3f81019b0 2024-05-21T15:32:27.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.186+0000 7ff3fe88b640 1 -- 172.21.15.38:0/217594427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f81a0f10 con 0x7ff3f81019b0 2024-05-21T15:32:27.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.187+0000 7ff3f5ffb640 1 -- 172.21.15.38:0/217594427 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e80188c0 con 0x7ff3f81019b0 2024-05-21T15:32:27.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.187+0000 7ff3fe88b640 1 -- 172.21.15.38:0/217594427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f81a13d0 con 0x7ff3f81019b0 2024-05-21T15:32:27.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.188+0000 7ff3fe88b640 1 -- 172.21.15.38:0/217594427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f8105ff0 con 0x7ff3f81019b0 2024-05-21T15:32:27.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.189+0000 7ff3f5ffb640 1 -- 172.21.15.38:0/217594427 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff3e800f450 con 0x7ff3f81019b0 2024-05-21T15:32:27.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.190+0000 7ff3f5ffb640 1 --2- 172.21.15.38:0/217594427 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3e40a2cb0 0x7ff3e40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:27.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.190+0000 7ff3effff640 1 --2- 172.21.15.38:0/217594427 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3e40a2cb0 0x7ff3e40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:27.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.190+0000 7ff3f5ffb640 1 -- 172.21.15.38:0/217594427 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff3e80ced90 con 0x7ff3f81019b0 2024-05-21T15:32:27.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.190+0000 7ff3effff640 1 --2- 172.21.15.38:0/217594427 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3e40a2cb0 0x7ff3e40a51c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0006010 tx=0x7ff3e0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:27.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.193+0000 7ff3f5ffb640 1 -- 172.21.15.38:0/217594427 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff3e8014030 con 0x7ff3f81019b0 2024-05-21T15:32:27.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.419+0000 7ff3fe88b640 1 -- 172.21.15.38:0/217594427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3f802d510 con 0x7ff3f81019b0 2024-05-21T15:32:27.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.419+0000 7ff3f5ffb640 1 -- 172.21.15.38:0/217594427 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9641 (secure 0 0 0) 0x7ff3e809c210 con 0x7ff3f81019b0 2024-05-21T15:32:27.420 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:27.420 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24551,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-21T15:32:27.420 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 12 2024-05-21T15:32:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.421+0000 7ff3fe88b640 1 -- 172.21.15.38:0/217594427 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3e40a2cb0 msgr2=0x7ff3e40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.421+0000 7ff3fe88b640 1 --2- 172.21.15.38:0/217594427 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3e40a2cb0 0x7ff3e40a51c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0006010 tx=0x7ff3e0031400 comp rx=0 tx=0).stop 2024-05-21T15:32:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.421+0000 7ff3fe88b640 1 -- 172.21.15.38:0/217594427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3f81019b0 msgr2=0x7ff3f81991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.421+0000 7ff3fe88b640 1 --2- 172.21.15.38:0/217594427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3f81019b0 0x7ff3f81991e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800c970 tx=0x7ff3e800ce30 comp rx=0 tx=0).stop 2024-05-21T15:32:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.421+0000 7ff3fe88b640 1 -- 172.21.15.38:0/217594427 shutdown_connections 2024-05-21T15:32:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.421+0000 7ff3fe88b640 1 --2- 172.21.15.38:0/217594427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3f81078f0 0x7ff3f81a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.421+0000 7ff3fe88b640 1 --2- 172.21.15.38:0/217594427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3f8102bf0 0x7ff3f8199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.421+0000 7ff3fe88b640 1 --2- 172.21.15.38:0/217594427 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3e40a2cb0 0x7ff3e40a51c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.421+0000 7ff3fe88b640 1 --2- 172.21.15.38:0/217594427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3f81019b0 0x7ff3f81991e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.421+0000 7ff3fe88b640 1 -- 172.21.15.38:0/217594427 >> 172.21.15.38:0/217594427 conn(0x7ff3f80fcba0 msgr2=0x7ff3f80ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.422+0000 7ff3fe88b640 1 -- 172.21.15.38:0/217594427 shutdown_connections 2024-05-21T15:32:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:27.422+0000 7ff3fe88b640 1 -- 172.21.15.38:0/217594427 wait complete. 2024-05-21T15:32:27.432 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:27.432 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:27.432 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:27 smithi038 ceph-mon[31995]: Deploying daemon mds.b on smithi045 2024-05-21T15:32:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:27 smithi038 ceph-mon[31995]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:27 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.38:6840/106491696,v1:172.21.15.38:6841/106491696] up:boot 2024-05-21T15:32:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:27 smithi038 ceph-mon[31995]: fsmap 11 up:standby 2024-05-21T15:32:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:27 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:32:27.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:27.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:27.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:27.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:27.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:27.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:27.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:27 smithi053 ceph-mon[36496]: Deploying daemon mds.b on smithi045 2024-05-21T15:32:27.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:27 smithi053 ceph-mon[36496]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:27.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:27 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.38:6840/106491696,v1:172.21.15.38:6841/106491696] up:boot 2024-05-21T15:32:27.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:27 smithi053 ceph-mon[36496]: fsmap 11 up:standby 2024-05-21T15:32:27.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:27 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:32:27.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:27.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:27.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:27.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:32:27.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:32:27.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:27.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:27 smithi045 ceph-mon[36409]: Deploying daemon mds.b on smithi045 2024-05-21T15:32:27.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:27 smithi045 ceph-mon[36409]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:27.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:27 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.38:6840/106491696,v1:172.21.15.38:6841/106491696] up:boot 2024-05-21T15:32:27.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:27 smithi045 ceph-mon[36409]: fsmap 11 up:standby 2024-05-21T15:32:27.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:27 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:32:28.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:28 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/217594427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:28.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:28 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/217594427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:28.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:28 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/217594427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:29.435 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:32:29.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:29 smithi053 ceph-mon[36496]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:29.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:29 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:29.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:29 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:29.459 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:29 smithi038 ceph-mon[31995]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:29.459 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:29 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:29.459 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:29 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:29.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:29 smithi045 ceph-mon[36409]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:29.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:29 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:29.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:29 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:29.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.557+0000 7f17d2338640 1 -- 172.21.15.38:0/2035506101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17cc102bf0 msgr2=0x7f17cc103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:29.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.557+0000 7f17d2338640 1 --2- 172.21.15.38:0/2035506101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17cc102bf0 0x7f17cc103000 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009900 tx=0x7f17c002a410 comp rx=0 tx=0).stop 2024-05-21T15:32:29.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.557+0000 7f17d2338640 1 -- 172.21.15.38:0/2035506101 shutdown_connections 2024-05-21T15:32:29.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.557+0000 7f17d2338640 1 --2- 172.21.15.38:0/2035506101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17cc1078f0 0x7f17cc109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.557+0000 7f17d2338640 1 --2- 172.21.15.38:0/2035506101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17cc102bf0 0x7f17cc103000 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.557+0000 7f17d2338640 1 --2- 172.21.15.38:0/2035506101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17cc1019b0 0x7f17cc1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.557+0000 7f17d2338640 1 -- 172.21.15.38:0/2035506101 >> 172.21.15.38:0/2035506101 conn(0x7f17cc0fcba0 msgr2=0x7f17cc0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:29.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.557+0000 7f17d2338640 1 -- 172.21.15.38:0/2035506101 shutdown_connections 2024-05-21T15:32:29.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.557+0000 7f17d2338640 1 -- 172.21.15.38:0/2035506101 wait complete. 2024-05-21T15:32:29.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.557+0000 7f17d2338640 1 Processor -- start 2024-05-21T15:32:29.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17d2338640 1 -- start start 2024-05-21T15:32:29.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17d2338640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17cc1019b0 0x7f17cc1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:29.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17d2338640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17cc102bf0 0x7f17cc199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:29.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17d2338640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17cc1078f0 0x7f17cc1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:29.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17d2338640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc10d620 con 0x7f17cc1019b0 2024-05-21T15:32:29.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17d2338640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc10d4a0 con 0x7f17cc1078f0 2024-05-21T15:32:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17d2338640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc10d7a0 con 0x7f17cc102bf0 2024-05-21T15:32:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17cbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17cc1019b0 0x7f17cc1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17cbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17cc1019b0 0x7f17cc1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37978/0 (socket says 172.21.15.38:37978) 2024-05-21T15:32:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17cbfff640 1 -- 172.21.15.38:0/2520882396 learned_addr learned my addr 172.21.15.38:0/2520882396 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17d08d5640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17cc1078f0 0x7f17cc1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17cb7fe640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17cc102bf0 0x7f17cc199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17cbfff640 1 -- 172.21.15.38:0/2520882396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17cc102bf0 msgr2=0x7f17cc199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17cbfff640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17cc102bf0 0x7f17cc199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17cbfff640 1 -- 172.21.15.38:0/2520882396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17cc1078f0 msgr2=0x7f17cc1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:29.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17cbfff640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17cc1078f0 0x7f17cc1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17cbfff640 1 -- 172.21.15.38:0/2520882396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc1a0d30 con 0x7f17cc1019b0 2024-05-21T15:32:29.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.558+0000 7f17d08d5640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17cc1078f0 0x7f17cc1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:32:29.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.559+0000 7f17cbfff640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17cc1019b0 0x7f17cc1991f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00e9f0 tx=0x7f17bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:29.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.559+0000 7f17c97fa640 1 -- 172.21.15.38:0/2520882396 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00ccc0 con 0x7f17cc1019b0 2024-05-21T15:32:29.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.559+0000 7f17d2338640 1 -- 172.21.15.38:0/2520882396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc1a0fc0 con 0x7f17cc1019b0 2024-05-21T15:32:29.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.559+0000 7f17c97fa640 1 -- 172.21.15.38:0/2520882396 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17bc0035f0 con 0x7f17cc1019b0 2024-05-21T15:32:29.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.559+0000 7f17c97fa640 1 -- 172.21.15.38:0/2520882396 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00ce60 con 0x7f17cc1019b0 2024-05-21T15:32:29.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.559+0000 7f17d2338640 1 -- 172.21.15.38:0/2520882396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc1a1430 con 0x7f17cc1019b0 2024-05-21T15:32:29.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.560+0000 7f17d2338640 1 -- 172.21.15.38:0/2520882396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17cc1a16f0 con 0x7f17cc1019b0 2024-05-21T15:32:29.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.562+0000 7f17c97fa640 1 -- 172.21.15.38:0/2520882396 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f17bc003b10 con 0x7f17cc1019b0 2024-05-21T15:32:29.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.562+0000 7f17c97fa640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f17b40a2c30 0x7f17b40a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:29.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.563+0000 7f17c97fa640 1 -- 172.21.15.38:0/2520882396 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f17bc0cefe0 con 0x7f17cc1019b0 2024-05-21T15:32:29.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.563+0000 7f17cb7fe640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f17b40a2c30 0x7f17b40a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:29.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.563+0000 7f17cb7fe640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f17b40a2c30 0x7f17b40a5140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009230 tx=0x7f17c0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:29.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.566+0000 7f17c97fa640 1 -- 172.21.15.38:0/2520882396 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f17bc09c380 con 0x7f17cc1019b0 2024-05-21T15:32:29.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.793+0000 7f17d2338640 1 -- 172.21.15.38:0/2520882396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17cc1a18c0 con 0x7f17cc1019b0 2024-05-21T15:32:29.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.793+0000 7f17c97fa640 1 -- 172.21.15.38:0/2520882396 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10467 (secure 0 0 0) 0x7f17bc09bc80 con 0x7f17cc1019b0 2024-05-21T15:32:29.794 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:29.794 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24551,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-21T15:32:29.794 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 13 2024-05-21T15:32:29.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.795+0000 7f17d2338640 1 -- 172.21.15.38:0/2520882396 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f17b40a2c30 msgr2=0x7f17b40a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:29.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.795+0000 7f17d2338640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f17b40a2c30 0x7f17b40a5140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009230 tx=0x7f17c0031400 comp rx=0 tx=0).stop 2024-05-21T15:32:29.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.796+0000 7f17d2338640 1 -- 172.21.15.38:0/2520882396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17cc1019b0 msgr2=0x7f17cc1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:29.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.796+0000 7f17d2338640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17cc1019b0 0x7f17cc1991f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00e9f0 tx=0x7f17bc00edf0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.796+0000 7f17d2338640 1 -- 172.21.15.38:0/2520882396 shutdown_connections 2024-05-21T15:32:29.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.796+0000 7f17d2338640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17cc1078f0 0x7f17cc1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.796+0000 7f17d2338640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17cc102bf0 0x7f17cc199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.796+0000 7f17d2338640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f17b40a2c30 0x7f17b40a5140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.796+0000 7f17d2338640 1 --2- 172.21.15.38:0/2520882396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17cc1019b0 0x7f17cc1991f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.796+0000 7f17d2338640 1 -- 172.21.15.38:0/2520882396 >> 172.21.15.38:0/2520882396 conn(0x7f17cc0fcba0 msgr2=0x7f17cc108c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:29.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.796+0000 7f17d2338640 1 -- 172.21.15.38:0/2520882396 shutdown_connections 2024-05-21T15:32:29.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.796+0000 7f17d2338640 1 -- 172.21.15.38:0/2520882396 wait complete. 2024-05-21T15:32:29.808 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:32:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.969+0000 7f5aed626640 1 -- 172.21.15.38:0/2469129428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ae81035b0 msgr2=0x7f5ae81059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.969+0000 7f5aed626640 1 --2- 172.21.15.38:0/2469129428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ae81035b0 0x7f5ae81059d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009900 tx=0x7f5ad802a410 comp rx=0 tx=0).stop 2024-05-21T15:32:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.969+0000 7f5aed626640 1 -- 172.21.15.38:0/2469129428 shutdown_connections 2024-05-21T15:32:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.969+0000 7f5aed626640 1 --2- 172.21.15.38:0/2469129428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ae8108870 0x7f5ae810ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.969+0000 7f5aed626640 1 --2- 172.21.15.38:0/2469129428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ae8105f10 0x7f5ae8108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.969+0000 7f5aed626640 1 --2- 172.21.15.38:0/2469129428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ae81035b0 0x7f5ae81059d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.969+0000 7f5aed626640 1 -- 172.21.15.38:0/2469129428 >> 172.21.15.38:0/2469129428 conn(0x7f5ae80fcbb0 msgr2=0x7f5ae80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.969+0000 7f5aed626640 1 -- 172.21.15.38:0/2469129428 shutdown_connections 2024-05-21T15:32:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.969+0000 7f5aed626640 1 -- 172.21.15.38:0/2469129428 wait complete. 2024-05-21T15:32:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.970+0000 7f5aed626640 1 Processor -- start 2024-05-21T15:32:29.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.970+0000 7f5aed626640 1 -- start start 2024-05-21T15:32:29.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.970+0000 7f5aed626640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ae81035b0 0x7f5ae81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.970+0000 7f5aed626640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ae8105f10 0x7f5ae8199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.970+0000 7f5aed626640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ae8108870 0x7f5ae81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.970+0000 7f5aed626640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae810e760 con 0x7f5ae8108870 2024-05-21T15:32:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.970+0000 7f5aed626640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae810e5e0 con 0x7f5ae81035b0 2024-05-21T15:32:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.970+0000 7f5aed626640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae810e8e0 con 0x7f5ae8105f10 2024-05-21T15:32:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.970+0000 7f5ae77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ae8108870 0x7f5ae81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.970+0000 7f5ae77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ae8108870 0x7f5ae81a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37982/0 (socket says 172.21.15.38:37982) 2024-05-21T15:32:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.970+0000 7f5ae77fe640 1 -- 172.21.15.38:0/4283889163 learned_addr learned my addr 172.21.15.38:0/4283889163 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.970+0000 7f5ae67fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ae8105f10 0x7f5ae8199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.970+0000 7f5ae6ffd640 1 --2- 172.21.15.38:0/4283889163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ae81035b0 0x7f5ae81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.971+0000 7f5ae77fe640 1 -- 172.21.15.38:0/4283889163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ae8105f10 msgr2=0x7f5ae8199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.971+0000 7f5ae77fe640 1 --2- 172.21.15.38:0/4283889163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ae8105f10 0x7f5ae8199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.971+0000 7f5ae77fe640 1 -- 172.21.15.38:0/4283889163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ae81035b0 msgr2=0x7f5ae81991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.971+0000 7f5ae77fe640 1 --2- 172.21.15.38:0/4283889163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ae81035b0 0x7f5ae81991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.971+0000 7f5ae77fe640 1 -- 172.21.15.38:0/4283889163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae81a0d20 con 0x7f5ae8108870 2024-05-21T15:32:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.971+0000 7f5ae67fc640 1 --2- 172.21.15.38:0/4283889163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ae8105f10 0x7f5ae8199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:32:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.971+0000 7f5ae77fe640 1 --2- 172.21.15.38:0/4283889163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ae8108870 0x7f5ae81a06b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000eb40 tx=0x7f5ae000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.971+0000 7f5acffff640 1 -- 172.21.15.38:0/4283889163 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0004b20 con 0x7f5ae8108870 2024-05-21T15:32:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.971+0000 7f5acffff640 1 -- 172.21.15.38:0/4283889163 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae0004cc0 con 0x7f5ae8108870 2024-05-21T15:32:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.971+0000 7f5aed626640 1 -- 172.21.15.38:0/4283889163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae81a0fb0 con 0x7f5ae8108870 2024-05-21T15:32:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.971+0000 7f5acffff640 1 -- 172.21.15.38:0/4283889163 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0010850 con 0x7f5ae8108870 2024-05-21T15:32:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.971+0000 7f5aed626640 1 -- 172.21.15.38:0/4283889163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae81a1470 con 0x7f5ae8108870 2024-05-21T15:32:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.972+0000 7f5aed626640 1 -- 172.21.15.38:0/4283889163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae80fe6d0 con 0x7f5ae8108870 2024-05-21T15:32:29.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.974+0000 7f5acffff640 1 -- 172.21.15.38:0/4283889163 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5ae0005000 con 0x7f5ae8108870 2024-05-21T15:32:29.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.974+0000 7f5acffff640 1 --2- 172.21.15.38:0/4283889163 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5ad00a2c30 0x7f5ad00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:29.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.975+0000 7f5ae6ffd640 1 --2- 172.21.15.38:0/4283889163 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5ad00a2c30 0x7f5ad00a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:29.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.975+0000 7f5acffff640 1 -- 172.21.15.38:0/4283889163 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5ae00cf2b0 con 0x7f5ae8108870 2024-05-21T15:32:29.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.975+0000 7f5ae6ffd640 1 --2- 172.21.15.38:0/4283889163 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5ad00a2c30 0x7f5ad00a5140 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8031440 tx=0x7f5ad80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:29.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:29.978+0000 7f5acffff640 1 -- 172.21.15.38:0/4283889163 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5ae009c6b0 con 0x7f5ae8108870 2024-05-21T15:32:30.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.174+0000 7f5aed626640 1 -- 172.21.15.38:0/4283889163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ae8062750 con 0x7f5ae8108870 2024-05-21T15:32:30.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.176+0000 7f5acffff640 1 -- 172.21.15.38:0/4283889163 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18349 (secure 0 0 0) 0x7f5ae00ba7c0 con 0x7f5ae8108870 2024-05-21T15:32:30.176 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:30.177 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":82,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:31:45.813596+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:32:30.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.178+0000 7f5aed626640 1 -- 172.21.15.38:0/4283889163 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5ad00a2c30 msgr2=0x7f5ad00a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.178+0000 7f5aed626640 1 --2- 172.21.15.38:0/4283889163 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5ad00a2c30 0x7f5ad00a5140 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8031440 tx=0x7f5ad80315b0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.178+0000 7f5aed626640 1 -- 172.21.15.38:0/4283889163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ae8108870 msgr2=0x7f5ae81a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.178+0000 7f5aed626640 1 --2- 172.21.15.38:0/4283889163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ae8108870 0x7f5ae81a06b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000eb40 tx=0x7f5ae000c2d0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.178+0000 7f5aed626640 1 -- 172.21.15.38:0/4283889163 shutdown_connections 2024-05-21T15:32:30.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.178+0000 7f5aed626640 1 --2- 172.21.15.38:0/4283889163 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5ad00a2c30 0x7f5ad00a5140 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.178+0000 7f5aed626640 1 --2- 172.21.15.38:0/4283889163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ae8108870 0x7f5ae81a06b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.178+0000 7f5aed626640 1 --2- 172.21.15.38:0/4283889163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ae8105f10 0x7f5ae8199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.178+0000 7f5aed626640 1 --2- 172.21.15.38:0/4283889163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ae81035b0 0x7f5ae81991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.178+0000 7f5aed626640 1 -- 172.21.15.38:0/4283889163 >> 172.21.15.38:0/4283889163 conn(0x7f5ae80fcbb0 msgr2=0x7f5ae8104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:30.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.178+0000 7f5aed626640 1 -- 172.21.15.38:0/4283889163 shutdown_connections 2024-05-21T15:32:30.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.178+0000 7f5aed626640 1 -- 172.21.15.38:0/4283889163 wait complete. 2024-05-21T15:32:30.190 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:27:50.659367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:32:30.191 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:32:30.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.311+0000 7f0f95461640 1 -- 172.21.15.38:0/4180219483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f90102a90 msgr2=0x7f0f90103790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.311+0000 7f0f95461640 1 --2- 172.21.15.38:0/4180219483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f90102a90 0x7f0f90103790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009900 tx=0x7f0f8002a410 comp rx=0 tx=0).stop 2024-05-21T15:32:30.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.311+0000 7f0f95461640 1 -- 172.21.15.38:0/4180219483 shutdown_connections 2024-05-21T15:32:30.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.311+0000 7f0f95461640 1 --2- 172.21.15.38:0/4180219483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f901089d0 0x7f0f9010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.311+0000 7f0f95461640 1 --2- 172.21.15.38:0/4180219483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f90103cd0 0x7f0f901040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.311+0000 7f0f95461640 1 --2- 172.21.15.38:0/4180219483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f90102a90 0x7f0f90103790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.311+0000 7f0f95461640 1 -- 172.21.15.38:0/4180219483 >> 172.21.15.38:0/4180219483 conn(0x7f0f900fdc80 msgr2=0x7f0f90100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:30.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.311+0000 7f0f95461640 1 -- 172.21.15.38:0/4180219483 shutdown_connections 2024-05-21T15:32:30.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.311+0000 7f0f95461640 1 -- 172.21.15.38:0/4180219483 wait complete. 2024-05-21T15:32:30.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.312+0000 7f0f95461640 1 Processor -- start 2024-05-21T15:32:30.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.312+0000 7f0f95461640 1 -- start start 2024-05-21T15:32:30.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.312+0000 7f0f95461640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f90102a90 0x7f0f901992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:30.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.312+0000 7f0f95461640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f90103cd0 0x7f0f90199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:30.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.312+0000 7f0f95461640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f901089d0 0x7f0f901a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:30.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.312+0000 7f0f95461640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9010e6e0 con 0x7f0f90102a90 2024-05-21T15:32:30.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.312+0000 7f0f95461640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9010e560 con 0x7f0f90103cd0 2024-05-21T15:32:30.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.312+0000 7f0f95461640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9010e860 con 0x7f0f901089d0 2024-05-21T15:32:30.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.312+0000 7f0f8effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f90102a90 0x7f0f901992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:30.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f8effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f90102a90 0x7f0f901992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37998/0 (socket says 172.21.15.38:37998) 2024-05-21T15:32:30.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f8effd640 1 -- 172.21.15.38:0/2739649979 learned_addr learned my addr 172.21.15.38:0/2739649979 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:30.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f8e7fc640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f90103cd0 0x7f0f90199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:30.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f8f7fe640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f901089d0 0x7f0f901a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:30.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f8effd640 1 -- 172.21.15.38:0/2739649979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f901089d0 msgr2=0x7f0f901a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f8effd640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f901089d0 0x7f0f901a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f8effd640 1 -- 172.21.15.38:0/2739649979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f90103cd0 msgr2=0x7f0f90199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f8effd640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f90103cd0 0x7f0f90199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f8effd640 1 -- 172.21.15.38:0/2739649979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f901a0dc0 con 0x7f0f90102a90 2024-05-21T15:32:30.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f8e7fc640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f90103cd0 0x7f0f90199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:32:30.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f8effd640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f90102a90 0x7f0f901992f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0f80005ab0 tx=0x7f0f80004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:30.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f87fff640 1 -- 172.21.15.38:0/2739649979 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8003f030 con 0x7f0f90102a90 2024-05-21T15:32:30.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f95461640 1 -- 172.21.15.38:0/2739649979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f901a1050 con 0x7f0f90102a90 2024-05-21T15:32:30.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f87fff640 1 -- 172.21.15.38:0/2739649979 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f80003ad0 con 0x7f0f90102a90 2024-05-21T15:32:30.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f87fff640 1 -- 172.21.15.38:0/2739649979 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80035070 con 0x7f0f90102a90 2024-05-21T15:32:30.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.313+0000 7f0f95461640 1 -- 172.21.15.38:0/2739649979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f901a1590 con 0x7f0f90102a90 2024-05-21T15:32:30.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.314+0000 7f0f95461640 1 -- 172.21.15.38:0/2739649979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f901077e0 con 0x7f0f90102a90 2024-05-21T15:32:30.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.316+0000 7f0f87fff640 1 -- 172.21.15.38:0/2739649979 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0f80042030 con 0x7f0f90102a90 2024-05-21T15:32:30.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.317+0000 7f0f87fff640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0f7c0a2b90 0x7f0f7c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.317+0000 7f0f87fff640 1 -- 172.21.15.38:0/2739649979 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0f800ef390 con 0x7f0f90102a90 2024-05-21T15:32:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.317+0000 7f0f8e7fc640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0f7c0a2b90 0x7f0f7c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.317+0000 7f0f8e7fc640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0f7c0a2b90 0x7f0f7c0a50a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0f9019a390 tx=0x7f0f78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.320+0000 7f0f87fff640 1 -- 172.21.15.38:0/2739649979 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0f800bc140 con 0x7f0f90102a90 2024-05-21T15:32:30.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:30 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] up:boot 2024-05-21T15:32:30.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:30 smithi053 ceph-mon[36496]: fsmap 12 up:standby 2024-05-21T15:32:30.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:30 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:32:30.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:30 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:30.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:30 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:30.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:30 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:30.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:30 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:32:30.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:30 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2520882396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:30.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:30 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4283889163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.508+0000 7f0f95461640 1 -- 172.21.15.38:0/2739649979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0f90062750 con 0x7f0f90102a90 2024-05-21T15:32:30.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.508+0000 7f0f87fff640 1 -- 172.21.15.38:0/2739649979 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f0f800c0440 con 0x7f0f90102a90 2024-05-21T15:32:30.509 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-05-21T15:32:30.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.510+0000 7f0f95461640 1 -- 172.21.15.38:0/2739649979 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0f7c0a2b90 msgr2=0x7f0f7c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.510+0000 7f0f95461640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0f7c0a2b90 0x7f0f7c0a50a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0f9019a390 tx=0x7f0f78005f90 comp rx=0 tx=0).stop 2024-05-21T15:32:30.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.510+0000 7f0f95461640 1 -- 172.21.15.38:0/2739649979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f90102a90 msgr2=0x7f0f901992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.510+0000 7f0f95461640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f90102a90 0x7f0f901992f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0f80005ab0 tx=0x7f0f80004270 comp rx=0 tx=0).stop 2024-05-21T15:32:30.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.510+0000 7f0f95461640 1 -- 172.21.15.38:0/2739649979 shutdown_connections 2024-05-21T15:32:30.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.510+0000 7f0f95461640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f901089d0 0x7f0f901a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.510+0000 7f0f95461640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f90103cd0 0x7f0f90199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.510+0000 7f0f95461640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0f7c0a2b90 0x7f0f7c0a50a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.510+0000 7f0f95461640 1 --2- 172.21.15.38:0/2739649979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f90102a90 0x7f0f901992f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.510+0000 7f0f95461640 1 -- 172.21.15.38:0/2739649979 >> 172.21.15.38:0/2739649979 conn(0x7f0f900fdc80 msgr2=0x7f0f901049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.510+0000 7f0f95461640 1 -- 172.21.15.38:0/2739649979 shutdown_connections 2024-05-21T15:32:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.511+0000 7f0f95461640 1 -- 172.21.15.38:0/2739649979 wait complete. 2024-05-21T15:32:30.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:30 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] up:boot 2024-05-21T15:32:30.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:30 smithi045 ceph-mon[36409]: fsmap 12 up:standby 2024-05-21T15:32:30.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:30 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:32:30.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:30 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:30.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:30 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:30.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:30 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:30.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:30 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:32:30.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:30 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2520882396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:30.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:30 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4283889163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:30.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:30 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] up:boot 2024-05-21T15:32:30.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:30 smithi038 ceph-mon[31995]: fsmap 12 up:standby 2024-05-21T15:32:30.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:30 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:32:30.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:30 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:30.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:30 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:30.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:30 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:30.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:30 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:32:30.522 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:30 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2520882396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:30.522 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:30 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4283889163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:30.523 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-21T15:32:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.638+0000 7f3c43ea2640 1 -- 172.21.15.38:0/2511332304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c3c1019a0 msgr2=0x7f3c3c1026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.638+0000 7f3c43ea2640 1 --2- 172.21.15.38:0/2511332304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c3c1019a0 0x7f3c3c1026a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009ca0 tx=0x7f3c2c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.638+0000 7f3c43ea2640 1 -- 172.21.15.38:0/2511332304 shutdown_connections 2024-05-21T15:32:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.638+0000 7f3c43ea2640 1 --2- 172.21.15.38:0/2511332304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c3c1078e0 0x7f3c3c109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.638+0000 7f3c43ea2640 1 --2- 172.21.15.38:0/2511332304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c102be0 0x7f3c3c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.638+0000 7f3c43ea2640 1 --2- 172.21.15.38:0/2511332304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c3c1019a0 0x7f3c3c1026a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.638+0000 7f3c43ea2640 1 -- 172.21.15.38:0/2511332304 >> 172.21.15.38:0/2511332304 conn(0x7f3c3c0fcbb0 msgr2=0x7f3c3c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.638+0000 7f3c43ea2640 1 -- 172.21.15.38:0/2511332304 shutdown_connections 2024-05-21T15:32:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.638+0000 7f3c43ea2640 1 -- 172.21.15.38:0/2511332304 wait complete. 2024-05-21T15:32:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.639+0000 7f3c43ea2640 1 Processor -- start 2024-05-21T15:32:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.639+0000 7f3c43ea2640 1 -- start start 2024-05-21T15:32:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.639+0000 7f3c43ea2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c1019a0 0x7f3c3c0747f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.639+0000 7f3c43ea2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c3c102be0 0x7f3c3c19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.639+0000 7f3c43ea2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c3c1078e0 0x7f3c3c1a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.639+0000 7f3c43ea2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c10d5f0 con 0x7f3c3c102be0 2024-05-21T15:32:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.639+0000 7f3c43ea2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c10d470 con 0x7f3c3c1019a0 2024-05-21T15:32:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.639+0000 7f3c43ea2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c10d770 con 0x7f3c3c1078e0 2024-05-21T15:32:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.639+0000 7f3c4143d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c3c102be0 0x7f3c3c19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.639+0000 7f3c4143d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c3c102be0 0x7f3c3c19dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38016/0 (socket says 172.21.15.38:38016) 2024-05-21T15:32:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.639+0000 7f3c4143d640 1 -- 172.21.15.38:0/936086339 learned_addr learned my addr 172.21.15.38:0/936086339 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.639+0000 7f3c4243f640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c3c1078e0 0x7f3c3c1a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.640+0000 7f3c41c3e640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c1019a0 0x7f3c3c0747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.640+0000 7f3c4143d640 1 -- 172.21.15.38:0/936086339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c3c1078e0 msgr2=0x7f3c3c1a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.640+0000 7f3c4143d640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c3c1078e0 0x7f3c3c1a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.640+0000 7f3c4143d640 1 -- 172.21.15.38:0/936086339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c1019a0 msgr2=0x7f3c3c0747f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.640+0000 7f3c4143d640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c1019a0 0x7f3c3c0747f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.640+0000 7f3c4143d640 1 -- 172.21.15.38:0/936086339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3c1a0d50 con 0x7f3c3c102be0 2024-05-21T15:32:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.640+0000 7f3c41c3e640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c1019a0 0x7f3c3c0747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:32:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.640+0000 7f3c4143d640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c3c102be0 0x7f3c3c19dfe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000e9f0 tx=0x7f3c3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.640+0000 7f3c3affd640 1 -- 172.21.15.38:0/936086339 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3000cd80 con 0x7f3c3c102be0 2024-05-21T15:32:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.640+0000 7f3c3affd640 1 -- 172.21.15.38:0/936086339 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c30010910 con 0x7f3c3c102be0 2024-05-21T15:32:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.640+0000 7f3c3affd640 1 -- 172.21.15.38:0/936086339 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30018990 con 0x7f3c3c102be0 2024-05-21T15:32:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.640+0000 7f3c43ea2640 1 -- 172.21.15.38:0/936086339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3c1a0fe0 con 0x7f3c3c102be0 2024-05-21T15:32:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.640+0000 7f3c43ea2640 1 -- 172.21.15.38:0/936086339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c3c1a14d0 con 0x7f3c3c102be0 2024-05-21T15:32:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.641+0000 7f3c43ea2640 1 -- 172.21.15.38:0/936086339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c3c1066f0 con 0x7f3c3c102be0 2024-05-21T15:32:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.642+0000 7f3c3affd640 1 -- 172.21.15.38:0/936086339 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3c30010ac0 con 0x7f3c3c102be0 2024-05-21T15:32:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.643+0000 7f3c3affd640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3c240a2c30 0x7f3c240a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.643+0000 7f3c41c3e640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3c240a2c30 0x7f3c240a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.643+0000 7f3c3affd640 1 -- 172.21.15.38:0/936086339 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3c300cf4f0 con 0x7f3c3c102be0 2024-05-21T15:32:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.644+0000 7f3c41c3e640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3c240a2c30 0x7f3c240a5140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c004af0 tx=0x7f3c2c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:30.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.646+0000 7f3c3affd640 1 -- 172.21.15.38:0/936086339 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3c3009c850 con 0x7f3c3c102be0 2024-05-21T15:32:30.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.841+0000 7f3c43ea2640 1 -- 172.21.15.38:0/936086339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3c3c062750 con 0x7f3c3c102be0 2024-05-21T15:32:30.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.842+0000 7f3c3affd640 1 -- 172.21.15.38:0/936086339 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f3c3009c150 con 0x7f3c3c102be0 2024-05-21T15:32:30.842 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:30.842 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-05-21T15:32:30.842 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:32:30.842 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-05-21T15:32:30.842 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-05-21T15:32:30.843 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:32:30.843 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-05-21T15:32:30.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.844+0000 7f3c43ea2640 1 -- 172.21.15.38:0/936086339 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3c240a2c30 msgr2=0x7f3c240a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.844+0000 7f3c43ea2640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3c240a2c30 0x7f3c240a5140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c004af0 tx=0x7f3c2c005fd0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.844+0000 7f3c43ea2640 1 -- 172.21.15.38:0/936086339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c3c102be0 msgr2=0x7f3c3c19dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.844+0000 7f3c43ea2640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c3c102be0 0x7f3c3c19dfe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000e9f0 tx=0x7f3c3000eeb0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.844+0000 7f3c43ea2640 1 -- 172.21.15.38:0/936086339 shutdown_connections 2024-05-21T15:32:30.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.844+0000 7f3c43ea2640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c3c1078e0 0x7f3c3c1a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.844+0000 7f3c43ea2640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3c240a2c30 0x7f3c240a5140 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.844+0000 7f3c43ea2640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c3c102be0 0x7f3c3c19dfe0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.844+0000 7f3c43ea2640 1 --2- 172.21.15.38:0/936086339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c1019a0 0x7f3c3c0747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.844+0000 7f3c43ea2640 1 -- 172.21.15.38:0/936086339 >> 172.21.15.38:0/936086339 conn(0x7f3c3c0fcbb0 msgr2=0x7f3c3c1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:30.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.844+0000 7f3c43ea2640 1 -- 172.21.15.38:0/936086339 shutdown_connections 2024-05-21T15:32:30.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.845+0000 7f3c43ea2640 1 -- 172.21.15.38:0/936086339 wait complete. 2024-05-21T15:32:30.858 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-21T15:32:30.858 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-21T15:32:30.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.989+0000 7fae6d06c640 1 -- 172.21.15.38:0/1121006235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae68104020 msgr2=0x7fae68104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.989+0000 7fae6d06c640 1 --2- 172.21.15.38:0/1121006235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae68104020 0x7fae68104430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fae58009900 tx=0x7fae5802a410 comp rx=0 tx=0).stop 2024-05-21T15:32:30.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.989+0000 7fae6d06c640 1 -- 172.21.15.38:0/1121006235 shutdown_connections 2024-05-21T15:32:30.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.989+0000 7fae6d06c640 1 --2- 172.21.15.38:0/1121006235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae681011d0 0x7fae6810ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.989+0000 7fae6d06c640 1 --2- 172.21.15.38:0/1121006235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae681006d0 0x7fae68100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.989+0000 7fae6d06c640 1 --2- 172.21.15.38:0/1121006235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae68104020 0x7fae68104430 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.989+0000 7fae6d06c640 1 -- 172.21.15.38:0/1121006235 >> 172.21.15.38:0/1121006235 conn(0x7fae680fc2f0 msgr2=0x7fae680fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:30.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.989+0000 7fae6d06c640 1 -- 172.21.15.38:0/1121006235 shutdown_connections 2024-05-21T15:32:30.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.989+0000 7fae6d06c640 1 -- 172.21.15.38:0/1121006235 wait complete. 2024-05-21T15:32:30.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.990+0000 7fae6d06c640 1 Processor -- start 2024-05-21T15:32:30.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.990+0000 7fae6d06c640 1 -- start start 2024-05-21T15:32:30.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.990+0000 7fae6d06c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae681006d0 0x7fae681992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:30.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.990+0000 7fae6d06c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae681011d0 0x7fae681997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:30.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.990+0000 7fae6d06c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae68104020 0x7fae681a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:30.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.990+0000 7fae6d06c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fae6810e730 con 0x7fae681011d0 2024-05-21T15:32:30.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.990+0000 7fae6d06c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fae6810e5b0 con 0x7fae681006d0 2024-05-21T15:32:30.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.990+0000 7fae6d06c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fae6810e8b0 con 0x7fae68104020 2024-05-21T15:32:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.990+0000 7fae66d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae681006d0 0x7fae681992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae6759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae68104020 0x7fae681a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae6659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae681011d0 0x7fae681997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae66d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae681006d0 0x7fae681992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:44002/0 (socket says 172.21.15.38:44002) 2024-05-21T15:32:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae66d9d640 1 -- 172.21.15.38:0/1769017384 learned_addr learned my addr 172.21.15.38:0/1769017384 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae6759e640 1 -- 172.21.15.38:0/1769017384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae681006d0 msgr2=0x7fae681992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae6759e640 1 --2- 172.21.15.38:0/1769017384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae681006d0 0x7fae681992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae6759e640 1 -- 172.21.15.38:0/1769017384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae681011d0 msgr2=0x7fae681997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae6759e640 1 --2- 172.21.15.38:0/1769017384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae681011d0 0x7fae681997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae6759e640 1 -- 172.21.15.38:0/1769017384 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae681a0de0 con 0x7fae68104020 2024-05-21T15:32:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae6759e640 1 --2- 172.21.15.38:0/1769017384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae68104020 0x7fae681a0770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fae60009fd0 tx=0x7fae6000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae4ffff640 1 -- 172.21.15.38:0/1769017384 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6000ce20 con 0x7fae68104020 2024-05-21T15:32:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae6d06c640 1 -- 172.21.15.38:0/1769017384 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae681017b0 con 0x7fae68104020 2024-05-21T15:32:30.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae4ffff640 1 -- 172.21.15.38:0/1769017384 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae60005020 con 0x7fae68104020 2024-05-21T15:32:30.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae4ffff640 1 -- 172.21.15.38:0/1769017384 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60010720 con 0x7fae68104020 2024-05-21T15:32:30.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.991+0000 7fae6d06c640 1 -- 172.21.15.38:0/1769017384 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae68101d50 con 0x7fae68104020 2024-05-21T15:32:30.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.993+0000 7fae6d06c640 1 -- 172.21.15.38:0/1769017384 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae681079b0 con 0x7fae68104020 2024-05-21T15:32:30.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.994+0000 7fae4ffff640 1 -- 172.21.15.38:0/1769017384 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fae60005540 con 0x7fae68104020 2024-05-21T15:32:30.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.995+0000 7fae4ffff640 1 --2- 172.21.15.38:0/1769017384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fae500a2b90 0x7fae500a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:30.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.995+0000 7fae4ffff640 1 -- 172.21.15.38:0/1769017384 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fae600cf250 con 0x7fae68104020 2024-05-21T15:32:30.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.995+0000 7fae66d9d640 1 --2- 172.21.15.38:0/1769017384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fae500a2b90 0x7fae500a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:30.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.996+0000 7fae66d9d640 1 --2- 172.21.15.38:0/1769017384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fae500a2b90 0x7fae500a50a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fae58000c00 tx=0x7fae5803c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:30.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:30.999+0000 7fae4ffff640 1 -- 172.21.15.38:0/1769017384 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fae6009c5b0 con 0x7fae68104020 2024-05-21T15:32:31.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:31.198+0000 7fae6d06c640 1 -- 172.21.15.38:0/1769017384 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fae6802d2a0 con 0x7fae68104020 2024-05-21T15:32:31.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:31 smithi053 ceph-mon[36496]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:31.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:31 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2739649979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:32:31.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:31 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/936086339' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-21T15:32:31.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:31.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:31.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:31.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:32.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:31 smithi045 ceph-mon[36409]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:32.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:31 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2739649979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:32:32.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:31 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/936086339' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-21T15:32:32.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:32.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:32.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:32.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:32.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:31 smithi038 ceph-mon[31995]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:32.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:31 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2739649979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:32:32.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:31 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/936086339' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-21T15:32:32.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:32.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:32.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:32.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.166+0000 7fae4ffff640 1 -- 172.21.15.38:0/1769017384 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7fae6009beb0 con 0x7fae68104020 2024-05-21T15:32:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-05-21T15:32:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.171+0000 7fae6d06c640 1 -- 172.21.15.38:0/1769017384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fae500a2b90 msgr2=0x7fae500a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.171+0000 7fae6d06c640 1 --2- 172.21.15.38:0/1769017384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fae500a2b90 0x7fae500a50a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fae58000c00 tx=0x7fae5803c040 comp rx=0 tx=0).stop 2024-05-21T15:32:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.171+0000 7fae6d06c640 1 -- 172.21.15.38:0/1769017384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae68104020 msgr2=0x7fae681a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.171+0000 7fae6d06c640 1 --2- 172.21.15.38:0/1769017384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae68104020 0x7fae681a0770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fae60009fd0 tx=0x7fae6000ef10 comp rx=0 tx=0).stop 2024-05-21T15:32:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.172+0000 7fae6d06c640 1 -- 172.21.15.38:0/1769017384 shutdown_connections 2024-05-21T15:32:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.172+0000 7fae6d06c640 1 --2- 172.21.15.38:0/1769017384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae68104020 0x7fae681a0770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.172+0000 7fae6d06c640 1 --2- 172.21.15.38:0/1769017384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fae500a2b90 0x7fae500a50a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.172+0000 7fae6d06c640 1 --2- 172.21.15.38:0/1769017384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fae681011d0 0x7fae681997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.172+0000 7fae6d06c640 1 --2- 172.21.15.38:0/1769017384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae681006d0 0x7fae681992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.172+0000 7fae6d06c640 1 -- 172.21.15.38:0/1769017384 >> 172.21.15.38:0/1769017384 conn(0x7fae680fc2f0 msgr2=0x7fae68104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.173+0000 7fae6d06c640 1 -- 172.21.15.38:0/1769017384 shutdown_connections 2024-05-21T15:32:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.173+0000 7fae6d06c640 1 -- 172.21.15.38:0/1769017384 wait complete. 2024-05-21T15:32:32.185 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-21T15:32:32.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.311+0000 7f419bbea640 1 -- 172.21.15.38:0/3609967776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4194108870 msgr2=0x7f419410ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:32.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.311+0000 7f419bbea640 1 --2- 172.21.15.38:0/3609967776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4194108870 0x7f419410ac90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f418c009ca0 tx=0x7f418c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:32:32.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.311+0000 7f419bbea640 1 -- 172.21.15.38:0/3609967776 shutdown_connections 2024-05-21T15:32:32.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.311+0000 7f419bbea640 1 --2- 172.21.15.38:0/3609967776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4194108870 0x7f419410ac90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:32.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.311+0000 7f419bbea640 1 --2- 172.21.15.38:0/3609967776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4194105f10 0x7f4194108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.311+0000 7f419bbea640 1 --2- 172.21.15.38:0/3609967776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41941035b0 0x7f41941059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.311+0000 7f419bbea640 1 -- 172.21.15.38:0/3609967776 >> 172.21.15.38:0/3609967776 conn(0x7f41940fcbb0 msgr2=0x7f41940ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.311+0000 7f419bbea640 1 -- 172.21.15.38:0/3609967776 shutdown_connections 2024-05-21T15:32:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.311+0000 7f419bbea640 1 -- 172.21.15.38:0/3609967776 wait complete. 2024-05-21T15:32:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f419bbea640 1 Processor -- start 2024-05-21T15:32:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f419bbea640 1 -- start start 2024-05-21T15:32:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f419bbea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41941035b0 0x7f4194199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f419bbea640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4194105f10 0x7f4194199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f419bbea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4194108870 0x7f41941a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f419bbea640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f419410e770 con 0x7f4194108870 2024-05-21T15:32:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f419bbea640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f419410e5f0 con 0x7f41941035b0 2024-05-21T15:32:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f419bbea640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f419410e8f0 con 0x7f4194105f10 2024-05-21T15:32:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f419a187640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4194108870 0x7f41941a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f4199185640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4194105f10 0x7f4194199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f419a187640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4194108870 0x7f41941a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38060/0 (socket says 172.21.15.38:38060) 2024-05-21T15:32:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f419a187640 1 -- 172.21.15.38:0/3169140938 learned_addr learned my addr 172.21.15.38:0/3169140938 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f4199986640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41941035b0 0x7f4194199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f4199185640 1 -- 172.21.15.38:0/3169140938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41941035b0 msgr2=0x7f4194199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f4199185640 1 --2- 172.21.15.38:0/3169140938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41941035b0 0x7f4194199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f4199185640 1 -- 172.21.15.38:0/3169140938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4194108870 msgr2=0x7f41941a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f4199185640 1 --2- 172.21.15.38:0/3169140938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4194108870 0x7f41941a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f4199185640 1 -- 172.21.15.38:0/3169140938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41941a0d90 con 0x7f4194105f10 2024-05-21T15:32:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.312+0000 7f419a187640 1 --2- 172.21.15.38:0/3169140938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4194108870 0x7f41941a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:32:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.313+0000 7f4199185640 1 --2- 172.21.15.38:0/3169140938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4194105f10 0x7f4194199790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f418400e9f0 tx=0x7f418400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.313+0000 7f418affd640 1 -- 172.21.15.38:0/3169140938 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418400ccc0 con 0x7f4194105f10 2024-05-21T15:32:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.313+0000 7f419bbea640 1 -- 172.21.15.38:0/3169140938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41941a1080 con 0x7f4194105f10 2024-05-21T15:32:32.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.313+0000 7f418affd640 1 -- 172.21.15.38:0/3169140938 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4184010910 con 0x7f4194105f10 2024-05-21T15:32:32.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.313+0000 7f418affd640 1 -- 172.21.15.38:0/3169140938 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184018960 con 0x7f4194105f10 2024-05-21T15:32:32.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.313+0000 7f419bbea640 1 -- 172.21.15.38:0/3169140938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41941a1620 con 0x7f4194105f10 2024-05-21T15:32:32.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.314+0000 7f419bbea640 1 -- 172.21.15.38:0/3169140938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41940fe6d0 con 0x7f4194105f10 2024-05-21T15:32:32.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.315+0000 7f418affd640 1 -- 172.21.15.38:0/3169140938 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4184010ac0 con 0x7f4194105f10 2024-05-21T15:32:32.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.316+0000 7f418affd640 1 --2- 172.21.15.38:0/3169140938 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f41800a2b90 0x7f41800a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:32.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.316+0000 7f4199986640 1 --2- 172.21.15.38:0/3169140938 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f41800a2b90 0x7f41800a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:32.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.316+0000 7f418affd640 1 -- 172.21.15.38:0/3169140938 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f41840cf4c0 con 0x7f4194105f10 2024-05-21T15:32:32.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.317+0000 7f4199986640 1 --2- 172.21.15.38:0/3169140938 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f41800a2b90 0x7f41800a50a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f41900095f0 tx=0x7f4190005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:32.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.319+0000 7f418affd640 1 -- 172.21.15.38:0/3169140938 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f418409c060 con 0x7f4194105f10 2024-05-21T15:32:32.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:32.512+0000 7f419bbea640 1 -- 172.21.15.38:0/3169140938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4194062750 con 0x7f4194105f10 2024-05-21T15:32:33.061 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:32 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1769017384' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T15:32:33.062 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:32 smithi053 ceph-mon[36496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T15:32:33.062 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:33.062 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:33.062 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:33.062 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:32:33.062 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:33.062 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:32:33.062 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:32 smithi053 ceph-mon[36496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-21T15:32:33.062 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:32 smithi053 ceph-mon[36496]: osdmap e83: 12 total, 12 up, 12 in 2024-05-21T15:32:33.062 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:33.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.170+0000 7f418affd640 1 -- 172.21.15.38:0/3169140938 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f41840a0360 con 0x7f4194105f10 2024-05-21T15:32:33.173 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-05-21T15:32:33.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.174+0000 7f419bbea640 1 -- 172.21.15.38:0/3169140938 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f41800a2b90 msgr2=0x7f41800a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:33.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.174+0000 7f419bbea640 1 --2- 172.21.15.38:0/3169140938 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f41800a2b90 0x7f41800a50a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f41900095f0 tx=0x7f4190005f90 comp rx=0 tx=0).stop 2024-05-21T15:32:33.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.174+0000 7f419bbea640 1 -- 172.21.15.38:0/3169140938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4194105f10 msgr2=0x7f4194199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:33.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.174+0000 7f419bbea640 1 --2- 172.21.15.38:0/3169140938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4194105f10 0x7f4194199790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f418400e9f0 tx=0x7f418400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:32:33.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.175+0000 7f419bbea640 1 -- 172.21.15.38:0/3169140938 shutdown_connections 2024-05-21T15:32:33.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.175+0000 7f419bbea640 1 --2- 172.21.15.38:0/3169140938 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f41800a2b90 0x7f41800a50a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:33.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.175+0000 7f419bbea640 1 --2- 172.21.15.38:0/3169140938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4194108870 0x7f41941a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:33.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.175+0000 7f419bbea640 1 --2- 172.21.15.38:0/3169140938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4194105f10 0x7f4194199790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:33.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.175+0000 7f419bbea640 1 --2- 172.21.15.38:0/3169140938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41941035b0 0x7f4194199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:33.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.175+0000 7f419bbea640 1 -- 172.21.15.38:0/3169140938 >> 172.21.15.38:0/3169140938 conn(0x7f41940fcbb0 msgr2=0x7f4194104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:33.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.175+0000 7f419bbea640 1 -- 172.21.15.38:0/3169140938 shutdown_connections 2024-05-21T15:32:33.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.175+0000 7f419bbea640 1 -- 172.21.15.38:0/3169140938 wait complete. 2024-05-21T15:32:33.193 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:32 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1769017384' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T15:32:33.193 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:32 smithi038 ceph-mon[31995]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T15:32:33.193 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:33.193 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:33.193 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:33.193 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:32:33.193 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:33.193 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:32:33.194 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:32 smithi038 ceph-mon[31995]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-21T15:32:33.194 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:32 smithi038 ceph-mon[31995]: osdmap e83: 12 total, 12 up, 12 in 2024-05-21T15:32:33.194 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:33.195 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-21T15:32:33.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:32 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1769017384' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T15:32:33.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:32 smithi045 ceph-mon[36409]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T15:32:33.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:33.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:33.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:33.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:32:33.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:33.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:32:33.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:32 smithi045 ceph-mon[36409]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-21T15:32:33.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:32 smithi045 ceph-mon[36409]: osdmap e83: 12 total, 12 up, 12 in 2024-05-21T15:32:33.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:33.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.318+0000 7f44ead65640 1 -- 172.21.15.38:0/1714290229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e4101990 msgr2=0x7f44e4102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:33.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.318+0000 7f44ead65640 1 --2- 172.21.15.38:0/1714290229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e4101990 0x7f44e4102690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009900 tx=0x7f44d802a410 comp rx=0 tx=0).stop 2024-05-21T15:32:33.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.318+0000 7f44ead65640 1 -- 172.21.15.38:0/1714290229 shutdown_connections 2024-05-21T15:32:33.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.318+0000 7f44ead65640 1 --2- 172.21.15.38:0/1714290229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e41078d0 0x7f44e4109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:33.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.318+0000 7f44ead65640 1 --2- 172.21.15.38:0/1714290229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e4102bd0 0x7f44e4102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:33.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.318+0000 7f44ead65640 1 --2- 172.21.15.38:0/1714290229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e4101990 0x7f44e4102690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:33.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.318+0000 7f44ead65640 1 -- 172.21.15.38:0/1714290229 >> 172.21.15.38:0/1714290229 conn(0x7f44e40fcb60 msgr2=0x7f44e40ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:33.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.318+0000 7f44ead65640 1 -- 172.21.15.38:0/1714290229 shutdown_connections 2024-05-21T15:32:33.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.318+0000 7f44ead65640 1 -- 172.21.15.38:0/1714290229 wait complete. 2024-05-21T15:32:33.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.319+0000 7f44ead65640 1 Processor -- start 2024-05-21T15:32:33.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.319+0000 7f44ead65640 1 -- start start 2024-05-21T15:32:33.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.319+0000 7f44ead65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e4101990 0x7f44e40fe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:33.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.319+0000 7f44ead65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e4102bd0 0x7f44e40fd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:33.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.320+0000 7f44ead65640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e41078d0 0x7f44e40fd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:33.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.320+0000 7f44ead65640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f44e410d5c0 con 0x7f44e41078d0 2024-05-21T15:32:33.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.320+0000 7f44ead65640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f44e410d440 con 0x7f44e4102bd0 2024-05-21T15:32:33.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.320+0000 7f44ead65640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f44e410d740 con 0x7f44e4101990 2024-05-21T15:32:33.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.320+0000 7f44e8b01640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e4101990 0x7f44e40fe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:33.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.320+0000 7f44e9302640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e41078d0 0x7f44e40fd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:33.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.320+0000 7f44e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e4102bd0 0x7f44e40fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:33.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.320+0000 7f44e9302640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e41078d0 0x7f44e40fd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38088/0 (socket says 172.21.15.38:38088) 2024-05-21T15:32:33.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.320+0000 7f44e8b01640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e4101990 0x7f44e40fe9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:44364/0 (socket says 172.21.15.38:44364) 2024-05-21T15:32:33.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.320+0000 7f44e9302640 1 -- 172.21.15.38:0/1090242560 learned_addr learned my addr 172.21.15.38:0/1090242560 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:33.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.321+0000 7f44e3fff640 1 -- 172.21.15.38:0/1090242560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e4101990 msgr2=0x7f44e40fe9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:33.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.321+0000 7f44e3fff640 1 --2- 172.21.15.38:0/1090242560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e4101990 0x7f44e40fe9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:33.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.321+0000 7f44e3fff640 1 -- 172.21.15.38:0/1090242560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e41078d0 msgr2=0x7f44e40fd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:33.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.321+0000 7f44e3fff640 1 --2- 172.21.15.38:0/1090242560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e41078d0 0x7f44e40fd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:33.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.321+0000 7f44e3fff640 1 -- 172.21.15.38:0/1090242560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e40fdd00 con 0x7f44e4102bd0 2024-05-21T15:32:33.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.321+0000 7f44e9302640 1 --2- 172.21.15.38:0/1090242560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e41078d0 0x7f44e40fd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:32:33.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.321+0000 7f44e3fff640 1 --2- 172.21.15.38:0/1090242560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e4102bd0 0x7f44e40fd090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f44d000e9f0 tx=0x7f44d000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:33.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.321+0000 7f44e1ffb640 1 -- 172.21.15.38:0/1090242560 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d000ccc0 con 0x7f44e4102bd0 2024-05-21T15:32:33.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.321+0000 7f44e1ffb640 1 -- 172.21.15.38:0/1090242560 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d0010910 con 0x7f44e4102bd0 2024-05-21T15:32:33.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.321+0000 7f44ead65640 1 -- 172.21.15.38:0/1090242560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e41a3380 con 0x7f44e4102bd0 2024-05-21T15:32:33.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.321+0000 7f44e1ffb640 1 -- 172.21.15.38:0/1090242560 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0018960 con 0x7f44e4102bd0 2024-05-21T15:32:33.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.321+0000 7f44ead65640 1 -- 172.21.15.38:0/1090242560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e41a3840 con 0x7f44e4102bd0 2024-05-21T15:32:33.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.322+0000 7f44ead65640 1 -- 172.21.15.38:0/1090242560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e41a3aa0 con 0x7f44e4102bd0 2024-05-21T15:32:33.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.324+0000 7f44e1ffb640 1 -- 172.21.15.38:0/1090242560 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f44d0010ac0 con 0x7f44e4102bd0 2024-05-21T15:32:33.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.324+0000 7f44e1ffb640 1 --2- 172.21.15.38:0/1090242560 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44d40a2b90 0x7f44d40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:33.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.324+0000 7f44e1ffb640 1 -- 172.21.15.38:0/1090242560 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f44d00cf2c0 con 0x7f44e4102bd0 2024-05-21T15:32:33.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.324+0000 7f44e8b01640 1 --2- 172.21.15.38:0/1090242560 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44d40a2b90 0x7f44d40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:33.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.325+0000 7f44e8b01640 1 --2- 172.21.15.38:0/1090242560 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44d40a2b90 0x7f44d40a50a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f44d8005ab0 tx=0x7f44d8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:33.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.329+0000 7f44e1ffb640 1 -- 172.21.15.38:0/1090242560 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f44d009bce0 con 0x7f44e4102bd0 2024-05-21T15:32:33.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:33.562+0000 7f44ead65640 1 -- 172.21.15.38:0/1090242560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f44e41a4120 con 0x7f44e4102bd0 2024-05-21T15:32:34.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.213+0000 7f44e1ffb640 1 -- 172.21.15.38:0/1090242560 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f44d009ffe0 con 0x7f44e4102bd0 2024-05-21T15:32:34.213 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 2 and data pool 3 2024-05-21T15:32:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.215+0000 7f44ead65640 1 -- 172.21.15.38:0/1090242560 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44d40a2b90 msgr2=0x7f44d40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.215+0000 7f44ead65640 1 --2- 172.21.15.38:0/1090242560 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44d40a2b90 0x7f44d40a50a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f44d8005ab0 tx=0x7f44d8031400 comp rx=0 tx=0).stop 2024-05-21T15:32:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.215+0000 7f44ead65640 1 -- 172.21.15.38:0/1090242560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e4102bd0 msgr2=0x7f44e40fd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.215+0000 7f44ead65640 1 --2- 172.21.15.38:0/1090242560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e4102bd0 0x7f44e40fd090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f44d000e9f0 tx=0x7f44d000eeb0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.216+0000 7f44ead65640 1 -- 172.21.15.38:0/1090242560 shutdown_connections 2024-05-21T15:32:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.216+0000 7f44ead65640 1 --2- 172.21.15.38:0/1090242560 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44d40a2b90 0x7f44d40a50a0 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f44d8005ab0 tx=0x7f44d8031400 comp rx=0 tx=0).stop 2024-05-21T15:32:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.216+0000 7f44ead65640 1 --2- 172.21.15.38:0/1090242560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e41078d0 0x7f44e40fd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.216+0000 7f44ead65640 1 --2- 172.21.15.38:0/1090242560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e4102bd0 0x7f44e40fd090 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.216+0000 7f44ead65640 1 --2- 172.21.15.38:0/1090242560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e4101990 0x7f44e40fe9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.216+0000 7f44ead65640 1 -- 172.21.15.38:0/1090242560 >> 172.21.15.38:0/1090242560 conn(0x7f44e40fcb60 msgr2=0x7f44e4108b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.216+0000 7f44ead65640 1 -- 172.21.15.38:0/1090242560 shutdown_connections 2024-05-21T15:32:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.216+0000 7f44ead65640 1 -- 172.21.15.38:0/1090242560 wait complete. 2024-05-21T15:32:34.230 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: pgmap v288: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:34.230 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3169140938' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T15:32:34.230 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T15:32:34.230 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-21T15:32:34.231 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: osdmap e84: 12 total, 12 up, 12 in 2024-05-21T15:32:34.231 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-21T15:32:34.231 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1090242560' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T15:32:34.231 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T15:32:34.231 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-21T15:32:34.231 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.231 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: osdmap e85: 12 total, 12 up, 12 in 2024-05-21T15:32:34.231 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.231 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.231 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.231 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.231 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.231 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:34.231 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:32:34.232 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.232 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:32:34.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: pgmap v288: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:34.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3169140938' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T15:32:34.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T15:32:34.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-21T15:32:34.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: osdmap e84: 12 total, 12 up, 12 in 2024-05-21T15:32:34.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-21T15:32:34.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1090242560' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T15:32:34.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T15:32:34.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-21T15:32:34.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: osdmap e85: 12 total, 12 up, 12 in 2024-05-21T15:32:34.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:34.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:32:34.263 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.349+0000 7efc3a0bc640 1 -- 172.21.15.38:0/3660641863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc341019b0 msgr2=0x7efc341026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:34.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.349+0000 7efc3a0bc640 1 --2- 172.21.15.38:0/3660641863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc341019b0 0x7efc341026b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efc24009900 tx=0x7efc2402a410 comp rx=0 tx=0).stop 2024-05-21T15:32:34.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.349+0000 7efc3a0bc640 1 -- 172.21.15.38:0/3660641863 shutdown_connections 2024-05-21T15:32:34.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.349+0000 7efc3a0bc640 1 --2- 172.21.15.38:0/3660641863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc341078f0 0x7efc34109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.349+0000 7efc3a0bc640 1 --2- 172.21.15.38:0/3660641863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc34102bf0 0x7efc34103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.349+0000 7efc3a0bc640 1 --2- 172.21.15.38:0/3660641863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc341019b0 0x7efc341026b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.349+0000 7efc3a0bc640 1 -- 172.21.15.38:0/3660641863 >> 172.21.15.38:0/3660641863 conn(0x7efc340fcba0 msgr2=0x7efc340ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.349+0000 7efc3a0bc640 1 -- 172.21.15.38:0/3660641863 shutdown_connections 2024-05-21T15:32:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.349+0000 7efc3a0bc640 1 -- 172.21.15.38:0/3660641863 wait complete. 2024-05-21T15:32:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.349+0000 7efc3a0bc640 1 Processor -- start 2024-05-21T15:32:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.349+0000 7efc3a0bc640 1 -- start start 2024-05-21T15:32:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc3a0bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc341019b0 0x7efc34199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc3a0bc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc34102bf0 0x7efc34199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc3a0bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc341078f0 0x7efc341a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc3a0bc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efc3410d600 con 0x7efc341019b0 2024-05-21T15:32:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc3a0bc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc3410d480 con 0x7efc34102bf0 2024-05-21T15:32:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc3a0bc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efc3410d780 con 0x7efc341078f0 2024-05-21T15:32:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc337fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc341019b0 0x7efc34199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc337fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc341019b0 0x7efc34199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38098/0 (socket says 172.21.15.38:38098) 2024-05-21T15:32:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc337fe640 1 -- 172.21.15.38:0/2669443578 learned_addr learned my addr 172.21.15.38:0/2669443578 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc2bfff640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc34102bf0 0x7efc34199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc33fff640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc341078f0 0x7efc341a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc337fe640 1 -- 172.21.15.38:0/2669443578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc341078f0 msgr2=0x7efc341a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc337fe640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc341078f0 0x7efc341a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc337fe640 1 -- 172.21.15.38:0/2669443578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc34102bf0 msgr2=0x7efc34199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc337fe640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc34102bf0 0x7efc34199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc337fe640 1 -- 172.21.15.38:0/2669443578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc341a0d80 con 0x7efc341019b0 2024-05-21T15:32:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc2bfff640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc34102bf0 0x7efc34199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:32:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.350+0000 7efc337fe640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc341019b0 0x7efc34199240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc24005ab0 tx=0x7efc24004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.351+0000 7efc317fa640 1 -- 172.21.15.38:0/2669443578 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2403f030 con 0x7efc341019b0 2024-05-21T15:32:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.351+0000 7efc317fa640 1 -- 172.21.15.38:0/2669443578 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc24003ad0 con 0x7efc341019b0 2024-05-21T15:32:34.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.351+0000 7efc3a0bc640 1 -- 172.21.15.38:0/2669443578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc341a1010 con 0x7efc341019b0 2024-05-21T15:32:34.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.351+0000 7efc317fa640 1 -- 172.21.15.38:0/2669443578 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24035070 con 0x7efc341019b0 2024-05-21T15:32:34.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.351+0000 7efc3a0bc640 1 -- 172.21.15.38:0/2669443578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc341a1500 con 0x7efc341019b0 2024-05-21T15:32:34.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.352+0000 7efc3a0bc640 1 -- 172.21.15.38:0/2669443578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc341a1760 con 0x7efc341019b0 2024-05-21T15:32:34.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.353+0000 7efc317fa640 1 -- 172.21.15.38:0/2669443578 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efc24042030 con 0x7efc341019b0 2024-05-21T15:32:34.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.354+0000 7efc317fa640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efc200a2c30 0x7efc200a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:34.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.354+0000 7efc2bfff640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efc200a2c30 0x7efc200a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:34.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.354+0000 7efc317fa640 1 -- 172.21.15.38:0/2669443578 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7efc240ef780 con 0x7efc341019b0 2024-05-21T15:32:34.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.355+0000 7efc2bfff640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efc200a2c30 0x7efc200a5140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efc3419a2e0 tx=0x7efc1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:34.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.358+0000 7efc317fa640 1 -- 172.21.15.38:0/2669443578 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efc240bc140 con 0x7efc341019b0 2024-05-21T15:32:34.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: pgmap v288: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:34.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3169140938' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T15:32:34.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T15:32:34.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-21T15:32:34.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: osdmap e84: 12 total, 12 up, 12 in 2024-05-21T15:32:34.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-21T15:32:34.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1090242560' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T15:32:34.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T15:32:34.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-21T15:32:34.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: osdmap e85: 12 total, 12 up, 12 in 2024-05-21T15:32:34.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:32:34.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:32:34.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:32:34.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.545+0000 7efc3a0bc640 1 -- 172.21.15.38:0/2669443578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc34106ff0 con 0x7efc341019b0 2024-05-21T15:32:34.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.546+0000 7efc317fa640 1 -- 172.21.15.38:0/2669443578 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22125 (secure 0 0 0) 0x7efc240c0440 con 0x7efc341019b0 2024-05-21T15:32:34.547 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:34.547 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":85,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:32:33.564202+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:32:34.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.548+0000 7efc3a0bc640 1 -- 172.21.15.38:0/2669443578 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efc200a2c30 msgr2=0x7efc200a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:34.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.548+0000 7efc3a0bc640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efc200a2c30 0x7efc200a5140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efc3419a2e0 tx=0x7efc1c005f90 comp rx=0 tx=0).stop 2024-05-21T15:32:34.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.548+0000 7efc3a0bc640 1 -- 172.21.15.38:0/2669443578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc341019b0 msgr2=0x7efc34199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:34.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.548+0000 7efc3a0bc640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc341019b0 0x7efc34199240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc24005ab0 tx=0x7efc24004270 comp rx=0 tx=0).stop 2024-05-21T15:32:34.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.548+0000 7efc3a0bc640 1 -- 172.21.15.38:0/2669443578 shutdown_connections 2024-05-21T15:32:34.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.548+0000 7efc3a0bc640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc341078f0 0x7efc341a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.548+0000 7efc3a0bc640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc34102bf0 0x7efc34199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.548+0000 7efc3a0bc640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efc200a2c30 0x7efc200a5140 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.548+0000 7efc3a0bc640 1 --2- 172.21.15.38:0/2669443578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc341019b0 0x7efc34199240 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.548+0000 7efc3a0bc640 1 -- 172.21.15.38:0/2669443578 >> 172.21.15.38:0/2669443578 conn(0x7efc340fcba0 msgr2=0x7efc34100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:34.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.549+0000 7efc3a0bc640 1 -- 172.21.15.38:0/2669443578 shutdown_connections 2024-05-21T15:32:34.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.549+0000 7efc3a0bc640 1 -- 172.21.15.38:0/2669443578 wait complete. 2024-05-21T15:32:34.561 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:32:34.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.678+0000 7fb9fe840640 1 -- 172.21.15.38:0/3602982042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9f8108a20 msgr2=0x7fb9f810ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:34.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.678+0000 7fb9fe840640 1 --2- 172.21.15.38:0/3602982042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9f8108a20 0x7fb9f810ae40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009ca0 tx=0x7fb9f002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.678+0000 7fb9fe840640 1 -- 172.21.15.38:0/3602982042 shutdown_connections 2024-05-21T15:32:34.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.678+0000 7fb9fe840640 1 --2- 172.21.15.38:0/3602982042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9f8108a20 0x7fb9f810ae40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.678+0000 7fb9fe840640 1 --2- 172.21.15.38:0/3602982042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9f8103d20 0x7fb9f8104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.678+0000 7fb9fe840640 1 --2- 172.21.15.38:0/3602982042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9f8102ae0 0x7fb9f81037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.678+0000 7fb9fe840640 1 -- 172.21.15.38:0/3602982042 >> 172.21.15.38:0/3602982042 conn(0x7fb9f80fdcf0 msgr2=0x7fb9f8100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:34.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.678+0000 7fb9fe840640 1 -- 172.21.15.38:0/3602982042 shutdown_connections 2024-05-21T15:32:34.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.678+0000 7fb9fe840640 1 -- 172.21.15.38:0/3602982042 wait complete. 2024-05-21T15:32:34.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.678+0000 7fb9fe840640 1 Processor -- start 2024-05-21T15:32:34.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9fe840640 1 -- start start 2024-05-21T15:32:34.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9fe840640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9f8102ae0 0x7fb9f81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:34.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9fe840640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9f8103d20 0x7fb9f8199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:34.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9fe840640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9f8108a20 0x7fb9f81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:34.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9fe840640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f810e760 con 0x7fb9f8102ae0 2024-05-21T15:32:34.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9fe840640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f810e5e0 con 0x7fb9f8108a20 2024-05-21T15:32:34.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9fe840640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f810e8e0 con 0x7fb9f8103d20 2024-05-21T15:32:34.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9f8102ae0 0x7fb9f81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:34.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9f8102ae0 0x7fb9f81991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38116/0 (socket says 172.21.15.38:38116) 2024-05-21T15:32:34.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9f7fff640 1 -- 172.21.15.38:0/2951177069 learned_addr learned my addr 172.21.15.38:0/2951177069 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:34.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9f77fe640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9f8103d20 0x7fb9f8199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:34.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9fcddd640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9f8108a20 0x7fb9f81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:34.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9f7fff640 1 -- 172.21.15.38:0/2951177069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9f8103d20 msgr2=0x7fb9f8199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:34.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9f7fff640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9f8103d20 0x7fb9f8199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9f7fff640 1 -- 172.21.15.38:0/2951177069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9f8108a20 msgr2=0x7fb9f81a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9f7fff640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9f8108a20 0x7fb9f81a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9f7fff640 1 -- 172.21.15.38:0/2951177069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f81a0cd0 con 0x7fb9f8102ae0 2024-05-21T15:32:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9f77fe640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9f8103d20 0x7fb9f8199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:32:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.679+0000 7fb9f7fff640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9f8102ae0 0x7fb9f81991e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800e9f0 tx=0x7fb9e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.680+0000 7fb9f57fa640 1 -- 172.21.15.38:0/2951177069 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e800ccc0 con 0x7fb9f8102ae0 2024-05-21T15:32:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.680+0000 7fb9f57fa640 1 -- 172.21.15.38:0/2951177069 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e80035f0 con 0x7fb9f8102ae0 2024-05-21T15:32:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.680+0000 7fb9f57fa640 1 -- 172.21.15.38:0/2951177069 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e800ce60 con 0x7fb9f8102ae0 2024-05-21T15:32:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.680+0000 7fb9fe840640 1 -- 172.21.15.38:0/2951177069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f81a0fc0 con 0x7fb9f8102ae0 2024-05-21T15:32:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.680+0000 7fb9fe840640 1 -- 172.21.15.38:0/2951177069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f81a1420 con 0x7fb9f8102ae0 2024-05-21T15:32:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.681+0000 7fb9fe840640 1 -- 172.21.15.38:0/2951177069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9f80691c0 con 0x7fb9f8102ae0 2024-05-21T15:32:34.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.682+0000 7fb9f57fa640 1 -- 172.21.15.38:0/2951177069 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb9e80037a0 con 0x7fb9f8102ae0 2024-05-21T15:32:34.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.683+0000 7fb9f57fa640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9e00a2cb0 0x7fb9e00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:34.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.683+0000 7fb9f77fe640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9e00a2cb0 0x7fb9e00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:34.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.683+0000 7fb9f57fa640 1 -- 172.21.15.38:0/2951177069 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb9e80cf5b0 con 0x7fb9f8102ae0 2024-05-21T15:32:34.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.684+0000 7fb9f77fe640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9e00a2cb0 0x7fb9e00a51c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb9f819a230 tx=0x7fb9ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:34.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.687+0000 7fb9f57fa640 1 -- 172.21.15.38:0/2951177069 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb9e809c640 con 0x7fb9f8102ae0 2024-05-21T15:32:34.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.873+0000 7fb9fe840640 1 -- 172.21.15.38:0/2951177069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9f8107880 con 0x7fb9f8102ae0 2024-05-21T15:32:34.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.875+0000 7fb9f57fa640 1 -- 172.21.15.38:0/2951177069 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22112 (secure 0 0 0) 0x7fb9f8107880 con 0x7fb9f8102ae0 2024-05-21T15:32:34.876 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:34.876 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":86,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:32:34.657863+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:32:34.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.877+0000 7fb9fe840640 1 -- 172.21.15.38:0/2951177069 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9e00a2cb0 msgr2=0x7fb9e00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:34.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.877+0000 7fb9fe840640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9e00a2cb0 0x7fb9e00a51c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb9f819a230 tx=0x7fb9ec005f90 comp rx=0 tx=0).stop 2024-05-21T15:32:34.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.877+0000 7fb9fe840640 1 -- 172.21.15.38:0/2951177069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9f8102ae0 msgr2=0x7fb9f81991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:34.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.877+0000 7fb9fe840640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9f8102ae0 0x7fb9f81991e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800e9f0 tx=0x7fb9e800eeb0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.877+0000 7fb9fe840640 1 -- 172.21.15.38:0/2951177069 shutdown_connections 2024-05-21T15:32:34.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.877+0000 7fb9fe840640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9f8108a20 0x7fb9f81a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.877+0000 7fb9fe840640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9f8103d20 0x7fb9f8199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.877+0000 7fb9fe840640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9e00a2cb0 0x7fb9e00a51c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.877+0000 7fb9fe840640 1 --2- 172.21.15.38:0/2951177069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9f8102ae0 0x7fb9f81991e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:34.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.877+0000 7fb9fe840640 1 -- 172.21.15.38:0/2951177069 >> 172.21.15.38:0/2951177069 conn(0x7fb9f80fdcf0 msgr2=0x7fb9f8104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:34.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.877+0000 7fb9fe840640 1 -- 172.21.15.38:0/2951177069 shutdown_connections 2024-05-21T15:32:34.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:34.877+0000 7fb9fe840640 1 -- 172.21.15.38:0/2951177069 wait complete. 2024-05-21T15:32:34.890 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-21T15:32:35.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.007+0000 7fa0f1032640 1 -- 172.21.15.38:0/1947997640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0ec108830 msgr2=0x7fa0ec10ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:35.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.007+0000 7fa0f1032640 1 --2- 172.21.15.38:0/1947997640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0ec108830 0x7fa0ec10ac50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009ca0 tx=0x7fa0e402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:32:35.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.007+0000 7fa0f1032640 1 -- 172.21.15.38:0/1947997640 shutdown_connections 2024-05-21T15:32:35.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.007+0000 7fa0f1032640 1 --2- 172.21.15.38:0/1947997640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0ec108830 0x7fa0ec10ac50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:35.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.007+0000 7fa0f1032640 1 --2- 172.21.15.38:0/1947997640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ec105ed0 0x7fa0ec1082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:35.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.007+0000 7fa0f1032640 1 --2- 172.21.15.38:0/1947997640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0ec103570 0x7fa0ec105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:35.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.007+0000 7fa0f1032640 1 -- 172.21.15.38:0/1947997640 >> 172.21.15.38:0/1947997640 conn(0x7fa0ec0fcb70 msgr2=0x7fa0ec0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:35.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.007+0000 7fa0f1032640 1 -- 172.21.15.38:0/1947997640 shutdown_connections 2024-05-21T15:32:35.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.007+0000 7fa0f1032640 1 -- 172.21.15.38:0/1947997640 wait complete. 2024-05-21T15:32:35.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.008+0000 7fa0f1032640 1 Processor -- start 2024-05-21T15:32:35.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.008+0000 7fa0f1032640 1 -- start start 2024-05-21T15:32:35.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.008+0000 7fa0f1032640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0ec103570 0x7fa0ec199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:35.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.008+0000 7fa0f1032640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ec105ed0 0x7fa0ec199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:35.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.008+0000 7fa0f1032640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0ec108830 0x7fa0ec1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:35.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.008+0000 7fa0f1032640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec10e700 con 0x7fa0ec108830 2024-05-21T15:32:35.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.008+0000 7fa0f1032640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec10e580 con 0x7fa0ec103570 2024-05-21T15:32:35.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.008+0000 7fa0f1032640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec10e880 con 0x7fa0ec105ed0 2024-05-21T15:32:35.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.008+0000 7fa0eb59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0ec108830 0x7fa0ec1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:35.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.008+0000 7fa0eb59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0ec108830 0x7fa0ec1a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38556/0 (socket says 172.21.15.38:38556) 2024-05-21T15:32:35.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.008+0000 7fa0eb59e640 1 -- 172.21.15.38:0/1828176724 learned_addr learned my addr 172.21.15.38:0/1828176724 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:35.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.009+0000 7fa0ead9d640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0ec103570 0x7fa0ec199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:35.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.009+0000 7fa0e3fff640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ec105ed0 0x7fa0ec199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:35.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.009+0000 7fa0eb59e640 1 -- 172.21.15.38:0/1828176724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ec105ed0 msgr2=0x7fa0ec199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:35.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.009+0000 7fa0eb59e640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ec105ed0 0x7fa0ec199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:35.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.009+0000 7fa0eb59e640 1 -- 172.21.15.38:0/1828176724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0ec103570 msgr2=0x7fa0ec199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:35.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.009+0000 7fa0eb59e640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0ec103570 0x7fa0ec199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:35.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.009+0000 7fa0eb59e640 1 -- 172.21.15.38:0/1828176724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec1a0d80 con 0x7fa0ec108830 2024-05-21T15:32:35.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.009+0000 7fa0e3fff640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ec105ed0 0x7fa0ec199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:32:35.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.009+0000 7fa0eb59e640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0ec108830 0x7fa0ec1a0710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009bf0 tx=0x7fa0e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:35.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.009+0000 7fa0e8d99640 1 -- 172.21.15.38:0/1828176724 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4003b70 con 0x7fa0ec108830 2024-05-21T15:32:35.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.009+0000 7fa0e8d99640 1 -- 172.21.15.38:0/1828176724 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e4003d10 con 0x7fa0ec108830 2024-05-21T15:32:35.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.009+0000 7fa0f1032640 1 -- 172.21.15.38:0/1828176724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec1a1010 con 0x7fa0ec108830 2024-05-21T15:32:35.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.009+0000 7fa0e8d99640 1 -- 172.21.15.38:0/1828176724 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e402fc80 con 0x7fa0ec108830 2024-05-21T15:32:35.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.009+0000 7fa0f1032640 1 -- 172.21.15.38:0/1828176724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec1a15b0 con 0x7fa0ec108830 2024-05-21T15:32:35.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.010+0000 7fa0f1032640 1 -- 172.21.15.38:0/1828176724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ec0fe690 con 0x7fa0ec108830 2024-05-21T15:32:35.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.012+0000 7fa0e8d99640 1 -- 172.21.15.38:0/1828176724 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa0e4004050 con 0x7fa0ec108830 2024-05-21T15:32:35.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.012+0000 7fa0e8d99640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0d80a2c10 0x7fa0d80a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:35.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.012+0000 7fa0ead9d640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0d80a2c10 0x7fa0d80a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:35.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.012+0000 7fa0e8d99640 1 -- 172.21.15.38:0/1828176724 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa0e40f0220 con 0x7fa0ec108830 2024-05-21T15:32:35.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.013+0000 7fa0ead9d640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0d80a2c10 0x7fa0d80a5120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0699d0 tx=0x7fa0dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:35.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.016+0000 7fa0e8d99640 1 -- 172.21.15.38:0/1828176724 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa0e40bd2d0 con 0x7fa0ec108830 2024-05-21T15:32:35.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:35.246+0000 7fa0f1032640 1 -- 172.21.15.38:0/1828176724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa0ec02d2a0 con 0x7fa0ec108830 2024-05-21T15:32:35.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:35 smithi045 ceph-mon[36409]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-21T15:32:35.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:35 smithi045 ceph-mon[36409]: pgmap v291: 97 pgs: 4 active+clean, 17 creating+peering, 76 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:35.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:35 smithi045 ceph-mon[36409]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-21T15:32:35.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:35 smithi045 ceph-mon[36409]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-21T15:32:35.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:35 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2669443578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:35.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:35 smithi045 ceph-mon[36409]: osdmap e86: 12 total, 12 up, 12 in 2024-05-21T15:32:35.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:35 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2951177069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:35.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:35 smithi038 ceph-mon[31995]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-21T15:32:35.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:35 smithi038 ceph-mon[31995]: pgmap v291: 97 pgs: 4 active+clean, 17 creating+peering, 76 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:35.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:35 smithi038 ceph-mon[31995]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-21T15:32:35.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:35 smithi038 ceph-mon[31995]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-21T15:32:35.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:35 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2669443578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:35.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:35 smithi038 ceph-mon[31995]: osdmap e86: 12 total, 12 up, 12 in 2024-05-21T15:32:35.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:35 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2951177069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:35.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:35 smithi053 ceph-mon[36496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-21T15:32:35.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:35 smithi053 ceph-mon[36496]: pgmap v291: 97 pgs: 4 active+clean, 17 creating+peering, 76 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:32:35.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:35 smithi053 ceph-mon[36496]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-21T15:32:35.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:35 smithi053 ceph-mon[36496]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-21T15:32:35.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:35 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2669443578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:35.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:35 smithi053 ceph-mon[36496]: osdmap e86: 12 total, 12 up, 12 in 2024-05-21T15:32:35.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:35 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2951177069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:36.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.237+0000 7fa0e8d99640 1 -- 172.21.15.38:0/1828176724 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fa0e40bcbd0 con 0x7fa0ec108830 2024-05-21T15:32:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.240+0000 7fa0f1032640 1 -- 172.21.15.38:0/1828176724 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0d80a2c10 msgr2=0x7fa0d80a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.240+0000 7fa0f1032640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0d80a2c10 0x7fa0d80a5120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0699d0 tx=0x7fa0dc005f90 comp rx=0 tx=0).stop 2024-05-21T15:32:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.240+0000 7fa0f1032640 1 -- 172.21.15.38:0/1828176724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0ec108830 msgr2=0x7fa0ec1a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.240+0000 7fa0f1032640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0ec108830 0x7fa0ec1a0710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009bf0 tx=0x7fa0e4004ab0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.240+0000 7fa0f1032640 1 -- 172.21.15.38:0/1828176724 shutdown_connections 2024-05-21T15:32:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.240+0000 7fa0f1032640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0d80a2c10 0x7fa0d80a5120 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.240+0000 7fa0f1032640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0ec108830 0x7fa0ec1a0710 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.240+0000 7fa0f1032640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ec105ed0 0x7fa0ec199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.240+0000 7fa0f1032640 1 --2- 172.21.15.38:0/1828176724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0ec103570 0x7fa0ec199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.240+0000 7fa0f1032640 1 -- 172.21.15.38:0/1828176724 >> 172.21.15.38:0/1828176724 conn(0x7fa0ec0fcb70 msgr2=0x7fa0ec104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:36.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.240+0000 7fa0f1032640 1 -- 172.21.15.38:0/1828176724 shutdown_connections 2024-05-21T15:32:36.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.240+0000 7fa0f1032640 1 -- 172.21.15.38:0/1828176724 wait complete. 2024-05-21T15:32:36.255 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-21T15:32:36.255 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-05-21T15:32:36.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.379+0000 7f641387a640 1 -- 172.21.15.38:0/4052216516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f640c102c00 msgr2=0x7f640c103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:36.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.379+0000 7f641387a640 1 --2- 172.21.15.38:0/4052216516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f640c102c00 0x7f640c103010 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009900 tx=0x7f63fc02a410 comp rx=0 tx=0).stop 2024-05-21T15:32:36.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.379+0000 7f641387a640 1 -- 172.21.15.38:0/4052216516 shutdown_connections 2024-05-21T15:32:36.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.379+0000 7f641387a640 1 --2- 172.21.15.38:0/4052216516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f640c107900 0x7f640c109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.379+0000 7f641387a640 1 --2- 172.21.15.38:0/4052216516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f640c102c00 0x7f640c103010 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.379+0000 7f641387a640 1 --2- 172.21.15.38:0/4052216516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f640c1019c0 0x7f640c1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.379+0000 7f641387a640 1 -- 172.21.15.38:0/4052216516 >> 172.21.15.38:0/4052216516 conn(0x7f640c0fcbb0 msgr2=0x7f640c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:36.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.379+0000 7f641387a640 1 -- 172.21.15.38:0/4052216516 shutdown_connections 2024-05-21T15:32:36.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.379+0000 7f641387a640 1 -- 172.21.15.38:0/4052216516 wait complete. 2024-05-21T15:32:36.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.380+0000 7f641387a640 1 Processor -- start 2024-05-21T15:32:36.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.380+0000 7f641387a640 1 -- start start 2024-05-21T15:32:36.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.380+0000 7f641387a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f640c1019c0 0x7f640c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:36.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.380+0000 7f641387a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f640c102c00 0x7f640c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:36.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.380+0000 7f641387a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f640c107900 0x7f640c1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:36.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.380+0000 7f641387a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10e7a0 con 0x7f640c1019c0 2024-05-21T15:32:36.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.380+0000 7f641387a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10e620 con 0x7f640c107900 2024-05-21T15:32:36.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.380+0000 7f641387a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10e920 con 0x7f640c102c00 2024-05-21T15:32:36.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.380+0000 7f6411616640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f640c1019c0 0x7f640c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:36.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.380+0000 7f6411616640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f640c1019c0 0x7f640c199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38582/0 (socket says 172.21.15.38:38582) 2024-05-21T15:32:36.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.380+0000 7f6411616640 1 -- 172.21.15.38:0/4100403250 learned_addr learned my addr 172.21.15.38:0/4100403250 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:36.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.381+0000 7f6410e15640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f640c102c00 0x7f640c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:36.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.381+0000 7f6411616640 1 -- 172.21.15.38:0/4100403250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f640c102c00 msgr2=0x7f640c199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:36.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.381+0000 7f6411616640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f640c102c00 0x7f640c199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.381+0000 7f6411e17640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f640c107900 0x7f640c1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:36.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.381+0000 7f6411616640 1 -- 172.21.15.38:0/4100403250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f640c107900 msgr2=0x7f640c1a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:36.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.381+0000 7f6411616640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f640c107900 0x7f640c1a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.381+0000 7f6411616640 1 -- 172.21.15.38:0/4100403250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c1a0cd0 con 0x7f640c1019c0 2024-05-21T15:32:36.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.381+0000 7f6410e15640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f640c102c00 0x7f640c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:32:36.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.381+0000 7f6411616640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f640c1019c0 0x7f640c199240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f640800c970 tx=0x7f640800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:36.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.381+0000 7f64027fc640 1 -- 172.21.15.38:0/4100403250 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408007c70 con 0x7f640c1019c0 2024-05-21T15:32:36.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.381+0000 7f64027fc640 1 -- 172.21.15.38:0/4100403250 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6408007e10 con 0x7f640c1019c0 2024-05-21T15:32:36.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.381+0000 7f641387a640 1 -- 172.21.15.38:0/4100403250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640c1a0fc0 con 0x7f640c1019c0 2024-05-21T15:32:36.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.381+0000 7f64027fc640 1 -- 172.21.15.38:0/4100403250 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64080188c0 con 0x7f640c1019c0 2024-05-21T15:32:36.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.381+0000 7f641387a640 1 -- 172.21.15.38:0/4100403250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640c1a1420 con 0x7f640c1019c0 2024-05-21T15:32:36.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.382+0000 7f641387a640 1 -- 172.21.15.38:0/4100403250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f640c0691f0 con 0x7f640c1019c0 2024-05-21T15:32:36.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.384+0000 7f64027fc640 1 -- 172.21.15.38:0/4100403250 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f640800f450 con 0x7f640c1019c0 2024-05-21T15:32:36.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.384+0000 7f64027fc640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f63f80a2b90 0x7f63f80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:36.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.384+0000 7f6410e15640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f63f80a2b90 0x7f63f80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:36.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.384+0000 7f64027fc640 1 -- 172.21.15.38:0/4100403250 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f64080cf000 con 0x7f640c1019c0 2024-05-21T15:32:36.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.385+0000 7f6410e15640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f63f80a2b90 0x7f63f80a50a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0096c0 tx=0x7f63fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:36.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.389+0000 7f64027fc640 1 -- 172.21.15.38:0/4100403250 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f640c0691f0 con 0x7f640c1019c0 2024-05-21T15:32:36.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:36 smithi045 ceph-mon[36409]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-21T15:32:36.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:36 smithi045 ceph-mon[36409]: Cluster is now healthy 2024-05-21T15:32:36.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:36 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] up:active 2024-05-21T15:32:36.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:36 smithi045 ceph-mon[36409]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:32:36.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:36 smithi045 ceph-mon[36409]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:32:36.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:36 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1828176724' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T15:32:36.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:36 smithi038 ceph-mon[31995]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-21T15:32:36.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:36 smithi038 ceph-mon[31995]: Cluster is now healthy 2024-05-21T15:32:36.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:36 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] up:active 2024-05-21T15:32:36.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:36 smithi038 ceph-mon[31995]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:32:36.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:36 smithi038 ceph-mon[31995]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:32:36.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:36 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1828176724' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T15:32:36.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.612+0000 7f641387a640 1 -- 172.21.15.38:0/4100403250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f640c02d380 con 0x7f640c1019c0 2024-05-21T15:32:36.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:36 smithi053 ceph-mon[36496]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-21T15:32:36.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:36 smithi053 ceph-mon[36496]: Cluster is now healthy 2024-05-21T15:32:36.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:36 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] up:active 2024-05-21T15:32:36.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:36 smithi053 ceph-mon[36496]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:32:36.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:36 smithi053 ceph-mon[36496]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:32:36.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:36 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1828176724' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T15:32:36.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.929+0000 7f64027fc640 1 -- 172.21.15.38:0/4100403250 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f6408014030 con 0x7f640c1019c0 2024-05-21T15:32:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.931+0000 7f641387a640 1 -- 172.21.15.38:0/4100403250 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f63f80a2b90 msgr2=0x7f63f80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.931+0000 7f641387a640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f63f80a2b90 0x7f63f80a50a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0096c0 tx=0x7f63fc000bc0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.931+0000 7f641387a640 1 -- 172.21.15.38:0/4100403250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f640c1019c0 msgr2=0x7f640c199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.931+0000 7f641387a640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f640c1019c0 0x7f640c199240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f640800c970 tx=0x7f640800ce30 comp rx=0 tx=0).stop 2024-05-21T15:32:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.931+0000 7f641387a640 1 -- 172.21.15.38:0/4100403250 shutdown_connections 2024-05-21T15:32:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.931+0000 7f641387a640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f640c107900 0x7f640c1a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.931+0000 7f641387a640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f640c102c00 0x7f640c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.931+0000 7f641387a640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f63f80a2b90 0x7f63f80a50a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.931+0000 7f641387a640 1 --2- 172.21.15.38:0/4100403250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f640c1019c0 0x7f640c199240 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.931+0000 7f641387a640 1 -- 172.21.15.38:0/4100403250 >> 172.21.15.38:0/4100403250 conn(0x7f640c0fcbb0 msgr2=0x7f640c1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.931+0000 7f641387a640 1 -- 172.21.15.38:0/4100403250 shutdown_connections 2024-05-21T15:32:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:36.931+0000 7f641387a640 1 -- 172.21.15.38:0/4100403250 wait complete. 2024-05-21T15:32:36.947 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-21T15:32:37.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.066+0000 7fea3881b640 1 -- 172.21.15.38:0/4210192841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea34102bd0 msgr2=0x7fea34102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:37.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.066+0000 7fea3881b640 1 --2- 172.21.15.38:0/4210192841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea34102bd0 0x7fea34102fe0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea28009900 tx=0x7fea2802a410 comp rx=0 tx=0).stop 2024-05-21T15:32:37.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.066+0000 7fea3881b640 1 -- 172.21.15.38:0/4210192841 shutdown_connections 2024-05-21T15:32:37.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.066+0000 7fea3881b640 1 --2- 172.21.15.38:0/4210192841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea341078d0 0x7fea34109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:37.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.066+0000 7fea3881b640 1 --2- 172.21.15.38:0/4210192841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea34102bd0 0x7fea34102fe0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:37.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.066+0000 7fea3881b640 1 --2- 172.21.15.38:0/4210192841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea34101990 0x7fea34102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:37.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.066+0000 7fea3881b640 1 -- 172.21.15.38:0/4210192841 >> 172.21.15.38:0/4210192841 conn(0x7fea340fcb80 msgr2=0x7fea340ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:37.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.066+0000 7fea3881b640 1 -- 172.21.15.38:0/4210192841 shutdown_connections 2024-05-21T15:32:37.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.066+0000 7fea3881b640 1 -- 172.21.15.38:0/4210192841 wait complete. 2024-05-21T15:32:37.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.067+0000 7fea3881b640 1 Processor -- start 2024-05-21T15:32:37.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.067+0000 7fea3881b640 1 -- start start 2024-05-21T15:32:37.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.067+0000 7fea3881b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea34101990 0x7fea341992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:37.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.067+0000 7fea3881b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea34102bd0 0x7fea34199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:37.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.067+0000 7fea3881b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea341078d0 0x7fea341a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:37.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea3881b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fea3410d650 con 0x7fea34101990 2024-05-21T15:32:37.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea3881b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea3410d4d0 con 0x7fea34102bd0 2024-05-21T15:32:37.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea3881b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fea3410d7d0 con 0x7fea341078d0 2024-05-21T15:32:37.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea3259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea34101990 0x7fea341992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:37.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea3259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea34101990 0x7fea341992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38604/0 (socket says 172.21.15.38:38604) 2024-05-21T15:32:37.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea3259c640 1 -- 172.21.15.38:0/1702841718 learned_addr learned my addr 172.21.15.38:0/1702841718 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:37.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea31d9b640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea34102bd0 0x7fea34199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea32d9d640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea341078d0 0x7fea341a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea3259c640 1 -- 172.21.15.38:0/1702841718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea341078d0 msgr2=0x7fea341a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea3259c640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea341078d0 0x7fea341a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea3259c640 1 -- 172.21.15.38:0/1702841718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea34102bd0 msgr2=0x7fea34199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea3259c640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea34102bd0 0x7fea34199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea3259c640 1 -- 172.21.15.38:0/1702841718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea341a0dd0 con 0x7fea34101990 2024-05-21T15:32:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea32d9d640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea341078d0 0x7fea341a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:32:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea3259c640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea34101990 0x7fea341992e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fea24005d00 tx=0x7fea2400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea237fe640 1 -- 172.21.15.38:0/1702841718 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24017070 con 0x7fea34101990 2024-05-21T15:32:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea237fe640 1 -- 172.21.15.38:0/1702841718 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea24009c30 con 0x7fea34101990 2024-05-21T15:32:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea237fe640 1 -- 172.21.15.38:0/1702841718 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2400f620 con 0x7fea34101990 2024-05-21T15:32:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea3881b640 1 -- 172.21.15.38:0/1702841718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea341a10c0 con 0x7fea34101990 2024-05-21T15:32:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.068+0000 7fea3881b640 1 -- 172.21.15.38:0/1702841718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea341a1610 con 0x7fea34101990 2024-05-21T15:32:37.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.069+0000 7fea3881b640 1 -- 172.21.15.38:0/1702841718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea341a1930 con 0x7fea34101990 2024-05-21T15:32:37.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.071+0000 7fea237fe640 1 -- 172.21.15.38:0/1702841718 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fea2401a070 con 0x7fea34101990 2024-05-21T15:32:37.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.071+0000 7fea237fe640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fea1c0a2c10 0x7fea1c0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:37.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.071+0000 7fea31d9b640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fea1c0a2c10 0x7fea1c0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:37.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.071+0000 7fea237fe640 1 -- 172.21.15.38:0/1702841718 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fea240ce970 con 0x7fea34101990 2024-05-21T15:32:37.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.072+0000 7fea31d9b640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fea1c0a2c10 0x7fea1c0a5120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fea28006010 tx=0x7fea28031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:37.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.076+0000 7fea237fe640 1 -- 172.21.15.38:0/1702841718 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fea24009de0 con 0x7fea34101990 2024-05-21T15:32:37.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.304+0000 7fea3881b640 1 -- 172.21.15.38:0/1702841718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fea341a1b00 con 0x7fea34101990 2024-05-21T15:32:37.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:37 smithi045 ceph-mon[36409]: pgmap v293: 97 pgs: 45 active+clean, 17 creating+peering, 35 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-05-21T15:32:37.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:37 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1828176724' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-21T15:32:37.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:37 smithi045 ceph-mon[36409]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:32:37.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:37 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4100403250' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T15:32:37.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:37 smithi045 ceph-mon[36409]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-21T15:32:37.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:37 smithi045 ceph-mon[36409]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-21T15:32:37.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:37 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4100403250' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-21T15:32:37.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:37 smithi045 ceph-mon[36409]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-21T15:32:37.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:37 smithi045 ceph-mon[36409]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-21T15:32:37.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:37 smithi038 ceph-mon[31995]: pgmap v293: 97 pgs: 45 active+clean, 17 creating+peering, 35 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-05-21T15:32:37.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:37 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1828176724' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-21T15:32:37.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:37 smithi038 ceph-mon[31995]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:32:37.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:37 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4100403250' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T15:32:37.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:37 smithi038 ceph-mon[31995]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-21T15:32:37.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:37 smithi038 ceph-mon[31995]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-21T15:32:37.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:37 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4100403250' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-21T15:32:37.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:37 smithi038 ceph-mon[31995]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-21T15:32:37.526 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:37 smithi038 ceph-mon[31995]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-21T15:32:37.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:37 smithi053 ceph-mon[36496]: pgmap v293: 97 pgs: 45 active+clean, 17 creating+peering, 35 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-05-21T15:32:37.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:37 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1828176724' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-21T15:32:37.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:37 smithi053 ceph-mon[36496]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:32:37.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:37 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4100403250' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T15:32:37.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:37 smithi053 ceph-mon[36496]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-21T15:32:37.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:37 smithi053 ceph-mon[36496]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-21T15:32:37.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:37 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4100403250' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-21T15:32:37.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:37 smithi053 ceph-mon[36496]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-21T15:32:37.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:37 smithi053 ceph-mon[36496]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-21T15:32:37.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.941+0000 7fea237fe640 1 -- 172.21.15.38:0/1702841718 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7fea2409ba20 con 0x7fea34101990 2024-05-21T15:32:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.944+0000 7fea3881b640 1 -- 172.21.15.38:0/1702841718 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fea1c0a2c10 msgr2=0x7fea1c0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.944+0000 7fea3881b640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fea1c0a2c10 0x7fea1c0a5120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fea28006010 tx=0x7fea28031400 comp rx=0 tx=0).stop 2024-05-21T15:32:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.944+0000 7fea3881b640 1 -- 172.21.15.38:0/1702841718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea34101990 msgr2=0x7fea341992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.944+0000 7fea3881b640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea34101990 0x7fea341992e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fea24005d00 tx=0x7fea2400dde0 comp rx=0 tx=0).stop 2024-05-21T15:32:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.944+0000 7fea3881b640 1 -- 172.21.15.38:0/1702841718 shutdown_connections 2024-05-21T15:32:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.944+0000 7fea3881b640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea341078d0 0x7fea341a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.944+0000 7fea3881b640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea34102bd0 0x7fea34199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.944+0000 7fea3881b640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fea1c0a2c10 0x7fea1c0a5120 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.944+0000 7fea3881b640 1 --2- 172.21.15.38:0/1702841718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea34101990 0x7fea341992e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.944+0000 7fea3881b640 1 -- 172.21.15.38:0/1702841718 >> 172.21.15.38:0/1702841718 conn(0x7fea340fcb80 msgr2=0x7fea34108d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.945+0000 7fea3881b640 1 -- 172.21.15.38:0/1702841718 shutdown_connections 2024-05-21T15:32:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:37.945+0000 7fea3881b640 1 -- 172.21.15.38:0/1702841718 wait complete. 2024-05-21T15:32:37.962 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-21T15:32:38.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.087+0000 7f470bade640 1 -- 172.21.15.38:0/2436549502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4704101990 msgr2=0x7f4704102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:38.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.087+0000 7f470bade640 1 --2- 172.21.15.38:0/2436549502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4704101990 0x7f4704102690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4700009900 tx=0x7f470002a410 comp rx=0 tx=0).stop 2024-05-21T15:32:38.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.087+0000 7f470bade640 1 -- 172.21.15.38:0/2436549502 shutdown_connections 2024-05-21T15:32:38.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.087+0000 7f470bade640 1 --2- 172.21.15.38:0/2436549502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47041078d0 0x7f4704109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.087+0000 7f470bade640 1 --2- 172.21.15.38:0/2436549502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4704102bd0 0x7f4704102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.087+0000 7f470bade640 1 --2- 172.21.15.38:0/2436549502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4704101990 0x7f4704102690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.087+0000 7f470bade640 1 -- 172.21.15.38:0/2436549502 >> 172.21.15.38:0/2436549502 conn(0x7f47040fcb80 msgr2=0x7f47040ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.087+0000 7f470bade640 1 -- 172.21.15.38:0/2436549502 shutdown_connections 2024-05-21T15:32:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.087+0000 7f470bade640 1 -- 172.21.15.38:0/2436549502 wait complete. 2024-05-21T15:32:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.087+0000 7f470bade640 1 Processor -- start 2024-05-21T15:32:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.087+0000 7f470bade640 1 -- start start 2024-05-21T15:32:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470bade640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4704101990 0x7f47040747f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470bade640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4704102bd0 0x7f470419e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470bade640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47041078d0 0x7f47041a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470bade640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f470410d610 con 0x7f47041078d0 2024-05-21T15:32:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470bade640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f470410d490 con 0x7f4704101990 2024-05-21T15:32:38.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470bade640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f470410d790 con 0x7f4704102bd0 2024-05-21T15:32:38.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470a07b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47041078d0 0x7f47041a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:38.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470a07b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47041078d0 0x7f47041a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38620/0 (socket says 172.21.15.38:38620) 2024-05-21T15:32:38.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470a07b640 1 -- 172.21.15.38:0/3306685366 learned_addr learned my addr 172.21.15.38:0/3306685366 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:38.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470987a640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4704101990 0x7f47040747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:38.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f4709079640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4704102bd0 0x7f470419e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:38.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470a07b640 1 -- 172.21.15.38:0/3306685366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4704102bd0 msgr2=0x7f470419e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:38.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470a07b640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4704102bd0 0x7f470419e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:38.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470a07b640 1 -- 172.21.15.38:0/3306685366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4704101990 msgr2=0x7f47040747f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:38.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470a07b640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4704101990 0x7f47040747f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:38.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470a07b640 1 -- 172.21.15.38:0/3306685366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47041a0dd0 con 0x7f47041078d0 2024-05-21T15:32:38.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f4709079640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4704102bd0 0x7f470419e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:32:38.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470a07b640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47041078d0 0x7f47041a05b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00eaa0 tx=0x7f46fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:38.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f46faffd640 1 -- 172.21.15.38:0/3306685366 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc00cdb0 con 0x7f47041078d0 2024-05-21T15:32:38.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f46faffd640 1 -- 172.21.15.38:0/3306685366 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46fc005020 con 0x7f47041078d0 2024-05-21T15:32:38.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.088+0000 7f470bade640 1 -- 172.21.15.38:0/3306685366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47041a10c0 con 0x7f47041078d0 2024-05-21T15:32:38.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.089+0000 7f46faffd640 1 -- 172.21.15.38:0/3306685366 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0116b0 con 0x7f47041078d0 2024-05-21T15:32:38.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.089+0000 7f470bade640 1 -- 172.21.15.38:0/3306685366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47041a15c0 con 0x7f47041078d0 2024-05-21T15:32:38.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.090+0000 7f470bade640 1 -- 172.21.15.38:0/3306685366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47041066e0 con 0x7f47041078d0 2024-05-21T15:32:38.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.091+0000 7f46faffd640 1 -- 172.21.15.38:0/3306685366 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f46fc005560 con 0x7f47041078d0 2024-05-21T15:32:38.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.092+0000 7f46faffd640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f46f00a2b90 0x7f46f00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:38.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.092+0000 7f470987a640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f46f00a2b90 0x7f46f00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:38.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.092+0000 7f46faffd640 1 -- 172.21.15.38:0/3306685366 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f46fc0cf550 con 0x7f47041078d0 2024-05-21T15:32:38.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.093+0000 7f470987a640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f46f00a2b90 0x7f46f00a50a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f470002aa00 tx=0x7f47000315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:38.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.096+0000 7f46faffd640 1 -- 172.21.15.38:0/3306685366 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f46fc016030 con 0x7f47041078d0 2024-05-21T15:32:38.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.321+0000 7f470bade640 1 -- 172.21.15.38:0/3306685366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f470402d2a0 con 0x7f47041078d0 2024-05-21T15:32:38.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:38 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1702841718' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T15:32:38.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:38 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1702841718' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-21T15:32:38.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:38 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] up:active 2024-05-21T15:32:38.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:38 smithi045 ceph-mon[36409]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-21T15:32:38.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:38 smithi045 ceph-mon[36409]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-21T15:32:38.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:38 smithi045 ceph-mon[36409]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-21T15:32:38.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:38 smithi045 ceph-mon[36409]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-21T15:32:38.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:38 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1702841718' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T15:32:38.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:38 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1702841718' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-21T15:32:38.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:38 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] up:active 2024-05-21T15:32:38.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:38 smithi038 ceph-mon[31995]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-21T15:32:38.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:38 smithi038 ceph-mon[31995]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-21T15:32:38.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:38 smithi038 ceph-mon[31995]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-21T15:32:38.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:38 smithi038 ceph-mon[31995]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-21T15:32:38.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:38 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1702841718' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T15:32:38.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:38 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1702841718' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-21T15:32:38.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:38 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] up:active 2024-05-21T15:32:38.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:38 smithi053 ceph-mon[36496]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-21T15:32:38.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:38 smithi053 ceph-mon[36496]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-21T15:32:38.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:38 smithi053 ceph-mon[36496]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-21T15:32:38.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:38 smithi053 ceph-mon[36496]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-21T15:32:38.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.956+0000 7f46faffd640 1 -- 172.21.15.38:0/3306685366 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f46fc09c5e0 con 0x7f47041078d0 2024-05-21T15:32:38.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.959+0000 7f470bade640 1 -- 172.21.15.38:0/3306685366 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f46f00a2b90 msgr2=0x7f46f00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:38.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.959+0000 7f470bade640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f46f00a2b90 0x7f46f00a50a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f470002aa00 tx=0x7f47000315b0 comp rx=0 tx=0).stop 2024-05-21T15:32:38.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.959+0000 7f470bade640 1 -- 172.21.15.38:0/3306685366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47041078d0 msgr2=0x7f47041a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:38.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.959+0000 7f470bade640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47041078d0 0x7f47041a05b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00eaa0 tx=0x7f46fc00ef60 comp rx=0 tx=0).stop 2024-05-21T15:32:38.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.959+0000 7f470bade640 1 -- 172.21.15.38:0/3306685366 shutdown_connections 2024-05-21T15:32:38.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.959+0000 7f470bade640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f46f00a2b90 0x7f46f00a50a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:38.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.959+0000 7f470bade640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47041078d0 0x7f47041a05b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:38.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.959+0000 7f470bade640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4704102bd0 0x7f470419e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:38.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.959+0000 7f470bade640 1 --2- 172.21.15.38:0/3306685366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4704101990 0x7f47040747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:38.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.959+0000 7f470bade640 1 -- 172.21.15.38:0/3306685366 >> 172.21.15.38:0/3306685366 conn(0x7f47040fcb80 msgr2=0x7f47041038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:38.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.959+0000 7f470bade640 1 -- 172.21.15.38:0/3306685366 shutdown_connections 2024-05-21T15:32:38.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:38.959+0000 7f470bade640 1 -- 172.21.15.38:0/3306685366 wait complete. 2024-05-21T15:32:38.977 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-21T15:32:38.977 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-21T15:32:39.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.101+0000 7f3720996640 1 -- 172.21.15.38:0/4245559870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c101990 msgr2=0x7f371c102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:39.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.101+0000 7f3720996640 1 --2- 172.21.15.38:0/4245559870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c101990 0x7f371c102690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f370c009900 tx=0x7f370c02a410 comp rx=0 tx=0).stop 2024-05-21T15:32:39.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.101+0000 7f3720996640 1 -- 172.21.15.38:0/4245559870 shutdown_connections 2024-05-21T15:32:39.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.101+0000 7f3720996640 1 --2- 172.21.15.38:0/4245559870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f371c1078d0 0x7f371c109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:39.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.101+0000 7f3720996640 1 --2- 172.21.15.38:0/4245559870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371c102bd0 0x7f371c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:39.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.101+0000 7f3720996640 1 --2- 172.21.15.38:0/4245559870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c101990 0x7f371c102690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:39.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.101+0000 7f3720996640 1 -- 172.21.15.38:0/4245559870 >> 172.21.15.38:0/4245559870 conn(0x7f371c0fcb80 msgr2=0x7f371c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:39.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.102+0000 7f3720996640 1 -- 172.21.15.38:0/4245559870 shutdown_connections 2024-05-21T15:32:39.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.102+0000 7f3720996640 1 -- 172.21.15.38:0/4245559870 wait complete. 2024-05-21T15:32:39.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.102+0000 7f3720996640 1 Processor -- start 2024-05-21T15:32:39.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.102+0000 7f3720996640 1 -- start start 2024-05-21T15:32:39.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f3720996640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f371c101990 0x7f371c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:39.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f3720996640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371c102bd0 0x7f371c199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:39.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f3720996640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c1078d0 0x7f371c1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:39.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f3720996640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f371c10d5c0 con 0x7f371c1078d0 2024-05-21T15:32:39.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f3720996640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f371c10d440 con 0x7f371c102bd0 2024-05-21T15:32:39.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f3720996640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f371c10d740 con 0x7f371c101990 2024-05-21T15:32:39.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f371ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c1078d0 0x7f371c1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:39.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f371ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c1078d0 0x7f371c1a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38640/0 (socket says 172.21.15.38:38640) 2024-05-21T15:32:39.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f371ad9d640 1 -- 172.21.15.38:0/4061275708 learned_addr learned my addr 172.21.15.38:0/4061275708 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:39.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f3719d9b640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371c102bd0 0x7f371c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:39.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f371a59c640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f371c101990 0x7f371c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:39.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f371ad9d640 1 -- 172.21.15.38:0/4061275708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f371c101990 msgr2=0x7f371c199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:39.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f371ad9d640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f371c101990 0x7f371c199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:39.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f371ad9d640 1 -- 172.21.15.38:0/4061275708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371c102bd0 msgr2=0x7f371c199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:39.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f371ad9d640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371c102bd0 0x7f371c199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:39.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f371ad9d640 1 -- 172.21.15.38:0/4061275708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371c1a0d50 con 0x7f371c1078d0 2024-05-21T15:32:39.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f371a59c640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f371c101990 0x7f371c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:32:39.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f371ad9d640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c1078d0 0x7f371c1a0790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f371400cce0 tx=0x7f371400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:39.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f370b7fe640 1 -- 172.21.15.38:0/4061275708 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371400f450 con 0x7f371c1078d0 2024-05-21T15:32:39.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f370b7fe640 1 -- 172.21.15.38:0/4061275708 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371400fad0 con 0x7f371c1078d0 2024-05-21T15:32:39.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f3720996640 1 -- 172.21.15.38:0/4061275708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371c1a0fe0 con 0x7f371c1078d0 2024-05-21T15:32:39.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f370b7fe640 1 -- 172.21.15.38:0/4061275708 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714018c10 con 0x7f371c1078d0 2024-05-21T15:32:39.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.103+0000 7f3720996640 1 -- 172.21.15.38:0/4061275708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371c1a1450 con 0x7f371c1078d0 2024-05-21T15:32:39.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.104+0000 7f3720996640 1 -- 172.21.15.38:0/4061275708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f371c1a1710 con 0x7f371c1078d0 2024-05-21T15:32:39.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.106+0000 7f370b7fe640 1 -- 172.21.15.38:0/4061275708 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f371400f5f0 con 0x7f371c1078d0 2024-05-21T15:32:39.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.107+0000 7f370b7fe640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f37040a2b90 0x7f37040a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:39.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.107+0000 7f371a59c640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f37040a2b90 0x7f37040a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:39.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.107+0000 7f370b7fe640 1 -- 172.21.15.38:0/4061275708 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f37140cf620 con 0x7f371c1078d0 2024-05-21T15:32:39.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.107+0000 7f371a59c640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f37040a2b90 0x7f37040a50a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f370c000c00 tx=0x7f370c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:39.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.111+0000 7f370b7fe640 1 -- 172.21.15.38:0/4061275708 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3714014030 con 0x7f371c1078d0 2024-05-21T15:32:39.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.325+0000 7f3720996640 1 -- 172.21.15.38:0/4061275708 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f371c109550 con 0x7f37040a2b90 2024-05-21T15:32:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:39 smithi045 ceph-mon[36409]: pgmap v294: 97 pgs: 89 active+clean, 8 creating+peering; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 10 op/s 2024-05-21T15:32:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:39 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3306685366' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T15:32:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:39 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3306685366' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-21T15:32:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:39 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] up:active 2024-05-21T15:32:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:39 smithi045 ceph-mon[36409]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-21T15:32:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:39 smithi045 ceph-mon[36409]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-21T15:32:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:39 smithi045 ceph-mon[36409]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-05-21T15:32:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:39 smithi045 ceph-mon[36409]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-21T15:32:39.520 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:39 smithi038 ceph-mon[31995]: pgmap v294: 97 pgs: 89 active+clean, 8 creating+peering; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 10 op/s 2024-05-21T15:32:39.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:39 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3306685366' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T15:32:39.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:39 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3306685366' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-21T15:32:39.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:39 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] up:active 2024-05-21T15:32:39.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:39 smithi038 ceph-mon[31995]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-21T15:32:39.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:39 smithi038 ceph-mon[31995]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-21T15:32:39.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:39 smithi038 ceph-mon[31995]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-05-21T15:32:39.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:39 smithi038 ceph-mon[31995]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-21T15:32:39.521 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:32:39 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:32:39.330+0000 7f4852ffd700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:32:39.521 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:32:39 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:32:39.330+0000 7f4852ffd700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:32:39.521 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:32:39 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:32:39.330+0000 7f4852ffd700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:32:39.522 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:32:39 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:32:39.330+0000 7f4852ffd700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:32:39.522 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:32:39 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x[32227]: 2024-05-21T15:32:39.330+0000 7f4852ffd700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:32:39.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:39 smithi053 ceph-mon[36496]: pgmap v294: 97 pgs: 89 active+clean, 8 creating+peering; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 10 op/s 2024-05-21T15:32:39.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:39 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3306685366' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T15:32:39.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:39 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3306685366' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-21T15:32:39.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:39 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] up:active 2024-05-21T15:32:39.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:39 smithi053 ceph-mon[36496]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-21T15:32:39.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:39 smithi053 ceph-mon[36496]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-21T15:32:39.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:39 smithi053 ceph-mon[36496]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-05-21T15:32:39.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:39 smithi053 ceph-mon[36496]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-21T15:32:39.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.989+0000 7f370b7fe640 1 -- 172.21.15.38:0/4061275708 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f371c109550 con 0x7f37040a2b90 2024-05-21T15:32:39.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.991+0000 7f3720996640 1 -- 172.21.15.38:0/4061275708 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f37040a2b90 msgr2=0x7f37040a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:39.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.991+0000 7f3720996640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f37040a2b90 0x7f37040a50a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f370c000c00 tx=0x7f370c031400 comp rx=0 tx=0).stop 2024-05-21T15:32:39.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.991+0000 7f3720996640 1 -- 172.21.15.38:0/4061275708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c1078d0 msgr2=0x7f371c1a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:39.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.991+0000 7f3720996640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c1078d0 0x7f371c1a0790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f371400cce0 tx=0x7f371400a620 comp rx=0 tx=0).stop 2024-05-21T15:32:39.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.991+0000 7f3720996640 1 -- 172.21.15.38:0/4061275708 shutdown_connections 2024-05-21T15:32:39.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.991+0000 7f3720996640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f37040a2b90 0x7f37040a50a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:39.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.991+0000 7f3720996640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f371c1078d0 0x7f371c1a0790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:39.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.991+0000 7f3720996640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371c102bd0 0x7f371c199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:39.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.991+0000 7f3720996640 1 --2- 172.21.15.38:0/4061275708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f371c101990 0x7f371c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:39.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.991+0000 7f3720996640 1 -- 172.21.15.38:0/4061275708 >> 172.21.15.38:0/4061275708 conn(0x7f371c0fcb80 msgr2=0x7f371c108e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:39.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.991+0000 7f3720996640 1 -- 172.21.15.38:0/4061275708 shutdown_connections 2024-05-21T15:32:39.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:39.991+0000 7f3720996640 1 -- 172.21.15.38:0/4061275708 wait complete. 2024-05-21T15:32:40.003 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-21T15:32:40.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.124+0000 7f8ad5b84640 1 -- 172.21.15.38:0/844401190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0102aa0 msgr2=0x7f8ad01037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.124+0000 7f8ad5b84640 1 --2- 172.21.15.38:0/844401190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0102aa0 0x7f8ad01037a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009900 tx=0x7f8ac002a410 comp rx=0 tx=0).stop 2024-05-21T15:32:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.124+0000 7f8ad5b84640 1 -- 172.21.15.38:0/844401190 shutdown_connections 2024-05-21T15:32:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.124+0000 7f8ad5b84640 1 --2- 172.21.15.38:0/844401190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad01089e0 0x7f8ad010ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.124+0000 7f8ad5b84640 1 --2- 172.21.15.38:0/844401190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ad0103ce0 0x7f8ad01040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.124+0000 7f8ad5b84640 1 --2- 172.21.15.38:0/844401190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0102aa0 0x7f8ad01037a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.124+0000 7f8ad5b84640 1 -- 172.21.15.38:0/844401190 >> 172.21.15.38:0/844401190 conn(0x7f8ad00fdc70 msgr2=0x7f8ad0100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.124+0000 7f8ad5b84640 1 -- 172.21.15.38:0/844401190 shutdown_connections 2024-05-21T15:32:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.124+0000 7f8ad5b84640 1 -- 172.21.15.38:0/844401190 wait complete. 2024-05-21T15:32:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.125+0000 7f8ad5b84640 1 Processor -- start 2024-05-21T15:32:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.125+0000 7f8ad5b84640 1 -- start start 2024-05-21T15:32:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.125+0000 7f8ad5b84640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad0102aa0 0x7f8ad01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.125+0000 7f8ad5b84640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0103ce0 0x7f8ad01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:40.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.125+0000 7f8ad5b84640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ad01089e0 0x7f8ad01a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.125+0000 7f8ad5b84640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad010e6d0 con 0x7f8ad0103ce0 2024-05-21T15:32:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.125+0000 7f8ad5b84640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad010e550 con 0x7f8ad0102aa0 2024-05-21T15:32:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.125+0000 7f8ad5b84640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad010e850 con 0x7f8ad01089e0 2024-05-21T15:32:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.125+0000 7f8ac7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0103ce0 0x7f8ad01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.125+0000 7f8ac7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0103ce0 0x7f8ad01997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38686/0 (socket says 172.21.15.38:38686) 2024-05-21T15:32:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.125+0000 7f8ac7fff640 1 -- 172.21.15.38:0/3073847052 learned_addr learned my addr 172.21.15.38:0/3073847052 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.125+0000 7f8acf7fe640 1 --2- 172.21.15.38:0/3073847052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad0102aa0 0x7f8ad01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.126+0000 7f8acffff640 1 --2- 172.21.15.38:0/3073847052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ad01089e0 0x7f8ad01a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.126+0000 7f8ac7fff640 1 -- 172.21.15.38:0/3073847052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ad01089e0 msgr2=0x7f8ad01a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.126+0000 7f8ac7fff640 1 --2- 172.21.15.38:0/3073847052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ad01089e0 0x7f8ad01a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.126+0000 7f8ac7fff640 1 -- 172.21.15.38:0/3073847052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad0102aa0 msgr2=0x7f8ad01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.126+0000 7f8ac7fff640 1 --2- 172.21.15.38:0/3073847052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad0102aa0 0x7f8ad01992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.126+0000 7f8ac7fff640 1 -- 172.21.15.38:0/3073847052 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad01a0de0 con 0x7f8ad0103ce0 2024-05-21T15:32:40.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.126+0000 7f8ac7fff640 1 --2- 172.21.15.38:0/3073847052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0103ce0 0x7f8ad01997e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800d950 tx=0x7f8ab800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:40.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.126+0000 7f8acd7fa640 1 -- 172.21.15.38:0/3073847052 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8017070 con 0x7f8ad0103ce0 2024-05-21T15:32:40.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.126+0000 7f8acd7fa640 1 -- 172.21.15.38:0/3073847052 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ab8009c30 con 0x7f8ad0103ce0 2024-05-21T15:32:40.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.126+0000 7f8acd7fa640 1 -- 172.21.15.38:0/3073847052 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800f6b0 con 0x7f8ad0103ce0 2024-05-21T15:32:40.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.126+0000 7f8ad5b84640 1 -- 172.21.15.38:0/3073847052 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad01a10d0 con 0x7f8ad0103ce0 2024-05-21T15:32:40.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.126+0000 7f8ad5b84640 1 -- 172.21.15.38:0/3073847052 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad01a1670 con 0x7f8ad0103ce0 2024-05-21T15:32:40.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.127+0000 7f8ad5b84640 1 -- 172.21.15.38:0/3073847052 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad01077f0 con 0x7f8ad0103ce0 2024-05-21T15:32:40.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.129+0000 7f8acd7fa640 1 -- 172.21.15.38:0/3073847052 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8ab80109d0 con 0x7f8ad0103ce0 2024-05-21T15:32:40.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.129+0000 7f8acd7fa640 1 --2- 172.21.15.38:0/3073847052 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8abc0a2c10 0x7f8abc0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:40.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.129+0000 7f8acf7fe640 1 --2- 172.21.15.38:0/3073847052 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8abc0a2c10 0x7f8abc0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:40.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.129+0000 7f8acd7fa640 1 -- 172.21.15.38:0/3073847052 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f8ab80ceab0 con 0x7f8ad0103ce0 2024-05-21T15:32:40.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.130+0000 7f8acf7fe640 1 --2- 172.21.15.38:0/3073847052 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8abc0a2c10 0x7f8abc0a5120 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0005ab0 tx=0x7f8ac0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:40.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.137+0000 7f8acd7fa640 1 -- 172.21.15.38:0/3073847052 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8ab8009de0 con 0x7f8ad0103ce0 2024-05-21T15:32:40.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.350+0000 7f8ad5b84640 1 -- 172.21.15.38:0/3073847052 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8ad0062020 con 0x7f8abc0a2c10 2024-05-21T15:32:40.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.378+0000 7f8acd7fa640 1 -- 172.21.15.38:0/3073847052 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ad0062020 con 0x7f8abc0a2c10 2024-05-21T15:32:40.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.380+0000 7f8ad5b84640 1 -- 172.21.15.38:0/3073847052 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8abc0a2c10 msgr2=0x7f8abc0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:40.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.380+0000 7f8ad5b84640 1 --2- 172.21.15.38:0/3073847052 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8abc0a2c10 0x7f8abc0a5120 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0005ab0 tx=0x7f8ac0031400 comp rx=0 tx=0).stop 2024-05-21T15:32:40.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.380+0000 7f8ad5b84640 1 -- 172.21.15.38:0/3073847052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0103ce0 msgr2=0x7f8ad01997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:40.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.380+0000 7f8ad5b84640 1 --2- 172.21.15.38:0/3073847052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0103ce0 0x7f8ad01997e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800d950 tx=0x7f8ab800de10 comp rx=0 tx=0).stop 2024-05-21T15:32:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.380+0000 7f8ad5b84640 1 -- 172.21.15.38:0/3073847052 shutdown_connections 2024-05-21T15:32:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.380+0000 7f8ad5b84640 1 --2- 172.21.15.38:0/3073847052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ad01089e0 0x7f8ad01a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.380+0000 7f8ad5b84640 1 --2- 172.21.15.38:0/3073847052 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8abc0a2c10 0x7f8abc0a5120 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.380+0000 7f8ad5b84640 1 --2- 172.21.15.38:0/3073847052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ad0103ce0 0x7f8ad01997e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.380+0000 7f8ad5b84640 1 --2- 172.21.15.38:0/3073847052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad0102aa0 0x7f8ad01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.380+0000 7f8ad5b84640 1 -- 172.21.15.38:0/3073847052 >> 172.21.15.38:0/3073847052 conn(0x7f8ad00fdc70 msgr2=0x7f8ad01049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.380+0000 7f8ad5b84640 1 -- 172.21.15.38:0/3073847052 shutdown_connections 2024-05-21T15:32:40.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.380+0000 7f8ad5b84640 1 -- 172.21.15.38:0/3073847052 wait complete. 2024-05-21T15:32:40.391 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-21T15:32:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.512+0000 7f305ccf7640 1 -- 172.21.15.38:0/3236433841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3058102e70 msgr2=0x7f3058101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.512+0000 7f305ccf7640 1 --2- 172.21.15.38:0/3236433841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3058102e70 0x7f3058101750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3048009900 tx=0x7f304802a410 comp rx=0 tx=0).stop 2024-05-21T15:32:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.513+0000 7f305ccf7640 1 -- 172.21.15.38:0/3236433841 shutdown_connections 2024-05-21T15:32:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.513+0000 7f305ccf7640 1 --2- 172.21.15.38:0/3236433841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30581079a0 0x7f3058109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.513+0000 7f305ccf7640 1 --2- 172.21.15.38:0/3236433841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3058103350 0x7f3058101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.513+0000 7f305ccf7640 1 --2- 172.21.15.38:0/3236433841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3058102e70 0x7f3058101750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.513+0000 7f305ccf7640 1 -- 172.21.15.38:0/3236433841 >> 172.21.15.38:0/3236433841 conn(0x7f30580fc2f0 msgr2=0x7f30580fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.513+0000 7f305ccf7640 1 -- 172.21.15.38:0/3236433841 shutdown_connections 2024-05-21T15:32:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.513+0000 7f305ccf7640 1 -- 172.21.15.38:0/3236433841 wait complete. 2024-05-21T15:32:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.513+0000 7f305ccf7640 1 Processor -- start 2024-05-21T15:32:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.513+0000 7f305ccf7640 1 -- start start 2024-05-21T15:32:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f305ccf7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3058102e70 0x7f3058198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f305ccf7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581079a0 0x7f3058199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f305ccf7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f305819e300 0x7f30581a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f305ccf7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f305810e700 con 0x7f305819e300 2024-05-21T15:32:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f305ccf7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f305810e580 con 0x7f30581079a0 2024-05-21T15:32:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f305ccf7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f305810e880 con 0x7f3058102e70 2024-05-21T15:32:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f3056d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f305819e300 0x7f30581a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f3055d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581079a0 0x7f3058199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f3056d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f305819e300 0x7f30581a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38696/0 (socket says 172.21.15.38:38696) 2024-05-21T15:32:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f3055d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581079a0 0x7f3058199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:35768/0 (socket says 172.21.15.38:35768) 2024-05-21T15:32:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f3056d9d640 1 -- 172.21.15.38:0/1461369702 learned_addr learned my addr 172.21.15.38:0/1461369702 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f305659c640 1 --2- 172.21.15.38:0/1461369702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3058102e70 0x7f3058198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f3056d9d640 1 -- 172.21.15.38:0/1461369702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3058102e70 msgr2=0x7f3058198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f3056d9d640 1 --2- 172.21.15.38:0/1461369702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3058102e70 0x7f3058198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f3056d9d640 1 -- 172.21.15.38:0/1461369702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581079a0 msgr2=0x7f3058199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f3056d9d640 1 --2- 172.21.15.38:0/1461369702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581079a0 0x7f3058199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f3056d9d640 1 -- 172.21.15.38:0/1461369702 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30581a0d90 con 0x7f305819e300 2024-05-21T15:32:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f3055d9b640 1 --2- 172.21.15.38:0/1461369702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581079a0 0x7f3058199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:32:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.514+0000 7f3056d9d640 1 --2- 172.21.15.38:0/1461369702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f305819e300 0x7f30581a0720 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f305000eb10 tx=0x7f305000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.515+0000 7f30477fe640 1 -- 172.21.15.38:0/1461369702 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050004b20 con 0x7f305819e300 2024-05-21T15:32:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.515+0000 7f305ccf7640 1 -- 172.21.15.38:0/1461369702 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30581a1080 con 0x7f305819e300 2024-05-21T15:32:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.515+0000 7f30477fe640 1 -- 172.21.15.38:0/1461369702 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3050004cc0 con 0x7f305819e300 2024-05-21T15:32:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.515+0000 7f30477fe640 1 -- 172.21.15.38:0/1461369702 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050010760 con 0x7f305819e300 2024-05-21T15:32:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.515+0000 7f305ccf7640 1 -- 172.21.15.38:0/1461369702 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30581a15d0 con 0x7f305819e300 2024-05-21T15:32:40.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.516+0000 7f305ccf7640 1 -- 172.21.15.38:0/1461369702 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30581060a0 con 0x7f305819e300 2024-05-21T15:32:40.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.517+0000 7f30477fe640 1 -- 172.21.15.38:0/1461369702 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3050005020 con 0x7f305819e300 2024-05-21T15:32:40.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.518+0000 7f30477fe640 1 --2- 172.21.15.38:0/1461369702 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f30400a2b90 0x7f30400a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:40.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.518+0000 7f30477fe640 1 -- 172.21.15.38:0/1461369702 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f30500cf540 con 0x7f305819e300 2024-05-21T15:32:40.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.518+0000 7f305659c640 1 --2- 172.21.15.38:0/1461369702 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f30400a2b90 0x7f30400a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:40.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.519+0000 7f305659c640 1 --2- 172.21.15.38:0/1461369702 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f30400a2b90 0x7f30400a50a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3048009230 tx=0x7f3048031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:40.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.523+0000 7f30477fe640 1 -- 172.21.15.38:0/1461369702 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f305009c550 con 0x7f305819e300 2024-05-21T15:32:40.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.739+0000 7f305ccf7640 1 -- 172.21.15.38:0/1461369702 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f3058061f90 con 0x7f30400a2b90 2024-05-21T15:32:40.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.744+0000 7f30477fe640 1 -- 172.21.15.38:0/1461369702 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3058061f90 con 0x7f30400a2b90 2024-05-21T15:32:40.745 INFO:teuthology.orchestra.run.smithi038.stdout:/volumes/qa/sv_0/97aed952-d14c-4ab2-bc20-61cb0703c323 2024-05-21T15:32:40.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.747+0000 7f305ccf7640 1 -- 172.21.15.38:0/1461369702 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f30400a2b90 msgr2=0x7f30400a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:40.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.747+0000 7f305ccf7640 1 --2- 172.21.15.38:0/1461369702 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f30400a2b90 0x7f30400a50a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3048009230 tx=0x7f3048031400 comp rx=0 tx=0).stop 2024-05-21T15:32:40.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.747+0000 7f305ccf7640 1 -- 172.21.15.38:0/1461369702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f305819e300 msgr2=0x7f30581a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:40.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.747+0000 7f305ccf7640 1 --2- 172.21.15.38:0/1461369702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f305819e300 0x7f30581a0720 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f305000eb10 tx=0x7f305000c2d0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.747+0000 7f305ccf7640 1 -- 172.21.15.38:0/1461369702 shutdown_connections 2024-05-21T15:32:40.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.747+0000 7f305ccf7640 1 --2- 172.21.15.38:0/1461369702 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f30400a2b90 0x7f30400a50a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.747+0000 7f305ccf7640 1 --2- 172.21.15.38:0/1461369702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f305819e300 0x7f30581a0720 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.747+0000 7f305ccf7640 1 --2- 172.21.15.38:0/1461369702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581079a0 0x7f3058199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.747+0000 7f305ccf7640 1 --2- 172.21.15.38:0/1461369702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3058102e70 0x7f3058198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.747+0000 7f305ccf7640 1 -- 172.21.15.38:0/1461369702 >> 172.21.15.38:0/1461369702 conn(0x7f30580fc2f0 msgr2=0x7f30580fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:40.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.747+0000 7f305ccf7640 1 -- 172.21.15.38:0/1461369702 shutdown_connections 2024-05-21T15:32:40.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.747+0000 7f305ccf7640 1 -- 172.21.15.38:0/1461369702 wait complete. 2024-05-21T15:32:40.759 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-21T15:32:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.879+0000 7f4f1f38f640 1 -- 172.21.15.38:0/647490778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f181019b0 msgr2=0x7f4f181026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.879+0000 7f4f1f38f640 1 --2- 172.21.15.38:0/647490778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f181019b0 0x7f4f181026b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009be0 tx=0x7f4f1402a6c0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.879+0000 7f4f1f38f640 1 -- 172.21.15.38:0/647490778 shutdown_connections 2024-05-21T15:32:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.879+0000 7f4f1f38f640 1 --2- 172.21.15.38:0/647490778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f181078f0 0x7f4f18109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.879+0000 7f4f1f38f640 1 --2- 172.21.15.38:0/647490778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f18102bf0 0x7f4f18103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.879+0000 7f4f1f38f640 1 --2- 172.21.15.38:0/647490778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f181019b0 0x7f4f181026b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.879+0000 7f4f1f38f640 1 -- 172.21.15.38:0/647490778 >> 172.21.15.38:0/647490778 conn(0x7f4f180fcbe0 msgr2=0x7f4f180ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.879+0000 7f4f1f38f640 1 -- 172.21.15.38:0/647490778 shutdown_connections 2024-05-21T15:32:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.879+0000 7f4f1f38f640 1 -- 172.21.15.38:0/647490778 wait complete. 2024-05-21T15:32:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.880+0000 7f4f1f38f640 1 Processor -- start 2024-05-21T15:32:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.880+0000 7f4f1f38f640 1 -- start start 2024-05-21T15:32:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.880+0000 7f4f1f38f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f181019b0 0x7f4f18199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.880+0000 7f4f1f38f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18102bf0 0x7f4f181997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.880+0000 7f4f1f38f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f181078f0 0x7f4f181a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.880+0000 7f4f1f38f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1810e6d0 con 0x7f4f181078f0 2024-05-21T15:32:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.880+0000 7f4f1f38f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1810e550 con 0x7f4f18102bf0 2024-05-21T15:32:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.880+0000 7f4f1f38f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1810e850 con 0x7f4f181019b0 2024-05-21T15:32:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.880+0000 7f4f1d92c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f181078f0 0x7f4f181a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.880+0000 7f4f1d92c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f181078f0 0x7f4f181a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38712/0 (socket says 172.21.15.38:38712) 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.880+0000 7f4f1d92c640 1 -- 172.21.15.38:0/2996535987 learned_addr learned my addr 172.21.15.38:0/2996535987 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.880+0000 7f4f1c92a640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18102bf0 0x7f4f181997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.880+0000 7f4f1d12b640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f181019b0 0x7f4f18199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.881+0000 7f4f1d92c640 1 -- 172.21.15.38:0/2996535987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f181019b0 msgr2=0x7f4f18199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.881+0000 7f4f1d92c640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f181019b0 0x7f4f18199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.881+0000 7f4f1d92c640 1 -- 172.21.15.38:0/2996535987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18102bf0 msgr2=0x7f4f181997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.881+0000 7f4f1d92c640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18102bf0 0x7f4f181997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.881+0000 7f4f1d92c640 1 -- 172.21.15.38:0/2996535987 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f181a0da0 con 0x7f4f181078f0 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.881+0000 7f4f1d12b640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f181019b0 0x7f4f18199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.881+0000 7f4f1d92c640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f181078f0 0x7f4f181a0730 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000e940 tx=0x7f4f1000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.881+0000 7f4f0a7fc640 1 -- 172.21.15.38:0/2996535987 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1000cd80 con 0x7f4f181078f0 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.881+0000 7f4f0a7fc640 1 -- 172.21.15.38:0/2996535987 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f100035f0 con 0x7f4f181078f0 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.881+0000 7f4f1f38f640 1 -- 172.21.15.38:0/2996535987 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f181a1090 con 0x7f4f181078f0 2024-05-21T15:32:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.881+0000 7f4f0a7fc640 1 -- 172.21.15.38:0/2996535987 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10010690 con 0x7f4f181078f0 2024-05-21T15:32:40.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.881+0000 7f4f1f38f640 1 -- 172.21.15.38:0/2996535987 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f181a1590 con 0x7f4f181078f0 2024-05-21T15:32:40.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.882+0000 7f4f1f38f640 1 -- 172.21.15.38:0/2996535987 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f18106700 con 0x7f4f181078f0 2024-05-21T15:32:40.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.884+0000 7f4f0a7fc640 1 -- 172.21.15.38:0/2996535987 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4f100037a0 con 0x7f4f181078f0 2024-05-21T15:32:40.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.884+0000 7f4f0a7fc640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f040a2b90 0x7f4f040a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:40.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.884+0000 7f4f0a7fc640 1 -- 172.21.15.38:0/2996535987 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4f100cf310 con 0x7f4f181078f0 2024-05-21T15:32:40.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.884+0000 7f4f1d12b640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f040a2b90 0x7f4f040a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:40.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.885+0000 7f4f1d12b640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f040a2b90 0x7f4f040a50a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4f14005cc0 tx=0x7f4f14009840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:40.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.888+0000 7f4f0a7fc640 1 -- 172.21.15.38:0/2996535987 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4f1009c450 con 0x7f4f181078f0 2024-05-21T15:32:40.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:40.993+0000 7f4f0a7fc640 1 -- 172.21.15.38:0/2996535987 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4f100983c0 con 0x7f4f181078f0 2024-05-21T15:32:41.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:40 smithi038 ceph-mon[31995]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:41.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:40 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] up:active 2024-05-21T15:32:41.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:40 smithi038 ceph-mon[31995]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-21T15:32:41.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:40 smithi038 ceph-mon[31995]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-21T15:32:41.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:40 smithi038 ceph-mon[31995]: Cluster is now healthy 2024-05-21T15:32:41.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:40 smithi038 ceph-mon[31995]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-05-21T15:32:41.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:40 smithi038 ceph-mon[31995]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-21T15:32:41.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:40 smithi038 ceph-mon[31995]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-05-21T15:32:41.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.111+0000 7f4f1f38f640 1 -- 172.21.15.38:0/2996535987 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4f18061f90 con 0x7f4f040a2b90 2024-05-21T15:32:41.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.136+0000 7f4f0a7fc640 1 -- 172.21.15.38:0/2996535987 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4f18061f90 con 0x7f4f040a2b90 2024-05-21T15:32:41.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.138+0000 7f4f1f38f640 1 -- 172.21.15.38:0/2996535987 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f040a2b90 msgr2=0x7f4f040a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.138+0000 7f4f1f38f640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f040a2b90 0x7f4f040a50a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4f14005cc0 tx=0x7f4f14009840 comp rx=0 tx=0).stop 2024-05-21T15:32:41.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.138+0000 7f4f1f38f640 1 -- 172.21.15.38:0/2996535987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f181078f0 msgr2=0x7f4f181a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.138+0000 7f4f1f38f640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f181078f0 0x7f4f181a0730 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000e940 tx=0x7f4f1000ee00 comp rx=0 tx=0).stop 2024-05-21T15:32:41.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.138+0000 7f4f1f38f640 1 -- 172.21.15.38:0/2996535987 shutdown_connections 2024-05-21T15:32:41.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.138+0000 7f4f1f38f640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f040a2b90 0x7f4f040a50a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.138+0000 7f4f1f38f640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f181078f0 0x7f4f181a0730 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.138+0000 7f4f1f38f640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18102bf0 0x7f4f181997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.138+0000 7f4f1f38f640 1 --2- 172.21.15.38:0/2996535987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f181019b0 0x7f4f18199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.138+0000 7f4f1f38f640 1 -- 172.21.15.38:0/2996535987 >> 172.21.15.38:0/2996535987 conn(0x7f4f180fcbe0 msgr2=0x7f4f181038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:41.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.138+0000 7f4f1f38f640 1 -- 172.21.15.38:0/2996535987 shutdown_connections 2024-05-21T15:32:41.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.138+0000 7f4f1f38f640 1 -- 172.21.15.38:0/2996535987 wait complete. 2024-05-21T15:32:41.149 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-21T15:32:41.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:40 smithi045 ceph-mon[36409]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:41.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:40 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] up:active 2024-05-21T15:32:41.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:40 smithi045 ceph-mon[36409]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-21T15:32:41.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:40 smithi045 ceph-mon[36409]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-21T15:32:41.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:40 smithi045 ceph-mon[36409]: Cluster is now healthy 2024-05-21T15:32:41.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:40 smithi045 ceph-mon[36409]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-05-21T15:32:41.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:40 smithi045 ceph-mon[36409]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-21T15:32:41.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:40 smithi045 ceph-mon[36409]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-05-21T15:32:41.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.269+0000 7f6743ab8640 1 -- 172.21.15.38:0/2425279085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f673c102eb0 msgr2=0x7f673c101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.269+0000 7f6743ab8640 1 --2- 172.21.15.38:0/2425279085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f673c102eb0 0x7f673c101790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6738009ca0 tx=0x7f673802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.269+0000 7f6743ab8640 1 -- 172.21.15.38:0/2425279085 shutdown_connections 2024-05-21T15:32:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.269+0000 7f6743ab8640 1 --2- 172.21.15.38:0/2425279085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f673c1079e0 0x7f673c109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.269+0000 7f6743ab8640 1 --2- 172.21.15.38:0/2425279085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f673c103390 0x7f673c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.269+0000 7f6743ab8640 1 --2- 172.21.15.38:0/2425279085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f673c102eb0 0x7f673c101790 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.269+0000 7f6743ab8640 1 -- 172.21.15.38:0/2425279085 >> 172.21.15.38:0/2425279085 conn(0x7f673c0fc330 msgr2=0x7f673c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.269+0000 7f6743ab8640 1 -- 172.21.15.38:0/2425279085 shutdown_connections 2024-05-21T15:32:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.269+0000 7f6743ab8640 1 -- 172.21.15.38:0/2425279085 wait complete. 2024-05-21T15:32:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.270+0000 7f6743ab8640 1 Processor -- start 2024-05-21T15:32:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.270+0000 7f6743ab8640 1 -- start start 2024-05-21T15:32:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.270+0000 7f6743ab8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f673c102eb0 0x7f673c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.270+0000 7f6743ab8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f673c1079e0 0x7f673c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.270+0000 7f6743ab8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f673c19e2f0 0x7f673c1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.270+0000 7f6743ab8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f673c10e740 con 0x7f673c1079e0 2024-05-21T15:32:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.270+0000 7f6743ab8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f673c10e5c0 con 0x7f673c102eb0 2024-05-21T15:32:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.270+0000 7f6743ab8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f673c10e8c0 con 0x7f673c19e2f0 2024-05-21T15:32:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.270+0000 7f6741053640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f673c1079e0 0x7f673c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.270+0000 7f6741854640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f673c102eb0 0x7f673c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.270+0000 7f6741053640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f673c1079e0 0x7f673c199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38736/0 (socket says 172.21.15.38:38736) 2024-05-21T15:32:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.270+0000 7f6741854640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f673c102eb0 0x7f673c198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:35790/0 (socket says 172.21.15.38:35790) 2024-05-21T15:32:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.270+0000 7f6741053640 1 -- 172.21.15.38:0/1591910649 learned_addr learned my addr 172.21.15.38:0/1591910649 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.271+0000 7f6742055640 1 --2- 172.21.15.38:0/1591910649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f673c19e2f0 0x7f673c1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.271+0000 7f6741053640 1 -- 172.21.15.38:0/1591910649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f673c19e2f0 msgr2=0x7f673c1a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.271+0000 7f6741053640 1 --2- 172.21.15.38:0/1591910649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f673c19e2f0 0x7f673c1a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.271+0000 7f6741053640 1 -- 172.21.15.38:0/1591910649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f673c102eb0 msgr2=0x7f673c198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.271+0000 7f6741053640 1 --2- 172.21.15.38:0/1591910649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f673c102eb0 0x7f673c198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.271+0000 7f6741053640 1 -- 172.21.15.38:0/1591910649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c1a0d80 con 0x7f673c1079e0 2024-05-21T15:32:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.271+0000 7f6741854640 1 --2- 172.21.15.38:0/1591910649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f673c102eb0 0x7f673c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:32:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.271+0000 7f6741053640 1 --2- 172.21.15.38:0/1591910649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f673c1079e0 0x7f673c199370 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f672c00d770 tx=0x7f672c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.271+0000 7f6732ffd640 1 -- 172.21.15.38:0/1591910649 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c017070 con 0x7f673c1079e0 2024-05-21T15:32:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.271+0000 7f6732ffd640 1 -- 172.21.15.38:0/1591910649 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672c005500 con 0x7f673c1079e0 2024-05-21T15:32:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.271+0000 7f6743ab8640 1 -- 172.21.15.38:0/1591910649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673c1a1070 con 0x7f673c1079e0 2024-05-21T15:32:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.271+0000 7f6732ffd640 1 -- 172.21.15.38:0/1591910649 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c004e00 con 0x7f673c1079e0 2024-05-21T15:32:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.271+0000 7f6743ab8640 1 -- 172.21.15.38:0/1591910649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673c1a1570 con 0x7f673c1079e0 2024-05-21T15:32:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.272+0000 7f6743ab8640 1 -- 172.21.15.38:0/1591910649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f673c1060e0 con 0x7f673c1079e0 2024-05-21T15:32:41.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.273+0000 7f6732ffd640 1 -- 172.21.15.38:0/1591910649 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f672c004fa0 con 0x7f673c1079e0 2024-05-21T15:32:41.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.274+0000 7f6732ffd640 1 --2- 172.21.15.38:0/1591910649 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f67280a2c20 0x7f67280a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:41.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.274+0000 7f6741854640 1 --2- 172.21.15.38:0/1591910649 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f67280a2c20 0x7f67280a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:41.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.274+0000 7f6732ffd640 1 -- 172.21.15.38:0/1591910649 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f672c0cebb0 con 0x7f673c1079e0 2024-05-21T15:32:41.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.275+0000 7f6741854640 1 --2- 172.21.15.38:0/1591910649 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f67280a2c20 0x7f67280a5130 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6738031c10 tx=0x7f6738005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:41.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.279+0000 7f6732ffd640 1 -- 172.21.15.38:0/1591910649 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f672c09ba60 con 0x7f673c1079e0 2024-05-21T15:32:41.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:40 smithi053 ceph-mon[36496]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:41.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:40 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] up:active 2024-05-21T15:32:41.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:40 smithi053 ceph-mon[36496]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-21T15:32:41.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:40 smithi053 ceph-mon[36496]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-21T15:32:41.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:40 smithi053 ceph-mon[36496]: Cluster is now healthy 2024-05-21T15:32:41.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:40 smithi053 ceph-mon[36496]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-05-21T15:32:41.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:40 smithi053 ceph-mon[36496]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-21T15:32:41.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:40 smithi053 ceph-mon[36496]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-05-21T15:32:41.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.493+0000 7f6743ab8640 1 -- 172.21.15.38:0/1591910649 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f673c061f90 con 0x7f67280a2c20 2024-05-21T15:32:41.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.497+0000 7f6732ffd640 1 -- 172.21.15.38:0/1591910649 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f673c061f90 con 0x7f67280a2c20 2024-05-21T15:32:41.498 INFO:teuthology.orchestra.run.smithi038.stdout:/volumes/qa/sv_1/52b7ee44-44a1-4389-8066-cc7f06d74d34 2024-05-21T15:32:41.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.499+0000 7f6743ab8640 1 -- 172.21.15.38:0/1591910649 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f67280a2c20 msgr2=0x7f67280a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.499+0000 7f6743ab8640 1 --2- 172.21.15.38:0/1591910649 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f67280a2c20 0x7f67280a5130 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6738031c10 tx=0x7f6738005fd0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.499+0000 7f6743ab8640 1 -- 172.21.15.38:0/1591910649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f673c1079e0 msgr2=0x7f673c199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.499+0000 7f6743ab8640 1 --2- 172.21.15.38:0/1591910649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f673c1079e0 0x7f673c199370 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f672c00d770 tx=0x7f672c00dc30 comp rx=0 tx=0).stop 2024-05-21T15:32:41.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.500+0000 7f6743ab8640 1 -- 172.21.15.38:0/1591910649 shutdown_connections 2024-05-21T15:32:41.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.500+0000 7f6743ab8640 1 --2- 172.21.15.38:0/1591910649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f673c19e2f0 0x7f673c1a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.500+0000 7f6743ab8640 1 --2- 172.21.15.38:0/1591910649 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f67280a2c20 0x7f67280a5130 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.500+0000 7f6743ab8640 1 --2- 172.21.15.38:0/1591910649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f673c1079e0 0x7f673c199370 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.500+0000 7f6743ab8640 1 --2- 172.21.15.38:0/1591910649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f673c102eb0 0x7f673c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.500+0000 7f6743ab8640 1 -- 172.21.15.38:0/1591910649 >> 172.21.15.38:0/1591910649 conn(0x7f673c0fc330 msgr2=0x7f673c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:41.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.500+0000 7f6743ab8640 1 -- 172.21.15.38:0/1591910649 shutdown_connections 2024-05-21T15:32:41.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.500+0000 7f6743ab8640 1 -- 172.21.15.38:0/1591910649 wait complete. 2024-05-21T15:32:41.511 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:32:41.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.626+0000 7f694a8f7640 1 -- 172.21.15.38:0/1622040031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6944104120 msgr2=0x7f69441022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.626+0000 7f694a8f7640 1 --2- 172.21.15.38:0/1622040031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6944104120 0x7f69441022a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6934009900 tx=0x7f693402a410 comp rx=0 tx=0).stop 2024-05-21T15:32:41.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.626+0000 7f694a8f7640 1 -- 172.21.15.38:0/1622040031 shutdown_connections 2024-05-21T15:32:41.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.626+0000 7f694a8f7640 1 --2- 172.21.15.38:0/1622040031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6944108a20 0x7f694410ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.626+0000 7f694a8f7640 1 --2- 172.21.15.38:0/1622040031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69441027e0 0x7f6944102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.626+0000 7f694a8f7640 1 --2- 172.21.15.38:0/1622040031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6944104120 0x7f69441022a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.626+0000 7f694a8f7640 1 -- 172.21.15.38:0/1622040031 >> 172.21.15.38:0/1622040031 conn(0x7f69440fdcf0 msgr2=0x7f6944100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:41.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.626+0000 7f694a8f7640 1 -- 172.21.15.38:0/1622040031 shutdown_connections 2024-05-21T15:32:41.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.626+0000 7f694a8f7640 1 -- 172.21.15.38:0/1622040031 wait complete. 2024-05-21T15:32:41.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.627+0000 7f694a8f7640 1 Processor -- start 2024-05-21T15:32:41.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.627+0000 7f694a8f7640 1 -- start start 2024-05-21T15:32:41.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.627+0000 7f694a8f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69441027e0 0x7f6944199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:41.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.627+0000 7f694a8f7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6944104120 0x7f69441998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:41.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.627+0000 7f694a8f7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6944108a20 0x7f69441a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:41.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.627+0000 7f694a8f7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f694410e820 con 0x7f69441027e0 2024-05-21T15:32:41.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.627+0000 7f694a8f7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f694410e6a0 con 0x7f6944104120 2024-05-21T15:32:41.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.627+0000 7f694a8f7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f694410e9a0 con 0x7f6944108a20 2024-05-21T15:32:41.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f6943fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69441027e0 0x7f6944199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:41.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f6943fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69441027e0 0x7f6944199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38750/0 (socket says 172.21.15.38:38750) 2024-05-21T15:32:41.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f6943fff640 1 -- 172.21.15.38:0/1987148534 learned_addr learned my addr 172.21.15.38:0/1987148534 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:41.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f69437fe640 1 --2- 172.21.15.38:0/1987148534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6944104120 0x7f69441998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:41.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f6948e94640 1 --2- 172.21.15.38:0/1987148534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6944108a20 0x7f69441a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:41.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f6943fff640 1 -- 172.21.15.38:0/1987148534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6944108a20 msgr2=0x7f69441a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f6943fff640 1 --2- 172.21.15.38:0/1987148534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6944108a20 0x7f69441a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f6943fff640 1 -- 172.21.15.38:0/1987148534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6944104120 msgr2=0x7f69441998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f6943fff640 1 --2- 172.21.15.38:0/1987148534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6944104120 0x7f69441998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f6943fff640 1 -- 172.21.15.38:0/1987148534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69441a0eb0 con 0x7f69441027e0 2024-05-21T15:32:41.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f6943fff640 1 --2- 172.21.15.38:0/1987148534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69441027e0 0x7f6944199370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6934006010 tx=0x7f6934031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:41.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f69417fa640 1 -- 172.21.15.38:0/1987148534 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693403f030 con 0x7f69441027e0 2024-05-21T15:32:41.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f69417fa640 1 -- 172.21.15.38:0/1987148534 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6934003ad0 con 0x7f69441027e0 2024-05-21T15:32:41.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f69417fa640 1 -- 172.21.15.38:0/1987148534 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934035070 con 0x7f69441027e0 2024-05-21T15:32:41.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f694a8f7640 1 -- 172.21.15.38:0/1987148534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69441a1140 con 0x7f69441027e0 2024-05-21T15:32:41.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.628+0000 7f694a8f7640 1 -- 172.21.15.38:0/1987148534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69441a15e0 con 0x7f69441027e0 2024-05-21T15:32:41.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.629+0000 7f694a8f7640 1 -- 172.21.15.38:0/1987148534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69440691c0 con 0x7f69441027e0 2024-05-21T15:32:41.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.630+0000 7f69417fa640 1 -- 172.21.15.38:0/1987148534 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6934042030 con 0x7f69441027e0 2024-05-21T15:32:41.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.631+0000 7f69417fa640 1 --2- 172.21.15.38:0/1987148534 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f692c0a2d50 0x7f692c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:41.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.631+0000 7f69417fa640 1 -- 172.21.15.38:0/1987148534 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f69340ef570 con 0x7f69441027e0 2024-05-21T15:32:41.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.631+0000 7f69437fe640 1 --2- 172.21.15.38:0/1987148534 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f692c0a2d50 0x7f692c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:41.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.632+0000 7f69437fe640 1 --2- 172.21.15.38:0/1987148534 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f692c0a2d50 0x7f692c0a5260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f694419a410 tx=0x7f693800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:41.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.636+0000 7f69417fa640 1 -- 172.21.15.38:0/1987148534 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f69340bc520 con 0x7f69441027e0 2024-05-21T15:32:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.861+0000 7f694a8f7640 1 -- 172.21.15.38:0/1987148534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f694402d2a0 con 0x7f69441027e0 2024-05-21T15:32:41.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.862+0000 7f69417fa640 1 -- 172.21.15.38:0/1987148534 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11914 (secure 0 0 0) 0x7f69340bbe20 con 0x7f69441027e0 2024-05-21T15:32:41.862 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:41.862 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:32:40.979368+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24550,24515,24602,24523,24551]},"id":1}]} 2024-05-21T15:32:41.862 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-05-21T15:32:41.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.864+0000 7f694a8f7640 1 -- 172.21.15.38:0/1987148534 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f692c0a2d50 msgr2=0x7f692c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.864+0000 7f694a8f7640 1 --2- 172.21.15.38:0/1987148534 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f692c0a2d50 0x7f692c0a5260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f694419a410 tx=0x7f693800b410 comp rx=0 tx=0).stop 2024-05-21T15:32:41.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.864+0000 7f694a8f7640 1 -- 172.21.15.38:0/1987148534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69441027e0 msgr2=0x7f6944199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.864+0000 7f694a8f7640 1 --2- 172.21.15.38:0/1987148534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69441027e0 0x7f6944199370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6934006010 tx=0x7f6934031b60 comp rx=0 tx=0).stop 2024-05-21T15:32:41.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.864+0000 7f694a8f7640 1 -- 172.21.15.38:0/1987148534 shutdown_connections 2024-05-21T15:32:41.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.864+0000 7f694a8f7640 1 --2- 172.21.15.38:0/1987148534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6944108a20 0x7f69441a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.864+0000 7f694a8f7640 1 --2- 172.21.15.38:0/1987148534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6944104120 0x7f69441998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.864+0000 7f694a8f7640 1 --2- 172.21.15.38:0/1987148534 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f692c0a2d50 0x7f692c0a5260 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.864+0000 7f694a8f7640 1 --2- 172.21.15.38:0/1987148534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69441027e0 0x7f6944199370 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.864+0000 7f694a8f7640 1 -- 172.21.15.38:0/1987148534 >> 172.21.15.38:0/1987148534 conn(0x7f69440fdcf0 msgr2=0x7f6944104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:41.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.864+0000 7f694a8f7640 1 -- 172.21.15.38:0/1987148534 shutdown_connections 2024-05-21T15:32:41.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.864+0000 7f694a8f7640 1 -- 172.21.15.38:0/1987148534 wait complete. 2024-05-21T15:32:41.876 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:32:41.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.992+0000 7facd718f640 1 -- 172.21.15.38:0/3016701705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd0108a20 msgr2=0x7facd010ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.992+0000 7facd718f640 1 --2- 172.21.15.38:0/3016701705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd0108a20 0x7facd010ae40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7facc8009ca0 tx=0x7facc802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.992+0000 7facd718f640 1 -- 172.21.15.38:0/3016701705 shutdown_connections 2024-05-21T15:32:41.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.992+0000 7facd718f640 1 --2- 172.21.15.38:0/3016701705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd0108a20 0x7facd010ae40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.992+0000 7facd718f640 1 --2- 172.21.15.38:0/3016701705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facd0103d20 0x7facd0104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.992+0000 7facd718f640 1 --2- 172.21.15.38:0/3016701705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facd0102ae0 0x7facd01037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.992+0000 7facd718f640 1 -- 172.21.15.38:0/3016701705 >> 172.21.15.38:0/3016701705 conn(0x7facd00fdcf0 msgr2=0x7facd0100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:41.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.992+0000 7facd718f640 1 -- 172.21.15.38:0/3016701705 shutdown_connections 2024-05-21T15:32:41.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.992+0000 7facd718f640 1 -- 172.21.15.38:0/3016701705 wait complete. 2024-05-21T15:32:41.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.993+0000 7facd718f640 1 Processor -- start 2024-05-21T15:32:41.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.993+0000 7facd718f640 1 -- start start 2024-05-21T15:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.993+0000 7facd718f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facd0102ae0 0x7facd0076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.993+0000 7facd718f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facd0103d20 0x7facd0074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.993+0000 7facd718f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd0108a20 0x7facd00754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.993+0000 7facd718f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7facd010e7b0 con 0x7facd0108a20 2024-05-21T15:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.993+0000 7facd718f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7facd010e630 con 0x7facd0103d20 2024-05-21T15:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.993+0000 7facd718f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7facd010e930 con 0x7facd0102ae0 2024-05-21T15:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.993+0000 7facd572c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd0108a20 0x7facd00754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.993+0000 7facd572c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd0108a20 0x7facd00754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38780/0 (socket says 172.21.15.38:38780) 2024-05-21T15:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.993+0000 7facd572c640 1 -- 172.21.15.38:0/2661159916 learned_addr learned my addr 172.21.15.38:0/2661159916 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.994+0000 7faccffff640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facd0103d20 0x7facd0074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.994+0000 7facd4f2b640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facd0102ae0 0x7facd0076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.994+0000 7facd572c640 1 -- 172.21.15.38:0/2661159916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facd0102ae0 msgr2=0x7facd0076910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.994+0000 7facd572c640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facd0102ae0 0x7facd0076910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.994+0000 7facd572c640 1 -- 172.21.15.38:0/2661159916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facd0103d20 msgr2=0x7facd0074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.994+0000 7facd572c640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facd0103d20 0x7facd0074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.994+0000 7facd572c640 1 -- 172.21.15.38:0/2661159916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd0075c20 con 0x7facd0108a20 2024-05-21T15:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.994+0000 7facd4f2b640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facd0102ae0 0x7facd0076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.994+0000 7facd572c640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd0108a20 0x7facd00754f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facc8009410 tx=0x7facc802f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.994+0000 7faccdffb640 1 -- 172.21.15.38:0/2661159916 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8003c70 con 0x7facd0108a20 2024-05-21T15:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.994+0000 7faccdffb640 1 -- 172.21.15.38:0/2661159916 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc8003e10 con 0x7facd0108a20 2024-05-21T15:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.994+0000 7facd718f640 1 -- 172.21.15.38:0/2661159916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd01a3250 con 0x7facd0108a20 2024-05-21T15:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.994+0000 7faccdffb640 1 -- 172.21.15.38:0/2661159916 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc802fd50 con 0x7facd0108a20 2024-05-21T15:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.994+0000 7facd718f640 1 -- 172.21.15.38:0/2661159916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd01a3710 con 0x7facd0108a20 2024-05-21T15:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.995+0000 7facd718f640 1 -- 172.21.15.38:0/2661159916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd0107120 con 0x7facd0108a20 2024-05-21T15:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.997+0000 7faccdffb640 1 -- 172.21.15.38:0/2661159916 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7facc8004040 con 0x7facd0108a20 2024-05-21T15:32:41.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.998+0000 7faccdffb640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7facc00a2d50 0x7facc00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:41.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.998+0000 7facd4f2b640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7facc00a2d50 0x7facc00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:41.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.998+0000 7faccdffb640 1 -- 172.21.15.38:0/2661159916 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7facc80f05a0 con 0x7facd0108a20 2024-05-21T15:32:41.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:41.998+0000 7facd4f2b640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7facc00a2d50 0x7facc00a5260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7facc40058c0 tx=0x7facc4005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:42.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.003+0000 7faccdffb640 1 -- 172.21.15.38:0/2661159916 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7facc80bd660 con 0x7facd0108a20 2024-05-21T15:32:42.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.191+0000 7facd718f640 1 -- 172.21.15.38:0/2661159916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facd002d470 con 0x7facd0108a20 2024-05-21T15:32:42.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.193+0000 7faccdffb640 1 -- 172.21.15.38:0/2661159916 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22112 (secure 0 0 0) 0x7facc80bcf60 con 0x7facd0108a20 2024-05-21T15:32:42.193 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:42.193 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":86,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:32:34.657863+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:32:42.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.195+0000 7facd718f640 1 -- 172.21.15.38:0/2661159916 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7facc00a2d50 msgr2=0x7facc00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.195+0000 7facd718f640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7facc00a2d50 0x7facc00a5260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7facc40058c0 tx=0x7facc4005d50 comp rx=0 tx=0).stop 2024-05-21T15:32:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.195+0000 7facd718f640 1 -- 172.21.15.38:0/2661159916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd0108a20 msgr2=0x7facd00754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.195+0000 7facd718f640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd0108a20 0x7facd00754f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facc8009410 tx=0x7facc802f450 comp rx=0 tx=0).stop 2024-05-21T15:32:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.195+0000 7facd718f640 1 -- 172.21.15.38:0/2661159916 shutdown_connections 2024-05-21T15:32:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.195+0000 7facd718f640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7facc00a2d50 0x7facc00a5260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.195+0000 7facd718f640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facd0108a20 0x7facd00754f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.195+0000 7facd718f640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facd0103d20 0x7facd0074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.195+0000 7facd718f640 1 --2- 172.21.15.38:0/2661159916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facd0102ae0 0x7facd0076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.195+0000 7facd718f640 1 -- 172.21.15.38:0/2661159916 >> 172.21.15.38:0/2661159916 conn(0x7facd00fdcf0 msgr2=0x7facd0100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.195+0000 7facd718f640 1 -- 172.21.15.38:0/2661159916 shutdown_connections 2024-05-21T15:32:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.195+0000 7facd718f640 1 -- 172.21.15.38:0/2661159916 wait complete. 2024-05-21T15:32:42.206 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:41 smithi038 ceph-mon[31995]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s wr, 16 op/s 2024-05-21T15:32:42.206 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:41 smithi038 ceph-mon[31995]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:42.206 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:41 smithi038 ceph-mon[31995]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:42.206 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:41 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] up:active 2024-05-21T15:32:42.206 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:41 smithi038 ceph-mon[31995]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-21T15:32:42.206 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:41 smithi038 ceph-mon[31995]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-21T15:32:42.206 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:41 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:32:42.207 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:41 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1987148534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:42.208 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-21T15:32:42.208 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T15:32:42.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:41 smithi045 ceph-mon[36409]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s wr, 16 op/s 2024-05-21T15:32:42.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:41 smithi045 ceph-mon[36409]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:42.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:41 smithi045 ceph-mon[36409]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:42.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:41 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] up:active 2024-05-21T15:32:42.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:41 smithi045 ceph-mon[36409]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-21T15:32:42.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:41 smithi045 ceph-mon[36409]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-21T15:32:42.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:41 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:32:42.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:41 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1987148534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:42.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.323+0000 7f5b4a164640 1 -- 172.21.15.38:0/1634934897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b441078f0 msgr2=0x7f5b44109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:42.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.323+0000 7f5b4a164640 1 --2- 172.21.15.38:0/1634934897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b441078f0 0x7f5b44109d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009ca0 tx=0x7f5b3c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.323+0000 7f5b4a164640 1 -- 172.21.15.38:0/1634934897 shutdown_connections 2024-05-21T15:32:42.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.323+0000 7f5b4a164640 1 --2- 172.21.15.38:0/1634934897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b441078f0 0x7f5b44109d10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.323+0000 7f5b4a164640 1 --2- 172.21.15.38:0/1634934897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b44102bf0 0x7f5b44103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.323+0000 7f5b4a164640 1 --2- 172.21.15.38:0/1634934897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b441019b0 0x7f5b441026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.323+0000 7f5b4a164640 1 -- 172.21.15.38:0/1634934897 >> 172.21.15.38:0/1634934897 conn(0x7f5b440fcba0 msgr2=0x7f5b440ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.323+0000 7f5b4a164640 1 -- 172.21.15.38:0/1634934897 shutdown_connections 2024-05-21T15:32:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.323+0000 7f5b4a164640 1 -- 172.21.15.38:0/1634934897 wait complete. 2024-05-21T15:32:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.324+0000 7f5b4a164640 1 Processor -- start 2024-05-21T15:32:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.324+0000 7f5b4a164640 1 -- start start 2024-05-21T15:32:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.324+0000 7f5b4a164640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b441019b0 0x7f5b44199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.324+0000 7f5b4a164640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b44102bf0 0x7f5b441996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.324+0000 7f5b4a164640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b441078f0 0x7f5b441a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.324+0000 7f5b4a164640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4410d630 con 0x7f5b441019b0 2024-05-21T15:32:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.324+0000 7f5b4a164640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4410d4b0 con 0x7f5b441078f0 2024-05-21T15:32:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.324+0000 7f5b4a164640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4410d7b0 con 0x7f5b44102bf0 2024-05-21T15:32:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b441019b0 0x7f5b44199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b441019b0 0x7f5b44199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38810/0 (socket says 172.21.15.38:38810) 2024-05-21T15:32:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b437fe640 1 -- 172.21.15.38:0/224149660 learned_addr learned my addr 172.21.15.38:0/224149660 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b42ffd640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b44102bf0 0x7f5b441996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b43fff640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b441078f0 0x7f5b441a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b437fe640 1 -- 172.21.15.38:0/224149660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b44102bf0 msgr2=0x7f5b441996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b437fe640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b44102bf0 0x7f5b441996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b437fe640 1 -- 172.21.15.38:0/224149660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b441078f0 msgr2=0x7f5b441a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b437fe640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b441078f0 0x7f5b441a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b437fe640 1 -- 172.21.15.38:0/224149660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b441a0cd0 con 0x7f5b441019b0 2024-05-21T15:32:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b42ffd640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b44102bf0 0x7f5b441996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:32:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b43fff640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b441078f0 0x7f5b441a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:32:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b437fe640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b441019b0 0x7f5b44199190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400e9f0 tx=0x7f5b3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b40ff9640 1 -- 172.21.15.38:0/224149660 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400ccc0 con 0x7f5b441019b0 2024-05-21T15:32:42.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b40ff9640 1 -- 172.21.15.38:0/224149660 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b340035f0 con 0x7f5b441019b0 2024-05-21T15:32:42.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b4a164640 1 -- 172.21.15.38:0/224149660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b441a0fc0 con 0x7f5b441019b0 2024-05-21T15:32:42.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b40ff9640 1 -- 172.21.15.38:0/224149660 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400ce60 con 0x7f5b441019b0 2024-05-21T15:32:42.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.325+0000 7f5b4a164640 1 -- 172.21.15.38:0/224149660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b441a1420 con 0x7f5b441019b0 2024-05-21T15:32:42.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.326+0000 7f5b4a164640 1 -- 172.21.15.38:0/224149660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b44106ff0 con 0x7f5b441019b0 2024-05-21T15:32:42.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.328+0000 7f5b40ff9640 1 -- 172.21.15.38:0/224149660 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5b340037a0 con 0x7f5b441019b0 2024-05-21T15:32:42.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.328+0000 7f5b40ff9640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5b2c0a2cd0 0x7f5b2c0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:42.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.329+0000 7f5b42ffd640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5b2c0a2cd0 0x7f5b2c0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:42.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.329+0000 7f5b40ff9640 1 -- 172.21.15.38:0/224149660 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5b340cf850 con 0x7f5b441019b0 2024-05-21T15:32:42.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.329+0000 7f5b42ffd640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5b2c0a2cd0 0x7f5b2c0a51e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b38005d00 tx=0x7f5b38005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:42.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.332+0000 7f5b40ff9640 1 -- 172.21.15.38:0/224149660 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b3409c700 con 0x7f5b441019b0 2024-05-21T15:32:42.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:41 smithi053 ceph-mon[36496]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s wr, 16 op/s 2024-05-21T15:32:42.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:41 smithi053 ceph-mon[36496]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:42.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:41 smithi053 ceph-mon[36496]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:42.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:41 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] up:active 2024-05-21T15:32:42.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:41 smithi053 ceph-mon[36496]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-21T15:32:42.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:41 smithi053 ceph-mon[36496]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-21T15:32:42.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:41 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:32:42.457 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:41 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1987148534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:32:42.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.514+0000 7f5b4a164640 1 -- 172.21.15.38:0/224149660 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b44061f90 con 0x7f5b2c0a2cd0 2024-05-21T15:32:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.522+0000 7f5b40ff9640 1 -- 172.21.15.38:0/224149660 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258953 (secure 0 0 0) 0x7f5b44061f90 con 0x7f5b2c0a2cd0 2024-05-21T15:32:42.526 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:42.528 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-05-21T15:32:42.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.525+0000 7f5b4a164640 1 -- 172.21.15.38:0/224149660 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5b2c0a2cd0 msgr2=0x7f5b2c0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:42.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.525+0000 7f5b4a164640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5b2c0a2cd0 0x7f5b2c0a51e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b38005d00 tx=0x7f5b38005c70 comp rx=0 tx=0).stop 2024-05-21T15:32:42.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.525+0000 7f5b4a164640 1 -- 172.21.15.38:0/224149660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b441019b0 msgr2=0x7f5b44199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:42.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.525+0000 7f5b4a164640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b441019b0 0x7f5b44199190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400e9f0 tx=0x7f5b3400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.525+0000 7f5b4a164640 1 -- 172.21.15.38:0/224149660 shutdown_connections 2024-05-21T15:32:42.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.525+0000 7f5b4a164640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b441078f0 0x7f5b441a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.525+0000 7f5b4a164640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b44102bf0 0x7f5b441996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.525+0000 7f5b4a164640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5b2c0a2cd0 0x7f5b2c0a51e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.525+0000 7f5b4a164640 1 --2- 172.21.15.38:0/224149660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b441019b0 0x7f5b44199190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.525+0000 7f5b4a164640 1 -- 172.21.15.38:0/224149660 >> 172.21.15.38:0/224149660 conn(0x7f5b440fcba0 msgr2=0x7f5b440fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:42.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.525+0000 7f5b4a164640 1 -- 172.21.15.38:0/224149660 shutdown_connections 2024-05-21T15:32:42.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.525+0000 7f5b4a164640 1 -- 172.21.15.38:0/224149660 wait complete. 2024-05-21T15:32:42.537 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-05-21T15:32:42.213458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597702,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":359,"num_write_kb":2993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372908,"kb_used_data":11276,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568916,"statfs":{"total":1151940427776,"available":1151558569984,"internally_reserved":0,"allocated":11546624,"data_stored":7109933,"data_compressed":109704,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7334,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":178,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.550967"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839664+0000","last_change":"2024-05-21T15:32:33.687244+0000","last_active":"2024-05-21T15:32:35.839664+0000","last_peered":"2024-05-21T15:32:35.839664+0000","last_clean":"2024-05-21T15:32:35.839664+0000","last_became_active":"2024-05-21T15:32:33.686515+0000","last_became_peered":"2024-05-21T15:32:33.686515+0000","last_unstale":"2024-05-21T15:32:35.839664+0000","last_undegraded":"2024-05-21T15:32:35.839664+0000","last_fullsized":"2024-05-21T15:32:35.839664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:10:18.396997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839287+0000","last_change":"2024-05-21T15:32:33.678497+0000","last_active":"2024-05-21T15:32:35.839287+0000","last_peered":"2024-05-21T15:32:35.839287+0000","last_clean":"2024-05-21T15:32:35.839287+0000","last_became_active":"2024-05-21T15:32:33.678001+0000","last_became_peered":"2024-05-21T15:32:33.678001+0000","last_unstale":"2024-05-21T15:32:35.839287+0000","last_undegraded":"2024-05-21T15:32:35.839287+0000","last_fullsized":"2024-05-21T15:32:35.839287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:14:48.255675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839765+0000","last_change":"2024-05-21T15:32:33.692444+0000","last_active":"2024-05-21T15:32:35.839765+0000","last_peered":"2024-05-21T15:32:35.839765+0000","last_clean":"2024-05-21T15:32:35.839765+0000","last_became_active":"2024-05-21T15:32:33.691662+0000","last_became_peered":"2024-05-21T15:32:33.691662+0000","last_unstale":"2024-05-21T15:32:35.839765+0000","last_undegraded":"2024-05-21T15:32:35.839765+0000","last_fullsized":"2024-05-21T15:32:35.839765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:26:25.929879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661999+0000","last_change":"2024-05-21T15:32:33.692297+0000","last_active":"2024-05-21T15:32:34.661999+0000","last_peered":"2024-05-21T15:32:34.661999+0000","last_clean":"2024-05-21T15:32:34.661999+0000","last_became_active":"2024-05-21T15:32:33.691925+0000","last_became_peered":"2024-05-21T15:32:33.691925+0000","last_unstale":"2024-05-21T15:32:34.661999+0000","last_undegraded":"2024-05-21T15:32:34.661999+0000","last_fullsized":"2024-05-21T15:32:34.661999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:40:45.448485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839363+0000","last_change":"2024-05-21T15:32:33.685511+0000","last_active":"2024-05-21T15:32:35.839363+0000","last_peered":"2024-05-21T15:32:35.839363+0000","last_clean":"2024-05-21T15:32:35.839363+0000","last_became_active":"2024-05-21T15:32:33.685182+0000","last_became_peered":"2024-05-21T15:32:33.685182+0000","last_unstale":"2024-05-21T15:32:35.839363+0000","last_undegraded":"2024-05-21T15:32:35.839363+0000","last_fullsized":"2024-05-21T15:32:35.839363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:59:06.680712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.737808+0000","last_change":"2024-05-21T15:32:33.682400+0000","last_active":"2024-05-21T15:32:34.737808+0000","last_peered":"2024-05-21T15:32:34.737808+0000","last_clean":"2024-05-21T15:32:34.737808+0000","last_became_active":"2024-05-21T15:32:33.682068+0000","last_became_peered":"2024-05-21T15:32:33.682068+0000","last_unstale":"2024-05-21T15:32:34.737808+0000","last_undegraded":"2024-05-21T15:32:34.737808+0000","last_fullsized":"2024-05-21T15:32:34.737808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:31:50.968706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838180+0000","last_change":"2024-05-21T15:32:33.690099+0000","last_active":"2024-05-21T15:32:34.838180+0000","last_peered":"2024-05-21T15:32:34.838180+0000","last_clean":"2024-05-21T15:32:34.838180+0000","last_became_active":"2024-05-21T15:32:33.689762+0000","last_became_peered":"2024-05-21T15:32:33.689762+0000","last_unstale":"2024-05-21T15:32:34.838180+0000","last_undegraded":"2024-05-21T15:32:34.838180+0000","last_fullsized":"2024-05-21T15:32:34.838180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:55:20.331097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661474+0000","last_change":"2024-05-21T15:32:33.691701+0000","last_active":"2024-05-21T15:32:34.661474+0000","last_peered":"2024-05-21T15:32:34.661474+0000","last_clean":"2024-05-21T15:32:34.661474+0000","last_became_active":"2024-05-21T15:32:33.691386+0000","last_became_peered":"2024-05-21T15:32:33.691386+0000","last_unstale":"2024-05-21T15:32:34.661474+0000","last_undegraded":"2024-05-21T15:32:34.661474+0000","last_fullsized":"2024-05-21T15:32:34.661474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:54:45.691392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839632+0000","last_change":"2024-05-21T15:32:33.677484+0000","last_active":"2024-05-21T15:32:35.839632+0000","last_peered":"2024-05-21T15:32:35.839632+0000","last_clean":"2024-05-21T15:32:35.839632+0000","last_became_active":"2024-05-21T15:32:33.677160+0000","last_became_peered":"2024-05-21T15:32:33.677160+0000","last_unstale":"2024-05-21T15:32:35.839632+0000","last_undegraded":"2024-05-21T15:32:35.839632+0000","last_fullsized":"2024-05-21T15:32:35.839632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:38:00.914393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.738016+0000","last_change":"2024-05-21T15:32:33.686957+0000","last_active":"2024-05-21T15:32:34.738016+0000","last_peered":"2024-05-21T15:32:34.738016+0000","last_clean":"2024-05-21T15:32:34.738016+0000","last_became_active":"2024-05-21T15:32:33.685338+0000","last_became_peered":"2024-05-21T15:32:33.685338+0000","last_unstale":"2024-05-21T15:32:34.738016+0000","last_undegraded":"2024-05-21T15:32:34.738016+0000","last_fullsized":"2024-05-21T15:32:34.738016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:04:58.115974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838581+0000","last_change":"2024-05-21T15:32:33.693311+0000","last_active":"2024-05-21T15:32:34.838581+0000","last_peered":"2024-05-21T15:32:34.838581+0000","last_clean":"2024-05-21T15:32:34.838581+0000","last_became_active":"2024-05-21T15:32:33.692959+0000","last_became_peered":"2024-05-21T15:32:33.692959+0000","last_unstale":"2024-05-21T15:32:34.838581+0000","last_undegraded":"2024-05-21T15:32:34.838581+0000","last_fullsized":"2024-05-21T15:32:34.838581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:35:12.918524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661979+0000","last_change":"2024-05-21T15:32:33.693254+0000","last_active":"2024-05-21T15:32:34.661979+0000","last_peered":"2024-05-21T15:32:34.661979+0000","last_clean":"2024-05-21T15:32:34.661979+0000","last_became_active":"2024-05-21T15:32:33.692901+0000","last_became_peered":"2024-05-21T15:32:33.692901+0000","last_unstale":"2024-05-21T15:32:34.661979+0000","last_undegraded":"2024-05-21T15:32:34.661979+0000","last_fullsized":"2024-05-21T15:32:34.661979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:36:30.591479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840022+0000","last_change":"2024-05-21T15:32:33.675685+0000","last_active":"2024-05-21T15:32:35.840022+0000","last_peered":"2024-05-21T15:32:35.840022+0000","last_clean":"2024-05-21T15:32:35.840022+0000","last_became_active":"2024-05-21T15:32:33.675351+0000","last_became_peered":"2024-05-21T15:32:33.675351+0000","last_unstale":"2024-05-21T15:32:35.840022+0000","last_undegraded":"2024-05-21T15:32:35.840022+0000","last_fullsized":"2024-05-21T15:32:35.840022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:30:22.731506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839064+0000","last_change":"2024-05-21T15:32:33.678315+0000","last_active":"2024-05-21T15:32:35.839064+0000","last_peered":"2024-05-21T15:32:35.839064+0000","last_clean":"2024-05-21T15:32:35.839064+0000","last_became_active":"2024-05-21T15:32:33.677787+0000","last_became_peered":"2024-05-21T15:32:33.677787+0000","last_unstale":"2024-05-21T15:32:35.839064+0000","last_undegraded":"2024-05-21T15:32:35.839064+0000","last_fullsized":"2024-05-21T15:32:35.839064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:17:24.826525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839212+0000","last_change":"2024-05-21T15:32:33.689994+0000","last_active":"2024-05-21T15:32:35.839212+0000","last_peered":"2024-05-21T15:32:35.839212+0000","last_clean":"2024-05-21T15:32:35.839212+0000","last_became_active":"2024-05-21T15:32:33.689607+0000","last_became_peered":"2024-05-21T15:32:33.689607+0000","last_unstale":"2024-05-21T15:32:35.839212+0000","last_undegraded":"2024-05-21T15:32:35.839212+0000","last_fullsized":"2024-05-21T15:32:35.839212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:22:07.081722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839092+0000","last_change":"2024-05-21T15:32:33.683597+0000","last_active":"2024-05-21T15:32:35.839092+0000","last_peered":"2024-05-21T15:32:35.839092+0000","last_clean":"2024-05-21T15:32:35.839092+0000","last_became_active":"2024-05-21T15:32:33.681948+0000","last_became_peered":"2024-05-21T15:32:33.681948+0000","last_unstale":"2024-05-21T15:32:35.839092+0000","last_undegraded":"2024-05-21T15:32:35.839092+0000","last_fullsized":"2024-05-21T15:32:35.839092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:26:50.086019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840072+0000","last_change":"2024-05-21T15:32:33.683329+0000","last_active":"2024-05-21T15:32:35.840072+0000","last_peered":"2024-05-21T15:32:35.840072+0000","last_clean":"2024-05-21T15:32:35.840072+0000","last_became_active":"2024-05-21T15:32:33.682947+0000","last_became_peered":"2024-05-21T15:32:33.682947+0000","last_unstale":"2024-05-21T15:32:35.840072+0000","last_undegraded":"2024-05-21T15:32:35.840072+0000","last_fullsized":"2024-05-21T15:32:35.840072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:24:27.502257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839889+0000","last_change":"2024-05-21T15:32:33.680391+0000","last_active":"2024-05-21T15:32:35.839889+0000","last_peered":"2024-05-21T15:32:35.839889+0000","last_clean":"2024-05-21T15:32:35.839889+0000","last_became_active":"2024-05-21T15:32:33.679996+0000","last_became_peered":"2024-05-21T15:32:33.679996+0000","last_unstale":"2024-05-21T15:32:35.839889+0000","last_undegraded":"2024-05-21T15:32:35.839889+0000","last_fullsized":"2024-05-21T15:32:35.839889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:56:43.658565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840687+0000","last_change":"2024-05-21T15:32:33.688783+0000","last_active":"2024-05-21T15:32:35.840687+0000","last_peered":"2024-05-21T15:32:35.840687+0000","last_clean":"2024-05-21T15:32:35.840687+0000","last_became_active":"2024-05-21T15:32:33.688006+0000","last_became_peered":"2024-05-21T15:32:33.688006+0000","last_unstale":"2024-05-21T15:32:35.840687+0000","last_undegraded":"2024-05-21T15:32:35.840687+0000","last_fullsized":"2024-05-21T15:32:35.840687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:17:44.489759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:40.999622+0000","last_change":"2024-05-21T15:32:33.182251+0000","last_active":"2024-05-21T15:32:40.999622+0000","last_peered":"2024-05-21T15:32:40.999622+0000","last_clean":"2024-05-21T15:32:40.999622+0000","last_became_active":"2024-05-21T15:32:33.181893+0000","last_became_peered":"2024-05-21T15:32:33.181893+0000","last_unstale":"2024-05-21T15:32:40.999622+0000","last_undegraded":"2024-05-21T15:32:40.999622+0000","last_fullsized":"2024-05-21T15:32:40.999622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:23:36.863186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661843+0000","last_change":"2024-05-21T15:32:33.691962+0000","last_active":"2024-05-21T15:32:34.661843+0000","last_peered":"2024-05-21T15:32:34.661843+0000","last_clean":"2024-05-21T15:32:34.661843+0000","last_became_active":"2024-05-21T15:32:33.691619+0000","last_became_peered":"2024-05-21T15:32:33.691619+0000","last_unstale":"2024-05-21T15:32:34.661843+0000","last_undegraded":"2024-05-21T15:32:34.661843+0000","last_fullsized":"2024-05-21T15:32:34.661843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:50:14.523567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.266086+0000","last_change":"2024-05-21T15:32:33.183772+0000","last_active":"2024-05-21T15:32:35.266086+0000","last_peered":"2024-05-21T15:32:35.266086+0000","last_clean":"2024-05-21T15:32:35.266086+0000","last_became_active":"2024-05-21T15:32:33.183440+0000","last_became_peered":"2024-05-21T15:32:33.183440+0000","last_unstale":"2024-05-21T15:32:35.266086+0000","last_undegraded":"2024-05-21T15:32:35.266086+0000","last_fullsized":"2024-05-21T15:32:35.266086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:53:00.714359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840384+0000","last_change":"2024-05-21T15:32:33.682544+0000","last_active":"2024-05-21T15:32:35.840384+0000","last_peered":"2024-05-21T15:32:35.840384+0000","last_clean":"2024-05-21T15:32:35.840384+0000","last_became_active":"2024-05-21T15:32:33.682161+0000","last_became_peered":"2024-05-21T15:32:33.682161+0000","last_unstale":"2024-05-21T15:32:35.840384+0000","last_undegraded":"2024-05-21T15:32:35.840384+0000","last_fullsized":"2024-05-21T15:32:35.840384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:20:38.143508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'11","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:39.986337+0000","last_change":"2024-05-21T15:32:33.182400+0000","last_active":"2024-05-21T15:32:39.986337+0000","last_peered":"2024-05-21T15:32:39.986337+0000","last_clean":"2024-05-21T15:32:39.986337+0000","last_became_active":"2024-05-21T15:32:33.181745+0000","last_became_peered":"2024-05-21T15:32:33.181745+0000","last_unstale":"2024-05-21T15:32:39.986337+0000","last_undegraded":"2024-05-21T15:32:39.986337+0000","last_fullsized":"2024-05-21T15:32:39.986337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:39:41.669529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839137+0000","last_change":"2024-05-21T15:32:33.682700+0000","last_active":"2024-05-21T15:32:35.839137+0000","last_peered":"2024-05-21T15:32:35.839137+0000","last_clean":"2024-05-21T15:32:35.839137+0000","last_became_active":"2024-05-21T15:32:33.680876+0000","last_became_peered":"2024-05-21T15:32:33.680876+0000","last_unstale":"2024-05-21T15:32:35.839137+0000","last_undegraded":"2024-05-21T15:32:35.839137+0000","last_fullsized":"2024-05-21T15:32:35.839137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:30.694520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:39.984416+0000","last_change":"2024-05-21T15:32:33.179114+0000","last_active":"2024-05-21T15:32:39.984416+0000","last_peered":"2024-05-21T15:32:39.984416+0000","last_clean":"2024-05-21T15:32:39.984416+0000","last_became_active":"2024-05-21T15:32:33.178485+0000","last_became_peered":"2024-05-21T15:32:33.178485+0000","last_unstale":"2024-05-21T15:32:39.984416+0000","last_undegraded":"2024-05-21T15:32:39.984416+0000","last_fullsized":"2024-05-21T15:32:39.984416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:19:56.501287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.663241+0000","last_change":"2024-05-21T15:32:33.691874+0000","last_active":"2024-05-21T15:32:34.663241+0000","last_peered":"2024-05-21T15:32:34.663241+0000","last_clean":"2024-05-21T15:32:34.663241+0000","last_became_active":"2024-05-21T15:32:33.691546+0000","last_became_peered":"2024-05-21T15:32:33.691546+0000","last_unstale":"2024-05-21T15:32:34.663241+0000","last_undegraded":"2024-05-21T15:32:34.663241+0000","last_fullsized":"2024-05-21T15:32:34.663241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:40:09.281792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:40.993419+0000","last_change":"2024-05-21T15:32:33.183457+0000","last_active":"2024-05-21T15:32:40.993419+0000","last_peered":"2024-05-21T15:32:40.993419+0000","last_clean":"2024-05-21T15:32:40.993419+0000","last_became_active":"2024-05-21T15:32:33.183127+0000","last_became_peered":"2024-05-21T15:32:33.183127+0000","last_unstale":"2024-05-21T15:32:40.993419+0000","last_undegraded":"2024-05-21T15:32:40.993419+0000","last_fullsized":"2024-05-21T15:32:40.993419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:41:05.179629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839739+0000","last_change":"2024-05-21T15:32:33.677260+0000","last_active":"2024-05-21T15:32:35.839739+0000","last_peered":"2024-05-21T15:32:35.839739+0000","last_clean":"2024-05-21T15:32:35.839739+0000","last_became_active":"2024-05-21T15:32:33.676905+0000","last_became_peered":"2024-05-21T15:32:33.676905+0000","last_unstale":"2024-05-21T15:32:35.839739+0000","last_undegraded":"2024-05-21T15:32:35.839739+0000","last_fullsized":"2024-05-21T15:32:35.839739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:51:41.543066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.737559+0000","last_change":"2024-05-21T15:32:33.181819+0000","last_active":"2024-05-21T15:32:34.737559+0000","last_peered":"2024-05-21T15:32:34.737559+0000","last_clean":"2024-05-21T15:32:34.737559+0000","last_became_active":"2024-05-21T15:32:33.181471+0000","last_became_peered":"2024-05-21T15:32:33.181471+0000","last_unstale":"2024-05-21T15:32:34.737559+0000","last_undegraded":"2024-05-21T15:32:34.737559+0000","last_fullsized":"2024-05-21T15:32:34.737559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:21:54.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.663463+0000","last_change":"2024-05-21T15:32:33.691072+0000","last_active":"2024-05-21T15:32:34.663463+0000","last_peered":"2024-05-21T15:32:34.663463+0000","last_clean":"2024-05-21T15:32:34.663463+0000","last_became_active":"2024-05-21T15:32:33.690663+0000","last_became_peered":"2024-05-21T15:32:33.690663+0000","last_unstale":"2024-05-21T15:32:34.663463+0000","last_undegraded":"2024-05-21T15:32:34.663463+0000","last_fullsized":"2024-05-21T15:32:34.663463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:52:44.549537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:37.960760+0000","last_change":"2024-05-21T15:32:33.179588+0000","last_active":"2024-05-21T15:32:37.960760+0000","last_peered":"2024-05-21T15:32:37.960760+0000","last_clean":"2024-05-21T15:32:37.960760+0000","last_became_active":"2024-05-21T15:32:33.179115+0000","last_became_peered":"2024-05-21T15:32:33.179115+0000","last_unstale":"2024-05-21T15:32:37.960760+0000","last_undegraded":"2024-05-21T15:32:37.960760+0000","last_fullsized":"2024-05-21T15:32:37.960760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:18:43.904584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661542+0000","last_change":"2024-05-21T15:32:33.689456+0000","last_active":"2024-05-21T15:32:34.661542+0000","last_peered":"2024-05-21T15:32:34.661542+0000","last_clean":"2024-05-21T15:32:34.661542+0000","last_became_active":"2024-05-21T15:32:33.689142+0000","last_became_peered":"2024-05-21T15:32:33.689142+0000","last_unstale":"2024-05-21T15:32:34.661542+0000","last_undegraded":"2024-05-21T15:32:34.661542+0000","last_fullsized":"2024-05-21T15:32:34.661542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:00:13.768272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:39.986300+0000","last_change":"2024-05-21T15:32:33.184865+0000","last_active":"2024-05-21T15:32:39.986300+0000","last_peered":"2024-05-21T15:32:39.986300+0000","last_clean":"2024-05-21T15:32:39.986300+0000","last_became_active":"2024-05-21T15:32:33.184540+0000","last_became_peered":"2024-05-21T15:32:33.184540+0000","last_unstale":"2024-05-21T15:32:39.986300+0000","last_undegraded":"2024-05-21T15:32:39.986300+0000","last_fullsized":"2024-05-21T15:32:39.986300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:09:20.072906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839350+0000","last_change":"2024-05-21T15:32:33.685478+0000","last_active":"2024-05-21T15:32:35.839350+0000","last_peered":"2024-05-21T15:32:35.839350+0000","last_clean":"2024-05-21T15:32:35.839350+0000","last_became_active":"2024-05-21T15:32:33.685137+0000","last_became_peered":"2024-05-21T15:32:33.685137+0000","last_unstale":"2024-05-21T15:32:35.839350+0000","last_undegraded":"2024-05-21T15:32:35.839350+0000","last_fullsized":"2024-05-21T15:32:35.839350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:39:40.815876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:36.945491+0000","last_change":"2024-05-21T15:32:33.179215+0000","last_active":"2024-05-21T15:32:36.945491+0000","last_peered":"2024-05-21T15:32:36.945491+0000","last_clean":"2024-05-21T15:32:36.945491+0000","last_became_active":"2024-05-21T15:32:33.178846+0000","last_became_peered":"2024-05-21T15:32:33.178846+0000","last_unstale":"2024-05-21T15:32:36.945491+0000","last_undegraded":"2024-05-21T15:32:36.945491+0000","last_fullsized":"2024-05-21T15:32:36.945491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:29:47.289968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839643+0000","last_change":"2024-05-21T15:32:33.684900+0000","last_active":"2024-05-21T15:32:35.839643+0000","last_peered":"2024-05-21T15:32:35.839643+0000","last_clean":"2024-05-21T15:32:35.839643+0000","last_became_active":"2024-05-21T15:32:33.681374+0000","last_became_peered":"2024-05-21T15:32:33.681374+0000","last_unstale":"2024-05-21T15:32:35.839643+0000","last_undegraded":"2024-05-21T15:32:35.839643+0000","last_fullsized":"2024-05-21T15:32:35.839643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:51:38.079726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:36.944183+0000","last_change":"2024-05-21T15:32:33.189945+0000","last_active":"2024-05-21T15:32:36.944183+0000","last_peered":"2024-05-21T15:32:36.944183+0000","last_clean":"2024-05-21T15:32:36.944183+0000","last_became_active":"2024-05-21T15:32:33.189557+0000","last_became_peered":"2024-05-21T15:32:33.189557+0000","last_unstale":"2024-05-21T15:32:36.944183+0000","last_undegraded":"2024-05-21T15:32:36.944183+0000","last_fullsized":"2024-05-21T15:32:36.944183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:08.427043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840394+0000","last_change":"2024-05-21T15:32:33.692896+0000","last_active":"2024-05-21T15:32:35.840394+0000","last_peered":"2024-05-21T15:32:35.840394+0000","last_clean":"2024-05-21T15:32:35.840394+0000","last_became_active":"2024-05-21T15:32:33.692527+0000","last_became_peered":"2024-05-21T15:32:33.692527+0000","last_unstale":"2024-05-21T15:32:35.840394+0000","last_undegraded":"2024-05-21T15:32:35.840394+0000","last_fullsized":"2024-05-21T15:32:35.840394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:45:45.418063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:38.975228+0000","last_change":"2024-05-21T15:32:33.172989+0000","last_active":"2024-05-21T15:32:38.975228+0000","last_peered":"2024-05-21T15:32:38.975228+0000","last_clean":"2024-05-21T15:32:38.975228+0000","last_became_active":"2024-05-21T15:32:33.172215+0000","last_became_peered":"2024-05-21T15:32:33.172215+0000","last_unstale":"2024-05-21T15:32:38.975228+0000","last_undegraded":"2024-05-21T15:32:38.975228+0000","last_fullsized":"2024-05-21T15:32:38.975228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:09:39.287636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839533+0000","last_change":"2024-05-21T15:32:33.678441+0000","last_active":"2024-05-21T15:32:35.839533+0000","last_peered":"2024-05-21T15:32:35.839533+0000","last_clean":"2024-05-21T15:32:35.839533+0000","last_became_active":"2024-05-21T15:32:33.677833+0000","last_became_peered":"2024-05-21T15:32:33.677833+0000","last_unstale":"2024-05-21T15:32:35.839533+0000","last_undegraded":"2024-05-21T15:32:35.839533+0000","last_fullsized":"2024-05-21T15:32:35.839533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:36.124064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:36.942609+0000","last_change":"2024-05-21T15:32:33.178153+0000","last_active":"2024-05-21T15:32:36.942609+0000","last_peered":"2024-05-21T15:32:36.942609+0000","last_clean":"2024-05-21T15:32:36.942609+0000","last_became_active":"2024-05-21T15:32:33.177792+0000","last_became_peered":"2024-05-21T15:32:33.177792+0000","last_unstale":"2024-05-21T15:32:36.942609+0000","last_undegraded":"2024-05-21T15:32:36.942609+0000","last_fullsized":"2024-05-21T15:32:36.942609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:43:41.865204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839941+0000","last_change":"2024-05-21T15:32:33.681328+0000","last_active":"2024-05-21T15:32:35.839941+0000","last_peered":"2024-05-21T15:32:35.839941+0000","last_clean":"2024-05-21T15:32:35.839941+0000","last_became_active":"2024-05-21T15:32:33.680975+0000","last_became_peered":"2024-05-21T15:32:33.680975+0000","last_unstale":"2024-05-21T15:32:35.839941+0000","last_undegraded":"2024-05-21T15:32:35.839941+0000","last_fullsized":"2024-05-21T15:32:35.839941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:50:27.199348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:37.963247+0000","last_change":"2024-05-21T15:32:33.182492+0000","last_active":"2024-05-21T15:32:37.963247+0000","last_peered":"2024-05-21T15:32:37.963247+0000","last_clean":"2024-05-21T15:32:37.963247+0000","last_became_active":"2024-05-21T15:32:33.182152+0000","last_became_peered":"2024-05-21T15:32:33.182152+0000","last_unstale":"2024-05-21T15:32:37.963247+0000","last_undegraded":"2024-05-21T15:32:37.963247+0000","last_fullsized":"2024-05-21T15:32:37.963247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:04:07.432742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839678+0000","last_change":"2024-05-21T15:32:33.686623+0000","last_active":"2024-05-21T15:32:35.839678+0000","last_peered":"2024-05-21T15:32:35.839678+0000","last_clean":"2024-05-21T15:32:35.839678+0000","last_became_active":"2024-05-21T15:32:33.685816+0000","last_became_peered":"2024-05-21T15:32:33.685816+0000","last_unstale":"2024-05-21T15:32:35.839678+0000","last_undegraded":"2024-05-21T15:32:35.839678+0000","last_fullsized":"2024-05-21T15:32:35.839678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:15:20.354113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:39.986495+0000","last_change":"2024-05-21T15:32:33.182218+0000","last_active":"2024-05-21T15:32:39.986495+0000","last_peered":"2024-05-21T15:32:39.986495+0000","last_clean":"2024-05-21T15:32:39.986495+0000","last_became_active":"2024-05-21T15:32:33.181883+0000","last_became_peered":"2024-05-21T15:32:33.181883+0000","last_unstale":"2024-05-21T15:32:39.986495+0000","last_undegraded":"2024-05-21T15:32:39.986495+0000","last_fullsized":"2024-05-21T15:32:39.986495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:24:07.963898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838515+0000","last_change":"2024-05-21T15:32:33.689936+0000","last_active":"2024-05-21T15:32:34.838515+0000","last_peered":"2024-05-21T15:32:34.838515+0000","last_clean":"2024-05-21T15:32:34.838515+0000","last_became_active":"2024-05-21T15:32:33.689452+0000","last_became_peered":"2024-05-21T15:32:33.689452+0000","last_unstale":"2024-05-21T15:32:34.838515+0000","last_undegraded":"2024-05-21T15:32:34.838515+0000","last_fullsized":"2024-05-21T15:32:34.838515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:00.420730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:40.998669+0000","last_change":"2024-05-21T15:32:33.181354+0000","last_active":"2024-05-21T15:32:40.998669+0000","last_peered":"2024-05-21T15:32:40.998669+0000","last_clean":"2024-05-21T15:32:40.998669+0000","last_became_active":"2024-05-21T15:32:33.180983+0000","last_became_peered":"2024-05-21T15:32:33.180983+0000","last_unstale":"2024-05-21T15:32:40.998669+0000","last_undegraded":"2024-05-21T15:32:40.998669+0000","last_fullsized":"2024-05-21T15:32:40.998669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:18:46.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838079+0000","last_change":"2024-05-21T15:32:33.690992+0000","last_active":"2024-05-21T15:32:34.838079+0000","last_peered":"2024-05-21T15:32:34.838079+0000","last_clean":"2024-05-21T15:32:34.838079+0000","last_became_active":"2024-05-21T15:32:33.690663+0000","last_became_peered":"2024-05-21T15:32:33.690663+0000","last_unstale":"2024-05-21T15:32:34.838079+0000","last_undegraded":"2024-05-21T15:32:34.838079+0000","last_fullsized":"2024-05-21T15:32:34.838079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:17:48.818803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:38.975485+0000","last_change":"2024-05-21T15:32:33.182804+0000","last_active":"2024-05-21T15:32:38.975485+0000","last_peered":"2024-05-21T15:32:38.975485+0000","last_clean":"2024-05-21T15:32:38.975485+0000","last_became_active":"2024-05-21T15:32:33.182449+0000","last_became_peered":"2024-05-21T15:32:33.182449+0000","last_unstale":"2024-05-21T15:32:38.975485+0000","last_undegraded":"2024-05-21T15:32:38.975485+0000","last_fullsized":"2024-05-21T15:32:38.975485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:33:56.320675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.662821+0000","last_change":"2024-05-21T15:32:33.691112+0000","last_active":"2024-05-21T15:32:34.662821+0000","last_peered":"2024-05-21T15:32:34.662821+0000","last_clean":"2024-05-21T15:32:34.662821+0000","last_became_active":"2024-05-21T15:32:33.690726+0000","last_became_peered":"2024-05-21T15:32:33.690726+0000","last_unstale":"2024-05-21T15:32:34.662821+0000","last_undegraded":"2024-05-21T15:32:34.662821+0000","last_fullsized":"2024-05-21T15:32:34.662821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:59:29.002636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840188+0000","last_change":"2024-05-21T15:32:33.179565+0000","last_active":"2024-05-21T15:32:35.840188+0000","last_peered":"2024-05-21T15:32:35.840188+0000","last_clean":"2024-05-21T15:32:35.840188+0000","last_became_active":"2024-05-21T15:32:33.179168+0000","last_became_peered":"2024-05-21T15:32:33.179168+0000","last_unstale":"2024-05-21T15:32:35.840188+0000","last_undegraded":"2024-05-21T15:32:35.840188+0000","last_fullsized":"2024-05-21T15:32:35.840188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:23:39.210931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839646+0000","last_change":"2024-05-21T15:32:33.691348+0000","last_active":"2024-05-21T15:32:35.839646+0000","last_peered":"2024-05-21T15:32:35.839646+0000","last_clean":"2024-05-21T15:32:35.839646+0000","last_became_active":"2024-05-21T15:32:33.691005+0000","last_became_peered":"2024-05-21T15:32:33.691005+0000","last_unstale":"2024-05-21T15:32:35.839646+0000","last_undegraded":"2024-05-21T15:32:35.839646+0000","last_fullsized":"2024-05-21T15:32:35.839646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:33:34.533026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839536+0000","last_change":"2024-05-21T15:32:33.184850+0000","last_active":"2024-05-21T15:32:35.839536+0000","last_peered":"2024-05-21T15:32:35.839536+0000","last_clean":"2024-05-21T15:32:35.839536+0000","last_became_active":"2024-05-21T15:32:33.184480+0000","last_became_peered":"2024-05-21T15:32:33.184480+0000","last_unstale":"2024-05-21T15:32:35.839536+0000","last_undegraded":"2024-05-21T15:32:35.839536+0000","last_fullsized":"2024-05-21T15:32:35.839536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:31.327820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.737977+0000","last_change":"2024-05-21T15:32:33.689520+0000","last_active":"2024-05-21T15:32:34.737977+0000","last_peered":"2024-05-21T15:32:34.737977+0000","last_clean":"2024-05-21T15:32:34.737977+0000","last_became_active":"2024-05-21T15:32:33.688219+0000","last_became_peered":"2024-05-21T15:32:33.688219+0000","last_unstale":"2024-05-21T15:32:34.737977+0000","last_undegraded":"2024-05-21T15:32:34.737977+0000","last_fullsized":"2024-05-21T15:32:34.737977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:36.546390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838313+0000","last_change":"2024-05-21T15:32:33.183183+0000","last_active":"2024-05-21T15:32:34.838313+0000","last_peered":"2024-05-21T15:32:34.838313+0000","last_clean":"2024-05-21T15:32:34.838313+0000","last_became_active":"2024-05-21T15:32:33.182753+0000","last_became_peered":"2024-05-21T15:32:33.182753+0000","last_unstale":"2024-05-21T15:32:34.838313+0000","last_undegraded":"2024-05-21T15:32:34.838313+0000","last_fullsized":"2024-05-21T15:32:34.838313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:44:22.509265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.737967+0000","last_change":"2024-05-21T15:32:33.684019+0000","last_active":"2024-05-21T15:32:34.737967+0000","last_peered":"2024-05-21T15:32:34.737967+0000","last_clean":"2024-05-21T15:32:34.737967+0000","last_became_active":"2024-05-21T15:32:33.683276+0000","last_became_peered":"2024-05-21T15:32:33.683276+0000","last_unstale":"2024-05-21T15:32:34.737967+0000","last_undegraded":"2024-05-21T15:32:34.737967+0000","last_fullsized":"2024-05-21T15:32:34.737967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:07:14.947240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.662950+0000","last_change":"2024-05-21T15:32:33.690956+0000","last_active":"2024-05-21T15:32:34.662950+0000","last_peered":"2024-05-21T15:32:34.662950+0000","last_clean":"2024-05-21T15:32:34.662950+0000","last_became_active":"2024-05-21T15:32:33.690630+0000","last_became_peered":"2024-05-21T15:32:33.690630+0000","last_unstale":"2024-05-21T15:32:34.662950+0000","last_undegraded":"2024-05-21T15:32:34.662950+0000","last_fullsized":"2024-05-21T15:32:34.662950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:24:07.050928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:37.963452+0000","last_change":"2024-05-21T15:32:33.175127+0000","last_active":"2024-05-21T15:32:37.963452+0000","last_peered":"2024-05-21T15:32:37.963452+0000","last_clean":"2024-05-21T15:32:37.963452+0000","last_became_active":"2024-05-21T15:32:33.174724+0000","last_became_peered":"2024-05-21T15:32:33.174724+0000","last_unstale":"2024-05-21T15:32:37.963452+0000","last_undegraded":"2024-05-21T15:32:37.963452+0000","last_fullsized":"2024-05-21T15:32:37.963452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:39:17.499743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661983+0000","last_change":"2024-05-21T15:32:33.693428+0000","last_active":"2024-05-21T15:32:34.661983+0000","last_peered":"2024-05-21T15:32:34.661983+0000","last_clean":"2024-05-21T15:32:34.661983+0000","last_became_active":"2024-05-21T15:32:33.693077+0000","last_became_peered":"2024-05-21T15:32:33.693077+0000","last_unstale":"2024-05-21T15:32:34.661983+0000","last_undegraded":"2024-05-21T15:32:34.661983+0000","last_fullsized":"2024-05-21T15:32:34.661983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:08:11.908874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:40.996136+0000","last_change":"2024-05-21T15:32:33.176267+0000","last_active":"2024-05-21T15:32:40.996136+0000","last_peered":"2024-05-21T15:32:40.996136+0000","last_clean":"2024-05-21T15:32:40.996136+0000","last_became_active":"2024-05-21T15:32:33.175856+0000","last_became_peered":"2024-05-21T15:32:33.175856+0000","last_unstale":"2024-05-21T15:32:40.996136+0000","last_undegraded":"2024-05-21T15:32:40.996136+0000","last_fullsized":"2024-05-21T15:32:40.996136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:58.113791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":339,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838073+0000","last_change":"2024-05-21T15:31:20.928694+0000","last_active":"2024-05-21T15:32:34.838073+0000","last_peered":"2024-05-21T15:32:34.838073+0000","last_clean":"2024-05-21T15:32:34.838073+0000","last_became_active":"2024-05-21T15:31:20.928115+0000","last_became_peered":"2024-05-21T15:31:20.928115+0000","last_unstale":"2024-05-21T15:32:34.838073+0000","last_undegraded":"2024-05-21T15:32:34.838073+0000","last_fullsized":"2024-05-21T15:32:34.838073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:27:50.788960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:27:50.788960+0000","last_clean_scrub_stamp":"2024-05-21T15:27:50.788960+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:55:10.393374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.838976+0000","last_change":"2024-05-21T15:32:33.689305+0000","last_active":"2024-05-21T15:32:35.838976+0000","last_peered":"2024-05-21T15:32:35.838976+0000","last_clean":"2024-05-21T15:32:35.838976+0000","last_became_active":"2024-05-21T15:32:33.688908+0000","last_became_peered":"2024-05-21T15:32:33.688908+0000","last_unstale":"2024-05-21T15:32:35.838976+0000","last_undegraded":"2024-05-21T15:32:35.838976+0000","last_fullsized":"2024-05-21T15:32:35.838976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:49:18.953116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:37.959859+0000","last_change":"2024-05-21T15:32:33.187781+0000","last_active":"2024-05-21T15:32:37.959859+0000","last_peered":"2024-05-21T15:32:37.959859+0000","last_clean":"2024-05-21T15:32:37.959859+0000","last_became_active":"2024-05-21T15:32:33.187504+0000","last_became_peered":"2024-05-21T15:32:33.187504+0000","last_unstale":"2024-05-21T15:32:37.959859+0000","last_undegraded":"2024-05-21T15:32:37.959859+0000","last_fullsized":"2024-05-21T15:32:37.959859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:30.400209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:38.976167+0000","last_change":"2024-05-21T15:32:33.188706+0000","last_active":"2024-05-21T15:32:38.976167+0000","last_peered":"2024-05-21T15:32:38.976167+0000","last_clean":"2024-05-21T15:32:38.976167+0000","last_became_active":"2024-05-21T15:32:33.187462+0000","last_became_peered":"2024-05-21T15:32:33.187462+0000","last_unstale":"2024-05-21T15:32:38.976167+0000","last_undegraded":"2024-05-21T15:32:38.976167+0000","last_fullsized":"2024-05-21T15:32:38.976167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:47:45.387721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840292+0000","last_change":"2024-05-21T15:32:33.672531+0000","last_active":"2024-05-21T15:32:35.840292+0000","last_peered":"2024-05-21T15:32:35.840292+0000","last_clean":"2024-05-21T15:32:35.840292+0000","last_became_active":"2024-05-21T15:32:33.671972+0000","last_became_peered":"2024-05-21T15:32:33.671972+0000","last_unstale":"2024-05-21T15:32:35.840292+0000","last_undegraded":"2024-05-21T15:32:35.840292+0000","last_fullsized":"2024-05-21T15:32:35.840292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:10:51.062530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:40.216114+0000","last_change":"2024-05-21T15:32:33.181274+0000","last_active":"2024-05-21T15:32:40.216114+0000","last_peered":"2024-05-21T15:32:40.216114+0000","last_clean":"2024-05-21T15:32:40.216114+0000","last_became_active":"2024-05-21T15:32:33.180885+0000","last_became_peered":"2024-05-21T15:32:33.180885+0000","last_unstale":"2024-05-21T15:32:40.216114+0000","last_undegraded":"2024-05-21T15:32:40.216114+0000","last_fullsized":"2024-05-21T15:32:40.216114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:10:13.547380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661999+0000","last_change":"2024-05-21T15:32:33.683749+0000","last_active":"2024-05-21T15:32:34.661999+0000","last_peered":"2024-05-21T15:32:34.661999+0000","last_clean":"2024-05-21T15:32:34.661999+0000","last_became_active":"2024-05-21T15:32:33.683393+0000","last_became_peered":"2024-05-21T15:32:33.683393+0000","last_unstale":"2024-05-21T15:32:34.661999+0000","last_undegraded":"2024-05-21T15:32:34.661999+0000","last_fullsized":"2024-05-21T15:32:34.661999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:07:53.009651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:39.984697+0000","last_change":"2024-05-21T15:32:33.184740+0000","last_active":"2024-05-21T15:32:39.984697+0000","last_peered":"2024-05-21T15:32:39.984697+0000","last_clean":"2024-05-21T15:32:39.984697+0000","last_became_active":"2024-05-21T15:32:33.184378+0000","last_became_peered":"2024-05-21T15:32:33.184378+0000","last_unstale":"2024-05-21T15:32:39.984697+0000","last_undegraded":"2024-05-21T15:32:39.984697+0000","last_fullsized":"2024-05-21T15:32:39.984697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:21:27.610513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840273+0000","last_change":"2024-05-21T15:32:33.692816+0000","last_active":"2024-05-21T15:32:35.840273+0000","last_peered":"2024-05-21T15:32:35.840273+0000","last_clean":"2024-05-21T15:32:35.840273+0000","last_became_active":"2024-05-21T15:32:33.692477+0000","last_became_peered":"2024-05-21T15:32:33.692477+0000","last_unstale":"2024-05-21T15:32:35.840273+0000","last_undegraded":"2024-05-21T15:32:35.840273+0000","last_fullsized":"2024-05-21T15:32:35.840273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:28:25.532884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:36.944469+0000","last_change":"2024-05-21T15:32:33.183344+0000","last_active":"2024-05-21T15:32:36.944469+0000","last_peered":"2024-05-21T15:32:36.944469+0000","last_clean":"2024-05-21T15:32:36.944469+0000","last_became_active":"2024-05-21T15:32:33.182977+0000","last_became_peered":"2024-05-21T15:32:33.182977+0000","last_unstale":"2024-05-21T15:32:36.944469+0000","last_undegraded":"2024-05-21T15:32:36.944469+0000","last_fullsized":"2024-05-21T15:32:36.944469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:27:24.354129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839339+0000","last_change":"2024-05-21T15:32:33.677549+0000","last_active":"2024-05-21T15:32:35.839339+0000","last_peered":"2024-05-21T15:32:35.839339+0000","last_clean":"2024-05-21T15:32:35.839339+0000","last_became_active":"2024-05-21T15:32:33.677213+0000","last_became_peered":"2024-05-21T15:32:33.677213+0000","last_unstale":"2024-05-21T15:32:35.839339+0000","last_undegraded":"2024-05-21T15:32:35.839339+0000","last_fullsized":"2024-05-21T15:32:35.839339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:57:06.560434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:39.981388+0000","last_change":"2024-05-21T15:32:33.190023+0000","last_active":"2024-05-21T15:32:39.981388+0000","last_peered":"2024-05-21T15:32:39.981388+0000","last_clean":"2024-05-21T15:32:39.981388+0000","last_became_active":"2024-05-21T15:32:33.189616+0000","last_became_peered":"2024-05-21T15:32:33.189616+0000","last_unstale":"2024-05-21T15:32:39.981388+0000","last_undegraded":"2024-05-21T15:32:39.981388+0000","last_fullsized":"2024-05-21T15:32:39.981388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:03.841414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840338+0000","last_change":"2024-05-21T15:32:33.685911+0000","last_active":"2024-05-21T15:32:35.840338+0000","last_peered":"2024-05-21T15:32:35.840338+0000","last_clean":"2024-05-21T15:32:35.840338+0000","last_became_active":"2024-05-21T15:32:33.684785+0000","last_became_peered":"2024-05-21T15:32:33.684785+0000","last_unstale":"2024-05-21T15:32:35.840338+0000","last_undegraded":"2024-05-21T15:32:35.840338+0000","last_fullsized":"2024-05-21T15:32:35.840338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:39:18.370289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:38.937086+0000","last_change":"2024-05-21T15:32:33.178354+0000","last_active":"2024-05-21T15:32:38.937086+0000","last_peered":"2024-05-21T15:32:38.937086+0000","last_clean":"2024-05-21T15:32:38.937086+0000","last_became_active":"2024-05-21T15:32:33.177989+0000","last_became_peered":"2024-05-21T15:32:33.177989+0000","last_unstale":"2024-05-21T15:32:38.937086+0000","last_undegraded":"2024-05-21T15:32:38.937086+0000","last_fullsized":"2024-05-21T15:32:38.937086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:58.650831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661712+0000","last_change":"2024-05-21T15:32:33.686467+0000","last_active":"2024-05-21T15:32:34.661712+0000","last_peered":"2024-05-21T15:32:34.661712+0000","last_clean":"2024-05-21T15:32:34.661712+0000","last_became_active":"2024-05-21T15:32:33.686135+0000","last_became_peered":"2024-05-21T15:32:33.686135+0000","last_unstale":"2024-05-21T15:32:34.661712+0000","last_undegraded":"2024-05-21T15:32:34.661712+0000","last_fullsized":"2024-05-21T15:32:34.661712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:53:54.045591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:36.945036+0000","last_change":"2024-05-21T15:32:33.182936+0000","last_active":"2024-05-21T15:32:36.945036+0000","last_peered":"2024-05-21T15:32:36.945036+0000","last_clean":"2024-05-21T15:32:36.945036+0000","last_became_active":"2024-05-21T15:32:33.182587+0000","last_became_peered":"2024-05-21T15:32:33.182587+0000","last_unstale":"2024-05-21T15:32:36.945036+0000","last_undegraded":"2024-05-21T15:32:36.945036+0000","last_fullsized":"2024-05-21T15:32:36.945036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:56.434249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838677+0000","last_change":"2024-05-21T15:32:33.692325+0000","last_active":"2024-05-21T15:32:34.838677+0000","last_peered":"2024-05-21T15:32:34.838677+0000","last_clean":"2024-05-21T15:32:34.838677+0000","last_became_active":"2024-05-21T15:32:33.691971+0000","last_became_peered":"2024-05-21T15:32:33.691971+0000","last_unstale":"2024-05-21T15:32:34.838677+0000","last_undegraded":"2024-05-21T15:32:34.838677+0000","last_fullsized":"2024-05-21T15:32:34.838677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:53:28.190063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:38.975895+0000","last_change":"2024-05-21T15:32:33.186925+0000","last_active":"2024-05-21T15:32:38.975895+0000","last_peered":"2024-05-21T15:32:38.975895+0000","last_clean":"2024-05-21T15:32:38.975895+0000","last_became_active":"2024-05-21T15:32:33.186605+0000","last_became_peered":"2024-05-21T15:32:33.186605+0000","last_unstale":"2024-05-21T15:32:38.975895+0000","last_undegraded":"2024-05-21T15:32:38.975895+0000","last_fullsized":"2024-05-21T15:32:38.975895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:32.752746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838710+0000","last_change":"2024-05-21T15:32:33.686132+0000","last_active":"2024-05-21T15:32:34.838710+0000","last_peered":"2024-05-21T15:32:34.838710+0000","last_clean":"2024-05-21T15:32:34.838710+0000","last_became_active":"2024-05-21T15:32:33.685745+0000","last_became_peered":"2024-05-21T15:32:33.685745+0000","last_unstale":"2024-05-21T15:32:34.838710+0000","last_undegraded":"2024-05-21T15:32:34.838710+0000","last_fullsized":"2024-05-21T15:32:34.838710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:51:49.522297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:39.985576+0000","last_change":"2024-05-21T15:32:33.176552+0000","last_active":"2024-05-21T15:32:39.985576+0000","last_peered":"2024-05-21T15:32:39.985576+0000","last_clean":"2024-05-21T15:32:39.985576+0000","last_became_active":"2024-05-21T15:32:33.176237+0000","last_became_peered":"2024-05-21T15:32:33.176237+0000","last_unstale":"2024-05-21T15:32:39.985576+0000","last_undegraded":"2024-05-21T15:32:39.985576+0000","last_fullsized":"2024-05-21T15:32:39.985576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:09.098344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838440+0000","last_change":"2024-05-21T15:32:33.692310+0000","last_active":"2024-05-21T15:32:34.838440+0000","last_peered":"2024-05-21T15:32:34.838440+0000","last_clean":"2024-05-21T15:32:34.838440+0000","last_became_active":"2024-05-21T15:32:33.691959+0000","last_became_peered":"2024-05-21T15:32:33.691959+0000","last_unstale":"2024-05-21T15:32:34.838440+0000","last_undegraded":"2024-05-21T15:32:34.838440+0000","last_fullsized":"2024-05-21T15:32:34.838440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:57:38.243926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839998+0000","last_change":"2024-05-21T15:32:33.183219+0000","last_active":"2024-05-21T15:32:35.839998+0000","last_peered":"2024-05-21T15:32:35.839998+0000","last_clean":"2024-05-21T15:32:35.839998+0000","last_became_active":"2024-05-21T15:32:33.182847+0000","last_became_peered":"2024-05-21T15:32:33.182847+0000","last_unstale":"2024-05-21T15:32:35.839998+0000","last_undegraded":"2024-05-21T15:32:35.839998+0000","last_fullsized":"2024-05-21T15:32:35.839998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:16:05.341406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838582+0000","last_change":"2024-05-21T15:32:33.682801+0000","last_active":"2024-05-21T15:32:34.838582+0000","last_peered":"2024-05-21T15:32:34.838582+0000","last_clean":"2024-05-21T15:32:34.838582+0000","last_became_active":"2024-05-21T15:32:33.682368+0000","last_became_peered":"2024-05-21T15:32:33.682368+0000","last_unstale":"2024-05-21T15:32:34.838582+0000","last_undegraded":"2024-05-21T15:32:34.838582+0000","last_fullsized":"2024-05-21T15:32:34.838582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:56:50.520672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840432+0000","last_change":"2024-05-21T15:32:33.684809+0000","last_active":"2024-05-21T15:32:35.840432+0000","last_peered":"2024-05-21T15:32:35.840432+0000","last_clean":"2024-05-21T15:32:35.840432+0000","last_became_active":"2024-05-21T15:32:33.684067+0000","last_became_peered":"2024-05-21T15:32:33.684067+0000","last_unstale":"2024-05-21T15:32:35.840432+0000","last_undegraded":"2024-05-21T15:32:35.840432+0000","last_fullsized":"2024-05-21T15:32:35.840432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:31:58.997470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839669+0000","last_change":"2024-05-21T15:32:33.676091+0000","last_active":"2024-05-21T15:32:35.839669+0000","last_peered":"2024-05-21T15:32:35.839669+0000","last_clean":"2024-05-21T15:32:35.839669+0000","last_became_active":"2024-05-21T15:32:33.675538+0000","last_became_peered":"2024-05-21T15:32:33.675538+0000","last_unstale":"2024-05-21T15:32:35.839669+0000","last_undegraded":"2024-05-21T15:32:35.839669+0000","last_fullsized":"2024-05-21T15:32:35.839669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:50.311152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839739+0000","last_change":"2024-05-21T15:32:33.688590+0000","last_active":"2024-05-21T15:32:35.839739+0000","last_peered":"2024-05-21T15:32:35.839739+0000","last_clean":"2024-05-21T15:32:35.839739+0000","last_became_active":"2024-05-21T15:32:33.687993+0000","last_became_peered":"2024-05-21T15:32:33.687993+0000","last_unstale":"2024-05-21T15:32:35.839739+0000","last_undegraded":"2024-05-21T15:32:35.839739+0000","last_fullsized":"2024-05-21T15:32:35.839739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:21.939483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840015+0000","last_change":"2024-05-21T15:32:33.689122+0000","last_active":"2024-05-21T15:32:35.840015+0000","last_peered":"2024-05-21T15:32:35.840015+0000","last_clean":"2024-05-21T15:32:35.840015+0000","last_became_active":"2024-05-21T15:32:33.688775+0000","last_became_peered":"2024-05-21T15:32:33.688775+0000","last_unstale":"2024-05-21T15:32:35.840015+0000","last_undegraded":"2024-05-21T15:32:35.840015+0000","last_fullsized":"2024-05-21T15:32:35.840015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:21.480005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840078+0000","last_change":"2024-05-21T15:32:33.689365+0000","last_active":"2024-05-21T15:32:35.840078+0000","last_peered":"2024-05-21T15:32:35.840078+0000","last_clean":"2024-05-21T15:32:35.840078+0000","last_became_active":"2024-05-21T15:32:33.688739+0000","last_became_peered":"2024-05-21T15:32:33.688739+0000","last_unstale":"2024-05-21T15:32:35.840078+0000","last_undegraded":"2024-05-21T15:32:35.840078+0000","last_fullsized":"2024-05-21T15:32:35.840078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:19:52.614155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840100+0000","last_change":"2024-05-21T15:32:33.681985+0000","last_active":"2024-05-21T15:32:35.840100+0000","last_peered":"2024-05-21T15:32:35.840100+0000","last_clean":"2024-05-21T15:32:35.840100+0000","last_became_active":"2024-05-21T15:32:33.681209+0000","last_became_peered":"2024-05-21T15:32:33.681209+0000","last_unstale":"2024-05-21T15:32:35.840100+0000","last_undegraded":"2024-05-21T15:32:35.840100+0000","last_fullsized":"2024-05-21T15:32:35.840100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:03:16.088136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840969+0000","last_change":"2024-05-21T15:32:33.673236+0000","last_active":"2024-05-21T15:32:35.840969+0000","last_peered":"2024-05-21T15:32:35.840969+0000","last_clean":"2024-05-21T15:32:35.840969+0000","last_became_active":"2024-05-21T15:32:33.672548+0000","last_became_peered":"2024-05-21T15:32:33.672548+0000","last_unstale":"2024-05-21T15:32:35.840969+0000","last_undegraded":"2024-05-21T15:32:35.840969+0000","last_fullsized":"2024-05-21T15:32:35.840969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:52:10.010219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661906+0000","last_change":"2024-05-21T15:32:33.695171+0000","last_active":"2024-05-21T15:32:34.661906+0000","last_peered":"2024-05-21T15:32:34.661906+0000","last_clean":"2024-05-21T15:32:34.661906+0000","last_became_active":"2024-05-21T15:32:33.694778+0000","last_became_peered":"2024-05-21T15:32:33.694778+0000","last_unstale":"2024-05-21T15:32:34.661906+0000","last_undegraded":"2024-05-21T15:32:34.661906+0000","last_fullsized":"2024-05-21T15:32:34.661906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:29.889437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.662826+0000","last_change":"2024-05-21T15:32:33.686992+0000","last_active":"2024-05-21T15:32:34.662826+0000","last_peered":"2024-05-21T15:32:34.662826+0000","last_clean":"2024-05-21T15:32:34.662826+0000","last_became_active":"2024-05-21T15:32:33.686673+0000","last_became_peered":"2024-05-21T15:32:33.686673+0000","last_unstale":"2024-05-21T15:32:34.662826+0000","last_undegraded":"2024-05-21T15:32:34.662826+0000","last_fullsized":"2024-05-21T15:32:34.662826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:18:20.668365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661502+0000","last_change":"2024-05-21T15:32:33.689433+0000","last_active":"2024-05-21T15:32:34.661502+0000","last_peered":"2024-05-21T15:32:34.661502+0000","last_clean":"2024-05-21T15:32:34.661502+0000","last_became_active":"2024-05-21T15:32:33.689106+0000","last_became_peered":"2024-05-21T15:32:33.689106+0000","last_unstale":"2024-05-21T15:32:34.661502+0000","last_undegraded":"2024-05-21T15:32:34.661502+0000","last_fullsized":"2024-05-21T15:32:34.661502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:45:17.841159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839573+0000","last_change":"2024-05-21T15:32:33.687763+0000","last_active":"2024-05-21T15:32:35.839573+0000","last_peered":"2024-05-21T15:32:35.839573+0000","last_clean":"2024-05-21T15:32:35.839573+0000","last_became_active":"2024-05-21T15:32:33.685103+0000","last_became_peered":"2024-05-21T15:32:33.685103+0000","last_unstale":"2024-05-21T15:32:35.839573+0000","last_undegraded":"2024-05-21T15:32:35.839573+0000","last_fullsized":"2024-05-21T15:32:35.839573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:56:39.454536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840217+0000","last_change":"2024-05-21T15:32:33.679225+0000","last_active":"2024-05-21T15:32:35.840217+0000","last_peered":"2024-05-21T15:32:35.840217+0000","last_clean":"2024-05-21T15:32:35.840217+0000","last_became_active":"2024-05-21T15:32:33.678890+0000","last_became_peered":"2024-05-21T15:32:33.678890+0000","last_unstale":"2024-05-21T15:32:35.840217+0000","last_undegraded":"2024-05-21T15:32:35.840217+0000","last_fullsized":"2024-05-21T15:32:35.840217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:12:23.721449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839788+0000","last_change":"2024-05-21T15:32:33.688659+0000","last_active":"2024-05-21T15:32:35.839788+0000","last_peered":"2024-05-21T15:32:35.839788+0000","last_clean":"2024-05-21T15:32:35.839788+0000","last_became_active":"2024-05-21T15:32:33.688293+0000","last_became_peered":"2024-05-21T15:32:33.688293+0000","last_unstale":"2024-05-21T15:32:35.839788+0000","last_undegraded":"2024-05-21T15:32:35.839788+0000","last_fullsized":"2024-05-21T15:32:35.839788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:08:49.543237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":287606,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7334,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":178,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":41405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612625,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":1034508,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":442694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":442310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":1033144,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462951,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":464219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":444141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":445156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":442002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888115,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":442272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":1034760,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":22647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T15:32:42.541 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T15:32:42.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.691+0000 7fefc501a640 1 -- 172.21.15.38:0/3900294371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefc0103430 msgr2=0x7fefc0109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:42.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.691+0000 7fefc501a640 1 --2- 172.21.15.38:0/3900294371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefc0103430 0x7fefc0109b70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009900 tx=0x7fefb802a3f0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.691+0000 7fefc501a640 1 -- 172.21.15.38:0/3900294371 shutdown_connections 2024-05-21T15:32:42.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.691+0000 7fefc501a640 1 --2- 172.21.15.38:0/3900294371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefc0103430 0x7fefc0109b70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.691+0000 7fefc501a640 1 --2- 172.21.15.38:0/3900294371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefc01028f0 0x7fefc0102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.691+0000 7fefc501a640 1 --2- 172.21.15.38:0/3900294371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefc0101f10 0x7fefc0102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.691+0000 7fefc501a640 1 -- 172.21.15.38:0/3900294371 >> 172.21.15.38:0/3900294371 conn(0x7fefc00fc330 msgr2=0x7fefc00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:42.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.691+0000 7fefc501a640 1 -- 172.21.15.38:0/3900294371 shutdown_connections 2024-05-21T15:32:42.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.691+0000 7fefc501a640 1 -- 172.21.15.38:0/3900294371 wait complete. 2024-05-21T15:32:42.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefc501a640 1 Processor -- start 2024-05-21T15:32:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefc501a640 1 -- start start 2024-05-21T15:32:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefc501a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefc0101f10 0x7fefc01011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefc501a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefc01028f0 0x7fefc019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:42.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefc501a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefc0103430 0x7fefc01a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:42.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefc501a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fefc010d650 con 0x7fefc0103430 2024-05-21T15:32:42.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefc501a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fefc010d4d0 con 0x7fefc0101f10 2024-05-21T15:32:42.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefc501a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fefc010d7d0 con 0x7fefc01028f0 2024-05-21T15:32:42.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefbf59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefc0103430 0x7fefc01a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:42.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefbf59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefc0103430 0x7fefc01a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38834/0 (socket says 172.21.15.38:38834) 2024-05-21T15:32:42.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefbf59e640 1 -- 172.21.15.38:0/3410447772 learned_addr learned my addr 172.21.15.38:0/3410447772 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:42.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefbed9d640 1 --2- 172.21.15.38:0/3410447772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefc0101f10 0x7fefc01011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:42.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefbe59c640 1 --2- 172.21.15.38:0/3410447772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefc01028f0 0x7fefc019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:42.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefbf59e640 1 -- 172.21.15.38:0/3410447772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefc01028f0 msgr2=0x7fefc019df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:42.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefbf59e640 1 --2- 172.21.15.38:0/3410447772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefc01028f0 0x7fefc019df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefbf59e640 1 -- 172.21.15.38:0/3410447772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefc0101f10 msgr2=0x7fefc01011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:42.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefbf59e640 1 --2- 172.21.15.38:0/3410447772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefc0101f10 0x7fefc01011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.692+0000 7fefbf59e640 1 -- 172.21.15.38:0/3410447772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc01a0c20 con 0x7fefc0103430 2024-05-21T15:32:42.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.693+0000 7fefbf59e640 1 --2- 172.21.15.38:0/3410447772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefc0103430 0x7fefc01a0490 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009230 tx=0x7fefb8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:42.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.693+0000 7fefb7fff640 1 -- 172.21.15.38:0/3410447772 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb803f030 con 0x7fefc0103430 2024-05-21T15:32:42.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.693+0000 7fefc501a640 1 -- 172.21.15.38:0/3410447772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc01a0eb0 con 0x7fefc0103430 2024-05-21T15:32:42.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.693+0000 7fefb7fff640 1 -- 172.21.15.38:0/3410447772 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb8003ce0 con 0x7fefc0103430 2024-05-21T15:32:42.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.693+0000 7fefb7fff640 1 -- 172.21.15.38:0/3410447772 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8035070 con 0x7fefc0103430 2024-05-21T15:32:42.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.693+0000 7fefc501a640 1 -- 172.21.15.38:0/3410447772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc01a1310 con 0x7fefc0103430 2024-05-21T15:32:42.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.694+0000 7fefc501a640 1 -- 172.21.15.38:0/3410447772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc01088c0 con 0x7fefc0103430 2024-05-21T15:32:42.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.695+0000 7fefb7fff640 1 -- 172.21.15.38:0/3410447772 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fefb8042030 con 0x7fefc0103430 2024-05-21T15:32:42.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.696+0000 7fefb7fff640 1 --2- 172.21.15.38:0/3410447772 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fefac0a2cd0 0x7fefac0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:42.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.696+0000 7fefb7fff640 1 -- 172.21.15.38:0/3410447772 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fefb80efac0 con 0x7fefc0103430 2024-05-21T15:32:42.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.696+0000 7fefbed9d640 1 --2- 172.21.15.38:0/3410447772 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fefac0a2cd0 0x7fefac0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:42.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.697+0000 7fefbed9d640 1 --2- 172.21.15.38:0/3410447772 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fefac0a2cd0 0x7fefac0a51e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fefc006a5a0 tx=0x7fefb0009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:42.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.701+0000 7fefb7fff640 1 -- 172.21.15.38:0/3410447772 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fefb80bca70 con 0x7fefc0103430 2024-05-21T15:32:42.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.881+0000 7fefc501a640 1 -- 172.21.15.38:0/3410447772 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fefc0061f90 con 0x7fefac0a2cd0 2024-05-21T15:32:42.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.888+0000 7fefb7fff640 1 -- 172.21.15.38:0/3410447772 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258953 (secure 0 0 0) 0x7fefc0061f90 con 0x7fefac0a2cd0 2024-05-21T15:32:42.891 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:42.892 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-05-21T15:32:42.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.891+0000 7fefc501a640 1 -- 172.21.15.38:0/3410447772 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fefac0a2cd0 msgr2=0x7fefac0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:42.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.891+0000 7fefc501a640 1 --2- 172.21.15.38:0/3410447772 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fefac0a2cd0 0x7fefac0a51e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fefc006a5a0 tx=0x7fefb0009870 comp rx=0 tx=0).stop 2024-05-21T15:32:42.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.891+0000 7fefc501a640 1 -- 172.21.15.38:0/3410447772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefc0103430 msgr2=0x7fefc01a0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:42.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.891+0000 7fefc501a640 1 --2- 172.21.15.38:0/3410447772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefc0103430 0x7fefc01a0490 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009230 tx=0x7fefb8004360 comp rx=0 tx=0).stop 2024-05-21T15:32:42.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.891+0000 7fefc501a640 1 -- 172.21.15.38:0/3410447772 shutdown_connections 2024-05-21T15:32:42.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.891+0000 7fefc501a640 1 --2- 172.21.15.38:0/3410447772 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fefac0a2cd0 0x7fefac0a51e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.891+0000 7fefc501a640 1 --2- 172.21.15.38:0/3410447772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefc0103430 0x7fefc01a0490 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.891+0000 7fefc501a640 1 --2- 172.21.15.38:0/3410447772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefc01028f0 0x7fefc019df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.891+0000 7fefc501a640 1 --2- 172.21.15.38:0/3410447772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefc0101f10 0x7fefc01011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:42.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.891+0000 7fefc501a640 1 -- 172.21.15.38:0/3410447772 >> 172.21.15.38:0/3410447772 conn(0x7fefc00fc330 msgr2=0x7fefc0105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:42.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.892+0000 7fefc501a640 1 -- 172.21.15.38:0/3410447772 shutdown_connections 2024-05-21T15:32:42.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:42.892+0000 7fefc501a640 1 -- 172.21.15.38:0/3410447772 wait complete. 2024-05-21T15:32:42.903 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-05-21T15:32:42.213458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597702,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":359,"num_write_kb":2993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372908,"kb_used_data":11276,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568916,"statfs":{"total":1151940427776,"available":1151558569984,"internally_reserved":0,"allocated":11546624,"data_stored":7109933,"data_compressed":109704,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7334,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":178,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.550967"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839664+0000","last_change":"2024-05-21T15:32:33.687244+0000","last_active":"2024-05-21T15:32:35.839664+0000","last_peered":"2024-05-21T15:32:35.839664+0000","last_clean":"2024-05-21T15:32:35.839664+0000","last_became_active":"2024-05-21T15:32:33.686515+0000","last_became_peered":"2024-05-21T15:32:33.686515+0000","last_unstale":"2024-05-21T15:32:35.839664+0000","last_undegraded":"2024-05-21T15:32:35.839664+0000","last_fullsized":"2024-05-21T15:32:35.839664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:10:18.396997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839287+0000","last_change":"2024-05-21T15:32:33.678497+0000","last_active":"2024-05-21T15:32:35.839287+0000","last_peered":"2024-05-21T15:32:35.839287+0000","last_clean":"2024-05-21T15:32:35.839287+0000","last_became_active":"2024-05-21T15:32:33.678001+0000","last_became_peered":"2024-05-21T15:32:33.678001+0000","last_unstale":"2024-05-21T15:32:35.839287+0000","last_undegraded":"2024-05-21T15:32:35.839287+0000","last_fullsized":"2024-05-21T15:32:35.839287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:14:48.255675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839765+0000","last_change":"2024-05-21T15:32:33.692444+0000","last_active":"2024-05-21T15:32:35.839765+0000","last_peered":"2024-05-21T15:32:35.839765+0000","last_clean":"2024-05-21T15:32:35.839765+0000","last_became_active":"2024-05-21T15:32:33.691662+0000","last_became_peered":"2024-05-21T15:32:33.691662+0000","last_unstale":"2024-05-21T15:32:35.839765+0000","last_undegraded":"2024-05-21T15:32:35.839765+0000","last_fullsized":"2024-05-21T15:32:35.839765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:26:25.929879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661999+0000","last_change":"2024-05-21T15:32:33.692297+0000","last_active":"2024-05-21T15:32:34.661999+0000","last_peered":"2024-05-21T15:32:34.661999+0000","last_clean":"2024-05-21T15:32:34.661999+0000","last_became_active":"2024-05-21T15:32:33.691925+0000","last_became_peered":"2024-05-21T15:32:33.691925+0000","last_unstale":"2024-05-21T15:32:34.661999+0000","last_undegraded":"2024-05-21T15:32:34.661999+0000","last_fullsized":"2024-05-21T15:32:34.661999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:40:45.448485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839363+0000","last_change":"2024-05-21T15:32:33.685511+0000","last_active":"2024-05-21T15:32:35.839363+0000","last_peered":"2024-05-21T15:32:35.839363+0000","last_clean":"2024-05-21T15:32:35.839363+0000","last_became_active":"2024-05-21T15:32:33.685182+0000","last_became_peered":"2024-05-21T15:32:33.685182+0000","last_unstale":"2024-05-21T15:32:35.839363+0000","last_undegraded":"2024-05-21T15:32:35.839363+0000","last_fullsized":"2024-05-21T15:32:35.839363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:59:06.680712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.737808+0000","last_change":"2024-05-21T15:32:33.682400+0000","last_active":"2024-05-21T15:32:34.737808+0000","last_peered":"2024-05-21T15:32:34.737808+0000","last_clean":"2024-05-21T15:32:34.737808+0000","last_became_active":"2024-05-21T15:32:33.682068+0000","last_became_peered":"2024-05-21T15:32:33.682068+0000","last_unstale":"2024-05-21T15:32:34.737808+0000","last_undegraded":"2024-05-21T15:32:34.737808+0000","last_fullsized":"2024-05-21T15:32:34.737808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:31:50.968706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838180+0000","last_change":"2024-05-21T15:32:33.690099+0000","last_active":"2024-05-21T15:32:34.838180+0000","last_peered":"2024-05-21T15:32:34.838180+0000","last_clean":"2024-05-21T15:32:34.838180+0000","last_became_active":"2024-05-21T15:32:33.689762+0000","last_became_peered":"2024-05-21T15:32:33.689762+0000","last_unstale":"2024-05-21T15:32:34.838180+0000","last_undegraded":"2024-05-21T15:32:34.838180+0000","last_fullsized":"2024-05-21T15:32:34.838180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:55:20.331097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661474+0000","last_change":"2024-05-21T15:32:33.691701+0000","last_active":"2024-05-21T15:32:34.661474+0000","last_peered":"2024-05-21T15:32:34.661474+0000","last_clean":"2024-05-21T15:32:34.661474+0000","last_became_active":"2024-05-21T15:32:33.691386+0000","last_became_peered":"2024-05-21T15:32:33.691386+0000","last_unstale":"2024-05-21T15:32:34.661474+0000","last_undegraded":"2024-05-21T15:32:34.661474+0000","last_fullsized":"2024-05-21T15:32:34.661474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:54:45.691392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839632+0000","last_change":"2024-05-21T15:32:33.677484+0000","last_active":"2024-05-21T15:32:35.839632+0000","last_peered":"2024-05-21T15:32:35.839632+0000","last_clean":"2024-05-21T15:32:35.839632+0000","last_became_active":"2024-05-21T15:32:33.677160+0000","last_became_peered":"2024-05-21T15:32:33.677160+0000","last_unstale":"2024-05-21T15:32:35.839632+0000","last_undegraded":"2024-05-21T15:32:35.839632+0000","last_fullsized":"2024-05-21T15:32:35.839632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:38:00.914393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.738016+0000","last_change":"2024-05-21T15:32:33.686957+0000","last_active":"2024-05-21T15:32:34.738016+0000","last_peered":"2024-05-21T15:32:34.738016+0000","last_clean":"2024-05-21T15:32:34.738016+0000","last_became_active":"2024-05-21T15:32:33.685338+0000","last_became_peered":"2024-05-21T15:32:33.685338+0000","last_unstale":"2024-05-21T15:32:34.738016+0000","last_undegraded":"2024-05-21T15:32:34.738016+0000","last_fullsized":"2024-05-21T15:32:34.738016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:04:58.115974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838581+0000","last_change":"2024-05-21T15:32:33.693311+0000","last_active":"2024-05-21T15:32:34.838581+0000","last_peered":"2024-05-21T15:32:34.838581+0000","last_clean":"2024-05-21T15:32:34.838581+0000","last_became_active":"2024-05-21T15:32:33.692959+0000","last_became_peered":"2024-05-21T15:32:33.692959+0000","last_unstale":"2024-05-21T15:32:34.838581+0000","last_undegraded":"2024-05-21T15:32:34.838581+0000","last_fullsized":"2024-05-21T15:32:34.838581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:35:12.918524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661979+0000","last_change":"2024-05-21T15:32:33.693254+0000","last_active":"2024-05-21T15:32:34.661979+0000","last_peered":"2024-05-21T15:32:34.661979+0000","last_clean":"2024-05-21T15:32:34.661979+0000","last_became_active":"2024-05-21T15:32:33.692901+0000","last_became_peered":"2024-05-21T15:32:33.692901+0000","last_unstale":"2024-05-21T15:32:34.661979+0000","last_undegraded":"2024-05-21T15:32:34.661979+0000","last_fullsized":"2024-05-21T15:32:34.661979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:36:30.591479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840022+0000","last_change":"2024-05-21T15:32:33.675685+0000","last_active":"2024-05-21T15:32:35.840022+0000","last_peered":"2024-05-21T15:32:35.840022+0000","last_clean":"2024-05-21T15:32:35.840022+0000","last_became_active":"2024-05-21T15:32:33.675351+0000","last_became_peered":"2024-05-21T15:32:33.675351+0000","last_unstale":"2024-05-21T15:32:35.840022+0000","last_undegraded":"2024-05-21T15:32:35.840022+0000","last_fullsized":"2024-05-21T15:32:35.840022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:30:22.731506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839064+0000","last_change":"2024-05-21T15:32:33.678315+0000","last_active":"2024-05-21T15:32:35.839064+0000","last_peered":"2024-05-21T15:32:35.839064+0000","last_clean":"2024-05-21T15:32:35.839064+0000","last_became_active":"2024-05-21T15:32:33.677787+0000","last_became_peered":"2024-05-21T15:32:33.677787+0000","last_unstale":"2024-05-21T15:32:35.839064+0000","last_undegraded":"2024-05-21T15:32:35.839064+0000","last_fullsized":"2024-05-21T15:32:35.839064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:17:24.826525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839212+0000","last_change":"2024-05-21T15:32:33.689994+0000","last_active":"2024-05-21T15:32:35.839212+0000","last_peered":"2024-05-21T15:32:35.839212+0000","last_clean":"2024-05-21T15:32:35.839212+0000","last_became_active":"2024-05-21T15:32:33.689607+0000","last_became_peered":"2024-05-21T15:32:33.689607+0000","last_unstale":"2024-05-21T15:32:35.839212+0000","last_undegraded":"2024-05-21T15:32:35.839212+0000","last_fullsized":"2024-05-21T15:32:35.839212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:22:07.081722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839092+0000","last_change":"2024-05-21T15:32:33.683597+0000","last_active":"2024-05-21T15:32:35.839092+0000","last_peered":"2024-05-21T15:32:35.839092+0000","last_clean":"2024-05-21T15:32:35.839092+0000","last_became_active":"2024-05-21T15:32:33.681948+0000","last_became_peered":"2024-05-21T15:32:33.681948+0000","last_unstale":"2024-05-21T15:32:35.839092+0000","last_undegraded":"2024-05-21T15:32:35.839092+0000","last_fullsized":"2024-05-21T15:32:35.839092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:26:50.086019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840072+0000","last_change":"2024-05-21T15:32:33.683329+0000","last_active":"2024-05-21T15:32:35.840072+0000","last_peered":"2024-05-21T15:32:35.840072+0000","last_clean":"2024-05-21T15:32:35.840072+0000","last_became_active":"2024-05-21T15:32:33.682947+0000","last_became_peered":"2024-05-21T15:32:33.682947+0000","last_unstale":"2024-05-21T15:32:35.840072+0000","last_undegraded":"2024-05-21T15:32:35.840072+0000","last_fullsized":"2024-05-21T15:32:35.840072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:24:27.502257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839889+0000","last_change":"2024-05-21T15:32:33.680391+0000","last_active":"2024-05-21T15:32:35.839889+0000","last_peered":"2024-05-21T15:32:35.839889+0000","last_clean":"2024-05-21T15:32:35.839889+0000","last_became_active":"2024-05-21T15:32:33.679996+0000","last_became_peered":"2024-05-21T15:32:33.679996+0000","last_unstale":"2024-05-21T15:32:35.839889+0000","last_undegraded":"2024-05-21T15:32:35.839889+0000","last_fullsized":"2024-05-21T15:32:35.839889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:56:43.658565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840687+0000","last_change":"2024-05-21T15:32:33.688783+0000","last_active":"2024-05-21T15:32:35.840687+0000","last_peered":"2024-05-21T15:32:35.840687+0000","last_clean":"2024-05-21T15:32:35.840687+0000","last_became_active":"2024-05-21T15:32:33.688006+0000","last_became_peered":"2024-05-21T15:32:33.688006+0000","last_unstale":"2024-05-21T15:32:35.840687+0000","last_undegraded":"2024-05-21T15:32:35.840687+0000","last_fullsized":"2024-05-21T15:32:35.840687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:17:44.489759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:40.999622+0000","last_change":"2024-05-21T15:32:33.182251+0000","last_active":"2024-05-21T15:32:40.999622+0000","last_peered":"2024-05-21T15:32:40.999622+0000","last_clean":"2024-05-21T15:32:40.999622+0000","last_became_active":"2024-05-21T15:32:33.181893+0000","last_became_peered":"2024-05-21T15:32:33.181893+0000","last_unstale":"2024-05-21T15:32:40.999622+0000","last_undegraded":"2024-05-21T15:32:40.999622+0000","last_fullsized":"2024-05-21T15:32:40.999622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:23:36.863186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661843+0000","last_change":"2024-05-21T15:32:33.691962+0000","last_active":"2024-05-21T15:32:34.661843+0000","last_peered":"2024-05-21T15:32:34.661843+0000","last_clean":"2024-05-21T15:32:34.661843+0000","last_became_active":"2024-05-21T15:32:33.691619+0000","last_became_peered":"2024-05-21T15:32:33.691619+0000","last_unstale":"2024-05-21T15:32:34.661843+0000","last_undegraded":"2024-05-21T15:32:34.661843+0000","last_fullsized":"2024-05-21T15:32:34.661843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:50:14.523567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.266086+0000","last_change":"2024-05-21T15:32:33.183772+0000","last_active":"2024-05-21T15:32:35.266086+0000","last_peered":"2024-05-21T15:32:35.266086+0000","last_clean":"2024-05-21T15:32:35.266086+0000","last_became_active":"2024-05-21T15:32:33.183440+0000","last_became_peered":"2024-05-21T15:32:33.183440+0000","last_unstale":"2024-05-21T15:32:35.266086+0000","last_undegraded":"2024-05-21T15:32:35.266086+0000","last_fullsized":"2024-05-21T15:32:35.266086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:53:00.714359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840384+0000","last_change":"2024-05-21T15:32:33.682544+0000","last_active":"2024-05-21T15:32:35.840384+0000","last_peered":"2024-05-21T15:32:35.840384+0000","last_clean":"2024-05-21T15:32:35.840384+0000","last_became_active":"2024-05-21T15:32:33.682161+0000","last_became_peered":"2024-05-21T15:32:33.682161+0000","last_unstale":"2024-05-21T15:32:35.840384+0000","last_undegraded":"2024-05-21T15:32:35.840384+0000","last_fullsized":"2024-05-21T15:32:35.840384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:20:38.143508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'11","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:39.986337+0000","last_change":"2024-05-21T15:32:33.182400+0000","last_active":"2024-05-21T15:32:39.986337+0000","last_peered":"2024-05-21T15:32:39.986337+0000","last_clean":"2024-05-21T15:32:39.986337+0000","last_became_active":"2024-05-21T15:32:33.181745+0000","last_became_peered":"2024-05-21T15:32:33.181745+0000","last_unstale":"2024-05-21T15:32:39.986337+0000","last_undegraded":"2024-05-21T15:32:39.986337+0000","last_fullsized":"2024-05-21T15:32:39.986337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:39:41.669529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839137+0000","last_change":"2024-05-21T15:32:33.682700+0000","last_active":"2024-05-21T15:32:35.839137+0000","last_peered":"2024-05-21T15:32:35.839137+0000","last_clean":"2024-05-21T15:32:35.839137+0000","last_became_active":"2024-05-21T15:32:33.680876+0000","last_became_peered":"2024-05-21T15:32:33.680876+0000","last_unstale":"2024-05-21T15:32:35.839137+0000","last_undegraded":"2024-05-21T15:32:35.839137+0000","last_fullsized":"2024-05-21T15:32:35.839137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:30.694520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:39.984416+0000","last_change":"2024-05-21T15:32:33.179114+0000","last_active":"2024-05-21T15:32:39.984416+0000","last_peered":"2024-05-21T15:32:39.984416+0000","last_clean":"2024-05-21T15:32:39.984416+0000","last_became_active":"2024-05-21T15:32:33.178485+0000","last_became_peered":"2024-05-21T15:32:33.178485+0000","last_unstale":"2024-05-21T15:32:39.984416+0000","last_undegraded":"2024-05-21T15:32:39.984416+0000","last_fullsized":"2024-05-21T15:32:39.984416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:19:56.501287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.663241+0000","last_change":"2024-05-21T15:32:33.691874+0000","last_active":"2024-05-21T15:32:34.663241+0000","last_peered":"2024-05-21T15:32:34.663241+0000","last_clean":"2024-05-21T15:32:34.663241+0000","last_became_active":"2024-05-21T15:32:33.691546+0000","last_became_peered":"2024-05-21T15:32:33.691546+0000","last_unstale":"2024-05-21T15:32:34.663241+0000","last_undegraded":"2024-05-21T15:32:34.663241+0000","last_fullsized":"2024-05-21T15:32:34.663241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:40:09.281792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:40.993419+0000","last_change":"2024-05-21T15:32:33.183457+0000","last_active":"2024-05-21T15:32:40.993419+0000","last_peered":"2024-05-21T15:32:40.993419+0000","last_clean":"2024-05-21T15:32:40.993419+0000","last_became_active":"2024-05-21T15:32:33.183127+0000","last_became_peered":"2024-05-21T15:32:33.183127+0000","last_unstale":"2024-05-21T15:32:40.993419+0000","last_undegraded":"2024-05-21T15:32:40.993419+0000","last_fullsized":"2024-05-21T15:32:40.993419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:41:05.179629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839739+0000","last_change":"2024-05-21T15:32:33.677260+0000","last_active":"2024-05-21T15:32:35.839739+0000","last_peered":"2024-05-21T15:32:35.839739+0000","last_clean":"2024-05-21T15:32:35.839739+0000","last_became_active":"2024-05-21T15:32:33.676905+0000","last_became_peered":"2024-05-21T15:32:33.676905+0000","last_unstale":"2024-05-21T15:32:35.839739+0000","last_undegraded":"2024-05-21T15:32:35.839739+0000","last_fullsized":"2024-05-21T15:32:35.839739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:51:41.543066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.737559+0000","last_change":"2024-05-21T15:32:33.181819+0000","last_active":"2024-05-21T15:32:34.737559+0000","last_peered":"2024-05-21T15:32:34.737559+0000","last_clean":"2024-05-21T15:32:34.737559+0000","last_became_active":"2024-05-21T15:32:33.181471+0000","last_became_peered":"2024-05-21T15:32:33.181471+0000","last_unstale":"2024-05-21T15:32:34.737559+0000","last_undegraded":"2024-05-21T15:32:34.737559+0000","last_fullsized":"2024-05-21T15:32:34.737559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:21:54.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.663463+0000","last_change":"2024-05-21T15:32:33.691072+0000","last_active":"2024-05-21T15:32:34.663463+0000","last_peered":"2024-05-21T15:32:34.663463+0000","last_clean":"2024-05-21T15:32:34.663463+0000","last_became_active":"2024-05-21T15:32:33.690663+0000","last_became_peered":"2024-05-21T15:32:33.690663+0000","last_unstale":"2024-05-21T15:32:34.663463+0000","last_undegraded":"2024-05-21T15:32:34.663463+0000","last_fullsized":"2024-05-21T15:32:34.663463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:52:44.549537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:37.960760+0000","last_change":"2024-05-21T15:32:33.179588+0000","last_active":"2024-05-21T15:32:37.960760+0000","last_peered":"2024-05-21T15:32:37.960760+0000","last_clean":"2024-05-21T15:32:37.960760+0000","last_became_active":"2024-05-21T15:32:33.179115+0000","last_became_peered":"2024-05-21T15:32:33.179115+0000","last_unstale":"2024-05-21T15:32:37.960760+0000","last_undegraded":"2024-05-21T15:32:37.960760+0000","last_fullsized":"2024-05-21T15:32:37.960760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:18:43.904584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661542+0000","last_change":"2024-05-21T15:32:33.689456+0000","last_active":"2024-05-21T15:32:34.661542+0000","last_peered":"2024-05-21T15:32:34.661542+0000","last_clean":"2024-05-21T15:32:34.661542+0000","last_became_active":"2024-05-21T15:32:33.689142+0000","last_became_peered":"2024-05-21T15:32:33.689142+0000","last_unstale":"2024-05-21T15:32:34.661542+0000","last_undegraded":"2024-05-21T15:32:34.661542+0000","last_fullsized":"2024-05-21T15:32:34.661542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:00:13.768272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:39.986300+0000","last_change":"2024-05-21T15:32:33.184865+0000","last_active":"2024-05-21T15:32:39.986300+0000","last_peered":"2024-05-21T15:32:39.986300+0000","last_clean":"2024-05-21T15:32:39.986300+0000","last_became_active":"2024-05-21T15:32:33.184540+0000","last_became_peered":"2024-05-21T15:32:33.184540+0000","last_unstale":"2024-05-21T15:32:39.986300+0000","last_undegraded":"2024-05-21T15:32:39.986300+0000","last_fullsized":"2024-05-21T15:32:39.986300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:09:20.072906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839350+0000","last_change":"2024-05-21T15:32:33.685478+0000","last_active":"2024-05-21T15:32:35.839350+0000","last_peered":"2024-05-21T15:32:35.839350+0000","last_clean":"2024-05-21T15:32:35.839350+0000","last_became_active":"2024-05-21T15:32:33.685137+0000","last_became_peered":"2024-05-21T15:32:33.685137+0000","last_unstale":"2024-05-21T15:32:35.839350+0000","last_undegraded":"2024-05-21T15:32:35.839350+0000","last_fullsized":"2024-05-21T15:32:35.839350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:39:40.815876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:36.945491+0000","last_change":"2024-05-21T15:32:33.179215+0000","last_active":"2024-05-21T15:32:36.945491+0000","last_peered":"2024-05-21T15:32:36.945491+0000","last_clean":"2024-05-21T15:32:36.945491+0000","last_became_active":"2024-05-21T15:32:33.178846+0000","last_became_peered":"2024-05-21T15:32:33.178846+0000","last_unstale":"2024-05-21T15:32:36.945491+0000","last_undegraded":"2024-05-21T15:32:36.945491+0000","last_fullsized":"2024-05-21T15:32:36.945491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:29:47.289968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839643+0000","last_change":"2024-05-21T15:32:33.684900+0000","last_active":"2024-05-21T15:32:35.839643+0000","last_peered":"2024-05-21T15:32:35.839643+0000","last_clean":"2024-05-21T15:32:35.839643+0000","last_became_active":"2024-05-21T15:32:33.681374+0000","last_became_peered":"2024-05-21T15:32:33.681374+0000","last_unstale":"2024-05-21T15:32:35.839643+0000","last_undegraded":"2024-05-21T15:32:35.839643+0000","last_fullsized":"2024-05-21T15:32:35.839643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:51:38.079726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:36.944183+0000","last_change":"2024-05-21T15:32:33.189945+0000","last_active":"2024-05-21T15:32:36.944183+0000","last_peered":"2024-05-21T15:32:36.944183+0000","last_clean":"2024-05-21T15:32:36.944183+0000","last_became_active":"2024-05-21T15:32:33.189557+0000","last_became_peered":"2024-05-21T15:32:33.189557+0000","last_unstale":"2024-05-21T15:32:36.944183+0000","last_undegraded":"2024-05-21T15:32:36.944183+0000","last_fullsized":"2024-05-21T15:32:36.944183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:08.427043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840394+0000","last_change":"2024-05-21T15:32:33.692896+0000","last_active":"2024-05-21T15:32:35.840394+0000","last_peered":"2024-05-21T15:32:35.840394+0000","last_clean":"2024-05-21T15:32:35.840394+0000","last_became_active":"2024-05-21T15:32:33.692527+0000","last_became_peered":"2024-05-21T15:32:33.692527+0000","last_unstale":"2024-05-21T15:32:35.840394+0000","last_undegraded":"2024-05-21T15:32:35.840394+0000","last_fullsized":"2024-05-21T15:32:35.840394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:45:45.418063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:38.975228+0000","last_change":"2024-05-21T15:32:33.172989+0000","last_active":"2024-05-21T15:32:38.975228+0000","last_peered":"2024-05-21T15:32:38.975228+0000","last_clean":"2024-05-21T15:32:38.975228+0000","last_became_active":"2024-05-21T15:32:33.172215+0000","last_became_peered":"2024-05-21T15:32:33.172215+0000","last_unstale":"2024-05-21T15:32:38.975228+0000","last_undegraded":"2024-05-21T15:32:38.975228+0000","last_fullsized":"2024-05-21T15:32:38.975228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:09:39.287636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839533+0000","last_change":"2024-05-21T15:32:33.678441+0000","last_active":"2024-05-21T15:32:35.839533+0000","last_peered":"2024-05-21T15:32:35.839533+0000","last_clean":"2024-05-21T15:32:35.839533+0000","last_became_active":"2024-05-21T15:32:33.677833+0000","last_became_peered":"2024-05-21T15:32:33.677833+0000","last_unstale":"2024-05-21T15:32:35.839533+0000","last_undegraded":"2024-05-21T15:32:35.839533+0000","last_fullsized":"2024-05-21T15:32:35.839533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:36.124064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:36.942609+0000","last_change":"2024-05-21T15:32:33.178153+0000","last_active":"2024-05-21T15:32:36.942609+0000","last_peered":"2024-05-21T15:32:36.942609+0000","last_clean":"2024-05-21T15:32:36.942609+0000","last_became_active":"2024-05-21T15:32:33.177792+0000","last_became_peered":"2024-05-21T15:32:33.177792+0000","last_unstale":"2024-05-21T15:32:36.942609+0000","last_undegraded":"2024-05-21T15:32:36.942609+0000","last_fullsized":"2024-05-21T15:32:36.942609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:43:41.865204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839941+0000","last_change":"2024-05-21T15:32:33.681328+0000","last_active":"2024-05-21T15:32:35.839941+0000","last_peered":"2024-05-21T15:32:35.839941+0000","last_clean":"2024-05-21T15:32:35.839941+0000","last_became_active":"2024-05-21T15:32:33.680975+0000","last_became_peered":"2024-05-21T15:32:33.680975+0000","last_unstale":"2024-05-21T15:32:35.839941+0000","last_undegraded":"2024-05-21T15:32:35.839941+0000","last_fullsized":"2024-05-21T15:32:35.839941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:50:27.199348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:37.963247+0000","last_change":"2024-05-21T15:32:33.182492+0000","last_active":"2024-05-21T15:32:37.963247+0000","last_peered":"2024-05-21T15:32:37.963247+0000","last_clean":"2024-05-21T15:32:37.963247+0000","last_became_active":"2024-05-21T15:32:33.182152+0000","last_became_peered":"2024-05-21T15:32:33.182152+0000","last_unstale":"2024-05-21T15:32:37.963247+0000","last_undegraded":"2024-05-21T15:32:37.963247+0000","last_fullsized":"2024-05-21T15:32:37.963247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:04:07.432742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839678+0000","last_change":"2024-05-21T15:32:33.686623+0000","last_active":"2024-05-21T15:32:35.839678+0000","last_peered":"2024-05-21T15:32:35.839678+0000","last_clean":"2024-05-21T15:32:35.839678+0000","last_became_active":"2024-05-21T15:32:33.685816+0000","last_became_peered":"2024-05-21T15:32:33.685816+0000","last_unstale":"2024-05-21T15:32:35.839678+0000","last_undegraded":"2024-05-21T15:32:35.839678+0000","last_fullsized":"2024-05-21T15:32:35.839678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:15:20.354113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:39.986495+0000","last_change":"2024-05-21T15:32:33.182218+0000","last_active":"2024-05-21T15:32:39.986495+0000","last_peered":"2024-05-21T15:32:39.986495+0000","last_clean":"2024-05-21T15:32:39.986495+0000","last_became_active":"2024-05-21T15:32:33.181883+0000","last_became_peered":"2024-05-21T15:32:33.181883+0000","last_unstale":"2024-05-21T15:32:39.986495+0000","last_undegraded":"2024-05-21T15:32:39.986495+0000","last_fullsized":"2024-05-21T15:32:39.986495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:24:07.963898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838515+0000","last_change":"2024-05-21T15:32:33.689936+0000","last_active":"2024-05-21T15:32:34.838515+0000","last_peered":"2024-05-21T15:32:34.838515+0000","last_clean":"2024-05-21T15:32:34.838515+0000","last_became_active":"2024-05-21T15:32:33.689452+0000","last_became_peered":"2024-05-21T15:32:33.689452+0000","last_unstale":"2024-05-21T15:32:34.838515+0000","last_undegraded":"2024-05-21T15:32:34.838515+0000","last_fullsized":"2024-05-21T15:32:34.838515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:00.420730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:40.998669+0000","last_change":"2024-05-21T15:32:33.181354+0000","last_active":"2024-05-21T15:32:40.998669+0000","last_peered":"2024-05-21T15:32:40.998669+0000","last_clean":"2024-05-21T15:32:40.998669+0000","last_became_active":"2024-05-21T15:32:33.180983+0000","last_became_peered":"2024-05-21T15:32:33.180983+0000","last_unstale":"2024-05-21T15:32:40.998669+0000","last_undegraded":"2024-05-21T15:32:40.998669+0000","last_fullsized":"2024-05-21T15:32:40.998669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:18:46.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838079+0000","last_change":"2024-05-21T15:32:33.690992+0000","last_active":"2024-05-21T15:32:34.838079+0000","last_peered":"2024-05-21T15:32:34.838079+0000","last_clean":"2024-05-21T15:32:34.838079+0000","last_became_active":"2024-05-21T15:32:33.690663+0000","last_became_peered":"2024-05-21T15:32:33.690663+0000","last_unstale":"2024-05-21T15:32:34.838079+0000","last_undegraded":"2024-05-21T15:32:34.838079+0000","last_fullsized":"2024-05-21T15:32:34.838079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:17:48.818803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:38.975485+0000","last_change":"2024-05-21T15:32:33.182804+0000","last_active":"2024-05-21T15:32:38.975485+0000","last_peered":"2024-05-21T15:32:38.975485+0000","last_clean":"2024-05-21T15:32:38.975485+0000","last_became_active":"2024-05-21T15:32:33.182449+0000","last_became_peered":"2024-05-21T15:32:33.182449+0000","last_unstale":"2024-05-21T15:32:38.975485+0000","last_undegraded":"2024-05-21T15:32:38.975485+0000","last_fullsized":"2024-05-21T15:32:38.975485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:33:56.320675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.662821+0000","last_change":"2024-05-21T15:32:33.691112+0000","last_active":"2024-05-21T15:32:34.662821+0000","last_peered":"2024-05-21T15:32:34.662821+0000","last_clean":"2024-05-21T15:32:34.662821+0000","last_became_active":"2024-05-21T15:32:33.690726+0000","last_became_peered":"2024-05-21T15:32:33.690726+0000","last_unstale":"2024-05-21T15:32:34.662821+0000","last_undegraded":"2024-05-21T15:32:34.662821+0000","last_fullsized":"2024-05-21T15:32:34.662821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:59:29.002636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840188+0000","last_change":"2024-05-21T15:32:33.179565+0000","last_active":"2024-05-21T15:32:35.840188+0000","last_peered":"2024-05-21T15:32:35.840188+0000","last_clean":"2024-05-21T15:32:35.840188+0000","last_became_active":"2024-05-21T15:32:33.179168+0000","last_became_peered":"2024-05-21T15:32:33.179168+0000","last_unstale":"2024-05-21T15:32:35.840188+0000","last_undegraded":"2024-05-21T15:32:35.840188+0000","last_fullsized":"2024-05-21T15:32:35.840188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:23:39.210931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839646+0000","last_change":"2024-05-21T15:32:33.691348+0000","last_active":"2024-05-21T15:32:35.839646+0000","last_peered":"2024-05-21T15:32:35.839646+0000","last_clean":"2024-05-21T15:32:35.839646+0000","last_became_active":"2024-05-21T15:32:33.691005+0000","last_became_peered":"2024-05-21T15:32:33.691005+0000","last_unstale":"2024-05-21T15:32:35.839646+0000","last_undegraded":"2024-05-21T15:32:35.839646+0000","last_fullsized":"2024-05-21T15:32:35.839646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:33:34.533026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839536+0000","last_change":"2024-05-21T15:32:33.184850+0000","last_active":"2024-05-21T15:32:35.839536+0000","last_peered":"2024-05-21T15:32:35.839536+0000","last_clean":"2024-05-21T15:32:35.839536+0000","last_became_active":"2024-05-21T15:32:33.184480+0000","last_became_peered":"2024-05-21T15:32:33.184480+0000","last_unstale":"2024-05-21T15:32:35.839536+0000","last_undegraded":"2024-05-21T15:32:35.839536+0000","last_fullsized":"2024-05-21T15:32:35.839536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:31.327820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.737977+0000","last_change":"2024-05-21T15:32:33.689520+0000","last_active":"2024-05-21T15:32:34.737977+0000","last_peered":"2024-05-21T15:32:34.737977+0000","last_clean":"2024-05-21T15:32:34.737977+0000","last_became_active":"2024-05-21T15:32:33.688219+0000","last_became_peered":"2024-05-21T15:32:33.688219+0000","last_unstale":"2024-05-21T15:32:34.737977+0000","last_undegraded":"2024-05-21T15:32:34.737977+0000","last_fullsized":"2024-05-21T15:32:34.737977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:36.546390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838313+0000","last_change":"2024-05-21T15:32:33.183183+0000","last_active":"2024-05-21T15:32:34.838313+0000","last_peered":"2024-05-21T15:32:34.838313+0000","last_clean":"2024-05-21T15:32:34.838313+0000","last_became_active":"2024-05-21T15:32:33.182753+0000","last_became_peered":"2024-05-21T15:32:33.182753+0000","last_unstale":"2024-05-21T15:32:34.838313+0000","last_undegraded":"2024-05-21T15:32:34.838313+0000","last_fullsized":"2024-05-21T15:32:34.838313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:44:22.509265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.737967+0000","last_change":"2024-05-21T15:32:33.684019+0000","last_active":"2024-05-21T15:32:34.737967+0000","last_peered":"2024-05-21T15:32:34.737967+0000","last_clean":"2024-05-21T15:32:34.737967+0000","last_became_active":"2024-05-21T15:32:33.683276+0000","last_became_peered":"2024-05-21T15:32:33.683276+0000","last_unstale":"2024-05-21T15:32:34.737967+0000","last_undegraded":"2024-05-21T15:32:34.737967+0000","last_fullsized":"2024-05-21T15:32:34.737967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:07:14.947240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.662950+0000","last_change":"2024-05-21T15:32:33.690956+0000","last_active":"2024-05-21T15:32:34.662950+0000","last_peered":"2024-05-21T15:32:34.662950+0000","last_clean":"2024-05-21T15:32:34.662950+0000","last_became_active":"2024-05-21T15:32:33.690630+0000","last_became_peered":"2024-05-21T15:32:33.690630+0000","last_unstale":"2024-05-21T15:32:34.662950+0000","last_undegraded":"2024-05-21T15:32:34.662950+0000","last_fullsized":"2024-05-21T15:32:34.662950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:24:07.050928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:37.963452+0000","last_change":"2024-05-21T15:32:33.175127+0000","last_active":"2024-05-21T15:32:37.963452+0000","last_peered":"2024-05-21T15:32:37.963452+0000","last_clean":"2024-05-21T15:32:37.963452+0000","last_became_active":"2024-05-21T15:32:33.174724+0000","last_became_peered":"2024-05-21T15:32:33.174724+0000","last_unstale":"2024-05-21T15:32:37.963452+0000","last_undegraded":"2024-05-21T15:32:37.963452+0000","last_fullsized":"2024-05-21T15:32:37.963452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:39:17.499743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661983+0000","last_change":"2024-05-21T15:32:33.693428+0000","last_active":"2024-05-21T15:32:34.661983+0000","last_peered":"2024-05-21T15:32:34.661983+0000","last_clean":"2024-05-21T15:32:34.661983+0000","last_became_active":"2024-05-21T15:32:33.693077+0000","last_became_peered":"2024-05-21T15:32:33.693077+0000","last_unstale":"2024-05-21T15:32:34.661983+0000","last_undegraded":"2024-05-21T15:32:34.661983+0000","last_fullsized":"2024-05-21T15:32:34.661983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:08:11.908874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:40.996136+0000","last_change":"2024-05-21T15:32:33.176267+0000","last_active":"2024-05-21T15:32:40.996136+0000","last_peered":"2024-05-21T15:32:40.996136+0000","last_clean":"2024-05-21T15:32:40.996136+0000","last_became_active":"2024-05-21T15:32:33.175856+0000","last_became_peered":"2024-05-21T15:32:33.175856+0000","last_unstale":"2024-05-21T15:32:40.996136+0000","last_undegraded":"2024-05-21T15:32:40.996136+0000","last_fullsized":"2024-05-21T15:32:40.996136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:58.113791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":339,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838073+0000","last_change":"2024-05-21T15:31:20.928694+0000","last_active":"2024-05-21T15:32:34.838073+0000","last_peered":"2024-05-21T15:32:34.838073+0000","last_clean":"2024-05-21T15:32:34.838073+0000","last_became_active":"2024-05-21T15:31:20.928115+0000","last_became_peered":"2024-05-21T15:31:20.928115+0000","last_unstale":"2024-05-21T15:32:34.838073+0000","last_undegraded":"2024-05-21T15:32:34.838073+0000","last_fullsized":"2024-05-21T15:32:34.838073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:27:50.788960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:27:50.788960+0000","last_clean_scrub_stamp":"2024-05-21T15:27:50.788960+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:55:10.393374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.838976+0000","last_change":"2024-05-21T15:32:33.689305+0000","last_active":"2024-05-21T15:32:35.838976+0000","last_peered":"2024-05-21T15:32:35.838976+0000","last_clean":"2024-05-21T15:32:35.838976+0000","last_became_active":"2024-05-21T15:32:33.688908+0000","last_became_peered":"2024-05-21T15:32:33.688908+0000","last_unstale":"2024-05-21T15:32:35.838976+0000","last_undegraded":"2024-05-21T15:32:35.838976+0000","last_fullsized":"2024-05-21T15:32:35.838976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:49:18.953116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:37.959859+0000","last_change":"2024-05-21T15:32:33.187781+0000","last_active":"2024-05-21T15:32:37.959859+0000","last_peered":"2024-05-21T15:32:37.959859+0000","last_clean":"2024-05-21T15:32:37.959859+0000","last_became_active":"2024-05-21T15:32:33.187504+0000","last_became_peered":"2024-05-21T15:32:33.187504+0000","last_unstale":"2024-05-21T15:32:37.959859+0000","last_undegraded":"2024-05-21T15:32:37.959859+0000","last_fullsized":"2024-05-21T15:32:37.959859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:30.400209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:38.976167+0000","last_change":"2024-05-21T15:32:33.188706+0000","last_active":"2024-05-21T15:32:38.976167+0000","last_peered":"2024-05-21T15:32:38.976167+0000","last_clean":"2024-05-21T15:32:38.976167+0000","last_became_active":"2024-05-21T15:32:33.187462+0000","last_became_peered":"2024-05-21T15:32:33.187462+0000","last_unstale":"2024-05-21T15:32:38.976167+0000","last_undegraded":"2024-05-21T15:32:38.976167+0000","last_fullsized":"2024-05-21T15:32:38.976167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:47:45.387721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840292+0000","last_change":"2024-05-21T15:32:33.672531+0000","last_active":"2024-05-21T15:32:35.840292+0000","last_peered":"2024-05-21T15:32:35.840292+0000","last_clean":"2024-05-21T15:32:35.840292+0000","last_became_active":"2024-05-21T15:32:33.671972+0000","last_became_peered":"2024-05-21T15:32:33.671972+0000","last_unstale":"2024-05-21T15:32:35.840292+0000","last_undegraded":"2024-05-21T15:32:35.840292+0000","last_fullsized":"2024-05-21T15:32:35.840292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:10:51.062530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:40.216114+0000","last_change":"2024-05-21T15:32:33.181274+0000","last_active":"2024-05-21T15:32:40.216114+0000","last_peered":"2024-05-21T15:32:40.216114+0000","last_clean":"2024-05-21T15:32:40.216114+0000","last_became_active":"2024-05-21T15:32:33.180885+0000","last_became_peered":"2024-05-21T15:32:33.180885+0000","last_unstale":"2024-05-21T15:32:40.216114+0000","last_undegraded":"2024-05-21T15:32:40.216114+0000","last_fullsized":"2024-05-21T15:32:40.216114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:10:13.547380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661999+0000","last_change":"2024-05-21T15:32:33.683749+0000","last_active":"2024-05-21T15:32:34.661999+0000","last_peered":"2024-05-21T15:32:34.661999+0000","last_clean":"2024-05-21T15:32:34.661999+0000","last_became_active":"2024-05-21T15:32:33.683393+0000","last_became_peered":"2024-05-21T15:32:33.683393+0000","last_unstale":"2024-05-21T15:32:34.661999+0000","last_undegraded":"2024-05-21T15:32:34.661999+0000","last_fullsized":"2024-05-21T15:32:34.661999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:07:53.009651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:39.984697+0000","last_change":"2024-05-21T15:32:33.184740+0000","last_active":"2024-05-21T15:32:39.984697+0000","last_peered":"2024-05-21T15:32:39.984697+0000","last_clean":"2024-05-21T15:32:39.984697+0000","last_became_active":"2024-05-21T15:32:33.184378+0000","last_became_peered":"2024-05-21T15:32:33.184378+0000","last_unstale":"2024-05-21T15:32:39.984697+0000","last_undegraded":"2024-05-21T15:32:39.984697+0000","last_fullsized":"2024-05-21T15:32:39.984697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:21:27.610513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840273+0000","last_change":"2024-05-21T15:32:33.692816+0000","last_active":"2024-05-21T15:32:35.840273+0000","last_peered":"2024-05-21T15:32:35.840273+0000","last_clean":"2024-05-21T15:32:35.840273+0000","last_became_active":"2024-05-21T15:32:33.692477+0000","last_became_peered":"2024-05-21T15:32:33.692477+0000","last_unstale":"2024-05-21T15:32:35.840273+0000","last_undegraded":"2024-05-21T15:32:35.840273+0000","last_fullsized":"2024-05-21T15:32:35.840273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:28:25.532884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:36.944469+0000","last_change":"2024-05-21T15:32:33.183344+0000","last_active":"2024-05-21T15:32:36.944469+0000","last_peered":"2024-05-21T15:32:36.944469+0000","last_clean":"2024-05-21T15:32:36.944469+0000","last_became_active":"2024-05-21T15:32:33.182977+0000","last_became_peered":"2024-05-21T15:32:33.182977+0000","last_unstale":"2024-05-21T15:32:36.944469+0000","last_undegraded":"2024-05-21T15:32:36.944469+0000","last_fullsized":"2024-05-21T15:32:36.944469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:27:24.354129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839339+0000","last_change":"2024-05-21T15:32:33.677549+0000","last_active":"2024-05-21T15:32:35.839339+0000","last_peered":"2024-05-21T15:32:35.839339+0000","last_clean":"2024-05-21T15:32:35.839339+0000","last_became_active":"2024-05-21T15:32:33.677213+0000","last_became_peered":"2024-05-21T15:32:33.677213+0000","last_unstale":"2024-05-21T15:32:35.839339+0000","last_undegraded":"2024-05-21T15:32:35.839339+0000","last_fullsized":"2024-05-21T15:32:35.839339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:57:06.560434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:39.981388+0000","last_change":"2024-05-21T15:32:33.190023+0000","last_active":"2024-05-21T15:32:39.981388+0000","last_peered":"2024-05-21T15:32:39.981388+0000","last_clean":"2024-05-21T15:32:39.981388+0000","last_became_active":"2024-05-21T15:32:33.189616+0000","last_became_peered":"2024-05-21T15:32:33.189616+0000","last_unstale":"2024-05-21T15:32:39.981388+0000","last_undegraded":"2024-05-21T15:32:39.981388+0000","last_fullsized":"2024-05-21T15:32:39.981388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:03.841414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840338+0000","last_change":"2024-05-21T15:32:33.685911+0000","last_active":"2024-05-21T15:32:35.840338+0000","last_peered":"2024-05-21T15:32:35.840338+0000","last_clean":"2024-05-21T15:32:35.840338+0000","last_became_active":"2024-05-21T15:32:33.684785+0000","last_became_peered":"2024-05-21T15:32:33.684785+0000","last_unstale":"2024-05-21T15:32:35.840338+0000","last_undegraded":"2024-05-21T15:32:35.840338+0000","last_fullsized":"2024-05-21T15:32:35.840338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:39:18.370289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:38.937086+0000","last_change":"2024-05-21T15:32:33.178354+0000","last_active":"2024-05-21T15:32:38.937086+0000","last_peered":"2024-05-21T15:32:38.937086+0000","last_clean":"2024-05-21T15:32:38.937086+0000","last_became_active":"2024-05-21T15:32:33.177989+0000","last_became_peered":"2024-05-21T15:32:33.177989+0000","last_unstale":"2024-05-21T15:32:38.937086+0000","last_undegraded":"2024-05-21T15:32:38.937086+0000","last_fullsized":"2024-05-21T15:32:38.937086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:58.650831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661712+0000","last_change":"2024-05-21T15:32:33.686467+0000","last_active":"2024-05-21T15:32:34.661712+0000","last_peered":"2024-05-21T15:32:34.661712+0000","last_clean":"2024-05-21T15:32:34.661712+0000","last_became_active":"2024-05-21T15:32:33.686135+0000","last_became_peered":"2024-05-21T15:32:33.686135+0000","last_unstale":"2024-05-21T15:32:34.661712+0000","last_undegraded":"2024-05-21T15:32:34.661712+0000","last_fullsized":"2024-05-21T15:32:34.661712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:53:54.045591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:36.945036+0000","last_change":"2024-05-21T15:32:33.182936+0000","last_active":"2024-05-21T15:32:36.945036+0000","last_peered":"2024-05-21T15:32:36.945036+0000","last_clean":"2024-05-21T15:32:36.945036+0000","last_became_active":"2024-05-21T15:32:33.182587+0000","last_became_peered":"2024-05-21T15:32:33.182587+0000","last_unstale":"2024-05-21T15:32:36.945036+0000","last_undegraded":"2024-05-21T15:32:36.945036+0000","last_fullsized":"2024-05-21T15:32:36.945036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:56.434249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838677+0000","last_change":"2024-05-21T15:32:33.692325+0000","last_active":"2024-05-21T15:32:34.838677+0000","last_peered":"2024-05-21T15:32:34.838677+0000","last_clean":"2024-05-21T15:32:34.838677+0000","last_became_active":"2024-05-21T15:32:33.691971+0000","last_became_peered":"2024-05-21T15:32:33.691971+0000","last_unstale":"2024-05-21T15:32:34.838677+0000","last_undegraded":"2024-05-21T15:32:34.838677+0000","last_fullsized":"2024-05-21T15:32:34.838677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:53:28.190063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:38.975895+0000","last_change":"2024-05-21T15:32:33.186925+0000","last_active":"2024-05-21T15:32:38.975895+0000","last_peered":"2024-05-21T15:32:38.975895+0000","last_clean":"2024-05-21T15:32:38.975895+0000","last_became_active":"2024-05-21T15:32:33.186605+0000","last_became_peered":"2024-05-21T15:32:33.186605+0000","last_unstale":"2024-05-21T15:32:38.975895+0000","last_undegraded":"2024-05-21T15:32:38.975895+0000","last_fullsized":"2024-05-21T15:32:38.975895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:32.752746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838710+0000","last_change":"2024-05-21T15:32:33.686132+0000","last_active":"2024-05-21T15:32:34.838710+0000","last_peered":"2024-05-21T15:32:34.838710+0000","last_clean":"2024-05-21T15:32:34.838710+0000","last_became_active":"2024-05-21T15:32:33.685745+0000","last_became_peered":"2024-05-21T15:32:33.685745+0000","last_unstale":"2024-05-21T15:32:34.838710+0000","last_undegraded":"2024-05-21T15:32:34.838710+0000","last_fullsized":"2024-05-21T15:32:34.838710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:51:49.522297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:39.985576+0000","last_change":"2024-05-21T15:32:33.176552+0000","last_active":"2024-05-21T15:32:39.985576+0000","last_peered":"2024-05-21T15:32:39.985576+0000","last_clean":"2024-05-21T15:32:39.985576+0000","last_became_active":"2024-05-21T15:32:33.176237+0000","last_became_peered":"2024-05-21T15:32:33.176237+0000","last_unstale":"2024-05-21T15:32:39.985576+0000","last_undegraded":"2024-05-21T15:32:39.985576+0000","last_fullsized":"2024-05-21T15:32:39.985576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:09.098344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838440+0000","last_change":"2024-05-21T15:32:33.692310+0000","last_active":"2024-05-21T15:32:34.838440+0000","last_peered":"2024-05-21T15:32:34.838440+0000","last_clean":"2024-05-21T15:32:34.838440+0000","last_became_active":"2024-05-21T15:32:33.691959+0000","last_became_peered":"2024-05-21T15:32:33.691959+0000","last_unstale":"2024-05-21T15:32:34.838440+0000","last_undegraded":"2024-05-21T15:32:34.838440+0000","last_fullsized":"2024-05-21T15:32:34.838440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:57:38.243926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839998+0000","last_change":"2024-05-21T15:32:33.183219+0000","last_active":"2024-05-21T15:32:35.839998+0000","last_peered":"2024-05-21T15:32:35.839998+0000","last_clean":"2024-05-21T15:32:35.839998+0000","last_became_active":"2024-05-21T15:32:33.182847+0000","last_became_peered":"2024-05-21T15:32:33.182847+0000","last_unstale":"2024-05-21T15:32:35.839998+0000","last_undegraded":"2024-05-21T15:32:35.839998+0000","last_fullsized":"2024-05-21T15:32:35.839998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:16:05.341406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.838582+0000","last_change":"2024-05-21T15:32:33.682801+0000","last_active":"2024-05-21T15:32:34.838582+0000","last_peered":"2024-05-21T15:32:34.838582+0000","last_clean":"2024-05-21T15:32:34.838582+0000","last_became_active":"2024-05-21T15:32:33.682368+0000","last_became_peered":"2024-05-21T15:32:33.682368+0000","last_unstale":"2024-05-21T15:32:34.838582+0000","last_undegraded":"2024-05-21T15:32:34.838582+0000","last_fullsized":"2024-05-21T15:32:34.838582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:56:50.520672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840432+0000","last_change":"2024-05-21T15:32:33.684809+0000","last_active":"2024-05-21T15:32:35.840432+0000","last_peered":"2024-05-21T15:32:35.840432+0000","last_clean":"2024-05-21T15:32:35.840432+0000","last_became_active":"2024-05-21T15:32:33.684067+0000","last_became_peered":"2024-05-21T15:32:33.684067+0000","last_unstale":"2024-05-21T15:32:35.840432+0000","last_undegraded":"2024-05-21T15:32:35.840432+0000","last_fullsized":"2024-05-21T15:32:35.840432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:31:58.997470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839669+0000","last_change":"2024-05-21T15:32:33.676091+0000","last_active":"2024-05-21T15:32:35.839669+0000","last_peered":"2024-05-21T15:32:35.839669+0000","last_clean":"2024-05-21T15:32:35.839669+0000","last_became_active":"2024-05-21T15:32:33.675538+0000","last_became_peered":"2024-05-21T15:32:33.675538+0000","last_unstale":"2024-05-21T15:32:35.839669+0000","last_undegraded":"2024-05-21T15:32:35.839669+0000","last_fullsized":"2024-05-21T15:32:35.839669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:50.311152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839739+0000","last_change":"2024-05-21T15:32:33.688590+0000","last_active":"2024-05-21T15:32:35.839739+0000","last_peered":"2024-05-21T15:32:35.839739+0000","last_clean":"2024-05-21T15:32:35.839739+0000","last_became_active":"2024-05-21T15:32:33.687993+0000","last_became_peered":"2024-05-21T15:32:33.687993+0000","last_unstale":"2024-05-21T15:32:35.839739+0000","last_undegraded":"2024-05-21T15:32:35.839739+0000","last_fullsized":"2024-05-21T15:32:35.839739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:21.939483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840015+0000","last_change":"2024-05-21T15:32:33.689122+0000","last_active":"2024-05-21T15:32:35.840015+0000","last_peered":"2024-05-21T15:32:35.840015+0000","last_clean":"2024-05-21T15:32:35.840015+0000","last_became_active":"2024-05-21T15:32:33.688775+0000","last_became_peered":"2024-05-21T15:32:33.688775+0000","last_unstale":"2024-05-21T15:32:35.840015+0000","last_undegraded":"2024-05-21T15:32:35.840015+0000","last_fullsized":"2024-05-21T15:32:35.840015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:21.480005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840078+0000","last_change":"2024-05-21T15:32:33.689365+0000","last_active":"2024-05-21T15:32:35.840078+0000","last_peered":"2024-05-21T15:32:35.840078+0000","last_clean":"2024-05-21T15:32:35.840078+0000","last_became_active":"2024-05-21T15:32:33.688739+0000","last_became_peered":"2024-05-21T15:32:33.688739+0000","last_unstale":"2024-05-21T15:32:35.840078+0000","last_undegraded":"2024-05-21T15:32:35.840078+0000","last_fullsized":"2024-05-21T15:32:35.840078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:19:52.614155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840100+0000","last_change":"2024-05-21T15:32:33.681985+0000","last_active":"2024-05-21T15:32:35.840100+0000","last_peered":"2024-05-21T15:32:35.840100+0000","last_clean":"2024-05-21T15:32:35.840100+0000","last_became_active":"2024-05-21T15:32:33.681209+0000","last_became_peered":"2024-05-21T15:32:33.681209+0000","last_unstale":"2024-05-21T15:32:35.840100+0000","last_undegraded":"2024-05-21T15:32:35.840100+0000","last_fullsized":"2024-05-21T15:32:35.840100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:03:16.088136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840969+0000","last_change":"2024-05-21T15:32:33.673236+0000","last_active":"2024-05-21T15:32:35.840969+0000","last_peered":"2024-05-21T15:32:35.840969+0000","last_clean":"2024-05-21T15:32:35.840969+0000","last_became_active":"2024-05-21T15:32:33.672548+0000","last_became_peered":"2024-05-21T15:32:33.672548+0000","last_unstale":"2024-05-21T15:32:35.840969+0000","last_undegraded":"2024-05-21T15:32:35.840969+0000","last_fullsized":"2024-05-21T15:32:35.840969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:52:10.010219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661906+0000","last_change":"2024-05-21T15:32:33.695171+0000","last_active":"2024-05-21T15:32:34.661906+0000","last_peered":"2024-05-21T15:32:34.661906+0000","last_clean":"2024-05-21T15:32:34.661906+0000","last_became_active":"2024-05-21T15:32:33.694778+0000","last_became_peered":"2024-05-21T15:32:33.694778+0000","last_unstale":"2024-05-21T15:32:34.661906+0000","last_undegraded":"2024-05-21T15:32:34.661906+0000","last_fullsized":"2024-05-21T15:32:34.661906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:29.889437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.662826+0000","last_change":"2024-05-21T15:32:33.686992+0000","last_active":"2024-05-21T15:32:34.662826+0000","last_peered":"2024-05-21T15:32:34.662826+0000","last_clean":"2024-05-21T15:32:34.662826+0000","last_became_active":"2024-05-21T15:32:33.686673+0000","last_became_peered":"2024-05-21T15:32:33.686673+0000","last_unstale":"2024-05-21T15:32:34.662826+0000","last_undegraded":"2024-05-21T15:32:34.662826+0000","last_fullsized":"2024-05-21T15:32:34.662826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:18:20.668365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:34.661502+0000","last_change":"2024-05-21T15:32:33.689433+0000","last_active":"2024-05-21T15:32:34.661502+0000","last_peered":"2024-05-21T15:32:34.661502+0000","last_clean":"2024-05-21T15:32:34.661502+0000","last_became_active":"2024-05-21T15:32:33.689106+0000","last_became_peered":"2024-05-21T15:32:33.689106+0000","last_unstale":"2024-05-21T15:32:34.661502+0000","last_undegraded":"2024-05-21T15:32:34.661502+0000","last_fullsized":"2024-05-21T15:32:34.661502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:45:17.841159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839573+0000","last_change":"2024-05-21T15:32:33.687763+0000","last_active":"2024-05-21T15:32:35.839573+0000","last_peered":"2024-05-21T15:32:35.839573+0000","last_clean":"2024-05-21T15:32:35.839573+0000","last_became_active":"2024-05-21T15:32:33.685103+0000","last_became_peered":"2024-05-21T15:32:33.685103+0000","last_unstale":"2024-05-21T15:32:35.839573+0000","last_undegraded":"2024-05-21T15:32:35.839573+0000","last_fullsized":"2024-05-21T15:32:35.839573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:56:39.454536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.840217+0000","last_change":"2024-05-21T15:32:33.679225+0000","last_active":"2024-05-21T15:32:35.840217+0000","last_peered":"2024-05-21T15:32:35.840217+0000","last_clean":"2024-05-21T15:32:35.840217+0000","last_became_active":"2024-05-21T15:32:33.678890+0000","last_became_peered":"2024-05-21T15:32:33.678890+0000","last_unstale":"2024-05-21T15:32:35.840217+0000","last_undegraded":"2024-05-21T15:32:35.840217+0000","last_fullsized":"2024-05-21T15:32:35.840217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:12:23.721449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-21T15:32:35.839788+0000","last_change":"2024-05-21T15:32:33.688659+0000","last_active":"2024-05-21T15:32:35.839788+0000","last_peered":"2024-05-21T15:32:35.839788+0000","last_clean":"2024-05-21T15:32:35.839788+0000","last_became_active":"2024-05-21T15:32:33.688293+0000","last_became_peered":"2024-05-21T15:32:33.688293+0000","last_unstale":"2024-05-21T15:32:35.839788+0000","last_undegraded":"2024-05-21T15:32:35.839788+0000","last_fullsized":"2024-05-21T15:32:35.839788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:08:49.543237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":287606,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7334,"num_objects":84,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":178,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":41405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612625,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":1034508,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":442694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":442310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":1033144,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462951,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":464219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":444141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":445156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":442002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888115,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":442272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":1034760,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":22647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T15:32:42.907 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-21T15:32:42.907 INFO:tasks.cephadm:Setting up client nodes... 2024-05-21T15:32:42.908 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-21T15:32:43.195 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:32:43.231 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:42 smithi053 ceph-mon[36496]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:43.231 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:42 smithi053 ceph-mon[36496]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:43.231 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:42 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2661159916' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:43.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:42 smithi045 ceph-mon[36409]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:43.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:42 smithi045 ceph-mon[36409]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:43.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:42 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2661159916' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:43.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:42 smithi038 ceph-mon[31995]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:43.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:42 smithi038 ceph-mon[31995]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:32:43.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:42 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2661159916' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:44.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:43 smithi045 ceph-mon[36409]: pgmap v296: 97 pgs: 97 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-05-21T15:32:44.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:43 smithi045 ceph-mon[36409]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:32:44.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:43 smithi045 ceph-mon[36409]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:32:44.262 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 21 15:32:44 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-b[66443]: 2024-05-21T15:32:44.210+0000 7f2c386f1700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-21T15:32:44.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:43 smithi038 ceph-mon[31995]: pgmap v296: 97 pgs: 97 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-05-21T15:32:44.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:43 smithi038 ceph-mon[31995]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:32:44.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:43 smithi038 ceph-mon[31995]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:32:44.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:43 smithi053 ceph-mon[36496]: pgmap v296: 97 pgs: 97 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-05-21T15:32:44.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:43 smithi053 ceph-mon[36496]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:32:44.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:43 smithi053 ceph-mon[36496]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:32:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.863+0000 7efe373d5700 1 -- 172.21.15.38:0/1369798353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3010b0a0 msgr2=0x7efe3010d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.863+0000 7efe373d5700 1 --2- 172.21.15.38:0/1369798353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3010b0a0 0x7efe3010d5b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efe28009990 tx=0x7efe28009ca0 comp rx=0 tx=0).stop 2024-05-21T15:32:44.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.863+0000 7efe373d5700 1 -- 172.21.15.38:0/1369798353 shutdown_connections 2024-05-21T15:32:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.863+0000 7efe373d5700 1 --2- 172.21.15.38:0/1369798353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3010b0a0 0x7efe3010d5b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.863+0000 7efe373d5700 1 --2- 172.21.15.38:0/1369798353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe30077670 0x7efe30075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.863+0000 7efe373d5700 1 --2- 172.21.15.38:0/1369798353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe30076080 0x7efe30076510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.863+0000 7efe373d5700 1 -- 172.21.15.38:0/1369798353 >> 172.21.15.38:0/1369798353 conn(0x7efe300fe850 msgr2=0x7efe30100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.864+0000 7efe373d5700 1 -- 172.21.15.38:0/1369798353 shutdown_connections 2024-05-21T15:32:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.864+0000 7efe373d5700 1 -- 172.21.15.38:0/1369798353 wait complete. 2024-05-21T15:32:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.864+0000 7efe373d5700 1 Processor -- start 2024-05-21T15:32:44.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.864+0000 7efe373d5700 1 -- start start 2024-05-21T15:32:44.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe373d5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe30076080 0x7efe3019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:44.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe373d5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe30077670 0x7efe3019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:44.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe373d5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3010b0a0 0x7efe301a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:44.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe373d5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efe30110fa0 con 0x7efe3010b0a0 2024-05-21T15:32:44.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe373d5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe30110e20 con 0x7efe30077670 2024-05-21T15:32:44.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe373d5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efe30111120 con 0x7efe30076080 2024-05-21T15:32:44.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe35972700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3010b0a0 0x7efe301a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:44.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe35171700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe30076080 0x7efe3019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:44.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe35972700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3010b0a0 0x7efe301a2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56772/0 (socket says 172.21.15.38:56772) 2024-05-21T15:32:44.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe35972700 1 -- 172.21.15.38:0/3680853463 learned_addr learned my addr 172.21.15.38:0/3680853463 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:44.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe34970700 1 --2- 172.21.15.38:0/3680853463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe30077670 0x7efe3019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:44.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe35972700 1 -- 172.21.15.38:0/3680853463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe30076080 msgr2=0x7efe3019b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:44.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe35972700 1 --2- 172.21.15.38:0/3680853463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe30076080 0x7efe3019b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:44.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe35972700 1 -- 172.21.15.38:0/3680853463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe30077670 msgr2=0x7efe3019bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:44.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe35972700 1 --2- 172.21.15.38:0/3680853463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe30077670 0x7efe3019bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:44.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe35972700 1 -- 172.21.15.38:0/3680853463 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe301a3550 con 0x7efe3010b0a0 2024-05-21T15:32:44.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe35171700 1 --2- 172.21.15.38:0/3680853463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe30076080 0x7efe3019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:32:44.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.865+0000 7efe35972700 1 --2- 172.21.15.38:0/3680853463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3010b0a0 0x7efe301a2ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efe2800b2e0 tx=0x7efe2800bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:44.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.866+0000 7efe2e7fc700 1 -- 172.21.15.38:0/3680853463 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28026070 con 0x7efe3010b0a0 2024-05-21T15:32:44.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.866+0000 7efe2e7fc700 1 -- 172.21.15.38:0/3680853463 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe28003d70 con 0x7efe3010b0a0 2024-05-21T15:32:44.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.866+0000 7efe373d5700 1 -- 172.21.15.38:0/3680853463 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe301a37e0 con 0x7efe3010b0a0 2024-05-21T15:32:44.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.866+0000 7efe2e7fc700 1 -- 172.21.15.38:0/3680853463 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28021410 con 0x7efe3010b0a0 2024-05-21T15:32:44.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.866+0000 7efe373d5700 1 -- 172.21.15.38:0/3680853463 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe301a3c30 con 0x7efe3010b0a0 2024-05-21T15:32:44.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.867+0000 7efe373d5700 1 -- 172.21.15.38:0/3680853463 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe30050170 con 0x7efe3010b0a0 2024-05-21T15:32:44.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.868+0000 7efe2e7fc700 1 -- 172.21.15.38:0/3680853463 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efe28029070 con 0x7efe3010b0a0 2024-05-21T15:32:44.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.869+0000 7efe2e7fc700 1 --2- 172.21.15.38:0/3680853463 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efe180a2cb0 0x7efe180a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:44.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.869+0000 7efe35171700 1 --2- 172.21.15.38:0/3680853463 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efe180a2cb0 0x7efe180a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:44.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.869+0000 7efe2e7fc700 1 -- 172.21.15.38:0/3680853463 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7efe280d04a0 con 0x7efe3010b0a0 2024-05-21T15:32:44.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.869+0000 7efe35171700 1 --2- 172.21.15.38:0/3680853463 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efe180a2cb0 0x7efe180a51c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efe30077370 tx=0x7efe2000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:44.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:44.872+0000 7efe2e7fc700 1 -- 172.21.15.38:0/3680853463 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efe28013070 con 0x7efe3010b0a0 2024-05-21T15:32:45.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:45.164+0000 7efe373d5700 1 -- 172.21.15.38:0/3680853463 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7efe3002dc60 con 0x7efe3010b0a0 2024-05-21T15:32:45.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:45.171+0000 7efe2e7fc700 1 -- 172.21.15.38:0/3680853463 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7efe2809d3e0 con 0x7efe3010b0a0 2024-05-21T15:32:45.171 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-05-21T15:32:45.171 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCdvkxmDIXgCRAArm7xlRxfeF+HxfRsjo5KGA== 2024-05-21T15:32:45.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:45.174+0000 7efe373d5700 1 -- 172.21.15.38:0/3680853463 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efe180a2cb0 msgr2=0x7efe180a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:45.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:45.174+0000 7efe373d5700 1 --2- 172.21.15.38:0/3680853463 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efe180a2cb0 0x7efe180a51c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efe30077370 tx=0x7efe2000b040 comp rx=0 tx=0).stop 2024-05-21T15:32:45.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:45.174+0000 7efe373d5700 1 -- 172.21.15.38:0/3680853463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3010b0a0 msgr2=0x7efe301a2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:45.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:45.174+0000 7efe373d5700 1 --2- 172.21.15.38:0/3680853463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3010b0a0 0x7efe301a2ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efe2800b2e0 tx=0x7efe2800bbb0 comp rx=0 tx=0).stop 2024-05-21T15:32:45.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:45.174+0000 7efe373d5700 1 -- 172.21.15.38:0/3680853463 shutdown_connections 2024-05-21T15:32:45.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:45.174+0000 7efe373d5700 1 --2- 172.21.15.38:0/3680853463 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efe180a2cb0 0x7efe180a51c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:45.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:45.174+0000 7efe373d5700 1 --2- 172.21.15.38:0/3680853463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe3010b0a0 0x7efe301a2ed0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:45.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:45.174+0000 7efe373d5700 1 --2- 172.21.15.38:0/3680853463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe30076080 0x7efe3019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:45.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:45.174+0000 7efe373d5700 1 --2- 172.21.15.38:0/3680853463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe30077670 0x7efe3019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:45.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:45.174+0000 7efe373d5700 1 -- 172.21.15.38:0/3680853463 >> 172.21.15.38:0/3680853463 conn(0x7efe300fe850 msgr2=0x7efe30107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:45.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:45.174+0000 7efe373d5700 1 -- 172.21.15.38:0/3680853463 shutdown_connections 2024-05-21T15:32:45.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:45.174+0000 7efe373d5700 1 -- 172.21.15.38:0/3680853463 wait complete. 2024-05-21T15:32:46.009 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:32:46.009 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-21T15:32:46.009 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-21T15:32:46.052 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-21T15:32:46.052 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-21T15:32:46.052 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph mgr dump --format=json 2024-05-21T15:32:46.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:45 smithi045 ceph-mon[36409]: pgmap v297: 97 pgs: 97 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-05-21T15:32:46.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:45 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3680853463' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-21T15:32:46.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:45 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3680853463' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T15:32:46.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:45 smithi038 ceph-mon[31995]: pgmap v297: 97 pgs: 97 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-05-21T15:32:46.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:45 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3680853463' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-21T15:32:46.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:45 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3680853463' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T15:32:46.343 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:32:46.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:45 smithi053 ceph-mon[36496]: pgmap v297: 97 pgs: 97 active+clean; 620 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-05-21T15:32:46.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:45 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3680853463' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-21T15:32:46.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:45 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3680853463' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T15:32:47.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:47 smithi053 ceph-mon[36496]: pgmap v298: 97 pgs: 97 active+clean; 621 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 9.1 KiB/s wr, 23 op/s 2024-05-21T15:32:47.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:47 smithi045 ceph-mon[36409]: pgmap v298: 97 pgs: 97 active+clean; 621 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 9.1 KiB/s wr, 23 op/s 2024-05-21T15:32:47.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:47 smithi038 ceph-mon[31995]: pgmap v298: 97 pgs: 97 active+clean; 621 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 9.1 KiB/s wr, 23 op/s 2024-05-21T15:32:48.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.438+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1032137159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54103460 msgr2=0x7f7d54103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:48.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.438+0000 7f7d5bf9d700 1 --2- 172.21.15.38:0/1032137159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54103460 0x7f7d54103870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7d500099c0 tx=0x7f7d50009cd0 comp rx=0 tx=0).stop 2024-05-21T15:32:48.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.438+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1032137159 shutdown_connections 2024-05-21T15:32:48.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.438+0000 7f7d5bf9d700 1 --2- 172.21.15.38:0/1032137159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d541086e0 0x7f7d54110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:48.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.438+0000 7f7d5bf9d700 1 --2- 172.21.15.38:0/1032137159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54103460 0x7f7d54103870 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:48.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.438+0000 7f7d5bf9d700 1 --2- 172.21.15.38:0/1032137159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d54107d90 0x7f7d541081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:48.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.438+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1032137159 >> 172.21.15.38:0/1032137159 conn(0x7f7d540fed70 msgr2=0x7f7d541011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:48.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.438+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1032137159 shutdown_connections 2024-05-21T15:32:48.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.438+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1032137159 wait complete. 2024-05-21T15:32:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d5bf9d700 1 Processor -- start 2024-05-21T15:32:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d5bf9d700 1 -- start start 2024-05-21T15:32:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d5bf9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54103460 0x7f7d54199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d5bf9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d54107d90 0x7f7d5419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d5bf9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d541086e0 0x7f7d5419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d5bf9d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7d54113460 con 0x7f7d54103460 2024-05-21T15:32:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d5bf9d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d541132e0 con 0x7f7d541086e0 2024-05-21T15:32:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d5bf9d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7d541135e0 con 0x7f7d54107d90 2024-05-21T15:32:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d59d39700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54103460 0x7f7d54199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d59d39700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54103460 0x7f7d54199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56784/0 (socket says 172.21.15.38:56784) 2024-05-21T15:32:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d59d39700 1 -- 172.21.15.38:0/1712907335 learned_addr learned my addr 172.21.15.38:0/1712907335 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d5a53a700 1 --2- 172.21.15.38:0/1712907335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d541086e0 0x7f7d5419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d59538700 1 --2- 172.21.15.38:0/1712907335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d54107d90 0x7f7d5419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d59d39700 1 -- 172.21.15.38:0/1712907335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d54107d90 msgr2=0x7f7d5419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d59d39700 1 --2- 172.21.15.38:0/1712907335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d54107d90 0x7f7d5419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d59d39700 1 -- 172.21.15.38:0/1712907335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d541086e0 msgr2=0x7f7d5419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d59d39700 1 --2- 172.21.15.38:0/1712907335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d541086e0 0x7f7d5419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.439+0000 7f7d59d39700 1 -- 172.21.15.38:0/1712907335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5419ec60 con 0x7f7d54103460 2024-05-21T15:32:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.440+0000 7f7d59d39700 1 --2- 172.21.15.38:0/1712907335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54103460 0x7f7d54199d90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7d50006010 tx=0x7f7d50009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.440+0000 7f7d46ffd700 1 -- 172.21.15.38:0/1712907335 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50026070 con 0x7f7d54103460 2024-05-21T15:32:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.440+0000 7f7d46ffd700 1 -- 172.21.15.38:0/1712907335 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d50004cd0 con 0x7f7d54103460 2024-05-21T15:32:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.440+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1712907335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5419eef0 con 0x7f7d54103460 2024-05-21T15:32:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.440+0000 7f7d46ffd700 1 -- 172.21.15.38:0/1712907335 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50003ba0 con 0x7f7d54103460 2024-05-21T15:32:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.440+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1712907335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5419f3d0 con 0x7f7d54103460 2024-05-21T15:32:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.441+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1712907335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d54050170 con 0x7f7d54103460 2024-05-21T15:32:48.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.442+0000 7f7d46ffd700 1 -- 172.21.15.38:0/1712907335 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7d50029070 con 0x7f7d54103460 2024-05-21T15:32:48.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.443+0000 7f7d46ffd700 1 --2- 172.21.15.38:0/1712907335 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7d400a2cb0 0x7f7d400a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:48.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.443+0000 7f7d59538700 1 --2- 172.21.15.38:0/1712907335 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7d400a2cb0 0x7f7d400a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:48.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.443+0000 7f7d46ffd700 1 -- 172.21.15.38:0/1712907335 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f7d500cfd80 con 0x7f7d54103460 2024-05-21T15:32:48.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.444+0000 7f7d59538700 1 --2- 172.21.15.38:0/1712907335 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7d400a2cb0 0x7f7d400a51c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7d54106530 tx=0x7f7d4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:48.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.447+0000 7f7d46ffd700 1 -- 172.21.15.38:0/1712907335 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7d50030080 con 0x7f7d54103460 2024-05-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.714+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1712907335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7d5402dc60 con 0x7f7d54103460 2024-05-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.719+0000 7f7d46ffd700 1 -- 172.21.15.38:0/1712907335 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263201 (secure 0 0 0) 0x7f7d50013070 con 0x7f7d54103460 2024-05-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.722+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1712907335 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7d400a2cb0 msgr2=0x7f7d400a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.722+0000 7f7d5bf9d700 1 --2- 172.21.15.38:0/1712907335 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7d400a2cb0 0x7f7d400a51c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7d54106530 tx=0x7f7d4800b410 comp rx=0 tx=0).stop 2024-05-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.722+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1712907335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54103460 msgr2=0x7f7d54199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.722+0000 7f7d5bf9d700 1 --2- 172.21.15.38:0/1712907335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54103460 0x7f7d54199d90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7d50006010 tx=0x7f7d50009f80 comp rx=0 tx=0).stop 2024-05-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.722+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1712907335 shutdown_connections 2024-05-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.722+0000 7f7d5bf9d700 1 --2- 172.21.15.38:0/1712907335 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7d400a2cb0 0x7f7d400a51c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.722+0000 7f7d5bf9d700 1 --2- 172.21.15.38:0/1712907335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d541086e0 0x7f7d5419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.722+0000 7f7d5bf9d700 1 --2- 172.21.15.38:0/1712907335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d54103460 0x7f7d54199d90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.722+0000 7f7d5bf9d700 1 --2- 172.21.15.38:0/1712907335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d54107d90 0x7f7d5419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.722+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1712907335 >> 172.21.15.38:0/1712907335 conn(0x7f7d540fed70 msgr2=0x7f7d54106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.722+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1712907335 shutdown_connections 2024-05-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:48.722+0000 7f7d5bf9d700 1 -- 172.21.15.38:0/1712907335 wait complete. 2024-05-21T15:32:49.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:49 smithi038 ceph-mon[31995]: pgmap v299: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-05-21T15:32:49.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:49 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1712907335' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-21T15:32:49.560 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1295226559},{"type":"v1","addr":"172.21.15.38:6801","nonce":1295226559}]},"active_addr":"172.21.15.38:6801/1295226559","active_change":"2024-05-21T15:25:22.118108+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24118,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24124,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.38:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":4233783744}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":2951450801}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":3399447322}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":4215114230}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":2419744470}]}]} 2024-05-21T15:32:49.567 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-21T15:32:49.567 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-21T15:32:49.567 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T15:32:49.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:49 smithi053 ceph-mon[36496]: pgmap v299: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-05-21T15:32:49.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:49 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1712907335' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-21T15:32:49.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:49 smithi045 ceph-mon[36409]: pgmap v299: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-05-21T15:32:49.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:49 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1712907335' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-21T15:32:49.827 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:32:51.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.560+0000 7fbf4389c700 1 -- 172.21.15.38:0/1702072854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c104340 msgr2=0x7fbf3c104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.560+0000 7fbf4389c700 1 --2- 172.21.15.38:0/1702072854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c104340 0x7fbf3c104750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800b0a0 tx=0x7fbf3800b3b0 comp rx=0 tx=0).stop 2024-05-21T15:32:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.560+0000 7fbf4389c700 1 -- 172.21.15.38:0/1702072854 shutdown_connections 2024-05-21T15:32:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.560+0000 7fbf4389c700 1 --2- 172.21.15.38:0/1702072854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c10d770 0x7fbf3c10fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.560+0000 7fbf4389c700 1 --2- 172.21.15.38:0/1702072854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c104340 0x7fbf3c104750 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.560+0000 7fbf4389c700 1 --2- 172.21.15.38:0/1702072854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf3c104c90 0x7fbf3c10d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.560+0000 7fbf4389c700 1 -- 172.21.15.38:0/1702072854 >> 172.21.15.38:0/1702072854 conn(0x7fbf3c0fdc70 msgr2=0x7fbf3c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:51.562 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:51 smithi038 ceph-mon[31995]: pgmap v300: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.5 KiB/s wr, 22 op/s 2024-05-21T15:32:51.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.560+0000 7fbf4389c700 1 -- 172.21.15.38:0/1702072854 shutdown_connections 2024-05-21T15:32:51.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.560+0000 7fbf4389c700 1 -- 172.21.15.38:0/1702072854 wait complete. 2024-05-21T15:32:51.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.561+0000 7fbf4389c700 1 Processor -- start 2024-05-21T15:32:51.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.561+0000 7fbf4389c700 1 -- start start 2024-05-21T15:32:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.561+0000 7fbf4389c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf3c104340 0x7fbf3c102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.561+0000 7fbf4389c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c104c90 0x7fbf3c102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.561+0000 7fbf4389c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c10d770 0x7fbf3c102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.561+0000 7fbf4389c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c113490 con 0x7fbf3c10d770 2024-05-21T15:32:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.561+0000 7fbf4389c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c113310 con 0x7fbf3c104c90 2024-05-21T15:32:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.561+0000 7fbf4389c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c113610 con 0x7fbf3c104340 2024-05-21T15:32:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.561+0000 7fbf41e39700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c10d770 0x7fbf3c102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.561+0000 7fbf41638700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf3c104340 0x7fbf3c102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.561+0000 7fbf41e39700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c10d770 0x7fbf3c102ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56802/0 (socket says 172.21.15.38:56802) 2024-05-21T15:32:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.561+0000 7fbf41e39700 1 -- 172.21.15.38:0/1433692810 learned_addr learned my addr 172.21.15.38:0/1433692810 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.561+0000 7fbf40e37700 1 --2- 172.21.15.38:0/1433692810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c104c90 0x7fbf3c102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.562+0000 7fbf41e39700 1 -- 172.21.15.38:0/1433692810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf3c104340 msgr2=0x7fbf3c102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:51.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.562+0000 7fbf41e39700 1 --2- 172.21.15.38:0/1433692810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf3c104340 0x7fbf3c102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:51.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.562+0000 7fbf41e39700 1 -- 172.21.15.38:0/1433692810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c104c90 msgr2=0x7fbf3c102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:51.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.562+0000 7fbf41e39700 1 --2- 172.21.15.38:0/1433692810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c104c90 0x7fbf3c102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:51.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.562+0000 7fbf41e39700 1 -- 172.21.15.38:0/1433692810 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c1a8180 con 0x7fbf3c10d770 2024-05-21T15:32:51.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.562+0000 7fbf40e37700 1 --2- 172.21.15.38:0/1433692810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c104c90 0x7fbf3c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:32:51.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.562+0000 7fbf41e39700 1 --2- 172.21.15.38:0/1433692810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c10d770 0x7fbf3c102ab0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400d980 tx=0x7fbf3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:51.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.562+0000 7fbf327fc700 1 -- 172.21.15.38:0/1433692810 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34017070 con 0x7fbf3c10d770 2024-05-21T15:32:51.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.562+0000 7fbf327fc700 1 -- 172.21.15.38:0/1433692810 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf34005020 con 0x7fbf3c10d770 2024-05-21T15:32:51.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.562+0000 7fbf4389c700 1 -- 172.21.15.38:0/1433692810 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3c1a8470 con 0x7fbf3c10d770 2024-05-21T15:32:51.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.562+0000 7fbf327fc700 1 -- 172.21.15.38:0/1433692810 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34009b50 con 0x7fbf3c10d770 2024-05-21T15:32:51.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.562+0000 7fbf4389c700 1 -- 172.21.15.38:0/1433692810 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3c1a8960 con 0x7fbf3c10d770 2024-05-21T15:32:51.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.563+0000 7fbf4389c700 1 -- 172.21.15.38:0/1433692810 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf3c050170 con 0x7fbf3c10d770 2024-05-21T15:32:51.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.564+0000 7fbf327fc700 1 -- 172.21.15.38:0/1433692810 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbf34009cf0 con 0x7fbf3c10d770 2024-05-21T15:32:51.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.565+0000 7fbf327fc700 1 --2- 172.21.15.38:0/1433692810 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf280a2cb0 0x7fbf280a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:51.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.565+0000 7fbf41638700 1 --2- 172.21.15.38:0/1433692810 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf280a2cb0 0x7fbf280a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:51.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.565+0000 7fbf327fc700 1 -- 172.21.15.38:0/1433692810 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fbf340cf330 con 0x7fbf3c10d770 2024-05-21T15:32:51.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.566+0000 7fbf41638700 1 --2- 172.21.15.38:0/1433692810 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf280a2cb0 0x7fbf280a51c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009fd0 tx=0x7fbf3800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:51.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.569+0000 7fbf327fc700 1 -- 172.21.15.38:0/1433692810 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbf34020020 con 0x7fbf3c10d770 2024-05-21T15:32:51.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:51 smithi053 ceph-mon[36496]: pgmap v300: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.5 KiB/s wr, 22 op/s 2024-05-21T15:32:51.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:51 smithi045 ceph-mon[36409]: pgmap v300: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.5 KiB/s wr, 22 op/s 2024-05-21T15:32:51.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.775+0000 7fbf4389c700 1 -- 172.21.15.38:0/1433692810 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf3c103560 con 0x7fbf3c10d770 2024-05-21T15:32:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.777+0000 7fbf327fc700 1 -- 172.21.15.38:0/1433692810 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22112 (secure 0 0 0) 0x7fbf3409c1e0 con 0x7fbf3c10d770 2024-05-21T15:32:51.778 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:51.778 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":86,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:32:34.657863+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:32:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.779+0000 7fbf4389c700 1 -- 172.21.15.38:0/1433692810 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf280a2cb0 msgr2=0x7fbf280a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.779+0000 7fbf4389c700 1 --2- 172.21.15.38:0/1433692810 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf280a2cb0 0x7fbf280a51c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009fd0 tx=0x7fbf3800b7b0 comp rx=0 tx=0).stop 2024-05-21T15:32:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.779+0000 7fbf4389c700 1 -- 172.21.15.38:0/1433692810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c10d770 msgr2=0x7fbf3c102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.779+0000 7fbf4389c700 1 --2- 172.21.15.38:0/1433692810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c10d770 0x7fbf3c102ab0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400d980 tx=0x7fbf3400dc90 comp rx=0 tx=0).stop 2024-05-21T15:32:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.779+0000 7fbf4389c700 1 -- 172.21.15.38:0/1433692810 shutdown_connections 2024-05-21T15:32:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.779+0000 7fbf4389c700 1 --2- 172.21.15.38:0/1433692810 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf280a2cb0 0x7fbf280a51c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.779+0000 7fbf4389c700 1 --2- 172.21.15.38:0/1433692810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c10d770 0x7fbf3c102ab0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.779+0000 7fbf4389c700 1 --2- 172.21.15.38:0/1433692810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf3c104340 0x7fbf3c102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.779+0000 7fbf4389c700 1 --2- 172.21.15.38:0/1433692810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c104c90 0x7fbf3c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.779+0000 7fbf4389c700 1 -- 172.21.15.38:0/1433692810 >> 172.21.15.38:0/1433692810 conn(0x7fbf3c0fdc70 msgr2=0x7fbf3c0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.779+0000 7fbf4389c700 1 -- 172.21.15.38:0/1433692810 shutdown_connections 2024-05-21T15:32:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:51.779+0000 7fbf4389c700 1 -- 172.21.15.38:0/1433692810 wait complete. 2024-05-21T15:32:52.520 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:52 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1433692810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:52.550 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-21T15:32:52.550 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T15:32:52.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:52 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1433692810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:52.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:52 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1433692810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:52.801 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:32:53.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:53 smithi038 ceph-mon[31995]: pgmap v301: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.3 KiB/s wr, 19 op/s 2024-05-21T15:32:53.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:53 smithi053 ceph-mon[36496]: pgmap v301: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.3 KiB/s wr, 19 op/s 2024-05-21T15:32:53.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:53 smithi045 ceph-mon[36409]: pgmap v301: 97 pgs: 97 active+clean; 626 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.3 KiB/s wr, 19 op/s 2024-05-21T15:32:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.521+0000 7fd36237b700 1 -- 172.21.15.38:0/1305799284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd35c1075e0 msgr2=0x7fd35c10ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.521+0000 7fd36237b700 1 --2- 172.21.15.38:0/1305799284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd35c1075e0 0x7fd35c10ef40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd35400b0d0 tx=0x7fd35400b3e0 comp rx=0 tx=0).stop 2024-05-21T15:32:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.521+0000 7fd36237b700 1 -- 172.21.15.38:0/1305799284 shutdown_connections 2024-05-21T15:32:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.521+0000 7fd36237b700 1 --2- 172.21.15.38:0/1305799284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd35c1075e0 0x7fd35c10ef40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:54.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.521+0000 7fd36237b700 1 --2- 172.21.15.38:0/1305799284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c102360 0x7fd35c102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:54.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.521+0000 7fd36237b700 1 --2- 172.21.15.38:0/1305799284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd35c106c90 0x7fd35c1070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:54.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.521+0000 7fd36237b700 1 -- 172.21.15.38:0/1305799284 >> 172.21.15.38:0/1305799284 conn(0x7fd35c0fddb0 msgr2=0x7fd35c1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:54.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.521+0000 7fd36237b700 1 -- 172.21.15.38:0/1305799284 shutdown_connections 2024-05-21T15:32:54.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.521+0000 7fd36237b700 1 -- 172.21.15.38:0/1305799284 wait complete. 2024-05-21T15:32:54.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.522+0000 7fd36237b700 1 Processor -- start 2024-05-21T15:32:54.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.522+0000 7fd36237b700 1 -- start start 2024-05-21T15:32:54.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.522+0000 7fd36237b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd35c102360 0x7fd35c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:54.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.522+0000 7fd36237b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c106c90 0x7fd35c19b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:54.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.522+0000 7fd36237b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd35c1075e0 0x7fd35c19b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:54.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.522+0000 7fd36237b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c112360 con 0x7fd35c102360 2024-05-21T15:32:54.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.522+0000 7fd36237b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1121e0 con 0x7fd35c1075e0 2024-05-21T15:32:54.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.522+0000 7fd36237b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1124e0 con 0x7fd35c106c90 2024-05-21T15:32:54.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.522+0000 7fd35bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd35c102360 0x7fd35c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:54.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.522+0000 7fd35bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd35c102360 0x7fd35c19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56832/0 (socket says 172.21.15.38:56832) 2024-05-21T15:32:54.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.522+0000 7fd35bfff700 1 -- 172.21.15.38:0/328158004 learned_addr learned my addr 172.21.15.38:0/328158004 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:32:54.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.523+0000 7fd353fff700 1 --2- 172.21.15.38:0/328158004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c106c90 0x7fd35c19b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:54.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.523+0000 7fd360918700 1 --2- 172.21.15.38:0/328158004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd35c1075e0 0x7fd35c19b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:54.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.523+0000 7fd35bfff700 1 -- 172.21.15.38:0/328158004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c106c90 msgr2=0x7fd35c19b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:54.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.523+0000 7fd35bfff700 1 --2- 172.21.15.38:0/328158004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c106c90 0x7fd35c19b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:54.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.523+0000 7fd35bfff700 1 -- 172.21.15.38:0/328158004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd35c1075e0 msgr2=0x7fd35c19b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:54.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.523+0000 7fd35bfff700 1 --2- 172.21.15.38:0/328158004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd35c1075e0 0x7fd35c19b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:54.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.523+0000 7fd35bfff700 1 -- 172.21.15.38:0/328158004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c1a5ed0 con 0x7fd35c102360 2024-05-21T15:32:54.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.523+0000 7fd35bfff700 1 --2- 172.21.15.38:0/328158004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd35c102360 0x7fd35c19ab80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00d980 tx=0x7fd34c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:54.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.523+0000 7fd359ffb700 1 -- 172.21.15.38:0/328158004 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c016070 con 0x7fd35c102360 2024-05-21T15:32:54.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.523+0000 7fd36237b700 1 -- 172.21.15.38:0/328158004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35c1a61c0 con 0x7fd35c102360 2024-05-21T15:32:54.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.523+0000 7fd359ffb700 1 -- 172.21.15.38:0/328158004 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd34c004d10 con 0x7fd35c102360 2024-05-21T15:32:54.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.523+0000 7fd359ffb700 1 -- 172.21.15.38:0/328158004 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0055f0 con 0x7fd35c102360 2024-05-21T15:32:54.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.523+0000 7fd36237b700 1 -- 172.21.15.38:0/328158004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35c1a6700 con 0x7fd35c102360 2024-05-21T15:32:54.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.524+0000 7fd36237b700 1 -- 172.21.15.38:0/328158004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd35c050170 con 0x7fd35c102360 2024-05-21T15:32:54.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.527+0000 7fd359ffb700 1 -- 172.21.15.38:0/328158004 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd34c004750 con 0x7fd35c102360 2024-05-21T15:32:54.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.527+0000 7fd359ffb700 1 --2- 172.21.15.38:0/328158004 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3480a2cb0 0x7fd3480a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:32:54.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.528+0000 7fd353fff700 1 --2- 172.21.15.38:0/328158004 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3480a2cb0 0x7fd3480a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:32:54.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.528+0000 7fd359ffb700 1 -- 172.21.15.38:0/328158004 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd34c0cf360 con 0x7fd35c102360 2024-05-21T15:32:54.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.528+0000 7fd353fff700 1 --2- 172.21.15.38:0/328158004 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3480a2cb0 0x7fd3480a51c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd35c1018c0 tx=0x7fd34400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:32:54.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.531+0000 7fd359ffb700 1 -- 172.21.15.38:0/328158004 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd34c01f020 con 0x7fd35c102360 2024-05-21T15:32:54.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.738+0000 7fd36237b700 1 -- 172.21.15.38:0/328158004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd35c02dc60 con 0x7fd35c102360 2024-05-21T15:32:54.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.740+0000 7fd359ffb700 1 -- 172.21.15.38:0/328158004 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22112 (secure 0 0 0) 0x7fd34c09c310 con 0x7fd35c102360 2024-05-21T15:32:54.740 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:32:54.740 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":86,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:32:34.657863+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:32:54.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.742+0000 7fd36237b700 1 -- 172.21.15.38:0/328158004 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3480a2cb0 msgr2=0x7fd3480a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:54.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.742+0000 7fd36237b700 1 --2- 172.21.15.38:0/328158004 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3480a2cb0 0x7fd3480a51c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd35c1018c0 tx=0x7fd34400a300 comp rx=0 tx=0).stop 2024-05-21T15:32:54.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.742+0000 7fd36237b700 1 -- 172.21.15.38:0/328158004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd35c102360 msgr2=0x7fd35c19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:32:54.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.742+0000 7fd36237b700 1 --2- 172.21.15.38:0/328158004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd35c102360 0x7fd35c19ab80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00d980 tx=0x7fd34c00dc90 comp rx=0 tx=0).stop 2024-05-21T15:32:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.742+0000 7fd36237b700 1 -- 172.21.15.38:0/328158004 shutdown_connections 2024-05-21T15:32:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.742+0000 7fd36237b700 1 --2- 172.21.15.38:0/328158004 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3480a2cb0 0x7fd3480a51c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.742+0000 7fd36237b700 1 --2- 172.21.15.38:0/328158004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd35c1075e0 0x7fd35c19b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.742+0000 7fd36237b700 1 --2- 172.21.15.38:0/328158004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd35c102360 0x7fd35c19ab80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.742+0000 7fd36237b700 1 --2- 172.21.15.38:0/328158004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c106c90 0x7fd35c19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:32:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.742+0000 7fd36237b700 1 -- 172.21.15.38:0/328158004 >> 172.21.15.38:0/328158004 conn(0x7fd35c0fddb0 msgr2=0x7fd35c1063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:32:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.742+0000 7fd36237b700 1 -- 172.21.15.38:0/328158004 shutdown_connections 2024-05-21T15:32:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:32:54.742+0000 7fd36237b700 1 -- 172.21.15.38:0/328158004 wait complete. 2024-05-21T15:32:55.405 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:55 smithi038 ceph-mon[31995]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-05-21T15:32:55.405 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:55 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/328158004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:55.437 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-21T15:32:55.438 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-21T15:32:55.438 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-21T15:32:55.439 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-21T15:32:55.439 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-21T15:32:55.439 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-21T15:32:55.440 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-21T15:32:55.440 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-21T15:32:55.441 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-21T15:32:55.441 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-21T15:32:55.441 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-21T15:32:55.442 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-21T15:32:55.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:55 smithi053 ceph-mon[36496]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-05-21T15:32:55.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:55 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/328158004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:55.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:55 smithi045 ceph-mon[36409]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-05-21T15:32:55.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:55 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/328158004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:32:56.226 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:32:56.234 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:32:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:32:56.237 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:32:56.246 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:32:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:32:57.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:57 smithi038 ceph-mon[31995]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-21T15:32:57.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:57 smithi053 ceph-mon[36496]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-21T15:32:57.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:57 smithi045 ceph-mon[36409]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-21T15:32:59.090 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:32:59.091 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:32:59.092 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:32:59.095 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:32:59.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:32:59 smithi038 ceph-mon[31995]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-05-21T15:32:59.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:32:59 smithi053 ceph-mon[36496]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-05-21T15:32:59.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:32:59 smithi045 ceph-mon[36409]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-05-21T15:33:01.411 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:01 smithi038 ceph-mon[31995]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-21T15:33:01.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:01 smithi053 ceph-mon[36496]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-21T15:33:01.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:01 smithi045 ceph-mon[36409]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-21T15:33:02.340 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:03.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.011+0000 7fc93f80c700 1 -- 172.21.15.38:0/3105037573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9381086f0 msgr2=0x7fc938110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:03.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.011+0000 7fc93f80c700 1 --2- 172.21.15.38:0/3105037573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9381086f0 0x7fc938110050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc93000b0d0 tx=0x7fc93000b3e0 comp rx=0 tx=0).stop 2024-05-21T15:33:03.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.011+0000 7fc93f80c700 1 -- 172.21.15.38:0/3105037573 shutdown_connections 2024-05-21T15:33:03.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.011+0000 7fc93f80c700 1 --2- 172.21.15.38:0/3105037573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9381086f0 0x7fc938110050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:03.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.011+0000 7fc93f80c700 1 --2- 172.21.15.38:0/3105037573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc938103470 0x7fc938103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:03.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.011+0000 7fc93f80c700 1 --2- 172.21.15.38:0/3105037573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc938107da0 0x7fc9381081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:03.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.011+0000 7fc93f80c700 1 -- 172.21.15.38:0/3105037573 >> 172.21.15.38:0/3105037573 conn(0x7fc9380feec0 msgr2=0x7fc9381012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:03.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.012+0000 7fc93f80c700 1 -- 172.21.15.38:0/3105037573 shutdown_connections 2024-05-21T15:33:03.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.012+0000 7fc93f80c700 1 -- 172.21.15.38:0/3105037573 wait complete. 2024-05-21T15:33:03.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.012+0000 7fc93f80c700 1 Processor -- start 2024-05-21T15:33:03.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.012+0000 7fc93f80c700 1 -- start start 2024-05-21T15:33:03.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93f80c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc938103470 0x7fc938199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:03.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93f80c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc938107da0 0x7fc93819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:03.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93f80c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9381086f0 0x7fc93819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:03.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93f80c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc938113470 con 0x7fc938107da0 2024-05-21T15:33:03.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93f80c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc9381132f0 con 0x7fc938103470 2024-05-21T15:33:03.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93f80c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc9381135f0 con 0x7fc9381086f0 2024-05-21T15:33:03.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93cda7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc938107da0 0x7fc93819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:03.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93cda7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc938107da0 0x7fc93819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38684/0 (socket says 172.21.15.38:38684) 2024-05-21T15:33:03.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93cda7700 1 -- 172.21.15.38:0/1566603137 learned_addr learned my addr 172.21.15.38:0/1566603137 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:03.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93dda9700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9381086f0 0x7fc93819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:03.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93d5a8700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc938103470 0x7fc938199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:03.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93dda9700 1 -- 172.21.15.38:0/1566603137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc938103470 msgr2=0x7fc938199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:03.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93dda9700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc938103470 0x7fc938199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:03.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93dda9700 1 -- 172.21.15.38:0/1566603137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc938107da0 msgr2=0x7fc93819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:03.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93dda9700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc938107da0 0x7fc93819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:03.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93dda9700 1 -- 172.21.15.38:0/1566603137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93819eb80 con 0x7fc9381086f0 2024-05-21T15:33:03.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93cda7700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc938107da0 0x7fc93819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:33:03.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93d5a8700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc938103470 0x7fc938199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:33:03.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.013+0000 7fc93dda9700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9381086f0 0x7fc93819e590 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc93000b770 tx=0x7fc930007a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:03.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.014+0000 7fc92e7fc700 1 -- 172.21.15.38:0/1566603137 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc930021070 con 0x7fc9381086f0 2024-05-21T15:33:03.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.014+0000 7fc92e7fc700 1 -- 172.21.15.38:0/1566603137 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc93001db30 con 0x7fc9381086f0 2024-05-21T15:33:03.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.014+0000 7fc92e7fc700 1 -- 172.21.15.38:0/1566603137 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93001bac0 con 0x7fc9381086f0 2024-05-21T15:33:03.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.014+0000 7fc93f80c700 1 -- 172.21.15.38:0/1566603137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93819ee10 con 0x7fc9381086f0 2024-05-21T15:33:03.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.014+0000 7fc93f80c700 1 -- 172.21.15.38:0/1566603137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93819f300 con 0x7fc9381086f0 2024-05-21T15:33:03.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.014+0000 7fc93f80c700 1 -- 172.21.15.38:0/1566603137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc938062dd0 con 0x7fc9381086f0 2024-05-21T15:33:03.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.016+0000 7fc92e7fc700 1 -- 172.21.15.38:0/1566603137 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc930024070 con 0x7fc9381086f0 2024-05-21T15:33:03.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.017+0000 7fc92e7fc700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc9240a2cb0 0x7fc9240a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:03.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.017+0000 7fc93d5a8700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc9240a2cb0 0x7fc9240a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:03.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.017+0000 7fc92e7fc700 1 -- 172.21.15.38:0/1566603137 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc93000f020 con 0x7fc9381086f0 2024-05-21T15:33:03.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.017+0000 7fc92e7fc700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766] conn(0x7fc9240ac4e0 0x7fc9240ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:03.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.017+0000 7fc92e7fc700 1 -- 172.21.15.38:0/1566603137 --> [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc9300020b0 con 0x7fc9240ac4e0 2024-05-21T15:33:03.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.017+0000 7fc92e7fc700 1 -- 172.21.15.38:0/1566603137 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fc9300d0ef0 con 0x7fc9381086f0 2024-05-21T15:33:03.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.017+0000 7fc93d5a8700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc9240a2cb0 0x7fc9240a51c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc934009cb0 tx=0x7fc934009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:03.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.017+0000 7fc93cda7700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766] conn(0x7fc9240ac4e0 0x7fc9240ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:03.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.018+0000 7fc93cda7700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766] conn(0x7fc9240ac4e0 0x7fc9240ae930 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:03.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.019+0000 7fc92e7fc700 1 -- 172.21.15.38:0/1566603137 <== osd.10 v2:172.21.15.45:6824/3908617766 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc9300020b0 con 0x7fc9240ac4e0 2024-05-21T15:33:03.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.036+0000 7fc93f80c700 1 -- 172.21.15.38:0/1566603137 --> [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc938063250 con 0x7fc9240ac4e0 2024-05-21T15:33:03.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.036+0000 7fc92e7fc700 1 -- 172.21.15.38:0/1566603137 <== osd.10 v2:172.21.15.45:6824/3908617766 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc938063250 con 0x7fc9240ac4e0 2024-05-21T15:33:03.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.036+0000 7fc93f80c700 1 -- 172.21.15.38:0/1566603137 >> [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766] conn(0x7fc9240ac4e0 msgr2=0x7fc9240ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:03.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.036+0000 7fc93f80c700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766] conn(0x7fc9240ac4e0 0x7fc9240ae930 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:03.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.037+0000 7fc93f80c700 1 -- 172.21.15.38:0/1566603137 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc9240a2cb0 msgr2=0x7fc9240a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:03.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.037+0000 7fc93f80c700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc9240a2cb0 0x7fc9240a51c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc934009cb0 tx=0x7fc934009260 comp rx=0 tx=0).stop 2024-05-21T15:33:03.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.037+0000 7fc93f80c700 1 -- 172.21.15.38:0/1566603137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9381086f0 msgr2=0x7fc93819e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:03.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.037+0000 7fc93f80c700 1 --2- 172.21.15.38:0/1566603137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9381086f0 0x7fc93819e590 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc93000b770 tx=0x7fc930007a20 comp rx=0 tx=0).stop 2024-05-21T15:33:03.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.037+0000 7fc93dda9700 1 -- 172.21.15.38:0/1566603137 reap_dead start 2024-05-21T15:33:03.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.037+0000 7fc93f80c700 1 -- 172.21.15.38:0/1566603137 shutdown_connections 2024-05-21T15:33:03.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.037+0000 7fc93f80c700 1 -- 172.21.15.38:0/1566603137 >> 172.21.15.38:0/1566603137 conn(0x7fc9380feec0 msgr2=0x7fc938107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:03.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.037+0000 7fc93f80c700 1 -- 172.21.15.38:0/1566603137 shutdown_connections 2024-05-21T15:33:03.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:03.037+0000 7fc93f80c700 1 -- 172.21.15.38:0/1566603137 wait complete. 2024-05-21T15:33:03.525 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:03 smithi038 ceph-mon[31995]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-21T15:33:03.563 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:03 smithi045 ceph-mon[36409]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-21T15:33:03.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:03 smithi053 ceph-mon[36496]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-21T15:33:04.872 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:05.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:05 smithi053 ceph-mon[36496]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-21T15:33:05.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:05 smithi045 ceph-mon[36409]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-21T15:33:05.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:05 smithi038 ceph-mon[31995]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-21T15:33:07.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.060+0000 7f6be3351700 1 -- 172.21.15.38:0/3227910887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bdc101610 msgr2=0x7f6bdc101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.060+0000 7f6be3351700 1 --2- 172.21.15.38:0/3227910887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bdc101610 0x7f6bdc101a20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009990 tx=0x7f6bcc009ca0 comp rx=0 tx=0).stop 2024-05-21T15:33:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.060+0000 7f6be3351700 1 -- 172.21.15.38:0/3227910887 shutdown_connections 2024-05-21T15:33:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.060+0000 7f6be3351700 1 --2- 172.21.15.38:0/3227910887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bdc101f60 0x7f6bdc10fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.060+0000 7f6be3351700 1 --2- 172.21.15.38:0/3227910887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bdc108180 0x7f6bdc108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.060+0000 7f6be3351700 1 --2- 172.21.15.38:0/3227910887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bdc101610 0x7f6bdc101a20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.060+0000 7f6be3351700 1 -- 172.21.15.38:0/3227910887 >> 172.21.15.38:0/3227910887 conn(0x7f6bdc0fd3c0 msgr2=0x7f6bdc0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.060+0000 7f6be3351700 1 -- 172.21.15.38:0/3227910887 shutdown_connections 2024-05-21T15:33:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.060+0000 7f6be3351700 1 -- 172.21.15.38:0/3227910887 wait complete. 2024-05-21T15:33:07.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.061+0000 7f6be3351700 1 Processor -- start 2024-05-21T15:33:07.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.061+0000 7f6be3351700 1 -- start start 2024-05-21T15:33:07.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.061+0000 7f6be3351700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bdc101610 0x7f6bdc199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:07.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.061+0000 7f6be3351700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bdc101f60 0x7f6bdc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:07.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.061+0000 7f6be3351700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bdc108180 0x7f6bdc19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:07.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.061+0000 7f6be3351700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc113570 con 0x7f6bdc108180 2024-05-21T15:33:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.061+0000 7f6be3351700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc1133f0 con 0x7f6bdc101610 2024-05-21T15:33:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.061+0000 7f6be3351700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc1136f0 con 0x7f6bdc101f60 2024-05-21T15:33:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.061+0000 7f6be18ee700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bdc108180 0x7f6bdc19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.061+0000 7f6be18ee700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bdc108180 0x7f6bdc19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47158/0 (socket says 172.21.15.38:47158) 2024-05-21T15:33:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.061+0000 7f6be18ee700 1 -- 172.21.15.38:0/2751684670 learned_addr learned my addr 172.21.15.38:0/2751684670 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.061+0000 7f6be08ec700 1 --2- 172.21.15.38:0/2751684670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bdc101f60 0x7f6bdc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.061+0000 7f6be10ed700 1 --2- 172.21.15.38:0/2751684670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bdc101610 0x7f6bdc199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.062+0000 7f6be18ee700 1 -- 172.21.15.38:0/2751684670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bdc101f60 msgr2=0x7f6bdc19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.062+0000 7f6be18ee700 1 --2- 172.21.15.38:0/2751684670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bdc101f60 0x7f6bdc19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.062+0000 7f6be18ee700 1 -- 172.21.15.38:0/2751684670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bdc101610 msgr2=0x7f6bdc199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.062+0000 7f6be18ee700 1 --2- 172.21.15.38:0/2751684670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bdc101610 0x7f6bdc199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.062+0000 7f6be18ee700 1 -- 172.21.15.38:0/2751684670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bdc19ed00 con 0x7f6bdc108180 2024-05-21T15:33:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.062+0000 7f6be18ee700 1 --2- 172.21.15.38:0/2751684670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bdc108180 0x7f6bdc19e660 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400ada0 tx=0x7f6bd400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.062+0000 7f6bd27fc700 1 -- 172.21.15.38:0/2751684670 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4017020 con 0x7f6bdc108180 2024-05-21T15:33:07.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.062+0000 7f6bd27fc700 1 -- 172.21.15.38:0/2751684670 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bd400cdb0 con 0x7f6bdc108180 2024-05-21T15:33:07.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.062+0000 7f6be3351700 1 -- 172.21.15.38:0/2751684670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bdc19eff0 con 0x7f6bdc108180 2024-05-21T15:33:07.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.062+0000 7f6bd27fc700 1 -- 172.21.15.38:0/2751684670 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd40053e0 con 0x7f6bdc108180 2024-05-21T15:33:07.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.062+0000 7f6be3351700 1 -- 172.21.15.38:0/2751684670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bdc1a6910 con 0x7f6bdc108180 2024-05-21T15:33:07.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.063+0000 7f6be3351700 1 -- 172.21.15.38:0/2751684670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6bdc062d50 con 0x7f6bdc108180 2024-05-21T15:33:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.065+0000 7f6bd27fc700 1 -- 172.21.15.38:0/2751684670 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6bd400c8d0 con 0x7f6bdc108180 2024-05-21T15:33:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.065+0000 7f6bd27fc700 1 --2- 172.21.15.38:0/2751684670 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6bc80a2d30 0x7f6bc80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.066+0000 7f6be10ed700 1 --2- 172.21.15.38:0/2751684670 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6bc80a2d30 0x7f6bc80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.066+0000 7f6bd27fc700 1 -- 172.21.15.38:0/2751684670 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6bd40cf0b0 con 0x7f6bdc108180 2024-05-21T15:33:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.066+0000 7f6bd27fc700 1 --2- 172.21.15.38:0/2751684670 >> [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649] conn(0x7f6bc80ac560 0x7f6bc80ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.066+0000 7f6bd27fc700 1 -- 172.21.15.38:0/2751684670 --> [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bd401ee50 con 0x7f6bc80ac560 2024-05-21T15:33:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.066+0000 7f6bd27fc700 1 -- 172.21.15.38:0/2751684670 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f6bd40cf590 con 0x7f6bdc108180 2024-05-21T15:33:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.066+0000 7f6be08ec700 1 --2- 172.21.15.38:0/2751684670 >> [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649] conn(0x7f6bc80ac560 0x7f6bc80ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:07.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.066+0000 7f6be10ed700 1 --2- 172.21.15.38:0/2751684670 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6bc80a2d30 0x7f6bc80a5240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc06b930 tx=0x7f6bd800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:07.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.066+0000 7f6be08ec700 1 --2- 172.21.15.38:0/2751684670 >> [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649] conn(0x7f6bc80ac560 0x7f6bc80ae9b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:07.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.067+0000 7f6bd27fc700 1 -- 172.21.15.38:0/2751684670 <== osd.9 v2:172.21.15.38:6826/2989402649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bd401ee50 con 0x7f6bc80ac560 2024-05-21T15:33:07.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.084+0000 7f6be3351700 1 -- 172.21.15.38:0/2751684670 --> [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6bdc063150 con 0x7f6bc80ac560 2024-05-21T15:33:07.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.084+0000 7f6bd27fc700 1 -- 172.21.15.38:0/2751684670 <== osd.9 v2:172.21.15.38:6826/2989402649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6bdc063150 con 0x7f6bc80ac560 2024-05-21T15:33:07.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.084+0000 7f6be3351700 1 -- 172.21.15.38:0/2751684670 >> [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649] conn(0x7f6bc80ac560 msgr2=0x7f6bc80ae9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:07.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.084+0000 7f6be3351700 1 --2- 172.21.15.38:0/2751684670 >> [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649] conn(0x7f6bc80ac560 0x7f6bc80ae9b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:07.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.084+0000 7f6be3351700 1 -- 172.21.15.38:0/2751684670 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6bc80a2d30 msgr2=0x7f6bc80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:07.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.084+0000 7f6be3351700 1 --2- 172.21.15.38:0/2751684670 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6bc80a2d30 0x7f6bc80a5240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc06b930 tx=0x7f6bd800b410 comp rx=0 tx=0).stop 2024-05-21T15:33:07.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.084+0000 7f6be3351700 1 -- 172.21.15.38:0/2751684670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bdc108180 msgr2=0x7f6bdc19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:07.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.084+0000 7f6be3351700 1 --2- 172.21.15.38:0/2751684670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6bdc108180 0x7f6bdc19e660 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400ada0 tx=0x7f6bd400c330 comp rx=0 tx=0).stop 2024-05-21T15:33:07.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.084+0000 7f6be18ee700 1 -- 172.21.15.38:0/2751684670 reap_dead start 2024-05-21T15:33:07.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.084+0000 7f6be3351700 1 -- 172.21.15.38:0/2751684670 shutdown_connections 2024-05-21T15:33:07.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.084+0000 7f6be3351700 1 -- 172.21.15.38:0/2751684670 >> 172.21.15.38:0/2751684670 conn(0x7f6bdc0fd3c0 msgr2=0x7f6bdc106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:07.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.085+0000 7f6be3351700 1 -- 172.21.15.38:0/2751684670 shutdown_connections 2024-05-21T15:33:07.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:07.085+0000 7f6be3351700 1 -- 172.21.15.38:0/2751684670 wait complete. 2024-05-21T15:33:07.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:07 smithi053 ceph-mon[36496]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:33:07.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:07 smithi045 ceph-mon[36409]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:33:07.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:07 smithi038 ceph-mon[31995]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:33:08.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.984+0000 7fdaadcde700 1 -- 172.21.15.38:0/3295046030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa81086d0 msgr2=0x7fdaa8110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:08.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.984+0000 7fdaadcde700 1 --2- 172.21.15.38:0/3295046030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa81086d0 0x7fdaa8110030 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000b0d0 tx=0x7fdaa000b3e0 comp rx=0 tx=0).stop 2024-05-21T15:33:08.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.984+0000 7fdaadcde700 1 -- 172.21.15.38:0/3295046030 shutdown_connections 2024-05-21T15:33:08.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.984+0000 7fdaadcde700 1 --2- 172.21.15.38:0/3295046030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa81086d0 0x7fdaa8110030 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:08.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.984+0000 7fdaadcde700 1 --2- 172.21.15.38:0/3295046030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa8103450 0x7fdaa8103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:08.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.984+0000 7fdaadcde700 1 --2- 172.21.15.38:0/3295046030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdaa8107d80 0x7fdaa8108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:08.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.984+0000 7fdaadcde700 1 -- 172.21.15.38:0/3295046030 >> 172.21.15.38:0/3295046030 conn(0x7fdaa80fed20 msgr2=0x7fdaa8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:08.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.984+0000 7fdaadcde700 1 -- 172.21.15.38:0/3295046030 shutdown_connections 2024-05-21T15:33:08.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.984+0000 7fdaadcde700 1 -- 172.21.15.38:0/3295046030 wait complete. 2024-05-21T15:33:08.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.985+0000 7fdaadcde700 1 Processor -- start 2024-05-21T15:33:08.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.985+0000 7fdaadcde700 1 -- start start 2024-05-21T15:33:08.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.985+0000 7fdaadcde700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa8103450 0x7fdaa8199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:08.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.985+0000 7fdaadcde700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa8107d80 0x7fdaa819a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:08.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.985+0000 7fdaadcde700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdaa81086d0 0x7fdaa819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:08.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.985+0000 7fdaadcde700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81134d0 con 0x7fdaa8103450 2024-05-21T15:33:08.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.985+0000 7fdaadcde700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8113350 con 0x7fdaa8107d80 2024-05-21T15:33:08.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.985+0000 7fdaadcde700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8113650 con 0x7fdaa81086d0 2024-05-21T15:33:08.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.985+0000 7fdaa77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa8103450 0x7fdaa8199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:08.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.985+0000 7fdaa77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa8103450 0x7fdaa8199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47178/0 (socket says 172.21.15.38:47178) 2024-05-21T15:33:08.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.985+0000 7fdaa77fe700 1 -- 172.21.15.38:0/2363891042 learned_addr learned my addr 172.21.15.38:0/2363891042 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:08.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaa6ffd700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa8107d80 0x7fdaa819a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:08.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaa7fff700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdaa81086d0 0x7fdaa819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:08.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaa77fe700 1 -- 172.21.15.38:0/2363891042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdaa81086d0 msgr2=0x7fdaa819e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:08.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaa77fe700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdaa81086d0 0x7fdaa819e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:08.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaa77fe700 1 -- 172.21.15.38:0/2363891042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa8107d80 msgr2=0x7fdaa819a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:08.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaa77fe700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa8107d80 0x7fdaa819a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:08.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaa77fe700 1 -- 172.21.15.38:0/2363891042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa819ebf0 con 0x7fdaa8103450 2024-05-21T15:33:08.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaa7fff700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdaa81086d0 0x7fdaa819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:33:08.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaa6ffd700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa8107d80 0x7fdaa819a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:33:08.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaa77fe700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa8103450 0x7fdaa8199d20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fda9800b530 tx=0x7fda9800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:08.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaa4ff9700 1 -- 172.21.15.38:0/2363891042 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98015020 con 0x7fdaa8103450 2024-05-21T15:33:08.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaa4ff9700 1 -- 172.21.15.38:0/2363891042 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda98011d10 con 0x7fdaa8103450 2024-05-21T15:33:08.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaadcde700 1 -- 172.21.15.38:0/2363891042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa819eee0 con 0x7fdaa8103450 2024-05-21T15:33:08.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaa4ff9700 1 -- 172.21.15.38:0/2363891042 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98010e50 con 0x7fdaa8103450 2024-05-21T15:33:08.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.986+0000 7fdaadcde700 1 -- 172.21.15.38:0/2363891042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa81a6800 con 0x7fdaa8103450 2024-05-21T15:33:08.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.987+0000 7fdaadcde700 1 -- 172.21.15.38:0/2363891042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdaa8062dd0 con 0x7fdaa8103450 2024-05-21T15:33:08.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.989+0000 7fdaa4ff9700 1 -- 172.21.15.38:0/2363891042 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fda98018020 con 0x7fdaa8103450 2024-05-21T15:33:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.989+0000 7fdaa4ff9700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fda900a2ed0 0x7fda900a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.990+0000 7fdaa6ffd700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fda900a2ed0 0x7fda900a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.990+0000 7fdaa4ff9700 1 -- 172.21.15.38:0/2363891042 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fda980ced90 con 0x7fdaa8103450 2024-05-21T15:33:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.990+0000 7fdaa4ff9700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282] conn(0x7fda900ac700 0x7fda900aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.990+0000 7fdaa4ff9700 1 -- 172.21.15.38:0/2363891042 --> [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda98012e80 con 0x7fda900ac700 2024-05-21T15:33:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.990+0000 7fdaa4ff9700 1 -- 172.21.15.38:0/2363891042 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fda980cf270 con 0x7fdaa8103450 2024-05-21T15:33:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.990+0000 7fdaa7fff700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282] conn(0x7fda900ac700 0x7fda900aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.990+0000 7fdaa6ffd700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fda900a2ed0 0x7fda900a53e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fda9c005fd0 tx=0x7fda9c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.990+0000 7fdaa7fff700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282] conn(0x7fda900ac700 0x7fda900aeb50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:08.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:08.991+0000 7fdaa4ff9700 1 -- 172.21.15.38:0/2363891042 <== osd.0 v2:172.21.15.38:6802/2546787282 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda98012e80 con 0x7fda900ac700 2024-05-21T15:33:09.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:09.008+0000 7fdaadcde700 1 -- 172.21.15.38:0/2363891042 --> [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdaa80631d0 con 0x7fda900ac700 2024-05-21T15:33:09.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:09.008+0000 7fdaa4ff9700 1 -- 172.21.15.38:0/2363891042 <== osd.0 v2:172.21.15.38:6802/2546787282 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdaa80631d0 con 0x7fda900ac700 2024-05-21T15:33:09.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:09.008+0000 7fdaadcde700 1 -- 172.21.15.38:0/2363891042 >> [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282] conn(0x7fda900ac700 msgr2=0x7fda900aeb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:09.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:09.008+0000 7fdaadcde700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282] conn(0x7fda900ac700 0x7fda900aeb50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:09.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:09.009+0000 7fdaadcde700 1 -- 172.21.15.38:0/2363891042 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fda900a2ed0 msgr2=0x7fda900a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:09.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:09.009+0000 7fdaadcde700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fda900a2ed0 0x7fda900a53e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fda9c005fd0 tx=0x7fda9c005ee0 comp rx=0 tx=0).stop 2024-05-21T15:33:09.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:09.009+0000 7fdaadcde700 1 -- 172.21.15.38:0/2363891042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa8103450 msgr2=0x7fdaa8199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:09.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:09.009+0000 7fdaadcde700 1 --2- 172.21.15.38:0/2363891042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa8103450 0x7fdaa8199d20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fda9800b530 tx=0x7fda9800b8f0 comp rx=0 tx=0).stop 2024-05-21T15:33:09.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:09.009+0000 7fdaa7fff700 1 -- 172.21.15.38:0/2363891042 reap_dead start 2024-05-21T15:33:09.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:09.009+0000 7fdaadcde700 1 -- 172.21.15.38:0/2363891042 shutdown_connections 2024-05-21T15:33:09.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:09.009+0000 7fdaadcde700 1 -- 172.21.15.38:0/2363891042 >> 172.21.15.38:0/2363891042 conn(0x7fdaa80fed20 msgr2=0x7fdaa8107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:09.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:09.009+0000 7fdaadcde700 1 -- 172.21.15.38:0/2363891042 shutdown_connections 2024-05-21T15:33:09.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:09.009+0000 7fdaadcde700 1 -- 172.21.15.38:0/2363891042 wait complete. 2024-05-21T15:33:09.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:09 smithi053 ceph-mon[36496]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-21T15:33:09.731 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:09 smithi038 ceph-mon[31995]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-21T15:33:09.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:09 smithi045 ceph-mon[36409]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-21T15:33:10.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.873+0000 7f86d235f700 1 -- 172.21.15.38:0/1440873110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc104380 msgr2=0x7f86cc104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:10.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.873+0000 7f86d235f700 1 --2- 172.21.15.38:0/1440873110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc104380 0x7f86cc104790 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009990 tx=0x7f86bc009ca0 comp rx=0 tx=0).stop 2024-05-21T15:33:10.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.873+0000 7f86d235f700 1 -- 172.21.15.38:0/1440873110 shutdown_connections 2024-05-21T15:33:10.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.873+0000 7f86d235f700 1 --2- 172.21.15.38:0/1440873110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86cc10d760 0x7f86cc10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:10.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.873+0000 7f86d235f700 1 --2- 172.21.15.38:0/1440873110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc104380 0x7f86cc104790 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:10.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.873+0000 7f86d235f700 1 --2- 172.21.15.38:0/1440873110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86cc104cd0 0x7f86cc10d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:10.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.873+0000 7f86d235f700 1 -- 172.21.15.38:0/1440873110 >> 172.21.15.38:0/1440873110 conn(0x7f86cc0fdc40 msgr2=0x7f86cc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:10.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.873+0000 7f86d235f700 1 -- 172.21.15.38:0/1440873110 shutdown_connections 2024-05-21T15:33:10.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.873+0000 7f86d235f700 1 -- 172.21.15.38:0/1440873110 wait complete. 2024-05-21T15:33:10.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.873+0000 7f86d235f700 1 Processor -- start 2024-05-21T15:33:10.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.874+0000 7f86d235f700 1 -- start start 2024-05-21T15:33:10.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.874+0000 7f86d235f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86cc104380 0x7f86cc199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.874+0000 7f86d235f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86cc104cd0 0x7f86cc19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.874+0000 7f86d235f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc10d760 0x7f86cc19e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.874+0000 7f86d235f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc113400 con 0x7f86cc10d760 2024-05-21T15:33:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.874+0000 7f86d235f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc113280 con 0x7f86cc104380 2024-05-21T15:33:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.874+0000 7f86d235f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc113580 con 0x7f86cc104cd0 2024-05-21T15:33:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.874+0000 7f86d08fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc10d760 0x7f86cc19e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.874+0000 7f86cbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86cc104380 0x7f86cc199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.874+0000 7f86d08fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc10d760 0x7f86cc19e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47212/0 (socket says 172.21.15.38:47212) 2024-05-21T15:33:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.874+0000 7f86d08fc700 1 -- 172.21.15.38:0/1649491275 learned_addr learned my addr 172.21.15.38:0/1649491275 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.874+0000 7f86cb7fe700 1 --2- 172.21.15.38:0/1649491275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86cc104cd0 0x7f86cc19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.875+0000 7f86cbfff700 1 -- 172.21.15.38:0/1649491275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86cc104cd0 msgr2=0x7f86cc19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.875+0000 7f86cbfff700 1 --2- 172.21.15.38:0/1649491275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86cc104cd0 0x7f86cc19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.875+0000 7f86cbfff700 1 -- 172.21.15.38:0/1649491275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc10d760 msgr2=0x7f86cc19e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.875+0000 7f86cbfff700 1 --2- 172.21.15.38:0/1649491275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc10d760 0x7f86cc19e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.875+0000 7f86cbfff700 1 -- 172.21.15.38:0/1649491275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86cc19ebe0 con 0x7f86cc104380 2024-05-21T15:33:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.875+0000 7f86d08fc700 1 --2- 172.21.15.38:0/1649491275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86cc10d760 0x7f86cc19e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:33:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.875+0000 7f86cbfff700 1 --2- 172.21.15.38:0/1649491275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86cc104380 0x7f86cc199d10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f86bc006010 tx=0x7f86bc003890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.875+0000 7f86c97fa700 1 -- 172.21.15.38:0/1649491275 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc027070 con 0x7f86cc104380 2024-05-21T15:33:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.875+0000 7f86d235f700 1 -- 172.21.15.38:0/1649491275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86cc19ee70 con 0x7f86cc104380 2024-05-21T15:33:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.875+0000 7f86d235f700 1 -- 172.21.15.38:0/1649491275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86cc1a67f0 con 0x7f86cc104380 2024-05-21T15:33:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.875+0000 7f86c97fa700 1 -- 172.21.15.38:0/1649491275 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86bc004050 con 0x7f86cc104380 2024-05-21T15:33:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.875+0000 7f86c97fa700 1 -- 172.21.15.38:0/1649491275 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc022c70 con 0x7f86cc104380 2024-05-21T15:33:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.876+0000 7f86d235f700 1 -- 172.21.15.38:0/1649491275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86b8001030 con 0x7f86cc104380 2024-05-21T15:33:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.878+0000 7f86c97fa700 1 -- 172.21.15.38:0/1649491275 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f86bc02a070 con 0x7f86cc104380 2024-05-21T15:33:10.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.879+0000 7f86c97fa700 1 --2- 172.21.15.38:0/1649491275 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86b40a2cb0 0x7f86b40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:10.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.879+0000 7f86cb7fe700 1 --2- 172.21.15.38:0/1649491275 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86b40a2cb0 0x7f86b40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:10.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.879+0000 7f86c97fa700 1 -- 172.21.15.38:0/1649491275 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f86bc0d0ee0 con 0x7f86cc104380 2024-05-21T15:33:10.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.879+0000 7f86c97fa700 1 --2- 172.21.15.38:0/1649491275 >> [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033] conn(0x7f86b40ac4e0 0x7f86b40ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:10.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.879+0000 7f86c97fa700 1 -- 172.21.15.38:0/1649491275 --> [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86bc001d30 con 0x7f86b40ac4e0 2024-05-21T15:33:10.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.879+0000 7f86c97fa700 1 -- 172.21.15.38:0/1649491275 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f86bc0d13c0 con 0x7f86cc104380 2024-05-21T15:33:10.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.879+0000 7f86cb7fe700 1 --2- 172.21.15.38:0/1649491275 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86b40a2cb0 0x7f86b40a51c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f86c0005fd0 tx=0x7f86c0005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:10.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.879+0000 7f86d08fc700 1 --2- 172.21.15.38:0/1649491275 >> [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033] conn(0x7f86b40ac4e0 0x7f86b40ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:10.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.880+0000 7f86d08fc700 1 --2- 172.21.15.38:0/1649491275 >> [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033] conn(0x7f86b40ac4e0 0x7f86b40ae930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:10.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.880+0000 7f86c97fa700 1 -- 172.21.15.38:0/1649491275 <== osd.3 v2:172.21.15.38:6810/1590427033 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86bc001d30 con 0x7f86b40ac4e0 2024-05-21T15:33:10.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.897+0000 7f86d235f700 1 -- 172.21.15.38:0/1649491275 --> [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86b8002e60 con 0x7f86b40ac4e0 2024-05-21T15:33:10.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.897+0000 7f86c97fa700 1 -- 172.21.15.38:0/1649491275 <== osd.3 v2:172.21.15.38:6810/1590427033 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f86b8002e60 con 0x7f86b40ac4e0 2024-05-21T15:33:10.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.898+0000 7f86d235f700 1 -- 172.21.15.38:0/1649491275 >> [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033] conn(0x7f86b40ac4e0 msgr2=0x7f86b40ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:10.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.898+0000 7f86d235f700 1 --2- 172.21.15.38:0/1649491275 >> [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033] conn(0x7f86b40ac4e0 0x7f86b40ae930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:10.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.898+0000 7f86d235f700 1 -- 172.21.15.38:0/1649491275 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86b40a2cb0 msgr2=0x7f86b40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:10.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.898+0000 7f86d235f700 1 --2- 172.21.15.38:0/1649491275 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86b40a2cb0 0x7f86b40a51c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f86c0005fd0 tx=0x7f86c0005f60 comp rx=0 tx=0).stop 2024-05-21T15:33:10.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.898+0000 7f86d235f700 1 -- 172.21.15.38:0/1649491275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86cc104380 msgr2=0x7f86cc199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:10.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.898+0000 7f86d235f700 1 --2- 172.21.15.38:0/1649491275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86cc104380 0x7f86cc199d10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f86bc006010 tx=0x7f86bc003890 comp rx=0 tx=0).stop 2024-05-21T15:33:10.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.898+0000 7f86d08fc700 1 -- 172.21.15.38:0/1649491275 reap_dead start 2024-05-21T15:33:10.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.898+0000 7f86d235f700 1 -- 172.21.15.38:0/1649491275 shutdown_connections 2024-05-21T15:33:10.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.898+0000 7f86d235f700 1 -- 172.21.15.38:0/1649491275 >> 172.21.15.38:0/1649491275 conn(0x7f86cc0fdc40 msgr2=0x7f86cc0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:10.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.898+0000 7f86d235f700 1 -- 172.21.15.38:0/1649491275 shutdown_connections 2024-05-21T15:33:10.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.898+0000 7f86d235f700 1 -- 172.21.15.38:0/1649491275 wait complete. 2024-05-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.755+0000 7f8191b87700 1 -- 172.21.15.38:0/3508192703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f818c106cb0 msgr2=0x7f818c1070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.755+0000 7f8191b87700 1 --2- 172.21.15.38:0/3508192703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f818c106cb0 0x7f818c1070c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f81740099c0 tx=0x7f8174009cd0 comp rx=0 tx=0).stop 2024-05-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.755+0000 7f8191b87700 1 -- 172.21.15.38:0/3508192703 shutdown_connections 2024-05-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.755+0000 7f8191b87700 1 --2- 172.21.15.38:0/3508192703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c107600 0x7f818c10ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.755+0000 7f8191b87700 1 --2- 172.21.15.38:0/3508192703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1023d0 0x7f818c1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.755+0000 7f8191b87700 1 --2- 172.21.15.38:0/3508192703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f818c106cb0 0x7f818c1070c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.755+0000 7f8191b87700 1 -- 172.21.15.38:0/3508192703 >> 172.21.15.38:0/3508192703 conn(0x7f818c077c80 msgr2=0x7f818c0780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.755+0000 7f8191b87700 1 -- 172.21.15.38:0/3508192703 shutdown_connections 2024-05-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.755+0000 7f8191b87700 1 -- 172.21.15.38:0/3508192703 wait complete. 2024-05-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.756+0000 7f8191b87700 1 Processor -- start 2024-05-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.756+0000 7f8191b87700 1 -- start start 2024-05-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f8191b87700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1023d0 0x7f818c103d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f8191b87700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c106cb0 0x7f818c104290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f8191b87700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f818c107600 0x7f818c1047d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f8191b87700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f818c112360 con 0x7f818c107600 2024-05-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f8191b87700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1121e0 con 0x7f818c1023d0 2024-05-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f8191b87700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1124e0 con 0x7f818c106cb0 2024-05-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f818bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f818c107600 0x7f818c1047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f8182ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c106cb0 0x7f818c104290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f818bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f818c107600 0x7f818c1047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47190/0 (socket says 172.21.15.38:47190) 2024-05-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f818bfff700 1 -- 172.21.15.38:0/2494020655 learned_addr learned my addr 172.21.15.38:0/2494020655 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f818b7fe700 1 --2- 172.21.15.38:0/2494020655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1023d0 0x7f818c103d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f8182ffd700 1 -- 172.21.15.38:0/2494020655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1023d0 msgr2=0x7f818c103d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f8182ffd700 1 --2- 172.21.15.38:0/2494020655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1023d0 0x7f818c103d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f8182ffd700 1 -- 172.21.15.38:0/2494020655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f818c107600 msgr2=0x7f818c1047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f8182ffd700 1 --2- 172.21.15.38:0/2494020655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f818c107600 0x7f818c1047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f8182ffd700 1 -- 172.21.15.38:0/2494020655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818c1a8120 con 0x7f818c106cb0 2024-05-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.757+0000 7f818bfff700 1 --2- 172.21.15.38:0/2494020655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f818c107600 0x7f818c1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.758+0000 7f8182ffd700 1 --2- 172.21.15.38:0/2494020655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c106cb0 0x7f818c104290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f81740098e0 tx=0x7f81740047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.758+0000 7f81897fa700 1 -- 172.21.15.38:0/2494020655 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174027070 con 0x7f818c106cb0 2024-05-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.758+0000 7f81897fa700 1 -- 172.21.15.38:0/2494020655 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8174003e40 con 0x7f818c106cb0 2024-05-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.758+0000 7f8191b87700 1 -- 172.21.15.38:0/2494020655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818c1a83b0 con 0x7f818c106cb0 2024-05-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.758+0000 7f81897fa700 1 -- 172.21.15.38:0/2494020655 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174022ba0 con 0x7f818c106cb0 2024-05-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.758+0000 7f8191b87700 1 -- 172.21.15.38:0/2494020655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f818c1a8800 con 0x7f818c106cb0 2024-05-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.759+0000 7f8191b87700 1 -- 172.21.15.38:0/2494020655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f818c062d10 con 0x7f818c106cb0 2024-05-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.760+0000 7f81897fa700 1 -- 172.21.15.38:0/2494020655 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f817402a070 con 0x7f818c106cb0 2024-05-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.761+0000 7f81897fa700 1 --2- 172.21.15.38:0/2494020655 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81780a2cb0 0x7f81780a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.761+0000 7f818b7fe700 1 --2- 172.21.15.38:0/2494020655 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81780a2cb0 0x7f81780a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.761+0000 7f81897fa700 1 -- 172.21.15.38:0/2494020655 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f81740d0cd0 con 0x7f818c106cb0 2024-05-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.761+0000 7f81897fa700 1 --2- 172.21.15.38:0/2494020655 >> [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193] conn(0x7f81780ac4e0 0x7f81780ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.761+0000 7f81897fa700 1 -- 172.21.15.38:0/2494020655 --> [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81740043d0 con 0x7f81780ac4e0 2024-05-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.761+0000 7f81897fa700 1 -- 172.21.15.38:0/2494020655 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f81740d1080 con 0x7f818c106cb0 2024-05-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.762+0000 7f818b7fe700 1 --2- 172.21.15.38:0/2494020655 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81780a2cb0 0x7f81780a51c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f817c005950 tx=0x7f817c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.762+0000 7f818bfff700 1 --2- 172.21.15.38:0/2494020655 >> [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193] conn(0x7f81780ac4e0 0x7f81780ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.762+0000 7f818bfff700 1 --2- 172.21.15.38:0/2494020655 >> [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193] conn(0x7f81780ac4e0 0x7f81780ae930 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.763+0000 7f81897fa700 1 -- 172.21.15.38:0/2494020655 <== osd.4 v2:172.21.15.45:6808/283670193 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f81740043d0 con 0x7f81780ac4e0 2024-05-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.799+0000 7f8191b87700 1 -- 172.21.15.38:0/2494020655 --> [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f818c063140 con 0x7f81780ac4e0 2024-05-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.799+0000 7f81897fa700 1 -- 172.21.15.38:0/2494020655 <== osd.4 v2:172.21.15.45:6808/283670193 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f818c063140 con 0x7f81780ac4e0 2024-05-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.800+0000 7f8191b87700 1 -- 172.21.15.38:0/2494020655 >> [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193] conn(0x7f81780ac4e0 msgr2=0x7f81780ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.800+0000 7f8191b87700 1 --2- 172.21.15.38:0/2494020655 >> [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193] conn(0x7f81780ac4e0 0x7f81780ae930 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.800+0000 7f8191b87700 1 -- 172.21.15.38:0/2494020655 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81780a2cb0 msgr2=0x7f81780a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.800+0000 7f8191b87700 1 --2- 172.21.15.38:0/2494020655 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81780a2cb0 0x7f81780a51c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f817c005950 tx=0x7f817c00b410 comp rx=0 tx=0).stop 2024-05-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.800+0000 7f8191b87700 1 -- 172.21.15.38:0/2494020655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c106cb0 msgr2=0x7f818c104290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.800+0000 7f8191b87700 1 --2- 172.21.15.38:0/2494020655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c106cb0 0x7f818c104290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f81740098e0 tx=0x7f81740047b0 comp rx=0 tx=0).stop 2024-05-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.800+0000 7f818bfff700 1 -- 172.21.15.38:0/2494020655 reap_dead start 2024-05-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.800+0000 7f8191b87700 1 -- 172.21.15.38:0/2494020655 shutdown_connections 2024-05-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.800+0000 7f8191b87700 1 -- 172.21.15.38:0/2494020655 >> 172.21.15.38:0/2494020655 conn(0x7f818c077c80 msgr2=0x7f818c074bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.800+0000 7f8191b87700 1 -- 172.21.15.38:0/2494020655 shutdown_connections 2024-05-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:10.800+0000 7f8191b87700 1 -- 172.21.15.38:0/2494020655 wait complete. 2024-05-21T15:33:11.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:11 smithi053 ceph-mon[36496]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:33:11.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:11 smithi045 ceph-mon[36409]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:33:11.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:11 smithi038 ceph-mon[31995]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:33:12.206 INFO:teuthology.orchestra.run.smithi038.stdout:313532612631 2024-05-21T15:33:12.207 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-21T15:33:12.708 INFO:teuthology.orchestra.run.smithi038.stdout:292057776157 2024-05-21T15:33:12.708 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-21T15:33:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.450+0000 7faf023fb700 1 -- 172.21.15.38:0/3025912332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc1086f0 msgr2=0x7faefc110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.450+0000 7faf023fb700 1 --2- 172.21.15.38:0/3025912332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc1086f0 0x7faefc110050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faef400b0d0 tx=0x7faef400b3e0 comp rx=0 tx=0).stop 2024-05-21T15:33:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.450+0000 7faf023fb700 1 -- 172.21.15.38:0/3025912332 shutdown_connections 2024-05-21T15:33:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.450+0000 7faf023fb700 1 --2- 172.21.15.38:0/3025912332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc1086f0 0x7faefc110050 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.450+0000 7faf023fb700 1 --2- 172.21.15.38:0/3025912332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faefc103470 0x7faefc103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.450+0000 7faf023fb700 1 --2- 172.21.15.38:0/3025912332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faefc107da0 0x7faefc1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.450+0000 7faf023fb700 1 -- 172.21.15.38:0/3025912332 >> 172.21.15.38:0/3025912332 conn(0x7faefc0feec0 msgr2=0x7faefc1012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.450+0000 7faf023fb700 1 -- 172.21.15.38:0/3025912332 shutdown_connections 2024-05-21T15:33:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.450+0000 7faf023fb700 1 -- 172.21.15.38:0/3025912332 wait complete. 2024-05-21T15:33:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faf023fb700 1 Processor -- start 2024-05-21T15:33:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faf023fb700 1 -- start start 2024-05-21T15:33:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faf023fb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc103470 0x7faefc199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faf023fb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faefc107da0 0x7faefc19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faf023fb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faefc1086f0 0x7faefc19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faf023fb700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faefc113470 con 0x7faefc103470 2024-05-21T15:33:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faf023fb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faefc1132f0 con 0x7faefc1086f0 2024-05-21T15:33:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faf023fb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faefc1135f0 con 0x7faefc107da0 2024-05-21T15:33:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faefbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc103470 0x7faefc199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faefbfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc103470 0x7faefc199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47260/0 (socket says 172.21.15.38:47260) 2024-05-21T15:33:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faefbfff700 1 -- 172.21.15.38:0/998792919 learned_addr learned my addr 172.21.15.38:0/998792919 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faef3fff700 1 --2- 172.21.15.38:0/998792919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faefc107da0 0x7faefc19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faf00998700 1 --2- 172.21.15.38:0/998792919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faefc1086f0 0x7faefc19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faefbfff700 1 -- 172.21.15.38:0/998792919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faefc107da0 msgr2=0x7faefc19a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.451+0000 7faefbfff700 1 --2- 172.21.15.38:0/998792919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faefc107da0 0x7faefc19a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.452+0000 7faefbfff700 1 -- 172.21.15.38:0/998792919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faefc1086f0 msgr2=0x7faefc19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.452+0000 7faefbfff700 1 --2- 172.21.15.38:0/998792919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faefc1086f0 0x7faefc19e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.452+0000 7faefbfff700 1 -- 172.21.15.38:0/998792919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc19eb40 con 0x7faefc103470 2024-05-21T15:33:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.452+0000 7faefbfff700 1 --2- 172.21.15.38:0/998792919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc103470 0x7faefc199d20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faeec00c9b0 tx=0x7faeec00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.452+0000 7faef9ffb700 1 -- 172.21.15.38:0/998792919 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec007b00 con 0x7faefc103470 2024-05-21T15:33:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.452+0000 7faef9ffb700 1 -- 172.21.15.38:0/998792919 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeec00ce50 con 0x7faefc103470 2024-05-21T15:33:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.452+0000 7faf023fb700 1 -- 172.21.15.38:0/998792919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faefc19ee30 con 0x7faefc103470 2024-05-21T15:33:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.452+0000 7faef9ffb700 1 -- 172.21.15.38:0/998792919 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec018660 con 0x7faefc103470 2024-05-21T15:33:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.452+0000 7faf023fb700 1 -- 172.21.15.38:0/998792919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faefc19f370 con 0x7faefc103470 2024-05-21T15:33:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.453+0000 7faf023fb700 1 -- 172.21.15.38:0/998792919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faefc062dd0 con 0x7faefc103470 2024-05-21T15:33:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.454+0000 7faef9ffb700 1 -- 172.21.15.38:0/998792919 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faeec00f450 con 0x7faefc103470 2024-05-21T15:33:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.455+0000 7faef9ffb700 1 --2- 172.21.15.38:0/998792919 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faee80a2cb0 0x7faee80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.455+0000 7faef3fff700 1 --2- 172.21.15.38:0/998792919 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faee80a2cb0 0x7faee80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.455+0000 7faef9ffb700 1 -- 172.21.15.38:0/998792919 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7faeec0cf380 con 0x7faefc103470 2024-05-21T15:33:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.455+0000 7faef9ffb700 1 --2- 172.21.15.38:0/998792919 >> [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902] conn(0x7faee80ac4e0 0x7faee80ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.455+0000 7faef9ffb700 1 -- 172.21.15.38:0/998792919 --> [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faee80af000 con 0x7faee80ac4e0 2024-05-21T15:33:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.455+0000 7faef9ffb700 1 -- 172.21.15.38:0/998792919 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7faeec0cf860 con 0x7faefc103470 2024-05-21T15:33:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.456+0000 7faef3fff700 1 --2- 172.21.15.38:0/998792919 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faee80a2cb0 0x7faee80a51c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faefc102880 tx=0x7faee400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.456+0000 7faf00998700 1 --2- 172.21.15.38:0/998792919 >> [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902] conn(0x7faee80ac4e0 0x7faee80ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.456+0000 7faf00998700 1 --2- 172.21.15.38:0/998792919 >> [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902] conn(0x7faee80ac4e0 0x7faee80ae930 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.457+0000 7faef9ffb700 1 -- 172.21.15.38:0/998792919 <== osd.5 v2:172.21.15.53:6808/1942044902 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faee80af000 con 0x7faee80ac4e0 2024-05-21T15:33:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.474+0000 7faf023fb700 1 -- 172.21.15.38:0/998792919 --> [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faefc063250 con 0x7faee80ac4e0 2024-05-21T15:33:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.474+0000 7faef9ffb700 1 -- 172.21.15.38:0/998792919 <== osd.5 v2:172.21.15.53:6808/1942044902 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faefc063250 con 0x7faee80ac4e0 2024-05-21T15:33:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.475+0000 7faf023fb700 1 -- 172.21.15.38:0/998792919 >> [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902] conn(0x7faee80ac4e0 msgr2=0x7faee80ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.475+0000 7faf023fb700 1 --2- 172.21.15.38:0/998792919 >> [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902] conn(0x7faee80ac4e0 0x7faee80ae930 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.475+0000 7faf023fb700 1 -- 172.21.15.38:0/998792919 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faee80a2cb0 msgr2=0x7faee80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.475+0000 7faf023fb700 1 --2- 172.21.15.38:0/998792919 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faee80a2cb0 0x7faee80a51c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faefc102880 tx=0x7faee400b410 comp rx=0 tx=0).stop 2024-05-21T15:33:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.475+0000 7faf023fb700 1 -- 172.21.15.38:0/998792919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc103470 msgr2=0x7faefc199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.475+0000 7faf023fb700 1 --2- 172.21.15.38:0/998792919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faefc103470 0x7faefc199d20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faeec00c9b0 tx=0x7faeec00ccc0 comp rx=0 tx=0).stop 2024-05-21T15:33:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.475+0000 7faf00998700 1 -- 172.21.15.38:0/998792919 reap_dead start 2024-05-21T15:33:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.475+0000 7faf023fb700 1 -- 172.21.15.38:0/998792919 shutdown_connections 2024-05-21T15:33:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.475+0000 7faf023fb700 1 -- 172.21.15.38:0/998792919 >> 172.21.15.38:0/998792919 conn(0x7faefc0feec0 msgr2=0x7faefc107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.475+0000 7faf023fb700 1 -- 172.21.15.38:0/998792919 shutdown_connections 2024-05-21T15:33:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.475+0000 7faf023fb700 1 -- 172.21.15.38:0/998792919 wait complete. 2024-05-21T15:33:13.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:13 smithi053 ceph-mon[36496]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:33:13.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:13 smithi045 ceph-mon[36409]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:33:13.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:13 smithi038 ceph-mon[31995]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:33:15.087 INFO:teuthology.orchestra.run.smithi038.stdout:34359738445 2024-05-21T15:33:15.088 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.289+0000 7fe91d59f700 1 -- 172.21.15.38:0/540306353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9181075c0 msgr2=0x7fe91810ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.289+0000 7fe91d59f700 1 --2- 172.21.15.38:0/540306353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9181075c0 0x7fe91810ef20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe91000b0d0 tx=0x7fe91000b3e0 comp rx=0 tx=0).stop 2024-05-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.289+0000 7fe91d59f700 1 -- 172.21.15.38:0/540306353 shutdown_connections 2024-05-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.289+0000 7fe91d59f700 1 --2- 172.21.15.38:0/540306353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9181075c0 0x7fe91810ef20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.289+0000 7fe91d59f700 1 --2- 172.21.15.38:0/540306353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe918102340 0x7fe918102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.289+0000 7fe91d59f700 1 --2- 172.21.15.38:0/540306353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe918106c70 0x7fe918107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.289+0000 7fe91d59f700 1 -- 172.21.15.38:0/540306353 >> 172.21.15.38:0/540306353 conn(0x7fe9180fdc40 msgr2=0x7fe918100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.289+0000 7fe91d59f700 1 -- 172.21.15.38:0/540306353 shutdown_connections 2024-05-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.289+0000 7fe91d59f700 1 -- 172.21.15.38:0/540306353 wait complete. 2024-05-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.289+0000 7fe91d59f700 1 Processor -- start 2024-05-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.290+0000 7fe91d59f700 1 -- start start 2024-05-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.290+0000 7fe91d59f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe918102340 0x7fe91819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.290+0000 7fe91d59f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe918106c70 0x7fe91819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.290+0000 7fe91d59f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9181075c0 0x7fe918199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.290+0000 7fe91d59f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe918112340 con 0x7fe9181075c0 2024-05-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.290+0000 7fe91d59f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe9181121c0 con 0x7fe918106c70 2024-05-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.290+0000 7fe91d59f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe9181124c0 con 0x7fe918102340 2024-05-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.290+0000 7fe9177fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9181075c0 0x7fe918199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.290+0000 7fe916ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe918102340 0x7fe91819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.290+0000 7fe9177fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9181075c0 0x7fe918199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38472/0 (socket says 172.21.15.38:38472) 2024-05-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.290+0000 7fe9177fe700 1 -- 172.21.15.38:0/2929412181 learned_addr learned my addr 172.21.15.38:0/2929412181 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.290+0000 7fe9167fc700 1 --2- 172.21.15.38:0/2929412181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe918106c70 0x7fe91819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.291+0000 7fe9177fe700 1 -- 172.21.15.38:0/2929412181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe918102340 msgr2=0x7fe91819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.291+0000 7fe9177fe700 1 --2- 172.21.15.38:0/2929412181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe918102340 0x7fe91819f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.291+0000 7fe9177fe700 1 -- 172.21.15.38:0/2929412181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe918106c70 msgr2=0x7fe91819fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.291+0000 7fe9177fe700 1 --2- 172.21.15.38:0/2929412181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe918106c70 0x7fe91819fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:15.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.291+0000 7fe9177fe700 1 -- 172.21.15.38:0/2929412181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91819a1b0 con 0x7fe9181075c0 2024-05-21T15:33:15.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.291+0000 7fe9177fe700 1 --2- 172.21.15.38:0/2929412181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9181075c0 0x7fe918199ad0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe910006070 tx=0x7fe910004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:15.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.291+0000 7fe8fffff700 1 -- 172.21.15.38:0/2929412181 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910021070 con 0x7fe9181075c0 2024-05-21T15:33:15.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.291+0000 7fe91d59f700 1 -- 172.21.15.38:0/2929412181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91819a440 con 0x7fe9181075c0 2024-05-21T15:33:15.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.291+0000 7fe8fffff700 1 -- 172.21.15.38:0/2929412181 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe910007b80 con 0x7fe9181075c0 2024-05-21T15:33:15.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.291+0000 7fe8fffff700 1 -- 172.21.15.38:0/2929412181 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9100052a0 con 0x7fe9181075c0 2024-05-21T15:33:15.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.291+0000 7fe91d59f700 1 -- 172.21.15.38:0/2929412181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9181a6740 con 0x7fe9181075c0 2024-05-21T15:33:15.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.292+0000 7fe91d59f700 1 -- 172.21.15.38:0/2929412181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe918062dd0 con 0x7fe9181075c0 2024-05-21T15:33:15.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.294+0000 7fe8fffff700 1 -- 172.21.15.38:0/2929412181 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe910024070 con 0x7fe9181075c0 2024-05-21T15:33:15.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.294+0000 7fe8fffff700 1 --2- 172.21.15.38:0/2929412181 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe9000a2cb0 0x7fe9000a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:15.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.294+0000 7fe8fffff700 1 -- 172.21.15.38:0/2929412181 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe91000f020 con 0x7fe9181075c0 2024-05-21T15:33:15.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.294+0000 7fe916ffd700 1 --2- 172.21.15.38:0/2929412181 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe9000a2cb0 0x7fe9000a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:15.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.295+0000 7fe8fffff700 1 --2- 172.21.15.38:0/2929412181 >> [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423] conn(0x7fe9000ac3b0 0x7fe9000ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:15.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.295+0000 7fe8fffff700 1 -- 172.21.15.38:0/2929412181 --> [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe910028e50 con 0x7fe9000ac3b0 2024-05-21T15:33:15.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.295+0000 7fe8fffff700 1 -- 172.21.15.38:0/2929412181 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe9100d0f90 con 0x7fe9181075c0 2024-05-21T15:33:15.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.295+0000 7fe9167fc700 1 --2- 172.21.15.38:0/2929412181 >> [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423] conn(0x7fe9000ac3b0 0x7fe9000ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:15.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.295+0000 7fe916ffd700 1 --2- 172.21.15.38:0/2929412181 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe9000a2cb0 0x7fe9000a51c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe91819ac10 tx=0x7fe90800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:15.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.295+0000 7fe9167fc700 1 --2- 172.21.15.38:0/2929412181 >> [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423] conn(0x7fe9000ac3b0 0x7fe9000ae800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:15.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.296+0000 7fe8fffff700 1 -- 172.21.15.38:0/2929412181 <== osd.11 v2:172.21.15.53:6824/2775995423 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe910028e50 con 0x7fe9000ac3b0 2024-05-21T15:33:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.313+0000 7fe91d59f700 1 -- 172.21.15.38:0/2929412181 --> [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe918063250 con 0x7fe9000ac3b0 2024-05-21T15:33:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.313+0000 7fe8fffff700 1 -- 172.21.15.38:0/2929412181 <== osd.11 v2:172.21.15.53:6824/2775995423 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe918063250 con 0x7fe9000ac3b0 2024-05-21T15:33:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.314+0000 7fe91d59f700 1 -- 172.21.15.38:0/2929412181 >> [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423] conn(0x7fe9000ac3b0 msgr2=0x7fe9000ae800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.314+0000 7fe91d59f700 1 --2- 172.21.15.38:0/2929412181 >> [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423] conn(0x7fe9000ac3b0 0x7fe9000ae800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.314+0000 7fe91d59f700 1 -- 172.21.15.38:0/2929412181 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe9000a2cb0 msgr2=0x7fe9000a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.314+0000 7fe91d59f700 1 --2- 172.21.15.38:0/2929412181 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe9000a2cb0 0x7fe9000a51c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe91819ac10 tx=0x7fe90800b410 comp rx=0 tx=0).stop 2024-05-21T15:33:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.314+0000 7fe91d59f700 1 -- 172.21.15.38:0/2929412181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9181075c0 msgr2=0x7fe918199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.314+0000 7fe91d59f700 1 --2- 172.21.15.38:0/2929412181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9181075c0 0x7fe918199ad0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe910006070 tx=0x7fe910004d00 comp rx=0 tx=0).stop 2024-05-21T15:33:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.314+0000 7fe9177fe700 1 -- 172.21.15.38:0/2929412181 reap_dead start 2024-05-21T15:33:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.314+0000 7fe91d59f700 1 -- 172.21.15.38:0/2929412181 shutdown_connections 2024-05-21T15:33:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.314+0000 7fe91d59f700 1 -- 172.21.15.38:0/2929412181 >> 172.21.15.38:0/2929412181 conn(0x7fe9180fdc40 msgr2=0x7fe91806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.314+0000 7fe91d59f700 1 -- 172.21.15.38:0/2929412181 shutdown_connections 2024-05-21T15:33:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:15.314+0000 7fe91d59f700 1 -- 172.21.15.38:0/2929412181 wait complete. 2024-05-21T15:33:15.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.349+0000 7f73e6783700 1 -- 172.21.15.38:0/3612044430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e01075d0 msgr2=0x7f73e010ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:15.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.349+0000 7f73e6783700 1 --2- 172.21.15.38:0/3612044430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e01075d0 0x7f73e010ef30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f73d800b0d0 tx=0x7f73d800b3e0 comp rx=0 tx=0).stop 2024-05-21T15:33:15.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.349+0000 7f73e6783700 1 -- 172.21.15.38:0/3612044430 shutdown_connections 2024-05-21T15:33:15.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.349+0000 7f73e6783700 1 --2- 172.21.15.38:0/3612044430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e01075d0 0x7f73e010ef30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:15.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.349+0000 7f73e6783700 1 --2- 172.21.15.38:0/3612044430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0102ae0 0x7f73e0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:15.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.349+0000 7f73e6783700 1 --2- 172.21.15.38:0/3612044430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73e0106c80 0x7f73e0107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:15.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.349+0000 7f73e6783700 1 -- 172.21.15.38:0/3612044430 >> 172.21.15.38:0/3612044430 conn(0x7f73e00fd570 msgr2=0x7f73e00ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:15.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.349+0000 7f73e6783700 1 -- 172.21.15.38:0/3612044430 shutdown_connections 2024-05-21T15:33:15.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.349+0000 7f73e6783700 1 -- 172.21.15.38:0/3612044430 wait complete. 2024-05-21T15:33:15.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.350+0000 7f73e6783700 1 Processor -- start 2024-05-21T15:33:15.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.350+0000 7f73e6783700 1 -- start start 2024-05-21T15:33:15.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.350+0000 7f73e6783700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e0102ae0 0x7f73e0199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:15.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.350+0000 7f73e6783700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0106c80 0x7f73e019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:15.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.350+0000 7f73e6783700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73e01075d0 0x7f73e019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:15.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.350+0000 7f73e6783700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0112350 con 0x7f73e0102ae0 2024-05-21T15:33:15.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.350+0000 7f73e6783700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01121d0 con 0x7f73e0106c80 2024-05-21T15:33:15.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.350+0000 7f73e6783700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01124d0 con 0x7f73e01075d0 2024-05-21T15:33:15.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.350+0000 7f73dffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e0102ae0 0x7f73e0199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:15.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.350+0000 7f73dffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e0102ae0 0x7f73e0199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47228/0 (socket says 172.21.15.38:47228) 2024-05-21T15:33:15.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.350+0000 7f73dffff700 1 -- 172.21.15.38:0/2720004 learned_addr learned my addr 172.21.15.38:0/2720004 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:15.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.350+0000 7f73df7fe700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0106c80 0x7f73e019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:15.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.350+0000 7f73e4d20700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73e01075d0 0x7f73e019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:15.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.351+0000 7f73dffff700 1 -- 172.21.15.38:0/2720004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73e01075d0 msgr2=0x7f73e019e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:15.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.351+0000 7f73dffff700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73e01075d0 0x7f73e019e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:15.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.351+0000 7f73dffff700 1 -- 172.21.15.38:0/2720004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0106c80 msgr2=0x7f73e019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:15.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.351+0000 7f73dffff700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0106c80 0x7f73e019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:15.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.351+0000 7f73dffff700 1 -- 172.21.15.38:0/2720004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e019ec00 con 0x7f73e0102ae0 2024-05-21T15:33:15.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.351+0000 7f73df7fe700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0106c80 0x7f73e019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:33:15.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.351+0000 7f73dffff700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e0102ae0 0x7f73e0199d30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f73d000d950 tx=0x7f73d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:15.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.351+0000 7f73dd7fa700 1 -- 172.21.15.38:0/2720004 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0016070 con 0x7f73e0102ae0 2024-05-21T15:33:15.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.351+0000 7f73dd7fa700 1 -- 172.21.15.38:0/2720004 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d000fd70 con 0x7f73e0102ae0 2024-05-21T15:33:15.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.351+0000 7f73e6783700 1 -- 172.21.15.38:0/2720004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e019eef0 con 0x7f73e0102ae0 2024-05-21T15:33:15.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.351+0000 7f73dd7fa700 1 -- 172.21.15.38:0/2720004 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d000e610 con 0x7f73e0102ae0 2024-05-21T15:33:15.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.351+0000 7f73e6783700 1 -- 172.21.15.38:0/2720004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e01a6810 con 0x7f73e0102ae0 2024-05-21T15:33:15.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.352+0000 7f73e6783700 1 -- 172.21.15.38:0/2720004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f73e00ffe60 con 0x7f73e0102ae0 2024-05-21T15:33:15.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.354+0000 7f73dd7fa700 1 -- 172.21.15.38:0/2720004 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f73d0019070 con 0x7f73e0102ae0 2024-05-21T15:33:15.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.354+0000 7f73dd7fa700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73c80a2cb0 0x7f73c80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:15.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.355+0000 7f73df7fe700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73c80a2cb0 0x7f73c80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:15.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.355+0000 7f73dd7fa700 1 -- 172.21.15.38:0/2720004 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f73d00cf020 con 0x7f73e0102ae0 2024-05-21T15:33:15.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.355+0000 7f73dd7fa700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925] conn(0x7f73c80ac540 0x7f73c80ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:15.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.355+0000 7f73dd7fa700 1 -- 172.21.15.38:0/2720004 --> [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73d0005690 con 0x7f73c80ac540 2024-05-21T15:33:15.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.355+0000 7f73dd7fa700 1 -- 172.21.15.38:0/2720004 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f73d00cf500 con 0x7f73e0102ae0 2024-05-21T15:33:15.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.355+0000 7f73df7fe700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73c80a2cb0 0x7f73c80a51c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f73e0100f70 tx=0x7f73d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:15.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.355+0000 7f73e4d20700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925] conn(0x7f73c80ac540 0x7f73c80ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:15.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.356+0000 7f73e4d20700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925] conn(0x7f73c80ac540 0x7f73c80ae990 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:15.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.357+0000 7f73dd7fa700 1 -- 172.21.15.38:0/2720004 <== osd.2 v2:172.21.15.53:6800/2547348925 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73d0005690 con 0x7f73c80ac540 2024-05-21T15:33:15.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.373+0000 7f73e6783700 1 -- 172.21.15.38:0/2720004 --> [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f73e0103860 con 0x7f73c80ac540 2024-05-21T15:33:15.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.374+0000 7f73dd7fa700 1 -- 172.21.15.38:0/2720004 <== osd.2 v2:172.21.15.53:6800/2547348925 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f73e0103860 con 0x7f73c80ac540 2024-05-21T15:33:15.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.374+0000 7f73e6783700 1 -- 172.21.15.38:0/2720004 >> [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925] conn(0x7f73c80ac540 msgr2=0x7f73c80ae990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:15.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.374+0000 7f73e6783700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925] conn(0x7f73c80ac540 0x7f73c80ae990 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:15.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.374+0000 7f73e6783700 1 -- 172.21.15.38:0/2720004 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73c80a2cb0 msgr2=0x7f73c80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:15.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.374+0000 7f73e6783700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73c80a2cb0 0x7f73c80a51c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f73e0100f70 tx=0x7f73d4009260 comp rx=0 tx=0).stop 2024-05-21T15:33:15.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.374+0000 7f73e6783700 1 -- 172.21.15.38:0/2720004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e0102ae0 msgr2=0x7f73e0199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:15.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.374+0000 7f73e6783700 1 --2- 172.21.15.38:0/2720004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e0102ae0 0x7f73e0199d30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f73d000d950 tx=0x7f73d000dc60 comp rx=0 tx=0).stop 2024-05-21T15:33:15.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.374+0000 7f73e4d20700 1 -- 172.21.15.38:0/2720004 reap_dead start 2024-05-21T15:33:15.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.374+0000 7f73e6783700 1 -- 172.21.15.38:0/2720004 shutdown_connections 2024-05-21T15:33:15.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.374+0000 7f73e6783700 1 -- 172.21.15.38:0/2720004 >> 172.21.15.38:0/2720004 conn(0x7f73e00fd570 msgr2=0x7f73e01062b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:15.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.374+0000 7f73e6783700 1 -- 172.21.15.38:0/2720004 shutdown_connections 2024-05-21T15:33:15.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:13.374+0000 7f73e6783700 1 -- 172.21.15.38:0/2720004 wait complete. 2024-05-21T15:33:15.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:15 smithi053 ceph-mon[36496]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-21T15:33:15.709 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:15 smithi038 ceph-mon[31995]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-21T15:33:15.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:15 smithi045 ceph-mon[36409]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-21T15:33:16.166 INFO:teuthology.orchestra.run.smithi038.stdout:77309411396 2024-05-21T15:33:16.166 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-21T15:33:17.267 INFO:teuthology.orchestra.run.smithi038.stdout:167503724596 2024-05-21T15:33:17.268 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-21T15:33:17.508 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:17 smithi038 ceph-mon[31995]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-21T15:33:17.666 INFO:teuthology.orchestra.run.smithi038.stdout:347892350996 2024-05-21T15:33:17.667 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-21T15:33:17.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:17 smithi053 ceph-mon[36496]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-21T15:33:17.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:17 smithi045 ceph-mon[36409]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-21T15:33:17.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.907+0000 7fe740c72700 1 -- 172.21.15.38:0/1130371509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c101840 msgr2=0x7fe73c101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:17.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.907+0000 7fe740c72700 1 --2- 172.21.15.38:0/1130371509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c101840 0x7fe73c101c50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe73000aaa0 tx=0x7fe73000adb0 comp rx=0 tx=0).stop 2024-05-21T15:33:17.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.907+0000 7fe740c72700 1 -- 172.21.15.38:0/1130371509 shutdown_connections 2024-05-21T15:33:17.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.907+0000 7fe740c72700 1 --2- 172.21.15.38:0/1130371509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe73c102190 0x7fe73c10e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:17.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.907+0000 7fe740c72700 1 --2- 172.21.15.38:0/1130371509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe73c107870 0x7fe73c107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:17.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.907+0000 7fe740c72700 1 --2- 172.21.15.38:0/1130371509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c101840 0x7fe73c101c50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:17.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.907+0000 7fe740c72700 1 -- 172.21.15.38:0/1130371509 >> 172.21.15.38:0/1130371509 conn(0x7fe73c0fd570 msgr2=0x7fe73c0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:17.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.908+0000 7fe740c72700 1 -- 172.21.15.38:0/1130371509 shutdown_connections 2024-05-21T15:33:17.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.908+0000 7fe740c72700 1 -- 172.21.15.38:0/1130371509 wait complete. 2024-05-21T15:33:17.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.909+0000 7fe740c72700 1 Processor -- start 2024-05-21T15:33:17.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.909+0000 7fe740c72700 1 -- start start 2024-05-21T15:33:17.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.909+0000 7fe740c72700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe73c101840 0x7fe73c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:17.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.909+0000 7fe740c72700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe73c102190 0x7fe73c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.909+0000 7fe740c72700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c107870 0x7fe73c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.909+0000 7fe740c72700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1112b0 con 0x7fe73c107870 2024-05-21T15:33:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.909+0000 7fe740c72700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c111130 con 0x7fe73c101840 2024-05-21T15:33:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.909+0000 7fe740c72700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c111430 con 0x7fe73c102190 2024-05-21T15:33:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.909+0000 7fe73ad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c107870 0x7fe73c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.909+0000 7fe73a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe73c101840 0x7fe73c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.909+0000 7fe73a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe73c101840 0x7fe73c199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:56902/0 (socket says 172.21.15.38:56902) 2024-05-21T15:33:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.909+0000 7fe73a59c700 1 -- 172.21.15.38:0/3082753142 learned_addr learned my addr 172.21.15.38:0/3082753142 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.909+0000 7fe73ad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c107870 0x7fe73c19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38492/0 (socket says 172.21.15.38:38492) 2024-05-21T15:33:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.909+0000 7fe739d9b700 1 --2- 172.21.15.38:0/3082753142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe73c102190 0x7fe73c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.910+0000 7fe73a59c700 1 -- 172.21.15.38:0/3082753142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe73c102190 msgr2=0x7fe73c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.910+0000 7fe73a59c700 1 --2- 172.21.15.38:0/3082753142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe73c102190 0x7fe73c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.910+0000 7fe73a59c700 1 -- 172.21.15.38:0/3082753142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c107870 msgr2=0x7fe73c19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:17.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.910+0000 7fe73a59c700 1 --2- 172.21.15.38:0/3082753142 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe73c107870 0x7fe73c19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:17.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.910+0000 7fe73a59c700 1 -- 172.21.15.38:0/3082753142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c19eca0 con 0x7fe73c101840 2024-05-21T15:33:17.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.910+0000 7fe73a59c700 1 --2- 172.21.15.38:0/3082753142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe73c101840 0x7fe73c199dd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00c8f0 tx=0x7fe72c00cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:17.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.910+0000 7fe72b7fe700 1 -- 172.21.15.38:0/3082753142 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c007940 con 0x7fe73c101840 2024-05-21T15:33:17.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.910+0000 7fe72b7fe700 1 -- 172.21.15.38:0/3082753142 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c00f450 con 0x7fe73c101840 2024-05-21T15:33:17.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.910+0000 7fe740c72700 1 -- 172.21.15.38:0/3082753142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c19ef90 con 0x7fe73c101840 2024-05-21T15:33:17.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.910+0000 7fe72b7fe700 1 -- 172.21.15.38:0/3082753142 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c018660 con 0x7fe73c101840 2024-05-21T15:33:17.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.910+0000 7fe740c72700 1 -- 172.21.15.38:0/3082753142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c1a68b0 con 0x7fe73c101840 2024-05-21T15:33:17.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.911+0000 7fe740c72700 1 -- 172.21.15.38:0/3082753142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe73c062dd0 con 0x7fe73c101840 2024-05-21T15:33:17.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.913+0000 7fe72b7fe700 1 -- 172.21.15.38:0/3082753142 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe72c00f600 con 0x7fe73c101840 2024-05-21T15:33:17.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.914+0000 7fe72b7fe700 1 --2- 172.21.15.38:0/3082753142 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7240a2e00 0x7fe7240a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:17.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.914+0000 7fe739d9b700 1 --2- 172.21.15.38:0/3082753142 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7240a2e00 0x7fe7240a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:17.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.914+0000 7fe72b7fe700 1 -- 172.21.15.38:0/3082753142 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe72c0cf250 con 0x7fe73c101840 2024-05-21T15:33:17.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.914+0000 7fe72b7fe700 1 --2- 172.21.15.38:0/3082753142 >> [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500] conn(0x7fe7240ac630 0x7fe7240aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:17.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.914+0000 7fe72b7fe700 1 -- 172.21.15.38:0/3082753142 --> [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7240af150 con 0x7fe7240ac630 2024-05-21T15:33:17.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.914+0000 7fe72b7fe700 1 -- 172.21.15.38:0/3082753142 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe72c0cf600 con 0x7fe73c101840 2024-05-21T15:33:17.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.914+0000 7fe739d9b700 1 --2- 172.21.15.38:0/3082753142 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7240a2e00 0x7fe7240a5310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe73c104d20 tx=0x7fe73000c650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:17.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.914+0000 7fe73ad9d700 1 --2- 172.21.15.38:0/3082753142 >> [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500] conn(0x7fe7240ac630 0x7fe7240aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:17.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.915+0000 7fe73ad9d700 1 --2- 172.21.15.38:0/3082753142 >> [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500] conn(0x7fe7240ac630 0x7fe7240aea80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:17.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.915+0000 7fe72b7fe700 1 -- 172.21.15.38:0/3082753142 <== osd.6 v2:172.21.15.38:6818/1680263500 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7240af150 con 0x7fe7240ac630 2024-05-21T15:33:17.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.932+0000 7fe740c72700 1 -- 172.21.15.38:0/3082753142 --> [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe73c0631d0 con 0x7fe7240ac630 2024-05-21T15:33:17.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.932+0000 7fe72b7fe700 1 -- 172.21.15.38:0/3082753142 <== osd.6 v2:172.21.15.38:6818/1680263500 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe73c0631d0 con 0x7fe7240ac630 2024-05-21T15:33:17.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.932+0000 7fe740c72700 1 -- 172.21.15.38:0/3082753142 >> [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500] conn(0x7fe7240ac630 msgr2=0x7fe7240aea80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:17.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.932+0000 7fe740c72700 1 --2- 172.21.15.38:0/3082753142 >> [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500] conn(0x7fe7240ac630 0x7fe7240aea80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:17.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.933+0000 7fe740c72700 1 -- 172.21.15.38:0/3082753142 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7240a2e00 msgr2=0x7fe7240a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:17.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.933+0000 7fe740c72700 1 --2- 172.21.15.38:0/3082753142 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7240a2e00 0x7fe7240a5310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe73c104d20 tx=0x7fe73000c650 comp rx=0 tx=0).stop 2024-05-21T15:33:17.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.933+0000 7fe740c72700 1 -- 172.21.15.38:0/3082753142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe73c101840 msgr2=0x7fe73c199dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:17.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.933+0000 7fe740c72700 1 --2- 172.21.15.38:0/3082753142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe73c101840 0x7fe73c199dd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00c8f0 tx=0x7fe72c00cc00 comp rx=0 tx=0).stop 2024-05-21T15:33:17.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.933+0000 7fe73ad9d700 1 -- 172.21.15.38:0/3082753142 reap_dead start 2024-05-21T15:33:17.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.933+0000 7fe740c72700 1 -- 172.21.15.38:0/3082753142 shutdown_connections 2024-05-21T15:33:17.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.933+0000 7fe740c72700 1 -- 172.21.15.38:0/3082753142 >> 172.21.15.38:0/3082753142 conn(0x7fe73c0fd570 msgr2=0x7fe73c106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.933+0000 7fe740c72700 1 -- 172.21.15.38:0/3082753142 shutdown_connections 2024-05-21T15:33:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:17.933+0000 7fe740c72700 1 -- 172.21.15.38:0/3082753142 wait complete. 2024-05-21T15:33:18.594 INFO:teuthology.orchestra.run.smithi038.stdout:146028888121 2024-05-21T15:33:18.594 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-21T15:33:19.182 INFO:teuthology.orchestra.run.smithi038.stdout:111669149757 2024-05-21T15:33:19.182 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-21T15:33:19.423 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:19 smithi038 ceph-mon[31995]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-05-21T15:33:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.422+0000 7f6770d32700 1 -- 172.21.15.38:0/1041104707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f676c106c90 msgr2=0x7f676c1070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.422+0000 7f6770d32700 1 --2- 172.21.15.38:0/1041104707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f676c106c90 0x7f676c1070a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f67540099c0 tx=0x7f6754009cd0 comp rx=0 tx=0).stop 2024-05-21T15:33:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.423+0000 7f6770d32700 1 -- 172.21.15.38:0/1041104707 shutdown_connections 2024-05-21T15:33:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.423+0000 7f6770d32700 1 --2- 172.21.15.38:0/1041104707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c1075e0 0x7f676c10ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.423+0000 7f6770d32700 1 --2- 172.21.15.38:0/1041104707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676c102360 0x7f676c102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.423+0000 7f6770d32700 1 --2- 172.21.15.38:0/1041104707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f676c106c90 0x7f676c1070a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.423+0000 7f6770d32700 1 -- 172.21.15.38:0/1041104707 >> 172.21.15.38:0/1041104707 conn(0x7f676c0fddb0 msgr2=0x7f676c1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.423+0000 7f6770d32700 1 -- 172.21.15.38:0/1041104707 shutdown_connections 2024-05-21T15:33:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.423+0000 7f6770d32700 1 -- 172.21.15.38:0/1041104707 wait complete. 2024-05-21T15:33:19.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.424+0000 7f6770d32700 1 Processor -- start 2024-05-21T15:33:19.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.424+0000 7f6770d32700 1 -- start start 2024-05-21T15:33:19.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.424+0000 7f6770d32700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676c102360 0x7f676c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:19.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.424+0000 7f6770d32700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c106c90 0x7f676c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:19.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.424+0000 7f6770d32700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f676c1075e0 0x7f676c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:19.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.424+0000 7f6770d32700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f676c1134c0 con 0x7f676c1075e0 2024-05-21T15:33:19.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.424+0000 7f6770d32700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f676c113340 con 0x7f676c102360 2024-05-21T15:33:19.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.424+0000 7f6770d32700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f676c113640 con 0x7f676c106c90 2024-05-21T15:33:19.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.425+0000 7f676ad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f676c1075e0 0x7f676c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:19.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.425+0000 7f6761d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c106c90 0x7f676c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:19.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.425+0000 7f676ad9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f676c1075e0 0x7f676c19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38518/0 (socket says 172.21.15.38:38518) 2024-05-21T15:33:19.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.425+0000 7f6761d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c106c90 0x7f676c19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:34826/0 (socket says 172.21.15.38:34826) 2024-05-21T15:33:19.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.425+0000 7f676ad9d700 1 -- 172.21.15.38:0/129471834 learned_addr learned my addr 172.21.15.38:0/129471834 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:19.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.425+0000 7f676a59c700 1 --2- 172.21.15.38:0/129471834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676c102360 0x7f676c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:19.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.425+0000 7f6761d9b700 1 -- 172.21.15.38:0/129471834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676c102360 msgr2=0x7f676c199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:19.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.425+0000 7f6761d9b700 1 --2- 172.21.15.38:0/129471834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676c102360 0x7f676c199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:19.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.425+0000 7f6761d9b700 1 -- 172.21.15.38:0/129471834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f676c1075e0 msgr2=0x7f676c19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:19.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.425+0000 7f6761d9b700 1 --2- 172.21.15.38:0/129471834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f676c1075e0 0x7f676c19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:19.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.425+0000 7f6761d9b700 1 -- 172.21.15.38:0/129471834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c19ec80 con 0x7f676c106c90 2024-05-21T15:33:19.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.425+0000 7f676ad9d700 1 --2- 172.21.15.38:0/129471834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f676c1075e0 0x7f676c19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:33:19.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.425+0000 7f6761d9b700 1 --2- 172.21.15.38:0/129471834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c106c90 0x7f676c19a350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f67540098e0 tx=0x7f67540047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:19.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.426+0000 7f6763fff700 1 -- 172.21.15.38:0/129471834 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754026070 con 0x7f676c106c90 2024-05-21T15:33:19.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.426+0000 7f6763fff700 1 -- 172.21.15.38:0/129471834 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6754003e40 con 0x7f676c106c90 2024-05-21T15:33:19.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.426+0000 7f6770d32700 1 -- 172.21.15.38:0/129471834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676c19ef10 con 0x7f676c106c90 2024-05-21T15:33:19.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.426+0000 7f6763fff700 1 -- 172.21.15.38:0/129471834 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675400fb50 con 0x7f676c106c90 2024-05-21T15:33:19.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.426+0000 7f6770d32700 1 -- 172.21.15.38:0/129471834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f676c19f3f0 con 0x7f676c106c90 2024-05-21T15:33:19.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.427+0000 7f6770d32700 1 -- 172.21.15.38:0/129471834 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f676c062dd0 con 0x7f676c106c90 2024-05-21T15:33:19.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.428+0000 7f6763fff700 1 -- 172.21.15.38:0/129471834 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6754029070 con 0x7f676c106c90 2024-05-21T15:33:19.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.429+0000 7f6763fff700 1 --2- 172.21.15.38:0/129471834 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f67580a2cb0 0x7f67580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:19.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.429+0000 7f676a59c700 1 --2- 172.21.15.38:0/129471834 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f67580a2cb0 0x7f67580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:19.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.429+0000 7f6763fff700 1 -- 172.21.15.38:0/129471834 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f675409abb0 con 0x7f676c106c90 2024-05-21T15:33:19.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.429+0000 7f6763fff700 1 --2- 172.21.15.38:0/129471834 >> [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746] conn(0x7f67580ad050 0x7f67580af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:19.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.429+0000 7f6763fff700 1 -- 172.21.15.38:0/129471834 --> [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67580afc10 con 0x7f67580ad050 2024-05-21T15:33:19.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.429+0000 7f6763fff700 1 -- 172.21.15.38:0/129471834 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f6754099540 con 0x7f676c106c90 2024-05-21T15:33:19.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.429+0000 7f676a59c700 1 --2- 172.21.15.38:0/129471834 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f67580a2cb0 0x7f67580a51c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f675c005950 tx=0x7f675c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:19.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.430+0000 7f676ad9d700 1 --2- 172.21.15.38:0/129471834 >> [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746] conn(0x7f67580ad050 0x7f67580af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:19.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.430+0000 7f676ad9d700 1 --2- 172.21.15.38:0/129471834 >> [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746] conn(0x7f67580ad050 0x7f67580af540 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:19.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.431+0000 7f6763fff700 1 -- 172.21.15.38:0/129471834 <== osd.7 v2:172.21.15.45:6816/424307746 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67580afc10 con 0x7f67580ad050 2024-05-21T15:33:19.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.449+0000 7f6770d32700 1 -- 172.21.15.38:0/129471834 --> [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f676c063210 con 0x7f67580ad050 2024-05-21T15:33:19.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.449+0000 7f6763fff700 1 -- 172.21.15.38:0/129471834 <== osd.7 v2:172.21.15.45:6816/424307746 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f676c063210 con 0x7f67580ad050 2024-05-21T15:33:19.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.449+0000 7f6770d32700 1 -- 172.21.15.38:0/129471834 >> [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746] conn(0x7f67580ad050 msgr2=0x7f67580af540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:19.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.449+0000 7f6770d32700 1 --2- 172.21.15.38:0/129471834 >> [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746] conn(0x7f67580ad050 0x7f67580af540 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:19.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.450+0000 7f6770d32700 1 -- 172.21.15.38:0/129471834 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f67580a2cb0 msgr2=0x7f67580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:19.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.450+0000 7f6770d32700 1 --2- 172.21.15.38:0/129471834 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f67580a2cb0 0x7f67580a51c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f675c005950 tx=0x7f675c00b410 comp rx=0 tx=0).stop 2024-05-21T15:33:19.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.450+0000 7f6770d32700 1 -- 172.21.15.38:0/129471834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c106c90 msgr2=0x7f676c19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:19.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.450+0000 7f6770d32700 1 --2- 172.21.15.38:0/129471834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c106c90 0x7f676c19a350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f67540098e0 tx=0x7f67540047b0 comp rx=0 tx=0).stop 2024-05-21T15:33:19.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.450+0000 7f676ad9d700 1 -- 172.21.15.38:0/129471834 reap_dead start 2024-05-21T15:33:19.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.450+0000 7f6770d32700 1 -- 172.21.15.38:0/129471834 shutdown_connections 2024-05-21T15:33:19.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.450+0000 7f6770d32700 1 -- 172.21.15.38:0/129471834 >> 172.21.15.38:0/129471834 conn(0x7f676c0fddb0 msgr2=0x7f676c106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:19.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.450+0000 7f6770d32700 1 -- 172.21.15.38:0/129471834 shutdown_connections 2024-05-21T15:33:19.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:19.450+0000 7f6770d32700 1 -- 172.21.15.38:0/129471834 wait complete. 2024-05-21T15:33:19.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:19 smithi053 ceph-mon[36496]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-05-21T15:33:19.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:19 smithi045 ceph-mon[36409]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-05-21T15:33:20.707 INFO:teuthology.orchestra.run.smithi038.stdout:201863462959 2024-05-21T15:33:20.707 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-21T15:33:21.355 INFO:teuthology.orchestra.run.smithi038.stdout:223338299435 2024-05-21T15:33:21.356 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-21T15:33:21.361 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:21.362 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:21.597 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:21 smithi038 ceph-mon[31995]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:33:21.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:21 smithi053 ceph-mon[36496]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:33:21.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:21 smithi045 ceph-mon[36409]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:33:22.694 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-21T15:33:22.694 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-21T15:33:22.694 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-21T15:33:22.694 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6, 3, 8]}]: dispatch 2024-05-21T15:33:22.694 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-05-21T15:33:22.694 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-05-21T15:33:22.694 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-21T15:33:22.695 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-21T15:33:22.695 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-05-21T15:33:22.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-21T15:33:22.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-21T15:33:22.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-21T15:33:22.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6, 3, 8]}]: dispatch 2024-05-21T15:33:22.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-05-21T15:33:22.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-05-21T15:33:22.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-21T15:33:22.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-21T15:33:22.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-05-21T15:33:22.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-21T15:33:22.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-21T15:33:22.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-21T15:33:22.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6, 3, 8]}]: dispatch 2024-05-21T15:33:22.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-05-21T15:33:22.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-05-21T15:33:22.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-21T15:33:22.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-21T15:33:22.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-05-21T15:33:23.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.465+0000 7f7ff0dac700 1 -- 172.21.15.38:0/3251661902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec10a8f0 msgr2=0x7f7fec112250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:23.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.465+0000 7f7ff0dac700 1 --2- 172.21.15.38:0/3251661902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec10a8f0 0x7f7fec112250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400b0d0 tx=0x7f7fe400b3e0 comp rx=0 tx=0).stop 2024-05-21T15:33:23.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.465+0000 7f7ff0dac700 1 -- 172.21.15.38:0/3251661902 shutdown_connections 2024-05-21T15:33:23.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.465+0000 7f7ff0dac700 1 --2- 172.21.15.38:0/3251661902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec10a8f0 0x7f7fec112250 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:23.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.465+0000 7f7ff0dac700 1 --2- 172.21.15.38:0/3251661902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fec105670 0x7f7fec105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:23.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.465+0000 7f7ff0dac700 1 --2- 172.21.15.38:0/3251661902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fec109fa0 0x7f7fec10a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:23.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.465+0000 7f7ff0dac700 1 -- 172.21.15.38:0/3251661902 >> 172.21.15.38:0/3251661902 conn(0x7f7fec100f40 msgr2=0x7f7fec103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:23.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.465+0000 7f7ff0dac700 1 -- 172.21.15.38:0/3251661902 shutdown_connections 2024-05-21T15:33:23.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.465+0000 7f7ff0dac700 1 -- 172.21.15.38:0/3251661902 wait complete. 2024-05-21T15:33:23.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.466+0000 7f7ff0dac700 1 Processor -- start 2024-05-21T15:33:23.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.466+0000 7f7ff0dac700 1 -- start start 2024-05-21T15:33:23.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7ff0dac700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105670 0x7f7fec19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:23.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7ff0dac700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fec109fa0 0x7f7fec19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7ff0dac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fec10a8f0 0x7f7fec1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7ff0dac700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec115690 con 0x7f7fec105670 2024-05-21T15:33:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7ff0dac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec115510 con 0x7f7fec10a8f0 2024-05-21T15:33:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7ff0dac700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec115810 con 0x7f7fec109fa0 2024-05-21T15:33:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7fea59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105670 0x7f7fec19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7fea59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105670 0x7f7fec19bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38548/0 (socket says 172.21.15.38:38548) 2024-05-21T15:33:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7fea59c700 1 -- 172.21.15.38:0/2330410561 learned_addr learned my addr 172.21.15.38:0/2330410561 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7fead9d700 1 --2- 172.21.15.38:0/2330410561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fec10a8f0 0x7f7fec1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7fe9d9b700 1 --2- 172.21.15.38:0/2330410561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fec109fa0 0x7f7fec19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7fea59c700 1 -- 172.21.15.38:0/2330410561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fec109fa0 msgr2=0x7f7fec19c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7fea59c700 1 --2- 172.21.15.38:0/2330410561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fec109fa0 0x7f7fec19c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7fea59c700 1 -- 172.21.15.38:0/2330410561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fec10a8f0 msgr2=0x7f7fec1a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7fea59c700 1 --2- 172.21.15.38:0/2330410561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fec10a8f0 0x7f7fec1a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:23.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7fea59c700 1 -- 172.21.15.38:0/2330410561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec1a0d20 con 0x7f7fec105670 2024-05-21T15:33:23.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7fea59c700 1 --2- 172.21.15.38:0/2330410561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105670 0x7f7fec19bf00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00d980 tx=0x7f7fdc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:23.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.467+0000 7f7fdb7fe700 1 -- 172.21.15.38:0/2330410561 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc017070 con 0x7f7fec105670 2024-05-21T15:33:23.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.468+0000 7f7ff0dac700 1 -- 172.21.15.38:0/2330410561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fec1a1010 con 0x7f7fec105670 2024-05-21T15:33:23.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.468+0000 7f7fdb7fe700 1 -- 172.21.15.38:0/2330410561 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fdc005020 con 0x7f7fec105670 2024-05-21T15:33:23.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.468+0000 7f7fdb7fe700 1 -- 172.21.15.38:0/2330410561 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc00fbf0 con 0x7f7fec105670 2024-05-21T15:33:23.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.468+0000 7f7ff0dac700 1 -- 172.21.15.38:0/2330410561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fec1a1500 con 0x7f7fec105670 2024-05-21T15:33:23.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.468+0000 7f7ff0dac700 1 -- 172.21.15.38:0/2330410561 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7fec062d90 con 0x7f7fec105670 2024-05-21T15:33:23.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.470+0000 7f7fdb7fe700 1 -- 172.21.15.38:0/2330410561 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7fdc0051d0 con 0x7f7fec105670 2024-05-21T15:33:23.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.471+0000 7f7fdb7fe700 1 --2- 172.21.15.38:0/2330410561 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7fd40a2cb0 0x7f7fd40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:23.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.471+0000 7f7fe9d9b700 1 --2- 172.21.15.38:0/2330410561 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7fd40a2cb0 0x7f7fd40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:23.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.471+0000 7f7fdb7fe700 1 -- 172.21.15.38:0/2330410561 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7fdc0cf600 con 0x7f7fec105670 2024-05-21T15:33:23.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.471+0000 7f7fdb7fe700 1 --2- 172.21.15.38:0/2330410561 >> [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940] conn(0x7f7fd40ac8c0 0x7f7fd40aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:23.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.471+0000 7f7fdb7fe700 1 -- 172.21.15.38:0/2330410561 --> [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7fd40af480 con 0x7f7fd40ac8c0 2024-05-21T15:33:23.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.471+0000 7f7fdb7fe700 1 -- 172.21.15.38:0/2330410561 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f7fdc0cfa40 con 0x7f7fec105670 2024-05-21T15:33:23.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.471+0000 7f7fe9d9b700 1 --2- 172.21.15.38:0/2330410561 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7fd40a2cb0 0x7f7fd40a51c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7fec108b10 tx=0x7f7fe000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:23.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.471+0000 7f7fead9d700 1 --2- 172.21.15.38:0/2330410561 >> [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940] conn(0x7f7fd40ac8c0 0x7f7fd40aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:23.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.472+0000 7f7fead9d700 1 --2- 172.21.15.38:0/2330410561 >> [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940] conn(0x7f7fd40ac8c0 0x7f7fd40aedb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:23.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.473+0000 7f7fdb7fe700 1 -- 172.21.15.38:0/2330410561 <== osd.1 v2:172.21.15.45:6800/3379390940 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fd40af480 con 0x7f7fd40ac8c0 2024-05-21T15:33:23.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.490+0000 7f7ff0dac700 1 -- 172.21.15.38:0/2330410561 --> [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7fec063190 con 0x7f7fd40ac8c0 2024-05-21T15:33:23.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.491+0000 7f7fdb7fe700 1 -- 172.21.15.38:0/2330410561 <== osd.1 v2:172.21.15.45:6800/3379390940 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7fec063190 con 0x7f7fd40ac8c0 2024-05-21T15:33:23.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.491+0000 7f7ff0dac700 1 -- 172.21.15.38:0/2330410561 >> [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940] conn(0x7f7fd40ac8c0 msgr2=0x7f7fd40aedb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:23.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.491+0000 7f7ff0dac700 1 --2- 172.21.15.38:0/2330410561 >> [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940] conn(0x7f7fd40ac8c0 0x7f7fd40aedb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:23.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.491+0000 7f7ff0dac700 1 -- 172.21.15.38:0/2330410561 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7fd40a2cb0 msgr2=0x7f7fd40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.491+0000 7f7ff0dac700 1 --2- 172.21.15.38:0/2330410561 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7fd40a2cb0 0x7f7fd40a51c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7fec108b10 tx=0x7f7fe000a380 comp rx=0 tx=0).stop 2024-05-21T15:33:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.491+0000 7f7ff0dac700 1 -- 172.21.15.38:0/2330410561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105670 msgr2=0x7f7fec19bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.491+0000 7f7ff0dac700 1 --2- 172.21.15.38:0/2330410561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105670 0x7f7fec19bf00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00d980 tx=0x7f7fdc00dc90 comp rx=0 tx=0).stop 2024-05-21T15:33:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.491+0000 7f7fead9d700 1 -- 172.21.15.38:0/2330410561 reap_dead start 2024-05-21T15:33:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.491+0000 7f7ff0dac700 1 -- 172.21.15.38:0/2330410561 shutdown_connections 2024-05-21T15:33:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.491+0000 7f7ff0dac700 1 -- 172.21.15.38:0/2330410561 >> 172.21.15.38:0/2330410561 conn(0x7f7fec100f40 msgr2=0x7f7fec109710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.491+0000 7f7ff0dac700 1 -- 172.21.15.38:0/2330410561 shutdown_connections 2024-05-21T15:33:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:23.491+0000 7f7ff0dac700 1 -- 172.21.15.38:0/2330410561 wait complete. 2024-05-21T15:33:23.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:23 smithi053 ceph-mon[36496]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:33:23.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-21T15:33:23.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-21T15:33:23.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-21T15:33:23.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6, 3, 8]}]': finished 2024-05-21T15:33:23.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-05-21T15:33:23.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-05-21T15:33:23.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-21T15:33:23.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-21T15:33:23.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:23 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-05-21T15:33:23.707 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:23 smithi053 ceph-mon[36496]: osdmap e87: 12 total, 12 up, 12 in 2024-05-21T15:33:23.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:23 smithi045 ceph-mon[36409]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:33:23.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-21T15:33:23.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-21T15:33:23.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-21T15:33:23.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6, 3, 8]}]': finished 2024-05-21T15:33:23.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-05-21T15:33:23.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-05-21T15:33:23.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-21T15:33:23.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-21T15:33:23.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:23 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-05-21T15:33:23.762 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:23 smithi045 ceph-mon[36409]: osdmap e87: 12 total, 12 up, 12 in 2024-05-21T15:33:23.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:23 smithi038 ceph-mon[31995]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:33:23.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-21T15:33:23.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-21T15:33:23.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-21T15:33:23.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6, 3, 8]}]': finished 2024-05-21T15:33:23.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-05-21T15:33:23.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-05-21T15:33:23.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-21T15:33:23.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-21T15:33:23.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:23 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-05-21T15:33:23.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:23 smithi038 ceph-mon[31995]: osdmap e87: 12 total, 12 up, 12 in 2024-05-21T15:33:24.070 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.096+0000 7feb1bd8d700 1 -- 172.21.15.38:0/36562122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb14102a20 msgr2=0x7feb14106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.096+0000 7feb1bd8d700 1 --2- 172.21.15.38:0/36562122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb14102a20 0x7feb14106f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feb040098f0 tx=0x7feb04009c00 comp rx=0 tx=0).stop 2024-05-21T15:33:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.096+0000 7feb1bd8d700 1 -- 172.21.15.38:0/36562122 shutdown_connections 2024-05-21T15:33:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.096+0000 7feb1bd8d700 1 --2- 172.21.15.38:0/36562122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb14107520 0x7feb141079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.096+0000 7feb1bd8d700 1 --2- 172.21.15.38:0/36562122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb141020d0 0x7feb141024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.096+0000 7feb1bd8d700 1 --2- 172.21.15.38:0/36562122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb14102a20 0x7feb14106f50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.096+0000 7feb1bd8d700 1 -- 172.21.15.38:0/36562122 >> 172.21.15.38:0/36562122 conn(0x7feb140fdc40 msgr2=0x7feb14100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.096+0000 7feb1bd8d700 1 -- 172.21.15.38:0/36562122 shutdown_connections 2024-05-21T15:33:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.097+0000 7feb1bd8d700 1 -- 172.21.15.38:0/36562122 wait complete. 2024-05-21T15:33:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.097+0000 7feb1bd8d700 1 Processor -- start 2024-05-21T15:33:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.097+0000 7feb1bd8d700 1 -- start start 2024-05-21T15:33:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb1bd8d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb141020d0 0x7feb141a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb1bd8d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb14102a20 0x7feb141a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb1bd8d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb14107520 0x7feb141a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb1bd8d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feb14119a90 con 0x7feb14102a20 2024-05-21T15:33:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb1bd8d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feb14119910 con 0x7feb14107520 2024-05-21T15:33:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb1bd8d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feb14119c10 con 0x7feb141020d0 2024-05-21T15:33:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb19328700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb14102a20 0x7feb141a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb19328700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb14102a20 0x7feb141a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38560/0 (socket says 172.21.15.38:38560) 2024-05-21T15:33:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb19328700 1 -- 172.21.15.38:0/945563211 learned_addr learned my addr 172.21.15.38:0/945563211 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb19b29700 1 --2- 172.21.15.38:0/945563211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb141020d0 0x7feb141a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb1a32a700 1 --2- 172.21.15.38:0/945563211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb14107520 0x7feb141a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb19328700 1 -- 172.21.15.38:0/945563211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb141020d0 msgr2=0x7feb141a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb19328700 1 --2- 172.21.15.38:0/945563211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb141020d0 0x7feb141a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb19328700 1 -- 172.21.15.38:0/945563211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb14107520 msgr2=0x7feb141a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb19328700 1 --2- 172.21.15.38:0/945563211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb14107520 0x7feb141a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb19328700 1 -- 172.21.15.38:0/945563211 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb141a52f0 con 0x7feb14102a20 2024-05-21T15:33:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.098+0000 7feb19328700 1 --2- 172.21.15.38:0/945563211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb14102a20 0x7feb141a09c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feb04009230 tx=0x7feb0400fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.099+0000 7feb0affd700 1 -- 172.21.15.38:0/945563211 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04005170 con 0x7feb14102a20 2024-05-21T15:33:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.099+0000 7feb0affd700 1 -- 172.21.15.38:0/945563211 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb0400b560 con 0x7feb14102a20 2024-05-21T15:33:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.099+0000 7feb0affd700 1 -- 172.21.15.38:0/945563211 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04003ac0 con 0x7feb14102a20 2024-05-21T15:33:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.099+0000 7feb1bd8d700 1 -- 172.21.15.38:0/945563211 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb141a5580 con 0x7feb14102a20 2024-05-21T15:33:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.099+0000 7feb1bd8d700 1 -- 172.21.15.38:0/945563211 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb141a5a10 con 0x7feb14102a20 2024-05-21T15:33:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.100+0000 7feb1bd8d700 1 -- 172.21.15.38:0/945563211 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feb14106870 con 0x7feb14102a20 2024-05-21T15:33:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.101+0000 7feb0affd700 1 -- 172.21.15.38:0/945563211 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7feb0400ba80 con 0x7feb14102a20 2024-05-21T15:33:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.102+0000 7feb0affd700 1 --2- 172.21.15.38:0/945563211 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7feb000a2d70 0x7feb000a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.102+0000 7feb19b29700 1 --2- 172.21.15.38:0/945563211 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7feb000a2d70 0x7feb000a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.102+0000 7feb0affd700 1 -- 172.21.15.38:0/945563211 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7feb040d0530 con 0x7feb14102a20 2024-05-21T15:33:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.102+0000 7feb0affd700 1 --2- 172.21.15.38:0/945563211 >> [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173] conn(0x7feb000ac980 0x7feb000aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.102+0000 7feb0affd700 1 -- 172.21.15.38:0/945563211 --> [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb0400b0b0 con 0x7feb000ac980 2024-05-21T15:33:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.102+0000 7feb0affd700 1 -- 172.21.15.38:0/945563211 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7feb040d08e0 con 0x7feb14102a20 2024-05-21T15:33:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.102+0000 7feb19b29700 1 --2- 172.21.15.38:0/945563211 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7feb000a2d70 0x7feb000a5280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feb10005fd0 tx=0x7feb10009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.103+0000 7feb1a32a700 1 --2- 172.21.15.38:0/945563211 >> [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173] conn(0x7feb000ac980 0x7feb000aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.103+0000 7feb1a32a700 1 --2- 172.21.15.38:0/945563211 >> [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173] conn(0x7feb000ac980 0x7feb000aee70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.104+0000 7feb0affd700 1 -- 172.21.15.38:0/945563211 <== osd.8 v2:172.21.15.53:6816/53196173 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb0400b0b0 con 0x7feb000ac980 2024-05-21T15:33:24.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.120+0000 7feb1bd8d700 1 -- 172.21.15.38:0/945563211 --> [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feb1406ad70 con 0x7feb000ac980 2024-05-21T15:33:24.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.121+0000 7feb0affd700 1 -- 172.21.15.38:0/945563211 <== osd.8 v2:172.21.15.53:6816/53196173 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7feb1406ad70 con 0x7feb000ac980 2024-05-21T15:33:24.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.121+0000 7feb1bd8d700 1 -- 172.21.15.38:0/945563211 >> [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173] conn(0x7feb000ac980 msgr2=0x7feb000aee70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:24.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.121+0000 7feb1bd8d700 1 --2- 172.21.15.38:0/945563211 >> [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173] conn(0x7feb000ac980 0x7feb000aee70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:24.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.121+0000 7feb1bd8d700 1 -- 172.21.15.38:0/945563211 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7feb000a2d70 msgr2=0x7feb000a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:24.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.121+0000 7feb1bd8d700 1 --2- 172.21.15.38:0/945563211 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7feb000a2d70 0x7feb000a5280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feb10005fd0 tx=0x7feb10009310 comp rx=0 tx=0).stop 2024-05-21T15:33:24.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.121+0000 7feb1bd8d700 1 -- 172.21.15.38:0/945563211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb14102a20 msgr2=0x7feb141a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:24.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.121+0000 7feb1bd8d700 1 --2- 172.21.15.38:0/945563211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb14102a20 0x7feb141a09c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feb04009230 tx=0x7feb0400fb20 comp rx=0 tx=0).stop 2024-05-21T15:33:24.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.121+0000 7feb1a32a700 1 -- 172.21.15.38:0/945563211 reap_dead start 2024-05-21T15:33:24.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.121+0000 7feb1bd8d700 1 -- 172.21.15.38:0/945563211 shutdown_connections 2024-05-21T15:33:24.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.121+0000 7feb1bd8d700 1 -- 172.21.15.38:0/945563211 >> 172.21.15.38:0/945563211 conn(0x7feb140fdc40 msgr2=0x7feb14104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:24.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.122+0000 7feb1bd8d700 1 -- 172.21.15.38:0/945563211 shutdown_connections 2024-05-21T15:33:24.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:24.122+0000 7feb1bd8d700 1 -- 172.21.15.38:0/945563211 wait complete. 2024-05-21T15:33:24.663 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:24 smithi038 ceph-mon[31995]: osdmap e88: 12 total, 12 up, 12 in 2024-05-21T15:33:24.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:24 smithi053 ceph-mon[36496]: osdmap e88: 12 total, 12 up, 12 in 2024-05-21T15:33:24.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:24 smithi045 ceph-mon[36409]: osdmap e88: 12 total, 12 up, 12 in 2024-05-21T15:33:24.968 INFO:teuthology.orchestra.run.smithi038.stdout:257698037798 2024-05-21T15:33:24.968 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-21T15:33:25.508 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:25 smithi038 ceph-mon[31995]: pgmap v319: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 13 B/s, 0 objects/s recovering 2024-05-21T15:33:25.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:25 smithi053 ceph-mon[36496]: pgmap v319: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 13 B/s, 0 objects/s recovering 2024-05-21T15:33:25.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.720+0000 7fc3b683d700 1 -- 172.21.15.38:0/1159069005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01042e0 msgr2=0x7fc3b01046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:25.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.720+0000 7fc3b683d700 1 --2- 172.21.15.38:0/1159069005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01042e0 0x7fc3b01046f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009990 tx=0x7fc3a0009ca0 comp rx=0 tx=0).stop 2024-05-21T15:33:25.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.721+0000 7fc3b683d700 1 -- 172.21.15.38:0/1159069005 shutdown_connections 2024-05-21T15:33:25.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.721+0000 7fc3b683d700 1 --2- 172.21.15.38:0/1159069005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b010d750 0x7fc3b010fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:25.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.721+0000 7fc3b683d700 1 --2- 172.21.15.38:0/1159069005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01042e0 0x7fc3b01046f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:25.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.721+0000 7fc3b683d700 1 --2- 172.21.15.38:0/1159069005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3b0104c30 0x7fc3b010d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:25.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.721+0000 7fc3b683d700 1 -- 172.21.15.38:0/1159069005 >> 172.21.15.38:0/1159069005 conn(0x7fc3b00fdc40 msgr2=0x7fc3b0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.721+0000 7fc3b683d700 1 -- 172.21.15.38:0/1159069005 shutdown_connections 2024-05-21T15:33:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.721+0000 7fc3b683d700 1 -- 172.21.15.38:0/1159069005 wait complete. 2024-05-21T15:33:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.721+0000 7fc3b683d700 1 Processor -- start 2024-05-21T15:33:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.721+0000 7fc3b683d700 1 -- start start 2024-05-21T15:33:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3b683d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01042e0 0x7fc3b0199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3b683d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b0104c30 0x7fc3b019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3b683d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3b010d750 0x7fc3b019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3b683d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b0113440 con 0x7fc3b01042e0 2024-05-21T15:33:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3b683d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b01132c0 con 0x7fc3b010d750 2024-05-21T15:33:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3b683d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b01135c0 con 0x7fc3b0104c30 2024-05-21T15:33:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3affff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01042e0 0x7fc3b0199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3affff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01042e0 0x7fc3b0199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59710/0 (socket says 172.21.15.38:59710) 2024-05-21T15:33:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3affff700 1 -- 172.21.15.38:0/887444102 learned_addr learned my addr 172.21.15.38:0/887444102 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3a7fff700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b0104c30 0x7fc3b019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:25.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3b4dda700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3b010d750 0x7fc3b019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:25.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3affff700 1 -- 172.21.15.38:0/887444102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b0104c30 msgr2=0x7fc3b019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:25.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3affff700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b0104c30 0x7fc3b019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:25.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3affff700 1 -- 172.21.15.38:0/887444102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3b010d750 msgr2=0x7fc3b019e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:25.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3affff700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3b010d750 0x7fc3b019e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:25.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3affff700 1 -- 172.21.15.38:0/887444102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b019ec00 con 0x7fc3b01042e0 2024-05-21T15:33:25.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3a7fff700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b0104c30 0x7fc3b019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:33:25.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3b4dda700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3b010d750 0x7fc3b019e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:33:25.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3affff700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01042e0 0x7fc3b0199d30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0005910 tx=0x7fc3a0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:25.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3adffb700 1 -- 172.21.15.38:0/887444102 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0027070 con 0x7fc3b01042e0 2024-05-21T15:33:25.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.722+0000 7fc3b683d700 1 -- 172.21.15.38:0/887444102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b019ee90 con 0x7fc3b01042e0 2024-05-21T15:33:25.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.723+0000 7fc3adffb700 1 -- 172.21.15.38:0/887444102 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3a0004050 con 0x7fc3b01042e0 2024-05-21T15:33:25.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.723+0000 7fc3adffb700 1 -- 172.21.15.38:0/887444102 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0022e30 con 0x7fc3b01042e0 2024-05-21T15:33:25.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.723+0000 7fc3b683d700 1 -- 172.21.15.38:0/887444102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b019f370 con 0x7fc3b01042e0 2024-05-21T15:33:25.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.723+0000 7fc3b683d700 1 -- 172.21.15.38:0/887444102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b0050170 con 0x7fc3b01042e0 2024-05-21T15:33:25.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.725+0000 7fc3adffb700 1 -- 172.21.15.38:0/887444102 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc3a002a070 con 0x7fc3b01042e0 2024-05-21T15:33:25.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.726+0000 7fc3adffb700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc39c0a2cb0 0x7fc39c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:25.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.726+0000 7fc3adffb700 1 -- 172.21.15.38:0/887444102 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fc3a00d0180 con 0x7fc3b01042e0 2024-05-21T15:33:25.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.726+0000 7fc3a7fff700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc39c0a2cb0 0x7fc39c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:25.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.726+0000 7fc3a7fff700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc39c0a2cb0 0x7fc39c0a51c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00fe6f0 tx=0x7fc39800b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:25.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.729+0000 7fc3adffb700 1 -- 172.21.15.38:0/887444102 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc3a0013070 con 0x7fc3b01042e0 2024-05-21T15:33:25.763 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:25 smithi045 ceph-mon[36409]: pgmap v319: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 13 B/s, 0 objects/s recovering 2024-05-21T15:33:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.945+0000 7fc3b683d700 1 -- 172.21.15.38:0/887444102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fc3b01a6810 con 0x7fc3b01042e0 2024-05-21T15:33:25.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.945+0000 7fc3adffb700 1 -- 172.21.15.38:0/887444102 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc3a009d000 con 0x7fc3b01042e0 2024-05-21T15:33:25.945 INFO:teuthology.orchestra.run.smithi038.stdout:292057776161 2024-05-21T15:33:25.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.947+0000 7fc3b683d700 1 -- 172.21.15.38:0/887444102 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc39c0a2cb0 msgr2=0x7fc39c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:25.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.947+0000 7fc3b683d700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc39c0a2cb0 0x7fc39c0a51c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00fe6f0 tx=0x7fc39800b5f0 comp rx=0 tx=0).stop 2024-05-21T15:33:25.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.947+0000 7fc3b683d700 1 -- 172.21.15.38:0/887444102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01042e0 msgr2=0x7fc3b0199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:25.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.947+0000 7fc3b683d700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01042e0 0x7fc3b0199d30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0005910 tx=0x7fc3a0009f90 comp rx=0 tx=0).stop 2024-05-21T15:33:25.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.947+0000 7fc3b683d700 1 -- 172.21.15.38:0/887444102 shutdown_connections 2024-05-21T15:33:25.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.947+0000 7fc3b683d700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc39c0a2cb0 0x7fc39c0a51c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:25.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.947+0000 7fc3b683d700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3b010d750 0x7fc3b019e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:25.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.947+0000 7fc3b683d700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b01042e0 0x7fc3b0199d30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:25.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.947+0000 7fc3b683d700 1 --2- 172.21.15.38:0/887444102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b0104c30 0x7fc3b019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:25.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.947+0000 7fc3b683d700 1 -- 172.21.15.38:0/887444102 >> 172.21.15.38:0/887444102 conn(0x7fc3b00fdc40 msgr2=0x7fc3b01071d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:25.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.947+0000 7fc3b683d700 1 -- 172.21.15.38:0/887444102 shutdown_connections 2024-05-21T15:33:25.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:25.947+0000 7fc3b683d700 1 -- 172.21.15.38:0/887444102 wait complete. 2024-05-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.508+0000 7f105fb2a700 1 -- 172.21.15.38:0/2794545790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1058104d10 msgr2=0x7f105810d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.508+0000 7f105fb2a700 1 --2- 172.21.15.38:0/2794545790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1058104d10 0x7f105810d260 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f104c009990 tx=0x7f104c009ca0 comp rx=0 tx=0).stop 2024-05-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.509+0000 7f105fb2a700 1 -- 172.21.15.38:0/2794545790 shutdown_connections 2024-05-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.509+0000 7f105fb2a700 1 --2- 172.21.15.38:0/2794545790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f105810d7a0 0x7f105810fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.509+0000 7f105fb2a700 1 --2- 172.21.15.38:0/2794545790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10581043c0 0x7f10581047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.509+0000 7f105fb2a700 1 --2- 172.21.15.38:0/2794545790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1058104d10 0x7f105810d260 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.509+0000 7f105fb2a700 1 -- 172.21.15.38:0/2794545790 >> 172.21.15.38:0/2794545790 conn(0x7f10580fddb0 msgr2=0x7f10581001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.509+0000 7f105fb2a700 1 -- 172.21.15.38:0/2794545790 shutdown_connections 2024-05-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.509+0000 7f105fb2a700 1 -- 172.21.15.38:0/2794545790 wait complete. 2024-05-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.509+0000 7f105fb2a700 1 Processor -- start 2024-05-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.509+0000 7f105fb2a700 1 -- start start 2024-05-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105fb2a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10581043c0 0x7f10581020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105fb2a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1058104d10 0x7f10581025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105fb2a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f105810d7a0 0x7f1058102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105fb2a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1058113430 con 0x7f1058104d10 2024-05-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105fb2a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f10581132b0 con 0x7f10581043c0 2024-05-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105fb2a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f10581135b0 con 0x7f105810d7a0 2024-05-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105d0c5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1058104d10 0x7f10581025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105d0c5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1058104d10 0x7f10581025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59732/0 (socket says 172.21.15.38:59732) 2024-05-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105d0c5700 1 -- 172.21.15.38:0/41451549 learned_addr learned my addr 172.21.15.38:0/41451549 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105d8c6700 1 --2- 172.21.15.38:0/41451549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10581043c0 0x7f10581020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105e0c7700 1 --2- 172.21.15.38:0/41451549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f105810d7a0 0x7f1058102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105d0c5700 1 -- 172.21.15.38:0/41451549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f105810d7a0 msgr2=0x7f1058102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105d0c5700 1 --2- 172.21.15.38:0/41451549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f105810d7a0 0x7f1058102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105d0c5700 1 -- 172.21.15.38:0/41451549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10581043c0 msgr2=0x7f10581020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105d0c5700 1 --2- 172.21.15.38:0/41451549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10581043c0 0x7f10581020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.510+0000 7f105d0c5700 1 -- 172.21.15.38:0/41451549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10581a80c0 con 0x7f1058104d10 2024-05-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.511+0000 7f105d0c5700 1 --2- 172.21.15.38:0/41451549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1058104d10 0x7f10581025f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f104c000c00 tx=0x7f104c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.511+0000 7f1056ffd700 1 -- 172.21.15.38:0/41451549 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c027070 con 0x7f1058104d10 2024-05-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.511+0000 7f1056ffd700 1 -- 172.21.15.38:0/41451549 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f104c003d70 con 0x7f1058104d10 2024-05-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.511+0000 7f1056ffd700 1 -- 172.21.15.38:0/41451549 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c00f460 con 0x7f1058104d10 2024-05-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.511+0000 7f105fb2a700 1 -- 172.21.15.38:0/41451549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10581a8350 con 0x7f1058104d10 2024-05-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.511+0000 7f105fb2a700 1 -- 172.21.15.38:0/41451549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10581a8830 con 0x7f1058104d10 2024-05-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.512+0000 7f105fb2a700 1 -- 172.21.15.38:0/41451549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1058050170 con 0x7f1058104d10 2024-05-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.513+0000 7f1056ffd700 1 -- 172.21.15.38:0/41451549 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f104c02a070 con 0x7f1058104d10 2024-05-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.514+0000 7f1056ffd700 1 --2- 172.21.15.38:0/41451549 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10400a2cb0 0x7f10400a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.514+0000 7f1056ffd700 1 -- 172.21.15.38:0/41451549 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f104c0cffd0 con 0x7f1058104d10 2024-05-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.515+0000 7f105d8c6700 1 --2- 172.21.15.38:0/41451549 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10400a2cb0 0x7f10400a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.515+0000 7f105d8c6700 1 --2- 172.21.15.38:0/41451549 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10400a2cb0 0x7f10400a51c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f104800f540 tx=0x7f104800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.520+0000 7f1056ffd700 1 -- 172.21.15.38:0/41451549 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f104c013070 con 0x7f1058104d10 2024-05-21T15:33:26.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:26 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/887444102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-21T15:33:26.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:26 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/887444102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-21T15:33:26.734 INFO:teuthology.orchestra.run.smithi038.stdout:77309411399 2024-05-21T15:33:26.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.733+0000 7f105fb2a700 1 -- 172.21.15.38:0/41451549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f10581a8b50 con 0x7f1058104d10 2024-05-21T15:33:26.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.734+0000 7f1056ffd700 1 -- 172.21.15.38:0/41451549 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f104c09ce50 con 0x7f1058104d10 2024-05-21T15:33:26.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.736+0000 7f105fb2a700 1 -- 172.21.15.38:0/41451549 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10400a2cb0 msgr2=0x7f10400a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:26.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.736+0000 7f105fb2a700 1 --2- 172.21.15.38:0/41451549 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10400a2cb0 0x7f10400a51c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f104800f540 tx=0x7f104800b040 comp rx=0 tx=0).stop 2024-05-21T15:33:26.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.736+0000 7f105fb2a700 1 -- 172.21.15.38:0/41451549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1058104d10 msgr2=0x7f10581025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:26.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.736+0000 7f105fb2a700 1 --2- 172.21.15.38:0/41451549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1058104d10 0x7f10581025f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f104c000c00 tx=0x7f104c00bab0 comp rx=0 tx=0).stop 2024-05-21T15:33:26.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.736+0000 7f105fb2a700 1 -- 172.21.15.38:0/41451549 shutdown_connections 2024-05-21T15:33:26.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.736+0000 7f105fb2a700 1 --2- 172.21.15.38:0/41451549 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10400a2cb0 0x7f10400a51c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:26.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.736+0000 7f105fb2a700 1 --2- 172.21.15.38:0/41451549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f105810d7a0 0x7f1058102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.736+0000 7f105fb2a700 1 --2- 172.21.15.38:0/41451549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10581043c0 0x7f10581020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.736+0000 7f105fb2a700 1 --2- 172.21.15.38:0/41451549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1058104d10 0x7f10581025f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.736+0000 7f105fb2a700 1 -- 172.21.15.38:0/41451549 >> 172.21.15.38:0/41451549 conn(0x7f10580fddb0 msgr2=0x7f10580ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.736+0000 7f105fb2a700 1 -- 172.21.15.38:0/41451549 shutdown_connections 2024-05-21T15:33:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:26.736+0000 7f105fb2a700 1 -- 172.21.15.38:0/41451549 wait complete. 2024-05-21T15:33:26.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:26 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/887444102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-21T15:33:27.019 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776157 got 292057776161 for osd.9 2024-05-21T15:33:27.019 DEBUG:teuthology.parallel:result is None 2024-05-21T15:33:27.596 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:27 smithi038 ceph-mon[31995]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-05-21T15:33:27.596 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:27 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/41451549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-21T15:33:27.623 INFO:teuthology.orchestra.run.smithi038.stdout:55834574923 2024-05-21T15:33:27.623 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-21T15:33:27.652 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:27.652 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:27.652 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:27.653 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:27.653 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:27.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:27 smithi053 ceph-mon[36496]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-05-21T15:33:27.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:27 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/41451549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-21T15:33:27.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:27 smithi045 ceph-mon[36409]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-05-21T15:33:27.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:27 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/41451549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-21T15:33:28.105 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:28.525 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411399 for osd.2 2024-05-21T15:33:28.525 DEBUG:teuthology.parallel:result is None 2024-05-21T15:33:29.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:29 smithi053 ceph-mon[36496]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 13 B/s, 1 objects/s recovering 2024-05-21T15:33:29.708 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:29 smithi038 ceph-mon[31995]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 13 B/s, 1 objects/s recovering 2024-05-21T15:33:29.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:29 smithi045 ceph-mon[36409]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 13 B/s, 1 objects/s recovering 2024-05-21T15:33:31.575 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:31 smithi038 ceph-mon[31995]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 13 B/s, 1 objects/s recovering 2024-05-21T15:33:31.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:31 smithi053 ceph-mon[36496]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 13 B/s, 1 objects/s recovering 2024-05-21T15:33:31.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:31 smithi045 ceph-mon[36409]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 13 B/s, 1 objects/s recovering 2024-05-21T15:33:33.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:33 smithi053 ceph-mon[36496]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 1 objects/s recovering 2024-05-21T15:33:33.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:33 smithi045 ceph-mon[36409]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 1 objects/s recovering 2024-05-21T15:33:33.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:33 smithi038 ceph-mon[31995]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 1 objects/s recovering 2024-05-21T15:33:34.703 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:34 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:33:34.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:34 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:33:34.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:34 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:33:34.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.921+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3116599303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d010fab0 msgr2=0x7ff3d0111ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:34.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.921+0000 7ff3d78d2700 1 --2- 172.21.15.38:0/3116599303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d010fab0 0x7ff3d0111ed0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800b0a0 tx=0x7ff3c800b3b0 comp rx=0 tx=0).stop 2024-05-21T15:33:34.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.921+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3116599303 shutdown_connections 2024-05-21T15:33:34.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.921+0000 7ff3d78d2700 1 --2- 172.21.15.38:0/3116599303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d010fab0 0x7ff3d0111ed0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:34.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.921+0000 7ff3d78d2700 1 --2- 172.21.15.38:0/3116599303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d0101f00 0x7ff3d0102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:34.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.921+0000 7ff3d78d2700 1 --2- 172.21.15.38:0/3116599303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d01028e0 0x7ff3d010f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:34.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.921+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3116599303 >> 172.21.15.38:0/3116599303 conn(0x7ff3d00fdc40 msgr2=0x7ff3d0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:34.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.921+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3116599303 shutdown_connections 2024-05-21T15:33:34.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.921+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3116599303 wait complete. 2024-05-21T15:33:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.921+0000 7ff3d78d2700 1 Processor -- start 2024-05-21T15:33:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d78d2700 1 -- start start 2024-05-21T15:33:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d78d2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d0101f00 0x7ff3d019bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d78d2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d01028e0 0x7ff3d019c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d78d2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d010fab0 0x7ff3d01a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d78d2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0115660 con 0x7ff3d010fab0 2024-05-21T15:33:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d78d2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01154e0 con 0x7ff3d01028e0 2024-05-21T15:33:34.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d78d2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01157e0 con 0x7ff3d0101f00 2024-05-21T15:33:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d5e6f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d010fab0 0x7ff3d01a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d4e6d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d01028e0 0x7ff3d019c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d5e6f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d010fab0 0x7ff3d01a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52890/0 (socket says 172.21.15.38:52890) 2024-05-21T15:33:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d5e6f700 1 -- 172.21.15.38:0/3118854386 learned_addr learned my addr 172.21.15.38:0/3118854386 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d566e700 1 --2- 172.21.15.38:0/3118854386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d0101f00 0x7ff3d019bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d4e6d700 1 -- 172.21.15.38:0/3118854386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d0101f00 msgr2=0x7ff3d019bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d4e6d700 1 --2- 172.21.15.38:0/3118854386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d0101f00 0x7ff3d019bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d4e6d700 1 -- 172.21.15.38:0/3118854386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d010fab0 msgr2=0x7ff3d01a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d4e6d700 1 --2- 172.21.15.38:0/3118854386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d010fab0 0x7ff3d01a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d4e6d700 1 -- 172.21.15.38:0/3118854386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d01a0da0 con 0x7ff3d01028e0 2024-05-21T15:33:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d5e6f700 1 --2- 172.21.15.38:0/3118854386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d010fab0 0x7ff3d01a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:33:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.922+0000 7ff3d4e6d700 1 --2- 172.21.15.38:0/3118854386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d01028e0 0x7ff3d019c470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0075bc0 tx=0x7ff3c000eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.923+0000 7ff3c67fc700 1 -- 172.21.15.38:0/3118854386 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0004220 con 0x7ff3d01028e0 2024-05-21T15:33:34.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.923+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3118854386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d01a1090 con 0x7ff3d01028e0 2024-05-21T15:33:34.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.923+0000 7ff3c67fc700 1 -- 172.21.15.38:0/3118854386 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c00043c0 con 0x7ff3d01028e0 2024-05-21T15:33:34.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.923+0000 7ff3c67fc700 1 -- 172.21.15.38:0/3118854386 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0003ce0 con 0x7ff3d01028e0 2024-05-21T15:33:34.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.923+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3118854386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d01a89b0 con 0x7ff3d01028e0 2024-05-21T15:33:34.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.924+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3118854386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d0050170 con 0x7ff3d01028e0 2024-05-21T15:33:34.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.925+0000 7ff3c67fc700 1 -- 172.21.15.38:0/3118854386 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff3c0010700 con 0x7ff3d01028e0 2024-05-21T15:33:34.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.926+0000 7ff3c67fc700 1 --2- 172.21.15.38:0/3118854386 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3bc0a2cb0 0x7ff3bc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:34.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.926+0000 7ff3d566e700 1 --2- 172.21.15.38:0/3118854386 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3bc0a2cb0 0x7ff3bc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:34.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.926+0000 7ff3c67fc700 1 -- 172.21.15.38:0/3118854386 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff3c00cf390 con 0x7ff3d01028e0 2024-05-21T15:33:34.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.926+0000 7ff3d566e700 1 --2- 172.21.15.38:0/3118854386 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3bc0a2cb0 0x7ff3bc0a51c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc005950 tx=0x7ff3cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:34.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:34.931+0000 7ff3c67fc700 1 -- 172.21.15.38:0/3118854386 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff3c009c180 con 0x7ff3d01028e0 2024-05-21T15:33:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:35.138+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3118854386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff3d01a8b80 con 0x7ff3d01028e0 2024-05-21T15:33:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:35.138+0000 7ff3c67fc700 1 -- 172.21.15.38:0/3118854386 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff3c009ba80 con 0x7ff3d01028e0 2024-05-21T15:33:35.138 INFO:teuthology.orchestra.run.smithi038.stdout:146028888126 2024-05-21T15:33:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:35.140+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3118854386 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3bc0a2cb0 msgr2=0x7ff3bc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:35.140+0000 7ff3d78d2700 1 --2- 172.21.15.38:0/3118854386 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3bc0a2cb0 0x7ff3bc0a51c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc005950 tx=0x7ff3cc00b410 comp rx=0 tx=0).stop 2024-05-21T15:33:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:35.140+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3118854386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d01028e0 msgr2=0x7ff3d019c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:35.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:35.140+0000 7ff3d78d2700 1 --2- 172.21.15.38:0/3118854386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d01028e0 0x7ff3d019c470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0075bc0 tx=0x7ff3c000eca0 comp rx=0 tx=0).stop 2024-05-21T15:33:35.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:35.140+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3118854386 shutdown_connections 2024-05-21T15:33:35.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:35.140+0000 7ff3d78d2700 1 --2- 172.21.15.38:0/3118854386 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3bc0a2cb0 0x7ff3bc0a51c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:35.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:35.140+0000 7ff3d78d2700 1 --2- 172.21.15.38:0/3118854386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3d010fab0 0x7ff3d01a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:35.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:35.140+0000 7ff3d78d2700 1 --2- 172.21.15.38:0/3118854386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3d0101f00 0x7ff3d019bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:35.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:35.140+0000 7ff3d78d2700 1 --2- 172.21.15.38:0/3118854386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d01028e0 0x7ff3d019c470 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:35.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:35.140+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3118854386 >> 172.21.15.38:0/3118854386 conn(0x7ff3d00fdc40 msgr2=0x7ff3d01112f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:35.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:35.140+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3118854386 shutdown_connections 2024-05-21T15:33:35.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:35.140+0000 7ff3d78d2700 1 -- 172.21.15.38:0/3118854386 wait complete. 2024-05-21T15:33:35.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:35 smithi053 ceph-mon[36496]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 10 B/s, 1 objects/s recovering 2024-05-21T15:33:35.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:35 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3118854386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-21T15:33:35.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:35 smithi045 ceph-mon[36409]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 10 B/s, 1 objects/s recovering 2024-05-21T15:33:35.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:35 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3118854386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-21T15:33:35.771 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:35 smithi038 ceph-mon[31995]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 10 B/s, 1 objects/s recovering 2024-05-21T15:33:35.771 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:35 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3118854386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-21T15:33:36.513 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888121 got 146028888126 for osd.4 2024-05-21T15:33:36.513 DEBUG:teuthology.parallel:result is None 2024-05-21T15:33:36.765 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:36.765 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:36.765 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:36.765 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:36 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:36.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:36 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:36.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:36 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:36.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:36 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:36.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:36 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:37.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:36 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:37.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:36 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:37.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:36 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:37.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:36 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:37.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.118+0000 7f81d869c700 1 -- 172.21.15.38:0/289832015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d0106c70 msgr2=0x7f81d0107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:37.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.118+0000 7f81d869c700 1 --2- 172.21.15.38:0/289832015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d0106c70 0x7f81d0107080 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f81c00099c0 tx=0x7f81c0009cd0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.118+0000 7f81d869c700 1 -- 172.21.15.38:0/289832015 shutdown_connections 2024-05-21T15:33:37.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.118+0000 7f81d869c700 1 --2- 172.21.15.38:0/289832015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81d01075c0 0x7f81d010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.118+0000 7f81d869c700 1 --2- 172.21.15.38:0/289832015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81d0102340 0x7f81d0102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.118+0000 7f81d869c700 1 --2- 172.21.15.38:0/289832015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d0106c70 0x7f81d0107080 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.118+0000 7f81d869c700 1 -- 172.21.15.38:0/289832015 >> 172.21.15.38:0/289832015 conn(0x7f81d00fdc70 msgr2=0x7f81d0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:37.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.118+0000 7f81d869c700 1 -- 172.21.15.38:0/289832015 shutdown_connections 2024-05-21T15:33:37.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.118+0000 7f81d869c700 1 -- 172.21.15.38:0/289832015 wait complete. 2024-05-21T15:33:37.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.119+0000 7f81d869c700 1 Processor -- start 2024-05-21T15:33:37.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.119+0000 7f81d869c700 1 -- start start 2024-05-21T15:33:37.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.119+0000 7f81d869c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81d0102340 0x7f81d0199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:37.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.119+0000 7f81d869c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d0106c70 0x7f81d019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:37.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.119+0000 7f81d869c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81d01075c0 0x7f81d019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:37.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.119+0000 7f81d869c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f81d0112340 con 0x7f81d0106c70 2024-05-21T15:33:37.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.119+0000 7f81d869c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01121c0 con 0x7f81d01075c0 2024-05-21T15:33:37.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.119+0000 7f81d869c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01124c0 con 0x7f81d0102340 2024-05-21T15:33:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.119+0000 7f81d5c37700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d0106c70 0x7f81d019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.119+0000 7f81d5c37700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d0106c70 0x7f81d019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52900/0 (socket says 172.21.15.38:52900) 2024-05-21T15:33:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.119+0000 7f81d5c37700 1 -- 172.21.15.38:0/3313306943 learned_addr learned my addr 172.21.15.38:0/3313306943 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.119+0000 7f81d6438700 1 --2- 172.21.15.38:0/3313306943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81d0102340 0x7f81d0199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.119+0000 7f81d6c39700 1 --2- 172.21.15.38:0/3313306943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81d01075c0 0x7f81d019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.120+0000 7f81d5c37700 1 -- 172.21.15.38:0/3313306943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81d0102340 msgr2=0x7f81d0199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.120+0000 7f81d5c37700 1 --2- 172.21.15.38:0/3313306943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81d0102340 0x7f81d0199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.120+0000 7f81d5c37700 1 -- 172.21.15.38:0/3313306943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81d01075c0 msgr2=0x7f81d019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.120+0000 7f81d5c37700 1 --2- 172.21.15.38:0/3313306943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81d01075c0 0x7f81d019e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.120+0000 7f81d5c37700 1 -- 172.21.15.38:0/3313306943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d019ebe0 con 0x7f81d0106c70 2024-05-21T15:33:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.120+0000 7f81d5c37700 1 --2- 172.21.15.38:0/3313306943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d0106c70 0x7f81d019a360 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f81c000c010 tx=0x7f81c00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.120+0000 7f81c77fe700 1 -- 172.21.15.38:0/3313306943 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0004830 con 0x7f81d0106c70 2024-05-21T15:33:37.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.120+0000 7f81c77fe700 1 -- 172.21.15.38:0/3313306943 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81c0005020 con 0x7f81d0106c70 2024-05-21T15:33:37.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.120+0000 7f81d869c700 1 -- 172.21.15.38:0/3313306943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d019ee70 con 0x7f81d0106c70 2024-05-21T15:33:37.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.120+0000 7f81c77fe700 1 -- 172.21.15.38:0/3313306943 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0003cb0 con 0x7f81d0106c70 2024-05-21T15:33:37.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.120+0000 7f81d869c700 1 -- 172.21.15.38:0/3313306943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d019f2f0 con 0x7f81d0106c70 2024-05-21T15:33:37.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.121+0000 7f81d869c700 1 -- 172.21.15.38:0/3313306943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d0050170 con 0x7f81d0106c70 2024-05-21T15:33:37.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.123+0000 7f81c77fe700 1 -- 172.21.15.38:0/3313306943 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f81c00049d0 con 0x7f81d0106c70 2024-05-21T15:33:37.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.123+0000 7f81c77fe700 1 --2- 172.21.15.38:0/3313306943 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81bc0a2cb0 0x7f81bc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:37.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.123+0000 7f81d6438700 1 --2- 172.21.15.38:0/3313306943 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81bc0a2cb0 0x7f81bc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:37.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.123+0000 7f81c77fe700 1 -- 172.21.15.38:0/3313306943 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f81c00d0540 con 0x7f81d0106c70 2024-05-21T15:33:37.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.124+0000 7f81d6438700 1 --2- 172.21.15.38:0/3313306943 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81bc0a2cb0 0x7f81bc0a51c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00ac50 tx=0x7f81cc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:37.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.127+0000 7f81c77fe700 1 -- 172.21.15.38:0/3313306943 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f81c0013070 con 0x7f81d0106c70 2024-05-21T15:33:37.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.336+0000 7f81d869c700 1 -- 172.21.15.38:0/3313306943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f81d002dc60 con 0x7f81d0106c70 2024-05-21T15:33:37.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.336+0000 7f81c77fe700 1 -- 172.21.15.38:0/3313306943 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f81c00095a0 con 0x7f81d0106c70 2024-05-21T15:33:37.337 INFO:teuthology.orchestra.run.smithi038.stdout:347892351001 2024-05-21T15:33:37.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.338+0000 7f0add551700 1 -- 172.21.15.38:0/4211894062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad81086f0 msgr2=0x7f0ad8110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:37.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.338+0000 7f0add551700 1 --2- 172.21.15.38:0/4211894062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad81086f0 0x7f0ad8110050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000b0d0 tx=0x7f0ad000b3e0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.338+0000 7f0add551700 1 -- 172.21.15.38:0/4211894062 shutdown_connections 2024-05-21T15:33:37.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.338+0000 7f0add551700 1 --2- 172.21.15.38:0/4211894062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad81086f0 0x7f0ad8110050 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.338+0000 7f0add551700 1 --2- 172.21.15.38:0/4211894062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad8103470 0x7f0ad8103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.338+0000 7f0add551700 1 --2- 172.21.15.38:0/4211894062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ad8107da0 0x7f0ad81081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.338+0000 7f0add551700 1 -- 172.21.15.38:0/4211894062 >> 172.21.15.38:0/4211894062 conn(0x7f0ad80feec0 msgr2=0x7f0ad81012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:37.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.338+0000 7f0add551700 1 -- 172.21.15.38:0/4211894062 shutdown_connections 2024-05-21T15:33:37.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.338+0000 7f0add551700 1 -- 172.21.15.38:0/4211894062 wait complete. 2024-05-21T15:33:37.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.339+0000 7f0add551700 1 Processor -- start 2024-05-21T15:33:37.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.339+0000 7f81d869c700 1 -- 172.21.15.38:0/3313306943 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81bc0a2cb0 msgr2=0x7f81bc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:37.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.339+0000 7f81d869c700 1 --2- 172.21.15.38:0/3313306943 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81bc0a2cb0 0x7f81bc0a51c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00ac50 tx=0x7f81cc00a380 comp rx=0 tx=0).stop 2024-05-21T15:33:37.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.339+0000 7f81d869c700 1 -- 172.21.15.38:0/3313306943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d0106c70 msgr2=0x7f81d019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:37.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.339+0000 7f81d869c700 1 --2- 172.21.15.38:0/3313306943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d0106c70 0x7f81d019a360 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f81c000c010 tx=0x7f81c00047b0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.339+0000 7f81d869c700 1 -- 172.21.15.38:0/3313306943 shutdown_connections 2024-05-21T15:33:37.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.339+0000 7f81d869c700 1 --2- 172.21.15.38:0/3313306943 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81bc0a2cb0 0x7f81bc0a51c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.339+0000 7f0add551700 1 -- start start 2024-05-21T15:33:37.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.339+0000 7f81d869c700 1 --2- 172.21.15.38:0/3313306943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81d01075c0 0x7f81d019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.339+0000 7f81d869c700 1 --2- 172.21.15.38:0/3313306943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81d0102340 0x7f81d0199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.339+0000 7f81d869c700 1 --2- 172.21.15.38:0/3313306943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d0106c70 0x7f81d019a360 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.339+0000 7f81d869c700 1 -- 172.21.15.38:0/3313306943 >> 172.21.15.38:0/3313306943 conn(0x7f81d00fdc70 msgr2=0x7f81d0074c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:37.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.339+0000 7f81d869c700 1 -- 172.21.15.38:0/3313306943 shutdown_connections 2024-05-21T15:33:37.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.340+0000 7f81d869c700 1 -- 172.21.15.38:0/3313306943 wait complete. 2024-05-21T15:33:37.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.340+0000 7f0add551700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad8103470 0x7f0ad8074940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:37.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.340+0000 7f0add551700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad8107da0 0x7f0ad8074e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:37.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.340+0000 7f0add551700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ad81086f0 0x7f0ad80753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:37.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.340+0000 7f0add551700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad81134c0 con 0x7f0ad8107da0 2024-05-21T15:33:37.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.340+0000 7f0add551700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad8113340 con 0x7f0ad8103470 2024-05-21T15:33:37.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.340+0000 7f0add551700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad8113640 con 0x7f0ad81086f0 2024-05-21T15:33:37.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.340+0000 7f0ace7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad8107da0 0x7f0ad8074e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:37.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.340+0000 7f0ad6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad8103470 0x7f0ad8074940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:37.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.340+0000 7f0ace7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad8107da0 0x7f0ad8074e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52922/0 (socket says 172.21.15.38:52922) 2024-05-21T15:33:37.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.340+0000 7f0ace7fc700 1 -- 172.21.15.38:0/3654143980 learned_addr learned my addr 172.21.15.38:0/3654143980 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:37.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.340+0000 7f0ad77fe700 1 --2- 172.21.15.38:0/3654143980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ad81086f0 0x7f0ad80753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:37.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.341+0000 7f0ad6ffd700 1 -- 172.21.15.38:0/3654143980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ad81086f0 msgr2=0x7f0ad80753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:37.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.341+0000 7f0ad6ffd700 1 --2- 172.21.15.38:0/3654143980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ad81086f0 0x7f0ad80753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.341+0000 7f0ad6ffd700 1 -- 172.21.15.38:0/3654143980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad8107da0 msgr2=0x7f0ad8074e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:37.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.341+0000 7f0ad6ffd700 1 --2- 172.21.15.38:0/3654143980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad8107da0 0x7f0ad8074e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.341+0000 7f0ad6ffd700 1 -- 172.21.15.38:0/3654143980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad81a8120 con 0x7f0ad8103470 2024-05-21T15:33:37.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.341+0000 7f0ad6ffd700 1 --2- 172.21.15.38:0/3654143980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad8103470 0x7f0ad8074940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800e9f0 tx=0x7f0ac800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:37.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.341+0000 7f0ad4ff9700 1 -- 172.21.15.38:0/3654143980 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac800cbd0 con 0x7f0ad8103470 2024-05-21T15:33:37.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.341+0000 7f0add551700 1 -- 172.21.15.38:0/3654143980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad81a8410 con 0x7f0ad8103470 2024-05-21T15:33:37.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.341+0000 7f0ad4ff9700 1 -- 172.21.15.38:0/3654143980 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ac800cd70 con 0x7f0ad8103470 2024-05-21T15:33:37.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.341+0000 7f0ad4ff9700 1 -- 172.21.15.38:0/3654143980 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8018710 con 0x7f0ad8103470 2024-05-21T15:33:37.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.341+0000 7f0add551700 1 -- 172.21.15.38:0/3654143980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad81a89a0 con 0x7f0ad8103470 2024-05-21T15:33:37.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.343+0000 7f0add551700 1 -- 172.21.15.38:0/3654143980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad8050170 con 0x7f0ad8103470 2024-05-21T15:33:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.345+0000 7f0ad4ff9700 1 -- 172.21.15.38:0/3654143980 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0ac8010430 con 0x7f0ad8103470 2024-05-21T15:33:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.345+0000 7f0ad4ff9700 1 --2- 172.21.15.38:0/3654143980 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0ac40a2cb0 0x7f0ac40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.346+0000 7f0ad4ff9700 1 -- 172.21.15.38:0/3654143980 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0ac80cf740 con 0x7f0ad8103470 2024-05-21T15:33:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.346+0000 7f0ace7fc700 1 --2- 172.21.15.38:0/3654143980 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0ac40a2cb0 0x7f0ac40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:37.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.346+0000 7f0ace7fc700 1 --2- 172.21.15.38:0/3654143980 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0ac40a2cb0 0x7f0ac40a51c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8106ad0 tx=0x7f0ac0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:37.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.349+0000 7f0ad4ff9700 1 -- 172.21.15.38:0/3654143980 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0ac809c530 con 0x7f0ad8103470 2024-05-21T15:33:37.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.569+0000 7f0add551700 1 -- 172.21.15.38:0/3654143980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0ad802dc60 con 0x7f0ad8103470 2024-05-21T15:33:37.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.570+0000 7f0ad4ff9700 1 -- 172.21.15.38:0/3654143980 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0ac809be30 con 0x7f0ad8103470 2024-05-21T15:33:37.569 INFO:teuthology.orchestra.run.smithi038.stdout:111669149762 2024-05-21T15:33:37.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.572+0000 7f0add551700 1 -- 172.21.15.38:0/3654143980 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0ac40a2cb0 msgr2=0x7f0ac40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:37.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.572+0000 7f0add551700 1 --2- 172.21.15.38:0/3654143980 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0ac40a2cb0 0x7f0ac40a51c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8106ad0 tx=0x7f0ac0009260 comp rx=0 tx=0).stop 2024-05-21T15:33:37.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.572+0000 7f0add551700 1 -- 172.21.15.38:0/3654143980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad8103470 msgr2=0x7f0ad8074940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:37.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.572+0000 7f0add551700 1 --2- 172.21.15.38:0/3654143980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad8103470 0x7f0ad8074940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800e9f0 tx=0x7f0ac800ed00 comp rx=0 tx=0).stop 2024-05-21T15:33:37.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.572+0000 7f0add551700 1 -- 172.21.15.38:0/3654143980 shutdown_connections 2024-05-21T15:33:37.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.572+0000 7f0add551700 1 --2- 172.21.15.38:0/3654143980 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0ac40a2cb0 0x7f0ac40a51c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.572+0000 7f0add551700 1 --2- 172.21.15.38:0/3654143980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ad81086f0 0x7f0ad80753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.572+0000 7f0add551700 1 --2- 172.21.15.38:0/3654143980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad8103470 0x7f0ad8074940 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.572+0000 7f0add551700 1 --2- 172.21.15.38:0/3654143980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ad8107da0 0x7f0ad8074e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:37.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.572+0000 7f0add551700 1 -- 172.21.15.38:0/3654143980 >> 172.21.15.38:0/3654143980 conn(0x7f0ad80feec0 msgr2=0x7f0ad8107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:37.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.572+0000 7f0add551700 1 -- 172.21.15.38:0/3654143980 shutdown_connections 2024-05-21T15:33:37.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:37.572+0000 7f0add551700 1 -- 172.21.15.38:0/3654143980 wait complete. 2024-05-21T15:33:38.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:37 smithi038 ceph-mon[31995]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:33:38.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:37 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3313306943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-21T15:33:38.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:37 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3654143980' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-21T15:33:38.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:37 smithi053 ceph-mon[36496]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:33:38.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:37 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3313306943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-21T15:33:38.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:37 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3654143980' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-21T15:33:38.243 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350996 got 347892351001 for osd.11 2024-05-21T15:33:38.243 DEBUG:teuthology.parallel:result is None 2024-05-21T15:33:38.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:37 smithi045 ceph-mon[36409]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:33:38.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:37 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3313306943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-21T15:33:38.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:37 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3654143980' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-21T15:33:38.676 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149757 got 111669149762 for osd.3 2024-05-21T15:33:38.677 DEBUG:teuthology.parallel:result is None 2024-05-21T15:33:38.896 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:39.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:39 smithi038 ceph-mon[31995]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:33:40.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.009+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2331825248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a381075c0 msgr2=0x7f3a3810ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:40.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.009+0000 7f3a3dd41700 1 --2- 172.21.15.38:0/2331825248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a381075c0 0x7f3a3810ef20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3a280099c0 tx=0x7f3a28009cd0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.009+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2331825248 shutdown_connections 2024-05-21T15:33:40.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.009+0000 7f3a3dd41700 1 --2- 172.21.15.38:0/2331825248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a381075c0 0x7f3a3810ef20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.009+0000 7f3a3dd41700 1 --2- 172.21.15.38:0/2331825248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38102340 0x7f3a38102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.009+0000 7f3a3dd41700 1 --2- 172.21.15.38:0/2331825248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a38106c70 0x7f3a38107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.009+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2331825248 >> 172.21.15.38:0/2331825248 conn(0x7f3a380fdc40 msgr2=0x7f3a38100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:40.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.009+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2331825248 shutdown_connections 2024-05-21T15:33:40.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.009+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2331825248 wait complete. 2024-05-21T15:33:40.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.010+0000 7f3a3dd41700 1 Processor -- start 2024-05-21T15:33:40.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.010+0000 7f3a3dd41700 1 -- start start 2024-05-21T15:33:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a3dd41700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38102340 0x7f3a3819f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a3dd41700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a38106c70 0x7f3a3819fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a3dd41700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a381075c0 0x7f3a38199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a3dd41700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38112340 con 0x7f3a38106c70 2024-05-21T15:33:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a3dd41700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3a381121c0 con 0x7f3a38102340 2024-05-21T15:33:40.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a3dd41700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3a381124c0 con 0x7f3a381075c0 2024-05-21T15:33:40.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a377fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38102340 0x7f3a3819f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:40.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a377fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38102340 0x7f3a3819f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:37850/0 (socket says 172.21.15.38:37850) 2024-05-21T15:33:40.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a377fe700 1 -- 172.21.15.38:0/2709050703 learned_addr learned my addr 172.21.15.38:0/2709050703 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:40.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a37fff700 1 --2- 172.21.15.38:0/2709050703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a381075c0 0x7f3a38199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:40.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a36ffd700 1 --2- 172.21.15.38:0/2709050703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a38106c70 0x7f3a3819fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:40.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a377fe700 1 -- 172.21.15.38:0/2709050703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a381075c0 msgr2=0x7f3a38199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:40.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a377fe700 1 --2- 172.21.15.38:0/2709050703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a381075c0 0x7f3a38199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a377fe700 1 -- 172.21.15.38:0/2709050703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a38106c70 msgr2=0x7f3a3819fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:40.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a377fe700 1 --2- 172.21.15.38:0/2709050703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a38106c70 0x7f3a3819fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.011+0000 7f3a377fe700 1 -- 172.21.15.38:0/2709050703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3819a1d0 con 0x7f3a38102340 2024-05-21T15:33:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.012+0000 7f3a377fe700 1 --2- 172.21.15.38:0/2709050703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38102340 0x7f3a3819f9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c9b0 tx=0x7f3a3000ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.012+0000 7f3a34ff9700 1 -- 172.21.15.38:0/2709050703 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30007b00 con 0x7f3a38102340 2024-05-21T15:33:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.012+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2709050703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3819a4c0 con 0x7f3a38102340 2024-05-21T15:33:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.012+0000 7f3a34ff9700 1 -- 172.21.15.38:0/2709050703 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a3000ce50 con 0x7f3a38102340 2024-05-21T15:33:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.012+0000 7f3a34ff9700 1 -- 172.21.15.38:0/2709050703 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30018660 con 0x7f3a38102340 2024-05-21T15:33:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.012+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2709050703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a381a6750 con 0x7f3a38102340 2024-05-21T15:33:40.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.013+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2709050703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a381037f0 con 0x7f3a38102340 2024-05-21T15:33:40.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.015+0000 7f3a34ff9700 1 -- 172.21.15.38:0/2709050703 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3a30043b10 con 0x7f3a38102340 2024-05-21T15:33:40.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.016+0000 7f3a34ff9700 1 --2- 172.21.15.38:0/2709050703 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3a240a2cb0 0x7f3a240a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:40.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.016+0000 7f3a34ff9700 1 -- 172.21.15.38:0/2709050703 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f3a300cf2c0 con 0x7f3a38102340 2024-05-21T15:33:40.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.016+0000 7f3a36ffd700 1 --2- 172.21.15.38:0/2709050703 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3a240a2cb0 0x7f3a240a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:40.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.017+0000 7f3a36ffd700 1 --2- 172.21.15.38:0/2709050703 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3a240a2cb0 0x7f3a240a51c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009550 tx=0x7f3a200091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:40.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.019+0000 7f3a34ff9700 1 -- 172.21.15.38:0/2709050703 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3a30014030 con 0x7f3a38102340 2024-05-21T15:33:40.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:39 smithi053 ceph-mon[36496]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:33:40.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.227+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2709050703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f3a3802dc60 con 0x7f3a38102340 2024-05-21T15:33:40.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.228+0000 7f3a34ff9700 1 -- 172.21.15.38:0/2709050703 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f3a3009c050 con 0x7f3a38102340 2024-05-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi038.stdout:313532612638 2024-05-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.230+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2709050703 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3a240a2cb0 msgr2=0x7f3a240a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.230+0000 7f3a3dd41700 1 --2- 172.21.15.38:0/2709050703 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3a240a2cb0 0x7f3a240a51c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009550 tx=0x7f3a200091b0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.230+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2709050703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38102340 msgr2=0x7f3a3819f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.230+0000 7f3a3dd41700 1 --2- 172.21.15.38:0/2709050703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38102340 0x7f3a3819f9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c9b0 tx=0x7f3a3000ccc0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.230+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2709050703 shutdown_connections 2024-05-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.230+0000 7f3a3dd41700 1 --2- 172.21.15.38:0/2709050703 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3a240a2cb0 0x7f3a240a51c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.230+0000 7f3a3dd41700 1 --2- 172.21.15.38:0/2709050703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a381075c0 0x7f3a38199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.230+0000 7f3a3dd41700 1 --2- 172.21.15.38:0/2709050703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a38102340 0x7f3a3819f9a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.230+0000 7f3a3dd41700 1 --2- 172.21.15.38:0/2709050703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a38106c70 0x7f3a3819fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.230+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2709050703 >> 172.21.15.38:0/2709050703 conn(0x7f3a380fdc40 msgr2=0x7f3a3806a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.231+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2709050703 shutdown_connections 2024-05-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.231+0000 7f3a3dd41700 1 -- 172.21.15.38:0/2709050703 wait complete. 2024-05-21T15:33:40.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:39 smithi045 ceph-mon[36409]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:33:40.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.445+0000 7f7e2e991700 1 -- 172.21.15.38:0/328499374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2810d770 msgr2=0x7f7e2810fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:40.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.445+0000 7f7e2e991700 1 --2- 172.21.15.38:0/328499374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2810d770 0x7f7e2810fc50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000b0a0 tx=0x7f7e2000b3b0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.445+0000 7f7e2e991700 1 -- 172.21.15.38:0/328499374 shutdown_connections 2024-05-21T15:33:40.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.445+0000 7f7e2e991700 1 --2- 172.21.15.38:0/328499374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2810d770 0x7f7e2810fc50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.445+0000 7f7e2e991700 1 --2- 172.21.15.38:0/328499374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e28104340 0x7f7e28104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.445+0000 7f7e2e991700 1 --2- 172.21.15.38:0/328499374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e28104c90 0x7f7e2810d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.445+0000 7f7e2e991700 1 -- 172.21.15.38:0/328499374 >> 172.21.15.38:0/328499374 conn(0x7f7e280fdc40 msgr2=0x7f7e28100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:40.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.445+0000 7f7e2e991700 1 -- 172.21.15.38:0/328499374 shutdown_connections 2024-05-21T15:33:40.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.445+0000 7f7e2e991700 1 -- 172.21.15.38:0/328499374 wait complete. 2024-05-21T15:33:40.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.446+0000 7f7e2e991700 1 Processor -- start 2024-05-21T15:33:40.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.446+0000 7f7e2e991700 1 -- start start 2024-05-21T15:33:40.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e2e991700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e28104340 0x7f7e2819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:40.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e2e991700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e28104c90 0x7f7e2819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:40.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e2e991700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2810d770 0x7f7e28199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:40.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e2e991700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e28113420 con 0x7f7e28104c90 2024-05-21T15:33:40.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e2e991700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e281132a0 con 0x7f7e28104340 2024-05-21T15:33:40.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e2e991700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e281135a0 con 0x7f7e2810d770 2024-05-21T15:33:40.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e277fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e28104c90 0x7f7e2819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:40.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e277fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e28104c90 0x7f7e2819fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52958/0 (socket says 172.21.15.38:52958) 2024-05-21T15:33:40.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e277fe700 1 -- 172.21.15.38:0/110880785 learned_addr learned my addr 172.21.15.38:0/110880785 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:40.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e27fff700 1 --2- 172.21.15.38:0/110880785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e28104340 0x7f7e2819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:40.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e2cf2e700 1 --2- 172.21.15.38:0/110880785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2810d770 0x7f7e28199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:40.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e277fe700 1 -- 172.21.15.38:0/110880785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2810d770 msgr2=0x7f7e28199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:40.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e277fe700 1 --2- 172.21.15.38:0/110880785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2810d770 0x7f7e28199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e277fe700 1 -- 172.21.15.38:0/110880785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e28104340 msgr2=0x7f7e2819f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:40.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e277fe700 1 --2- 172.21.15.38:0/110880785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e28104340 0x7f7e2819f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.447+0000 7f7e277fe700 1 -- 172.21.15.38:0/110880785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2819a1e0 con 0x7f7e28104c90 2024-05-21T15:33:40.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.448+0000 7f7e277fe700 1 --2- 172.21.15.38:0/110880785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e28104c90 0x7f7e2819fe20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009ef0 tx=0x7f7e1000ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:40.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.448+0000 7f7e257fa700 1 -- 172.21.15.38:0/110880785 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1001e070 con 0x7f7e28104c90 2024-05-21T15:33:40.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.448+0000 7f7e2e991700 1 -- 172.21.15.38:0/110880785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2819a4d0 con 0x7f7e28104c90 2024-05-21T15:33:40.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.448+0000 7f7e257fa700 1 -- 172.21.15.38:0/110880785 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e10007c50 con 0x7f7e28104c90 2024-05-21T15:33:40.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.448+0000 7f7e257fa700 1 -- 172.21.15.38:0/110880785 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1000eb90 con 0x7f7e28104c90 2024-05-21T15:33:40.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.448+0000 7f7e2e991700 1 -- 172.21.15.38:0/110880785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e28101f60 con 0x7f7e28104c90 2024-05-21T15:33:40.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.449+0000 7f7e2e991700 1 -- 172.21.15.38:0/110880785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e28050170 con 0x7f7e28104c90 2024-05-21T15:33:40.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.450+0000 7f7e257fa700 1 -- 172.21.15.38:0/110880785 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7e10021070 con 0x7f7e28104c90 2024-05-21T15:33:40.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.451+0000 7f7e257fa700 1 --2- 172.21.15.38:0/110880785 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e140a2cb0 0x7f7e140a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:40.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.451+0000 7f7e27fff700 1 --2- 172.21.15.38:0/110880785 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e140a2cb0 0x7f7e140a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:40.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.451+0000 7f7e257fa700 1 -- 172.21.15.38:0/110880785 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7e100d24f0 con 0x7f7e28104c90 2024-05-21T15:33:40.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.452+0000 7f7e27fff700 1 --2- 172.21.15.38:0/110880785 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e140a2cb0 0x7f7e140a51c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7e2819ac40 tx=0x7f7e180073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:40.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.456+0000 7f7e257fa700 1 -- 172.21.15.38:0/110880785 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7e1009f280 con 0x7f7e28104c90 2024-05-21T15:33:40.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.664+0000 7f7e2e991700 1 -- 172.21.15.38:0/110880785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7e28102220 con 0x7f7e28104c90 2024-05-21T15:33:40.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.665+0000 7f7e257fa700 1 -- 172.21.15.38:0/110880785 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7e10007e00 con 0x7f7e28104c90 2024-05-21T15:33:40.665 INFO:teuthology.orchestra.run.smithi038.stdout:167503724601 2024-05-21T15:33:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.667+0000 7f7e2e991700 1 -- 172.21.15.38:0/110880785 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e140a2cb0 msgr2=0x7f7e140a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.668+0000 7f7e2e991700 1 --2- 172.21.15.38:0/110880785 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e140a2cb0 0x7f7e140a51c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7e2819ac40 tx=0x7f7e180073d0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.668+0000 7f7e2e991700 1 -- 172.21.15.38:0/110880785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e28104c90 msgr2=0x7f7e2819fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.668+0000 7f7e2e991700 1 --2- 172.21.15.38:0/110880785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e28104c90 0x7f7e2819fe20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009ef0 tx=0x7f7e1000ff50 comp rx=0 tx=0).stop 2024-05-21T15:33:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.668+0000 7f7e2e991700 1 -- 172.21.15.38:0/110880785 shutdown_connections 2024-05-21T15:33:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.668+0000 7f7e2e991700 1 --2- 172.21.15.38:0/110880785 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e140a2cb0 0x7f7e140a51c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.668+0000 7f7e2e991700 1 --2- 172.21.15.38:0/110880785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2810d770 0x7f7e28199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.668+0000 7f7e2e991700 1 --2- 172.21.15.38:0/110880785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e28104340 0x7f7e2819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.668+0000 7f7e2e991700 1 --2- 172.21.15.38:0/110880785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e28104c90 0x7f7e2819fe20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.668+0000 7f7e2e991700 1 -- 172.21.15.38:0/110880785 >> 172.21.15.38:0/110880785 conn(0x7f7e280fdc40 msgr2=0x7f7e280fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.668+0000 7f7e2e991700 1 -- 172.21.15.38:0/110880785 shutdown_connections 2024-05-21T15:33:40.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:40.668+0000 7f7e2e991700 1 -- 172.21.15.38:0/110880785 wait complete. 2024-05-21T15:33:41.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:40 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2709050703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-21T15:33:41.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:40 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/110880785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-21T15:33:41.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:40 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2709050703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-21T15:33:41.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:40 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/110880785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-21T15:33:41.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:40 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2709050703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-21T15:33:41.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:40 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/110880785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-21T15:33:41.488 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724596 got 167503724601 for osd.5 2024-05-21T15:33:41.488 DEBUG:teuthology.parallel:result is None 2024-05-21T15:33:41.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:41 smithi038 ceph-mon[31995]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:33:42.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:41 smithi053 ceph-mon[36496]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:33:42.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:41 smithi045 ceph-mon[36409]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:33:42.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.892+0000 7fa22da91700 1 -- 172.21.15.38:0/4183406561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa228107080 msgr2=0x7fa228107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:42.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.892+0000 7fa22da91700 1 --2- 172.21.15.38:0/4183406561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa228107080 0x7fa228107490 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa218009990 tx=0x7fa218009ca0 comp rx=0 tx=0).stop 2024-05-21T15:33:42.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.892+0000 7fa22da91700 1 -- 172.21.15.38:0/4183406561 shutdown_connections 2024-05-21T15:33:42.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.892+0000 7fa22da91700 1 --2- 172.21.15.38:0/4183406561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa228103130 0x7fa22810eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:42.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.892+0000 7fa22da91700 1 --2- 172.21.15.38:0/4183406561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa228107080 0x7fa228107490 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:42.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.892+0000 7fa22da91700 1 --2- 172.21.15.38:0/4183406561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2281027e0 0x7fa228102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:42.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.892+0000 7fa22da91700 1 -- 172.21.15.38:0/4183406561 >> 172.21.15.38:0/4183406561 conn(0x7fa2280fd360 msgr2=0x7fa2280ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:42.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.892+0000 7fa22da91700 1 -- 172.21.15.38:0/4183406561 shutdown_connections 2024-05-21T15:33:42.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.892+0000 7fa22da91700 1 -- 172.21.15.38:0/4183406561 wait complete. 2024-05-21T15:33:42.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.893+0000 7fa22da91700 1 Processor -- start 2024-05-21T15:33:42.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.893+0000 7fa22da91700 1 -- start start 2024-05-21T15:33:42.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.893+0000 7fa22da91700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2281027e0 0x7fa228199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:42.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.893+0000 7fa22da91700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa228103130 0x7fa22819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:42.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.893+0000 7fa22da91700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa228107080 0x7fa22819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:42.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.893+0000 7fa22da91700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa228113510 con 0x7fa2281027e0 2024-05-21T15:33:42.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.893+0000 7fa22da91700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa228113390 con 0x7fa228107080 2024-05-21T15:33:42.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.893+0000 7fa22da91700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa228113690 con 0x7fa228103130 2024-05-21T15:33:42.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa2277fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2281027e0 0x7fa228199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:42.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa2277fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2281027e0 0x7fa228199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52978/0 (socket says 172.21.15.38:52978) 2024-05-21T15:33:42.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa2277fe700 1 -- 172.21.15.38:0/280042524 learned_addr learned my addr 172.21.15.38:0/280042524 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:42.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa226ffd700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa228103130 0x7fa22819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:42.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa227fff700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa228107080 0x7fa22819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:42.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa2277fe700 1 -- 172.21.15.38:0/280042524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa228103130 msgr2=0x7fa22819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:42.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa2277fe700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa228103130 0x7fa22819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:42.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa2277fe700 1 -- 172.21.15.38:0/280042524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa228107080 msgr2=0x7fa22819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:42.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa2277fe700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa228107080 0x7fa22819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:42.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa2277fe700 1 -- 172.21.15.38:0/280042524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22819ec90 con 0x7fa2281027e0 2024-05-21T15:33:42.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa226ffd700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa228103130 0x7fa22819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:33:42.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa227fff700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa228107080 0x7fa22819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:33:42.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa2277fe700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2281027e0 0x7fa228199dc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa2180098e0 tx=0x7fa218009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:42.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa224ff9700 1 -- 172.21.15.38:0/280042524 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218027070 con 0x7fa2281027e0 2024-05-21T15:33:42.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa224ff9700 1 -- 172.21.15.38:0/280042524 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa218004050 con 0x7fa2281027e0 2024-05-21T15:33:42.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa224ff9700 1 -- 172.21.15.38:0/280042524 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218022d30 con 0x7fa2281027e0 2024-05-21T15:33:42.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa22da91700 1 -- 172.21.15.38:0/280042524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22819ef20 con 0x7fa2281027e0 2024-05-21T15:33:42.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.894+0000 7fa22da91700 1 -- 172.21.15.38:0/280042524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2281a68a0 con 0x7fa2281027e0 2024-05-21T15:33:42.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.895+0000 7fa22da91700 1 -- 172.21.15.38:0/280042524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa22810a620 con 0x7fa2281027e0 2024-05-21T15:33:42.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.897+0000 7fa224ff9700 1 -- 172.21.15.38:0/280042524 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa21802a070 con 0x7fa2281027e0 2024-05-21T15:33:42.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.897+0000 7fa224ff9700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2140a2cb0 0x7fa2140a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:42.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.898+0000 7fa226ffd700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2140a2cb0 0x7fa2140a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:42.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.898+0000 7fa224ff9700 1 -- 172.21.15.38:0/280042524 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fa2180d0950 con 0x7fa2281027e0 2024-05-21T15:33:42.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.898+0000 7fa226ffd700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2140a2cb0 0x7fa2140a51c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa2281065a0 tx=0x7fa210008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:42.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:42.901+0000 7fa224ff9700 1 -- 172.21.15.38:0/280042524 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa218031080 con 0x7fa2281027e0 2024-05-21T15:33:43.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.105+0000 7f4f4a472700 1 -- 172.21.15.38:0/343145582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f44103460 msgr2=0x7f4f44103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:43.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.105+0000 7f4f4a472700 1 --2- 172.21.15.38:0/343145582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f44103460 0x7f4f44103870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4f340099c0 tx=0x7f4f34009cd0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.105+0000 7f4f4a472700 1 -- 172.21.15.38:0/343145582 shutdown_connections 2024-05-21T15:33:43.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.105+0000 7f4f4a472700 1 --2- 172.21.15.38:0/343145582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f441086e0 0x7f4f44110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.105+0000 7f4f4a472700 1 --2- 172.21.15.38:0/343145582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f44103460 0x7f4f44103870 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.105+0000 7f4f4a472700 1 --2- 172.21.15.38:0/343145582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f44107d90 0x7f4f441081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.105+0000 7f4f4a472700 1 -- 172.21.15.38:0/343145582 >> 172.21.15.38:0/343145582 conn(0x7f4f440fecf0 msgr2=0x7f4f44101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:43.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.105+0000 7f4f4a472700 1 -- 172.21.15.38:0/343145582 shutdown_connections 2024-05-21T15:33:43.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.105+0000 7f4f4a472700 1 -- 172.21.15.38:0/343145582 wait complete. 2024-05-21T15:33:43.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.106+0000 7f4f4a472700 1 Processor -- start 2024-05-21T15:33:43.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.106+0000 7f4f4a472700 1 -- start start 2024-05-21T15:33:43.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f4a472700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f44103460 0x7f4f44199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:43.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f4a472700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f44107d90 0x7f4f4419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:43.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f4a472700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f441086e0 0x7f4f4419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:43.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f4a472700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4f44113460 con 0x7f4f441086e0 2024-05-21T15:33:43.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f4a472700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441132e0 con 0x7f4f44103460 2024-05-21T15:33:43.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f4a472700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441135e0 con 0x7f4f44107d90 2024-05-21T15:33:43.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f48a0f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f441086e0 0x7f4f4419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:43.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f48a0f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f441086e0 0x7f4f4419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53002/0 (socket says 172.21.15.38:53002) 2024-05-21T15:33:43.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f48a0f700 1 -- 172.21.15.38:0/625576747 learned_addr learned my addr 172.21.15.38:0/625576747 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:43.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f437fe700 1 --2- 172.21.15.38:0/625576747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f44107d90 0x7f4f4419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:43.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f43fff700 1 --2- 172.21.15.38:0/625576747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f44103460 0x7f4f44199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:43.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f48a0f700 1 -- 172.21.15.38:0/625576747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f44107d90 msgr2=0x7f4f4419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:43.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f48a0f700 1 --2- 172.21.15.38:0/625576747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f44107d90 0x7f4f4419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f48a0f700 1 -- 172.21.15.38:0/625576747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f44103460 msgr2=0x7f4f44199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:43.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f48a0f700 1 --2- 172.21.15.38:0/625576747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f44103460 0x7f4f44199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f48a0f700 1 -- 172.21.15.38:0/625576747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4419ec50 con 0x7f4f441086e0 2024-05-21T15:33:43.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.107+0000 7f4f48a0f700 1 --2- 172.21.15.38:0/625576747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f441086e0 0x7f4f4419e5b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00ef30 tx=0x7f4f3c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:43.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.108+0000 7f4f417fa700 1 -- 172.21.15.38:0/625576747 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0092e0 con 0x7f4f441086e0 2024-05-21T15:33:43.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.108+0000 7f4f4a472700 1 -- 172.21.15.38:0/625576747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4419eee0 con 0x7f4f441086e0 2024-05-21T15:33:43.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.108+0000 7f4f417fa700 1 -- 172.21.15.38:0/625576747 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f3c004bb0 con 0x7f4f441086e0 2024-05-21T15:33:43.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.108+0000 7f4f417fa700 1 -- 172.21.15.38:0/625576747 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c004280 con 0x7f4f441086e0 2024-05-21T15:33:43.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.108+0000 7f4f4a472700 1 -- 172.21.15.38:0/625576747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f441a6860 con 0x7f4f441086e0 2024-05-21T15:33:43.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.109+0000 7f4f4a472700 1 -- 172.21.15.38:0/625576747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f44050170 con 0x7f4f441086e0 2024-05-21T15:33:43.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.111+0000 7f4f417fa700 1 -- 172.21.15.38:0/625576747 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4f3c01b030 con 0x7f4f441086e0 2024-05-21T15:33:43.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.112+0000 7fa22da91700 1 -- 172.21.15.38:0/280042524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa22819b120 con 0x7fa2281027e0 2024-05-21T15:33:43.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.112+0000 7f4f417fa700 1 --2- 172.21.15.38:0/625576747 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f2c0a2cb0 0x7f4f2c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:43.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.112+0000 7f4f43fff700 1 --2- 172.21.15.38:0/625576747 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f2c0a2cb0 0x7f4f2c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:43.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.112+0000 7f4f417fa700 1 -- 172.21.15.38:0/625576747 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f4f3c0cf610 con 0x7f4f441086e0 2024-05-21T15:33:43.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.112+0000 7f4f43fff700 1 --2- 172.21.15.38:0/625576747 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f2c0a2cb0 0x7f4f2c0a51c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f34003800 tx=0x7f4f34005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:43.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.116+0000 7fa224ff9700 1 -- 172.21.15.38:0/280042524 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa218013070 con 0x7fa2281027e0 2024-05-21T15:33:43.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.116+0000 7f4f417fa700 1 -- 172.21.15.38:0/625576747 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4f3c022020 con 0x7f4f441086e0 2024-05-21T15:33:43.175 INFO:teuthology.orchestra.run.smithi038.stdout:34359738452 2024-05-21T15:33:43.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.118+0000 7fa22da91700 1 -- 172.21.15.38:0/280042524 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2140a2cb0 msgr2=0x7fa2140a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:43.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.118+0000 7fa22da91700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2140a2cb0 0x7fa2140a51c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa2281065a0 tx=0x7fa210008040 comp rx=0 tx=0).stop 2024-05-21T15:33:43.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.118+0000 7fa22da91700 1 -- 172.21.15.38:0/280042524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2281027e0 msgr2=0x7fa228199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.118+0000 7fa22da91700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2281027e0 0x7fa228199dc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa2180098e0 tx=0x7fa218009f90 comp rx=0 tx=0).stop 2024-05-21T15:33:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.118+0000 7fa22da91700 1 -- 172.21.15.38:0/280042524 shutdown_connections 2024-05-21T15:33:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.118+0000 7fa22da91700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2140a2cb0 0x7fa2140a51c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.118+0000 7fa22da91700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa228107080 0x7fa22819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.118+0000 7fa22da91700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2281027e0 0x7fa228199dc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.118+0000 7fa22da91700 1 --2- 172.21.15.38:0/280042524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa228103130 0x7fa22819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.118+0000 7fa22da91700 1 -- 172.21.15.38:0/280042524 >> 172.21.15.38:0/280042524 conn(0x7fa2280fd360 msgr2=0x7fa228104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.119+0000 7fa22da91700 1 -- 172.21.15.38:0/280042524 shutdown_connections 2024-05-21T15:33:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.119+0000 7fa22da91700 1 -- 172.21.15.38:0/280042524 wait complete. 2024-05-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.326+0000 7f4f4a472700 1 -- 172.21.15.38:0/625576747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4f4402dc60 con 0x7f4f441086e0 2024-05-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.326+0000 7f4f417fa700 1 -- 172.21.15.38:0/625576747 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4f3c010040 con 0x7f4f441086e0 2024-05-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi038.stdout:223338299439 2024-05-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.328+0000 7f4f4a472700 1 -- 172.21.15.38:0/625576747 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f2c0a2cb0 msgr2=0x7f4f2c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.328+0000 7f4f4a472700 1 --2- 172.21.15.38:0/625576747 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f2c0a2cb0 0x7f4f2c0a51c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f34003800 tx=0x7f4f34005fb0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.328+0000 7f4f4a472700 1 -- 172.21.15.38:0/625576747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f441086e0 msgr2=0x7f4f4419e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.328+0000 7f4f4a472700 1 --2- 172.21.15.38:0/625576747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f441086e0 0x7f4f4419e5b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00ef30 tx=0x7f4f3c00c3a0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.328+0000 7f4f4a472700 1 -- 172.21.15.38:0/625576747 shutdown_connections 2024-05-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.328+0000 7f4f4a472700 1 --2- 172.21.15.38:0/625576747 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4f2c0a2cb0 0x7f4f2c0a51c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.328+0000 7f4f4a472700 1 --2- 172.21.15.38:0/625576747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f441086e0 0x7f4f4419e5b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.328+0000 7f4f4a472700 1 --2- 172.21.15.38:0/625576747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f44103460 0x7f4f44199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.328+0000 7f4f4a472700 1 --2- 172.21.15.38:0/625576747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f44107d90 0x7f4f4419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.328+0000 7f4f4a472700 1 -- 172.21.15.38:0/625576747 >> 172.21.15.38:0/625576747 conn(0x7f4f440fecf0 msgr2=0x7f4f44107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.329+0000 7f4f4a472700 1 -- 172.21.15.38:0/625576747 shutdown_connections 2024-05-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:43.329+0000 7f4f4a472700 1 -- 172.21.15.38:0/625576747 wait complete. 2024-05-21T15:33:44.014 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:43 smithi038 ceph-mon[31995]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:33:44.014 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:43 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/280042524' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-21T15:33:44.014 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:43 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/625576747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-21T15:33:44.044 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299435 got 223338299439 for osd.7 2024-05-21T15:33:44.044 DEBUG:teuthology.parallel:result is None 2024-05-21T15:33:44.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:43 smithi053 ceph-mon[36496]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:33:44.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:43 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/280042524' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-21T15:33:44.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:43 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/625576747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-21T15:33:44.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:43 smithi045 ceph-mon[36409]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:33:44.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:43 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/280042524' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-21T15:33:44.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:43 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/625576747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-21T15:33:44.725 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612631 got 313532612638 for osd.10 2024-05-21T15:33:44.725 DEBUG:teuthology.parallel:result is None 2024-05-21T15:33:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:44.986 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:45.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.002+0000 7f88881c8700 1 -- 172.21.15.38:0/787086702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88801075c0 msgr2=0x7f888010ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:45.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.002+0000 7f88881c8700 1 --2- 172.21.15.38:0/787086702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88801075c0 0x7f888010ef20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f887800b0d0 tx=0x7f887800b3e0 comp rx=0 tx=0).stop 2024-05-21T15:33:45.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.002+0000 7f88881c8700 1 -- 172.21.15.38:0/787086702 shutdown_connections 2024-05-21T15:33:45.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.002+0000 7f88881c8700 1 --2- 172.21.15.38:0/787086702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88801075c0 0x7f888010ef20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:45.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.002+0000 7f88881c8700 1 --2- 172.21.15.38:0/787086702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8880102340 0x7f8880102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:45.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.002+0000 7f88881c8700 1 --2- 172.21.15.38:0/787086702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8880106c70 0x7f8880107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:45.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.002+0000 7f88881c8700 1 -- 172.21.15.38:0/787086702 >> 172.21.15.38:0/787086702 conn(0x7f88800fdc40 msgr2=0x7f8880100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:45.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.002+0000 7f88881c8700 1 -- 172.21.15.38:0/787086702 shutdown_connections 2024-05-21T15:33:45.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.003+0000 7f88881c8700 1 -- 172.21.15.38:0/787086702 wait complete. 2024-05-21T15:33:45.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.004+0000 7f88881c8700 1 Processor -- start 2024-05-21T15:33:45.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.004+0000 7f88881c8700 1 -- start start 2024-05-21T15:33:45.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.004+0000 7f88881c8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8880102340 0x7f888019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:45.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.004+0000 7f88881c8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8880106c70 0x7f888019b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:45.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.004+0000 7f88881c8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88801075c0 0x7f888019b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:45.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.004+0000 7f88881c8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8880112340 con 0x7f8880102340 2024-05-21T15:33:45.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.004+0000 7f88881c8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f88801121c0 con 0x7f8880106c70 2024-05-21T15:33:45.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.004+0000 7f88881c8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f88801124c0 con 0x7f88801075c0 2024-05-21T15:33:45.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.004+0000 7f8885f64700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8880102340 0x7f888019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:45.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.004+0000 7f8885f64700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8880102340 0x7f888019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40658/0 (socket says 172.21.15.38:40658) 2024-05-21T15:33:45.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.004+0000 7f8885f64700 1 -- 172.21.15.38:0/1655492672 learned_addr learned my addr 172.21.15.38:0/1655492672 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.005+0000 7f8886765700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88801075c0 0x7f888019b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.005+0000 7f8885763700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8880106c70 0x7f888019b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.005+0000 7f8885f64700 1 -- 172.21.15.38:0/1655492672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88801075c0 msgr2=0x7f888019b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.005+0000 7f8885f64700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88801075c0 0x7f888019b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.005+0000 7f8885f64700 1 -- 172.21.15.38:0/1655492672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8880106c70 msgr2=0x7f888019b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.005+0000 7f8885f64700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8880106c70 0x7f888019b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.005+0000 7f8885f64700 1 -- 172.21.15.38:0/1655492672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88801a5ed0 con 0x7f8880102340 2024-05-21T15:33:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.005+0000 7f8886765700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88801075c0 0x7f888019b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:33:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.005+0000 7f8885f64700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8880102340 0x7f888019ab80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f887c012db0 tx=0x7f887c00c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.005+0000 7f8876ffd700 1 -- 172.21.15.38:0/1655492672 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c01f020 con 0x7f8880102340 2024-05-21T15:33:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.006+0000 7f8876ffd700 1 -- 172.21.15.38:0/1655492672 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f887c0125b0 con 0x7f8880102340 2024-05-21T15:33:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.006+0000 7f8876ffd700 1 -- 172.21.15.38:0/1655492672 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c00f7d0 con 0x7f8880102340 2024-05-21T15:33:45.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.006+0000 7f88881c8700 1 -- 172.21.15.38:0/1655492672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88801989d0 con 0x7f8880102340 2024-05-21T15:33:45.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.006+0000 7f88881c8700 1 -- 172.21.15.38:0/1655492672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8880198ec0 con 0x7f8880102340 2024-05-21T15:33:45.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.007+0000 7f88881c8700 1 -- 172.21.15.38:0/1655492672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8880050170 con 0x7f8880102340 2024-05-21T15:33:45.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.010+0000 7f8876ffd700 1 -- 172.21.15.38:0/1655492672 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f887c022020 con 0x7f8880102340 2024-05-21T15:33:45.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.011+0000 7f8876ffd700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f886c0a2cb0 0x7f886c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:45.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.011+0000 7f8876ffd700 1 -- 172.21.15.38:0/1655492672 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f887c0d7e20 con 0x7f8880102340 2024-05-21T15:33:45.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.012+0000 7f8885763700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f886c0a2cb0 0x7f886c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:45.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.012+0000 7f8885763700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f886c0a2cb0 0x7f886c0a51c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8880101770 tx=0x7f887000d4b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:45.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.016+0000 7f8876ffd700 1 -- 172.21.15.38:0/1655492672 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f887c029020 con 0x7f8880102340 2024-05-21T15:33:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.226+0000 7f88881c8700 1 -- 172.21.15.38:0/1655492672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f888002dc60 con 0x7f8880102340 2024-05-21T15:33:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.227+0000 7f8876ffd700 1 -- 172.21.15.38:0/1655492672 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f887c0a4c10 con 0x7f8880102340 2024-05-21T15:33:45.227 INFO:teuthology.orchestra.run.smithi038.stdout:201863462965 2024-05-21T15:33:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.229+0000 7f88881c8700 1 -- 172.21.15.38:0/1655492672 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f886c0a2cb0 msgr2=0x7f886c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.229+0000 7f88881c8700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f886c0a2cb0 0x7f886c0a51c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8880101770 tx=0x7f887000d4b0 comp rx=0 tx=0).stop 2024-05-21T15:33:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.229+0000 7f88881c8700 1 -- 172.21.15.38:0/1655492672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8880102340 msgr2=0x7f888019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.229+0000 7f88881c8700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8880102340 0x7f888019ab80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f887c012db0 tx=0x7f887c00c5b0 comp rx=0 tx=0).stop 2024-05-21T15:33:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.229+0000 7f88881c8700 1 -- 172.21.15.38:0/1655492672 shutdown_connections 2024-05-21T15:33:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.229+0000 7f88881c8700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f886c0a2cb0 0x7f886c0a51c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.229+0000 7f88881c8700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88801075c0 0x7f888019b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:45.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.229+0000 7f88881c8700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8880102340 0x7f888019ab80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:45.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.229+0000 7f88881c8700 1 --2- 172.21.15.38:0/1655492672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8880106c70 0x7f888019b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:45.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.229+0000 7f88881c8700 1 -- 172.21.15.38:0/1655492672 >> 172.21.15.38:0/1655492672 conn(0x7f88800fdc40 msgr2=0x7f88801063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:45.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.229+0000 7f88881c8700 1 -- 172.21.15.38:0/1655492672 shutdown_connections 2024-05-21T15:33:45.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:45.229+0000 7f88881c8700 1 -- 172.21.15.38:0/1655492672 wait complete. 2024-05-21T15:33:45.429 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738452 for osd.0 2024-05-21T15:33:45.429 DEBUG:teuthology.parallel:result is None 2024-05-21T15:33:46.009 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:45 smithi038 ceph-mon[31995]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:33:46.009 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:45 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1655492672' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-21T15:33:46.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:45 smithi053 ceph-mon[36496]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:33:46.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:45 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1655492672' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-21T15:33:46.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:45 smithi045 ceph-mon[36409]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:33:46.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:45 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1655492672' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-21T15:33:46.979 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462959 got 201863462965 for osd.6 2024-05-21T15:33:46.980 DEBUG:teuthology.parallel:result is None 2024-05-21T15:33:48.103 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:47 smithi038 ceph-mon[31995]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:33:48.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:47 smithi053 ceph-mon[36496]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:33:48.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:47 smithi045 ceph-mon[36409]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:33:49.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.024+0000 7f40dfec9700 1 -- 172.21.15.38:0/684004793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d8107d80 msgr2=0x7f40d8108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:49.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.024+0000 7f40dfec9700 1 --2- 172.21.15.38:0/684004793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d8107d80 0x7f40d8108190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f40c80099c0 tx=0x7f40c8009cd0 comp rx=0 tx=0).stop 2024-05-21T15:33:49.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.024+0000 7f40dfec9700 1 -- 172.21.15.38:0/684004793 shutdown_connections 2024-05-21T15:33:49.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.024+0000 7f40dfec9700 1 --2- 172.21.15.38:0/684004793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40d81086d0 0x7f40d8110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:49.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.024+0000 7f40dfec9700 1 --2- 172.21.15.38:0/684004793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40d8103450 0x7f40d8103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:49.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.024+0000 7f40dfec9700 1 --2- 172.21.15.38:0/684004793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d8107d80 0x7f40d8108190 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:49.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.024+0000 7f40dfec9700 1 -- 172.21.15.38:0/684004793 >> 172.21.15.38:0/684004793 conn(0x7f40d80fed20 msgr2=0x7f40d8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:49.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.024+0000 7f40dfec9700 1 -- 172.21.15.38:0/684004793 shutdown_connections 2024-05-21T15:33:49.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.024+0000 7f40dfec9700 1 -- 172.21.15.38:0/684004793 wait complete. 2024-05-21T15:33:49.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40dfec9700 1 Processor -- start 2024-05-21T15:33:49.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40dfec9700 1 -- start start 2024-05-21T15:33:49.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40dfec9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40d8103450 0x7f40d8199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:49.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40dfec9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40d8107d80 0x7f40d819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:49.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40dfec9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d81086d0 0x7f40d819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:49.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40dfec9700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f40d8113430 con 0x7f40d81086d0 2024-05-21T15:33:49.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40dfec9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f40d81132b0 con 0x7f40d8107d80 2024-05-21T15:33:49.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40dfec9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f40d81135b0 con 0x7f40d8103450 2024-05-21T15:33:49.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40de466700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d81086d0 0x7f40d819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:49.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40de466700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d81086d0 0x7f40d819e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40678/0 (socket says 172.21.15.38:40678) 2024-05-21T15:33:49.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40de466700 1 -- 172.21.15.38:0/471770803 learned_addr learned my addr 172.21.15.38:0/471770803 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:49.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40ddc65700 1 --2- 172.21.15.38:0/471770803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40d8103450 0x7f40d8199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:49.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40dd464700 1 --2- 172.21.15.38:0/471770803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40d8107d80 0x7f40d819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:49.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40de466700 1 -- 172.21.15.38:0/471770803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40d8103450 msgr2=0x7f40d8199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:49.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40de466700 1 --2- 172.21.15.38:0/471770803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40d8103450 0x7f40d8199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:49.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40de466700 1 -- 172.21.15.38:0/471770803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40d8107d80 msgr2=0x7f40d819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:49.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40de466700 1 --2- 172.21.15.38:0/471770803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40d8107d80 0x7f40d819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:49.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.025+0000 7f40de466700 1 -- 172.21.15.38:0/471770803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d819ec30 con 0x7f40d81086d0 2024-05-21T15:33:49.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.026+0000 7f40de466700 1 --2- 172.21.15.38:0/471770803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d81086d0 0x7f40d819e590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f40d0007ee0 tx=0x7f40d000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:49.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.026+0000 7f40ceffd700 1 -- 172.21.15.38:0/471770803 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d000dd20 con 0x7f40d81086d0 2024-05-21T15:33:49.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.026+0000 7f40dfec9700 1 -- 172.21.15.38:0/471770803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d819ef20 con 0x7f40d81086d0 2024-05-21T15:33:49.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.026+0000 7f40ceffd700 1 -- 172.21.15.38:0/471770803 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40d0004020 con 0x7f40d81086d0 2024-05-21T15:33:49.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.026+0000 7f40ceffd700 1 -- 172.21.15.38:0/471770803 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d0015660 con 0x7f40d81086d0 2024-05-21T15:33:49.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.026+0000 7f40dfec9700 1 -- 172.21.15.38:0/471770803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d81a6840 con 0x7f40d81086d0 2024-05-21T15:33:49.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.027+0000 7f40dfec9700 1 -- 172.21.15.38:0/471770803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d8050170 con 0x7f40d81086d0 2024-05-21T15:33:49.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.028+0000 7f40ceffd700 1 -- 172.21.15.38:0/471770803 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f40d00041d0 con 0x7f40d81086d0 2024-05-21T15:33:49.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.029+0000 7f40ceffd700 1 --2- 172.21.15.38:0/471770803 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f40c40a2d30 0x7f40c40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:49.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.029+0000 7f40ddc65700 1 --2- 172.21.15.38:0/471770803 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f40c40a2d30 0x7f40c40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:49.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.029+0000 7f40ceffd700 1 -- 172.21.15.38:0/471770803 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f40d00ceb50 con 0x7f40d81086d0 2024-05-21T15:33:49.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.030+0000 7f40ddc65700 1 --2- 172.21.15.38:0/471770803 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f40c40a2d30 0x7f40c40a5240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f40d4005950 tx=0x7f40d400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:49.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.034+0000 7f40ceffd700 1 -- 172.21.15.38:0/471770803 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f40d0010070 con 0x7f40d81086d0 2024-05-21T15:33:49.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.243+0000 7f40dfec9700 1 -- 172.21.15.38:0/471770803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f40d802dc60 con 0x7f40d81086d0 2024-05-21T15:33:49.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.243+0000 7f40ceffd700 1 -- 172.21.15.38:0/471770803 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f40d009b940 con 0x7f40d81086d0 2024-05-21T15:33:49.244 INFO:teuthology.orchestra.run.smithi038.stdout:55834574928 2024-05-21T15:33:49.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.246+0000 7f40dfec9700 1 -- 172.21.15.38:0/471770803 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f40c40a2d30 msgr2=0x7f40c40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:49.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.246+0000 7f40dfec9700 1 --2- 172.21.15.38:0/471770803 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f40c40a2d30 0x7f40c40a5240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f40d4005950 tx=0x7f40d400b410 comp rx=0 tx=0).stop 2024-05-21T15:33:49.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.246+0000 7f40dfec9700 1 -- 172.21.15.38:0/471770803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d81086d0 msgr2=0x7f40d819e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:49.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.246+0000 7f40dfec9700 1 --2- 172.21.15.38:0/471770803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d81086d0 0x7f40d819e590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f40d0007ee0 tx=0x7f40d000d3b0 comp rx=0 tx=0).stop 2024-05-21T15:33:49.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.246+0000 7f40dfec9700 1 -- 172.21.15.38:0/471770803 shutdown_connections 2024-05-21T15:33:49.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.246+0000 7f40dfec9700 1 --2- 172.21.15.38:0/471770803 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f40c40a2d30 0x7f40c40a5240 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:49.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.246+0000 7f40dfec9700 1 --2- 172.21.15.38:0/471770803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40d81086d0 0x7f40d819e590 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:49.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.246+0000 7f40dfec9700 1 --2- 172.21.15.38:0/471770803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40d8103450 0x7f40d8199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:49.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.246+0000 7f40dfec9700 1 --2- 172.21.15.38:0/471770803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40d8107d80 0x7f40d819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:49.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.246+0000 7f40dfec9700 1 -- 172.21.15.38:0/471770803 >> 172.21.15.38:0/471770803 conn(0x7f40d80fed20 msgr2=0x7f40d8109630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:49.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.246+0000 7f40dfec9700 1 -- 172.21.15.38:0/471770803 shutdown_connections 2024-05-21T15:33:49.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:49.246+0000 7f40dfec9700 1 -- 172.21.15.38:0/471770803 wait complete. 2024-05-21T15:33:49.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:49 smithi038 ceph-mon[31995]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:33:49.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:49 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/471770803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-21T15:33:50.015 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574923 got 55834574928 for osd.1 2024-05-21T15:33:50.016 DEBUG:teuthology.parallel:result is None 2024-05-21T15:33:50.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:49 smithi053 ceph-mon[36496]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:33:50.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:49 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/471770803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-21T15:33:50.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:49 smithi045 ceph-mon[36409]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:33:50.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:49 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/471770803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-21T15:33:50.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.506+0000 7fa7954f8700 1 -- 172.21.15.38:0/1891605690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa79010d790 msgr2=0x7fa79010fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:50.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.506+0000 7fa7954f8700 1 --2- 172.21.15.38:0/1891605690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa79010d790 0x7fa79010fc70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa78800b0a0 tx=0x7fa78800b3b0 comp rx=0 tx=0).stop 2024-05-21T15:33:50.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.506+0000 7fa7954f8700 1 -- 172.21.15.38:0/1891605690 shutdown_connections 2024-05-21T15:33:50.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.506+0000 7fa7954f8700 1 --2- 172.21.15.38:0/1891605690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa79010d790 0x7fa79010fc70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:50.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.506+0000 7fa7954f8700 1 --2- 172.21.15.38:0/1891605690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa790104360 0x7fa790104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:50.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.506+0000 7fa7954f8700 1 --2- 172.21.15.38:0/1891605690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa790104cb0 0x7fa79010d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:50.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.506+0000 7fa7954f8700 1 -- 172.21.15.38:0/1891605690 >> 172.21.15.38:0/1891605690 conn(0x7fa7900fddb0 msgr2=0x7fa7901001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:50.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.506+0000 7fa7954f8700 1 -- 172.21.15.38:0/1891605690 shutdown_connections 2024-05-21T15:33:50.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.506+0000 7fa7954f8700 1 -- 172.21.15.38:0/1891605690 wait complete. 2024-05-21T15:33:50.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.506+0000 7fa7954f8700 1 Processor -- start 2024-05-21T15:33:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.507+0000 7fa7954f8700 1 -- start start 2024-05-21T15:33:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.507+0000 7fa7954f8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa790104360 0x7fa790199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.507+0000 7fa7954f8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa790104cb0 0x7fa79019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.507+0000 7fa7954f8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa79010d790 0x7fa79019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.507+0000 7fa7954f8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa790113460 con 0x7fa790104360 2024-05-21T15:33:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.507+0000 7fa7954f8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901132e0 con 0x7fa79010d790 2024-05-21T15:33:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.507+0000 7fa7954f8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901135e0 con 0x7fa790104cb0 2024-05-21T15:33:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.507+0000 7fa78effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa790104360 0x7fa790199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.507+0000 7fa78effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa790104360 0x7fa790199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40694/0 (socket says 172.21.15.38:40694) 2024-05-21T15:33:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.507+0000 7fa78effd700 1 -- 172.21.15.38:0/3321619483 learned_addr learned my addr 172.21.15.38:0/3321619483 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.507+0000 7fa78e7fc700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa790104cb0 0x7fa79019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.507+0000 7fa78f7fe700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa79010d790 0x7fa79019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:50.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.508+0000 7fa78effd700 1 -- 172.21.15.38:0/3321619483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa790104cb0 msgr2=0x7fa79019a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.508+0000 7fa78effd700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa790104cb0 0x7fa79019a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.508+0000 7fa78effd700 1 -- 172.21.15.38:0/3321619483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa79010d790 msgr2=0x7fa79019e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.508+0000 7fa78effd700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa79010d790 0x7fa79019e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.508+0000 7fa78effd700 1 -- 172.21.15.38:0/3321619483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79019ebb0 con 0x7fa790104360 2024-05-21T15:33:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.508+0000 7fa78f7fe700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa79010d790 0x7fa79019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:33:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.508+0000 7fa78effd700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa790104360 0x7fa790199d90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa78000c9b0 tx=0x7fa78000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.508+0000 7fa777fff700 1 -- 172.21.15.38:0/3321619483 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780007b00 con 0x7fa790104360 2024-05-21T15:33:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.508+0000 7fa7954f8700 1 -- 172.21.15.38:0/3321619483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79019eea0 con 0x7fa790104360 2024-05-21T15:33:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.508+0000 7fa7954f8700 1 -- 172.21.15.38:0/3321619483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7901a6870 con 0x7fa790104360 2024-05-21T15:33:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.508+0000 7fa777fff700 1 -- 172.21.15.38:0/3321619483 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa780004d10 con 0x7fa790104360 2024-05-21T15:33:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.508+0000 7fa777fff700 1 -- 172.21.15.38:0/3321619483 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780005650 con 0x7fa790104360 2024-05-21T15:33:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.509+0000 7fa7954f8700 1 -- 172.21.15.38:0/3321619483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa790050170 con 0x7fa790104360 2024-05-21T15:33:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.511+0000 7fa777fff700 1 -- 172.21.15.38:0/3321619483 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa780007ca0 con 0x7fa790104360 2024-05-21T15:33:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.511+0000 7fa777fff700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa7780a2cb0 0x7fa7780a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.512+0000 7fa78e7fc700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa7780a2cb0 0x7fa7780a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.512+0000 7fa777fff700 1 -- 172.21.15.38:0/3321619483 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fa7800cf450 con 0x7fa790104360 2024-05-21T15:33:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.512+0000 7fa78e7fc700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa7780a2cb0 0x7fa7780a51c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa790101240 tx=0x7fa78400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:50.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.515+0000 7fa777fff700 1 -- 172.21.15.38:0/3321619483 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa780014030 con 0x7fa790104360 2024-05-21T15:33:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.732+0000 7fa7954f8700 1 -- 172.21.15.38:0/3321619483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fa7901a6a40 con 0x7fa790104360 2024-05-21T15:33:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.732+0000 7fa777fff700 1 -- 172.21.15.38:0/3321619483 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa78000ee90 con 0x7fa790104360 2024-05-21T15:33:50.732 INFO:teuthology.orchestra.run.smithi038.stdout:257698037803 2024-05-21T15:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.735+0000 7fa7954f8700 1 -- 172.21.15.38:0/3321619483 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa7780a2cb0 msgr2=0x7fa7780a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.735+0000 7fa7954f8700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa7780a2cb0 0x7fa7780a51c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa790101240 tx=0x7fa78400b3c0 comp rx=0 tx=0).stop 2024-05-21T15:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.735+0000 7fa7954f8700 1 -- 172.21.15.38:0/3321619483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa790104360 msgr2=0x7fa790199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.735+0000 7fa7954f8700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa790104360 0x7fa790199d90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa78000c9b0 tx=0x7fa78000ccc0 comp rx=0 tx=0).stop 2024-05-21T15:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.735+0000 7fa7954f8700 1 -- 172.21.15.38:0/3321619483 shutdown_connections 2024-05-21T15:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.735+0000 7fa7954f8700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa7780a2cb0 0x7fa7780a51c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.735+0000 7fa7954f8700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa79010d790 0x7fa79019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.735+0000 7fa7954f8700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa790104360 0x7fa790199d90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.735+0000 7fa7954f8700 1 --2- 172.21.15.38:0/3321619483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa790104cb0 0x7fa79019a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.735+0000 7fa7954f8700 1 -- 172.21.15.38:0/3321619483 >> 172.21.15.38:0/3321619483 conn(0x7fa7900fddb0 msgr2=0x7fa7900fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.735+0000 7fa7954f8700 1 -- 172.21.15.38:0/3321619483 shutdown_connections 2024-05-21T15:33:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:50.735+0000 7fa7954f8700 1 -- 172.21.15.38:0/3321619483 wait complete. 2024-05-21T15:33:51.139 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:50 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3321619483' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-21T15:33:51.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:50 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3321619483' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-21T15:33:51.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:50 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3321619483' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-21T15:33:51.423 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037798 got 257698037803 for osd.8 2024-05-21T15:33:51.423 DEBUG:teuthology.parallel:result is None 2024-05-21T15:33:51.423 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-21T15:33:51.423 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T15:33:51.684 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:52.073 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:51 smithi038 ceph-mon[31995]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:33:52.073 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:51 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:52.074 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:51 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:52.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:51 smithi053 ceph-mon[36496]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:33:52.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:51 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:52.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:51 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:52.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:51 smithi045 ceph-mon[36409]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:33:52.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:51 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:52.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:51 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.393+0000 7fa06ca1d700 1 -- 172.21.15.38:0/3520519583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa068104380 msgr2=0x7fa068104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.393+0000 7fa06ca1d700 1 --2- 172.21.15.38:0/3520519583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa068104380 0x7fa068104790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa058009990 tx=0x7fa058009ca0 comp rx=0 tx=0).stop 2024-05-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.393+0000 7fa06ca1d700 1 -- 172.21.15.38:0/3520519583 shutdown_connections 2024-05-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.393+0000 7fa06ca1d700 1 --2- 172.21.15.38:0/3520519583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06810d760 0x7fa06810fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.393+0000 7fa06ca1d700 1 --2- 172.21.15.38:0/3520519583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa068104380 0x7fa068104790 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.393+0000 7fa06ca1d700 1 --2- 172.21.15.38:0/3520519583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa068104cd0 0x7fa06810d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.393+0000 7fa06ca1d700 1 -- 172.21.15.38:0/3520519583 >> 172.21.15.38:0/3520519583 conn(0x7fa0680fdc40 msgr2=0x7fa068100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.393+0000 7fa06ca1d700 1 -- 172.21.15.38:0/3520519583 shutdown_connections 2024-05-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.394+0000 7fa06ca1d700 1 -- 172.21.15.38:0/3520519583 wait complete. 2024-05-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.394+0000 7fa06ca1d700 1 Processor -- start 2024-05-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.394+0000 7fa06ca1d700 1 -- start start 2024-05-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa06ca1d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa068104380 0x7fa068102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa06ca1d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa068104cd0 0x7fa0681025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa06ca1d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06810d760 0x7fa068102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa06ca1d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa068113400 con 0x7fa068104380 2024-05-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa06ca1d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa068113280 con 0x7fa06810d760 2024-05-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa06ca1d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa068113580 con 0x7fa068104cd0 2024-05-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa06659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa068104380 0x7fa068102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa06659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa068104380 0x7fa068102070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40710/0 (socket says 172.21.15.38:40710) 2024-05-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa06659c700 1 -- 172.21.15.38:0/771554447 learned_addr learned my addr 172.21.15.38:0/771554447 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa066d9d700 1 --2- 172.21.15.38:0/771554447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06810d760 0x7fa068102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa065d9b700 1 --2- 172.21.15.38:0/771554447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa068104cd0 0x7fa0681025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa06659c700 1 -- 172.21.15.38:0/771554447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa068104cd0 msgr2=0x7fa0681025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa06659c700 1 --2- 172.21.15.38:0/771554447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa068104cd0 0x7fa0681025b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa06659c700 1 -- 172.21.15.38:0/771554447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06810d760 msgr2=0x7fa068102af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa06659c700 1 --2- 172.21.15.38:0/771554447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06810d760 0x7fa068102af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.395+0000 7fa06659c700 1 -- 172.21.15.38:0/771554447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0681a8070 con 0x7fa068104380 2024-05-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.396+0000 7fa06659c700 1 --2- 172.21.15.38:0/771554447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa068104380 0x7fa068102070 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa058000c00 tx=0x7fa0580047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.396+0000 7fa0577fe700 1 -- 172.21.15.38:0/771554447 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058027070 con 0x7fa068104380 2024-05-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.396+0000 7fa06ca1d700 1 -- 172.21.15.38:0/771554447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0681a8300 con 0x7fa068104380 2024-05-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.396+0000 7fa06ca1d700 1 -- 172.21.15.38:0/771554447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0681a8830 con 0x7fa068104380 2024-05-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.396+0000 7fa0577fe700 1 -- 172.21.15.38:0/771554447 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa058003e40 con 0x7fa068104380 2024-05-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.396+0000 7fa0577fe700 1 -- 172.21.15.38:0/771554447 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058022d10 con 0x7fa068104380 2024-05-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.397+0000 7fa06ca1d700 1 -- 172.21.15.38:0/771554447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa068050170 con 0x7fa068104380 2024-05-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.399+0000 7fa0577fe700 1 -- 172.21.15.38:0/771554447 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa05802a070 con 0x7fa068104380 2024-05-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.399+0000 7fa0577fe700 1 --2- 172.21.15.38:0/771554447 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0500a2c20 0x7fa0500a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.399+0000 7fa065d9b700 1 --2- 172.21.15.38:0/771554447 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0500a2c20 0x7fa0500a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.399+0000 7fa0577fe700 1 -- 172.21.15.38:0/771554447 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fa0580d0580 con 0x7fa068104380 2024-05-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.400+0000 7fa065d9b700 1 --2- 172.21.15.38:0/771554447 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0500a2c20 0x7fa0500a5130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa0681011a0 tx=0x7fa05c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.403+0000 7fa0577fe700 1 -- 172.21.15.38:0/771554447 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa058013070 con 0x7fa068104380 2024-05-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.604+0000 7fa06ca1d700 1 -- 172.21.15.38:0/771554447 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0680fe900 con 0x7fa0500a2c20 2024-05-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.611+0000 7fa0577fe700 1 -- 172.21.15.38:0/771554447 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259058 (secure 0 0 0) 0x7fa0680fe900 con 0x7fa0500a2c20 2024-05-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.614+0000 7fa06ca1d700 1 -- 172.21.15.38:0/771554447 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0500a2c20 msgr2=0x7fa0500a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.614+0000 7fa06ca1d700 1 --2- 172.21.15.38:0/771554447 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0500a2c20 0x7fa0500a5130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa0681011a0 tx=0x7fa05c00a300 comp rx=0 tx=0).stop 2024-05-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.614+0000 7fa06ca1d700 1 -- 172.21.15.38:0/771554447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa068104380 msgr2=0x7fa068102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.614+0000 7fa06ca1d700 1 --2- 172.21.15.38:0/771554447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa068104380 0x7fa068102070 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa058000c00 tx=0x7fa0580047b0 comp rx=0 tx=0).stop 2024-05-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.614+0000 7fa06ca1d700 1 -- 172.21.15.38:0/771554447 shutdown_connections 2024-05-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.614+0000 7fa06ca1d700 1 --2- 172.21.15.38:0/771554447 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0500a2c20 0x7fa0500a5130 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.614+0000 7fa06ca1d700 1 --2- 172.21.15.38:0/771554447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06810d760 0x7fa068102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.614+0000 7fa06ca1d700 1 --2- 172.21.15.38:0/771554447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa068104380 0x7fa068102070 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.614+0000 7fa06ca1d700 1 --2- 172.21.15.38:0/771554447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa068104cd0 0x7fa0681025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.614+0000 7fa06ca1d700 1 -- 172.21.15.38:0/771554447 >> 172.21.15.38:0/771554447 conn(0x7fa0680fdc40 msgr2=0x7fa0680fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.614+0000 7fa06ca1d700 1 -- 172.21.15.38:0/771554447 shutdown_connections 2024-05-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:53.615+0000 7fa06ca1d700 1 -- 172.21.15.38:0/771554447 wait complete. 2024-05-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-05-21T15:33:54.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:53 smithi053 ceph-mon[36496]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:33:54.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:53 smithi053 ceph-mon[36496]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-21T15:33:54.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:33:54.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:33:54.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:53 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:54.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:53 smithi045 ceph-mon[36409]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:33:54.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:53 smithi045 ceph-mon[36409]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-21T15:33:54.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:33:54.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:33:54.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:53 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:54.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:53 smithi038 ceph-mon[31995]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:33:54.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:53 smithi038 ceph-mon[31995]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-21T15:33:54.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:33:54.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:33:54.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:53 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:33:54.311 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-05-21T15:33:52.236336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656880,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":846,"num_read_kb":538,"num_write":418,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373508,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568316,"statfs":{"total":1151940427776,"available":1151557955584,"internally_reserved":0,"allocated":12079104,"data_stored":7476060,"data_compressed":184854,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003623"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371915+0000","last_change":"2024-05-21T15:32:33.687244+0000","last_active":"2024-05-21T15:33:23.371915+0000","last_peered":"2024-05-21T15:33:23.371915+0000","last_clean":"2024-05-21T15:33:23.371915+0000","last_became_active":"2024-05-21T15:32:33.686515+0000","last_became_peered":"2024-05-21T15:32:33.686515+0000","last_unstale":"2024-05-21T15:33:23.371915+0000","last_undegraded":"2024-05-21T15:33:23.371915+0000","last_fullsized":"2024-05-21T15:33:23.371915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:10:18.396997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375764+0000","last_change":"2024-05-21T15:32:33.678497+0000","last_active":"2024-05-21T15:33:23.375764+0000","last_peered":"2024-05-21T15:33:23.375764+0000","last_clean":"2024-05-21T15:33:23.375764+0000","last_became_active":"2024-05-21T15:32:33.678001+0000","last_became_peered":"2024-05-21T15:32:33.678001+0000","last_unstale":"2024-05-21T15:33:23.375764+0000","last_undegraded":"2024-05-21T15:33:23.375764+0000","last_fullsized":"2024-05-21T15:33:23.375764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:14:48.255675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375514+0000","last_change":"2024-05-21T15:32:33.692444+0000","last_active":"2024-05-21T15:33:23.375514+0000","last_peered":"2024-05-21T15:33:23.375514+0000","last_clean":"2024-05-21T15:33:23.375514+0000","last_became_active":"2024-05-21T15:32:33.691662+0000","last_became_peered":"2024-05-21T15:32:33.691662+0000","last_unstale":"2024-05-21T15:33:23.375514+0000","last_undegraded":"2024-05-21T15:33:23.375514+0000","last_fullsized":"2024-05-21T15:33:23.375514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:26:25.929879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.482157+0000","last_change":"2024-05-21T15:32:33.692297+0000","last_active":"2024-05-21T15:33:23.482157+0000","last_peered":"2024-05-21T15:33:23.482157+0000","last_clean":"2024-05-21T15:33:23.482157+0000","last_became_active":"2024-05-21T15:32:33.691925+0000","last_became_peered":"2024-05-21T15:32:33.691925+0000","last_unstale":"2024-05-21T15:33:23.482157+0000","last_undegraded":"2024-05-21T15:33:23.482157+0000","last_fullsized":"2024-05-21T15:33:23.482157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:40:45.448485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374293+0000","last_change":"2024-05-21T15:32:33.685511+0000","last_active":"2024-05-21T15:33:23.374293+0000","last_peered":"2024-05-21T15:33:23.374293+0000","last_clean":"2024-05-21T15:33:23.374293+0000","last_became_active":"2024-05-21T15:32:33.685182+0000","last_became_peered":"2024-05-21T15:32:33.685182+0000","last_unstale":"2024-05-21T15:33:23.374293+0000","last_undegraded":"2024-05-21T15:33:23.374293+0000","last_fullsized":"2024-05-21T15:33:23.374293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:59:06.680712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371963+0000","last_change":"2024-05-21T15:32:33.682400+0000","last_active":"2024-05-21T15:33:23.371963+0000","last_peered":"2024-05-21T15:33:23.371963+0000","last_clean":"2024-05-21T15:33:23.371963+0000","last_became_active":"2024-05-21T15:32:33.682068+0000","last_became_peered":"2024-05-21T15:32:33.682068+0000","last_unstale":"2024-05-21T15:33:23.371963+0000","last_undegraded":"2024-05-21T15:33:23.371963+0000","last_fullsized":"2024-05-21T15:33:23.371963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:31:50.968706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.373937+0000","last_change":"2024-05-21T15:32:33.690099+0000","last_active":"2024-05-21T15:33:23.373937+0000","last_peered":"2024-05-21T15:33:23.373937+0000","last_clean":"2024-05-21T15:33:23.373937+0000","last_became_active":"2024-05-21T15:32:33.689762+0000","last_became_peered":"2024-05-21T15:32:33.689762+0000","last_unstale":"2024-05-21T15:33:23.373937+0000","last_undegraded":"2024-05-21T15:33:23.373937+0000","last_fullsized":"2024-05-21T15:33:23.373937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:55:20.331097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.380040+0000","last_change":"2024-05-21T15:32:33.691701+0000","last_active":"2024-05-21T15:33:23.380040+0000","last_peered":"2024-05-21T15:33:23.380040+0000","last_clean":"2024-05-21T15:33:23.380040+0000","last_became_active":"2024-05-21T15:32:33.691386+0000","last_became_peered":"2024-05-21T15:32:33.691386+0000","last_unstale":"2024-05-21T15:33:23.380040+0000","last_undegraded":"2024-05-21T15:33:23.380040+0000","last_fullsized":"2024-05-21T15:33:23.380040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:54:45.691392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.609521+0000","last_change":"2024-05-21T15:32:33.677484+0000","last_active":"2024-05-21T15:33:23.609521+0000","last_peered":"2024-05-21T15:33:23.609521+0000","last_clean":"2024-05-21T15:33:23.609521+0000","last_became_active":"2024-05-21T15:32:33.677160+0000","last_became_peered":"2024-05-21T15:32:33.677160+0000","last_unstale":"2024-05-21T15:33:23.609521+0000","last_undegraded":"2024-05-21T15:33:23.609521+0000","last_fullsized":"2024-05-21T15:33:23.609521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:38:00.914393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.372244+0000","last_change":"2024-05-21T15:32:33.686957+0000","last_active":"2024-05-21T15:33:23.372244+0000","last_peered":"2024-05-21T15:33:23.372244+0000","last_clean":"2024-05-21T15:33:23.372244+0000","last_became_active":"2024-05-21T15:32:33.685338+0000","last_became_peered":"2024-05-21T15:32:33.685338+0000","last_unstale":"2024-05-21T15:33:23.372244+0000","last_undegraded":"2024-05-21T15:33:23.372244+0000","last_fullsized":"2024-05-21T15:33:23.372244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:04:58.115974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374263+0000","last_change":"2024-05-21T15:32:33.693311+0000","last_active":"2024-05-21T15:33:23.374263+0000","last_peered":"2024-05-21T15:33:23.374263+0000","last_clean":"2024-05-21T15:33:23.374263+0000","last_became_active":"2024-05-21T15:32:33.692959+0000","last_became_peered":"2024-05-21T15:32:33.692959+0000","last_unstale":"2024-05-21T15:33:23.374263+0000","last_undegraded":"2024-05-21T15:33:23.374263+0000","last_fullsized":"2024-05-21T15:33:23.374263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:35:12.918524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.481881+0000","last_change":"2024-05-21T15:32:33.693254+0000","last_active":"2024-05-21T15:33:23.481881+0000","last_peered":"2024-05-21T15:33:23.481881+0000","last_clean":"2024-05-21T15:33:23.481881+0000","last_became_active":"2024-05-21T15:32:33.692901+0000","last_became_peered":"2024-05-21T15:32:33.692901+0000","last_unstale":"2024-05-21T15:33:23.481881+0000","last_undegraded":"2024-05-21T15:33:23.481881+0000","last_fullsized":"2024-05-21T15:33:23.481881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:36:30.591479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.383656+0000","last_change":"2024-05-21T15:33:23.383656+0000","last_active":"2024-05-21T15:33:23.383656+0000","last_peered":"2024-05-21T15:33:23.383656+0000","last_clean":"2024-05-21T15:33:23.383656+0000","last_became_active":"2024-05-21T15:33:23.383113+0000","last_became_peered":"2024-05-21T15:33:23.383113+0000","last_unstale":"2024-05-21T15:33:23.383656+0000","last_undegraded":"2024-05-21T15:33:23.383656+0000","last_fullsized":"2024-05-21T15:33:23.383656+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:14:37.327480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375276+0000","last_change":"2024-05-21T15:32:33.678315+0000","last_active":"2024-05-21T15:33:23.375276+0000","last_peered":"2024-05-21T15:33:23.375276+0000","last_clean":"2024-05-21T15:33:23.375276+0000","last_became_active":"2024-05-21T15:32:33.677787+0000","last_became_peered":"2024-05-21T15:32:33.677787+0000","last_unstale":"2024-05-21T15:33:23.375276+0000","last_undegraded":"2024-05-21T15:33:23.375276+0000","last_fullsized":"2024-05-21T15:33:23.375276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:17:24.826525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.380244+0000","last_change":"2024-05-21T15:33:23.380244+0000","last_active":"2024-05-21T15:33:23.380244+0000","last_peered":"2024-05-21T15:33:23.380244+0000","last_clean":"2024-05-21T15:33:23.380244+0000","last_became_active":"2024-05-21T15:33:23.379769+0000","last_became_peered":"2024-05-21T15:33:23.379769+0000","last_unstale":"2024-05-21T15:33:23.380244+0000","last_undegraded":"2024-05-21T15:33:23.380244+0000","last_fullsized":"2024-05-21T15:33:23.380244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:16:07.820636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.608610+0000","last_change":"2024-05-21T15:32:33.683597+0000","last_active":"2024-05-21T15:33:23.608610+0000","last_peered":"2024-05-21T15:33:23.608610+0000","last_clean":"2024-05-21T15:33:23.608610+0000","last_became_active":"2024-05-21T15:32:33.681948+0000","last_became_peered":"2024-05-21T15:32:33.681948+0000","last_unstale":"2024-05-21T15:33:23.608610+0000","last_undegraded":"2024-05-21T15:33:23.608610+0000","last_fullsized":"2024-05-21T15:33:23.608610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:26:50.086019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.372056+0000","last_change":"2024-05-21T15:32:33.683329+0000","last_active":"2024-05-21T15:33:23.372056+0000","last_peered":"2024-05-21T15:33:23.372056+0000","last_clean":"2024-05-21T15:33:23.372056+0000","last_became_active":"2024-05-21T15:32:33.682947+0000","last_became_peered":"2024-05-21T15:32:33.682947+0000","last_unstale":"2024-05-21T15:33:23.372056+0000","last_undegraded":"2024-05-21T15:33:23.372056+0000","last_fullsized":"2024-05-21T15:33:23.372056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:24:27.502257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371161+0000","last_change":"2024-05-21T15:32:33.680391+0000","last_active":"2024-05-21T15:33:23.371161+0000","last_peered":"2024-05-21T15:33:23.371161+0000","last_clean":"2024-05-21T15:33:23.371161+0000","last_became_active":"2024-05-21T15:32:33.679996+0000","last_became_peered":"2024-05-21T15:32:33.679996+0000","last_unstale":"2024-05-21T15:33:23.371161+0000","last_undegraded":"2024-05-21T15:33:23.371161+0000","last_fullsized":"2024-05-21T15:33:23.371161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:56:43.658565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375597+0000","last_change":"2024-05-21T15:32:33.688783+0000","last_active":"2024-05-21T15:33:23.375597+0000","last_peered":"2024-05-21T15:33:23.375597+0000","last_clean":"2024-05-21T15:33:23.375597+0000","last_became_active":"2024-05-21T15:32:33.688006+0000","last_became_peered":"2024-05-21T15:32:33.688006+0000","last_unstale":"2024-05-21T15:33:23.375597+0000","last_undegraded":"2024-05-21T15:33:23.375597+0000","last_fullsized":"2024-05-21T15:33:23.375597+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:17:44.489759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:51.096731+0000","last_change":"2024-05-21T15:32:33.182251+0000","last_active":"2024-05-21T15:33:51.096731+0000","last_peered":"2024-05-21T15:33:51.096731+0000","last_clean":"2024-05-21T15:33:51.096731+0000","last_became_active":"2024-05-21T15:32:33.181893+0000","last_became_peered":"2024-05-21T15:32:33.181893+0000","last_unstale":"2024-05-21T15:33:51.096731+0000","last_undegraded":"2024-05-21T15:33:51.096731+0000","last_fullsized":"2024-05-21T15:33:51.096731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:23:36.863186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.381529+0000","last_change":"2024-05-21T15:32:33.691962+0000","last_active":"2024-05-21T15:33:23.381529+0000","last_peered":"2024-05-21T15:33:23.381529+0000","last_clean":"2024-05-21T15:33:23.381529+0000","last_became_active":"2024-05-21T15:32:33.691619+0000","last_became_peered":"2024-05-21T15:32:33.691619+0000","last_unstale":"2024-05-21T15:33:23.381529+0000","last_undegraded":"2024-05-21T15:33:23.381529+0000","last_fullsized":"2024-05-21T15:33:23.381529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:50:14.523567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.381842+0000","last_change":"2024-05-21T15:32:33.183772+0000","last_active":"2024-05-21T15:33:23.381842+0000","last_peered":"2024-05-21T15:33:23.381842+0000","last_clean":"2024-05-21T15:33:23.381842+0000","last_became_active":"2024-05-21T15:32:33.183440+0000","last_became_peered":"2024-05-21T15:32:33.183440+0000","last_unstale":"2024-05-21T15:33:23.381842+0000","last_undegraded":"2024-05-21T15:33:23.381842+0000","last_fullsized":"2024-05-21T15:33:23.381842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:53:00.714359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.376127+0000","last_change":"2024-05-21T15:32:33.682544+0000","last_active":"2024-05-21T15:33:23.376127+0000","last_peered":"2024-05-21T15:33:23.376127+0000","last_clean":"2024-05-21T15:33:23.376127+0000","last_became_active":"2024-05-21T15:32:33.682161+0000","last_became_peered":"2024-05-21T15:32:33.682161+0000","last_unstale":"2024-05-21T15:33:23.376127+0000","last_undegraded":"2024-05-21T15:33:23.376127+0000","last_fullsized":"2024-05-21T15:33:23.376127+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:20:38.143508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375834+0000","last_change":"2024-05-21T15:32:33.182400+0000","last_active":"2024-05-21T15:33:23.375834+0000","last_peered":"2024-05-21T15:33:23.375834+0000","last_clean":"2024-05-21T15:33:23.375834+0000","last_became_active":"2024-05-21T15:32:33.181745+0000","last_became_peered":"2024-05-21T15:32:33.181745+0000","last_unstale":"2024-05-21T15:33:23.375834+0000","last_undegraded":"2024-05-21T15:33:23.375834+0000","last_fullsized":"2024-05-21T15:33:23.375834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:39:41.669529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.608783+0000","last_change":"2024-05-21T15:32:33.682700+0000","last_active":"2024-05-21T15:33:23.608783+0000","last_peered":"2024-05-21T15:33:23.608783+0000","last_clean":"2024-05-21T15:33:23.608783+0000","last_became_active":"2024-05-21T15:32:33.680876+0000","last_became_peered":"2024-05-21T15:32:33.680876+0000","last_unstale":"2024-05-21T15:33:23.608783+0000","last_undegraded":"2024-05-21T15:33:23.608783+0000","last_fullsized":"2024-05-21T15:33:23.608783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:30.694520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375325+0000","last_change":"2024-05-21T15:32:33.179114+0000","last_active":"2024-05-21T15:33:23.375325+0000","last_peered":"2024-05-21T15:33:23.375325+0000","last_clean":"2024-05-21T15:33:23.375325+0000","last_became_active":"2024-05-21T15:32:33.178485+0000","last_became_peered":"2024-05-21T15:32:33.178485+0000","last_unstale":"2024-05-21T15:33:23.375325+0000","last_undegraded":"2024-05-21T15:33:23.375325+0000","last_fullsized":"2024-05-21T15:33:23.375325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:19:56.501287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.609227+0000","last_change":"2024-05-21T15:32:33.691874+0000","last_active":"2024-05-21T15:33:23.609227+0000","last_peered":"2024-05-21T15:33:23.609227+0000","last_clean":"2024-05-21T15:33:23.609227+0000","last_became_active":"2024-05-21T15:32:33.691546+0000","last_became_peered":"2024-05-21T15:32:33.691546+0000","last_unstale":"2024-05-21T15:33:23.609227+0000","last_undegraded":"2024-05-21T15:33:23.609227+0000","last_fullsized":"2024-05-21T15:33:23.609227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:40:09.281792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:50.096530+0000","last_change":"2024-05-21T15:32:33.183457+0000","last_active":"2024-05-21T15:33:50.096530+0000","last_peered":"2024-05-21T15:33:50.096530+0000","last_clean":"2024-05-21T15:33:50.096530+0000","last_became_active":"2024-05-21T15:32:33.183127+0000","last_became_peered":"2024-05-21T15:32:33.183127+0000","last_unstale":"2024-05-21T15:33:50.096530+0000","last_undegraded":"2024-05-21T15:33:50.096530+0000","last_fullsized":"2024-05-21T15:33:50.096530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:41:05.179629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374650+0000","last_change":"2024-05-21T15:32:33.677260+0000","last_active":"2024-05-21T15:33:23.374650+0000","last_peered":"2024-05-21T15:33:23.374650+0000","last_clean":"2024-05-21T15:33:23.374650+0000","last_became_active":"2024-05-21T15:32:33.676905+0000","last_became_peered":"2024-05-21T15:32:33.676905+0000","last_unstale":"2024-05-21T15:33:23.374650+0000","last_undegraded":"2024-05-21T15:33:23.374650+0000","last_fullsized":"2024-05-21T15:33:23.374650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:51:41.543066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371864+0000","last_change":"2024-05-21T15:32:33.181819+0000","last_active":"2024-05-21T15:33:23.371864+0000","last_peered":"2024-05-21T15:33:23.371864+0000","last_clean":"2024-05-21T15:33:23.371864+0000","last_became_active":"2024-05-21T15:32:33.181471+0000","last_became_peered":"2024-05-21T15:32:33.181471+0000","last_unstale":"2024-05-21T15:33:23.371864+0000","last_undegraded":"2024-05-21T15:33:23.371864+0000","last_fullsized":"2024-05-21T15:33:23.371864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:21:54.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.609455+0000","last_change":"2024-05-21T15:32:33.691072+0000","last_active":"2024-05-21T15:33:23.609455+0000","last_peered":"2024-05-21T15:33:23.609455+0000","last_clean":"2024-05-21T15:33:23.609455+0000","last_became_active":"2024-05-21T15:32:33.690663+0000","last_became_peered":"2024-05-21T15:32:33.690663+0000","last_unstale":"2024-05-21T15:33:23.609455+0000","last_undegraded":"2024-05-21T15:33:23.609455+0000","last_fullsized":"2024-05-21T15:33:23.609455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:52:44.549537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375504+0000","last_change":"2024-05-21T15:32:33.179588+0000","last_active":"2024-05-21T15:33:23.375504+0000","last_peered":"2024-05-21T15:33:23.375504+0000","last_clean":"2024-05-21T15:33:23.375504+0000","last_became_active":"2024-05-21T15:32:33.179115+0000","last_became_peered":"2024-05-21T15:32:33.179115+0000","last_unstale":"2024-05-21T15:33:23.375504+0000","last_undegraded":"2024-05-21T15:33:23.375504+0000","last_fullsized":"2024-05-21T15:33:23.375504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:18:43.904584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.380167+0000","last_change":"2024-05-21T15:32:33.689456+0000","last_active":"2024-05-21T15:33:23.380167+0000","last_peered":"2024-05-21T15:33:23.380167+0000","last_clean":"2024-05-21T15:33:23.380167+0000","last_became_active":"2024-05-21T15:32:33.689142+0000","last_became_peered":"2024-05-21T15:32:33.689142+0000","last_unstale":"2024-05-21T15:33:23.380167+0000","last_undegraded":"2024-05-21T15:33:23.380167+0000","last_fullsized":"2024-05-21T15:33:23.380167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:00:13.768272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.608862+0000","last_change":"2024-05-21T15:32:33.184865+0000","last_active":"2024-05-21T15:33:23.608862+0000","last_peered":"2024-05-21T15:33:23.608862+0000","last_clean":"2024-05-21T15:33:23.608862+0000","last_became_active":"2024-05-21T15:32:33.184540+0000","last_became_peered":"2024-05-21T15:32:33.184540+0000","last_unstale":"2024-05-21T15:33:23.608862+0000","last_undegraded":"2024-05-21T15:33:23.608862+0000","last_fullsized":"2024-05-21T15:33:23.608862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:09:20.072906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.370919+0000","last_change":"2024-05-21T15:32:33.685478+0000","last_active":"2024-05-21T15:33:23.370919+0000","last_peered":"2024-05-21T15:33:23.370919+0000","last_clean":"2024-05-21T15:33:23.370919+0000","last_became_active":"2024-05-21T15:32:33.685137+0000","last_became_peered":"2024-05-21T15:32:33.685137+0000","last_unstale":"2024-05-21T15:33:23.370919+0000","last_undegraded":"2024-05-21T15:33:23.370919+0000","last_fullsized":"2024-05-21T15:33:23.370919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:39:40.815876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":157,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:47.091699+0000","last_change":"2024-05-21T15:32:33.179215+0000","last_active":"2024-05-21T15:33:47.091699+0000","last_peered":"2024-05-21T15:33:47.091699+0000","last_clean":"2024-05-21T15:33:47.091699+0000","last_became_active":"2024-05-21T15:32:33.178846+0000","last_became_peered":"2024-05-21T15:32:33.178846+0000","last_unstale":"2024-05-21T15:33:47.091699+0000","last_undegraded":"2024-05-21T15:33:47.091699+0000","last_fullsized":"2024-05-21T15:33:47.091699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:29:47.289968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.609586+0000","last_change":"2024-05-21T15:32:33.684900+0000","last_active":"2024-05-21T15:33:23.609586+0000","last_peered":"2024-05-21T15:33:23.609586+0000","last_clean":"2024-05-21T15:33:23.609586+0000","last_became_active":"2024-05-21T15:32:33.681374+0000","last_became_peered":"2024-05-21T15:32:33.681374+0000","last_unstale":"2024-05-21T15:33:23.609586+0000","last_undegraded":"2024-05-21T15:33:23.609586+0000","last_fullsized":"2024-05-21T15:33:23.609586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:51:38.079726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374211+0000","last_change":"2024-05-21T15:32:33.189945+0000","last_active":"2024-05-21T15:33:23.374211+0000","last_peered":"2024-05-21T15:33:23.374211+0000","last_clean":"2024-05-21T15:33:23.374211+0000","last_became_active":"2024-05-21T15:32:33.189557+0000","last_became_peered":"2024-05-21T15:32:33.189557+0000","last_unstale":"2024-05-21T15:33:23.374211+0000","last_undegraded":"2024-05-21T15:33:23.374211+0000","last_fullsized":"2024-05-21T15:33:23.374211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:08.427043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.376014+0000","last_change":"2024-05-21T15:32:33.692896+0000","last_active":"2024-05-21T15:33:23.376014+0000","last_peered":"2024-05-21T15:33:23.376014+0000","last_clean":"2024-05-21T15:33:23.376014+0000","last_became_active":"2024-05-21T15:32:33.692527+0000","last_became_peered":"2024-05-21T15:32:33.692527+0000","last_unstale":"2024-05-21T15:33:23.376014+0000","last_undegraded":"2024-05-21T15:33:23.376014+0000","last_fullsized":"2024-05-21T15:33:23.376014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:45:45.418063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.609353+0000","last_change":"2024-05-21T15:32:33.172989+0000","last_active":"2024-05-21T15:33:23.609353+0000","last_peered":"2024-05-21T15:33:23.609353+0000","last_clean":"2024-05-21T15:33:23.609353+0000","last_became_active":"2024-05-21T15:32:33.172215+0000","last_became_peered":"2024-05-21T15:32:33.172215+0000","last_unstale":"2024-05-21T15:33:23.609353+0000","last_undegraded":"2024-05-21T15:33:23.609353+0000","last_fullsized":"2024-05-21T15:33:23.609353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:09:39.287636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.609025+0000","last_change":"2024-05-21T15:32:33.678441+0000","last_active":"2024-05-21T15:33:23.609025+0000","last_peered":"2024-05-21T15:33:23.609025+0000","last_clean":"2024-05-21T15:33:23.609025+0000","last_became_active":"2024-05-21T15:32:33.677833+0000","last_became_peered":"2024-05-21T15:32:33.677833+0000","last_unstale":"2024-05-21T15:33:23.609025+0000","last_undegraded":"2024-05-21T15:33:23.609025+0000","last_fullsized":"2024-05-21T15:33:23.609025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:36.124064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'29","reported_seq":188,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:47.108482+0000","last_change":"2024-05-21T15:32:33.178153+0000","last_active":"2024-05-21T15:33:47.108482+0000","last_peered":"2024-05-21T15:33:47.108482+0000","last_clean":"2024-05-21T15:33:47.108482+0000","last_became_active":"2024-05-21T15:32:33.177792+0000","last_became_peered":"2024-05-21T15:32:33.177792+0000","last_unstale":"2024-05-21T15:33:47.108482+0000","last_undegraded":"2024-05-21T15:33:47.108482+0000","last_fullsized":"2024-05-21T15:33:47.108482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:43:41.865204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":140,"num_read_kb":128,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375589+0000","last_change":"2024-05-21T15:32:33.681328+0000","last_active":"2024-05-21T15:33:23.375589+0000","last_peered":"2024-05-21T15:33:23.375589+0000","last_clean":"2024-05-21T15:33:23.375589+0000","last_became_active":"2024-05-21T15:32:33.680975+0000","last_became_peered":"2024-05-21T15:32:33.680975+0000","last_unstale":"2024-05-21T15:33:23.375589+0000","last_undegraded":"2024-05-21T15:33:23.375589+0000","last_fullsized":"2024-05-21T15:33:23.375589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:50:27.199348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375005+0000","last_change":"2024-05-21T15:32:33.182492+0000","last_active":"2024-05-21T15:33:23.375005+0000","last_peered":"2024-05-21T15:33:23.375005+0000","last_clean":"2024-05-21T15:33:23.375005+0000","last_became_active":"2024-05-21T15:32:33.182152+0000","last_became_peered":"2024-05-21T15:32:33.182152+0000","last_unstale":"2024-05-21T15:33:23.375005+0000","last_undegraded":"2024-05-21T15:33:23.375005+0000","last_fullsized":"2024-05-21T15:33:23.375005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:04:07.432742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.376516+0000","last_change":"2024-05-21T15:32:33.686623+0000","last_active":"2024-05-21T15:33:23.376516+0000","last_peered":"2024-05-21T15:33:23.376516+0000","last_clean":"2024-05-21T15:33:23.376516+0000","last_became_active":"2024-05-21T15:32:33.685816+0000","last_became_peered":"2024-05-21T15:32:33.685816+0000","last_unstale":"2024-05-21T15:33:23.376516+0000","last_undegraded":"2024-05-21T15:33:23.376516+0000","last_fullsized":"2024-05-21T15:33:23.376516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:15:20.354113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.608969+0000","last_change":"2024-05-21T15:32:33.182218+0000","last_active":"2024-05-21T15:33:23.608969+0000","last_peered":"2024-05-21T15:33:23.608969+0000","last_clean":"2024-05-21T15:33:23.608969+0000","last_became_active":"2024-05-21T15:32:33.181883+0000","last_became_peered":"2024-05-21T15:32:33.181883+0000","last_unstale":"2024-05-21T15:33:23.608969+0000","last_undegraded":"2024-05-21T15:33:23.608969+0000","last_fullsized":"2024-05-21T15:33:23.608969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:24:07.963898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374311+0000","last_change":"2024-05-21T15:32:33.689936+0000","last_active":"2024-05-21T15:33:23.374311+0000","last_peered":"2024-05-21T15:33:23.374311+0000","last_clean":"2024-05-21T15:33:23.374311+0000","last_became_active":"2024-05-21T15:32:33.689452+0000","last_became_peered":"2024-05-21T15:32:33.689452+0000","last_unstale":"2024-05-21T15:33:23.374311+0000","last_undegraded":"2024-05-21T15:33:23.374311+0000","last_fullsized":"2024-05-21T15:33:23.374311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:00.420730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:50.096727+0000","last_change":"2024-05-21T15:32:33.181354+0000","last_active":"2024-05-21T15:33:50.096727+0000","last_peered":"2024-05-21T15:33:50.096727+0000","last_clean":"2024-05-21T15:33:50.096727+0000","last_became_active":"2024-05-21T15:32:33.180983+0000","last_became_peered":"2024-05-21T15:32:33.180983+0000","last_unstale":"2024-05-21T15:33:50.096727+0000","last_undegraded":"2024-05-21T15:33:50.096727+0000","last_fullsized":"2024-05-21T15:33:50.096727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:18:46.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.377305+0000","last_change":"2024-05-21T15:33:23.377305+0000","last_active":"2024-05-21T15:33:23.377305+0000","last_peered":"2024-05-21T15:33:23.377305+0000","last_clean":"2024-05-21T15:33:23.377305+0000","last_became_active":"2024-05-21T15:33:23.376921+0000","last_became_peered":"2024-05-21T15:33:23.376921+0000","last_unstale":"2024-05-21T15:33:23.377305+0000","last_undegraded":"2024-05-21T15:33:23.377305+0000","last_fullsized":"2024-05-21T15:33:23.377305+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:19.874725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":61,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.496401+0000","last_change":"2024-05-21T15:33:23.496401+0000","last_active":"2024-05-21T15:33:23.496401+0000","last_peered":"2024-05-21T15:33:23.496401+0000","last_clean":"2024-05-21T15:33:23.496401+0000","last_became_active":"2024-05-21T15:33:23.483584+0000","last_became_peered":"2024-05-21T15:33:23.483584+0000","last_unstale":"2024-05-21T15:33:23.496401+0000","last_undegraded":"2024-05-21T15:33:23.496401+0000","last_fullsized":"2024-05-21T15:33:23.496401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:35:32.411825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.608729+0000","last_change":"2024-05-21T15:32:33.691112+0000","last_active":"2024-05-21T15:33:23.608729+0000","last_peered":"2024-05-21T15:33:23.608729+0000","last_clean":"2024-05-21T15:33:23.608729+0000","last_became_active":"2024-05-21T15:32:33.690726+0000","last_became_peered":"2024-05-21T15:32:33.690726+0000","last_unstale":"2024-05-21T15:33:23.608729+0000","last_undegraded":"2024-05-21T15:33:23.608729+0000","last_fullsized":"2024-05-21T15:33:23.608729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:59:29.002636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375970+0000","last_change":"2024-05-21T15:32:33.179565+0000","last_active":"2024-05-21T15:33:23.375970+0000","last_peered":"2024-05-21T15:33:23.375970+0000","last_clean":"2024-05-21T15:33:23.375970+0000","last_became_active":"2024-05-21T15:32:33.179168+0000","last_became_peered":"2024-05-21T15:32:33.179168+0000","last_unstale":"2024-05-21T15:33:23.375970+0000","last_undegraded":"2024-05-21T15:33:23.375970+0000","last_fullsized":"2024-05-21T15:33:23.375970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:23:39.210931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371649+0000","last_change":"2024-05-21T15:32:33.691348+0000","last_active":"2024-05-21T15:33:23.371649+0000","last_peered":"2024-05-21T15:33:23.371649+0000","last_clean":"2024-05-21T15:33:23.371649+0000","last_became_active":"2024-05-21T15:32:33.691005+0000","last_became_peered":"2024-05-21T15:32:33.691005+0000","last_unstale":"2024-05-21T15:33:23.371649+0000","last_undegraded":"2024-05-21T15:33:23.371649+0000","last_fullsized":"2024-05-21T15:33:23.371649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:33:34.533026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371037+0000","last_change":"2024-05-21T15:32:33.184850+0000","last_active":"2024-05-21T15:33:23.371037+0000","last_peered":"2024-05-21T15:33:23.371037+0000","last_clean":"2024-05-21T15:33:23.371037+0000","last_became_active":"2024-05-21T15:32:33.184480+0000","last_became_peered":"2024-05-21T15:32:33.184480+0000","last_unstale":"2024-05-21T15:33:23.371037+0000","last_undegraded":"2024-05-21T15:33:23.371037+0000","last_fullsized":"2024-05-21T15:33:23.371037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:31.327820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.372154+0000","last_change":"2024-05-21T15:32:33.689520+0000","last_active":"2024-05-21T15:33:23.372154+0000","last_peered":"2024-05-21T15:33:23.372154+0000","last_clean":"2024-05-21T15:33:23.372154+0000","last_became_active":"2024-05-21T15:32:33.688219+0000","last_became_peered":"2024-05-21T15:32:33.688219+0000","last_unstale":"2024-05-21T15:33:23.372154+0000","last_undegraded":"2024-05-21T15:33:23.372154+0000","last_fullsized":"2024-05-21T15:33:23.372154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:36.546390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:47.091577+0000","last_change":"2024-05-21T15:32:33.183183+0000","last_active":"2024-05-21T15:33:47.091577+0000","last_peered":"2024-05-21T15:33:47.091577+0000","last_clean":"2024-05-21T15:33:47.091577+0000","last_became_active":"2024-05-21T15:32:33.182753+0000","last_became_peered":"2024-05-21T15:32:33.182753+0000","last_unstale":"2024-05-21T15:33:47.091577+0000","last_undegraded":"2024-05-21T15:33:47.091577+0000","last_fullsized":"2024-05-21T15:33:47.091577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:44:22.509265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.372179+0000","last_change":"2024-05-21T15:32:33.684019+0000","last_active":"2024-05-21T15:33:23.372179+0000","last_peered":"2024-05-21T15:33:23.372179+0000","last_clean":"2024-05-21T15:33:23.372179+0000","last_became_active":"2024-05-21T15:32:33.683276+0000","last_became_peered":"2024-05-21T15:32:33.683276+0000","last_unstale":"2024-05-21T15:33:23.372179+0000","last_undegraded":"2024-05-21T15:33:23.372179+0000","last_fullsized":"2024-05-21T15:33:23.372179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:07:14.947240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.609033+0000","last_change":"2024-05-21T15:32:33.690956+0000","last_active":"2024-05-21T15:33:23.609033+0000","last_peered":"2024-05-21T15:33:23.609033+0000","last_clean":"2024-05-21T15:33:23.609033+0000","last_became_active":"2024-05-21T15:32:33.690630+0000","last_became_peered":"2024-05-21T15:32:33.690630+0000","last_unstale":"2024-05-21T15:33:23.609033+0000","last_undegraded":"2024-05-21T15:33:23.609033+0000","last_fullsized":"2024-05-21T15:33:23.609033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:24:07.050928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:46.090258+0000","last_change":"2024-05-21T15:32:33.175127+0000","last_active":"2024-05-21T15:33:46.090258+0000","last_peered":"2024-05-21T15:33:46.090258+0000","last_clean":"2024-05-21T15:33:46.090258+0000","last_became_active":"2024-05-21T15:32:33.174724+0000","last_became_peered":"2024-05-21T15:32:33.174724+0000","last_unstale":"2024-05-21T15:33:46.090258+0000","last_undegraded":"2024-05-21T15:33:46.090258+0000","last_fullsized":"2024-05-21T15:33:46.090258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:39:17.499743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.381060+0000","last_change":"2024-05-21T15:32:33.693428+0000","last_active":"2024-05-21T15:33:23.381060+0000","last_peered":"2024-05-21T15:33:23.381060+0000","last_clean":"2024-05-21T15:33:23.381060+0000","last_became_active":"2024-05-21T15:32:33.693077+0000","last_became_peered":"2024-05-21T15:32:33.693077+0000","last_unstale":"2024-05-21T15:33:23.381060+0000","last_undegraded":"2024-05-21T15:33:23.381060+0000","last_fullsized":"2024-05-21T15:33:23.381060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:08:11.908874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.482302+0000","last_change":"2024-05-21T15:32:33.176267+0000","last_active":"2024-05-21T15:33:23.482302+0000","last_peered":"2024-05-21T15:33:23.482302+0000","last_clean":"2024-05-21T15:33:23.482302+0000","last_became_active":"2024-05-21T15:32:33.175856+0000","last_became_peered":"2024-05-21T15:32:33.175856+0000","last_unstale":"2024-05-21T15:33:23.482302+0000","last_undegraded":"2024-05-21T15:33:23.482302+0000","last_fullsized":"2024-05-21T15:33:23.482302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:58.113791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":343,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374396+0000","last_change":"2024-05-21T15:31:20.928694+0000","last_active":"2024-05-21T15:33:23.374396+0000","last_peered":"2024-05-21T15:33:23.374396+0000","last_clean":"2024-05-21T15:33:23.374396+0000","last_became_active":"2024-05-21T15:31:20.928115+0000","last_became_peered":"2024-05-21T15:31:20.928115+0000","last_unstale":"2024-05-21T15:33:23.374396+0000","last_undegraded":"2024-05-21T15:33:23.374396+0000","last_fullsized":"2024-05-21T15:33:23.374396+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:27:50.788960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:27:50.788960+0000","last_clean_scrub_stamp":"2024-05-21T15:27:50.788960+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:55:10.393374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375249+0000","last_change":"2024-05-21T15:32:33.689305+0000","last_active":"2024-05-21T15:33:23.375249+0000","last_peered":"2024-05-21T15:33:23.375249+0000","last_clean":"2024-05-21T15:33:23.375249+0000","last_became_active":"2024-05-21T15:32:33.688908+0000","last_became_peered":"2024-05-21T15:32:33.688908+0000","last_unstale":"2024-05-21T15:33:23.375249+0000","last_undegraded":"2024-05-21T15:33:23.375249+0000","last_fullsized":"2024-05-21T15:33:23.375249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:49:18.953116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374568+0000","last_change":"2024-05-21T15:32:33.187781+0000","last_active":"2024-05-21T15:33:23.374568+0000","last_peered":"2024-05-21T15:33:23.374568+0000","last_clean":"2024-05-21T15:33:23.374568+0000","last_became_active":"2024-05-21T15:32:33.187504+0000","last_became_peered":"2024-05-21T15:32:33.187504+0000","last_unstale":"2024-05-21T15:33:23.374568+0000","last_undegraded":"2024-05-21T15:33:23.374568+0000","last_fullsized":"2024-05-21T15:33:23.374568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:30.400209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375721+0000","last_change":"2024-05-21T15:32:33.188706+0000","last_active":"2024-05-21T15:33:23.375721+0000","last_peered":"2024-05-21T15:33:23.375721+0000","last_clean":"2024-05-21T15:33:23.375721+0000","last_became_active":"2024-05-21T15:32:33.187462+0000","last_became_peered":"2024-05-21T15:32:33.187462+0000","last_unstale":"2024-05-21T15:33:23.375721+0000","last_undegraded":"2024-05-21T15:33:23.375721+0000","last_fullsized":"2024-05-21T15:33:23.375721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:47:45.387721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375432+0000","last_change":"2024-05-21T15:32:33.672531+0000","last_active":"2024-05-21T15:33:23.375432+0000","last_peered":"2024-05-21T15:33:23.375432+0000","last_clean":"2024-05-21T15:33:23.375432+0000","last_became_active":"2024-05-21T15:32:33.671972+0000","last_became_peered":"2024-05-21T15:32:33.671972+0000","last_unstale":"2024-05-21T15:33:23.375432+0000","last_undegraded":"2024-05-21T15:33:23.375432+0000","last_fullsized":"2024-05-21T15:33:23.375432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:10:51.062530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371212+0000","last_change":"2024-05-21T15:32:33.181274+0000","last_active":"2024-05-21T15:33:23.371212+0000","last_peered":"2024-05-21T15:33:23.371212+0000","last_clean":"2024-05-21T15:33:23.371212+0000","last_became_active":"2024-05-21T15:32:33.180885+0000","last_became_peered":"2024-05-21T15:32:33.180885+0000","last_unstale":"2024-05-21T15:33:23.371212+0000","last_undegraded":"2024-05-21T15:33:23.371212+0000","last_fullsized":"2024-05-21T15:33:23.371212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:10:13.547380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.382009+0000","last_change":"2024-05-21T15:32:33.683749+0000","last_active":"2024-05-21T15:33:23.382009+0000","last_peered":"2024-05-21T15:33:23.382009+0000","last_clean":"2024-05-21T15:33:23.382009+0000","last_became_active":"2024-05-21T15:32:33.683393+0000","last_became_peered":"2024-05-21T15:32:33.683393+0000","last_unstale":"2024-05-21T15:33:23.382009+0000","last_undegraded":"2024-05-21T15:33:23.382009+0000","last_fullsized":"2024-05-21T15:33:23.382009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:07:53.009651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.482059+0000","last_change":"2024-05-21T15:32:33.184740+0000","last_active":"2024-05-21T15:33:23.482059+0000","last_peered":"2024-05-21T15:33:23.482059+0000","last_clean":"2024-05-21T15:33:23.482059+0000","last_became_active":"2024-05-21T15:32:33.184378+0000","last_became_peered":"2024-05-21T15:32:33.184378+0000","last_unstale":"2024-05-21T15:33:23.482059+0000","last_undegraded":"2024-05-21T15:33:23.482059+0000","last_fullsized":"2024-05-21T15:33:23.482059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:21:27.610513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.381090+0000","last_change":"2024-05-21T15:33:23.381090+0000","last_active":"2024-05-21T15:33:23.381090+0000","last_peered":"2024-05-21T15:33:23.381090+0000","last_clean":"2024-05-21T15:33:23.381090+0000","last_became_active":"2024-05-21T15:33:23.380680+0000","last_became_peered":"2024-05-21T15:33:23.380680+0000","last_unstale":"2024-05-21T15:33:23.381090+0000","last_undegraded":"2024-05-21T15:33:23.381090+0000","last_fullsized":"2024-05-21T15:33:23.381090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:26:03.371476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.608845+0000","last_change":"2024-05-21T15:32:33.183344+0000","last_active":"2024-05-21T15:33:23.608845+0000","last_peered":"2024-05-21T15:33:23.608845+0000","last_clean":"2024-05-21T15:33:23.608845+0000","last_became_active":"2024-05-21T15:32:33.182977+0000","last_became_peered":"2024-05-21T15:32:33.182977+0000","last_unstale":"2024-05-21T15:33:23.608845+0000","last_undegraded":"2024-05-21T15:33:23.608845+0000","last_fullsized":"2024-05-21T15:33:23.608845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:27:24.354129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374293+0000","last_change":"2024-05-21T15:32:33.677549+0000","last_active":"2024-05-21T15:33:23.374293+0000","last_peered":"2024-05-21T15:33:23.374293+0000","last_clean":"2024-05-21T15:33:23.374293+0000","last_became_active":"2024-05-21T15:32:33.677213+0000","last_became_peered":"2024-05-21T15:32:33.677213+0000","last_unstale":"2024-05-21T15:33:23.374293+0000","last_undegraded":"2024-05-21T15:33:23.374293+0000","last_fullsized":"2024-05-21T15:33:23.374293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:57:06.560434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.380435+0000","last_change":"2024-05-21T15:32:33.190023+0000","last_active":"2024-05-21T15:33:23.380435+0000","last_peered":"2024-05-21T15:33:23.380435+0000","last_clean":"2024-05-21T15:33:23.380435+0000","last_became_active":"2024-05-21T15:32:33.189616+0000","last_became_peered":"2024-05-21T15:32:33.189616+0000","last_unstale":"2024-05-21T15:33:23.380435+0000","last_undegraded":"2024-05-21T15:33:23.380435+0000","last_fullsized":"2024-05-21T15:33:23.380435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:03.841414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371428+0000","last_change":"2024-05-21T15:32:33.685911+0000","last_active":"2024-05-21T15:33:23.371428+0000","last_peered":"2024-05-21T15:33:23.371428+0000","last_clean":"2024-05-21T15:33:23.371428+0000","last_became_active":"2024-05-21T15:32:33.684785+0000","last_became_peered":"2024-05-21T15:32:33.684785+0000","last_unstale":"2024-05-21T15:33:23.371428+0000","last_undegraded":"2024-05-21T15:33:23.371428+0000","last_fullsized":"2024-05-21T15:33:23.371428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:39:18.370289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374839+0000","last_change":"2024-05-21T15:32:33.178354+0000","last_active":"2024-05-21T15:33:23.374839+0000","last_peered":"2024-05-21T15:33:23.374839+0000","last_clean":"2024-05-21T15:33:23.374839+0000","last_became_active":"2024-05-21T15:32:33.177989+0000","last_became_peered":"2024-05-21T15:32:33.177989+0000","last_unstale":"2024-05-21T15:33:23.374839+0000","last_undegraded":"2024-05-21T15:33:23.374839+0000","last_fullsized":"2024-05-21T15:33:23.374839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:58.650831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.380349+0000","last_change":"2024-05-21T15:32:33.686467+0000","last_active":"2024-05-21T15:33:23.380349+0000","last_peered":"2024-05-21T15:33:23.380349+0000","last_clean":"2024-05-21T15:33:23.380349+0000","last_became_active":"2024-05-21T15:32:33.686135+0000","last_became_peered":"2024-05-21T15:32:33.686135+0000","last_unstale":"2024-05-21T15:33:23.380349+0000","last_undegraded":"2024-05-21T15:33:23.380349+0000","last_fullsized":"2024-05-21T15:33:23.380349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:53:54.045591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.383702+0000","last_change":"2024-05-21T15:33:23.383157+0000","last_active":"2024-05-21T15:33:23.383702+0000","last_peered":"2024-05-21T15:33:23.383702+0000","last_clean":"2024-05-21T15:33:23.383702+0000","last_became_active":"2024-05-21T15:33:23.376834+0000","last_became_peered":"2024-05-21T15:33:23.376834+0000","last_unstale":"2024-05-21T15:33:23.383702+0000","last_undegraded":"2024-05-21T15:33:23.383702+0000","last_fullsized":"2024-05-21T15:33:23.383702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:01:19.349486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374119+0000","last_change":"2024-05-21T15:32:33.692325+0000","last_active":"2024-05-21T15:33:23.374119+0000","last_peered":"2024-05-21T15:33:23.374119+0000","last_clean":"2024-05-21T15:33:23.374119+0000","last_became_active":"2024-05-21T15:32:33.691971+0000","last_became_peered":"2024-05-21T15:32:33.691971+0000","last_unstale":"2024-05-21T15:33:23.374119+0000","last_undegraded":"2024-05-21T15:33:23.374119+0000","last_fullsized":"2024-05-21T15:33:23.374119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:53:28.190063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:48.110320+0000","last_change":"2024-05-21T15:32:33.186925+0000","last_active":"2024-05-21T15:33:48.110320+0000","last_peered":"2024-05-21T15:33:48.110320+0000","last_clean":"2024-05-21T15:33:48.110320+0000","last_became_active":"2024-05-21T15:32:33.186605+0000","last_became_peered":"2024-05-21T15:32:33.186605+0000","last_unstale":"2024-05-21T15:33:48.110320+0000","last_undegraded":"2024-05-21T15:33:48.110320+0000","last_fullsized":"2024-05-21T15:33:48.110320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:32.752746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374493+0000","last_change":"2024-05-21T15:32:33.686132+0000","last_active":"2024-05-21T15:33:23.374493+0000","last_peered":"2024-05-21T15:33:23.374493+0000","last_clean":"2024-05-21T15:33:23.374493+0000","last_became_active":"2024-05-21T15:32:33.685745+0000","last_became_peered":"2024-05-21T15:32:33.685745+0000","last_unstale":"2024-05-21T15:33:23.374493+0000","last_undegraded":"2024-05-21T15:33:23.374493+0000","last_fullsized":"2024-05-21T15:33:23.374493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:51:49.522297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.609134+0000","last_change":"2024-05-21T15:32:33.176552+0000","last_active":"2024-05-21T15:33:23.609134+0000","last_peered":"2024-05-21T15:33:23.609134+0000","last_clean":"2024-05-21T15:33:23.609134+0000","last_became_active":"2024-05-21T15:32:33.176237+0000","last_became_peered":"2024-05-21T15:32:33.176237+0000","last_unstale":"2024-05-21T15:33:23.609134+0000","last_undegraded":"2024-05-21T15:33:23.609134+0000","last_fullsized":"2024-05-21T15:33:23.609134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:09.098344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374365+0000","last_change":"2024-05-21T15:32:33.692310+0000","last_active":"2024-05-21T15:33:23.374365+0000","last_peered":"2024-05-21T15:33:23.374365+0000","last_clean":"2024-05-21T15:33:23.374365+0000","last_became_active":"2024-05-21T15:32:33.691959+0000","last_became_peered":"2024-05-21T15:32:33.691959+0000","last_unstale":"2024-05-21T15:33:23.374365+0000","last_undegraded":"2024-05-21T15:33:23.374365+0000","last_fullsized":"2024-05-21T15:33:23.374365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:57:38.243926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.614280+0000","last_change":"2024-05-21T15:33:23.614280+0000","last_active":"2024-05-21T15:33:23.614280+0000","last_peered":"2024-05-21T15:33:23.614280+0000","last_clean":"2024-05-21T15:33:23.614280+0000","last_became_active":"2024-05-21T15:33:23.610364+0000","last_became_peered":"2024-05-21T15:33:23.610364+0000","last_unstale":"2024-05-21T15:33:23.614280+0000","last_undegraded":"2024-05-21T15:33:23.614280+0000","last_fullsized":"2024-05-21T15:33:23.614280+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:40:42.990949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374327+0000","last_change":"2024-05-21T15:32:33.682801+0000","last_active":"2024-05-21T15:33:23.374327+0000","last_peered":"2024-05-21T15:33:23.374327+0000","last_clean":"2024-05-21T15:33:23.374327+0000","last_became_active":"2024-05-21T15:32:33.682368+0000","last_became_peered":"2024-05-21T15:32:33.682368+0000","last_unstale":"2024-05-21T15:33:23.374327+0000","last_undegraded":"2024-05-21T15:33:23.374327+0000","last_fullsized":"2024-05-21T15:33:23.374327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:56:50.520672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371528+0000","last_change":"2024-05-21T15:32:33.684809+0000","last_active":"2024-05-21T15:33:23.371528+0000","last_peered":"2024-05-21T15:33:23.371528+0000","last_clean":"2024-05-21T15:33:23.371528+0000","last_became_active":"2024-05-21T15:32:33.684067+0000","last_became_peered":"2024-05-21T15:32:33.684067+0000","last_unstale":"2024-05-21T15:33:23.371528+0000","last_undegraded":"2024-05-21T15:33:23.371528+0000","last_fullsized":"2024-05-21T15:33:23.371528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:31:58.997470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375461+0000","last_change":"2024-05-21T15:32:33.676091+0000","last_active":"2024-05-21T15:33:23.375461+0000","last_peered":"2024-05-21T15:33:23.375461+0000","last_clean":"2024-05-21T15:33:23.375461+0000","last_became_active":"2024-05-21T15:32:33.675538+0000","last_became_peered":"2024-05-21T15:32:33.675538+0000","last_unstale":"2024-05-21T15:33:23.375461+0000","last_undegraded":"2024-05-21T15:33:23.375461+0000","last_fullsized":"2024-05-21T15:33:23.375461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:50.311152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.373902+0000","last_change":"2024-05-21T15:32:33.688590+0000","last_active":"2024-05-21T15:33:23.373902+0000","last_peered":"2024-05-21T15:33:23.373902+0000","last_clean":"2024-05-21T15:33:23.373902+0000","last_became_active":"2024-05-21T15:32:33.687993+0000","last_became_peered":"2024-05-21T15:32:33.687993+0000","last_unstale":"2024-05-21T15:33:23.373902+0000","last_undegraded":"2024-05-21T15:33:23.373902+0000","last_fullsized":"2024-05-21T15:33:23.373902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:21.939483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.380890+0000","last_change":"2024-05-21T15:33:23.380890+0000","last_active":"2024-05-21T15:33:23.380890+0000","last_peered":"2024-05-21T15:33:23.380890+0000","last_clean":"2024-05-21T15:33:23.380890+0000","last_became_active":"2024-05-21T15:33:23.380503+0000","last_became_peered":"2024-05-21T15:33:23.380503+0000","last_unstale":"2024-05-21T15:33:23.380890+0000","last_undegraded":"2024-05-21T15:33:23.380890+0000","last_fullsized":"2024-05-21T15:33:23.380890+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:41.070336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375260+0000","last_change":"2024-05-21T15:32:33.689365+0000","last_active":"2024-05-21T15:33:23.375260+0000","last_peered":"2024-05-21T15:33:23.375260+0000","last_clean":"2024-05-21T15:33:23.375260+0000","last_became_active":"2024-05-21T15:32:33.688739+0000","last_became_peered":"2024-05-21T15:32:33.688739+0000","last_unstale":"2024-05-21T15:33:23.375260+0000","last_undegraded":"2024-05-21T15:33:23.375260+0000","last_fullsized":"2024-05-21T15:33:23.375260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:19:52.614155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371884+0000","last_change":"2024-05-21T15:32:33.681985+0000","last_active":"2024-05-21T15:33:23.371884+0000","last_peered":"2024-05-21T15:33:23.371884+0000","last_clean":"2024-05-21T15:33:23.371884+0000","last_became_active":"2024-05-21T15:32:33.681209+0000","last_became_peered":"2024-05-21T15:32:33.681209+0000","last_unstale":"2024-05-21T15:33:23.371884+0000","last_undegraded":"2024-05-21T15:33:23.371884+0000","last_fullsized":"2024-05-21T15:33:23.371884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:03:16.088136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375886+0000","last_change":"2024-05-21T15:32:33.673236+0000","last_active":"2024-05-21T15:33:23.375886+0000","last_peered":"2024-05-21T15:33:23.375886+0000","last_clean":"2024-05-21T15:33:23.375886+0000","last_became_active":"2024-05-21T15:32:33.672548+0000","last_became_peered":"2024-05-21T15:32:33.672548+0000","last_unstale":"2024-05-21T15:33:23.375886+0000","last_undegraded":"2024-05-21T15:33:23.375886+0000","last_fullsized":"2024-05-21T15:33:23.375886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:52:10.010219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.483327+0000","last_change":"2024-05-21T15:33:23.482874+0000","last_active":"2024-05-21T15:33:23.483327+0000","last_peered":"2024-05-21T15:33:23.483327+0000","last_clean":"2024-05-21T15:33:23.483327+0000","last_became_active":"2024-05-21T15:33:23.482589+0000","last_became_peered":"2024-05-21T15:33:23.482589+0000","last_unstale":"2024-05-21T15:33:23.483327+0000","last_undegraded":"2024-05-21T15:33:23.483327+0000","last_fullsized":"2024-05-21T15:33:23.483327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:19:45.870562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.608907+0000","last_change":"2024-05-21T15:32:33.686992+0000","last_active":"2024-05-21T15:33:23.608907+0000","last_peered":"2024-05-21T15:33:23.608907+0000","last_clean":"2024-05-21T15:33:23.608907+0000","last_became_active":"2024-05-21T15:32:33.686673+0000","last_became_peered":"2024-05-21T15:32:33.686673+0000","last_unstale":"2024-05-21T15:33:23.608907+0000","last_undegraded":"2024-05-21T15:33:23.608907+0000","last_fullsized":"2024-05-21T15:33:23.608907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:18:20.668365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.380311+0000","last_change":"2024-05-21T15:32:33.689433+0000","last_active":"2024-05-21T15:33:23.380311+0000","last_peered":"2024-05-21T15:33:23.380311+0000","last_clean":"2024-05-21T15:33:23.380311+0000","last_became_active":"2024-05-21T15:32:33.689106+0000","last_became_peered":"2024-05-21T15:32:33.689106+0000","last_unstale":"2024-05-21T15:33:23.380311+0000","last_undegraded":"2024-05-21T15:33:23.380311+0000","last_fullsized":"2024-05-21T15:33:23.380311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:45:17.841159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.608826+0000","last_change":"2024-05-21T15:32:33.687763+0000","last_active":"2024-05-21T15:33:23.608826+0000","last_peered":"2024-05-21T15:33:23.608826+0000","last_clean":"2024-05-21T15:33:23.608826+0000","last_became_active":"2024-05-21T15:32:33.685103+0000","last_became_peered":"2024-05-21T15:32:33.685103+0000","last_unstale":"2024-05-21T15:33:23.608826+0000","last_undegraded":"2024-05-21T15:33:23.608826+0000","last_fullsized":"2024-05-21T15:33:23.608826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:56:39.454536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375636+0000","last_change":"2024-05-21T15:32:33.679225+0000","last_active":"2024-05-21T15:33:23.375636+0000","last_peered":"2024-05-21T15:33:23.375636+0000","last_clean":"2024-05-21T15:33:23.375636+0000","last_became_active":"2024-05-21T15:32:33.678890+0000","last_became_peered":"2024-05-21T15:32:33.678890+0000","last_unstale":"2024-05-21T15:33:23.375636+0000","last_undegraded":"2024-05-21T15:33:23.375636+0000","last_fullsized":"2024-05-21T15:33:23.375636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:12:23.721449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371767+0000","last_change":"2024-05-21T15:32:33.688659+0000","last_active":"2024-05-21T15:33:23.371767+0000","last_peered":"2024-05-21T15:33:23.371767+0000","last_clean":"2024-05-21T15:33:23.371767+0000","last_became_active":"2024-05-21T15:32:33.688293+0000","last_became_peered":"2024-05-21T15:32:33.688293+0000","last_unstale":"2024-05-21T15:33:23.371767+0000","last_undegraded":"2024-05-21T15:33:23.371767+0000","last_fullsized":"2024-05-21T15:33:23.371767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:08:49.543237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":287606,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66278,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":720,"num_read_kb":429,"num_write":235,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":198834,"data_compressed":5562,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351004,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":460368,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612640,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1110016,"data_stored":1110471,"data_compressed":44221,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":459907,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":978944,"data_stored":460285,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1052412,"data_compressed":42367,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462966,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":459624,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":520002,"data_compressed":7653,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":461354,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":520501,"data_compressed":7653,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":459449,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888130,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":459597,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":1052090,"data_compressed":42367,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61217,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61716,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61201,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T15:33:54.316 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T15:33:54.569 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:55.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:54 smithi038 ceph-mon[31995]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:33:55.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:54 smithi053 ceph-mon[36496]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:33:55.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:54 smithi045 ceph-mon[36409]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:33:56.015 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:55 smithi038 ceph-mon[31995]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-21T15:33:56.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:55 smithi053 ceph-mon[36496]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-21T15:33:56.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:55 smithi045 ceph-mon[36409]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-21T15:33:56.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.261+0000 7f35d25f0700 1 -- 172.21.15.38:0/319135032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35cc108150 msgr2=0x7f35cc108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:56.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.261+0000 7f35d25f0700 1 --2- 172.21.15.38:0/319135032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35cc108150 0x7f35cc108560 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009990 tx=0x7f35bc009ca0 comp rx=0 tx=0).stop 2024-05-21T15:33:56.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.261+0000 7f35d25f0700 1 -- 172.21.15.38:0/319135032 shutdown_connections 2024-05-21T15:33:56.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.261+0000 7f35d25f0700 1 --2- 172.21.15.38:0/319135032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc1031b0 0x7f35cc10ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:56.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.261+0000 7f35d25f0700 1 --2- 172.21.15.38:0/319135032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35cc108150 0x7f35cc108560 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:56.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.261+0000 7f35d25f0700 1 --2- 172.21.15.38:0/319135032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35cc102860 0x7f35cc102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:56.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.261+0000 7f35d25f0700 1 -- 172.21.15.38:0/319135032 >> 172.21.15.38:0/319135032 conn(0x7f35cc0fd3c0 msgr2=0x7f35cc0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:56.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.261+0000 7f35d25f0700 1 -- 172.21.15.38:0/319135032 shutdown_connections 2024-05-21T15:33:56.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.261+0000 7f35d25f0700 1 -- 172.21.15.38:0/319135032 wait complete. 2024-05-21T15:33:56.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.262+0000 7f35d25f0700 1 Processor -- start 2024-05-21T15:33:56.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.262+0000 7f35d25f0700 1 -- start start 2024-05-21T15:33:56.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.262+0000 7f35d25f0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc102860 0x7f35cc10f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:56.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.262+0000 7f35d25f0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35cc1031b0 0x7f35cc10fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:56.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.262+0000 7f35d25f0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35cc108150 0x7f35cc1103a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:56.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.262+0000 7f35d25f0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc113500 con 0x7f35cc1031b0 2024-05-21T15:33:56.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.262+0000 7f35d25f0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc113380 con 0x7f35cc102860 2024-05-21T15:33:56.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.262+0000 7f35d25f0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc113680 con 0x7f35cc108150 2024-05-21T15:33:56.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.262+0000 7f35cb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35cc1031b0 0x7f35cc10fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:56.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.262+0000 7f35cb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35cc1031b0 0x7f35cc10fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51192/0 (socket says 172.21.15.38:51192) 2024-05-21T15:33:56.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.262+0000 7f35cb7fe700 1 -- 172.21.15.38:0/358354027 learned_addr learned my addr 172.21.15.38:0/358354027 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:56.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.263+0000 7f35d0b8d700 1 --2- 172.21.15.38:0/358354027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35cc108150 0x7f35cc1103a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:56.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.262+0000 7f35cbfff700 1 --2- 172.21.15.38:0/358354027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc102860 0x7f35cc10f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:56.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.263+0000 7f35cb7fe700 1 -- 172.21.15.38:0/358354027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35cc108150 msgr2=0x7f35cc1103a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.263+0000 7f35cb7fe700 1 --2- 172.21.15.38:0/358354027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35cc108150 0x7f35cc1103a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.263+0000 7f35cb7fe700 1 -- 172.21.15.38:0/358354027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc102860 msgr2=0x7f35cc10f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.263+0000 7f35cb7fe700 1 --2- 172.21.15.38:0/358354027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc102860 0x7f35cc10f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.263+0000 7f35cb7fe700 1 -- 172.21.15.38:0/358354027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc1a8190 con 0x7f35cc1031b0 2024-05-21T15:33:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.263+0000 7f35cb7fe700 1 --2- 172.21.15.38:0/358354027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35cc1031b0 0x7f35cc10fe60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f35c000d980 tx=0x7f35c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.263+0000 7f35c97fa700 1 -- 172.21.15.38:0/358354027 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0017070 con 0x7f35cc1031b0 2024-05-21T15:33:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.263+0000 7f35d25f0700 1 -- 172.21.15.38:0/358354027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35cc1a8480 con 0x7f35cc1031b0 2024-05-21T15:33:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.263+0000 7f35c97fa700 1 -- 172.21.15.38:0/358354027 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c000fd70 con 0x7f35cc1031b0 2024-05-21T15:33:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.263+0000 7f35c97fa700 1 -- 172.21.15.38:0/358354027 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c000e3b0 con 0x7f35cc1031b0 2024-05-21T15:33:56.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.263+0000 7f35d25f0700 1 -- 172.21.15.38:0/358354027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35cc1a89c0 con 0x7f35cc1031b0 2024-05-21T15:33:56.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.264+0000 7f35d25f0700 1 -- 172.21.15.38:0/358354027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35cc10a5c0 con 0x7f35cc1031b0 2024-05-21T15:33:56.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.266+0000 7f35c97fa700 1 -- 172.21.15.38:0/358354027 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f35c000f890 con 0x7f35cc1031b0 2024-05-21T15:33:56.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.266+0000 7f35c97fa700 1 --2- 172.21.15.38:0/358354027 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f35b40a2c20 0x7f35b40a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:56.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.267+0000 7f35cbfff700 1 --2- 172.21.15.38:0/358354027 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f35b40a2c20 0x7f35b40a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:56.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.267+0000 7f35c97fa700 1 -- 172.21.15.38:0/358354027 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f35c00ceb80 con 0x7f35cc1031b0 2024-05-21T15:33:56.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.267+0000 7f35cbfff700 1 --2- 172.21.15.38:0/358354027 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f35b40a2c20 0x7f35b40a5130 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00b5c0 tx=0x7f35bc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:56.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.273+0000 7f35c97fa700 1 -- 172.21.15.38:0/358354027 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f35c009ba70 con 0x7f35cc1031b0 2024-05-21T15:33:56.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.473+0000 7f35d25f0700 1 -- 172.21.15.38:0/358354027 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35cc110e50 con 0x7f35b40a2c20 2024-05-21T15:33:56.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.481+0000 7f35c97fa700 1 -- 172.21.15.38:0/358354027 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259251 (secure 0 0 0) 0x7f35cc110e50 con 0x7f35b40a2c20 2024-05-21T15:33:56.484 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:33:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.485+0000 7f35d25f0700 1 -- 172.21.15.38:0/358354027 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f35b40a2c20 msgr2=0x7f35b40a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.485+0000 7f35d25f0700 1 --2- 172.21.15.38:0/358354027 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f35b40a2c20 0x7f35b40a5130 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00b5c0 tx=0x7f35bc01a040 comp rx=0 tx=0).stop 2024-05-21T15:33:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.485+0000 7f35d25f0700 1 -- 172.21.15.38:0/358354027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35cc1031b0 msgr2=0x7f35cc10fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.485+0000 7f35d25f0700 1 --2- 172.21.15.38:0/358354027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35cc1031b0 0x7f35cc10fe60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f35c000d980 tx=0x7f35c000dc90 comp rx=0 tx=0).stop 2024-05-21T15:33:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.485+0000 7f35d25f0700 1 -- 172.21.15.38:0/358354027 shutdown_connections 2024-05-21T15:33:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.485+0000 7f35d25f0700 1 --2- 172.21.15.38:0/358354027 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f35b40a2c20 0x7f35b40a5130 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:56.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.485+0000 7f35d25f0700 1 --2- 172.21.15.38:0/358354027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35cc108150 0x7f35cc1103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:56.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.485+0000 7f35d25f0700 1 --2- 172.21.15.38:0/358354027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc102860 0x7f35cc10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:56.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.485+0000 7f35d25f0700 1 --2- 172.21.15.38:0/358354027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35cc1031b0 0x7f35cc10fe60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:56.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.486+0000 7f35d25f0700 1 -- 172.21.15.38:0/358354027 >> 172.21.15.38:0/358354027 conn(0x7f35cc0fd3c0 msgr2=0x7f35cc0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:56.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.486+0000 7f35d25f0700 1 -- 172.21.15.38:0/358354027 shutdown_connections 2024-05-21T15:33:56.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:56.486+0000 7f35d25f0700 1 -- 172.21.15.38:0/358354027 wait complete. 2024-05-21T15:33:56.487 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-05-21T15:33:57.195 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-05-21T15:33:56.237726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657270,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":882,"num_read_kb":559,"num_write":423,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":714,"ondisk_log_size":714,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373456,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568368,"statfs":{"total":1151940427776,"available":1151558008832,"internally_reserved":0,"allocated":12079104,"data_stored":7477089,"data_compressed":184854,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":51,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003836"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.257218+0000","last_change":"2024-05-21T15:32:33.687244+0000","last_active":"2024-05-21T15:33:52.257218+0000","last_peered":"2024-05-21T15:33:52.257218+0000","last_clean":"2024-05-21T15:33:52.257218+0000","last_became_active":"2024-05-21T15:32:33.686515+0000","last_became_peered":"2024-05-21T15:32:33.686515+0000","last_unstale":"2024-05-21T15:33:52.257218+0000","last_undegraded":"2024-05-21T15:33:52.257218+0000","last_fullsized":"2024-05-21T15:33:52.257218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:10:18.396997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375764+0000","last_change":"2024-05-21T15:32:33.678497+0000","last_active":"2024-05-21T15:33:23.375764+0000","last_peered":"2024-05-21T15:33:23.375764+0000","last_clean":"2024-05-21T15:33:23.375764+0000","last_became_active":"2024-05-21T15:32:33.678001+0000","last_became_peered":"2024-05-21T15:32:33.678001+0000","last_unstale":"2024-05-21T15:33:23.375764+0000","last_undegraded":"2024-05-21T15:33:23.375764+0000","last_fullsized":"2024-05-21T15:33:23.375764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:14:48.255675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375514+0000","last_change":"2024-05-21T15:32:33.692444+0000","last_active":"2024-05-21T15:33:23.375514+0000","last_peered":"2024-05-21T15:33:23.375514+0000","last_clean":"2024-05-21T15:33:23.375514+0000","last_became_active":"2024-05-21T15:32:33.691662+0000","last_became_peered":"2024-05-21T15:32:33.691662+0000","last_unstale":"2024-05-21T15:33:23.375514+0000","last_undegraded":"2024-05-21T15:33:23.375514+0000","last_fullsized":"2024-05-21T15:33:23.375514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:26:25.929879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.482157+0000","last_change":"2024-05-21T15:32:33.692297+0000","last_active":"2024-05-21T15:33:23.482157+0000","last_peered":"2024-05-21T15:33:23.482157+0000","last_clean":"2024-05-21T15:33:23.482157+0000","last_became_active":"2024-05-21T15:32:33.691925+0000","last_became_peered":"2024-05-21T15:32:33.691925+0000","last_unstale":"2024-05-21T15:33:23.482157+0000","last_undegraded":"2024-05-21T15:33:23.482157+0000","last_fullsized":"2024-05-21T15:33:23.482157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:40:45.448485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.220632+0000","last_change":"2024-05-21T15:32:33.685511+0000","last_active":"2024-05-21T15:33:52.220632+0000","last_peered":"2024-05-21T15:33:52.220632+0000","last_clean":"2024-05-21T15:33:52.220632+0000","last_became_active":"2024-05-21T15:32:33.685182+0000","last_became_peered":"2024-05-21T15:32:33.685182+0000","last_unstale":"2024-05-21T15:33:52.220632+0000","last_undegraded":"2024-05-21T15:33:52.220632+0000","last_fullsized":"2024-05-21T15:33:52.220632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:59:06.680712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.235643+0000","last_change":"2024-05-21T15:32:33.682400+0000","last_active":"2024-05-21T15:33:52.235643+0000","last_peered":"2024-05-21T15:33:52.235643+0000","last_clean":"2024-05-21T15:33:52.235643+0000","last_became_active":"2024-05-21T15:32:33.682068+0000","last_became_peered":"2024-05-21T15:32:33.682068+0000","last_unstale":"2024-05-21T15:33:52.235643+0000","last_undegraded":"2024-05-21T15:33:52.235643+0000","last_fullsized":"2024-05-21T15:33:52.235643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:31:50.968706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.242566+0000","last_change":"2024-05-21T15:32:33.690099+0000","last_active":"2024-05-21T15:33:52.242566+0000","last_peered":"2024-05-21T15:33:52.242566+0000","last_clean":"2024-05-21T15:33:52.242566+0000","last_became_active":"2024-05-21T15:32:33.689762+0000","last_became_peered":"2024-05-21T15:32:33.689762+0000","last_unstale":"2024-05-21T15:33:52.242566+0000","last_undegraded":"2024-05-21T15:33:52.242566+0000","last_fullsized":"2024-05-21T15:33:52.242566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:55:20.331097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.246040+0000","last_change":"2024-05-21T15:32:33.691701+0000","last_active":"2024-05-21T15:33:52.246040+0000","last_peered":"2024-05-21T15:33:52.246040+0000","last_clean":"2024-05-21T15:33:52.246040+0000","last_became_active":"2024-05-21T15:32:33.691386+0000","last_became_peered":"2024-05-21T15:32:33.691386+0000","last_unstale":"2024-05-21T15:33:52.246040+0000","last_undegraded":"2024-05-21T15:33:52.246040+0000","last_fullsized":"2024-05-21T15:33:52.246040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:54:45.691392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.255874+0000","last_change":"2024-05-21T15:32:33.677484+0000","last_active":"2024-05-21T15:33:52.255874+0000","last_peered":"2024-05-21T15:33:52.255874+0000","last_clean":"2024-05-21T15:33:52.255874+0000","last_became_active":"2024-05-21T15:32:33.677160+0000","last_became_peered":"2024-05-21T15:32:33.677160+0000","last_unstale":"2024-05-21T15:33:52.255874+0000","last_undegraded":"2024-05-21T15:33:52.255874+0000","last_fullsized":"2024-05-21T15:33:52.255874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:38:00.914393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.246642+0000","last_change":"2024-05-21T15:32:33.686957+0000","last_active":"2024-05-21T15:33:52.246642+0000","last_peered":"2024-05-21T15:33:52.246642+0000","last_clean":"2024-05-21T15:33:52.246642+0000","last_became_active":"2024-05-21T15:32:33.685338+0000","last_became_peered":"2024-05-21T15:32:33.685338+0000","last_unstale":"2024-05-21T15:33:52.246642+0000","last_undegraded":"2024-05-21T15:33:52.246642+0000","last_fullsized":"2024-05-21T15:33:52.246642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:04:58.115974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.248094+0000","last_change":"2024-05-21T15:32:33.693311+0000","last_active":"2024-05-21T15:33:52.248094+0000","last_peered":"2024-05-21T15:33:52.248094+0000","last_clean":"2024-05-21T15:33:52.248094+0000","last_became_active":"2024-05-21T15:32:33.692959+0000","last_became_peered":"2024-05-21T15:32:33.692959+0000","last_unstale":"2024-05-21T15:33:52.248094+0000","last_undegraded":"2024-05-21T15:33:52.248094+0000","last_fullsized":"2024-05-21T15:33:52.248094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:35:12.918524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.481881+0000","last_change":"2024-05-21T15:32:33.693254+0000","last_active":"2024-05-21T15:33:23.481881+0000","last_peered":"2024-05-21T15:33:23.481881+0000","last_clean":"2024-05-21T15:33:23.481881+0000","last_became_active":"2024-05-21T15:32:33.692901+0000","last_became_peered":"2024-05-21T15:32:33.692901+0000","last_unstale":"2024-05-21T15:33:23.481881+0000","last_undegraded":"2024-05-21T15:33:23.481881+0000","last_fullsized":"2024-05-21T15:33:23.481881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:36:30.591479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.242538+0000","last_change":"2024-05-21T15:33:23.383656+0000","last_active":"2024-05-21T15:33:52.242538+0000","last_peered":"2024-05-21T15:33:52.242538+0000","last_clean":"2024-05-21T15:33:52.242538+0000","last_became_active":"2024-05-21T15:33:23.383113+0000","last_became_peered":"2024-05-21T15:33:23.383113+0000","last_unstale":"2024-05-21T15:33:52.242538+0000","last_undegraded":"2024-05-21T15:33:52.242538+0000","last_fullsized":"2024-05-21T15:33:52.242538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:14:37.327480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375276+0000","last_change":"2024-05-21T15:32:33.678315+0000","last_active":"2024-05-21T15:33:23.375276+0000","last_peered":"2024-05-21T15:33:23.375276+0000","last_clean":"2024-05-21T15:33:23.375276+0000","last_became_active":"2024-05-21T15:32:33.677787+0000","last_became_peered":"2024-05-21T15:32:33.677787+0000","last_unstale":"2024-05-21T15:33:23.375276+0000","last_undegraded":"2024-05-21T15:33:23.375276+0000","last_fullsized":"2024-05-21T15:33:23.375276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:17:24.826525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.380244+0000","last_change":"2024-05-21T15:33:23.380244+0000","last_active":"2024-05-21T15:33:23.380244+0000","last_peered":"2024-05-21T15:33:23.380244+0000","last_clean":"2024-05-21T15:33:23.380244+0000","last_became_active":"2024-05-21T15:33:23.379769+0000","last_became_peered":"2024-05-21T15:33:23.379769+0000","last_unstale":"2024-05-21T15:33:23.380244+0000","last_undegraded":"2024-05-21T15:33:23.380244+0000","last_fullsized":"2024-05-21T15:33:23.380244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:16:07.820636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.253790+0000","last_change":"2024-05-21T15:32:33.683597+0000","last_active":"2024-05-21T15:33:52.253790+0000","last_peered":"2024-05-21T15:33:52.253790+0000","last_clean":"2024-05-21T15:33:52.253790+0000","last_became_active":"2024-05-21T15:32:33.681948+0000","last_became_peered":"2024-05-21T15:32:33.681948+0000","last_unstale":"2024-05-21T15:33:52.253790+0000","last_undegraded":"2024-05-21T15:33:52.253790+0000","last_fullsized":"2024-05-21T15:33:52.253790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:26:50.086019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.257499+0000","last_change":"2024-05-21T15:32:33.683329+0000","last_active":"2024-05-21T15:33:52.257499+0000","last_peered":"2024-05-21T15:33:52.257499+0000","last_clean":"2024-05-21T15:33:52.257499+0000","last_became_active":"2024-05-21T15:32:33.682947+0000","last_became_peered":"2024-05-21T15:32:33.682947+0000","last_unstale":"2024-05-21T15:33:52.257499+0000","last_undegraded":"2024-05-21T15:33:52.257499+0000","last_fullsized":"2024-05-21T15:33:52.257499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:24:27.502257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371161+0000","last_change":"2024-05-21T15:32:33.680391+0000","last_active":"2024-05-21T15:33:23.371161+0000","last_peered":"2024-05-21T15:33:23.371161+0000","last_clean":"2024-05-21T15:33:23.371161+0000","last_became_active":"2024-05-21T15:32:33.679996+0000","last_became_peered":"2024-05-21T15:32:33.679996+0000","last_unstale":"2024-05-21T15:33:23.371161+0000","last_undegraded":"2024-05-21T15:33:23.371161+0000","last_fullsized":"2024-05-21T15:33:23.371161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:56:43.658565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.240913+0000","last_change":"2024-05-21T15:32:33.688783+0000","last_active":"2024-05-21T15:33:52.240913+0000","last_peered":"2024-05-21T15:33:52.240913+0000","last_clean":"2024-05-21T15:33:52.240913+0000","last_became_active":"2024-05-21T15:32:33.688006+0000","last_became_peered":"2024-05-21T15:32:33.688006+0000","last_unstale":"2024-05-21T15:33:52.240913+0000","last_undegraded":"2024-05-21T15:33:52.240913+0000","last_fullsized":"2024-05-21T15:33:52.240913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:17:44.489759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:51.096731+0000","last_change":"2024-05-21T15:32:33.182251+0000","last_active":"2024-05-21T15:33:51.096731+0000","last_peered":"2024-05-21T15:33:51.096731+0000","last_clean":"2024-05-21T15:33:51.096731+0000","last_became_active":"2024-05-21T15:32:33.181893+0000","last_became_peered":"2024-05-21T15:32:33.181893+0000","last_unstale":"2024-05-21T15:33:51.096731+0000","last_undegraded":"2024-05-21T15:33:51.096731+0000","last_fullsized":"2024-05-21T15:33:51.096731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:23:36.863186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.238788+0000","last_change":"2024-05-21T15:32:33.691962+0000","last_active":"2024-05-21T15:33:52.238788+0000","last_peered":"2024-05-21T15:33:52.238788+0000","last_clean":"2024-05-21T15:33:52.238788+0000","last_became_active":"2024-05-21T15:32:33.691619+0000","last_became_peered":"2024-05-21T15:32:33.691619+0000","last_unstale":"2024-05-21T15:33:52.238788+0000","last_undegraded":"2024-05-21T15:33:52.238788+0000","last_fullsized":"2024-05-21T15:33:52.238788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:50:14.523567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.381842+0000","last_change":"2024-05-21T15:32:33.183772+0000","last_active":"2024-05-21T15:33:23.381842+0000","last_peered":"2024-05-21T15:33:23.381842+0000","last_clean":"2024-05-21T15:33:23.381842+0000","last_became_active":"2024-05-21T15:32:33.183440+0000","last_became_peered":"2024-05-21T15:32:33.183440+0000","last_unstale":"2024-05-21T15:33:23.381842+0000","last_undegraded":"2024-05-21T15:33:23.381842+0000","last_fullsized":"2024-05-21T15:33:23.381842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:53:00.714359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.376127+0000","last_change":"2024-05-21T15:32:33.682544+0000","last_active":"2024-05-21T15:33:23.376127+0000","last_peered":"2024-05-21T15:33:23.376127+0000","last_clean":"2024-05-21T15:33:23.376127+0000","last_became_active":"2024-05-21T15:32:33.682161+0000","last_became_peered":"2024-05-21T15:32:33.682161+0000","last_unstale":"2024-05-21T15:33:23.376127+0000","last_undegraded":"2024-05-21T15:33:23.376127+0000","last_fullsized":"2024-05-21T15:33:23.376127+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:20:38.143508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375834+0000","last_change":"2024-05-21T15:32:33.182400+0000","last_active":"2024-05-21T15:33:23.375834+0000","last_peered":"2024-05-21T15:33:23.375834+0000","last_clean":"2024-05-21T15:33:23.375834+0000","last_became_active":"2024-05-21T15:32:33.181745+0000","last_became_peered":"2024-05-21T15:32:33.181745+0000","last_unstale":"2024-05-21T15:33:23.375834+0000","last_undegraded":"2024-05-21T15:33:23.375834+0000","last_fullsized":"2024-05-21T15:33:23.375834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:39:41.669529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.257179+0000","last_change":"2024-05-21T15:32:33.682700+0000","last_active":"2024-05-21T15:33:52.257179+0000","last_peered":"2024-05-21T15:33:52.257179+0000","last_clean":"2024-05-21T15:33:52.257179+0000","last_became_active":"2024-05-21T15:32:33.680876+0000","last_became_peered":"2024-05-21T15:32:33.680876+0000","last_unstale":"2024-05-21T15:33:52.257179+0000","last_undegraded":"2024-05-21T15:33:52.257179+0000","last_fullsized":"2024-05-21T15:33:52.257179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:30.694520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375325+0000","last_change":"2024-05-21T15:32:33.179114+0000","last_active":"2024-05-21T15:33:23.375325+0000","last_peered":"2024-05-21T15:33:23.375325+0000","last_clean":"2024-05-21T15:33:23.375325+0000","last_became_active":"2024-05-21T15:32:33.178485+0000","last_became_peered":"2024-05-21T15:32:33.178485+0000","last_unstale":"2024-05-21T15:33:23.375325+0000","last_undegraded":"2024-05-21T15:33:23.375325+0000","last_fullsized":"2024-05-21T15:33:23.375325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:19:56.501287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.609227+0000","last_change":"2024-05-21T15:32:33.691874+0000","last_active":"2024-05-21T15:33:23.609227+0000","last_peered":"2024-05-21T15:33:23.609227+0000","last_clean":"2024-05-21T15:33:23.609227+0000","last_became_active":"2024-05-21T15:32:33.691546+0000","last_became_peered":"2024-05-21T15:32:33.691546+0000","last_unstale":"2024-05-21T15:33:23.609227+0000","last_undegraded":"2024-05-21T15:33:23.609227+0000","last_fullsized":"2024-05-21T15:33:23.609227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:40:09.281792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:50.096530+0000","last_change":"2024-05-21T15:32:33.183457+0000","last_active":"2024-05-21T15:33:50.096530+0000","last_peered":"2024-05-21T15:33:50.096530+0000","last_clean":"2024-05-21T15:33:50.096530+0000","last_became_active":"2024-05-21T15:32:33.183127+0000","last_became_peered":"2024-05-21T15:32:33.183127+0000","last_unstale":"2024-05-21T15:33:50.096530+0000","last_undegraded":"2024-05-21T15:33:50.096530+0000","last_fullsized":"2024-05-21T15:33:50.096530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:41:05.179629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.229518+0000","last_change":"2024-05-21T15:32:33.677260+0000","last_active":"2024-05-21T15:33:52.229518+0000","last_peered":"2024-05-21T15:33:52.229518+0000","last_clean":"2024-05-21T15:33:52.229518+0000","last_became_active":"2024-05-21T15:32:33.676905+0000","last_became_peered":"2024-05-21T15:32:33.676905+0000","last_unstale":"2024-05-21T15:33:52.229518+0000","last_undegraded":"2024-05-21T15:33:52.229518+0000","last_fullsized":"2024-05-21T15:33:52.229518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:51:41.543066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371864+0000","last_change":"2024-05-21T15:32:33.181819+0000","last_active":"2024-05-21T15:33:23.371864+0000","last_peered":"2024-05-21T15:33:23.371864+0000","last_clean":"2024-05-21T15:33:23.371864+0000","last_became_active":"2024-05-21T15:32:33.181471+0000","last_became_peered":"2024-05-21T15:32:33.181471+0000","last_unstale":"2024-05-21T15:33:23.371864+0000","last_undegraded":"2024-05-21T15:33:23.371864+0000","last_fullsized":"2024-05-21T15:33:23.371864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:21:54.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.609455+0000","last_change":"2024-05-21T15:32:33.691072+0000","last_active":"2024-05-21T15:33:23.609455+0000","last_peered":"2024-05-21T15:33:23.609455+0000","last_clean":"2024-05-21T15:33:23.609455+0000","last_became_active":"2024-05-21T15:32:33.690663+0000","last_became_peered":"2024-05-21T15:32:33.690663+0000","last_unstale":"2024-05-21T15:33:23.609455+0000","last_undegraded":"2024-05-21T15:33:23.609455+0000","last_fullsized":"2024-05-21T15:33:23.609455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:52:44.549537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375504+0000","last_change":"2024-05-21T15:32:33.179588+0000","last_active":"2024-05-21T15:33:23.375504+0000","last_peered":"2024-05-21T15:33:23.375504+0000","last_clean":"2024-05-21T15:33:23.375504+0000","last_became_active":"2024-05-21T15:32:33.179115+0000","last_became_peered":"2024-05-21T15:32:33.179115+0000","last_unstale":"2024-05-21T15:33:23.375504+0000","last_undegraded":"2024-05-21T15:33:23.375504+0000","last_fullsized":"2024-05-21T15:33:23.375504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:18:43.904584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.236221+0000","last_change":"2024-05-21T15:32:33.689456+0000","last_active":"2024-05-21T15:33:52.236221+0000","last_peered":"2024-05-21T15:33:52.236221+0000","last_clean":"2024-05-21T15:33:52.236221+0000","last_became_active":"2024-05-21T15:32:33.689142+0000","last_became_peered":"2024-05-21T15:32:33.689142+0000","last_unstale":"2024-05-21T15:33:52.236221+0000","last_undegraded":"2024-05-21T15:33:52.236221+0000","last_fullsized":"2024-05-21T15:33:52.236221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:00:13.768272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.608862+0000","last_change":"2024-05-21T15:32:33.184865+0000","last_active":"2024-05-21T15:33:23.608862+0000","last_peered":"2024-05-21T15:33:23.608862+0000","last_clean":"2024-05-21T15:33:23.608862+0000","last_became_active":"2024-05-21T15:32:33.184540+0000","last_became_peered":"2024-05-21T15:32:33.184540+0000","last_unstale":"2024-05-21T15:33:23.608862+0000","last_undegraded":"2024-05-21T15:33:23.608862+0000","last_fullsized":"2024-05-21T15:33:23.608862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:09:20.072906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.370919+0000","last_change":"2024-05-21T15:32:33.685478+0000","last_active":"2024-05-21T15:33:23.370919+0000","last_peered":"2024-05-21T15:33:23.370919+0000","last_clean":"2024-05-21T15:33:23.370919+0000","last_became_active":"2024-05-21T15:32:33.685137+0000","last_became_peered":"2024-05-21T15:32:33.685137+0000","last_unstale":"2024-05-21T15:33:23.370919+0000","last_undegraded":"2024-05-21T15:33:23.370919+0000","last_fullsized":"2024-05-21T15:33:23.370919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:39:40.815876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":168,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.116830+0000","last_change":"2024-05-21T15:32:33.179215+0000","last_active":"2024-05-21T15:33:52.116830+0000","last_peered":"2024-05-21T15:33:52.116830+0000","last_clean":"2024-05-21T15:33:52.116830+0000","last_became_active":"2024-05-21T15:32:33.178846+0000","last_became_peered":"2024-05-21T15:32:33.178846+0000","last_unstale":"2024-05-21T15:33:52.116830+0000","last_undegraded":"2024-05-21T15:33:52.116830+0000","last_fullsized":"2024-05-21T15:33:52.116830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:29:47.289968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.256916+0000","last_change":"2024-05-21T15:32:33.684900+0000","last_active":"2024-05-21T15:33:52.256916+0000","last_peered":"2024-05-21T15:33:52.256916+0000","last_clean":"2024-05-21T15:33:52.256916+0000","last_became_active":"2024-05-21T15:32:33.681374+0000","last_became_peered":"2024-05-21T15:32:33.681374+0000","last_unstale":"2024-05-21T15:33:52.256916+0000","last_undegraded":"2024-05-21T15:33:52.256916+0000","last_fullsized":"2024-05-21T15:33:52.256916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:51:38.079726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374211+0000","last_change":"2024-05-21T15:32:33.189945+0000","last_active":"2024-05-21T15:33:23.374211+0000","last_peered":"2024-05-21T15:33:23.374211+0000","last_clean":"2024-05-21T15:33:23.374211+0000","last_became_active":"2024-05-21T15:32:33.189557+0000","last_became_peered":"2024-05-21T15:32:33.189557+0000","last_unstale":"2024-05-21T15:33:23.374211+0000","last_undegraded":"2024-05-21T15:33:23.374211+0000","last_fullsized":"2024-05-21T15:33:23.374211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:08.427043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.376014+0000","last_change":"2024-05-21T15:32:33.692896+0000","last_active":"2024-05-21T15:33:23.376014+0000","last_peered":"2024-05-21T15:33:23.376014+0000","last_clean":"2024-05-21T15:33:23.376014+0000","last_became_active":"2024-05-21T15:32:33.692527+0000","last_became_peered":"2024-05-21T15:32:33.692527+0000","last_unstale":"2024-05-21T15:33:23.376014+0000","last_undegraded":"2024-05-21T15:33:23.376014+0000","last_fullsized":"2024-05-21T15:33:23.376014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:45:45.418063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.609353+0000","last_change":"2024-05-21T15:32:33.172989+0000","last_active":"2024-05-21T15:33:23.609353+0000","last_peered":"2024-05-21T15:33:23.609353+0000","last_clean":"2024-05-21T15:33:23.609353+0000","last_became_active":"2024-05-21T15:32:33.172215+0000","last_became_peered":"2024-05-21T15:32:33.172215+0000","last_unstale":"2024-05-21T15:33:23.609353+0000","last_undegraded":"2024-05-21T15:33:23.609353+0000","last_fullsized":"2024-05-21T15:33:23.609353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:09:39.287636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.256656+0000","last_change":"2024-05-21T15:32:33.678441+0000","last_active":"2024-05-21T15:33:52.256656+0000","last_peered":"2024-05-21T15:33:52.256656+0000","last_clean":"2024-05-21T15:33:52.256656+0000","last_became_active":"2024-05-21T15:32:33.677833+0000","last_became_peered":"2024-05-21T15:32:33.677833+0000","last_unstale":"2024-05-21T15:33:52.256656+0000","last_undegraded":"2024-05-21T15:33:52.256656+0000","last_fullsized":"2024-05-21T15:33:52.256656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:36.124064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'32","reported_seq":201,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.195455+0000","last_change":"2024-05-21T15:32:33.178153+0000","last_active":"2024-05-21T15:33:52.195455+0000","last_peered":"2024-05-21T15:33:52.195455+0000","last_clean":"2024-05-21T15:33:52.195455+0000","last_became_active":"2024-05-21T15:32:33.177792+0000","last_became_peered":"2024-05-21T15:32:33.177792+0000","last_unstale":"2024-05-21T15:33:52.195455+0000","last_undegraded":"2024-05-21T15:33:52.195455+0000","last_fullsized":"2024-05-21T15:33:52.195455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:43:41.865204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59714,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":150,"num_read_kb":133,"num_write":34,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.244790+0000","last_change":"2024-05-21T15:32:33.681328+0000","last_active":"2024-05-21T15:33:52.244790+0000","last_peered":"2024-05-21T15:33:52.244790+0000","last_clean":"2024-05-21T15:33:52.244790+0000","last_became_active":"2024-05-21T15:32:33.680975+0000","last_became_peered":"2024-05-21T15:32:33.680975+0000","last_unstale":"2024-05-21T15:33:52.244790+0000","last_undegraded":"2024-05-21T15:33:52.244790+0000","last_fullsized":"2024-05-21T15:33:52.244790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:50:27.199348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375005+0000","last_change":"2024-05-21T15:32:33.182492+0000","last_active":"2024-05-21T15:33:23.375005+0000","last_peered":"2024-05-21T15:33:23.375005+0000","last_clean":"2024-05-21T15:33:23.375005+0000","last_became_active":"2024-05-21T15:32:33.182152+0000","last_became_peered":"2024-05-21T15:32:33.182152+0000","last_unstale":"2024-05-21T15:33:23.375005+0000","last_undegraded":"2024-05-21T15:33:23.375005+0000","last_fullsized":"2024-05-21T15:33:23.375005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:04:07.432742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.243222+0000","last_change":"2024-05-21T15:32:33.686623+0000","last_active":"2024-05-21T15:33:52.243222+0000","last_peered":"2024-05-21T15:33:52.243222+0000","last_clean":"2024-05-21T15:33:52.243222+0000","last_became_active":"2024-05-21T15:32:33.685816+0000","last_became_peered":"2024-05-21T15:32:33.685816+0000","last_unstale":"2024-05-21T15:33:52.243222+0000","last_undegraded":"2024-05-21T15:33:52.243222+0000","last_fullsized":"2024-05-21T15:33:52.243222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:15:20.354113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.608969+0000","last_change":"2024-05-21T15:32:33.182218+0000","last_active":"2024-05-21T15:33:23.608969+0000","last_peered":"2024-05-21T15:33:23.608969+0000","last_clean":"2024-05-21T15:33:23.608969+0000","last_became_active":"2024-05-21T15:32:33.181883+0000","last_became_peered":"2024-05-21T15:32:33.181883+0000","last_unstale":"2024-05-21T15:33:23.608969+0000","last_undegraded":"2024-05-21T15:33:23.608969+0000","last_fullsized":"2024-05-21T15:33:23.608969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:24:07.963898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.252508+0000","last_change":"2024-05-21T15:32:33.689936+0000","last_active":"2024-05-21T15:33:52.252508+0000","last_peered":"2024-05-21T15:33:52.252508+0000","last_clean":"2024-05-21T15:33:52.252508+0000","last_became_active":"2024-05-21T15:32:33.689452+0000","last_became_peered":"2024-05-21T15:32:33.689452+0000","last_unstale":"2024-05-21T15:33:52.252508+0000","last_undegraded":"2024-05-21T15:33:52.252508+0000","last_fullsized":"2024-05-21T15:33:52.252508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:00.420730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:50.096727+0000","last_change":"2024-05-21T15:32:33.181354+0000","last_active":"2024-05-21T15:33:50.096727+0000","last_peered":"2024-05-21T15:33:50.096727+0000","last_clean":"2024-05-21T15:33:50.096727+0000","last_became_active":"2024-05-21T15:32:33.180983+0000","last_became_peered":"2024-05-21T15:32:33.180983+0000","last_unstale":"2024-05-21T15:33:50.096727+0000","last_undegraded":"2024-05-21T15:33:50.096727+0000","last_fullsized":"2024-05-21T15:33:50.096727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:18:46.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.243188+0000","last_change":"2024-05-21T15:33:23.377305+0000","last_active":"2024-05-21T15:33:52.243188+0000","last_peered":"2024-05-21T15:33:52.243188+0000","last_clean":"2024-05-21T15:33:52.243188+0000","last_became_active":"2024-05-21T15:33:23.376921+0000","last_became_peered":"2024-05-21T15:33:23.376921+0000","last_unstale":"2024-05-21T15:33:52.243188+0000","last_undegraded":"2024-05-21T15:33:52.243188+0000","last_fullsized":"2024-05-21T15:33:52.243188+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:19.874725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":61,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.496401+0000","last_change":"2024-05-21T15:33:23.496401+0000","last_active":"2024-05-21T15:33:23.496401+0000","last_peered":"2024-05-21T15:33:23.496401+0000","last_clean":"2024-05-21T15:33:23.496401+0000","last_became_active":"2024-05-21T15:33:23.483584+0000","last_became_peered":"2024-05-21T15:33:23.483584+0000","last_unstale":"2024-05-21T15:33:23.496401+0000","last_undegraded":"2024-05-21T15:33:23.496401+0000","last_fullsized":"2024-05-21T15:33:23.496401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:35:32.411825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.608729+0000","last_change":"2024-05-21T15:32:33.691112+0000","last_active":"2024-05-21T15:33:23.608729+0000","last_peered":"2024-05-21T15:33:23.608729+0000","last_clean":"2024-05-21T15:33:23.608729+0000","last_became_active":"2024-05-21T15:32:33.690726+0000","last_became_peered":"2024-05-21T15:32:33.690726+0000","last_unstale":"2024-05-21T15:33:23.608729+0000","last_undegraded":"2024-05-21T15:33:23.608729+0000","last_fullsized":"2024-05-21T15:33:23.608729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:59:29.002636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375970+0000","last_change":"2024-05-21T15:32:33.179565+0000","last_active":"2024-05-21T15:33:23.375970+0000","last_peered":"2024-05-21T15:33:23.375970+0000","last_clean":"2024-05-21T15:33:23.375970+0000","last_became_active":"2024-05-21T15:32:33.179168+0000","last_became_peered":"2024-05-21T15:32:33.179168+0000","last_unstale":"2024-05-21T15:33:23.375970+0000","last_undegraded":"2024-05-21T15:33:23.375970+0000","last_fullsized":"2024-05-21T15:33:23.375970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:23:39.210931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.247065+0000","last_change":"2024-05-21T15:32:33.691348+0000","last_active":"2024-05-21T15:33:52.247065+0000","last_peered":"2024-05-21T15:33:52.247065+0000","last_clean":"2024-05-21T15:33:52.247065+0000","last_became_active":"2024-05-21T15:32:33.691005+0000","last_became_peered":"2024-05-21T15:32:33.691005+0000","last_unstale":"2024-05-21T15:33:52.247065+0000","last_undegraded":"2024-05-21T15:33:52.247065+0000","last_fullsized":"2024-05-21T15:33:52.247065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:33:34.533026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371037+0000","last_change":"2024-05-21T15:32:33.184850+0000","last_active":"2024-05-21T15:33:23.371037+0000","last_peered":"2024-05-21T15:33:23.371037+0000","last_clean":"2024-05-21T15:33:23.371037+0000","last_became_active":"2024-05-21T15:32:33.184480+0000","last_became_peered":"2024-05-21T15:32:33.184480+0000","last_unstale":"2024-05-21T15:33:23.371037+0000","last_undegraded":"2024-05-21T15:33:23.371037+0000","last_fullsized":"2024-05-21T15:33:23.371037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:31.327820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.241169+0000","last_change":"2024-05-21T15:32:33.689520+0000","last_active":"2024-05-21T15:33:52.241169+0000","last_peered":"2024-05-21T15:33:52.241169+0000","last_clean":"2024-05-21T15:33:52.241169+0000","last_became_active":"2024-05-21T15:32:33.688219+0000","last_became_peered":"2024-05-21T15:32:33.688219+0000","last_unstale":"2024-05-21T15:33:52.241169+0000","last_undegraded":"2024-05-21T15:33:52.241169+0000","last_fullsized":"2024-05-21T15:33:52.241169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:36.546390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.098805+0000","last_change":"2024-05-21T15:32:33.183183+0000","last_active":"2024-05-21T15:33:52.098805+0000","last_peered":"2024-05-21T15:33:52.098805+0000","last_clean":"2024-05-21T15:33:52.098805+0000","last_became_active":"2024-05-21T15:32:33.182753+0000","last_became_peered":"2024-05-21T15:32:33.182753+0000","last_unstale":"2024-05-21T15:33:52.098805+0000","last_undegraded":"2024-05-21T15:33:52.098805+0000","last_fullsized":"2024-05-21T15:33:52.098805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:44:22.509265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.230738+0000","last_change":"2024-05-21T15:32:33.684019+0000","last_active":"2024-05-21T15:33:52.230738+0000","last_peered":"2024-05-21T15:33:52.230738+0000","last_clean":"2024-05-21T15:33:52.230738+0000","last_became_active":"2024-05-21T15:32:33.683276+0000","last_became_peered":"2024-05-21T15:32:33.683276+0000","last_unstale":"2024-05-21T15:33:52.230738+0000","last_undegraded":"2024-05-21T15:33:52.230738+0000","last_fullsized":"2024-05-21T15:33:52.230738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:07:14.947240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.609033+0000","last_change":"2024-05-21T15:32:33.690956+0000","last_active":"2024-05-21T15:33:23.609033+0000","last_peered":"2024-05-21T15:33:23.609033+0000","last_clean":"2024-05-21T15:33:23.609033+0000","last_became_active":"2024-05-21T15:32:33.690630+0000","last_became_peered":"2024-05-21T15:32:33.690630+0000","last_unstale":"2024-05-21T15:33:23.609033+0000","last_undegraded":"2024-05-21T15:33:23.609033+0000","last_fullsized":"2024-05-21T15:33:23.609033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:24:07.050928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:51.097379+0000","last_change":"2024-05-21T15:32:33.175127+0000","last_active":"2024-05-21T15:33:51.097379+0000","last_peered":"2024-05-21T15:33:51.097379+0000","last_clean":"2024-05-21T15:33:51.097379+0000","last_became_active":"2024-05-21T15:32:33.174724+0000","last_became_peered":"2024-05-21T15:32:33.174724+0000","last_unstale":"2024-05-21T15:33:51.097379+0000","last_undegraded":"2024-05-21T15:33:51.097379+0000","last_fullsized":"2024-05-21T15:33:51.097379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:39:17.499743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.239636+0000","last_change":"2024-05-21T15:32:33.693428+0000","last_active":"2024-05-21T15:33:52.239636+0000","last_peered":"2024-05-21T15:33:52.239636+0000","last_clean":"2024-05-21T15:33:52.239636+0000","last_became_active":"2024-05-21T15:32:33.693077+0000","last_became_peered":"2024-05-21T15:32:33.693077+0000","last_unstale":"2024-05-21T15:33:52.239636+0000","last_undegraded":"2024-05-21T15:33:52.239636+0000","last_fullsized":"2024-05-21T15:33:52.239636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:08:11.908874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.482302+0000","last_change":"2024-05-21T15:32:33.176267+0000","last_active":"2024-05-21T15:33:23.482302+0000","last_peered":"2024-05-21T15:33:23.482302+0000","last_clean":"2024-05-21T15:33:23.482302+0000","last_became_active":"2024-05-21T15:32:33.175856+0000","last_became_peered":"2024-05-21T15:32:33.175856+0000","last_unstale":"2024-05-21T15:33:23.482302+0000","last_undegraded":"2024-05-21T15:33:23.482302+0000","last_fullsized":"2024-05-21T15:33:23.482302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:58.113791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":343,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374396+0000","last_change":"2024-05-21T15:31:20.928694+0000","last_active":"2024-05-21T15:33:23.374396+0000","last_peered":"2024-05-21T15:33:23.374396+0000","last_clean":"2024-05-21T15:33:23.374396+0000","last_became_active":"2024-05-21T15:31:20.928115+0000","last_became_peered":"2024-05-21T15:31:20.928115+0000","last_unstale":"2024-05-21T15:33:23.374396+0000","last_undegraded":"2024-05-21T15:33:23.374396+0000","last_fullsized":"2024-05-21T15:33:23.374396+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:27:50.788960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:27:50.788960+0000","last_clean_scrub_stamp":"2024-05-21T15:27:50.788960+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:55:10.393374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.230102+0000","last_change":"2024-05-21T15:32:33.689305+0000","last_active":"2024-05-21T15:33:52.230102+0000","last_peered":"2024-05-21T15:33:52.230102+0000","last_clean":"2024-05-21T15:33:52.230102+0000","last_became_active":"2024-05-21T15:32:33.688908+0000","last_became_peered":"2024-05-21T15:32:33.688908+0000","last_unstale":"2024-05-21T15:33:52.230102+0000","last_undegraded":"2024-05-21T15:33:52.230102+0000","last_fullsized":"2024-05-21T15:33:52.230102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:49:18.953116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374568+0000","last_change":"2024-05-21T15:32:33.187781+0000","last_active":"2024-05-21T15:33:23.374568+0000","last_peered":"2024-05-21T15:33:23.374568+0000","last_clean":"2024-05-21T15:33:23.374568+0000","last_became_active":"2024-05-21T15:32:33.187504+0000","last_became_peered":"2024-05-21T15:32:33.187504+0000","last_unstale":"2024-05-21T15:33:23.374568+0000","last_undegraded":"2024-05-21T15:33:23.374568+0000","last_fullsized":"2024-05-21T15:33:23.374568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:30.400209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.375721+0000","last_change":"2024-05-21T15:32:33.188706+0000","last_active":"2024-05-21T15:33:23.375721+0000","last_peered":"2024-05-21T15:33:23.375721+0000","last_clean":"2024-05-21T15:33:23.375721+0000","last_became_active":"2024-05-21T15:32:33.187462+0000","last_became_peered":"2024-05-21T15:32:33.187462+0000","last_unstale":"2024-05-21T15:33:23.375721+0000","last_undegraded":"2024-05-21T15:33:23.375721+0000","last_fullsized":"2024-05-21T15:33:23.375721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:47:45.387721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.229365+0000","last_change":"2024-05-21T15:32:33.672531+0000","last_active":"2024-05-21T15:33:52.229365+0000","last_peered":"2024-05-21T15:33:52.229365+0000","last_clean":"2024-05-21T15:33:52.229365+0000","last_became_active":"2024-05-21T15:32:33.671972+0000","last_became_peered":"2024-05-21T15:32:33.671972+0000","last_unstale":"2024-05-21T15:33:52.229365+0000","last_undegraded":"2024-05-21T15:33:52.229365+0000","last_fullsized":"2024-05-21T15:33:52.229365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:10:51.062530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.371212+0000","last_change":"2024-05-21T15:32:33.181274+0000","last_active":"2024-05-21T15:33:23.371212+0000","last_peered":"2024-05-21T15:33:23.371212+0000","last_clean":"2024-05-21T15:33:23.371212+0000","last_became_active":"2024-05-21T15:32:33.180885+0000","last_became_peered":"2024-05-21T15:32:33.180885+0000","last_unstale":"2024-05-21T15:33:23.371212+0000","last_undegraded":"2024-05-21T15:33:23.371212+0000","last_fullsized":"2024-05-21T15:33:23.371212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:10:13.547380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.243561+0000","last_change":"2024-05-21T15:32:33.683749+0000","last_active":"2024-05-21T15:33:52.243561+0000","last_peered":"2024-05-21T15:33:52.243561+0000","last_clean":"2024-05-21T15:33:52.243561+0000","last_became_active":"2024-05-21T15:32:33.683393+0000","last_became_peered":"2024-05-21T15:32:33.683393+0000","last_unstale":"2024-05-21T15:33:52.243561+0000","last_undegraded":"2024-05-21T15:33:52.243561+0000","last_fullsized":"2024-05-21T15:33:52.243561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:07:53.009651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.482059+0000","last_change":"2024-05-21T15:32:33.184740+0000","last_active":"2024-05-21T15:33:23.482059+0000","last_peered":"2024-05-21T15:33:23.482059+0000","last_clean":"2024-05-21T15:33:23.482059+0000","last_became_active":"2024-05-21T15:32:33.184378+0000","last_became_peered":"2024-05-21T15:32:33.184378+0000","last_unstale":"2024-05-21T15:33:23.482059+0000","last_undegraded":"2024-05-21T15:33:23.482059+0000","last_fullsized":"2024-05-21T15:33:23.482059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:21:27.610513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.381090+0000","last_change":"2024-05-21T15:33:23.381090+0000","last_active":"2024-05-21T15:33:23.381090+0000","last_peered":"2024-05-21T15:33:23.381090+0000","last_clean":"2024-05-21T15:33:23.381090+0000","last_became_active":"2024-05-21T15:33:23.380680+0000","last_became_peered":"2024-05-21T15:33:23.380680+0000","last_unstale":"2024-05-21T15:33:23.381090+0000","last_undegraded":"2024-05-21T15:33:23.381090+0000","last_fullsized":"2024-05-21T15:33:23.381090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:26:03.371476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.608845+0000","last_change":"2024-05-21T15:32:33.183344+0000","last_active":"2024-05-21T15:33:23.608845+0000","last_peered":"2024-05-21T15:33:23.608845+0000","last_clean":"2024-05-21T15:33:23.608845+0000","last_became_active":"2024-05-21T15:32:33.182977+0000","last_became_peered":"2024-05-21T15:32:33.182977+0000","last_unstale":"2024-05-21T15:33:23.608845+0000","last_undegraded":"2024-05-21T15:33:23.608845+0000","last_fullsized":"2024-05-21T15:33:23.608845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:27:24.354129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.200096+0000","last_change":"2024-05-21T15:32:33.677549+0000","last_active":"2024-05-21T15:33:52.200096+0000","last_peered":"2024-05-21T15:33:52.200096+0000","last_clean":"2024-05-21T15:33:52.200096+0000","last_became_active":"2024-05-21T15:32:33.677213+0000","last_became_peered":"2024-05-21T15:32:33.677213+0000","last_unstale":"2024-05-21T15:33:52.200096+0000","last_undegraded":"2024-05-21T15:33:52.200096+0000","last_fullsized":"2024-05-21T15:33:52.200096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:57:06.560434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.380435+0000","last_change":"2024-05-21T15:32:33.190023+0000","last_active":"2024-05-21T15:33:23.380435+0000","last_peered":"2024-05-21T15:33:23.380435+0000","last_clean":"2024-05-21T15:33:23.380435+0000","last_became_active":"2024-05-21T15:32:33.189616+0000","last_became_peered":"2024-05-21T15:32:33.189616+0000","last_unstale":"2024-05-21T15:33:23.380435+0000","last_undegraded":"2024-05-21T15:33:23.380435+0000","last_fullsized":"2024-05-21T15:33:23.380435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:03.841414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.245365+0000","last_change":"2024-05-21T15:32:33.685911+0000","last_active":"2024-05-21T15:33:52.245365+0000","last_peered":"2024-05-21T15:33:52.245365+0000","last_clean":"2024-05-21T15:33:52.245365+0000","last_became_active":"2024-05-21T15:32:33.684785+0000","last_became_peered":"2024-05-21T15:32:33.684785+0000","last_unstale":"2024-05-21T15:33:52.245365+0000","last_undegraded":"2024-05-21T15:33:52.245365+0000","last_fullsized":"2024-05-21T15:33:52.245365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:39:18.370289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.374839+0000","last_change":"2024-05-21T15:32:33.178354+0000","last_active":"2024-05-21T15:33:23.374839+0000","last_peered":"2024-05-21T15:33:23.374839+0000","last_clean":"2024-05-21T15:33:23.374839+0000","last_became_active":"2024-05-21T15:32:33.177989+0000","last_became_peered":"2024-05-21T15:32:33.177989+0000","last_unstale":"2024-05-21T15:33:23.374839+0000","last_undegraded":"2024-05-21T15:33:23.374839+0000","last_fullsized":"2024-05-21T15:33:23.374839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:58.650831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.232607+0000","last_change":"2024-05-21T15:32:33.686467+0000","last_active":"2024-05-21T15:33:52.232607+0000","last_peered":"2024-05-21T15:33:52.232607+0000","last_clean":"2024-05-21T15:33:52.232607+0000","last_became_active":"2024-05-21T15:32:33.686135+0000","last_became_peered":"2024-05-21T15:32:33.686135+0000","last_unstale":"2024-05-21T15:33:52.232607+0000","last_undegraded":"2024-05-21T15:33:52.232607+0000","last_fullsized":"2024-05-21T15:33:52.232607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:53:54.045591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.383702+0000","last_change":"2024-05-21T15:33:23.383157+0000","last_active":"2024-05-21T15:33:23.383702+0000","last_peered":"2024-05-21T15:33:23.383702+0000","last_clean":"2024-05-21T15:33:23.383702+0000","last_became_active":"2024-05-21T15:33:23.376834+0000","last_became_peered":"2024-05-21T15:33:23.376834+0000","last_unstale":"2024-05-21T15:33:23.383702+0000","last_undegraded":"2024-05-21T15:33:23.383702+0000","last_fullsized":"2024-05-21T15:33:23.383702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:01:19.349486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.249586+0000","last_change":"2024-05-21T15:32:33.692325+0000","last_active":"2024-05-21T15:33:52.249586+0000","last_peered":"2024-05-21T15:33:52.249586+0000","last_clean":"2024-05-21T15:33:52.249586+0000","last_became_active":"2024-05-21T15:32:33.691971+0000","last_became_peered":"2024-05-21T15:32:33.691971+0000","last_unstale":"2024-05-21T15:33:52.249586+0000","last_undegraded":"2024-05-21T15:33:52.249586+0000","last_fullsized":"2024-05-21T15:33:52.249586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:53:28.190063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":103,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:53.119785+0000","last_change":"2024-05-21T15:32:33.186925+0000","last_active":"2024-05-21T15:33:53.119785+0000","last_peered":"2024-05-21T15:33:53.119785+0000","last_clean":"2024-05-21T15:33:53.119785+0000","last_became_active":"2024-05-21T15:32:33.186605+0000","last_became_peered":"2024-05-21T15:32:33.186605+0000","last_unstale":"2024-05-21T15:33:53.119785+0000","last_undegraded":"2024-05-21T15:33:53.119785+0000","last_fullsized":"2024-05-21T15:33:53.119785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:32.752746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.248234+0000","last_change":"2024-05-21T15:32:33.686132+0000","last_active":"2024-05-21T15:33:52.248234+0000","last_peered":"2024-05-21T15:33:52.248234+0000","last_clean":"2024-05-21T15:33:52.248234+0000","last_became_active":"2024-05-21T15:32:33.685745+0000","last_became_peered":"2024-05-21T15:32:33.685745+0000","last_unstale":"2024-05-21T15:33:52.248234+0000","last_undegraded":"2024-05-21T15:33:52.248234+0000","last_fullsized":"2024-05-21T15:33:52.248234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:51:49.522297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.609134+0000","last_change":"2024-05-21T15:32:33.176552+0000","last_active":"2024-05-21T15:33:23.609134+0000","last_peered":"2024-05-21T15:33:23.609134+0000","last_clean":"2024-05-21T15:33:23.609134+0000","last_became_active":"2024-05-21T15:32:33.176237+0000","last_became_peered":"2024-05-21T15:32:33.176237+0000","last_unstale":"2024-05-21T15:33:23.609134+0000","last_undegraded":"2024-05-21T15:33:23.609134+0000","last_fullsized":"2024-05-21T15:33:23.609134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:09.098344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.243974+0000","last_change":"2024-05-21T15:32:33.692310+0000","last_active":"2024-05-21T15:33:52.243974+0000","last_peered":"2024-05-21T15:33:52.243974+0000","last_clean":"2024-05-21T15:33:52.243974+0000","last_became_active":"2024-05-21T15:32:33.691959+0000","last_became_peered":"2024-05-21T15:32:33.691959+0000","last_unstale":"2024-05-21T15:33:52.243974+0000","last_undegraded":"2024-05-21T15:33:52.243974+0000","last_fullsized":"2024-05-21T15:33:52.243974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:57:38.243926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.614280+0000","last_change":"2024-05-21T15:33:23.614280+0000","last_active":"2024-05-21T15:33:23.614280+0000","last_peered":"2024-05-21T15:33:23.614280+0000","last_clean":"2024-05-21T15:33:23.614280+0000","last_became_active":"2024-05-21T15:33:23.610364+0000","last_became_peered":"2024-05-21T15:33:23.610364+0000","last_unstale":"2024-05-21T15:33:23.614280+0000","last_undegraded":"2024-05-21T15:33:23.614280+0000","last_fullsized":"2024-05-21T15:33:23.614280+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:40:42.990949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.246347+0000","last_change":"2024-05-21T15:32:33.682801+0000","last_active":"2024-05-21T15:33:52.246347+0000","last_peered":"2024-05-21T15:33:52.246347+0000","last_clean":"2024-05-21T15:33:52.246347+0000","last_became_active":"2024-05-21T15:32:33.682368+0000","last_became_peered":"2024-05-21T15:32:33.682368+0000","last_unstale":"2024-05-21T15:33:52.246347+0000","last_undegraded":"2024-05-21T15:33:52.246347+0000","last_fullsized":"2024-05-21T15:33:52.246347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:56:50.520672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.247424+0000","last_change":"2024-05-21T15:32:33.684809+0000","last_active":"2024-05-21T15:33:52.247424+0000","last_peered":"2024-05-21T15:33:52.247424+0000","last_clean":"2024-05-21T15:33:52.247424+0000","last_became_active":"2024-05-21T15:32:33.684067+0000","last_became_peered":"2024-05-21T15:32:33.684067+0000","last_unstale":"2024-05-21T15:33:52.247424+0000","last_undegraded":"2024-05-21T15:33:52.247424+0000","last_fullsized":"2024-05-21T15:33:52.247424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:31:58.997470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.242092+0000","last_change":"2024-05-21T15:32:33.676091+0000","last_active":"2024-05-21T15:33:52.242092+0000","last_peered":"2024-05-21T15:33:52.242092+0000","last_clean":"2024-05-21T15:33:52.242092+0000","last_became_active":"2024-05-21T15:32:33.675538+0000","last_became_peered":"2024-05-21T15:32:33.675538+0000","last_unstale":"2024-05-21T15:33:52.242092+0000","last_undegraded":"2024-05-21T15:33:52.242092+0000","last_fullsized":"2024-05-21T15:33:52.242092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:50.311152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.239952+0000","last_change":"2024-05-21T15:32:33.688590+0000","last_active":"2024-05-21T15:33:52.239952+0000","last_peered":"2024-05-21T15:33:52.239952+0000","last_clean":"2024-05-21T15:33:52.239952+0000","last_became_active":"2024-05-21T15:32:33.687993+0000","last_became_peered":"2024-05-21T15:32:33.687993+0000","last_unstale":"2024-05-21T15:33:52.239952+0000","last_undegraded":"2024-05-21T15:33:52.239952+0000","last_fullsized":"2024-05-21T15:33:52.239952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:21.939483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.380890+0000","last_change":"2024-05-21T15:33:23.380890+0000","last_active":"2024-05-21T15:33:23.380890+0000","last_peered":"2024-05-21T15:33:23.380890+0000","last_clean":"2024-05-21T15:33:23.380890+0000","last_became_active":"2024-05-21T15:33:23.380503+0000","last_became_peered":"2024-05-21T15:33:23.380503+0000","last_unstale":"2024-05-21T15:33:23.380890+0000","last_undegraded":"2024-05-21T15:33:23.380890+0000","last_fullsized":"2024-05-21T15:33:23.380890+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:41.070336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.237380+0000","last_change":"2024-05-21T15:32:33.689365+0000","last_active":"2024-05-21T15:33:52.237380+0000","last_peered":"2024-05-21T15:33:52.237380+0000","last_clean":"2024-05-21T15:33:52.237380+0000","last_became_active":"2024-05-21T15:32:33.688739+0000","last_became_peered":"2024-05-21T15:32:33.688739+0000","last_unstale":"2024-05-21T15:33:52.237380+0000","last_undegraded":"2024-05-21T15:33:52.237380+0000","last_fullsized":"2024-05-21T15:33:52.237380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:19:52.614155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.247363+0000","last_change":"2024-05-21T15:32:33.681985+0000","last_active":"2024-05-21T15:33:52.247363+0000","last_peered":"2024-05-21T15:33:52.247363+0000","last_clean":"2024-05-21T15:33:52.247363+0000","last_became_active":"2024-05-21T15:32:33.681209+0000","last_became_peered":"2024-05-21T15:32:33.681209+0000","last_unstale":"2024-05-21T15:33:52.247363+0000","last_undegraded":"2024-05-21T15:33:52.247363+0000","last_fullsized":"2024-05-21T15:33:52.247363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:03:16.088136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.254124+0000","last_change":"2024-05-21T15:32:33.673236+0000","last_active":"2024-05-21T15:33:52.254124+0000","last_peered":"2024-05-21T15:33:52.254124+0000","last_clean":"2024-05-21T15:33:52.254124+0000","last_became_active":"2024-05-21T15:32:33.672548+0000","last_became_peered":"2024-05-21T15:32:33.672548+0000","last_unstale":"2024-05-21T15:33:52.254124+0000","last_undegraded":"2024-05-21T15:33:52.254124+0000","last_fullsized":"2024-05-21T15:33:52.254124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:52:10.010219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.483327+0000","last_change":"2024-05-21T15:33:23.482874+0000","last_active":"2024-05-21T15:33:23.483327+0000","last_peered":"2024-05-21T15:33:23.483327+0000","last_clean":"2024-05-21T15:33:23.483327+0000","last_became_active":"2024-05-21T15:33:23.482589+0000","last_became_peered":"2024-05-21T15:33:23.482589+0000","last_unstale":"2024-05-21T15:33:23.483327+0000","last_undegraded":"2024-05-21T15:33:23.483327+0000","last_fullsized":"2024-05-21T15:33:23.483327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:19:45.870562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:23.608907+0000","last_change":"2024-05-21T15:32:33.686992+0000","last_active":"2024-05-21T15:33:23.608907+0000","last_peered":"2024-05-21T15:33:23.608907+0000","last_clean":"2024-05-21T15:33:23.608907+0000","last_became_active":"2024-05-21T15:32:33.686673+0000","last_became_peered":"2024-05-21T15:32:33.686673+0000","last_unstale":"2024-05-21T15:33:23.608907+0000","last_undegraded":"2024-05-21T15:33:23.608907+0000","last_fullsized":"2024-05-21T15:33:23.608907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:18:20.668365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.239354+0000","last_change":"2024-05-21T15:32:33.689433+0000","last_active":"2024-05-21T15:33:52.239354+0000","last_peered":"2024-05-21T15:33:52.239354+0000","last_clean":"2024-05-21T15:33:52.239354+0000","last_became_active":"2024-05-21T15:32:33.689106+0000","last_became_peered":"2024-05-21T15:32:33.689106+0000","last_unstale":"2024-05-21T15:33:52.239354+0000","last_undegraded":"2024-05-21T15:33:52.239354+0000","last_fullsized":"2024-05-21T15:33:52.239354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:45:17.841159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.255049+0000","last_change":"2024-05-21T15:32:33.687763+0000","last_active":"2024-05-21T15:33:52.255049+0000","last_peered":"2024-05-21T15:33:52.255049+0000","last_clean":"2024-05-21T15:33:52.255049+0000","last_became_active":"2024-05-21T15:32:33.685103+0000","last_became_peered":"2024-05-21T15:32:33.685103+0000","last_unstale":"2024-05-21T15:33:52.255049+0000","last_undegraded":"2024-05-21T15:33:52.255049+0000","last_fullsized":"2024-05-21T15:33:52.255049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:56:39.454536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.243760+0000","last_change":"2024-05-21T15:32:33.679225+0000","last_active":"2024-05-21T15:33:52.243760+0000","last_peered":"2024-05-21T15:33:52.243760+0000","last_clean":"2024-05-21T15:33:52.243760+0000","last_became_active":"2024-05-21T15:32:33.678890+0000","last_became_peered":"2024-05-21T15:32:33.678890+0000","last_unstale":"2024-05-21T15:33:52.243760+0000","last_undegraded":"2024-05-21T15:33:52.243760+0000","last_fullsized":"2024-05-21T15:33:52.243760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:12:23.721449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:33:52.247135+0000","last_change":"2024-05-21T15:32:33.688659+0000","last_active":"2024-05-21T15:33:52.247135+0000","last_peered":"2024-05-21T15:33:52.247135+0000","last_clean":"2024-05-21T15:33:52.247135+0000","last_became_active":"2024-05-21T15:32:33.688293+0000","last_became_peered":"2024-05-21T15:32:33.688293+0000","last_unstale":"2024-05-21T15:33:52.247135+0000","last_undegraded":"2024-05-21T15:33:52.247135+0000","last_fullsized":"2024-05-21T15:33:52.247135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:08:49.543237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":368,"ondisk_log_size":368,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":287606,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66668,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":756,"num_read_kb":450,"num_write":240,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":199863,"data_compressed":5562,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":460509,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":1110720,"data_compressed":44221,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":459907,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037805,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":460285,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1052412,"data_compressed":42367,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459624,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":520251,"data_compressed":7653,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":461354,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":520891,"data_compressed":7653,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":459449,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888130,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":459597,"data_compressed":5799,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":1052090,"data_compressed":42367,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61466,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62106,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61450,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T15:33:57.200 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-21T15:33:57.200 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-21T15:33:57.200 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-21T15:33:57.200 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph health --format=json 2024-05-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:33:58.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:57 smithi053 ceph-mon[36496]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:33:58.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:57 smithi053 ceph-mon[36496]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:33:58.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:57 smithi045 ceph-mon[36409]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:33:58.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:57 smithi045 ceph-mon[36409]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:33:58.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:57 smithi038 ceph-mon[31995]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:33:58.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:57 smithi038 ceph-mon[31995]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:33:59.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.182+0000 7f6e648c8700 1 -- 172.21.15.38:0/3229589979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e60101f00 msgr2=0x7f6e60102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:59.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.182+0000 7f6e648c8700 1 --2- 172.21.15.38:0/3229589979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e60101f00 0x7f6e60102310 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009990 tx=0x7f6e50009ca0 comp rx=0 tx=0).stop 2024-05-21T15:33:59.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.182+0000 7f6e648c8700 1 -- 172.21.15.38:0/3229589979 shutdown_connections 2024-05-21T15:33:59.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.182+0000 7f6e648c8700 1 --2- 172.21.15.38:0/3229589979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e6010fab0 0x7f6e60111ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:59.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.182+0000 7f6e648c8700 1 --2- 172.21.15.38:0/3229589979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e60101f00 0x7f6e60102310 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:59.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.182+0000 7f6e648c8700 1 --2- 172.21.15.38:0/3229589979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e60102850 0x7f6e6010f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:59.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.182+0000 7f6e648c8700 1 -- 172.21.15.38:0/3229589979 >> 172.21.15.38:0/3229589979 conn(0x7f6e600fdc70 msgr2=0x7f6e60100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:59.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.182+0000 7f6e648c8700 1 -- 172.21.15.38:0/3229589979 shutdown_connections 2024-05-21T15:33:59.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.182+0000 7f6e648c8700 1 -- 172.21.15.38:0/3229589979 wait complete. 2024-05-21T15:33:59.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.183+0000 7f6e648c8700 1 Processor -- start 2024-05-21T15:33:59.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.183+0000 7f6e648c8700 1 -- start start 2024-05-21T15:33:59.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e648c8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e60101f00 0x7f6e6019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:59.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e648c8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e60102850 0x7f6e6019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:59.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e648c8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6010fab0 0x7f6e601a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:59.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e648c8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e60115670 con 0x7f6e6010fab0 2024-05-21T15:33:59.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e648c8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601154f0 con 0x7f6e60101f00 2024-05-21T15:33:59.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e648c8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601157f0 con 0x7f6e60102850 2024-05-21T15:33:59.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e5ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6010fab0 0x7f6e601a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:59.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e5dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e60102850 0x7f6e6019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:59.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e5e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e60101f00 0x7f6e6019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:59.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e5dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e60102850 0x7f6e6019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:42406/0 (socket says 172.21.15.38:42406) 2024-05-21T15:33:59.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e5e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e60101f00 0x7f6e6019bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:60206/0 (socket says 172.21.15.38:60206) 2024-05-21T15:33:59.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e5dd9b700 1 -- 172.21.15.38:0/276885697 learned_addr learned my addr 172.21.15.38:0/276885697 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:33:59.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e5ed9d700 1 -- 172.21.15.38:0/276885697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e60102850 msgr2=0x7f6e6019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:59.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e5ed9d700 1 --2- 172.21.15.38:0/276885697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e60102850 0x7f6e6019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:59.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e5ed9d700 1 -- 172.21.15.38:0/276885697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e60101f00 msgr2=0x7f6e6019bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:59.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e5ed9d700 1 --2- 172.21.15.38:0/276885697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e60101f00 0x7f6e6019bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:59.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e5ed9d700 1 -- 172.21.15.38:0/276885697 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e601a0e70 con 0x7f6e6010fab0 2024-05-21T15:33:59.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.184+0000 7f6e5ed9d700 1 --2- 172.21.15.38:0/276885697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6010fab0 0x7f6e601a07d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800ab40 tx=0x7f6e5800ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:59.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.185+0000 7f6e4f7fe700 1 -- 172.21.15.38:0/276885697 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58017020 con 0x7f6e6010fab0 2024-05-21T15:33:59.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.185+0000 7f6e648c8700 1 -- 172.21.15.38:0/276885697 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e601a1160 con 0x7f6e6010fab0 2024-05-21T15:33:59.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.185+0000 7f6e4f7fe700 1 -- 172.21.15.38:0/276885697 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e58004bb0 con 0x7f6e6010fab0 2024-05-21T15:33:59.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.185+0000 7f6e4f7fe700 1 -- 172.21.15.38:0/276885697 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58005260 con 0x7f6e6010fab0 2024-05-21T15:33:59.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.185+0000 7f6e648c8700 1 -- 172.21.15.38:0/276885697 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e601a8a80 con 0x7f6e6010fab0 2024-05-21T15:33:59.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.186+0000 7f6e648c8700 1 -- 172.21.15.38:0/276885697 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e60050170 con 0x7f6e6010fab0 2024-05-21T15:33:59.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.187+0000 7f6e4f7fe700 1 -- 172.21.15.38:0/276885697 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6e5800c580 con 0x7f6e6010fab0 2024-05-21T15:33:59.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.188+0000 7f6e4f7fe700 1 --2- 172.21.15.38:0/276885697 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e480a2c20 0x7f6e480a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:33:59.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.188+0000 7f6e5e59c700 1 --2- 172.21.15.38:0/276885697 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e480a2c20 0x7f6e480a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:33:59.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.188+0000 7f6e4f7fe700 1 -- 172.21.15.38:0/276885697 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6e580ce560 con 0x7f6e6010fab0 2024-05-21T15:33:59.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.188+0000 7f6e5e59c700 1 --2- 172.21.15.38:0/276885697 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e480a2c20 0x7f6e480a5130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000b5c0 tx=0x7f6e5001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:33:59.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.192+0000 7f6e4f7fe700 1 -- 172.21.15.38:0/276885697 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6e58021020 con 0x7f6e6010fab0 2024-05-21T15:33:59.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.445+0000 7f6e648c8700 1 -- 172.21.15.38:0/276885697 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6e6002dc60 con 0x7f6e6010fab0 2024-05-21T15:33:59.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.445+0000 7f6e4f7fe700 1 -- 172.21.15.38:0/276885697 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6e5809b540 con 0x7f6e6010fab0 2024-05-21T15:33:59.445 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:33:59.445 INFO:teuthology.orchestra.run.smithi038.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-21T15:33:59.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.447+0000 7f6e648c8700 1 -- 172.21.15.38:0/276885697 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e480a2c20 msgr2=0x7f6e480a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:59.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.447+0000 7f6e648c8700 1 --2- 172.21.15.38:0/276885697 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e480a2c20 0x7f6e480a5130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000b5c0 tx=0x7f6e5001a040 comp rx=0 tx=0).stop 2024-05-21T15:33:59.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.447+0000 7f6e648c8700 1 -- 172.21.15.38:0/276885697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6010fab0 msgr2=0x7f6e601a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:33:59.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.447+0000 7f6e648c8700 1 --2- 172.21.15.38:0/276885697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6010fab0 0x7f6e601a07d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800ab40 tx=0x7f6e5800ae50 comp rx=0 tx=0).stop 2024-05-21T15:33:59.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.447+0000 7f6e648c8700 1 -- 172.21.15.38:0/276885697 shutdown_connections 2024-05-21T15:33:59.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.447+0000 7f6e648c8700 1 --2- 172.21.15.38:0/276885697 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e480a2c20 0x7f6e480a5130 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:59.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.447+0000 7f6e648c8700 1 --2- 172.21.15.38:0/276885697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6010fab0 0x7f6e601a07d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:59.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.447+0000 7f6e648c8700 1 --2- 172.21.15.38:0/276885697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e60101f00 0x7f6e6019bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:59.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.447+0000 7f6e648c8700 1 --2- 172.21.15.38:0/276885697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e60102850 0x7f6e6019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:33:59.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.447+0000 7f6e648c8700 1 -- 172.21.15.38:0/276885697 >> 172.21.15.38:0/276885697 conn(0x7f6e600fdc70 msgr2=0x7f6e60111590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:33:59.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.447+0000 7f6e648c8700 1 -- 172.21.15.38:0/276885697 shutdown_connections 2024-05-21T15:33:59.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:33:59.448+0000 7f6e648c8700 1 -- 172.21.15.38:0/276885697 wait complete. 2024-05-21T15:34:00.123 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:59 smithi038 ceph-mon[31995]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-21T15:34:00.123 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:33:59 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/276885697' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-21T15:34:00.152 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-21T15:34:00.153 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:34:00.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:59 smithi053 ceph-mon[36496]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-21T15:34:00.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:33:59 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/276885697' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-21T15:34:00.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:59 smithi045 ceph-mon[36409]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-21T15:34:00.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:33:59 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/276885697' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-21T15:34:00.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.272+0000 7f3c35a7a640 1 -- 172.21.15.38:0/381427377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c30102bf0 msgr2=0x7f3c30103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:00.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.272+0000 7f3c35a7a640 1 --2- 172.21.15.38:0/381427377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c30102bf0 0x7f3c30103000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009900 tx=0x7f3c2402a410 comp rx=0 tx=0).stop 2024-05-21T15:34:00.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.272+0000 7f3c35a7a640 1 -- 172.21.15.38:0/381427377 shutdown_connections 2024-05-21T15:34:00.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.272+0000 7f3c35a7a640 1 --2- 172.21.15.38:0/381427377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c301078f0 0x7f3c30109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.272+0000 7f3c35a7a640 1 --2- 172.21.15.38:0/381427377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c30102bf0 0x7f3c30103000 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.272+0000 7f3c35a7a640 1 --2- 172.21.15.38:0/381427377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c301019b0 0x7f3c301026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.272+0000 7f3c35a7a640 1 -- 172.21.15.38:0/381427377 >> 172.21.15.38:0/381427377 conn(0x7f3c300fcba0 msgr2=0x7f3c300ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:00.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.272+0000 7f3c35a7a640 1 -- 172.21.15.38:0/381427377 shutdown_connections 2024-05-21T15:34:00.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.272+0000 7f3c35a7a640 1 -- 172.21.15.38:0/381427377 wait complete. 2024-05-21T15:34:00.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.273+0000 7f3c35a7a640 1 Processor -- start 2024-05-21T15:34:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.273+0000 7f3c35a7a640 1 -- start start 2024-05-21T15:34:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.273+0000 7f3c35a7a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c301019b0 0x7f3c30198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.273+0000 7f3c35a7a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c30102bf0 0x7f3c30198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.273+0000 7f3c35a7a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c301078f0 0x7f3c3019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.273+0000 7f3c35a7a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3010d670 con 0x7f3c301078f0 2024-05-21T15:34:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.273+0000 7f3c35a7a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3010d4f0 con 0x7f3c30102bf0 2024-05-21T15:34:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.273+0000 7f3c35a7a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3010d7f0 con 0x7f3c301019b0 2024-05-21T15:34:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c301078f0 0x7f3c3019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c301078f0 0x7f3c3019f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51242/0 (socket says 172.21.15.38:51242) 2024-05-21T15:34:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2ffff640 1 -- 172.21.15.38:0/3250467545 learned_addr learned my addr 172.21.15.38:0/3250467545 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2f7fe640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c301019b0 0x7f3c30198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2effd640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c30102bf0 0x7f3c30198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2ffff640 1 -- 172.21.15.38:0/3250467545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c301019b0 msgr2=0x7f3c30198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2ffff640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c301019b0 0x7f3c30198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2ffff640 1 -- 172.21.15.38:0/3250467545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c30102bf0 msgr2=0x7f3c30198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2ffff640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c30102bf0 0x7f3c30198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2ffff640 1 -- 172.21.15.38:0/3250467545 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3019fbb0 con 0x7f3c301078f0 2024-05-21T15:34:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2f7fe640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c301019b0 0x7f3c30198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2ffff640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c301078f0 0x7f3c3019f5f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3c28007cb0 tx=0x7f3c2800e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2cff9640 1 -- 172.21.15.38:0/3250467545 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800eb30 con 0x7f3c301078f0 2024-05-21T15:34:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c35a7a640 1 -- 172.21.15.38:0/3250467545 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c30073de0 con 0x7f3c301078f0 2024-05-21T15:34:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2cff9640 1 -- 172.21.15.38:0/3250467545 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2800ecd0 con 0x7f3c301078f0 2024-05-21T15:34:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c2cff9640 1 -- 172.21.15.38:0/3250467545 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28015650 con 0x7f3c301078f0 2024-05-21T15:34:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.274+0000 7f3c35a7a640 1 -- 172.21.15.38:0/3250467545 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c30074330 con 0x7f3c301078f0 2024-05-21T15:34:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.276+0000 7f3c35a7a640 1 -- 172.21.15.38:0/3250467545 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c300691c0 con 0x7f3c301078f0 2024-05-21T15:34:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.277+0000 7f3c2cff9640 1 -- 172.21.15.38:0/3250467545 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3c28004a40 con 0x7f3c301078f0 2024-05-21T15:34:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.278+0000 7f3c2cff9640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3c180a2c30 0x7f3c180a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.278+0000 7f3c2f7fe640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3c180a2c30 0x7f3c180a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.278+0000 7f3c2cff9640 1 -- 172.21.15.38:0/3250467545 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f3c280d1010 con 0x7f3c301078f0 2024-05-21T15:34:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.279+0000 7f3c2f7fe640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3c180a2c30 0x7f3c180a5140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3c200058c0 tx=0x7f3c20005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.282+0000 7f3c2cff9640 1 -- 172.21.15.38:0/3250467545 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3c2809df60 con 0x7f3c301078f0 2024-05-21T15:34:00.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.467+0000 7f3c35a7a640 1 -- 172.21.15.38:0/3250467545 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c30106170 con 0x7f3c301078f0 2024-05-21T15:34:00.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.469+0000 7f3c2cff9640 1 -- 172.21.15.38:0/3250467545 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22609 (secure 0 0 0) 0x7f3c30106170 con 0x7f3c301078f0 2024-05-21T15:34:00.469 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:00.469 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:33:23.367336+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:34:00.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.470+0000 7f3c35a7a640 1 -- 172.21.15.38:0/3250467545 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3c180a2c30 msgr2=0x7f3c180a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:00.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.470+0000 7f3c35a7a640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3c180a2c30 0x7f3c180a5140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3c200058c0 tx=0x7f3c20005850 comp rx=0 tx=0).stop 2024-05-21T15:34:00.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.470+0000 7f3c35a7a640 1 -- 172.21.15.38:0/3250467545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c301078f0 msgr2=0x7f3c3019f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:00.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.470+0000 7f3c35a7a640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c301078f0 0x7f3c3019f5f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3c28007cb0 tx=0x7f3c2800e420 comp rx=0 tx=0).stop 2024-05-21T15:34:00.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.471+0000 7f3c35a7a640 1 -- 172.21.15.38:0/3250467545 shutdown_connections 2024-05-21T15:34:00.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.471+0000 7f3c35a7a640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3c180a2c30 0x7f3c180a5140 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.471+0000 7f3c35a7a640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c301078f0 0x7f3c3019f5f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.471+0000 7f3c35a7a640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c30102bf0 0x7f3c30198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.471+0000 7f3c35a7a640 1 --2- 172.21.15.38:0/3250467545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c301019b0 0x7f3c30198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.471+0000 7f3c35a7a640 1 -- 172.21.15.38:0/3250467545 >> 172.21.15.38:0/3250467545 conn(0x7f3c300fcba0 msgr2=0x7f3c301038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:00.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.471+0000 7f3c35a7a640 1 -- 172.21.15.38:0/3250467545 shutdown_connections 2024-05-21T15:34:00.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.471+0000 7f3c35a7a640 1 -- 172.21.15.38:0/3250467545 wait complete. 2024-05-21T15:34:00.482 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:27:50.659367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:32:31.645647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:32:32.955451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:34:00.483 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:34:00.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.601+0000 7f2439af7640 1 -- 172.21.15.38:0/1370421456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2434101980 msgr2=0x7f2434102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:00.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.601+0000 7f2439af7640 1 --2- 172.21.15.38:0/1370421456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2434101980 0x7f2434102680 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2424009900 tx=0x7f242402a410 comp rx=0 tx=0).stop 2024-05-21T15:34:00.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.601+0000 7f2439af7640 1 -- 172.21.15.38:0/1370421456 shutdown_connections 2024-05-21T15:34:00.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.601+0000 7f2439af7640 1 --2- 172.21.15.38:0/1370421456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24341078c0 0x7f2434109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.601+0000 7f2439af7640 1 --2- 172.21.15.38:0/1370421456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2434102bc0 0x7f2434102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.601+0000 7f2439af7640 1 --2- 172.21.15.38:0/1370421456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2434101980 0x7f2434102680 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.601+0000 7f2439af7640 1 -- 172.21.15.38:0/1370421456 >> 172.21.15.38:0/1370421456 conn(0x7f24340fcb70 msgr2=0x7f24340ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:00.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.601+0000 7f2439af7640 1 -- 172.21.15.38:0/1370421456 shutdown_connections 2024-05-21T15:34:00.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.601+0000 7f2439af7640 1 -- 172.21.15.38:0/1370421456 wait complete. 2024-05-21T15:34:00.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.602+0000 7f2439af7640 1 Processor -- start 2024-05-21T15:34:00.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.602+0000 7f2439af7640 1 -- start start 2024-05-21T15:34:00.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.602+0000 7f2439af7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2434101980 0x7f243410c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:00.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.602+0000 7f2439af7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2434102bc0 0x7f243410ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:00.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.602+0000 7f2439af7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24341078c0 0x7f243410b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:00.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.602+0000 7f2439af7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f243410e6f0 con 0x7f2434102bc0 2024-05-21T15:34:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.602+0000 7f2439af7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f243410e570 con 0x7f2434101980 2024-05-21T15:34:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.602+0000 7f2439af7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f243410e870 con 0x7f24341078c0 2024-05-21T15:34:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2432ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2434102bc0 0x7f243410ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2432ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2434102bc0 0x7f243410ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51256/0 (socket says 172.21.15.38:51256) 2024-05-21T15:34:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2432ffd640 1 -- 172.21.15.38:0/532175078 learned_addr learned my addr 172.21.15.38:0/532175078 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f24337fe640 1 --2- 172.21.15.38:0/532175078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2434101980 0x7f243410c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2433fff640 1 --2- 172.21.15.38:0/532175078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24341078c0 0x7f243410b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2432ffd640 1 -- 172.21.15.38:0/532175078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24341078c0 msgr2=0x7f243410b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2432ffd640 1 --2- 172.21.15.38:0/532175078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24341078c0 0x7f243410b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2432ffd640 1 -- 172.21.15.38:0/532175078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2434101980 msgr2=0x7f243410c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2432ffd640 1 --2- 172.21.15.38:0/532175078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2434101980 0x7f243410c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2432ffd640 1 -- 172.21.15.38:0/532175078 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243410b7f0 con 0x7f2434102bc0 2024-05-21T15:34:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2432ffd640 1 --2- 172.21.15.38:0/532175078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2434102bc0 0x7f243410ab80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f242800e9f0 tx=0x7f242800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2430ff9640 1 -- 172.21.15.38:0/532175078 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242800cd80 con 0x7f2434102bc0 2024-05-21T15:34:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2439af7640 1 -- 172.21.15.38:0/532175078 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24341a3350 con 0x7f2434102bc0 2024-05-21T15:34:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2430ff9640 1 -- 172.21.15.38:0/532175078 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2428005500 con 0x7f2434102bc0 2024-05-21T15:34:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2430ff9640 1 -- 172.21.15.38:0/532175078 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428010690 con 0x7f2434102bc0 2024-05-21T15:34:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.603+0000 7f2439af7640 1 -- 172.21.15.38:0/532175078 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24341a37c0 con 0x7f2434102bc0 2024-05-21T15:34:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.605+0000 7f2439af7640 1 -- 172.21.15.38:0/532175078 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2434068d70 con 0x7f2434102bc0 2024-05-21T15:34:00.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.606+0000 7f2430ff9640 1 -- 172.21.15.38:0/532175078 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2428004f60 con 0x7f2434102bc0 2024-05-21T15:34:00.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.606+0000 7f2430ff9640 1 --2- 172.21.15.38:0/532175078 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f241c0a2b90 0x7f241c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:00.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.606+0000 7f2430ff9640 1 -- 172.21.15.38:0/532175078 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f24280cf470 con 0x7f2434102bc0 2024-05-21T15:34:00.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.606+0000 7f24337fe640 1 --2- 172.21.15.38:0/532175078 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f241c0a2b90 0x7f241c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:00.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.607+0000 7f24337fe640 1 --2- 172.21.15.38:0/532175078 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f241c0a2b90 0x7f241c0a50a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2424005ab0 tx=0x7f2424031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:00.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.611+0000 7f2430ff9640 1 -- 172.21.15.38:0/532175078 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f242809c3c0 con 0x7f2434102bc0 2024-05-21T15:34:00.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.794+0000 7f2439af7640 1 -- 172.21.15.38:0/532175078 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2434062750 con 0x7f2434102bc0 2024-05-21T15:34:00.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.795+0000 7f2430ff9640 1 -- 172.21.15.38:0/532175078 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f242809bcc0 con 0x7f2434102bc0 2024-05-21T15:34:00.794 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-05-21T15:34:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.796+0000 7f2439af7640 1 -- 172.21.15.38:0/532175078 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f241c0a2b90 msgr2=0x7f241c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.796+0000 7f2439af7640 1 --2- 172.21.15.38:0/532175078 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f241c0a2b90 0x7f241c0a50a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2424005ab0 tx=0x7f2424031400 comp rx=0 tx=0).stop 2024-05-21T15:34:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.796+0000 7f2439af7640 1 -- 172.21.15.38:0/532175078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2434102bc0 msgr2=0x7f243410ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.796+0000 7f2439af7640 1 --2- 172.21.15.38:0/532175078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2434102bc0 0x7f243410ab80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f242800e9f0 tx=0x7f242800edf0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.796+0000 7f2439af7640 1 -- 172.21.15.38:0/532175078 shutdown_connections 2024-05-21T15:34:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.796+0000 7f2439af7640 1 --2- 172.21.15.38:0/532175078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24341078c0 0x7f243410b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.796+0000 7f2439af7640 1 --2- 172.21.15.38:0/532175078 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f241c0a2b90 0x7f241c0a50a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.796+0000 7f2439af7640 1 --2- 172.21.15.38:0/532175078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2434102bc0 0x7f243410ab80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.796+0000 7f2439af7640 1 --2- 172.21.15.38:0/532175078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2434101980 0x7f243410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.796+0000 7f2439af7640 1 -- 172.21.15.38:0/532175078 >> 172.21.15.38:0/532175078 conn(0x7f24340fcb70 msgr2=0x7f24341038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.797+0000 7f2439af7640 1 -- 172.21.15.38:0/532175078 shutdown_connections 2024-05-21T15:34:00.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.797+0000 7f2439af7640 1 -- 172.21.15.38:0/532175078 wait complete. 2024-05-21T15:34:00.808 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:34:00.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.927+0000 7f193c66c640 1 -- 172.21.15.38:0/4150030185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1934102c00 msgr2=0x7f1934103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:00.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.927+0000 7f193c66c640 1 --2- 172.21.15.38:0/4150030185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1934102c00 0x7f1934103010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1928009900 tx=0x7f192802a410 comp rx=0 tx=0).stop 2024-05-21T15:34:00.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.927+0000 7f193c66c640 1 -- 172.21.15.38:0/4150030185 shutdown_connections 2024-05-21T15:34:00.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.927+0000 7f193c66c640 1 --2- 172.21.15.38:0/4150030185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1934107900 0x7f1934109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.927+0000 7f193c66c640 1 --2- 172.21.15.38:0/4150030185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1934102c00 0x7f1934103010 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.927+0000 7f193c66c640 1 --2- 172.21.15.38:0/4150030185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19341019c0 0x7f19341026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.927+0000 7f193c66c640 1 -- 172.21.15.38:0/4150030185 >> 172.21.15.38:0/4150030185 conn(0x7f19340fcb90 msgr2=0x7f19340ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:00.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.927+0000 7f193c66c640 1 -- 172.21.15.38:0/4150030185 shutdown_connections 2024-05-21T15:34:00.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.927+0000 7f193c66c640 1 -- 172.21.15.38:0/4150030185 wait complete. 2024-05-21T15:34:00.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.927+0000 7f193c66c640 1 Processor -- start 2024-05-21T15:34:00.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.927+0000 7f193c66c640 1 -- start start 2024-05-21T15:34:00.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193c66c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19341019c0 0x7f1934199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:00.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193c66c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1934102c00 0x7f19341997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:00.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193c66c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1934107900 0x7f19341a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:00.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193c66c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f193410d630 con 0x7f1934107900 2024-05-21T15:34:00.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193c66c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f193410d4b0 con 0x7f1934102c00 2024-05-21T15:34:00.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193c66c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f193410d7b0 con 0x7f19341019c0 2024-05-21T15:34:00.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193ac09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1934107900 0x7f19341a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:00.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193a408640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19341019c0 0x7f1934199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:00.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193ac09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1934107900 0x7f19341a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51270/0 (socket says 172.21.15.38:51270) 2024-05-21T15:34:00.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f1939c07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1934102c00 0x7f19341997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:00.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193ac09640 1 -- 172.21.15.38:0/1344729818 learned_addr learned my addr 172.21.15.38:0/1344729818 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:00.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193ac09640 1 -- 172.21.15.38:0/1344729818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19341019c0 msgr2=0x7f1934199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:00.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193ac09640 1 --2- 172.21.15.38:0/1344729818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19341019c0 0x7f1934199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193ac09640 1 -- 172.21.15.38:0/1344729818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1934102c00 msgr2=0x7f19341997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:00.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193ac09640 1 --2- 172.21.15.38:0/1344729818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1934102c00 0x7f19341997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:00.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193ac09640 1 -- 172.21.15.38:0/1344729818 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19341a0db0 con 0x7f1934107900 2024-05-21T15:34:00.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193a408640 1 --2- 172.21.15.38:0/1344729818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19341019c0 0x7f1934199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:34:00.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.928+0000 7f193ac09640 1 --2- 172.21.15.38:0/1344729818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1934107900 0x7f19341a0740 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f192400eaa0 tx=0x7f192400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:00.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.929+0000 7f19337fe640 1 -- 172.21.15.38:0/1344729818 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192400cdb0 con 0x7f1934107900 2024-05-21T15:34:00.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.929+0000 7f19337fe640 1 -- 172.21.15.38:0/1344729818 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1924005020 con 0x7f1934107900 2024-05-21T15:34:00.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.929+0000 7f19337fe640 1 -- 172.21.15.38:0/1344729818 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19240116b0 con 0x7f1934107900 2024-05-21T15:34:00.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.929+0000 7f193c66c640 1 -- 172.21.15.38:0/1344729818 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19341a10a0 con 0x7f1934107900 2024-05-21T15:34:00.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.929+0000 7f193c66c640 1 -- 172.21.15.38:0/1344729818 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19341a15f0 con 0x7f1934107900 2024-05-21T15:34:00.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.930+0000 7f193c66c640 1 -- 172.21.15.38:0/1344729818 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1934107000 con 0x7f1934107900 2024-05-21T15:34:00.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.931+0000 7f19337fe640 1 -- 172.21.15.38:0/1344729818 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1924005560 con 0x7f1934107900 2024-05-21T15:34:00.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.932+0000 7f19337fe640 1 --2- 172.21.15.38:0/1344729818 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f191c0a2c10 0x7f191c0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:00.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.932+0000 7f193a408640 1 --2- 172.21.15.38:0/1344729818 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f191c0a2c10 0x7f191c0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:00.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.932+0000 7f19337fe640 1 -- 172.21.15.38:0/1344729818 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f19240cf580 con 0x7f1934107900 2024-05-21T15:34:00.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.932+0000 7f193a408640 1 --2- 172.21.15.38:0/1344729818 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f191c0a2c10 0x7f191c0a5120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f19341025c0 tx=0x7f192c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:00.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:00.935+0000 7f19337fe640 1 -- 172.21.15.38:0/1344729818 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1924016030 con 0x7f1934107900 2024-05-21T15:34:01.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.119+0000 7f193c66c640 1 -- 172.21.15.38:0/1344729818 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f193402d430 con 0x7f1934107900 2024-05-21T15:34:01.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.120+0000 7f19337fe640 1 -- 172.21.15.38:0/1344729818 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f193402d430 con 0x7f1934107900 2024-05-21T15:34:01.119 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-05-21T15:34:01.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.121+0000 7f193c66c640 1 -- 172.21.15.38:0/1344729818 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f191c0a2c10 msgr2=0x7f191c0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.121+0000 7f193c66c640 1 --2- 172.21.15.38:0/1344729818 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f191c0a2c10 0x7f191c0a5120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f19341025c0 tx=0x7f192c005f90 comp rx=0 tx=0).stop 2024-05-21T15:34:01.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.121+0000 7f193c66c640 1 -- 172.21.15.38:0/1344729818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1934107900 msgr2=0x7f19341a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.121+0000 7f193c66c640 1 --2- 172.21.15.38:0/1344729818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1934107900 0x7f19341a0740 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f192400eaa0 tx=0x7f192400ef60 comp rx=0 tx=0).stop 2024-05-21T15:34:01.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.121+0000 7f193c66c640 1 -- 172.21.15.38:0/1344729818 shutdown_connections 2024-05-21T15:34:01.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.121+0000 7f193c66c640 1 --2- 172.21.15.38:0/1344729818 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f191c0a2c10 0x7f191c0a5120 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.121+0000 7f193c66c640 1 --2- 172.21.15.38:0/1344729818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1934107900 0x7f19341a0740 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.121+0000 7f193c66c640 1 --2- 172.21.15.38:0/1344729818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1934102c00 0x7f19341997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.121+0000 7f193c66c640 1 --2- 172.21.15.38:0/1344729818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19341019c0 0x7f1934199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.121+0000 7f193c66c640 1 -- 172.21.15.38:0/1344729818 >> 172.21.15.38:0/1344729818 conn(0x7f19340fcb90 msgr2=0x7f1934074660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:01.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.122+0000 7f193c66c640 1 -- 172.21.15.38:0/1344729818 shutdown_connections 2024-05-21T15:34:01.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.122+0000 7f193c66c640 1 -- 172.21.15.38:0/1344729818 wait complete. 2024-05-21T15:34:01.132 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:00 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3250467545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:01.132 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:00 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/532175078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:01.133 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:34:01.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:00 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3250467545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:01.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:00 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/532175078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:01.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.251+0000 7fd490c32640 1 -- 172.21.15.38:0/2034133345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd48c1078c0 msgr2=0x7fd48c109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.251+0000 7fd490c32640 1 --2- 172.21.15.38:0/2034133345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd48c1078c0 0x7fd48c109ce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd484009ca0 tx=0x7fd48402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.251+0000 7fd490c32640 1 -- 172.21.15.38:0/2034133345 shutdown_connections 2024-05-21T15:34:01.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.251+0000 7fd490c32640 1 --2- 172.21.15.38:0/2034133345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd48c1078c0 0x7fd48c109ce0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.251+0000 7fd490c32640 1 --2- 172.21.15.38:0/2034133345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd48c102bc0 0x7fd48c102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.251+0000 7fd490c32640 1 --2- 172.21.15.38:0/2034133345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd48c101980 0x7fd48c102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.251+0000 7fd490c32640 1 -- 172.21.15.38:0/2034133345 >> 172.21.15.38:0/2034133345 conn(0x7fd48c0fcb70 msgr2=0x7fd48c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:01.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.251+0000 7fd490c32640 1 -- 172.21.15.38:0/2034133345 shutdown_connections 2024-05-21T15:34:01.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.251+0000 7fd490c32640 1 -- 172.21.15.38:0/2034133345 wait complete. 2024-05-21T15:34:01.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd490c32640 1 Processor -- start 2024-05-21T15:34:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd490c32640 1 -- start start 2024-05-21T15:34:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd490c32640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd48c101980 0x7fd48c1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd490c32640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd48c102bc0 0x7fd48c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd490c32640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd48c1078c0 0x7fd48c19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd490c32640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10d600 con 0x7fd48c1078c0 2024-05-21T15:34:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd490c32640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10d480 con 0x7fd48c102bc0 2024-05-21T15:34:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd490c32640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10d780 con 0x7fd48c101980 2024-05-21T15:34:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd48ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd48c1078c0 0x7fd48c19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd48ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd48c1078c0 0x7fd48c19f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51280/0 (socket says 172.21.15.38:51280) 2024-05-21T15:34:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd48ad9d640 1 -- 172.21.15.38:0/54062989 learned_addr learned my addr 172.21.15.38:0/54062989 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd48a59c640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd48c101980 0x7fd48c1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd483fff640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd48c102bc0 0x7fd48c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd48ad9d640 1 -- 172.21.15.38:0/54062989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd48c101980 msgr2=0x7fd48c1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd48ad9d640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd48c101980 0x7fd48c1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd48ad9d640 1 -- 172.21.15.38:0/54062989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd48c102bc0 msgr2=0x7fd48c198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd48ad9d640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd48c102bc0 0x7fd48c198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd48ad9d640 1 -- 172.21.15.38:0/54062989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c19fc20 con 0x7fd48c1078c0 2024-05-21T15:34:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.252+0000 7fd483fff640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd48c102bc0 0x7fd48c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.253+0000 7fd48ad9d640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd48c1078c0 0x7fd48c19f5b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd484005cc0 tx=0x7fd484009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.253+0000 7fd4837fe640 1 -- 172.21.15.38:0/54062989 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484003c30 con 0x7fd48c1078c0 2024-05-21T15:34:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.253+0000 7fd4837fe640 1 -- 172.21.15.38:0/54062989 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd484003dd0 con 0x7fd48c1078c0 2024-05-21T15:34:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.253+0000 7fd490c32640 1 -- 172.21.15.38:0/54062989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c19feb0 con 0x7fd48c1078c0 2024-05-21T15:34:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.253+0000 7fd4837fe640 1 -- 172.21.15.38:0/54062989 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48402fc50 con 0x7fd48c1078c0 2024-05-21T15:34:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.253+0000 7fd490c32640 1 -- 172.21.15.38:0/54062989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1a0310 con 0x7fd48c1078c0 2024-05-21T15:34:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.254+0000 7fd490c32640 1 -- 172.21.15.38:0/54062989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c068d70 con 0x7fd48c1078c0 2024-05-21T15:34:01.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.255+0000 7fd4837fe640 1 -- 172.21.15.38:0/54062989 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd484004050 con 0x7fd48c1078c0 2024-05-21T15:34:01.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.256+0000 7fd4837fe640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd4780a2b90 0x7fd4780a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:01.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.256+0000 7fd48a59c640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd4780a2b90 0x7fd4780a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:01.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.256+0000 7fd4837fe640 1 -- 172.21.15.38:0/54062989 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fd4840f04f0 con 0x7fd48c1078c0 2024-05-21T15:34:01.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.256+0000 7fd48a59c640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd4780a2b90 0x7fd4780a50a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd47c005fd0 tx=0x7fd47c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:01.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.261+0000 7fd4837fe640 1 -- 172.21.15.38:0/54062989 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd4840bd440 con 0x7fd48c1078c0 2024-05-21T15:34:01.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:00 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3250467545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:01.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:00 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/532175078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:01.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.443+0000 7fd490c32640 1 -- 172.21.15.38:0/54062989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd48c062750 con 0x7fd48c1078c0 2024-05-21T15:34:01.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.444+0000 7fd4837fe640 1 -- 172.21.15.38:0/54062989 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fd4840bcd40 con 0x7fd48c1078c0 2024-05-21T15:34:01.444 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-05-21T15:34:01.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.446+0000 7fd490c32640 1 -- 172.21.15.38:0/54062989 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd4780a2b90 msgr2=0x7fd4780a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.446+0000 7fd490c32640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd4780a2b90 0x7fd4780a50a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd47c005fd0 tx=0x7fd47c005850 comp rx=0 tx=0).stop 2024-05-21T15:34:01.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.446+0000 7fd490c32640 1 -- 172.21.15.38:0/54062989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd48c1078c0 msgr2=0x7fd48c19f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.446+0000 7fd490c32640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd48c1078c0 0x7fd48c19f5b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd484005cc0 tx=0x7fd484009890 comp rx=0 tx=0).stop 2024-05-21T15:34:01.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.446+0000 7fd490c32640 1 -- 172.21.15.38:0/54062989 shutdown_connections 2024-05-21T15:34:01.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.446+0000 7fd490c32640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd4780a2b90 0x7fd4780a50a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.446+0000 7fd490c32640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd48c1078c0 0x7fd48c19f5b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.446+0000 7fd490c32640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd48c102bc0 0x7fd48c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.446+0000 7fd490c32640 1 --2- 172.21.15.38:0/54062989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd48c101980 0x7fd48c1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.446+0000 7fd490c32640 1 -- 172.21.15.38:0/54062989 >> 172.21.15.38:0/54062989 conn(0x7fd48c0fcb70 msgr2=0x7fd48c1038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:01.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.446+0000 7fd490c32640 1 -- 172.21.15.38:0/54062989 shutdown_connections 2024-05-21T15:34:01.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.446+0000 7fd490c32640 1 -- 172.21.15.38:0/54062989 wait complete. 2024-05-21T15:34:01.457 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:34:01.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.572+0000 7fad3c256640 1 -- 172.21.15.38:0/1457784303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad341019b0 msgr2=0x7fad341026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.572+0000 7fad3c256640 1 --2- 172.21.15.38:0/1457784303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad341019b0 0x7fad341026b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fad30009900 tx=0x7fad3002a410 comp rx=0 tx=0).stop 2024-05-21T15:34:01.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.573+0000 7fad3c256640 1 -- 172.21.15.38:0/1457784303 shutdown_connections 2024-05-21T15:34:01.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.573+0000 7fad3c256640 1 --2- 172.21.15.38:0/1457784303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad341078f0 0x7fad34109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.573+0000 7fad3c256640 1 --2- 172.21.15.38:0/1457784303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad34102bf0 0x7fad34103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.573+0000 7fad3c256640 1 --2- 172.21.15.38:0/1457784303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad341019b0 0x7fad341026b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.573+0000 7fad3c256640 1 -- 172.21.15.38:0/1457784303 >> 172.21.15.38:0/1457784303 conn(0x7fad340fcba0 msgr2=0x7fad340ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.573+0000 7fad3c256640 1 -- 172.21.15.38:0/1457784303 shutdown_connections 2024-05-21T15:34:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.573+0000 7fad3c256640 1 -- 172.21.15.38:0/1457784303 wait complete. 2024-05-21T15:34:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.573+0000 7fad3c256640 1 Processor -- start 2024-05-21T15:34:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.573+0000 7fad3c256640 1 -- start start 2024-05-21T15:34:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.573+0000 7fad3c256640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad341019b0 0x7fad34199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3c256640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad34102bf0 0x7fad34199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3c256640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad341078f0 0x7fad341a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3c256640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fad3410d600 con 0x7fad341078f0 2024-05-21T15:34:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3c256640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad3410d480 con 0x7fad341019b0 2024-05-21T15:34:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3c256640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fad3410d780 con 0x7fad34102bf0 2024-05-21T15:34:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3a7f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad341078f0 0x7fad341a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3a7f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad341078f0 0x7fad341a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51300/0 (socket says 172.21.15.38:51300) 2024-05-21T15:34:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3a7f3640 1 -- 172.21.15.38:0/913850951 learned_addr learned my addr 172.21.15.38:0/913850951 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad39ff2640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad341019b0 0x7fad34199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad397f1640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad34102bf0 0x7fad34199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3a7f3640 1 -- 172.21.15.38:0/913850951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad34102bf0 msgr2=0x7fad34199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3a7f3640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad34102bf0 0x7fad34199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3a7f3640 1 -- 172.21.15.38:0/913850951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad341019b0 msgr2=0x7fad34199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3a7f3640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad341019b0 0x7fad34199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3a7f3640 1 -- 172.21.15.38:0/913850951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad341a0d80 con 0x7fad341078f0 2024-05-21T15:34:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad397f1640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad34102bf0 0x7fad34199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3a7f3640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad341078f0 0x7fad341a0710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00eaa0 tx=0x7fad2c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad2affd640 1 -- 172.21.15.38:0/913850951 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c00cdb0 con 0x7fad341078f0 2024-05-21T15:34:01.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad2affd640 1 -- 172.21.15.38:0/913850951 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad2c005020 con 0x7fad341078f0 2024-05-21T15:34:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3c256640 1 -- 172.21.15.38:0/913850951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad341a1070 con 0x7fad341078f0 2024-05-21T15:34:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad2affd640 1 -- 172.21.15.38:0/913850951 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c0116b0 con 0x7fad341078f0 2024-05-21T15:34:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.574+0000 7fad3c256640 1 -- 172.21.15.38:0/913850951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad341a15c0 con 0x7fad341078f0 2024-05-21T15:34:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.575+0000 7fad3c256640 1 -- 172.21.15.38:0/913850951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad341a1880 con 0x7fad341078f0 2024-05-21T15:34:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.577+0000 7fad2affd640 1 -- 172.21.15.38:0/913850951 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fad2c005560 con 0x7fad341078f0 2024-05-21T15:34:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.578+0000 7fad2affd640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fad200a2c30 0x7fad200a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.578+0000 7fad39ff2640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fad200a2c30 0x7fad200a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.578+0000 7fad2affd640 1 -- 172.21.15.38:0/913850951 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fad2c0cf600 con 0x7fad341078f0 2024-05-21T15:34:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.578+0000 7fad39ff2640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fad200a2c30 0x7fad200a5140 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fad30005ab0 tx=0x7fad30031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:01.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.581+0000 7fad2affd640 1 -- 172.21.15.38:0/913850951 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fad2c016030 con 0x7fad341078f0 2024-05-21T15:34:01.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.803+0000 7fad3c256640 1 -- 172.21.15.38:0/913850951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad341a1a50 con 0x7fad341078f0 2024-05-21T15:34:01.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.803+0000 7fad2affd640 1 -- 172.21.15.38:0/913850951 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11914 (secure 0 0 0) 0x7fad2c09c4f0 con 0x7fad341078f0 2024-05-21T15:34:01.803 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:01.803 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:32:40.979368+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24550,24515,24602,24523,24551]},"id":1}]} 2024-05-21T15:34:01.803 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-05-21T15:34:01.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.805+0000 7fad3c256640 1 -- 172.21.15.38:0/913850951 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fad200a2c30 msgr2=0x7fad200a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.805+0000 7fad3c256640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fad200a2c30 0x7fad200a5140 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fad30005ab0 tx=0x7fad30031400 comp rx=0 tx=0).stop 2024-05-21T15:34:01.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.805+0000 7fad3c256640 1 -- 172.21.15.38:0/913850951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad341078f0 msgr2=0x7fad341a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.805+0000 7fad3c256640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad341078f0 0x7fad341a0710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00eaa0 tx=0x7fad2c00ef60 comp rx=0 tx=0).stop 2024-05-21T15:34:01.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.805+0000 7fad3c256640 1 -- 172.21.15.38:0/913850951 shutdown_connections 2024-05-21T15:34:01.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.806+0000 7fad3c256640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fad200a2c30 0x7fad200a5140 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.806+0000 7fad3c256640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad341078f0 0x7fad341a0710 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.806+0000 7fad3c256640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad34102bf0 0x7fad34199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.806+0000 7fad3c256640 1 --2- 172.21.15.38:0/913850951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad341019b0 0x7fad34199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.806+0000 7fad3c256640 1 -- 172.21.15.38:0/913850951 >> 172.21.15.38:0/913850951 conn(0x7fad340fcba0 msgr2=0x7fad34108dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:01.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.806+0000 7fad3c256640 1 -- 172.21.15.38:0/913850951 shutdown_connections 2024-05-21T15:34:01.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.806+0000 7fad3c256640 1 -- 172.21.15.38:0/913850951 wait complete. 2024-05-21T15:34:01.817 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:34:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.933+0000 7f239d979640 1 -- 172.21.15.38:0/1783803539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23981019b0 msgr2=0x7f23981026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.933+0000 7f239d979640 1 --2- 172.21.15.38:0/1783803539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23981019b0 0x7f23981026b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2388009900 tx=0x7f238802a410 comp rx=0 tx=0).stop 2024-05-21T15:34:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.933+0000 7f239d979640 1 -- 172.21.15.38:0/1783803539 shutdown_connections 2024-05-21T15:34:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.933+0000 7f239d979640 1 --2- 172.21.15.38:0/1783803539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23981078f0 0x7f2398109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.933+0000 7f239d979640 1 --2- 172.21.15.38:0/1783803539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2398102bf0 0x7f2398103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.933+0000 7f239d979640 1 --2- 172.21.15.38:0/1783803539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23981019b0 0x7f23981026b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.933+0000 7f239d979640 1 -- 172.21.15.38:0/1783803539 >> 172.21.15.38:0/1783803539 conn(0x7f23980fcba0 msgr2=0x7f23980ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.933+0000 7f239d979640 1 -- 172.21.15.38:0/1783803539 shutdown_connections 2024-05-21T15:34:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.933+0000 7f239d979640 1 -- 172.21.15.38:0/1783803539 wait complete. 2024-05-21T15:34:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.934+0000 7f239d979640 1 Processor -- start 2024-05-21T15:34:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.934+0000 7f239d979640 1 -- start start 2024-05-21T15:34:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.934+0000 7f239d979640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23981019b0 0x7f23981991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.934+0000 7f239d979640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2398102bf0 0x7f2398199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.934+0000 7f239d979640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23981078f0 0x7f23981a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.934+0000 7f239d979640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f239810e730 con 0x7f2398102bf0 2024-05-21T15:34:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.934+0000 7f239d979640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f239810e5b0 con 0x7f23981078f0 2024-05-21T15:34:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.934+0000 7f239d979640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f239810e8b0 con 0x7f23981019b0 2024-05-21T15:34:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.934+0000 7f238ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2398102bf0 0x7f2398199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.934+0000 7f238ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2398102bf0 0x7f2398199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51314/0 (socket says 172.21.15.38:51314) 2024-05-21T15:34:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.934+0000 7f238ffff640 1 -- 172.21.15.38:0/3188273809 learned_addr learned my addr 172.21.15.38:0/3188273809 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.934+0000 7f2396ffd640 1 --2- 172.21.15.38:0/3188273809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23981019b0 0x7f23981991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.935+0000 7f23977fe640 1 --2- 172.21.15.38:0/3188273809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23981078f0 0x7f23981a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.935+0000 7f238ffff640 1 -- 172.21.15.38:0/3188273809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23981019b0 msgr2=0x7f23981991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.935+0000 7f238ffff640 1 --2- 172.21.15.38:0/3188273809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23981019b0 0x7f23981991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.935+0000 7f238ffff640 1 -- 172.21.15.38:0/3188273809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23981078f0 msgr2=0x7f23981a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.935+0000 7f238ffff640 1 --2- 172.21.15.38:0/3188273809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23981078f0 0x7f23981a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.935+0000 7f238ffff640 1 -- 172.21.15.38:0/3188273809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23981a0d30 con 0x7f2398102bf0 2024-05-21T15:34:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.935+0000 7f238ffff640 1 --2- 172.21.15.38:0/3188273809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2398102bf0 0x7f2398199730 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f238000e9f0 tx=0x7f238000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.935+0000 7f239c977640 1 -- 172.21.15.38:0/3188273809 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238000cd80 con 0x7f2398102bf0 2024-05-21T15:34:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.935+0000 7f239c977640 1 -- 172.21.15.38:0/3188273809 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2380010910 con 0x7f2398102bf0 2024-05-21T15:34:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.935+0000 7f239d979640 1 -- 172.21.15.38:0/3188273809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23981a0fc0 con 0x7f2398102bf0 2024-05-21T15:34:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.935+0000 7f239c977640 1 -- 172.21.15.38:0/3188273809 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380018990 con 0x7f2398102bf0 2024-05-21T15:34:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.935+0000 7f239d979640 1 -- 172.21.15.38:0/3188273809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23981a14d0 con 0x7f2398102bf0 2024-05-21T15:34:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.936+0000 7f239d979640 1 -- 172.21.15.38:0/3188273809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2398106700 con 0x7f2398102bf0 2024-05-21T15:34:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.937+0000 7f239c977640 1 -- 172.21.15.38:0/3188273809 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2380010ac0 con 0x7f2398102bf0 2024-05-21T15:34:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.938+0000 7f239c977640 1 --2- 172.21.15.38:0/3188273809 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f23840a2cb0 0x7f23840a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.938+0000 7f2396ffd640 1 --2- 172.21.15.38:0/3188273809 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f23840a2cb0 0x7f23840a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:01.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.938+0000 7f239c977640 1 -- 172.21.15.38:0/3188273809 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f23800cf470 con 0x7f2398102bf0 2024-05-21T15:34:01.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.939+0000 7f2396ffd640 1 --2- 172.21.15.38:0/3188273809 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f23840a2cb0 0x7f23840a51c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2388009230 tx=0x7f2388031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:01.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:01.943+0000 7f239c977640 1 -- 172.21.15.38:0/3188273809 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f238009c360 con 0x7f2398102bf0 2024-05-21T15:34:02.126 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:01 smithi053 ceph-mon[36496]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:34:02.126 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:01 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1344729818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:02.126 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:01 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/54062989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:02.126 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:01 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/913850951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:02.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.130+0000 7f239d979640 1 -- 172.21.15.38:0/3188273809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2398062750 con 0x7f2398102bf0 2024-05-21T15:34:02.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.132+0000 7f239c977640 1 -- 172.21.15.38:0/3188273809 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22609 (secure 0 0 0) 0x7f238009bc60 con 0x7f2398102bf0 2024-05-21T15:34:02.132 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:02.132 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:33:23.367336+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:34:02.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.134+0000 7f239d979640 1 -- 172.21.15.38:0/3188273809 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f23840a2cb0 msgr2=0x7f23840a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.134+0000 7f239d979640 1 --2- 172.21.15.38:0/3188273809 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f23840a2cb0 0x7f23840a51c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2388009230 tx=0x7f2388031400 comp rx=0 tx=0).stop 2024-05-21T15:34:02.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.134+0000 7f239d979640 1 -- 172.21.15.38:0/3188273809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2398102bf0 msgr2=0x7f2398199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.134+0000 7f239d979640 1 --2- 172.21.15.38:0/3188273809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2398102bf0 0x7f2398199730 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f238000e9f0 tx=0x7f238000eeb0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.134+0000 7f239d979640 1 -- 172.21.15.38:0/3188273809 shutdown_connections 2024-05-21T15:34:02.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.134+0000 7f239d979640 1 --2- 172.21.15.38:0/3188273809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23981078f0 0x7f23981a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.134+0000 7f239d979640 1 --2- 172.21.15.38:0/3188273809 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f23840a2cb0 0x7f23840a51c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.134+0000 7f239d979640 1 --2- 172.21.15.38:0/3188273809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2398102bf0 0x7f2398199730 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.134+0000 7f239d979640 1 --2- 172.21.15.38:0/3188273809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23981019b0 0x7f23981991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.134+0000 7f239d979640 1 -- 172.21.15.38:0/3188273809 >> 172.21.15.38:0/3188273809 conn(0x7f23980fcba0 msgr2=0x7f23981038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:02.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.134+0000 7f239d979640 1 -- 172.21.15.38:0/3188273809 shutdown_connections 2024-05-21T15:34:02.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.134+0000 7f239d979640 1 -- 172.21.15.38:0/3188273809 wait complete. 2024-05-21T15:34:02.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:01 smithi038 ceph-mon[31995]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:34:02.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:01 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1344729818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:02.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:01 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/54062989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:02.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:01 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/913850951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:02.145 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:27:50.659367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:32:31.645647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:32:32.955451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:34:02.146 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:34:02.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:01 smithi045 ceph-mon[36409]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:34:02.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:01 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1344729818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:02.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:01 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/54062989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:02.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:01 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/913850951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:02.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.265+0000 7f98814f7640 1 -- 172.21.15.38:0/1148026966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f987c105f10 msgr2=0x7f987c108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.265+0000 7f98814f7640 1 --2- 172.21.15.38:0/1148026966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f987c105f10 0x7f987c108330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9864009900 tx=0x7f986402a410 comp rx=0 tx=0).stop 2024-05-21T15:34:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.265+0000 7f98814f7640 1 -- 172.21.15.38:0/1148026966 shutdown_connections 2024-05-21T15:34:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.265+0000 7f98814f7640 1 --2- 172.21.15.38:0/1148026966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f987c108870 0x7f987c10ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.265+0000 7f98814f7640 1 --2- 172.21.15.38:0/1148026966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f987c105f10 0x7f987c108330 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.265+0000 7f98814f7640 1 --2- 172.21.15.38:0/1148026966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c1035b0 0x7f987c1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.265+0000 7f98814f7640 1 -- 172.21.15.38:0/1148026966 >> 172.21.15.38:0/1148026966 conn(0x7f987c0fcbb0 msgr2=0x7f987c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.265+0000 7f98814f7640 1 -- 172.21.15.38:0/1148026966 shutdown_connections 2024-05-21T15:34:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.266+0000 7f98814f7640 1 -- 172.21.15.38:0/1148026966 wait complete. 2024-05-21T15:34:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.266+0000 7f98814f7640 1 Processor -- start 2024-05-21T15:34:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.266+0000 7f98814f7640 1 -- start start 2024-05-21T15:34:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.266+0000 7f98814f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f987c1035b0 0x7f987c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.266+0000 7f98814f7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c105f10 0x7f987c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.266+0000 7f98814f7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f987c108870 0x7f987c1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.266+0000 7f98814f7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f987c10e790 con 0x7f987c1035b0 2024-05-21T15:34:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.266+0000 7f98814f7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f987c10e610 con 0x7f987c105f10 2024-05-21T15:34:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.266+0000 7f98814f7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f987c10e910 con 0x7f987c108870 2024-05-21T15:34:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.266+0000 7f987affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f987c1035b0 0x7f987c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.266+0000 7f987affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f987c1035b0 0x7f987c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51332/0 (socket says 172.21.15.38:51332) 2024-05-21T15:34:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.267+0000 7f987affd640 1 -- 172.21.15.38:0/2719144856 learned_addr learned my addr 172.21.15.38:0/2719144856 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.267+0000 7f987b7fe640 1 --2- 172.21.15.38:0/2719144856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f987c108870 0x7f987c1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.267+0000 7f9873fff640 1 --2- 172.21.15.38:0/2719144856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c105f10 0x7f987c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.267+0000 7f987affd640 1 -- 172.21.15.38:0/2719144856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f987c108870 msgr2=0x7f987c1a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.267+0000 7f987affd640 1 --2- 172.21.15.38:0/2719144856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f987c108870 0x7f987c1a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.267+0000 7f987affd640 1 -- 172.21.15.38:0/2719144856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c105f10 msgr2=0x7f987c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.267+0000 7f987affd640 1 --2- 172.21.15.38:0/2719144856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c105f10 0x7f987c1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.267+0000 7f987affd640 1 -- 172.21.15.38:0/2719144856 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c1a0d00 con 0x7f987c1035b0 2024-05-21T15:34:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.267+0000 7f987affd640 1 --2- 172.21.15.38:0/2719144856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f987c1035b0 0x7f987c199270 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c970 tx=0x7f986c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.267+0000 7f9878ff9640 1 -- 172.21.15.38:0/2719144856 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c014070 con 0x7f987c1035b0 2024-05-21T15:34:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.267+0000 7f9878ff9640 1 -- 172.21.15.38:0/2719144856 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f986c004440 con 0x7f987c1035b0 2024-05-21T15:34:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.267+0000 7f9878ff9640 1 -- 172.21.15.38:0/2719144856 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c00f450 con 0x7f987c1035b0 2024-05-21T15:34:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.267+0000 7f98814f7640 1 -- 172.21.15.38:0/2719144856 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987c1a0ff0 con 0x7f987c1035b0 2024-05-21T15:34:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.267+0000 7f98814f7640 1 -- 172.21.15.38:0/2719144856 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f987c1a14b0 con 0x7f987c1035b0 2024-05-21T15:34:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.268+0000 7f98814f7640 1 -- 172.21.15.38:0/2719144856 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f987c0fe6d0 con 0x7f987c1035b0 2024-05-21T15:34:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.269+0000 7f9878ff9640 1 -- 172.21.15.38:0/2719144856 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f986c017070 con 0x7f987c1035b0 2024-05-21T15:34:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.271+0000 7f9878ff9640 1 --2- 172.21.15.38:0/2719144856 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f98680a2c10 0x7f98680a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.271+0000 7f9873fff640 1 --2- 172.21.15.38:0/2719144856 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f98680a2c10 0x7f98680a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.271+0000 7f9878ff9640 1 -- 172.21.15.38:0/2719144856 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f986c0cf230 con 0x7f987c1035b0 2024-05-21T15:34:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.271+0000 7f9873fff640 1 --2- 172.21.15.38:0/2719144856 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f98680a2c10 0x7f98680a5120 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f986402a930 tx=0x7f9864005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:02.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.274+0000 7f9878ff9640 1 -- 172.21.15.38:0/2719144856 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f986c01e020 con 0x7f987c1035b0 2024-05-21T15:34:02.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.460+0000 7f98814f7640 1 -- 172.21.15.38:0/2719144856 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f987c02d4f0 con 0x7f987c1035b0 2024-05-21T15:34:02.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.461+0000 7f9878ff9640 1 -- 172.21.15.38:0/2719144856 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f986c09c200 con 0x7f987c1035b0 2024-05-21T15:34:02.461 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-05-21T15:34:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.463+0000 7f98814f7640 1 -- 172.21.15.38:0/2719144856 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f98680a2c10 msgr2=0x7f98680a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.463+0000 7f98814f7640 1 --2- 172.21.15.38:0/2719144856 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f98680a2c10 0x7f98680a5120 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f986402a930 tx=0x7f9864005fd0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.463+0000 7f98814f7640 1 -- 172.21.15.38:0/2719144856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f987c1035b0 msgr2=0x7f987c199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.463+0000 7f98814f7640 1 --2- 172.21.15.38:0/2719144856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f987c1035b0 0x7f987c199270 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c970 tx=0x7f986c00ce30 comp rx=0 tx=0).stop 2024-05-21T15:34:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.463+0000 7f98814f7640 1 -- 172.21.15.38:0/2719144856 shutdown_connections 2024-05-21T15:34:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.463+0000 7f98814f7640 1 --2- 172.21.15.38:0/2719144856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f987c108870 0x7f987c1a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.463+0000 7f98814f7640 1 --2- 172.21.15.38:0/2719144856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c105f10 0x7f987c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.463+0000 7f98814f7640 1 --2- 172.21.15.38:0/2719144856 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f98680a2c10 0x7f98680a5120 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.463+0000 7f98814f7640 1 --2- 172.21.15.38:0/2719144856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f987c1035b0 0x7f987c199270 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.463+0000 7f98814f7640 1 -- 172.21.15.38:0/2719144856 >> 172.21.15.38:0/2719144856 conn(0x7f987c0fcbb0 msgr2=0x7f987c104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.463+0000 7f98814f7640 1 -- 172.21.15.38:0/2719144856 shutdown_connections 2024-05-21T15:34:02.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.463+0000 7f98814f7640 1 -- 172.21.15.38:0/2719144856 wait complete. 2024-05-21T15:34:02.475 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:34:02.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.592+0000 7f5b21f77640 1 -- 172.21.15.38:0/4210436788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c1078f0 msgr2=0x7f5b1c109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.593+0000 7f5b21f77640 1 --2- 172.21.15.38:0/4210436788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c1078f0 0x7f5b1c109d10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009900 tx=0x7f5b1402a410 comp rx=0 tx=0).stop 2024-05-21T15:34:02.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.593+0000 7f5b21f77640 1 -- 172.21.15.38:0/4210436788 shutdown_connections 2024-05-21T15:34:02.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.593+0000 7f5b21f77640 1 --2- 172.21.15.38:0/4210436788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c1078f0 0x7f5b1c109d10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.593+0000 7f5b21f77640 1 --2- 172.21.15.38:0/4210436788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b1c102bf0 0x7f5b1c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.593+0000 7f5b21f77640 1 --2- 172.21.15.38:0/4210436788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b1c1019b0 0x7f5b1c1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.593+0000 7f5b21f77640 1 -- 172.21.15.38:0/4210436788 >> 172.21.15.38:0/4210436788 conn(0x7f5b1c0fcbe0 msgr2=0x7f5b1c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:02.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.593+0000 7f5b21f77640 1 -- 172.21.15.38:0/4210436788 shutdown_connections 2024-05-21T15:34:02.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.593+0000 7f5b21f77640 1 -- 172.21.15.38:0/4210436788 wait complete. 2024-05-21T15:34:02.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.593+0000 7f5b21f77640 1 Processor -- start 2024-05-21T15:34:02.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.593+0000 7f5b21f77640 1 -- start start 2024-05-21T15:34:02.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b21f77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b1c1019b0 0x7f5b1c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:02.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b21f77640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c102bf0 0x7f5b1c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:02.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b21f77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b1c1078f0 0x7f5b1c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:02.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b21f77640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c10d660 con 0x7f5b1c102bf0 2024-05-21T15:34:02.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b21f77640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c10d4e0 con 0x7f5b1c1019b0 2024-05-21T15:34:02.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b21f77640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c10d7e0 con 0x7f5b1c1078f0 2024-05-21T15:34:02.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b1affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c102bf0 0x7f5b1c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:02.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b1affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c102bf0 0x7f5b1c198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51356/0 (socket says 172.21.15.38:51356) 2024-05-21T15:34:02.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b1affd640 1 -- 172.21.15.38:0/1575647921 learned_addr learned my addr 172.21.15.38:0/1575647921 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:02.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b1bfff640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b1c1078f0 0x7f5b1c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:02.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b1b7fe640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b1c1019b0 0x7f5b1c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:02.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b1affd640 1 -- 172.21.15.38:0/1575647921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b1c1078f0 msgr2=0x7f5b1c19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b1affd640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b1c1078f0 0x7f5b1c19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b1affd640 1 -- 172.21.15.38:0/1575647921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b1c1019b0 msgr2=0x7f5b1c198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b1affd640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b1c1019b0 0x7f5b1c198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b1affd640 1 -- 172.21.15.38:0/1575647921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1c19fb90 con 0x7f5b1c102bf0 2024-05-21T15:34:02.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b1b7fe640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b1c1019b0 0x7f5b1c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:02.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b1affd640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c102bf0 0x7f5b1c198640 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c920 tx=0x7f5b0400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:02.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b18ff9640 1 -- 172.21.15.38:0/1575647921 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04007c70 con 0x7f5b1c102bf0 2024-05-21T15:34:02.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b18ff9640 1 -- 172.21.15.38:0/1575647921 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b04007e10 con 0x7f5b1c102bf0 2024-05-21T15:34:02.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.594+0000 7f5b21f77640 1 -- 172.21.15.38:0/1575647921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1c19fe80 con 0x7f5b1c102bf0 2024-05-21T15:34:02.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.595+0000 7f5b18ff9640 1 -- 172.21.15.38:0/1575647921 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04018890 con 0x7f5b1c102bf0 2024-05-21T15:34:02.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.595+0000 7f5b21f77640 1 -- 172.21.15.38:0/1575647921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1c1a02f0 con 0x7f5b1c102bf0 2024-05-21T15:34:02.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.596+0000 7f5b21f77640 1 -- 172.21.15.38:0/1575647921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b1c106700 con 0x7f5b1c102bf0 2024-05-21T15:34:02.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.597+0000 7f5b18ff9640 1 -- 172.21.15.38:0/1575647921 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5b04018a70 con 0x7f5b1c102bf0 2024-05-21T15:34:02.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.598+0000 7f5b18ff9640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5b080a2b90 0x7f5b080a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:02.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.598+0000 7f5b1b7fe640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5b080a2b90 0x7f5b080a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:02.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.598+0000 7f5b18ff9640 1 -- 172.21.15.38:0/1575647921 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5b040cf200 con 0x7f5b1c102bf0 2024-05-21T15:34:02.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.598+0000 7f5b1b7fe640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5b080a2b90 0x7f5b080a50a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c1025b0 tx=0x7f5b0c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:02.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.602+0000 7f5b18ff9640 1 -- 172.21.15.38:0/1575647921 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b04014030 con 0x7f5b1c102bf0 2024-05-21T15:34:02.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.787+0000 7f5b21f77640 1 -- 172.21.15.38:0/1575647921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5b1c062750 con 0x7f5b1c102bf0 2024-05-21T15:34:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.787+0000 7f5b18ff9640 1 -- 172.21.15.38:0/1575647921 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f5b0409c0f0 con 0x7f5b1c102bf0 2024-05-21T15:34:02.787 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-05-21T15:34:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.789+0000 7f5b21f77640 1 -- 172.21.15.38:0/1575647921 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5b080a2b90 msgr2=0x7f5b080a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.789+0000 7f5b21f77640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5b080a2b90 0x7f5b080a50a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c1025b0 tx=0x7f5b0c009500 comp rx=0 tx=0).stop 2024-05-21T15:34:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.789+0000 7f5b21f77640 1 -- 172.21.15.38:0/1575647921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c102bf0 msgr2=0x7f5b1c198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.789+0000 7f5b21f77640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c102bf0 0x7f5b1c198640 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c920 tx=0x7f5b0400cde0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.789+0000 7f5b21f77640 1 -- 172.21.15.38:0/1575647921 shutdown_connections 2024-05-21T15:34:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.789+0000 7f5b21f77640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b1c1078f0 0x7f5b1c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.789+0000 7f5b21f77640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5b080a2b90 0x7f5b080a50a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.789+0000 7f5b21f77640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b1c102bf0 0x7f5b1c198640 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.789+0000 7f5b21f77640 1 --2- 172.21.15.38:0/1575647921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b1c1019b0 0x7f5b1c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.789+0000 7f5b21f77640 1 -- 172.21.15.38:0/1575647921 >> 172.21.15.38:0/1575647921 conn(0x7f5b1c0fcbe0 msgr2=0x7f5b1c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:02.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.789+0000 7f5b21f77640 1 -- 172.21.15.38:0/1575647921 shutdown_connections 2024-05-21T15:34:02.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.789+0000 7f5b21f77640 1 -- 172.21.15.38:0/1575647921 wait complete. 2024-05-21T15:34:02.800 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:34:02.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.918+0000 7f746ee75640 1 -- 172.21.15.38:0/4276081714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7468100e20 msgr2=0x7f7468105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.918+0000 7f746ee75640 1 --2- 172.21.15.38:0/4276081714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7468100e20 0x7f7468105ae0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7458009ca0 tx=0x7f745802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.918+0000 7f746ee75640 1 -- 172.21.15.38:0/4276081714 shutdown_connections 2024-05-21T15:34:02.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.918+0000 7f746ee75640 1 --2- 172.21.15.38:0/4276081714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7468108740 0x7f746810ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.918+0000 7f746ee75640 1 --2- 172.21.15.38:0/4276081714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7468101320 0x7f74681080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.918+0000 7f746ee75640 1 --2- 172.21.15.38:0/4276081714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7468100e20 0x7f7468105ae0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.918+0000 7f746ee75640 1 -- 172.21.15.38:0/4276081714 >> 172.21.15.38:0/4276081714 conn(0x7f74680fc330 msgr2=0x7f74680fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:02.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.918+0000 7f746ee75640 1 -- 172.21.15.38:0/4276081714 shutdown_connections 2024-05-21T15:34:02.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.918+0000 7f746ee75640 1 -- 172.21.15.38:0/4276081714 wait complete. 2024-05-21T15:34:02.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746ee75640 1 Processor -- start 2024-05-21T15:34:02.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746ee75640 1 -- start start 2024-05-21T15:34:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746ee75640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7468100e20 0x7f7468199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746ee75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7468101320 0x7f74681997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746ee75640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7468108740 0x7f74681a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746ee75640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f746810e6f0 con 0x7f7468100e20 2024-05-21T15:34:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746ee75640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f746810e570 con 0x7f7468101320 2024-05-21T15:34:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746ee75640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f746810e870 con 0x7f7468108740 2024-05-21T15:34:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746cc11640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7468100e20 0x7f7468199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746cc11640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7468100e20 0x7f7468199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51388/0 (socket says 172.21.15.38:51388) 2024-05-21T15:34:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746cc11640 1 -- 172.21.15.38:0/4139492453 learned_addr learned my addr 172.21.15.38:0/4139492453 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746d412640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7468108740 0x7f74681a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f7467fff640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7468101320 0x7f74681997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746cc11640 1 -- 172.21.15.38:0/4139492453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7468108740 msgr2=0x7f74681a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746cc11640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7468108740 0x7f74681a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746cc11640 1 -- 172.21.15.38:0/4139492453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7468101320 msgr2=0x7f74681997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:02.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746cc11640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7468101320 0x7f74681997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:02.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746cc11640 1 -- 172.21.15.38:0/4139492453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74681a0cf0 con 0x7f7468100e20 2024-05-21T15:34:02.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746d412640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7468108740 0x7f74681a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:02.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f7467fff640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7468101320 0x7f74681997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:02.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.919+0000 7f746cc11640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7468100e20 0x7f7468199260 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f745802ade0 tx=0x7f7458031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:02.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.920+0000 7f7465ffb640 1 -- 172.21.15.38:0/4139492453 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745802fae0 con 0x7f7468100e20 2024-05-21T15:34:02.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.920+0000 7f7465ffb640 1 -- 172.21.15.38:0/4139492453 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f745802fc80 con 0x7f7468100e20 2024-05-21T15:34:02.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.920+0000 7f746ee75640 1 -- 172.21.15.38:0/4139492453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74681a0f80 con 0x7f7468100e20 2024-05-21T15:34:02.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.920+0000 7f7465ffb640 1 -- 172.21.15.38:0/4139492453 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458046750 con 0x7f7468100e20 2024-05-21T15:34:02.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.920+0000 7f746ee75640 1 -- 172.21.15.38:0/4139492453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74681a13e0 con 0x7f7468100e20 2024-05-21T15:34:02.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.921+0000 7f746ee75640 1 -- 172.21.15.38:0/4139492453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74680691f0 con 0x7f7468100e20 2024-05-21T15:34:02.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.922+0000 7f7465ffb640 1 -- 172.21.15.38:0/4139492453 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f74580468f0 con 0x7f7468100e20 2024-05-21T15:34:02.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.923+0000 7f7465ffb640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f74500a2b90 0x7f74500a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:02.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.923+0000 7f7465ffb640 1 -- 172.21.15.38:0/4139492453 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f74580f0970 con 0x7f7468100e20 2024-05-21T15:34:02.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.923+0000 7f7467fff640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f74500a2b90 0x7f74500a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:02.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.924+0000 7f7467fff640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f74500a2b90 0x7f74500a50a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f746819a300 tx=0x7f745c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:02.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:02.927+0000 7f7465ffb640 1 -- 172.21.15.38:0/4139492453 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7458081850 con 0x7f7468100e20 2024-05-21T15:34:03.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.110+0000 7f746ee75640 1 -- 172.21.15.38:0/4139492453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f746802d360 con 0x7f7468100e20 2024-05-21T15:34:03.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.111+0000 7f7465ffb640 1 -- 172.21.15.38:0/4139492453 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f74680691f0 con 0x7f7468100e20 2024-05-21T15:34:03.111 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-05-21T15:34:03.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.113+0000 7f746ee75640 1 -- 172.21.15.38:0/4139492453 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f74500a2b90 msgr2=0x7f74500a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.113+0000 7f746ee75640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f74500a2b90 0x7f74500a50a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f746819a300 tx=0x7f745c005f90 comp rx=0 tx=0).stop 2024-05-21T15:34:03.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.113+0000 7f746ee75640 1 -- 172.21.15.38:0/4139492453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7468100e20 msgr2=0x7f7468199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.113+0000 7f746ee75640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7468100e20 0x7f7468199260 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f745802ade0 tx=0x7f7458031ee0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.113+0000 7f746ee75640 1 -- 172.21.15.38:0/4139492453 shutdown_connections 2024-05-21T15:34:03.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.113+0000 7f746ee75640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7468108740 0x7f74681a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.113+0000 7f746ee75640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7468101320 0x7f74681997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.113+0000 7f746ee75640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f74500a2b90 0x7f74500a50a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.113+0000 7f746ee75640 1 --2- 172.21.15.38:0/4139492453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7468100e20 0x7f7468199260 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.113+0000 7f746ee75640 1 -- 172.21.15.38:0/4139492453 >> 172.21.15.38:0/4139492453 conn(0x7f74680fc330 msgr2=0x7f74681092d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:03.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.113+0000 7f746ee75640 1 -- 172.21.15.38:0/4139492453 shutdown_connections 2024-05-21T15:34:03.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.113+0000 7f746ee75640 1 -- 172.21.15.38:0/4139492453 wait complete. 2024-05-21T15:34:03.123 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:02 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3188273809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:03.123 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:02 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2719144856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:03.124 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:02 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1575647921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:03.125 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:34:03.162 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:02 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3188273809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:03.162 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:02 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2719144856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:03.162 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:02 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1575647921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:03.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:02 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3188273809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:03.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:02 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2719144856' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:03.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:02 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1575647921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:03.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.240+0000 7fd65babc640 1 -- 172.21.15.38:0/2290503620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd654105c10 msgr2=0x7fd6541080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.240+0000 7fd65babc640 1 --2- 172.21.15.38:0/2290503620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd654105c10 0x7fd6541080b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd644009900 tx=0x7fd64402a410 comp rx=0 tx=0).stop 2024-05-21T15:34:03.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.240+0000 7fd65babc640 1 -- 172.21.15.38:0/2290503620 shutdown_connections 2024-05-21T15:34:03.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.240+0000 7fd65babc640 1 --2- 172.21.15.38:0/2290503620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd654108740 0x7fd65410ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.240+0000 7fd65babc640 1 --2- 172.21.15.38:0/2290503620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd654105c10 0x7fd6541080b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.240+0000 7fd65babc640 1 --2- 172.21.15.38:0/2290503620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd654103220 0x7fd654105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.240+0000 7fd65babc640 1 -- 172.21.15.38:0/2290503620 >> 172.21.15.38:0/2290503620 conn(0x7fd6540fcba0 msgr2=0x7fd6540ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:03.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.240+0000 7fd65babc640 1 -- 172.21.15.38:0/2290503620 shutdown_connections 2024-05-21T15:34:03.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.240+0000 7fd65babc640 1 -- 172.21.15.38:0/2290503620 wait complete. 2024-05-21T15:34:03.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd65babc640 1 Processor -- start 2024-05-21T15:34:03.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd65babc640 1 -- start start 2024-05-21T15:34:03.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd65babc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd654103220 0x7fd6541991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:03.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd65babc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd654105c10 0x7fd6541996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:03.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd65babc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd654108740 0x7fd6541a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:03.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd65babc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd65410e720 con 0x7fd654105c10 2024-05-21T15:34:03.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd65babc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd65410e5a0 con 0x7fd654103220 2024-05-21T15:34:03.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd65babc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd65410e8a0 con 0x7fd654108740 2024-05-21T15:34:03.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd659057640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd654105c10 0x7fd6541996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:03.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd659057640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd654105c10 0x7fd6541996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51418/0 (socket says 172.21.15.38:51418) 2024-05-21T15:34:03.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd659057640 1 -- 172.21.15.38:0/1480270791 learned_addr learned my addr 172.21.15.38:0/1480270791 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:03.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd65a059640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd654108740 0x7fd6541a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:03.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd659858640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd654103220 0x7fd6541991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:03.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd659057640 1 -- 172.21.15.38:0/1480270791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd654108740 msgr2=0x7fd6541a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd659057640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd654108740 0x7fd6541a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd659057640 1 -- 172.21.15.38:0/1480270791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd654103220 msgr2=0x7fd6541991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd659057640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd654103220 0x7fd6541991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd659057640 1 -- 172.21.15.38:0/1480270791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6541a0cf0 con 0x7fd654105c10 2024-05-21T15:34:03.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd659858640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd654103220 0x7fd6541991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:03.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd65a059640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd654108740 0x7fd6541a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:03.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.241+0000 7fd659057640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd654105c10 0x7fd6541996f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd644006010 tx=0x7fd644031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:03.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.242+0000 7fd64affd640 1 -- 172.21.15.38:0/1480270791 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64403f030 con 0x7fd654105c10 2024-05-21T15:34:03.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.242+0000 7fd65babc640 1 -- 172.21.15.38:0/1480270791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6541a0f80 con 0x7fd654105c10 2024-05-21T15:34:03.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.242+0000 7fd64affd640 1 -- 172.21.15.38:0/1480270791 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd644003ad0 con 0x7fd654105c10 2024-05-21T15:34:03.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.242+0000 7fd64affd640 1 -- 172.21.15.38:0/1480270791 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd644035070 con 0x7fd654105c10 2024-05-21T15:34:03.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.242+0000 7fd65babc640 1 -- 172.21.15.38:0/1480270791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6541a13e0 con 0x7fd654105c10 2024-05-21T15:34:03.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.243+0000 7fd65babc640 1 -- 172.21.15.38:0/1480270791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6540fe6c0 con 0x7fd654105c10 2024-05-21T15:34:03.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.244+0000 7fd64affd640 1 -- 172.21.15.38:0/1480270791 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd644042030 con 0x7fd654105c10 2024-05-21T15:34:03.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.245+0000 7fd64affd640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd6400a2c30 0x7fd6400a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:03.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.245+0000 7fd64affd640 1 -- 172.21.15.38:0/1480270791 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fd6440efce0 con 0x7fd654105c10 2024-05-21T15:34:03.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.245+0000 7fd659858640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd6400a2c30 0x7fd6400a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:03.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.246+0000 7fd659858640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd6400a2c30 0x7fd6400a5140 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd6500058c0 tx=0x7fd650005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:03.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.248+0000 7fd64affd640 1 -- 172.21.15.38:0/1480270791 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd6440bcbd0 con 0x7fd654105c10 2024-05-21T15:34:03.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.473+0000 7fd65babc640 1 -- 172.21.15.38:0/1480270791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd65402d2a0 con 0x7fd654105c10 2024-05-21T15:34:03.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.474+0000 7fd64affd640 1 -- 172.21.15.38:0/1480270791 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11914 (secure 0 0 0) 0x7fd6440bc4d0 con 0x7fd654105c10 2024-05-21T15:34:03.473 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:03.474 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:32:40.979368+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24550,24515,24602,24523,24551]},"id":1}]} 2024-05-21T15:34:03.474 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-05-21T15:34:03.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.476+0000 7fd65babc640 1 -- 172.21.15.38:0/1480270791 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd6400a2c30 msgr2=0x7fd6400a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.476+0000 7fd65babc640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd6400a2c30 0x7fd6400a5140 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd6500058c0 tx=0x7fd650005850 comp rx=0 tx=0).stop 2024-05-21T15:34:03.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.476+0000 7fd65babc640 1 -- 172.21.15.38:0/1480270791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd654105c10 msgr2=0x7fd6541996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.476+0000 7fd65babc640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd654105c10 0x7fd6541996f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd644006010 tx=0x7fd644031b60 comp rx=0 tx=0).stop 2024-05-21T15:34:03.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.476+0000 7fd65babc640 1 -- 172.21.15.38:0/1480270791 shutdown_connections 2024-05-21T15:34:03.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.476+0000 7fd65babc640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd654108740 0x7fd6541a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.476+0000 7fd65babc640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd6400a2c30 0x7fd6400a5140 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.476+0000 7fd65babc640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd654105c10 0x7fd6541996f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.476+0000 7fd65babc640 1 --2- 172.21.15.38:0/1480270791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd654103220 0x7fd6541991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.476+0000 7fd65babc640 1 -- 172.21.15.38:0/1480270791 >> 172.21.15.38:0/1480270791 conn(0x7fd6540fcba0 msgr2=0x7fd654103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:03.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.476+0000 7fd65babc640 1 -- 172.21.15.38:0/1480270791 shutdown_connections 2024-05-21T15:34:03.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.476+0000 7fd65babc640 1 -- 172.21.15.38:0/1480270791 wait complete. 2024-05-21T15:34:03.488 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:34:03.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.602+0000 7f18f5565640 1 -- 172.21.15.38:0/1075036593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f0107900 msgr2=0x7f18f0109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.602+0000 7f18f5565640 1 --2- 172.21.15.38:0/1075036593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f0107900 0x7f18f0109d20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009ca0 tx=0x7f18e802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 -- 172.21.15.38:0/1075036593 shutdown_connections 2024-05-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 --2- 172.21.15.38:0/1075036593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f0107900 0x7f18f0109d20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 --2- 172.21.15.38:0/1075036593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18f0102c00 0x7f18f0103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 --2- 172.21.15.38:0/1075036593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f01019c0 0x7f18f01026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 -- 172.21.15.38:0/1075036593 >> 172.21.15.38:0/1075036593 conn(0x7f18f00fcbb0 msgr2=0x7f18f00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 -- 172.21.15.38:0/1075036593 shutdown_connections 2024-05-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 -- 172.21.15.38:0/1075036593 wait complete. 2024-05-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 Processor -- start 2024-05-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 -- start start 2024-05-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f01019c0 0x7f18f01991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18f0102c00 0x7f18f0199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f0107900 0x7f18f01a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f18f010e740 con 0x7f18f01019c0 2024-05-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18f010e5c0 con 0x7f18f0107900 2024-05-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.603+0000 7f18f5565640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f18f010e8c0 con 0x7f18f0102c00 2024-05-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f01019c0 0x7f18f01991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f01019c0 0x7f18f01991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51432/0 (socket says 172.21.15.38:51432) 2024-05-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18eeffd640 1 -- 172.21.15.38:0/4109332927 learned_addr learned my addr 172.21.15.38:0/4109332927 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18ef7fe640 1 --2- 172.21.15.38:0/4109332927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f0107900 0x7f18f01a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18e7fff640 1 --2- 172.21.15.38:0/4109332927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18f0102c00 0x7f18f0199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18eeffd640 1 -- 172.21.15.38:0/4109332927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18f0102c00 msgr2=0x7f18f0199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18eeffd640 1 --2- 172.21.15.38:0/4109332927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18f0102c00 0x7f18f0199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18eeffd640 1 -- 172.21.15.38:0/4109332927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f0107900 msgr2=0x7f18f01a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18eeffd640 1 --2- 172.21.15.38:0/4109332927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f0107900 0x7f18f01a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18eeffd640 1 -- 172.21.15.38:0/4109332927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f01a0d30 con 0x7f18f01019c0 2024-05-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18eeffd640 1 --2- 172.21.15.38:0/4109332927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f01019c0 0x7f18f01991f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f18e000d740 tx=0x7f18e000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18ecff9640 1 -- 172.21.15.38:0/4109332927 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0017070 con 0x7f18f01019c0 2024-05-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18ecff9640 1 -- 172.21.15.38:0/4109332927 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e000de50 con 0x7f18f01019c0 2024-05-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18ecff9640 1 -- 172.21.15.38:0/4109332927 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0003e00 con 0x7f18f01019c0 2024-05-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18f5565640 1 -- 172.21.15.38:0/4109332927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f01a0fc0 con 0x7f18f01019c0 2024-05-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.604+0000 7f18f5565640 1 -- 172.21.15.38:0/4109332927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f01a1430 con 0x7f18f01019c0 2024-05-21T15:34:03.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.605+0000 7f18f5565640 1 -- 172.21.15.38:0/4109332927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f0106000 con 0x7f18f01019c0 2024-05-21T15:34:03.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.606+0000 7f18ecff9640 1 -- 172.21.15.38:0/4109332927 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f18e0003fa0 con 0x7f18f01019c0 2024-05-21T15:34:03.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.607+0000 7f18ecff9640 1 --2- 172.21.15.38:0/4109332927 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f18dc0a2cb0 0x7f18dc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:03.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.607+0000 7f18e7fff640 1 --2- 172.21.15.38:0/4109332927 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f18dc0a2cb0 0x7f18dc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:03.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.607+0000 7f18ecff9640 1 -- 172.21.15.38:0/4109332927 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f18e00cea90 con 0x7f18f01019c0 2024-05-21T15:34:03.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.608+0000 7f18e7fff640 1 --2- 172.21.15.38:0/4109332927 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f18dc0a2cb0 0x7f18dc0a51c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f18f019a290 tx=0x7f18d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:03.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.612+0000 7f18ecff9640 1 -- 172.21.15.38:0/4109332927 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f18e009b9e0 con 0x7f18f01019c0 2024-05-21T15:34:03.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.798+0000 7f18f5565640 1 -- 172.21.15.38:0/4109332927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18f0062780 con 0x7f18f01019c0 2024-05-21T15:34:03.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.799+0000 7f18ecff9640 1 -- 172.21.15.38:0/4109332927 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22609 (secure 0 0 0) 0x7f18e009b2e0 con 0x7f18f01019c0 2024-05-21T15:34:03.800 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:03.800 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:33:23.367336+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:34:03.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.801+0000 7f18f5565640 1 -- 172.21.15.38:0/4109332927 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f18dc0a2cb0 msgr2=0x7f18dc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.801+0000 7f18f5565640 1 --2- 172.21.15.38:0/4109332927 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f18dc0a2cb0 0x7f18dc0a51c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f18f019a290 tx=0x7f18d8005f90 comp rx=0 tx=0).stop 2024-05-21T15:34:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.801+0000 7f18f5565640 1 -- 172.21.15.38:0/4109332927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f01019c0 msgr2=0x7f18f01991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.801+0000 7f18f5565640 1 --2- 172.21.15.38:0/4109332927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f01019c0 0x7f18f01991f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f18e000d740 tx=0x7f18e000dc00 comp rx=0 tx=0).stop 2024-05-21T15:34:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.801+0000 7f18f5565640 1 -- 172.21.15.38:0/4109332927 shutdown_connections 2024-05-21T15:34:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.801+0000 7f18f5565640 1 --2- 172.21.15.38:0/4109332927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f0107900 0x7f18f01a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.801+0000 7f18f5565640 1 --2- 172.21.15.38:0/4109332927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18f0102c00 0x7f18f0199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.801+0000 7f18f5565640 1 --2- 172.21.15.38:0/4109332927 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f18dc0a2cb0 0x7f18dc0a51c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.801+0000 7f18f5565640 1 --2- 172.21.15.38:0/4109332927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18f01019c0 0x7f18f01991f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.801+0000 7f18f5565640 1 -- 172.21.15.38:0/4109332927 >> 172.21.15.38:0/4109332927 conn(0x7f18f00fcbb0 msgr2=0x7f18f00fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.802+0000 7f18f5565640 1 -- 172.21.15.38:0/4109332927 shutdown_connections 2024-05-21T15:34:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.802+0000 7f18f5565640 1 -- 172.21.15.38:0/4109332927 wait complete. 2024-05-21T15:34:03.814 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:34:03.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.929+0000 7f75c0284640 1 -- 172.21.15.38:0/2763360221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b8108a00 msgr2=0x7f75b810ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.929+0000 7f75c0284640 1 --2- 172.21.15.38:0/2763360221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b8108a00 0x7f75b810ae20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009ca0 tx=0x7f75b002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.929+0000 7f75c0284640 1 -- 172.21.15.38:0/2763360221 shutdown_connections 2024-05-21T15:34:03.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.929+0000 7f75c0284640 1 --2- 172.21.15.38:0/2763360221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b8108a00 0x7f75b810ae20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.929+0000 7f75c0284640 1 --2- 172.21.15.38:0/2763360221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b8103d00 0x7f75b8104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.929+0000 7f75c0284640 1 --2- 172.21.15.38:0/2763360221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75b8102ac0 0x7f75b81037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.929+0000 7f75c0284640 1 -- 172.21.15.38:0/2763360221 >> 172.21.15.38:0/2763360221 conn(0x7f75b80fdcb0 msgr2=0x7f75b8100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:03.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.929+0000 7f75c0284640 1 -- 172.21.15.38:0/2763360221 shutdown_connections 2024-05-21T15:34:03.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.929+0000 7f75c0284640 1 -- 172.21.15.38:0/2763360221 wait complete. 2024-05-21T15:34:03.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75c0284640 1 Processor -- start 2024-05-21T15:34:03.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75c0284640 1 -- start start 2024-05-21T15:34:03.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75c0284640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75b8102ac0 0x7f75b8199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:03.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75c0284640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b8103d00 0x7f75b81996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:03.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75c0284640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b8108a00 0x7f75b81a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:03.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75c0284640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f75b810e740 con 0x7f75b8103d00 2024-05-21T15:34:03.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75c0284640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f75b810e5c0 con 0x7f75b8108a00 2024-05-21T15:34:03.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75c0284640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f75b810e8c0 con 0x7f75b8102ac0 2024-05-21T15:34:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75bd81f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b8103d00 0x7f75b81996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75bd81f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b8103d00 0x7f75b81996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51452/0 (socket says 172.21.15.38:51452) 2024-05-21T15:34:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75bd81f640 1 -- 172.21.15.38:0/1567685085 learned_addr learned my addr 172.21.15.38:0/1567685085 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75be821640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b8108a00 0x7f75b81a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75be020640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75b8102ac0 0x7f75b8199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75bd81f640 1 -- 172.21.15.38:0/1567685085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75b8102ac0 msgr2=0x7f75b8199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75bd81f640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75b8102ac0 0x7f75b8199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75bd81f640 1 -- 172.21.15.38:0/1567685085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b8108a00 msgr2=0x7f75b81a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75bd81f640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b8108a00 0x7f75b81a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:03.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75bd81f640 1 -- 172.21.15.38:0/1567685085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b81a0bf0 con 0x7f75b8103d00 2024-05-21T15:34:03.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.930+0000 7f75be821640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b8108a00 0x7f75b81a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:03.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.931+0000 7f75bd81f640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b8103d00 0x7f75b81996a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f75a800c920 tx=0x7f75a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:03.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.931+0000 7f75af7fe640 1 -- 172.21.15.38:0/1567685085 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8007c70 con 0x7f75b8103d00 2024-05-21T15:34:03.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.931+0000 7f75af7fe640 1 -- 172.21.15.38:0/1567685085 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a8007e10 con 0x7f75b8103d00 2024-05-21T15:34:03.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.931+0000 7f75af7fe640 1 -- 172.21.15.38:0/1567685085 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8018890 con 0x7f75b8103d00 2024-05-21T15:34:03.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.931+0000 7f75c0284640 1 -- 172.21.15.38:0/1567685085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b81a0ee0 con 0x7f75b8103d00 2024-05-21T15:34:03.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.931+0000 7f75c0284640 1 -- 172.21.15.38:0/1567685085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b81a1350 con 0x7f75b8103d00 2024-05-21T15:34:03.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.932+0000 7f75c0284640 1 -- 172.21.15.38:0/1567685085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b8107100 con 0x7f75b8103d00 2024-05-21T15:34:03.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.933+0000 7f75af7fe640 1 -- 172.21.15.38:0/1567685085 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f75a8018a50 con 0x7f75b8103d00 2024-05-21T15:34:03.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.934+0000 7f75af7fe640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f75a40a2c30 0x7f75a40a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:03.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.934+0000 7f75be020640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f75a40a2c30 0x7f75a40a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:03.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.934+0000 7f75af7fe640 1 -- 172.21.15.38:0/1567685085 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f75a80cf4f0 con 0x7f75b8103d00 2024-05-21T15:34:03.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.934+0000 7f75be020640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f75a40a2c30 0x7f75a40a5140 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f75b4005fd0 tx=0x7f75b4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:03.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:03.937+0000 7f75af7fe640 1 -- 172.21.15.38:0/1567685085 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f75a8014030 con 0x7f75b8103d00 2024-05-21T15:34:04.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.160+0000 7f75c0284640 1 -- 172.21.15.38:0/1567685085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75b802d2a0 con 0x7f75b8103d00 2024-05-21T15:34:04.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.161+0000 7f75af7fe640 1 -- 172.21.15.38:0/1567685085 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11914 (secure 0 0 0) 0x7f75a809c4d0 con 0x7f75b8103d00 2024-05-21T15:34:04.161 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:04.161 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:32:40.979368+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24550,24515,24602,24523,24551]},"id":1}]} 2024-05-21T15:34:04.161 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-05-21T15:34:04.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.163+0000 7f75c0284640 1 -- 172.21.15.38:0/1567685085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f75a40a2c30 msgr2=0x7f75a40a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.163+0000 7f75c0284640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f75a40a2c30 0x7f75a40a5140 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f75b4005fd0 tx=0x7f75b4005850 comp rx=0 tx=0).stop 2024-05-21T15:34:04.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.163+0000 7f75c0284640 1 -- 172.21.15.38:0/1567685085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b8103d00 msgr2=0x7f75b81996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.163+0000 7f75c0284640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b8103d00 0x7f75b81996a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f75a800c920 tx=0x7f75a800cde0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.163+0000 7f75c0284640 1 -- 172.21.15.38:0/1567685085 shutdown_connections 2024-05-21T15:34:04.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.163+0000 7f75c0284640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b8108a00 0x7f75b81a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.163+0000 7f75c0284640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f75a40a2c30 0x7f75a40a5140 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.163+0000 7f75c0284640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75b8103d00 0x7f75b81996a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.163+0000 7f75c0284640 1 --2- 172.21.15.38:0/1567685085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75b8102ac0 0x7f75b8199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.163+0000 7f75c0284640 1 -- 172.21.15.38:0/1567685085 >> 172.21.15.38:0/1567685085 conn(0x7f75b80fdcb0 msgr2=0x7f75b81000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.163+0000 7f75c0284640 1 -- 172.21.15.38:0/1567685085 shutdown_connections 2024-05-21T15:34:04.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.163+0000 7f75c0284640 1 -- 172.21.15.38:0/1567685085 wait complete. 2024-05-21T15:34:04.173 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:03 smithi038 ceph-mon[31995]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:34:04.173 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:03 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4139492453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:04.173 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:03 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1480270791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:04.173 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:03 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4109332927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:04.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:03 smithi045 ceph-mon[36409]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:34:04.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:03 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4139492453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:04.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:03 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1480270791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:04.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:03 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4109332927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:04.176 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-21T15:32:33.564616+0000', 'modified': '2024-05-21T15:32:40.979368+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24602, 'mds_1': 24551, 'mds_2': 24550, 'mds_3': 24523, 'mds_4': 24515}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14655': {'gid': 14655, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6839/2313367718', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6838', 'nonce': 2313367718}, {'type': 'v1', 'addr': '172.21.15.38:6839', 'nonce': 2313367718}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14667': {'gid': 14667, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6841/106491696', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6840', 'nonce': 106491696}, {'type': 'v1', 'addr': '172.21.15.38:6841', 'nonce': 106491696}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6833/1390207425', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 1390207425}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 1390207425}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6833/2551213098', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 2551213098}, {'type': 'v1', 'addr': '172.21.15.45:6833', 'nonce': 2551213098}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6835/1330413091', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 1330413091}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 1330413091}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'e', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.45:6835/3817948392', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 3817948392}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 3817948392}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.53:6837/2684842704', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 2684842704}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 2684842704}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.53:6839/3594548281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 3594548281}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 3594548281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24551': {'gid': 24551, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.45:6837/977772487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6836', 'nonce': 977772487}, {'type': 'v1', 'addr': '172.21.15.45:6837', 'nonce': 977772487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24602': {'gid': 24602, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6839/2790647513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6838', 'nonce': 2790647513}, {'type': 'v1', 'addr': '172.21.15.45:6839', 'nonce': 2790647513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24602, 'qdb_cluster': [24550, 24515, 24602, 24523, 24551]} 2024-05-21T15:34:04.176 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-21T15:34:04.176 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-05-21T15:34:04.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:03 smithi053 ceph-mon[36496]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:34:04.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:03 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4139492453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:04.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:03 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1480270791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:04.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:03 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4109332927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:04.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.292+0000 7ff52f23f640 1 -- 172.21.15.38:0/35455823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528102ae0 msgr2=0x7ff5281037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.292+0000 7ff52f23f640 1 --2- 172.21.15.38:0/35455823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528102ae0 0x7ff5281037e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009900 tx=0x7ff51c02a410 comp rx=0 tx=0).stop 2024-05-21T15:34:04.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.292+0000 7ff52f23f640 1 -- 172.21.15.38:0/35455823 shutdown_connections 2024-05-21T15:34:04.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.292+0000 7ff52f23f640 1 --2- 172.21.15.38:0/35455823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff528108a20 0x7ff52810ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.292+0000 7ff52f23f640 1 --2- 172.21.15.38:0/35455823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff528103d20 0x7ff528104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.292+0000 7ff52f23f640 1 --2- 172.21.15.38:0/35455823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528102ae0 0x7ff5281037e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.292+0000 7ff52f23f640 1 -- 172.21.15.38:0/35455823 >> 172.21.15.38:0/35455823 conn(0x7ff5280fdcf0 msgr2=0x7ff528100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.292+0000 7ff52f23f640 1 -- 172.21.15.38:0/35455823 shutdown_connections 2024-05-21T15:34:04.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.292+0000 7ff52f23f640 1 -- 172.21.15.38:0/35455823 wait complete. 2024-05-21T15:34:04.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.293+0000 7ff52f23f640 1 Processor -- start 2024-05-21T15:34:04.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.293+0000 7ff52f23f640 1 -- start start 2024-05-21T15:34:04.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.293+0000 7ff52f23f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff528102ae0 0x7ff528199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.293+0000 7ff52f23f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff528103d20 0x7ff5281997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.293+0000 7ff52f23f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528108a20 0x7ff5281a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.293+0000 7ff52f23f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff52810e730 con 0x7ff528108a20 2024-05-21T15:34:04.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.293+0000 7ff52f23f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff52810e5b0 con 0x7ff528103d20 2024-05-21T15:34:04.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.293+0000 7ff52f23f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff52810e8b0 con 0x7ff528102ae0 2024-05-21T15:34:04.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff52d7dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528108a20 0x7ff5281a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff52d7dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528108a20 0x7ff5281a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51476/0 (socket says 172.21.15.38:51476) 2024-05-21T15:34:04.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff52d7dc640 1 -- 172.21.15.38:0/2159406151 learned_addr learned my addr 172.21.15.38:0/2159406151 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff527fff640 1 --2- 172.21.15.38:0/2159406151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff528103d20 0x7ff5281997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff52cfdb640 1 --2- 172.21.15.38:0/2159406151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff528102ae0 0x7ff528199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff52d7dc640 1 -- 172.21.15.38:0/2159406151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff528102ae0 msgr2=0x7ff528199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff52d7dc640 1 --2- 172.21.15.38:0/2159406151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff528102ae0 0x7ff528199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff52d7dc640 1 -- 172.21.15.38:0/2159406151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff528103d20 msgr2=0x7ff5281997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff52d7dc640 1 --2- 172.21.15.38:0/2159406151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff528103d20 0x7ff5281997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff52d7dc640 1 -- 172.21.15.38:0/2159406151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5281a0db0 con 0x7ff528108a20 2024-05-21T15:34:04.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff52d7dc640 1 --2- 172.21.15.38:0/2159406151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528108a20 0x7ff5281a0740 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff52000eb10 tx=0x7ff52000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff525ffb640 1 -- 172.21.15.38:0/2159406151 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520004b20 con 0x7ff528108a20 2024-05-21T15:34:04.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff52f23f640 1 -- 172.21.15.38:0/2159406151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5281a10a0 con 0x7ff528108a20 2024-05-21T15:34:04.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff525ffb640 1 -- 172.21.15.38:0/2159406151 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff520004cc0 con 0x7ff528108a20 2024-05-21T15:34:04.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.294+0000 7ff525ffb640 1 -- 172.21.15.38:0/2159406151 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520010760 con 0x7ff528108a20 2024-05-21T15:34:04.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.295+0000 7ff52f23f640 1 -- 172.21.15.38:0/2159406151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5281a15f0 con 0x7ff528108a20 2024-05-21T15:34:04.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.298+0000 7ff525ffb640 1 -- 172.21.15.38:0/2159406151 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff520005020 con 0x7ff528108a20 2024-05-21T15:34:04.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.298+0000 7ff5361bd740 1 Processor -- start 2024-05-21T15:34:04.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.298+0000 7ff5361bd740 1 -- start start 2024-05-21T15:34:04.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.298+0000 7ff5361bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd9b270 0x55c8be13b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.298+0000 7ff5361bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd723b0 0x55c8be13b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.298+0000 7ff5361bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.298+0000 7ff5361bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c8be09f4d0 con 0x55c8bdd7a590 2024-05-21T15:34:04.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.298+0000 7ff5361bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c8be0473e0 con 0x55c8bdd9b270 2024-05-21T15:34:04.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.298+0000 7ff5361bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c8be09b490 con 0x55c8bdd723b0 2024-05-21T15:34:04.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.298+0000 7ff527fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.298+0000 7ff527fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be1428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51480/0 (socket says 172.21.15.38:51480) 2024-05-21T15:34:04.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.298+0000 7ff527fff640 1 -- 172.21.15.38:0/2190273658 learned_addr learned my addr 172.21.15.38:0/2190273658 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.298+0000 7ff52cfdb640 1 --2- 172.21.15.38:0/2190273658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd723b0 0x55c8be13b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.298+0000 7ff527fff640 1 --2- 172.21.15.38:0/2190273658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd9b270 0x55c8be13b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.299+0000 7ff525ffb640 1 --2- 172.21.15.38:0/2159406151 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff5180a2b90 0x7ff5180a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.299+0000 7ff52d7dc640 1 --2- 172.21.15.38:0/2159406151 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff5180a2b90 0x7ff5180a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.299+0000 7ff527fff640 1 -- 172.21.15.38:0/2190273658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd723b0 msgr2=0x55c8be13b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.299+0000 7ff527fff640 1 --2- 172.21.15.38:0/2190273658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd723b0 0x55c8be13b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.299+0000 7ff527fff640 1 -- 172.21.15.38:0/2190273658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd9b270 msgr2=0x55c8be13b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.299+0000 7ff527fff640 1 --2- 172.21.15.38:0/2190273658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd9b270 0x55c8be13b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.299+0000 7ff527fff640 1 -- 172.21.15.38:0/2190273658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8bddb32d0 con 0x55c8bdd7a590 2024-05-21T15:34:04.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.299+0000 7ff52cfdb640 1 --2- 172.21.15.38:0/2190273658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd723b0 0x55c8be13b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:34:04.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.299+0000 7ff525ffb640 1 -- 172.21.15.38:0/2159406151 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff5200cf6f0 con 0x7ff528108a20 2024-05-21T15:34:04.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.299+0000 7ff527fff640 1 --2- 172.21.15.38:0/2190273658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be1428f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff514009fd0 tx=0x7ff514004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b716d4f61b963a1a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.299+0000 7ff512ffd640 1 -- 172.21.15.38:0/2190273658 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514004030 con 0x55c8bdd7a590 2024-05-21T15:34:04.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.299+0000 7ff52d7dc640 1 --2- 172.21.15.38:0/2159406151 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff5180a2b90 0x7ff5180a50a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff52009bed0 tx=0x7ff52006f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.299+0000 7ff512ffd640 1 -- 172.21.15.38:0/2190273658 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff51400fc70 con 0x55c8bdd7a590 2024-05-21T15:34:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.299+0000 7ff512ffd640 1 -- 172.21.15.38:0/2190273658 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514003a90 con 0x55c8bdd7a590 2024-05-21T15:34:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.300+0000 7ff5361bd740 1 -- 172.21.15.38:0/2190273658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 msgr2=0x55c8be1428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.300+0000 7ff5361bd740 1 --2- 172.21.15.38:0/2190273658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be1428f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff514009fd0 tx=0x7ff514004730 comp rx=0 tx=0).stop 2024-05-21T15:34:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.300+0000 7ff5361bd740 1 -- 172.21.15.38:0/2190273658 shutdown_connections 2024-05-21T15:34:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.300+0000 7ff5361bd740 1 --2- 172.21.15.38:0/2190273658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be1428f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.300+0000 7ff5361bd740 1 --2- 172.21.15.38:0/2190273658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd723b0 0x55c8be13b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.300+0000 7ff5361bd740 1 --2- 172.21.15.38:0/2190273658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd9b270 0x55c8be13b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.300+0000 7ff5361bd740 1 -- 172.21.15.38:0/2190273658 >> 172.21.15.38:0/2190273658 conn(0x55c8bdd8f3c0 msgr2=0x55c8be136e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.300+0000 7ff5361bd740 1 -- 172.21.15.38:0/2190273658 shutdown_connections 2024-05-21T15:34:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.300+0000 7ff5361bd740 1 -- 172.21.15.38:0/2190273658 wait complete. 2024-05-21T15:34:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.300+0000 7ff5361bd740 1 -- start start 2024-05-21T15:34:04.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff5361bd740 1 Processor -- start 2024-05-21T15:34:04.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff5361bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd723b0 0x55c8be145240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff5361bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be145780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff5361bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd9b270 0x55c8be14c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff5361bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c8be143340 con 0x55c8bdd7a590 2024-05-21T15:34:04.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff5361bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c8be05a8e0 con 0x55c8bdd9b270 2024-05-21T15:34:04.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff5361bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c8be1434c0 con 0x55c8bdd723b0 2024-05-21T15:34:04.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff5361bd740 10 client.0 _ll_register_callbacks cb 0x55c8be0279c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff52cfdb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff52cfdb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be145780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51492/0 (socket says 172.21.15.38:51492) 2024-05-21T15:34:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff52cfdb640 1 -- 172.21.15.38:0/3204755983 learned_addr learned my addr 172.21.15.38:0/3204755983 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff527fff640 1 --2- 172.21.15.38:0/3204755983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd9b270 0x55c8be14c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff527fff640 1 --2- 172.21.15.38:0/3204755983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd723b0 0x55c8be145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff52cfdb640 1 -- 172.21.15.38:0/3204755983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd723b0 msgr2=0x55c8be145240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff52cfdb640 1 --2- 172.21.15.38:0/3204755983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd723b0 0x55c8be145240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff52cfdb640 1 -- 172.21.15.38:0/3204755983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd9b270 msgr2=0x55c8be14c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff52cfdb640 1 --2- 172.21.15.38:0/3204755983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd9b270 0x55c8be14c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.301+0000 7ff52cfdb640 1 -- 172.21.15.38:0/3204755983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8bde7d6b0 con 0x55c8bdd7a590 2024-05-21T15:34:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.302+0000 7ff52cfdb640 1 --2- 172.21.15.38:0/3204755983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be145780 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009230 tx=0x7ff51c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331e0181a1726baa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.302+0000 7ff507fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.302+0000 7ff507fff640 1 -- 172.21.15.38:0/3204755983 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c035040 con 0x55c8bdd7a590 2024-05-21T15:34:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.302+0000 7ff507fff640 1 -- 172.21.15.38:0/3204755983 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff51c004040 con 0x55c8bdd7a590 2024-05-21T15:34:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.302+0000 7ff5361bd740 1 -- 172.21.15.38:0/3204755983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff51400fc70 con 0x55c8bdd7a590 2024-05-21T15:34:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.302+0000 7ff507fff640 1 -- 172.21.15.38:0/3204755983 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff51c0319f0 con 0x55c8bdd7a590 2024-05-21T15:34:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.302+0000 7ff5361bd740 10 client.15087 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.302+0000 7ff5361bd740 1 -- 172.21.15.38:0/3204755983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c8be14f680 con 0x55c8bdd7a590 2024-05-21T15:34:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.302+0000 7ff507fff640 1 -- 172.21.15.38:0/3204755983 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff51400fc70 con 0x55c8bdd7a590 2024-05-21T15:34:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.302+0000 7ff507fff640 1 -- 172.21.15.38:0/3204755983 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff51c0049c0 con 0x55c8bdd7a590 2024-05-21T15:34:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.302+0000 7ff5361bd740 10 client.15087 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.303+0000 7ff5361bd740 10 client.15087 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-21T15:34:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.303+0000 7ff5361bd740 20 client.15087 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:04.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.303+0000 7ff5361bd740 1 --2- 172.21.15.38:0/3204755983 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55c8bdd59e40 0x55c8be1530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.303+0000 7ff5361bd740 4 client.15087 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:34:04.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.303+0000 7ff5361bd740 1 -- 172.21.15.38:0/3204755983 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8be0acc00 con 0x55c8bdd59e40 2024-05-21T15:34:04.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.303+0000 7ff527fff640 1 --2- 172.21.15.38:0/3204755983 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55c8bdd59e40 0x55c8be1530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.303+0000 7ff527fff640 1 --2- 172.21.15.38:0/3204755983 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55c8bdd59e40 0x55c8be1530d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fdc0d37205f44b60 server_cookie=cf6c3b2cd7948743 in_seq=0 out_seq=0 2024-05-21T15:34:04.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.303+0000 7ff507fff640 10 client.15087 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:34:04.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff507fff640 1 -- 172.21.15.38:0/3204755983 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c8be0acc00 con 0x55c8bdd59e40 2024-05-21T15:34:04.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff507fff640 10 client.15087 handle_command_reply: tid=0 2024-05-21T15:34:04.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff5361bd740 1 client.15087 shutdown 2024-05-21T15:34:04.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff5361bd740 1 -- 172.21.15.38:0/3204755983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 msgr2=0x55c8be145780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff5361bd740 1 --2- 172.21.15.38:0/3204755983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be145780 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009230 tx=0x7ff51c005f10 comp rx=0 tx=0).stop 2024-05-21T15:34:04.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff5361bd740 1 -- 172.21.15.38:0/3204755983 shutdown_connections 2024-05-21T15:34:04.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff5361bd740 1 --2- 172.21.15.38:0/3204755983 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55c8bdd59e40 0x55c8be1530d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff5361bd740 1 --2- 172.21.15.38:0/3204755983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd9b270 0x55c8be14c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff507fff640 0 client.15087 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:34:04.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff5361bd740 1 --2- 172.21.15.38:0/3204755983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be145780 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff5361bd740 1 --2- 172.21.15.38:0/3204755983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd723b0 0x55c8be145240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff5361bd740 1 -- 172.21.15.38:0/3204755983 >> 172.21.15.38:0/3204755983 conn(0x55c8bdd8f3c0 msgr2=0x55c8be135ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff5361bd740 1 -- 172.21.15.38:0/3204755983 shutdown_connections 2024-05-21T15:34:04.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff5361bd740 1 -- 172.21.15.38:0/3204755983 wait complete. 2024-05-21T15:34:04.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.304+0000 7ff5361bd740 20 client.15087 trim_cache size 0 max 16384 2024-05-21T15:34:04.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.323+0000 7ff5361bd740 1 Processor -- start 2024-05-21T15:34:04.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.323+0000 7ff5361bd740 1 -- start start 2024-05-21T15:34:04.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.323+0000 7ff5361bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd59e40 0x55c8be143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.323+0000 7ff5361bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd723b0 0x55c8be14c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.323+0000 7ff5361bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be143da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.323+0000 7ff5361bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c8be151410 con 0x55c8bdd7a590 2024-05-21T15:34:04.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.323+0000 7ff5361bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c8be151ae0 con 0x55c8bdd723b0 2024-05-21T15:34:04.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.323+0000 7ff5361bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c8be1521b0 con 0x55c8bdd59e40 2024-05-21T15:34:04.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.323+0000 7ff527fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.323+0000 7ff527fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be143da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51500/0 (socket says 172.21.15.38:51500) 2024-05-21T15:34:04.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.323+0000 7ff527fff640 1 -- 172.21.15.38:0/1722149126 learned_addr learned my addr 172.21.15.38:0/1722149126 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.323+0000 7ff52cfdb640 1 --2- 172.21.15.38:0/1722149126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd723b0 0x55c8be14c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.324+0000 7ff527fff640 1 --2- 172.21.15.38:0/1722149126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd59e40 0x55c8be143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.324+0000 7ff527fff640 1 -- 172.21.15.38:0/1722149126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd59e40 msgr2=0x55c8be143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.324+0000 7ff527fff640 1 --2- 172.21.15.38:0/1722149126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd59e40 0x55c8be143860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.324+0000 7ff527fff640 1 -- 172.21.15.38:0/1722149126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd723b0 msgr2=0x55c8be14c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.324+0000 7ff527fff640 1 --2- 172.21.15.38:0/1722149126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd723b0 0x55c8be14c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.324+0000 7ff527fff640 1 -- 172.21.15.38:0/1722149126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8be1442e0 con 0x55c8bdd7a590 2024-05-21T15:34:04.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.324+0000 7ff527fff640 1 --2- 172.21.15.38:0/1722149126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be143da0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff514005fd0 tx=0x7ff5140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f115bd59d0319372 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.324+0000 7ff5127fc640 1 -- 172.21.15.38:0/1722149126 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514011050 con 0x55c8bdd7a590 2024-05-21T15:34:04.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.324+0000 7ff5127fc640 1 -- 172.21.15.38:0/1722149126 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5140111f0 con 0x55c8bdd7a590 2024-05-21T15:34:04.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.324+0000 7ff5127fc640 1 -- 172.21.15.38:0/1722149126 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5140047a0 con 0x55c8bdd7a590 2024-05-21T15:34:04.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.325+0000 7ff5361bd740 1 -- 172.21.15.38:0/1722149126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 msgr2=0x55c8be143da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.325+0000 7ff5361bd740 1 --2- 172.21.15.38:0/1722149126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be143da0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff514005fd0 tx=0x7ff5140046c0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.325+0000 7ff5361bd740 1 -- 172.21.15.38:0/1722149126 shutdown_connections 2024-05-21T15:34:04.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.325+0000 7ff5361bd740 1 --2- 172.21.15.38:0/1722149126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd7a590 0x55c8be143da0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.325+0000 7ff5361bd740 1 --2- 172.21.15.38:0/1722149126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd723b0 0x55c8be14c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.325+0000 7ff5361bd740 1 --2- 172.21.15.38:0/1722149126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd59e40 0x55c8be143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.325+0000 7ff5361bd740 1 -- 172.21.15.38:0/1722149126 >> 172.21.15.38:0/1722149126 conn(0x55c8bdd8f3c0 msgr2=0x55c8be13f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.325+0000 7ff5361bd740 1 -- 172.21.15.38:0/1722149126 shutdown_connections 2024-05-21T15:34:04.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.325+0000 7ff5361bd740 1 -- 172.21.15.38:0/1722149126 wait complete. 2024-05-21T15:34:04.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.325+0000 7ff5361bd740 1 -- start start 2024-05-21T15:34:04.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.325+0000 7ff5361bd740 1 Processor -- start 2024-05-21T15:34:04.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff5361bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd59e40 0x55c8be147650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff5361bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd723b0 0x55c8be147b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff5361bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd7a590 0x55c8be1b9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff5361bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c8be152f50 con 0x55c8bdd59e40 2024-05-21T15:34:04.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff5361bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c8be152880 con 0x55c8bdd723b0 2024-05-21T15:34:04.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff5361bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c8be153620 con 0x55c8bdd7a590 2024-05-21T15:34:04.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff5361bd740 10 client.0 _ll_register_callbacks cb 0x7ff51c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:04.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff527fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd59e40 0x55c8be147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff527fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd59e40 0x55c8be147650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51502/0 (socket says 172.21.15.38:51502) 2024-05-21T15:34:04.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff527fff640 1 -- 172.21.15.38:0/1900619754 learned_addr learned my addr 172.21.15.38:0/1900619754 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff52cfdb640 1 --2- 172.21.15.38:0/1900619754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd723b0 0x55c8be147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff527fff640 1 --2- 172.21.15.38:0/1900619754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd7a590 0x55c8be1b9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff527fff640 1 -- 172.21.15.38:0/1900619754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd7a590 msgr2=0x55c8be1b9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff527fff640 1 --2- 172.21.15.38:0/1900619754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd7a590 0x55c8be1b9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff527fff640 1 -- 172.21.15.38:0/1900619754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd723b0 msgr2=0x55c8be147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff527fff640 1 --2- 172.21.15.38:0/1900619754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd723b0 0x55c8be147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff527fff640 1 -- 172.21.15.38:0/1900619754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8be1ba480 con 0x55c8bdd59e40 2024-05-21T15:34:04.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff527fff640 1 --2- 172.21.15.38:0/1900619754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd59e40 0x55c8be147650 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff5140114b0 tx=0x7ff51400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e03c031f883b6c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff507fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:04.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff507fff640 1 -- 172.21.15.38:0/1900619754 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514005320 con 0x55c8bdd59e40 2024-05-21T15:34:04.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff507fff640 1 -- 172.21.15.38:0/1900619754 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7ff514019040 con 0x55c8bdd59e40 2024-05-21T15:34:04.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.326+0000 7ff5361bd740 1 -- 172.21.15.38:0/1900619754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5140111f0 con 0x55c8bdd59e40 2024-05-21T15:34:04.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.327+0000 7ff507fff640 1 -- 172.21.15.38:0/1900619754 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff5140054c0 con 0x55c8bdd59e40 2024-05-21T15:34:04.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.327+0000 7ff5361bd740 10 client.15093 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:04.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.327+0000 7ff5361bd740 1 -- 172.21.15.38:0/1900619754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c8be1bb640 con 0x55c8bdd59e40 2024-05-21T15:34:04.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.327+0000 7ff507fff640 1 -- 172.21.15.38:0/1900619754 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5140111f0 con 0x55c8bdd59e40 2024-05-21T15:34:04.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.327+0000 7ff507fff640 1 -- 172.21.15.38:0/1900619754 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff514004c50 con 0x55c8bdd59e40 2024-05-21T15:34:04.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.327+0000 7ff5361bd740 10 client.15093 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:04.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.327+0000 7ff5361bd740 10 client.15093 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-21T15:34:04.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.327+0000 7ff5361bd740 20 client.15093 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:04.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.327+0000 7ff5361bd740 1 --2- 172.21.15.38:0/1900619754 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55c8bdd9b270 0x55c8be1bc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.327+0000 7ff5361bd740 4 client.15093 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:34:04.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.327+0000 7ff5361bd740 1 -- 172.21.15.38:0/1900619754 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "status"}) -- 0x55c8be0a6f50 con 0x55c8bdd9b270 2024-05-21T15:34:04.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.327+0000 7ff52cfdb640 1 --2- 172.21.15.38:0/1900619754 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55c8bdd9b270 0x55c8be1bc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff507fff640 10 client.15093 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:34:04.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff52cfdb640 1 --2- 172.21.15.38:0/1900619754 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55c8bdd9b270 0x55c8be1bc7d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da07e4221008c7f6 server_cookie=37fc969cf0169d1c in_seq=0 out_seq=0 2024-05-21T15:34:04.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff507fff640 1 -- 172.21.15.38:0/1900619754 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c8be0a6f50 con 0x55c8bdd9b270 2024-05-21T15:34:04.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff507fff640 10 client.15093 handle_command_reply: tid=0 2024-05-21T15:34:04.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff5361bd740 1 client.15093 shutdown 2024-05-21T15:34:04.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff5361bd740 1 -- 172.21.15.38:0/1900619754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd59e40 msgr2=0x55c8be147650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff5361bd740 1 --2- 172.21.15.38:0/1900619754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd59e40 0x55c8be147650 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff5140114b0 tx=0x7ff51400fb30 comp rx=0 tx=0).stop 2024-05-21T15:34:04.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff5361bd740 1 -- 172.21.15.38:0/1900619754 shutdown_connections 2024-05-21T15:34:04.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff5361bd740 1 --2- 172.21.15.38:0/1900619754 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55c8bdd9b270 0x55c8be1bc7d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff5361bd740 1 --2- 172.21.15.38:0/1900619754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8bdd7a590 0x55c8be1b9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff5361bd740 1 --2- 172.21.15.38:0/1900619754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8bdd723b0 0x55c8be147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff5361bd740 1 --2- 172.21.15.38:0/1900619754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c8bdd59e40 0x55c8be147650 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff5361bd740 1 -- 172.21.15.38:0/1900619754 >> 172.21.15.38:0/1900619754 conn(0x55c8bdd8f3c0 msgr2=0x55c8be1367f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff507fff640 0 client.15093 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:34:04.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff5361bd740 1 -- 172.21.15.38:0/1900619754 shutdown_connections 2024-05-21T15:34:04.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff5361bd740 1 -- 172.21.15.38:0/1900619754 wait complete. 2024-05-21T15:34:04.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.328+0000 7ff5361bd740 20 client.15093 trim_cache size 0 max 16384 2024-05-21T15:34:04.336 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:34:04.336 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "102ec4fa-1786-11ef-bc9a-c7b262605968", 2024-05-21T15:34:04.336 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 4, 2024-05-21T15:34:04.336 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24515, 2024-05-21T15:34:04.336 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-05-21T15:34:04.336 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-05-21T15:34:04.336 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-05-21T15:34:04.336 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 84.355164462999994, 2024-05-21T15:34:04.336 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-05-21T15:34:04.336 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-05-21T15:34:04.336 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-05-21T15:34:04.336 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 115.854478902 2024-05-21T15:34:04.337 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:34:04.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.329+0000 7ff52f23f640 1 -- 172.21.15.38:0/2159406151 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff5180a2b90 msgr2=0x7ff5180a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.329+0000 7ff52f23f640 1 --2- 172.21.15.38:0/2159406151 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff5180a2b90 0x7ff5180a50a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff52009bed0 tx=0x7ff52006f000 comp rx=0 tx=0).stop 2024-05-21T15:34:04.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.329+0000 7ff52f23f640 1 -- 172.21.15.38:0/2159406151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528108a20 msgr2=0x7ff5281a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.329+0000 7ff52f23f640 1 --2- 172.21.15.38:0/2159406151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528108a20 0x7ff5281a0740 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff52000eb10 tx=0x7ff52000c2d0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.329+0000 7ff52f23f640 1 -- 172.21.15.38:0/2159406151 shutdown_connections 2024-05-21T15:34:04.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.329+0000 7ff52f23f640 1 --2- 172.21.15.38:0/2159406151 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff5180a2b90 0x7ff5180a50a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.329+0000 7ff52f23f640 1 --2- 172.21.15.38:0/2159406151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff528108a20 0x7ff5281a0740 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.329+0000 7ff52f23f640 1 --2- 172.21.15.38:0/2159406151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff528103d20 0x7ff5281997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.329+0000 7ff52f23f640 1 --2- 172.21.15.38:0/2159406151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff528102ae0 0x7ff528199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.329+0000 7ff52f23f640 1 -- 172.21.15.38:0/2159406151 >> 172.21.15.38:0/2159406151 conn(0x7ff5280fdcf0 msgr2=0x7ff528104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.329+0000 7ff52f23f640 1 -- 172.21.15.38:0/2159406151 shutdown_connections 2024-05-21T15:34:04.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.329+0000 7ff52f23f640 1 -- 172.21.15.38:0/2159406151 wait complete. 2024-05-21T15:34:04.340 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-21T15:34:04.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.456+0000 7fb972b6b640 1 -- 172.21.15.38:0/2325007217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb96c102bc0 msgr2=0x7fb96c102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.456+0000 7fb972b6b640 1 --2- 172.21.15.38:0/2325007217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb96c102bc0 0x7fb96c102fd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009900 tx=0x7fb95c02a410 comp rx=0 tx=0).stop 2024-05-21T15:34:04.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.456+0000 7fb972b6b640 1 -- 172.21.15.38:0/2325007217 shutdown_connections 2024-05-21T15:34:04.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.456+0000 7fb972b6b640 1 --2- 172.21.15.38:0/2325007217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb96c1078c0 0x7fb96c109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.456+0000 7fb972b6b640 1 --2- 172.21.15.38:0/2325007217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb96c102bc0 0x7fb96c102fd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.456+0000 7fb972b6b640 1 --2- 172.21.15.38:0/2325007217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb96c101980 0x7fb96c102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.456+0000 7fb972b6b640 1 -- 172.21.15.38:0/2325007217 >> 172.21.15.38:0/2325007217 conn(0x7fb96c0fcb70 msgr2=0x7fb96c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.456+0000 7fb972b6b640 1 -- 172.21.15.38:0/2325007217 shutdown_connections 2024-05-21T15:34:04.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.456+0000 7fb972b6b640 1 -- 172.21.15.38:0/2325007217 wait complete. 2024-05-21T15:34:04.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb972b6b640 1 Processor -- start 2024-05-21T15:34:04.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb972b6b640 1 -- start start 2024-05-21T15:34:04.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb972b6b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb96c101980 0x7fb96c069a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb972b6b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb96c102bc0 0x7fb96c19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb972b6b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb96c1078c0 0x7fb96c1a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb972b6b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c10d600 con 0x7fb96c1078c0 2024-05-21T15:34:04.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb972b6b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c10d480 con 0x7fb96c101980 2024-05-21T15:34:04.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb972b6b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c10d780 con 0x7fb96c102bc0 2024-05-21T15:34:04.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb971108640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb96c1078c0 0x7fb96c1a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb971108640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb96c1078c0 0x7fb96c1a0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51528/0 (socket says 172.21.15.38:51528) 2024-05-21T15:34:04.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb971108640 1 -- 172.21.15.38:0/3796412689 learned_addr learned my addr 172.21.15.38:0/3796412689 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb970907640 1 --2- 172.21.15.38:0/3796412689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb96c101980 0x7fb96c069a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb96bfff640 1 --2- 172.21.15.38:0/3796412689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb96c102bc0 0x7fb96c19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb971108640 1 -- 172.21.15.38:0/3796412689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb96c102bc0 msgr2=0x7fb96c19dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb971108640 1 --2- 172.21.15.38:0/3796412689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb96c102bc0 0x7fb96c19dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb971108640 1 -- 172.21.15.38:0/3796412689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb96c101980 msgr2=0x7fb96c069a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb971108640 1 --2- 172.21.15.38:0/3796412689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb96c101980 0x7fb96c069a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb971108640 1 -- 172.21.15.38:0/3796412689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c1a0d60 con 0x7fb96c1078c0 2024-05-21T15:34:04.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.457+0000 7fb971108640 1 --2- 172.21.15.38:0/3796412689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb96c1078c0 0x7fb96c1a0540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb96400eb10 tx=0x7fb96400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.458+0000 7fb969ffb640 1 -- 172.21.15.38:0/3796412689 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964004b20 con 0x7fb96c1078c0 2024-05-21T15:34:04.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.458+0000 7fb969ffb640 1 -- 172.21.15.38:0/3796412689 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb964004cc0 con 0x7fb96c1078c0 2024-05-21T15:34:04.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.458+0000 7fb972b6b640 1 -- 172.21.15.38:0/3796412689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96c1a0ff0 con 0x7fb96c1078c0 2024-05-21T15:34:04.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.458+0000 7fb969ffb640 1 -- 172.21.15.38:0/3796412689 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964010760 con 0x7fb96c1078c0 2024-05-21T15:34:04.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.458+0000 7fb972b6b640 1 -- 172.21.15.38:0/3796412689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96c1a14b0 con 0x7fb96c1078c0 2024-05-21T15:34:04.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.460+0000 7fb979a0c740 1 Processor -- start 2024-05-21T15:34:04.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.460+0000 7fb969ffb640 1 -- 172.21.15.38:0/3796412689 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb964005020 con 0x7fb96c1078c0 2024-05-21T15:34:04.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.460+0000 7fb979a0c740 1 -- start start 2024-05-21T15:34:04.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb979a0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a77a6270 0x5652a7b46470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb979a0c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a777d3b0 0x5652a7b469f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb979a0c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a7785590 0x5652a7b4d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb979a0c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5652a7aaa4d0 con 0x5652a777d3b0 2024-05-21T15:34:04.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb979a0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5652a7a523e0 con 0x5652a77a6270 2024-05-21T15:34:04.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb979a0c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5652a7aa6490 con 0x5652a7785590 2024-05-21T15:34:04.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb970907640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a777d3b0 0x5652a7b469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb970907640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a777d3b0 0x5652a7b469f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51538/0 (socket says 172.21.15.38:51538) 2024-05-21T15:34:04.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb970907640 1 -- 172.21.15.38:0/893618149 learned_addr learned my addr 172.21.15.38:0/893618149 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb96bfff640 1 --2- 172.21.15.38:0/893618149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a7785590 0x5652a7b4d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb96bfff640 1 --2- 172.21.15.38:0/893618149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a77a6270 0x5652a7b46470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb970907640 1 -- 172.21.15.38:0/893618149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a7785590 msgr2=0x5652a7b4d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb970907640 1 --2- 172.21.15.38:0/893618149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a7785590 0x5652a7b4d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb970907640 1 -- 172.21.15.38:0/893618149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a77a6270 msgr2=0x5652a7b46470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb970907640 1 --2- 172.21.15.38:0/893618149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a77a6270 0x5652a7b46470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb970907640 1 -- 172.21.15.38:0/893618149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652a77be2d0 con 0x5652a777d3b0 2024-05-21T15:34:04.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb96bfff640 1 --2- 172.21.15.38:0/893618149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a77a6270 0x5652a7b46470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:04.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.461+0000 7fb969ffb640 1 --2- 172.21.15.38:0/3796412689 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9580a2b90 0x7fb9580a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.462+0000 7fb970907640 1 --2- 172.21.15.38:0/893618149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a777d3b0 0x5652a7b469f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb954009c60 tx=0x7fb954011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27e47c0f73be931 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.462+0000 7fb96bfff640 1 --2- 172.21.15.38:0/3796412689 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9580a2b90 0x7fb9580a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.462+0000 7fb969ffb640 1 -- 172.21.15.38:0/3796412689 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb9640cf6f0 con 0x7fb96c1078c0 2024-05-21T15:34:04.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.462+0000 7fb962ffd640 1 -- 172.21.15.38:0/893618149 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954004220 con 0x5652a777d3b0 2024-05-21T15:34:04.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.462+0000 7fb962ffd640 1 -- 172.21.15.38:0/893618149 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9540043c0 con 0x5652a777d3b0 2024-05-21T15:34:04.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.462+0000 7fb962ffd640 1 -- 172.21.15.38:0/893618149 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954004e40 con 0x5652a777d3b0 2024-05-21T15:34:04.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.463+0000 7fb979a0c740 1 -- 172.21.15.38:0/893618149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a777d3b0 msgr2=0x5652a7b469f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.463+0000 7fb96bfff640 1 --2- 172.21.15.38:0/3796412689 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9580a2b90 0x7fb9580a50a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009230 tx=0x7fb95c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.463+0000 7fb979a0c740 1 --2- 172.21.15.38:0/893618149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a777d3b0 0x5652a7b469f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb954009c60 tx=0x7fb954011810 comp rx=0 tx=0).stop 2024-05-21T15:34:04.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.463+0000 7fb979a0c740 1 -- 172.21.15.38:0/893618149 shutdown_connections 2024-05-21T15:34:04.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.463+0000 7fb979a0c740 1 --2- 172.21.15.38:0/893618149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a7785590 0x5652a7b4d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.463+0000 7fb979a0c740 1 --2- 172.21.15.38:0/893618149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a777d3b0 0x5652a7b469f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.463+0000 7fb979a0c740 1 --2- 172.21.15.38:0/893618149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a77a6270 0x5652a7b46470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.463+0000 7fb979a0c740 1 -- 172.21.15.38:0/893618149 >> 172.21.15.38:0/893618149 conn(0x5652a779a3c0 msgr2=0x5652a7b41e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.463+0000 7fb979a0c740 1 -- 172.21.15.38:0/893618149 shutdown_connections 2024-05-21T15:34:04.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.463+0000 7fb979a0c740 1 -- 172.21.15.38:0/893618149 wait complete. 2024-05-21T15:34:04.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.463+0000 7fb979a0c740 1 -- start start 2024-05-21T15:34:04.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.463+0000 7fb979a0c740 1 Processor -- start 2024-05-21T15:34:04.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb979a0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 0x5652a7b50240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb979a0c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a7785590 0x5652a7b50780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb979a0c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a77a6270 0x5652a7b576e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb979a0c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5652a7b4e340 con 0x5652a7785590 2024-05-21T15:34:04.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb979a0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5652a7a658e0 con 0x5652a777d3b0 2024-05-21T15:34:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb979a0c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5652a7b4e4c0 con 0x5652a77a6270 2024-05-21T15:34:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb979a0c740 10 client.0 _ll_register_callbacks cb 0x5652a7a329c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb96bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a7785590 0x5652a7b50780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb96bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a7785590 0x5652a7b50780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51540/0 (socket says 172.21.15.38:51540) 2024-05-21T15:34:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb96bfff640 1 -- 172.21.15.38:0/3947979015 learned_addr learned my addr 172.21.15.38:0/3947979015 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb970907640 1 --2- 172.21.15.38:0/3947979015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 0x5652a7b50240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb971108640 1 --2- 172.21.15.38:0/3947979015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a77a6270 0x5652a7b576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb96bfff640 1 -- 172.21.15.38:0/3947979015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a77a6270 msgr2=0x5652a7b576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb96bfff640 1 --2- 172.21.15.38:0/3947979015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a77a6270 0x5652a7b576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb96bfff640 1 -- 172.21.15.38:0/3947979015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 msgr2=0x5652a7b50240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb96bfff640 1 --2- 172.21.15.38:0/3947979015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 0x5652a7b50240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb96bfff640 1 -- 172.21.15.38:0/3947979015 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652a78886b0 con 0x5652a7785590 2024-05-21T15:34:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb96bfff640 1 --2- 172.21.15.38:0/3947979015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a7785590 0x5652a7b50780 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb95c033000 tx=0x7fb95c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d02601999f75b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb94bfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.464+0000 7fb94bfff640 1 -- 172.21.15.38:0/3947979015 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c04e070 con 0x5652a7785590 2024-05-21T15:34:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.465+0000 7fb94bfff640 1 -- 172.21.15.38:0/3947979015 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb95c004470 con 0x5652a7785590 2024-05-21T15:34:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.465+0000 7fb979a0c740 1 -- 172.21.15.38:0/3947979015 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9540043c0 con 0x5652a7785590 2024-05-21T15:34:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.465+0000 7fb94bfff640 1 -- 172.21.15.38:0/3947979015 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb95c035040 con 0x5652a7785590 2024-05-21T15:34:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.465+0000 7fb979a0c740 10 client.15105 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.465+0000 7fb979a0c740 1 -- 172.21.15.38:0/3947979015 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5652a7b5a680 con 0x5652a7785590 2024-05-21T15:34:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.465+0000 7fb94bfff640 1 -- 172.21.15.38:0/3947979015 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9540043c0 con 0x5652a7785590 2024-05-21T15:34:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.465+0000 7fb94bfff640 1 -- 172.21.15.38:0/3947979015 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb95c0035f0 con 0x5652a7785590 2024-05-21T15:34:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.465+0000 7fb979a0c740 10 client.15105 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.465+0000 7fb979a0c740 10 client.15105 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T15:34:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.465+0000 7fb979a0c740 20 client.15105 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.465+0000 7fb979a0c740 1 --2- 172.21.15.38:0/3947979015 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5652a7764e40 0x5652a7b5e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.465+0000 7fb979a0c740 4 client.15105 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:34:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.465+0000 7fb979a0c740 1 -- 172.21.15.38:0/3947979015 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652a7ab7c00 con 0x5652a7764e40 2024-05-21T15:34:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.466+0000 7fb970907640 1 --2- 172.21.15.38:0/3947979015 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5652a7764e40 0x5652a7b5e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.466+0000 7fb970907640 1 --2- 172.21.15.38:0/3947979015 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5652a7764e40 0x5652a7b5e150 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c0819410f43c9438 server_cookie=afb7d545dd4c4d8 in_seq=0 out_seq=0 2024-05-21T15:34:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.466+0000 7fb94bfff640 10 client.15105 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:34:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb94bfff640 1 -- 172.21.15.38:0/3947979015 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5652a7ab7c00 con 0x5652a7764e40 2024-05-21T15:34:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb94bfff640 10 client.15105 handle_command_reply: tid=0 2024-05-21T15:34:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb979a0c740 1 client.15105 shutdown 2024-05-21T15:34:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb979a0c740 1 -- 172.21.15.38:0/3947979015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a7785590 msgr2=0x5652a7b50780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb979a0c740 1 --2- 172.21.15.38:0/3947979015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a7785590 0x5652a7b50780 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb95c033000 tx=0x7fb95c02a410 comp rx=0 tx=0).stop 2024-05-21T15:34:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb979a0c740 1 -- 172.21.15.38:0/3947979015 shutdown_connections 2024-05-21T15:34:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb979a0c740 1 --2- 172.21.15.38:0/3947979015 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5652a7764e40 0x5652a7b5e150 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb979a0c740 1 --2- 172.21.15.38:0/3947979015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a77a6270 0x5652a7b576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb94bfff640 0 client.15105 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:34:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb979a0c740 1 --2- 172.21.15.38:0/3947979015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a7785590 0x5652a7b50780 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb979a0c740 1 --2- 172.21.15.38:0/3947979015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 0x5652a7b50240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb979a0c740 1 -- 172.21.15.38:0/3947979015 >> 172.21.15.38:0/3947979015 conn(0x5652a779a3c0 msgr2=0x5652a7b40ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb979a0c740 1 -- 172.21.15.38:0/3947979015 shutdown_connections 2024-05-21T15:34:04.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb979a0c740 1 -- 172.21.15.38:0/3947979015 wait complete. 2024-05-21T15:34:04.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.467+0000 7fb979a0c740 20 client.15105 trim_cache size 0 max 16384 2024-05-21T15:34:04.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.486+0000 7fb979a0c740 1 Processor -- start 2024-05-21T15:34:04.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.486+0000 7fb979a0c740 1 -- start start 2024-05-21T15:34:04.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.486+0000 7fb979a0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 0x5652a7b4fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.486+0000 7fb979a0c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a7785590 0x5652a7b50350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.486+0000 7fb979a0c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a77a6270 0x5652a7b50890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.486+0000 7fb979a0c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5652a7b5c490 con 0x5652a7785590 2024-05-21T15:34:04.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.486+0000 7fb979a0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5652a7b5cb60 con 0x5652a777d3b0 2024-05-21T15:34:04.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.486+0000 7fb979a0c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5652a7b5d230 con 0x5652a77a6270 2024-05-21T15:34:04.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.486+0000 7fb96bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a7785590 0x5652a7b50350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.486+0000 7fb96bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a7785590 0x5652a7b50350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51552/0 (socket says 172.21.15.38:51552) 2024-05-21T15:34:04.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.486+0000 7fb96bfff640 1 -- 172.21.15.38:0/444879230 learned_addr learned my addr 172.21.15.38:0/444879230 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.486+0000 7fb970907640 1 --2- 172.21.15.38:0/444879230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 0x5652a7b4fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.486+0000 7fb971108640 1 --2- 172.21.15.38:0/444879230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a77a6270 0x5652a7b50890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.487+0000 7fb96bfff640 1 -- 172.21.15.38:0/444879230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a77a6270 msgr2=0x5652a7b50890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.487+0000 7fb96bfff640 1 --2- 172.21.15.38:0/444879230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a77a6270 0x5652a7b50890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.487+0000 7fb96bfff640 1 -- 172.21.15.38:0/444879230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 msgr2=0x5652a7b4fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.487+0000 7fb96bfff640 1 --2- 172.21.15.38:0/444879230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 0x5652a7b4fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.487+0000 7fb96bfff640 1 -- 172.21.15.38:0/444879230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652a7b46020 con 0x5652a7785590 2024-05-21T15:34:04.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.487+0000 7fb970907640 1 --2- 172.21.15.38:0/444879230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 0x5652a7b4fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:04.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.487+0000 7fb96bfff640 1 --2- 172.21.15.38:0/444879230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a7785590 0x5652a7b50350 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb95c02a930 tx=0x7fb95c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56404e0bd688128a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.487+0000 7fb9627fc640 1 -- 172.21.15.38:0/444879230 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c04e040 con 0x5652a7785590 2024-05-21T15:34:04.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.487+0000 7fb9627fc640 1 -- 172.21.15.38:0/444879230 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb95c005510 con 0x5652a7785590 2024-05-21T15:34:04.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.487+0000 7fb9627fc640 1 -- 172.21.15.38:0/444879230 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c0049c0 con 0x5652a7785590 2024-05-21T15:34:04.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.488+0000 7fb979a0c740 1 -- 172.21.15.38:0/444879230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a7785590 msgr2=0x5652a7b50350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.488+0000 7fb979a0c740 1 --2- 172.21.15.38:0/444879230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a7785590 0x5652a7b50350 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb95c02a930 tx=0x7fb95c005d50 comp rx=0 tx=0).stop 2024-05-21T15:34:04.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.488+0000 7fb979a0c740 1 -- 172.21.15.38:0/444879230 shutdown_connections 2024-05-21T15:34:04.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.488+0000 7fb979a0c740 1 --2- 172.21.15.38:0/444879230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a77a6270 0x5652a7b50890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.488+0000 7fb979a0c740 1 --2- 172.21.15.38:0/444879230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a7785590 0x5652a7b50350 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.488+0000 7fb979a0c740 1 --2- 172.21.15.38:0/444879230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 0x5652a7b4fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.488+0000 7fb979a0c740 1 -- 172.21.15.38:0/444879230 >> 172.21.15.38:0/444879230 conn(0x5652a779a3c0 msgr2=0x5652a7b4a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.488+0000 7fb979a0c740 1 -- 172.21.15.38:0/444879230 shutdown_connections 2024-05-21T15:34:04.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.488+0000 7fb979a0c740 1 -- 172.21.15.38:0/444879230 wait complete. 2024-05-21T15:34:04.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.488+0000 7fb979a0c740 1 -- start start 2024-05-21T15:34:04.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.488+0000 7fb979a0c740 1 Processor -- start 2024-05-21T15:34:04.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb979a0c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 0x5652a7b52f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb979a0c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a7785590 0x5652a7b534a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb979a0c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a77a6270 0x5652a7bc4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb979a0c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5652a7b5dfd0 con 0x5652a77a6270 2024-05-21T15:34:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb979a0c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5652a7b5d900 con 0x5652a777d3b0 2024-05-21T15:34:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb979a0c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5652a7b5e6a0 con 0x5652a7785590 2024-05-21T15:34:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb979a0c740 10 client.0 _ll_register_callbacks cb 0x7fb95c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb971108640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a77a6270 0x5652a7bc4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb971108640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a77a6270 0x5652a7bc4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51554/0 (socket says 172.21.15.38:51554) 2024-05-21T15:34:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb971108640 1 -- 172.21.15.38:0/2504874260 learned_addr learned my addr 172.21.15.38:0/2504874260 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb970907640 1 --2- 172.21.15.38:0/2504874260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 0x5652a7b52f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb96bfff640 1 --2- 172.21.15.38:0/2504874260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a7785590 0x5652a7b534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb971108640 1 -- 172.21.15.38:0/2504874260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a7785590 msgr2=0x5652a7b534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb971108640 1 --2- 172.21.15.38:0/2504874260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a7785590 0x5652a7b534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb971108640 1 -- 172.21.15.38:0/2504874260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 msgr2=0x5652a7b52f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb971108640 1 --2- 172.21.15.38:0/2504874260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 0x5652a7b52f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb971108640 1 -- 172.21.15.38:0/2504874260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652a7bc5580 con 0x5652a77a6270 2024-05-21T15:34:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb971108640 1 --2- 172.21.15.38:0/2504874260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a77a6270 0x5652a7bc4fc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb9640983a0 tx=0x7fb96409ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850df7051b734a78 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb94bfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:04.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb94bfff640 1 -- 172.21.15.38:0/2504874260 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9640bad80 con 0x5652a77a6270 2024-05-21T15:34:04.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb94bfff640 1 -- 172.21.15.38:0/2504874260 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb964005310 con 0x5652a77a6270 2024-05-21T15:34:04.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.489+0000 7fb979a0c740 1 -- 172.21.15.38:0/2504874260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb95c005510 con 0x5652a77a6270 2024-05-21T15:34:04.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.490+0000 7fb94bfff640 1 -- 172.21.15.38:0/2504874260 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb96409bf40 con 0x5652a77a6270 2024-05-21T15:34:04.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.490+0000 7fb979a0c740 10 client.15111 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:04.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.490+0000 7fb979a0c740 1 -- 172.21.15.38:0/2504874260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5652a7bc6870 con 0x5652a77a6270 2024-05-21T15:34:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.490+0000 7fb94bfff640 1 -- 172.21.15.38:0/2504874260 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb95c005510 con 0x5652a77a6270 2024-05-21T15:34:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.490+0000 7fb94bfff640 1 -- 172.21.15.38:0/2504874260 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb96409b4b0 con 0x5652a77a6270 2024-05-21T15:34:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.490+0000 7fb979a0c740 10 client.15111 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.490+0000 7fb979a0c740 10 client.15111 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T15:34:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.490+0000 7fb979a0c740 20 client.15111 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.490+0000 7fb979a0c740 1 --2- 172.21.15.38:0/2504874260 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5652a773a280 0x5652a7bca150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.490+0000 7fb979a0c740 4 client.15111 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:34:04.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.490+0000 7fb979a0c740 1 -- 172.21.15.38:0/2504874260 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "status"}) -- 0x5652a7ab1f50 con 0x5652a773a280 2024-05-21T15:34:04.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.490+0000 7fb970907640 1 --2- 172.21.15.38:0/2504874260 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5652a773a280 0x5652a7bca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb970907640 1 --2- 172.21.15.38:0/2504874260 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5652a773a280 0x5652a7bca150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=79fda74a0feefec4 server_cookie=c771b96858c32d21 in_seq=0 out_seq=0 2024-05-21T15:34:04.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb94bfff640 10 client.15111 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:34:04.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb94bfff640 1 -- 172.21.15.38:0/2504874260 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5652a7ab1f50 con 0x5652a773a280 2024-05-21T15:34:04.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb94bfff640 10 client.15111 handle_command_reply: tid=0 2024-05-21T15:34:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb979a0c740 1 client.15111 shutdown 2024-05-21T15:34:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb979a0c740 1 -- 172.21.15.38:0/2504874260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a77a6270 msgr2=0x5652a7bc4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb979a0c740 1 --2- 172.21.15.38:0/2504874260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a77a6270 0x5652a7bc4fc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb9640983a0 tx=0x7fb96409ba80 comp rx=0 tx=0).stop 2024-05-21T15:34:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb979a0c740 1 -- 172.21.15.38:0/2504874260 shutdown_connections 2024-05-21T15:34:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb979a0c740 1 --2- 172.21.15.38:0/2504874260 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5652a773a280 0x5652a7bca150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb979a0c740 1 --2- 172.21.15.38:0/2504874260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5652a77a6270 0x5652a7bc4fc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb94bfff640 0 client.15111 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:34:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb979a0c740 1 --2- 172.21.15.38:0/2504874260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652a7785590 0x5652a7b534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb979a0c740 1 --2- 172.21.15.38:0/2504874260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652a777d3b0 0x5652a7b52f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb979a0c740 1 -- 172.21.15.38:0/2504874260 >> 172.21.15.38:0/2504874260 conn(0x5652a779a3c0 msgr2=0x5652a7b4fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb979a0c740 1 -- 172.21.15.38:0/2504874260 shutdown_connections 2024-05-21T15:34:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb979a0c740 1 -- 172.21.15.38:0/2504874260 wait complete. 2024-05-21T15:34:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.491+0000 7fb979a0c740 20 client.15111 trim_cache size 0 max 16384 2024-05-21T15:34:04.512 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:34:04.512 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "102ec4fa-1786-11ef-bc9a-c7b262605968", 2024-05-21T15:34:04.512 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 3, 2024-05-21T15:34:04.512 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24523, 2024-05-21T15:34:04.512 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-05-21T15:34:04.512 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-05-21T15:34:04.512 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-05-21T15:34:04.512 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 85.527754895000001, 2024-05-21T15:34:04.512 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-05-21T15:34:04.512 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-05-21T15:34:04.512 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-05-21T15:34:04.512 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 110.09175441399999 2024-05-21T15:34:04.512 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:34:04.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.492+0000 7fb972b6b640 1 -- 172.21.15.38:0/3796412689 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9580a2b90 msgr2=0x7fb9580a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.492+0000 7fb972b6b640 1 --2- 172.21.15.38:0/3796412689 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9580a2b90 0x7fb9580a50a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009230 tx=0x7fb95c031400 comp rx=0 tx=0).stop 2024-05-21T15:34:04.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.492+0000 7fb972b6b640 1 -- 172.21.15.38:0/3796412689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb96c1078c0 msgr2=0x7fb96c1a0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.492+0000 7fb972b6b640 1 --2- 172.21.15.38:0/3796412689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb96c1078c0 0x7fb96c1a0540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb96400eb10 tx=0x7fb96400c2d0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.492+0000 7fb972b6b640 1 -- 172.21.15.38:0/3796412689 shutdown_connections 2024-05-21T15:34:04.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.492+0000 7fb972b6b640 1 --2- 172.21.15.38:0/3796412689 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9580a2b90 0x7fb9580a50a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.492+0000 7fb972b6b640 1 --2- 172.21.15.38:0/3796412689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb96c1078c0 0x7fb96c1a0540 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.492+0000 7fb972b6b640 1 --2- 172.21.15.38:0/3796412689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb96c102bc0 0x7fb96c19dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.492+0000 7fb972b6b640 1 --2- 172.21.15.38:0/3796412689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb96c101980 0x7fb96c069a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.492+0000 7fb972b6b640 1 -- 172.21.15.38:0/3796412689 >> 172.21.15.38:0/3796412689 conn(0x7fb96c0fcb70 msgr2=0x7fb96c1038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.492+0000 7fb972b6b640 1 -- 172.21.15.38:0/3796412689 shutdown_connections 2024-05-21T15:34:04.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.492+0000 7fb972b6b640 1 -- 172.21.15.38:0/3796412689 wait complete. 2024-05-21T15:34:04.515 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-21T15:34:04.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.631+0000 7fe798172640 1 -- 172.21.15.38:0/2257615919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe790102bf0 msgr2=0x7fe790103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.631+0000 7fe798172640 1 --2- 172.21.15.38:0/2257615919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe790102bf0 0x7fe790103000 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe780009900 tx=0x7fe78002a410 comp rx=0 tx=0).stop 2024-05-21T15:34:04.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.631+0000 7fe798172640 1 -- 172.21.15.38:0/2257615919 shutdown_connections 2024-05-21T15:34:04.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.631+0000 7fe798172640 1 --2- 172.21.15.38:0/2257615919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901078f0 0x7fe790109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.631+0000 7fe798172640 1 --2- 172.21.15.38:0/2257615919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe790102bf0 0x7fe790103000 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.631+0000 7fe798172640 1 --2- 172.21.15.38:0/2257615919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7901019b0 0x7fe7901026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.631+0000 7fe798172640 1 -- 172.21.15.38:0/2257615919 >> 172.21.15.38:0/2257615919 conn(0x7fe7900fcbe0 msgr2=0x7fe7900ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.631+0000 7fe798172640 1 -- 172.21.15.38:0/2257615919 shutdown_connections 2024-05-21T15:34:04.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.631+0000 7fe798172640 1 -- 172.21.15.38:0/2257615919 wait complete. 2024-05-21T15:34:04.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe798172640 1 Processor -- start 2024-05-21T15:34:04.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe798172640 1 -- start start 2024-05-21T15:34:04.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe798172640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7901019b0 0x7fe7901981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe798172640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe790102bf0 0x7fe790198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe798172640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901078f0 0x7fe79019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe798172640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe79010d620 con 0x7fe7901019b0 2024-05-21T15:34:04.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe798172640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe79010d4a0 con 0x7fe7901078f0 2024-05-21T15:34:04.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe798172640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe79010d7a0 con 0x7fe790102bf0 2024-05-21T15:34:04.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe795f0e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7901019b0 0x7fe7901981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe795f0e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7901019b0 0x7fe7901981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51578/0 (socket says 172.21.15.38:51578) 2024-05-21T15:34:04.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe795f0e640 1 -- 172.21.15.38:0/1244238673 learned_addr learned my addr 172.21.15.38:0/1244238673 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe79670f640 1 --2- 172.21.15.38:0/1244238673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901078f0 0x7fe79019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe79570d640 1 --2- 172.21.15.38:0/1244238673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe790102bf0 0x7fe790198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe795f0e640 1 -- 172.21.15.38:0/1244238673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe790102bf0 msgr2=0x7fe790198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.632+0000 7fe795f0e640 1 --2- 172.21.15.38:0/1244238673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe790102bf0 0x7fe790198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.633+0000 7fe795f0e640 1 -- 172.21.15.38:0/1244238673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901078f0 msgr2=0x7fe79019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.633+0000 7fe795f0e640 1 --2- 172.21.15.38:0/1244238673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901078f0 0x7fe79019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.633+0000 7fe795f0e640 1 -- 172.21.15.38:0/1244238673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79019fcc0 con 0x7fe7901019b0 2024-05-21T15:34:04.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.633+0000 7fe795f0e640 1 --2- 172.21.15.38:0/1244238673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7901019b0 0x7fe7901981f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00e9f0 tx=0x7fe78c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.633+0000 7fe786ffd640 1 -- 172.21.15.38:0/1244238673 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00ccc0 con 0x7fe7901019b0 2024-05-21T15:34:04.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.633+0000 7fe786ffd640 1 -- 172.21.15.38:0/1244238673 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe78c0035f0 con 0x7fe7901019b0 2024-05-21T15:34:04.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.633+0000 7fe798172640 1 -- 172.21.15.38:0/1244238673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79019ffb0 con 0x7fe7901019b0 2024-05-21T15:34:04.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.633+0000 7fe786ffd640 1 -- 172.21.15.38:0/1244238673 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00ce60 con 0x7fe7901019b0 2024-05-21T15:34:04.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.633+0000 7fe798172640 1 -- 172.21.15.38:0/1244238673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7901a03c0 con 0x7fe7901019b0 2024-05-21T15:34:04.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe786ffd640 1 -- 172.21.15.38:0/1244238673 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe78c0037a0 con 0x7fe7901019b0 2024-05-21T15:34:04.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe79f017740 1 Processor -- start 2024-05-21T15:34:04.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe79f017740 1 -- start start 2024-05-21T15:34:04.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe79f017740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ef8270 0x558dc6298470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe79f017740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ecf3b0 0x558dc62989f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe79f017740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ed7590 0x558dc629f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe79f017740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558dc61fc4d0 con 0x558dc5ed7590 2024-05-21T15:34:04.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe79f017740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558dc61a43e0 con 0x558dc5ef8270 2024-05-21T15:34:04.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe79f017740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558dc61f8490 con 0x558dc5ecf3b0 2024-05-21T15:34:04.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe79570d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ed7590 0x558dc629f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe79670f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ef8270 0x558dc6298470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe79570d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ed7590 0x558dc629f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51586/0 (socket says 172.21.15.38:51586) 2024-05-21T15:34:04.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe79670f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ef8270 0x558dc6298470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:60584/0 (socket says 172.21.15.38:60584) 2024-05-21T15:34:04.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe79570d640 1 -- 172.21.15.38:0/2798205170 learned_addr learned my addr 172.21.15.38:0/2798205170 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.636+0000 7fe795f0e640 1 --2- 172.21.15.38:0/2798205170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ecf3b0 0x558dc62989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.637+0000 7fe79670f640 1 -- 172.21.15.38:0/2798205170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ecf3b0 msgr2=0x558dc62989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.637+0000 7fe79670f640 1 --2- 172.21.15.38:0/2798205170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ecf3b0 0x558dc62989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.637+0000 7fe79670f640 1 -- 172.21.15.38:0/2798205170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ed7590 msgr2=0x558dc629f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.637+0000 7fe79670f640 1 --2- 172.21.15.38:0/2798205170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ed7590 0x558dc629f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.637+0000 7fe79670f640 1 -- 172.21.15.38:0/2798205170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc5f102d0 con 0x558dc5ef8270 2024-05-21T15:34:04.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.637+0000 7fe786ffd640 1 --2- 172.21.15.38:0/1244238673 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe77c0a2b90 0x7fe77c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.637+0000 7fe79670f640 1 --2- 172.21.15.38:0/2798205170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ef8270 0x558dc6298470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe788009f00 tx=0x7fe788011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae60375524233da0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.637+0000 7fe795f0e640 1 --2- 172.21.15.38:0/1244238673 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe77c0a2b90 0x7fe77c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.637+0000 7fe786ffd640 1 -- 172.21.15.38:0/1244238673 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fe78c0cf600 con 0x7fe7901019b0 2024-05-21T15:34:04.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.637+0000 7fe77bfff640 1 -- 172.21.15.38:0/2798205170 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788004d20 con 0x558dc5ef8270 2024-05-21T15:34:04.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.637+0000 7fe77bfff640 1 -- 172.21.15.38:0/2798205170 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe788004030 con 0x558dc5ef8270 2024-05-21T15:34:04.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.637+0000 7fe77bfff640 1 -- 172.21.15.38:0/2798205170 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788005280 con 0x558dc5ef8270 2024-05-21T15:34:04.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.638+0000 7fe79f017740 1 -- 172.21.15.38:0/2798205170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ef8270 msgr2=0x558dc6298470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.638+0000 7fe79f017740 1 --2- 172.21.15.38:0/2798205170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ef8270 0x558dc6298470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe788009f00 tx=0x7fe788011810 comp rx=0 tx=0).stop 2024-05-21T15:34:04.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.638+0000 7fe795f0e640 1 --2- 172.21.15.38:0/1244238673 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe77c0a2b90 0x7fe77c0a50a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x558dc6299850 tx=0x7fe78c003f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.638+0000 7fe79f017740 1 -- 172.21.15.38:0/2798205170 shutdown_connections 2024-05-21T15:34:04.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.638+0000 7fe79f017740 1 --2- 172.21.15.38:0/2798205170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ed7590 0x558dc629f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.638+0000 7fe79f017740 1 --2- 172.21.15.38:0/2798205170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ecf3b0 0x558dc62989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.638+0000 7fe79f017740 1 --2- 172.21.15.38:0/2798205170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ef8270 0x558dc6298470 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.638+0000 7fe79f017740 1 -- 172.21.15.38:0/2798205170 >> 172.21.15.38:0/2798205170 conn(0x558dc5eec3c0 msgr2=0x558dc6293e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.638+0000 7fe79f017740 1 -- 172.21.15.38:0/2798205170 shutdown_connections 2024-05-21T15:34:04.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.638+0000 7fe79f017740 1 -- 172.21.15.38:0/2798205170 wait complete. 2024-05-21T15:34:04.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.638+0000 7fe79f017740 1 -- start start 2024-05-21T15:34:04.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79f017740 1 Processor -- start 2024-05-21T15:34:04.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79f017740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ecf3b0 0x558dc62a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79f017740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ed7590 0x558dc62a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79f017740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ef8270 0x558dc62a9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79f017740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558dc62a0340 con 0x558dc5ef8270 2024-05-21T15:34:04.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79f017740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558dc61b78e0 con 0x558dc5ed7590 2024-05-21T15:34:04.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79f017740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558dc62a04c0 con 0x558dc5ecf3b0 2024-05-21T15:34:04.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79f017740 10 client.0 _ll_register_callbacks cb 0x558dc61849c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:04.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79670f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ef8270 0x558dc62a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79670f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ef8270 0x558dc62a9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51590/0 (socket says 172.21.15.38:51590) 2024-05-21T15:34:04.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79670f640 1 -- 172.21.15.38:0/1051417673 learned_addr learned my addr 172.21.15.38:0/1051417673 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79570d640 1 --2- 172.21.15.38:0/1051417673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ed7590 0x558dc62a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79670f640 1 --2- 172.21.15.38:0/1051417673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ecf3b0 0x558dc62a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79670f640 1 -- 172.21.15.38:0/1051417673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ecf3b0 msgr2=0x558dc62a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79670f640 1 --2- 172.21.15.38:0/1051417673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ecf3b0 0x558dc62a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79670f640 1 -- 172.21.15.38:0/1051417673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ed7590 msgr2=0x558dc62a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79670f640 1 --2- 172.21.15.38:0/1051417673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ed7590 0x558dc62a27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79670f640 1 -- 172.21.15.38:0/1051417673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc5fda6b0 con 0x558dc5ef8270 2024-05-21T15:34:04.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.639+0000 7fe79570d640 1 --2- 172.21.15.38:0/1051417673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ed7590 0x558dc62a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:34:04.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe79670f640 1 --2- 172.21.15.38:0/1051417673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ef8270 0x558dc62a9740 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe788000f80 tx=0x7fe788009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8916a2c0360a35a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe7797fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:04.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe7797fa640 1 -- 172.21.15.38:0/1051417673 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788004c40 con 0x558dc5ef8270 2024-05-21T15:34:04.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe7797fa640 1 -- 172.21.15.38:0/1051417673 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fe788004de0 con 0x558dc5ef8270 2024-05-21T15:34:04.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe79f017740 1 -- 172.21.15.38:0/1051417673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe788004030 con 0x558dc5ef8270 2024-05-21T15:34:04.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe7797fa640 1 -- 172.21.15.38:0/1051417673 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe788004fe0 con 0x558dc5ef8270 2024-05-21T15:34:04.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe79f017740 10 client.15123 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:04.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe79f017740 1 -- 172.21.15.38:0/1051417673 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558dc62ac670 con 0x558dc5ef8270 2024-05-21T15:34:04.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe7797fa640 1 -- 172.21.15.38:0/1051417673 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe788004030 con 0x558dc5ef8270 2024-05-21T15:34:04.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe7797fa640 1 -- 172.21.15.38:0/1051417673 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe788018070 con 0x558dc5ef8270 2024-05-21T15:34:04.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe79f017740 10 client.15123 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:04.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe79f017740 10 client.15123 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T15:34:04.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe79f017740 20 client.15123 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:04.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe79f017740 1 --2- 172.21.15.38:0/1051417673 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558dc5eb6e40 0x558dc62afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe79f017740 4 client.15123 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:34:04.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.640+0000 7fe79f017740 1 -- 172.21.15.38:0/1051417673 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dc6209c00 con 0x558dc5eb6e40 2024-05-21T15:34:04.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.641+0000 7fe79570d640 1 --2- 172.21.15.38:0/1051417673 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558dc5eb6e40 0x558dc62afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.641+0000 7fe7797fa640 10 client.15123 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:34:04.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.641+0000 7fe79570d640 1 --2- 172.21.15.38:0/1051417673 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558dc5eb6e40 0x558dc62afe90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3a43ba0780029f2 server_cookie=aac224e035000518 in_seq=0 out_seq=0 2024-05-21T15:34:04.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.642+0000 7fe7797fa640 1 -- 172.21.15.38:0/1051417673 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558dc6209c00 con 0x558dc5eb6e40 2024-05-21T15:34:04.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.642+0000 7fe7797fa640 10 client.15123 handle_command_reply: tid=0 2024-05-21T15:34:04.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.642+0000 7fe79f017740 1 client.15123 shutdown 2024-05-21T15:34:04.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.642+0000 7fe79f017740 1 -- 172.21.15.38:0/1051417673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ef8270 msgr2=0x558dc62a9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.642+0000 7fe79f017740 1 --2- 172.21.15.38:0/1051417673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ef8270 0x558dc62a9740 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe788000f80 tx=0x7fe788009c20 comp rx=0 tx=0).stop 2024-05-21T15:34:04.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.642+0000 7fe79f017740 1 -- 172.21.15.38:0/1051417673 shutdown_connections 2024-05-21T15:34:04.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.642+0000 7fe79f017740 1 --2- 172.21.15.38:0/1051417673 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558dc5eb6e40 0x558dc62afe90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.642+0000 7fe79f017740 1 --2- 172.21.15.38:0/1051417673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ef8270 0x558dc62a9740 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.642+0000 7fe79f017740 1 --2- 172.21.15.38:0/1051417673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ed7590 0x558dc62a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.642+0000 7fe7797fa640 0 client.15123 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:34:04.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.642+0000 7fe79f017740 1 --2- 172.21.15.38:0/1051417673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ecf3b0 0x558dc62a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.642+0000 7fe79f017740 1 -- 172.21.15.38:0/1051417673 >> 172.21.15.38:0/1051417673 conn(0x558dc5eec3c0 msgr2=0x558dc6292ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.642+0000 7fe79f017740 1 -- 172.21.15.38:0/1051417673 shutdown_connections 2024-05-21T15:34:04.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.642+0000 7fe79f017740 1 -- 172.21.15.38:0/1051417673 wait complete. 2024-05-21T15:34:04.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.643+0000 7fe79f017740 20 client.15123 trim_cache size 0 max 16384 2024-05-21T15:34:04.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.661+0000 7fe79f017740 1 Processor -- start 2024-05-21T15:34:04.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.661+0000 7fe79f017740 1 -- start start 2024-05-21T15:34:04.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79f017740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ecf3b0 0x558dc62983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79f017740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ed7590 0x558dc62a9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79f017740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ef8270 0x558dc629c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79f017740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558dc62ae1d0 con 0x558dc5ecf3b0 2024-05-21T15:34:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79f017740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558dc62ae8a0 con 0x558dc5ef8270 2024-05-21T15:34:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79f017740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558dc62aef70 con 0x558dc5ed7590 2024-05-21T15:34:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79670f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ecf3b0 0x558dc62983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79670f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ecf3b0 0x558dc62983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51606/0 (socket says 172.21.15.38:51606) 2024-05-21T15:34:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79670f640 1 -- 172.21.15.38:0/2174679099 learned_addr learned my addr 172.21.15.38:0/2174679099 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79670f640 1 --2- 172.21.15.38:0/2174679099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ef8270 0x558dc629c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79670f640 1 -- 172.21.15.38:0/2174679099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ed7590 msgr2=0x558dc62a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79670f640 1 --2- 172.21.15.38:0/2174679099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ed7590 0x558dc62a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79670f640 1 -- 172.21.15.38:0/2174679099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ef8270 msgr2=0x558dc629c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79670f640 1 --2- 172.21.15.38:0/2174679099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ef8270 0x558dc629c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79670f640 1 -- 172.21.15.38:0/2174679099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc6292220 con 0x558dc5ecf3b0 2024-05-21T15:34:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe79670f640 1 --2- 172.21.15.38:0/2174679099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ecf3b0 0x558dc62983f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe788005d70 tx=0x7fe788009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3dd4e01dce6aa54 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe77b7fe640 1 -- 172.21.15.38:0/2174679099 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788003a30 con 0x558dc5ecf3b0 2024-05-21T15:34:04.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe77b7fe640 1 -- 172.21.15.38:0/2174679099 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe78801f040 con 0x558dc5ecf3b0 2024-05-21T15:34:04.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.662+0000 7fe77b7fe640 1 -- 172.21.15.38:0/2174679099 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78800f6e0 con 0x558dc5ecf3b0 2024-05-21T15:34:04.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.663+0000 7fe79f017740 1 -- 172.21.15.38:0/2174679099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ecf3b0 msgr2=0x558dc62983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.663+0000 7fe79f017740 1 --2- 172.21.15.38:0/2174679099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ecf3b0 0x558dc62983f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe788005d70 tx=0x7fe788009b30 comp rx=0 tx=0).stop 2024-05-21T15:34:04.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.663+0000 7fe79f017740 1 -- 172.21.15.38:0/2174679099 shutdown_connections 2024-05-21T15:34:04.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.663+0000 7fe79f017740 1 --2- 172.21.15.38:0/2174679099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ef8270 0x558dc629c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.663+0000 7fe79f017740 1 --2- 172.21.15.38:0/2174679099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ed7590 0x558dc62a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.663+0000 7fe79f017740 1 --2- 172.21.15.38:0/2174679099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ecf3b0 0x558dc62983f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.663+0000 7fe79f017740 1 -- 172.21.15.38:0/2174679099 >> 172.21.15.38:0/2174679099 conn(0x558dc5eec3c0 msgr2=0x558dc62937f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.663+0000 7fe79f017740 1 -- 172.21.15.38:0/2174679099 shutdown_connections 2024-05-21T15:34:04.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.663+0000 7fe79f017740 1 -- 172.21.15.38:0/2174679099 wait complete. 2024-05-21T15:34:04.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.663+0000 7fe79f017740 1 -- start start 2024-05-21T15:34:04.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.663+0000 7fe79f017740 1 Processor -- start 2024-05-21T15:34:04.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79f017740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ecf3b0 0x558dc62a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79f017740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ed7590 0x558dc62a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79f017740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ef8270 0x558dc6316d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79f017740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558dc62afd10 con 0x558dc5ecf3b0 2024-05-21T15:34:04.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79f017740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558dc62af640 con 0x558dc5ed7590 2024-05-21T15:34:04.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79f017740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558dc62b03e0 con 0x558dc5ef8270 2024-05-21T15:34:04.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79f017740 10 client.0 _ll_register_callbacks cb 0x7fe7880093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:04.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79670f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ecf3b0 0x558dc62a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79670f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ecf3b0 0x558dc62a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51620/0 (socket says 172.21.15.38:51620) 2024-05-21T15:34:04.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79670f640 1 -- 172.21.15.38:0/2107674253 learned_addr learned my addr 172.21.15.38:0/2107674253 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79570d640 1 --2- 172.21.15.38:0/2107674253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ed7590 0x558dc62a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79670f640 1 --2- 172.21.15.38:0/2107674253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ef8270 0x558dc6316d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79670f640 1 -- 172.21.15.38:0/2107674253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ef8270 msgr2=0x558dc6316d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79670f640 1 --2- 172.21.15.38:0/2107674253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ef8270 0x558dc6316d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79670f640 1 -- 172.21.15.38:0/2107674253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ed7590 msgr2=0x558dc62a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79670f640 1 --2- 172.21.15.38:0/2107674253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ed7590 0x558dc62a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79670f640 1 -- 172.21.15.38:0/2107674253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc6317240 con 0x558dc5ecf3b0 2024-05-21T15:34:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79570d640 1 --2- 172.21.15.38:0/2107674253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ed7590 0x558dc62a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:34:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe79670f640 1 --2- 172.21.15.38:0/2107674253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ecf3b0 0x558dc62a46b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe788000f80 tx=0x7fe788009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7854a9a01b7cc13 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.664+0000 7fe7797fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.665+0000 7fe7797fa640 1 -- 172.21.15.38:0/2107674253 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788003c90 con 0x558dc5ecf3b0 2024-05-21T15:34:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.665+0000 7fe7797fa640 1 -- 172.21.15.38:0/2107674253 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fe788003e30 con 0x558dc5ecf3b0 2024-05-21T15:34:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.665+0000 7fe79f017740 1 -- 172.21.15.38:0/2107674253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe78801f040 con 0x558dc5ecf3b0 2024-05-21T15:34:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.665+0000 7fe7797fa640 1 -- 172.21.15.38:0/2107674253 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe788004030 con 0x558dc5ecf3b0 2024-05-21T15:34:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.665+0000 7fe79f017740 10 client.15129 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.665+0000 7fe79f017740 1 -- 172.21.15.38:0/2107674253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558dc6318460 con 0x558dc5ecf3b0 2024-05-21T15:34:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.665+0000 7fe7797fa640 1 -- 172.21.15.38:0/2107674253 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe78801f040 con 0x558dc5ecf3b0 2024-05-21T15:34:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.665+0000 7fe7797fa640 1 -- 172.21.15.38:0/2107674253 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe788018070 con 0x558dc5ecf3b0 2024-05-21T15:34:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.665+0000 7fe79f017740 10 client.15129 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.665+0000 7fe79f017740 10 client.15129 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T15:34:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.665+0000 7fe79f017740 20 client.15129 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.665+0000 7fe79f017740 1 --2- 172.21.15.38:0/2107674253 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558dc5e8c280 0x558dc631b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.665+0000 7fe79f017740 4 client.15129 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:34:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.665+0000 7fe79f017740 1 -- 172.21.15.38:0/2107674253 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "status"}) -- 0x558dc6203f50 con 0x558dc5e8c280 2024-05-21T15:34:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.666+0000 7fe79570d640 1 --2- 172.21.15.38:0/2107674253 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558dc5e8c280 0x558dc631b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.666+0000 7fe7797fa640 10 client.15129 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:34:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.666+0000 7fe79570d640 1 --2- 172.21.15.38:0/2107674253 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558dc5e8c280 0x558dc631b7e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b656eeda315da2e3 server_cookie=cf5d191094c01576 in_seq=0 out_seq=0 2024-05-21T15:34:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.666+0000 7fe7797fa640 1 -- 172.21.15.38:0/2107674253 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558dc6203f50 con 0x558dc5e8c280 2024-05-21T15:34:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.666+0000 7fe7797fa640 10 client.15129 handle_command_reply: tid=0 2024-05-21T15:34:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.666+0000 7fe79f017740 1 client.15129 shutdown 2024-05-21T15:34:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.666+0000 7fe79f017740 1 -- 172.21.15.38:0/2107674253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ecf3b0 msgr2=0x558dc62a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.666+0000 7fe79f017740 1 --2- 172.21.15.38:0/2107674253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ecf3b0 0x558dc62a46b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe788000f80 tx=0x7fe788009500 comp rx=0 tx=0).stop 2024-05-21T15:34:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.666+0000 7fe79f017740 1 -- 172.21.15.38:0/2107674253 shutdown_connections 2024-05-21T15:34:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.666+0000 7fe79f017740 1 --2- 172.21.15.38:0/2107674253 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558dc5e8c280 0x558dc631b7e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe79f017740 1 --2- 172.21.15.38:0/2107674253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558dc5ef8270 0x558dc6316d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe7797fa640 0 client.15129 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:34:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe79f017740 1 --2- 172.21.15.38:0/2107674253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558dc5ed7590 0x558dc62a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe79f017740 1 --2- 172.21.15.38:0/2107674253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558dc5ecf3b0 0x558dc62a46b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe79f017740 1 -- 172.21.15.38:0/2107674253 >> 172.21.15.38:0/2107674253 conn(0x558dc5eec3c0 msgr2=0x558dc62937f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe79f017740 1 -- 172.21.15.38:0/2107674253 shutdown_connections 2024-05-21T15:34:04.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe79f017740 1 -- 172.21.15.38:0/2107674253 wait complete. 2024-05-21T15:34:04.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe79f017740 20 client.15129 trim_cache size 0 max 16384 2024-05-21T15:34:04.673 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:34:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "102ec4fa-1786-11ef-bc9a-c7b262605968", 2024-05-21T15:34:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 2, 2024-05-21T15:34:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24550, 2024-05-21T15:34:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-05-21T15:34:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-05-21T15:34:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-05-21T15:34:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 86.716365115000002, 2024-05-21T15:34:04.673 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-05-21T15:34:04.674 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-05-21T15:34:04.674 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-05-21T15:34:04.674 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 104.46647030299999 2024-05-21T15:34:04.674 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:34:04.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe798172640 1 -- 172.21.15.38:0/1244238673 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe77c0a2b90 msgr2=0x7fe77c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe798172640 1 --2- 172.21.15.38:0/1244238673 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe77c0a2b90 0x7fe77c0a50a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x558dc6299850 tx=0x7fe78c003f10 comp rx=0 tx=0).stop 2024-05-21T15:34:04.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe798172640 1 -- 172.21.15.38:0/1244238673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7901019b0 msgr2=0x7fe7901981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe798172640 1 --2- 172.21.15.38:0/1244238673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7901019b0 0x7fe7901981f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00e9f0 tx=0x7fe78c00eeb0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe798172640 1 -- 172.21.15.38:0/1244238673 shutdown_connections 2024-05-21T15:34:04.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe798172640 1 --2- 172.21.15.38:0/1244238673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901078f0 0x7fe79019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe798172640 1 --2- 172.21.15.38:0/1244238673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe790102bf0 0x7fe790198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe798172640 1 --2- 172.21.15.38:0/1244238673 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe77c0a2b90 0x7fe77c0a50a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe798172640 1 --2- 172.21.15.38:0/1244238673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7901019b0 0x7fe7901981f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe798172640 1 -- 172.21.15.38:0/1244238673 >> 172.21.15.38:0/1244238673 conn(0x7fe7900fcbe0 msgr2=0x7fe7901038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe798172640 1 -- 172.21.15.38:0/1244238673 shutdown_connections 2024-05-21T15:34:04.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.667+0000 7fe798172640 1 -- 172.21.15.38:0/1244238673 wait complete. 2024-05-21T15:34:04.678 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-05-21T15:34:04.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.795+0000 7fb3d54e2640 1 -- 172.21.15.38:0/2465703796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3d0103580 msgr2=0x7fb3d01059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.795+0000 7fb3d54e2640 1 --2- 172.21.15.38:0/2465703796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3d0103580 0x7fb3d01059a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009900 tx=0x7fb3c002a410 comp rx=0 tx=0).stop 2024-05-21T15:34:04.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.795+0000 7fb3d54e2640 1 -- 172.21.15.38:0/2465703796 shutdown_connections 2024-05-21T15:34:04.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.795+0000 7fb3d54e2640 1 --2- 172.21.15.38:0/2465703796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d0108840 0x7fb3d010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.795+0000 7fb3d54e2640 1 --2- 172.21.15.38:0/2465703796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3d0105ee0 0x7fb3d0108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.795+0000 7fb3d54e2640 1 --2- 172.21.15.38:0/2465703796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3d0103580 0x7fb3d01059a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.795+0000 7fb3d54e2640 1 -- 172.21.15.38:0/2465703796 >> 172.21.15.38:0/2465703796 conn(0x7fb3d00fcb80 msgr2=0x7fb3d00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.795+0000 7fb3d54e2640 1 -- 172.21.15.38:0/2465703796 shutdown_connections 2024-05-21T15:34:04.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.795+0000 7fb3d54e2640 1 -- 172.21.15.38:0/2465703796 wait complete. 2024-05-21T15:34:04.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.795+0000 7fb3d54e2640 1 Processor -- start 2024-05-21T15:34:04.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3d54e2640 1 -- start start 2024-05-21T15:34:04.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3d54e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d0103580 0x7fb3d01029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3d54e2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3d0105ee0 0x7fb3d0101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3d54e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3d0108840 0x7fb3d01015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3d54e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d010e6c0 con 0x7fb3d0108840 2024-05-21T15:34:04.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3d54e2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d010e540 con 0x7fb3d0105ee0 2024-05-21T15:34:04.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3d54e2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d010e840 con 0x7fb3d0103580 2024-05-21T15:34:04.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3d0108840 0x7fb3d01015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3d0108840 0x7fb3d01015b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49896/0 (socket says 172.21.15.38:49896) 2024-05-21T15:34:04.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3cf7fe640 1 -- 172.21.15.38:0/1509435849 learned_addr learned my addr 172.21.15.38:0/1509435849 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3d0105ee0 0x7fb3d0101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3ceffd640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d0103580 0x7fb3d01029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3cf7fe640 1 -- 172.21.15.38:0/1509435849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d0103580 msgr2=0x7fb3d01029d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3cf7fe640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d0103580 0x7fb3d01029d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3cf7fe640 1 -- 172.21.15.38:0/1509435849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3d0105ee0 msgr2=0x7fb3d0101070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3cf7fe640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3d0105ee0 0x7fb3d0101070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3cf7fe640 1 -- 172.21.15.38:0/1509435849 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d0101ce0 con 0x7fb3d0108840 2024-05-21T15:34:04.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.796+0000 7fb3ceffd640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d0103580 0x7fb3d01029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:04.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.797+0000 7fb3cf7fe640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3d0108840 0x7fb3d01015b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800dac0 tx=0x7fb3c800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.797+0000 7fb3c7fff640 1 -- 172.21.15.38:0/1509435849 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8017070 con 0x7fb3d0108840 2024-05-21T15:34:04.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.797+0000 7fb3c7fff640 1 -- 172.21.15.38:0/1509435849 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3c800be30 con 0x7fb3d0108840 2024-05-21T15:34:04.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.797+0000 7fb3c7fff640 1 -- 172.21.15.38:0/1509435849 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c800f890 con 0x7fb3d0108840 2024-05-21T15:34:04.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.797+0000 7fb3d54e2640 1 -- 172.21.15.38:0/1509435849 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d01a3330 con 0x7fb3d0108840 2024-05-21T15:34:04.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.797+0000 7fb3d54e2640 1 -- 172.21.15.38:0/1509435849 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d01a37a0 con 0x7fb3d0108840 2024-05-21T15:34:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.799+0000 7fb3dc388740 1 Processor -- start 2024-05-21T15:34:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.799+0000 7fb3c7fff640 1 -- 172.21.15.38:0/1509435849 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb3c800fad0 con 0x7fb3d0108840 2024-05-21T15:34:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.799+0000 7fb3dc388740 1 -- start start 2024-05-21T15:34:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3dc388740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca939270 0x563bcacd9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3dc388740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca9103b0 0x563bcacd99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3dc388740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca918590 0x563bcace08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3dc388740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563bcac3d4d0 con 0x563bca918590 2024-05-21T15:34:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3dc388740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563bcabe53e0 con 0x563bca939270 2024-05-21T15:34:04.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3dc388740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563bcac39490 con 0x563bca9103b0 2024-05-21T15:34:04.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3ce7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca939270 0x563bcacd9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3ce7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca939270 0x563bcacd9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:40884/0 (socket says 172.21.15.38:40884) 2024-05-21T15:34:04.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3ce7fc640 1 -- 172.21.15.38:0/3496915577 learned_addr learned my addr 172.21.15.38:0/3496915577 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/3496915577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca918590 0x563bcace08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3ceffd640 1 --2- 172.21.15.38:0/3496915577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca9103b0 0x563bcacd99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3c7fff640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb3bc0a2b90 0x7fb3bc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3ce7fc640 1 -- 172.21.15.38:0/3496915577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca9103b0 msgr2=0x563bcacd99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/3496915577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca9103b0 0x563bcacd99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3cf7fe640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb3bc0a2b90 0x7fb3bc0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3ce7fc640 1 -- 172.21.15.38:0/3496915577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca918590 msgr2=0x563bcace08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/3496915577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca918590 0x563bcace08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.800+0000 7fb3ce7fc640 1 -- 172.21.15.38:0/3496915577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bca9512d0 con 0x563bca939270 2024-05-21T15:34:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.801+0000 7fb3c7fff640 1 -- 172.21.15.38:0/1509435849 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb3c80cf020 con 0x7fb3d0108840 2024-05-21T15:34:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.801+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/3496915577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca939270 0x563bcacd9470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8005bc0 tx=0x7fb3b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4369173a4344b082 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.801+0000 7fb3c57fa640 1 -- 172.21.15.38:0/3496915577 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8004030 con 0x563bca939270 2024-05-21T15:34:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.801+0000 7fb3cf7fe640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb3bc0a2b90 0x7fb3bc0a50a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb3c806e900 tx=0x7fb3c8086040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.801+0000 7fb3c57fa640 1 -- 172.21.15.38:0/3496915577 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b8005500 con 0x563bca939270 2024-05-21T15:34:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.801+0000 7fb3c57fa640 1 -- 172.21.15.38:0/3496915577 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8004ce0 con 0x563bca939270 2024-05-21T15:34:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.802+0000 7fb3dc388740 1 -- 172.21.15.38:0/3496915577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca939270 msgr2=0x563bcacd9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.802+0000 7fb3dc388740 1 --2- 172.21.15.38:0/3496915577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca939270 0x563bcacd9470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8005bc0 tx=0x7fb3b80046c0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.802+0000 7fb3dc388740 1 -- 172.21.15.38:0/3496915577 shutdown_connections 2024-05-21T15:34:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.802+0000 7fb3dc388740 1 --2- 172.21.15.38:0/3496915577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca918590 0x563bcace08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.802+0000 7fb3dc388740 1 --2- 172.21.15.38:0/3496915577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca9103b0 0x563bcacd99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.802+0000 7fb3dc388740 1 --2- 172.21.15.38:0/3496915577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca939270 0x563bcacd9470 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.802+0000 7fb3dc388740 1 -- 172.21.15.38:0/3496915577 >> 172.21.15.38:0/3496915577 conn(0x563bca92d3c0 msgr2=0x563bcacd4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.802+0000 7fb3dc388740 1 -- 172.21.15.38:0/3496915577 shutdown_connections 2024-05-21T15:34:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.802+0000 7fb3dc388740 1 -- 172.21.15.38:0/3496915577 wait complete. 2024-05-21T15:34:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.802+0000 7fb3dc388740 1 -- start start 2024-05-21T15:34:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.802+0000 7fb3dc388740 1 Processor -- start 2024-05-21T15:34:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3dc388740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca9103b0 0x563bcace3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3dc388740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca918590 0x563bcace3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3dc388740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca939270 0x563bcacea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3dc388740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563bcace1340 con 0x563bca918590 2024-05-21T15:34:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3dc388740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563bcabf88e0 con 0x563bca9103b0 2024-05-21T15:34:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3dc388740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563bcace14c0 con 0x563bca939270 2024-05-21T15:34:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3dc388740 10 client.0 _ll_register_callbacks cb 0x563bcabc59c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3ceffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca918590 0x563bcace3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3ce7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca9103b0 0x563bcace3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3ce7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca9103b0 0x563bcace3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:40890/0 (socket says 172.21.15.38:40890) 2024-05-21T15:34:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3ce7fc640 1 -- 172.21.15.38:0/4206229555 learned_addr learned my addr 172.21.15.38:0/4206229555 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3ceffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca918590 0x563bcace3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49910/0 (socket says 172.21.15.38:49910) 2024-05-21T15:34:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/4206229555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca939270 0x563bcacea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3ceffd640 1 -- 172.21.15.38:0/4206229555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca939270 msgr2=0x563bcacea6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3ceffd640 1 --2- 172.21.15.38:0/4206229555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca939270 0x563bcacea6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3ceffd640 1 -- 172.21.15.38:0/4206229555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca9103b0 msgr2=0x563bcace3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3ceffd640 1 --2- 172.21.15.38:0/4206229555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca9103b0 0x563bcace3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3ceffd640 1 -- 172.21.15.38:0/4206229555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bcaa1b6b0 con 0x563bca918590 2024-05-21T15:34:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/4206229555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca9103b0 0x563bcace3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:34:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3ceffd640 1 --2- 172.21.15.38:0/4206229555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca918590 0x563bcace3780 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0006010 tx=0x7fb3c0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c22cab0afc8f1be server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.803+0000 7fb3b67fc640 1 -- 172.21.15.38:0/4206229555 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0035040 con 0x563bca918590 2024-05-21T15:34:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.804+0000 7fb3dc388740 1 -- 172.21.15.38:0/4206229555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b8005500 con 0x563bca918590 2024-05-21T15:34:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.804+0000 7fb3b67fc640 1 -- 172.21.15.38:0/4206229555 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb3c0004300 con 0x563bca918590 2024-05-21T15:34:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.804+0000 7fb3b67fc640 1 -- 172.21.15.38:0/4206229555 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb3c0005030 con 0x563bca918590 2024-05-21T15:34:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.804+0000 7fb3dc388740 10 client.15141 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.804+0000 7fb3dc388740 1 -- 172.21.15.38:0/4206229555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563bcaced6d0 con 0x563bca918590 2024-05-21T15:34:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.804+0000 7fb3b67fc640 1 -- 172.21.15.38:0/4206229555 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b8005500 con 0x563bca918590 2024-05-21T15:34:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.804+0000 7fb3b67fc640 1 -- 172.21.15.38:0/4206229555 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb3c0004bb0 con 0x563bca918590 2024-05-21T15:34:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.804+0000 7fb3dc388740 10 client.15141 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.804+0000 7fb3dc388740 10 client.15141 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T15:34:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.804+0000 7fb3dc388740 20 client.15141 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.804+0000 7fb3dc388740 1 --2- 172.21.15.38:0/4206229555 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x563bca8f7e40 0x563bcacf10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.804+0000 7fb3dc388740 4 client.15141 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:34:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.804+0000 7fb3dc388740 1 -- 172.21.15.38:0/4206229555 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bcac4ac00 con 0x563bca8f7e40 2024-05-21T15:34:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.804+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/4206229555 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x563bca8f7e40 0x563bcacf10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.805+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/4206229555 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x563bca8f7e40 0x563bcacf10f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1338cebba06953d server_cookie=96e5cb6eb589016a in_seq=0 out_seq=0 2024-05-21T15:34:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.805+0000 7fb3b67fc640 10 client.15141 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:34:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3b67fc640 1 -- 172.21.15.38:0/4206229555 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563bcac4ac00 con 0x563bca8f7e40 2024-05-21T15:34:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3b67fc640 10 client.15141 handle_command_reply: tid=0 2024-05-21T15:34:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3dc388740 1 client.15141 shutdown 2024-05-21T15:34:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3dc388740 1 -- 172.21.15.38:0/4206229555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca918590 msgr2=0x563bcace3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3dc388740 1 --2- 172.21.15.38:0/4206229555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca918590 0x563bcace3780 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0006010 tx=0x7fb3c0031a20 comp rx=0 tx=0).stop 2024-05-21T15:34:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3dc388740 1 -- 172.21.15.38:0/4206229555 shutdown_connections 2024-05-21T15:34:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3dc388740 1 --2- 172.21.15.38:0/4206229555 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x563bca8f7e40 0x563bcacf10f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3dc388740 1 --2- 172.21.15.38:0/4206229555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca939270 0x563bcacea6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3dc388740 1 --2- 172.21.15.38:0/4206229555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca918590 0x563bcace3780 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3dc388740 1 --2- 172.21.15.38:0/4206229555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca9103b0 0x563bcace3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3b67fc640 0 client.15141 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:34:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3dc388740 1 -- 172.21.15.38:0/4206229555 >> 172.21.15.38:0/4206229555 conn(0x563bca92d3c0 msgr2=0x563bcacd3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3dc388740 1 -- 172.21.15.38:0/4206229555 shutdown_connections 2024-05-21T15:34:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3dc388740 1 -- 172.21.15.38:0/4206229555 wait complete. 2024-05-21T15:34:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.806+0000 7fb3dc388740 20 client.15141 trim_cache size 0 max 16384 2024-05-21T15:34:04.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.825+0000 7fb3dc388740 1 Processor -- start 2024-05-21T15:34:04.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.825+0000 7fb3dc388740 1 -- start start 2024-05-21T15:34:04.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.825+0000 7fb3dc388740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca9103b0 0x563bcace1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.825+0000 7fb3dc388740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca918590 0x563bcace1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.825+0000 7fb3dc388740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca939270 0x563bcacea600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.825+0000 7fb3dc388740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563bcacef430 con 0x563bca9103b0 2024-05-21T15:34:04.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.825+0000 7fb3dc388740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563bcacefb00 con 0x563bca939270 2024-05-21T15:34:04.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.825+0000 7fb3dc388740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563bcacf01d0 con 0x563bca918590 2024-05-21T15:34:04.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3ce7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca9103b0 0x563bcace1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3ce7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca9103b0 0x563bcace1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49922/0 (socket says 172.21.15.38:49922) 2024-05-21T15:34:04.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3ce7fc640 1 -- 172.21.15.38:0/1858067854 learned_addr learned my addr 172.21.15.38:0/1858067854 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/1858067854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca939270 0x563bcacea600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3ceffd640 1 --2- 172.21.15.38:0/1858067854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca918590 0x563bcace1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3ce7fc640 1 -- 172.21.15.38:0/1858067854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca918590 msgr2=0x563bcace1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/1858067854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca918590 0x563bcace1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3ce7fc640 1 -- 172.21.15.38:0/1858067854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca939270 msgr2=0x563bcacea600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/1858067854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca939270 0x563bcacea600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3ce7fc640 1 -- 172.21.15.38:0/1858067854 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bcace2ff0 con 0x563bca9103b0 2024-05-21T15:34:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3ceffd640 1 --2- 172.21.15.38:0/1858067854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca918590 0x563bcace1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/1858067854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca9103b0 0x563bcace1860 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80117f0 tx=0x7fb3b8005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2265231fb235ec2f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3c4ff9640 1 -- 172.21.15.38:0/1858067854 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8028040 con 0x563bca9103b0 2024-05-21T15:34:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3c4ff9640 1 -- 172.21.15.38:0/1858067854 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b8005450 con 0x563bca9103b0 2024-05-21T15:34:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.826+0000 7fb3c4ff9640 1 -- 172.21.15.38:0/1858067854 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b800fd90 con 0x563bca9103b0 2024-05-21T15:34:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.827+0000 7fb3dc388740 1 -- 172.21.15.38:0/1858067854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca9103b0 msgr2=0x563bcace1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.827+0000 7fb3dc388740 1 --2- 172.21.15.38:0/1858067854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca9103b0 0x563bcace1860 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80117f0 tx=0x7fb3b8005d30 comp rx=0 tx=0).stop 2024-05-21T15:34:04.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.827+0000 7fb3dc388740 1 -- 172.21.15.38:0/1858067854 shutdown_connections 2024-05-21T15:34:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.827+0000 7fb3dc388740 1 --2- 172.21.15.38:0/1858067854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca939270 0x563bcacea600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.827+0000 7fb3dc388740 1 --2- 172.21.15.38:0/1858067854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca918590 0x563bcace1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.827+0000 7fb3dc388740 1 --2- 172.21.15.38:0/1858067854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca9103b0 0x563bcace1860 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.827+0000 7fb3dc388740 1 -- 172.21.15.38:0/1858067854 >> 172.21.15.38:0/1858067854 conn(0x563bca92d3c0 msgr2=0x563bcacdc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.827+0000 7fb3dc388740 1 -- 172.21.15.38:0/1858067854 shutdown_connections 2024-05-21T15:34:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.827+0000 7fb3dc388740 1 -- 172.21.15.38:0/1858067854 wait complete. 2024-05-21T15:34:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.827+0000 7fb3dc388740 1 -- start start 2024-05-21T15:34:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3dc388740 1 Processor -- start 2024-05-21T15:34:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3dc388740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca9103b0 0x563bcace5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3dc388740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca918590 0x563bcace6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3dc388740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca939270 0x563bcad57f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3dc388740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563bcacf0f70 con 0x563bca939270 2024-05-21T15:34:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3dc388740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563bcacf08a0 con 0x563bca918590 2024-05-21T15:34:04.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3dc388740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563bcacf1640 con 0x563bca9103b0 2024-05-21T15:34:04.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3dc388740 10 client.0 _ll_register_callbacks cb 0x7fb3c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:04.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3ce7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca939270 0x563bcad57f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3ce7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca939270 0x563bcad57f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49930/0 (socket says 172.21.15.38:49930) 2024-05-21T15:34:04.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3ce7fc640 1 -- 172.21.15.38:0/2609381606 learned_addr learned my addr 172.21.15.38:0/2609381606 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3ceffd640 1 --2- 172.21.15.38:0/2609381606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca918590 0x563bcace6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/2609381606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca9103b0 0x563bcace5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3ce7fc640 1 -- 172.21.15.38:0/2609381606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca9103b0 msgr2=0x563bcace5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/2609381606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca9103b0 0x563bcace5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3ce7fc640 1 -- 172.21.15.38:0/2609381606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca918590 msgr2=0x563bcace6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/2609381606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca918590 0x563bcace6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3ce7fc640 1 -- 172.21.15.38:0/2609381606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bcaceb150 con 0x563bca939270 2024-05-21T15:34:04.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3ce7fc640 1 --2- 172.21.15.38:0/2609381606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca939270 0x563bcad57f60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8004e90 tx=0x7fb3b8036040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d00fdf6643ff2c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.828+0000 7fb3b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3b67fc640 1 -- 172.21.15.38:0/2609381606 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8005250 con 0x563bca939270 2024-05-21T15:34:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3b67fc640 1 -- 172.21.15.38:0/2609381606 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb3b800fb10 con 0x563bca939270 2024-05-21T15:34:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3dc388740 1 -- 172.21.15.38:0/2609381606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b8005450 con 0x563bca939270 2024-05-21T15:34:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3b67fc640 1 -- 172.21.15.38:0/2609381606 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb3b800fd10 con 0x563bca939270 2024-05-21T15:34:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3dc388740 10 client.15147 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3dc388740 1 -- 172.21.15.38:0/2609381606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563bcad593f0 con 0x563bca939270 2024-05-21T15:34:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3b67fc640 1 -- 172.21.15.38:0/2609381606 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b8005450 con 0x563bca939270 2024-05-21T15:34:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3b67fc640 1 -- 172.21.15.38:0/2609381606 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb3b802f070 con 0x563bca939270 2024-05-21T15:34:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3dc388740 10 client.15147 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3dc388740 10 client.15147 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T15:34:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3dc388740 20 client.15147 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3dc388740 1 --2- 172.21.15.38:0/2609381606 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x563bca8cd280 0x563bcad5c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3dc388740 4 client.15147 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:34:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3dc388740 1 -- 172.21.15.38:0/2609381606 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "status"}) -- 0x563bcac44f50 con 0x563bca8cd280 2024-05-21T15:34:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.829+0000 7fb3ceffd640 1 --2- 172.21.15.38:0/2609381606 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x563bca8cd280 0x563bcad5c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3b67fc640 10 client.15147 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:34:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3ceffd640 1 --2- 172.21.15.38:0/2609381606 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x563bca8cd280 0x563bcad5c760 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1272f29a2799795 server_cookie=de58c2ebb16a90c8 in_seq=0 out_seq=0 2024-05-21T15:34:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3b67fc640 1 -- 172.21.15.38:0/2609381606 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563bcac44f50 con 0x563bca8cd280 2024-05-21T15:34:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3b67fc640 10 client.15147 handle_command_reply: tid=0 2024-05-21T15:34:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3dc388740 1 client.15147 shutdown 2024-05-21T15:34:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3dc388740 1 -- 172.21.15.38:0/2609381606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca939270 msgr2=0x563bcad57f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3dc388740 1 --2- 172.21.15.38:0/2609381606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca939270 0x563bcad57f60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8004e90 tx=0x7fb3b8036040 comp rx=0 tx=0).stop 2024-05-21T15:34:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3dc388740 1 -- 172.21.15.38:0/2609381606 shutdown_connections 2024-05-21T15:34:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3dc388740 1 --2- 172.21.15.38:0/2609381606 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x563bca8cd280 0x563bcad5c760 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3dc388740 1 --2- 172.21.15.38:0/2609381606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563bca939270 0x563bcad57f60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3b67fc640 0 client.15147 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:34:04.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3dc388740 1 --2- 172.21.15.38:0/2609381606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bca918590 0x563bcace6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3dc388740 1 --2- 172.21.15.38:0/2609381606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bca9103b0 0x563bcace5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3dc388740 1 -- 172.21.15.38:0/2609381606 >> 172.21.15.38:0/2609381606 conn(0x563bca92d3c0 msgr2=0x563bcace1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3dc388740 1 -- 172.21.15.38:0/2609381606 shutdown_connections 2024-05-21T15:34:04.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3dc388740 1 -- 172.21.15.38:0/2609381606 wait complete. 2024-05-21T15:34:04.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.830+0000 7fb3dc388740 20 client.15147 trim_cache size 0 max 16384 2024-05-21T15:34:04.837 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:34:04.837 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "102ec4fa-1786-11ef-bc9a-c7b262605968", 2024-05-21T15:34:04.837 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 1, 2024-05-21T15:34:04.837 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24551, 2024-05-21T15:34:04.837 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-05-21T15:34:04.838 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-05-21T15:34:04.838 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-05-21T15:34:04.838 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 87.900093631999994, 2024-05-21T15:34:04.838 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-05-21T15:34:04.838 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-05-21T15:34:04.838 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-05-21T15:34:04.838 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 107.441456714 2024-05-21T15:34:04.838 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:34:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.831+0000 7fb3d54e2640 1 -- 172.21.15.38:0/1509435849 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb3bc0a2b90 msgr2=0x7fb3bc0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.831+0000 7fb3d54e2640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb3bc0a2b90 0x7fb3bc0a50a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb3c806e900 tx=0x7fb3c8086040 comp rx=0 tx=0).stop 2024-05-21T15:34:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.831+0000 7fb3d54e2640 1 -- 172.21.15.38:0/1509435849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3d0108840 msgr2=0x7fb3d01015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.831+0000 7fb3d54e2640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3d0108840 0x7fb3d01015b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800dac0 tx=0x7fb3c800df80 comp rx=0 tx=0).stop 2024-05-21T15:34:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.831+0000 7fb3d54e2640 1 -- 172.21.15.38:0/1509435849 shutdown_connections 2024-05-21T15:34:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.831+0000 7fb3d54e2640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb3bc0a2b90 0x7fb3bc0a50a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.831+0000 7fb3d54e2640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3d0108840 0x7fb3d01015b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.831+0000 7fb3d54e2640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3d0105ee0 0x7fb3d0101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.831+0000 7fb3d54e2640 1 --2- 172.21.15.38:0/1509435849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d0103580 0x7fb3d01029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.831+0000 7fb3d54e2640 1 -- 172.21.15.38:0/1509435849 >> 172.21.15.38:0/1509435849 conn(0x7fb3d00fcb80 msgr2=0x7fb3d0104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.831+0000 7fb3d54e2640 1 -- 172.21.15.38:0/1509435849 shutdown_connections 2024-05-21T15:34:04.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.831+0000 7fb3d54e2640 1 -- 172.21.15.38:0/1509435849 wait complete. 2024-05-21T15:34:04.842 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-21T15:34:04.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.959+0000 7efc82cc3640 1 -- 172.21.15.38:0/3896047429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7c104120 msgr2=0x7efc7c1022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.959+0000 7efc82cc3640 1 --2- 172.21.15.38:0/3896047429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7c104120 0x7efc7c1022a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009900 tx=0x7efc6c02a410 comp rx=0 tx=0).stop 2024-05-21T15:34:04.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.959+0000 7efc82cc3640 1 -- 172.21.15.38:0/3896047429 shutdown_connections 2024-05-21T15:34:04.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.959+0000 7efc82cc3640 1 --2- 172.21.15.38:0/3896047429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc7c108a20 0x7efc7c10ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.959+0000 7efc82cc3640 1 --2- 172.21.15.38:0/3896047429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7c1027e0 0x7efc7c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.959+0000 7efc82cc3640 1 --2- 172.21.15.38:0/3896047429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7c104120 0x7efc7c1022a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.959+0000 7efc82cc3640 1 -- 172.21.15.38:0/3896047429 >> 172.21.15.38:0/3896047429 conn(0x7efc7c0fdcf0 msgr2=0x7efc7c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.959+0000 7efc82cc3640 1 -- 172.21.15.38:0/3896047429 shutdown_connections 2024-05-21T15:34:04.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.959+0000 7efc82cc3640 1 -- 172.21.15.38:0/3896047429 wait complete. 2024-05-21T15:34:04.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.959+0000 7efc82cc3640 1 Processor -- start 2024-05-21T15:34:04.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.959+0000 7efc82cc3640 1 -- start start 2024-05-21T15:34:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc82cc3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7c1027e0 0x7efc7c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc82cc3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc7c104120 0x7efc7c1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc82cc3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7c108a20 0x7efc7c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc82cc3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c10e890 con 0x7efc7c1027e0 2024-05-21T15:34:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc82cc3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c10e710 con 0x7efc7c104120 2024-05-21T15:34:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc82cc3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c10ea10 con 0x7efc7c108a20 2024-05-21T15:34:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc80a5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7c1027e0 0x7efc7c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc80a5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7c1027e0 0x7efc7c1993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49952/0 (socket says 172.21.15.38:49952) 2024-05-21T15:34:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc80a5f640 1 -- 172.21.15.38:0/1044117268 learned_addr learned my addr 172.21.15.38:0/1044117268 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc7bfff640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc7c104120 0x7efc7c1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc81260640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7c108a20 0x7efc7c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc7bfff640 1 -- 172.21.15.38:0/1044117268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7c108a20 msgr2=0x7efc7c1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc7bfff640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7c108a20 0x7efc7c1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc7bfff640 1 -- 172.21.15.38:0/1044117268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7c1027e0 msgr2=0x7efc7c1993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc7bfff640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7c1027e0 0x7efc7c1993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc7bfff640 1 -- 172.21.15.38:0/1044117268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7c1a0ef0 con 0x7efc7c104120 2024-05-21T15:34:04.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc80a5f640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7c1027e0 0x7efc7c1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:34:04.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.960+0000 7efc7bfff640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc7c104120 0x7efc7c1998f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc6400e9f0 tx=0x7efc6400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.961+0000 7efc79ffb640 1 -- 172.21.15.38:0/1044117268 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6400ccc0 con 0x7efc7c104120 2024-05-21T15:34:04.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.961+0000 7efc82cc3640 1 -- 172.21.15.38:0/1044117268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7c1a11e0 con 0x7efc7c104120 2024-05-21T15:34:04.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.961+0000 7efc79ffb640 1 -- 172.21.15.38:0/1044117268 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc64005020 con 0x7efc7c104120 2024-05-21T15:34:04.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.961+0000 7efc79ffb640 1 -- 172.21.15.38:0/1044117268 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc640049e0 con 0x7efc7c104120 2024-05-21T15:34:04.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.961+0000 7efc82cc3640 1 -- 172.21.15.38:0/1044117268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7c1a16e0 con 0x7efc7c104120 2024-05-21T15:34:04.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.963+0000 7efc79ffb640 1 -- 172.21.15.38:0/1044117268 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc64005560 con 0x7efc7c104120 2024-05-21T15:34:04.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.963+0000 7efc89b63740 1 Processor -- start 2024-05-21T15:34:04.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.963+0000 7efc89b63740 1 -- start start 2024-05-21T15:34:04.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.963+0000 7efc89b63740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd737270 0x5626fdad7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc89b63740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd70e3b0 0x5626fdad79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc89b63740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdade8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc89b63740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5626fda3b4d0 con 0x5626fd716590 2024-05-21T15:34:04.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc89b63740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5626fd9e33e0 con 0x5626fd737270 2024-05-21T15:34:04.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc89b63740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5626fda37490 con 0x5626fd70e3b0 2024-05-21T15:34:04.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc7bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdade8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc81260640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd737270 0x5626fdad7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc7bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdade8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49954/0 (socket says 172.21.15.38:49954) 2024-05-21T15:34:04.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc7bfff640 1 -- 172.21.15.38:0/828930118 learned_addr learned my addr 172.21.15.38:0/828930118 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc81260640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd737270 0x5626fdad7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:40934/0 (socket says 172.21.15.38:40934) 2024-05-21T15:34:04.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc80a5f640 1 --2- 172.21.15.38:0/828930118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd70e3b0 0x5626fdad79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc79ffb640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efc680a2b90 0x7efc680a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc81260640 1 -- 172.21.15.38:0/828930118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd70e3b0 msgr2=0x5626fdad79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc81260640 1 --2- 172.21.15.38:0/828930118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd70e3b0 0x5626fdad79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc81260640 1 -- 172.21.15.38:0/828930118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 msgr2=0x5626fdade8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc81260640 1 --2- 172.21.15.38:0/828930118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdade8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc81260640 1 -- 172.21.15.38:0/828930118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626fd74f2d0 con 0x5626fd737270 2024-05-21T15:34:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc7bfff640 1 --2- 172.21.15.38:0/828930118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdade8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:34:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc79ffb640 1 -- 172.21.15.38:0/1044117268 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7efc640cf0d0 con 0x7efc7c104120 2024-05-21T15:34:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc81260640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efc680a2b90 0x7efc680a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.964+0000 7efc81260640 1 --2- 172.21.15.38:0/828930118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd737270 0x5626fdad7470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc74010420 tx=0x7efc740108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=963d2f9fbd1109e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.965+0000 7efc72ffd640 1 -- 172.21.15.38:0/828930118 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74024070 con 0x5626fd737270 2024-05-21T15:34:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.965+0000 7efc81260640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efc680a2b90 0x7efc680a50a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc74021000 tx=0x7efc7401a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.965+0000 7efc72ffd640 1 -- 172.21.15.38:0/828930118 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc7400ed70 con 0x5626fd737270 2024-05-21T15:34:04.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.965+0000 7efc72ffd640 1 -- 172.21.15.38:0/828930118 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7401ee10 con 0x5626fd737270 2024-05-21T15:34:04.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.965+0000 7efc89b63740 1 -- 172.21.15.38:0/828930118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd737270 msgr2=0x5626fdad7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.965+0000 7efc89b63740 1 --2- 172.21.15.38:0/828930118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd737270 0x5626fdad7470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc74010420 tx=0x7efc740108e0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.965+0000 7efc89b63740 1 -- 172.21.15.38:0/828930118 shutdown_connections 2024-05-21T15:34:04.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.965+0000 7efc89b63740 1 --2- 172.21.15.38:0/828930118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdade8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.965+0000 7efc89b63740 1 --2- 172.21.15.38:0/828930118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd70e3b0 0x5626fdad79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.965+0000 7efc89b63740 1 --2- 172.21.15.38:0/828930118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd737270 0x5626fdad7470 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.965+0000 7efc89b63740 1 -- 172.21.15.38:0/828930118 >> 172.21.15.38:0/828930118 conn(0x5626fd72b3c0 msgr2=0x5626fdad2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.965+0000 7efc89b63740 1 -- 172.21.15.38:0/828930118 shutdown_connections 2024-05-21T15:34:04.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.965+0000 7efc89b63740 1 -- 172.21.15.38:0/828930118 wait complete. 2024-05-21T15:34:04.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.966+0000 7efc89b63740 1 -- start start 2024-05-21T15:34:04.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.966+0000 7efc89b63740 1 Processor -- start 2024-05-21T15:34:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.966+0000 7efc89b63740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd70e3b0 0x5626fdae12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.966+0000 7efc89b63740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdae17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.966+0000 7efc89b63740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd737270 0x5626fdae8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.966+0000 7efc89b63740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5626fdadf340 con 0x5626fd716590 2024-05-21T15:34:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.966+0000 7efc89b63740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5626fd9f68e0 con 0x5626fd737270 2024-05-21T15:34:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.966+0000 7efc89b63740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5626fdadf4c0 con 0x5626fd70e3b0 2024-05-21T15:34:04.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.966+0000 7efc89b63740 10 client.0 _ll_register_callbacks cb 0x5626fd9c39c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.966+0000 7efc7bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdae17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.966+0000 7efc7bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdae17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49966/0 (socket says 172.21.15.38:49966) 2024-05-21T15:34:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.966+0000 7efc7bfff640 1 -- 172.21.15.38:0/2957828601 learned_addr learned my addr 172.21.15.38:0/2957828601 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.966+0000 7efc81260640 1 --2- 172.21.15.38:0/2957828601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd737270 0x5626fdae8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc80a5f640 1 --2- 172.21.15.38:0/2957828601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd70e3b0 0x5626fdae12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc7bfff640 1 -- 172.21.15.38:0/2957828601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd70e3b0 msgr2=0x5626fdae12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc7bfff640 1 --2- 172.21.15.38:0/2957828601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd70e3b0 0x5626fdae12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc7bfff640 1 -- 172.21.15.38:0/2957828601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd737270 msgr2=0x5626fdae8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc7bfff640 1 --2- 172.21.15.38:0/2957828601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd737270 0x5626fdae8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc7bfff640 1 -- 172.21.15.38:0/2957828601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626fd8196b0 con 0x5626fd716590 2024-05-21T15:34:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc80a5f640 1 --2- 172.21.15.38:0/2957828601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd70e3b0 0x5626fdae12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:04.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc7bfff640 1 --2- 172.21.15.38:0/2957828601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdae17e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc6400e9c0 tx=0x7efc64097c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8832baa35464d5d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:04.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc5bfff640 1 -- 172.21.15.38:0/2957828601 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6401c070 con 0x5626fd716590 2024-05-21T15:34:04.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc5bfff640 1 -- 172.21.15.38:0/2957828601 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7efc640d2e00 con 0x5626fd716590 2024-05-21T15:34:04.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc89b63740 1 -- 172.21.15.38:0/2957828601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc7400ed70 con 0x5626fd716590 2024-05-21T15:34:04.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc5bfff640 1 -- 172.21.15.38:0/2957828601 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efc6409bfc0 con 0x5626fd716590 2024-05-21T15:34:04.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc89b63740 10 client.15159 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:04.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc89b63740 1 -- 172.21.15.38:0/2957828601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626fdaeb740 con 0x5626fd716590 2024-05-21T15:34:04.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.967+0000 7efc5bfff640 1 -- 172.21.15.38:0/2957828601 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc7400ed70 con 0x5626fd716590 2024-05-21T15:34:04.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.968+0000 7efc5bfff640 1 -- 172.21.15.38:0/2957828601 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efc6409bb20 con 0x5626fd716590 2024-05-21T15:34:04.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.968+0000 7efc89b63740 10 client.15159 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:04.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.968+0000 7efc89b63740 10 client.15159 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T15:34:04.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.968+0000 7efc89b63740 20 client.15159 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:04.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.968+0000 7efc89b63740 1 --2- 172.21.15.38:0/2957828601 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5626fd6f5e40 0x5626fdaef130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.968+0000 7efc89b63740 4 client.15159 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:34:04.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.968+0000 7efc89b63740 1 -- 172.21.15.38:0/2957828601 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626fda48c00 con 0x5626fd6f5e40 2024-05-21T15:34:04.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.968+0000 7efc80a5f640 1 --2- 172.21.15.38:0/2957828601 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5626fd6f5e40 0x5626fdaef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.968+0000 7efc5bfff640 10 client.15159 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:04.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.968+0000 7efc80a5f640 1 --2- 172.21.15.38:0/2957828601 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5626fd6f5e40 0x5626fdaef130 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3119233a0e4b926d server_cookie=2275b73532c7baf7 in_seq=0 out_seq=0 2024-05-21T15:34:04.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.969+0000 7efc5bfff640 1 -- 172.21.15.38:0/2957828601 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5626fda48c00 con 0x5626fd6f5e40 2024-05-21T15:34:04.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.969+0000 7efc5bfff640 10 client.15159 handle_command_reply: tid=0 2024-05-21T15:34:04.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.969+0000 7efc89b63740 1 client.15159 shutdown 2024-05-21T15:34:04.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.969+0000 7efc89b63740 1 -- 172.21.15.38:0/2957828601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 msgr2=0x5626fdae17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.969+0000 7efc89b63740 1 --2- 172.21.15.38:0/2957828601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdae17e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc6400e9c0 tx=0x7efc64097c80 comp rx=0 tx=0).stop 2024-05-21T15:34:04.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.969+0000 7efc89b63740 1 -- 172.21.15.38:0/2957828601 shutdown_connections 2024-05-21T15:34:04.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.969+0000 7efc89b63740 1 --2- 172.21.15.38:0/2957828601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd737270 0x5626fdae8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.969+0000 7efc89b63740 1 --2- 172.21.15.38:0/2957828601 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5626fd6f5e40 0x5626fdaef130 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.969+0000 7efc89b63740 1 --2- 172.21.15.38:0/2957828601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdae17e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.969+0000 7efc5bfff640 0 client.15159 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:04.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.969+0000 7efc89b63740 1 --2- 172.21.15.38:0/2957828601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd70e3b0 0x5626fdae12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.969+0000 7efc89b63740 1 -- 172.21.15.38:0/2957828601 >> 172.21.15.38:0/2957828601 conn(0x5626fd72b3c0 msgr2=0x5626fdad1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.970+0000 7efc89b63740 1 -- 172.21.15.38:0/2957828601 shutdown_connections 2024-05-21T15:34:04.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.970+0000 7efc89b63740 1 -- 172.21.15.38:0/2957828601 wait complete. 2024-05-21T15:34:04.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.970+0000 7efc89b63740 20 client.15159 trim_cache size 0 max 16384 2024-05-21T15:34:04.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.988+0000 7efc89b63740 1 Processor -- start 2024-05-21T15:34:04.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.988+0000 7efc89b63740 1 -- start start 2024-05-21T15:34:04.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc89b63740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd70e3b0 0x5626fdadffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc89b63740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdae8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc89b63740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd737270 0x5626fdae8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc89b63740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5626fdaed470 con 0x5626fd716590 2024-05-21T15:34:04.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc89b63740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5626fdaedb40 con 0x5626fd70e3b0 2024-05-21T15:34:04.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc89b63740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5626fdaee210 con 0x5626fd737270 2024-05-21T15:34:04.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc7bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdae8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc7bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdae8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49970/0 (socket says 172.21.15.38:49970) 2024-05-21T15:34:04.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc7bfff640 1 -- 172.21.15.38:0/3698416850 learned_addr learned my addr 172.21.15.38:0/3698416850 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc80a5f640 1 --2- 172.21.15.38:0/3698416850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd70e3b0 0x5626fdadffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc81260640 1 --2- 172.21.15.38:0/3698416850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd737270 0x5626fdae8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc7bfff640 1 -- 172.21.15.38:0/3698416850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd737270 msgr2=0x5626fdae8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc7bfff640 1 --2- 172.21.15.38:0/3698416850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd737270 0x5626fdae8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc7bfff640 1 -- 172.21.15.38:0/3698416850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd70e3b0 msgr2=0x5626fdadffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc7bfff640 1 --2- 172.21.15.38:0/3698416850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd70e3b0 0x5626fdadffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc7bfff640 1 -- 172.21.15.38:0/3698416850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626fdad7020 con 0x5626fd716590 2024-05-21T15:34:04.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc7bfff640 1 --2- 172.21.15.38:0/3698416850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdae8660 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efc64098240 tx=0x7efc640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8132ddde603f0c3d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc727fc640 1 -- 172.21.15.38:0/3698416850 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6401c070 con 0x5626fd716590 2024-05-21T15:34:04.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc727fc640 1 -- 172.21.15.38:0/3698416850 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc64084e50 con 0x5626fd716590 2024-05-21T15:34:04.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.989+0000 7efc727fc640 1 -- 172.21.15.38:0/3698416850 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6409bb20 con 0x5626fd716590 2024-05-21T15:34:04.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.990+0000 7efc89b63740 1 -- 172.21.15.38:0/3698416850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 msgr2=0x5626fdae8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.990+0000 7efc89b63740 1 --2- 172.21.15.38:0/3698416850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdae8660 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efc64098240 tx=0x7efc640046c0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.990+0000 7efc89b63740 1 -- 172.21.15.38:0/3698416850 shutdown_connections 2024-05-21T15:34:04.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.990+0000 7efc89b63740 1 --2- 172.21.15.38:0/3698416850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd737270 0x5626fdae8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.990+0000 7efc89b63740 1 --2- 172.21.15.38:0/3698416850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd716590 0x5626fdae8660 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.990+0000 7efc89b63740 1 --2- 172.21.15.38:0/3698416850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd70e3b0 0x5626fdadffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.990+0000 7efc89b63740 1 -- 172.21.15.38:0/3698416850 >> 172.21.15.38:0/3698416850 conn(0x5626fd72b3c0 msgr2=0x5626fdadf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:04.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.990+0000 7efc89b63740 1 -- 172.21.15.38:0/3698416850 shutdown_connections 2024-05-21T15:34:04.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.990+0000 7efc89b63740 1 -- 172.21.15.38:0/3698416850 wait complete. 2024-05-21T15:34:04.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.990+0000 7efc89b63740 1 -- start start 2024-05-21T15:34:04.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc89b63740 1 Processor -- start 2024-05-21T15:34:04.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc89b63740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd70e3b0 0x5626fdae44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc89b63740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd716590 0x5626fdae4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc89b63740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd737270 0x5626fdae7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc89b63740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5626fdaeefb0 con 0x5626fd737270 2024-05-21T15:34:04.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc89b63740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5626fdaee8e0 con 0x5626fd70e3b0 2024-05-21T15:34:04.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc89b63740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5626fdaef680 con 0x5626fd716590 2024-05-21T15:34:04.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc89b63740 10 client.0 _ll_register_callbacks cb 0x7efc640053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:04.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc81260640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd737270 0x5626fdae7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc80a5f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd70e3b0 0x5626fdae44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc81260640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd737270 0x5626fdae7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49980/0 (socket says 172.21.15.38:49980) 2024-05-21T15:34:04.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc81260640 1 -- 172.21.15.38:0/3196585270 learned_addr learned my addr 172.21.15.38:0/3196585270 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:04.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc7bfff640 1 --2- 172.21.15.38:0/3196585270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd716590 0x5626fdae4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc80a5f640 1 -- 172.21.15.38:0/3196585270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd716590 msgr2=0x5626fdae4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc80a5f640 1 --2- 172.21.15.38:0/3196585270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd716590 0x5626fdae4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc80a5f640 1 -- 172.21.15.38:0/3196585270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd737270 msgr2=0x5626fdae7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:04.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc80a5f640 1 --2- 172.21.15.38:0/3196585270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd737270 0x5626fdae7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc80a5f640 1 -- 172.21.15.38:0/3196585270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626fdae7f20 con 0x5626fd70e3b0 2024-05-21T15:34:04.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.991+0000 7efc81260640 1 --2- 172.21.15.38:0/3196585270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd737270 0x5626fdae7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:34:04.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.992+0000 7efc80a5f640 1 --2- 172.21.15.38:0/3196585270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd70e3b0 0x5626fdae44e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efc6c005ab0 tx=0x7efc6c0052b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b6738f3bdfcb6e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:04.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.992+0000 7efc5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:34:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.992+0000 7efc5bfff640 1 -- 172.21.15.38:0/3196585270 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c035040 con 0x5626fd70e3b0 2024-05-21T15:34:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.992+0000 7efc89b63740 1 -- 172.21.15.38:0/3196585270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc64084e50 con 0x5626fd70e3b0 2024-05-21T15:34:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.992+0000 7efc5bfff640 1 -- 172.21.15.38:0/3196585270 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7efc6c004940 con 0x5626fd70e3b0 2024-05-21T15:34:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.992+0000 7efc5bfff640 1 -- 172.21.15.38:0/3196585270 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efc6c004b40 con 0x5626fd70e3b0 2024-05-21T15:34:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.992+0000 7efc89b63740 10 client.24995 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.992+0000 7efc89b63740 1 -- 172.21.15.38:0/3196585270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626fdae9260 con 0x5626fd70e3b0 2024-05-21T15:34:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.992+0000 7efc5bfff640 1 -- 172.21.15.38:0/3196585270 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc64084e50 con 0x5626fd70e3b0 2024-05-21T15:34:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc5bfff640 1 -- 172.21.15.38:0/3196585270 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efc6c045290 con 0x5626fd70e3b0 2024-05-21T15:34:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc89b63740 10 client.24995 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc89b63740 10 client.24995 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T15:34:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc89b63740 20 client.24995 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc89b63740 1 --2- 172.21.15.38:0/3196585270 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5626fd6cb280 0x5626fdb5b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc89b63740 4 client.24995 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:34:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc89b63740 1 -- 172.21.15.38:0/3196585270 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "status"}) -- 0x5626fda42f50 con 0x5626fd6cb280 2024-05-21T15:34:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc7bfff640 1 --2- 172.21.15.38:0/3196585270 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5626fd6cb280 0x5626fdb5b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc7bfff640 1 --2- 172.21.15.38:0/3196585270 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5626fd6cb280 0x5626fdb5b680 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f03a1be12d20081 server_cookie=de8450fc07b4133a in_seq=0 out_seq=0 2024-05-21T15:34:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc5bfff640 10 client.24995 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc5bfff640 1 -- 172.21.15.38:0/3196585270 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5626fda42f50 con 0x5626fd6cb280 2024-05-21T15:34:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc5bfff640 10 client.24995 handle_command_reply: tid=0 2024-05-21T15:34:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc89b63740 1 client.24995 shutdown 2024-05-21T15:34:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc89b63740 1 -- 172.21.15.38:0/3196585270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd70e3b0 msgr2=0x5626fdae44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.993+0000 7efc89b63740 1 --2- 172.21.15.38:0/3196585270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd70e3b0 0x5626fdae44e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efc6c005ab0 tx=0x7efc6c0052b0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc89b63740 1 -- 172.21.15.38:0/3196585270 shutdown_connections 2024-05-21T15:34:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc89b63740 1 --2- 172.21.15.38:0/3196585270 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5626fd6cb280 0x5626fdb5b680 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc89b63740 1 --2- 172.21.15.38:0/3196585270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5626fd737270 0x5626fdae7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc89b63740 1 --2- 172.21.15.38:0/3196585270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5626fd716590 0x5626fdae4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc5bfff640 0 client.24995 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:05.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc89b63740 1 --2- 172.21.15.38:0/3196585270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626fd70e3b0 0x5626fdae44e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:05.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc89b63740 1 -- 172.21.15.38:0/3196585270 >> 172.21.15.38:0/3196585270 conn(0x5626fd72b3c0 msgr2=0x5626fdad27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:05.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc89b63740 1 -- 172.21.15.38:0/3196585270 shutdown_connections 2024-05-21T15:34:05.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc89b63740 1 -- 172.21.15.38:0/3196585270 wait complete. 2024-05-21T15:34:05.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc89b63740 20 client.24995 trim_cache size 0 max 16384 2024-05-21T15:34:05.000 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:34:05.000 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "102ec4fa-1786-11ef-bc9a-c7b262605968", 2024-05-21T15:34:05.000 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-05-21T15:34:05.000 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24602, 2024-05-21T15:34:05.000 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-05-21T15:34:05.000 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-05-21T15:34:05.001 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-05-21T15:34:05.001 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 90.783662824999993, 2024-05-21T15:34:05.001 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-05-21T15:34:05.001 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-05-21T15:34:05.001 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-05-21T15:34:05.001 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 95.933463048999997 2024-05-21T15:34:05.001 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:34:05.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc82cc3640 1 -- 172.21.15.38:0/1044117268 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efc680a2b90 msgr2=0x7efc680a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:05.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc82cc3640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efc680a2b90 0x7efc680a50a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc74021000 tx=0x7efc7401a000 comp rx=0 tx=0).stop 2024-05-21T15:34:05.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc82cc3640 1 -- 172.21.15.38:0/1044117268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc7c104120 msgr2=0x7efc7c1998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:05.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc82cc3640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc7c104120 0x7efc7c1998f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc6400e9f0 tx=0x7efc6400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:34:05.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc82cc3640 1 -- 172.21.15.38:0/1044117268 shutdown_connections 2024-05-21T15:34:05.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc82cc3640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7c108a20 0x7efc7c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:05.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc82cc3640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc7c104120 0x7efc7c1998f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:05.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc82cc3640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efc680a2b90 0x7efc680a50a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:05.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc82cc3640 1 --2- 172.21.15.38:0/1044117268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc7c1027e0 0x7efc7c1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc82cc3640 1 -- 172.21.15.38:0/1044117268 >> 172.21.15.38:0/1044117268 conn(0x7efc7c0fdcf0 msgr2=0x7efc7c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc82cc3640 1 -- 172.21.15.38:0/1044117268 shutdown_connections 2024-05-21T15:34:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:04.994+0000 7efc82cc3640 1 -- 172.21.15.38:0/1044117268 wait complete. 2024-05-21T15:34:05.005 INFO:tasks.cephadm:Setup complete, yielding 2024-05-21T15:34:05.006 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-21T15:34:05.016 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-05-21T15:34:05.017 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch status' 2024-05-21T15:34:05.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:04 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1567685085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:05.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:04 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1567685085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:05.264 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:34:05.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:04 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1567685085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:06.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:05 smithi038 ceph-mon[31995]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T15:34:06.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:05 smithi053 ceph-mon[36496]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T15:34:06.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:05 smithi045 ceph-mon[36409]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T15:34:07.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.107+0000 7f7e968a2700 1 -- 172.21.15.38:0/1447217100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e90103ba0 msgr2=0x7f7e9010a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:07.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.107+0000 7f7e968a2700 1 --2- 172.21.15.38:0/1447217100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e90103ba0 0x7f7e9010a450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800b0a0 tx=0x7f7e8800b3b0 comp rx=0 tx=0).stop 2024-05-21T15:34:07.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.107+0000 7f7e968a2700 1 -- 172.21.15.38:0/1447217100 shutdown_connections 2024-05-21T15:34:07.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.107+0000 7f7e968a2700 1 --2- 172.21.15.38:0/1447217100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e90103ba0 0x7f7e9010a450 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:07.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.107+0000 7f7e968a2700 1 --2- 172.21.15.38:0/1447217100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e901020d0 0x7f7e901024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:07.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.107+0000 7f7e968a2700 1 --2- 172.21.15.38:0/1447217100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e90103200 0x7f7e90103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:07.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.107+0000 7f7e968a2700 1 -- 172.21.15.38:0/1447217100 >> 172.21.15.38:0/1447217100 conn(0x7f7e900fd710 msgr2=0x7f7e900ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:07.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.107+0000 7f7e968a2700 1 -- 172.21.15.38:0/1447217100 shutdown_connections 2024-05-21T15:34:07.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.107+0000 7f7e968a2700 1 -- 172.21.15.38:0/1447217100 wait complete. 2024-05-21T15:34:07.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.107+0000 7f7e968a2700 1 Processor -- start 2024-05-21T15:34:07.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.107+0000 7f7e968a2700 1 -- start start 2024-05-21T15:34:07.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e968a2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e901020d0 0x7f7e90198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:07.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e968a2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e90103200 0x7f7e90198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:07.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e968a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e90103ba0 0x7f7e9019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:07.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e968a2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9010dca0 con 0x7f7e901020d0 2024-05-21T15:34:07.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e968a2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9010db20 con 0x7f7e90103ba0 2024-05-21T15:34:07.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e968a2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9010de20 con 0x7f7e90103200 2024-05-21T15:34:07.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e901020d0 0x7f7e90198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:07.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e901020d0 0x7f7e90198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50004/0 (socket says 172.21.15.38:50004) 2024-05-21T15:34:07.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e8ffff700 1 -- 172.21.15.38:0/3214731 learned_addr learned my addr 172.21.15.38:0/3214731 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:07.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e8f7fe700 1 --2- 172.21.15.38:0/3214731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e90103200 0x7f7e90198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:07.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e94e3f700 1 --2- 172.21.15.38:0/3214731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e90103ba0 0x7f7e9019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:07.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e8ffff700 1 -- 172.21.15.38:0/3214731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e90103200 msgr2=0x7f7e90198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:07.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e8ffff700 1 --2- 172.21.15.38:0/3214731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e90103200 0x7f7e90198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:07.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e8ffff700 1 -- 172.21.15.38:0/3214731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e90103ba0 msgr2=0x7f7e9019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:07.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e8ffff700 1 --2- 172.21.15.38:0/3214731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e90103ba0 0x7f7e9019fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:07.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.108+0000 7f7e8ffff700 1 -- 172.21.15.38:0/3214731 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e901a0260 con 0x7f7e901020d0 2024-05-21T15:34:07.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.109+0000 7f7e8ffff700 1 --2- 172.21.15.38:0/3214731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e901020d0 0x7f7e90198750 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7e800098c0 tx=0x7f7e80009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:07.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.109+0000 7f7e8d7fa700 1 -- 172.21.15.38:0/3214731 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000d020 con 0x7f7e901020d0 2024-05-21T15:34:07.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.109+0000 7f7e968a2700 1 -- 172.21.15.38:0/3214731 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e900748d0 con 0x7f7e901020d0 2024-05-21T15:34:07.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.109+0000 7f7e8d7fa700 1 -- 172.21.15.38:0/3214731 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e80004500 con 0x7f7e901020d0 2024-05-21T15:34:07.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.109+0000 7f7e8d7fa700 1 -- 172.21.15.38:0/3214731 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80003680 con 0x7f7e901020d0 2024-05-21T15:34:07.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.109+0000 7f7e968a2700 1 -- 172.21.15.38:0/3214731 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e90074e10 con 0x7f7e901020d0 2024-05-21T15:34:07.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.110+0000 7f7e968a2700 1 -- 172.21.15.38:0/3214731 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e90050170 con 0x7f7e901020d0 2024-05-21T15:34:07.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.111+0000 7f7e8d7fa700 1 -- 172.21.15.38:0/3214731 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7e8001f020 con 0x7f7e901020d0 2024-05-21T15:34:07.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.112+0000 7f7e8d7fa700 1 --2- 172.21.15.38:0/3214731 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e780a2c20 0x7f7e780a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:07.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.112+0000 7f7e8f7fe700 1 --2- 172.21.15.38:0/3214731 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e780a2c20 0x7f7e780a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:07.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.112+0000 7f7e8d7fa700 1 -- 172.21.15.38:0/3214731 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7e800d0c00 con 0x7f7e901020d0 2024-05-21T15:34:07.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.113+0000 7f7e8f7fe700 1 --2- 172.21.15.38:0/3214731 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e780a2c20 0x7f7e780a5130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7e84005950 tx=0x7f7e840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:07.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.116+0000 7f7e8d7fa700 1 -- 172.21.15.38:0/3214731 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7e80026030 con 0x7f7e901020d0 2024-05-21T15:34:07.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.343+0000 7f7e968a2700 1 -- 172.21.15.38:0/3214731 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f7e90075070 con 0x7f7e780a2c20 2024-05-21T15:34:07.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.344+0000 7f7e8d7fa700 1 -- 172.21.15.38:0/3214731 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f7e90075070 con 0x7f7e780a2c20 2024-05-21T15:34:07.344 INFO:teuthology.orchestra.run.smithi038.stdout:Backend: cephadm 2024-05-21T15:34:07.344 INFO:teuthology.orchestra.run.smithi038.stdout:Available: Yes 2024-05-21T15:34:07.344 INFO:teuthology.orchestra.run.smithi038.stdout:Paused: No 2024-05-21T15:34:07.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.346+0000 7f7e968a2700 1 -- 172.21.15.38:0/3214731 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e780a2c20 msgr2=0x7f7e780a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:07.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.346+0000 7f7e968a2700 1 --2- 172.21.15.38:0/3214731 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e780a2c20 0x7f7e780a5130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7e84005950 tx=0x7f7e840058e0 comp rx=0 tx=0).stop 2024-05-21T15:34:07.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.346+0000 7f7e968a2700 1 -- 172.21.15.38:0/3214731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e901020d0 msgr2=0x7f7e90198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:07.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.346+0000 7f7e968a2700 1 --2- 172.21.15.38:0/3214731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e901020d0 0x7f7e90198750 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7e800098c0 tx=0x7f7e80009c80 comp rx=0 tx=0).stop 2024-05-21T15:34:07.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.347+0000 7f7e968a2700 1 -- 172.21.15.38:0/3214731 shutdown_connections 2024-05-21T15:34:07.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.347+0000 7f7e968a2700 1 --2- 172.21.15.38:0/3214731 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e780a2c20 0x7f7e780a5130 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:07.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.347+0000 7f7e968a2700 1 --2- 172.21.15.38:0/3214731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e90103ba0 0x7f7e9019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:07.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.347+0000 7f7e968a2700 1 --2- 172.21.15.38:0/3214731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e901020d0 0x7f7e90198750 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:07.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.347+0000 7f7e968a2700 1 --2- 172.21.15.38:0/3214731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e90103200 0x7f7e90198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:07.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.347+0000 7f7e968a2700 1 -- 172.21.15.38:0/3214731 >> 172.21.15.38:0/3214731 conn(0x7f7e900fd710 msgr2=0x7f7e90106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:07.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.347+0000 7f7e968a2700 1 -- 172.21.15.38:0/3214731 shutdown_connections 2024-05-21T15:34:07.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:07.347+0000 7f7e968a2700 1 -- 172.21.15.38:0/3214731 wait complete. 2024-05-21T15:34:08.119 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-21T15:34:08.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:07 smithi053 ceph-mon[36496]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-21T15:34:08.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:07 smithi045 ceph-mon[36409]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-21T15:34:08.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:07 smithi038 ceph-mon[31995]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:34:09.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:08 smithi038 ceph-mon[31995]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:09.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:08 smithi053 ceph-mon[36496]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:09.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:08 smithi045 ceph-mon[36409]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:10.195 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:09 smithi038 ceph-mon[31995]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-21T15:34:10.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:09 smithi053 ceph-mon[36496]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-21T15:34:10.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.218+0000 7f014fe09700 1 -- 172.21.15.38:0/1265551731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0148103b90 msgr2=0x7f014810a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:10.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.218+0000 7f014fe09700 1 --2- 172.21.15.38:0/1265551731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0148103b90 0x7f014810a440 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f014000b0a0 tx=0x7f014000b3b0 comp rx=0 tx=0).stop 2024-05-21T15:34:10.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.218+0000 7f014fe09700 1 -- 172.21.15.38:0/1265551731 shutdown_connections 2024-05-21T15:34:10.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.218+0000 7f014fe09700 1 --2- 172.21.15.38:0/1265551731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0148103b90 0x7f014810a440 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:10.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.218+0000 7f014fe09700 1 --2- 172.21.15.38:0/1265551731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01481020c0 0x7f01481024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:10.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.218+0000 7f014fe09700 1 --2- 172.21.15.38:0/1265551731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01481031f0 0x7f0148103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:10.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.218+0000 7f014fe09700 1 -- 172.21.15.38:0/1265551731 >> 172.21.15.38:0/1265551731 conn(0x7f01480fd700 msgr2=0x7f01480ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:10.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.218+0000 7f014fe09700 1 -- 172.21.15.38:0/1265551731 shutdown_connections 2024-05-21T15:34:10.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.218+0000 7f014fe09700 1 -- 172.21.15.38:0/1265551731 wait complete. 2024-05-21T15:34:10.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.218+0000 7f014fe09700 1 Processor -- start 2024-05-21T15:34:10.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.218+0000 7f014fe09700 1 -- start start 2024-05-21T15:34:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014fe09700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01481020c0 0x7f014806aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014fe09700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01481031f0 0x7f014819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014fe09700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0148103b90 0x7f01481a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014fe09700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f014810dd00 con 0x7f01481020c0 2024-05-21T15:34:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014fe09700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f014810db80 con 0x7f01481031f0 2024-05-21T15:34:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014fe09700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f014810de80 con 0x7f0148103b90 2024-05-21T15:34:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014dba5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01481020c0 0x7f014806aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014dba5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01481020c0 0x7f014806aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50024/0 (socket says 172.21.15.38:50024) 2024-05-21T15:34:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014dba5700 1 -- 172.21.15.38:0/359703008 learned_addr learned my addr 172.21.15.38:0/359703008 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014e3a6700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0148103b90 0x7f01481a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:10.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014d3a4700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01481031f0 0x7f014819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014dba5700 1 -- 172.21.15.38:0/359703008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0148103b90 msgr2=0x7f01481a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014dba5700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0148103b90 0x7f01481a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014dba5700 1 -- 172.21.15.38:0/359703008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01481031f0 msgr2=0x7f014819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014dba5700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01481031f0 0x7f014819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014dba5700 1 -- 172.21.15.38:0/359703008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01481a13c0 con 0x7f01481020c0 2024-05-21T15:34:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014d3a4700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01481031f0 0x7f014819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.219+0000 7f014dba5700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01481020c0 0x7f014806aef0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f013c00c970 tx=0x7f013c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.220+0000 7f013affd700 1 -- 172.21.15.38:0/359703008 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c00ce60 con 0x7f01481020c0 2024-05-21T15:34:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.220+0000 7f014fe09700 1 -- 172.21.15.38:0/359703008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01481a16b0 con 0x7f01481020c0 2024-05-21T15:34:10.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.220+0000 7f013affd700 1 -- 172.21.15.38:0/359703008 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f013c007de0 con 0x7f01481020c0 2024-05-21T15:34:10.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.220+0000 7f013affd700 1 -- 172.21.15.38:0/359703008 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c003d00 con 0x7f01481020c0 2024-05-21T15:34:10.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.220+0000 7f014fe09700 1 -- 172.21.15.38:0/359703008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01481a1c40 con 0x7f01481020c0 2024-05-21T15:34:10.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.221+0000 7f014fe09700 1 -- 172.21.15.38:0/359703008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0148050170 con 0x7f01481020c0 2024-05-21T15:34:10.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.222+0000 7f013affd700 1 -- 172.21.15.38:0/359703008 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f013c007900 con 0x7f01481020c0 2024-05-21T15:34:10.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.223+0000 7f013affd700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f01340a2c20 0x7f01340a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:10.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.223+0000 7f014d3a4700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f01340a2c20 0x7f01340a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:10.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.223+0000 7f013affd700 1 -- 172.21.15.38:0/359703008 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f013c0ceff0 con 0x7f01481020c0 2024-05-21T15:34:10.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.223+0000 7f014d3a4700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f01340a2c20 0x7f01340a5130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f014806a8a0 tx=0x7f0144008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:10.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.226+0000 7f013affd700 1 -- 172.21.15.38:0/359703008 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f013c014030 con 0x7f01481020c0 2024-05-21T15:34:10.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:09 smithi045 ceph-mon[36409]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-21T15:34:10.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.442+0000 7f014fe09700 1 -- 172.21.15.38:0/359703008 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0148062a80 con 0x7f01340a2c20 2024-05-21T15:34:10.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.456+0000 7f013affd700 1 -- 172.21.15.38:0/359703008 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f0148062a80 con 0x7f01340a2c20 2024-05-21T15:34:10.456 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-21T15:34:10.456 INFO:teuthology.orchestra.run.smithi038.stdout:mds.a smithi038 running (104s) 18s ago 104s 84.4M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 0ddaa288b363 2024-05-21T15:34:10.456 INFO:teuthology.orchestra.run.smithi038.stdout:mds.b smithi045 running (101s) 34s ago 101s 92.3M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 4d4648d5453d 2024-05-21T15:34:10.456 INFO:teuthology.orchestra.run.smithi038.stdout:mds.c smithi053 running (116s) 34s ago 115s 85.0M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 d992e7fc917d 2024-05-21T15:34:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:mds.d smithi038 running (2m) 18s ago 2m 81.6M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 a77c7155721b 2024-05-21T15:34:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:mds.e smithi045 running (2m) 34s ago 2m 87.3M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 d12932c6e8e9 2024-05-21T15:34:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:mds.f smithi053 running (119s) 34s ago 118s 84.3M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 22b42be6955b 2024-05-21T15:34:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:mds.g smithi038 running (107s) 18s ago 107s 84.4M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 24307d2a9a6a 2024-05-21T15:34:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:mds.h smithi045 running (113s) 34s ago 113s 85.2M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 d464e15f31c2 2024-05-21T15:34:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:mds.i smithi053 running (110s) 34s ago 110s 87.5M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 80998c82a686 2024-05-21T15:34:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:mds.j smithi038 running (2m) 18s ago 2m 81.5M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 e998518fd25d 2024-05-21T15:34:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:mds.k smithi045 running (2m) 34s ago 2m 84.3M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 cf789efac309 2024-05-21T15:34:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:mds.l smithi053 running (2m) 34s ago 2m 83.3M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 ea5eb1669df5 2024-05-21T15:34:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.x smithi038 *:9283,8765 running (9m) 18s ago 9m 583M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 f79809031e3a 2024-05-21T15:34:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.y smithi045 *:8443,8765 running (7m) 34s ago 7m 521M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 65f2b4e1696a 2024-05-21T15:34:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.z smithi053 *:8443,8765 running (7m) 34s ago 7m 520M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 75e9738b9c6d 2024-05-21T15:34:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:mon.a smithi038 running (9m) 18s ago 9m 116M 2048M 19.0.0-2744-gd2b507c7 cfcc61b2d476 df34e728985d 2024-05-21T15:34:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:mon.b smithi045 running (7m) 34s ago 7m 105M 2048M 19.0.0-2744-gd2b507c7 cfcc61b2d476 fb1f297bbcde 2024-05-21T15:34:10.458 INFO:teuthology.orchestra.run.smithi038.stdout:mon.c smithi053 running (8m) 34s ago 8m 107M 2048M 19.0.0-2744-gd2b507c7 cfcc61b2d476 a6c1844a3a25 2024-05-21T15:34:10.458 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (7m) 18s ago 7m 123M 4273M 19.0.0-2744-gd2b507c7 cfcc61b2d476 3c2b383373b2 2024-05-21T15:34:10.458 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi045 running (6m) 34s ago 6m 112M 4273M 19.0.0-2744-gd2b507c7 cfcc61b2d476 173c9125b57e 2024-05-21T15:34:10.458 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi053 running (6m) 34s ago 6m 113M 4273M 19.0.0-2744-gd2b507c7 cfcc61b2d476 ba771b3286d0 2024-05-21T15:34:10.458 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi038 running (6m) 18s ago 6m 123M 4273M 19.0.0-2744-gd2b507c7 cfcc61b2d476 6f372c2fb2d1 2024-05-21T15:34:10.458 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi045 running (5m) 34s ago 5m 122M 4273M 19.0.0-2744-gd2b507c7 cfcc61b2d476 6f86f02149c3 2024-05-21T15:34:10.458 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi053 running (5m) 34s ago 5m 124M 4273M 19.0.0-2744-gd2b507c7 cfcc61b2d476 54053abdd0d3 2024-05-21T15:34:10.458 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 smithi038 running (4m) 18s ago 4m 108M 4273M 19.0.0-2744-gd2b507c7 cfcc61b2d476 6a097f6fd6fe 2024-05-21T15:34:10.458 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 smithi045 running (4m) 34s ago 4m 114M 4273M 19.0.0-2744-gd2b507c7 cfcc61b2d476 469d0cab7f74 2024-05-21T15:34:10.459 INFO:teuthology.orchestra.run.smithi038.stdout:osd.8 smithi053 running (3m) 34s ago 3m 120M 4273M 19.0.0-2744-gd2b507c7 cfcc61b2d476 d2c2e989f77e 2024-05-21T15:34:10.459 INFO:teuthology.orchestra.run.smithi038.stdout:osd.9 smithi038 running (3m) 18s ago 3m 119M 4273M 19.0.0-2744-gd2b507c7 cfcc61b2d476 6be7680a5c9b 2024-05-21T15:34:10.459 INFO:teuthology.orchestra.run.smithi038.stdout:osd.10 smithi045 running (2m) 34s ago 2m 112M 4273M 19.0.0-2744-gd2b507c7 cfcc61b2d476 f54f4ab9db6e 2024-05-21T15:34:10.459 INFO:teuthology.orchestra.run.smithi038.stdout:osd.11 smithi053 running (2m) 34s ago 2m 107M 4273M 19.0.0-2744-gd2b507c7 cfcc61b2d476 8067a1c59cc5 2024-05-21T15:34:10.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.459+0000 7f014fe09700 1 -- 172.21.15.38:0/359703008 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f01340a2c20 msgr2=0x7f01340a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:10.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.459+0000 7f014fe09700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f01340a2c20 0x7f01340a5130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f014806a8a0 tx=0x7f0144008040 comp rx=0 tx=0).stop 2024-05-21T15:34:10.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.459+0000 7f014fe09700 1 -- 172.21.15.38:0/359703008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01481020c0 msgr2=0x7f014806aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:10.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.459+0000 7f014fe09700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01481020c0 0x7f014806aef0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f013c00c970 tx=0x7f013c00cc80 comp rx=0 tx=0).stop 2024-05-21T15:34:10.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.459+0000 7f014fe09700 1 -- 172.21.15.38:0/359703008 shutdown_connections 2024-05-21T15:34:10.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.459+0000 7f014fe09700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f01340a2c20 0x7f01340a5130 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:10.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.459+0000 7f014fe09700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0148103b90 0x7f01481a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:10.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.459+0000 7f014fe09700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01481020c0 0x7f014806aef0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:10.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.459+0000 7f014fe09700 1 --2- 172.21.15.38:0/359703008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01481031f0 0x7f014819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:10.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.459+0000 7f014fe09700 1 -- 172.21.15.38:0/359703008 >> 172.21.15.38:0/359703008 conn(0x7f01480fd700 msgr2=0x7f0148106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:10.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.459+0000 7f014fe09700 1 -- 172.21.15.38:0/359703008 shutdown_connections 2024-05-21T15:34:10.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:10.459+0000 7f014fe09700 1 -- 172.21.15.38:0/359703008 wait complete. 2024-05-21T15:34:11.155 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-21T15:34:11.411 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:34:12.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:11 smithi053 ceph-mon[36496]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:34:12.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:11 smithi053 ceph-mon[36496]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:12.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:11 smithi045 ceph-mon[36409]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:34:12.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:11 smithi045 ceph-mon[36409]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:12.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:11 smithi038 ceph-mon[31995]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:34:12.277 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:11 smithi038 ceph-mon[31995]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.229+0000 7f4c2deab700 1 -- 172.21.15.38:0/4277330439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c28103200 msgr2=0x7f4c28103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.229+0000 7f4c2deab700 1 --2- 172.21.15.38:0/4277330439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c28103200 0x7f4c28103610 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4c18009990 tx=0x7f4c18009ca0 comp rx=0 tx=0).stop 2024-05-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.229+0000 7f4c2deab700 1 -- 172.21.15.38:0/4277330439 shutdown_connections 2024-05-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.229+0000 7f4c2deab700 1 --2- 172.21.15.38:0/4277330439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c28103ba0 0x7f4c2810a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.229+0000 7f4c2deab700 1 --2- 172.21.15.38:0/4277330439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c281020d0 0x7f4c281024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.229+0000 7f4c2deab700 1 --2- 172.21.15.38:0/4277330439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c28103200 0x7f4c28103610 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.229+0000 7f4c2deab700 1 -- 172.21.15.38:0/4277330439 >> 172.21.15.38:0/4277330439 conn(0x7f4c280fd710 msgr2=0x7f4c280ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.229+0000 7f4c2deab700 1 -- 172.21.15.38:0/4277330439 shutdown_connections 2024-05-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.229+0000 7f4c2deab700 1 -- 172.21.15.38:0/4277330439 wait complete. 2024-05-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.230+0000 7f4c2deab700 1 Processor -- start 2024-05-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.230+0000 7f4c2deab700 1 -- start start 2024-05-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.230+0000 7f4c2deab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c281020d0 0x7f4c28199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.230+0000 7f4c2deab700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c28103200 0x7f4c28199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.230+0000 7f4c2deab700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c28103ba0 0x7f4c281a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.230+0000 7f4c2deab700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2810dc90 con 0x7f4c28103ba0 2024-05-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.230+0000 7f4c2deab700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2810db10 con 0x7f4c281020d0 2024-05-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.230+0000 7f4c2deab700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2810de10 con 0x7f4c28103200 2024-05-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.230+0000 7f4c27fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c28103ba0 0x7f4c281a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.230+0000 7f4c27fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c28103ba0 0x7f4c281a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50052/0 (socket says 172.21.15.38:50052) 2024-05-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.230+0000 7f4c27fff700 1 -- 172.21.15.38:0/3114657327 learned_addr learned my addr 172.21.15.38:0/3114657327 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.231+0000 7f4c26ffd700 1 --2- 172.21.15.38:0/3114657327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c28103200 0x7f4c28199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.231+0000 7f4c277fe700 1 --2- 172.21.15.38:0/3114657327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c281020d0 0x7f4c28199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.231+0000 7f4c27fff700 1 -- 172.21.15.38:0/3114657327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c28103200 msgr2=0x7f4c28199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.231+0000 7f4c27fff700 1 --2- 172.21.15.38:0/3114657327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c28103200 0x7f4c28199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.231+0000 7f4c27fff700 1 -- 172.21.15.38:0/3114657327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c281020d0 msgr2=0x7f4c28199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.231+0000 7f4c27fff700 1 --2- 172.21.15.38:0/3114657327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c281020d0 0x7f4c28199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.231+0000 7f4c27fff700 1 -- 172.21.15.38:0/3114657327 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c281a13d0 con 0x7f4c28103ba0 2024-05-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.231+0000 7f4c27fff700 1 --2- 172.21.15.38:0/3114657327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c28103ba0 0x7f4c281a0d50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4c20007ee0 tx=0x7f4c2000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.231+0000 7f4c24ff9700 1 -- 172.21.15.38:0/3114657327 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20004bb0 con 0x7f4c28103ba0 2024-05-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.231+0000 7f4c24ff9700 1 -- 172.21.15.38:0/3114657327 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c20004d50 con 0x7f4c28103ba0 2024-05-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.231+0000 7f4c2deab700 1 -- 172.21.15.38:0/3114657327 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c281a16c0 con 0x7f4c28103ba0 2024-05-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.231+0000 7f4c24ff9700 1 -- 172.21.15.38:0/3114657327 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20005960 con 0x7f4c28103ba0 2024-05-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.231+0000 7f4c2deab700 1 -- 172.21.15.38:0/3114657327 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c281a1bb0 con 0x7f4c28103ba0 2024-05-21T15:34:13.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.232+0000 7f4c2deab700 1 -- 172.21.15.38:0/3114657327 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c28050170 con 0x7f4c28103ba0 2024-05-21T15:34:13.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.234+0000 7f4c24ff9700 1 -- 172.21.15.38:0/3114657327 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4c2001a020 con 0x7f4c28103ba0 2024-05-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.234+0000 7f4c24ff9700 1 --2- 172.21.15.38:0/3114657327 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4c140a2c20 0x7f4c140a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.234+0000 7f4c277fe700 1 --2- 172.21.15.38:0/3114657327 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4c140a2c20 0x7f4c140a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.234+0000 7f4c24ff9700 1 -- 172.21.15.38:0/3114657327 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f4c200ce790 con 0x7f4c28103ba0 2024-05-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.235+0000 7f4c277fe700 1 --2- 172.21.15.38:0/3114657327 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4c140a2c20 0x7f4c140a5130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4c28103100 tx=0x7f4c1000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:13.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.239+0000 7f4c24ff9700 1 -- 172.21.15.38:0/3114657327 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4c20010070 con 0x7f4c28103ba0 2024-05-21T15:34:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.456+0000 7f4c2deab700 1 -- 172.21.15.38:0/3114657327 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f4c280ff6a0 con 0x7f4c140a2c20 2024-05-21T15:34:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.462+0000 7f4c24ff9700 1 -- 172.21.15.38:0/3114657327 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f4c280ff6a0 con 0x7f4c140a2c20 2024-05-21T15:34:13.462 INFO:teuthology.orchestra.run.smithi038.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-21T15:34:13.462 INFO:teuthology.orchestra.run.smithi038.stdout:mds.all 12/12 37s ago 2m smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l;count:12 2024-05-21T15:34:13.463 INFO:teuthology.orchestra.run.smithi038.stdout:mgr 3/3 37s ago 7m smithi038=x;smithi045=y;smithi053=z;count:3 2024-05-21T15:34:13.463 INFO:teuthology.orchestra.run.smithi038.stdout:mon 3/3 37s ago 8m smithi038:172.21.15.38=a;smithi045:172.21.15.45=b;smithi053:172.21.15.53=c;count:3 2024-05-21T15:34:13.463 INFO:teuthology.orchestra.run.smithi038.stdout:osd 12 37s ago - 2024-05-21T15:34:13.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.465+0000 7f4c2deab700 1 -- 172.21.15.38:0/3114657327 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4c140a2c20 msgr2=0x7f4c140a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:13.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.465+0000 7f4c2deab700 1 --2- 172.21.15.38:0/3114657327 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4c140a2c20 0x7f4c140a5130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4c28103100 tx=0x7f4c1000b410 comp rx=0 tx=0).stop 2024-05-21T15:34:13.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.465+0000 7f4c2deab700 1 -- 172.21.15.38:0/3114657327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c28103ba0 msgr2=0x7f4c281a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:13.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.465+0000 7f4c2deab700 1 --2- 172.21.15.38:0/3114657327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c28103ba0 0x7f4c281a0d50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4c20007ee0 tx=0x7f4c2000d3b0 comp rx=0 tx=0).stop 2024-05-21T15:34:13.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.465+0000 7f4c2deab700 1 -- 172.21.15.38:0/3114657327 shutdown_connections 2024-05-21T15:34:13.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.465+0000 7f4c2deab700 1 --2- 172.21.15.38:0/3114657327 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4c140a2c20 0x7f4c140a5130 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:13.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.465+0000 7f4c2deab700 1 --2- 172.21.15.38:0/3114657327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c28103ba0 0x7f4c281a0d50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:13.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.465+0000 7f4c2deab700 1 --2- 172.21.15.38:0/3114657327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c281020d0 0x7f4c28199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:13.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.465+0000 7f4c2deab700 1 --2- 172.21.15.38:0/3114657327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c28103200 0x7f4c28199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:13.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.465+0000 7f4c2deab700 1 -- 172.21.15.38:0/3114657327 >> 172.21.15.38:0/3114657327 conn(0x7f4c280fd710 msgr2=0x7f4c280fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:13.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.465+0000 7f4c2deab700 1 -- 172.21.15.38:0/3114657327 shutdown_connections 2024-05-21T15:34:13.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:13.465+0000 7f4c2deab700 1 -- 172.21.15.38:0/3114657327 wait complete. 2024-05-21T15:34:14.129 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:13 smithi038 ceph-mon[31995]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:34:14.163 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-21T15:34:14.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:13 smithi053 ceph-mon[36496]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:34:14.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:13 smithi045 ceph-mon[36409]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:34:14.419 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:34:15.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:14 smithi053 ceph-mon[36496]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:15.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:14 smithi038 ceph-mon[31995]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:15.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:14 smithi045 ceph-mon[36409]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:16.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.088+0000 7f5221a9f700 1 -- 172.21.15.38:0/3224390940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f521c108f80 msgr2=0x7f521c10b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.088+0000 7f5221a9f700 1 --2- 172.21.15.38:0/3224390940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f521c108f80 0x7f521c10b3a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5214009990 tx=0x7f5214009ca0 comp rx=0 tx=0).stop 2024-05-21T15:34:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.088+0000 7f5221a9f700 1 -- 172.21.15.38:0/3224390940 shutdown_connections 2024-05-21T15:34:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.088+0000 7f5221a9f700 1 --2- 172.21.15.38:0/3224390940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f521c108f80 0x7f521c10b3a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.088+0000 7f5221a9f700 1 --2- 172.21.15.38:0/3224390940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c103cc0 0x7f521c1060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.088+0000 7f5221a9f700 1 --2- 172.21.15.38:0/3224390940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f521c106620 0x7f521c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.088+0000 7f5221a9f700 1 -- 172.21.15.38:0/3224390940 >> 172.21.15.38:0/3224390940 conn(0x7f521c0fd700 msgr2=0x7f521c0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.088+0000 7f5221a9f700 1 -- 172.21.15.38:0/3224390940 shutdown_connections 2024-05-21T15:34:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.088+0000 7f5221a9f700 1 -- 172.21.15.38:0/3224390940 wait complete. 2024-05-21T15:34:16.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.089+0000 7f5221a9f700 1 Processor -- start 2024-05-21T15:34:16.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.089+0000 7f5221a9f700 1 -- start start 2024-05-21T15:34:16.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.089+0000 7f5221a9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f521c103cc0 0x7f521c19b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:16.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.089+0000 7f5221a9f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c106620 0x7f521c19bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:16.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.089+0000 7f5221a9f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f521c108f80 0x7f521c1a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:16.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.089+0000 7f5221a9f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f521c10ed80 con 0x7f521c108f80 2024-05-21T15:34:16.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.089+0000 7f5221a9f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f521c10ec00 con 0x7f521c103cc0 2024-05-21T15:34:16.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.089+0000 7f5221a9f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f521c10ef00 con 0x7f521c106620 2024-05-21T15:34:16.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f521bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f521c108f80 0x7f521c1a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:16.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f521bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f521c108f80 0x7f521c1a2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48842/0 (socket says 172.21.15.38:48842) 2024-05-21T15:34:16.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f521bfff700 1 -- 172.21.15.38:0/121555034 learned_addr learned my addr 172.21.15.38:0/121555034 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:16.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f521affd700 1 --2- 172.21.15.38:0/121555034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c106620 0x7f521c19bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:16.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f521b7fe700 1 --2- 172.21.15.38:0/121555034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f521c103cc0 0x7f521c19b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:16.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f521bfff700 1 -- 172.21.15.38:0/121555034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c106620 msgr2=0x7f521c19bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:16.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f521bfff700 1 --2- 172.21.15.38:0/121555034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c106620 0x7f521c19bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:16.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f521bfff700 1 -- 172.21.15.38:0/121555034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f521c103cc0 msgr2=0x7f521c19b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:16.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f521bfff700 1 --2- 172.21.15.38:0/121555034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f521c103cc0 0x7f521c19b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:16.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f521bfff700 1 -- 172.21.15.38:0/121555034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521c1a34c0 con 0x7f521c108f80 2024-05-21T15:34:16.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f521bfff700 1 --2- 172.21.15.38:0/121555034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f521c108f80 0x7f521c1a2ed0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5214003900 tx=0x7f52140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:16.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f5218ff9700 1 -- 172.21.15.38:0/121555034 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214003c70 con 0x7f521c108f80 2024-05-21T15:34:16.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f5218ff9700 1 -- 172.21.15.38:0/121555034 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5214004830 con 0x7f521c108f80 2024-05-21T15:34:16.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f5218ff9700 1 -- 172.21.15.38:0/121555034 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52140053a0 con 0x7f521c108f80 2024-05-21T15:34:16.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f5221a9f700 1 -- 172.21.15.38:0/121555034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f521c1a3750 con 0x7f521c108f80 2024-05-21T15:34:16.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.090+0000 7f5221a9f700 1 -- 172.21.15.38:0/121555034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f521c1a3ba0 con 0x7f521c108f80 2024-05-21T15:34:16.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.091+0000 7f5221a9f700 1 -- 172.21.15.38:0/121555034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f521c050170 con 0x7f521c108f80 2024-05-21T15:34:16.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.093+0000 7f5218ff9700 1 -- 172.21.15.38:0/121555034 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5214005540 con 0x7f521c108f80 2024-05-21T15:34:16.094 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:15 smithi038 ceph-mon[31995]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:34:16.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.094+0000 7f5218ff9700 1 --2- 172.21.15.38:0/121555034 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f52080a2ca0 0x7f52080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:16.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.094+0000 7f521b7fe700 1 --2- 172.21.15.38:0/121555034 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f52080a2ca0 0x7f52080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:16.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.094+0000 7f5218ff9700 1 -- 172.21.15.38:0/121555034 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f52140d03d0 con 0x7f521c108f80 2024-05-21T15:34:16.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.095+0000 7f521b7fe700 1 --2- 172.21.15.38:0/121555034 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f52080a2ca0 0x7f52080a51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f521c06aeb0 tx=0x7f520c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:16.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.097+0000 7f5218ff9700 1 -- 172.21.15.38:0/121555034 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5214013070 con 0x7f521c108f80 2024-05-21T15:34:16.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:15 smithi053 ceph-mon[36496]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:34:16.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:15 smithi045 ceph-mon[36409]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:34:16.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.310+0000 7f5221a9f700 1 -- 172.21.15.38:0/121555034 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f521c062b40 con 0x7f52080a2ca0 2024-05-21T15:34:16.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.311+0000 7f5218ff9700 1 -- 172.21.15.38:0/121555034 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7f521c062b40 con 0x7f52080a2ca0 2024-05-21T15:34:16.311 INFO:teuthology.orchestra.run.smithi038.stdout:HOST ADDR LABELS STATUS 2024-05-21T15:34:16.311 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 172.21.15.38 2024-05-21T15:34:16.311 INFO:teuthology.orchestra.run.smithi038.stdout:smithi045 172.21.15.45 2024-05-21T15:34:16.311 INFO:teuthology.orchestra.run.smithi038.stdout:smithi053 172.21.15.53 2024-05-21T15:34:16.311 INFO:teuthology.orchestra.run.smithi038.stdout:3 hosts in cluster 2024-05-21T15:34:16.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.313+0000 7f5221a9f700 1 -- 172.21.15.38:0/121555034 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f52080a2ca0 msgr2=0x7f52080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:16.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.313+0000 7f5221a9f700 1 --2- 172.21.15.38:0/121555034 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f52080a2ca0 0x7f52080a51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f521c06aeb0 tx=0x7f520c009040 comp rx=0 tx=0).stop 2024-05-21T15:34:16.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.313+0000 7f5221a9f700 1 -- 172.21.15.38:0/121555034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f521c108f80 msgr2=0x7f521c1a2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:16.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.313+0000 7f5221a9f700 1 --2- 172.21.15.38:0/121555034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f521c108f80 0x7f521c1a2ed0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5214003900 tx=0x7f52140047b0 comp rx=0 tx=0).stop 2024-05-21T15:34:16.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.313+0000 7f5221a9f700 1 -- 172.21.15.38:0/121555034 shutdown_connections 2024-05-21T15:34:16.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.313+0000 7f5221a9f700 1 --2- 172.21.15.38:0/121555034 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f52080a2ca0 0x7f52080a51b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:16.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.313+0000 7f5221a9f700 1 --2- 172.21.15.38:0/121555034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f521c108f80 0x7f521c1a2ed0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:16.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.313+0000 7f5221a9f700 1 --2- 172.21.15.38:0/121555034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f521c103cc0 0x7f521c19b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:16.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.313+0000 7f5221a9f700 1 --2- 172.21.15.38:0/121555034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c106620 0x7f521c19bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:16.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.314+0000 7f5221a9f700 1 -- 172.21.15.38:0/121555034 >> 172.21.15.38:0/121555034 conn(0x7f521c0fd700 msgr2=0x7f521c107c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:16.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.314+0000 7f5221a9f700 1 -- 172.21.15.38:0/121555034 shutdown_connections 2024-05-21T15:34:16.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:16.314+0000 7f5221a9f700 1 -- 172.21.15.38:0/121555034 wait complete. 2024-05-21T15:34:17.146 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-21T15:34:17.396 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:34:18.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:17 smithi045 ceph-mon[36409]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:34:18.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:17 smithi045 ceph-mon[36409]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:18.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:17 smithi038 ceph-mon[31995]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:34:18.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:17 smithi038 ceph-mon[31995]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:18.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:17 smithi053 ceph-mon[36496]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:34:18.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:17 smithi053 ceph-mon[36496]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.246+0000 7f5be6298700 1 -- 172.21.15.38:0/1246397518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0108f90 msgr2=0x7f5be010b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.246+0000 7f5be6298700 1 --2- 172.21.15.38:0/1246397518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0108f90 0x7f5be010b3b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800b0a0 tx=0x7f5bd800b3b0 comp rx=0 tx=0).stop 2024-05-21T15:34:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.246+0000 7f5be6298700 1 -- 172.21.15.38:0/1246397518 shutdown_connections 2024-05-21T15:34:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.246+0000 7f5be6298700 1 --2- 172.21.15.38:0/1246397518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0108f90 0x7f5be010b3b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.246+0000 7f5be6298700 1 --2- 172.21.15.38:0/1246397518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5be0103cd0 0x7f5be01060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.246+0000 7f5be6298700 1 --2- 172.21.15.38:0/1246397518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be0106630 0x7f5be0108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.246+0000 7f5be6298700 1 -- 172.21.15.38:0/1246397518 >> 172.21.15.38:0/1246397518 conn(0x7f5be00fd710 msgr2=0x7f5be00ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.246+0000 7f5be6298700 1 -- 172.21.15.38:0/1246397518 shutdown_connections 2024-05-21T15:34:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.246+0000 7f5be6298700 1 -- 172.21.15.38:0/1246397518 wait complete. 2024-05-21T15:34:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.247+0000 7f5be6298700 1 Processor -- start 2024-05-21T15:34:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.247+0000 7f5be6298700 1 -- start start 2024-05-21T15:34:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.247+0000 7f5be6298700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5be0103cd0 0x7f5be01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.247+0000 7f5be6298700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be0106630 0x7f5be0199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.247+0000 7f5be6298700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0108f90 0x7f5be01a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.247+0000 7f5be6298700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5be010ed90 con 0x7f5be0108f90 2024-05-21T15:34:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.247+0000 7f5be6298700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5be010ec10 con 0x7f5be0106630 2024-05-21T15:34:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.247+0000 7f5be6298700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5be010ef10 con 0x7f5be0103cd0 2024-05-21T15:34:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.247+0000 7f5be4835700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0108f90 0x7f5be01a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5be4835700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0108f90 0x7f5be01a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48856/0 (socket says 172.21.15.38:48856) 2024-05-21T15:34:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5be4835700 1 -- 172.21.15.38:0/1417917071 learned_addr learned my addr 172.21.15.38:0/1417917071 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5bd7fff700 1 --2- 172.21.15.38:0/1417917071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be0106630 0x7f5be0199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5bdffff700 1 --2- 172.21.15.38:0/1417917071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5be0103cd0 0x7f5be01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5be4835700 1 -- 172.21.15.38:0/1417917071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5be0103cd0 msgr2=0x7f5be01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5be4835700 1 --2- 172.21.15.38:0/1417917071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5be0103cd0 0x7f5be01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5be4835700 1 -- 172.21.15.38:0/1417917071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be0106630 msgr2=0x7f5be0199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5be4835700 1 --2- 172.21.15.38:0/1417917071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be0106630 0x7f5be0199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5be4835700 1 -- 172.21.15.38:0/1417917071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be01a1340 con 0x7f5be0108f90 2024-05-21T15:34:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5be4835700 1 --2- 172.21.15.38:0/1417917071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0108f90 0x7f5be01a0cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80062a0 tx=0x7f5bd8004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5bddffb700 1 -- 172.21.15.38:0/1417917071 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8021070 con 0x7f5be0108f90 2024-05-21T15:34:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5be6298700 1 -- 172.21.15.38:0/1417917071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be01a15d0 con 0x7f5be0108f90 2024-05-21T15:34:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5bddffb700 1 -- 172.21.15.38:0/1417917071 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd8007c40 con 0x7f5be0108f90 2024-05-21T15:34:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5bddffb700 1 -- 172.21.15.38:0/1417917071 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd801d450 con 0x7f5be0108f90 2024-05-21T15:34:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.248+0000 7f5be6298700 1 -- 172.21.15.38:0/1417917071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be01a1ad0 con 0x7f5be0108f90 2024-05-21T15:34:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.249+0000 7f5be6298700 1 -- 172.21.15.38:0/1417917071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be0050170 con 0x7f5be0108f90 2024-05-21T15:34:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.251+0000 7f5bddffb700 1 -- 172.21.15.38:0/1417917071 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5bd80040b0 con 0x7f5be0108f90 2024-05-21T15:34:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.251+0000 7f5bddffb700 1 --2- 172.21.15.38:0/1417917071 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5bcc0a2c20 0x7f5bcc0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.251+0000 7f5bdffff700 1 --2- 172.21.15.38:0/1417917071 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5bcc0a2c20 0x7f5bcc0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.251+0000 7f5bddffb700 1 -- 172.21.15.38:0/1417917071 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f5bd800f020 con 0x7f5be0108f90 2024-05-21T15:34:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.252+0000 7f5bdffff700 1 --2- 172.21.15.38:0/1417917071 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5bcc0a2c20 0x7f5bcc0a5130 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0005950 tx=0x7f5bd000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:19.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.256+0000 7f5bddffb700 1 -- 172.21.15.38:0/1417917071 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5bd802a020 con 0x7f5be0108f90 2024-05-21T15:34:19.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.470+0000 7f5be6298700 1 -- 172.21.15.38:0/1417917071 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f5be0062a80 con 0x7f5bcc0a2c20 2024-05-21T15:34:19.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.472+0000 7f5bddffb700 1 -- 172.21.15.38:0/1417917071 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f5be0062a80 con 0x7f5bcc0a2c20 2024-05-21T15:34:19.471 INFO:teuthology.orchestra.run.smithi038.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-21T15:34:19.471 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736103BB4P0KGN 3726G No 3m ago Has a FileSystem, LVM detected 2024-05-21T15:34:19.471 INFO:teuthology.orchestra.run.smithi038.stdout:smithi045 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120098480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-05-21T15:34:19.471 INFO:teuthology.orchestra.run.smithi038.stdout:smithi053 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300G8480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-05-21T15:34:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.474+0000 7f5be6298700 1 -- 172.21.15.38:0/1417917071 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5bcc0a2c20 msgr2=0x7f5bcc0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.474+0000 7f5be6298700 1 --2- 172.21.15.38:0/1417917071 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5bcc0a2c20 0x7f5bcc0a5130 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0005950 tx=0x7f5bd000b410 comp rx=0 tx=0).stop 2024-05-21T15:34:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.474+0000 7f5be6298700 1 -- 172.21.15.38:0/1417917071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0108f90 msgr2=0x7f5be01a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.474+0000 7f5be6298700 1 --2- 172.21.15.38:0/1417917071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0108f90 0x7f5be01a0cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80062a0 tx=0x7f5bd8004d70 comp rx=0 tx=0).stop 2024-05-21T15:34:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.474+0000 7f5be6298700 1 -- 172.21.15.38:0/1417917071 shutdown_connections 2024-05-21T15:34:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.474+0000 7f5be6298700 1 --2- 172.21.15.38:0/1417917071 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5bcc0a2c20 0x7f5bcc0a5130 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.474+0000 7f5be6298700 1 --2- 172.21.15.38:0/1417917071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0108f90 0x7f5be01a0cc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.474+0000 7f5be6298700 1 --2- 172.21.15.38:0/1417917071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5be0103cd0 0x7f5be01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.474+0000 7f5be6298700 1 --2- 172.21.15.38:0/1417917071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be0106630 0x7f5be0199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.474+0000 7f5be6298700 1 -- 172.21.15.38:0/1417917071 >> 172.21.15.38:0/1417917071 conn(0x7f5be00fd710 msgr2=0x7f5be0104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.475+0000 7f5be6298700 1 -- 172.21.15.38:0/1417917071 shutdown_connections 2024-05-21T15:34:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:19.475+0000 7f5be6298700 1 -- 172.21.15.38:0/1417917071 wait complete. 2024-05-21T15:34:20.195 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:19 smithi038 ceph-mon[31995]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:34:20.227 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-21T15:34:20.236 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-05-21T15:34:20.237 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-21T15:34:20.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:19 smithi045 ceph-mon[36409]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:34:20.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:19 smithi053 ceph-mon[36496]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:34:20.490 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:34:21.229 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:20 smithi038 ceph-mon[31995]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:21.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:20 smithi045 ceph-mon[36409]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:21.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:20 smithi053 ceph-mon[36496]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:34:22.201 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:21 smithi038 ceph-mon[31995]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:34:22.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:21 smithi045 ceph-mon[36409]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:34:22.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.320+0000 7f08e16bd700 1 -- 172.21.15.38:0/2557091978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc103ba0 msgr2=0x7f08dc10a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:22.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.320+0000 7f08e16bd700 1 --2- 172.21.15.38:0/2557091978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc103ba0 0x7f08dc10a450 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f08d400b0a0 tx=0x7f08d400b3b0 comp rx=0 tx=0).stop 2024-05-21T15:34:22.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.320+0000 7f08e16bd700 1 -- 172.21.15.38:0/2557091978 shutdown_connections 2024-05-21T15:34:22.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.320+0000 7f08e16bd700 1 --2- 172.21.15.38:0/2557091978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc103ba0 0x7f08dc10a450 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:22.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.320+0000 7f08e16bd700 1 --2- 172.21.15.38:0/2557091978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08dc1020d0 0x7f08dc1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:22.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.320+0000 7f08e16bd700 1 --2- 172.21.15.38:0/2557091978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08dc103200 0x7f08dc103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:22.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.320+0000 7f08e16bd700 1 -- 172.21.15.38:0/2557091978 >> 172.21.15.38:0/2557091978 conn(0x7f08dc0fd710 msgr2=0x7f08dc0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:22.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.321+0000 7f08e16bd700 1 -- 172.21.15.38:0/2557091978 shutdown_connections 2024-05-21T15:34:22.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.321+0000 7f08e16bd700 1 -- 172.21.15.38:0/2557091978 wait complete. 2024-05-21T15:34:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.321+0000 7f08e16bd700 1 Processor -- start 2024-05-21T15:34:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.321+0000 7f08e16bd700 1 -- start start 2024-05-21T15:34:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08e16bd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc1020d0 0x7f08dc0752d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08e16bd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08dc103200 0x7f08dc19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08e16bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08dc103ba0 0x7f08dc1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08e16bd700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc10dca0 con 0x7f08dc1020d0 2024-05-21T15:34:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08e16bd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc10db20 con 0x7f08dc103ba0 2024-05-21T15:34:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08e16bd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc10de20 con 0x7f08dc103200 2024-05-21T15:34:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08daffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc1020d0 0x7f08dc0752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08daffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc1020d0 0x7f08dc0752d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48874/0 (socket says 172.21.15.38:48874) 2024-05-21T15:34:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08daffd700 1 -- 172.21.15.38:0/1851871613 learned_addr learned my addr 172.21.15.38:0/1851871613 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08da7fc700 1 --2- 172.21.15.38:0/1851871613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08dc103200 0x7f08dc19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08db7fe700 1 --2- 172.21.15.38:0/1851871613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08dc103ba0 0x7f08dc1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08daffd700 1 -- 172.21.15.38:0/1851871613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08dc103200 msgr2=0x7f08dc19e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08daffd700 1 --2- 172.21.15.38:0/1851871613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08dc103200 0x7f08dc19e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08daffd700 1 -- 172.21.15.38:0/1851871613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08dc103ba0 msgr2=0x7f08dc1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08daffd700 1 --2- 172.21.15.38:0/1851871613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08dc103ba0 0x7f08dc1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08daffd700 1 -- 172.21.15.38:0/1851871613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc1a1340 con 0x7f08dc1020d0 2024-05-21T15:34:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.322+0000 7f08daffd700 1 --2- 172.21.15.38:0/1851871613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc1020d0 0x7f08dc0752d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00d950 tx=0x7f08cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.323+0000 7f08c3fff700 1 -- 172.21.15.38:0/1851871613 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc017070 con 0x7f08dc1020d0 2024-05-21T15:34:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.323+0000 7f08c3fff700 1 -- 172.21.15.38:0/1851871613 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08cc00fb70 con 0x7f08dc1020d0 2024-05-21T15:34:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.323+0000 7f08e16bd700 1 -- 172.21.15.38:0/1851871613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08dc1a1630 con 0x7f08dc1020d0 2024-05-21T15:34:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.323+0000 7f08c3fff700 1 -- 172.21.15.38:0/1851871613 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc003b40 con 0x7f08dc1020d0 2024-05-21T15:34:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.323+0000 7f08e16bd700 1 -- 172.21.15.38:0/1851871613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08dc1a1b70 con 0x7f08dc1020d0 2024-05-21T15:34:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.324+0000 7f08e16bd700 1 -- 172.21.15.38:0/1851871613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08dc050170 con 0x7f08dc1020d0 2024-05-21T15:34:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.325+0000 7f08c3fff700 1 -- 172.21.15.38:0/1851871613 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f08cc003ce0 con 0x7f08dc1020d0 2024-05-21T15:34:22.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.326+0000 7f08c3fff700 1 --2- 172.21.15.38:0/1851871613 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f08c40a2c20 0x7f08c40a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:22.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.326+0000 7f08da7fc700 1 --2- 172.21.15.38:0/1851871613 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f08c40a2c20 0x7f08c40a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:22.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.327+0000 7f08c3fff700 1 -- 172.21.15.38:0/1851871613 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f08cc0cf250 con 0x7f08dc1020d0 2024-05-21T15:34:22.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.327+0000 7f08da7fc700 1 --2- 172.21.15.38:0/1851871613 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f08c40a2c20 0x7f08c40a5130 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f08d0005950 tx=0x7f08d000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.330+0000 7f08c3fff700 1 -- 172.21.15.38:0/1851871613 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f08cc020030 con 0x7f08dc1020d0 2024-05-21T15:34:22.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:21 smithi053 ceph-mon[36496]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.582+0000 7f08e16bd700 1 -- 172.21.15.38:0/1851871613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f08dc063250 con 0x7f08dc1020d0 2024-05-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.583+0000 7f08c3fff700 1 -- 172.21.15.38:0/1851871613 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3078 (secure 0 0 0) 0x7f08cc09c1a0 con 0x7f08dc1020d0 2024-05-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi038.stdout:e26 2024-05-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-05-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-05-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-05-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 26 2024-05-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-05-21T15:32:33.564616+0000 2024-05-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-05-21T15:32:40.979368+0000 2024-05-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-05-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-05-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 300 2024-05-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-05-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-05-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-05-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-05-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-05-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-05-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 5 2024-05-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1,2,3,4 2024-05-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24602,1=24551,2=24550,3=24523,4=24515} 2024-05-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-05-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-05-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-05-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-05-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-05-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-05-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-05-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-05-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 0 2024-05-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 24602 members: 24550,24515,24602,24523,24551 2024-05-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.b{0:24602} state up:active seq 3 addr [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.f{0:24511} state up:standby-replay seq 1 addr [v2:172.21.15.53:6834/1330413091,v1:172.21.15.53:6835/1330413091] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.h{1:24551} state up:active seq 6 addr [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.k{1:24479} state up:standby-replay seq 1 addr [v2:172.21.15.45:6832/2551213098,v1:172.21.15.45:6833/2551213098] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.i{2:24550} state up:active seq 6 addr [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.l{2:24445} state up:standby-replay seq 1 addr [v2:172.21.15.53:6832/1390207425,v1:172.21.15.53:6833/1390207425] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.c{3:24523} state up:active seq 8 addr [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.a{3:14667} state up:standby-replay seq 1 addr [v2:172.21.15.38:6840/106491696,v1:172.21.15.38:6841/106491696] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.e{4:24515} state up:active seq 9 addr [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.g{4:14655} state up:standby-replay seq 1 addr [v2:172.21.15.38:6838/2313367718,v1:172.21.15.38:6839/2313367718] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-05-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.j{-1:14562} state up:standby seq 1 addr [v2:172.21.15.38:6834/3555626860,v1:172.21.15.38:6835/3555626860] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.d{-1:14574} state up:standby seq 1 addr [v2:172.21.15.38:6836/2818680145,v1:172.21.15.38:6837/2818680145] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.585+0000 7f08e16bd700 1 -- 172.21.15.38:0/1851871613 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f08c40a2c20 msgr2=0x7f08c40a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.585+0000 7f08e16bd700 1 --2- 172.21.15.38:0/1851871613 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f08c40a2c20 0x7f08c40a5130 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f08d0005950 tx=0x7f08d000a300 comp rx=0 tx=0).stop 2024-05-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.585+0000 7f08e16bd700 1 -- 172.21.15.38:0/1851871613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc1020d0 msgr2=0x7f08dc0752d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.585+0000 7f08e16bd700 1 --2- 172.21.15.38:0/1851871613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc1020d0 0x7f08dc0752d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00d950 tx=0x7f08cc00dc60 comp rx=0 tx=0).stop 2024-05-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.585+0000 7f08e16bd700 1 -- 172.21.15.38:0/1851871613 shutdown_connections 2024-05-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.585+0000 7f08e16bd700 1 --2- 172.21.15.38:0/1851871613 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f08c40a2c20 0x7f08c40a5130 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.585+0000 7f08e16bd700 1 --2- 172.21.15.38:0/1851871613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08dc103ba0 0x7f08dc1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.585+0000 7f08e16bd700 1 --2- 172.21.15.38:0/1851871613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08dc1020d0 0x7f08dc0752d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.585+0000 7f08e16bd700 1 --2- 172.21.15.38:0/1851871613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08dc103200 0x7f08dc19e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.585+0000 7f08e16bd700 1 -- 172.21.15.38:0/1851871613 >> 172.21.15.38:0/1851871613 conn(0x7f08dc0fd710 msgr2=0x7f08dc106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.585+0000 7f08e16bd700 1 -- 172.21.15.38:0/1851871613 shutdown_connections 2024-05-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:22.585+0000 7f08e16bd700 1 -- 172.21.15.38:0/1851871613 wait complete. 2024-05-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-05-21T15:34:23.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-05-21T15:34:23.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:22 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-05-21T15:34:23.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:22 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1851871613' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-21T15:34:23.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-05-21T15:34:23.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:22 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-05-21T15:34:23.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:22 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1851871613' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-21T15:34:23.344 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-21T15:34:23.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-05-21T15:34:23.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:22 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-05-21T15:34:23.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:22 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1851871613' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-21T15:34:23.594 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:34:24.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:24 smithi045 ceph-mon[36409]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:34:24.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-05-21T15:34:24.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:24 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-05-21T15:34:24.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:24 smithi045 ceph-mon[36409]: osdmap e89: 12 total, 12 up, 12 in 2024-05-21T15:34:24.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:24 smithi038 ceph-mon[31995]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:34:24.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-05-21T15:34:24.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:24 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-05-21T15:34:24.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:24 smithi038 ceph-mon[31995]: osdmap e89: 12 total, 12 up, 12 in 2024-05-21T15:34:24.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:24 smithi053 ceph-mon[36496]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:34:24.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-05-21T15:34:24.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:24 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-05-21T15:34:24.456 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:24 smithi053 ceph-mon[36496]: osdmap e89: 12 total, 12 up, 12 in 2024-05-21T15:34:25.306 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:25 smithi038 ceph-mon[31995]: osdmap e90: 12 total, 12 up, 12 in 2024-05-21T15:34:25.306 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:25 smithi038 ceph-mon[31995]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:34:25.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.334+0000 7f1042a08700 1 -- 172.21.15.38:0/25329087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f103c103860 msgr2=0x7f103c10b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:25.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.334+0000 7f1042a08700 1 --2- 172.21.15.38:0/25329087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f103c103860 0x7f103c10b570 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f103400b0a0 tx=0x7f103400b3b0 comp rx=0 tx=0).stop 2024-05-21T15:34:25.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.334+0000 7f1042a08700 1 -- 172.21.15.38:0/25329087 shutdown_connections 2024-05-21T15:34:25.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.334+0000 7f1042a08700 1 --2- 172.21.15.38:0/25329087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f103c103860 0x7f103c10b570 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:25.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.334+0000 7f1042a08700 1 --2- 172.21.15.38:0/25329087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f103c104830 0x7f103c104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:25.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.334+0000 7f1042a08700 1 --2- 172.21.15.38:0/25329087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103c102de0 0x7f103c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:25.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.334+0000 7f1042a08700 1 -- 172.21.15.38:0/25329087 >> 172.21.15.38:0/25329087 conn(0x7f103c0fe850 msgr2=0x7f103c100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:25.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.334+0000 7f1042a08700 1 -- 172.21.15.38:0/25329087 shutdown_connections 2024-05-21T15:34:25.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.334+0000 7f1042a08700 1 -- 172.21.15.38:0/25329087 wait complete. 2024-05-21T15:34:25.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.334+0000 7f1042a08700 1 Processor -- start 2024-05-21T15:34:25.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1042a08700 1 -- start start 2024-05-21T15:34:25.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1042a08700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103c102de0 0x7f103c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:25.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1042a08700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f103c103860 0x7f103c199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:25.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1042a08700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f103c104830 0x7f103c1a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:25.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1042a08700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f103c10ef10 con 0x7f103c104830 2024-05-21T15:34:25.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1042a08700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f103c10ed90 con 0x7f103c103860 2024-05-21T15:34:25.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1042a08700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f103c10f090 con 0x7f103c102de0 2024-05-21T15:34:25.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1040fa5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f103c104830 0x7f103c1a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:25.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1040fa5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f103c104830 0x7f103c1a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59718/0 (socket says 172.21.15.38:59718) 2024-05-21T15:34:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1040fa5700 1 -- 172.21.15.38:0/928422439 learned_addr learned my addr 172.21.15.38:0/928422439 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f103bfff700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103c102de0 0x7f103c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f103b7fe700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f103c103860 0x7f103c199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1040fa5700 1 -- 172.21.15.38:0/928422439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103c102de0 msgr2=0x7f103c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1040fa5700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103c102de0 0x7f103c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1040fa5700 1 -- 172.21.15.38:0/928422439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f103c103860 msgr2=0x7f103c199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1040fa5700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f103c103860 0x7f103c199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f1040fa5700 1 -- 172.21.15.38:0/928422439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c1a14a0 con 0x7f103c104830 2024-05-21T15:34:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.335+0000 7f103b7fe700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f103c103860 0x7f103c199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.336+0000 7f1040fa5700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f103c104830 0x7f103c1a0e20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1034009fd0 tx=0x7f1034004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.336+0000 7f10397fa700 1 -- 172.21.15.38:0/928422439 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034021070 con 0x7f103c104830 2024-05-21T15:34:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.336+0000 7f10397fa700 1 -- 172.21.15.38:0/928422439 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1034007c80 con 0x7f103c104830 2024-05-21T15:34:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.336+0000 7f10397fa700 1 -- 172.21.15.38:0/928422439 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10340055e0 con 0x7f103c104830 2024-05-21T15:34:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.336+0000 7f1042a08700 1 -- 172.21.15.38:0/928422439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103c1a1730 con 0x7f103c104830 2024-05-21T15:34:25.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.336+0000 7f1042a08700 1 -- 172.21.15.38:0/928422439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103c1a1be0 con 0x7f103c104830 2024-05-21T15:34:25.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.337+0000 7f1042a08700 1 -- 172.21.15.38:0/928422439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f103c050170 con 0x7f103c104830 2024-05-21T15:34:25.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.338+0000 7f10397fa700 1 -- 172.21.15.38:0/928422439 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1034024070 con 0x7f103c104830 2024-05-21T15:34:25.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.339+0000 7f10397fa700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10240a2c20 0x7f10240a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:25.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.339+0000 7f103bfff700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10240a2c20 0x7f10240a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:25.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.339+0000 7f10397fa700 1 -- 172.21.15.38:0/928422439 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f103400f020 con 0x7f103c104830 2024-05-21T15:34:25.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.340+0000 7f103bfff700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10240a2c20 0x7f10240a5130 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f102c005fd0 tx=0x7f102c005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.344+0000 7f10397fa700 1 -- 172.21.15.38:0/928422439 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f103402b020 con 0x7f103c104830 2024-05-21T15:34:25.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:25 smithi053 ceph-mon[36496]: osdmap e90: 12 total, 12 up, 12 in 2024-05-21T15:34:25.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:25 smithi053 ceph-mon[36496]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:34:25.510 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:25 smithi045 ceph-mon[36409]: osdmap e90: 12 total, 12 up, 12 in 2024-05-21T15:34:25.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:25 smithi045 ceph-mon[36409]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:34:25.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.550+0000 7f1042a08700 1 -- 172.21.15.38:0/928422439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f103c02e010 con 0x7f103c104830 2024-05-21T15:34:25.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.551+0000 7f10397fa700 1 -- 172.21.15.38:0/928422439 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5330 (secure 0 0 0) 0x7f1034024360 con 0x7f103c104830 2024-05-21T15:34:25.551 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 90 2024-05-21T15:34:25.551 INFO:teuthology.orchestra.run.smithi038.stdout:fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:34:25.551 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-05-21T15:24:24.082082+0000 2024-05-21T15:34:25.551 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-05-21T15:34:23.996759+0000 2024-05-21T15:34:25.551 INFO:teuthology.orchestra.run.smithi038.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-21T15:34:25.551 INFO:teuthology.orchestra.run.smithi038.stdout:crush_version 26 2024-05-21T15:34:25.551 INFO:teuthology.orchestra.run.smithi038.stdout:full_ratio 0.95 2024-05-21T15:34:25.552 INFO:teuthology.orchestra.run.smithi038.stdout:backfillfull_ratio 0.9 2024-05-21T15:34:25.552 INFO:teuthology.orchestra.run.smithi038.stdout:nearfull_ratio 0.85 2024-05-21T15:34:25.552 INFO:teuthology.orchestra.run.smithi038.stdout:require_min_compat_client luminous 2024-05-21T15:34:25.552 INFO:teuthology.orchestra.run.smithi038.stdout:min_compat_client luminous 2024-05-21T15:34:25.552 INFO:teuthology.orchestra.run.smithi038.stdout:require_osd_release squid 2024-05-21T15:34:25.552 INFO:teuthology.orchestra.run.smithi038.stdout:stretch_mode_enabled false 2024-05-21T15:34:25.552 INFO:teuthology.orchestra.run.smithi038.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-21T15:34:25.552 INFO:teuthology.orchestra.run.smithi038.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-21T15:34:25.552 INFO:teuthology.orchestra.run.smithi038.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-21T15:34:25.552 INFO:teuthology.orchestra.run.smithi038.stdout:max_osd 12 2024-05-21T15:34:25.552 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6802/2546787282,v1:172.21.15.38:6803/2546787282] [v2:172.21.15.38:6804/2546787282,v1:172.21.15.38:6805/2546787282] exists,up f6fb6df5-af66-434f-9f35-f571243a0e39 2024-05-21T15:34:25.552 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6800/3379390940,v1:172.21.15.45:6801/3379390940] [v2:172.21.15.45:6802/3379390940,v1:172.21.15.45:6803/3379390940] exists,up 61df021e-6d15-4875-baab-e7ff8c388a17 2024-05-21T15:34:25.552 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6800/2547348925,v1:172.21.15.53:6801/2547348925] [v2:172.21.15.53:6802/2547348925,v1:172.21.15.53:6803/2547348925] exists,up 7dba2ab3-c388-47ea-b756-68611a94f77b 2024-05-21T15:34:25.553 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6810/1590427033,v1:172.21.15.38:6811/1590427033] [v2:172.21.15.38:6812/1590427033,v1:172.21.15.38:6813/1590427033] exists,up a3de4eed-679b-4828-ae1e-468585b7c1f3 2024-05-21T15:34:25.553 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6808/283670193,v1:172.21.15.45:6809/283670193] [v2:172.21.15.45:6810/283670193,v1:172.21.15.45:6811/283670193] exists,up b728a85c-8f55-4a40-a5e5-9ce853372338 2024-05-21T15:34:25.553 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6808/1942044902,v1:172.21.15.53:6809/1942044902] [v2:172.21.15.53:6810/1942044902,v1:172.21.15.53:6811/1942044902] exists,up b94853cc-2ef8-410f-b900-023c6d74ea56 2024-05-21T15:34:25.553 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6818/1680263500,v1:172.21.15.38:6819/1680263500] [v2:172.21.15.38:6820/1680263500,v1:172.21.15.38:6821/1680263500] exists,up d609b60e-d3c8-482f-850d-f313e97c8c2e 2024-05-21T15:34:25.553 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6816/424307746,v1:172.21.15.45:6817/424307746] [v2:172.21.15.45:6818/424307746,v1:172.21.15.45:6819/424307746] exists,up 6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7 2024-05-21T15:34:25.553 INFO:teuthology.orchestra.run.smithi038.stdout:osd.8 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6816/53196173,v1:172.21.15.53:6817/53196173] [v2:172.21.15.53:6818/53196173,v1:172.21.15.53:6819/53196173] exists,up bc9538ac-44ea-4518-86b7-1a4dc9f10d57 2024-05-21T15:34:25.553 INFO:teuthology.orchestra.run.smithi038.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6826/2989402649,v1:172.21.15.38:6827/2989402649] [v2:172.21.15.38:6828/2989402649,v1:172.21.15.38:6829/2989402649] exists,up e8885020-dc81-4f22-8592-45c5179c24ad 2024-05-21T15:34:25.553 INFO:teuthology.orchestra.run.smithi038.stdout:osd.10 up in weight 1 up_from 73 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6824/3908617766,v1:172.21.15.45:6825/3908617766] [v2:172.21.15.45:6826/3908617766,v1:172.21.15.45:6827/3908617766] exists,up 56587d67-8492-49d1-8e4f-1ee050b7f9cc 2024-05-21T15:34:25.553 INFO:teuthology.orchestra.run.smithi038.stdout:osd.11 up in weight 1 up_from 81 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6824/2775995423,v1:172.21.15.53:6825/2775995423] [v2:172.21.15.53:6826/2775995423,v1:172.21.15.53:6827/2775995423] exists,up b27f85e0-a264-4ee8-9e24-251bc01305c2 2024-05-21T15:34:25.553 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 2.b [8,11] 2024-05-21T15:34:25.553 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 2.e [2,1] 2024-05-21T15:34:25.553 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 2.10 [2,4] 2024-05-21T15:34:25.553 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 2.12 [5,11] 2024-05-21T15:34:25.554 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 2.1a [8,4] 2024-05-21T15:34:25.554 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.6 [4,6,3,8] 2024-05-21T15:34:25.554 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.10 [5,9] 2024-05-21T15:34:25.554 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.23 [4,7] 2024-05-21T15:34:25.554 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.27 [3,9] 2024-05-21T15:34:25.554 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.31 [4,8] 2024-05-21T15:34:25.554 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.33 [5,9] 2024-05-21T15:34:25.554 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/592277081 expires 2024-05-22T15:25:22.117984+0000 2024-05-21T15:34:25.554 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/1186842456 expires 2024-05-22T15:25:22.117984+0000 2024-05-21T15:34:25.554 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6801/1481949523 expires 2024-05-22T15:25:22.117984+0000 2024-05-21T15:34:25.554 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6800/1481949523 expires 2024-05-22T15:25:22.117984+0000 2024-05-21T15:34:25.554 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/1771432089 expires 2024-05-22T15:24:54.404192+0000 2024-05-21T15:34:25.554 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6801/3523378851 expires 2024-05-22T15:24:54.404192+0000 2024-05-21T15:34:25.555 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/2903150778 expires 2024-05-22T15:24:54.404192+0000 2024-05-21T15:34:25.555 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6800/3523378851 expires 2024-05-22T15:24:54.404192+0000 2024-05-21T15:34:25.555 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/2666943685 expires 2024-05-22T15:24:54.404192+0000 2024-05-21T15:34:25.555 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/3586196771 expires 2024-05-22T15:25:22.117984+0000 2024-05-21T15:34:25.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.554+0000 7f1042a08700 1 -- 172.21.15.38:0/928422439 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10240a2c20 msgr2=0x7f10240a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:25.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.554+0000 7f1042a08700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10240a2c20 0x7f10240a5130 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f102c005fd0 tx=0x7f102c005dd0 comp rx=0 tx=0).stop 2024-05-21T15:34:25.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.554+0000 7f1042a08700 1 -- 172.21.15.38:0/928422439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f103c104830 msgr2=0x7f103c1a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:25.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.554+0000 7f1042a08700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f103c104830 0x7f103c1a0e20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1034009fd0 tx=0x7f1034004e00 comp rx=0 tx=0).stop 2024-05-21T15:34:25.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.554+0000 7f1042a08700 1 -- 172.21.15.38:0/928422439 shutdown_connections 2024-05-21T15:34:25.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.554+0000 7f1042a08700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10240a2c20 0x7f10240a5130 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:25.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.554+0000 7f1042a08700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f103c104830 0x7f103c1a0e20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:25.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.554+0000 7f1042a08700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f103c102de0 0x7f103c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:25.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.554+0000 7f1042a08700 1 --2- 172.21.15.38:0/928422439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f103c103860 0x7f103c199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:25.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.554+0000 7f1042a08700 1 -- 172.21.15.38:0/928422439 >> 172.21.15.38:0/928422439 conn(0x7f103c0fe850 msgr2=0x7f103c107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:25.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.554+0000 7f1042a08700 1 -- 172.21.15.38:0/928422439 shutdown_connections 2024-05-21T15:34:25.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:25.554+0000 7f1042a08700 1 -- 172.21.15.38:0/928422439 wait complete. 2024-05-21T15:34:26.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:26 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/928422439' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-21T15:34:26.308 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-21T15:34:26.319 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:34:26.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.440+0000 7f5df5a67640 1 -- 172.21.15.38:0/3568085919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5df0105bd0 msgr2=0x7f5df01080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:26.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.440+0000 7f5df5a67640 1 --2- 172.21.15.38:0/3568085919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5df0105bd0 0x7f5df01080b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009900 tx=0x7f5de402a410 comp rx=0 tx=0).stop 2024-05-21T15:34:26.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.440+0000 7f5df5a67640 1 -- 172.21.15.38:0/3568085919 shutdown_connections 2024-05-21T15:34:26.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.440+0000 7f5df5a67640 1 --2- 172.21.15.38:0/3568085919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df0108740 0x7f5df010ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.440+0000 7f5df5a67640 1 --2- 172.21.15.38:0/3568085919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5df0105bd0 0x7f5df01080b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.440+0000 7f5df5a67640 1 --2- 172.21.15.38:0/3568085919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df01031e0 0x7f5df0105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.440+0000 7f5df5a67640 1 -- 172.21.15.38:0/3568085919 >> 172.21.15.38:0/3568085919 conn(0x7f5df00fcba0 msgr2=0x7f5df00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:26.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.440+0000 7f5df5a67640 1 -- 172.21.15.38:0/3568085919 shutdown_connections 2024-05-21T15:34:26.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.440+0000 7f5df5a67640 1 -- 172.21.15.38:0/3568085919 wait complete. 2024-05-21T15:34:26.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.440+0000 7f5df5a67640 1 Processor -- start 2024-05-21T15:34:26.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5df5a67640 1 -- start start 2024-05-21T15:34:26.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5df5a67640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df01031e0 0x7f5df0199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:26.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5df5a67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df0105bd0 0x7f5df0199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:26.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5df5a67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5df0108740 0x7f5df01a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:26.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5df5a67640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5df010e700 con 0x7f5df0108740 2024-05-21T15:34:26.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5df5a67640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5df010e580 con 0x7f5df0105bd0 2024-05-21T15:34:26.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5df5a67640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5df010e880 con 0x7f5df01031e0 2024-05-21T15:34:26.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5deffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5df0108740 0x7f5df01a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:26.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5deffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5df0108740 0x7f5df01a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59740/0 (socket says 172.21.15.38:59740) 2024-05-21T15:34:26.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5deffff640 1 -- 172.21.15.38:0/2483176102 learned_addr learned my addr 172.21.15.38:0/2483176102 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:26.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5def7fe640 1 --2- 172.21.15.38:0/2483176102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df01031e0 0x7f5df0199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:26.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5deeffd640 1 --2- 172.21.15.38:0/2483176102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df0105bd0 0x7f5df0199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:26.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5deffff640 1 -- 172.21.15.38:0/2483176102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df01031e0 msgr2=0x7f5df0199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:26.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5deffff640 1 --2- 172.21.15.38:0/2483176102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df01031e0 0x7f5df0199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5deffff640 1 -- 172.21.15.38:0/2483176102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df0105bd0 msgr2=0x7f5df0199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:26.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5deffff640 1 --2- 172.21.15.38:0/2483176102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df0105bd0 0x7f5df0199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5deffff640 1 -- 172.21.15.38:0/2483176102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df01a0ca0 con 0x7f5df0108740 2024-05-21T15:34:26.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.441+0000 7f5deffff640 1 --2- 172.21.15.38:0/2483176102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5df0108740 0x7f5df01a06e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5de800cce0 tx=0x7f5de800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:26.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.442+0000 7f5decff9640 1 -- 172.21.15.38:0/2483176102 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de800f450 con 0x7f5df0108740 2024-05-21T15:34:26.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.442+0000 7f5decff9640 1 -- 172.21.15.38:0/2483176102 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de800fad0 con 0x7f5df0108740 2024-05-21T15:34:26.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.442+0000 7f5df5a67640 1 -- 172.21.15.38:0/2483176102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df01a0f90 con 0x7f5df0108740 2024-05-21T15:34:26.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.442+0000 7f5decff9640 1 -- 172.21.15.38:0/2483176102 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8018c10 con 0x7f5df0108740 2024-05-21T15:34:26.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.442+0000 7f5df5a67640 1 -- 172.21.15.38:0/2483176102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df01a13f0 con 0x7f5df0108740 2024-05-21T15:34:26.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.443+0000 7f5df5a67640 1 -- 172.21.15.38:0/2483176102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df00fe6c0 con 0x7f5df0108740 2024-05-21T15:34:26.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.444+0000 7f5decff9640 1 -- 172.21.15.38:0/2483176102 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5de800f5f0 con 0x7f5df0108740 2024-05-21T15:34:26.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.445+0000 7f5decff9640 1 --2- 172.21.15.38:0/2483176102 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5dd80a2c30 0x7f5dd80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:26.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.445+0000 7f5def7fe640 1 --2- 172.21.15.38:0/2483176102 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5dd80a2c30 0x7f5dd80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:26.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.445+0000 7f5decff9640 1 -- 172.21.15.38:0/2483176102 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f5de80cf820 con 0x7f5df0108740 2024-05-21T15:34:26.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.445+0000 7f5def7fe640 1 --2- 172.21.15.38:0/2483176102 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5dd80a2c30 0x7f5dd80a5140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5df0074740 tx=0x7f5de0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:26.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.448+0000 7f5decff9640 1 -- 172.21.15.38:0/2483176102 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5de8014030 con 0x7f5df0108740 2024-05-21T15:34:26.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:26 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/928422439' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-21T15:34:26.510 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:26 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/928422439' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-21T15:34:26.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.636+0000 7f5df5a67640 1 -- 172.21.15.38:0/2483176102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5df002d470 con 0x7f5df0108740 2024-05-21T15:34:26.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.637+0000 7f5decff9640 1 -- 172.21.15.38:0/2483176102 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22704 (secure 0 0 0) 0x7f5de809c760 con 0x7f5df0108740 2024-05-21T15:34:26.637 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:26.638 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":90,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:34:23.996759+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:34:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.639+0000 7f5df5a67640 1 -- 172.21.15.38:0/2483176102 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5dd80a2c30 msgr2=0x7f5dd80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.639+0000 7f5df5a67640 1 --2- 172.21.15.38:0/2483176102 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5dd80a2c30 0x7f5dd80a5140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5df0074740 tx=0x7f5de0005f90 comp rx=0 tx=0).stop 2024-05-21T15:34:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.639+0000 7f5df5a67640 1 -- 172.21.15.38:0/2483176102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5df0108740 msgr2=0x7f5df01a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.639+0000 7f5df5a67640 1 --2- 172.21.15.38:0/2483176102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5df0108740 0x7f5df01a06e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5de800cce0 tx=0x7f5de800a620 comp rx=0 tx=0).stop 2024-05-21T15:34:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.640+0000 7f5df5a67640 1 -- 172.21.15.38:0/2483176102 shutdown_connections 2024-05-21T15:34:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.640+0000 7f5df5a67640 1 --2- 172.21.15.38:0/2483176102 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5dd80a2c30 0x7f5dd80a5140 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.640+0000 7f5df5a67640 1 --2- 172.21.15.38:0/2483176102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5df0108740 0x7f5df01a06e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.640+0000 7f5df5a67640 1 --2- 172.21.15.38:0/2483176102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df0105bd0 0x7f5df0199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.640+0000 7f5df5a67640 1 --2- 172.21.15.38:0/2483176102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df01031e0 0x7f5df0199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.640+0000 7f5df5a67640 1 -- 172.21.15.38:0/2483176102 >> 172.21.15.38:0/2483176102 conn(0x7f5df00fcba0 msgr2=0x7f5df0104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.640+0000 7f5df5a67640 1 -- 172.21.15.38:0/2483176102 shutdown_connections 2024-05-21T15:34:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.640+0000 7f5df5a67640 1 -- 172.21.15.38:0/2483176102 wait complete. 2024-05-21T15:34:26.652 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:27:50.659367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:32:31.645647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:32:32.955451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:34:26.652 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:34:26.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.771+0000 7f70547ce640 1 -- 172.21.15.38:0/2270052910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f704c108840 msgr2=0x7f704c10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:26.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.771+0000 7f70547ce640 1 --2- 172.21.15.38:0/2270052910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f704c108840 0x7f704c10ac60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7044009900 tx=0x7f704402a410 comp rx=0 tx=0).stop 2024-05-21T15:34:26.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.771+0000 7f70547ce640 1 -- 172.21.15.38:0/2270052910 shutdown_connections 2024-05-21T15:34:26.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.771+0000 7f70547ce640 1 --2- 172.21.15.38:0/2270052910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f704c108840 0x7f704c10ac60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.771+0000 7f70547ce640 1 --2- 172.21.15.38:0/2270052910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c105ee0 0x7f704c108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.771+0000 7f70547ce640 1 --2- 172.21.15.38:0/2270052910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f704c103580 0x7f704c1059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.771+0000 7f70547ce640 1 -- 172.21.15.38:0/2270052910 >> 172.21.15.38:0/2270052910 conn(0x7f704c0fcb80 msgr2=0x7f704c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:26.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.771+0000 7f70547ce640 1 -- 172.21.15.38:0/2270052910 shutdown_connections 2024-05-21T15:34:26.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.771+0000 7f70547ce640 1 -- 172.21.15.38:0/2270052910 wait complete. 2024-05-21T15:34:26.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f70547ce640 1 Processor -- start 2024-05-21T15:34:26.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f70547ce640 1 -- start start 2024-05-21T15:34:26.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f70547ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c103580 0x7f704c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:26.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f70547ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f704c105ee0 0x7f704c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:26.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f70547ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f704c108840 0x7f704c1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f70547ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10e710 con 0x7f704c105ee0 2024-05-21T15:34:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f70547ce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10e590 con 0x7f704c103580 2024-05-21T15:34:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f70547ce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10e890 con 0x7f704c108840 2024-05-21T15:34:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f7051d69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f704c105ee0 0x7f704c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f7051d69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f704c105ee0 0x7f704c1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59754/0 (socket says 172.21.15.38:59754) 2024-05-21T15:34:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f7051d69640 1 -- 172.21.15.38:0/4089387715 learned_addr learned my addr 172.21.15.38:0/4089387715 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f705256a640 1 --2- 172.21.15.38:0/4089387715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c103580 0x7f704c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f7051d69640 1 -- 172.21.15.38:0/4089387715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f704c108840 msgr2=0x7f704c1a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f7051d69640 1 --2- 172.21.15.38:0/4089387715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f704c108840 0x7f704c1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f7051d69640 1 -- 172.21.15.38:0/4089387715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c103580 msgr2=0x7f704c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f7051d69640 1 --2- 172.21.15.38:0/4089387715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c103580 0x7f704c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f7051d69640 1 -- 172.21.15.38:0/4089387715 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c1a0da0 con 0x7f704c105ee0 2024-05-21T15:34:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f705256a640 1 --2- 172.21.15.38:0/4089387715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c103580 0x7f704c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.772+0000 7f7051d69640 1 --2- 172.21.15.38:0/4089387715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f704c105ee0 0x7f704c1997a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f703c00d950 tx=0x7f703c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.773+0000 7f70437fe640 1 -- 172.21.15.38:0/4089387715 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c017070 con 0x7f704c105ee0 2024-05-21T15:34:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.773+0000 7f70437fe640 1 -- 172.21.15.38:0/4089387715 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703c009c30 con 0x7f704c105ee0 2024-05-21T15:34:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.773+0000 7f70547ce640 1 -- 172.21.15.38:0/4089387715 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c1a1090 con 0x7f704c105ee0 2024-05-21T15:34:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.773+0000 7f70437fe640 1 -- 172.21.15.38:0/4089387715 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c00f6b0 con 0x7f704c105ee0 2024-05-21T15:34:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.773+0000 7f70547ce640 1 -- 172.21.15.38:0/4089387715 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c1a15e0 con 0x7f704c105ee0 2024-05-21T15:34:26.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.774+0000 7f70547ce640 1 -- 172.21.15.38:0/4089387715 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f704c0fe6a0 con 0x7f704c105ee0 2024-05-21T15:34:26.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.775+0000 7f70437fe640 1 -- 172.21.15.38:0/4089387715 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f703c0109d0 con 0x7f704c105ee0 2024-05-21T15:34:26.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.776+0000 7f70437fe640 1 --2- 172.21.15.38:0/4089387715 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f70380a2b90 0x7f70380a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:26.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.776+0000 7f705256a640 1 --2- 172.21.15.38:0/4089387715 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f70380a2b90 0x7f70380a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:26.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.776+0000 7f70437fe640 1 -- 172.21.15.38:0/4089387715 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f703c0cede0 con 0x7f704c105ee0 2024-05-21T15:34:26.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.776+0000 7f705256a640 1 --2- 172.21.15.38:0/4089387715 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f70380a2b90 0x7f70380a50a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f704c0747a0 tx=0x7f7048009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:26.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.779+0000 7f70437fe640 1 -- 172.21.15.38:0/4089387715 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f703c009de0 con 0x7f704c105ee0 2024-05-21T15:34:26.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.961+0000 7f70547ce640 1 -- 172.21.15.38:0/4089387715 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f704c062750 con 0x7f704c105ee0 2024-05-21T15:34:26.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.962+0000 7f70437fe640 1 -- 172.21.15.38:0/4089387715 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f703c09bc90 con 0x7f704c105ee0 2024-05-21T15:34:26.961 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-05-21T15:34:26.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.963+0000 7f70547ce640 1 -- 172.21.15.38:0/4089387715 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f70380a2b90 msgr2=0x7f70380a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:26.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.963+0000 7f70547ce640 1 --2- 172.21.15.38:0/4089387715 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f70380a2b90 0x7f70380a50a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f704c0747a0 tx=0x7f7048009520 comp rx=0 tx=0).stop 2024-05-21T15:34:26.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.963+0000 7f70547ce640 1 -- 172.21.15.38:0/4089387715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f704c105ee0 msgr2=0x7f704c1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:26.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.963+0000 7f70547ce640 1 --2- 172.21.15.38:0/4089387715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f704c105ee0 0x7f704c1997a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f703c00d950 tx=0x7f703c00de10 comp rx=0 tx=0).stop 2024-05-21T15:34:26.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.964+0000 7f70547ce640 1 -- 172.21.15.38:0/4089387715 shutdown_connections 2024-05-21T15:34:26.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.964+0000 7f70547ce640 1 --2- 172.21.15.38:0/4089387715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f704c108840 0x7f704c1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.964+0000 7f70547ce640 1 --2- 172.21.15.38:0/4089387715 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f70380a2b90 0x7f70380a50a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.964+0000 7f70547ce640 1 --2- 172.21.15.38:0/4089387715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f704c105ee0 0x7f704c1997a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.964+0000 7f70547ce640 1 --2- 172.21.15.38:0/4089387715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c103580 0x7f704c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:26.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.964+0000 7f70547ce640 1 -- 172.21.15.38:0/4089387715 >> 172.21.15.38:0/4089387715 conn(0x7f704c0fcb80 msgr2=0x7f704c104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:26.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.964+0000 7f70547ce640 1 -- 172.21.15.38:0/4089387715 shutdown_connections 2024-05-21T15:34:26.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:26.964+0000 7f70547ce640 1 -- 172.21.15.38:0/4089387715 wait complete. 2024-05-21T15:34:26.975 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:34:27.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.095+0000 7fe090c7a640 1 -- 172.21.15.38:0/2410430204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08c108740 msgr2=0x7fe08c10ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.095+0000 7fe090c7a640 1 --2- 172.21.15.38:0/2410430204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08c108740 0x7fe08c10ac50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe084009ca0 tx=0x7fe08402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.095+0000 7fe090c7a640 1 -- 172.21.15.38:0/2410430204 shutdown_connections 2024-05-21T15:34:27.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.095+0000 7fe090c7a640 1 --2- 172.21.15.38:0/2410430204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08c108740 0x7fe08c10ac50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.095+0000 7fe090c7a640 1 --2- 172.21.15.38:0/2410430204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe08c105bd0 0x7fe08c1080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.095+0000 7fe090c7a640 1 --2- 172.21.15.38:0/2410430204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe08c100e40 0x7fe08c101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.095+0000 7fe090c7a640 1 -- 172.21.15.38:0/2410430204 >> 172.21.15.38:0/2410430204 conn(0x7fe08c0fc330 msgr2=0x7fe08c0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:27.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.096+0000 7fe090c7a640 1 -- 172.21.15.38:0/2410430204 shutdown_connections 2024-05-21T15:34:27.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.096+0000 7fe090c7a640 1 -- 172.21.15.38:0/2410430204 wait complete. 2024-05-21T15:34:27.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.096+0000 7fe090c7a640 1 Processor -- start 2024-05-21T15:34:27.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.096+0000 7fe090c7a640 1 -- start start 2024-05-21T15:34:27.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe090c7a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe08c100e40 0x7fe08c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:27.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe090c7a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe08c105bd0 0x7fe08c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:27.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe090c7a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08c108740 0x7fe08c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:27.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe090c7a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c10e720 con 0x7fe08c108740 2024-05-21T15:34:27.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe090c7a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c10e5a0 con 0x7fe08c100e40 2024-05-21T15:34:27.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe090c7a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c10e8a0 con 0x7fe08c105bd0 2024-05-21T15:34:27.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe08ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08c108740 0x7fe08c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:27.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe08ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08c108740 0x7fe08c1a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59764/0 (socket says 172.21.15.38:59764) 2024-05-21T15:34:27.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe08ad9d640 1 -- 172.21.15.38:0/1650175668 learned_addr learned my addr 172.21.15.38:0/1650175668 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:27.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe08a59c640 1 --2- 172.21.15.38:0/1650175668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe08c100e40 0x7fe08c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:27.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe089d9b640 1 --2- 172.21.15.38:0/1650175668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe08c105bd0 0x7fe08c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:27.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe08ad9d640 1 -- 172.21.15.38:0/1650175668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe08c105bd0 msgr2=0x7fe08c199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe08ad9d640 1 --2- 172.21.15.38:0/1650175668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe08c105bd0 0x7fe08c199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe08ad9d640 1 -- 172.21.15.38:0/1650175668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe08c100e40 msgr2=0x7fe08c199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe08ad9d640 1 --2- 172.21.15.38:0/1650175668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe08c100e40 0x7fe08c199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe08ad9d640 1 -- 172.21.15.38:0/1650175668 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08c1a0d40 con 0x7fe08c108740 2024-05-21T15:34:27.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe08ad9d640 1 --2- 172.21.15.38:0/1650175668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08c108740 0x7fe08c1a06d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe084005e30 tx=0x7fe084031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.097+0000 7fe07b7fe640 1 -- 172.21.15.38:0/1650175668 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084003b70 con 0x7fe08c108740 2024-05-21T15:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.098+0000 7fe07b7fe640 1 -- 172.21.15.38:0/1650175668 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe084003d10 con 0x7fe08c108740 2024-05-21T15:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.098+0000 7fe090c7a640 1 -- 172.21.15.38:0/1650175668 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08c1a0fd0 con 0x7fe08c108740 2024-05-21T15:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.098+0000 7fe07b7fe640 1 -- 172.21.15.38:0/1650175668 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08402fb70 con 0x7fe08c108740 2024-05-21T15:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.098+0000 7fe090c7a640 1 -- 172.21.15.38:0/1650175668 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe08c1a1430 con 0x7fe08c108740 2024-05-21T15:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.099+0000 7fe090c7a640 1 -- 172.21.15.38:0/1650175668 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe08c0691c0 con 0x7fe08c108740 2024-05-21T15:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.101+0000 7fe07b7fe640 1 -- 172.21.15.38:0/1650175668 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe08402fd10 con 0x7fe08c108740 2024-05-21T15:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.101+0000 7fe07b7fe640 1 --2- 172.21.15.38:0/1650175668 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe0740a2b90 0x7fe0740a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.102+0000 7fe08a59c640 1 --2- 172.21.15.38:0/1650175668 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe0740a2b90 0x7fe0740a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.102+0000 7fe07b7fe640 1 -- 172.21.15.38:0/1650175668 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe0840f02d0 con 0x7fe08c108740 2024-05-21T15:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.102+0000 7fe08a59c640 1 --2- 172.21.15.38:0/1650175668 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe0740a2b90 0x7fe0740a50a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe08c06a600 tx=0x7fe07c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:27.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.105+0000 7fe07b7fe640 1 -- 172.21.15.38:0/1650175668 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe0840bd270 con 0x7fe08c108740 2024-05-21T15:34:27.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:27 smithi038 ceph-mon[31995]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s; 11 B/s, 0 objects/s recovering 2024-05-21T15:34:27.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:27 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2483176102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:27.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:27 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4089387715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:27.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.291+0000 7fe090c7a640 1 -- 172.21.15.38:0/1650175668 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe08c02d360 con 0x7fe08c108740 2024-05-21T15:34:27.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.292+0000 7fe07b7fe640 1 -- 172.21.15.38:0/1650175668 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fe0840bcb70 con 0x7fe08c108740 2024-05-21T15:34:27.291 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-05-21T15:34:27.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.294+0000 7fe090c7a640 1 -- 172.21.15.38:0/1650175668 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe0740a2b90 msgr2=0x7fe0740a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.294+0000 7fe090c7a640 1 --2- 172.21.15.38:0/1650175668 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe0740a2b90 0x7fe0740a50a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe08c06a600 tx=0x7fe07c005f90 comp rx=0 tx=0).stop 2024-05-21T15:34:27.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.294+0000 7fe090c7a640 1 -- 172.21.15.38:0/1650175668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08c108740 msgr2=0x7fe08c1a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.294+0000 7fe090c7a640 1 --2- 172.21.15.38:0/1650175668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08c108740 0x7fe08c1a06d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe084005e30 tx=0x7fe084031ee0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.294+0000 7fe090c7a640 1 -- 172.21.15.38:0/1650175668 shutdown_connections 2024-05-21T15:34:27.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.294+0000 7fe090c7a640 1 --2- 172.21.15.38:0/1650175668 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe0740a2b90 0x7fe0740a50a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.294+0000 7fe090c7a640 1 --2- 172.21.15.38:0/1650175668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08c108740 0x7fe08c1a06d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.294+0000 7fe090c7a640 1 --2- 172.21.15.38:0/1650175668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe08c105bd0 0x7fe08c199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.294+0000 7fe090c7a640 1 --2- 172.21.15.38:0/1650175668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe08c100e40 0x7fe08c199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.294+0000 7fe090c7a640 1 -- 172.21.15.38:0/1650175668 >> 172.21.15.38:0/1650175668 conn(0x7fe08c0fc330 msgr2=0x7fe08c106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:27.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.294+0000 7fe090c7a640 1 -- 172.21.15.38:0/1650175668 shutdown_connections 2024-05-21T15:34:27.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.295+0000 7fe090c7a640 1 -- 172.21.15.38:0/1650175668 wait complete. 2024-05-21T15:34:27.306 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.424+0000 7ff323185640 1 -- 172.21.15.38:0/2182558054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c103420 msgr2=0x7ff31c109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.424+0000 7ff323185640 1 --2- 172.21.15.38:0/2182558054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c103420 0x7ff31c109b60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff314009ca0 tx=0x7ff31402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.424+0000 7ff323185640 1 -- 172.21.15.38:0/2182558054 shutdown_connections 2024-05-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.424+0000 7ff323185640 1 --2- 172.21.15.38:0/2182558054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c103420 0x7ff31c109b60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.424+0000 7ff323185640 1 --2- 172.21.15.38:0/2182558054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff31c102920 0x7ff31c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.424+0000 7ff323185640 1 --2- 172.21.15.38:0/2182558054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff31c101f40 0x7ff31c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.424+0000 7ff323185640 1 -- 172.21.15.38:0/2182558054 >> 172.21.15.38:0/2182558054 conn(0x7ff31c0fc320 msgr2=0x7ff31c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.424+0000 7ff323185640 1 -- 172.21.15.38:0/2182558054 shutdown_connections 2024-05-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.424+0000 7ff323185640 1 -- 172.21.15.38:0/2182558054 wait complete. 2024-05-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.424+0000 7ff323185640 1 Processor -- start 2024-05-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.424+0000 7ff323185640 1 -- start start 2024-05-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff323185640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff31c101f40 0x7ff31c101240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff323185640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c102920 0x7ff31c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff323185640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff31c103420 0x7ff31c1a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff323185640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c10d630 con 0x7ff31c102920 2024-05-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff323185640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c10d4b0 con 0x7ff31c101f40 2024-05-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff323185640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c10d7b0 con 0x7ff31c103420 2024-05-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff31bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c102920 0x7ff31c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff31bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c102920 0x7ff31c19df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59770/0 (socket says 172.21.15.38:59770) 2024-05-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff31bfff640 1 -- 172.21.15.38:0/719876817 learned_addr learned my addr 172.21.15.38:0/719876817 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff320f21640 1 --2- 172.21.15.38:0/719876817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff31c101f40 0x7ff31c101240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff321722640 1 --2- 172.21.15.38:0/719876817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff31c103420 0x7ff31c1a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff31bfff640 1 -- 172.21.15.38:0/719876817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff31c103420 msgr2=0x7ff31c1a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff31bfff640 1 --2- 172.21.15.38:0/719876817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff31c103420 0x7ff31c1a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff31bfff640 1 -- 172.21.15.38:0/719876817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff31c101f40 msgr2=0x7ff31c101240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff31bfff640 1 --2- 172.21.15.38:0/719876817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff31c101f40 0x7ff31c101240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff31bfff640 1 -- 172.21.15.38:0/719876817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c1a0d60 con 0x7ff31c102920 2024-05-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff31bfff640 1 --2- 172.21.15.38:0/719876817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c102920 0x7ff31c19df90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff31000d770 tx=0x7ff31000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff319ffb640 1 -- 172.21.15.38:0/719876817 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310017070 con 0x7ff31c102920 2024-05-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff319ffb640 1 -- 172.21.15.38:0/719876817 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff310005500 con 0x7ff31c102920 2024-05-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff323185640 1 -- 172.21.15.38:0/719876817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31c1a0ff0 con 0x7ff31c102920 2024-05-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.425+0000 7ff319ffb640 1 -- 172.21.15.38:0/719876817 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310004e00 con 0x7ff31c102920 2024-05-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.426+0000 7ff323185640 1 -- 172.21.15.38:0/719876817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff31c1a1490 con 0x7ff31c102920 2024-05-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.427+0000 7ff323185640 1 -- 172.21.15.38:0/719876817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff31c1088f0 con 0x7ff31c102920 2024-05-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.428+0000 7ff319ffb640 1 -- 172.21.15.38:0/719876817 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff310004fa0 con 0x7ff31c102920 2024-05-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.428+0000 7ff319ffb640 1 --2- 172.21.15.38:0/719876817 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3040a2b90 0x7ff3040a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.429+0000 7ff320f21640 1 --2- 172.21.15.38:0/719876817 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3040a2b90 0x7ff3040a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.429+0000 7ff319ffb640 1 -- 172.21.15.38:0/719876817 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff3100ceac0 con 0x7ff31c102920 2024-05-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.429+0000 7ff320f21640 1 --2- 172.21.15.38:0/719876817 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3040a2b90 0x7ff3040a50a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff31c06a330 tx=0x7ff30c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.433+0000 7ff319ffb640 1 -- 172.21.15.38:0/719876817 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff31009b320 con 0x7ff31c102920 2024-05-21T15:34:27.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:27 smithi053 ceph-mon[36496]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s; 11 B/s, 0 objects/s recovering 2024-05-21T15:34:27.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:27 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2483176102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:27.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:27 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4089387715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:27.510 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:27 smithi045 ceph-mon[36409]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s; 11 B/s, 0 objects/s recovering 2024-05-21T15:34:27.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:27 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2483176102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:27.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:27 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4089387715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.618+0000 7ff323185640 1 -- 172.21.15.38:0/719876817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff31c02d4c0 con 0x7ff31c102920 2024-05-21T15:34:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.619+0000 7ff319ffb640 1 -- 172.21.15.38:0/719876817 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7ff31009f620 con 0x7ff31c102920 2024-05-21T15:34:27.619 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-05-21T15:34:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.621+0000 7ff323185640 1 -- 172.21.15.38:0/719876817 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3040a2b90 msgr2=0x7ff3040a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.621+0000 7ff323185640 1 --2- 172.21.15.38:0/719876817 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3040a2b90 0x7ff3040a50a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff31c06a330 tx=0x7ff30c00b390 comp rx=0 tx=0).stop 2024-05-21T15:34:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.621+0000 7ff323185640 1 -- 172.21.15.38:0/719876817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c102920 msgr2=0x7ff31c19df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.621+0000 7ff323185640 1 --2- 172.21.15.38:0/719876817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c102920 0x7ff31c19df90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff31000d770 tx=0x7ff31000dc30 comp rx=0 tx=0).stop 2024-05-21T15:34:27.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.621+0000 7ff323185640 1 -- 172.21.15.38:0/719876817 shutdown_connections 2024-05-21T15:34:27.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.621+0000 7ff323185640 1 --2- 172.21.15.38:0/719876817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff31c103420 0x7ff31c1a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.621+0000 7ff323185640 1 --2- 172.21.15.38:0/719876817 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3040a2b90 0x7ff3040a50a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.621+0000 7ff323185640 1 --2- 172.21.15.38:0/719876817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c102920 0x7ff31c19df90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.621+0000 7ff323185640 1 --2- 172.21.15.38:0/719876817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff31c101f40 0x7ff31c101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.621+0000 7ff323185640 1 -- 172.21.15.38:0/719876817 >> 172.21.15.38:0/719876817 conn(0x7ff31c0fc320 msgr2=0x7ff31c105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:27.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.621+0000 7ff323185640 1 -- 172.21.15.38:0/719876817 shutdown_connections 2024-05-21T15:34:27.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.621+0000 7ff323185640 1 -- 172.21.15.38:0/719876817 wait complete. 2024-05-21T15:34:27.632 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:34:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.747+0000 7f867b38f640 1 -- 172.21.15.38:0/1874526288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8674101140 msgr2=0x7f8674101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.747+0000 7f867b38f640 1 --2- 172.21.15.38:0/1874526288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8674101140 0x7f8674101e40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8670009900 tx=0x7f867002a410 comp rx=0 tx=0).stop 2024-05-21T15:34:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.747+0000 7f867b38f640 1 -- 172.21.15.38:0/1874526288 shutdown_connections 2024-05-21T15:34:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.747+0000 7f867b38f640 1 --2- 172.21.15.38:0/1874526288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8674107080 0x7f86741094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.747+0000 7f867b38f640 1 --2- 172.21.15.38:0/1874526288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8674102380 0x7f8674102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.747+0000 7f867b38f640 1 --2- 172.21.15.38:0/1874526288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8674101140 0x7f8674101e40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.747+0000 7f867b38f640 1 -- 172.21.15.38:0/1874526288 >> 172.21.15.38:0/1874526288 conn(0x7f86740fc330 msgr2=0x7f86740fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.748+0000 7f867b38f640 1 -- 172.21.15.38:0/1874526288 shutdown_connections 2024-05-21T15:34:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.748+0000 7f867b38f640 1 -- 172.21.15.38:0/1874526288 wait complete. 2024-05-21T15:34:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.748+0000 7f867b38f640 1 Processor -- start 2024-05-21T15:34:27.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.748+0000 7f867b38f640 1 -- start start 2024-05-21T15:34:27.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.748+0000 7f867b38f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8674101140 0x7f867410aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.748+0000 7f867b38f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8674102380 0x7f86741a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.748+0000 7f867b38f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8674107080 0x7f86741a2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.748+0000 7f867b38f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f867410d680 con 0x7f8674101140 2024-05-21T15:34:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.748+0000 7f867b38f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f867410d500 con 0x7f8674102380 2024-05-21T15:34:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.748+0000 7f867b38f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f867410d800 con 0x7f8674107080 2024-05-21T15:34:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.748+0000 7f867912b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8674101140 0x7f867410aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.748+0000 7f867912b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8674101140 0x7f867410aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59794/0 (socket says 172.21.15.38:59794) 2024-05-21T15:34:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.749+0000 7f867912b640 1 -- 172.21.15.38:0/3163715467 learned_addr learned my addr 172.21.15.38:0/3163715467 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.749+0000 7f867992c640 1 --2- 172.21.15.38:0/3163715467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8674107080 0x7f86741a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.749+0000 7f867892a640 1 --2- 172.21.15.38:0/3163715467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8674102380 0x7f86741a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.749+0000 7f867912b640 1 -- 172.21.15.38:0/3163715467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8674107080 msgr2=0x7f86741a2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.749+0000 7f867912b640 1 --2- 172.21.15.38:0/3163715467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8674107080 0x7f86741a2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.749+0000 7f867912b640 1 -- 172.21.15.38:0/3163715467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8674102380 msgr2=0x7f86741a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.749+0000 7f867912b640 1 --2- 172.21.15.38:0/3163715467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8674102380 0x7f86741a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.749+0000 7f867912b640 1 -- 172.21.15.38:0/3163715467 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86741a2f80 con 0x7f8674101140 2024-05-21T15:34:27.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.749+0000 7f867912b640 1 --2- 172.21.15.38:0/3163715467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8674101140 0x7f867410aa00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8670031440 tx=0x7f8670032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:27.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.749+0000 7f866a7fc640 1 -- 172.21.15.38:0/3163715467 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867003f030 con 0x7f8674101140 2024-05-21T15:34:27.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.749+0000 7f866a7fc640 1 -- 172.21.15.38:0/3163715467 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8670003ce0 con 0x7f8674101140 2024-05-21T15:34:27.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.749+0000 7f867b38f640 1 -- 172.21.15.38:0/3163715467 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86741a3210 con 0x7f8674101140 2024-05-21T15:34:27.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.749+0000 7f866a7fc640 1 -- 172.21.15.38:0/3163715467 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8670035070 con 0x7f8674101140 2024-05-21T15:34:27.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.749+0000 7f867b38f640 1 -- 172.21.15.38:0/3163715467 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86741a3700 con 0x7f8674101140 2024-05-21T15:34:27.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.750+0000 7f867b38f640 1 -- 172.21.15.38:0/3163715467 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8674105780 con 0x7f8674101140 2024-05-21T15:34:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.752+0000 7f866a7fc640 1 -- 172.21.15.38:0/3163715467 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8670042030 con 0x7f8674101140 2024-05-21T15:34:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.752+0000 7f866a7fc640 1 --2- 172.21.15.38:0/3163715467 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86600a2c30 0x7f86600a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.753+0000 7f867892a640 1 --2- 172.21.15.38:0/3163715467 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86600a2c30 0x7f86600a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.753+0000 7f866a7fc640 1 -- 172.21.15.38:0/3163715467 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f86700efb10 con 0x7f8674101140 2024-05-21T15:34:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.753+0000 7f867892a640 1 --2- 172.21.15.38:0/3163715467 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86600a2c30 0x7f86600a5140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8664005fd0 tx=0x7f8664005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.756+0000 7f866a7fc640 1 -- 172.21.15.38:0/3163715467 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f86700bca20 con 0x7f8674101140 2024-05-21T15:34:27.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.979+0000 7f867b38f640 1 -- 172.21.15.38:0/3163715467 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f867402d2a0 con 0x7f8674101140 2024-05-21T15:34:27.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.980+0000 7f866a7fc640 1 -- 172.21.15.38:0/3163715467 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11914 (secure 0 0 0) 0x7f86700bc320 con 0x7f8674101140 2024-05-21T15:34:27.980 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:27.980 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:32:40.979368+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24550,24515,24602,24523,24551]},"id":1}]} 2024-05-21T15:34:27.980 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-05-21T15:34:27.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.982+0000 7f867b38f640 1 -- 172.21.15.38:0/3163715467 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86600a2c30 msgr2=0x7f86600a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.982+0000 7f867b38f640 1 --2- 172.21.15.38:0/3163715467 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86600a2c30 0x7f86600a5140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8664005fd0 tx=0x7f8664005850 comp rx=0 tx=0).stop 2024-05-21T15:34:27.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.982+0000 7f867b38f640 1 -- 172.21.15.38:0/3163715467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8674101140 msgr2=0x7f867410aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:27.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.982+0000 7f867b38f640 1 --2- 172.21.15.38:0/3163715467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8674101140 0x7f867410aa00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8670031440 tx=0x7f8670032ce0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.982+0000 7f867b38f640 1 -- 172.21.15.38:0/3163715467 shutdown_connections 2024-05-21T15:34:27.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.982+0000 7f867b38f640 1 --2- 172.21.15.38:0/3163715467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8674107080 0x7f86741a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.982+0000 7f867b38f640 1 --2- 172.21.15.38:0/3163715467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8674102380 0x7f86741a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.982+0000 7f867b38f640 1 --2- 172.21.15.38:0/3163715467 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86600a2c30 0x7f86600a5140 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.982+0000 7f867b38f640 1 --2- 172.21.15.38:0/3163715467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8674101140 0x7f867410aa00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:27.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.982+0000 7f867b38f640 1 -- 172.21.15.38:0/3163715467 >> 172.21.15.38:0/3163715467 conn(0x7f86740fc330 msgr2=0x7f86740fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:27.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.982+0000 7f867b38f640 1 -- 172.21.15.38:0/3163715467 shutdown_connections 2024-05-21T15:34:27.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:27.982+0000 7f867b38f640 1 -- 172.21.15.38:0/3163715467 wait complete. 2024-05-21T15:34:27.994 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:34:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.111+0000 7fcaa7c06640 1 -- 172.21.15.38:0/656062415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa0108750 msgr2=0x7fcaa010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.111+0000 7fcaa7c06640 1 --2- 172.21.15.38:0/656062415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa0108750 0x7fcaa010ac60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fca98009ca0 tx=0x7fca9802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.111+0000 7fcaa7c06640 1 -- 172.21.15.38:0/656062415 shutdown_connections 2024-05-21T15:34:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.111+0000 7fcaa7c06640 1 --2- 172.21.15.38:0/656062415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa0108750 0x7fcaa010ac60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.111+0000 7fcaa7c06640 1 --2- 172.21.15.38:0/656062415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa0105be0 0x7fcaa01080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.111+0000 7fcaa7c06640 1 --2- 172.21.15.38:0/656062415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa0100e00 0x7fcaa0101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.111+0000 7fcaa7c06640 1 -- 172.21.15.38:0/656062415 >> 172.21.15.38:0/656062415 conn(0x7fcaa00fc330 msgr2=0x7fcaa00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.112+0000 7fcaa7c06640 1 -- 172.21.15.38:0/656062415 shutdown_connections 2024-05-21T15:34:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.112+0000 7fcaa7c06640 1 -- 172.21.15.38:0/656062415 wait complete. 2024-05-21T15:34:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.112+0000 7fcaa7c06640 1 Processor -- start 2024-05-21T15:34:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.112+0000 7fcaa7c06640 1 -- start start 2024-05-21T15:34:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.112+0000 7fcaa7c06640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa0100e00 0x7fcaa0199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.112+0000 7fcaa7c06640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa0105be0 0x7fcaa0199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.112+0000 7fcaa7c06640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa0108750 0x7fcaa01a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.112+0000 7fcaa7c06640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa010e710 con 0x7fcaa0105be0 2024-05-21T15:34:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.112+0000 7fcaa7c06640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa010e590 con 0x7fcaa0108750 2024-05-21T15:34:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.112+0000 7fcaa7c06640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa010e890 con 0x7fcaa0100e00 2024-05-21T15:34:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fcaa51a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa0105be0 0x7fcaa0199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:28.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fcaa51a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa0105be0 0x7fcaa0199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59822/0 (socket says 172.21.15.38:59822) 2024-05-21T15:34:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fcaa51a1640 1 -- 172.21.15.38:0/2886157079 learned_addr learned my addr 172.21.15.38:0/2886157079 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fcaa61a3640 1 --2- 172.21.15.38:0/2886157079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa0108750 0x7fcaa01a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fcaa59a2640 1 --2- 172.21.15.38:0/2886157079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa0100e00 0x7fcaa0199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fcaa51a1640 1 -- 172.21.15.38:0/2886157079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa0100e00 msgr2=0x7fcaa0199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fcaa51a1640 1 --2- 172.21.15.38:0/2886157079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa0100e00 0x7fcaa0199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fcaa51a1640 1 -- 172.21.15.38:0/2886157079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa0108750 msgr2=0x7fcaa01a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fcaa51a1640 1 --2- 172.21.15.38:0/2886157079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa0108750 0x7fcaa01a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fcaa51a1640 1 -- 172.21.15.38:0/2886157079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa01a0c70 con 0x7fcaa0105be0 2024-05-21T15:34:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fcaa51a1640 1 --2- 172.21.15.38:0/2886157079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa0105be0 0x7fcaa0199670 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fca9000d950 tx=0x7fca9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fca96ffd640 1 -- 172.21.15.38:0/2886157079 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90017030 con 0x7fcaa0105be0 2024-05-21T15:34:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fca96ffd640 1 -- 172.21.15.38:0/2886157079 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca90004020 con 0x7fcaa0105be0 2024-05-21T15:34:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fcaa7c06640 1 -- 172.21.15.38:0/2886157079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa01a0f60 con 0x7fcaa0105be0 2024-05-21T15:34:28.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fca96ffd640 1 -- 172.21.15.38:0/2886157079 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca900047a0 con 0x7fcaa0105be0 2024-05-21T15:34:28.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.113+0000 7fcaa7c06640 1 -- 172.21.15.38:0/2886157079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa01a13d0 con 0x7fcaa0105be0 2024-05-21T15:34:28.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.114+0000 7fcaa7c06640 1 -- 172.21.15.38:0/2886157079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa00fde50 con 0x7fcaa0105be0 2024-05-21T15:34:28.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.116+0000 7fca96ffd640 1 -- 172.21.15.38:0/2886157079 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca900041d0 con 0x7fcaa0105be0 2024-05-21T15:34:28.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.117+0000 7fca96ffd640 1 --2- 172.21.15.38:0/2886157079 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fca8c0a2c30 0x7fca8c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:28.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.117+0000 7fcaa59a2640 1 --2- 172.21.15.38:0/2886157079 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fca8c0a2c30 0x7fca8c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:28.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.117+0000 7fca96ffd640 1 -- 172.21.15.38:0/2886157079 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fca900cf060 con 0x7fcaa0105be0 2024-05-21T15:34:28.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.117+0000 7fcaa59a2640 1 --2- 172.21.15.38:0/2886157079 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fca8c0a2c30 0x7fca8c0a5140 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0103ad0 tx=0x7fca9c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:28.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.121+0000 7fca96ffd640 1 -- 172.21.15.38:0/2886157079 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fca9009bf70 con 0x7fcaa0105be0 2024-05-21T15:34:28.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:28 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1650175668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:28.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:28 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/719876817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:28.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:28 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3163715467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.313+0000 7fcaa7c06640 1 -- 172.21.15.38:0/2886157079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcaa002d470 con 0x7fcaa0105be0 2024-05-21T15:34:28.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.315+0000 7fca96ffd640 1 -- 172.21.15.38:0/2886157079 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22704 (secure 0 0 0) 0x7fca9009b870 con 0x7fcaa0105be0 2024-05-21T15:34:28.315 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:28.315 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":90,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:34:23.996759+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:34:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.317+0000 7fcaa7c06640 1 -- 172.21.15.38:0/2886157079 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fca8c0a2c30 msgr2=0x7fca8c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.317+0000 7fcaa7c06640 1 --2- 172.21.15.38:0/2886157079 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fca8c0a2c30 0x7fca8c0a5140 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0103ad0 tx=0x7fca9c00b390 comp rx=0 tx=0).stop 2024-05-21T15:34:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.317+0000 7fcaa7c06640 1 -- 172.21.15.38:0/2886157079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa0105be0 msgr2=0x7fcaa0199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.317+0000 7fcaa7c06640 1 --2- 172.21.15.38:0/2886157079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa0105be0 0x7fcaa0199670 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fca9000d950 tx=0x7fca9000de10 comp rx=0 tx=0).stop 2024-05-21T15:34:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.317+0000 7fcaa7c06640 1 -- 172.21.15.38:0/2886157079 shutdown_connections 2024-05-21T15:34:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.317+0000 7fcaa7c06640 1 --2- 172.21.15.38:0/2886157079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa0108750 0x7fcaa01a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.317+0000 7fcaa7c06640 1 --2- 172.21.15.38:0/2886157079 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fca8c0a2c30 0x7fca8c0a5140 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.317+0000 7fcaa7c06640 1 --2- 172.21.15.38:0/2886157079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaa0105be0 0x7fcaa0199670 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.317+0000 7fcaa7c06640 1 --2- 172.21.15.38:0/2886157079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa0100e00 0x7fcaa0199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.317+0000 7fcaa7c06640 1 -- 172.21.15.38:0/2886157079 >> 172.21.15.38:0/2886157079 conn(0x7fcaa00fc330 msgr2=0x7fcaa0106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.317+0000 7fcaa7c06640 1 -- 172.21.15.38:0/2886157079 shutdown_connections 2024-05-21T15:34:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.317+0000 7fcaa7c06640 1 -- 172.21.15.38:0/2886157079 wait complete. 2024-05-21T15:34:28.328 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:27:50.659367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:32:31.645647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:32:32.955451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:34:28.329 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:34:28.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.447+0000 7f664907a640 1 -- 172.21.15.38:0/2256733870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6644102ac0 msgr2=0x7f66441037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.447+0000 7f664907a640 1 --2- 172.21.15.38:0/2256733870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6644102ac0 0x7f66441037c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6634009900 tx=0x7f663402a410 comp rx=0 tx=0).stop 2024-05-21T15:34:28.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.447+0000 7f664907a640 1 -- 172.21.15.38:0/2256733870 shutdown_connections 2024-05-21T15:34:28.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.447+0000 7f664907a640 1 --2- 172.21.15.38:0/2256733870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6644108a00 0x7f664410ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.447+0000 7f664907a640 1 --2- 172.21.15.38:0/2256733870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6644103d00 0x7f6644104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.447+0000 7f664907a640 1 --2- 172.21.15.38:0/2256733870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6644102ac0 0x7f66441037c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.447+0000 7f664907a640 1 -- 172.21.15.38:0/2256733870 >> 172.21.15.38:0/2256733870 conn(0x7f66440fdcb0 msgr2=0x7f6644100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:28.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.447+0000 7f664907a640 1 -- 172.21.15.38:0/2256733870 shutdown_connections 2024-05-21T15:34:28.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.447+0000 7f664907a640 1 -- 172.21.15.38:0/2256733870 wait complete. 2024-05-21T15:34:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.448+0000 7f664907a640 1 Processor -- start 2024-05-21T15:34:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.448+0000 7f664907a640 1 -- start start 2024-05-21T15:34:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.448+0000 7f664907a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6644102ac0 0x7f6644199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.448+0000 7f664907a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6644103d00 0x7f66441997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.448+0000 7f664907a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6644108a00 0x7f66441a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.448+0000 7f664907a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f664410e710 con 0x7f6644102ac0 2024-05-21T15:34:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.448+0000 7f664907a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f664410e590 con 0x7f6644103d00 2024-05-21T15:34:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.448+0000 7f664907a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f664410e890 con 0x7f6644108a00 2024-05-21T15:34:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.448+0000 7f6642d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6644102ac0 0x7f6644199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.448+0000 7f6642d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6644102ac0 0x7f6644199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59836/0 (socket says 172.21.15.38:59836) 2024-05-21T15:34:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.448+0000 7f6642d9d640 1 -- 172.21.15.38:0/2993036033 learned_addr learned my addr 172.21.15.38:0/2993036033 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.448+0000 7f664259c640 1 --2- 172.21.15.38:0/2993036033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6644103d00 0x7f66441997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.448+0000 7f664359e640 1 --2- 172.21.15.38:0/2993036033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6644108a00 0x7f66441a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:28.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.449+0000 7f6642d9d640 1 -- 172.21.15.38:0/2993036033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6644108a00 msgr2=0x7f66441a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.449+0000 7f6642d9d640 1 --2- 172.21.15.38:0/2993036033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6644108a00 0x7f66441a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.449+0000 7f6642d9d640 1 -- 172.21.15.38:0/2993036033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6644103d00 msgr2=0x7f66441997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.449+0000 7f6642d9d640 1 --2- 172.21.15.38:0/2993036033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6644103d00 0x7f66441997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.449+0000 7f6642d9d640 1 -- 172.21.15.38:0/2993036033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66441a0dc0 con 0x7f6644102ac0 2024-05-21T15:34:28.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.449+0000 7f6642d9d640 1 --2- 172.21.15.38:0/2993036033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6644102ac0 0x7f6644199280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6634006010 tx=0x7f6634031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:28.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.449+0000 7f662bfff640 1 -- 172.21.15.38:0/2993036033 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663403f030 con 0x7f6644102ac0 2024-05-21T15:34:28.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.449+0000 7f662bfff640 1 -- 172.21.15.38:0/2993036033 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6634003ad0 con 0x7f6644102ac0 2024-05-21T15:34:28.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.449+0000 7f664907a640 1 -- 172.21.15.38:0/2993036033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66441a1050 con 0x7f6644102ac0 2024-05-21T15:34:28.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.449+0000 7f662bfff640 1 -- 172.21.15.38:0/2993036033 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634035070 con 0x7f6644102ac0 2024-05-21T15:34:28.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.449+0000 7f664907a640 1 -- 172.21.15.38:0/2993036033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66441a1540 con 0x7f6644102ac0 2024-05-21T15:34:28.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.450+0000 7f664907a640 1 -- 172.21.15.38:0/2993036033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6644068d70 con 0x7f6644102ac0 2024-05-21T15:34:28.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.451+0000 7f662bfff640 1 -- 172.21.15.38:0/2993036033 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6634042030 con 0x7f6644102ac0 2024-05-21T15:34:28.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.452+0000 7f662bfff640 1 --2- 172.21.15.38:0/2993036033 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f662c0a2b90 0x7f662c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:28.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.452+0000 7f662bfff640 1 -- 172.21.15.38:0/2993036033 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f66340ef690 con 0x7f6644102ac0 2024-05-21T15:34:28.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.452+0000 7f664259c640 1 --2- 172.21.15.38:0/2993036033 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f662c0a2b90 0x7f662c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:28.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.453+0000 7f664259c640 1 --2- 172.21.15.38:0/2993036033 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f662c0a2b90 0x7f662c0a50a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f664419a320 tx=0x7f663800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:28.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:28 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1650175668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:28.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:28 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/719876817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:28.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:28 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3163715467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:28.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.457+0000 7f662bfff640 1 -- 172.21.15.38:0/2993036033 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f66340bc540 con 0x7f6644102ac0 2024-05-21T15:34:28.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:28 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1650175668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:28.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:28 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/719876817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:28.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:28 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3163715467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:28.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.641+0000 7f664907a640 1 -- 172.21.15.38:0/2993036033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6644062750 con 0x7f6644102ac0 2024-05-21T15:34:28.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.641+0000 7f662bfff640 1 -- 172.21.15.38:0/2993036033 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f66340bbe40 con 0x7f6644102ac0 2024-05-21T15:34:28.641 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-05-21T15:34:28.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.643+0000 7f664907a640 1 -- 172.21.15.38:0/2993036033 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f662c0a2b90 msgr2=0x7f662c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.643+0000 7f664907a640 1 --2- 172.21.15.38:0/2993036033 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f662c0a2b90 0x7f662c0a50a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f664419a320 tx=0x7f663800b390 comp rx=0 tx=0).stop 2024-05-21T15:34:28.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.643+0000 7f664907a640 1 -- 172.21.15.38:0/2993036033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6644102ac0 msgr2=0x7f6644199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.643+0000 7f664907a640 1 --2- 172.21.15.38:0/2993036033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6644102ac0 0x7f6644199280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6634006010 tx=0x7f6634031b60 comp rx=0 tx=0).stop 2024-05-21T15:34:28.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.643+0000 7f664907a640 1 -- 172.21.15.38:0/2993036033 shutdown_connections 2024-05-21T15:34:28.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.643+0000 7f664907a640 1 --2- 172.21.15.38:0/2993036033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6644108a00 0x7f66441a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.643+0000 7f664907a640 1 --2- 172.21.15.38:0/2993036033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6644103d00 0x7f66441997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.643+0000 7f664907a640 1 --2- 172.21.15.38:0/2993036033 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f662c0a2b90 0x7f662c0a50a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.643+0000 7f664907a640 1 --2- 172.21.15.38:0/2993036033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6644102ac0 0x7f6644199280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.643+0000 7f664907a640 1 -- 172.21.15.38:0/2993036033 >> 172.21.15.38:0/2993036033 conn(0x7f66440fdcb0 msgr2=0x7f66441049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:28.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.643+0000 7f664907a640 1 -- 172.21.15.38:0/2993036033 shutdown_connections 2024-05-21T15:34:28.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.643+0000 7f664907a640 1 -- 172.21.15.38:0/2993036033 wait complete. 2024-05-21T15:34:28.655 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:34:28.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.773+0000 7f1465777640 1 -- 172.21.15.38:0/2929992358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14601078f0 msgr2=0x7f1460109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.773+0000 7f1465777640 1 --2- 172.21.15.38:0/2929992358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14601078f0 0x7f1460109d10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1458009ca0 tx=0x7f145802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.773+0000 7f1465777640 1 -- 172.21.15.38:0/2929992358 shutdown_connections 2024-05-21T15:34:28.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.773+0000 7f1465777640 1 --2- 172.21.15.38:0/2929992358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14601078f0 0x7f1460109d10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.773+0000 7f1465777640 1 --2- 172.21.15.38:0/2929992358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1460102bf0 0x7f1460103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.773+0000 7f1465777640 1 --2- 172.21.15.38:0/2929992358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14601019b0 0x7f14601026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.773+0000 7f1465777640 1 -- 172.21.15.38:0/2929992358 >> 172.21.15.38:0/2929992358 conn(0x7f14600fcbe0 msgr2=0x7f14600ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:28.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.773+0000 7f1465777640 1 -- 172.21.15.38:0/2929992358 shutdown_connections 2024-05-21T15:34:28.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.773+0000 7f1465777640 1 -- 172.21.15.38:0/2929992358 wait complete. 2024-05-21T15:34:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1465777640 1 Processor -- start 2024-05-21T15:34:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1465777640 1 -- start start 2024-05-21T15:34:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1465777640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14601019b0 0x7f1460198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1465777640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1460102bf0 0x7f14601986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1465777640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14601078f0 0x7f146019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1465777640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f146010d680 con 0x7f1460102bf0 2024-05-21T15:34:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1465777640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f146010d500 con 0x7f14601019b0 2024-05-21T15:34:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1465777640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f146010d800 con 0x7f14601078f0 2024-05-21T15:34:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1457fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1460102bf0 0x7f14601986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1457fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1460102bf0 0x7f14601986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59850/0 (socket says 172.21.15.38:59850) 2024-05-21T15:34:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1457fff640 1 -- 172.21.15.38:0/824454627 learned_addr learned my addr 172.21.15.38:0/824454627 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f145effd640 1 --2- 172.21.15.38:0/824454627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14601019b0 0x7f1460198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:28.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f145f7fe640 1 --2- 172.21.15.38:0/824454627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14601078f0 0x7f146019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:28.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1457fff640 1 -- 172.21.15.38:0/824454627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14601078f0 msgr2=0x7f146019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1457fff640 1 --2- 172.21.15.38:0/824454627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14601078f0 0x7f146019f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1457fff640 1 -- 172.21.15.38:0/824454627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14601019b0 msgr2=0x7f1460198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1457fff640 1 --2- 172.21.15.38:0/824454627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14601019b0 0x7f1460198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.774+0000 7f1457fff640 1 -- 172.21.15.38:0/824454627 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146019fca0 con 0x7f1460102bf0 2024-05-21T15:34:28.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.775+0000 7f1457fff640 1 --2- 172.21.15.38:0/824454627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1460102bf0 0x7f14601986a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f144800d950 tx=0x7f144800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:28.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.775+0000 7f145cff9640 1 -- 172.21.15.38:0/824454627 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448017030 con 0x7f1460102bf0 2024-05-21T15:34:28.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.775+0000 7f145cff9640 1 -- 172.21.15.38:0/824454627 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1448004020 con 0x7f1460102bf0 2024-05-21T15:34:28.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.775+0000 7f1465777640 1 -- 172.21.15.38:0/824454627 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146019ff90 con 0x7f1460102bf0 2024-05-21T15:34:28.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.775+0000 7f145cff9640 1 -- 172.21.15.38:0/824454627 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14480047a0 con 0x7f1460102bf0 2024-05-21T15:34:28.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.775+0000 7f1465777640 1 -- 172.21.15.38:0/824454627 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14601a0450 con 0x7f1460102bf0 2024-05-21T15:34:28.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.776+0000 7f1465777640 1 -- 172.21.15.38:0/824454627 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1460106700 con 0x7f1460102bf0 2024-05-21T15:34:28.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.777+0000 7f145cff9640 1 -- 172.21.15.38:0/824454627 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f14480041d0 con 0x7f1460102bf0 2024-05-21T15:34:28.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.778+0000 7f145cff9640 1 --2- 172.21.15.38:0/824454627 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f144c0a2c10 0x7f144c0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:28.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.778+0000 7f145effd640 1 --2- 172.21.15.38:0/824454627 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f144c0a2c10 0x7f144c0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:28.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.778+0000 7f145cff9640 1 -- 172.21.15.38:0/824454627 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f14480cf0f0 con 0x7f1460102bf0 2024-05-21T15:34:28.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.779+0000 7f145effd640 1 --2- 172.21.15.38:0/824454627 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f144c0a2c10 0x7f144c0a5120 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1450005fd0 tx=0x7f145000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:28.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.781+0000 7f145cff9640 1 -- 172.21.15.38:0/824454627 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f144809c020 con 0x7f1460102bf0 2024-05-21T15:34:28.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.966+0000 7f1465777640 1 -- 172.21.15.38:0/824454627 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f146002d4f0 con 0x7f1460102bf0 2024-05-21T15:34:28.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.966+0000 7f145cff9640 1 -- 172.21.15.38:0/824454627 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f144809b920 con 0x7f1460102bf0 2024-05-21T15:34:28.966 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-05-21T15:34:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.968+0000 7f1465777640 1 -- 172.21.15.38:0/824454627 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f144c0a2c10 msgr2=0x7f144c0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.968+0000 7f1465777640 1 --2- 172.21.15.38:0/824454627 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f144c0a2c10 0x7f144c0a5120 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1450005fd0 tx=0x7f145000b390 comp rx=0 tx=0).stop 2024-05-21T15:34:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.968+0000 7f1465777640 1 -- 172.21.15.38:0/824454627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1460102bf0 msgr2=0x7f14601986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.968+0000 7f1465777640 1 --2- 172.21.15.38:0/824454627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1460102bf0 0x7f14601986a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f144800d950 tx=0x7f144800de10 comp rx=0 tx=0).stop 2024-05-21T15:34:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.968+0000 7f1465777640 1 -- 172.21.15.38:0/824454627 shutdown_connections 2024-05-21T15:34:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.968+0000 7f1465777640 1 --2- 172.21.15.38:0/824454627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14601078f0 0x7f146019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.968+0000 7f1465777640 1 --2- 172.21.15.38:0/824454627 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f144c0a2c10 0x7f144c0a5120 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.968+0000 7f1465777640 1 --2- 172.21.15.38:0/824454627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1460102bf0 0x7f14601986a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.968+0000 7f1465777640 1 --2- 172.21.15.38:0/824454627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14601019b0 0x7f1460198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.968+0000 7f1465777640 1 -- 172.21.15.38:0/824454627 >> 172.21.15.38:0/824454627 conn(0x7f14600fcbe0 msgr2=0x7f14601038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.968+0000 7f1465777640 1 -- 172.21.15.38:0/824454627 shutdown_connections 2024-05-21T15:34:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:28.968+0000 7f1465777640 1 -- 172.21.15.38:0/824454627 wait complete. 2024-05-21T15:34:28.980 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:34:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.098+0000 7f5a8ec73640 1 -- 172.21.15.38:0/467686377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a88102bf0 msgr2=0x7f5a88103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.098+0000 7f5a8ec73640 1 --2- 172.21.15.38:0/467686377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a88102bf0 0x7f5a88103000 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009900 tx=0x7f5a7402a410 comp rx=0 tx=0).stop 2024-05-21T15:34:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.098+0000 7f5a8ec73640 1 -- 172.21.15.38:0/467686377 shutdown_connections 2024-05-21T15:34:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.098+0000 7f5a8ec73640 1 --2- 172.21.15.38:0/467686377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a881078f0 0x7f5a88109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.098+0000 7f5a8ec73640 1 --2- 172.21.15.38:0/467686377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a88102bf0 0x7f5a88103000 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.098+0000 7f5a8ec73640 1 --2- 172.21.15.38:0/467686377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a881019b0 0x7f5a881026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.098+0000 7f5a8ec73640 1 -- 172.21.15.38:0/467686377 >> 172.21.15.38:0/467686377 conn(0x7f5a880fcbe0 msgr2=0x7f5a880ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.098+0000 7f5a8ec73640 1 -- 172.21.15.38:0/467686377 shutdown_connections 2024-05-21T15:34:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.098+0000 7f5a8ec73640 1 -- 172.21.15.38:0/467686377 wait complete. 2024-05-21T15:34:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.099+0000 7f5a8ec73640 1 Processor -- start 2024-05-21T15:34:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.099+0000 7f5a8ec73640 1 -- start start 2024-05-21T15:34:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.099+0000 7f5a8ec73640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a881019b0 0x7f5a88199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.099+0000 7f5a8ec73640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a88102bf0 0x7f5a88199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.099+0000 7f5a8ec73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a881078f0 0x7f5a881a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.099+0000 7f5a8ec73640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8810e730 con 0x7f5a881078f0 2024-05-21T15:34:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.099+0000 7f5a8ec73640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8810e5b0 con 0x7f5a881019b0 2024-05-21T15:34:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.099+0000 7f5a8ec73640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8810e8b0 con 0x7f5a88102bf0 2024-05-21T15:34:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.099+0000 7f5a8d210640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a881078f0 0x7f5a881a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.099+0000 7f5a8d210640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a881078f0 0x7f5a881a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59872/0 (socket says 172.21.15.38:59872) 2024-05-21T15:34:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.099+0000 7f5a8d210640 1 -- 172.21.15.38:0/2245582515 learned_addr learned my addr 172.21.15.38:0/2245582515 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.099+0000 7f5a87fff640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a88102bf0 0x7f5a88199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.100+0000 7f5a8ca0f640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a881019b0 0x7f5a88199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.100+0000 7f5a8d210640 1 -- 172.21.15.38:0/2245582515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a88102bf0 msgr2=0x7f5a88199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.100+0000 7f5a8d210640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a88102bf0 0x7f5a88199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.100+0000 7f5a8d210640 1 -- 172.21.15.38:0/2245582515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a881019b0 msgr2=0x7f5a88199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.100+0000 7f5a8d210640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a881019b0 0x7f5a88199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.100+0000 7f5a8d210640 1 -- 172.21.15.38:0/2245582515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a881a0d90 con 0x7f5a881078f0 2024-05-21T15:34:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.100+0000 7f5a8ca0f640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a881019b0 0x7f5a88199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.100+0000 7f5a8d210640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a881078f0 0x7f5a881a0720 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000eaa0 tx=0x7f5a8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.100+0000 7f5a85ffb640 1 -- 172.21.15.38:0/2245582515 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8000cdb0 con 0x7f5a881078f0 2024-05-21T15:34:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.100+0000 7f5a8ec73640 1 -- 172.21.15.38:0/2245582515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a881a1080 con 0x7f5a881078f0 2024-05-21T15:34:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.100+0000 7f5a85ffb640 1 -- 172.21.15.38:0/2245582515 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a80005020 con 0x7f5a881078f0 2024-05-21T15:34:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.100+0000 7f5a85ffb640 1 -- 172.21.15.38:0/2245582515 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a800116b0 con 0x7f5a881078f0 2024-05-21T15:34:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.100+0000 7f5a8ec73640 1 -- 172.21.15.38:0/2245582515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a881a15d0 con 0x7f5a881078f0 2024-05-21T15:34:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.101+0000 7f5a8ec73640 1 -- 172.21.15.38:0/2245582515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a88106700 con 0x7f5a881078f0 2024-05-21T15:34:29.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.103+0000 7f5a85ffb640 1 -- 172.21.15.38:0/2245582515 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5a80005560 con 0x7f5a881078f0 2024-05-21T15:34:29.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.104+0000 7f5a85ffb640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5a780a2b90 0x7f5a780a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:29.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.104+0000 7f5a8ca0f640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5a780a2b90 0x7f5a780a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:29.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.104+0000 7f5a85ffb640 1 -- 172.21.15.38:0/2245582515 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f5a800cf280 con 0x7f5a881078f0 2024-05-21T15:34:29.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.104+0000 7f5a8ca0f640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5a780a2b90 0x7f5a780a50a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c005d00 tx=0x7f5a7c005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:29.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.108+0000 7f5a85ffb640 1 -- 172.21.15.38:0/2245582515 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5a80016030 con 0x7f5a881078f0 2024-05-21T15:34:29.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:29 smithi038 ceph-mon[31995]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 11 B/s, 0 objects/s recovering 2024-05-21T15:34:29.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:29 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2886157079' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:29.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:29 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2993036033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:29.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:29 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/824454627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:29.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.293+0000 7f5a8ec73640 1 -- 172.21.15.38:0/2245582515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5a88062750 con 0x7f5a881078f0 2024-05-21T15:34:29.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.294+0000 7f5a85ffb640 1 -- 172.21.15.38:0/2245582515 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f5a8009c190 con 0x7f5a881078f0 2024-05-21T15:34:29.293 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-05-21T15:34:29.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.296+0000 7f5a8ec73640 1 -- 172.21.15.38:0/2245582515 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5a780a2b90 msgr2=0x7f5a780a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.296+0000 7f5a8ec73640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5a780a2b90 0x7f5a780a50a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c005d00 tx=0x7f5a7c005ef0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.296+0000 7f5a8ec73640 1 -- 172.21.15.38:0/2245582515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a881078f0 msgr2=0x7f5a881a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.296+0000 7f5a8ec73640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a881078f0 0x7f5a881a0720 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000eaa0 tx=0x7f5a8000ef60 comp rx=0 tx=0).stop 2024-05-21T15:34:29.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.296+0000 7f5a8ec73640 1 -- 172.21.15.38:0/2245582515 shutdown_connections 2024-05-21T15:34:29.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.296+0000 7f5a8ec73640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5a780a2b90 0x7f5a780a50a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.296+0000 7f5a8ec73640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a881078f0 0x7f5a881a0720 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.296+0000 7f5a8ec73640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a88102bf0 0x7f5a88199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.296+0000 7f5a8ec73640 1 --2- 172.21.15.38:0/2245582515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a881019b0 0x7f5a88199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.296+0000 7f5a8ec73640 1 -- 172.21.15.38:0/2245582515 >> 172.21.15.38:0/2245582515 conn(0x7f5a880fcbe0 msgr2=0x7f5a881038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:29.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.296+0000 7f5a8ec73640 1 -- 172.21.15.38:0/2245582515 shutdown_connections 2024-05-21T15:34:29.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.296+0000 7f5a8ec73640 1 -- 172.21.15.38:0/2245582515 wait complete. 2024-05-21T15:34:29.307 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:34:29.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.422+0000 7f317dfb5640 1 -- 172.21.15.38:0/3707021658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31781078f0 msgr2=0x7f3178109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.422+0000 7f317dfb5640 1 --2- 172.21.15.38:0/3707021658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31781078f0 0x7f3178109d10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3170009ca0 tx=0x7f317002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.422+0000 7f317dfb5640 1 -- 172.21.15.38:0/3707021658 shutdown_connections 2024-05-21T15:34:29.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.422+0000 7f317dfb5640 1 --2- 172.21.15.38:0/3707021658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31781078f0 0x7f3178109d10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.422+0000 7f317dfb5640 1 --2- 172.21.15.38:0/3707021658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3178102bf0 0x7f3178103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.422+0000 7f317dfb5640 1 --2- 172.21.15.38:0/3707021658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31781019b0 0x7f31781026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.422+0000 7f317dfb5640 1 -- 172.21.15.38:0/3707021658 >> 172.21.15.38:0/3707021658 conn(0x7f31780fcba0 msgr2=0x7f31780ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:29.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.422+0000 7f317dfb5640 1 -- 172.21.15.38:0/3707021658 shutdown_connections 2024-05-21T15:34:29.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.422+0000 7f317dfb5640 1 -- 172.21.15.38:0/3707021658 wait complete. 2024-05-21T15:34:29.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.423+0000 7f317dfb5640 1 Processor -- start 2024-05-21T15:34:29.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.423+0000 7f317dfb5640 1 -- start start 2024-05-21T15:34:29.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.423+0000 7f317dfb5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31781019b0 0x7f317810c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:29.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.423+0000 7f317dfb5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3178102bf0 0x7f317810a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:29.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.423+0000 7f317dfb5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31781078f0 0x7f317810ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:29.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.423+0000 7f317dfb5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f317810e730 con 0x7f3178102bf0 2024-05-21T15:34:29.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.423+0000 7f317dfb5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f317810e5b0 con 0x7f31781078f0 2024-05-21T15:34:29.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.423+0000 7f317dfb5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f317810e8b0 con 0x7f31781019b0 2024-05-21T15:34:29.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.423+0000 7f3176ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3178102bf0 0x7f317810a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:29.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.423+0000 7f3176ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3178102bf0 0x7f317810a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59898/0 (socket says 172.21.15.38:59898) 2024-05-21T15:34:29.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.423+0000 7f3176ffd640 1 -- 172.21.15.38:0/546824479 learned_addr learned my addr 172.21.15.38:0/546824479 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:29.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.424+0000 7f31777fe640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31781019b0 0x7f317810c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:29.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.424+0000 7f3177fff640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31781078f0 0x7f317810ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:29.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.424+0000 7f3176ffd640 1 -- 172.21.15.38:0/546824479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31781019b0 msgr2=0x7f317810c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.424+0000 7f3176ffd640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31781019b0 0x7f317810c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.424+0000 7f3176ffd640 1 -- 172.21.15.38:0/546824479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31781078f0 msgr2=0x7f317810ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.424+0000 7f3176ffd640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31781078f0 0x7f317810ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.424+0000 7f3176ffd640 1 -- 172.21.15.38:0/546824479 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317810b590 con 0x7f3178102bf0 2024-05-21T15:34:29.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.424+0000 7f3177fff640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31781078f0 0x7f317810ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:29.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.424+0000 7f3176ffd640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3178102bf0 0x7f317810a920 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f316c00d950 tx=0x7f316c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:29.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.424+0000 7f3174ff9640 1 -- 172.21.15.38:0/546824479 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c017070 con 0x7f3178102bf0 2024-05-21T15:34:29.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.424+0000 7f317dfb5640 1 -- 172.21.15.38:0/546824479 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31781a31f0 con 0x7f3178102bf0 2024-05-21T15:34:29.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.424+0000 7f3174ff9640 1 -- 172.21.15.38:0/546824479 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f316c009c30 con 0x7f3178102bf0 2024-05-21T15:34:29.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.424+0000 7f3174ff9640 1 -- 172.21.15.38:0/546824479 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c00f6b0 con 0x7f3178102bf0 2024-05-21T15:34:29.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.424+0000 7f317dfb5640 1 -- 172.21.15.38:0/546824479 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31781a36b0 con 0x7f3178102bf0 2024-05-21T15:34:29.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.425+0000 7f317dfb5640 1 -- 172.21.15.38:0/546824479 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3178106700 con 0x7f3178102bf0 2024-05-21T15:34:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.426+0000 7f3174ff9640 1 -- 172.21.15.38:0/546824479 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f316c0109d0 con 0x7f3178102bf0 2024-05-21T15:34:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.427+0000 7f3174ff9640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f31600a2c30 0x7f31600a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.427+0000 7f31777fe640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f31600a2c30 0x7f31600a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.427+0000 7f3174ff9640 1 -- 172.21.15.38:0/546824479 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f316c0cee10 con 0x7f3178102bf0 2024-05-21T15:34:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.428+0000 7f31777fe640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f31600a2c30 0x7f31600a5140 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3168005fd0 tx=0x7f3168005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:29.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.432+0000 7f3174ff9640 1 -- 172.21.15.38:0/546824479 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f316c0d2030 con 0x7f3178102bf0 2024-05-21T15:34:29.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:29 smithi053 ceph-mon[36496]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 11 B/s, 0 objects/s recovering 2024-05-21T15:34:29.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:29 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2886157079' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:29.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:29 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2993036033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:29.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:29 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/824454627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:29.510 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:29 smithi045 ceph-mon[36409]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 11 B/s, 0 objects/s recovering 2024-05-21T15:34:29.510 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:29 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2886157079' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:29.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:29 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2993036033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:29.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:29 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/824454627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:29.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.658+0000 7f317dfb5640 1 -- 172.21.15.38:0/546824479 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f317802d2a0 con 0x7f3178102bf0 2024-05-21T15:34:29.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.659+0000 7f3174ff9640 1 -- 172.21.15.38:0/546824479 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11914 (secure 0 0 0) 0x7f316c009de0 con 0x7f3178102bf0 2024-05-21T15:34:29.659 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:29.659 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:32:40.979368+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24550,24515,24602,24523,24551]},"id":1}]} 2024-05-21T15:34:29.659 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-05-21T15:34:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.661+0000 7f317dfb5640 1 -- 172.21.15.38:0/546824479 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f31600a2c30 msgr2=0x7f31600a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.661+0000 7f317dfb5640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f31600a2c30 0x7f31600a5140 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3168005fd0 tx=0x7f3168005850 comp rx=0 tx=0).stop 2024-05-21T15:34:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.661+0000 7f317dfb5640 1 -- 172.21.15.38:0/546824479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3178102bf0 msgr2=0x7f317810a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.661+0000 7f317dfb5640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3178102bf0 0x7f317810a920 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f316c00d950 tx=0x7f316c00de10 comp rx=0 tx=0).stop 2024-05-21T15:34:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.661+0000 7f317dfb5640 1 -- 172.21.15.38:0/546824479 shutdown_connections 2024-05-21T15:34:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.661+0000 7f317dfb5640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31781078f0 0x7f317810ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.661+0000 7f317dfb5640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f31600a2c30 0x7f31600a5140 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.661+0000 7f317dfb5640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3178102bf0 0x7f317810a920 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.661+0000 7f317dfb5640 1 --2- 172.21.15.38:0/546824479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31781019b0 0x7f317810c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.661+0000 7f317dfb5640 1 -- 172.21.15.38:0/546824479 >> 172.21.15.38:0/546824479 conn(0x7f31780fcba0 msgr2=0x7f31781038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:29.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.661+0000 7f317dfb5640 1 -- 172.21.15.38:0/546824479 shutdown_connections 2024-05-21T15:34:29.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.661+0000 7f317dfb5640 1 -- 172.21.15.38:0/546824479 wait complete. 2024-05-21T15:34:29.673 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:34:29.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.788+0000 7f27e539d640 1 -- 172.21.15.38:0/2239998125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27e0102490 msgr2=0x7f27e01028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.788+0000 7f27e539d640 1 --2- 172.21.15.38:0/2239998125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27e0102490 0x7f27e01028a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009cd0 tx=0x7f27d802a8e0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.788+0000 7f27e539d640 1 -- 172.21.15.38:0/2239998125 shutdown_connections 2024-05-21T15:34:29.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.788+0000 7f27e539d640 1 --2- 172.21.15.38:0/2239998125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27e0102490 0x7f27e01028a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.788+0000 7f27e539d640 1 --2- 172.21.15.38:0/2239998125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e0101b40 0x7f27e0101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.788+0000 7f27e539d640 1 --2- 172.21.15.38:0/2239998125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01011f0 0x7f27e0101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.788+0000 7f27e539d640 1 -- 172.21.15.38:0/2239998125 >> 172.21.15.38:0/2239998125 conn(0x7f27e00fca00 msgr2=0x7f27e00feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:29.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.788+0000 7f27e539d640 1 -- 172.21.15.38:0/2239998125 shutdown_connections 2024-05-21T15:34:29.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.788+0000 7f27e539d640 1 -- 172.21.15.38:0/2239998125 wait complete. 2024-05-21T15:34:29.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27e539d640 1 Processor -- start 2024-05-21T15:34:29.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27e539d640 1 -- start start 2024-05-21T15:34:29.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27e539d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e01011f0 0x7f27e019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:29.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27e539d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27e0101b40 0x7f27e019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:29.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27e539d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e0102490 0x7f27e01a2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:29.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27e539d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01107a0 con 0x7f27e0101b40 2024-05-21T15:34:29.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27e539d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0110620 con 0x7f27e01011f0 2024-05-21T15:34:29.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27e539d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0110920 con 0x7f27e0102490 2024-05-21T15:34:29.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27e0101b40 0x7f27e019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27e0101b40 0x7f27e019b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59914/0 (socket says 172.21.15.38:59914) 2024-05-21T15:34:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27d7fff640 1 -- 172.21.15.38:0/908951201 learned_addr learned my addr 172.21.15.38:0/908951201 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27dffff640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e01011f0 0x7f27e019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27e4b9c640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e0102490 0x7f27e01a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27d7fff640 1 -- 172.21.15.38:0/908951201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e0102490 msgr2=0x7f27e01a2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27d7fff640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e0102490 0x7f27e01a2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27d7fff640 1 -- 172.21.15.38:0/908951201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e01011f0 msgr2=0x7f27e019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27d7fff640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e01011f0 0x7f27e019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27d7fff640 1 -- 172.21.15.38:0/908951201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e01a2df0 con 0x7f27e0101b40 2024-05-21T15:34:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.789+0000 7f27e4b9c640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e0102490 0x7f27e01a2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.790+0000 7f27d7fff640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27e0101b40 0x7f27e019b7f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f27c800d950 tx=0x7f27c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.790+0000 7f27ddffb640 1 -- 172.21.15.38:0/908951201 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8017070 con 0x7f27e0101b40 2024-05-21T15:34:29.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.790+0000 7f27ddffb640 1 -- 172.21.15.38:0/908951201 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c8009c30 con 0x7f27e0101b40 2024-05-21T15:34:29.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.790+0000 7f27e539d640 1 -- 172.21.15.38:0/908951201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e01a30e0 con 0x7f27e0101b40 2024-05-21T15:34:29.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.790+0000 7f27ddffb640 1 -- 172.21.15.38:0/908951201 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c800f6b0 con 0x7f27e0101b40 2024-05-21T15:34:29.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.790+0000 7f27e539d640 1 -- 172.21.15.38:0/908951201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e01a3630 con 0x7f27e0101b40 2024-05-21T15:34:29.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.791+0000 7f27e539d640 1 -- 172.21.15.38:0/908951201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e010bb20 con 0x7f27e0101b40 2024-05-21T15:34:29.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.792+0000 7f27ddffb640 1 -- 172.21.15.38:0/908951201 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f27c80109d0 con 0x7f27e0101b40 2024-05-21T15:34:29.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.793+0000 7f27ddffb640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27cc0a2c30 0x7f27cc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:29.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.793+0000 7f27dffff640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27cc0a2c30 0x7f27cc0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:29.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.793+0000 7f27ddffb640 1 -- 172.21.15.38:0/908951201 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f27c80cecd0 con 0x7f27e0101b40 2024-05-21T15:34:29.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.793+0000 7f27dffff640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27cc0a2c30 0x7f27cc0a5140 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f27e006a980 tx=0x7f27d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:29.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.796+0000 7f27ddffb640 1 -- 172.21.15.38:0/908951201 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f27c8009de0 con 0x7f27e0101b40 2024-05-21T15:34:29.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.981+0000 7f27e539d640 1 -- 172.21.15.38:0/908951201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27e002d470 con 0x7f27e0101b40 2024-05-21T15:34:29.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.983+0000 7f27ddffb640 1 -- 172.21.15.38:0/908951201 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22704 (secure 0 0 0) 0x7f27c809bbe0 con 0x7f27e0101b40 2024-05-21T15:34:29.983 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:29.983 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":90,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:34:23.996759+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:34:29.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.985+0000 7f27e539d640 1 -- 172.21.15.38:0/908951201 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27cc0a2c30 msgr2=0x7f27cc0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.985+0000 7f27e539d640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27cc0a2c30 0x7f27cc0a5140 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f27e006a980 tx=0x7f27d0005f90 comp rx=0 tx=0).stop 2024-05-21T15:34:29.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.985+0000 7f27e539d640 1 -- 172.21.15.38:0/908951201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27e0101b40 msgr2=0x7f27e019b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:29.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.985+0000 7f27e539d640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27e0101b40 0x7f27e019b7f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f27c800d950 tx=0x7f27c800de10 comp rx=0 tx=0).stop 2024-05-21T15:34:29.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.985+0000 7f27e539d640 1 -- 172.21.15.38:0/908951201 shutdown_connections 2024-05-21T15:34:29.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.985+0000 7f27e539d640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e0102490 0x7f27e01a2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.985+0000 7f27e539d640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27cc0a2c30 0x7f27cc0a5140 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.985+0000 7f27e539d640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27e0101b40 0x7f27e019b7f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.985+0000 7f27e539d640 1 --2- 172.21.15.38:0/908951201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27e01011f0 0x7f27e019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:29.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.985+0000 7f27e539d640 1 -- 172.21.15.38:0/908951201 >> 172.21.15.38:0/908951201 conn(0x7f27e00fca00 msgr2=0x7f27e00fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:29.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.985+0000 7f27e539d640 1 -- 172.21.15.38:0/908951201 shutdown_connections 2024-05-21T15:34:29.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:29.985+0000 7f27e539d640 1 -- 172.21.15.38:0/908951201 wait complete. 2024-05-21T15:34:29.997 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-21T15:32:33.564616+0000', 'modified': '2024-05-21T15:32:40.979368+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24602, 'mds_1': 24551, 'mds_2': 24550, 'mds_3': 24523, 'mds_4': 24515}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14655': {'gid': 14655, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6839/2313367718', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6838', 'nonce': 2313367718}, {'type': 'v1', 'addr': '172.21.15.38:6839', 'nonce': 2313367718}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14667': {'gid': 14667, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6841/106491696', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6840', 'nonce': 106491696}, {'type': 'v1', 'addr': '172.21.15.38:6841', 'nonce': 106491696}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6833/1390207425', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 1390207425}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 1390207425}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6833/2551213098', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 2551213098}, {'type': 'v1', 'addr': '172.21.15.45:6833', 'nonce': 2551213098}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6835/1330413091', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 1330413091}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 1330413091}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'e', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.45:6835/3817948392', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 3817948392}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 3817948392}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.53:6837/2684842704', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 2684842704}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 2684842704}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.53:6839/3594548281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 3594548281}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 3594548281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24551': {'gid': 24551, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.45:6837/977772487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6836', 'nonce': 977772487}, {'type': 'v1', 'addr': '172.21.15.45:6837', 'nonce': 977772487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24602': {'gid': 24602, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6839/2790647513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6838', 'nonce': 2790647513}, {'type': 'v1', 'addr': '172.21.15.45:6839', 'nonce': 2790647513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24602, 'qdb_cluster': [24550, 24515, 24602, 24523, 24551]} 2024-05-21T15:34:29.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-21T15:34:29.998 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-05-21T15:34:30.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.115+0000 7f05cee5a640 1 -- 172.21.15.38:0/1139530304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05c8108a20 msgr2=0x7f05c810ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.115+0000 7f05cee5a640 1 --2- 172.21.15.38:0/1139530304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05c8108a20 0x7f05c810ae40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009900 tx=0x7f05c002a410 comp rx=0 tx=0).stop 2024-05-21T15:34:30.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.115+0000 7f05cee5a640 1 -- 172.21.15.38:0/1139530304 shutdown_connections 2024-05-21T15:34:30.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.115+0000 7f05cee5a640 1 --2- 172.21.15.38:0/1139530304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05c8108a20 0x7f05c810ae40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.115+0000 7f05cee5a640 1 --2- 172.21.15.38:0/1139530304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c8103d20 0x7f05c8104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.115+0000 7f05cee5a640 1 --2- 172.21.15.38:0/1139530304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c8102ae0 0x7f05c81037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.115+0000 7f05cee5a640 1 -- 172.21.15.38:0/1139530304 >> 172.21.15.38:0/1139530304 conn(0x7f05c80fdcf0 msgr2=0x7f05c8100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.115+0000 7f05cee5a640 1 -- 172.21.15.38:0/1139530304 shutdown_connections 2024-05-21T15:34:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.115+0000 7f05cee5a640 1 -- 172.21.15.38:0/1139530304 wait complete. 2024-05-21T15:34:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.116+0000 7f05cee5a640 1 Processor -- start 2024-05-21T15:34:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.116+0000 7f05cee5a640 1 -- start start 2024-05-21T15:34:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.116+0000 7f05cee5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05c8102ae0 0x7f05c8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.116+0000 7f05cee5a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c8103d20 0x7f05c8199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.116+0000 7f05cee5a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c8108a20 0x7f05c81a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.116+0000 7f05cee5a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05c810e760 con 0x7f05c8102ae0 2024-05-21T15:34:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.116+0000 7f05cee5a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f05c810e5e0 con 0x7f05c8103d20 2024-05-21T15:34:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.116+0000 7f05cee5a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f05c810e8e0 con 0x7f05c8108a20 2024-05-21T15:34:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.116+0000 7f05ccbf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05c8102ae0 0x7f05c8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05ccbf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05c8102ae0 0x7f05c8199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59930/0 (socket says 172.21.15.38:59930) 2024-05-21T15:34:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05ccbf6640 1 -- 172.21.15.38:0/4137019132 learned_addr learned my addr 172.21.15.38:0/4137019132 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05cd3f7640 1 --2- 172.21.15.38:0/4137019132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c8108a20 0x7f05c81a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05ccbf6640 1 -- 172.21.15.38:0/4137019132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c8108a20 msgr2=0x7f05c81a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05ccbf6640 1 --2- 172.21.15.38:0/4137019132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c8108a20 0x7f05c81a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05ccbf6640 1 -- 172.21.15.38:0/4137019132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c8103d20 msgr2=0x7f05c8199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05ccbf6640 1 --2- 172.21.15.38:0/4137019132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c8103d20 0x7f05c8199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05ccbf6640 1 -- 172.21.15.38:0/4137019132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c81a0d80 con 0x7f05c8102ae0 2024-05-21T15:34:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05cd3f7640 1 --2- 172.21.15.38:0/4137019132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c8108a20 0x7f05c81a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05ccbf6640 1 --2- 172.21.15.38:0/4137019132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05c8102ae0 0x7f05c8199240 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f05b0005d00 tx=0x7f05b000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05c5ffb640 1 -- 172.21.15.38:0/4137019132 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0017070 con 0x7f05c8102ae0 2024-05-21T15:34:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05c5ffb640 1 -- 172.21.15.38:0/4137019132 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b0009c30 con 0x7f05c8102ae0 2024-05-21T15:34:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05c5ffb640 1 -- 172.21.15.38:0/4137019132 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b000f620 con 0x7f05c8102ae0 2024-05-21T15:34:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05cee5a640 1 -- 172.21.15.38:0/4137019132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c81a1070 con 0x7f05c8102ae0 2024-05-21T15:34:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.117+0000 7f05cee5a640 1 -- 172.21.15.38:0/4137019132 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c81a1570 con 0x7f05c8102ae0 2024-05-21T15:34:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.120+0000 7f05c5ffb640 1 -- 172.21.15.38:0/4137019132 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f05b001a070 con 0x7f05c8102ae0 2024-05-21T15:34:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.120+0000 7f05d5ddb740 1 Processor -- start 2024-05-21T15:34:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.120+0000 7f05d5ddb740 1 -- start start 2024-05-21T15:34:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.120+0000 7f05d5ddb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79587b270 0x55a795c1b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.120+0000 7f05d5ddb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 0x55a795c1b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.120+0000 7f05d5ddb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c228f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.120+0000 7f05d5ddb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a795b7f4d0 con 0x55a79585a590 2024-05-21T15:34:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.120+0000 7f05d5ddb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a795b273e0 con 0x55a7958523b0 2024-05-21T15:34:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.120+0000 7f05d5ddb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a795b7b490 con 0x55a79587b270 2024-05-21T15:34:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05c7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c228f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05c7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c228f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59934/0 (socket says 172.21.15.38:59934) 2024-05-21T15:34:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05cd3f7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79587b270 0x55a795c1b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05c7fff640 1 -- 172.21.15.38:0/1725966774 learned_addr learned my addr 172.21.15.38:0/1725966774 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05ccbf6640 1 --2- 172.21.15.38:0/1725966774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 0x55a795c1b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05c7fff640 1 -- 172.21.15.38:0/1725966774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79587b270 msgr2=0x55a795c1b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05c7fff640 1 --2- 172.21.15.38:0/1725966774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79587b270 0x55a795c1b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05c5ffb640 1 --2- 172.21.15.38:0/4137019132 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05b40a2b90 0x7f05b40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05c7fff640 1 -- 172.21.15.38:0/1725966774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 msgr2=0x55a795c1b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05c7fff640 1 --2- 172.21.15.38:0/1725966774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 0x55a795c1b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05c7fff640 1 -- 172.21.15.38:0/1725966774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7958932d0 con 0x55a79585a590 2024-05-21T15:34:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05cd3f7640 1 --2- 172.21.15.38:0/4137019132 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05b40a2b90 0x7f05b40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05c5ffb640 1 -- 172.21.15.38:0/4137019132 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f05b00ceba0 con 0x7f05c8102ae0 2024-05-21T15:34:30.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05c7fff640 1 --2- 172.21.15.38:0/1725966774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c228f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009e70 tx=0x7f05b800d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20b02db39a83bd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05beffd640 1 -- 172.21.15.38:0/1725966774 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8016070 con 0x55a79585a590 2024-05-21T15:34:30.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05beffd640 1 -- 172.21.15.38:0/1725966774 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b8003b70 con 0x55a79585a590 2024-05-21T15:34:30.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.121+0000 7f05beffd640 1 -- 172.21.15.38:0/1725966774 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b80106c0 con 0x55a79585a590 2024-05-21T15:34:30.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.122+0000 7f05d5ddb740 1 -- 172.21.15.38:0/1725966774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 msgr2=0x55a795c228f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.122+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/1725966774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c228f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009e70 tx=0x7f05b800d6b0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.122+0000 7f05cd3f7640 1 --2- 172.21.15.38:0/4137019132 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05b40a2b90 0x7f05b40a50a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f05c0031610 tx=0x7f05c00314e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.122+0000 7f05d5ddb740 1 -- 172.21.15.38:0/1725966774 shutdown_connections 2024-05-21T15:34:30.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.122+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/1725966774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c228f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.122+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/1725966774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 0x55a795c1b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.122+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/1725966774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79587b270 0x55a795c1b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.122+0000 7f05d5ddb740 1 -- 172.21.15.38:0/1725966774 >> 172.21.15.38:0/1725966774 conn(0x55a79586f3c0 msgr2=0x55a795c16e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.122+0000 7f05d5ddb740 1 -- 172.21.15.38:0/1725966774 shutdown_connections 2024-05-21T15:34:30.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.122+0000 7f05d5ddb740 1 -- 172.21.15.38:0/1725966774 wait complete. 2024-05-21T15:34:30.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.123+0000 7f05d5ddb740 1 -- start start 2024-05-21T15:34:30.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.123+0000 7f05d5ddb740 1 Processor -- start 2024-05-21T15:34:30.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.123+0000 7f05d5ddb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 0x55a795c25230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.123+0000 7f05d5ddb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79585a590 0x55a795c25770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.123+0000 7f05d5ddb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79587b270 0x55a795c2c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.123+0000 7f05d5ddb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a795c23340 con 0x55a79587b270 2024-05-21T15:34:30.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.123+0000 7f05d5ddb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a795b3a8e0 con 0x55a7958523b0 2024-05-21T15:34:30.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.123+0000 7f05d5ddb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a795c234c0 con 0x55a79585a590 2024-05-21T15:34:30.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.123+0000 7f05d5ddb740 10 client.0 _ll_register_callbacks cb 0x55a795b079c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:30.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.123+0000 7f05cd3f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79587b270 0x55a795c2c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.123+0000 7f05cd3f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79587b270 0x55a795c2c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59950/0 (socket says 172.21.15.38:59950) 2024-05-21T15:34:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.123+0000 7f05cd3f7640 1 -- 172.21.15.38:0/213408622 learned_addr learned my addr 172.21.15.38:0/213408622 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05ccbf6640 1 --2- 172.21.15.38:0/213408622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 0x55a795c25230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05c7fff640 1 --2- 172.21.15.38:0/213408622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79585a590 0x55a795c25770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05cd3f7640 1 -- 172.21.15.38:0/213408622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79585a590 msgr2=0x55a795c25770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05cd3f7640 1 --2- 172.21.15.38:0/213408622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79585a590 0x55a795c25770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05cd3f7640 1 -- 172.21.15.38:0/213408622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 msgr2=0x55a795c25230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05cd3f7640 1 --2- 172.21.15.38:0/213408622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 0x55a795c25230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05cd3f7640 1 -- 172.21.15.38:0/213408622 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a79595d6b0 con 0x55a79587b270 2024-05-21T15:34:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05cd3f7640 1 --2- 172.21.15.38:0/213408622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79587b270 0x55a795c2c6d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009900 tx=0x7f05c0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7049d5d43df1e73 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05a7fff640 1 -- 172.21.15.38:0/213408622 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c004c070 con 0x55a79587b270 2024-05-21T15:34:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05a7fff640 1 -- 172.21.15.38:0/213408622 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f05c0005460 con 0x55a79587b270 2024-05-21T15:34:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05d5ddb740 1 -- 172.21.15.38:0/213408622 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b8003b70 con 0x55a79587b270 2024-05-21T15:34:30.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05a7fff640 1 -- 172.21.15.38:0/213408622 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f05c0058050 con 0x55a79587b270 2024-05-21T15:34:30.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05d5ddb740 10 client.15285 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:30.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.124+0000 7f05d5ddb740 1 -- 172.21.15.38:0/213408622 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a795c2f670 con 0x55a79587b270 2024-05-21T15:34:30.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.125+0000 7f05a7fff640 1 -- 172.21.15.38:0/213408622 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b8003b70 con 0x55a79587b270 2024-05-21T15:34:30.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.125+0000 7f05a7fff640 1 -- 172.21.15.38:0/213408622 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f05c0003a30 con 0x55a79587b270 2024-05-21T15:34:30.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.125+0000 7f05d5ddb740 10 client.15285 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:30.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.125+0000 7f05d5ddb740 10 client.15285 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-21T15:34:30.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.125+0000 7f05d5ddb740 20 client.15285 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:30.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.125+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/213408622 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55a795839e40 0x55a795c33150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.125+0000 7f05d5ddb740 4 client.15285 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:34:30.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.125+0000 7f05d5ddb740 1 -- 172.21.15.38:0/213408622 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a795b8cc00 con 0x55a795839e40 2024-05-21T15:34:30.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.125+0000 7f05ccbf6640 1 --2- 172.21.15.38:0/213408622 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55a795839e40 0x55a795c33150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.126+0000 7f05ccbf6640 1 --2- 172.21.15.38:0/213408622 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55a795839e40 0x55a795c33150 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=550d49b3e28fc7df server_cookie=e8015d2ec3177b0e in_seq=0 out_seq=0 2024-05-21T15:34:30.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.126+0000 7f05a7fff640 10 client.15285 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:34:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.126+0000 7f05a7fff640 1 -- 172.21.15.38:0/213408622 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a795b8cc00 con 0x55a795839e40 2024-05-21T15:34:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.126+0000 7f05a7fff640 10 client.15285 handle_command_reply: tid=0 2024-05-21T15:34:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.126+0000 7f05d5ddb740 1 client.15285 shutdown 2024-05-21T15:34:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.126+0000 7f05d5ddb740 1 -- 172.21.15.38:0/213408622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79587b270 msgr2=0x55a795c2c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.126+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/213408622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79587b270 0x55a795c2c6d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009900 tx=0x7f05c0009260 comp rx=0 tx=0).stop 2024-05-21T15:34:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.127+0000 7f05d5ddb740 1 -- 172.21.15.38:0/213408622 shutdown_connections 2024-05-21T15:34:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.127+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/213408622 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55a795839e40 0x55a795c33150 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.127+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/213408622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79587b270 0x55a795c2c6d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.127+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/213408622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79585a590 0x55a795c25770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.127+0000 7f05a7fff640 0 client.15285 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:34:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.127+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/213408622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 0x55a795c25230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.127+0000 7f05d5ddb740 1 -- 172.21.15.38:0/213408622 >> 172.21.15.38:0/213408622 conn(0x55a79586f3c0 msgr2=0x55a795c15ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.127+0000 7f05d5ddb740 1 -- 172.21.15.38:0/213408622 shutdown_connections 2024-05-21T15:34:30.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.127+0000 7f05d5ddb740 1 -- 172.21.15.38:0/213408622 wait complete. 2024-05-21T15:34:30.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.127+0000 7f05d5ddb740 20 client.15285 trim_cache size 0 max 16384 2024-05-21T15:34:30.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05d5ddb740 1 Processor -- start 2024-05-21T15:34:30.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05d5ddb740 1 -- start start 2024-05-21T15:34:30.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05d5ddb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7958523b0 0x55a795c1ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05d5ddb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c24e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05d5ddb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79587b270 0x55a795c25340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05d5ddb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a795c31490 con 0x55a79585a590 2024-05-21T15:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05d5ddb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a795c31b60 con 0x55a79587b270 2024-05-21T15:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05d5ddb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a795c32230 con 0x55a7958523b0 2024-05-21T15:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05c7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c24e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05c7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c24e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59962/0 (socket says 172.21.15.38:59962) 2024-05-21T15:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05c7fff640 1 -- 172.21.15.38:0/3881536850 learned_addr learned my addr 172.21.15.38:0/3881536850 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05c7fff640 1 -- 172.21.15.38:0/3881536850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7958523b0 msgr2=0x55a795c1ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05ccbf6640 1 --2- 172.21.15.38:0/3881536850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7958523b0 0x55a795c1ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05c7fff640 1 --2- 172.21.15.38:0/3881536850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7958523b0 0x55a795c1ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05c7fff640 1 -- 172.21.15.38:0/3881536850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79587b270 msgr2=0x55a795c25340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05c7fff640 1 --2- 172.21.15.38:0/3881536850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79587b270 0x55a795c25340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05c7fff640 1 -- 172.21.15.38:0/3881536850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a795c1c260 con 0x55a79585a590 2024-05-21T15:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.146+0000 7f05c7fff640 1 --2- 172.21.15.38:0/3881536850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c24e00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009e70 tx=0x7f05b8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8ef1c3910c0ec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.147+0000 7f05be7fc640 1 -- 172.21.15.38:0/3881536850 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8016050 con 0x55a79585a590 2024-05-21T15:34:30.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.147+0000 7f05be7fc640 1 -- 172.21.15.38:0/3881536850 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b8009920 con 0x55a79585a590 2024-05-21T15:34:30.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.147+0000 7f05be7fc640 1 -- 172.21.15.38:0/3881536850 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8005930 con 0x55a79585a590 2024-05-21T15:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.147+0000 7f05d5ddb740 1 -- 172.21.15.38:0/3881536850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 msgr2=0x55a795c24e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.147+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/3881536850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c24e00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009e70 tx=0x7f05b8000f40 comp rx=0 tx=0).stop 2024-05-21T15:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.147+0000 7f05d5ddb740 1 -- 172.21.15.38:0/3881536850 shutdown_connections 2024-05-21T15:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.147+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/3881536850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a79587b270 0x55a795c25340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.147+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/3881536850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c24e00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.147+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/3881536850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7958523b0 0x55a795c1ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.147+0000 7f05d5ddb740 1 -- 172.21.15.38:0/3881536850 >> 172.21.15.38:0/3881536850 conn(0x55a79586f3c0 msgr2=0x55a795c1e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.147+0000 7f05d5ddb740 1 -- 172.21.15.38:0/3881536850 shutdown_connections 2024-05-21T15:34:30.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.147+0000 7f05d5ddb740 1 -- 172.21.15.38:0/3881536850 wait complete. 2024-05-21T15:34:30.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.148+0000 7f05d5ddb740 1 -- start start 2024-05-21T15:34:30.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.148+0000 7f05d5ddb740 1 Processor -- start 2024-05-21T15:34:30.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.148+0000 7f05d5ddb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 0x55a795c270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.148+0000 7f05d5ddb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c27600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.148+0000 7f05d5ddb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79587b270 0x55a795c2a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.148+0000 7f05d5ddb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a795c32fd0 con 0x55a79585a590 2024-05-21T15:34:30.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.148+0000 7f05d5ddb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a795c32900 con 0x55a7958523b0 2024-05-21T15:34:30.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.148+0000 7f05d5ddb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a795c336a0 con 0x55a79587b270 2024-05-21T15:34:30.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.148+0000 7f05d5ddb740 10 client.0 _ll_register_callbacks cb 0x7f05c0032c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:30.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.148+0000 7f05c7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c27600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.148+0000 7f05c7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c27600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59974/0 (socket says 172.21.15.38:59974) 2024-05-21T15:34:30.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.148+0000 7f05c7fff640 1 -- 172.21.15.38:0/287168115 learned_addr learned my addr 172.21.15.38:0/287168115 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.148+0000 7f05cd3f7640 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79587b270 0x55a795c2a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.148+0000 7f05ccbf6640 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 0x55a795c270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05c7fff640 1 -- 172.21.15.38:0/287168115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79587b270 msgr2=0x55a795c2a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05c7fff640 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79587b270 0x55a795c2a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05c7fff640 1 -- 172.21.15.38:0/287168115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 msgr2=0x55a795c270c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05c7fff640 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 0x55a795c270c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05c7fff640 1 -- 172.21.15.38:0/287168115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a795c2aa90 con 0x55a79585a590 2024-05-21T15:34:30.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05ccbf6640 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 0x55a795c270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:30.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05cd3f7640 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79587b270 0x55a795c2a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:30.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05c7fff640 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c27600 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f05b80051f0 tx=0x7f05b8004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdc7c898977003a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05a7fff640 1 -- 172.21.15.38:0/287168115 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8005840 con 0x55a79585a590 2024-05-21T15:34:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05a7fff640 1 -- 172.21.15.38:0/287168115 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f05b8021050 con 0x55a79585a590 2024-05-21T15:34:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05d5ddb740 1 -- 172.21.15.38:0/287168115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b8009920 con 0x55a79585a590 2024-05-21T15:34:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05a7fff640 1 -- 172.21.15.38:0/287168115 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f05b802d470 con 0x55a79585a590 2024-05-21T15:34:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05d5ddb740 10 client.15291 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05d5ddb740 1 -- 172.21.15.38:0/287168115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a795c2bd10 con 0x55a79585a590 2024-05-21T15:34:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.149+0000 7f05a7fff640 1 -- 172.21.15.38:0/287168115 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b8009920 con 0x55a79585a590 2024-05-21T15:34:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.150+0000 7f05a7fff640 1 -- 172.21.15.38:0/287168115 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f05b8035030 con 0x55a79585a590 2024-05-21T15:34:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.150+0000 7f05d5ddb740 10 client.15291 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.150+0000 7f05d5ddb740 10 client.15291 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-21T15:34:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.150+0000 7f05d5ddb740 20 client.15291 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.150+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55a79580f280 0x55a795c9ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.150+0000 7f05d5ddb740 4 client.15291 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:34:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.150+0000 7f05d5ddb740 1 -- 172.21.15.38:0/287168115 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "status"}) -- 0x55a795b86f50 con 0x55a79580f280 2024-05-21T15:34:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.150+0000 7f05ccbf6640 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55a79580f280 0x55a795c9ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.150+0000 7f05a7fff640 10 client.15291 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:34:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.150+0000 7f05ccbf6640 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55a79580f280 0x55a795c9ec20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b727c662cc06caae server_cookie=2340e239e4857894 in_seq=0 out_seq=0 2024-05-21T15:34:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.150+0000 7f05a7fff640 1 -- 172.21.15.38:0/287168115 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a795b86f50 con 0x55a79580f280 2024-05-21T15:34:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.150+0000 7f05a7fff640 10 client.15291 handle_command_reply: tid=0 2024-05-21T15:34:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05d5ddb740 1 client.15291 shutdown 2024-05-21T15:34:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05d5ddb740 1 -- 172.21.15.38:0/287168115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 msgr2=0x55a795c27600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c27600 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f05b80051f0 tx=0x7f05b8004d50 comp rx=0 tx=0).stop 2024-05-21T15:34:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05d5ddb740 1 -- 172.21.15.38:0/287168115 shutdown_connections 2024-05-21T15:34:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55a79580f280 0x55a795c9ec20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a79587b270 0x55a795c2a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a79585a590 0x55a795c27600 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05a7fff640 0 client.15291 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:34:30.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05d5ddb740 1 --2- 172.21.15.38:0/287168115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7958523b0 0x55a795c270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05d5ddb740 1 -- 172.21.15.38:0/287168115 >> 172.21.15.38:0/287168115 conn(0x55a79586f3c0 msgr2=0x55a795c24e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05d5ddb740 1 -- 172.21.15.38:0/287168115 shutdown_connections 2024-05-21T15:34:30.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05d5ddb740 1 -- 172.21.15.38:0/287168115 wait complete. 2024-05-21T15:34:30.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05d5ddb740 20 client.15291 trim_cache size 0 max 16384 2024-05-21T15:34:30.157 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:34:30.157 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "102ec4fa-1786-11ef-bc9a-c7b262605968", 2024-05-21T15:34:30.157 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 4, 2024-05-21T15:34:30.158 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24515, 2024-05-21T15:34:30.158 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-05-21T15:34:30.158 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-05-21T15:34:30.158 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-05-21T15:34:30.158 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 110.177776765, 2024-05-21T15:34:30.158 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-05-21T15:34:30.158 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 90, 2024-05-21T15:34:30.158 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-05-21T15:34:30.158 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 141.67709061900001 2024-05-21T15:34:30.158 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:34:30.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05cee5a640 1 -- 172.21.15.38:0/4137019132 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05b40a2b90 msgr2=0x7f05b40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05cee5a640 1 --2- 172.21.15.38:0/4137019132 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05b40a2b90 0x7f05b40a50a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f05c0031610 tx=0x7f05c00314e0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05cee5a640 1 -- 172.21.15.38:0/4137019132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05c8102ae0 msgr2=0x7f05c8199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.151+0000 7f05cee5a640 1 --2- 172.21.15.38:0/4137019132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05c8102ae0 0x7f05c8199240 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f05b0005d00 tx=0x7f05b000dde0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.152+0000 7f05cee5a640 1 -- 172.21.15.38:0/4137019132 shutdown_connections 2024-05-21T15:34:30.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.152+0000 7f05cee5a640 1 --2- 172.21.15.38:0/4137019132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05c8108a20 0x7f05c81a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.152+0000 7f05cee5a640 1 --2- 172.21.15.38:0/4137019132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c8103d20 0x7f05c8199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.152+0000 7f05cee5a640 1 --2- 172.21.15.38:0/4137019132 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05b40a2b90 0x7f05b40a50a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.152+0000 7f05cee5a640 1 --2- 172.21.15.38:0/4137019132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05c8102ae0 0x7f05c8199240 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.152+0000 7f05cee5a640 1 -- 172.21.15.38:0/4137019132 >> 172.21.15.38:0/4137019132 conn(0x7f05c80fdcf0 msgr2=0x7f05c8104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.152+0000 7f05cee5a640 1 -- 172.21.15.38:0/4137019132 shutdown_connections 2024-05-21T15:34:30.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.152+0000 7f05cee5a640 1 -- 172.21.15.38:0/4137019132 wait complete. 2024-05-21T15:34:30.163 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-21T15:34:30.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:30 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2245582515' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:30.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:30 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/546824479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:30.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:30 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/908951201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.279+0000 7f73eed8f640 1 -- 172.21.15.38:0/667824026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e8103d00 msgr2=0x7f73e8104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.279+0000 7f73eed8f640 1 --2- 172.21.15.38:0/667824026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e8103d00 0x7f73e8104110 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009900 tx=0x7f73d802a410 comp rx=0 tx=0).stop 2024-05-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.279+0000 7f73eed8f640 1 -- 172.21.15.38:0/667824026 shutdown_connections 2024-05-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.279+0000 7f73eed8f640 1 --2- 172.21.15.38:0/667824026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73e8108a00 0x7f73e810ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.279+0000 7f73eed8f640 1 --2- 172.21.15.38:0/667824026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e8103d00 0x7f73e8104110 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.279+0000 7f73eed8f640 1 --2- 172.21.15.38:0/667824026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e8102ac0 0x7f73e81037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.279+0000 7f73eed8f640 1 -- 172.21.15.38:0/667824026 >> 172.21.15.38:0/667824026 conn(0x7f73e80fdcf0 msgr2=0x7f73e8100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.279+0000 7f73eed8f640 1 -- 172.21.15.38:0/667824026 shutdown_connections 2024-05-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.279+0000 7f73eed8f640 1 -- 172.21.15.38:0/667824026 wait complete. 2024-05-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.280+0000 7f73eed8f640 1 Processor -- start 2024-05-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.280+0000 7f73eed8f640 1 -- start start 2024-05-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.280+0000 7f73eed8f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e8102ac0 0x7f73e81992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.280+0000 7f73eed8f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e8103d00 0x7f73e81997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.280+0000 7f73eed8f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73e8108a00 0x7f73e81a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.280+0000 7f73eed8f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f73e810e710 con 0x7f73e8103d00 2024-05-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.280+0000 7f73eed8f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73e810e590 con 0x7f73e8102ac0 2024-05-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.280+0000 7f73eed8f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f73e810e890 con 0x7f73e8108a00 2024-05-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.280+0000 7f73dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e8103d00 0x7f73e81997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.280+0000 7f73dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e8103d00 0x7f73e81997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60004/0 (socket says 172.21.15.38:60004) 2024-05-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.280+0000 7f73dffff640 1 -- 172.21.15.38:0/1673450314 learned_addr learned my addr 172.21.15.38:0/1673450314 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.280+0000 7f73ed32c640 1 --2- 172.21.15.38:0/1673450314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73e8108a00 0x7f73e81a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.281+0000 7f73ecb2b640 1 --2- 172.21.15.38:0/1673450314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e8102ac0 0x7f73e81992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.281+0000 7f73dffff640 1 -- 172.21.15.38:0/1673450314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73e8108a00 msgr2=0x7f73e81a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.281+0000 7f73dffff640 1 --2- 172.21.15.38:0/1673450314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73e8108a00 0x7f73e81a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.281+0000 7f73dffff640 1 -- 172.21.15.38:0/1673450314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e8102ac0 msgr2=0x7f73e81992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.281+0000 7f73dffff640 1 --2- 172.21.15.38:0/1673450314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e8102ac0 0x7f73e81992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.281+0000 7f73dffff640 1 -- 172.21.15.38:0/1673450314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e81a0df0 con 0x7f73e8103d00 2024-05-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.281+0000 7f73dffff640 1 --2- 172.21.15.38:0/1673450314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e8103d00 0x7f73e81997f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f73d8006010 tx=0x7f73d8031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.281+0000 7f73ddffb640 1 -- 172.21.15.38:0/1673450314 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d803f030 con 0x7f73e8103d00 2024-05-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.281+0000 7f73ddffb640 1 -- 172.21.15.38:0/1673450314 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d8003ad0 con 0x7f73e8103d00 2024-05-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.281+0000 7f73eed8f640 1 -- 172.21.15.38:0/1673450314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e81a1080 con 0x7f73e8103d00 2024-05-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.281+0000 7f73ddffb640 1 -- 172.21.15.38:0/1673450314 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8035070 con 0x7f73e8103d00 2024-05-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.281+0000 7f73eed8f640 1 -- 172.21.15.38:0/1673450314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e81a1570 con 0x7f73e8103d00 2024-05-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73f5c3b740 1 Processor -- start 2024-05-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73f5c3b740 1 -- start start 2024-05-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73ddffb640 1 -- 172.21.15.38:0/1673450314 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f73d8042030 con 0x7f73e8103d00 2024-05-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73f5c3b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6479270 0x55a6b6819470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73f5c3b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b68199f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73f5c3b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 0x55a6b68208f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73f5c3b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a6b677d4d0 con 0x55a6b64503b0 2024-05-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73f5c3b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a6b67253e0 con 0x55a6b6458590 2024-05-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73f5c3b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a6b6779490 con 0x55a6b6479270 2024-05-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73ecb2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b68199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73ecb2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b68199f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60008/0 (socket says 172.21.15.38:60008) 2024-05-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73ecb2b640 1 -- 172.21.15.38:0/2174297090 learned_addr learned my addr 172.21.15.38:0/2174297090 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73dffff640 1 --2- 172.21.15.38:0/2174297090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 0x55a6b68208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73ed32c640 1 --2- 172.21.15.38:0/2174297090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6479270 0x55a6b6819470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73ecb2b640 1 -- 172.21.15.38:0/2174297090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6479270 msgr2=0x55a6b6819470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73ecb2b640 1 --2- 172.21.15.38:0/2174297090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6479270 0x55a6b6819470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73ecb2b640 1 -- 172.21.15.38:0/2174297090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 msgr2=0x55a6b68208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73ecb2b640 1 --2- 172.21.15.38:0/2174297090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 0x55a6b68208f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73ecb2b640 1 -- 172.21.15.38:0/2174297090 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6b64912d0 con 0x55a6b64503b0 2024-05-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73ed32c640 1 --2- 172.21.15.38:0/2174297090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6479270 0x55a6b6819470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.285+0000 7f73ddffb640 1 --2- 172.21.15.38:0/1673450314 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73d00a2b90 0x7f73d00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.286+0000 7f73ed32c640 1 --2- 172.21.15.38:0/1673450314 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73d00a2b90 0x7f73d00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.286+0000 7f73ddffb640 1 -- 172.21.15.38:0/1673450314 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f73d80efa10 con 0x7f73e8103d00 2024-05-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.286+0000 7f73ecb2b640 1 --2- 172.21.15.38:0/2174297090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b68199f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009bf0 tx=0x7f73e40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698c2474b92bdbaf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.286+0000 7f73ceffd640 1 -- 172.21.15.38:0/2174297090 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4004030 con 0x55a6b64503b0 2024-05-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.286+0000 7f73ceffd640 1 -- 172.21.15.38:0/2174297090 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e40041d0 con 0x55a6b64503b0 2024-05-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.286+0000 7f73ceffd640 1 -- 172.21.15.38:0/2174297090 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4012690 con 0x55a6b64503b0 2024-05-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.287+0000 7f73f5c3b740 1 -- 172.21.15.38:0/2174297090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 msgr2=0x55a6b68199f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.287+0000 7f73ed32c640 1 --2- 172.21.15.38:0/1673450314 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73d00a2b90 0x7f73d00a50a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55a6b681a640 tx=0x7f73e00096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.287+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/2174297090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b68199f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009bf0 tx=0x7f73e40117b0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.287+0000 7f73f5c3b740 1 -- 172.21.15.38:0/2174297090 shutdown_connections 2024-05-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.287+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/2174297090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 0x55a6b68208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.287+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/2174297090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b68199f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.287+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/2174297090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6479270 0x55a6b6819470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.287+0000 7f73f5c3b740 1 -- 172.21.15.38:0/2174297090 >> 172.21.15.38:0/2174297090 conn(0x55a6b646d3c0 msgr2=0x55a6b6814e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.287+0000 7f73f5c3b740 1 -- 172.21.15.38:0/2174297090 shutdown_connections 2024-05-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.287+0000 7f73f5c3b740 1 -- 172.21.15.38:0/2174297090 wait complete. 2024-05-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.287+0000 7f73f5c3b740 1 -- start start 2024-05-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.287+0000 7f73f5c3b740 1 Processor -- start 2024-05-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73f5c3b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b6823290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73f5c3b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 0x55a6b68237d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73f5c3b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6479270 0x55a6b682a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73f5c3b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a6b6821340 con 0x55a6b64503b0 2024-05-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73f5c3b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a6b67388e0 con 0x55a6b6458590 2024-05-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73f5c3b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a6b68214c0 con 0x55a6b6479270 2024-05-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73f5c3b740 10 client.0 _ll_register_callbacks cb 0x55a6b67059c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73ecb2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b6823290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73ecb2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b6823290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60010/0 (socket says 172.21.15.38:60010) 2024-05-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73ecb2b640 1 -- 172.21.15.38:0/4086861798 learned_addr learned my addr 172.21.15.38:0/4086861798 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73ed32c640 1 --2- 172.21.15.38:0/4086861798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6479270 0x55a6b682a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73dffff640 1 --2- 172.21.15.38:0/4086861798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 0x55a6b68237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73ecb2b640 1 -- 172.21.15.38:0/4086861798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6479270 msgr2=0x55a6b682a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73ecb2b640 1 --2- 172.21.15.38:0/4086861798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6479270 0x55a6b682a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73ecb2b640 1 -- 172.21.15.38:0/4086861798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 msgr2=0x55a6b68237d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73ecb2b640 1 --2- 172.21.15.38:0/4086861798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 0x55a6b68237d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73ecb2b640 1 -- 172.21.15.38:0/4086861798 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6b655b6b0 con 0x55a6b64503b0 2024-05-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73ecb2b640 1 --2- 172.21.15.38:0/4086861798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b6823290 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009770 tx=0x7f73e4009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427021e915c55319 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.288+0000 7f73c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.289+0000 7f73c7fff640 1 -- 172.21.15.38:0/4086861798 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4028020 con 0x55a6b64503b0 2024-05-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.289+0000 7f73c7fff640 1 -- 172.21.15.38:0/4086861798 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f73e40037f0 con 0x55a6b64503b0 2024-05-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.289+0000 7f73f5c3b740 1 -- 172.21.15.38:0/4086861798 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e40041d0 con 0x55a6b64503b0 2024-05-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.289+0000 7f73c7fff640 1 -- 172.21.15.38:0/4086861798 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73e4033050 con 0x55a6b64503b0 2024-05-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.289+0000 7f73f5c3b740 10 client.15303 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.289+0000 7f73f5c3b740 1 -- 172.21.15.38:0/4086861798 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a6b682d660 con 0x55a6b64503b0 2024-05-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.289+0000 7f73c7fff640 1 -- 172.21.15.38:0/4086861798 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e40041d0 con 0x55a6b64503b0 2024-05-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.289+0000 7f73c7fff640 1 -- 172.21.15.38:0/4086861798 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f73e4024530 con 0x55a6b64503b0 2024-05-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.289+0000 7f73f5c3b740 10 client.15303 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.289+0000 7f73f5c3b740 10 client.15303 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.289+0000 7f73f5c3b740 20 client.15303 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.289+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/4086861798 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55a6b6437e40 0x55a6b6830db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.289+0000 7f73f5c3b740 4 client.15303 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.289+0000 7f73f5c3b740 1 -- 172.21.15.38:0/4086861798 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6b678ac00 con 0x55a6b6437e40 2024-05-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.290+0000 7f73dffff640 1 --2- 172.21.15.38:0/4086861798 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55a6b6437e40 0x55a6b6830db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.290+0000 7f73dffff640 1 --2- 172.21.15.38:0/4086861798 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55a6b6437e40 0x55a6b6830db0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fbac3285dda50106 server_cookie=fb99e65a7b60b9e0 in_seq=0 out_seq=0 2024-05-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.290+0000 7f73c7fff640 10 client.15303 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73c7fff640 1 -- 172.21.15.38:0/4086861798 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a6b678ac00 con 0x55a6b6437e40 2024-05-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73c7fff640 10 client.15303 handle_command_reply: tid=0 2024-05-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73f5c3b740 1 client.15303 shutdown 2024-05-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73f5c3b740 1 -- 172.21.15.38:0/4086861798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 msgr2=0x55a6b6823290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/4086861798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b6823290 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009770 tx=0x7f73e4009930 comp rx=0 tx=0).stop 2024-05-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73f5c3b740 1 -- 172.21.15.38:0/4086861798 shutdown_connections 2024-05-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/4086861798 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55a6b6437e40 0x55a6b6830db0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/4086861798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6479270 0x55a6b682a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/4086861798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 0x55a6b68237d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/4086861798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b6823290 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73c7fff640 0 client.15303 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73f5c3b740 1 -- 172.21.15.38:0/4086861798 >> 172.21.15.38:0/4086861798 conn(0x55a6b646d3c0 msgr2=0x55a6b6813ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73f5c3b740 1 -- 172.21.15.38:0/4086861798 shutdown_connections 2024-05-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73f5c3b740 1 -- 172.21.15.38:0/4086861798 wait complete. 2024-05-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.291+0000 7f73f5c3b740 20 client.15303 trim_cache size 0 max 16384 2024-05-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73f5c3b740 1 Processor -- start 2024-05-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73f5c3b740 1 -- start start 2024-05-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73f5c3b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b682acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73f5c3b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6458590 0x55a6b681d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73f5c3b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6479270 0x55a6b681dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73f5c3b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a6b682f0f0 con 0x55a6b64503b0 2024-05-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73f5c3b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a6b682f7c0 con 0x55a6b6479270 2024-05-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73f5c3b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a6b682fe90 con 0x55a6b6458590 2024-05-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73ecb2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b682acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73ecb2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b682acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60018/0 (socket says 172.21.15.38:60018) 2024-05-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73ecb2b640 1 -- 172.21.15.38:0/810775663 learned_addr learned my addr 172.21.15.38:0/810775663 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73ed32c640 1 --2- 172.21.15.38:0/810775663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6479270 0x55a6b681dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73ecb2b640 1 -- 172.21.15.38:0/810775663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6458590 msgr2=0x55a6b681d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73ecb2b640 1 --2- 172.21.15.38:0/810775663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6458590 0x55a6b681d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73ecb2b640 1 -- 172.21.15.38:0/810775663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6479270 msgr2=0x55a6b681dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73ecb2b640 1 --2- 172.21.15.38:0/810775663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6479270 0x55a6b681dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73ecb2b640 1 -- 172.21.15.38:0/810775663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6b6830a10 con 0x55a6b64503b0 2024-05-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.310+0000 7f73ed32c640 1 --2- 172.21.15.38:0/810775663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6479270 0x55a6b681dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.311+0000 7f73ecb2b640 1 --2- 172.21.15.38:0/810775663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b682acc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f73e4000c00 tx=0x7f73e4009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28ab3cc961d6989 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.311+0000 7f73ce7fc640 1 -- 172.21.15.38:0/810775663 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4028040 con 0x55a6b64503b0 2024-05-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.311+0000 7f73ce7fc640 1 -- 172.21.15.38:0/810775663 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e4024b90 con 0x55a6b64503b0 2024-05-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.311+0000 7f73ce7fc640 1 -- 172.21.15.38:0/810775663 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e401f040 con 0x55a6b64503b0 2024-05-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.311+0000 7f73f5c3b740 1 -- 172.21.15.38:0/810775663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 msgr2=0x55a6b682acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.312+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/810775663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b682acc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f73e4000c00 tx=0x7f73e4009610 comp rx=0 tx=0).stop 2024-05-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.312+0000 7f73f5c3b740 1 -- 172.21.15.38:0/810775663 shutdown_connections 2024-05-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.312+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/810775663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6479270 0x55a6b681dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.312+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/810775663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b6458590 0x55a6b681d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.312+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/810775663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b64503b0 0x55a6b682acc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.312+0000 7f73f5c3b740 1 -- 172.21.15.38:0/810775663 >> 172.21.15.38:0/810775663 conn(0x55a6b646d3c0 msgr2=0x55a6b68218c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.312+0000 7f73f5c3b740 1 -- 172.21.15.38:0/810775663 shutdown_connections 2024-05-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.312+0000 7f73f5c3b740 1 -- 172.21.15.38:0/810775663 wait complete. 2024-05-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.312+0000 7f73f5c3b740 1 -- start start 2024-05-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.312+0000 7f73f5c3b740 1 Processor -- start 2024-05-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.312+0000 7f73f5c3b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b64503b0 0x55a6b6824d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.312+0000 7f73f5c3b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 0x55a6b6825270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73f5c3b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b6479270 0x55a6b6897c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73f5c3b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a6b6830c30 con 0x55a6b6479270 2024-05-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73f5c3b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a6b6830560 con 0x55a6b6458590 2024-05-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73f5c3b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a6b6831300 con 0x55a6b64503b0 2024-05-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73f5c3b740 10 client.0 _ll_register_callbacks cb 0x7f73d80f5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73ed32c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b6479270 0x55a6b6897c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73ed32c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b6479270 0x55a6b6897c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60022/0 (socket says 172.21.15.38:60022) 2024-05-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73ed32c640 1 -- 172.21.15.38:0/576725595 learned_addr learned my addr 172.21.15.38:0/576725595 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73ecb2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b64503b0 0x55a6b6824d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73ed32c640 1 -- 172.21.15.38:0/576725595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b64503b0 msgr2=0x55a6b6824d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73ed32c640 1 --2- 172.21.15.38:0/576725595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b64503b0 0x55a6b6824d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73dffff640 1 --2- 172.21.15.38:0/576725595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 0x55a6b6825270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73ed32c640 1 -- 172.21.15.38:0/576725595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 msgr2=0x55a6b6825270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73ed32c640 1 --2- 172.21.15.38:0/576725595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 0x55a6b6825270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73ed32c640 1 -- 172.21.15.38:0/576725595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6b6898160 con 0x55a6b6479270 2024-05-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73ed32c640 1 --2- 172.21.15.38:0/576725595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b6479270 0x55a6b6897c20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f73e0012750 tx=0x7f73e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67664036b66c258f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73c7fff640 1 -- 172.21.15.38:0/576725595 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e002b060 con 0x55a6b6479270 2024-05-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73c7fff640 1 -- 172.21.15.38:0/576725595 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f73e0010a60 con 0x55a6b6479270 2024-05-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73f5c3b740 1 -- 172.21.15.38:0/576725595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e4024b90 con 0x55a6b6479270 2024-05-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73c7fff640 1 -- 172.21.15.38:0/576725595 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73e0010c60 con 0x55a6b6479270 2024-05-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.313+0000 7f73f5c3b740 10 client.15309 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.314+0000 7f73f5c3b740 1 -- 172.21.15.38:0/576725595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a6b68993b0 con 0x55a6b6479270 2024-05-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.314+0000 7f73c7fff640 1 -- 172.21.15.38:0/576725595 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e4024b90 con 0x55a6b6479270 2024-05-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.314+0000 7f73c7fff640 1 -- 172.21.15.38:0/576725595 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f73e0025070 con 0x55a6b6479270 2024-05-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.314+0000 7f73f5c3b740 10 client.15309 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.314+0000 7f73f5c3b740 10 client.15309 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.314+0000 7f73f5c3b740 20 client.15309 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.314+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/576725595 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55a6b640d280 0x55a6b689cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.314+0000 7f73f5c3b740 4 client.15309 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.314+0000 7f73f5c3b740 1 -- 172.21.15.38:0/576725595 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "status"}) -- 0x55a6b6784f50 con 0x55a6b640d280 2024-05-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.314+0000 7f73ecb2b640 1 --2- 172.21.15.38:0/576725595 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55a6b640d280 0x55a6b689cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73ecb2b640 1 --2- 172.21.15.38:0/576725595 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55a6b640d280 0x55a6b689cd10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3d84a474287fe994 server_cookie=d8f93deb42bed7a0 in_seq=0 out_seq=0 2024-05-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73c7fff640 10 client.15309 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73c7fff640 1 -- 172.21.15.38:0/576725595 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a6b6784f50 con 0x55a6b640d280 2024-05-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73c7fff640 10 client.15309 handle_command_reply: tid=0 2024-05-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73f5c3b740 1 client.15309 shutdown 2024-05-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73f5c3b740 1 -- 172.21.15.38:0/576725595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b6479270 msgr2=0x55a6b6897c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/576725595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b6479270 0x55a6b6897c20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f73e0012750 tx=0x7f73e0004a40 comp rx=0 tx=0).stop 2024-05-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73f5c3b740 1 -- 172.21.15.38:0/576725595 shutdown_connections 2024-05-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/576725595 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55a6b640d280 0x55a6b689cd10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/576725595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6b6479270 0x55a6b6897c20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/576725595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6b6458590 0x55a6b6825270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73f5c3b740 1 --2- 172.21.15.38:0/576725595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6b64503b0 0x55a6b6824d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73c7fff640 0 client.15309 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:34:30.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73f5c3b740 1 -- 172.21.15.38:0/576725595 >> 172.21.15.38:0/576725595 conn(0x55a6b646d3c0 msgr2=0x55a6b681d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73f5c3b740 1 -- 172.21.15.38:0/576725595 shutdown_connections 2024-05-21T15:34:30.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73f5c3b740 1 -- 172.21.15.38:0/576725595 wait complete. 2024-05-21T15:34:30.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.315+0000 7f73f5c3b740 20 client.15309 trim_cache size 0 max 16384 2024-05-21T15:34:30.322 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:34:30.323 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "102ec4fa-1786-11ef-bc9a-c7b262605968", 2024-05-21T15:34:30.323 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 3, 2024-05-21T15:34:30.323 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24523, 2024-05-21T15:34:30.323 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-05-21T15:34:30.323 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-05-21T15:34:30.323 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-05-21T15:34:30.323 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 111.35188433899999, 2024-05-21T15:34:30.323 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-05-21T15:34:30.323 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 90, 2024-05-21T15:34:30.323 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-05-21T15:34:30.323 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 135.915883329 2024-05-21T15:34:30.323 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:34:30.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.316+0000 7f73eed8f640 1 -- 172.21.15.38:0/1673450314 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73d00a2b90 msgr2=0x7f73d00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.316+0000 7f73eed8f640 1 --2- 172.21.15.38:0/1673450314 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73d00a2b90 0x7f73d00a50a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55a6b681a640 tx=0x7f73e00096a0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.316+0000 7f73eed8f640 1 -- 172.21.15.38:0/1673450314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e8103d00 msgr2=0x7f73e81997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.316+0000 7f73eed8f640 1 --2- 172.21.15.38:0/1673450314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e8103d00 0x7f73e81997f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f73d8006010 tx=0x7f73d8031b60 comp rx=0 tx=0).stop 2024-05-21T15:34:30.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.316+0000 7f73eed8f640 1 -- 172.21.15.38:0/1673450314 shutdown_connections 2024-05-21T15:34:30.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.316+0000 7f73eed8f640 1 --2- 172.21.15.38:0/1673450314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73e8108a00 0x7f73e81a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.316+0000 7f73eed8f640 1 --2- 172.21.15.38:0/1673450314 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f73d00a2b90 0x7f73d00a50a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.316+0000 7f73eed8f640 1 --2- 172.21.15.38:0/1673450314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73e8103d00 0x7f73e81997f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.316+0000 7f73eed8f640 1 --2- 172.21.15.38:0/1673450314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e8102ac0 0x7f73e81992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.316+0000 7f73eed8f640 1 -- 172.21.15.38:0/1673450314 >> 172.21.15.38:0/1673450314 conn(0x7f73e80fdcf0 msgr2=0x7f73e81049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.316+0000 7f73eed8f640 1 -- 172.21.15.38:0/1673450314 shutdown_connections 2024-05-21T15:34:30.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.316+0000 7f73eed8f640 1 -- 172.21.15.38:0/1673450314 wait complete. 2024-05-21T15:34:30.327 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-21T15:34:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.442+0000 7f27bf6f9640 1 -- 172.21.15.38:0/2747883225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b8102ab0 msgr2=0x7f27b81037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.442+0000 7f27bf6f9640 1 --2- 172.21.15.38:0/2747883225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b8102ab0 0x7f27b81037b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009900 tx=0x7f27b402a410 comp rx=0 tx=0).stop 2024-05-21T15:34:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.443+0000 7f27bf6f9640 1 -- 172.21.15.38:0/2747883225 shutdown_connections 2024-05-21T15:34:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.443+0000 7f27bf6f9640 1 --2- 172.21.15.38:0/2747883225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27b81089f0 0x7f27b810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.443+0000 7f27bf6f9640 1 --2- 172.21.15.38:0/2747883225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27b8103cf0 0x7f27b8104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.443+0000 7f27bf6f9640 1 --2- 172.21.15.38:0/2747883225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b8102ab0 0x7f27b81037b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.443+0000 7f27bf6f9640 1 -- 172.21.15.38:0/2747883225 >> 172.21.15.38:0/2747883225 conn(0x7f27b80fdcc0 msgr2=0x7f27b8100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.443+0000 7f27bf6f9640 1 -- 172.21.15.38:0/2747883225 shutdown_connections 2024-05-21T15:34:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.443+0000 7f27bf6f9640 1 -- 172.21.15.38:0/2747883225 wait complete. 2024-05-21T15:34:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.443+0000 7f27bf6f9640 1 Processor -- start 2024-05-21T15:34:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.443+0000 7f27bf6f9640 1 -- start start 2024-05-21T15:34:30.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bf6f9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27b8102ab0 0x7f27b8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bf6f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b8103cf0 0x7f27b81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bf6f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27b81089f0 0x7f27b81a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bf6f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f27b810e700 con 0x7f27b8103cf0 2024-05-21T15:34:30.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bf6f9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27b810e580 con 0x7f27b81089f0 2024-05-21T15:34:30.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bf6f9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f27b810e880 con 0x7f27b8102ab0 2024-05-21T15:34:30.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bcc94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b8103cf0 0x7f27b81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bcc94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b8103cf0 0x7f27b81997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60030/0 (socket says 172.21.15.38:60030) 2024-05-21T15:34:30.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bcc94640 1 -- 172.21.15.38:0/529080174 learned_addr learned my addr 172.21.15.38:0/529080174 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bd495640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27b8102ab0 0x7f27b8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bdc96640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27b81089f0 0x7f27b81a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bcc94640 1 -- 172.21.15.38:0/529080174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27b8102ab0 msgr2=0x7f27b8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bcc94640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27b8102ab0 0x7f27b8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bcc94640 1 -- 172.21.15.38:0/529080174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27b81089f0 msgr2=0x7f27b81a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bcc94640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27b81089f0 0x7f27b81a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bcc94640 1 -- 172.21.15.38:0/529080174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b81a0dc0 con 0x7f27b8103cf0 2024-05-21T15:34:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bdc96640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27b81089f0 0x7f27b81a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bcc94640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b8103cf0 0x7f27b81997c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f27a800e9f0 tx=0x7f27a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27ae7fc640 1 -- 172.21.15.38:0/529080174 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a800cd80 con 0x7f27b8103cf0 2024-05-21T15:34:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27bf6f9640 1 -- 172.21.15.38:0/529080174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b81a10b0 con 0x7f27b8103cf0 2024-05-21T15:34:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27ae7fc640 1 -- 172.21.15.38:0/529080174 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27a8005500 con 0x7f27b8103cf0 2024-05-21T15:34:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.444+0000 7f27ae7fc640 1 -- 172.21.15.38:0/529080174 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8010690 con 0x7f27b8103cf0 2024-05-21T15:34:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.445+0000 7f27bf6f9640 1 -- 172.21.15.38:0/529080174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b81a1600 con 0x7f27b8103cf0 2024-05-21T15:34:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.447+0000 7f27ae7fc640 1 -- 172.21.15.38:0/529080174 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f27a80046c0 con 0x7f27b8103cf0 2024-05-21T15:34:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.447+0000 7f27c659b740 1 Processor -- start 2024-05-21T15:34:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.447+0000 7f27c659b740 1 -- start start 2024-05-21T15:34:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27c659b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d86270 0x556145126470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27c659b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 0x5561451269f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27c659b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d65590 0x55614512d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27c659b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55614508a4d0 con 0x556144d86270 2024-05-21T15:34:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27c659b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5561450323e0 con 0x556144d65590 2024-05-21T15:34:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27c659b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556145086490 con 0x556144d5d3b0 2024-05-21T15:34:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27bdc96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d86270 0x556145126470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27bdc96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d86270 0x556145126470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60036/0 (socket says 172.21.15.38:60036) 2024-05-21T15:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27bdc96640 1 -- 172.21.15.38:0/1575431604 learned_addr learned my addr 172.21.15.38:0/1575431604 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27bd495640 1 --2- 172.21.15.38:0/1575431604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 0x5561451269f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27bcc94640 1 --2- 172.21.15.38:0/1575431604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d65590 0x55614512d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27bdc96640 1 -- 172.21.15.38:0/1575431604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 msgr2=0x5561451269f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27bdc96640 1 --2- 172.21.15.38:0/1575431604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 0x5561451269f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27bdc96640 1 -- 172.21.15.38:0/1575431604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d65590 msgr2=0x55614512d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27bdc96640 1 --2- 172.21.15.38:0/1575431604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d65590 0x55614512d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27bdc96640 1 -- 172.21.15.38:0/1575431604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556144d9e2d0 con 0x556144d86270 2024-05-21T15:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27bdc96640 1 --2- 172.21.15.38:0/1575431604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d86270 0x556145126470 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009f90 tx=0x7f27b000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb0e74aef821517 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.448+0000 7f27ae7fc640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27a40a2b90 0x7f27a40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.449+0000 7f27a37fe640 1 -- 172.21.15.38:0/1575431604 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0003b70 con 0x556144d86270 2024-05-21T15:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.449+0000 7f27bd495640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27a40a2b90 0x7f27a40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.449+0000 7f27a37fe640 1 -- 172.21.15.38:0/1575431604 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b0003d10 con 0x556144d86270 2024-05-21T15:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.449+0000 7f27ae7fc640 1 -- 172.21.15.38:0/529080174 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f27a80cf520 con 0x7f27b8103cf0 2024-05-21T15:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.449+0000 7f27a37fe640 1 -- 172.21.15.38:0/1575431604 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0011a60 con 0x556144d86270 2024-05-21T15:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.449+0000 7f27c659b740 1 -- 172.21.15.38:0/1575431604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d86270 msgr2=0x556145126470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.449+0000 7f27bd495640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27a40a2b90 0x7f27a40a50a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f27b4005ab0 tx=0x7f27b4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.450+0000 7f27c659b740 1 --2- 172.21.15.38:0/1575431604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d86270 0x556145126470 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009f90 tx=0x7f27b000f7f0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.450+0000 7f27c659b740 1 -- 172.21.15.38:0/1575431604 shutdown_connections 2024-05-21T15:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.450+0000 7f27c659b740 1 --2- 172.21.15.38:0/1575431604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d65590 0x55614512d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.450+0000 7f27c659b740 1 --2- 172.21.15.38:0/1575431604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 0x5561451269f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.450+0000 7f27c659b740 1 --2- 172.21.15.38:0/1575431604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d86270 0x556145126470 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.450+0000 7f27c659b740 1 -- 172.21.15.38:0/1575431604 >> 172.21.15.38:0/1575431604 conn(0x556144d7a3c0 msgr2=0x556145121e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.450+0000 7f27c659b740 1 -- 172.21.15.38:0/1575431604 shutdown_connections 2024-05-21T15:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.450+0000 7f27c659b740 1 -- 172.21.15.38:0/1575431604 wait complete. 2024-05-21T15:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.450+0000 7f27c659b740 1 -- start start 2024-05-21T15:34:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.450+0000 7f27c659b740 1 Processor -- start 2024-05-21T15:34:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.450+0000 7f27c659b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 0x556145130350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.450+0000 7f27c659b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d65590 0x556145130890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27c659b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d86270 0x5561451377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27c659b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55614512e340 con 0x556144d86270 2024-05-21T15:34:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27c659b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5561450458e0 con 0x556144d65590 2024-05-21T15:34:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27c659b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55614512e4c0 con 0x556144d5d3b0 2024-05-21T15:34:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27c659b740 10 client.0 _ll_register_callbacks cb 0x5561450129c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27bdc96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d86270 0x5561451377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27bdc96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d86270 0x5561451377f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60040/0 (socket says 172.21.15.38:60040) 2024-05-21T15:34:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27bdc96640 1 -- 172.21.15.38:0/2487180113 learned_addr learned my addr 172.21.15.38:0/2487180113 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27bd495640 1 --2- 172.21.15.38:0/2487180113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 0x556145130350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27bcc94640 1 --2- 172.21.15.38:0/2487180113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d65590 0x556145130890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27bdc96640 1 -- 172.21.15.38:0/2487180113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 msgr2=0x556145130350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27bdc96640 1 --2- 172.21.15.38:0/2487180113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 0x556145130350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27bdc96640 1 -- 172.21.15.38:0/2487180113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d65590 msgr2=0x556145130890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27bdc96640 1 --2- 172.21.15.38:0/2487180113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d65590 0x556145130890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27bdc96640 1 -- 172.21.15.38:0/2487180113 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556144e686b0 con 0x556144d86270 2024-05-21T15:34:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27bcc94640 1 --2- 172.21.15.38:0/2487180113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d65590 0x556145130890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27bdc96640 1 --2- 172.21.15.38:0/2487180113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d86270 0x5561451377f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f27b000fcf0 tx=0x7f27b0009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3e383baac40987 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27a0ff9640 1 -- 172.21.15.38:0/2487180113 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b002a060 con 0x556144d86270 2024-05-21T15:34:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27a0ff9640 1 -- 172.21.15.38:0/2487180113 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f27b0004a40 con 0x556144d86270 2024-05-21T15:34:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27c659b740 1 -- 172.21.15.38:0/2487180113 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b0003d10 con 0x556144d86270 2024-05-21T15:34:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.451+0000 7f27a0ff9640 1 -- 172.21.15.38:0/2487180113 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f27b0004c40 con 0x556144d86270 2024-05-21T15:34:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.452+0000 7f27c659b740 10 client.15321 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.452+0000 7f27c659b740 1 -- 172.21.15.38:0/2487180113 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55614513a720 con 0x556144d86270 2024-05-21T15:34:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.452+0000 7f27a0ff9640 1 -- 172.21.15.38:0/2487180113 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b0003d10 con 0x556144d86270 2024-05-21T15:34:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.452+0000 7f27a0ff9640 1 -- 172.21.15.38:0/2487180113 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f27b0004300 con 0x556144d86270 2024-05-21T15:34:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.452+0000 7f27c659b740 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.452+0000 7f27c659b740 10 client.15321 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T15:34:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.452+0000 7f27c659b740 20 client.15321 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.452+0000 7f27c659b740 1 --2- 172.21.15.38:0/2487180113 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x556144d44e40 0x55614513de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.452+0000 7f27c659b740 4 client.15321 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:34:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.452+0000 7f27c659b740 1 -- 172.21.15.38:0/2487180113 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556145097c00 con 0x556144d44e40 2024-05-21T15:34:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.452+0000 7f27bd495640 1 --2- 172.21.15.38:0/2487180113 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x556144d44e40 0x55614513de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.453+0000 7f27bd495640 1 --2- 172.21.15.38:0/2487180113 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x556144d44e40 0x55614513de40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e69b93584569cdf server_cookie=8bcc9ba81cce5181 in_seq=0 out_seq=0 2024-05-21T15:34:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.453+0000 7f27a0ff9640 10 client.15321 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:34:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27a0ff9640 1 -- 172.21.15.38:0/2487180113 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556145097c00 con 0x556144d44e40 2024-05-21T15:34:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27a0ff9640 10 client.15321 handle_command_reply: tid=0 2024-05-21T15:34:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27c659b740 1 client.15321 shutdown 2024-05-21T15:34:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27c659b740 1 -- 172.21.15.38:0/2487180113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d86270 msgr2=0x5561451377f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27c659b740 1 --2- 172.21.15.38:0/2487180113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d86270 0x5561451377f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f27b000fcf0 tx=0x7f27b0009d50 comp rx=0 tx=0).stop 2024-05-21T15:34:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27c659b740 1 -- 172.21.15.38:0/2487180113 shutdown_connections 2024-05-21T15:34:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27c659b740 1 --2- 172.21.15.38:0/2487180113 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x556144d44e40 0x55614513de40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27c659b740 1 --2- 172.21.15.38:0/2487180113 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d86270 0x5561451377f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27c659b740 1 --2- 172.21.15.38:0/2487180113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d65590 0x556145130890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27a0ff9640 0 client.15321 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:34:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27c659b740 1 --2- 172.21.15.38:0/2487180113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 0x556145130350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27c659b740 1 -- 172.21.15.38:0/2487180113 >> 172.21.15.38:0/2487180113 conn(0x556144d7a3c0 msgr2=0x556145120ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27c659b740 1 -- 172.21.15.38:0/2487180113 shutdown_connections 2024-05-21T15:34:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27c659b740 1 -- 172.21.15.38:0/2487180113 wait complete. 2024-05-21T15:34:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.454+0000 7f27c659b740 20 client.15321 trim_cache size 0 max 16384 2024-05-21T15:34:30.461 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:30 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2245582515' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:30.461 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:30 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/546824479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:30.461 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:30 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/908951201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27c659b740 1 Processor -- start 2024-05-21T15:34:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27c659b740 1 -- start start 2024-05-21T15:34:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27c659b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d44e40 0x55614512adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27c659b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 0x556145121f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27c659b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d65590 0x556145137710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27c659b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55614513c180 con 0x556144d65590 2024-05-21T15:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27c659b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55614513c850 con 0x556144d44e40 2024-05-21T15:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27c659b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55614513cf20 con 0x556144d5d3b0 2024-05-21T15:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27bdc96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d65590 0x556145137710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27bdc96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d65590 0x556145137710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60048/0 (socket says 172.21.15.38:60048) 2024-05-21T15:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27bdc96640 1 -- 172.21.15.38:0/133078793 learned_addr learned my addr 172.21.15.38:0/133078793 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27bcc94640 1 --2- 172.21.15.38:0/133078793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 0x556145121f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27bd495640 1 --2- 172.21.15.38:0/133078793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d44e40 0x55614512adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27bdc96640 1 -- 172.21.15.38:0/133078793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 msgr2=0x556145121f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27bdc96640 1 --2- 172.21.15.38:0/133078793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 0x556145121f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27bdc96640 1 -- 172.21.15.38:0/133078793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d44e40 msgr2=0x55614512adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27bdc96640 1 --2- 172.21.15.38:0/133078793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d44e40 0x55614512adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27bdc96640 1 -- 172.21.15.38:0/133078793 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556145126020 con 0x556144d65590 2024-05-21T15:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.473+0000 7f27bdc96640 1 --2- 172.21.15.38:0/133078793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d65590 0x556145137710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f27b00046a0 tx=0x7f27b0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab0b314a2b9ed4a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.474+0000 7f27a2ffd640 1 -- 172.21.15.38:0/133078793 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b002a070 con 0x556144d65590 2024-05-21T15:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.474+0000 7f27a2ffd640 1 -- 172.21.15.38:0/133078793 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b0004de0 con 0x556144d65590 2024-05-21T15:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.474+0000 7f27a2ffd640 1 -- 172.21.15.38:0/133078793 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b00056d0 con 0x556144d65590 2024-05-21T15:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.474+0000 7f27c659b740 1 -- 172.21.15.38:0/133078793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d65590 msgr2=0x556145137710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.474+0000 7f27c659b740 1 --2- 172.21.15.38:0/133078793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d65590 0x556145137710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f27b00046a0 tx=0x7f27b0009500 comp rx=0 tx=0).stop 2024-05-21T15:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.474+0000 7f27c659b740 1 -- 172.21.15.38:0/133078793 shutdown_connections 2024-05-21T15:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.474+0000 7f27c659b740 1 --2- 172.21.15.38:0/133078793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d65590 0x556145137710 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.474+0000 7f27c659b740 1 --2- 172.21.15.38:0/133078793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d5d3b0 0x556145121f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.474+0000 7f27c659b740 1 --2- 172.21.15.38:0/133078793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d44e40 0x55614512adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.474+0000 7f27c659b740 1 -- 172.21.15.38:0/133078793 >> 172.21.15.38:0/133078793 conn(0x556144d7a3c0 msgr2=0x5561451217f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.474+0000 7f27c659b740 1 -- 172.21.15.38:0/133078793 shutdown_connections 2024-05-21T15:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.474+0000 7f27c659b740 1 -- 172.21.15.38:0/133078793 wait complete. 2024-05-21T15:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.475+0000 7f27c659b740 1 -- start start 2024-05-21T15:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.475+0000 7f27c659b740 1 Processor -- start 2024-05-21T15:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.475+0000 7f27c659b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d44e40 0x556145132ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.475+0000 7f27c659b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d5d3b0 0x5561451330e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.475+0000 7f27c659b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d65590 0x5561451a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.475+0000 7f27c659b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55614513dcc0 con 0x556144d44e40 2024-05-21T15:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.475+0000 7f27c659b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55614513d5f0 con 0x556144d5d3b0 2024-05-21T15:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.475+0000 7f27c659b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55614513e390 con 0x556144d65590 2024-05-21T15:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.475+0000 7f27c659b740 10 client.0 _ll_register_callbacks cb 0x7f27b0015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.475+0000 7f27bd495640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d44e40 0x556145132ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.475+0000 7f27bd495640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d44e40 0x556145132ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60064/0 (socket says 172.21.15.38:60064) 2024-05-21T15:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.475+0000 7f27bd495640 1 -- 172.21.15.38:0/1564354233 learned_addr learned my addr 172.21.15.38:0/1564354233 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.475+0000 7f27bcc94640 1 --2- 172.21.15.38:0/1564354233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d5d3b0 0x5561451330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.475+0000 7f27bdc96640 1 --2- 172.21.15.38:0/1564354233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d65590 0x5561451a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27bd495640 1 -- 172.21.15.38:0/1564354233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d65590 msgr2=0x5561451a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27bd495640 1 --2- 172.21.15.38:0/1564354233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d65590 0x5561451a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27bd495640 1 -- 172.21.15.38:0/1564354233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d5d3b0 msgr2=0x5561451330e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27bd495640 1 --2- 172.21.15.38:0/1564354233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d5d3b0 0x5561451330e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27bd495640 1 -- 172.21.15.38:0/1564354233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556145138060 con 0x556144d44e40 2024-05-21T15:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27bdc96640 1 --2- 172.21.15.38:0/1564354233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d65590 0x5561451a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27bd495640 1 --2- 172.21.15.38:0/1564354233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d44e40 0x556145132ba0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009230 tx=0x7f27b402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c1b85a635911b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27a0ff9640 1 -- 172.21.15.38:0/1564354233 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4059060 con 0x556144d44e40 2024-05-21T15:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27a0ff9640 1 -- 172.21.15.38:0/1564354233 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f27b4005040 con 0x556144d44e40 2024-05-21T15:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27c659b740 1 -- 172.21.15.38:0/1564354233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b0004de0 con 0x556144d44e40 2024-05-21T15:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27a0ff9640 1 -- 172.21.15.38:0/1564354233 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f27b4035040 con 0x556144d44e40 2024-05-21T15:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27c659b740 10 client.15327 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27c659b740 1 -- 172.21.15.38:0/1564354233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5561451a60a0 con 0x556144d44e40 2024-05-21T15:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27a0ff9640 1 -- 172.21.15.38:0/1564354233 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b0004de0 con 0x556144d44e40 2024-05-21T15:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27a0ff9640 1 -- 172.21.15.38:0/1564354233 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f27b4053070 con 0x556144d44e40 2024-05-21T15:34:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.476+0000 7f27c659b740 10 client.15327 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.477+0000 7f27c659b740 10 client.15327 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T15:34:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.477+0000 7f27c659b740 20 client.15327 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.477+0000 7f27c659b740 1 --2- 172.21.15.38:0/1564354233 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x556144d86270 0x5561451a70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.477+0000 7f27c659b740 4 client.15327 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:34:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.477+0000 7f27c659b740 1 -- 172.21.15.38:0/1564354233 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "status"}) -- 0x556145091f50 con 0x556144d86270 2024-05-21T15:34:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.477+0000 7f27bcc94640 1 --2- 172.21.15.38:0/1564354233 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x556144d86270 0x5561451a70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.477+0000 7f27bcc94640 1 --2- 172.21.15.38:0/1564354233 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x556144d86270 0x5561451a70d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d04c53aac4e41695 server_cookie=ba41be69fb7414e2 in_seq=0 out_seq=0 2024-05-21T15:34:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.477+0000 7f27a0ff9640 10 client.15327 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:34:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.477+0000 7f27a0ff9640 1 -- 172.21.15.38:0/1564354233 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x556145091f50 con 0x556144d86270 2024-05-21T15:34:30.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.477+0000 7f27a0ff9640 10 client.15327 handle_command_reply: tid=0 2024-05-21T15:34:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.477+0000 7f27c659b740 1 client.15327 shutdown 2024-05-21T15:34:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27c659b740 1 -- 172.21.15.38:0/1564354233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d44e40 msgr2=0x556145132ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27c659b740 1 --2- 172.21.15.38:0/1564354233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d44e40 0x556145132ba0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009230 tx=0x7f27b402a410 comp rx=0 tx=0).stop 2024-05-21T15:34:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27c659b740 1 -- 172.21.15.38:0/1564354233 shutdown_connections 2024-05-21T15:34:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27c659b740 1 --2- 172.21.15.38:0/1564354233 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x556144d86270 0x5561451a70d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27c659b740 1 --2- 172.21.15.38:0/1564354233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556144d65590 0x5561451a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27c659b740 1 --2- 172.21.15.38:0/1564354233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556144d5d3b0 0x5561451330e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27a0ff9640 0 client.15327 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:34:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27c659b740 1 --2- 172.21.15.38:0/1564354233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556144d44e40 0x556145132ba0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27c659b740 1 -- 172.21.15.38:0/1564354233 >> 172.21.15.38:0/1564354233 conn(0x556144d7a3c0 msgr2=0x556145120470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27c659b740 1 -- 172.21.15.38:0/1564354233 shutdown_connections 2024-05-21T15:34:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27c659b740 1 -- 172.21.15.38:0/1564354233 wait complete. 2024-05-21T15:34:30.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27c659b740 20 client.15327 trim_cache size 0 max 16384 2024-05-21T15:34:30.485 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:34:30.485 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "102ec4fa-1786-11ef-bc9a-c7b262605968", 2024-05-21T15:34:30.485 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 2, 2024-05-21T15:34:30.485 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24550, 2024-05-21T15:34:30.485 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-05-21T15:34:30.485 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-05-21T15:34:30.485 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-05-21T15:34:30.485 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 112.527533925, 2024-05-21T15:34:30.485 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-05-21T15:34:30.485 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 90, 2024-05-21T15:34:30.486 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-05-21T15:34:30.486 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 130.277635433 2024-05-21T15:34:30.486 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:34:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27bf6f9640 1 -- 172.21.15.38:0/529080174 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27a40a2b90 msgr2=0x7f27a40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27bf6f9640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27a40a2b90 0x7f27a40a50a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f27b4005ab0 tx=0x7f27b4031400 comp rx=0 tx=0).stop 2024-05-21T15:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27bf6f9640 1 -- 172.21.15.38:0/529080174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b8103cf0 msgr2=0x7f27b81997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27bf6f9640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b8103cf0 0x7f27b81997c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f27a800e9f0 tx=0x7f27a800edf0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27bf6f9640 1 -- 172.21.15.38:0/529080174 shutdown_connections 2024-05-21T15:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27bf6f9640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27b81089f0 0x7f27b81a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27bf6f9640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27a40a2b90 0x7f27a40a50a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27bf6f9640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b8103cf0 0x7f27b81997c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27bf6f9640 1 --2- 172.21.15.38:0/529080174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27b8102ab0 0x7f27b8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.478+0000 7f27bf6f9640 1 -- 172.21.15.38:0/529080174 >> 172.21.15.38:0/529080174 conn(0x7f27b80fdcc0 msgr2=0x7f27b81049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.479+0000 7f27bf6f9640 1 -- 172.21.15.38:0/529080174 shutdown_connections 2024-05-21T15:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.479+0000 7f27bf6f9640 1 -- 172.21.15.38:0/529080174 wait complete. 2024-05-21T15:34:30.489 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-05-21T15:34:30.510 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:30 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2245582515' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:30.510 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:30 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/546824479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:30.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:30 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/908951201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:30.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.605+0000 7fbff07e3640 1 -- 172.21.15.38:0/3215785140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfe8105ee0 msgr2=0x7fbfe8108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.605+0000 7fbff07e3640 1 --2- 172.21.15.38:0/3215785140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfe8105ee0 0x7fbfe8108300 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009900 tx=0x7fbfd802a410 comp rx=0 tx=0).stop 2024-05-21T15:34:30.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.605+0000 7fbff07e3640 1 -- 172.21.15.38:0/3215785140 shutdown_connections 2024-05-21T15:34:30.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.605+0000 7fbff07e3640 1 --2- 172.21.15.38:0/3215785140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe8108840 0x7fbfe810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.605+0000 7fbff07e3640 1 --2- 172.21.15.38:0/3215785140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfe8105ee0 0x7fbfe8108300 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.605+0000 7fbff07e3640 1 --2- 172.21.15.38:0/3215785140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfe8103580 0x7fbfe81059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.605+0000 7fbff07e3640 1 -- 172.21.15.38:0/3215785140 >> 172.21.15.38:0/3215785140 conn(0x7fbfe80fcb80 msgr2=0x7fbfe80ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.606+0000 7fbff07e3640 1 -- 172.21.15.38:0/3215785140 shutdown_connections 2024-05-21T15:34:30.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.606+0000 7fbff07e3640 1 -- 172.21.15.38:0/3215785140 wait complete. 2024-05-21T15:34:30.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.606+0000 7fbff07e3640 1 Processor -- start 2024-05-21T15:34:30.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.606+0000 7fbff07e3640 1 -- start start 2024-05-21T15:34:30.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.606+0000 7fbff07e3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfe8103580 0x7fbfe81992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.606+0000 7fbff07e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfe8105ee0 0x7fbfe81997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.606+0000 7fbff07e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe8108840 0x7fbfe81a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.606+0000 7fbff07e3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe810e6f0 con 0x7fbfe8105ee0 2024-05-21T15:34:30.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.606+0000 7fbff07e3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe810e570 con 0x7fbfe8108840 2024-05-21T15:34:30.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.606+0000 7fbff07e3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe810e870 con 0x7fbfe8103580 2024-05-21T15:34:30.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfedd7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfe8105ee0 0x7fbfe81997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfedd7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfe8105ee0 0x7fbfe81997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60082/0 (socket says 172.21.15.38:60082) 2024-05-21T15:34:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfedd7e640 1 -- 172.21.15.38:0/3238494440 learned_addr learned my addr 172.21.15.38:0/3238494440 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfeed80640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe8108840 0x7fbfe81a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfee57f640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfe8103580 0x7fbfe81992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfedd7e640 1 -- 172.21.15.38:0/3238494440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfe8103580 msgr2=0x7fbfe81992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfedd7e640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfe8103580 0x7fbfe81992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfedd7e640 1 -- 172.21.15.38:0/3238494440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe8108840 msgr2=0x7fbfe81a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfedd7e640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe8108840 0x7fbfe81a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfedd7e640 1 -- 172.21.15.38:0/3238494440 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe81a0de0 con 0x7fbfe8105ee0 2024-05-21T15:34:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfee57f640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfe8103580 0x7fbfe81992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfedd7e640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfe8105ee0 0x7fbfe81997e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8006010 tx=0x7fbfd80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfdf7fe640 1 -- 172.21.15.38:0/3238494440 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd803f030 con 0x7fbfe8105ee0 2024-05-21T15:34:30.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfdf7fe640 1 -- 172.21.15.38:0/3238494440 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfd8003ce0 con 0x7fbfe8105ee0 2024-05-21T15:34:30.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbff07e3640 1 -- 172.21.15.38:0/3238494440 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe81a1070 con 0x7fbfe8105ee0 2024-05-21T15:34:30.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbfdf7fe640 1 -- 172.21.15.38:0/3238494440 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8035070 con 0x7fbfe8105ee0 2024-05-21T15:34:30.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.607+0000 7fbff07e3640 1 -- 172.21.15.38:0/3238494440 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe81a1560 con 0x7fbfe8105ee0 2024-05-21T15:34:30.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.610+0000 7fbff7767740 1 Processor -- start 2024-05-21T15:34:30.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbff7767740 1 -- start start 2024-05-21T15:34:30.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbfdf7fe640 1 -- 172.21.15.38:0/3238494440 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbfd8042030 con 0x7fbfe8105ee0 2024-05-21T15:34:30.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbff7767740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 0x559462870470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbff7767740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624a73b0 0x5594628709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbff7767740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624af590 0x5594628778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbff7767740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5594627d44d0 con 0x5594624af590 2024-05-21T15:34:30.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbff7767740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55946277c3e0 con 0x5594624a73b0 2024-05-21T15:34:30.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbff7767740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5594627d0490 con 0x5594624d0270 2024-05-21T15:34:30.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbfedd7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624af590 0x5594628778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbfee57f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624a73b0 0x5594628709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbfedd7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624af590 0x5594628778f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60092/0 (socket says 172.21.15.38:60092) 2024-05-21T15:34:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbfee57f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624a73b0 0x5594628709f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:56572/0 (socket says 172.21.15.38:56572) 2024-05-21T15:34:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbfedd7e640 1 -- 172.21.15.38:0/2639539401 learned_addr learned my addr 172.21.15.38:0/2639539401 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbfee57f640 1 -- 172.21.15.38:0/2639539401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 msgr2=0x559462870470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbfee57f640 1 --2- 172.21.15.38:0/2639539401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 0x559462870470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbfee57f640 1 -- 172.21.15.38:0/2639539401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624af590 msgr2=0x5594628778f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbfee57f640 1 --2- 172.21.15.38:0/2639539401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624af590 0x5594628778f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.611+0000 7fbfee57f640 1 -- 172.21.15.38:0/2639539401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594624e82d0 con 0x5594624a73b0 2024-05-21T15:34:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.612+0000 7fbfedd7e640 1 --2- 172.21.15.38:0/2639539401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624af590 0x5594628778f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:34:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.612+0000 7fbfdf7fe640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbfd40a2b90 0x7fbfd40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.612+0000 7fbfee57f640 1 --2- 172.21.15.38:0/2639539401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624a73b0 0x5594628709f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009c00 tx=0x7fbfe40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0dbb17f6b57fde6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.612+0000 7fbfeed80640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbfd40a2b90 0x7fbfd40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.612+0000 7fbfdf7fe640 1 -- 172.21.15.38:0/3238494440 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbfd80ef540 con 0x7fbfe8105ee0 2024-05-21T15:34:30.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.612+0000 7fbfdcff9640 1 -- 172.21.15.38:0/2639539401 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4012450 con 0x5594624a73b0 2024-05-21T15:34:30.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.612+0000 7fbfdcff9640 1 -- 172.21.15.38:0/2639539401 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe4012ad0 con 0x5594624a73b0 2024-05-21T15:34:30.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.612+0000 7fbfdcff9640 1 -- 172.21.15.38:0/2639539401 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe400fa80 con 0x5594624a73b0 2024-05-21T15:34:30.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.613+0000 7fbff7767740 1 -- 172.21.15.38:0/2639539401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624a73b0 msgr2=0x5594628709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.613+0000 7fbff7767740 1 --2- 172.21.15.38:0/2639539401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624a73b0 0x5594628709f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009c00 tx=0x7fbfe40117b0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.613+0000 7fbfeed80640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbfd40a2b90 0x7fbfd40a50a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x559462871640 tx=0x7fbfe0009b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.613+0000 7fbff7767740 1 -- 172.21.15.38:0/2639539401 shutdown_connections 2024-05-21T15:34:30.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.613+0000 7fbff7767740 1 --2- 172.21.15.38:0/2639539401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624af590 0x5594628778f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.613+0000 7fbff7767740 1 --2- 172.21.15.38:0/2639539401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624a73b0 0x5594628709f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.613+0000 7fbff7767740 1 --2- 172.21.15.38:0/2639539401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 0x559462870470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.613+0000 7fbff7767740 1 -- 172.21.15.38:0/2639539401 >> 172.21.15.38:0/2639539401 conn(0x5594624c43c0 msgr2=0x55946286be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.613+0000 7fbff7767740 1 -- 172.21.15.38:0/2639539401 shutdown_connections 2024-05-21T15:34:30.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.613+0000 7fbff7767740 1 -- 172.21.15.38:0/2639539401 wait complete. 2024-05-21T15:34:30.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbff7767740 1 -- start start 2024-05-21T15:34:30.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbff7767740 1 Processor -- start 2024-05-21T15:34:30.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbff7767740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624a73b0 0x55946287a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbff7767740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624af590 0x55946287a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbff7767740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 0x559462881710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbff7767740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559462878340 con 0x5594624af590 2024-05-21T15:34:30.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbff7767740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55946278f8e0 con 0x5594624a73b0 2024-05-21T15:34:30.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbff7767740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5594628784c0 con 0x5594624d0270 2024-05-21T15:34:30.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbff7767740 10 client.0 _ll_register_callbacks cb 0x55946275c9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:30.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbfedd7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624af590 0x55946287a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbfedd7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624af590 0x55946287a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60102/0 (socket says 172.21.15.38:60102) 2024-05-21T15:34:30.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbfedd7e640 1 -- 172.21.15.38:0/1061388280 learned_addr learned my addr 172.21.15.38:0/1061388280 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbfee57f640 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624a73b0 0x55946287a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbfedd7e640 1 -- 172.21.15.38:0/1061388280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 msgr2=0x559462881710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbfeed80640 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 0x559462881710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbfedd7e640 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 0x559462881710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbfedd7e640 1 -- 172.21.15.38:0/1061388280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624a73b0 msgr2=0x55946287a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbfedd7e640 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624a73b0 0x55946287a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbfedd7e640 1 -- 172.21.15.38:0/1061388280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594625b26b0 con 0x5594624af590 2024-05-21T15:34:30.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbfee57f640 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624a73b0 0x55946287a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:30.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.614+0000 7fbfeed80640 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 0x559462881710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:30.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbfedd7e640 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624af590 0x55946287a7b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80c4490 tx=0x7fbfd80092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab45e75e479fcbe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbfd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:30.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbfd1ffb640 1 -- 172.21.15.38:0/1061388280 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8049070 con 0x5594624af590 2024-05-21T15:34:30.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbfd1ffb640 1 -- 172.21.15.38:0/1061388280 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbfd80b83c0 con 0x5594624af590 2024-05-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbff7767740 1 -- 172.21.15.38:0/1061388280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfe4012ad0 con 0x5594624af590 2024-05-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbfd1ffb640 1 -- 172.21.15.38:0/1061388280 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbfd80bbb20 con 0x5594624af590 2024-05-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbff7767740 10 client.15339 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbff7767740 1 -- 172.21.15.38:0/1061388280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559462884640 con 0x5594624af590 2024-05-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbfd1ffb640 1 -- 172.21.15.38:0/1061388280 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe4012ad0 con 0x5594624af590 2024-05-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbfd1ffb640 1 -- 172.21.15.38:0/1061388280 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbfd80bbfb0 con 0x5594624af590 2024-05-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbff7767740 10 client.15339 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbff7767740 10 client.15339 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbff7767740 20 client.15339 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbff7767740 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55946248ee40 0x559462888190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbff7767740 4 client.15339 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:34:30.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.615+0000 7fbff7767740 1 -- 172.21.15.38:0/1061388280 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594627e1c00 con 0x55946248ee40 2024-05-21T15:34:30.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.616+0000 7fbfee57f640 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55946248ee40 0x559462888190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.616+0000 7fbfee57f640 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55946248ee40 0x559462888190 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3ae147884ce59d4 server_cookie=7e7fcba27626ce79 in_seq=0 out_seq=0 2024-05-21T15:34:30.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.616+0000 7fbfd1ffb640 10 client.15339 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:34:30.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbfd1ffb640 1 -- 172.21.15.38:0/1061388280 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5594627e1c00 con 0x55946248ee40 2024-05-21T15:34:30.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbfd1ffb640 10 client.15339 handle_command_reply: tid=0 2024-05-21T15:34:30.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbff7767740 1 client.15339 shutdown 2024-05-21T15:34:30.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbff7767740 1 -- 172.21.15.38:0/1061388280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624af590 msgr2=0x55946287a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbff7767740 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624af590 0x55946287a7b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80c4490 tx=0x7fbfd80092f0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbff7767740 1 -- 172.21.15.38:0/1061388280 shutdown_connections 2024-05-21T15:34:30.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbff7767740 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55946248ee40 0x559462888190 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbff7767740 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 0x559462881710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbff7767740 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624af590 0x55946287a7b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbfd1ffb640 0 client.15339 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:34:30.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbff7767740 1 --2- 172.21.15.38:0/1061388280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624a73b0 0x55946287a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbff7767740 1 -- 172.21.15.38:0/1061388280 >> 172.21.15.38:0/1061388280 conn(0x5594624c43c0 msgr2=0x55946286aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbff7767740 1 -- 172.21.15.38:0/1061388280 shutdown_connections 2024-05-21T15:34:30.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbff7767740 1 -- 172.21.15.38:0/1061388280 wait complete. 2024-05-21T15:34:30.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.617+0000 7fbff7767740 20 client.15339 trim_cache size 0 max 16384 2024-05-21T15:34:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.636+0000 7fbff7767740 1 Processor -- start 2024-05-21T15:34:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.636+0000 7fbff7767740 1 -- start start 2024-05-21T15:34:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.636+0000 7fbff7767740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624a73b0 0x55946286b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.636+0000 7fbff7767740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624af590 0x55946286bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.636+0000 7fbff7767740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624d0270 0x559462874640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.636+0000 7fbff7767740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5594628864d0 con 0x5594624a73b0 2024-05-21T15:34:30.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.636+0000 7fbff7767740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559462886ba0 con 0x5594624d0270 2024-05-21T15:34:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.636+0000 7fbff7767740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559462887270 con 0x5594624af590 2024-05-21T15:34:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.636+0000 7fbfee57f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624a73b0 0x55946286b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.636+0000 7fbfee57f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624a73b0 0x55946286b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60116/0 (socket says 172.21.15.38:60116) 2024-05-21T15:34:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.636+0000 7fbfee57f640 1 -- 172.21.15.38:0/1658333052 learned_addr learned my addr 172.21.15.38:0/1658333052 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.636+0000 7fbfedd7e640 1 --2- 172.21.15.38:0/1658333052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624af590 0x55946286bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.637+0000 7fbfee57f640 1 -- 172.21.15.38:0/1658333052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624af590 msgr2=0x55946286bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.637+0000 7fbfee57f640 1 --2- 172.21.15.38:0/1658333052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624af590 0x55946286bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.637+0000 7fbfee57f640 1 -- 172.21.15.38:0/1658333052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624d0270 msgr2=0x559462874640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.637+0000 7fbfee57f640 1 --2- 172.21.15.38:0/1658333052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624d0270 0x559462874640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.637+0000 7fbfee57f640 1 -- 172.21.15.38:0/1658333052 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559462870020 con 0x5594624a73b0 2024-05-21T15:34:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.637+0000 7fbfee57f640 1 --2- 172.21.15.38:0/1658333052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624a73b0 0x55946286b7f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x559462871850 tx=0x7fbfe4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db4ba04916eb492 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.637+0000 7fbfd3fff640 1 -- 172.21.15.38:0/1658333052 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4032070 con 0x5594624a73b0 2024-05-21T15:34:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.637+0000 7fbfd3fff640 1 -- 172.21.15.38:0/1658333052 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe4009e10 con 0x5594624a73b0 2024-05-21T15:34:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.637+0000 7fbfd3fff640 1 -- 172.21.15.38:0/1658333052 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe402c320 con 0x5594624a73b0 2024-05-21T15:34:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 -- 172.21.15.38:0/1658333052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624a73b0 msgr2=0x55946286b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 --2- 172.21.15.38:0/1658333052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624a73b0 0x55946286b7f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x559462871850 tx=0x7fbfe4009b50 comp rx=0 tx=0).stop 2024-05-21T15:34:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 -- 172.21.15.38:0/1658333052 shutdown_connections 2024-05-21T15:34:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 --2- 172.21.15.38:0/1658333052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624d0270 0x559462874640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 --2- 172.21.15.38:0/1658333052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624af590 0x55946286bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 --2- 172.21.15.38:0/1658333052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624a73b0 0x55946286b7f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 -- 172.21.15.38:0/1658333052 >> 172.21.15.38:0/1658333052 conn(0x5594624c43c0 msgr2=0x559462881a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 -- 172.21.15.38:0/1658333052 shutdown_connections 2024-05-21T15:34:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 -- 172.21.15.38:0/1658333052 wait complete. 2024-05-21T15:34:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 -- start start 2024-05-21T15:34:30.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 Processor -- start 2024-05-21T15:34:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624a73b0 0x559462881d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624af590 0x5594628f1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 0x5594628f3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559462888010 con 0x5594624a73b0 2024-05-21T15:34:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559462887940 con 0x5594624af590 2024-05-21T15:34:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5594628886e0 con 0x5594624d0270 2024-05-21T15:34:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.638+0000 7fbff7767740 10 client.0 _ll_register_callbacks cb 0x7fbfd8005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfee57f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624a73b0 0x559462881d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfee57f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624a73b0 0x559462881d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60130/0 (socket says 172.21.15.38:60130) 2024-05-21T15:34:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfee57f640 1 -- 172.21.15.38:0/3713112963 learned_addr learned my addr 172.21.15.38:0/3713112963 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfeed80640 1 --2- 172.21.15.38:0/3713112963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 0x5594628f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfedd7e640 1 --2- 172.21.15.38:0/3713112963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624af590 0x5594628f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfee57f640 1 -- 172.21.15.38:0/3713112963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 msgr2=0x5594628f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfee57f640 1 --2- 172.21.15.38:0/3713112963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 0x5594628f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfee57f640 1 -- 172.21.15.38:0/3713112963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624af590 msgr2=0x5594628f1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfee57f640 1 --2- 172.21.15.38:0/3713112963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624af590 0x5594628f1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfee57f640 1 -- 172.21.15.38:0/3713112963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594628f3c70 con 0x5594624a73b0 2024-05-21T15:34:30.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfee57f640 1 --2- 172.21.15.38:0/3713112963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624a73b0 0x559462881d30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4030040 tx=0x7fbfe4009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83c23354769fc42 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:30.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfd1ffb640 1 -- 172.21.15.38:0/3713112963 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4032040 con 0x5594624a73b0 2024-05-21T15:34:30.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfd1ffb640 1 -- 172.21.15.38:0/3713112963 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbfe4005050 con 0x5594624a73b0 2024-05-21T15:34:30.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbff7767740 1 -- 172.21.15.38:0/3713112963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfe4009e10 con 0x5594624a73b0 2024-05-21T15:34:30.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbfd1ffb640 1 -- 172.21.15.38:0/3713112963 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbfe4005250 con 0x5594624a73b0 2024-05-21T15:34:30.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbff7767740 10 client.15345 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:30.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.639+0000 7fbff7767740 1 -- 172.21.15.38:0/3713112963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594628f4ef0 con 0x5594624a73b0 2024-05-21T15:34:30.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.640+0000 7fbfd1ffb640 1 -- 172.21.15.38:0/3713112963 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe4009e10 con 0x5594624a73b0 2024-05-21T15:34:30.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.640+0000 7fbfd1ffb640 1 -- 172.21.15.38:0/3713112963 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbfe40035f0 con 0x5594624a73b0 2024-05-21T15:34:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.640+0000 7fbff7767740 10 client.15345 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.640+0000 7fbff7767740 10 client.15345 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T15:34:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.640+0000 7fbff7767740 20 client.15345 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.640+0000 7fbff7767740 1 --2- 172.21.15.38:0/3713112963 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x559462464280 0x5594628f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.640+0000 7fbff7767740 4 client.15345 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:34:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.640+0000 7fbff7767740 1 -- 172.21.15.38:0/3713112963 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "status"}) -- 0x5594627dbf50 con 0x559462464280 2024-05-21T15:34:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.640+0000 7fbfedd7e640 1 --2- 172.21.15.38:0/3713112963 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x559462464280 0x5594628f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.640+0000 7fbfd1ffb640 10 client.15345 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:34:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.640+0000 7fbfedd7e640 1 --2- 172.21.15.38:0/3713112963 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x559462464280 0x5594628f80d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4127d39cb6cd8a2b server_cookie=e9aed91985a68fb5 in_seq=0 out_seq=0 2024-05-21T15:34:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbfd1ffb640 1 -- 172.21.15.38:0/3713112963 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5594627dbf50 con 0x559462464280 2024-05-21T15:34:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbfd1ffb640 10 client.15345 handle_command_reply: tid=0 2024-05-21T15:34:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff7767740 1 client.15345 shutdown 2024-05-21T15:34:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff7767740 1 -- 172.21.15.38:0/3713112963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624a73b0 msgr2=0x559462881d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff7767740 1 --2- 172.21.15.38:0/3713112963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624a73b0 0x559462881d30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4030040 tx=0x7fbfe4009860 comp rx=0 tx=0).stop 2024-05-21T15:34:30.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff7767740 1 -- 172.21.15.38:0/3713112963 shutdown_connections 2024-05-21T15:34:30.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff7767740 1 --2- 172.21.15.38:0/3713112963 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x559462464280 0x5594628f80d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff7767740 1 --2- 172.21.15.38:0/3713112963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594624d0270 0x5594628f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbfd1ffb640 0 client.15345 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:34:30.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff7767740 1 --2- 172.21.15.38:0/3713112963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594624af590 0x5594628f1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff7767740 1 --2- 172.21.15.38:0/3713112963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594624a73b0 0x559462881d30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff7767740 1 -- 172.21.15.38:0/3713112963 >> 172.21.15.38:0/3713112963 conn(0x5594624c43c0 msgr2=0x55946286b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff7767740 1 -- 172.21.15.38:0/3713112963 shutdown_connections 2024-05-21T15:34:30.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff7767740 1 -- 172.21.15.38:0/3713112963 wait complete. 2024-05-21T15:34:30.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff7767740 20 client.15345 trim_cache size 0 max 16384 2024-05-21T15:34:30.647 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:34:30.648 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "102ec4fa-1786-11ef-bc9a-c7b262605968", 2024-05-21T15:34:30.648 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 1, 2024-05-21T15:34:30.648 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24551, 2024-05-21T15:34:30.648 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-05-21T15:34:30.648 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-05-21T15:34:30.648 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-05-21T15:34:30.648 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 113.710663655, 2024-05-21T15:34:30.648 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-05-21T15:34:30.648 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 90, 2024-05-21T15:34:30.648 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-05-21T15:34:30.648 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 133.252019778 2024-05-21T15:34:30.648 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:34:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff07e3640 1 -- 172.21.15.38:0/3238494440 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbfd40a2b90 msgr2=0x7fbfd40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff07e3640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbfd40a2b90 0x7fbfd40a50a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x559462871640 tx=0x7fbfe0009b10 comp rx=0 tx=0).stop 2024-05-21T15:34:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff07e3640 1 -- 172.21.15.38:0/3238494440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfe8105ee0 msgr2=0x7fbfe81997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.641+0000 7fbff07e3640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfe8105ee0 0x7fbfe81997e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8006010 tx=0x7fbfd80042f0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.642+0000 7fbff07e3640 1 -- 172.21.15.38:0/3238494440 shutdown_connections 2024-05-21T15:34:30.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.642+0000 7fbff07e3640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe8108840 0x7fbfe81a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.642+0000 7fbff07e3640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbfd40a2b90 0x7fbfd40a50a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.642+0000 7fbff07e3640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfe8105ee0 0x7fbfe81997e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.642+0000 7fbff07e3640 1 --2- 172.21.15.38:0/3238494440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfe8103580 0x7fbfe81992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.642+0000 7fbff07e3640 1 -- 172.21.15.38:0/3238494440 >> 172.21.15.38:0/3238494440 conn(0x7fbfe80fcb80 msgr2=0x7fbfe8104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.642+0000 7fbff07e3640 1 -- 172.21.15.38:0/3238494440 shutdown_connections 2024-05-21T15:34:30.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.642+0000 7fbff07e3640 1 -- 172.21.15.38:0/3238494440 wait complete. 2024-05-21T15:34:30.652 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-21T15:34:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.768+0000 7ff678e3f640 1 -- 172.21.15.38:0/1016775905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff674102bc0 msgr2=0x7ff674102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.768+0000 7ff678e3f640 1 --2- 172.21.15.38:0/1016775905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff674102bc0 0x7ff674102fd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff668009ca0 tx=0x7ff66802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.768+0000 7ff678e3f640 1 -- 172.21.15.38:0/1016775905 shutdown_connections 2024-05-21T15:34:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.768+0000 7ff678e3f640 1 --2- 172.21.15.38:0/1016775905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6741078c0 0x7ff674109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.768+0000 7ff678e3f640 1 --2- 172.21.15.38:0/1016775905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff674102bc0 0x7ff674102fd0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.768+0000 7ff678e3f640 1 --2- 172.21.15.38:0/1016775905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff674101980 0x7ff674102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.768+0000 7ff678e3f640 1 -- 172.21.15.38:0/1016775905 >> 172.21.15.38:0/1016775905 conn(0x7ff6740fcb70 msgr2=0x7ff6740ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.768+0000 7ff678e3f640 1 -- 172.21.15.38:0/1016775905 shutdown_connections 2024-05-21T15:34:30.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.768+0000 7ff678e3f640 1 -- 172.21.15.38:0/1016775905 wait complete. 2024-05-21T15:34:30.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.769+0000 7ff678e3f640 1 Processor -- start 2024-05-21T15:34:30.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.769+0000 7ff678e3f640 1 -- start start 2024-05-21T15:34:30.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.769+0000 7ff678e3f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff674101980 0x7ff6740699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.769+0000 7ff678e3f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff674102bc0 0x7ff67419e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.769+0000 7ff678e3f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6741078c0 0x7ff6741a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.769+0000 7ff678e3f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff67410d5d0 con 0x7ff674102bc0 2024-05-21T15:34:30.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.769+0000 7ff678e3f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff67410d450 con 0x7ff6741078c0 2024-05-21T15:34:30.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.769+0000 7ff678e3f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff67410d750 con 0x7ff674101980 2024-05-21T15:34:30.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.769+0000 7ff671d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff674102bc0 0x7ff67419e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.769+0000 7ff671d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff674102bc0 0x7ff67419e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60160/0 (socket says 172.21.15.38:60160) 2024-05-21T15:34:30.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.769+0000 7ff671d9b640 1 -- 172.21.15.38:0/1688911823 learned_addr learned my addr 172.21.15.38:0/1688911823 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.770+0000 7ff67259c640 1 --2- 172.21.15.38:0/1688911823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff674101980 0x7ff6740699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.770+0000 7ff671d9b640 1 -- 172.21.15.38:0/1688911823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff674101980 msgr2=0x7ff6740699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.770+0000 7ff671d9b640 1 --2- 172.21.15.38:0/1688911823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff674101980 0x7ff6740699b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.770+0000 7ff671d9b640 1 -- 172.21.15.38:0/1688911823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6741078c0 msgr2=0x7ff6741a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.770+0000 7ff671d9b640 1 --2- 172.21.15.38:0/1688911823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6741078c0 0x7ff6741a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.770+0000 7ff671d9b640 1 -- 172.21.15.38:0/1688911823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6741a0d80 con 0x7ff674102bc0 2024-05-21T15:34:30.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.770+0000 7ff671d9b640 1 --2- 172.21.15.38:0/1688911823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff674102bc0 0x7ff67419e060 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff668005e30 tx=0x7ff668031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.770+0000 7ff6637fe640 1 -- 172.21.15.38:0/1688911823 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66802fb60 con 0x7ff674102bc0 2024-05-21T15:34:30.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.770+0000 7ff6637fe640 1 -- 172.21.15.38:0/1688911823 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff66802fd00 con 0x7ff674102bc0 2024-05-21T15:34:30.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.770+0000 7ff678e3f640 1 -- 172.21.15.38:0/1688911823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6741a1010 con 0x7ff674102bc0 2024-05-21T15:34:30.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.770+0000 7ff6637fe640 1 -- 172.21.15.38:0/1688911823 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668046870 con 0x7ff674102bc0 2024-05-21T15:34:30.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.770+0000 7ff678e3f640 1 -- 172.21.15.38:0/1688911823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6741a15b0 con 0x7ff674102bc0 2024-05-21T15:34:30.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.773+0000 7ff67fdbd740 1 Processor -- start 2024-05-21T15:34:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.773+0000 7ff6637fe640 1 -- 172.21.15.38:0/1688911823 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff668030bc0 con 0x7ff674102bc0 2024-05-21T15:34:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.773+0000 7ff67fdbd740 1 -- start start 2024-05-21T15:34:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.773+0000 7ff67fdbd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a60270 0x55bae5e00470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.773+0000 7ff67fdbd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e009f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.773+0000 7ff67fdbd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a3f590 0x55bae5e078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.773+0000 7ff67fdbd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bae5d644d0 con 0x55bae5a373b0 2024-05-21T15:34:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.773+0000 7ff67fdbd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bae5d0c3e0 con 0x55bae5a3f590 2024-05-21T15:34:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.773+0000 7ff67fdbd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bae5d60490 con 0x55bae5a60270 2024-05-21T15:34:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.773+0000 7ff67259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.773+0000 7ff67259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60170/0 (socket says 172.21.15.38:60170) 2024-05-21T15:34:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.773+0000 7ff67259c640 1 -- 172.21.15.38:0/2681880763 learned_addr learned my addr 172.21.15.38:0/2681880763 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.773+0000 7ff672d9d640 1 --2- 172.21.15.38:0/2681880763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a60270 0x55bae5e00470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.774+0000 7ff671d9b640 1 --2- 172.21.15.38:0/2681880763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a3f590 0x55bae5e078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.774+0000 7ff6637fe640 1 --2- 172.21.15.38:0/1688911823 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff65c0a2c10 0x7ff65c0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.774+0000 7ff67259c640 1 -- 172.21.15.38:0/2681880763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a60270 msgr2=0x55bae5e00470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.774+0000 7ff67259c640 1 --2- 172.21.15.38:0/2681880763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a60270 0x55bae5e00470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.774+0000 7ff67259c640 1 -- 172.21.15.38:0/2681880763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a3f590 msgr2=0x55bae5e078f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.774+0000 7ff67259c640 1 --2- 172.21.15.38:0/2681880763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a3f590 0x55bae5e078f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.774+0000 7ff67259c640 1 -- 172.21.15.38:0/2681880763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bae5a782d0 con 0x55bae5a373b0 2024-05-21T15:34:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.774+0000 7ff672d9d640 1 --2- 172.21.15.38:0/2681880763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a60270 0x55bae5e00470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:34:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.774+0000 7ff6637fe640 1 -- 172.21.15.38:0/1688911823 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff6680f05e0 con 0x7ff674102bc0 2024-05-21T15:34:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.774+0000 7ff672d9d640 1 --2- 172.21.15.38:0/1688911823 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff65c0a2c10 0x7ff65c0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.774+0000 7ff67259c640 1 --2- 172.21.15.38:0/2681880763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e009f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff66400bc50 tx=0x7ff66400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc0fdf70fc18fde server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.774+0000 7ff660ff9640 1 -- 172.21.15.38:0/2681880763 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664015070 con 0x55bae5a373b0 2024-05-21T15:34:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.774+0000 7ff660ff9640 1 -- 172.21.15.38:0/2681880763 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6640035f0 con 0x55bae5a373b0 2024-05-21T15:34:30.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.774+0000 7ff660ff9640 1 -- 172.21.15.38:0/2681880763 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6640116a0 con 0x55bae5a373b0 2024-05-21T15:34:30.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.775+0000 7ff67fdbd740 1 -- 172.21.15.38:0/2681880763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 msgr2=0x55bae5e009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.775+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/2681880763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e009f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff66400bc50 tx=0x7ff66400f730 comp rx=0 tx=0).stop 2024-05-21T15:34:30.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.775+0000 7ff672d9d640 1 --2- 172.21.15.38:0/1688911823 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff65c0a2c10 0x7ff65c0a5120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00b590 tx=0x7ff66c00b4e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.775+0000 7ff67fdbd740 1 -- 172.21.15.38:0/2681880763 shutdown_connections 2024-05-21T15:34:30.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.775+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/2681880763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a3f590 0x55bae5e078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.775+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/2681880763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e009f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.775+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/2681880763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a60270 0x55bae5e00470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.775+0000 7ff67fdbd740 1 -- 172.21.15.38:0/2681880763 >> 172.21.15.38:0/2681880763 conn(0x55bae5a543c0 msgr2=0x55bae5dfbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.775+0000 7ff67fdbd740 1 -- 172.21.15.38:0/2681880763 shutdown_connections 2024-05-21T15:34:30.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.775+0000 7ff67fdbd740 1 -- 172.21.15.38:0/2681880763 wait complete. 2024-05-21T15:34:30.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67fdbd740 1 -- start start 2024-05-21T15:34:30.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67fdbd740 1 Processor -- start 2024-05-21T15:34:30.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67fdbd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e0a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67fdbd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a3f590 0x55bae5e0a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67fdbd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a60270 0x55bae5e11760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67fdbd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bae5e08340 con 0x55bae5a373b0 2024-05-21T15:34:30.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67fdbd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bae5d1f8e0 con 0x55bae5a60270 2024-05-21T15:34:30.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67fdbd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bae5e084c0 con 0x55bae5a3f590 2024-05-21T15:34:30.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67fdbd740 10 client.0 _ll_register_callbacks cb 0x55bae5cec9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:30.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e0a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e0a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60186/0 (socket says 172.21.15.38:60186) 2024-05-21T15:34:30.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67259c640 1 -- 172.21.15.38:0/229385743 learned_addr learned my addr 172.21.15.38:0/229385743 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff672d9d640 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a60270 0x55bae5e11760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff671d9b640 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a3f590 0x55bae5e0a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67259c640 1 -- 172.21.15.38:0/229385743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a3f590 msgr2=0x55bae5e0a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67259c640 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a3f590 0x55bae5e0a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67259c640 1 -- 172.21.15.38:0/229385743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a60270 msgr2=0x55bae5e11760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.776+0000 7ff67259c640 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a60270 0x55bae5e11760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.777+0000 7ff67259c640 1 -- 172.21.15.38:0/229385743 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bae5b426b0 con 0x55bae5a373b0 2024-05-21T15:34:30.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.777+0000 7ff672d9d640 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a60270 0x55bae5e11760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:30.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.777+0000 7ff671d9b640 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a3f590 0x55bae5e0a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:30.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.777+0000 7ff67259c640 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e0a2c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff66400fc30 tx=0x7ff66400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56bcb151a173a5a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.777+0000 7ff659ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:30.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.777+0000 7ff659ffb640 1 -- 172.21.15.38:0/229385743 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664028060 con 0x55bae5a373b0 2024-05-21T15:34:30.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.777+0000 7ff659ffb640 1 -- 172.21.15.38:0/229385743 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff6640039b0 con 0x55bae5a373b0 2024-05-21T15:34:30.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.777+0000 7ff67fdbd740 1 -- 172.21.15.38:0/229385743 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6640035f0 con 0x55bae5a373b0 2024-05-21T15:34:30.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.777+0000 7ff659ffb640 1 -- 172.21.15.38:0/229385743 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff664015070 con 0x55bae5a373b0 2024-05-21T15:34:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.777+0000 7ff67fdbd740 10 client.15357 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.777+0000 7ff67fdbd740 1 -- 172.21.15.38:0/229385743 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bae5e14690 con 0x55bae5a373b0 2024-05-21T15:34:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.777+0000 7ff659ffb640 1 -- 172.21.15.38:0/229385743 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6640035f0 con 0x55bae5a373b0 2024-05-21T15:34:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.777+0000 7ff659ffb640 1 -- 172.21.15.38:0/229385743 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff664011aa0 con 0x55bae5a373b0 2024-05-21T15:34:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.777+0000 7ff67fdbd740 10 client.15357 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.778+0000 7ff67fdbd740 10 client.15357 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T15:34:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.778+0000 7ff67fdbd740 20 client.15357 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.778+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bae5a1ee40 0x55bae5e17d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.778+0000 7ff67fdbd740 4 client.15357 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:34:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.778+0000 7ff67fdbd740 1 -- 172.21.15.38:0/229385743 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bae5d71c00 con 0x55bae5a1ee40 2024-05-21T15:34:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.778+0000 7ff671d9b640 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bae5a1ee40 0x55bae5e17d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.778+0000 7ff671d9b640 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bae5a1ee40 0x55bae5e17d40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d7e58b80e1210b server_cookie=a4cab6a9de143fb8 in_seq=0 out_seq=0 2024-05-21T15:34:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.778+0000 7ff659ffb640 10 client.15357 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:30.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff659ffb640 1 -- 172.21.15.38:0/229385743 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bae5d71c00 con 0x55bae5a1ee40 2024-05-21T15:34:30.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff659ffb640 10 client.15357 handle_command_reply: tid=0 2024-05-21T15:34:30.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff67fdbd740 1 client.15357 shutdown 2024-05-21T15:34:30.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff67fdbd740 1 -- 172.21.15.38:0/229385743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 msgr2=0x55bae5e0a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e0a2c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff66400fc30 tx=0x7ff66400b9c0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff67fdbd740 1 -- 172.21.15.38:0/229385743 shutdown_connections 2024-05-21T15:34:30.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a60270 0x55bae5e11760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a3f590 0x55bae5e0a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bae5a1ee40 0x55bae5e17d40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/229385743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e0a2c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff67fdbd740 1 -- 172.21.15.38:0/229385743 >> 172.21.15.38:0/229385743 conn(0x55bae5a543c0 msgr2=0x55bae5dfaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff659ffb640 0 client.15357 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:30.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff67fdbd740 1 -- 172.21.15.38:0/229385743 shutdown_connections 2024-05-21T15:34:30.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff67fdbd740 1 -- 172.21.15.38:0/229385743 wait complete. 2024-05-21T15:34:30.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.779+0000 7ff67fdbd740 20 client.15357 trim_cache size 0 max 16384 2024-05-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67fdbd740 1 Processor -- start 2024-05-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67fdbd740 1 -- start start 2024-05-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67fdbd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e08fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67fdbd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a3f590 0x55bae5dfbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67fdbd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a60270 0x55bae5e11cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67fdbd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bae5e16080 con 0x55bae5a373b0 2024-05-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67fdbd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bae5e16750 con 0x55bae5a60270 2024-05-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67fdbd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bae5e16e20 con 0x55bae5a3f590 2024-05-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e08fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e08fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60196/0 (socket says 172.21.15.38:60196) 2024-05-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67259c640 1 -- 172.21.15.38:0/144054489 learned_addr learned my addr 172.21.15.38:0/144054489 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67259c640 1 -- 172.21.15.38:0/144054489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a3f590 msgr2=0x55bae5dfbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67259c640 1 --2- 172.21.15.38:0/144054489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a3f590 0x55bae5dfbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67259c640 1 -- 172.21.15.38:0/144054489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a60270 msgr2=0x55bae5e11cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67259c640 1 --2- 172.21.15.38:0/144054489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a60270 0x55bae5e11cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.798+0000 7ff67259c640 1 -- 172.21.15.38:0/144054489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bae5e0aec0 con 0x55bae5a373b0 2024-05-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.799+0000 7ff67259c640 1 --2- 172.21.15.38:0/144054489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e08fa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff664011e70 tx=0x7ff664005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d794117315d4e2f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.799+0000 7ff65bfff640 1 -- 172.21.15.38:0/144054489 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664028040 con 0x55bae5a373b0 2024-05-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.799+0000 7ff65bfff640 1 -- 172.21.15.38:0/144054489 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff66400bd40 con 0x55bae5a373b0 2024-05-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.799+0000 7ff65bfff640 1 -- 172.21.15.38:0/144054489 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664004100 con 0x55bae5a373b0 2024-05-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.799+0000 7ff67fdbd740 1 -- 172.21.15.38:0/144054489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 msgr2=0x55bae5e08fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.799+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/144054489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e08fa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff664011e70 tx=0x7ff664005f50 comp rx=0 tx=0).stop 2024-05-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.799+0000 7ff67fdbd740 1 -- 172.21.15.38:0/144054489 shutdown_connections 2024-05-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.799+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/144054489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a60270 0x55bae5e11cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.799+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/144054489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a3f590 0x55bae5dfbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.799+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/144054489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a373b0 0x55bae5e08fa0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.799+0000 7ff67fdbd740 1 -- 172.21.15.38:0/144054489 >> 172.21.15.38:0/144054489 conn(0x55bae5a543c0 msgr2=0x55bae5e08890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff67fdbd740 1 -- 172.21.15.38:0/144054489 shutdown_connections 2024-05-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff67fdbd740 1 -- 172.21.15.38:0/144054489 wait complete. 2024-05-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff67fdbd740 1 -- start start 2024-05-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff67fdbd740 1 Processor -- start 2024-05-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff67fdbd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a373b0 0x55bae5e0bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff67fdbd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a3f590 0x55bae5e0c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff67fdbd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a60270 0x55bae5e7ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff67fdbd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bae5e17bc0 con 0x55bae5a60270 2024-05-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff67fdbd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bae5e174f0 con 0x55bae5a3f590 2024-05-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff67fdbd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bae5e18290 con 0x55bae5a373b0 2024-05-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff67fdbd740 10 client.0 _ll_register_callbacks cb 0x7ff6680b9090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff672d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a60270 0x55bae5e7ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff672d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a60270 0x55bae5e7ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60200/0 (socket says 172.21.15.38:60200) 2024-05-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff672d9d640 1 -- 172.21.15.38:0/1710383236 learned_addr learned my addr 172.21.15.38:0/1710383236 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.800+0000 7ff67259c640 1 --2- 172.21.15.38:0/1710383236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a373b0 0x55bae5e0bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff672d9d640 1 -- 172.21.15.38:0/1710383236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a373b0 msgr2=0x55bae5e0bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff671d9b640 1 --2- 172.21.15.38:0/1710383236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a3f590 0x55bae5e0c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff672d9d640 1 --2- 172.21.15.38:0/1710383236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a373b0 0x55bae5e0bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff672d9d640 1 -- 172.21.15.38:0/1710383236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a3f590 msgr2=0x55bae5e0c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff672d9d640 1 --2- 172.21.15.38:0/1710383236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a3f590 0x55bae5e0c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff672d9d640 1 -- 172.21.15.38:0/1710383236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bae5e7f0f0 con 0x55bae5a60270 2024-05-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff672d9d640 1 --2- 172.21.15.38:0/1710383236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a60270 0x55bae5e7ebb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00bc70 tx=0x7ff66c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f8b76af7c42dcc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff659ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff659ffb640 1 -- 172.21.15.38:0/1710383236 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c01d690 con 0x55bae5a60270 2024-05-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff659ffb640 1 -- 172.21.15.38:0/1710383236 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff66c01d830 con 0x55bae5a60270 2024-05-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff67fdbd740 1 -- 172.21.15.38:0/1710383236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff66400bd40 con 0x55bae5a60270 2024-05-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff659ffb640 1 -- 172.21.15.38:0/1710383236 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff66c01da30 con 0x55bae5a60270 2024-05-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff67fdbd740 10 client.15363 fetch_fsmap learned FSMap version 26 2024-05-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff67fdbd740 1 -- 172.21.15.38:0/1710383236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bae5e803f0 con 0x55bae5a60270 2024-05-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff659ffb640 1 -- 172.21.15.38:0/1710383236 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff66400bd40 con 0x55bae5a60270 2024-05-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff659ffb640 1 -- 172.21.15.38:0/1710383236 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff66c027420 con 0x55bae5a60270 2024-05-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff67fdbd740 10 client.15363 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.801+0000 7ff67fdbd740 10 client.15363 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.802+0000 7ff67fdbd740 20 client.15363 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.802+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/1710383236 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bae59f4280 0x55bae5e83cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.802+0000 7ff67fdbd740 4 client.15363 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.802+0000 7ff67fdbd740 1 -- 172.21.15.38:0/1710383236 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "status"}) -- 0x55bae5d6bf50 con 0x55bae59f4280 2024-05-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.802+0000 7ff67259c640 1 --2- 172.21.15.38:0/1710383236 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bae59f4280 0x55bae5e83cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.802+0000 7ff67259c640 1 --2- 172.21.15.38:0/1710383236 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bae59f4280 0x55bae5e83cc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b958033d9df8ff server_cookie=c51c6c6705b36573 in_seq=0 out_seq=0 2024-05-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.802+0000 7ff659ffb640 10 client.15363 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.802+0000 7ff659ffb640 1 -- 172.21.15.38:0/1710383236 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55bae5d6bf50 con 0x55bae59f4280 2024-05-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.802+0000 7ff659ffb640 10 client.15363 handle_command_reply: tid=0 2024-05-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.802+0000 7ff67fdbd740 1 client.15363 shutdown 2024-05-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff67fdbd740 1 -- 172.21.15.38:0/1710383236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a60270 msgr2=0x55bae5e7ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/1710383236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a60270 0x55bae5e7ebb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00bc70 tx=0x7ff66c012820 comp rx=0 tx=0).stop 2024-05-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff67fdbd740 1 -- 172.21.15.38:0/1710383236 shutdown_connections 2024-05-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/1710383236 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bae59f4280 0x55bae5e83cc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/1710383236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae5a60270 0x55bae5e7ebb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff659ffb640 0 client.15363 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/1710383236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bae5a3f590 0x55bae5e0c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff67fdbd740 1 --2- 172.21.15.38:0/1710383236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae5a373b0 0x55bae5e0bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff67fdbd740 1 -- 172.21.15.38:0/1710383236 >> 172.21.15.38:0/1710383236 conn(0x55bae5a543c0 msgr2=0x55bae5e03680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff67fdbd740 1 -- 172.21.15.38:0/1710383236 shutdown_connections 2024-05-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff67fdbd740 1 -- 172.21.15.38:0/1710383236 wait complete. 2024-05-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff67fdbd740 20 client.15363 trim_cache size 0 max 16384 2024-05-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "102ec4fa-1786-11ef-bc9a-c7b262605968", 2024-05-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-05-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24602, 2024-05-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-05-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-05-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-05-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 116.592677916, 2024-05-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-05-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 90, 2024-05-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 86, 2024-05-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 121.742479093 2024-05-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff678e3f640 1 -- 172.21.15.38:0/1688911823 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff65c0a2c10 msgr2=0x7ff65c0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff678e3f640 1 --2- 172.21.15.38:0/1688911823 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff65c0a2c10 0x7ff65c0a5120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00b590 tx=0x7ff66c00b4e0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff678e3f640 1 -- 172.21.15.38:0/1688911823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff674102bc0 msgr2=0x7ff67419e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff678e3f640 1 --2- 172.21.15.38:0/1688911823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff674102bc0 0x7ff67419e060 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff668005e30 tx=0x7ff668031ee0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff678e3f640 1 -- 172.21.15.38:0/1688911823 shutdown_connections 2024-05-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff678e3f640 1 --2- 172.21.15.38:0/1688911823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6741078c0 0x7ff6741a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff678e3f640 1 --2- 172.21.15.38:0/1688911823 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff65c0a2c10 0x7ff65c0a5120 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff678e3f640 1 --2- 172.21.15.38:0/1688911823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff674102bc0 0x7ff67419e060 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff678e3f640 1 --2- 172.21.15.38:0/1688911823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff674101980 0x7ff6740699b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff678e3f640 1 -- 172.21.15.38:0/1688911823 >> 172.21.15.38:0/1688911823 conn(0x7ff6740fcb70 msgr2=0x7ff6741038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff678e3f640 1 -- 172.21.15.38:0/1688911823 shutdown_connections 2024-05-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:30.803+0000 7ff678e3f640 1 -- 172.21.15.38:0/1688911823 wait complete. 2024-05-21T15:34:30.814 INFO:teuthology.run_tasks:Running task sequential... 2024-05-21T15:34:30.824 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-21T15:34:30.824 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-21T15:34:30.824 INFO:teuthology.task.print:Enabling mgr modules 2024-05-21T15:34:30.824 INFO:teuthology.run_tasks:Running task kclient... 2024-05-21T15:34:30.964 INFO:tasks.kclient:Mounting kernel clients... 2024-05-21T15:34:30.964 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-21T15:34:30.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:34:30.965 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-05-21T15:34:30.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:34:30.982 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-05-21T15:34:31.051 INFO:teuthology.orchestra.run.smithi038.stderr:Cannot find device "ceph-brx" 2024-05-21T15:34:31.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:34:31.053 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-21T15:34:31.053 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-21T15:34:31.053 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-21T15:34:31.053 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-21T15:34:31.342 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:34:31.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:31 smithi053 ceph-mon[36496]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 11 B/s, 0 objects/s recovering 2024-05-21T15:34:31.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:31 smithi038 ceph-mon[31995]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 11 B/s, 0 objects/s recovering 2024-05-21T15:34:31.510 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:31 smithi045 ceph-mon[36409]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 11 B/s, 0 objects/s recovering 2024-05-21T15:34:33.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.258+0000 7f4daf03a700 1 -- 172.21.15.38:0/2868047687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da81075c0 msgr2=0x7f4da810ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:33.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.258+0000 7f4daf03a700 1 --2- 172.21.15.38:0/2868047687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da81075c0 0x7f4da810ef20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4da000b0d0 tx=0x7f4da000b3e0 comp rx=0 tx=0).stop 2024-05-21T15:34:33.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.258+0000 7f4daf03a700 1 -- 172.21.15.38:0/2868047687 shutdown_connections 2024-05-21T15:34:33.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.258+0000 7f4daf03a700 1 --2- 172.21.15.38:0/2868047687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da81075c0 0x7f4da810ef20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:33.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.258+0000 7f4daf03a700 1 --2- 172.21.15.38:0/2868047687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da8102340 0x7f4da8102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:33.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.258+0000 7f4daf03a700 1 --2- 172.21.15.38:0/2868047687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4da8106c70 0x7f4da8107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:33.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.258+0000 7f4daf03a700 1 -- 172.21.15.38:0/2868047687 >> 172.21.15.38:0/2868047687 conn(0x7f4da80fdc40 msgr2=0x7f4da8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:33.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.258+0000 7f4daf03a700 1 -- 172.21.15.38:0/2868047687 shutdown_connections 2024-05-21T15:34:33.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.258+0000 7f4daf03a700 1 -- 172.21.15.38:0/2868047687 wait complete. 2024-05-21T15:34:33.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.259+0000 7f4daf03a700 1 Processor -- start 2024-05-21T15:34:33.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.259+0000 7f4daf03a700 1 -- start start 2024-05-21T15:34:33.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.259+0000 7f4daf03a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4da8102340 0x7f4da819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:33.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4daf03a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8106c70 0x7f4da819b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:33.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4daf03a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da81075c0 0x7f4da819b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:33.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4daf03a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4da8112340 con 0x7f4da8106c70 2024-05-21T15:34:33.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4daf03a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4da81121c0 con 0x7f4da8102340 2024-05-21T15:34:33.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4daf03a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4da81124c0 con 0x7f4da81075c0 2024-05-21T15:34:33.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4d9ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8106c70 0x7f4da819b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:33.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4d9ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8106c70 0x7f4da819b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60222/0 (socket says 172.21.15.38:60222) 2024-05-21T15:34:33.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4d9ffff700 1 -- 172.21.15.38:0/204320553 learned_addr learned my addr 172.21.15.38:0/204320553 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:33.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4dad5d7700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da81075c0 0x7f4da819b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:33.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4dacdd6700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4da8102340 0x7f4da819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:33.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4d9ffff700 1 -- 172.21.15.38:0/204320553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da81075c0 msgr2=0x7f4da819b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:33.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4d9ffff700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da81075c0 0x7f4da819b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:33.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4d9ffff700 1 -- 172.21.15.38:0/204320553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4da8102340 msgr2=0x7f4da819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:33.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4d9ffff700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4da8102340 0x7f4da819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:33.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4d9ffff700 1 -- 172.21.15.38:0/204320553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da81a5e70 con 0x7f4da8106c70 2024-05-21T15:34:33.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4dacdd6700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4da8102340 0x7f4da819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:33.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.260+0000 7f4d9ffff700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8106c70 0x7f4da819b040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800d980 tx=0x7f4d9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:33.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.261+0000 7f4d9dffb700 1 -- 172.21.15.38:0/204320553 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98017070 con 0x7f4da8106c70 2024-05-21T15:34:33.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.261+0000 7f4d9dffb700 1 -- 172.21.15.38:0/204320553 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d98009e10 con 0x7f4da8106c70 2024-05-21T15:34:33.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.261+0000 7f4d9dffb700 1 -- 172.21.15.38:0/204320553 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800f620 con 0x7f4da8106c70 2024-05-21T15:34:33.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.261+0000 7f4daf03a700 1 -- 172.21.15.38:0/204320553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da81a6160 con 0x7f4da8106c70 2024-05-21T15:34:33.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.261+0000 7f4daf03a700 1 -- 172.21.15.38:0/204320553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da81a6650 con 0x7f4da8106c70 2024-05-21T15:34:33.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.262+0000 7f4daf03a700 1 -- 172.21.15.38:0/204320553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da8050170 con 0x7f4da8106c70 2024-05-21T15:34:33.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.263+0000 7f4d9dffb700 1 -- 172.21.15.38:0/204320553 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d98010460 con 0x7f4da8106c70 2024-05-21T15:34:33.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.264+0000 7f4d9dffb700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4d900a2c20 0x7f4d900a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:33.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.265+0000 7f4dacdd6700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4d900a2c20 0x7f4d900a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:33.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.265+0000 7f4d9dffb700 1 -- 172.21.15.38:0/204320553 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f4d980ceb10 con 0x7f4da8106c70 2024-05-21T15:34:33.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.265+0000 7f4dacdd6700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4d900a2c20 0x7f4d900a5130 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009cb0 tx=0x7f4da4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:33.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.268+0000 7f4d9dffb700 1 -- 172.21.15.38:0/204320553 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4d9809ba20 con 0x7f4da8106c70 2024-05-21T15:34:33.426 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:33 smithi038 ceph-mon[31995]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-05-21T15:34:33.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.525+0000 7f4daf03a700 1 -- 172.21.15.38:0/204320553 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4da802dc60 con 0x7f4da8106c70 2024-05-21T15:34:33.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.525+0000 7f4d9dffb700 1 -- 172.21.15.38:0/204320553 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f4d9809b320 con 0x7f4da8106c70 2024-05-21T15:34:33.525 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-21T15:34:33.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.528+0000 7f4daf03a700 1 -- 172.21.15.38:0/204320553 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4d900a2c20 msgr2=0x7f4d900a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:33.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.528+0000 7f4daf03a700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4d900a2c20 0x7f4d900a5130 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009cb0 tx=0x7f4da4009260 comp rx=0 tx=0).stop 2024-05-21T15:34:33.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.528+0000 7f4daf03a700 1 -- 172.21.15.38:0/204320553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8106c70 msgr2=0x7f4da819b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:33.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.528+0000 7f4daf03a700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8106c70 0x7f4da819b040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800d980 tx=0x7f4d9800dc90 comp rx=0 tx=0).stop 2024-05-21T15:34:33.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.528+0000 7f4daf03a700 1 -- 172.21.15.38:0/204320553 shutdown_connections 2024-05-21T15:34:33.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.528+0000 7f4daf03a700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4d900a2c20 0x7f4d900a5130 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:33.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.528+0000 7f4daf03a700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da81075c0 0x7f4da819b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:33.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.528+0000 7f4daf03a700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4da8102340 0x7f4da819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:33.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.528+0000 7f4daf03a700 1 --2- 172.21.15.38:0/204320553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da8106c70 0x7f4da819b040 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:33.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.528+0000 7f4daf03a700 1 -- 172.21.15.38:0/204320553 >> 172.21.15.38:0/204320553 conn(0x7f4da80fdc40 msgr2=0x7f4da8106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:33.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.528+0000 7f4daf03a700 1 -- 172.21.15.38:0/204320553 shutdown_connections 2024-05-21T15:34:33.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:33.528+0000 7f4daf03a700 1 -- 172.21.15.38:0/204320553 wait complete. 2024-05-21T15:34:33.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:33 smithi053 ceph-mon[36496]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-05-21T15:34:33.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:33 smithi045 ceph-mon[36409]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-05-21T15:34:34.298 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-21T15:34:34.298 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-21T15:34:34.298 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-05-21T15:34:34.298 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-21T15:34:34.298 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-21T15:34:34.298 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-21T15:34:34.299 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-21T15:34:34.299 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-21T15:34:34.299 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-21T15:34:34.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:34:34.299 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-05-21T15:34:34.318 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-21T15:34:34.318 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-21T15:34:34.318 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-05-21T15:34:34.318 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-05-21T15:34:34.319 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-05-21T15:34:34.319 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-05-21T15:34:34.319 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-21T15:34:34.319 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-05-21T15:34:34.319 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-05-21T15:34:34.319 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-21T15:34:34.319 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-05-21T15:34:34.319 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-05-21T15:34:34.319 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-21T15:34:34.319 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-05-21T15:34:34.319 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-05-21T15:34:34.319 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-21T15:34:34.319 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42117sec preferred_lft 42117sec 2024-05-21T15:34:34.319 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-05-21T15:34:34.320 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-05-21T15:34:34.320 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-21T15:34:34.320 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-05-21T15:34:34.320 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-05-21T15:34:34.320 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-21T15:34:34.320 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T15:34:34.321 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-05-21T15:34:34.321 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add name ceph-brx type bridge 2024-05-21T15:34:34.321 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr flush dev ceph-brx 2024-05-21T15:34:34.321 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set ceph-brx up 2024-05-21T15:34:34.321 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-21T15:34:34.321 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-05-21T15:34:34.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T15:34:34.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:34:34.519 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:34 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/204320553' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-21T15:34:34.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:34:34.525 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-21T15:34:34.552 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-05-21T15:34:34.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:34:34.554 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-05-21T15:34:34.630 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-05-21T15:34:34.630 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-21T15:34:34.630 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-21T15:34:34.630 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-21T15:34:34.630 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-21T15:34:34.630 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T15:34:34.630 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-05-21T15:34:34.631 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-21T15:34:34.631 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-21T15:34:34.631 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-21T15:34:34.631 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-05-21T15:34:34.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:34 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/204320553' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-21T15:34:34.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T15:34:34.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:34 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/204320553' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-21T15:34:34.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:34:34.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:34:34.945 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-05-21T15:34:34.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:34:34.962 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-05-21T15:34:35.017 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T15:34:35.018 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-05-21T15:34:35.018 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-21T15:34:35.018 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-21T15:34:35.018 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-05-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:34:35.138 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-21T15:34:35.139 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T15:34:35.139 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-05-21T15:34:35.139 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-21T15:34:35.139 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-21T15:34:35.139 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-21T15:34:35.139 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-21T15:34:35.139 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-21T15:34:35.139 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-05-21T15:34:35.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T15:34:35.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:34:35.270 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T15:34:35.270 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-05-21T15:34:35.270 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-05-21T15:34:35.270 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.0 master ceph-brx 2024-05-21T15:34:35.270 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-05-21T15:34:35.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T15:34:35.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:34:35.351 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-21T15:34:35.351 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-21T15:34:35.367 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-21T15:34:35.367 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-21T15:34:35.367 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-21T15:34:35.423 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-21T15:34:35.423 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-21T15:34:35.423 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/52b7ee44-44a1-4389-8066-cc7f06d74d34 2024-05-21T15:34:35.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:34:35.423 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/52b7ee44-44a1-4389-8066-cc7f06d74d34 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-21T15:34:35.486 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:35 smithi038 ceph-mon[31995]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 8 B/s, 0 objects/s recovering 2024-05-21T15:34:35.535 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-21T15:34:35.535 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-05-21T15:34:35.664 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-21T15:34:35.664 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-05-21T15:34:35.665 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.45:3300,172.21.15.53:3300" 2024-05-21T15:34:35.665 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with new device syntax: 0@102ec4fa-1786-11ef-bc9a-c7b262605968.cephfs=/volumes/qa/sv_1/52b7ee44-44a1-4389-8066-cc7f06d74d34 2024-05-21T15:34:35.665 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.38:3300/172.21.15.45:3300/172.21.15.53:3300" will pass to kernel 2024-05-21T15:34:35.666 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-21T15:34:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-05-21T15:34:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-21T15:34:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-05-21T15:34:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-21T15:34:35.667 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-21T15:34:35.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:34:35.668 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-21T15:34:35.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:35 smithi053 ceph-mon[36496]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 8 B/s, 0 objects/s recovering 2024-05-21T15:34:35.735 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-21T15:34:35.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:35 smithi045 ceph-mon[36409]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 8 B/s, 0 objects/s recovering 2024-05-21T15:34:35.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-21T15:34:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:34:35.835 INFO:teuthology.orchestra.run.smithi038.stdout:client25139 2024-05-21T15:34:35.835 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:34:35.835 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-21T15:34:35.852 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-21T15:34:35.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-21T15:34:35.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:34:35.951 INFO:teuthology.orchestra.run.smithi038.stdout:client25139 2024-05-21T15:34:35.951 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-21T15:34:36.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-21T15:34:36.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:34:36.010 INFO:teuthology.orchestra.run.smithi038.stdout:client25139 2024-05-21T15:34:36.010 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968.client25139/status') 2024-05-21T15:34:36.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T15:34:36.071 INFO:teuthology.orchestra.run.smithi038.stdout:instance: client.25139 (3)172.21.15.38:0/4172329064 2024-05-21T15:34:36.071 INFO:teuthology.orchestra.run.smithi038.stdout:blocklisted: false 2024-05-21T15:34:36.072 INFO:teuthology.orchestra.run.smithi038.stderr:0+1 records in 2024-05-21T15:34:36.072 INFO:teuthology.orchestra.run.smithi038.stderr:0+1 records out 2024-05-21T15:34:36.072 INFO:teuthology.orchestra.run.smithi038.stderr:71 bytes copied, 2.475e-05 s, 2.9 MB/s 2024-05-21T15:34:36.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:34:36.079 INFO:teuthology.run_tasks:Running task exec... 2024-05-21T15:34:36.089 INFO:teuthology.task.exec:Executing custom commands... 2024-05-21T15:34:36.089 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-05-21T15:34:36.089 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-21T15:34:36.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.205+0000 7f28f7b23640 1 -- 172.21.15.38:0/3468146259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f0103d20 msgr2=0x7f28f0104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:36.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.205+0000 7f28f7b23640 1 --2- 172.21.15.38:0/3468146259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f0103d20 0x7f28f0104130 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009900 tx=0x7f28e002a410 comp rx=0 tx=0).stop 2024-05-21T15:34:36.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.205+0000 7f28f7b23640 1 -- 172.21.15.38:0/3468146259 shutdown_connections 2024-05-21T15:34:36.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.205+0000 7f28f7b23640 1 --2- 172.21.15.38:0/3468146259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28f0108a20 0x7f28f010ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:36.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.205+0000 7f28f7b23640 1 --2- 172.21.15.38:0/3468146259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f0103d20 0x7f28f0104130 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:36.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.205+0000 7f28f7b23640 1 --2- 172.21.15.38:0/3468146259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f0102ae0 0x7f28f01037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:36.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.205+0000 7f28f7b23640 1 -- 172.21.15.38:0/3468146259 >> 172.21.15.38:0/3468146259 conn(0x7f28f00fdcb0 msgr2=0x7f28f0100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:36.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.206+0000 7f28f7b23640 1 -- 172.21.15.38:0/3468146259 shutdown_connections 2024-05-21T15:34:36.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.206+0000 7f28f7b23640 1 -- 172.21.15.38:0/3468146259 wait complete. 2024-05-21T15:34:36.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.206+0000 7f28f7b23640 1 Processor -- start 2024-05-21T15:34:36.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.206+0000 7f28f7b23640 1 -- start start 2024-05-21T15:34:36.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.206+0000 7f28f7b23640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f0102ae0 0x7f28f0199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:36.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.206+0000 7f28f7b23640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28f0103d20 0x7f28f0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:36.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.206+0000 7f28f7b23640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f0108a20 0x7f28f01a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:36.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.206+0000 7f28f7b23640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f28f010e730 con 0x7f28f0108a20 2024-05-21T15:34:36.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.206+0000 7f28f7b23640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28f010e5b0 con 0x7f28f0102ae0 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.206+0000 7f28f7b23640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f28f010e8b0 con 0x7f28f0103d20 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28f60c0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f0108a20 0x7f28f01a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28f60c0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f0108a20 0x7f28f01a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59734/0 (socket says 172.21.15.38:59734) 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28f60c0640 1 -- 172.21.15.38:0/1874151176 learned_addr learned my addr 172.21.15.38:0/1874151176 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28f50be640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28f0103d20 0x7f28f0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28f58bf640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f0102ae0 0x7f28f0199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28f60c0640 1 -- 172.21.15.38:0/1874151176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28f0103d20 msgr2=0x7f28f0199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28f60c0640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28f0103d20 0x7f28f0199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28f60c0640 1 -- 172.21.15.38:0/1874151176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f0102ae0 msgr2=0x7f28f0199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28f60c0640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f0102ae0 0x7f28f0199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28f60c0640 1 -- 172.21.15.38:0/1874151176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f01a0c90 con 0x7f28f0108a20 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28f50be640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28f0103d20 0x7f28f0199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28f60c0640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f0108a20 0x7f28f01a06d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f28e800cce0 tx=0x7f28e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28e6ffd640 1 -- 172.21.15.38:0/1874151176 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e800f450 con 0x7f28f0108a20 2024-05-21T15:34:36.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28e6ffd640 1 -- 172.21.15.38:0/1874151176 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e800fad0 con 0x7f28f0108a20 2024-05-21T15:34:36.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28f7b23640 1 -- 172.21.15.38:0/1874151176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f01a0f80 con 0x7f28f0108a20 2024-05-21T15:34:36.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28e6ffd640 1 -- 172.21.15.38:0/1874151176 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8018c10 con 0x7f28f0108a20 2024-05-21T15:34:36.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.207+0000 7f28f7b23640 1 -- 172.21.15.38:0/1874151176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f01a1390 con 0x7f28f0108a20 2024-05-21T15:34:36.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.208+0000 7f28f7b23640 1 -- 172.21.15.38:0/1874151176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f004f750 con 0x7f28f0108a20 2024-05-21T15:34:36.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.210+0000 7f28e6ffd640 1 -- 172.21.15.38:0/1874151176 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f28e800f5f0 con 0x7f28f0108a20 2024-05-21T15:34:36.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.211+0000 7f28e6ffd640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28dc0a2c30 0x7f28dc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:36.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.211+0000 7f28f58bf640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28dc0a2c30 0x7f28dc0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:36.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.211+0000 7f28e6ffd640 1 -- 172.21.15.38:0/1874151176 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f28e80cf890 con 0x7f28f0108a20 2024-05-21T15:34:36.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.211+0000 7f28f58bf640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28dc0a2c30 0x7f28dc0a5140 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f28ec005db0 tx=0x7f28ec005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:36.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.215+0000 7f28e6ffd640 1 -- 172.21.15.38:0/1874151176 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f28f004f750 con 0x7f28f0108a20 2024-05-21T15:34:36.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:36.439+0000 7f28f7b23640 1 -- 172.21.15.38:0/1874151176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f28f002d360 con 0x7f28f0108a20 2024-05-21T15:34:37.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.332+0000 7f28e6ffd640 1 -- 172.21.15.38:0/1874151176 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f28e8014030 con 0x7f28f0108a20 2024-05-21T15:34:37.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.335+0000 7f28f7b23640 1 -- 172.21.15.38:0/1874151176 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28dc0a2c30 msgr2=0x7f28dc0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:37.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.335+0000 7f28f7b23640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28dc0a2c30 0x7f28dc0a5140 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f28ec005db0 tx=0x7f28ec005d40 comp rx=0 tx=0).stop 2024-05-21T15:34:37.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.335+0000 7f28f7b23640 1 -- 172.21.15.38:0/1874151176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f0108a20 msgr2=0x7f28f01a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:37.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.335+0000 7f28f7b23640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f0108a20 0x7f28f01a06d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f28e800cce0 tx=0x7f28e800a620 comp rx=0 tx=0).stop 2024-05-21T15:34:37.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.335+0000 7f28f7b23640 1 -- 172.21.15.38:0/1874151176 shutdown_connections 2024-05-21T15:34:37.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.335+0000 7f28f7b23640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28dc0a2c30 0x7f28dc0a5140 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.335+0000 7f28f7b23640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28f0108a20 0x7f28f01a06d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.335+0000 7f28f7b23640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28f0103d20 0x7f28f0199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.335+0000 7f28f7b23640 1 --2- 172.21.15.38:0/1874151176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f0102ae0 0x7f28f0199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.335+0000 7f28f7b23640 1 -- 172.21.15.38:0/1874151176 >> 172.21.15.38:0/1874151176 conn(0x7f28f00fdcb0 msgr2=0x7f28f0104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:37.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.335+0000 7f28f7b23640 1 -- 172.21.15.38:0/1874151176 shutdown_connections 2024-05-21T15:34:37.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.335+0000 7f28f7b23640 1 -- 172.21.15.38:0/1874151176 wait complete. 2024-05-21T15:34:37.348 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-21T15:34:37.359 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-21T15:34:37.472 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:34:37.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.601+0000 7f81b41e3640 1 -- 172.21.15.38:0/1919806511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81ac1035a0 msgr2=0x7f81ac1059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:37.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.601+0000 7f81b41e3640 1 --2- 172.21.15.38:0/1919806511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81ac1035a0 0x7f81ac1059c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009ca0 tx=0x7f81a802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.601+0000 7f81b41e3640 1 -- 172.21.15.38:0/1919806511 shutdown_connections 2024-05-21T15:34:37.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.601+0000 7f81b41e3640 1 --2- 172.21.15.38:0/1919806511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81ac108860 0x7f81ac10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.601+0000 7f81b41e3640 1 --2- 172.21.15.38:0/1919806511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81ac105f00 0x7f81ac108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.601+0000 7f81b41e3640 1 --2- 172.21.15.38:0/1919806511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81ac1035a0 0x7f81ac1059c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.601+0000 7f81b41e3640 1 -- 172.21.15.38:0/1919806511 >> 172.21.15.38:0/1919806511 conn(0x7f81ac0fcba0 msgr2=0x7f81ac0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:37.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.601+0000 7f81b41e3640 1 -- 172.21.15.38:0/1919806511 shutdown_connections 2024-05-21T15:34:37.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.601+0000 7f81b41e3640 1 -- 172.21.15.38:0/1919806511 wait complete. 2024-05-21T15:34:37.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.602+0000 7f81b41e3640 1 Processor -- start 2024-05-21T15:34:37.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.602+0000 7f81b41e3640 1 -- start start 2024-05-21T15:34:37.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.602+0000 7f81b41e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81ac1035a0 0x7f81ac199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:37.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.602+0000 7f81b41e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81ac105f00 0x7f81ac199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:37.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.602+0000 7f81b41e3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81ac108860 0x7f81ac1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:37.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.602+0000 7f81b41e3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac10e700 con 0x7f81ac105f00 2024-05-21T15:34:37.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.602+0000 7f81b41e3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac10e580 con 0x7f81ac1035a0 2024-05-21T15:34:37.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.602+0000 7f81b41e3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac10e880 con 0x7f81ac108860 2024-05-21T15:34:37.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.602+0000 7f81b177e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81ac105f00 0x7f81ac199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:37.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.602+0000 7f81b177e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81ac105f00 0x7f81ac199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59768/0 (socket says 172.21.15.38:59768) 2024-05-21T15:34:37.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.602+0000 7f81b177e640 1 -- 172.21.15.38:0/63268389 learned_addr learned my addr 172.21.15.38:0/63268389 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:37.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.603+0000 7f81b1f7f640 1 --2- 172.21.15.38:0/63268389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81ac1035a0 0x7f81ac199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:37.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.603+0000 7f81b2780640 1 --2- 172.21.15.38:0/63268389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81ac108860 0x7f81ac1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:37.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.603+0000 7f81b177e640 1 -- 172.21.15.38:0/63268389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81ac108860 msgr2=0x7f81ac1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:37.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.603+0000 7f81b177e640 1 --2- 172.21.15.38:0/63268389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81ac108860 0x7f81ac1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.603+0000 7f81b177e640 1 -- 172.21.15.38:0/63268389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81ac1035a0 msgr2=0x7f81ac199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.603+0000 7f81b177e640 1 --2- 172.21.15.38:0/63268389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81ac1035a0 0x7f81ac199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.603+0000 7f81b177e640 1 -- 172.21.15.38:0/63268389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ac1a0d50 con 0x7f81ac105f00 2024-05-21T15:34:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.603+0000 7f81b177e640 1 --2- 172.21.15.38:0/63268389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81ac105f00 0x7f81ac199750 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f819c00e9f0 tx=0x7f819c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.603+0000 7f81a2ffd640 1 -- 172.21.15.38:0/63268389 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c00cd80 con 0x7f81ac105f00 2024-05-21T15:34:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.603+0000 7f81a2ffd640 1 -- 172.21.15.38:0/63268389 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819c005500 con 0x7f81ac105f00 2024-05-21T15:34:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.603+0000 7f81b41e3640 1 -- 172.21.15.38:0/63268389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ac1a0fe0 con 0x7f81ac105f00 2024-05-21T15:34:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.603+0000 7f81a2ffd640 1 -- 172.21.15.38:0/63268389 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c010690 con 0x7f81ac105f00 2024-05-21T15:34:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.603+0000 7f81b41e3640 1 -- 172.21.15.38:0/63268389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ac1a1480 con 0x7f81ac105f00 2024-05-21T15:34:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.604+0000 7f81b41e3640 1 -- 172.21.15.38:0/63268389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81ac0fe6c0 con 0x7f81ac105f00 2024-05-21T15:34:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.606+0000 7f81a2ffd640 1 -- 172.21.15.38:0/63268389 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f819c0046c0 con 0x7f81ac105f00 2024-05-21T15:34:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.606+0000 7f81a2ffd640 1 --2- 172.21.15.38:0/63268389 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81980a2c30 0x7f81980a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.607+0000 7f81a2ffd640 1 -- 172.21.15.38:0/63268389 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f819c0cf840 con 0x7f81ac105f00 2024-05-21T15:34:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.607+0000 7f81b1f7f640 1 --2- 172.21.15.38:0/63268389 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81980a2c30 0x7f81980a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:37.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.607+0000 7f81b1f7f640 1 --2- 172.21.15.38:0/63268389 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81980a2c30 0x7f81980a5140 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f81a8031c10 tx=0x7f81a8000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:37.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.610+0000 7f81a2ffd640 1 -- 172.21.15.38:0/63268389 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f819c053850 con 0x7f81ac105f00 2024-05-21T15:34:37.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:37 smithi053 ceph-mon[36496]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 7 B/s, 0 objects/s recovering 2024-05-21T15:34:37.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:37 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1874151176' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-21T15:34:37.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:37 smithi045 ceph-mon[36409]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 7 B/s, 0 objects/s recovering 2024-05-21T15:34:37.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:37 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1874151176' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-21T15:34:37.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:37 smithi038 ceph-mon[31995]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 7 B/s, 0 objects/s recovering 2024-05-21T15:34:37.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:37 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1874151176' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-21T15:34:37.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.799+0000 7f81b41e3640 1 -- 172.21.15.38:0/63268389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81ac02d470 con 0x7f81ac105f00 2024-05-21T15:34:37.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.800+0000 7f81a2ffd640 1 -- 172.21.15.38:0/63268389 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22704 (secure 0 0 0) 0x7f819c01f340 con 0x7f81ac105f00 2024-05-21T15:34:37.800 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:37.801 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":90,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:34:23.996759+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:34:37.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.803+0000 7f81b41e3640 1 -- 172.21.15.38:0/63268389 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81980a2c30 msgr2=0x7f81980a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:37.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.803+0000 7f81b41e3640 1 --2- 172.21.15.38:0/63268389 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81980a2c30 0x7f81980a5140 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f81a8031c10 tx=0x7f81a8000f40 comp rx=0 tx=0).stop 2024-05-21T15:34:37.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.803+0000 7f81b41e3640 1 -- 172.21.15.38:0/63268389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81ac105f00 msgr2=0x7f81ac199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:37.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.803+0000 7f81b41e3640 1 --2- 172.21.15.38:0/63268389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81ac105f00 0x7f81ac199750 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f819c00e9f0 tx=0x7f819c00edf0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.803+0000 7f81b41e3640 1 -- 172.21.15.38:0/63268389 shutdown_connections 2024-05-21T15:34:37.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.803+0000 7f81b41e3640 1 --2- 172.21.15.38:0/63268389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81ac108860 0x7f81ac1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.803+0000 7f81b41e3640 1 --2- 172.21.15.38:0/63268389 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f81980a2c30 0x7f81980a5140 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.803+0000 7f81b41e3640 1 --2- 172.21.15.38:0/63268389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81ac105f00 0x7f81ac199750 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.803+0000 7f81b41e3640 1 --2- 172.21.15.38:0/63268389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81ac1035a0 0x7f81ac199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.803+0000 7f81b41e3640 1 -- 172.21.15.38:0/63268389 >> 172.21.15.38:0/63268389 conn(0x7f81ac0fcba0 msgr2=0x7f81ac104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:37.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.803+0000 7f81b41e3640 1 -- 172.21.15.38:0/63268389 shutdown_connections 2024-05-21T15:34:37.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.803+0000 7f81b41e3640 1 -- 172.21.15.38:0/63268389 wait complete. 2024-05-21T15:34:37.814 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:27:50.659367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:32:31.645647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:32:32.955451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:34:37.814 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:34:37.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.932+0000 7f374d1e0640 1 -- 172.21.15.38:0/2677337193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3748101990 msgr2=0x7f3748102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:37.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.932+0000 7f374d1e0640 1 --2- 172.21.15.38:0/2677337193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3748101990 0x7f3748102690 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3738009900 tx=0x7f373802a410 comp rx=0 tx=0).stop 2024-05-21T15:34:37.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.932+0000 7f374d1e0640 1 -- 172.21.15.38:0/2677337193 shutdown_connections 2024-05-21T15:34:37.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.932+0000 7f374d1e0640 1 --2- 172.21.15.38:0/2677337193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37481078d0 0x7f3748109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.932+0000 7f374d1e0640 1 --2- 172.21.15.38:0/2677337193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3748102bd0 0x7f3748102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.932+0000 7f374d1e0640 1 --2- 172.21.15.38:0/2677337193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3748101990 0x7f3748102690 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.932+0000 7f374d1e0640 1 -- 172.21.15.38:0/2677337193 >> 172.21.15.38:0/2677337193 conn(0x7f37480fcba0 msgr2=0x7f37480ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:37.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.933+0000 7f374d1e0640 1 -- 172.21.15.38:0/2677337193 shutdown_connections 2024-05-21T15:34:37.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.933+0000 7f374d1e0640 1 -- 172.21.15.38:0/2677337193 wait complete. 2024-05-21T15:34:37.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.933+0000 7f374d1e0640 1 Processor -- start 2024-05-21T15:34:37.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.933+0000 7f374d1e0640 1 -- start start 2024-05-21T15:34:37.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374d1e0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3748101990 0x7f37481992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:37.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374d1e0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3748102bd0 0x7f37481997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:37.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374d1e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37481078d0 0x7f37481a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374d1e0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f374810d630 con 0x7f37481078d0 2024-05-21T15:34:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374d1e0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f374810d4b0 con 0x7f3748101990 2024-05-21T15:34:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374d1e0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f374810d7b0 con 0x7f3748102bd0 2024-05-21T15:34:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37481078d0 0x7f37481a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37481078d0 0x7f37481a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59782/0 (socket says 172.21.15.38:59782) 2024-05-21T15:34:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374759e640 1 -- 172.21.15.38:0/2688275235 learned_addr learned my addr 172.21.15.38:0/2688275235 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f3746d9d640 1 --2- 172.21.15.38:0/2688275235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3748101990 0x7f37481992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374659c640 1 --2- 172.21.15.38:0/2688275235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3748102bd0 0x7f37481997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374759e640 1 -- 172.21.15.38:0/2688275235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3748102bd0 msgr2=0x7f37481997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374759e640 1 --2- 172.21.15.38:0/2688275235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3748102bd0 0x7f37481997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374759e640 1 -- 172.21.15.38:0/2688275235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3748101990 msgr2=0x7f37481992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374759e640 1 --2- 172.21.15.38:0/2688275235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3748101990 0x7f37481992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374759e640 1 -- 172.21.15.38:0/2688275235 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37481a0de0 con 0x7f37481078d0 2024-05-21T15:34:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.934+0000 7f374759e640 1 --2- 172.21.15.38:0/2688275235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37481078d0 0x7f37481a0770 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f374000eaa0 tx=0x7f374000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.935+0000 7f372ffff640 1 -- 172.21.15.38:0/2688275235 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374000cdb0 con 0x7f37481078d0 2024-05-21T15:34:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.935+0000 7f374d1e0640 1 -- 172.21.15.38:0/2688275235 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37481a10d0 con 0x7f37481078d0 2024-05-21T15:34:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.935+0000 7f372ffff640 1 -- 172.21.15.38:0/2688275235 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3740005020 con 0x7f37481078d0 2024-05-21T15:34:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.935+0000 7f374d1e0640 1 -- 172.21.15.38:0/2688275235 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37481a1670 con 0x7f37481078d0 2024-05-21T15:34:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.935+0000 7f372ffff640 1 -- 172.21.15.38:0/2688275235 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374001f060 con 0x7f37481078d0 2024-05-21T15:34:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.936+0000 7f374d1e0640 1 -- 172.21.15.38:0/2688275235 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37481a1930 con 0x7f37481078d0 2024-05-21T15:34:37.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.937+0000 7f372ffff640 1 -- 172.21.15.38:0/2688275235 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f37400054d0 con 0x7f37481078d0 2024-05-21T15:34:37.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.938+0000 7f372ffff640 1 --2- 172.21.15.38:0/2688275235 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f37300a2b90 0x7f37300a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:37.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.938+0000 7f372ffff640 1 -- 172.21.15.38:0/2688275235 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f37400cf540 con 0x7f37481078d0 2024-05-21T15:34:37.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.938+0000 7f3746d9d640 1 --2- 172.21.15.38:0/2688275235 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f37300a2b90 0x7f37300a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:37.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.938+0000 7f3746d9d640 1 --2- 172.21.15.38:0/2688275235 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f37300a2b90 0x7f37300a50a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f373802aa00 tx=0x7f37380315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:37.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:37.941+0000 7f372ffff640 1 -- 172.21.15.38:0/2688275235 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3740016030 con 0x7f37481078d0 2024-05-21T15:34:38.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.126+0000 7f374d1e0640 1 -- 172.21.15.38:0/2688275235 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f37481a1b00 con 0x7f37481078d0 2024-05-21T15:34:38.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.127+0000 7f372ffff640 1 -- 172.21.15.38:0/2688275235 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f374009be00 con 0x7f37481078d0 2024-05-21T15:34:38.126 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-05-21T15:34:38.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.129+0000 7f374d1e0640 1 -- 172.21.15.38:0/2688275235 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f37300a2b90 msgr2=0x7f37300a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.129+0000 7f374d1e0640 1 --2- 172.21.15.38:0/2688275235 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f37300a2b90 0x7f37300a50a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f373802aa00 tx=0x7f37380315b0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.129+0000 7f374d1e0640 1 -- 172.21.15.38:0/2688275235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37481078d0 msgr2=0x7f37481a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.129+0000 7f374d1e0640 1 --2- 172.21.15.38:0/2688275235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37481078d0 0x7f37481a0770 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f374000eaa0 tx=0x7f374000ef60 comp rx=0 tx=0).stop 2024-05-21T15:34:38.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.129+0000 7f374d1e0640 1 -- 172.21.15.38:0/2688275235 shutdown_connections 2024-05-21T15:34:38.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.129+0000 7f374d1e0640 1 --2- 172.21.15.38:0/2688275235 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f37300a2b90 0x7f37300a50a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.129+0000 7f374d1e0640 1 --2- 172.21.15.38:0/2688275235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37481078d0 0x7f37481a0770 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.129+0000 7f374d1e0640 1 --2- 172.21.15.38:0/2688275235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3748102bd0 0x7f37481997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.129+0000 7f374d1e0640 1 --2- 172.21.15.38:0/2688275235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3748101990 0x7f37481992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.129+0000 7f374d1e0640 1 -- 172.21.15.38:0/2688275235 >> 172.21.15.38:0/2688275235 conn(0x7f37480fcba0 msgr2=0x7f3748108d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:38.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.129+0000 7f374d1e0640 1 -- 172.21.15.38:0/2688275235 shutdown_connections 2024-05-21T15:34:38.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.129+0000 7f374d1e0640 1 -- 172.21.15.38:0/2688275235 wait complete. 2024-05-21T15:34:38.140 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:34:38.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.260+0000 7f0e15d36640 1 -- 172.21.15.38:0/2384684665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e10108830 msgr2=0x7f0e1010ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.260+0000 7f0e15d36640 1 --2- 172.21.15.38:0/2384684665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e10108830 0x7f0e1010ac50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009900 tx=0x7f0e0802a410 comp rx=0 tx=0).stop 2024-05-21T15:34:38.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.260+0000 7f0e15d36640 1 -- 172.21.15.38:0/2384684665 shutdown_connections 2024-05-21T15:34:38.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.260+0000 7f0e15d36640 1 --2- 172.21.15.38:0/2384684665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e10108830 0x7f0e1010ac50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.260+0000 7f0e15d36640 1 --2- 172.21.15.38:0/2384684665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e10105ed0 0x7f0e101082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.260+0000 7f0e15d36640 1 --2- 172.21.15.38:0/2384684665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e10103570 0x7f0e10105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.260+0000 7f0e15d36640 1 -- 172.21.15.38:0/2384684665 >> 172.21.15.38:0/2384684665 conn(0x7f0e100fcb70 msgr2=0x7f0e100ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:38.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.260+0000 7f0e15d36640 1 -- 172.21.15.38:0/2384684665 shutdown_connections 2024-05-21T15:34:38.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.260+0000 7f0e15d36640 1 -- 172.21.15.38:0/2384684665 wait complete. 2024-05-21T15:34:38.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.260+0000 7f0e15d36640 1 Processor -- start 2024-05-21T15:34:38.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e15d36640 1 -- start start 2024-05-21T15:34:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e15d36640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e10103570 0x7f0e10199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e15d36640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e10105ed0 0x7f0e10199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e15d36640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e10108830 0x7f0e101a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e15d36640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1010e750 con 0x7f0e10108830 2024-05-21T15:34:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e15d36640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1010e5d0 con 0x7f0e10105ed0 2024-05-21T15:34:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e15d36640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1010e8d0 con 0x7f0e10103570 2024-05-21T15:34:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e0ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e10108830 0x7f0e101a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e0ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e10108830 0x7f0e101a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59798/0 (socket says 172.21.15.38:59798) 2024-05-21T15:34:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e0ffff640 1 -- 172.21.15.38:0/733671948 learned_addr learned my addr 172.21.15.38:0/733671948 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e0effd640 1 --2- 172.21.15.38:0/733671948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e10105ed0 0x7f0e10199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e0f7fe640 1 --2- 172.21.15.38:0/733671948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e10103570 0x7f0e10199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e0ffff640 1 -- 172.21.15.38:0/733671948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e10103570 msgr2=0x7f0e10199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e0ffff640 1 --2- 172.21.15.38:0/733671948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e10103570 0x7f0e10199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e0ffff640 1 -- 172.21.15.38:0/733671948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e10105ed0 msgr2=0x7f0e10199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e0ffff640 1 --2- 172.21.15.38:0/733671948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e10105ed0 0x7f0e10199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.261+0000 7f0e0ffff640 1 -- 172.21.15.38:0/733671948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e101a0d40 con 0x7f0e10108830 2024-05-21T15:34:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.262+0000 7f0e0ffff640 1 --2- 172.21.15.38:0/733671948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e10108830 0x7f0e101a06d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0e0802a930 tx=0x7f0e08032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.262+0000 7f0e0cff9640 1 -- 172.21.15.38:0/733671948 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0803f030 con 0x7f0e10108830 2024-05-21T15:34:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.262+0000 7f0e0cff9640 1 -- 172.21.15.38:0/733671948 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e08032e50 con 0x7f0e10108830 2024-05-21T15:34:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.262+0000 7f0e15d36640 1 -- 172.21.15.38:0/733671948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e101a0fd0 con 0x7f0e10108830 2024-05-21T15:34:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.262+0000 7f0e0cff9640 1 -- 172.21.15.38:0/733671948 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08035070 con 0x7f0e10108830 2024-05-21T15:34:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.262+0000 7f0e15d36640 1 -- 172.21.15.38:0/733671948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e101a13e0 con 0x7f0e10108830 2024-05-21T15:34:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.263+0000 7f0e15d36640 1 -- 172.21.15.38:0/733671948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e100fe690 con 0x7f0e10108830 2024-05-21T15:34:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.264+0000 7f0e0cff9640 1 -- 172.21.15.38:0/733671948 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0e08042030 con 0x7f0e10108830 2024-05-21T15:34:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.265+0000 7f0e0cff9640 1 --2- 172.21.15.38:0/733671948 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0df80a2b90 0x7f0df80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:38.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.265+0000 7f0e0cff9640 1 -- 172.21.15.38:0/733671948 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f0e080efa60 con 0x7f0e10108830 2024-05-21T15:34:38.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.265+0000 7f0e0f7fe640 1 --2- 172.21.15.38:0/733671948 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0df80a2b90 0x7f0df80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:38.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.265+0000 7f0e0f7fe640 1 --2- 172.21.15.38:0/733671948 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0df80a2b90 0x7f0df80a50a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0e100699d0 tx=0x7f0e00009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:38.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.268+0000 7f0e0cff9640 1 -- 172.21.15.38:0/733671948 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0e080bc970 con 0x7f0e10108830 2024-05-21T15:34:38.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.460+0000 7f0e15d36640 1 -- 172.21.15.38:0/733671948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0e1002d4f0 con 0x7f0e10108830 2024-05-21T15:34:38.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.460+0000 7f0e0cff9640 1 -- 172.21.15.38:0/733671948 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f0e080bc270 con 0x7f0e10108830 2024-05-21T15:34:38.460 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-05-21T15:34:38.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.463+0000 7f0e15d36640 1 -- 172.21.15.38:0/733671948 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0df80a2b90 msgr2=0x7f0df80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.463+0000 7f0e15d36640 1 --2- 172.21.15.38:0/733671948 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0df80a2b90 0x7f0df80a50a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0e100699d0 tx=0x7f0e00009870 comp rx=0 tx=0).stop 2024-05-21T15:34:38.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.463+0000 7f0e15d36640 1 -- 172.21.15.38:0/733671948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e10108830 msgr2=0x7f0e101a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.463+0000 7f0e15d36640 1 --2- 172.21.15.38:0/733671948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e10108830 0x7f0e101a06d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0e0802a930 tx=0x7f0e08032c60 comp rx=0 tx=0).stop 2024-05-21T15:34:38.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.463+0000 7f0e15d36640 1 -- 172.21.15.38:0/733671948 shutdown_connections 2024-05-21T15:34:38.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.463+0000 7f0e15d36640 1 --2- 172.21.15.38:0/733671948 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0df80a2b90 0x7f0df80a50a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.463+0000 7f0e15d36640 1 --2- 172.21.15.38:0/733671948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e10108830 0x7f0e101a06d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.463+0000 7f0e15d36640 1 --2- 172.21.15.38:0/733671948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e10105ed0 0x7f0e10199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.463+0000 7f0e15d36640 1 --2- 172.21.15.38:0/733671948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e10103570 0x7f0e10199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.463+0000 7f0e15d36640 1 -- 172.21.15.38:0/733671948 >> 172.21.15.38:0/733671948 conn(0x7f0e100fcb70 msgr2=0x7f0e10104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:38.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.463+0000 7f0e15d36640 1 -- 172.21.15.38:0/733671948 shutdown_connections 2024-05-21T15:34:38.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.463+0000 7f0e15d36640 1 -- 172.21.15.38:0/733671948 wait complete. 2024-05-21T15:34:38.472 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:38 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1874151176' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-21T15:34:38.473 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:38 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:34:38.473 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:38 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/63268389' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:38.473 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:38 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2688275235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:38.474 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:34:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.593+0000 7fed15208640 1 -- 172.21.15.38:0/3138683072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed10101140 msgr2=0x7fed10101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.593+0000 7fed15208640 1 --2- 172.21.15.38:0/3138683072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed10101140 0x7fed10101e40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fed0000aa10 tx=0x7fed0002b720 comp rx=0 tx=0).stop 2024-05-21T15:34:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.593+0000 7fed15208640 1 -- 172.21.15.38:0/3138683072 shutdown_connections 2024-05-21T15:34:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.593+0000 7fed15208640 1 --2- 172.21.15.38:0/3138683072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed10107080 0x7fed101094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.593+0000 7fed15208640 1 --2- 172.21.15.38:0/3138683072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed10102380 0x7fed10102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.593+0000 7fed15208640 1 --2- 172.21.15.38:0/3138683072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed10101140 0x7fed10101e40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.593+0000 7fed15208640 1 -- 172.21.15.38:0/3138683072 >> 172.21.15.38:0/3138683072 conn(0x7fed100fc330 msgr2=0x7fed100fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.593+0000 7fed15208640 1 -- 172.21.15.38:0/3138683072 shutdown_connections 2024-05-21T15:34:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.593+0000 7fed15208640 1 -- 172.21.15.38:0/3138683072 wait complete. 2024-05-21T15:34:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.594+0000 7fed15208640 1 Processor -- start 2024-05-21T15:34:38.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.594+0000 7fed15208640 1 -- start start 2024-05-21T15:34:38.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.594+0000 7fed15208640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed10101140 0x7fed101992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:38.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.594+0000 7fed15208640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed10102380 0x7fed101997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:38.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.594+0000 7fed15208640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed10107080 0x7fed101a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:38.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.594+0000 7fed15208640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed1010c530 con 0x7fed10107080 2024-05-21T15:34:38.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.594+0000 7fed15208640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fed1010c3b0 con 0x7fed10102380 2024-05-21T15:34:38.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.594+0000 7fed15208640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed1010c6b0 con 0x7fed10101140 2024-05-21T15:34:38.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.594+0000 7fed0f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed10107080 0x7fed101a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:38.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.594+0000 7fed0f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed10107080 0x7fed101a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59808/0 (socket says 172.21.15.38:59808) 2024-05-21T15:34:38.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.594+0000 7fed0f59e640 1 -- 172.21.15.38:0/3829991800 learned_addr learned my addr 172.21.15.38:0/3829991800 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:38.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.594+0000 7fed0e59c640 1 --2- 172.21.15.38:0/3829991800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed10102380 0x7fed101997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:38.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.595+0000 7fed0ed9d640 1 --2- 172.21.15.38:0/3829991800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed10101140 0x7fed101992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:38.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.595+0000 7fed0f59e640 1 -- 172.21.15.38:0/3829991800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed10101140 msgr2=0x7fed101992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.595+0000 7fed0f59e640 1 --2- 172.21.15.38:0/3829991800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed10101140 0x7fed101992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.595+0000 7fed0f59e640 1 -- 172.21.15.38:0/3829991800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed10102380 msgr2=0x7fed101997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.595+0000 7fed0f59e640 1 --2- 172.21.15.38:0/3829991800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed10102380 0x7fed101997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.595+0000 7fed0f59e640 1 -- 172.21.15.38:0/3829991800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed101a0d30 con 0x7fed10107080 2024-05-21T15:34:38.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.595+0000 7fed0f59e640 1 --2- 172.21.15.38:0/3829991800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed10107080 0x7fed101a0770 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fed0800cad0 tx=0x7fed0800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:38.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.595+0000 7fecf7fff640 1 -- 172.21.15.38:0/3829991800 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0800ad60 con 0x7fed10107080 2024-05-21T15:34:38.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.595+0000 7fecf7fff640 1 -- 172.21.15.38:0/3829991800 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed08005020 con 0x7fed10107080 2024-05-21T15:34:38.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.595+0000 7fed15208640 1 -- 172.21.15.38:0/3829991800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed101a0fc0 con 0x7fed10107080 2024-05-21T15:34:38.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.595+0000 7fecf7fff640 1 -- 172.21.15.38:0/3829991800 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0800f6b0 con 0x7fed10107080 2024-05-21T15:34:38.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.595+0000 7fed15208640 1 -- 172.21.15.38:0/3829991800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed101a1430 con 0x7fed10107080 2024-05-21T15:34:38.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.596+0000 7fed15208640 1 -- 172.21.15.38:0/3829991800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed101a16c0 con 0x7fed10107080 2024-05-21T15:34:38.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.597+0000 7fecf7fff640 1 -- 172.21.15.38:0/3829991800 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fed08005560 con 0x7fed10107080 2024-05-21T15:34:38.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.598+0000 7fecf7fff640 1 --2- 172.21.15.38:0/3829991800 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fecf80a2b90 0x7fecf80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:38.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.598+0000 7fed0ed9d640 1 --2- 172.21.15.38:0/3829991800 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fecf80a2b90 0x7fecf80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:38.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.598+0000 7fecf7fff640 1 -- 172.21.15.38:0/3829991800 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fed080cf060 con 0x7fed10107080 2024-05-21T15:34:38.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.599+0000 7fed0ed9d640 1 --2- 172.21.15.38:0/3829991800 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fecf80a2b90 0x7fecf80a50a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fed0002bd10 tx=0x7fed000306c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:38.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.603+0000 7fecf7fff640 1 -- 172.21.15.38:0/3829991800 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fed08014030 con 0x7fed10107080 2024-05-21T15:34:38.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:38 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1874151176' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-21T15:34:38.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:38 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:34:38.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:38 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/63268389' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:38.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:38 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2688275235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:38.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:38 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1874151176' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-21T15:34:38.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:38 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:34:38.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:38 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/63268389' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:38.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:38 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2688275235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:38.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.789+0000 7fed15208640 1 -- 172.21.15.38:0/3829991800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fed101a1890 con 0x7fed10107080 2024-05-21T15:34:38.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.789+0000 7fecf7fff640 1 -- 172.21.15.38:0/3829991800 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fed0809bf70 con 0x7fed10107080 2024-05-21T15:34:38.789 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-05-21T15:34:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.791+0000 7fed15208640 1 -- 172.21.15.38:0/3829991800 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fecf80a2b90 msgr2=0x7fecf80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.791+0000 7fed15208640 1 --2- 172.21.15.38:0/3829991800 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fecf80a2b90 0x7fecf80a50a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fed0002bd10 tx=0x7fed000306c0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.791+0000 7fed15208640 1 -- 172.21.15.38:0/3829991800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed10107080 msgr2=0x7fed101a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.791+0000 7fed15208640 1 --2- 172.21.15.38:0/3829991800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed10107080 0x7fed101a0770 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fed0800cad0 tx=0x7fed0800ced0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.791+0000 7fed15208640 1 -- 172.21.15.38:0/3829991800 shutdown_connections 2024-05-21T15:34:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.791+0000 7fed15208640 1 --2- 172.21.15.38:0/3829991800 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fecf80a2b90 0x7fecf80a50a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.791+0000 7fed15208640 1 --2- 172.21.15.38:0/3829991800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed10107080 0x7fed101a0770 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.791+0000 7fed15208640 1 --2- 172.21.15.38:0/3829991800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed10102380 0x7fed101997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.791+0000 7fed15208640 1 --2- 172.21.15.38:0/3829991800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed10101140 0x7fed101992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.791+0000 7fed15208640 1 -- 172.21.15.38:0/3829991800 >> 172.21.15.38:0/3829991800 conn(0x7fed100fc330 msgr2=0x7fed10105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:38.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.791+0000 7fed15208640 1 -- 172.21.15.38:0/3829991800 shutdown_connections 2024-05-21T15:34:38.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.791+0000 7fed15208640 1 -- 172.21.15.38:0/3829991800 wait complete. 2024-05-21T15:34:38.802 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:34:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.925+0000 7fc0509bc640 1 -- 172.21.15.38:0/1668176172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc04c108860 msgr2=0x7fc04c10ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.925+0000 7fc0509bc640 1 --2- 172.21.15.38:0/1668176172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc04c108860 0x7fc04c10ac80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc044009ca0 tx=0x7fc04402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.925+0000 7fc0509bc640 1 -- 172.21.15.38:0/1668176172 shutdown_connections 2024-05-21T15:34:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.925+0000 7fc0509bc640 1 --2- 172.21.15.38:0/1668176172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc04c108860 0x7fc04c10ac80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.925+0000 7fc0509bc640 1 --2- 172.21.15.38:0/1668176172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc04c105f00 0x7fc04c108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.925+0000 7fc0509bc640 1 --2- 172.21.15.38:0/1668176172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc04c1035a0 0x7fc04c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.925+0000 7fc0509bc640 1 -- 172.21.15.38:0/1668176172 >> 172.21.15.38:0/1668176172 conn(0x7fc04c0fcba0 msgr2=0x7fc04c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.925+0000 7fc0509bc640 1 -- 172.21.15.38:0/1668176172 shutdown_connections 2024-05-21T15:34:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.925+0000 7fc0509bc640 1 -- 172.21.15.38:0/1668176172 wait complete. 2024-05-21T15:34:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.926+0000 7fc0509bc640 1 Processor -- start 2024-05-21T15:34:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.926+0000 7fc0509bc640 1 -- start start 2024-05-21T15:34:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.926+0000 7fc0509bc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc04c1035a0 0x7fc04c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.926+0000 7fc0509bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc04c105f00 0x7fc04c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.926+0000 7fc0509bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc04c108860 0x7fc04c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.926+0000 7fc0509bc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c10e710 con 0x7fc04c105f00 2024-05-21T15:34:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.926+0000 7fc0509bc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c10e590 con 0x7fc04c1035a0 2024-05-21T15:34:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.926+0000 7fc0509bc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c10e890 con 0x7fc04c108860 2024-05-21T15:34:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.926+0000 7fc041d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc04c105f00 0x7fc04c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.926+0000 7fc041d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc04c105f00 0x7fc04c1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59834/0 (socket says 172.21.15.38:59834) 2024-05-21T15:34:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.926+0000 7fc041d9b640 1 -- 172.21.15.38:0/738009942 learned_addr learned my addr 172.21.15.38:0/738009942 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.927+0000 7fc04a59c640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc04c1035a0 0x7fc04c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:38.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.927+0000 7fc04ad9d640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc04c108860 0x7fc04c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:38.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.927+0000 7fc041d9b640 1 -- 172.21.15.38:0/738009942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc04c108860 msgr2=0x7fc04c1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.927+0000 7fc041d9b640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc04c108860 0x7fc04c1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.927+0000 7fc041d9b640 1 -- 172.21.15.38:0/738009942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc04c1035a0 msgr2=0x7fc04c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:38.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.927+0000 7fc041d9b640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc04c1035a0 0x7fc04c199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:38.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.927+0000 7fc041d9b640 1 -- 172.21.15.38:0/738009942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c1a0cb0 con 0x7fc04c105f00 2024-05-21T15:34:38.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.927+0000 7fc04a59c640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc04c1035a0 0x7fc04c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:38.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.927+0000 7fc041d9b640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc04c105f00 0x7fc04c1996b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc03400d950 tx=0x7fc03400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:38.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.927+0000 7fc043fff640 1 -- 172.21.15.38:0/738009942 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034017030 con 0x7fc04c105f00 2024-05-21T15:34:38.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.927+0000 7fc0509bc640 1 -- 172.21.15.38:0/738009942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04c1a0fa0 con 0x7fc04c105f00 2024-05-21T15:34:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.927+0000 7fc043fff640 1 -- 172.21.15.38:0/738009942 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc034004020 con 0x7fc04c105f00 2024-05-21T15:34:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.927+0000 7fc043fff640 1 -- 172.21.15.38:0/738009942 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0340047a0 con 0x7fc04c105f00 2024-05-21T15:34:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.927+0000 7fc0509bc640 1 -- 172.21.15.38:0/738009942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04c1a1460 con 0x7fc04c105f00 2024-05-21T15:34:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.928+0000 7fc0509bc640 1 -- 172.21.15.38:0/738009942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04c0fe6c0 con 0x7fc04c105f00 2024-05-21T15:34:38.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.930+0000 7fc043fff640 1 -- 172.21.15.38:0/738009942 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc0340041d0 con 0x7fc04c105f00 2024-05-21T15:34:38.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.930+0000 7fc043fff640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc0380a2c30 0x7fc0380a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:38.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.930+0000 7fc04a59c640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc0380a2c30 0x7fc0380a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:38.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.930+0000 7fc043fff640 1 -- 172.21.15.38:0/738009942 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fc0340cf1d0 con 0x7fc04c105f00 2024-05-21T15:34:38.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.931+0000 7fc04a59c640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc0380a2c30 0x7fc0380a5140 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0058c0 tx=0x7fc03c005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:38.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:38.934+0000 7fc043fff640 1 -- 172.21.15.38:0/738009942 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc03409c0e0 con 0x7fc04c105f00 2024-05-21T15:34:39.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.120+0000 7fc0509bc640 1 -- 172.21.15.38:0/738009942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc04c02d420 con 0x7fc04c105f00 2024-05-21T15:34:39.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.122+0000 7fc043fff640 1 -- 172.21.15.38:0/738009942 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22704 (secure 0 0 0) 0x7fc03409b9e0 con 0x7fc04c105f00 2024-05-21T15:34:39.121 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:39.122 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":90,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:34:23.996759+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:34:39.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.124+0000 7fc0509bc640 1 -- 172.21.15.38:0/738009942 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc0380a2c30 msgr2=0x7fc0380a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.124+0000 7fc0509bc640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc0380a2c30 0x7fc0380a5140 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0058c0 tx=0x7fc03c005d30 comp rx=0 tx=0).stop 2024-05-21T15:34:39.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.124+0000 7fc0509bc640 1 -- 172.21.15.38:0/738009942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc04c105f00 msgr2=0x7fc04c1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.124+0000 7fc0509bc640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc04c105f00 0x7fc04c1996b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc03400d950 tx=0x7fc03400de10 comp rx=0 tx=0).stop 2024-05-21T15:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.124+0000 7fc0509bc640 1 -- 172.21.15.38:0/738009942 shutdown_connections 2024-05-21T15:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.124+0000 7fc0509bc640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc04c108860 0x7fc04c1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.124+0000 7fc0509bc640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc0380a2c30 0x7fc0380a5140 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.124+0000 7fc0509bc640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc04c105f00 0x7fc04c1996b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.124+0000 7fc0509bc640 1 --2- 172.21.15.38:0/738009942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc04c1035a0 0x7fc04c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.124+0000 7fc0509bc640 1 -- 172.21.15.38:0/738009942 >> 172.21.15.38:0/738009942 conn(0x7fc04c0fcba0 msgr2=0x7fc04c1043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.124+0000 7fc0509bc640 1 -- 172.21.15.38:0/738009942 shutdown_connections 2024-05-21T15:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.124+0000 7fc0509bc640 1 -- 172.21.15.38:0/738009942 wait complete. 2024-05-21T15:34:39.136 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:27:50.659367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:32:31.645647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:32:32.955451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:34:39.136 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:34:39.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.255+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3260304892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b94108710 msgr2=0x7f8b9410ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.255+0000 7f8b9a0eb640 1 --2- 172.21.15.38:0/3260304892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b94108710 0x7f8b9410ac20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009ca0 tx=0x7f8b8c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.255+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3260304892 shutdown_connections 2024-05-21T15:34:39.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.255+0000 7f8b9a0eb640 1 --2- 172.21.15.38:0/3260304892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b94108710 0x7f8b9410ac20 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.255+0000 7f8b9a0eb640 1 --2- 172.21.15.38:0/3260304892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b94105be0 0x7f8b94108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.255+0000 7f8b9a0eb640 1 --2- 172.21.15.38:0/3260304892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b941031f0 0x7f8b94105610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.255+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3260304892 >> 172.21.15.38:0/3260304892 conn(0x7f8b940fcb70 msgr2=0x7f8b940ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.255+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3260304892 shutdown_connections 2024-05-21T15:34:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.255+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3260304892 wait complete. 2024-05-21T15:34:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.256+0000 7f8b9a0eb640 1 Processor -- start 2024-05-21T15:34:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.256+0000 7f8b9a0eb640 1 -- start start 2024-05-21T15:34:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.256+0000 7f8b9a0eb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b941031f0 0x7f8b94199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.256+0000 7f8b9a0eb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b94105be0 0x7f8b94199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.256+0000 7f8b9a0eb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b94108710 0x7f8b941a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.256+0000 7f8b9a0eb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9410e6f0 con 0x7f8b94108710 2024-05-21T15:34:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.256+0000 7f8b9a0eb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9410e570 con 0x7f8b941031f0 2024-05-21T15:34:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.256+0000 7f8b9a0eb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9410e870 con 0x7f8b94105be0 2024-05-21T15:34:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.256+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b94108710 0x7f8b941a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.256+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b94108710 0x7f8b941a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59860/0 (socket says 172.21.15.38:59860) 2024-05-21T15:34:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.256+0000 7f8b93fff640 1 -- 172.21.15.38:0/3375374255 learned_addr learned my addr 172.21.15.38:0/3375374255 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.257+0000 7f8b92ffd640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b94105be0 0x7f8b94199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.257+0000 7f8b937fe640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b941031f0 0x7f8b94199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.257+0000 7f8b93fff640 1 -- 172.21.15.38:0/3375374255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b94105be0 msgr2=0x7f8b94199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.257+0000 7f8b93fff640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b94105be0 0x7f8b94199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.257+0000 7f8b93fff640 1 -- 172.21.15.38:0/3375374255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b941031f0 msgr2=0x7f8b94199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.257+0000 7f8b93fff640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b941031f0 0x7f8b94199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.257+0000 7f8b93fff640 1 -- 172.21.15.38:0/3375374255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b941a0d50 con 0x7f8b94108710 2024-05-21T15:34:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.257+0000 7f8b92ffd640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b94105be0 0x7f8b94199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.257+0000 7f8b93fff640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b94108710 0x7f8b941a06e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009430 tx=0x7f8b8c02f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.257+0000 7f8b90ff9640 1 -- 172.21.15.38:0/3375374255 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c003c30 con 0x7f8b94108710 2024-05-21T15:34:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.257+0000 7f8b90ff9640 1 -- 172.21.15.38:0/3375374255 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b8c003dd0 con 0x7f8b94108710 2024-05-21T15:34:39.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.257+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3375374255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b941a0fe0 con 0x7f8b94108710 2024-05-21T15:34:39.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.257+0000 7f8b90ff9640 1 -- 172.21.15.38:0/3375374255 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0047e0 con 0x7f8b94108710 2024-05-21T15:34:39.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.257+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3375374255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b941a1440 con 0x7f8b94108710 2024-05-21T15:34:39.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.258+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3375374255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b940fe690 con 0x7f8b94108710 2024-05-21T15:34:39.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.260+0000 7f8b90ff9640 1 -- 172.21.15.38:0/3375374255 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8b8c004b40 con 0x7f8b94108710 2024-05-21T15:34:39.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.260+0000 7f8b90ff9640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8b7c0a2b90 0x7f8b7c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:39.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.261+0000 7f8b937fe640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8b7c0a2b90 0x7f8b7c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:39.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.261+0000 7f8b90ff9640 1 -- 172.21.15.38:0/3375374255 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8b8c0f0790 con 0x7f8b94108710 2024-05-21T15:34:39.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.261+0000 7f8b937fe640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8b7c0a2b90 0x7f8b7c0a50a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8b840058c0 tx=0x7f8b8400a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:39.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.264+0000 7f8b90ff9640 1 -- 172.21.15.38:0/3375374255 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8b8c0bd6a0 con 0x7f8b94108710 2024-05-21T15:34:39.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.454+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3375374255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8b9402d430 con 0x7f8b94108710 2024-05-21T15:34:39.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.454+0000 7f8b90ff9640 1 -- 172.21.15.38:0/3375374255 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f8b8c0bcfa0 con 0x7f8b94108710 2024-05-21T15:34:39.454 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-05-21T15:34:39.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.456+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3375374255 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8b7c0a2b90 msgr2=0x7f8b7c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.456+0000 7f8b9a0eb640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8b7c0a2b90 0x7f8b7c0a50a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8b840058c0 tx=0x7f8b8400a4c0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.456+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3375374255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b94108710 msgr2=0x7f8b941a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.456+0000 7f8b9a0eb640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b94108710 0x7f8b941a06e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009430 tx=0x7f8b8c02f610 comp rx=0 tx=0).stop 2024-05-21T15:34:39.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.457+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3375374255 shutdown_connections 2024-05-21T15:34:39.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.457+0000 7f8b9a0eb640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8b7c0a2b90 0x7f8b7c0a50a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.457+0000 7f8b9a0eb640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b94108710 0x7f8b941a06e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.457+0000 7f8b9a0eb640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b94105be0 0x7f8b94199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.457+0000 7f8b9a0eb640 1 --2- 172.21.15.38:0/3375374255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b941031f0 0x7f8b94199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.457+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3375374255 >> 172.21.15.38:0/3375374255 conn(0x7f8b940fcb70 msgr2=0x7f8b94103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:39.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.457+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3375374255 shutdown_connections 2024-05-21T15:34:39.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.457+0000 7f8b9a0eb640 1 -- 172.21.15.38:0/3375374255 wait complete. 2024-05-21T15:34:39.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:39 smithi038 ceph-mon[31995]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:34:39.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:39 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/733671948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:39.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:39 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3829991800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:39.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:39 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/738009942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:39.467 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.586+0000 7f4d88c1b640 1 -- 172.21.15.38:0/1279166638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d84104120 msgr2=0x7f4d841022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.586+0000 7f4d88c1b640 1 --2- 172.21.15.38:0/1279166638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d84104120 0x7f4d841022a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009900 tx=0x7f4d7402a410 comp rx=0 tx=0).stop 2024-05-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.586+0000 7f4d88c1b640 1 -- 172.21.15.38:0/1279166638 shutdown_connections 2024-05-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.586+0000 7f4d88c1b640 1 --2- 172.21.15.38:0/1279166638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d84108a20 0x7f4d8410ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.586+0000 7f4d88c1b640 1 --2- 172.21.15.38:0/1279166638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d841027e0 0x7f4d84102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.586+0000 7f4d88c1b640 1 --2- 172.21.15.38:0/1279166638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d84104120 0x7f4d841022a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.586+0000 7f4d88c1b640 1 -- 172.21.15.38:0/1279166638 >> 172.21.15.38:0/1279166638 conn(0x7f4d840fdcf0 msgr2=0x7f4d84100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.586+0000 7f4d88c1b640 1 -- 172.21.15.38:0/1279166638 shutdown_connections 2024-05-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.587+0000 7f4d88c1b640 1 -- 172.21.15.38:0/1279166638 wait complete. 2024-05-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.587+0000 7f4d88c1b640 1 Processor -- start 2024-05-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.587+0000 7f4d88c1b640 1 -- start start 2024-05-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d88c1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d841027e0 0x7f4d84199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d88c1b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d84104120 0x7f4d841998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d88c1b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d84108a20 0x7f4d841a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d88c1b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8410e840 con 0x7f4d84108a20 2024-05-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d88c1b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8410e6c0 con 0x7f4d841027e0 2024-05-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d88c1b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8410e9c0 con 0x7f4d84104120 2024-05-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d82d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d84108a20 0x7f4d841a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d82d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d84108a20 0x7f4d841a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59884/0 (socket says 172.21.15.38:59884) 2024-05-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d82d9d640 1 -- 172.21.15.38:0/672264610 learned_addr learned my addr 172.21.15.38:0/672264610 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d81d9b640 1 --2- 172.21.15.38:0/672264610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d84104120 0x7f4d841998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d8259c640 1 --2- 172.21.15.38:0/672264610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d841027e0 0x7f4d84199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d82d9d640 1 -- 172.21.15.38:0/672264610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d84104120 msgr2=0x7f4d841998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d82d9d640 1 --2- 172.21.15.38:0/672264610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d84104120 0x7f4d841998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d82d9d640 1 -- 172.21.15.38:0/672264610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d841027e0 msgr2=0x7f4d84199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d82d9d640 1 --2- 172.21.15.38:0/672264610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d841027e0 0x7f4d84199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d82d9d640 1 -- 172.21.15.38:0/672264610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d841a0e10 con 0x7f4d84108a20 2024-05-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d82d9d640 1 --2- 172.21.15.38:0/672264610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d84108a20 0x7f4d841a0850 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00eaa0 tx=0x7f4d7c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d737fe640 1 -- 172.21.15.38:0/672264610 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c004d20 con 0x7f4d84108a20 2024-05-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.588+0000 7f4d737fe640 1 -- 172.21.15.38:0/672264610 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d7c005020 con 0x7f4d84108a20 2024-05-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.589+0000 7f4d88c1b640 1 -- 172.21.15.38:0/672264610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d841a1100 con 0x7f4d84108a20 2024-05-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.589+0000 7f4d737fe640 1 -- 172.21.15.38:0/672264610 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c0116d0 con 0x7f4d84108a20 2024-05-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.589+0000 7f4d88c1b640 1 -- 172.21.15.38:0/672264610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d841a1650 con 0x7f4d84108a20 2024-05-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.590+0000 7f4d88c1b640 1 -- 172.21.15.38:0/672264610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d840691c0 con 0x7f4d84108a20 2024-05-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.592+0000 7f4d737fe640 1 -- 172.21.15.38:0/672264610 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d7c0051d0 con 0x7f4d84108a20 2024-05-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.592+0000 7f4d737fe640 1 --2- 172.21.15.38:0/672264610 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4d6c0a2b90 0x7f4d6c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.593+0000 7f4d8259c640 1 --2- 172.21.15.38:0/672264610 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4d6c0a2b90 0x7f4d6c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.593+0000 7f4d737fe640 1 -- 172.21.15.38:0/672264610 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f4d7c0cf670 con 0x7f4d84108a20 2024-05-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.593+0000 7f4d8259c640 1 --2- 172.21.15.38:0/672264610 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4d6c0a2b90 0x7f4d6c0a50a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009230 tx=0x7f4d74031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.596+0000 7f4d737fe640 1 -- 172.21.15.38:0/672264610 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4d7c016030 con 0x7f4d84108a20 2024-05-21T15:34:39.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:39 smithi053 ceph-mon[36496]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:34:39.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:39 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/733671948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:39.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:39 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3829991800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:39.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:39 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/738009942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:39.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:39 smithi045 ceph-mon[36409]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:34:39.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:39 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/733671948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:39.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:39 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3829991800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:39.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:39 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/738009942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:39.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.779+0000 7f4d88c1b640 1 -- 172.21.15.38:0/672264610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4d8402d2a0 con 0x7f4d84108a20 2024-05-21T15:34:39.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.780+0000 7f4d737fe640 1 -- 172.21.15.38:0/672264610 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f4d7c09c580 con 0x7f4d84108a20 2024-05-21T15:34:39.779 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-05-21T15:34:39.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.782+0000 7f4d88c1b640 1 -- 172.21.15.38:0/672264610 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4d6c0a2b90 msgr2=0x7f4d6c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.782+0000 7f4d88c1b640 1 --2- 172.21.15.38:0/672264610 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4d6c0a2b90 0x7f4d6c0a50a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009230 tx=0x7f4d74031400 comp rx=0 tx=0).stop 2024-05-21T15:34:39.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.782+0000 7f4d88c1b640 1 -- 172.21.15.38:0/672264610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d84108a20 msgr2=0x7f4d841a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.782+0000 7f4d88c1b640 1 --2- 172.21.15.38:0/672264610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d84108a20 0x7f4d841a0850 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00eaa0 tx=0x7f4d7c00eea0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.782+0000 7f4d88c1b640 1 -- 172.21.15.38:0/672264610 shutdown_connections 2024-05-21T15:34:39.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.782+0000 7f4d88c1b640 1 --2- 172.21.15.38:0/672264610 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4d6c0a2b90 0x7f4d6c0a50a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.782+0000 7f4d88c1b640 1 --2- 172.21.15.38:0/672264610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d84108a20 0x7f4d841a0850 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.782+0000 7f4d88c1b640 1 --2- 172.21.15.38:0/672264610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d84104120 0x7f4d841998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.782+0000 7f4d88c1b640 1 --2- 172.21.15.38:0/672264610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d841027e0 0x7f4d84199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.782+0000 7f4d88c1b640 1 -- 172.21.15.38:0/672264610 >> 172.21.15.38:0/672264610 conn(0x7f4d840fdcf0 msgr2=0x7f4d84104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:39.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.782+0000 7f4d88c1b640 1 -- 172.21.15.38:0/672264610 shutdown_connections 2024-05-21T15:34:39.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.782+0000 7f4d88c1b640 1 -- 172.21.15.38:0/672264610 wait complete. 2024-05-21T15:34:39.793 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:34:39.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.911+0000 7f1a83279640 1 -- 172.21.15.38:0/1602059533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a7c101980 msgr2=0x7f1a7c102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.911+0000 7f1a83279640 1 --2- 172.21.15.38:0/1602059533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a7c101980 0x7f1a7c102680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009900 tx=0x7f1a7802a410 comp rx=0 tx=0).stop 2024-05-21T15:34:39.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.911+0000 7f1a83279640 1 -- 172.21.15.38:0/1602059533 shutdown_connections 2024-05-21T15:34:39.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.911+0000 7f1a83279640 1 --2- 172.21.15.38:0/1602059533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a7c1078c0 0x7f1a7c109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.911+0000 7f1a83279640 1 --2- 172.21.15.38:0/1602059533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a7c102bc0 0x7f1a7c102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.911+0000 7f1a83279640 1 --2- 172.21.15.38:0/1602059533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a7c101980 0x7f1a7c102680 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.911+0000 7f1a83279640 1 -- 172.21.15.38:0/1602059533 >> 172.21.15.38:0/1602059533 conn(0x7f1a7c0fcb70 msgr2=0x7f1a7c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:39.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.911+0000 7f1a83279640 1 -- 172.21.15.38:0/1602059533 shutdown_connections 2024-05-21T15:34:39.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.911+0000 7f1a83279640 1 -- 172.21.15.38:0/1602059533 wait complete. 2024-05-21T15:34:39.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a83279640 1 Processor -- start 2024-05-21T15:34:39.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a83279640 1 -- start start 2024-05-21T15:34:39.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a83279640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a7c101980 0x7f1a7c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:39.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a83279640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a7c102bc0 0x7f1a7c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:39.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a83279640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a7c1078c0 0x7f1a7c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:39.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a83279640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c10d5d0 con 0x7f1a7c1078c0 2024-05-21T15:34:39.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a83279640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c10d450 con 0x7f1a7c102bc0 2024-05-21T15:34:39.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a83279640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c10d750 con 0x7f1a7c101980 2024-05-21T15:34:39.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a81816640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a7c1078c0 0x7f1a7c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:39.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a81816640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a7c1078c0 0x7f1a7c19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59908/0 (socket says 172.21.15.38:59908) 2024-05-21T15:34:39.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a81816640 1 -- 172.21.15.38:0/2259230639 learned_addr learned my addr 172.21.15.38:0/2259230639 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:39.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a80814640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a7c102bc0 0x7f1a7c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:39.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a81015640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a7c101980 0x7f1a7c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:39.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a81816640 1 -- 172.21.15.38:0/2259230639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a7c101980 msgr2=0x7f1a7c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a81816640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a7c101980 0x7f1a7c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a81816640 1 -- 172.21.15.38:0/2259230639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a7c102bc0 msgr2=0x7f1a7c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:39.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a81816640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a7c102bc0 0x7f1a7c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:39.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a81816640 1 -- 172.21.15.38:0/2259230639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c19fcb0 con 0x7f1a7c1078c0 2024-05-21T15:34:39.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.912+0000 7f1a80814640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a7c102bc0 0x7f1a7c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:39.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.913+0000 7f1a81816640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a7c1078c0 0x7f1a7c19f640 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400eb10 tx=0x7f1a7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:39.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.913+0000 7f1a727fc640 1 -- 172.21.15.38:0/2259230639 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74004b20 con 0x7f1a7c1078c0 2024-05-21T15:34:39.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.913+0000 7f1a83279640 1 -- 172.21.15.38:0/2259230639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7c19ffa0 con 0x7f1a7c1078c0 2024-05-21T15:34:39.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.913+0000 7f1a727fc640 1 -- 172.21.15.38:0/2259230639 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a74004cc0 con 0x7f1a7c1078c0 2024-05-21T15:34:39.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.913+0000 7f1a727fc640 1 -- 172.21.15.38:0/2259230639 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74010760 con 0x7f1a7c1078c0 2024-05-21T15:34:39.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.913+0000 7f1a83279640 1 -- 172.21.15.38:0/2259230639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7c1a03b0 con 0x7f1a7c1078c0 2024-05-21T15:34:39.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.914+0000 7f1a83279640 1 -- 172.21.15.38:0/2259230639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a7c1066d0 con 0x7f1a7c1078c0 2024-05-21T15:34:39.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.915+0000 7f1a727fc640 1 -- 172.21.15.38:0/2259230639 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1a74010900 con 0x7f1a7c1078c0 2024-05-21T15:34:39.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.916+0000 7f1a727fc640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1a680a2b90 0x7f1a680a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:39.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.916+0000 7f1a727fc640 1 -- 172.21.15.38:0/2259230639 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f1a740cf910 con 0x7f1a7c1078c0 2024-05-21T15:34:39.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.916+0000 7f1a81015640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1a680a2b90 0x7f1a680a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:39.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.917+0000 7f1a81015640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1a680a2b90 0x7f1a680a50a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009230 tx=0x7f1a78031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:39.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:39.920+0000 7f1a727fc640 1 -- 172.21.15.38:0/2259230639 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1a74053850 con 0x7f1a7c1078c0 2024-05-21T15:34:40.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.103+0000 7f1a83279640 1 -- 172.21.15.38:0/2259230639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1a7c02d430 con 0x7f1a7c1078c0 2024-05-21T15:34:40.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.104+0000 7f1a727fc640 1 -- 172.21.15.38:0/2259230639 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f1a7409c820 con 0x7f1a7c1078c0 2024-05-21T15:34:40.107 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-05-21T15:34:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.106+0000 7f1a83279640 1 -- 172.21.15.38:0/2259230639 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1a680a2b90 msgr2=0x7f1a680a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.106+0000 7f1a83279640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1a680a2b90 0x7f1a680a50a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009230 tx=0x7f1a78031400 comp rx=0 tx=0).stop 2024-05-21T15:34:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.106+0000 7f1a83279640 1 -- 172.21.15.38:0/2259230639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a7c1078c0 msgr2=0x7f1a7c19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.106+0000 7f1a83279640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a7c1078c0 0x7f1a7c19f640 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400eb10 tx=0x7f1a7400c2d0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.106+0000 7f1a83279640 1 -- 172.21.15.38:0/2259230639 shutdown_connections 2024-05-21T15:34:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.106+0000 7f1a83279640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1a680a2b90 0x7f1a680a50a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.106+0000 7f1a83279640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a7c1078c0 0x7f1a7c19f640 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.106+0000 7f1a83279640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a7c102bc0 0x7f1a7c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.106+0000 7f1a83279640 1 --2- 172.21.15.38:0/2259230639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a7c101980 0x7f1a7c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.106+0000 7f1a83279640 1 -- 172.21.15.38:0/2259230639 >> 172.21.15.38:0/2259230639 conn(0x7f1a7c0fcb70 msgr2=0x7f1a7c1038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.106+0000 7f1a83279640 1 -- 172.21.15.38:0/2259230639 shutdown_connections 2024-05-21T15:34:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.106+0000 7f1a83279640 1 -- 172.21.15.38:0/2259230639 wait complete. 2024-05-21T15:34:40.117 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-21T15:34:40.117 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:34:40.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.233+0000 7f51ad414640 1 -- 172.21.15.38:0/1690019838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a8102c00 msgr2=0x7f51a8103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.233+0000 7f51ad414640 1 --2- 172.21.15.38:0/1690019838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a8102c00 0x7f51a8103010 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f519c009900 tx=0x7f519c02a410 comp rx=0 tx=0).stop 2024-05-21T15:34:40.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.233+0000 7f51ad414640 1 -- 172.21.15.38:0/1690019838 shutdown_connections 2024-05-21T15:34:40.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.233+0000 7f51ad414640 1 --2- 172.21.15.38:0/1690019838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51a8107900 0x7f51a8109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.233+0000 7f51ad414640 1 --2- 172.21.15.38:0/1690019838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a8102c00 0x7f51a8103010 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.233+0000 7f51ad414640 1 --2- 172.21.15.38:0/1690019838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a81019c0 0x7f51a81026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.233+0000 7f51ad414640 1 -- 172.21.15.38:0/1690019838 >> 172.21.15.38:0/1690019838 conn(0x7f51a80fcbb0 msgr2=0x7f51a80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:40.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.233+0000 7f51ad414640 1 -- 172.21.15.38:0/1690019838 shutdown_connections 2024-05-21T15:34:40.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.233+0000 7f51ad414640 1 -- 172.21.15.38:0/1690019838 wait complete. 2024-05-21T15:34:40.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.234+0000 7f51ad414640 1 Processor -- start 2024-05-21T15:34:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.234+0000 7f51ad414640 1 -- start start 2024-05-21T15:34:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.234+0000 7f51ad414640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a81019c0 0x7f51a8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.234+0000 7f51ad414640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51a8102c00 0x7f51a81997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.234+0000 7f51ad414640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a8107900 0x7f51a81a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.234+0000 7f51ad414640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f51a810e740 con 0x7f51a8107900 2024-05-21T15:34:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.234+0000 7f51ad414640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51a810e5c0 con 0x7f51a8102c00 2024-05-21T15:34:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.234+0000 7f51ad414640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f51a810e8c0 con 0x7f51a81019c0 2024-05-21T15:34:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.234+0000 7f51a77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a8107900 0x7f51a81a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.234+0000 7f51a77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a8107900 0x7f51a81a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59932/0 (socket says 172.21.15.38:59932) 2024-05-21T15:34:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.234+0000 7f51a77fe640 1 -- 172.21.15.38:0/2974519708 learned_addr learned my addr 172.21.15.38:0/2974519708 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.234+0000 7f51a6ffd640 1 --2- 172.21.15.38:0/2974519708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a81019c0 0x7f51a8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.234+0000 7f51a67fc640 1 --2- 172.21.15.38:0/2974519708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51a8102c00 0x7f51a81997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.235+0000 7f51a77fe640 1 -- 172.21.15.38:0/2974519708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a81019c0 msgr2=0x7f51a8199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.235+0000 7f51a77fe640 1 --2- 172.21.15.38:0/2974519708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a81019c0 0x7f51a8199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.235+0000 7f51a77fe640 1 -- 172.21.15.38:0/2974519708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51a8102c00 msgr2=0x7f51a81997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.235+0000 7f51a77fe640 1 --2- 172.21.15.38:0/2974519708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51a8102c00 0x7f51a81997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.235+0000 7f51a77fe640 1 -- 172.21.15.38:0/2974519708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a81a0d50 con 0x7f51a8107900 2024-05-21T15:34:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.235+0000 7f51a77fe640 1 --2- 172.21.15.38:0/2974519708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a8107900 0x7f51a81a06e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f51a000e9b0 tx=0x7f51a000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.235+0000 7f518ffff640 1 -- 172.21.15.38:0/2974519708 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a000cd80 con 0x7f51a8107900 2024-05-21T15:34:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.235+0000 7f51ad414640 1 -- 172.21.15.38:0/2974519708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a81a0fe0 con 0x7f51a8107900 2024-05-21T15:34:40.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.235+0000 7f518ffff640 1 -- 172.21.15.38:0/2974519708 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51a00035f0 con 0x7f51a8107900 2024-05-21T15:34:40.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.235+0000 7f518ffff640 1 -- 172.21.15.38:0/2974519708 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0010670 con 0x7f51a8107900 2024-05-21T15:34:40.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.235+0000 7f51ad414640 1 -- 172.21.15.38:0/2974519708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a81a14a0 con 0x7f51a8107900 2024-05-21T15:34:40.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.236+0000 7f51ad414640 1 -- 172.21.15.38:0/2974519708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a8106710 con 0x7f51a8107900 2024-05-21T15:34:40.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.237+0000 7f518ffff640 1 -- 172.21.15.38:0/2974519708 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f51a00037a0 con 0x7f51a8107900 2024-05-21T15:34:40.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.238+0000 7f518ffff640 1 --2- 172.21.15.38:0/2974519708 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51900a2c30 0x7f51900a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:40.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.238+0000 7f518ffff640 1 -- 172.21.15.38:0/2974519708 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f51a00cf310 con 0x7f51a8107900 2024-05-21T15:34:40.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.238+0000 7f51a6ffd640 1 --2- 172.21.15.38:0/2974519708 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51900a2c30 0x7f51900a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:40.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.239+0000 7f51a6ffd640 1 --2- 172.21.15.38:0/2974519708 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51900a2c30 0x7f51900a5140 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f51a81025c0 tx=0x7f5198009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:40.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.243+0000 7f518ffff640 1 -- 172.21.15.38:0/2974519708 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f51a009c220 con 0x7f51a8107900 2024-05-21T15:34:40.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.473+0000 7f51ad414640 1 -- 172.21.15.38:0/2974519708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51a802d2a0 con 0x7f51a8107900 2024-05-21T15:34:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.473+0000 7f518ffff640 1 -- 172.21.15.38:0/2974519708 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f51a009bb20 con 0x7f51a8107900 2024-05-21T15:34:40.473 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:40.473 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:34:36.440701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24551,24523,24602,24550,24515]},"id":1}]} 2024-05-21T15:34:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-05-21T15:34:40.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.475+0000 7f51ad414640 1 -- 172.21.15.38:0/2974519708 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51900a2c30 msgr2=0x7f51900a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.475+0000 7f51ad414640 1 --2- 172.21.15.38:0/2974519708 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51900a2c30 0x7f51900a5140 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f51a81025c0 tx=0x7f5198009720 comp rx=0 tx=0).stop 2024-05-21T15:34:40.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.475+0000 7f51ad414640 1 -- 172.21.15.38:0/2974519708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a8107900 msgr2=0x7f51a81a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.475+0000 7f51ad414640 1 --2- 172.21.15.38:0/2974519708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a8107900 0x7f51a81a06e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f51a000e9b0 tx=0x7f51a000ee70 comp rx=0 tx=0).stop 2024-05-21T15:34:40.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.475+0000 7f51ad414640 1 -- 172.21.15.38:0/2974519708 shutdown_connections 2024-05-21T15:34:40.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.475+0000 7f51ad414640 1 --2- 172.21.15.38:0/2974519708 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51900a2c30 0x7f51900a5140 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.475+0000 7f51ad414640 1 --2- 172.21.15.38:0/2974519708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51a8107900 0x7f51a81a06e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.475+0000 7f51ad414640 1 --2- 172.21.15.38:0/2974519708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51a8102c00 0x7f51a81997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.475+0000 7f51ad414640 1 --2- 172.21.15.38:0/2974519708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a81019c0 0x7f51a8199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.475+0000 7f51ad414640 1 -- 172.21.15.38:0/2974519708 >> 172.21.15.38:0/2974519708 conn(0x7f51a80fcbb0 msgr2=0x7f51a81038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:40.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.476+0000 7f51ad414640 1 -- 172.21.15.38:0/2974519708 shutdown_connections 2024-05-21T15:34:40.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.476+0000 7f51ad414640 1 -- 172.21.15.38:0/2974519708 wait complete. 2024-05-21T15:34:40.486 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:40 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3375374255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:40.486 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:40 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/672264610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:40.486 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:40 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2259230639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:40.487 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-21T15:34:40.487 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-21T15:34:40.487 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T15:34:40.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.603+0000 7f46cf4e2640 1 -- 172.21.15.38:0/3248324779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c8102bf0 msgr2=0x7f46c8103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.603+0000 7f46cf4e2640 1 --2- 172.21.15.38:0/3248324779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c8102bf0 0x7f46c8103000 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009900 tx=0x7f46b802a410 comp rx=0 tx=0).stop 2024-05-21T15:34:40.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.603+0000 7f46cf4e2640 1 -- 172.21.15.38:0/3248324779 shutdown_connections 2024-05-21T15:34:40.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.603+0000 7f46cf4e2640 1 --2- 172.21.15.38:0/3248324779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c81078f0 0x7f46c8109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.603+0000 7f46cf4e2640 1 --2- 172.21.15.38:0/3248324779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c8102bf0 0x7f46c8103000 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.603+0000 7f46cf4e2640 1 --2- 172.21.15.38:0/3248324779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46c81019b0 0x7f46c81026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.603+0000 7f46cf4e2640 1 -- 172.21.15.38:0/3248324779 >> 172.21.15.38:0/3248324779 conn(0x7f46c80fcba0 msgr2=0x7f46c80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:40.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.603+0000 7f46cf4e2640 1 -- 172.21.15.38:0/3248324779 shutdown_connections 2024-05-21T15:34:40.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.603+0000 7f46cf4e2640 1 -- 172.21.15.38:0/3248324779 wait complete. 2024-05-21T15:34:40.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.603+0000 7f46cf4e2640 1 Processor -- start 2024-05-21T15:34:40.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.603+0000 7f46cf4e2640 1 -- start start 2024-05-21T15:34:40.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cf4e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46c81019b0 0x7f46c81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:40.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cf4e2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c8102bf0 0x7f46c8199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:40.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cf4e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c81078f0 0x7f46c81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:40.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cf4e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f46c810e730 con 0x7f46c81078f0 2024-05-21T15:34:40.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cf4e2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f46c810e5b0 con 0x7f46c8102bf0 2024-05-21T15:34:40.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cf4e2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f46c810e8b0 con 0x7f46c81019b0 2024-05-21T15:34:40.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cda7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c81078f0 0x7f46c81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:40.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cda7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c81078f0 0x7f46c81a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59960/0 (socket says 172.21.15.38:59960) 2024-05-21T15:34:40.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cda7f640 1 -- 172.21.15.38:0/1135421191 learned_addr learned my addr 172.21.15.38:0/1135421191 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:40.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cca7d640 1 --2- 172.21.15.38:0/1135421191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c8102bf0 0x7f46c8199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:40.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cd27e640 1 --2- 172.21.15.38:0/1135421191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46c81019b0 0x7f46c81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:40.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cda7f640 1 -- 172.21.15.38:0/1135421191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46c81019b0 msgr2=0x7f46c81991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cda7f640 1 --2- 172.21.15.38:0/1135421191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46c81019b0 0x7f46c81991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cda7f640 1 -- 172.21.15.38:0/1135421191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c8102bf0 msgr2=0x7f46c8199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cda7f640 1 --2- 172.21.15.38:0/1135421191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c8102bf0 0x7f46c8199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cda7f640 1 -- 172.21.15.38:0/1135421191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c81a0d00 con 0x7f46c81078f0 2024-05-21T15:34:40.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46cda7f640 1 --2- 172.21.15.38:0/1135421191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c81078f0 0x7f46c81a0690 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f46c000eaa0 tx=0x7f46c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:40.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46be7fc640 1 -- 172.21.15.38:0/1135421191 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c000cdb0 con 0x7f46c81078f0 2024-05-21T15:34:40.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.604+0000 7f46be7fc640 1 -- 172.21.15.38:0/1135421191 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c0005020 con 0x7f46c81078f0 2024-05-21T15:34:40.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.605+0000 7f46cf4e2640 1 -- 172.21.15.38:0/1135421191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c81a0ff0 con 0x7f46c81078f0 2024-05-21T15:34:40.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.605+0000 7f46be7fc640 1 -- 172.21.15.38:0/1135421191 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c00116b0 con 0x7f46c81078f0 2024-05-21T15:34:40.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.605+0000 7f46cf4e2640 1 -- 172.21.15.38:0/1135421191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c81a1400 con 0x7f46c81078f0 2024-05-21T15:34:40.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.605+0000 7f46cf4e2640 1 -- 172.21.15.38:0/1135421191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c80691f0 con 0x7f46c81078f0 2024-05-21T15:34:40.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.607+0000 7f46be7fc640 1 -- 172.21.15.38:0/1135421191 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f46c0005560 con 0x7f46c81078f0 2024-05-21T15:34:40.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.608+0000 7f46be7fc640 1 --2- 172.21.15.38:0/1135421191 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f46b40a2c30 0x7f46b40a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:40.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.608+0000 7f46cd27e640 1 --2- 172.21.15.38:0/1135421191 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f46b40a2c30 0x7f46b40a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:40.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.608+0000 7f46be7fc640 1 -- 172.21.15.38:0/1135421191 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f46c00cf620 con 0x7f46c81078f0 2024-05-21T15:34:40.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.608+0000 7f46cd27e640 1 --2- 172.21.15.38:0/1135421191 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f46b40a2c30 0x7f46b40a5140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f46c81025b0 tx=0x7f46c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:40.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.611+0000 7f46be7fc640 1 -- 172.21.15.38:0/1135421191 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f46c80691f0 con 0x7f46c81078f0 2024-05-21T15:34:40.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:40 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3375374255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:40.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:40 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/672264610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:40.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:40 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2259230639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:40.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:40 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3375374255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:40.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:40 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/672264610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:40.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:40 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2259230639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:40.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.793+0000 7f46cf4e2640 1 -- 172.21.15.38:0/1135421191 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46c8061f90 con 0x7f46b40a2c30 2024-05-21T15:34:40.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.800+0000 7f46be7fc640 1 -- 172.21.15.38:0/1135421191 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259204 (secure 0 0 0) 0x7f46c8061f90 con 0x7f46b40a2c30 2024-05-21T15:34:40.802 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:40.804 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-05-21T15:34:40.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.803+0000 7f46cf4e2640 1 -- 172.21.15.38:0/1135421191 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f46b40a2c30 msgr2=0x7f46b40a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.803+0000 7f46cf4e2640 1 --2- 172.21.15.38:0/1135421191 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f46b40a2c30 0x7f46b40a5140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f46c81025b0 tx=0x7f46c4005f90 comp rx=0 tx=0).stop 2024-05-21T15:34:40.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.803+0000 7f46cf4e2640 1 -- 172.21.15.38:0/1135421191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c81078f0 msgr2=0x7f46c81a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.803+0000 7f46cf4e2640 1 --2- 172.21.15.38:0/1135421191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c81078f0 0x7f46c81a0690 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f46c000eaa0 tx=0x7f46c000ef60 comp rx=0 tx=0).stop 2024-05-21T15:34:40.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.803+0000 7f46cf4e2640 1 -- 172.21.15.38:0/1135421191 shutdown_connections 2024-05-21T15:34:40.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.803+0000 7f46cf4e2640 1 --2- 172.21.15.38:0/1135421191 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f46b40a2c30 0x7f46b40a5140 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.803+0000 7f46cf4e2640 1 --2- 172.21.15.38:0/1135421191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c81078f0 0x7f46c81a0690 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.803+0000 7f46cf4e2640 1 --2- 172.21.15.38:0/1135421191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c8102bf0 0x7f46c8199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.803+0000 7f46cf4e2640 1 --2- 172.21.15.38:0/1135421191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46c81019b0 0x7f46c81991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.803+0000 7f46cf4e2640 1 -- 172.21.15.38:0/1135421191 >> 172.21.15.38:0/1135421191 conn(0x7f46c80fcba0 msgr2=0x7f46c81038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:40.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.803+0000 7f46cf4e2640 1 -- 172.21.15.38:0/1135421191 shutdown_connections 2024-05-21T15:34:40.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.803+0000 7f46cf4e2640 1 -- 172.21.15.38:0/1135421191 wait complete. 2024-05-21T15:34:40.814 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-05-21T15:34:40.249259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657578,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1332,"num_read_kb":784,"num_write":429,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180938,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":852,"ondisk_log_size":852,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373788,"kb_used_data":12088,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568036,"statfs":{"total":1151940427776,"available":1151557668864,"internally_reserved":0,"allocated":12378112,"data_stored":7688251,"data_compressed":254358,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":51,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003152"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.020232+0000","last_change":"2024-05-21T15:32:33.687244+0000","last_active":"2024-05-21T15:34:24.020232+0000","last_peered":"2024-05-21T15:34:24.020232+0000","last_clean":"2024-05-21T15:34:24.020232+0000","last_became_active":"2024-05-21T15:32:33.686515+0000","last_became_peered":"2024-05-21T15:32:33.686515+0000","last_unstale":"2024-05-21T15:34:24.020232+0000","last_undegraded":"2024-05-21T15:34:24.020232+0000","last_fullsized":"2024-05-21T15:34:24.020232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:10:18.396997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143144+0000","last_change":"2024-05-21T15:32:33.678497+0000","last_active":"2024-05-21T15:34:24.143144+0000","last_peered":"2024-05-21T15:34:24.143144+0000","last_clean":"2024-05-21T15:34:24.143144+0000","last_became_active":"2024-05-21T15:32:33.678001+0000","last_became_peered":"2024-05-21T15:32:33.678001+0000","last_unstale":"2024-05-21T15:34:24.143144+0000","last_undegraded":"2024-05-21T15:34:24.143144+0000","last_fullsized":"2024-05-21T15:34:24.143144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:14:48.255675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143214+0000","last_change":"2024-05-21T15:32:33.692444+0000","last_active":"2024-05-21T15:34:24.143214+0000","last_peered":"2024-05-21T15:34:24.143214+0000","last_clean":"2024-05-21T15:34:24.143214+0000","last_became_active":"2024-05-21T15:32:33.691662+0000","last_became_peered":"2024-05-21T15:32:33.691662+0000","last_unstale":"2024-05-21T15:34:24.143214+0000","last_undegraded":"2024-05-21T15:34:24.143214+0000","last_fullsized":"2024-05-21T15:34:24.143214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:26:25.929879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143818+0000","last_change":"2024-05-21T15:32:33.692297+0000","last_active":"2024-05-21T15:34:24.143818+0000","last_peered":"2024-05-21T15:34:24.143818+0000","last_clean":"2024-05-21T15:34:24.143818+0000","last_became_active":"2024-05-21T15:32:33.691925+0000","last_became_peered":"2024-05-21T15:32:33.691925+0000","last_unstale":"2024-05-21T15:34:24.143818+0000","last_undegraded":"2024-05-21T15:34:24.143818+0000","last_fullsized":"2024-05-21T15:34:24.143818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:40:45.448485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021213+0000","last_change":"2024-05-21T15:32:33.685511+0000","last_active":"2024-05-21T15:34:24.021213+0000","last_peered":"2024-05-21T15:34:24.021213+0000","last_clean":"2024-05-21T15:34:24.021213+0000","last_became_active":"2024-05-21T15:32:33.685182+0000","last_became_peered":"2024-05-21T15:32:33.685182+0000","last_unstale":"2024-05-21T15:34:24.021213+0000","last_undegraded":"2024-05-21T15:34:24.021213+0000","last_fullsized":"2024-05-21T15:34:24.021213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:59:06.680712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019738+0000","last_change":"2024-05-21T15:32:33.682400+0000","last_active":"2024-05-21T15:34:24.019738+0000","last_peered":"2024-05-21T15:34:24.019738+0000","last_clean":"2024-05-21T15:34:24.019738+0000","last_became_active":"2024-05-21T15:32:33.682068+0000","last_became_peered":"2024-05-21T15:32:33.682068+0000","last_unstale":"2024-05-21T15:34:24.019738+0000","last_undegraded":"2024-05-21T15:34:24.019738+0000","last_fullsized":"2024-05-21T15:34:24.019738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:31:50.968706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021817+0000","last_change":"2024-05-21T15:32:33.690099+0000","last_active":"2024-05-21T15:34:24.021817+0000","last_peered":"2024-05-21T15:34:24.021817+0000","last_clean":"2024-05-21T15:34:24.021817+0000","last_became_active":"2024-05-21T15:32:33.689762+0000","last_became_peered":"2024-05-21T15:32:33.689762+0000","last_unstale":"2024-05-21T15:34:24.021817+0000","last_undegraded":"2024-05-21T15:34:24.021817+0000","last_fullsized":"2024-05-21T15:34:24.021817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:55:20.331097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021396+0000","last_change":"2024-05-21T15:32:33.691701+0000","last_active":"2024-05-21T15:34:24.021396+0000","last_peered":"2024-05-21T15:34:24.021396+0000","last_clean":"2024-05-21T15:34:24.021396+0000","last_became_active":"2024-05-21T15:32:33.691386+0000","last_became_peered":"2024-05-21T15:32:33.691386+0000","last_unstale":"2024-05-21T15:34:24.021396+0000","last_undegraded":"2024-05-21T15:34:24.021396+0000","last_fullsized":"2024-05-21T15:34:24.021396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:54:45.691392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.018266+0000","last_change":"2024-05-21T15:32:33.677484+0000","last_active":"2024-05-21T15:34:24.018266+0000","last_peered":"2024-05-21T15:34:24.018266+0000","last_clean":"2024-05-21T15:34:24.018266+0000","last_became_active":"2024-05-21T15:32:33.677160+0000","last_became_peered":"2024-05-21T15:32:33.677160+0000","last_unstale":"2024-05-21T15:34:24.018266+0000","last_undegraded":"2024-05-21T15:34:24.018266+0000","last_fullsized":"2024-05-21T15:34:24.018266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:38:00.914393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019574+0000","last_change":"2024-05-21T15:32:33.686957+0000","last_active":"2024-05-21T15:34:24.019574+0000","last_peered":"2024-05-21T15:34:24.019574+0000","last_clean":"2024-05-21T15:34:24.019574+0000","last_became_active":"2024-05-21T15:32:33.685338+0000","last_became_peered":"2024-05-21T15:32:33.685338+0000","last_unstale":"2024-05-21T15:34:24.019574+0000","last_undegraded":"2024-05-21T15:34:24.019574+0000","last_fullsized":"2024-05-21T15:34:24.019574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:04:58.115974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022284+0000","last_change":"2024-05-21T15:32:33.693311+0000","last_active":"2024-05-21T15:34:24.022284+0000","last_peered":"2024-05-21T15:34:24.022284+0000","last_clean":"2024-05-21T15:34:24.022284+0000","last_became_active":"2024-05-21T15:32:33.692959+0000","last_became_peered":"2024-05-21T15:32:33.692959+0000","last_unstale":"2024-05-21T15:34:24.022284+0000","last_undegraded":"2024-05-21T15:34:24.022284+0000","last_fullsized":"2024-05-21T15:34:24.022284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:35:12.918524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143679+0000","last_change":"2024-05-21T15:32:33.693254+0000","last_active":"2024-05-21T15:34:24.143679+0000","last_peered":"2024-05-21T15:34:24.143679+0000","last_clean":"2024-05-21T15:34:24.143679+0000","last_became_active":"2024-05-21T15:32:33.692901+0000","last_became_peered":"2024-05-21T15:32:33.692901+0000","last_unstale":"2024-05-21T15:34:24.143679+0000","last_undegraded":"2024-05-21T15:34:24.143679+0000","last_fullsized":"2024-05-21T15:34:24.143679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:36:30.591479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016565+0000","last_change":"2024-05-21T15:33:23.383656+0000","last_active":"2024-05-21T15:34:24.016565+0000","last_peered":"2024-05-21T15:34:24.016565+0000","last_clean":"2024-05-21T15:34:24.016565+0000","last_became_active":"2024-05-21T15:33:23.383113+0000","last_became_peered":"2024-05-21T15:33:23.383113+0000","last_unstale":"2024-05-21T15:34:24.016565+0000","last_undegraded":"2024-05-21T15:34:24.016565+0000","last_fullsized":"2024-05-21T15:34:24.016565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:14:37.327480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142760+0000","last_change":"2024-05-21T15:32:33.678315+0000","last_active":"2024-05-21T15:34:24.142760+0000","last_peered":"2024-05-21T15:34:24.142760+0000","last_clean":"2024-05-21T15:34:24.142760+0000","last_became_active":"2024-05-21T15:32:33.677787+0000","last_became_peered":"2024-05-21T15:32:33.677787+0000","last_unstale":"2024-05-21T15:34:24.142760+0000","last_undegraded":"2024-05-21T15:34:24.142760+0000","last_fullsized":"2024-05-21T15:34:24.142760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:17:24.826525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142879+0000","last_change":"2024-05-21T15:33:23.380244+0000","last_active":"2024-05-21T15:34:24.142879+0000","last_peered":"2024-05-21T15:34:24.142879+0000","last_clean":"2024-05-21T15:34:24.142879+0000","last_became_active":"2024-05-21T15:33:23.379769+0000","last_became_peered":"2024-05-21T15:33:23.379769+0000","last_unstale":"2024-05-21T15:34:24.142879+0000","last_undegraded":"2024-05-21T15:34:24.142879+0000","last_fullsized":"2024-05-21T15:34:24.142879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:16:07.820636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017488+0000","last_change":"2024-05-21T15:32:33.683597+0000","last_active":"2024-05-21T15:34:24.017488+0000","last_peered":"2024-05-21T15:34:24.017488+0000","last_clean":"2024-05-21T15:34:24.017488+0000","last_became_active":"2024-05-21T15:32:33.681948+0000","last_became_peered":"2024-05-21T15:32:33.681948+0000","last_unstale":"2024-05-21T15:34:24.017488+0000","last_undegraded":"2024-05-21T15:34:24.017488+0000","last_fullsized":"2024-05-21T15:34:24.017488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:26:50.086019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.020426+0000","last_change":"2024-05-21T15:32:33.683329+0000","last_active":"2024-05-21T15:34:24.020426+0000","last_peered":"2024-05-21T15:34:24.020426+0000","last_clean":"2024-05-21T15:34:24.020426+0000","last_became_active":"2024-05-21T15:32:33.682947+0000","last_became_peered":"2024-05-21T15:32:33.682947+0000","last_unstale":"2024-05-21T15:34:24.020426+0000","last_undegraded":"2024-05-21T15:34:24.020426+0000","last_fullsized":"2024-05-21T15:34:24.020426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:24:27.502257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142833+0000","last_change":"2024-05-21T15:32:33.680391+0000","last_active":"2024-05-21T15:34:24.142833+0000","last_peered":"2024-05-21T15:34:24.142833+0000","last_clean":"2024-05-21T15:34:24.142833+0000","last_became_active":"2024-05-21T15:32:33.679996+0000","last_became_peered":"2024-05-21T15:32:33.679996+0000","last_unstale":"2024-05-21T15:34:24.142833+0000","last_undegraded":"2024-05-21T15:34:24.142833+0000","last_fullsized":"2024-05-21T15:34:24.142833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:56:43.658565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142946+0000","last_change":"2024-05-21T15:32:33.688783+0000","last_active":"2024-05-21T15:34:24.142946+0000","last_peered":"2024-05-21T15:34:24.142946+0000","last_clean":"2024-05-21T15:34:24.142946+0000","last_became_active":"2024-05-21T15:32:33.688006+0000","last_became_peered":"2024-05-21T15:32:33.688006+0000","last_unstale":"2024-05-21T15:34:24.142946+0000","last_undegraded":"2024-05-21T15:34:24.142946+0000","last_fullsized":"2024-05-21T15:34:24.142946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:17:44.489759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":155,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:37.162842+0000","last_change":"2024-05-21T15:32:33.182251+0000","last_active":"2024-05-21T15:34:37.162842+0000","last_peered":"2024-05-21T15:34:37.162842+0000","last_clean":"2024-05-21T15:34:37.162842+0000","last_became_active":"2024-05-21T15:32:33.181893+0000","last_became_peered":"2024-05-21T15:32:33.181893+0000","last_unstale":"2024-05-21T15:34:37.162842+0000","last_undegraded":"2024-05-21T15:34:37.162842+0000","last_fullsized":"2024-05-21T15:34:37.162842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:23:36.863186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021774+0000","last_change":"2024-05-21T15:32:33.691962+0000","last_active":"2024-05-21T15:34:24.021774+0000","last_peered":"2024-05-21T15:34:24.021774+0000","last_clean":"2024-05-21T15:34:24.021774+0000","last_became_active":"2024-05-21T15:32:33.691619+0000","last_became_peered":"2024-05-21T15:32:33.691619+0000","last_unstale":"2024-05-21T15:34:24.021774+0000","last_undegraded":"2024-05-21T15:34:24.021774+0000","last_fullsized":"2024-05-21T15:34:24.021774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:50:14.523567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022035+0000","last_change":"2024-05-21T15:32:33.183772+0000","last_active":"2024-05-21T15:34:24.022035+0000","last_peered":"2024-05-21T15:34:24.022035+0000","last_clean":"2024-05-21T15:34:24.022035+0000","last_became_active":"2024-05-21T15:32:33.183440+0000","last_became_peered":"2024-05-21T15:32:33.183440+0000","last_unstale":"2024-05-21T15:34:24.022035+0000","last_undegraded":"2024-05-21T15:34:24.022035+0000","last_fullsized":"2024-05-21T15:34:24.022035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:53:00.714359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143490+0000","last_change":"2024-05-21T15:32:33.682544+0000","last_active":"2024-05-21T15:34:24.143490+0000","last_peered":"2024-05-21T15:34:24.143490+0000","last_clean":"2024-05-21T15:34:24.143490+0000","last_became_active":"2024-05-21T15:32:33.682161+0000","last_became_peered":"2024-05-21T15:32:33.682161+0000","last_unstale":"2024-05-21T15:34:24.143490+0000","last_undegraded":"2024-05-21T15:34:24.143490+0000","last_fullsized":"2024-05-21T15:34:24.143490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:20:38.143508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021961+0000","last_change":"2024-05-21T15:32:33.182400+0000","last_active":"2024-05-21T15:34:24.021961+0000","last_peered":"2024-05-21T15:34:24.021961+0000","last_clean":"2024-05-21T15:34:24.021961+0000","last_became_active":"2024-05-21T15:32:33.181745+0000","last_became_peered":"2024-05-21T15:32:33.181745+0000","last_unstale":"2024-05-21T15:34:24.021961+0000","last_undegraded":"2024-05-21T15:34:24.021961+0000","last_fullsized":"2024-05-21T15:34:24.021961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:39:41.669529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017631+0000","last_change":"2024-05-21T15:32:33.682700+0000","last_active":"2024-05-21T15:34:24.017631+0000","last_peered":"2024-05-21T15:34:24.017631+0000","last_clean":"2024-05-21T15:34:24.017631+0000","last_became_active":"2024-05-21T15:32:33.680876+0000","last_became_peered":"2024-05-21T15:32:33.680876+0000","last_unstale":"2024-05-21T15:34:24.017631+0000","last_undegraded":"2024-05-21T15:34:24.017631+0000","last_fullsized":"2024-05-21T15:34:24.017631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:30.694520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021728+0000","last_change":"2024-05-21T15:32:33.179114+0000","last_active":"2024-05-21T15:34:24.021728+0000","last_peered":"2024-05-21T15:34:24.021728+0000","last_clean":"2024-05-21T15:34:24.021728+0000","last_became_active":"2024-05-21T15:32:33.178485+0000","last_became_peered":"2024-05-21T15:32:33.178485+0000","last_unstale":"2024-05-21T15:34:24.021728+0000","last_undegraded":"2024-05-21T15:34:24.021728+0000","last_fullsized":"2024-05-21T15:34:24.021728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:19:56.501287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017868+0000","last_change":"2024-05-21T15:32:33.691874+0000","last_active":"2024-05-21T15:34:24.017868+0000","last_peered":"2024-05-21T15:34:24.017868+0000","last_clean":"2024-05-21T15:34:24.017868+0000","last_became_active":"2024-05-21T15:32:33.691546+0000","last_became_peered":"2024-05-21T15:32:33.691546+0000","last_unstale":"2024-05-21T15:34:24.017868+0000","last_undegraded":"2024-05-21T15:34:24.017868+0000","last_fullsized":"2024-05-21T15:34:24.017868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:40:09.281792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:35.164975+0000","last_change":"2024-05-21T15:32:33.183457+0000","last_active":"2024-05-21T15:34:35.164975+0000","last_peered":"2024-05-21T15:34:35.164975+0000","last_clean":"2024-05-21T15:34:35.164975+0000","last_became_active":"2024-05-21T15:32:33.183127+0000","last_became_peered":"2024-05-21T15:32:33.183127+0000","last_unstale":"2024-05-21T15:34:35.164975+0000","last_undegraded":"2024-05-21T15:34:35.164975+0000","last_fullsized":"2024-05-21T15:34:35.164975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:41:05.179629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021554+0000","last_change":"2024-05-21T15:32:33.677260+0000","last_active":"2024-05-21T15:34:24.021554+0000","last_peered":"2024-05-21T15:34:24.021554+0000","last_clean":"2024-05-21T15:34:24.021554+0000","last_became_active":"2024-05-21T15:32:33.676905+0000","last_became_peered":"2024-05-21T15:32:33.676905+0000","last_unstale":"2024-05-21T15:34:24.021554+0000","last_undegraded":"2024-05-21T15:34:24.021554+0000","last_fullsized":"2024-05-21T15:34:24.021554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:51:41.543066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.150318+0000","last_change":"2024-05-21T15:34:24.150318+0000","last_active":"2024-05-21T15:34:24.150318+0000","last_peered":"2024-05-21T15:34:24.150318+0000","last_clean":"2024-05-21T15:34:24.150318+0000","last_became_active":"2024-05-21T15:34:24.144598+0000","last_became_peered":"2024-05-21T15:34:24.144598+0000","last_unstale":"2024-05-21T15:34:24.150318+0000","last_undegraded":"2024-05-21T15:34:24.150318+0000","last_fullsized":"2024-05-21T15:34:24.150318+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:12:00.320432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017867+0000","last_change":"2024-05-21T15:32:33.691072+0000","last_active":"2024-05-21T15:34:24.017867+0000","last_peered":"2024-05-21T15:34:24.017867+0000","last_clean":"2024-05-21T15:34:24.017867+0000","last_became_active":"2024-05-21T15:32:33.690663+0000","last_became_peered":"2024-05-21T15:32:33.690663+0000","last_unstale":"2024-05-21T15:34:24.017867+0000","last_undegraded":"2024-05-21T15:34:24.017867+0000","last_fullsized":"2024-05-21T15:34:24.017867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:52:44.549537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016485+0000","last_change":"2024-05-21T15:32:33.179588+0000","last_active":"2024-05-21T15:34:24.016485+0000","last_peered":"2024-05-21T15:34:24.016485+0000","last_clean":"2024-05-21T15:34:24.016485+0000","last_became_active":"2024-05-21T15:32:33.179115+0000","last_became_peered":"2024-05-21T15:32:33.179115+0000","last_unstale":"2024-05-21T15:34:24.016485+0000","last_undegraded":"2024-05-21T15:34:24.016485+0000","last_fullsized":"2024-05-21T15:34:24.016485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:18:43.904584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021488+0000","last_change":"2024-05-21T15:32:33.689456+0000","last_active":"2024-05-21T15:34:24.021488+0000","last_peered":"2024-05-21T15:34:24.021488+0000","last_clean":"2024-05-21T15:34:24.021488+0000","last_became_active":"2024-05-21T15:32:33.689142+0000","last_became_peered":"2024-05-21T15:32:33.689142+0000","last_unstale":"2024-05-21T15:34:24.021488+0000","last_undegraded":"2024-05-21T15:34:24.021488+0000","last_fullsized":"2024-05-21T15:34:24.021488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:00:13.768272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017507+0000","last_change":"2024-05-21T15:32:33.184865+0000","last_active":"2024-05-21T15:34:24.017507+0000","last_peered":"2024-05-21T15:34:24.017507+0000","last_clean":"2024-05-21T15:34:24.017507+0000","last_became_active":"2024-05-21T15:32:33.184540+0000","last_became_peered":"2024-05-21T15:32:33.184540+0000","last_unstale":"2024-05-21T15:34:24.017507+0000","last_undegraded":"2024-05-21T15:34:24.017507+0000","last_fullsized":"2024-05-21T15:34:24.017507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:09:20.072906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142492+0000","last_change":"2024-05-21T15:32:33.685478+0000","last_active":"2024-05-21T15:34:24.142492+0000","last_peered":"2024-05-21T15:34:24.142492+0000","last_clean":"2024-05-21T15:34:24.142492+0000","last_became_active":"2024-05-21T15:32:33.685137+0000","last_became_peered":"2024-05-21T15:32:33.685137+0000","last_unstale":"2024-05-21T15:34:24.142492+0000","last_undegraded":"2024-05-21T15:34:24.142492+0000","last_fullsized":"2024-05-21T15:34:24.142492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:39:40.815876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":262,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:37.167571+0000","last_change":"2024-05-21T15:32:33.179215+0000","last_active":"2024-05-21T15:34:37.167571+0000","last_peered":"2024-05-21T15:34:37.167571+0000","last_clean":"2024-05-21T15:34:37.167571+0000","last_became_active":"2024-05-21T15:32:33.178846+0000","last_became_peered":"2024-05-21T15:32:33.178846+0000","last_unstale":"2024-05-21T15:34:37.167571+0000","last_undegraded":"2024-05-21T15:34:37.167571+0000","last_fullsized":"2024-05-21T15:34:37.167571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:29:47.289968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":234,"num_read_kb":234,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.018136+0000","last_change":"2024-05-21T15:32:33.684900+0000","last_active":"2024-05-21T15:34:24.018136+0000","last_peered":"2024-05-21T15:34:24.018136+0000","last_clean":"2024-05-21T15:34:24.018136+0000","last_became_active":"2024-05-21T15:32:33.681374+0000","last_became_peered":"2024-05-21T15:32:33.681374+0000","last_unstale":"2024-05-21T15:34:24.018136+0000","last_undegraded":"2024-05-21T15:34:24.018136+0000","last_fullsized":"2024-05-21T15:34:24.018136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:51:38.079726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022157+0000","last_change":"2024-05-21T15:32:33.189945+0000","last_active":"2024-05-21T15:34:24.022157+0000","last_peered":"2024-05-21T15:34:24.022157+0000","last_clean":"2024-05-21T15:34:24.022157+0000","last_became_active":"2024-05-21T15:32:33.189557+0000","last_became_peered":"2024-05-21T15:32:33.189557+0000","last_unstale":"2024-05-21T15:34:24.022157+0000","last_undegraded":"2024-05-21T15:34:24.022157+0000","last_fullsized":"2024-05-21T15:34:24.022157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:08.427043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143483+0000","last_change":"2024-05-21T15:32:33.692896+0000","last_active":"2024-05-21T15:34:24.143483+0000","last_peered":"2024-05-21T15:34:24.143483+0000","last_clean":"2024-05-21T15:34:24.143483+0000","last_became_active":"2024-05-21T15:32:33.692527+0000","last_became_peered":"2024-05-21T15:32:33.692527+0000","last_unstale":"2024-05-21T15:34:24.143483+0000","last_undegraded":"2024-05-21T15:34:24.143483+0000","last_fullsized":"2024-05-21T15:34:24.143483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:45:45.418063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017955+0000","last_change":"2024-05-21T15:32:33.172989+0000","last_active":"2024-05-21T15:34:24.017955+0000","last_peered":"2024-05-21T15:34:24.017955+0000","last_clean":"2024-05-21T15:34:24.017955+0000","last_became_active":"2024-05-21T15:32:33.172215+0000","last_became_peered":"2024-05-21T15:32:33.172215+0000","last_unstale":"2024-05-21T15:34:24.017955+0000","last_undegraded":"2024-05-21T15:34:24.017955+0000","last_fullsized":"2024-05-21T15:34:24.017955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:09:39.287636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017840+0000","last_change":"2024-05-21T15:32:33.678441+0000","last_active":"2024-05-21T15:34:24.017840+0000","last_peered":"2024-05-21T15:34:24.017840+0000","last_clean":"2024-05-21T15:34:24.017840+0000","last_became_active":"2024-05-21T15:32:33.677833+0000","last_became_peered":"2024-05-21T15:32:33.677833+0000","last_unstale":"2024-05-21T15:34:24.017840+0000","last_undegraded":"2024-05-21T15:34:24.017840+0000","last_fullsized":"2024-05-21T15:34:24.017840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:36.124064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'34","reported_seq":299,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:37.189944+0000","last_change":"2024-05-21T15:32:33.178153+0000","last_active":"2024-05-21T15:34:37.189944+0000","last_peered":"2024-05-21T15:34:37.189944+0000","last_clean":"2024-05-21T15:34:37.189944+0000","last_became_active":"2024-05-21T15:32:33.177792+0000","last_became_peered":"2024-05-21T15:32:33.177792+0000","last_unstale":"2024-05-21T15:34:37.189944+0000","last_undegraded":"2024-05-21T15:34:37.189944+0000","last_fullsized":"2024-05-21T15:34:37.189944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:43:41.865204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60022,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":242,"num_read_kb":180,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021430+0000","last_change":"2024-05-21T15:32:33.681328+0000","last_active":"2024-05-21T15:34:24.021430+0000","last_peered":"2024-05-21T15:34:24.021430+0000","last_clean":"2024-05-21T15:34:24.021430+0000","last_became_active":"2024-05-21T15:32:33.680975+0000","last_became_peered":"2024-05-21T15:32:33.680975+0000","last_unstale":"2024-05-21T15:34:24.021430+0000","last_undegraded":"2024-05-21T15:34:24.021430+0000","last_fullsized":"2024-05-21T15:34:24.021430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:50:27.199348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142791+0000","last_change":"2024-05-21T15:32:33.182492+0000","last_active":"2024-05-21T15:34:24.142791+0000","last_peered":"2024-05-21T15:34:24.142791+0000","last_clean":"2024-05-21T15:34:24.142791+0000","last_became_active":"2024-05-21T15:32:33.182152+0000","last_became_peered":"2024-05-21T15:32:33.182152+0000","last_unstale":"2024-05-21T15:34:24.142791+0000","last_undegraded":"2024-05-21T15:34:24.142791+0000","last_fullsized":"2024-05-21T15:34:24.142791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:04:07.432742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016470+0000","last_change":"2024-05-21T15:32:33.686623+0000","last_active":"2024-05-21T15:34:24.016470+0000","last_peered":"2024-05-21T15:34:24.016470+0000","last_clean":"2024-05-21T15:34:24.016470+0000","last_became_active":"2024-05-21T15:32:33.685816+0000","last_became_peered":"2024-05-21T15:32:33.685816+0000","last_unstale":"2024-05-21T15:34:24.016470+0000","last_undegraded":"2024-05-21T15:34:24.016470+0000","last_fullsized":"2024-05-21T15:34:24.016470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:15:20.354113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.029185+0000","last_change":"2024-05-21T15:34:24.028748+0000","last_active":"2024-05-21T15:34:24.029185+0000","last_peered":"2024-05-21T15:34:24.029185+0000","last_clean":"2024-05-21T15:34:24.029185+0000","last_became_active":"2024-05-21T15:34:24.022920+0000","last_became_peered":"2024-05-21T15:34:24.022920+0000","last_unstale":"2024-05-21T15:34:24.029185+0000","last_undegraded":"2024-05-21T15:34:24.029185+0000","last_fullsized":"2024-05-21T15:34:24.029185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:09:04.712896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021976+0000","last_change":"2024-05-21T15:32:33.689936+0000","last_active":"2024-05-21T15:34:24.021976+0000","last_peered":"2024-05-21T15:34:24.021976+0000","last_clean":"2024-05-21T15:34:24.021976+0000","last_became_active":"2024-05-21T15:32:33.689452+0000","last_became_peered":"2024-05-21T15:32:33.689452+0000","last_unstale":"2024-05-21T15:34:24.021976+0000","last_undegraded":"2024-05-21T15:34:24.021976+0000","last_fullsized":"2024-05-21T15:34:24.021976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:00.420730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:35.166161+0000","last_change":"2024-05-21T15:32:33.181354+0000","last_active":"2024-05-21T15:34:35.166161+0000","last_peered":"2024-05-21T15:34:35.166161+0000","last_clean":"2024-05-21T15:34:35.166161+0000","last_became_active":"2024-05-21T15:32:33.180983+0000","last_became_peered":"2024-05-21T15:32:33.180983+0000","last_unstale":"2024-05-21T15:34:35.166161+0000","last_undegraded":"2024-05-21T15:34:35.166161+0000","last_fullsized":"2024-05-21T15:34:35.166161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:18:46.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021643+0000","last_change":"2024-05-21T15:33:23.377305+0000","last_active":"2024-05-21T15:34:24.021643+0000","last_peered":"2024-05-21T15:34:24.021643+0000","last_clean":"2024-05-21T15:34:24.021643+0000","last_became_active":"2024-05-21T15:33:23.376921+0000","last_became_peered":"2024-05-21T15:33:23.376921+0000","last_unstale":"2024-05-21T15:34:24.021643+0000","last_undegraded":"2024-05-21T15:34:24.021643+0000","last_fullsized":"2024-05-21T15:34:24.021643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:19.874725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021379+0000","last_change":"2024-05-21T15:33:23.496401+0000","last_active":"2024-05-21T15:34:24.021379+0000","last_peered":"2024-05-21T15:34:24.021379+0000","last_clean":"2024-05-21T15:34:24.021379+0000","last_became_active":"2024-05-21T15:33:23.483584+0000","last_became_peered":"2024-05-21T15:33:23.483584+0000","last_unstale":"2024-05-21T15:34:24.021379+0000","last_undegraded":"2024-05-21T15:34:24.021379+0000","last_fullsized":"2024-05-21T15:34:24.021379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:35:32.411825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017462+0000","last_change":"2024-05-21T15:32:33.691112+0000","last_active":"2024-05-21T15:34:24.017462+0000","last_peered":"2024-05-21T15:34:24.017462+0000","last_clean":"2024-05-21T15:34:24.017462+0000","last_became_active":"2024-05-21T15:32:33.690726+0000","last_became_peered":"2024-05-21T15:32:33.690726+0000","last_unstale":"2024-05-21T15:34:24.017462+0000","last_undegraded":"2024-05-21T15:34:24.017462+0000","last_fullsized":"2024-05-21T15:34:24.017462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:59:29.002636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021659+0000","last_change":"2024-05-21T15:32:33.179565+0000","last_active":"2024-05-21T15:34:24.021659+0000","last_peered":"2024-05-21T15:34:24.021659+0000","last_clean":"2024-05-21T15:34:24.021659+0000","last_became_active":"2024-05-21T15:32:33.179168+0000","last_became_peered":"2024-05-21T15:32:33.179168+0000","last_unstale":"2024-05-21T15:34:24.021659+0000","last_undegraded":"2024-05-21T15:34:24.021659+0000","last_fullsized":"2024-05-21T15:34:24.021659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:23:39.210931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.020065+0000","last_change":"2024-05-21T15:32:33.691348+0000","last_active":"2024-05-21T15:34:24.020065+0000","last_peered":"2024-05-21T15:34:24.020065+0000","last_clean":"2024-05-21T15:34:24.020065+0000","last_became_active":"2024-05-21T15:32:33.691005+0000","last_became_peered":"2024-05-21T15:32:33.691005+0000","last_unstale":"2024-05-21T15:34:24.020065+0000","last_undegraded":"2024-05-21T15:34:24.020065+0000","last_fullsized":"2024-05-21T15:34:24.020065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:33:34.533026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142969+0000","last_change":"2024-05-21T15:32:33.184850+0000","last_active":"2024-05-21T15:34:24.142969+0000","last_peered":"2024-05-21T15:34:24.142969+0000","last_clean":"2024-05-21T15:34:24.142969+0000","last_became_active":"2024-05-21T15:32:33.184480+0000","last_became_peered":"2024-05-21T15:32:33.184480+0000","last_unstale":"2024-05-21T15:34:24.142969+0000","last_undegraded":"2024-05-21T15:34:24.142969+0000","last_fullsized":"2024-05-21T15:34:24.142969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:31.327820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019449+0000","last_change":"2024-05-21T15:32:33.689520+0000","last_active":"2024-05-21T15:34:24.019449+0000","last_peered":"2024-05-21T15:34:24.019449+0000","last_clean":"2024-05-21T15:34:24.019449+0000","last_became_active":"2024-05-21T15:32:33.688219+0000","last_became_peered":"2024-05-21T15:32:33.688219+0000","last_unstale":"2024-05-21T15:34:24.019449+0000","last_undegraded":"2024-05-21T15:34:24.019449+0000","last_fullsized":"2024-05-21T15:34:24.019449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:36.546390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:38.164983+0000","last_change":"2024-05-21T15:32:33.183183+0000","last_active":"2024-05-21T15:34:38.164983+0000","last_peered":"2024-05-21T15:34:38.164983+0000","last_clean":"2024-05-21T15:34:38.164983+0000","last_became_active":"2024-05-21T15:32:33.182753+0000","last_became_peered":"2024-05-21T15:32:33.182753+0000","last_unstale":"2024-05-21T15:34:38.164983+0000","last_undegraded":"2024-05-21T15:34:38.164983+0000","last_fullsized":"2024-05-21T15:34:38.164983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:44:22.509265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019138+0000","last_change":"2024-05-21T15:32:33.684019+0000","last_active":"2024-05-21T15:34:24.019138+0000","last_peered":"2024-05-21T15:34:24.019138+0000","last_clean":"2024-05-21T15:34:24.019138+0000","last_became_active":"2024-05-21T15:32:33.683276+0000","last_became_peered":"2024-05-21T15:32:33.683276+0000","last_unstale":"2024-05-21T15:34:24.019138+0000","last_undegraded":"2024-05-21T15:34:24.019138+0000","last_fullsized":"2024-05-21T15:34:24.019138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:07:14.947240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017746+0000","last_change":"2024-05-21T15:32:33.690956+0000","last_active":"2024-05-21T15:34:24.017746+0000","last_peered":"2024-05-21T15:34:24.017746+0000","last_clean":"2024-05-21T15:34:24.017746+0000","last_became_active":"2024-05-21T15:32:33.690630+0000","last_became_peered":"2024-05-21T15:32:33.690630+0000","last_unstale":"2024-05-21T15:34:24.017746+0000","last_undegraded":"2024-05-21T15:34:24.017746+0000","last_fullsized":"2024-05-21T15:34:24.017746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:24:07.050928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":147,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:37.168923+0000","last_change":"2024-05-21T15:32:33.175127+0000","last_active":"2024-05-21T15:34:37.168923+0000","last_peered":"2024-05-21T15:34:37.168923+0000","last_clean":"2024-05-21T15:34:37.168923+0000","last_became_active":"2024-05-21T15:32:33.174724+0000","last_became_peered":"2024-05-21T15:32:33.174724+0000","last_unstale":"2024-05-21T15:34:37.168923+0000","last_undegraded":"2024-05-21T15:34:37.168923+0000","last_fullsized":"2024-05-21T15:34:37.168923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:39:17.499743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021701+0000","last_change":"2024-05-21T15:32:33.693428+0000","last_active":"2024-05-21T15:34:24.021701+0000","last_peered":"2024-05-21T15:34:24.021701+0000","last_clean":"2024-05-21T15:34:24.021701+0000","last_became_active":"2024-05-21T15:32:33.693077+0000","last_became_peered":"2024-05-21T15:32:33.693077+0000","last_unstale":"2024-05-21T15:34:24.021701+0000","last_undegraded":"2024-05-21T15:34:24.021701+0000","last_fullsized":"2024-05-21T15:34:24.021701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:08:11.908874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143504+0000","last_change":"2024-05-21T15:32:33.176267+0000","last_active":"2024-05-21T15:34:24.143504+0000","last_peered":"2024-05-21T15:34:24.143504+0000","last_clean":"2024-05-21T15:34:24.143504+0000","last_became_active":"2024-05-21T15:32:33.175856+0000","last_became_peered":"2024-05-21T15:32:33.175856+0000","last_unstale":"2024-05-21T15:34:24.143504+0000","last_undegraded":"2024-05-21T15:34:24.143504+0000","last_fullsized":"2024-05-21T15:34:24.143504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:58.113791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021740+0000","last_change":"2024-05-21T15:31:20.928694+0000","last_active":"2024-05-21T15:34:24.021740+0000","last_peered":"2024-05-21T15:34:24.021740+0000","last_clean":"2024-05-21T15:34:24.021740+0000","last_became_active":"2024-05-21T15:31:20.928115+0000","last_became_peered":"2024-05-21T15:31:20.928115+0000","last_unstale":"2024-05-21T15:34:24.021740+0000","last_undegraded":"2024-05-21T15:34:24.021740+0000","last_fullsized":"2024-05-21T15:34:24.021740+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:27:50.788960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:27:50.788960+0000","last_clean_scrub_stamp":"2024-05-21T15:27:50.788960+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:55:10.393374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016306+0000","last_change":"2024-05-21T15:32:33.689305+0000","last_active":"2024-05-21T15:34:24.016306+0000","last_peered":"2024-05-21T15:34:24.016306+0000","last_clean":"2024-05-21T15:34:24.016306+0000","last_became_active":"2024-05-21T15:32:33.688908+0000","last_became_peered":"2024-05-21T15:32:33.688908+0000","last_unstale":"2024-05-21T15:34:24.016306+0000","last_undegraded":"2024-05-21T15:34:24.016306+0000","last_fullsized":"2024-05-21T15:34:24.016306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:49:18.953116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022046+0000","last_change":"2024-05-21T15:32:33.187781+0000","last_active":"2024-05-21T15:34:24.022046+0000","last_peered":"2024-05-21T15:34:24.022046+0000","last_clean":"2024-05-21T15:34:24.022046+0000","last_became_active":"2024-05-21T15:32:33.187504+0000","last_became_peered":"2024-05-21T15:32:33.187504+0000","last_unstale":"2024-05-21T15:34:24.022046+0000","last_undegraded":"2024-05-21T15:34:24.022046+0000","last_fullsized":"2024-05-21T15:34:24.022046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:30.400209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021855+0000","last_change":"2024-05-21T15:32:33.188706+0000","last_active":"2024-05-21T15:34:24.021855+0000","last_peered":"2024-05-21T15:34:24.021855+0000","last_clean":"2024-05-21T15:34:24.021855+0000","last_became_active":"2024-05-21T15:32:33.187462+0000","last_became_peered":"2024-05-21T15:32:33.187462+0000","last_unstale":"2024-05-21T15:34:24.021855+0000","last_undegraded":"2024-05-21T15:34:24.021855+0000","last_fullsized":"2024-05-21T15:34:24.021855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:47:45.387721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016776+0000","last_change":"2024-05-21T15:32:33.672531+0000","last_active":"2024-05-21T15:34:24.016776+0000","last_peered":"2024-05-21T15:34:24.016776+0000","last_clean":"2024-05-21T15:34:24.016776+0000","last_became_active":"2024-05-21T15:32:33.671972+0000","last_became_peered":"2024-05-21T15:32:33.671972+0000","last_unstale":"2024-05-21T15:34:24.016776+0000","last_undegraded":"2024-05-21T15:34:24.016776+0000","last_fullsized":"2024-05-21T15:34:24.016776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:10:51.062530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'14","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143042+0000","last_change":"2024-05-21T15:32:33.181274+0000","last_active":"2024-05-21T15:34:24.143042+0000","last_peered":"2024-05-21T15:34:24.143042+0000","last_clean":"2024-05-21T15:34:24.143042+0000","last_became_active":"2024-05-21T15:32:33.180885+0000","last_became_peered":"2024-05-21T15:32:33.180885+0000","last_unstale":"2024-05-21T15:34:24.143042+0000","last_undegraded":"2024-05-21T15:34:24.143042+0000","last_fullsized":"2024-05-21T15:34:24.143042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:10:13.547380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022012+0000","last_change":"2024-05-21T15:32:33.683749+0000","last_active":"2024-05-21T15:34:24.022012+0000","last_peered":"2024-05-21T15:34:24.022012+0000","last_clean":"2024-05-21T15:34:24.022012+0000","last_became_active":"2024-05-21T15:32:33.683393+0000","last_became_peered":"2024-05-21T15:32:33.683393+0000","last_unstale":"2024-05-21T15:34:24.022012+0000","last_undegraded":"2024-05-21T15:34:24.022012+0000","last_fullsized":"2024-05-21T15:34:24.022012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:07:53.009651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143760+0000","last_change":"2024-05-21T15:32:33.184740+0000","last_active":"2024-05-21T15:34:24.143760+0000","last_peered":"2024-05-21T15:34:24.143760+0000","last_clean":"2024-05-21T15:34:24.143760+0000","last_became_active":"2024-05-21T15:32:33.184378+0000","last_became_peered":"2024-05-21T15:32:33.184378+0000","last_unstale":"2024-05-21T15:34:24.143760+0000","last_undegraded":"2024-05-21T15:34:24.143760+0000","last_fullsized":"2024-05-21T15:34:24.143760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:21:27.610513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143379+0000","last_change":"2024-05-21T15:33:23.381090+0000","last_active":"2024-05-21T15:34:24.143379+0000","last_peered":"2024-05-21T15:34:24.143379+0000","last_clean":"2024-05-21T15:34:24.143379+0000","last_became_active":"2024-05-21T15:33:23.380680+0000","last_became_peered":"2024-05-21T15:33:23.380680+0000","last_unstale":"2024-05-21T15:34:24.143379+0000","last_undegraded":"2024-05-21T15:34:24.143379+0000","last_fullsized":"2024-05-21T15:34:24.143379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:26:03.371476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017604+0000","last_change":"2024-05-21T15:32:33.183344+0000","last_active":"2024-05-21T15:34:24.017604+0000","last_peered":"2024-05-21T15:34:24.017604+0000","last_clean":"2024-05-21T15:34:24.017604+0000","last_became_active":"2024-05-21T15:32:33.182977+0000","last_became_peered":"2024-05-21T15:32:33.182977+0000","last_unstale":"2024-05-21T15:34:24.017604+0000","last_undegraded":"2024-05-21T15:34:24.017604+0000","last_fullsized":"2024-05-21T15:34:24.017604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:27:24.354129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021213+0000","last_change":"2024-05-21T15:32:33.677549+0000","last_active":"2024-05-21T15:34:24.021213+0000","last_peered":"2024-05-21T15:34:24.021213+0000","last_clean":"2024-05-21T15:34:24.021213+0000","last_became_active":"2024-05-21T15:32:33.677213+0000","last_became_peered":"2024-05-21T15:32:33.677213+0000","last_unstale":"2024-05-21T15:34:24.021213+0000","last_undegraded":"2024-05-21T15:34:24.021213+0000","last_fullsized":"2024-05-21T15:34:24.021213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:57:06.560434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021547+0000","last_change":"2024-05-21T15:32:33.190023+0000","last_active":"2024-05-21T15:34:24.021547+0000","last_peered":"2024-05-21T15:34:24.021547+0000","last_clean":"2024-05-21T15:34:24.021547+0000","last_became_active":"2024-05-21T15:32:33.189616+0000","last_became_peered":"2024-05-21T15:32:33.189616+0000","last_unstale":"2024-05-21T15:34:24.021547+0000","last_undegraded":"2024-05-21T15:34:24.021547+0000","last_fullsized":"2024-05-21T15:34:24.021547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:03.841414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019511+0000","last_change":"2024-05-21T15:32:33.685911+0000","last_active":"2024-05-21T15:34:24.019511+0000","last_peered":"2024-05-21T15:34:24.019511+0000","last_clean":"2024-05-21T15:34:24.019511+0000","last_became_active":"2024-05-21T15:32:33.684785+0000","last_became_peered":"2024-05-21T15:32:33.684785+0000","last_unstale":"2024-05-21T15:34:24.019511+0000","last_undegraded":"2024-05-21T15:34:24.019511+0000","last_fullsized":"2024-05-21T15:34:24.019511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:39:18.370289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142681+0000","last_change":"2024-05-21T15:32:33.178354+0000","last_active":"2024-05-21T15:34:24.142681+0000","last_peered":"2024-05-21T15:34:24.142681+0000","last_clean":"2024-05-21T15:34:24.142681+0000","last_became_active":"2024-05-21T15:32:33.177989+0000","last_became_peered":"2024-05-21T15:32:33.177989+0000","last_unstale":"2024-05-21T15:34:24.142681+0000","last_undegraded":"2024-05-21T15:34:24.142681+0000","last_fullsized":"2024-05-21T15:34:24.142681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:58.650831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021792+0000","last_change":"2024-05-21T15:32:33.686467+0000","last_active":"2024-05-21T15:34:24.021792+0000","last_peered":"2024-05-21T15:34:24.021792+0000","last_clean":"2024-05-21T15:34:24.021792+0000","last_became_active":"2024-05-21T15:32:33.686135+0000","last_became_peered":"2024-05-21T15:32:33.686135+0000","last_unstale":"2024-05-21T15:34:24.021792+0000","last_undegraded":"2024-05-21T15:34:24.021792+0000","last_fullsized":"2024-05-21T15:34:24.021792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:53:54.045591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019621+0000","last_change":"2024-05-21T15:33:23.383157+0000","last_active":"2024-05-21T15:34:24.019621+0000","last_peered":"2024-05-21T15:34:24.019621+0000","last_clean":"2024-05-21T15:34:24.019621+0000","last_became_active":"2024-05-21T15:33:23.376834+0000","last_became_peered":"2024-05-21T15:33:23.376834+0000","last_unstale":"2024-05-21T15:34:24.019621+0000","last_undegraded":"2024-05-21T15:34:24.019621+0000","last_fullsized":"2024-05-21T15:34:24.019621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:01:19.349486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022263+0000","last_change":"2024-05-21T15:32:33.692325+0000","last_active":"2024-05-21T15:34:24.022263+0000","last_peered":"2024-05-21T15:34:24.022263+0000","last_clean":"2024-05-21T15:34:24.022263+0000","last_became_active":"2024-05-21T15:32:33.691971+0000","last_became_peered":"2024-05-21T15:32:33.691971+0000","last_unstale":"2024-05-21T15:34:24.022263+0000","last_undegraded":"2024-05-21T15:34:24.022263+0000","last_fullsized":"2024-05-21T15:34:24.022263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:53:28.190063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:38.192163+0000","last_change":"2024-05-21T15:32:33.186925+0000","last_active":"2024-05-21T15:34:38.192163+0000","last_peered":"2024-05-21T15:34:38.192163+0000","last_clean":"2024-05-21T15:34:38.192163+0000","last_became_active":"2024-05-21T15:32:33.186605+0000","last_became_peered":"2024-05-21T15:32:33.186605+0000","last_unstale":"2024-05-21T15:34:38.192163+0000","last_undegraded":"2024-05-21T15:34:38.192163+0000","last_fullsized":"2024-05-21T15:34:38.192163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:32.752746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022284+0000","last_change":"2024-05-21T15:32:33.686132+0000","last_active":"2024-05-21T15:34:24.022284+0000","last_peered":"2024-05-21T15:34:24.022284+0000","last_clean":"2024-05-21T15:34:24.022284+0000","last_became_active":"2024-05-21T15:32:33.685745+0000","last_became_peered":"2024-05-21T15:32:33.685745+0000","last_unstale":"2024-05-21T15:34:24.022284+0000","last_undegraded":"2024-05-21T15:34:24.022284+0000","last_fullsized":"2024-05-21T15:34:24.022284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:51:49.522297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017771+0000","last_change":"2024-05-21T15:32:33.176552+0000","last_active":"2024-05-21T15:34:24.017771+0000","last_peered":"2024-05-21T15:34:24.017771+0000","last_clean":"2024-05-21T15:34:24.017771+0000","last_became_active":"2024-05-21T15:32:33.176237+0000","last_became_peered":"2024-05-21T15:32:33.176237+0000","last_unstale":"2024-05-21T15:34:24.017771+0000","last_undegraded":"2024-05-21T15:34:24.017771+0000","last_fullsized":"2024-05-21T15:34:24.017771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:09.098344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022243+0000","last_change":"2024-05-21T15:32:33.692310+0000","last_active":"2024-05-21T15:34:24.022243+0000","last_peered":"2024-05-21T15:34:24.022243+0000","last_clean":"2024-05-21T15:34:24.022243+0000","last_became_active":"2024-05-21T15:32:33.691959+0000","last_became_peered":"2024-05-21T15:32:33.691959+0000","last_unstale":"2024-05-21T15:34:24.022243+0000","last_undegraded":"2024-05-21T15:34:24.022243+0000","last_fullsized":"2024-05-21T15:34:24.022243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:57:38.243926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016714+0000","last_change":"2024-05-21T15:33:23.614280+0000","last_active":"2024-05-21T15:34:24.016714+0000","last_peered":"2024-05-21T15:34:24.016714+0000","last_clean":"2024-05-21T15:34:24.016714+0000","last_became_active":"2024-05-21T15:33:23.610364+0000","last_became_peered":"2024-05-21T15:33:23.610364+0000","last_unstale":"2024-05-21T15:34:24.016714+0000","last_undegraded":"2024-05-21T15:34:24.016714+0000","last_fullsized":"2024-05-21T15:34:24.016714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:40:42.990949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022268+0000","last_change":"2024-05-21T15:32:33.682801+0000","last_active":"2024-05-21T15:34:24.022268+0000","last_peered":"2024-05-21T15:34:24.022268+0000","last_clean":"2024-05-21T15:34:24.022268+0000","last_became_active":"2024-05-21T15:32:33.682368+0000","last_became_peered":"2024-05-21T15:32:33.682368+0000","last_unstale":"2024-05-21T15:34:24.022268+0000","last_undegraded":"2024-05-21T15:34:24.022268+0000","last_fullsized":"2024-05-21T15:34:24.022268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:56:50.520672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019605+0000","last_change":"2024-05-21T15:32:33.684809+0000","last_active":"2024-05-21T15:34:24.019605+0000","last_peered":"2024-05-21T15:34:24.019605+0000","last_clean":"2024-05-21T15:34:24.019605+0000","last_became_active":"2024-05-21T15:32:33.684067+0000","last_became_peered":"2024-05-21T15:32:33.684067+0000","last_unstale":"2024-05-21T15:34:24.019605+0000","last_undegraded":"2024-05-21T15:34:24.019605+0000","last_fullsized":"2024-05-21T15:34:24.019605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:31:58.997470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142775+0000","last_change":"2024-05-21T15:32:33.676091+0000","last_active":"2024-05-21T15:34:24.142775+0000","last_peered":"2024-05-21T15:34:24.142775+0000","last_clean":"2024-05-21T15:34:24.142775+0000","last_became_active":"2024-05-21T15:32:33.675538+0000","last_became_peered":"2024-05-21T15:32:33.675538+0000","last_unstale":"2024-05-21T15:34:24.142775+0000","last_undegraded":"2024-05-21T15:34:24.142775+0000","last_fullsized":"2024-05-21T15:34:24.142775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:50.311152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142726+0000","last_change":"2024-05-21T15:32:33.688590+0000","last_active":"2024-05-21T15:34:24.142726+0000","last_peered":"2024-05-21T15:34:24.142726+0000","last_clean":"2024-05-21T15:34:24.142726+0000","last_became_active":"2024-05-21T15:32:33.687993+0000","last_became_peered":"2024-05-21T15:32:33.687993+0000","last_unstale":"2024-05-21T15:34:24.142726+0000","last_undegraded":"2024-05-21T15:34:24.142726+0000","last_fullsized":"2024-05-21T15:34:24.142726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:21.939483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143010+0000","last_change":"2024-05-21T15:33:23.380890+0000","last_active":"2024-05-21T15:34:24.143010+0000","last_peered":"2024-05-21T15:34:24.143010+0000","last_clean":"2024-05-21T15:34:24.143010+0000","last_became_active":"2024-05-21T15:33:23.380503+0000","last_became_peered":"2024-05-21T15:33:23.380503+0000","last_unstale":"2024-05-21T15:34:24.143010+0000","last_undegraded":"2024-05-21T15:34:24.143010+0000","last_fullsized":"2024-05-21T15:34:24.143010+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:41.070336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016602+0000","last_change":"2024-05-21T15:32:33.689365+0000","last_active":"2024-05-21T15:34:24.016602+0000","last_peered":"2024-05-21T15:34:24.016602+0000","last_clean":"2024-05-21T15:34:24.016602+0000","last_became_active":"2024-05-21T15:32:33.688739+0000","last_became_peered":"2024-05-21T15:32:33.688739+0000","last_unstale":"2024-05-21T15:34:24.016602+0000","last_undegraded":"2024-05-21T15:34:24.016602+0000","last_fullsized":"2024-05-21T15:34:24.016602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:19:52.614155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.020003+0000","last_change":"2024-05-21T15:32:33.681985+0000","last_active":"2024-05-21T15:34:24.020003+0000","last_peered":"2024-05-21T15:34:24.020003+0000","last_clean":"2024-05-21T15:34:24.020003+0000","last_became_active":"2024-05-21T15:32:33.681209+0000","last_became_peered":"2024-05-21T15:32:33.681209+0000","last_unstale":"2024-05-21T15:34:24.020003+0000","last_undegraded":"2024-05-21T15:34:24.020003+0000","last_fullsized":"2024-05-21T15:34:24.020003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:03:16.088136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143161+0000","last_change":"2024-05-21T15:32:33.673236+0000","last_active":"2024-05-21T15:34:24.143161+0000","last_peered":"2024-05-21T15:34:24.143161+0000","last_clean":"2024-05-21T15:34:24.143161+0000","last_became_active":"2024-05-21T15:32:33.672548+0000","last_became_peered":"2024-05-21T15:32:33.672548+0000","last_unstale":"2024-05-21T15:34:24.143161+0000","last_undegraded":"2024-05-21T15:34:24.143161+0000","last_fullsized":"2024-05-21T15:34:24.143161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:52:10.010219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143155+0000","last_change":"2024-05-21T15:33:23.482874+0000","last_active":"2024-05-21T15:34:24.143155+0000","last_peered":"2024-05-21T15:34:24.143155+0000","last_clean":"2024-05-21T15:34:24.143155+0000","last_became_active":"2024-05-21T15:33:23.482589+0000","last_became_peered":"2024-05-21T15:33:23.482589+0000","last_unstale":"2024-05-21T15:34:24.143155+0000","last_undegraded":"2024-05-21T15:34:24.143155+0000","last_fullsized":"2024-05-21T15:34:24.143155+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:19:45.870562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017577+0000","last_change":"2024-05-21T15:32:33.686992+0000","last_active":"2024-05-21T15:34:24.017577+0000","last_peered":"2024-05-21T15:34:24.017577+0000","last_clean":"2024-05-21T15:34:24.017577+0000","last_became_active":"2024-05-21T15:32:33.686673+0000","last_became_peered":"2024-05-21T15:32:33.686673+0000","last_unstale":"2024-05-21T15:34:24.017577+0000","last_undegraded":"2024-05-21T15:34:24.017577+0000","last_fullsized":"2024-05-21T15:34:24.017577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:18:20.668365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021467+0000","last_change":"2024-05-21T15:32:33.689433+0000","last_active":"2024-05-21T15:34:24.021467+0000","last_peered":"2024-05-21T15:34:24.021467+0000","last_clean":"2024-05-21T15:34:24.021467+0000","last_became_active":"2024-05-21T15:32:33.689106+0000","last_became_peered":"2024-05-21T15:32:33.689106+0000","last_unstale":"2024-05-21T15:34:24.021467+0000","last_undegraded":"2024-05-21T15:34:24.021467+0000","last_fullsized":"2024-05-21T15:34:24.021467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:45:17.841159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017934+0000","last_change":"2024-05-21T15:32:33.687763+0000","last_active":"2024-05-21T15:34:24.017934+0000","last_peered":"2024-05-21T15:34:24.017934+0000","last_clean":"2024-05-21T15:34:24.017934+0000","last_became_active":"2024-05-21T15:32:33.685103+0000","last_became_peered":"2024-05-21T15:32:33.685103+0000","last_unstale":"2024-05-21T15:34:24.017934+0000","last_undegraded":"2024-05-21T15:34:24.017934+0000","last_fullsized":"2024-05-21T15:34:24.017934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:56:39.454536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016565+0000","last_change":"2024-05-21T15:32:33.679225+0000","last_active":"2024-05-21T15:34:24.016565+0000","last_peered":"2024-05-21T15:34:24.016565+0000","last_clean":"2024-05-21T15:34:24.016565+0000","last_became_active":"2024-05-21T15:32:33.678890+0000","last_became_peered":"2024-05-21T15:32:33.678890+0000","last_unstale":"2024-05-21T15:34:24.016565+0000","last_undegraded":"2024-05-21T15:34:24.016565+0000","last_fullsized":"2024-05-21T15:34:24.016565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:12:23.721449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019618+0000","last_change":"2024-05-21T15:32:33.688659+0000","last_active":"2024-05-21T15:34:24.019618+0000","last_peered":"2024-05-21T15:34:24.019618+0000","last_clean":"2024-05-21T15:34:24.019618+0000","last_became_active":"2024-05-21T15:32:33.688293+0000","last_became_peered":"2024-05-21T15:32:33.688293+0000","last_unstale":"2024-05-21T15:34:24.019618+0000","last_undegraded":"2024-05-21T15:34:24.019618+0000","last_fullsized":"2024-05-21T15:34:24.019618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:08:49.543237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":287606,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66976,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1206,"num_read_kb":675,"num_write":246,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":202,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":206301,"data_compressed":5562,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":477685,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1128114,"data_compressed":50013,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776176,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":476993,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":477371,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":1069498,"data_compressed":48159,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":476710,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":537337,"data_compressed":13445,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":478581,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":543658,"data_compressed":13445,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":476535,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888139,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":999424,"data_stored":476683,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724613,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":1069086,"data_compressed":48159,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61774,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67787,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61758,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T15:34:40.819 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T15:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.970+0000 7f9315137640 1 -- 172.21.15.38:0/1806417785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93101035a0 msgr2=0x7f93101059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.970+0000 7f9315137640 1 --2- 172.21.15.38:0/1806417785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93101035a0 0x7f93101059c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9300009900 tx=0x7f930002a410 comp rx=0 tx=0).stop 2024-05-21T15:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.970+0000 7f9315137640 1 -- 172.21.15.38:0/1806417785 shutdown_connections 2024-05-21T15:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.970+0000 7f9315137640 1 --2- 172.21.15.38:0/1806417785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9310108860 0x7f931010ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.970+0000 7f9315137640 1 --2- 172.21.15.38:0/1806417785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9310105f00 0x7f9310108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.970+0000 7f9315137640 1 --2- 172.21.15.38:0/1806417785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93101035a0 0x7f93101059c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.970+0000 7f9315137640 1 -- 172.21.15.38:0/1806417785 >> 172.21.15.38:0/1806417785 conn(0x7f93100fcba0 msgr2=0x7f93100ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.970+0000 7f9315137640 1 -- 172.21.15.38:0/1806417785 shutdown_connections 2024-05-21T15:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.970+0000 7f9315137640 1 -- 172.21.15.38:0/1806417785 wait complete. 2024-05-21T15:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.971+0000 7f9315137640 1 Processor -- start 2024-05-21T15:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.971+0000 7f9315137640 1 -- start start 2024-05-21T15:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.971+0000 7f9315137640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93101035a0 0x7f93101991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.971+0000 7f9315137640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9310105f00 0x7f9310199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.971+0000 7f9315137640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9310108860 0x7f93101a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.971+0000 7f9315137640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f931010e700 con 0x7f9310105f00 2024-05-21T15:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.971+0000 7f9315137640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f931010e580 con 0x7f93101035a0 2024-05-21T15:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.971+0000 7f9315137640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f931010e880 con 0x7f9310108860 2024-05-21T15:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.971+0000 7f930e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9310105f00 0x7f9310199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.971+0000 7f930e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9310105f00 0x7f9310199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59976/0 (socket says 172.21.15.38:59976) 2024-05-21T15:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.971+0000 7f930e59c640 1 -- 172.21.15.38:0/257018213 learned_addr learned my addr 172.21.15.38:0/257018213 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.971+0000 7f930ed9d640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93101035a0 0x7f93101991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.971+0000 7f930f59e640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9310108860 0x7f93101a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.972+0000 7f930e59c640 1 -- 172.21.15.38:0/257018213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9310108860 msgr2=0x7f93101a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.972+0000 7f930e59c640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9310108860 0x7f93101a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.972+0000 7f930e59c640 1 -- 172.21.15.38:0/257018213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93101035a0 msgr2=0x7f93101991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.972+0000 7f930e59c640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93101035a0 0x7f93101991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.972+0000 7f930e59c640 1 -- 172.21.15.38:0/257018213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93101a0d30 con 0x7f9310105f00 2024-05-21T15:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.972+0000 7f930ed9d640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93101035a0 0x7f93101991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.972+0000 7f930e59c640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9310105f00 0x7f9310199730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f930400d950 tx=0x7f930400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.972+0000 7f92f7fff640 1 -- 172.21.15.38:0/257018213 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304017070 con 0x7f9310105f00 2024-05-21T15:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.972+0000 7f9315137640 1 -- 172.21.15.38:0/257018213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93101a0fc0 con 0x7f9310105f00 2024-05-21T15:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.972+0000 7f92f7fff640 1 -- 172.21.15.38:0/257018213 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9304009c30 con 0x7f9310105f00 2024-05-21T15:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.972+0000 7f92f7fff640 1 -- 172.21.15.38:0/257018213 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930400f6b0 con 0x7f9310105f00 2024-05-21T15:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.972+0000 7f9315137640 1 -- 172.21.15.38:0/257018213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93101a14b0 con 0x7f9310105f00 2024-05-21T15:34:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.973+0000 7f9315137640 1 -- 172.21.15.38:0/257018213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93100fe6c0 con 0x7f9310105f00 2024-05-21T15:34:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.975+0000 7f92f7fff640 1 -- 172.21.15.38:0/257018213 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f93040109d0 con 0x7f9310105f00 2024-05-21T15:34:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.976+0000 7f92f7fff640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f92f80a2c30 0x7f92f80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.976+0000 7f930ed9d640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f92f80a2c30 0x7f92f80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:40.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.976+0000 7f92f7fff640 1 -- 172.21.15.38:0/257018213 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f93040ced60 con 0x7f9310105f00 2024-05-21T15:34:40.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.976+0000 7f930ed9d640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f92f80a2c30 0x7f92f80a5140 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9300000c00 tx=0x7f93000315d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:40.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:40.979+0000 7f92f7fff640 1 -- 172.21.15.38:0/257018213 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9304009de0 con 0x7f9310105f00 2024-05-21T15:34:41.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.163+0000 7f9315137640 1 -- 172.21.15.38:0/257018213 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9310061f90 con 0x7f92f80a2c30 2024-05-21T15:34:41.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.170+0000 7f92f7fff640 1 -- 172.21.15.38:0/257018213 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259204 (secure 0 0 0) 0x7f9310061f90 con 0x7f92f80a2c30 2024-05-21T15:34:41.172 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:41.173 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-05-21T15:34:41.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.173+0000 7f9315137640 1 -- 172.21.15.38:0/257018213 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f92f80a2c30 msgr2=0x7f92f80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:41.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.173+0000 7f9315137640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f92f80a2c30 0x7f92f80a5140 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9300000c00 tx=0x7f93000315d0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.173+0000 7f9315137640 1 -- 172.21.15.38:0/257018213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9310105f00 msgr2=0x7f9310199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:41.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.173+0000 7f9315137640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9310105f00 0x7f9310199730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f930400d950 tx=0x7f930400de10 comp rx=0 tx=0).stop 2024-05-21T15:34:41.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.173+0000 7f9315137640 1 -- 172.21.15.38:0/257018213 shutdown_connections 2024-05-21T15:34:41.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.173+0000 7f9315137640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9310108860 0x7f93101a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.173+0000 7f9315137640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f92f80a2c30 0x7f92f80a5140 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.173+0000 7f9315137640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9310105f00 0x7f9310199730 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.173+0000 7f9315137640 1 --2- 172.21.15.38:0/257018213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93101035a0 0x7f93101991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.173+0000 7f9315137640 1 -- 172.21.15.38:0/257018213 >> 172.21.15.38:0/257018213 conn(0x7f93100fcba0 msgr2=0x7f9310104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:41.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.173+0000 7f9315137640 1 -- 172.21.15.38:0/257018213 shutdown_connections 2024-05-21T15:34:41.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.173+0000 7f9315137640 1 -- 172.21.15.38:0/257018213 wait complete. 2024-05-21T15:34:41.184 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-05-21T15:34:40.249259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657578,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1332,"num_read_kb":784,"num_write":429,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180938,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":852,"ondisk_log_size":852,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373788,"kb_used_data":12088,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568036,"statfs":{"total":1151940427776,"available":1151557668864,"internally_reserved":0,"allocated":12378112,"data_stored":7688251,"data_compressed":254358,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":51,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003152"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.020232+0000","last_change":"2024-05-21T15:32:33.687244+0000","last_active":"2024-05-21T15:34:24.020232+0000","last_peered":"2024-05-21T15:34:24.020232+0000","last_clean":"2024-05-21T15:34:24.020232+0000","last_became_active":"2024-05-21T15:32:33.686515+0000","last_became_peered":"2024-05-21T15:32:33.686515+0000","last_unstale":"2024-05-21T15:34:24.020232+0000","last_undegraded":"2024-05-21T15:34:24.020232+0000","last_fullsized":"2024-05-21T15:34:24.020232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:10:18.396997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143144+0000","last_change":"2024-05-21T15:32:33.678497+0000","last_active":"2024-05-21T15:34:24.143144+0000","last_peered":"2024-05-21T15:34:24.143144+0000","last_clean":"2024-05-21T15:34:24.143144+0000","last_became_active":"2024-05-21T15:32:33.678001+0000","last_became_peered":"2024-05-21T15:32:33.678001+0000","last_unstale":"2024-05-21T15:34:24.143144+0000","last_undegraded":"2024-05-21T15:34:24.143144+0000","last_fullsized":"2024-05-21T15:34:24.143144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:14:48.255675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143214+0000","last_change":"2024-05-21T15:32:33.692444+0000","last_active":"2024-05-21T15:34:24.143214+0000","last_peered":"2024-05-21T15:34:24.143214+0000","last_clean":"2024-05-21T15:34:24.143214+0000","last_became_active":"2024-05-21T15:32:33.691662+0000","last_became_peered":"2024-05-21T15:32:33.691662+0000","last_unstale":"2024-05-21T15:34:24.143214+0000","last_undegraded":"2024-05-21T15:34:24.143214+0000","last_fullsized":"2024-05-21T15:34:24.143214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:26:25.929879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143818+0000","last_change":"2024-05-21T15:32:33.692297+0000","last_active":"2024-05-21T15:34:24.143818+0000","last_peered":"2024-05-21T15:34:24.143818+0000","last_clean":"2024-05-21T15:34:24.143818+0000","last_became_active":"2024-05-21T15:32:33.691925+0000","last_became_peered":"2024-05-21T15:32:33.691925+0000","last_unstale":"2024-05-21T15:34:24.143818+0000","last_undegraded":"2024-05-21T15:34:24.143818+0000","last_fullsized":"2024-05-21T15:34:24.143818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:40:45.448485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021213+0000","last_change":"2024-05-21T15:32:33.685511+0000","last_active":"2024-05-21T15:34:24.021213+0000","last_peered":"2024-05-21T15:34:24.021213+0000","last_clean":"2024-05-21T15:34:24.021213+0000","last_became_active":"2024-05-21T15:32:33.685182+0000","last_became_peered":"2024-05-21T15:32:33.685182+0000","last_unstale":"2024-05-21T15:34:24.021213+0000","last_undegraded":"2024-05-21T15:34:24.021213+0000","last_fullsized":"2024-05-21T15:34:24.021213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:59:06.680712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019738+0000","last_change":"2024-05-21T15:32:33.682400+0000","last_active":"2024-05-21T15:34:24.019738+0000","last_peered":"2024-05-21T15:34:24.019738+0000","last_clean":"2024-05-21T15:34:24.019738+0000","last_became_active":"2024-05-21T15:32:33.682068+0000","last_became_peered":"2024-05-21T15:32:33.682068+0000","last_unstale":"2024-05-21T15:34:24.019738+0000","last_undegraded":"2024-05-21T15:34:24.019738+0000","last_fullsized":"2024-05-21T15:34:24.019738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:31:50.968706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021817+0000","last_change":"2024-05-21T15:32:33.690099+0000","last_active":"2024-05-21T15:34:24.021817+0000","last_peered":"2024-05-21T15:34:24.021817+0000","last_clean":"2024-05-21T15:34:24.021817+0000","last_became_active":"2024-05-21T15:32:33.689762+0000","last_became_peered":"2024-05-21T15:32:33.689762+0000","last_unstale":"2024-05-21T15:34:24.021817+0000","last_undegraded":"2024-05-21T15:34:24.021817+0000","last_fullsized":"2024-05-21T15:34:24.021817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:55:20.331097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021396+0000","last_change":"2024-05-21T15:32:33.691701+0000","last_active":"2024-05-21T15:34:24.021396+0000","last_peered":"2024-05-21T15:34:24.021396+0000","last_clean":"2024-05-21T15:34:24.021396+0000","last_became_active":"2024-05-21T15:32:33.691386+0000","last_became_peered":"2024-05-21T15:32:33.691386+0000","last_unstale":"2024-05-21T15:34:24.021396+0000","last_undegraded":"2024-05-21T15:34:24.021396+0000","last_fullsized":"2024-05-21T15:34:24.021396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:54:45.691392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.018266+0000","last_change":"2024-05-21T15:32:33.677484+0000","last_active":"2024-05-21T15:34:24.018266+0000","last_peered":"2024-05-21T15:34:24.018266+0000","last_clean":"2024-05-21T15:34:24.018266+0000","last_became_active":"2024-05-21T15:32:33.677160+0000","last_became_peered":"2024-05-21T15:32:33.677160+0000","last_unstale":"2024-05-21T15:34:24.018266+0000","last_undegraded":"2024-05-21T15:34:24.018266+0000","last_fullsized":"2024-05-21T15:34:24.018266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:38:00.914393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019574+0000","last_change":"2024-05-21T15:32:33.686957+0000","last_active":"2024-05-21T15:34:24.019574+0000","last_peered":"2024-05-21T15:34:24.019574+0000","last_clean":"2024-05-21T15:34:24.019574+0000","last_became_active":"2024-05-21T15:32:33.685338+0000","last_became_peered":"2024-05-21T15:32:33.685338+0000","last_unstale":"2024-05-21T15:34:24.019574+0000","last_undegraded":"2024-05-21T15:34:24.019574+0000","last_fullsized":"2024-05-21T15:34:24.019574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:04:58.115974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022284+0000","last_change":"2024-05-21T15:32:33.693311+0000","last_active":"2024-05-21T15:34:24.022284+0000","last_peered":"2024-05-21T15:34:24.022284+0000","last_clean":"2024-05-21T15:34:24.022284+0000","last_became_active":"2024-05-21T15:32:33.692959+0000","last_became_peered":"2024-05-21T15:32:33.692959+0000","last_unstale":"2024-05-21T15:34:24.022284+0000","last_undegraded":"2024-05-21T15:34:24.022284+0000","last_fullsized":"2024-05-21T15:34:24.022284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:35:12.918524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143679+0000","last_change":"2024-05-21T15:32:33.693254+0000","last_active":"2024-05-21T15:34:24.143679+0000","last_peered":"2024-05-21T15:34:24.143679+0000","last_clean":"2024-05-21T15:34:24.143679+0000","last_became_active":"2024-05-21T15:32:33.692901+0000","last_became_peered":"2024-05-21T15:32:33.692901+0000","last_unstale":"2024-05-21T15:34:24.143679+0000","last_undegraded":"2024-05-21T15:34:24.143679+0000","last_fullsized":"2024-05-21T15:34:24.143679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:36:30.591479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016565+0000","last_change":"2024-05-21T15:33:23.383656+0000","last_active":"2024-05-21T15:34:24.016565+0000","last_peered":"2024-05-21T15:34:24.016565+0000","last_clean":"2024-05-21T15:34:24.016565+0000","last_became_active":"2024-05-21T15:33:23.383113+0000","last_became_peered":"2024-05-21T15:33:23.383113+0000","last_unstale":"2024-05-21T15:34:24.016565+0000","last_undegraded":"2024-05-21T15:34:24.016565+0000","last_fullsized":"2024-05-21T15:34:24.016565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:14:37.327480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142760+0000","last_change":"2024-05-21T15:32:33.678315+0000","last_active":"2024-05-21T15:34:24.142760+0000","last_peered":"2024-05-21T15:34:24.142760+0000","last_clean":"2024-05-21T15:34:24.142760+0000","last_became_active":"2024-05-21T15:32:33.677787+0000","last_became_peered":"2024-05-21T15:32:33.677787+0000","last_unstale":"2024-05-21T15:34:24.142760+0000","last_undegraded":"2024-05-21T15:34:24.142760+0000","last_fullsized":"2024-05-21T15:34:24.142760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:17:24.826525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142879+0000","last_change":"2024-05-21T15:33:23.380244+0000","last_active":"2024-05-21T15:34:24.142879+0000","last_peered":"2024-05-21T15:34:24.142879+0000","last_clean":"2024-05-21T15:34:24.142879+0000","last_became_active":"2024-05-21T15:33:23.379769+0000","last_became_peered":"2024-05-21T15:33:23.379769+0000","last_unstale":"2024-05-21T15:34:24.142879+0000","last_undegraded":"2024-05-21T15:34:24.142879+0000","last_fullsized":"2024-05-21T15:34:24.142879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:16:07.820636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017488+0000","last_change":"2024-05-21T15:32:33.683597+0000","last_active":"2024-05-21T15:34:24.017488+0000","last_peered":"2024-05-21T15:34:24.017488+0000","last_clean":"2024-05-21T15:34:24.017488+0000","last_became_active":"2024-05-21T15:32:33.681948+0000","last_became_peered":"2024-05-21T15:32:33.681948+0000","last_unstale":"2024-05-21T15:34:24.017488+0000","last_undegraded":"2024-05-21T15:34:24.017488+0000","last_fullsized":"2024-05-21T15:34:24.017488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:26:50.086019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.020426+0000","last_change":"2024-05-21T15:32:33.683329+0000","last_active":"2024-05-21T15:34:24.020426+0000","last_peered":"2024-05-21T15:34:24.020426+0000","last_clean":"2024-05-21T15:34:24.020426+0000","last_became_active":"2024-05-21T15:32:33.682947+0000","last_became_peered":"2024-05-21T15:32:33.682947+0000","last_unstale":"2024-05-21T15:34:24.020426+0000","last_undegraded":"2024-05-21T15:34:24.020426+0000","last_fullsized":"2024-05-21T15:34:24.020426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:24:27.502257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142833+0000","last_change":"2024-05-21T15:32:33.680391+0000","last_active":"2024-05-21T15:34:24.142833+0000","last_peered":"2024-05-21T15:34:24.142833+0000","last_clean":"2024-05-21T15:34:24.142833+0000","last_became_active":"2024-05-21T15:32:33.679996+0000","last_became_peered":"2024-05-21T15:32:33.679996+0000","last_unstale":"2024-05-21T15:34:24.142833+0000","last_undegraded":"2024-05-21T15:34:24.142833+0000","last_fullsized":"2024-05-21T15:34:24.142833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:56:43.658565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142946+0000","last_change":"2024-05-21T15:32:33.688783+0000","last_active":"2024-05-21T15:34:24.142946+0000","last_peered":"2024-05-21T15:34:24.142946+0000","last_clean":"2024-05-21T15:34:24.142946+0000","last_became_active":"2024-05-21T15:32:33.688006+0000","last_became_peered":"2024-05-21T15:32:33.688006+0000","last_unstale":"2024-05-21T15:34:24.142946+0000","last_undegraded":"2024-05-21T15:34:24.142946+0000","last_fullsized":"2024-05-21T15:34:24.142946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:17:44.489759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":155,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:37.162842+0000","last_change":"2024-05-21T15:32:33.182251+0000","last_active":"2024-05-21T15:34:37.162842+0000","last_peered":"2024-05-21T15:34:37.162842+0000","last_clean":"2024-05-21T15:34:37.162842+0000","last_became_active":"2024-05-21T15:32:33.181893+0000","last_became_peered":"2024-05-21T15:32:33.181893+0000","last_unstale":"2024-05-21T15:34:37.162842+0000","last_undegraded":"2024-05-21T15:34:37.162842+0000","last_fullsized":"2024-05-21T15:34:37.162842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:23:36.863186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021774+0000","last_change":"2024-05-21T15:32:33.691962+0000","last_active":"2024-05-21T15:34:24.021774+0000","last_peered":"2024-05-21T15:34:24.021774+0000","last_clean":"2024-05-21T15:34:24.021774+0000","last_became_active":"2024-05-21T15:32:33.691619+0000","last_became_peered":"2024-05-21T15:32:33.691619+0000","last_unstale":"2024-05-21T15:34:24.021774+0000","last_undegraded":"2024-05-21T15:34:24.021774+0000","last_fullsized":"2024-05-21T15:34:24.021774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:50:14.523567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022035+0000","last_change":"2024-05-21T15:32:33.183772+0000","last_active":"2024-05-21T15:34:24.022035+0000","last_peered":"2024-05-21T15:34:24.022035+0000","last_clean":"2024-05-21T15:34:24.022035+0000","last_became_active":"2024-05-21T15:32:33.183440+0000","last_became_peered":"2024-05-21T15:32:33.183440+0000","last_unstale":"2024-05-21T15:34:24.022035+0000","last_undegraded":"2024-05-21T15:34:24.022035+0000","last_fullsized":"2024-05-21T15:34:24.022035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:53:00.714359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143490+0000","last_change":"2024-05-21T15:32:33.682544+0000","last_active":"2024-05-21T15:34:24.143490+0000","last_peered":"2024-05-21T15:34:24.143490+0000","last_clean":"2024-05-21T15:34:24.143490+0000","last_became_active":"2024-05-21T15:32:33.682161+0000","last_became_peered":"2024-05-21T15:32:33.682161+0000","last_unstale":"2024-05-21T15:34:24.143490+0000","last_undegraded":"2024-05-21T15:34:24.143490+0000","last_fullsized":"2024-05-21T15:34:24.143490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:20:38.143508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021961+0000","last_change":"2024-05-21T15:32:33.182400+0000","last_active":"2024-05-21T15:34:24.021961+0000","last_peered":"2024-05-21T15:34:24.021961+0000","last_clean":"2024-05-21T15:34:24.021961+0000","last_became_active":"2024-05-21T15:32:33.181745+0000","last_became_peered":"2024-05-21T15:32:33.181745+0000","last_unstale":"2024-05-21T15:34:24.021961+0000","last_undegraded":"2024-05-21T15:34:24.021961+0000","last_fullsized":"2024-05-21T15:34:24.021961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:39:41.669529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017631+0000","last_change":"2024-05-21T15:32:33.682700+0000","last_active":"2024-05-21T15:34:24.017631+0000","last_peered":"2024-05-21T15:34:24.017631+0000","last_clean":"2024-05-21T15:34:24.017631+0000","last_became_active":"2024-05-21T15:32:33.680876+0000","last_became_peered":"2024-05-21T15:32:33.680876+0000","last_unstale":"2024-05-21T15:34:24.017631+0000","last_undegraded":"2024-05-21T15:34:24.017631+0000","last_fullsized":"2024-05-21T15:34:24.017631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:30.694520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021728+0000","last_change":"2024-05-21T15:32:33.179114+0000","last_active":"2024-05-21T15:34:24.021728+0000","last_peered":"2024-05-21T15:34:24.021728+0000","last_clean":"2024-05-21T15:34:24.021728+0000","last_became_active":"2024-05-21T15:32:33.178485+0000","last_became_peered":"2024-05-21T15:32:33.178485+0000","last_unstale":"2024-05-21T15:34:24.021728+0000","last_undegraded":"2024-05-21T15:34:24.021728+0000","last_fullsized":"2024-05-21T15:34:24.021728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:19:56.501287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017868+0000","last_change":"2024-05-21T15:32:33.691874+0000","last_active":"2024-05-21T15:34:24.017868+0000","last_peered":"2024-05-21T15:34:24.017868+0000","last_clean":"2024-05-21T15:34:24.017868+0000","last_became_active":"2024-05-21T15:32:33.691546+0000","last_became_peered":"2024-05-21T15:32:33.691546+0000","last_unstale":"2024-05-21T15:34:24.017868+0000","last_undegraded":"2024-05-21T15:34:24.017868+0000","last_fullsized":"2024-05-21T15:34:24.017868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:40:09.281792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:35.164975+0000","last_change":"2024-05-21T15:32:33.183457+0000","last_active":"2024-05-21T15:34:35.164975+0000","last_peered":"2024-05-21T15:34:35.164975+0000","last_clean":"2024-05-21T15:34:35.164975+0000","last_became_active":"2024-05-21T15:32:33.183127+0000","last_became_peered":"2024-05-21T15:32:33.183127+0000","last_unstale":"2024-05-21T15:34:35.164975+0000","last_undegraded":"2024-05-21T15:34:35.164975+0000","last_fullsized":"2024-05-21T15:34:35.164975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:41:05.179629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021554+0000","last_change":"2024-05-21T15:32:33.677260+0000","last_active":"2024-05-21T15:34:24.021554+0000","last_peered":"2024-05-21T15:34:24.021554+0000","last_clean":"2024-05-21T15:34:24.021554+0000","last_became_active":"2024-05-21T15:32:33.676905+0000","last_became_peered":"2024-05-21T15:32:33.676905+0000","last_unstale":"2024-05-21T15:34:24.021554+0000","last_undegraded":"2024-05-21T15:34:24.021554+0000","last_fullsized":"2024-05-21T15:34:24.021554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:51:41.543066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.150318+0000","last_change":"2024-05-21T15:34:24.150318+0000","last_active":"2024-05-21T15:34:24.150318+0000","last_peered":"2024-05-21T15:34:24.150318+0000","last_clean":"2024-05-21T15:34:24.150318+0000","last_became_active":"2024-05-21T15:34:24.144598+0000","last_became_peered":"2024-05-21T15:34:24.144598+0000","last_unstale":"2024-05-21T15:34:24.150318+0000","last_undegraded":"2024-05-21T15:34:24.150318+0000","last_fullsized":"2024-05-21T15:34:24.150318+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:12:00.320432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017867+0000","last_change":"2024-05-21T15:32:33.691072+0000","last_active":"2024-05-21T15:34:24.017867+0000","last_peered":"2024-05-21T15:34:24.017867+0000","last_clean":"2024-05-21T15:34:24.017867+0000","last_became_active":"2024-05-21T15:32:33.690663+0000","last_became_peered":"2024-05-21T15:32:33.690663+0000","last_unstale":"2024-05-21T15:34:24.017867+0000","last_undegraded":"2024-05-21T15:34:24.017867+0000","last_fullsized":"2024-05-21T15:34:24.017867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:52:44.549537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016485+0000","last_change":"2024-05-21T15:32:33.179588+0000","last_active":"2024-05-21T15:34:24.016485+0000","last_peered":"2024-05-21T15:34:24.016485+0000","last_clean":"2024-05-21T15:34:24.016485+0000","last_became_active":"2024-05-21T15:32:33.179115+0000","last_became_peered":"2024-05-21T15:32:33.179115+0000","last_unstale":"2024-05-21T15:34:24.016485+0000","last_undegraded":"2024-05-21T15:34:24.016485+0000","last_fullsized":"2024-05-21T15:34:24.016485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:18:43.904584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021488+0000","last_change":"2024-05-21T15:32:33.689456+0000","last_active":"2024-05-21T15:34:24.021488+0000","last_peered":"2024-05-21T15:34:24.021488+0000","last_clean":"2024-05-21T15:34:24.021488+0000","last_became_active":"2024-05-21T15:32:33.689142+0000","last_became_peered":"2024-05-21T15:32:33.689142+0000","last_unstale":"2024-05-21T15:34:24.021488+0000","last_undegraded":"2024-05-21T15:34:24.021488+0000","last_fullsized":"2024-05-21T15:34:24.021488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:00:13.768272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017507+0000","last_change":"2024-05-21T15:32:33.184865+0000","last_active":"2024-05-21T15:34:24.017507+0000","last_peered":"2024-05-21T15:34:24.017507+0000","last_clean":"2024-05-21T15:34:24.017507+0000","last_became_active":"2024-05-21T15:32:33.184540+0000","last_became_peered":"2024-05-21T15:32:33.184540+0000","last_unstale":"2024-05-21T15:34:24.017507+0000","last_undegraded":"2024-05-21T15:34:24.017507+0000","last_fullsized":"2024-05-21T15:34:24.017507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:09:20.072906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142492+0000","last_change":"2024-05-21T15:32:33.685478+0000","last_active":"2024-05-21T15:34:24.142492+0000","last_peered":"2024-05-21T15:34:24.142492+0000","last_clean":"2024-05-21T15:34:24.142492+0000","last_became_active":"2024-05-21T15:32:33.685137+0000","last_became_peered":"2024-05-21T15:32:33.685137+0000","last_unstale":"2024-05-21T15:34:24.142492+0000","last_undegraded":"2024-05-21T15:34:24.142492+0000","last_fullsized":"2024-05-21T15:34:24.142492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:39:40.815876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":262,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:37.167571+0000","last_change":"2024-05-21T15:32:33.179215+0000","last_active":"2024-05-21T15:34:37.167571+0000","last_peered":"2024-05-21T15:34:37.167571+0000","last_clean":"2024-05-21T15:34:37.167571+0000","last_became_active":"2024-05-21T15:32:33.178846+0000","last_became_peered":"2024-05-21T15:32:33.178846+0000","last_unstale":"2024-05-21T15:34:37.167571+0000","last_undegraded":"2024-05-21T15:34:37.167571+0000","last_fullsized":"2024-05-21T15:34:37.167571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:29:47.289968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":234,"num_read_kb":234,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.018136+0000","last_change":"2024-05-21T15:32:33.684900+0000","last_active":"2024-05-21T15:34:24.018136+0000","last_peered":"2024-05-21T15:34:24.018136+0000","last_clean":"2024-05-21T15:34:24.018136+0000","last_became_active":"2024-05-21T15:32:33.681374+0000","last_became_peered":"2024-05-21T15:32:33.681374+0000","last_unstale":"2024-05-21T15:34:24.018136+0000","last_undegraded":"2024-05-21T15:34:24.018136+0000","last_fullsized":"2024-05-21T15:34:24.018136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:51:38.079726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022157+0000","last_change":"2024-05-21T15:32:33.189945+0000","last_active":"2024-05-21T15:34:24.022157+0000","last_peered":"2024-05-21T15:34:24.022157+0000","last_clean":"2024-05-21T15:34:24.022157+0000","last_became_active":"2024-05-21T15:32:33.189557+0000","last_became_peered":"2024-05-21T15:32:33.189557+0000","last_unstale":"2024-05-21T15:34:24.022157+0000","last_undegraded":"2024-05-21T15:34:24.022157+0000","last_fullsized":"2024-05-21T15:34:24.022157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:08.427043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143483+0000","last_change":"2024-05-21T15:32:33.692896+0000","last_active":"2024-05-21T15:34:24.143483+0000","last_peered":"2024-05-21T15:34:24.143483+0000","last_clean":"2024-05-21T15:34:24.143483+0000","last_became_active":"2024-05-21T15:32:33.692527+0000","last_became_peered":"2024-05-21T15:32:33.692527+0000","last_unstale":"2024-05-21T15:34:24.143483+0000","last_undegraded":"2024-05-21T15:34:24.143483+0000","last_fullsized":"2024-05-21T15:34:24.143483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:45:45.418063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017955+0000","last_change":"2024-05-21T15:32:33.172989+0000","last_active":"2024-05-21T15:34:24.017955+0000","last_peered":"2024-05-21T15:34:24.017955+0000","last_clean":"2024-05-21T15:34:24.017955+0000","last_became_active":"2024-05-21T15:32:33.172215+0000","last_became_peered":"2024-05-21T15:32:33.172215+0000","last_unstale":"2024-05-21T15:34:24.017955+0000","last_undegraded":"2024-05-21T15:34:24.017955+0000","last_fullsized":"2024-05-21T15:34:24.017955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:09:39.287636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017840+0000","last_change":"2024-05-21T15:32:33.678441+0000","last_active":"2024-05-21T15:34:24.017840+0000","last_peered":"2024-05-21T15:34:24.017840+0000","last_clean":"2024-05-21T15:34:24.017840+0000","last_became_active":"2024-05-21T15:32:33.677833+0000","last_became_peered":"2024-05-21T15:32:33.677833+0000","last_unstale":"2024-05-21T15:34:24.017840+0000","last_undegraded":"2024-05-21T15:34:24.017840+0000","last_fullsized":"2024-05-21T15:34:24.017840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:36.124064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'34","reported_seq":299,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:37.189944+0000","last_change":"2024-05-21T15:32:33.178153+0000","last_active":"2024-05-21T15:34:37.189944+0000","last_peered":"2024-05-21T15:34:37.189944+0000","last_clean":"2024-05-21T15:34:37.189944+0000","last_became_active":"2024-05-21T15:32:33.177792+0000","last_became_peered":"2024-05-21T15:32:33.177792+0000","last_unstale":"2024-05-21T15:34:37.189944+0000","last_undegraded":"2024-05-21T15:34:37.189944+0000","last_fullsized":"2024-05-21T15:34:37.189944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:43:41.865204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60022,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":242,"num_read_kb":180,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021430+0000","last_change":"2024-05-21T15:32:33.681328+0000","last_active":"2024-05-21T15:34:24.021430+0000","last_peered":"2024-05-21T15:34:24.021430+0000","last_clean":"2024-05-21T15:34:24.021430+0000","last_became_active":"2024-05-21T15:32:33.680975+0000","last_became_peered":"2024-05-21T15:32:33.680975+0000","last_unstale":"2024-05-21T15:34:24.021430+0000","last_undegraded":"2024-05-21T15:34:24.021430+0000","last_fullsized":"2024-05-21T15:34:24.021430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:50:27.199348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142791+0000","last_change":"2024-05-21T15:32:33.182492+0000","last_active":"2024-05-21T15:34:24.142791+0000","last_peered":"2024-05-21T15:34:24.142791+0000","last_clean":"2024-05-21T15:34:24.142791+0000","last_became_active":"2024-05-21T15:32:33.182152+0000","last_became_peered":"2024-05-21T15:32:33.182152+0000","last_unstale":"2024-05-21T15:34:24.142791+0000","last_undegraded":"2024-05-21T15:34:24.142791+0000","last_fullsized":"2024-05-21T15:34:24.142791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:04:07.432742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016470+0000","last_change":"2024-05-21T15:32:33.686623+0000","last_active":"2024-05-21T15:34:24.016470+0000","last_peered":"2024-05-21T15:34:24.016470+0000","last_clean":"2024-05-21T15:34:24.016470+0000","last_became_active":"2024-05-21T15:32:33.685816+0000","last_became_peered":"2024-05-21T15:32:33.685816+0000","last_unstale":"2024-05-21T15:34:24.016470+0000","last_undegraded":"2024-05-21T15:34:24.016470+0000","last_fullsized":"2024-05-21T15:34:24.016470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:15:20.354113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.029185+0000","last_change":"2024-05-21T15:34:24.028748+0000","last_active":"2024-05-21T15:34:24.029185+0000","last_peered":"2024-05-21T15:34:24.029185+0000","last_clean":"2024-05-21T15:34:24.029185+0000","last_became_active":"2024-05-21T15:34:24.022920+0000","last_became_peered":"2024-05-21T15:34:24.022920+0000","last_unstale":"2024-05-21T15:34:24.029185+0000","last_undegraded":"2024-05-21T15:34:24.029185+0000","last_fullsized":"2024-05-21T15:34:24.029185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:09:04.712896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021976+0000","last_change":"2024-05-21T15:32:33.689936+0000","last_active":"2024-05-21T15:34:24.021976+0000","last_peered":"2024-05-21T15:34:24.021976+0000","last_clean":"2024-05-21T15:34:24.021976+0000","last_became_active":"2024-05-21T15:32:33.689452+0000","last_became_peered":"2024-05-21T15:32:33.689452+0000","last_unstale":"2024-05-21T15:34:24.021976+0000","last_undegraded":"2024-05-21T15:34:24.021976+0000","last_fullsized":"2024-05-21T15:34:24.021976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:00.420730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:35.166161+0000","last_change":"2024-05-21T15:32:33.181354+0000","last_active":"2024-05-21T15:34:35.166161+0000","last_peered":"2024-05-21T15:34:35.166161+0000","last_clean":"2024-05-21T15:34:35.166161+0000","last_became_active":"2024-05-21T15:32:33.180983+0000","last_became_peered":"2024-05-21T15:32:33.180983+0000","last_unstale":"2024-05-21T15:34:35.166161+0000","last_undegraded":"2024-05-21T15:34:35.166161+0000","last_fullsized":"2024-05-21T15:34:35.166161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:18:46.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021643+0000","last_change":"2024-05-21T15:33:23.377305+0000","last_active":"2024-05-21T15:34:24.021643+0000","last_peered":"2024-05-21T15:34:24.021643+0000","last_clean":"2024-05-21T15:34:24.021643+0000","last_became_active":"2024-05-21T15:33:23.376921+0000","last_became_peered":"2024-05-21T15:33:23.376921+0000","last_unstale":"2024-05-21T15:34:24.021643+0000","last_undegraded":"2024-05-21T15:34:24.021643+0000","last_fullsized":"2024-05-21T15:34:24.021643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:19.874725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021379+0000","last_change":"2024-05-21T15:33:23.496401+0000","last_active":"2024-05-21T15:34:24.021379+0000","last_peered":"2024-05-21T15:34:24.021379+0000","last_clean":"2024-05-21T15:34:24.021379+0000","last_became_active":"2024-05-21T15:33:23.483584+0000","last_became_peered":"2024-05-21T15:33:23.483584+0000","last_unstale":"2024-05-21T15:34:24.021379+0000","last_undegraded":"2024-05-21T15:34:24.021379+0000","last_fullsized":"2024-05-21T15:34:24.021379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:35:32.411825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017462+0000","last_change":"2024-05-21T15:32:33.691112+0000","last_active":"2024-05-21T15:34:24.017462+0000","last_peered":"2024-05-21T15:34:24.017462+0000","last_clean":"2024-05-21T15:34:24.017462+0000","last_became_active":"2024-05-21T15:32:33.690726+0000","last_became_peered":"2024-05-21T15:32:33.690726+0000","last_unstale":"2024-05-21T15:34:24.017462+0000","last_undegraded":"2024-05-21T15:34:24.017462+0000","last_fullsized":"2024-05-21T15:34:24.017462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:59:29.002636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021659+0000","last_change":"2024-05-21T15:32:33.179565+0000","last_active":"2024-05-21T15:34:24.021659+0000","last_peered":"2024-05-21T15:34:24.021659+0000","last_clean":"2024-05-21T15:34:24.021659+0000","last_became_active":"2024-05-21T15:32:33.179168+0000","last_became_peered":"2024-05-21T15:32:33.179168+0000","last_unstale":"2024-05-21T15:34:24.021659+0000","last_undegraded":"2024-05-21T15:34:24.021659+0000","last_fullsized":"2024-05-21T15:34:24.021659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:23:39.210931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.020065+0000","last_change":"2024-05-21T15:32:33.691348+0000","last_active":"2024-05-21T15:34:24.020065+0000","last_peered":"2024-05-21T15:34:24.020065+0000","last_clean":"2024-05-21T15:34:24.020065+0000","last_became_active":"2024-05-21T15:32:33.691005+0000","last_became_peered":"2024-05-21T15:32:33.691005+0000","last_unstale":"2024-05-21T15:34:24.020065+0000","last_undegraded":"2024-05-21T15:34:24.020065+0000","last_fullsized":"2024-05-21T15:34:24.020065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:33:34.533026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142969+0000","last_change":"2024-05-21T15:32:33.184850+0000","last_active":"2024-05-21T15:34:24.142969+0000","last_peered":"2024-05-21T15:34:24.142969+0000","last_clean":"2024-05-21T15:34:24.142969+0000","last_became_active":"2024-05-21T15:32:33.184480+0000","last_became_peered":"2024-05-21T15:32:33.184480+0000","last_unstale":"2024-05-21T15:34:24.142969+0000","last_undegraded":"2024-05-21T15:34:24.142969+0000","last_fullsized":"2024-05-21T15:34:24.142969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:31.327820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019449+0000","last_change":"2024-05-21T15:32:33.689520+0000","last_active":"2024-05-21T15:34:24.019449+0000","last_peered":"2024-05-21T15:34:24.019449+0000","last_clean":"2024-05-21T15:34:24.019449+0000","last_became_active":"2024-05-21T15:32:33.688219+0000","last_became_peered":"2024-05-21T15:32:33.688219+0000","last_unstale":"2024-05-21T15:34:24.019449+0000","last_undegraded":"2024-05-21T15:34:24.019449+0000","last_fullsized":"2024-05-21T15:34:24.019449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:36.546390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:38.164983+0000","last_change":"2024-05-21T15:32:33.183183+0000","last_active":"2024-05-21T15:34:38.164983+0000","last_peered":"2024-05-21T15:34:38.164983+0000","last_clean":"2024-05-21T15:34:38.164983+0000","last_became_active":"2024-05-21T15:32:33.182753+0000","last_became_peered":"2024-05-21T15:32:33.182753+0000","last_unstale":"2024-05-21T15:34:38.164983+0000","last_undegraded":"2024-05-21T15:34:38.164983+0000","last_fullsized":"2024-05-21T15:34:38.164983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:44:22.509265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019138+0000","last_change":"2024-05-21T15:32:33.684019+0000","last_active":"2024-05-21T15:34:24.019138+0000","last_peered":"2024-05-21T15:34:24.019138+0000","last_clean":"2024-05-21T15:34:24.019138+0000","last_became_active":"2024-05-21T15:32:33.683276+0000","last_became_peered":"2024-05-21T15:32:33.683276+0000","last_unstale":"2024-05-21T15:34:24.019138+0000","last_undegraded":"2024-05-21T15:34:24.019138+0000","last_fullsized":"2024-05-21T15:34:24.019138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:07:14.947240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017746+0000","last_change":"2024-05-21T15:32:33.690956+0000","last_active":"2024-05-21T15:34:24.017746+0000","last_peered":"2024-05-21T15:34:24.017746+0000","last_clean":"2024-05-21T15:34:24.017746+0000","last_became_active":"2024-05-21T15:32:33.690630+0000","last_became_peered":"2024-05-21T15:32:33.690630+0000","last_unstale":"2024-05-21T15:34:24.017746+0000","last_undegraded":"2024-05-21T15:34:24.017746+0000","last_fullsized":"2024-05-21T15:34:24.017746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:24:07.050928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":147,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:37.168923+0000","last_change":"2024-05-21T15:32:33.175127+0000","last_active":"2024-05-21T15:34:37.168923+0000","last_peered":"2024-05-21T15:34:37.168923+0000","last_clean":"2024-05-21T15:34:37.168923+0000","last_became_active":"2024-05-21T15:32:33.174724+0000","last_became_peered":"2024-05-21T15:32:33.174724+0000","last_unstale":"2024-05-21T15:34:37.168923+0000","last_undegraded":"2024-05-21T15:34:37.168923+0000","last_fullsized":"2024-05-21T15:34:37.168923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:39:17.499743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021701+0000","last_change":"2024-05-21T15:32:33.693428+0000","last_active":"2024-05-21T15:34:24.021701+0000","last_peered":"2024-05-21T15:34:24.021701+0000","last_clean":"2024-05-21T15:34:24.021701+0000","last_became_active":"2024-05-21T15:32:33.693077+0000","last_became_peered":"2024-05-21T15:32:33.693077+0000","last_unstale":"2024-05-21T15:34:24.021701+0000","last_undegraded":"2024-05-21T15:34:24.021701+0000","last_fullsized":"2024-05-21T15:34:24.021701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:08:11.908874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143504+0000","last_change":"2024-05-21T15:32:33.176267+0000","last_active":"2024-05-21T15:34:24.143504+0000","last_peered":"2024-05-21T15:34:24.143504+0000","last_clean":"2024-05-21T15:34:24.143504+0000","last_became_active":"2024-05-21T15:32:33.175856+0000","last_became_peered":"2024-05-21T15:32:33.175856+0000","last_unstale":"2024-05-21T15:34:24.143504+0000","last_undegraded":"2024-05-21T15:34:24.143504+0000","last_fullsized":"2024-05-21T15:34:24.143504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:58.113791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021740+0000","last_change":"2024-05-21T15:31:20.928694+0000","last_active":"2024-05-21T15:34:24.021740+0000","last_peered":"2024-05-21T15:34:24.021740+0000","last_clean":"2024-05-21T15:34:24.021740+0000","last_became_active":"2024-05-21T15:31:20.928115+0000","last_became_peered":"2024-05-21T15:31:20.928115+0000","last_unstale":"2024-05-21T15:34:24.021740+0000","last_undegraded":"2024-05-21T15:34:24.021740+0000","last_fullsized":"2024-05-21T15:34:24.021740+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:27:50.788960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:27:50.788960+0000","last_clean_scrub_stamp":"2024-05-21T15:27:50.788960+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:55:10.393374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016306+0000","last_change":"2024-05-21T15:32:33.689305+0000","last_active":"2024-05-21T15:34:24.016306+0000","last_peered":"2024-05-21T15:34:24.016306+0000","last_clean":"2024-05-21T15:34:24.016306+0000","last_became_active":"2024-05-21T15:32:33.688908+0000","last_became_peered":"2024-05-21T15:32:33.688908+0000","last_unstale":"2024-05-21T15:34:24.016306+0000","last_undegraded":"2024-05-21T15:34:24.016306+0000","last_fullsized":"2024-05-21T15:34:24.016306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:49:18.953116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022046+0000","last_change":"2024-05-21T15:32:33.187781+0000","last_active":"2024-05-21T15:34:24.022046+0000","last_peered":"2024-05-21T15:34:24.022046+0000","last_clean":"2024-05-21T15:34:24.022046+0000","last_became_active":"2024-05-21T15:32:33.187504+0000","last_became_peered":"2024-05-21T15:32:33.187504+0000","last_unstale":"2024-05-21T15:34:24.022046+0000","last_undegraded":"2024-05-21T15:34:24.022046+0000","last_fullsized":"2024-05-21T15:34:24.022046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:30.400209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021855+0000","last_change":"2024-05-21T15:32:33.188706+0000","last_active":"2024-05-21T15:34:24.021855+0000","last_peered":"2024-05-21T15:34:24.021855+0000","last_clean":"2024-05-21T15:34:24.021855+0000","last_became_active":"2024-05-21T15:32:33.187462+0000","last_became_peered":"2024-05-21T15:32:33.187462+0000","last_unstale":"2024-05-21T15:34:24.021855+0000","last_undegraded":"2024-05-21T15:34:24.021855+0000","last_fullsized":"2024-05-21T15:34:24.021855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:47:45.387721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016776+0000","last_change":"2024-05-21T15:32:33.672531+0000","last_active":"2024-05-21T15:34:24.016776+0000","last_peered":"2024-05-21T15:34:24.016776+0000","last_clean":"2024-05-21T15:34:24.016776+0000","last_became_active":"2024-05-21T15:32:33.671972+0000","last_became_peered":"2024-05-21T15:32:33.671972+0000","last_unstale":"2024-05-21T15:34:24.016776+0000","last_undegraded":"2024-05-21T15:34:24.016776+0000","last_fullsized":"2024-05-21T15:34:24.016776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:10:51.062530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'14","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143042+0000","last_change":"2024-05-21T15:32:33.181274+0000","last_active":"2024-05-21T15:34:24.143042+0000","last_peered":"2024-05-21T15:34:24.143042+0000","last_clean":"2024-05-21T15:34:24.143042+0000","last_became_active":"2024-05-21T15:32:33.180885+0000","last_became_peered":"2024-05-21T15:32:33.180885+0000","last_unstale":"2024-05-21T15:34:24.143042+0000","last_undegraded":"2024-05-21T15:34:24.143042+0000","last_fullsized":"2024-05-21T15:34:24.143042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:10:13.547380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022012+0000","last_change":"2024-05-21T15:32:33.683749+0000","last_active":"2024-05-21T15:34:24.022012+0000","last_peered":"2024-05-21T15:34:24.022012+0000","last_clean":"2024-05-21T15:34:24.022012+0000","last_became_active":"2024-05-21T15:32:33.683393+0000","last_became_peered":"2024-05-21T15:32:33.683393+0000","last_unstale":"2024-05-21T15:34:24.022012+0000","last_undegraded":"2024-05-21T15:34:24.022012+0000","last_fullsized":"2024-05-21T15:34:24.022012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:07:53.009651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143760+0000","last_change":"2024-05-21T15:32:33.184740+0000","last_active":"2024-05-21T15:34:24.143760+0000","last_peered":"2024-05-21T15:34:24.143760+0000","last_clean":"2024-05-21T15:34:24.143760+0000","last_became_active":"2024-05-21T15:32:33.184378+0000","last_became_peered":"2024-05-21T15:32:33.184378+0000","last_unstale":"2024-05-21T15:34:24.143760+0000","last_undegraded":"2024-05-21T15:34:24.143760+0000","last_fullsized":"2024-05-21T15:34:24.143760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:21:27.610513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143379+0000","last_change":"2024-05-21T15:33:23.381090+0000","last_active":"2024-05-21T15:34:24.143379+0000","last_peered":"2024-05-21T15:34:24.143379+0000","last_clean":"2024-05-21T15:34:24.143379+0000","last_became_active":"2024-05-21T15:33:23.380680+0000","last_became_peered":"2024-05-21T15:33:23.380680+0000","last_unstale":"2024-05-21T15:34:24.143379+0000","last_undegraded":"2024-05-21T15:34:24.143379+0000","last_fullsized":"2024-05-21T15:34:24.143379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:26:03.371476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017604+0000","last_change":"2024-05-21T15:32:33.183344+0000","last_active":"2024-05-21T15:34:24.017604+0000","last_peered":"2024-05-21T15:34:24.017604+0000","last_clean":"2024-05-21T15:34:24.017604+0000","last_became_active":"2024-05-21T15:32:33.182977+0000","last_became_peered":"2024-05-21T15:32:33.182977+0000","last_unstale":"2024-05-21T15:34:24.017604+0000","last_undegraded":"2024-05-21T15:34:24.017604+0000","last_fullsized":"2024-05-21T15:34:24.017604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:27:24.354129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021213+0000","last_change":"2024-05-21T15:32:33.677549+0000","last_active":"2024-05-21T15:34:24.021213+0000","last_peered":"2024-05-21T15:34:24.021213+0000","last_clean":"2024-05-21T15:34:24.021213+0000","last_became_active":"2024-05-21T15:32:33.677213+0000","last_became_peered":"2024-05-21T15:32:33.677213+0000","last_unstale":"2024-05-21T15:34:24.021213+0000","last_undegraded":"2024-05-21T15:34:24.021213+0000","last_fullsized":"2024-05-21T15:34:24.021213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:57:06.560434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021547+0000","last_change":"2024-05-21T15:32:33.190023+0000","last_active":"2024-05-21T15:34:24.021547+0000","last_peered":"2024-05-21T15:34:24.021547+0000","last_clean":"2024-05-21T15:34:24.021547+0000","last_became_active":"2024-05-21T15:32:33.189616+0000","last_became_peered":"2024-05-21T15:32:33.189616+0000","last_unstale":"2024-05-21T15:34:24.021547+0000","last_undegraded":"2024-05-21T15:34:24.021547+0000","last_fullsized":"2024-05-21T15:34:24.021547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:03.841414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019511+0000","last_change":"2024-05-21T15:32:33.685911+0000","last_active":"2024-05-21T15:34:24.019511+0000","last_peered":"2024-05-21T15:34:24.019511+0000","last_clean":"2024-05-21T15:34:24.019511+0000","last_became_active":"2024-05-21T15:32:33.684785+0000","last_became_peered":"2024-05-21T15:32:33.684785+0000","last_unstale":"2024-05-21T15:34:24.019511+0000","last_undegraded":"2024-05-21T15:34:24.019511+0000","last_fullsized":"2024-05-21T15:34:24.019511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:39:18.370289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142681+0000","last_change":"2024-05-21T15:32:33.178354+0000","last_active":"2024-05-21T15:34:24.142681+0000","last_peered":"2024-05-21T15:34:24.142681+0000","last_clean":"2024-05-21T15:34:24.142681+0000","last_became_active":"2024-05-21T15:32:33.177989+0000","last_became_peered":"2024-05-21T15:32:33.177989+0000","last_unstale":"2024-05-21T15:34:24.142681+0000","last_undegraded":"2024-05-21T15:34:24.142681+0000","last_fullsized":"2024-05-21T15:34:24.142681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:58.650831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021792+0000","last_change":"2024-05-21T15:32:33.686467+0000","last_active":"2024-05-21T15:34:24.021792+0000","last_peered":"2024-05-21T15:34:24.021792+0000","last_clean":"2024-05-21T15:34:24.021792+0000","last_became_active":"2024-05-21T15:32:33.686135+0000","last_became_peered":"2024-05-21T15:32:33.686135+0000","last_unstale":"2024-05-21T15:34:24.021792+0000","last_undegraded":"2024-05-21T15:34:24.021792+0000","last_fullsized":"2024-05-21T15:34:24.021792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:53:54.045591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019621+0000","last_change":"2024-05-21T15:33:23.383157+0000","last_active":"2024-05-21T15:34:24.019621+0000","last_peered":"2024-05-21T15:34:24.019621+0000","last_clean":"2024-05-21T15:34:24.019621+0000","last_became_active":"2024-05-21T15:33:23.376834+0000","last_became_peered":"2024-05-21T15:33:23.376834+0000","last_unstale":"2024-05-21T15:34:24.019621+0000","last_undegraded":"2024-05-21T15:34:24.019621+0000","last_fullsized":"2024-05-21T15:34:24.019621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:01:19.349486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022263+0000","last_change":"2024-05-21T15:32:33.692325+0000","last_active":"2024-05-21T15:34:24.022263+0000","last_peered":"2024-05-21T15:34:24.022263+0000","last_clean":"2024-05-21T15:34:24.022263+0000","last_became_active":"2024-05-21T15:32:33.691971+0000","last_became_peered":"2024-05-21T15:32:33.691971+0000","last_unstale":"2024-05-21T15:34:24.022263+0000","last_undegraded":"2024-05-21T15:34:24.022263+0000","last_fullsized":"2024-05-21T15:34:24.022263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:53:28.190063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:38.192163+0000","last_change":"2024-05-21T15:32:33.186925+0000","last_active":"2024-05-21T15:34:38.192163+0000","last_peered":"2024-05-21T15:34:38.192163+0000","last_clean":"2024-05-21T15:34:38.192163+0000","last_became_active":"2024-05-21T15:32:33.186605+0000","last_became_peered":"2024-05-21T15:32:33.186605+0000","last_unstale":"2024-05-21T15:34:38.192163+0000","last_undegraded":"2024-05-21T15:34:38.192163+0000","last_fullsized":"2024-05-21T15:34:38.192163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:32.752746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022284+0000","last_change":"2024-05-21T15:32:33.686132+0000","last_active":"2024-05-21T15:34:24.022284+0000","last_peered":"2024-05-21T15:34:24.022284+0000","last_clean":"2024-05-21T15:34:24.022284+0000","last_became_active":"2024-05-21T15:32:33.685745+0000","last_became_peered":"2024-05-21T15:32:33.685745+0000","last_unstale":"2024-05-21T15:34:24.022284+0000","last_undegraded":"2024-05-21T15:34:24.022284+0000","last_fullsized":"2024-05-21T15:34:24.022284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:51:49.522297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017771+0000","last_change":"2024-05-21T15:32:33.176552+0000","last_active":"2024-05-21T15:34:24.017771+0000","last_peered":"2024-05-21T15:34:24.017771+0000","last_clean":"2024-05-21T15:34:24.017771+0000","last_became_active":"2024-05-21T15:32:33.176237+0000","last_became_peered":"2024-05-21T15:32:33.176237+0000","last_unstale":"2024-05-21T15:34:24.017771+0000","last_undegraded":"2024-05-21T15:34:24.017771+0000","last_fullsized":"2024-05-21T15:34:24.017771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:09.098344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022243+0000","last_change":"2024-05-21T15:32:33.692310+0000","last_active":"2024-05-21T15:34:24.022243+0000","last_peered":"2024-05-21T15:34:24.022243+0000","last_clean":"2024-05-21T15:34:24.022243+0000","last_became_active":"2024-05-21T15:32:33.691959+0000","last_became_peered":"2024-05-21T15:32:33.691959+0000","last_unstale":"2024-05-21T15:34:24.022243+0000","last_undegraded":"2024-05-21T15:34:24.022243+0000","last_fullsized":"2024-05-21T15:34:24.022243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:57:38.243926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016714+0000","last_change":"2024-05-21T15:33:23.614280+0000","last_active":"2024-05-21T15:34:24.016714+0000","last_peered":"2024-05-21T15:34:24.016714+0000","last_clean":"2024-05-21T15:34:24.016714+0000","last_became_active":"2024-05-21T15:33:23.610364+0000","last_became_peered":"2024-05-21T15:33:23.610364+0000","last_unstale":"2024-05-21T15:34:24.016714+0000","last_undegraded":"2024-05-21T15:34:24.016714+0000","last_fullsized":"2024-05-21T15:34:24.016714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:40:42.990949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022268+0000","last_change":"2024-05-21T15:32:33.682801+0000","last_active":"2024-05-21T15:34:24.022268+0000","last_peered":"2024-05-21T15:34:24.022268+0000","last_clean":"2024-05-21T15:34:24.022268+0000","last_became_active":"2024-05-21T15:32:33.682368+0000","last_became_peered":"2024-05-21T15:32:33.682368+0000","last_unstale":"2024-05-21T15:34:24.022268+0000","last_undegraded":"2024-05-21T15:34:24.022268+0000","last_fullsized":"2024-05-21T15:34:24.022268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:56:50.520672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019605+0000","last_change":"2024-05-21T15:32:33.684809+0000","last_active":"2024-05-21T15:34:24.019605+0000","last_peered":"2024-05-21T15:34:24.019605+0000","last_clean":"2024-05-21T15:34:24.019605+0000","last_became_active":"2024-05-21T15:32:33.684067+0000","last_became_peered":"2024-05-21T15:32:33.684067+0000","last_unstale":"2024-05-21T15:34:24.019605+0000","last_undegraded":"2024-05-21T15:34:24.019605+0000","last_fullsized":"2024-05-21T15:34:24.019605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:31:58.997470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142775+0000","last_change":"2024-05-21T15:32:33.676091+0000","last_active":"2024-05-21T15:34:24.142775+0000","last_peered":"2024-05-21T15:34:24.142775+0000","last_clean":"2024-05-21T15:34:24.142775+0000","last_became_active":"2024-05-21T15:32:33.675538+0000","last_became_peered":"2024-05-21T15:32:33.675538+0000","last_unstale":"2024-05-21T15:34:24.142775+0000","last_undegraded":"2024-05-21T15:34:24.142775+0000","last_fullsized":"2024-05-21T15:34:24.142775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:50.311152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142726+0000","last_change":"2024-05-21T15:32:33.688590+0000","last_active":"2024-05-21T15:34:24.142726+0000","last_peered":"2024-05-21T15:34:24.142726+0000","last_clean":"2024-05-21T15:34:24.142726+0000","last_became_active":"2024-05-21T15:32:33.687993+0000","last_became_peered":"2024-05-21T15:32:33.687993+0000","last_unstale":"2024-05-21T15:34:24.142726+0000","last_undegraded":"2024-05-21T15:34:24.142726+0000","last_fullsized":"2024-05-21T15:34:24.142726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:21.939483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143010+0000","last_change":"2024-05-21T15:33:23.380890+0000","last_active":"2024-05-21T15:34:24.143010+0000","last_peered":"2024-05-21T15:34:24.143010+0000","last_clean":"2024-05-21T15:34:24.143010+0000","last_became_active":"2024-05-21T15:33:23.380503+0000","last_became_peered":"2024-05-21T15:33:23.380503+0000","last_unstale":"2024-05-21T15:34:24.143010+0000","last_undegraded":"2024-05-21T15:34:24.143010+0000","last_fullsized":"2024-05-21T15:34:24.143010+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:41.070336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016602+0000","last_change":"2024-05-21T15:32:33.689365+0000","last_active":"2024-05-21T15:34:24.016602+0000","last_peered":"2024-05-21T15:34:24.016602+0000","last_clean":"2024-05-21T15:34:24.016602+0000","last_became_active":"2024-05-21T15:32:33.688739+0000","last_became_peered":"2024-05-21T15:32:33.688739+0000","last_unstale":"2024-05-21T15:34:24.016602+0000","last_undegraded":"2024-05-21T15:34:24.016602+0000","last_fullsized":"2024-05-21T15:34:24.016602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:19:52.614155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.020003+0000","last_change":"2024-05-21T15:32:33.681985+0000","last_active":"2024-05-21T15:34:24.020003+0000","last_peered":"2024-05-21T15:34:24.020003+0000","last_clean":"2024-05-21T15:34:24.020003+0000","last_became_active":"2024-05-21T15:32:33.681209+0000","last_became_peered":"2024-05-21T15:32:33.681209+0000","last_unstale":"2024-05-21T15:34:24.020003+0000","last_undegraded":"2024-05-21T15:34:24.020003+0000","last_fullsized":"2024-05-21T15:34:24.020003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:03:16.088136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143161+0000","last_change":"2024-05-21T15:32:33.673236+0000","last_active":"2024-05-21T15:34:24.143161+0000","last_peered":"2024-05-21T15:34:24.143161+0000","last_clean":"2024-05-21T15:34:24.143161+0000","last_became_active":"2024-05-21T15:32:33.672548+0000","last_became_peered":"2024-05-21T15:32:33.672548+0000","last_unstale":"2024-05-21T15:34:24.143161+0000","last_undegraded":"2024-05-21T15:34:24.143161+0000","last_fullsized":"2024-05-21T15:34:24.143161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:52:10.010219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143155+0000","last_change":"2024-05-21T15:33:23.482874+0000","last_active":"2024-05-21T15:34:24.143155+0000","last_peered":"2024-05-21T15:34:24.143155+0000","last_clean":"2024-05-21T15:34:24.143155+0000","last_became_active":"2024-05-21T15:33:23.482589+0000","last_became_peered":"2024-05-21T15:33:23.482589+0000","last_unstale":"2024-05-21T15:34:24.143155+0000","last_undegraded":"2024-05-21T15:34:24.143155+0000","last_fullsized":"2024-05-21T15:34:24.143155+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:19:45.870562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017577+0000","last_change":"2024-05-21T15:32:33.686992+0000","last_active":"2024-05-21T15:34:24.017577+0000","last_peered":"2024-05-21T15:34:24.017577+0000","last_clean":"2024-05-21T15:34:24.017577+0000","last_became_active":"2024-05-21T15:32:33.686673+0000","last_became_peered":"2024-05-21T15:32:33.686673+0000","last_unstale":"2024-05-21T15:34:24.017577+0000","last_undegraded":"2024-05-21T15:34:24.017577+0000","last_fullsized":"2024-05-21T15:34:24.017577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:18:20.668365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021467+0000","last_change":"2024-05-21T15:32:33.689433+0000","last_active":"2024-05-21T15:34:24.021467+0000","last_peered":"2024-05-21T15:34:24.021467+0000","last_clean":"2024-05-21T15:34:24.021467+0000","last_became_active":"2024-05-21T15:32:33.689106+0000","last_became_peered":"2024-05-21T15:32:33.689106+0000","last_unstale":"2024-05-21T15:34:24.021467+0000","last_undegraded":"2024-05-21T15:34:24.021467+0000","last_fullsized":"2024-05-21T15:34:24.021467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:45:17.841159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017934+0000","last_change":"2024-05-21T15:32:33.687763+0000","last_active":"2024-05-21T15:34:24.017934+0000","last_peered":"2024-05-21T15:34:24.017934+0000","last_clean":"2024-05-21T15:34:24.017934+0000","last_became_active":"2024-05-21T15:32:33.685103+0000","last_became_peered":"2024-05-21T15:32:33.685103+0000","last_unstale":"2024-05-21T15:34:24.017934+0000","last_undegraded":"2024-05-21T15:34:24.017934+0000","last_fullsized":"2024-05-21T15:34:24.017934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:56:39.454536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016565+0000","last_change":"2024-05-21T15:32:33.679225+0000","last_active":"2024-05-21T15:34:24.016565+0000","last_peered":"2024-05-21T15:34:24.016565+0000","last_clean":"2024-05-21T15:34:24.016565+0000","last_became_active":"2024-05-21T15:32:33.678890+0000","last_became_peered":"2024-05-21T15:32:33.678890+0000","last_unstale":"2024-05-21T15:34:24.016565+0000","last_undegraded":"2024-05-21T15:34:24.016565+0000","last_fullsized":"2024-05-21T15:34:24.016565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:12:23.721449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019618+0000","last_change":"2024-05-21T15:32:33.688659+0000","last_active":"2024-05-21T15:34:24.019618+0000","last_peered":"2024-05-21T15:34:24.019618+0000","last_clean":"2024-05-21T15:34:24.019618+0000","last_became_active":"2024-05-21T15:32:33.688293+0000","last_became_peered":"2024-05-21T15:32:33.688293+0000","last_unstale":"2024-05-21T15:34:24.019618+0000","last_undegraded":"2024-05-21T15:34:24.019618+0000","last_fullsized":"2024-05-21T15:34:24.019618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:08:49.543237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":287606,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66976,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1206,"num_read_kb":675,"num_write":246,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":202,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":206301,"data_compressed":5562,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":477685,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1128114,"data_compressed":50013,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776176,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":476993,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":477371,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":1069498,"data_compressed":48159,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":476710,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":537337,"data_compressed":13445,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":478581,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":543658,"data_compressed":13445,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":476535,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888139,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":999424,"data_stored":476683,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724613,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":1069086,"data_compressed":48159,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61774,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67787,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61758,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T15:34:41.188 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-21T15:34:41.189 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.340+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2981789036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b58108a20 msgr2=0x7f7b5810ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.340+0000 7f7b5eed8640 1 --2- 172.21.15.38:0/2981789036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b58108a20 0x7f7b5810ae40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009900 tx=0x7f7b4002a410 comp rx=0 tx=0).stop 2024-05-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.340+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2981789036 shutdown_connections 2024-05-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.340+0000 7f7b5eed8640 1 --2- 172.21.15.38:0/2981789036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b58108a20 0x7f7b5810ae40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.340+0000 7f7b5eed8640 1 --2- 172.21.15.38:0/2981789036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b581027e0 0x7f7b58102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.340+0000 7f7b5eed8640 1 --2- 172.21.15.38:0/2981789036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b58104120 0x7f7b581022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.340+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2981789036 >> 172.21.15.38:0/2981789036 conn(0x7f7b580fdcf0 msgr2=0x7f7b58100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.340+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2981789036 shutdown_connections 2024-05-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.341+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2981789036 wait complete. 2024-05-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.341+0000 7f7b5eed8640 1 Processor -- start 2024-05-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.341+0000 7f7b5eed8640 1 -- start start 2024-05-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.341+0000 7f7b5eed8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b581027e0 0x7f7b58199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.341+0000 7f7b5eed8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b58104120 0x7f7b58199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.341+0000 7f7b5eed8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b58108a20 0x7f7b581a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.341+0000 7f7b5eed8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5810e870 con 0x7f7b58104120 2024-05-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.341+0000 7f7b5eed8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5810e6f0 con 0x7f7b581027e0 2024-05-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.341+0000 7f7b5eed8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5810e9f0 con 0x7f7b58108a20 2024-05-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b57fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b58104120 0x7f7b58199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b57fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b58104120 0x7f7b58199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59992/0 (socket says 172.21.15.38:59992) 2024-05-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b57fff640 1 -- 172.21.15.38:0/2094140759 learned_addr learned my addr 172.21.15.38:0/2094140759 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b5d475640 1 --2- 172.21.15.38:0/2094140759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b58108a20 0x7f7b581a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b5cc74640 1 --2- 172.21.15.38:0/2094140759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b581027e0 0x7f7b58199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b57fff640 1 -- 172.21.15.38:0/2094140759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b58108a20 msgr2=0x7f7b581a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b57fff640 1 --2- 172.21.15.38:0/2094140759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b58108a20 0x7f7b581a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b57fff640 1 -- 172.21.15.38:0/2094140759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b581027e0 msgr2=0x7f7b58199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b57fff640 1 --2- 172.21.15.38:0/2094140759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b581027e0 0x7f7b58199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b57fff640 1 -- 172.21.15.38:0/2094140759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b581a0e90 con 0x7f7b58104120 2024-05-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b57fff640 1 --2- 172.21.15.38:0/2094140759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b58104120 0x7f7b58199890 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800ebc0 tx=0x7f7b4800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b55ffb640 1 -- 172.21.15.38:0/2094140759 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48010630 con 0x7f7b58104120 2024-05-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b55ffb640 1 -- 172.21.15.38:0/2094140759 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b48010cb0 con 0x7f7b58104120 2024-05-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2094140759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b581a1180 con 0x7f7b58104120 2024-05-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.342+0000 7f7b55ffb640 1 -- 172.21.15.38:0/2094140759 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48018bd0 con 0x7f7b58104120 2024-05-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.343+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2094140759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b581a16d0 con 0x7f7b58104120 2024-05-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.344+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2094140759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b580691c0 con 0x7f7b58104120 2024-05-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.345+0000 7f7b55ffb640 1 -- 172.21.15.38:0/2094140759 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7b480107d0 con 0x7f7b58104120 2024-05-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.346+0000 7f7b55ffb640 1 --2- 172.21.15.38:0/2094140759 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7b440a2c30 0x7f7b440a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.346+0000 7f7b5cc74640 1 --2- 172.21.15.38:0/2094140759 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7b440a2c30 0x7f7b440a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.346+0000 7f7b55ffb640 1 -- 172.21.15.38:0/2094140759 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7b480cf870 con 0x7f7b58104120 2024-05-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.346+0000 7f7b5cc74640 1 --2- 172.21.15.38:0/2094140759 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7b440a2c30 0x7f7b440a5140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7b50005fd0 tx=0x7f7b50005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:41.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.350+0000 7f7b55ffb640 1 -- 172.21.15.38:0/2094140759 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7b4809c720 con 0x7f7b58104120 2024-05-21T15:34:41.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:41 smithi038 ceph-mon[31995]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:34:41.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:41 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2974519708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:41.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:41 smithi038 ceph-mon[31995]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:34:41.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.536+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2094140759 --> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b5802d580 con 0x7f7b440a2c30 2024-05-21T15:34:41.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.543+0000 7f7b55ffb640 1 -- 172.21.15.38:0/2094140759 <== mgr.14152 v2:172.21.15.38:6800/1295226559 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259204 (secure 0 0 0) 0x7f7b5802d580 con 0x7f7b440a2c30 2024-05-21T15:34:41.546 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:41.547 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-05-21T15:34:41.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.546+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2094140759 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7b440a2c30 msgr2=0x7f7b440a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:41.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.546+0000 7f7b5eed8640 1 --2- 172.21.15.38:0/2094140759 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7b440a2c30 0x7f7b440a5140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7b50005fd0 tx=0x7f7b50005850 comp rx=0 tx=0).stop 2024-05-21T15:34:41.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.546+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2094140759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b58104120 msgr2=0x7f7b58199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:41.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.546+0000 7f7b5eed8640 1 --2- 172.21.15.38:0/2094140759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b58104120 0x7f7b58199890 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800ebc0 tx=0x7f7b4800c6a0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.546+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2094140759 shutdown_connections 2024-05-21T15:34:41.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.546+0000 7f7b5eed8640 1 --2- 172.21.15.38:0/2094140759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b58108a20 0x7f7b581a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.546+0000 7f7b5eed8640 1 --2- 172.21.15.38:0/2094140759 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7b440a2c30 0x7f7b440a5140 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.546+0000 7f7b5eed8640 1 --2- 172.21.15.38:0/2094140759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b58104120 0x7f7b58199890 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.546+0000 7f7b5eed8640 1 --2- 172.21.15.38:0/2094140759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b581027e0 0x7f7b58199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.546+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2094140759 >> 172.21.15.38:0/2094140759 conn(0x7f7b580fdcf0 msgr2=0x7f7b58100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:41.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.547+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2094140759 shutdown_connections 2024-05-21T15:34:41.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.547+0000 7f7b5eed8640 1 -- 172.21.15.38:0/2094140759 wait complete. 2024-05-21T15:34:41.558 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-05-21T15:34:40.249259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657578,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1332,"num_read_kb":784,"num_write":429,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180938,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":852,"ondisk_log_size":852,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373788,"kb_used_data":12088,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568036,"statfs":{"total":1151940427776,"available":1151557668864,"internally_reserved":0,"allocated":12378112,"data_stored":7688251,"data_compressed":254358,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":51,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003152"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.020232+0000","last_change":"2024-05-21T15:32:33.687244+0000","last_active":"2024-05-21T15:34:24.020232+0000","last_peered":"2024-05-21T15:34:24.020232+0000","last_clean":"2024-05-21T15:34:24.020232+0000","last_became_active":"2024-05-21T15:32:33.686515+0000","last_became_peered":"2024-05-21T15:32:33.686515+0000","last_unstale":"2024-05-21T15:34:24.020232+0000","last_undegraded":"2024-05-21T15:34:24.020232+0000","last_fullsized":"2024-05-21T15:34:24.020232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:10:18.396997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143144+0000","last_change":"2024-05-21T15:32:33.678497+0000","last_active":"2024-05-21T15:34:24.143144+0000","last_peered":"2024-05-21T15:34:24.143144+0000","last_clean":"2024-05-21T15:34:24.143144+0000","last_became_active":"2024-05-21T15:32:33.678001+0000","last_became_peered":"2024-05-21T15:32:33.678001+0000","last_unstale":"2024-05-21T15:34:24.143144+0000","last_undegraded":"2024-05-21T15:34:24.143144+0000","last_fullsized":"2024-05-21T15:34:24.143144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:14:48.255675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143214+0000","last_change":"2024-05-21T15:32:33.692444+0000","last_active":"2024-05-21T15:34:24.143214+0000","last_peered":"2024-05-21T15:34:24.143214+0000","last_clean":"2024-05-21T15:34:24.143214+0000","last_became_active":"2024-05-21T15:32:33.691662+0000","last_became_peered":"2024-05-21T15:32:33.691662+0000","last_unstale":"2024-05-21T15:34:24.143214+0000","last_undegraded":"2024-05-21T15:34:24.143214+0000","last_fullsized":"2024-05-21T15:34:24.143214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:26:25.929879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143818+0000","last_change":"2024-05-21T15:32:33.692297+0000","last_active":"2024-05-21T15:34:24.143818+0000","last_peered":"2024-05-21T15:34:24.143818+0000","last_clean":"2024-05-21T15:34:24.143818+0000","last_became_active":"2024-05-21T15:32:33.691925+0000","last_became_peered":"2024-05-21T15:32:33.691925+0000","last_unstale":"2024-05-21T15:34:24.143818+0000","last_undegraded":"2024-05-21T15:34:24.143818+0000","last_fullsized":"2024-05-21T15:34:24.143818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:40:45.448485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021213+0000","last_change":"2024-05-21T15:32:33.685511+0000","last_active":"2024-05-21T15:34:24.021213+0000","last_peered":"2024-05-21T15:34:24.021213+0000","last_clean":"2024-05-21T15:34:24.021213+0000","last_became_active":"2024-05-21T15:32:33.685182+0000","last_became_peered":"2024-05-21T15:32:33.685182+0000","last_unstale":"2024-05-21T15:34:24.021213+0000","last_undegraded":"2024-05-21T15:34:24.021213+0000","last_fullsized":"2024-05-21T15:34:24.021213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:59:06.680712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019738+0000","last_change":"2024-05-21T15:32:33.682400+0000","last_active":"2024-05-21T15:34:24.019738+0000","last_peered":"2024-05-21T15:34:24.019738+0000","last_clean":"2024-05-21T15:34:24.019738+0000","last_became_active":"2024-05-21T15:32:33.682068+0000","last_became_peered":"2024-05-21T15:32:33.682068+0000","last_unstale":"2024-05-21T15:34:24.019738+0000","last_undegraded":"2024-05-21T15:34:24.019738+0000","last_fullsized":"2024-05-21T15:34:24.019738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:31:50.968706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021817+0000","last_change":"2024-05-21T15:32:33.690099+0000","last_active":"2024-05-21T15:34:24.021817+0000","last_peered":"2024-05-21T15:34:24.021817+0000","last_clean":"2024-05-21T15:34:24.021817+0000","last_became_active":"2024-05-21T15:32:33.689762+0000","last_became_peered":"2024-05-21T15:32:33.689762+0000","last_unstale":"2024-05-21T15:34:24.021817+0000","last_undegraded":"2024-05-21T15:34:24.021817+0000","last_fullsized":"2024-05-21T15:34:24.021817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:55:20.331097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021396+0000","last_change":"2024-05-21T15:32:33.691701+0000","last_active":"2024-05-21T15:34:24.021396+0000","last_peered":"2024-05-21T15:34:24.021396+0000","last_clean":"2024-05-21T15:34:24.021396+0000","last_became_active":"2024-05-21T15:32:33.691386+0000","last_became_peered":"2024-05-21T15:32:33.691386+0000","last_unstale":"2024-05-21T15:34:24.021396+0000","last_undegraded":"2024-05-21T15:34:24.021396+0000","last_fullsized":"2024-05-21T15:34:24.021396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:54:45.691392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.018266+0000","last_change":"2024-05-21T15:32:33.677484+0000","last_active":"2024-05-21T15:34:24.018266+0000","last_peered":"2024-05-21T15:34:24.018266+0000","last_clean":"2024-05-21T15:34:24.018266+0000","last_became_active":"2024-05-21T15:32:33.677160+0000","last_became_peered":"2024-05-21T15:32:33.677160+0000","last_unstale":"2024-05-21T15:34:24.018266+0000","last_undegraded":"2024-05-21T15:34:24.018266+0000","last_fullsized":"2024-05-21T15:34:24.018266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:38:00.914393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019574+0000","last_change":"2024-05-21T15:32:33.686957+0000","last_active":"2024-05-21T15:34:24.019574+0000","last_peered":"2024-05-21T15:34:24.019574+0000","last_clean":"2024-05-21T15:34:24.019574+0000","last_became_active":"2024-05-21T15:32:33.685338+0000","last_became_peered":"2024-05-21T15:32:33.685338+0000","last_unstale":"2024-05-21T15:34:24.019574+0000","last_undegraded":"2024-05-21T15:34:24.019574+0000","last_fullsized":"2024-05-21T15:34:24.019574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:04:58.115974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022284+0000","last_change":"2024-05-21T15:32:33.693311+0000","last_active":"2024-05-21T15:34:24.022284+0000","last_peered":"2024-05-21T15:34:24.022284+0000","last_clean":"2024-05-21T15:34:24.022284+0000","last_became_active":"2024-05-21T15:32:33.692959+0000","last_became_peered":"2024-05-21T15:32:33.692959+0000","last_unstale":"2024-05-21T15:34:24.022284+0000","last_undegraded":"2024-05-21T15:34:24.022284+0000","last_fullsized":"2024-05-21T15:34:24.022284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:35:12.918524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143679+0000","last_change":"2024-05-21T15:32:33.693254+0000","last_active":"2024-05-21T15:34:24.143679+0000","last_peered":"2024-05-21T15:34:24.143679+0000","last_clean":"2024-05-21T15:34:24.143679+0000","last_became_active":"2024-05-21T15:32:33.692901+0000","last_became_peered":"2024-05-21T15:32:33.692901+0000","last_unstale":"2024-05-21T15:34:24.143679+0000","last_undegraded":"2024-05-21T15:34:24.143679+0000","last_fullsized":"2024-05-21T15:34:24.143679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:36:30.591479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016565+0000","last_change":"2024-05-21T15:33:23.383656+0000","last_active":"2024-05-21T15:34:24.016565+0000","last_peered":"2024-05-21T15:34:24.016565+0000","last_clean":"2024-05-21T15:34:24.016565+0000","last_became_active":"2024-05-21T15:33:23.383113+0000","last_became_peered":"2024-05-21T15:33:23.383113+0000","last_unstale":"2024-05-21T15:34:24.016565+0000","last_undegraded":"2024-05-21T15:34:24.016565+0000","last_fullsized":"2024-05-21T15:34:24.016565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:14:37.327480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142760+0000","last_change":"2024-05-21T15:32:33.678315+0000","last_active":"2024-05-21T15:34:24.142760+0000","last_peered":"2024-05-21T15:34:24.142760+0000","last_clean":"2024-05-21T15:34:24.142760+0000","last_became_active":"2024-05-21T15:32:33.677787+0000","last_became_peered":"2024-05-21T15:32:33.677787+0000","last_unstale":"2024-05-21T15:34:24.142760+0000","last_undegraded":"2024-05-21T15:34:24.142760+0000","last_fullsized":"2024-05-21T15:34:24.142760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:17:24.826525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142879+0000","last_change":"2024-05-21T15:33:23.380244+0000","last_active":"2024-05-21T15:34:24.142879+0000","last_peered":"2024-05-21T15:34:24.142879+0000","last_clean":"2024-05-21T15:34:24.142879+0000","last_became_active":"2024-05-21T15:33:23.379769+0000","last_became_peered":"2024-05-21T15:33:23.379769+0000","last_unstale":"2024-05-21T15:34:24.142879+0000","last_undegraded":"2024-05-21T15:34:24.142879+0000","last_fullsized":"2024-05-21T15:34:24.142879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:16:07.820636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017488+0000","last_change":"2024-05-21T15:32:33.683597+0000","last_active":"2024-05-21T15:34:24.017488+0000","last_peered":"2024-05-21T15:34:24.017488+0000","last_clean":"2024-05-21T15:34:24.017488+0000","last_became_active":"2024-05-21T15:32:33.681948+0000","last_became_peered":"2024-05-21T15:32:33.681948+0000","last_unstale":"2024-05-21T15:34:24.017488+0000","last_undegraded":"2024-05-21T15:34:24.017488+0000","last_fullsized":"2024-05-21T15:34:24.017488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:26:50.086019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.020426+0000","last_change":"2024-05-21T15:32:33.683329+0000","last_active":"2024-05-21T15:34:24.020426+0000","last_peered":"2024-05-21T15:34:24.020426+0000","last_clean":"2024-05-21T15:34:24.020426+0000","last_became_active":"2024-05-21T15:32:33.682947+0000","last_became_peered":"2024-05-21T15:32:33.682947+0000","last_unstale":"2024-05-21T15:34:24.020426+0000","last_undegraded":"2024-05-21T15:34:24.020426+0000","last_fullsized":"2024-05-21T15:34:24.020426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:24:27.502257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142833+0000","last_change":"2024-05-21T15:32:33.680391+0000","last_active":"2024-05-21T15:34:24.142833+0000","last_peered":"2024-05-21T15:34:24.142833+0000","last_clean":"2024-05-21T15:34:24.142833+0000","last_became_active":"2024-05-21T15:32:33.679996+0000","last_became_peered":"2024-05-21T15:32:33.679996+0000","last_unstale":"2024-05-21T15:34:24.142833+0000","last_undegraded":"2024-05-21T15:34:24.142833+0000","last_fullsized":"2024-05-21T15:34:24.142833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:56:43.658565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142946+0000","last_change":"2024-05-21T15:32:33.688783+0000","last_active":"2024-05-21T15:34:24.142946+0000","last_peered":"2024-05-21T15:34:24.142946+0000","last_clean":"2024-05-21T15:34:24.142946+0000","last_became_active":"2024-05-21T15:32:33.688006+0000","last_became_peered":"2024-05-21T15:32:33.688006+0000","last_unstale":"2024-05-21T15:34:24.142946+0000","last_undegraded":"2024-05-21T15:34:24.142946+0000","last_fullsized":"2024-05-21T15:34:24.142946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:17:44.489759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":155,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:37.162842+0000","last_change":"2024-05-21T15:32:33.182251+0000","last_active":"2024-05-21T15:34:37.162842+0000","last_peered":"2024-05-21T15:34:37.162842+0000","last_clean":"2024-05-21T15:34:37.162842+0000","last_became_active":"2024-05-21T15:32:33.181893+0000","last_became_peered":"2024-05-21T15:32:33.181893+0000","last_unstale":"2024-05-21T15:34:37.162842+0000","last_undegraded":"2024-05-21T15:34:37.162842+0000","last_fullsized":"2024-05-21T15:34:37.162842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:23:36.863186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021774+0000","last_change":"2024-05-21T15:32:33.691962+0000","last_active":"2024-05-21T15:34:24.021774+0000","last_peered":"2024-05-21T15:34:24.021774+0000","last_clean":"2024-05-21T15:34:24.021774+0000","last_became_active":"2024-05-21T15:32:33.691619+0000","last_became_peered":"2024-05-21T15:32:33.691619+0000","last_unstale":"2024-05-21T15:34:24.021774+0000","last_undegraded":"2024-05-21T15:34:24.021774+0000","last_fullsized":"2024-05-21T15:34:24.021774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:50:14.523567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022035+0000","last_change":"2024-05-21T15:32:33.183772+0000","last_active":"2024-05-21T15:34:24.022035+0000","last_peered":"2024-05-21T15:34:24.022035+0000","last_clean":"2024-05-21T15:34:24.022035+0000","last_became_active":"2024-05-21T15:32:33.183440+0000","last_became_peered":"2024-05-21T15:32:33.183440+0000","last_unstale":"2024-05-21T15:34:24.022035+0000","last_undegraded":"2024-05-21T15:34:24.022035+0000","last_fullsized":"2024-05-21T15:34:24.022035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:53:00.714359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143490+0000","last_change":"2024-05-21T15:32:33.682544+0000","last_active":"2024-05-21T15:34:24.143490+0000","last_peered":"2024-05-21T15:34:24.143490+0000","last_clean":"2024-05-21T15:34:24.143490+0000","last_became_active":"2024-05-21T15:32:33.682161+0000","last_became_peered":"2024-05-21T15:32:33.682161+0000","last_unstale":"2024-05-21T15:34:24.143490+0000","last_undegraded":"2024-05-21T15:34:24.143490+0000","last_fullsized":"2024-05-21T15:34:24.143490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:20:38.143508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021961+0000","last_change":"2024-05-21T15:32:33.182400+0000","last_active":"2024-05-21T15:34:24.021961+0000","last_peered":"2024-05-21T15:34:24.021961+0000","last_clean":"2024-05-21T15:34:24.021961+0000","last_became_active":"2024-05-21T15:32:33.181745+0000","last_became_peered":"2024-05-21T15:32:33.181745+0000","last_unstale":"2024-05-21T15:34:24.021961+0000","last_undegraded":"2024-05-21T15:34:24.021961+0000","last_fullsized":"2024-05-21T15:34:24.021961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:39:41.669529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017631+0000","last_change":"2024-05-21T15:32:33.682700+0000","last_active":"2024-05-21T15:34:24.017631+0000","last_peered":"2024-05-21T15:34:24.017631+0000","last_clean":"2024-05-21T15:34:24.017631+0000","last_became_active":"2024-05-21T15:32:33.680876+0000","last_became_peered":"2024-05-21T15:32:33.680876+0000","last_unstale":"2024-05-21T15:34:24.017631+0000","last_undegraded":"2024-05-21T15:34:24.017631+0000","last_fullsized":"2024-05-21T15:34:24.017631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:30.694520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021728+0000","last_change":"2024-05-21T15:32:33.179114+0000","last_active":"2024-05-21T15:34:24.021728+0000","last_peered":"2024-05-21T15:34:24.021728+0000","last_clean":"2024-05-21T15:34:24.021728+0000","last_became_active":"2024-05-21T15:32:33.178485+0000","last_became_peered":"2024-05-21T15:32:33.178485+0000","last_unstale":"2024-05-21T15:34:24.021728+0000","last_undegraded":"2024-05-21T15:34:24.021728+0000","last_fullsized":"2024-05-21T15:34:24.021728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:19:56.501287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017868+0000","last_change":"2024-05-21T15:32:33.691874+0000","last_active":"2024-05-21T15:34:24.017868+0000","last_peered":"2024-05-21T15:34:24.017868+0000","last_clean":"2024-05-21T15:34:24.017868+0000","last_became_active":"2024-05-21T15:32:33.691546+0000","last_became_peered":"2024-05-21T15:32:33.691546+0000","last_unstale":"2024-05-21T15:34:24.017868+0000","last_undegraded":"2024-05-21T15:34:24.017868+0000","last_fullsized":"2024-05-21T15:34:24.017868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:40:09.281792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:35.164975+0000","last_change":"2024-05-21T15:32:33.183457+0000","last_active":"2024-05-21T15:34:35.164975+0000","last_peered":"2024-05-21T15:34:35.164975+0000","last_clean":"2024-05-21T15:34:35.164975+0000","last_became_active":"2024-05-21T15:32:33.183127+0000","last_became_peered":"2024-05-21T15:32:33.183127+0000","last_unstale":"2024-05-21T15:34:35.164975+0000","last_undegraded":"2024-05-21T15:34:35.164975+0000","last_fullsized":"2024-05-21T15:34:35.164975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:41:05.179629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021554+0000","last_change":"2024-05-21T15:32:33.677260+0000","last_active":"2024-05-21T15:34:24.021554+0000","last_peered":"2024-05-21T15:34:24.021554+0000","last_clean":"2024-05-21T15:34:24.021554+0000","last_became_active":"2024-05-21T15:32:33.676905+0000","last_became_peered":"2024-05-21T15:32:33.676905+0000","last_unstale":"2024-05-21T15:34:24.021554+0000","last_undegraded":"2024-05-21T15:34:24.021554+0000","last_fullsized":"2024-05-21T15:34:24.021554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:51:41.543066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.150318+0000","last_change":"2024-05-21T15:34:24.150318+0000","last_active":"2024-05-21T15:34:24.150318+0000","last_peered":"2024-05-21T15:34:24.150318+0000","last_clean":"2024-05-21T15:34:24.150318+0000","last_became_active":"2024-05-21T15:34:24.144598+0000","last_became_peered":"2024-05-21T15:34:24.144598+0000","last_unstale":"2024-05-21T15:34:24.150318+0000","last_undegraded":"2024-05-21T15:34:24.150318+0000","last_fullsized":"2024-05-21T15:34:24.150318+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:12:00.320432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017867+0000","last_change":"2024-05-21T15:32:33.691072+0000","last_active":"2024-05-21T15:34:24.017867+0000","last_peered":"2024-05-21T15:34:24.017867+0000","last_clean":"2024-05-21T15:34:24.017867+0000","last_became_active":"2024-05-21T15:32:33.690663+0000","last_became_peered":"2024-05-21T15:32:33.690663+0000","last_unstale":"2024-05-21T15:34:24.017867+0000","last_undegraded":"2024-05-21T15:34:24.017867+0000","last_fullsized":"2024-05-21T15:34:24.017867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:52:44.549537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016485+0000","last_change":"2024-05-21T15:32:33.179588+0000","last_active":"2024-05-21T15:34:24.016485+0000","last_peered":"2024-05-21T15:34:24.016485+0000","last_clean":"2024-05-21T15:34:24.016485+0000","last_became_active":"2024-05-21T15:32:33.179115+0000","last_became_peered":"2024-05-21T15:32:33.179115+0000","last_unstale":"2024-05-21T15:34:24.016485+0000","last_undegraded":"2024-05-21T15:34:24.016485+0000","last_fullsized":"2024-05-21T15:34:24.016485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:18:43.904584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021488+0000","last_change":"2024-05-21T15:32:33.689456+0000","last_active":"2024-05-21T15:34:24.021488+0000","last_peered":"2024-05-21T15:34:24.021488+0000","last_clean":"2024-05-21T15:34:24.021488+0000","last_became_active":"2024-05-21T15:32:33.689142+0000","last_became_peered":"2024-05-21T15:32:33.689142+0000","last_unstale":"2024-05-21T15:34:24.021488+0000","last_undegraded":"2024-05-21T15:34:24.021488+0000","last_fullsized":"2024-05-21T15:34:24.021488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:00:13.768272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017507+0000","last_change":"2024-05-21T15:32:33.184865+0000","last_active":"2024-05-21T15:34:24.017507+0000","last_peered":"2024-05-21T15:34:24.017507+0000","last_clean":"2024-05-21T15:34:24.017507+0000","last_became_active":"2024-05-21T15:32:33.184540+0000","last_became_peered":"2024-05-21T15:32:33.184540+0000","last_unstale":"2024-05-21T15:34:24.017507+0000","last_undegraded":"2024-05-21T15:34:24.017507+0000","last_fullsized":"2024-05-21T15:34:24.017507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:09:20.072906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142492+0000","last_change":"2024-05-21T15:32:33.685478+0000","last_active":"2024-05-21T15:34:24.142492+0000","last_peered":"2024-05-21T15:34:24.142492+0000","last_clean":"2024-05-21T15:34:24.142492+0000","last_became_active":"2024-05-21T15:32:33.685137+0000","last_became_peered":"2024-05-21T15:32:33.685137+0000","last_unstale":"2024-05-21T15:34:24.142492+0000","last_undegraded":"2024-05-21T15:34:24.142492+0000","last_fullsized":"2024-05-21T15:34:24.142492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:39:40.815876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":262,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:37.167571+0000","last_change":"2024-05-21T15:32:33.179215+0000","last_active":"2024-05-21T15:34:37.167571+0000","last_peered":"2024-05-21T15:34:37.167571+0000","last_clean":"2024-05-21T15:34:37.167571+0000","last_became_active":"2024-05-21T15:32:33.178846+0000","last_became_peered":"2024-05-21T15:32:33.178846+0000","last_unstale":"2024-05-21T15:34:37.167571+0000","last_undegraded":"2024-05-21T15:34:37.167571+0000","last_fullsized":"2024-05-21T15:34:37.167571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:29:47.289968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":234,"num_read_kb":234,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.018136+0000","last_change":"2024-05-21T15:32:33.684900+0000","last_active":"2024-05-21T15:34:24.018136+0000","last_peered":"2024-05-21T15:34:24.018136+0000","last_clean":"2024-05-21T15:34:24.018136+0000","last_became_active":"2024-05-21T15:32:33.681374+0000","last_became_peered":"2024-05-21T15:32:33.681374+0000","last_unstale":"2024-05-21T15:34:24.018136+0000","last_undegraded":"2024-05-21T15:34:24.018136+0000","last_fullsized":"2024-05-21T15:34:24.018136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:51:38.079726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022157+0000","last_change":"2024-05-21T15:32:33.189945+0000","last_active":"2024-05-21T15:34:24.022157+0000","last_peered":"2024-05-21T15:34:24.022157+0000","last_clean":"2024-05-21T15:34:24.022157+0000","last_became_active":"2024-05-21T15:32:33.189557+0000","last_became_peered":"2024-05-21T15:32:33.189557+0000","last_unstale":"2024-05-21T15:34:24.022157+0000","last_undegraded":"2024-05-21T15:34:24.022157+0000","last_fullsized":"2024-05-21T15:34:24.022157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:08.427043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143483+0000","last_change":"2024-05-21T15:32:33.692896+0000","last_active":"2024-05-21T15:34:24.143483+0000","last_peered":"2024-05-21T15:34:24.143483+0000","last_clean":"2024-05-21T15:34:24.143483+0000","last_became_active":"2024-05-21T15:32:33.692527+0000","last_became_peered":"2024-05-21T15:32:33.692527+0000","last_unstale":"2024-05-21T15:34:24.143483+0000","last_undegraded":"2024-05-21T15:34:24.143483+0000","last_fullsized":"2024-05-21T15:34:24.143483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:45:45.418063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017955+0000","last_change":"2024-05-21T15:32:33.172989+0000","last_active":"2024-05-21T15:34:24.017955+0000","last_peered":"2024-05-21T15:34:24.017955+0000","last_clean":"2024-05-21T15:34:24.017955+0000","last_became_active":"2024-05-21T15:32:33.172215+0000","last_became_peered":"2024-05-21T15:32:33.172215+0000","last_unstale":"2024-05-21T15:34:24.017955+0000","last_undegraded":"2024-05-21T15:34:24.017955+0000","last_fullsized":"2024-05-21T15:34:24.017955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:09:39.287636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017840+0000","last_change":"2024-05-21T15:32:33.678441+0000","last_active":"2024-05-21T15:34:24.017840+0000","last_peered":"2024-05-21T15:34:24.017840+0000","last_clean":"2024-05-21T15:34:24.017840+0000","last_became_active":"2024-05-21T15:32:33.677833+0000","last_became_peered":"2024-05-21T15:32:33.677833+0000","last_unstale":"2024-05-21T15:34:24.017840+0000","last_undegraded":"2024-05-21T15:34:24.017840+0000","last_fullsized":"2024-05-21T15:34:24.017840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:36.124064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'34","reported_seq":299,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:37.189944+0000","last_change":"2024-05-21T15:32:33.178153+0000","last_active":"2024-05-21T15:34:37.189944+0000","last_peered":"2024-05-21T15:34:37.189944+0000","last_clean":"2024-05-21T15:34:37.189944+0000","last_became_active":"2024-05-21T15:32:33.177792+0000","last_became_peered":"2024-05-21T15:32:33.177792+0000","last_unstale":"2024-05-21T15:34:37.189944+0000","last_undegraded":"2024-05-21T15:34:37.189944+0000","last_fullsized":"2024-05-21T15:34:37.189944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:43:41.865204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60022,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":242,"num_read_kb":180,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021430+0000","last_change":"2024-05-21T15:32:33.681328+0000","last_active":"2024-05-21T15:34:24.021430+0000","last_peered":"2024-05-21T15:34:24.021430+0000","last_clean":"2024-05-21T15:34:24.021430+0000","last_became_active":"2024-05-21T15:32:33.680975+0000","last_became_peered":"2024-05-21T15:32:33.680975+0000","last_unstale":"2024-05-21T15:34:24.021430+0000","last_undegraded":"2024-05-21T15:34:24.021430+0000","last_fullsized":"2024-05-21T15:34:24.021430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:50:27.199348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142791+0000","last_change":"2024-05-21T15:32:33.182492+0000","last_active":"2024-05-21T15:34:24.142791+0000","last_peered":"2024-05-21T15:34:24.142791+0000","last_clean":"2024-05-21T15:34:24.142791+0000","last_became_active":"2024-05-21T15:32:33.182152+0000","last_became_peered":"2024-05-21T15:32:33.182152+0000","last_unstale":"2024-05-21T15:34:24.142791+0000","last_undegraded":"2024-05-21T15:34:24.142791+0000","last_fullsized":"2024-05-21T15:34:24.142791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:04:07.432742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016470+0000","last_change":"2024-05-21T15:32:33.686623+0000","last_active":"2024-05-21T15:34:24.016470+0000","last_peered":"2024-05-21T15:34:24.016470+0000","last_clean":"2024-05-21T15:34:24.016470+0000","last_became_active":"2024-05-21T15:32:33.685816+0000","last_became_peered":"2024-05-21T15:32:33.685816+0000","last_unstale":"2024-05-21T15:34:24.016470+0000","last_undegraded":"2024-05-21T15:34:24.016470+0000","last_fullsized":"2024-05-21T15:34:24.016470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:15:20.354113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.029185+0000","last_change":"2024-05-21T15:34:24.028748+0000","last_active":"2024-05-21T15:34:24.029185+0000","last_peered":"2024-05-21T15:34:24.029185+0000","last_clean":"2024-05-21T15:34:24.029185+0000","last_became_active":"2024-05-21T15:34:24.022920+0000","last_became_peered":"2024-05-21T15:34:24.022920+0000","last_unstale":"2024-05-21T15:34:24.029185+0000","last_undegraded":"2024-05-21T15:34:24.029185+0000","last_fullsized":"2024-05-21T15:34:24.029185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:09:04.712896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021976+0000","last_change":"2024-05-21T15:32:33.689936+0000","last_active":"2024-05-21T15:34:24.021976+0000","last_peered":"2024-05-21T15:34:24.021976+0000","last_clean":"2024-05-21T15:34:24.021976+0000","last_became_active":"2024-05-21T15:32:33.689452+0000","last_became_peered":"2024-05-21T15:32:33.689452+0000","last_unstale":"2024-05-21T15:34:24.021976+0000","last_undegraded":"2024-05-21T15:34:24.021976+0000","last_fullsized":"2024-05-21T15:34:24.021976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:00.420730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:35.166161+0000","last_change":"2024-05-21T15:32:33.181354+0000","last_active":"2024-05-21T15:34:35.166161+0000","last_peered":"2024-05-21T15:34:35.166161+0000","last_clean":"2024-05-21T15:34:35.166161+0000","last_became_active":"2024-05-21T15:32:33.180983+0000","last_became_peered":"2024-05-21T15:32:33.180983+0000","last_unstale":"2024-05-21T15:34:35.166161+0000","last_undegraded":"2024-05-21T15:34:35.166161+0000","last_fullsized":"2024-05-21T15:34:35.166161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:18:46.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021643+0000","last_change":"2024-05-21T15:33:23.377305+0000","last_active":"2024-05-21T15:34:24.021643+0000","last_peered":"2024-05-21T15:34:24.021643+0000","last_clean":"2024-05-21T15:34:24.021643+0000","last_became_active":"2024-05-21T15:33:23.376921+0000","last_became_peered":"2024-05-21T15:33:23.376921+0000","last_unstale":"2024-05-21T15:34:24.021643+0000","last_undegraded":"2024-05-21T15:34:24.021643+0000","last_fullsized":"2024-05-21T15:34:24.021643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:19.874725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021379+0000","last_change":"2024-05-21T15:33:23.496401+0000","last_active":"2024-05-21T15:34:24.021379+0000","last_peered":"2024-05-21T15:34:24.021379+0000","last_clean":"2024-05-21T15:34:24.021379+0000","last_became_active":"2024-05-21T15:33:23.483584+0000","last_became_peered":"2024-05-21T15:33:23.483584+0000","last_unstale":"2024-05-21T15:34:24.021379+0000","last_undegraded":"2024-05-21T15:34:24.021379+0000","last_fullsized":"2024-05-21T15:34:24.021379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:35:32.411825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017462+0000","last_change":"2024-05-21T15:32:33.691112+0000","last_active":"2024-05-21T15:34:24.017462+0000","last_peered":"2024-05-21T15:34:24.017462+0000","last_clean":"2024-05-21T15:34:24.017462+0000","last_became_active":"2024-05-21T15:32:33.690726+0000","last_became_peered":"2024-05-21T15:32:33.690726+0000","last_unstale":"2024-05-21T15:34:24.017462+0000","last_undegraded":"2024-05-21T15:34:24.017462+0000","last_fullsized":"2024-05-21T15:34:24.017462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:59:29.002636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021659+0000","last_change":"2024-05-21T15:32:33.179565+0000","last_active":"2024-05-21T15:34:24.021659+0000","last_peered":"2024-05-21T15:34:24.021659+0000","last_clean":"2024-05-21T15:34:24.021659+0000","last_became_active":"2024-05-21T15:32:33.179168+0000","last_became_peered":"2024-05-21T15:32:33.179168+0000","last_unstale":"2024-05-21T15:34:24.021659+0000","last_undegraded":"2024-05-21T15:34:24.021659+0000","last_fullsized":"2024-05-21T15:34:24.021659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:23:39.210931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.020065+0000","last_change":"2024-05-21T15:32:33.691348+0000","last_active":"2024-05-21T15:34:24.020065+0000","last_peered":"2024-05-21T15:34:24.020065+0000","last_clean":"2024-05-21T15:34:24.020065+0000","last_became_active":"2024-05-21T15:32:33.691005+0000","last_became_peered":"2024-05-21T15:32:33.691005+0000","last_unstale":"2024-05-21T15:34:24.020065+0000","last_undegraded":"2024-05-21T15:34:24.020065+0000","last_fullsized":"2024-05-21T15:34:24.020065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:33:34.533026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142969+0000","last_change":"2024-05-21T15:32:33.184850+0000","last_active":"2024-05-21T15:34:24.142969+0000","last_peered":"2024-05-21T15:34:24.142969+0000","last_clean":"2024-05-21T15:34:24.142969+0000","last_became_active":"2024-05-21T15:32:33.184480+0000","last_became_peered":"2024-05-21T15:32:33.184480+0000","last_unstale":"2024-05-21T15:34:24.142969+0000","last_undegraded":"2024-05-21T15:34:24.142969+0000","last_fullsized":"2024-05-21T15:34:24.142969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:31.327820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019449+0000","last_change":"2024-05-21T15:32:33.689520+0000","last_active":"2024-05-21T15:34:24.019449+0000","last_peered":"2024-05-21T15:34:24.019449+0000","last_clean":"2024-05-21T15:34:24.019449+0000","last_became_active":"2024-05-21T15:32:33.688219+0000","last_became_peered":"2024-05-21T15:32:33.688219+0000","last_unstale":"2024-05-21T15:34:24.019449+0000","last_undegraded":"2024-05-21T15:34:24.019449+0000","last_fullsized":"2024-05-21T15:34:24.019449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:36.546390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:38.164983+0000","last_change":"2024-05-21T15:32:33.183183+0000","last_active":"2024-05-21T15:34:38.164983+0000","last_peered":"2024-05-21T15:34:38.164983+0000","last_clean":"2024-05-21T15:34:38.164983+0000","last_became_active":"2024-05-21T15:32:33.182753+0000","last_became_peered":"2024-05-21T15:32:33.182753+0000","last_unstale":"2024-05-21T15:34:38.164983+0000","last_undegraded":"2024-05-21T15:34:38.164983+0000","last_fullsized":"2024-05-21T15:34:38.164983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:44:22.509265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019138+0000","last_change":"2024-05-21T15:32:33.684019+0000","last_active":"2024-05-21T15:34:24.019138+0000","last_peered":"2024-05-21T15:34:24.019138+0000","last_clean":"2024-05-21T15:34:24.019138+0000","last_became_active":"2024-05-21T15:32:33.683276+0000","last_became_peered":"2024-05-21T15:32:33.683276+0000","last_unstale":"2024-05-21T15:34:24.019138+0000","last_undegraded":"2024-05-21T15:34:24.019138+0000","last_fullsized":"2024-05-21T15:34:24.019138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:07:14.947240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017746+0000","last_change":"2024-05-21T15:32:33.690956+0000","last_active":"2024-05-21T15:34:24.017746+0000","last_peered":"2024-05-21T15:34:24.017746+0000","last_clean":"2024-05-21T15:34:24.017746+0000","last_became_active":"2024-05-21T15:32:33.690630+0000","last_became_peered":"2024-05-21T15:32:33.690630+0000","last_unstale":"2024-05-21T15:34:24.017746+0000","last_undegraded":"2024-05-21T15:34:24.017746+0000","last_fullsized":"2024-05-21T15:34:24.017746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:24:07.050928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":147,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:37.168923+0000","last_change":"2024-05-21T15:32:33.175127+0000","last_active":"2024-05-21T15:34:37.168923+0000","last_peered":"2024-05-21T15:34:37.168923+0000","last_clean":"2024-05-21T15:34:37.168923+0000","last_became_active":"2024-05-21T15:32:33.174724+0000","last_became_peered":"2024-05-21T15:32:33.174724+0000","last_unstale":"2024-05-21T15:34:37.168923+0000","last_undegraded":"2024-05-21T15:34:37.168923+0000","last_fullsized":"2024-05-21T15:34:37.168923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:39:17.499743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021701+0000","last_change":"2024-05-21T15:32:33.693428+0000","last_active":"2024-05-21T15:34:24.021701+0000","last_peered":"2024-05-21T15:34:24.021701+0000","last_clean":"2024-05-21T15:34:24.021701+0000","last_became_active":"2024-05-21T15:32:33.693077+0000","last_became_peered":"2024-05-21T15:32:33.693077+0000","last_unstale":"2024-05-21T15:34:24.021701+0000","last_undegraded":"2024-05-21T15:34:24.021701+0000","last_fullsized":"2024-05-21T15:34:24.021701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:08:11.908874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143504+0000","last_change":"2024-05-21T15:32:33.176267+0000","last_active":"2024-05-21T15:34:24.143504+0000","last_peered":"2024-05-21T15:34:24.143504+0000","last_clean":"2024-05-21T15:34:24.143504+0000","last_became_active":"2024-05-21T15:32:33.175856+0000","last_became_peered":"2024-05-21T15:32:33.175856+0000","last_unstale":"2024-05-21T15:34:24.143504+0000","last_undegraded":"2024-05-21T15:34:24.143504+0000","last_fullsized":"2024-05-21T15:34:24.143504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:58.113791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021740+0000","last_change":"2024-05-21T15:31:20.928694+0000","last_active":"2024-05-21T15:34:24.021740+0000","last_peered":"2024-05-21T15:34:24.021740+0000","last_clean":"2024-05-21T15:34:24.021740+0000","last_became_active":"2024-05-21T15:31:20.928115+0000","last_became_peered":"2024-05-21T15:31:20.928115+0000","last_unstale":"2024-05-21T15:34:24.021740+0000","last_undegraded":"2024-05-21T15:34:24.021740+0000","last_fullsized":"2024-05-21T15:34:24.021740+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:27:50.788960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:27:50.788960+0000","last_clean_scrub_stamp":"2024-05-21T15:27:50.788960+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:55:10.393374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016306+0000","last_change":"2024-05-21T15:32:33.689305+0000","last_active":"2024-05-21T15:34:24.016306+0000","last_peered":"2024-05-21T15:34:24.016306+0000","last_clean":"2024-05-21T15:34:24.016306+0000","last_became_active":"2024-05-21T15:32:33.688908+0000","last_became_peered":"2024-05-21T15:32:33.688908+0000","last_unstale":"2024-05-21T15:34:24.016306+0000","last_undegraded":"2024-05-21T15:34:24.016306+0000","last_fullsized":"2024-05-21T15:34:24.016306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:49:18.953116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022046+0000","last_change":"2024-05-21T15:32:33.187781+0000","last_active":"2024-05-21T15:34:24.022046+0000","last_peered":"2024-05-21T15:34:24.022046+0000","last_clean":"2024-05-21T15:34:24.022046+0000","last_became_active":"2024-05-21T15:32:33.187504+0000","last_became_peered":"2024-05-21T15:32:33.187504+0000","last_unstale":"2024-05-21T15:34:24.022046+0000","last_undegraded":"2024-05-21T15:34:24.022046+0000","last_fullsized":"2024-05-21T15:34:24.022046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:30.400209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021855+0000","last_change":"2024-05-21T15:32:33.188706+0000","last_active":"2024-05-21T15:34:24.021855+0000","last_peered":"2024-05-21T15:34:24.021855+0000","last_clean":"2024-05-21T15:34:24.021855+0000","last_became_active":"2024-05-21T15:32:33.187462+0000","last_became_peered":"2024-05-21T15:32:33.187462+0000","last_unstale":"2024-05-21T15:34:24.021855+0000","last_undegraded":"2024-05-21T15:34:24.021855+0000","last_fullsized":"2024-05-21T15:34:24.021855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:47:45.387721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016776+0000","last_change":"2024-05-21T15:32:33.672531+0000","last_active":"2024-05-21T15:34:24.016776+0000","last_peered":"2024-05-21T15:34:24.016776+0000","last_clean":"2024-05-21T15:34:24.016776+0000","last_became_active":"2024-05-21T15:32:33.671972+0000","last_became_peered":"2024-05-21T15:32:33.671972+0000","last_unstale":"2024-05-21T15:34:24.016776+0000","last_undegraded":"2024-05-21T15:34:24.016776+0000","last_fullsized":"2024-05-21T15:34:24.016776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:10:51.062530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'14","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143042+0000","last_change":"2024-05-21T15:32:33.181274+0000","last_active":"2024-05-21T15:34:24.143042+0000","last_peered":"2024-05-21T15:34:24.143042+0000","last_clean":"2024-05-21T15:34:24.143042+0000","last_became_active":"2024-05-21T15:32:33.180885+0000","last_became_peered":"2024-05-21T15:32:33.180885+0000","last_unstale":"2024-05-21T15:34:24.143042+0000","last_undegraded":"2024-05-21T15:34:24.143042+0000","last_fullsized":"2024-05-21T15:34:24.143042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:10:13.547380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022012+0000","last_change":"2024-05-21T15:32:33.683749+0000","last_active":"2024-05-21T15:34:24.022012+0000","last_peered":"2024-05-21T15:34:24.022012+0000","last_clean":"2024-05-21T15:34:24.022012+0000","last_became_active":"2024-05-21T15:32:33.683393+0000","last_became_peered":"2024-05-21T15:32:33.683393+0000","last_unstale":"2024-05-21T15:34:24.022012+0000","last_undegraded":"2024-05-21T15:34:24.022012+0000","last_fullsized":"2024-05-21T15:34:24.022012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:07:53.009651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143760+0000","last_change":"2024-05-21T15:32:33.184740+0000","last_active":"2024-05-21T15:34:24.143760+0000","last_peered":"2024-05-21T15:34:24.143760+0000","last_clean":"2024-05-21T15:34:24.143760+0000","last_became_active":"2024-05-21T15:32:33.184378+0000","last_became_peered":"2024-05-21T15:32:33.184378+0000","last_unstale":"2024-05-21T15:34:24.143760+0000","last_undegraded":"2024-05-21T15:34:24.143760+0000","last_fullsized":"2024-05-21T15:34:24.143760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:21:27.610513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143379+0000","last_change":"2024-05-21T15:33:23.381090+0000","last_active":"2024-05-21T15:34:24.143379+0000","last_peered":"2024-05-21T15:34:24.143379+0000","last_clean":"2024-05-21T15:34:24.143379+0000","last_became_active":"2024-05-21T15:33:23.380680+0000","last_became_peered":"2024-05-21T15:33:23.380680+0000","last_unstale":"2024-05-21T15:34:24.143379+0000","last_undegraded":"2024-05-21T15:34:24.143379+0000","last_fullsized":"2024-05-21T15:34:24.143379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:26:03.371476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017604+0000","last_change":"2024-05-21T15:32:33.183344+0000","last_active":"2024-05-21T15:34:24.017604+0000","last_peered":"2024-05-21T15:34:24.017604+0000","last_clean":"2024-05-21T15:34:24.017604+0000","last_became_active":"2024-05-21T15:32:33.182977+0000","last_became_peered":"2024-05-21T15:32:33.182977+0000","last_unstale":"2024-05-21T15:34:24.017604+0000","last_undegraded":"2024-05-21T15:34:24.017604+0000","last_fullsized":"2024-05-21T15:34:24.017604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:27:24.354129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021213+0000","last_change":"2024-05-21T15:32:33.677549+0000","last_active":"2024-05-21T15:34:24.021213+0000","last_peered":"2024-05-21T15:34:24.021213+0000","last_clean":"2024-05-21T15:34:24.021213+0000","last_became_active":"2024-05-21T15:32:33.677213+0000","last_became_peered":"2024-05-21T15:32:33.677213+0000","last_unstale":"2024-05-21T15:34:24.021213+0000","last_undegraded":"2024-05-21T15:34:24.021213+0000","last_fullsized":"2024-05-21T15:34:24.021213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:57:06.560434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021547+0000","last_change":"2024-05-21T15:32:33.190023+0000","last_active":"2024-05-21T15:34:24.021547+0000","last_peered":"2024-05-21T15:34:24.021547+0000","last_clean":"2024-05-21T15:34:24.021547+0000","last_became_active":"2024-05-21T15:32:33.189616+0000","last_became_peered":"2024-05-21T15:32:33.189616+0000","last_unstale":"2024-05-21T15:34:24.021547+0000","last_undegraded":"2024-05-21T15:34:24.021547+0000","last_fullsized":"2024-05-21T15:34:24.021547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:03.841414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019511+0000","last_change":"2024-05-21T15:32:33.685911+0000","last_active":"2024-05-21T15:34:24.019511+0000","last_peered":"2024-05-21T15:34:24.019511+0000","last_clean":"2024-05-21T15:34:24.019511+0000","last_became_active":"2024-05-21T15:32:33.684785+0000","last_became_peered":"2024-05-21T15:32:33.684785+0000","last_unstale":"2024-05-21T15:34:24.019511+0000","last_undegraded":"2024-05-21T15:34:24.019511+0000","last_fullsized":"2024-05-21T15:34:24.019511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:39:18.370289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142681+0000","last_change":"2024-05-21T15:32:33.178354+0000","last_active":"2024-05-21T15:34:24.142681+0000","last_peered":"2024-05-21T15:34:24.142681+0000","last_clean":"2024-05-21T15:34:24.142681+0000","last_became_active":"2024-05-21T15:32:33.177989+0000","last_became_peered":"2024-05-21T15:32:33.177989+0000","last_unstale":"2024-05-21T15:34:24.142681+0000","last_undegraded":"2024-05-21T15:34:24.142681+0000","last_fullsized":"2024-05-21T15:34:24.142681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:58.650831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021792+0000","last_change":"2024-05-21T15:32:33.686467+0000","last_active":"2024-05-21T15:34:24.021792+0000","last_peered":"2024-05-21T15:34:24.021792+0000","last_clean":"2024-05-21T15:34:24.021792+0000","last_became_active":"2024-05-21T15:32:33.686135+0000","last_became_peered":"2024-05-21T15:32:33.686135+0000","last_unstale":"2024-05-21T15:34:24.021792+0000","last_undegraded":"2024-05-21T15:34:24.021792+0000","last_fullsized":"2024-05-21T15:34:24.021792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:53:54.045591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019621+0000","last_change":"2024-05-21T15:33:23.383157+0000","last_active":"2024-05-21T15:34:24.019621+0000","last_peered":"2024-05-21T15:34:24.019621+0000","last_clean":"2024-05-21T15:34:24.019621+0000","last_became_active":"2024-05-21T15:33:23.376834+0000","last_became_peered":"2024-05-21T15:33:23.376834+0000","last_unstale":"2024-05-21T15:34:24.019621+0000","last_undegraded":"2024-05-21T15:34:24.019621+0000","last_fullsized":"2024-05-21T15:34:24.019621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:01:19.349486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022263+0000","last_change":"2024-05-21T15:32:33.692325+0000","last_active":"2024-05-21T15:34:24.022263+0000","last_peered":"2024-05-21T15:34:24.022263+0000","last_clean":"2024-05-21T15:34:24.022263+0000","last_became_active":"2024-05-21T15:32:33.691971+0000","last_became_peered":"2024-05-21T15:32:33.691971+0000","last_unstale":"2024-05-21T15:34:24.022263+0000","last_undegraded":"2024-05-21T15:34:24.022263+0000","last_fullsized":"2024-05-21T15:34:24.022263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:53:28.190063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:38.192163+0000","last_change":"2024-05-21T15:32:33.186925+0000","last_active":"2024-05-21T15:34:38.192163+0000","last_peered":"2024-05-21T15:34:38.192163+0000","last_clean":"2024-05-21T15:34:38.192163+0000","last_became_active":"2024-05-21T15:32:33.186605+0000","last_became_peered":"2024-05-21T15:32:33.186605+0000","last_unstale":"2024-05-21T15:34:38.192163+0000","last_undegraded":"2024-05-21T15:34:38.192163+0000","last_fullsized":"2024-05-21T15:34:38.192163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:32.752746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022284+0000","last_change":"2024-05-21T15:32:33.686132+0000","last_active":"2024-05-21T15:34:24.022284+0000","last_peered":"2024-05-21T15:34:24.022284+0000","last_clean":"2024-05-21T15:34:24.022284+0000","last_became_active":"2024-05-21T15:32:33.685745+0000","last_became_peered":"2024-05-21T15:32:33.685745+0000","last_unstale":"2024-05-21T15:34:24.022284+0000","last_undegraded":"2024-05-21T15:34:24.022284+0000","last_fullsized":"2024-05-21T15:34:24.022284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:51:49.522297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017771+0000","last_change":"2024-05-21T15:32:33.176552+0000","last_active":"2024-05-21T15:34:24.017771+0000","last_peered":"2024-05-21T15:34:24.017771+0000","last_clean":"2024-05-21T15:34:24.017771+0000","last_became_active":"2024-05-21T15:32:33.176237+0000","last_became_peered":"2024-05-21T15:32:33.176237+0000","last_unstale":"2024-05-21T15:34:24.017771+0000","last_undegraded":"2024-05-21T15:34:24.017771+0000","last_fullsized":"2024-05-21T15:34:24.017771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:09.098344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022243+0000","last_change":"2024-05-21T15:32:33.692310+0000","last_active":"2024-05-21T15:34:24.022243+0000","last_peered":"2024-05-21T15:34:24.022243+0000","last_clean":"2024-05-21T15:34:24.022243+0000","last_became_active":"2024-05-21T15:32:33.691959+0000","last_became_peered":"2024-05-21T15:32:33.691959+0000","last_unstale":"2024-05-21T15:34:24.022243+0000","last_undegraded":"2024-05-21T15:34:24.022243+0000","last_fullsized":"2024-05-21T15:34:24.022243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:57:38.243926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016714+0000","last_change":"2024-05-21T15:33:23.614280+0000","last_active":"2024-05-21T15:34:24.016714+0000","last_peered":"2024-05-21T15:34:24.016714+0000","last_clean":"2024-05-21T15:34:24.016714+0000","last_became_active":"2024-05-21T15:33:23.610364+0000","last_became_peered":"2024-05-21T15:33:23.610364+0000","last_unstale":"2024-05-21T15:34:24.016714+0000","last_undegraded":"2024-05-21T15:34:24.016714+0000","last_fullsized":"2024-05-21T15:34:24.016714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:32.156479+0000","last_clean_scrub_stamp":"2024-05-21T15:32:32.156479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:40:42.990949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.022268+0000","last_change":"2024-05-21T15:32:33.682801+0000","last_active":"2024-05-21T15:34:24.022268+0000","last_peered":"2024-05-21T15:34:24.022268+0000","last_clean":"2024-05-21T15:34:24.022268+0000","last_became_active":"2024-05-21T15:32:33.682368+0000","last_became_peered":"2024-05-21T15:32:33.682368+0000","last_unstale":"2024-05-21T15:34:24.022268+0000","last_undegraded":"2024-05-21T15:34:24.022268+0000","last_fullsized":"2024-05-21T15:34:24.022268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:56:50.520672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019605+0000","last_change":"2024-05-21T15:32:33.684809+0000","last_active":"2024-05-21T15:34:24.019605+0000","last_peered":"2024-05-21T15:34:24.019605+0000","last_clean":"2024-05-21T15:34:24.019605+0000","last_became_active":"2024-05-21T15:32:33.684067+0000","last_became_peered":"2024-05-21T15:32:33.684067+0000","last_unstale":"2024-05-21T15:34:24.019605+0000","last_undegraded":"2024-05-21T15:34:24.019605+0000","last_fullsized":"2024-05-21T15:34:24.019605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:31:58.997470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142775+0000","last_change":"2024-05-21T15:32:33.676091+0000","last_active":"2024-05-21T15:34:24.142775+0000","last_peered":"2024-05-21T15:34:24.142775+0000","last_clean":"2024-05-21T15:34:24.142775+0000","last_became_active":"2024-05-21T15:32:33.675538+0000","last_became_peered":"2024-05-21T15:32:33.675538+0000","last_unstale":"2024-05-21T15:34:24.142775+0000","last_undegraded":"2024-05-21T15:34:24.142775+0000","last_fullsized":"2024-05-21T15:34:24.142775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:15:50.311152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.142726+0000","last_change":"2024-05-21T15:32:33.688590+0000","last_active":"2024-05-21T15:34:24.142726+0000","last_peered":"2024-05-21T15:34:24.142726+0000","last_clean":"2024-05-21T15:34:24.142726+0000","last_became_active":"2024-05-21T15:32:33.687993+0000","last_became_peered":"2024-05-21T15:32:33.687993+0000","last_unstale":"2024-05-21T15:34:24.142726+0000","last_undegraded":"2024-05-21T15:34:24.142726+0000","last_fullsized":"2024-05-21T15:34:24.142726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:21.939483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143010+0000","last_change":"2024-05-21T15:33:23.380890+0000","last_active":"2024-05-21T15:34:24.143010+0000","last_peered":"2024-05-21T15:34:24.143010+0000","last_clean":"2024-05-21T15:34:24.143010+0000","last_became_active":"2024-05-21T15:33:23.380503+0000","last_became_peered":"2024-05-21T15:33:23.380503+0000","last_unstale":"2024-05-21T15:34:24.143010+0000","last_undegraded":"2024-05-21T15:34:24.143010+0000","last_fullsized":"2024-05-21T15:34:24.143010+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:41.070336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016602+0000","last_change":"2024-05-21T15:32:33.689365+0000","last_active":"2024-05-21T15:34:24.016602+0000","last_peered":"2024-05-21T15:34:24.016602+0000","last_clean":"2024-05-21T15:34:24.016602+0000","last_became_active":"2024-05-21T15:32:33.688739+0000","last_became_peered":"2024-05-21T15:32:33.688739+0000","last_unstale":"2024-05-21T15:34:24.016602+0000","last_undegraded":"2024-05-21T15:34:24.016602+0000","last_fullsized":"2024-05-21T15:34:24.016602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:19:52.614155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.020003+0000","last_change":"2024-05-21T15:32:33.681985+0000","last_active":"2024-05-21T15:34:24.020003+0000","last_peered":"2024-05-21T15:34:24.020003+0000","last_clean":"2024-05-21T15:34:24.020003+0000","last_became_active":"2024-05-21T15:32:33.681209+0000","last_became_peered":"2024-05-21T15:32:33.681209+0000","last_unstale":"2024-05-21T15:34:24.020003+0000","last_undegraded":"2024-05-21T15:34:24.020003+0000","last_fullsized":"2024-05-21T15:34:24.020003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:03:16.088136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143161+0000","last_change":"2024-05-21T15:32:33.673236+0000","last_active":"2024-05-21T15:34:24.143161+0000","last_peered":"2024-05-21T15:34:24.143161+0000","last_clean":"2024-05-21T15:34:24.143161+0000","last_became_active":"2024-05-21T15:32:33.672548+0000","last_became_peered":"2024-05-21T15:32:33.672548+0000","last_unstale":"2024-05-21T15:34:24.143161+0000","last_undegraded":"2024-05-21T15:34:24.143161+0000","last_fullsized":"2024-05-21T15:34:24.143161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:52:10.010219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.143155+0000","last_change":"2024-05-21T15:33:23.482874+0000","last_active":"2024-05-21T15:34:24.143155+0000","last_peered":"2024-05-21T15:34:24.143155+0000","last_clean":"2024-05-21T15:34:24.143155+0000","last_became_active":"2024-05-21T15:33:23.482589+0000","last_became_peered":"2024-05-21T15:33:23.482589+0000","last_unstale":"2024-05-21T15:34:24.143155+0000","last_undegraded":"2024-05-21T15:34:24.143155+0000","last_fullsized":"2024-05-21T15:34:24.143155+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:19:45.870562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017577+0000","last_change":"2024-05-21T15:32:33.686992+0000","last_active":"2024-05-21T15:34:24.017577+0000","last_peered":"2024-05-21T15:34:24.017577+0000","last_clean":"2024-05-21T15:34:24.017577+0000","last_became_active":"2024-05-21T15:32:33.686673+0000","last_became_peered":"2024-05-21T15:32:33.686673+0000","last_unstale":"2024-05-21T15:34:24.017577+0000","last_undegraded":"2024-05-21T15:34:24.017577+0000","last_fullsized":"2024-05-21T15:34:24.017577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:18:20.668365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.021467+0000","last_change":"2024-05-21T15:32:33.689433+0000","last_active":"2024-05-21T15:34:24.021467+0000","last_peered":"2024-05-21T15:34:24.021467+0000","last_clean":"2024-05-21T15:34:24.021467+0000","last_became_active":"2024-05-21T15:32:33.689106+0000","last_became_peered":"2024-05-21T15:32:33.689106+0000","last_unstale":"2024-05-21T15:34:24.021467+0000","last_undegraded":"2024-05-21T15:34:24.021467+0000","last_fullsized":"2024-05-21T15:34:24.021467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:45:17.841159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.017934+0000","last_change":"2024-05-21T15:32:33.687763+0000","last_active":"2024-05-21T15:34:24.017934+0000","last_peered":"2024-05-21T15:34:24.017934+0000","last_clean":"2024-05-21T15:34:24.017934+0000","last_became_active":"2024-05-21T15:32:33.685103+0000","last_became_peered":"2024-05-21T15:32:33.685103+0000","last_unstale":"2024-05-21T15:34:24.017934+0000","last_undegraded":"2024-05-21T15:34:24.017934+0000","last_fullsized":"2024-05-21T15:34:24.017934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:56:39.454536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.016565+0000","last_change":"2024-05-21T15:32:33.679225+0000","last_active":"2024-05-21T15:34:24.016565+0000","last_peered":"2024-05-21T15:34:24.016565+0000","last_clean":"2024-05-21T15:34:24.016565+0000","last_became_active":"2024-05-21T15:32:33.678890+0000","last_became_peered":"2024-05-21T15:32:33.678890+0000","last_unstale":"2024-05-21T15:34:24.016565+0000","last_undegraded":"2024-05-21T15:34:24.016565+0000","last_fullsized":"2024-05-21T15:34:24.016565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:12:23.721449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-21T15:34:24.019618+0000","last_change":"2024-05-21T15:32:33.688659+0000","last_active":"2024-05-21T15:34:24.019618+0000","last_peered":"2024-05-21T15:34:24.019618+0000","last_clean":"2024-05-21T15:34:24.019618+0000","last_became_active":"2024-05-21T15:32:33.688293+0000","last_became_peered":"2024-05-21T15:32:33.688293+0000","last_unstale":"2024-05-21T15:34:24.019618+0000","last_undegraded":"2024-05-21T15:34:24.019618+0000","last_fullsized":"2024-05-21T15:34:24.019618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:32:33.162154+0000","last_clean_scrub_stamp":"2024-05-21T15:32:33.162154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:08:49.543237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":287606,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66976,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1206,"num_read_kb":675,"num_write":246,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":202,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":206301,"data_compressed":5562,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":477685,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1128114,"data_compressed":50013,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776176,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":476993,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":477371,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":1069498,"data_compressed":48159,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":476710,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":537337,"data_compressed":13445,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":478581,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":543658,"data_compressed":13445,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":476535,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888139,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":999424,"data_stored":476683,"data_compressed":11591,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724613,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":1069086,"data_compressed":48159,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34922,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36568,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61774,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67787,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61758,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T15:34:41.562 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:34:41.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:41 smithi053 ceph-mon[36496]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:34:41.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:41 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2974519708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:41.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:41 smithi053 ceph-mon[36496]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:34:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.714+0000 7f518f6e9640 1 -- 172.21.15.38:0/2569732405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5188102bf0 msgr2=0x7f5188103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.714+0000 7f518f6e9640 1 --2- 172.21.15.38:0/2569732405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5188102bf0 0x7f5188103000 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5178009900 tx=0x7f517802a410 comp rx=0 tx=0).stop 2024-05-21T15:34:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.714+0000 7f518f6e9640 1 -- 172.21.15.38:0/2569732405 shutdown_connections 2024-05-21T15:34:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.714+0000 7f518f6e9640 1 --2- 172.21.15.38:0/2569732405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51881078f0 0x7f5188109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.714+0000 7f518f6e9640 1 --2- 172.21.15.38:0/2569732405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5188102bf0 0x7f5188103000 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.714+0000 7f518f6e9640 1 --2- 172.21.15.38:0/2569732405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51881019b0 0x7f51881026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.714+0000 7f518f6e9640 1 -- 172.21.15.38:0/2569732405 >> 172.21.15.38:0/2569732405 conn(0x7f51880fcba0 msgr2=0x7f51880ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.714+0000 7f518f6e9640 1 -- 172.21.15.38:0/2569732405 shutdown_connections 2024-05-21T15:34:41.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.714+0000 7f518f6e9640 1 -- 172.21.15.38:0/2569732405 wait complete. 2024-05-21T15:34:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518f6e9640 1 Processor -- start 2024-05-21T15:34:41.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518f6e9640 1 -- start start 2024-05-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518f6e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51881019b0 0x7f5188199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518f6e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5188102bf0 0x7f5188199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518f6e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51881078f0 0x7f51881a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518f6e9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f518810d630 con 0x7f51881019b0 2024-05-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518f6e9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f518810d4b0 con 0x7f51881078f0 2024-05-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518f6e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f518810d7b0 con 0x7f5188102bf0 2024-05-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518d485640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51881019b0 0x7f5188199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518d485640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51881019b0 0x7f5188199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60012/0 (socket says 172.21.15.38:60012) 2024-05-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518d485640 1 -- 172.21.15.38:0/1172108092 learned_addr learned my addr 172.21.15.38:0/1172108092 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518cc84640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5188102bf0 0x7f5188199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518dc86640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51881078f0 0x7f51881a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518d485640 1 -- 172.21.15.38:0/1172108092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5188102bf0 msgr2=0x7f5188199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518d485640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5188102bf0 0x7f5188199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518d485640 1 -- 172.21.15.38:0/1172108092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51881078f0 msgr2=0x7f51881a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:41.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518d485640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51881078f0 0x7f51881a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518d485640 1 -- 172.21.15.38:0/1172108092 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51881a0d50 con 0x7f51881019b0 2024-05-21T15:34:41.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518cc84640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5188102bf0 0x7f5188199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:41.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.715+0000 7f518dc86640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51881078f0 0x7f51881a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:41.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.716+0000 7f518d485640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51881019b0 0x7f5188199210 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f518400e9f0 tx=0x7f518400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:41.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.716+0000 7f517e7fc640 1 -- 172.21.15.38:0/1172108092 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518400ccc0 con 0x7f51881019b0 2024-05-21T15:34:41.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.716+0000 7f517e7fc640 1 -- 172.21.15.38:0/1172108092 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51840035f0 con 0x7f51881019b0 2024-05-21T15:34:41.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.716+0000 7f518f6e9640 1 -- 172.21.15.38:0/1172108092 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51881a0fe0 con 0x7f51881019b0 2024-05-21T15:34:41.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.716+0000 7f517e7fc640 1 -- 172.21.15.38:0/1172108092 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518400ce60 con 0x7f51881019b0 2024-05-21T15:34:41.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.716+0000 7f518f6e9640 1 -- 172.21.15.38:0/1172108092 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51881a14f0 con 0x7f51881019b0 2024-05-21T15:34:41.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.717+0000 7f518f6e9640 1 -- 172.21.15.38:0/1172108092 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5188106ff0 con 0x7f51881019b0 2024-05-21T15:34:41.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.718+0000 7f517e7fc640 1 -- 172.21.15.38:0/1172108092 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f51840037a0 con 0x7f51881019b0 2024-05-21T15:34:41.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.719+0000 7f517e7fc640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51740a2cb0 0x7f51740a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:41.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.719+0000 7f518cc84640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51740a2cb0 0x7f51740a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:41.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.719+0000 7f517e7fc640 1 -- 172.21.15.38:0/1172108092 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f51840cf550 con 0x7f51881019b0 2024-05-21T15:34:41.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.720+0000 7f518cc84640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51740a2cb0 0x7f51740a51c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5178000c00 tx=0x7f51780095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:41.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.723+0000 7f517e7fc640 1 -- 172.21.15.38:0/1172108092 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f518409c400 con 0x7f51881019b0 2024-05-21T15:34:41.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:41 smithi045 ceph-mon[36409]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:34:41.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:41 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2974519708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:41.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:41 smithi045 ceph-mon[36409]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:34:41.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.909+0000 7f518f6e9640 1 -- 172.21.15.38:0/1172108092 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f518802d470 con 0x7f51881019b0 2024-05-21T15:34:41.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.911+0000 7f517e7fc640 1 -- 172.21.15.38:0/1172108092 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22704 (secure 0 0 0) 0x7f518409bd00 con 0x7f51881019b0 2024-05-21T15:34:41.911 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:41.911 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":90,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:34:23.996759+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:34:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.913+0000 7f518f6e9640 1 -- 172.21.15.38:0/1172108092 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51740a2cb0 msgr2=0x7f51740a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.913+0000 7f518f6e9640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51740a2cb0 0x7f51740a51c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5178000c00 tx=0x7f51780095f0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.913+0000 7f518f6e9640 1 -- 172.21.15.38:0/1172108092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51881019b0 msgr2=0x7f5188199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.913+0000 7f518f6e9640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51881019b0 0x7f5188199210 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f518400e9f0 tx=0x7f518400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.913+0000 7f518f6e9640 1 -- 172.21.15.38:0/1172108092 shutdown_connections 2024-05-21T15:34:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.913+0000 7f518f6e9640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51881078f0 0x7f51881a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.913+0000 7f518f6e9640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5188102bf0 0x7f5188199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.913+0000 7f518f6e9640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51740a2cb0 0x7f51740a51c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.913+0000 7f518f6e9640 1 --2- 172.21.15.38:0/1172108092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51881019b0 0x7f5188199210 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:41.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.913+0000 7f518f6e9640 1 -- 172.21.15.38:0/1172108092 >> 172.21.15.38:0/1172108092 conn(0x7f51880fcba0 msgr2=0x7f51880fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:41.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.913+0000 7f518f6e9640 1 -- 172.21.15.38:0/1172108092 shutdown_connections 2024-05-21T15:34:41.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:41.913+0000 7f518f6e9640 1 -- 172.21.15.38:0/1172108092 wait complete. 2024-05-21T15:34:41.925 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:27:50.659367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:32:31.645647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:32:32.955451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:34:41.925 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:34:42.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.044+0000 7efce32f0640 1 -- 172.21.15.38:0/4264228307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcdc103d20 msgr2=0x7efcdc104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.044+0000 7efce32f0640 1 --2- 172.21.15.38:0/4264228307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcdc103d20 0x7efcdc104130 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efccc009900 tx=0x7efccc02a410 comp rx=0 tx=0).stop 2024-05-21T15:34:42.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.044+0000 7efce32f0640 1 -- 172.21.15.38:0/4264228307 shutdown_connections 2024-05-21T15:34:42.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.044+0000 7efce32f0640 1 --2- 172.21.15.38:0/4264228307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcdc108a20 0x7efcdc10ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.044+0000 7efce32f0640 1 --2- 172.21.15.38:0/4264228307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcdc103d20 0x7efcdc104130 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.044+0000 7efce32f0640 1 --2- 172.21.15.38:0/4264228307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcdc102ae0 0x7efcdc1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.044+0000 7efce32f0640 1 -- 172.21.15.38:0/4264228307 >> 172.21.15.38:0/4264228307 conn(0x7efcdc0fdcf0 msgr2=0x7efcdc100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:42.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.045+0000 7efce32f0640 1 -- 172.21.15.38:0/4264228307 shutdown_connections 2024-05-21T15:34:42.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.045+0000 7efce32f0640 1 -- 172.21.15.38:0/4264228307 wait complete. 2024-05-21T15:34:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.045+0000 7efce32f0640 1 Processor -- start 2024-05-21T15:34:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.045+0000 7efce32f0640 1 -- start start 2024-05-21T15:34:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.045+0000 7efce32f0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcdc102ae0 0x7efcdc076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.045+0000 7efce32f0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcdc103d20 0x7efcdc074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce32f0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcdc108a20 0x7efcdc0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce32f0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc10e7b0 con 0x7efcdc102ae0 2024-05-21T15:34:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce32f0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc10e630 con 0x7efcdc103d20 2024-05-21T15:34:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce32f0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc10e930 con 0x7efcdc108a20 2024-05-21T15:34:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce108c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcdc102ae0 0x7efcdc076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:42.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce108c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcdc102ae0 0x7efcdc076910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60032/0 (socket says 172.21.15.38:60032) 2024-05-21T15:34:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce108c640 1 -- 172.21.15.38:0/1721196921 learned_addr learned my addr 172.21.15.38:0/1721196921 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce188d640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcdc108a20 0x7efcdc0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce088b640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcdc103d20 0x7efcdc074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce108c640 1 -- 172.21.15.38:0/1721196921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcdc108a20 msgr2=0x7efcdc0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce108c640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcdc108a20 0x7efcdc0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce108c640 1 -- 172.21.15.38:0/1721196921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcdc103d20 msgr2=0x7efcdc074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce108c640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcdc103d20 0x7efcdc074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce108c640 1 -- 172.21.15.38:0/1721196921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcdc075c20 con 0x7efcdc102ae0 2024-05-21T15:34:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce088b640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcdc103d20 0x7efcdc074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce108c640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcdc102ae0 0x7efcdc076910 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efcd800e9f0 tx=0x7efcd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efcd27fc640 1 -- 172.21.15.38:0/1721196921 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd800ccc0 con 0x7efcdc102ae0 2024-05-21T15:34:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efce32f0640 1 -- 172.21.15.38:0/1721196921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcdc1a3320 con 0x7efcdc102ae0 2024-05-21T15:34:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efcd27fc640 1 -- 172.21.15.38:0/1721196921 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcd80035f0 con 0x7efcdc102ae0 2024-05-21T15:34:42.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.046+0000 7efcd27fc640 1 -- 172.21.15.38:0/1721196921 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd800ce60 con 0x7efcdc102ae0 2024-05-21T15:34:42.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.047+0000 7efce32f0640 1 -- 172.21.15.38:0/1721196921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcdc1a37e0 con 0x7efcdc102ae0 2024-05-21T15:34:42.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.047+0000 7efce32f0640 1 -- 172.21.15.38:0/1721196921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcdc107830 con 0x7efcdc102ae0 2024-05-21T15:34:42.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.049+0000 7efcd27fc640 1 -- 172.21.15.38:0/1721196921 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efcd80037a0 con 0x7efcdc102ae0 2024-05-21T15:34:42.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.049+0000 7efcd27fc640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efcc80a2b90 0x7efcc80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:42.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.050+0000 7efce088b640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efcc80a2b90 0x7efcc80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:42.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.050+0000 7efcd27fc640 1 -- 172.21.15.38:0/1721196921 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7efcd80cf440 con 0x7efcdc102ae0 2024-05-21T15:34:42.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.050+0000 7efce088b640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efcc80a2b90 0x7efcc80a50a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efccc006010 tx=0x7efccc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:42.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.054+0000 7efcd27fc640 1 -- 172.21.15.38:0/1721196921 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efcd809c350 con 0x7efcdc102ae0 2024-05-21T15:34:42.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.239+0000 7efce32f0640 1 -- 172.21.15.38:0/1721196921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efcdc062750 con 0x7efcdc102ae0 2024-05-21T15:34:42.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.240+0000 7efcd27fc640 1 -- 172.21.15.38:0/1721196921 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7efcd809bc50 con 0x7efcdc102ae0 2024-05-21T15:34:42.239 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-05-21T15:34:42.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.242+0000 7efce32f0640 1 -- 172.21.15.38:0/1721196921 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efcc80a2b90 msgr2=0x7efcc80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.242+0000 7efce32f0640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efcc80a2b90 0x7efcc80a50a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efccc006010 tx=0x7efccc000bc0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.242+0000 7efce32f0640 1 -- 172.21.15.38:0/1721196921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcdc102ae0 msgr2=0x7efcdc076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.242+0000 7efce32f0640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcdc102ae0 0x7efcdc076910 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efcd800e9f0 tx=0x7efcd800eeb0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.242+0000 7efce32f0640 1 -- 172.21.15.38:0/1721196921 shutdown_connections 2024-05-21T15:34:42.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.242+0000 7efce32f0640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcdc108a20 0x7efcdc0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.242+0000 7efce32f0640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcdc103d20 0x7efcdc074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.242+0000 7efce32f0640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7efcc80a2b90 0x7efcc80a50a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.242+0000 7efce32f0640 1 --2- 172.21.15.38:0/1721196921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcdc102ae0 0x7efcdc076910 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.242+0000 7efce32f0640 1 -- 172.21.15.38:0/1721196921 >> 172.21.15.38:0/1721196921 conn(0x7efcdc0fdcf0 msgr2=0x7efcdc104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:42.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.242+0000 7efce32f0640 1 -- 172.21.15.38:0/1721196921 shutdown_connections 2024-05-21T15:34:42.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.242+0000 7efce32f0640 1 -- 172.21.15.38:0/1721196921 wait complete. 2024-05-21T15:34:42.253 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:34:42.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.375+0000 7f44e8656640 1 -- 172.21.15.38:0/2845700278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e0102c00 msgr2=0x7f44e0103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.375+0000 7f44e8656640 1 --2- 172.21.15.38:0/2845700278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e0102c00 0x7f44e0103010 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009900 tx=0x7f44d002a410 comp rx=0 tx=0).stop 2024-05-21T15:34:42.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.375+0000 7f44e8656640 1 -- 172.21.15.38:0/2845700278 shutdown_connections 2024-05-21T15:34:42.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.375+0000 7f44e8656640 1 --2- 172.21.15.38:0/2845700278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e0107900 0x7f44e0109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.375+0000 7f44e8656640 1 --2- 172.21.15.38:0/2845700278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e0102c00 0x7f44e0103010 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.375+0000 7f44e8656640 1 --2- 172.21.15.38:0/2845700278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e01019c0 0x7f44e01026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.375+0000 7f44e8656640 1 -- 172.21.15.38:0/2845700278 >> 172.21.15.38:0/2845700278 conn(0x7f44e00fcbb0 msgr2=0x7f44e00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:42.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.375+0000 7f44e8656640 1 -- 172.21.15.38:0/2845700278 shutdown_connections 2024-05-21T15:34:42.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.375+0000 7f44e8656640 1 -- 172.21.15.38:0/2845700278 wait complete. 2024-05-21T15:34:42.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.375+0000 7f44e8656640 1 Processor -- start 2024-05-21T15:34:42.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.375+0000 7f44e8656640 1 -- start start 2024-05-21T15:34:42.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e8656640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e01019c0 0x7f44e006ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e8656640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e0102c00 0x7f44e019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e8656640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e0107900 0x7f44e01a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e8656640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f44e010d660 con 0x7f44e0107900 2024-05-21T15:34:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e8656640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f44e010d4e0 con 0x7f44e01019c0 2024-05-21T15:34:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e8656640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f44e010d7e0 con 0x7f44e0102c00 2024-05-21T15:34:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e6bf3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e0107900 0x7f44e01a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e6bf3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e0107900 0x7f44e01a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60050/0 (socket says 172.21.15.38:60050) 2024-05-21T15:34:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e6bf3640 1 -- 172.21.15.38:0/31783281 learned_addr learned my addr 172.21.15.38:0/31783281 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e63f2640 1 --2- 172.21.15.38:0/31783281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e01019c0 0x7f44e006ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e5bf1640 1 --2- 172.21.15.38:0/31783281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e0102c00 0x7f44e019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e6bf3640 1 -- 172.21.15.38:0/31783281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e0102c00 msgr2=0x7f44e019e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e6bf3640 1 --2- 172.21.15.38:0/31783281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e0102c00 0x7f44e019e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e6bf3640 1 -- 172.21.15.38:0/31783281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e01019c0 msgr2=0x7f44e006ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e6bf3640 1 --2- 172.21.15.38:0/31783281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e01019c0 0x7f44e006ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e6bf3640 1 -- 172.21.15.38:0/31783281 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e01a0e20 con 0x7f44e0107900 2024-05-21T15:34:42.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.376+0000 7f44e6bf3640 1 --2- 172.21.15.38:0/31783281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e0107900 0x7f44e01a0600 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f44d800eb10 tx=0x7f44d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:42.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.377+0000 7f44d77fe640 1 -- 172.21.15.38:0/31783281 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8004b20 con 0x7f44e0107900 2024-05-21T15:34:42.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.377+0000 7f44d77fe640 1 -- 172.21.15.38:0/31783281 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d8004cc0 con 0x7f44e0107900 2024-05-21T15:34:42.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.377+0000 7f44e8656640 1 -- 172.21.15.38:0/31783281 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e01a1110 con 0x7f44e0107900 2024-05-21T15:34:42.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.377+0000 7f44d77fe640 1 -- 172.21.15.38:0/31783281 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8010760 con 0x7f44e0107900 2024-05-21T15:34:42.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.377+0000 7f44e8656640 1 -- 172.21.15.38:0/31783281 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e01a1660 con 0x7f44e0107900 2024-05-21T15:34:42.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.378+0000 7f44e8656640 1 -- 172.21.15.38:0/31783281 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e0106710 con 0x7f44e0107900 2024-05-21T15:34:42.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.379+0000 7f44d77fe640 1 -- 172.21.15.38:0/31783281 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f44d8005020 con 0x7f44e0107900 2024-05-21T15:34:42.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.380+0000 7f44d77fe640 1 --2- 172.21.15.38:0/31783281 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44cc0a2c10 0x7f44cc0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:42.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.380+0000 7f44e63f2640 1 --2- 172.21.15.38:0/31783281 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44cc0a2c10 0x7f44cc0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:42.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.380+0000 7f44d77fe640 1 -- 172.21.15.38:0/31783281 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f44d80cf670 con 0x7f44e0107900 2024-05-21T15:34:42.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.380+0000 7f44e63f2640 1 --2- 172.21.15.38:0/31783281 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44cc0a2c10 0x7f44cc0a5120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0095f0 tx=0x7f44dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:42.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.384+0000 7f44d77fe640 1 -- 172.21.15.38:0/31783281 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f44d809c580 con 0x7f44e0107900 2024-05-21T15:34:42.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:42 smithi038 ceph-mon[31995]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:34:42.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:42 smithi038 ceph-mon[31995]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:34:42.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:42 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1172108092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:42.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:42 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1721196921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:42.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.573+0000 7f44e8656640 1 -- 172.21.15.38:0/31783281 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f44e002d4c0 con 0x7f44e0107900 2024-05-21T15:34:42.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.574+0000 7f44d77fe640 1 -- 172.21.15.38:0/31783281 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f44d809be80 con 0x7f44e0107900 2024-05-21T15:34:42.573 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-05-21T15:34:42.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.576+0000 7f44e8656640 1 -- 172.21.15.38:0/31783281 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44cc0a2c10 msgr2=0x7f44cc0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.576+0000 7f44e8656640 1 --2- 172.21.15.38:0/31783281 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44cc0a2c10 0x7f44cc0a5120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0095f0 tx=0x7f44dc005f90 comp rx=0 tx=0).stop 2024-05-21T15:34:42.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.576+0000 7f44e8656640 1 -- 172.21.15.38:0/31783281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e0107900 msgr2=0x7f44e01a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.576+0000 7f44e8656640 1 --2- 172.21.15.38:0/31783281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e0107900 0x7f44e01a0600 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f44d800eb10 tx=0x7f44d800c2d0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.576+0000 7f44e8656640 1 -- 172.21.15.38:0/31783281 shutdown_connections 2024-05-21T15:34:42.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.576+0000 7f44e8656640 1 --2- 172.21.15.38:0/31783281 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f44cc0a2c10 0x7f44cc0a5120 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.576+0000 7f44e8656640 1 --2- 172.21.15.38:0/31783281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e0107900 0x7f44e01a0600 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.576+0000 7f44e8656640 1 --2- 172.21.15.38:0/31783281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e0102c00 0x7f44e019e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.576+0000 7f44e8656640 1 --2- 172.21.15.38:0/31783281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44e01019c0 0x7f44e006ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.576+0000 7f44e8656640 1 -- 172.21.15.38:0/31783281 >> 172.21.15.38:0/31783281 conn(0x7f44e00fcbb0 msgr2=0x7f44e01038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:42.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.576+0000 7f44e8656640 1 -- 172.21.15.38:0/31783281 shutdown_connections 2024-05-21T15:34:42.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.576+0000 7f44e8656640 1 -- 172.21.15.38:0/31783281 wait complete. 2024-05-21T15:34:42.587 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:34:42.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:42 smithi053 ceph-mon[36496]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:34:42.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:42 smithi053 ceph-mon[36496]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:34:42.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:42 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1172108092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:42.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:42 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1721196921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:42.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.706+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3307564895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102ab0 msgr2=0x7fd7e81037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.706+0000 7fd7ee6e9640 1 --2- 172.21.15.38:0/3307564895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102ab0 0x7fd7e81037b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009900 tx=0x7fd7d802a410 comp rx=0 tx=0).stop 2024-05-21T15:34:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.706+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3307564895 shutdown_connections 2024-05-21T15:34:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.706+0000 7fd7ee6e9640 1 --2- 172.21.15.38:0/3307564895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e81089f0 0x7fd7e810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.706+0000 7fd7ee6e9640 1 --2- 172.21.15.38:0/3307564895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e8103cf0 0x7fd7e8104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.706+0000 7fd7ee6e9640 1 --2- 172.21.15.38:0/3307564895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102ab0 0x7fd7e81037b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.706+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3307564895 >> 172.21.15.38:0/3307564895 conn(0x7fd7e80fdcc0 msgr2=0x7fd7e8100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.706+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3307564895 shutdown_connections 2024-05-21T15:34:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.706+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3307564895 wait complete. 2024-05-21T15:34:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.707+0000 7fd7ee6e9640 1 Processor -- start 2024-05-21T15:34:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.707+0000 7fd7ee6e9640 1 -- start start 2024-05-21T15:34:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.707+0000 7fd7ee6e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102ab0 0x7fd7e81992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.707+0000 7fd7ee6e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e8103cf0 0x7fd7e8199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.707+0000 7fd7ee6e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e81089f0 0x7fd7e81a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:42.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.707+0000 7fd7ee6e9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e810e700 con 0x7fd7e8102ab0 2024-05-21T15:34:42.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.707+0000 7fd7ee6e9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e810e580 con 0x7fd7e81089f0 2024-05-21T15:34:42.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.707+0000 7fd7ee6e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e810e880 con 0x7fd7e8103cf0 2024-05-21T15:34:42.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.707+0000 7fd7e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102ab0 0x7fd7e81992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:42.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.707+0000 7fd7e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102ab0 0x7fd7e81992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60068/0 (socket says 172.21.15.38:60068) 2024-05-21T15:34:42.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.707+0000 7fd7e7fff640 1 -- 172.21.15.38:0/3887336143 learned_addr learned my addr 172.21.15.38:0/3887336143 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:42.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.708+0000 7fd7dffff640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e8103cf0 0x7fd7e8199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:42.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.708+0000 7fd7ecc86640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e81089f0 0x7fd7e81a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:42.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.708+0000 7fd7e7fff640 1 -- 172.21.15.38:0/3887336143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e8103cf0 msgr2=0x7fd7e8199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.708+0000 7fd7e7fff640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e8103cf0 0x7fd7e8199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.708+0000 7fd7e7fff640 1 -- 172.21.15.38:0/3887336143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e81089f0 msgr2=0x7fd7e81a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.708+0000 7fd7e7fff640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e81089f0 0x7fd7e81a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.708+0000 7fd7e7fff640 1 -- 172.21.15.38:0/3887336143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e81a0d50 con 0x7fd7e8102ab0 2024-05-21T15:34:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.708+0000 7fd7ecc86640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e81089f0 0x7fd7e81a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.708+0000 7fd7e7fff640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102ab0 0x7fd7e81992c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8006010 tx=0x7fd7d8031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.708+0000 7fd7e5ffb640 1 -- 172.21.15.38:0/3887336143 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d803f030 con 0x7fd7e8102ab0 2024-05-21T15:34:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.708+0000 7fd7e5ffb640 1 -- 172.21.15.38:0/3887336143 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d8003ad0 con 0x7fd7e8102ab0 2024-05-21T15:34:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.708+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3887336143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e81a0fe0 con 0x7fd7e8102ab0 2024-05-21T15:34:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.708+0000 7fd7e5ffb640 1 -- 172.21.15.38:0/3887336143 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8035070 con 0x7fd7e8102ab0 2024-05-21T15:34:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.708+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3887336143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e81a1440 con 0x7fd7e8102ab0 2024-05-21T15:34:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.709+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3887336143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e8107800 con 0x7fd7e8102ab0 2024-05-21T15:34:42.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.711+0000 7fd7e5ffb640 1 -- 172.21.15.38:0/3887336143 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd7d8042030 con 0x7fd7e8102ab0 2024-05-21T15:34:42.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.711+0000 7fd7e5ffb640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd7d40a2c10 0x7fd7d40a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:42.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.711+0000 7fd7dffff640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd7d40a2c10 0x7fd7d40a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:42.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.711+0000 7fd7e5ffb640 1 -- 172.21.15.38:0/3887336143 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fd7d80ef7d0 con 0x7fd7e8102ab0 2024-05-21T15:34:42.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.712+0000 7fd7dffff640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd7d40a2c10 0x7fd7d40a5120 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd7e819a360 tx=0x7fd7d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:42.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.715+0000 7fd7e5ffb640 1 -- 172.21.15.38:0/3887336143 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd7d80bc6e0 con 0x7fd7e8102ab0 2024-05-21T15:34:42.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:42 smithi045 ceph-mon[36409]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:34:42.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:42 smithi045 ceph-mon[36409]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:34:42.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:42 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1172108092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:42.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:42 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1721196921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:34:42.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.898+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3887336143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd7e802d430 con 0x7fd7e8102ab0 2024-05-21T15:34:42.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.899+0000 7fd7e5ffb640 1 -- 172.21.15.38:0/3887336143 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fd7d80bbfe0 con 0x7fd7e8102ab0 2024-05-21T15:34:42.898 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-05-21T15:34:42.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.901+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3887336143 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd7d40a2c10 msgr2=0x7fd7d40a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.901+0000 7fd7ee6e9640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd7d40a2c10 0x7fd7d40a5120 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd7e819a360 tx=0x7fd7d0005f90 comp rx=0 tx=0).stop 2024-05-21T15:34:42.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.901+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3887336143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102ab0 msgr2=0x7fd7e81992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:42.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.901+0000 7fd7ee6e9640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102ab0 0x7fd7e81992c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8006010 tx=0x7fd7d8031b60 comp rx=0 tx=0).stop 2024-05-21T15:34:42.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.901+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3887336143 shutdown_connections 2024-05-21T15:34:42.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.901+0000 7fd7ee6e9640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e81089f0 0x7fd7e81a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.901+0000 7fd7ee6e9640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7e8103cf0 0x7fd7e8199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.901+0000 7fd7ee6e9640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd7d40a2c10 0x7fd7d40a5120 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.901+0000 7fd7ee6e9640 1 --2- 172.21.15.38:0/3887336143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7e8102ab0 0x7fd7e81992c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:42.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.901+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3887336143 >> 172.21.15.38:0/3887336143 conn(0x7fd7e80fdcc0 msgr2=0x7fd7e81049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:42.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.901+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3887336143 shutdown_connections 2024-05-21T15:34:42.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:42.901+0000 7fd7ee6e9640 1 -- 172.21.15.38:0/3887336143 wait complete. 2024-05-21T15:34:42.912 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:34:43.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.027+0000 7f56b2a84640 1 -- 172.21.15.38:0/4169549166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ac102bf0 msgr2=0x7f56ac103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.027+0000 7f56b2a84640 1 --2- 172.21.15.38:0/4169549166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ac102bf0 0x7f56ac103000 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f569c009ca0 tx=0x7f569c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.027+0000 7f56b2a84640 1 -- 172.21.15.38:0/4169549166 shutdown_connections 2024-05-21T15:34:43.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.027+0000 7f56b2a84640 1 --2- 172.21.15.38:0/4169549166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ac1078f0 0x7f56ac109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.027+0000 7f56b2a84640 1 --2- 172.21.15.38:0/4169549166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ac102bf0 0x7f56ac103000 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.027+0000 7f56b2a84640 1 --2- 172.21.15.38:0/4169549166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56ac1019b0 0x7f56ac1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.027+0000 7f56b2a84640 1 -- 172.21.15.38:0/4169549166 >> 172.21.15.38:0/4169549166 conn(0x7f56ac0fcba0 msgr2=0x7f56ac0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.027+0000 7f56b2a84640 1 -- 172.21.15.38:0/4169549166 shutdown_connections 2024-05-21T15:34:43.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.027+0000 7f56b2a84640 1 -- 172.21.15.38:0/4169549166 wait complete. 2024-05-21T15:34:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56b2a84640 1 Processor -- start 2024-05-21T15:34:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56b2a84640 1 -- start start 2024-05-21T15:34:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56b2a84640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ac1019b0 0x7f56ac0747f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56b2a84640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ac102bf0 0x7f56ac19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56b2a84640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56ac1078f0 0x7f56ac1a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56b2a84640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac10d630 con 0x7f56ac1019b0 2024-05-21T15:34:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56b2a84640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac10d4b0 con 0x7f56ac1078f0 2024-05-21T15:34:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56b2a84640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac10d7b0 con 0x7f56ac102bf0 2024-05-21T15:34:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56b0820640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ac1019b0 0x7f56ac0747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56b0820640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ac1019b0 0x7f56ac0747f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60090/0 (socket says 172.21.15.38:60090) 2024-05-21T15:34:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56b0820640 1 -- 172.21.15.38:0/3113486547 learned_addr learned my addr 172.21.15.38:0/3113486547 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56abfff640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ac102bf0 0x7f56ac19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56b1021640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56ac1078f0 0x7f56ac1a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56b0820640 1 -- 172.21.15.38:0/3113486547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ac102bf0 msgr2=0x7f56ac19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.028+0000 7f56b0820640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ac102bf0 0x7f56ac19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.029+0000 7f56b0820640 1 -- 172.21.15.38:0/3113486547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56ac1078f0 msgr2=0x7f56ac1a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.029+0000 7f56b0820640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56ac1078f0 0x7f56ac1a04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.029+0000 7f56b0820640 1 -- 172.21.15.38:0/3113486547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac1a0c40 con 0x7f56ac1019b0 2024-05-21T15:34:43.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.029+0000 7f56abfff640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ac102bf0 0x7f56ac19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:43.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.029+0000 7f56b0820640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ac1019b0 0x7f56ac0747f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f569400c970 tx=0x7f569400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.029+0000 7f56a9ffb640 1 -- 172.21.15.38:0/3113486547 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694007c70 con 0x7f56ac1019b0 2024-05-21T15:34:43.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.029+0000 7f56a9ffb640 1 -- 172.21.15.38:0/3113486547 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5694007e10 con 0x7f56ac1019b0 2024-05-21T15:34:43.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.029+0000 7f56b2a84640 1 -- 172.21.15.38:0/3113486547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ac1a0f30 con 0x7f56ac1019b0 2024-05-21T15:34:43.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.029+0000 7f56a9ffb640 1 -- 172.21.15.38:0/3113486547 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56940188c0 con 0x7f56ac1019b0 2024-05-21T15:34:43.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.029+0000 7f56b2a84640 1 -- 172.21.15.38:0/3113486547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ac1a1390 con 0x7f56ac1019b0 2024-05-21T15:34:43.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.030+0000 7f56b2a84640 1 -- 172.21.15.38:0/3113486547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ac0691c0 con 0x7f56ac1019b0 2024-05-21T15:34:43.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.032+0000 7f56a9ffb640 1 -- 172.21.15.38:0/3113486547 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f569400f450 con 0x7f56ac1019b0 2024-05-21T15:34:43.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.032+0000 7f56a9ffb640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f56980a2c30 0x7f56980a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.032+0000 7f56a9ffb640 1 -- 172.21.15.38:0/3113486547 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f56940cf3a0 con 0x7f56ac1019b0 2024-05-21T15:34:43.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.032+0000 7f56abfff640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f56980a2c30 0x7f56980a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.033+0000 7f56abfff640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f56980a2c30 0x7f56980a5140 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f569c0094e0 tx=0x7f569c000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.036+0000 7f56a9ffb640 1 -- 172.21.15.38:0/3113486547 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5694014030 con 0x7f56ac1019b0 2024-05-21T15:34:43.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.260+0000 7f56b2a84640 1 -- 172.21.15.38:0/3113486547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56ac02d2a0 con 0x7f56ac1019b0 2024-05-21T15:34:43.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.261+0000 7f56a9ffb640 1 -- 172.21.15.38:0/3113486547 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f569409c2b0 con 0x7f56ac1019b0 2024-05-21T15:34:43.261 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:43.261 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:34:36.440701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24551,24523,24602,24550,24515]},"id":1}]} 2024-05-21T15:34:43.261 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-05-21T15:34:43.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.263+0000 7f56b2a84640 1 -- 172.21.15.38:0/3113486547 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f56980a2c30 msgr2=0x7f56980a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.263+0000 7f56b2a84640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f56980a2c30 0x7f56980a5140 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f569c0094e0 tx=0x7f569c000f40 comp rx=0 tx=0).stop 2024-05-21T15:34:43.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.263+0000 7f56b2a84640 1 -- 172.21.15.38:0/3113486547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ac1019b0 msgr2=0x7f56ac0747f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.263+0000 7f56b2a84640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ac1019b0 0x7f56ac0747f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f569400c970 tx=0x7f569400ce30 comp rx=0 tx=0).stop 2024-05-21T15:34:43.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.263+0000 7f56b2a84640 1 -- 172.21.15.38:0/3113486547 shutdown_connections 2024-05-21T15:34:43.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.263+0000 7f56b2a84640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56ac1078f0 0x7f56ac1a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.263+0000 7f56b2a84640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56ac102bf0 0x7f56ac19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.263+0000 7f56b2a84640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f56980a2c30 0x7f56980a5140 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.263+0000 7f56b2a84640 1 --2- 172.21.15.38:0/3113486547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56ac1019b0 0x7f56ac0747f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.263+0000 7f56b2a84640 1 -- 172.21.15.38:0/3113486547 >> 172.21.15.38:0/3113486547 conn(0x7f56ac0fcba0 msgr2=0x7f56ac0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.263+0000 7f56b2a84640 1 -- 172.21.15.38:0/3113486547 shutdown_connections 2024-05-21T15:34:43.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.263+0000 7f56b2a84640 1 -- 172.21.15.38:0/3113486547 wait complete. 2024-05-21T15:34:43.275 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:34:43.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.390+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4186852505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af4102bf0 msgr2=0x7f3af4103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.390+0000 7f3af9e7a640 1 --2- 172.21.15.38:0/4186852505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af4102bf0 0x7f3af4103000 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8009900 tx=0x7f3ae802a410 comp rx=0 tx=0).stop 2024-05-21T15:34:43.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.390+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4186852505 shutdown_connections 2024-05-21T15:34:43.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.390+0000 7f3af9e7a640 1 --2- 172.21.15.38:0/4186852505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af41078f0 0x7f3af4109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.390+0000 7f3af9e7a640 1 --2- 172.21.15.38:0/4186852505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af4102bf0 0x7f3af4103000 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.390+0000 7f3af9e7a640 1 --2- 172.21.15.38:0/4186852505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3af41019b0 0x7f3af41026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.390+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4186852505 >> 172.21.15.38:0/4186852505 conn(0x7f3af40fcba0 msgr2=0x7f3af40ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.390+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4186852505 shutdown_connections 2024-05-21T15:34:43.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.390+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4186852505 wait complete. 2024-05-21T15:34:43.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af9e7a640 1 Processor -- start 2024-05-21T15:34:43.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af9e7a640 1 -- start start 2024-05-21T15:34:43.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af9e7a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af41019b0 0x7f3af4199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af9e7a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af4102bf0 0x7f3af4199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af9e7a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3af41078f0 0x7f3af41a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af9e7a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410e730 con 0x7f3af41019b0 2024-05-21T15:34:43.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af9e7a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410e5b0 con 0x7f3af4102bf0 2024-05-21T15:34:43.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af9e7a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410e8b0 con 0x7f3af41078f0 2024-05-21T15:34:43.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af41019b0 0x7f3af4199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af41019b0 0x7f3af4199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60108/0 (socket says 172.21.15.38:60108) 2024-05-21T15:34:43.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af37fe640 1 -- 172.21.15.38:0/4153065470 learned_addr learned my addr 172.21.15.38:0/4153065470 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:43.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af2ffd640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af4102bf0 0x7f3af4199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af3fff640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3af41078f0 0x7f3af41a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af37fe640 1 -- 172.21.15.38:0/4153065470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3af41078f0 msgr2=0x7f3af41a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af37fe640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3af41078f0 0x7f3af41a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af37fe640 1 -- 172.21.15.38:0/4153065470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af4102bf0 msgr2=0x7f3af4199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af37fe640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af4102bf0 0x7f3af4199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af37fe640 1 -- 172.21.15.38:0/4153065470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af41a0d40 con 0x7f3af41019b0 2024-05-21T15:34:43.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.391+0000 7f3af2ffd640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af4102bf0 0x7f3af4199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:43.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.392+0000 7f3af37fe640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af41019b0 0x7f3af4199200 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4005d00 tx=0x7f3ae400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.392+0000 7f3af0ff9640 1 -- 172.21.15.38:0/4153065470 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4017070 con 0x7f3af41019b0 2024-05-21T15:34:43.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.392+0000 7f3af0ff9640 1 -- 172.21.15.38:0/4153065470 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae4009c30 con 0x7f3af41019b0 2024-05-21T15:34:43.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.392+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4153065470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af41a0fd0 con 0x7f3af41019b0 2024-05-21T15:34:43.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.392+0000 7f3af0ff9640 1 -- 172.21.15.38:0/4153065470 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae400f620 con 0x7f3af41019b0 2024-05-21T15:34:43.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.392+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4153065470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af41a1490 con 0x7f3af41019b0 2024-05-21T15:34:43.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.393+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4153065470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af4106700 con 0x7f3af41019b0 2024-05-21T15:34:43.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.394+0000 7f3af0ff9640 1 -- 172.21.15.38:0/4153065470 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ae401a070 con 0x7f3af41019b0 2024-05-21T15:34:43.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.395+0000 7f3af0ff9640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3adc0a2cb0 0x7f3adc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.395+0000 7f3af0ff9640 1 -- 172.21.15.38:0/4153065470 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f3ae40ceb10 con 0x7f3af41019b0 2024-05-21T15:34:43.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.395+0000 7f3af2ffd640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3adc0a2cb0 0x7f3adc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.396+0000 7f3af2ffd640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3adc0a2cb0 0x7f3adc0a51c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8000c00 tx=0x7f3ae80095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.400+0000 7f3af0ff9640 1 -- 172.21.15.38:0/4153065470 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3ae4009de0 con 0x7f3af41019b0 2024-05-21T15:34:43.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:43 smithi038 ceph-mon[31995]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:34:43.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:43 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/31783281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:43.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:43 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3887336143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:43.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:43 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3113486547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:43.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.586+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4153065470 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3af4062750 con 0x7f3af41019b0 2024-05-21T15:34:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.587+0000 7f3af0ff9640 1 -- 172.21.15.38:0/4153065470 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22704 (secure 0 0 0) 0x7f3ae409ba20 con 0x7f3af41019b0 2024-05-21T15:34:43.589 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:34:43.590 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":90,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:34:23.996759+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:34:43.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.590+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4153065470 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3adc0a2cb0 msgr2=0x7f3adc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.590+0000 7f3af9e7a640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3adc0a2cb0 0x7f3adc0a51c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8000c00 tx=0x7f3ae80095f0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.590+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4153065470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af41019b0 msgr2=0x7f3af4199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.590+0000 7f3af9e7a640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af41019b0 0x7f3af4199200 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4005d00 tx=0x7f3ae400dde0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.590+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4153065470 shutdown_connections 2024-05-21T15:34:43.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.590+0000 7f3af9e7a640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3af41078f0 0x7f3af41a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.590+0000 7f3af9e7a640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3af4102bf0 0x7f3af4199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.590+0000 7f3af9e7a640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3adc0a2cb0 0x7f3adc0a51c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.590+0000 7f3af9e7a640 1 --2- 172.21.15.38:0/4153065470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3af41019b0 0x7f3af4199200 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.590+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4153065470 >> 172.21.15.38:0/4153065470 conn(0x7f3af40fcba0 msgr2=0x7f3af41038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.590+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4153065470 shutdown_connections 2024-05-21T15:34:43.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.590+0000 7f3af9e7a640 1 -- 172.21.15.38:0/4153065470 wait complete. 2024-05-21T15:34:43.601 DEBUG:tasks.fwd_scrub:Yielding 2024-05-21T15:34:43.601 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-21T15:34:43.611 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-05-21T15:34:43.611 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-21T15:34:43.612 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-21T15:34:43.612 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:34:43.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:34:43.612 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:34:43.614 INFO:teuthology.run_tasks:Running task workunit... 2024-05-21T15:34:43.702 INFO:tasks.workunit:Pulling workunits from ref d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:34:43.703 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-21T15:34:43.703 INFO:tasks.workunit:timeout=3h 2024-05-21T15:34:43.703 INFO:tasks.workunit:cleanup=True 2024-05-21T15:34:43.703 DEBUG:teuthology.orchestra.run.smithi038:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-21T15:34:43.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:43 smithi053 ceph-mon[36496]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:34:43.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:43 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/31783281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:43.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:43 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3887336143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:43.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:43 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3113486547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:43.720 INFO:teuthology.orchestra.run.smithi038.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-21T15:34:43.720 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-21T15:34:43.720 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-05-21T15:34:43.720 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:34:43.720 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-21T15:34:43.720 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-05-21 15:32:41.117040850 +0000 2024-05-21T15:34:43.720 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-05-21 15:32:41.117040850 +0000 2024-05-21T15:34:43.720 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-05-21 15:34:35.733142479 +0000 2024-05-21T15:34:43.720 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-05-21 15:32:41.117040850 +0000 2024-05-21T15:34:43.720 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-21T15:34:43.721 DEBUG:teuthology.orchestra.run.smithi038:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-21T15:34:43.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.732+0000 7f291c292640 1 -- 172.21.15.38:0/1223096804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2914102bc0 msgr2=0x7f2914102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.732+0000 7f291c292640 1 --2- 172.21.15.38:0/1223096804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2914102bc0 0x7f2914102fd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2908009900 tx=0x7f290802a410 comp rx=0 tx=0).stop 2024-05-21T15:34:43.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.732+0000 7f291c292640 1 -- 172.21.15.38:0/1223096804 shutdown_connections 2024-05-21T15:34:43.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.732+0000 7f291c292640 1 --2- 172.21.15.38:0/1223096804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29141078c0 0x7f2914109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.732+0000 7f291c292640 1 --2- 172.21.15.38:0/1223096804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2914102bc0 0x7f2914102fd0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.732+0000 7f291c292640 1 --2- 172.21.15.38:0/1223096804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2914101980 0x7f2914102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.732+0000 7f291c292640 1 -- 172.21.15.38:0/1223096804 >> 172.21.15.38:0/1223096804 conn(0x7f29140fcb70 msgr2=0x7f29140ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.732+0000 7f291c292640 1 -- 172.21.15.38:0/1223096804 shutdown_connections 2024-05-21T15:34:43.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.732+0000 7f291c292640 1 -- 172.21.15.38:0/1223096804 wait complete. 2024-05-21T15:34:43.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.733+0000 7f291c292640 1 Processor -- start 2024-05-21T15:34:43.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.733+0000 7f291c292640 1 -- start start 2024-05-21T15:34:43.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.733+0000 7f291c292640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2914101980 0x7f2914199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.733+0000 7f291c292640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2914102bc0 0x7f29141997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.733+0000 7f291c292640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141078c0 0x7f29141a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.733+0000 7f291c292640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f291410d600 con 0x7f29141078c0 2024-05-21T15:34:43.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.733+0000 7f291c292640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f291410d480 con 0x7f2914101980 2024-05-21T15:34:43.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.733+0000 7f291c292640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f291410d780 con 0x7f2914102bc0 2024-05-21T15:34:43.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.733+0000 7f291a82f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141078c0 0x7f29141a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.733+0000 7f291a82f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141078c0 0x7f29141a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60124/0 (socket says 172.21.15.38:60124) 2024-05-21T15:34:43.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.733+0000 7f291a82f640 1 -- 172.21.15.38:0/1850459841 learned_addr learned my addr 172.21.15.38:0/1850459841 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:43.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.733+0000 7f291982d640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2914102bc0 0x7f29141997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.733+0000 7f291a02e640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2914101980 0x7f2914199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.734+0000 7f291a82f640 1 -- 172.21.15.38:0/1850459841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2914102bc0 msgr2=0x7f29141997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.734+0000 7f291a82f640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2914102bc0 0x7f29141997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.734+0000 7f291a82f640 1 -- 172.21.15.38:0/1850459841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2914101980 msgr2=0x7f2914199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.734+0000 7f291a82f640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2914101980 0x7f2914199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.734+0000 7f291a82f640 1 -- 172.21.15.38:0/1850459841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29141a0db0 con 0x7f29141078c0 2024-05-21T15:34:43.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.734+0000 7f291a02e640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2914101980 0x7f2914199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:43.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.734+0000 7f291a82f640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141078c0 0x7f29141a0740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f290c00c970 tx=0x7f290c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.734+0000 7f29137fe640 1 -- 172.21.15.38:0/1850459841 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c007bb0 con 0x7f29141078c0 2024-05-21T15:34:43.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.734+0000 7f29137fe640 1 -- 172.21.15.38:0/1850459841 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f290c007d50 con 0x7f29141078c0 2024-05-21T15:34:43.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.734+0000 7f291c292640 1 -- 172.21.15.38:0/1850459841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29141a10a0 con 0x7f29141078c0 2024-05-21T15:34:43.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.734+0000 7f29137fe640 1 -- 172.21.15.38:0/1850459841 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c00f450 con 0x7f29141078c0 2024-05-21T15:34:43.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.734+0000 7f291c292640 1 -- 172.21.15.38:0/1850459841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29141a15f0 con 0x7f29141078c0 2024-05-21T15:34:43.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.737+0000 7f2923219740 1 Processor -- start 2024-05-21T15:34:43.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.737+0000 7f29137fe640 1 -- 172.21.15.38:0/1850459841 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f290c00f5f0 con 0x7f29141078c0 2024-05-21T15:34:43.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.737+0000 7f2923219740 1 -- start start 2024-05-21T15:34:43.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.737+0000 7f2923219740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a22f410 0x55802a5cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.737+0000 7f2923219740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a206550 0x55802a5cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.737+0000 7f2923219740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a20e730 0x55802a5d6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.737+0000 7f2923219740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55802a533990 con 0x55802a20e730 2024-05-21T15:34:43.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.737+0000 7f2923219740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55802a4db5b0 con 0x55802a22f410 2024-05-21T15:34:43.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.737+0000 7f2923219740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55802a52f950 con 0x55802a206550 2024-05-21T15:34:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f291982d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a20e730 0x55802a5d6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f291982d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a20e730 0x55802a5d6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60126/0 (socket says 172.21.15.38:60126) 2024-05-21T15:34:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f291982d640 1 -- 172.21.15.38:0/2770924903 learned_addr learned my addr 172.21.15.38:0/2770924903 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f291a02e640 1 --2- 172.21.15.38:0/2770924903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a206550 0x55802a5cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f291982d640 1 --2- 172.21.15.38:0/2770924903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a22f410 0x55802a5cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f29137fe640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28fc0a2b90 0x7f28fc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f291a02e640 1 -- 172.21.15.38:0/2770924903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a22f410 msgr2=0x55802a5cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f291a02e640 1 --2- 172.21.15.38:0/2770924903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a22f410 0x55802a5cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f291a02e640 1 -- 172.21.15.38:0/2770924903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a20e730 msgr2=0x55802a5d6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f291a02e640 1 --2- 172.21.15.38:0/2770924903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a20e730 0x55802a5d6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f291a02e640 1 -- 172.21.15.38:0/2770924903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55802a30f7e0 con 0x55802a206550 2024-05-21T15:34:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f291a82f640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28fc0a2b90 0x7f28fc0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f29137fe640 1 -- 172.21.15.38:0/1850459841 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f290c0ced60 con 0x7f29141078c0 2024-05-21T15:34:43.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f291a02e640 1 --2- 172.21.15.38:0/2770924903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a206550 0x55802a5cfa20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2914101250 tx=0x7f2904010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a92ebc5dd20e355 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.738+0000 7f2910bfe640 1 -- 172.21.15.38:0/2770924903 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904016070 con 0x55802a206550 2024-05-21T15:34:43.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.739+0000 7f2910bfe640 1 -- 172.21.15.38:0/2770924903 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2904004d20 con 0x55802a206550 2024-05-21T15:34:43.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.739+0000 7f2910bfe640 1 -- 172.21.15.38:0/2770924903 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29040127e0 con 0x55802a206550 2024-05-21T15:34:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.739+0000 7f2923219740 1 -- 172.21.15.38:0/2770924903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a206550 msgr2=0x55802a5cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.739+0000 7f291a82f640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28fc0a2b90 0x7f28fc0a50a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f290c09b500 tx=0x7f290c0661f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.739+0000 7f2923219740 1 --2- 172.21.15.38:0/2770924903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a206550 0x55802a5cfa20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2914101250 tx=0x7f2904010800 comp rx=0 tx=0).stop 2024-05-21T15:34:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.739+0000 7f2923219740 1 -- 172.21.15.38:0/2770924903 shutdown_connections 2024-05-21T15:34:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.739+0000 7f2923219740 1 --2- 172.21.15.38:0/2770924903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a20e730 0x55802a5d6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.739+0000 7f2923219740 1 --2- 172.21.15.38:0/2770924903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a206550 0x55802a5cfa20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.739+0000 7f2923219740 1 --2- 172.21.15.38:0/2770924903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a22f410 0x55802a5cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.739+0000 7f2923219740 1 -- 172.21.15.38:0/2770924903 >> 172.21.15.38:0/2770924903 conn(0x55802a223560 msgr2=0x55802a5cae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.740+0000 7f2923219740 1 -- 172.21.15.38:0/2770924903 shutdown_connections 2024-05-21T15:34:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.740+0000 7f2923219740 1 -- 172.21.15.38:0/2770924903 wait complete. 2024-05-21T15:34:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.740+0000 7f2923219740 1 -- start start 2024-05-21T15:34:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.740+0000 7f2923219740 1 Processor -- start 2024-05-21T15:34:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.740+0000 7f2923219740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a206550 0x55802a5d9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.740+0000 7f2923219740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a20e730 0x55802a5d9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.740+0000 7f2923219740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a22f410 0x55802a5e08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.740+0000 7f2923219740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55802a5d7370 con 0x55802a20e730 2024-05-21T15:34:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.740+0000 7f2923219740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55802a4eeb40 con 0x55802a206550 2024-05-21T15:34:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.740+0000 7f2923219740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55802a5d74f0 con 0x55802a22f410 2024-05-21T15:34:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.740+0000 7f2923219740 10 client.0 _ll_register_callbacks cb 0x55802a4e30f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.740+0000 7f291a02e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a20e730 0x55802a5d9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.740+0000 7f291a02e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a20e730 0x55802a5d9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60142/0 (socket says 172.21.15.38:60142) 2024-05-21T15:34:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.740+0000 7f291a02e640 1 -- 172.21.15.38:0/1713472154 learned_addr learned my addr 172.21.15.38:0/1713472154 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f291982d640 1 --2- 172.21.15.38:0/1713472154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a22f410 0x55802a5e08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f291982d640 1 --2- 172.21.15.38:0/1713472154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a206550 0x55802a5d9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f291a02e640 1 -- 172.21.15.38:0/1713472154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a22f410 msgr2=0x55802a5e08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f291a02e640 1 --2- 172.21.15.38:0/1713472154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a22f410 0x55802a5e08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f291a02e640 1 -- 172.21.15.38:0/1713472154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a206550 msgr2=0x55802a5d9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f291a02e640 1 --2- 172.21.15.38:0/1713472154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a206550 0x55802a5d9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f291a02e640 1 -- 172.21.15.38:0/1713472154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55802a36ff70 con 0x55802a20e730 2024-05-21T15:34:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f291982d640 1 --2- 172.21.15.38:0/1713472154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a22f410 0x55802a5e08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f291a02e640 1 --2- 172.21.15.38:0/1713472154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a20e730 0x55802a5d9970 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2904004c70 tx=0x7f2904004ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ed4c37b09da40 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f28f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f28f9ffb640 1 -- 172.21.15.38:0/1713472154 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904016070 con 0x55802a20e730 2024-05-21T15:34:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f2923219740 1 -- 172.21.15.38:0/1713472154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2904004d20 con 0x55802a20e730 2024-05-21T15:34:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f28f9ffb640 1 -- 172.21.15.38:0/1713472154 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2904003970 con 0x55802a20e730 2024-05-21T15:34:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f28f9ffb640 1 -- 172.21.15.38:0/1713472154 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2904003b70 con 0x55802a20e730 2024-05-21T15:34:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f2923219740 10 client.15495 fetch_fsmap learned FSMap version 27 2024-05-21T15:34:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.741+0000 7f2923219740 1 -- 172.21.15.38:0/1713472154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55802a207170 con 0x55802a20e730 2024-05-21T15:34:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.742+0000 7f28f9ffb640 1 -- 172.21.15.38:0/1713472154 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2904004d20 con 0x55802a20e730 2024-05-21T15:34:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.742+0000 7f28f9ffb640 1 -- 172.21.15.38:0/1713472154 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f290402d070 con 0x55802a20e730 2024-05-21T15:34:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.742+0000 7f2923219740 10 client.15495 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:34:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.742+0000 7f2923219740 10 client.15495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:34:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.742+0000 7f2923219740 20 client.15495 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.742+0000 7f2923219740 1 --2- 172.21.15.38:0/1713472154 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55802a1ee1f0 0x55802a5e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.742+0000 7f2923219740 4 client.15495 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:34:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.742+0000 7f2923219740 1 -- 172.21.15.38:0/1713472154 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55802a5410c0 con 0x55802a1ee1f0 2024-05-21T15:34:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.742+0000 7f291982d640 1 --2- 172.21.15.38:0/1713472154 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55802a1ee1f0 0x55802a5e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.743+0000 7f291982d640 1 --2- 172.21.15.38:0/1713472154 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55802a1ee1f0 0x55802a5e6aa0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445635f55e894483 server_cookie=1c6f2c3d9bc6e2be in_seq=0 out_seq=0 2024-05-21T15:34:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.743+0000 7f28f9ffb640 10 client.15495 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.743+0000 7f28f9ffb640 1 -- 172.21.15.38:0/1713472154 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55802a5410c0 con 0x55802a1ee1f0 2024-05-21T15:34:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.743+0000 7f28f9ffb640 10 client.15495 handle_command_reply: tid=0 2024-05-21T15:34:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.743+0000 7f2923219740 1 client.15495 shutdown 2024-05-21T15:34:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.744+0000 7f2923219740 1 -- 172.21.15.38:0/1713472154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a20e730 msgr2=0x55802a5d9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.744+0000 7f2923219740 1 --2- 172.21.15.38:0/1713472154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a20e730 0x55802a5d9970 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2904004c70 tx=0x7f2904004ca0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.744+0000 7f2923219740 1 -- 172.21.15.38:0/1713472154 shutdown_connections 2024-05-21T15:34:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.744+0000 7f2923219740 1 --2- 172.21.15.38:0/1713472154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a22f410 0x55802a5e08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.744+0000 7f2923219740 1 --2- 172.21.15.38:0/1713472154 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55802a1ee1f0 0x55802a5e6aa0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.744+0000 7f2923219740 1 --2- 172.21.15.38:0/1713472154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a20e730 0x55802a5d9970 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.744+0000 7f2923219740 1 --2- 172.21.15.38:0/1713472154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a206550 0x55802a5d9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.744+0000 7f28f9ffb640 0 client.15495 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.744+0000 7f2923219740 1 -- 172.21.15.38:0/1713472154 >> 172.21.15.38:0/1713472154 conn(0x55802a223560 msgr2=0x55802a5c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.744+0000 7f2923219740 1 -- 172.21.15.38:0/1713472154 shutdown_connections 2024-05-21T15:34:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.744+0000 7f2923219740 1 -- 172.21.15.38:0/1713472154 wait complete. 2024-05-21T15:34:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.744+0000 7f2923219740 20 client.15495 trim_cache size 0 max 16384 2024-05-21T15:34:43.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:43 smithi045 ceph-mon[36409]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:34:43.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:43 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/31783281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:34:43.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:43 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3887336143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:34:43.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:43 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3113486547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:34:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.762+0000 7f2923219740 1 Processor -- start 2024-05-21T15:34:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.762+0000 7f2923219740 1 -- start start 2024-05-21T15:34:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f2923219740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a206550 0x55802a5d7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f2923219740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a20e730 0x55802a5d7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f2923219740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a22f410 0x55802a5ca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f2923219740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f29141aa4f0 con 0x55802a206550 2024-05-21T15:34:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f2923219740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29141ac030 con 0x55802a20e730 2024-05-21T15:34:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f2923219740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29141aabc0 con 0x55802a22f410 2024-05-21T15:34:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f291982d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a206550 0x55802a5d7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f291982d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a206550 0x55802a5d7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60154/0 (socket says 172.21.15.38:60154) 2024-05-21T15:34:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f291982d640 1 -- 172.21.15.38:0/3712079310 learned_addr learned my addr 172.21.15.38:0/3712079310 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f291a02e640 1 --2- 172.21.15.38:0/3712079310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a20e730 0x55802a5d7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f291982d640 1 --2- 172.21.15.38:0/3712079310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a22f410 0x55802a5ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f291982d640 1 -- 172.21.15.38:0/3712079310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a22f410 msgr2=0x55802a5ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f291982d640 1 --2- 172.21.15.38:0/3712079310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a22f410 0x55802a5ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f291982d640 1 -- 172.21.15.38:0/3712079310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a20e730 msgr2=0x55802a5d7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f291982d640 1 --2- 172.21.15.38:0/3712079310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a20e730 0x55802a5d7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f291982d640 1 -- 172.21.15.38:0/3712079310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55802a5e6aa0 con 0x55802a206550 2024-05-21T15:34:43.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f291a02e640 1 --2- 172.21.15.38:0/3712079310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a20e730 0x55802a5d7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:34:43.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.763+0000 7f291982d640 1 --2- 172.21.15.38:0/3712079310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a206550 0x55802a5d7a70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2908005ab0 tx=0x7f2908000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185939cc325eb53e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.764+0000 7f28fbfff640 1 -- 172.21.15.38:0/3712079310 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908035040 con 0x55802a206550 2024-05-21T15:34:43.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.764+0000 7f28fbfff640 1 -- 172.21.15.38:0/3712079310 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2908005030 con 0x55802a206550 2024-05-21T15:34:43.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.764+0000 7f28fbfff640 1 -- 172.21.15.38:0/3712079310 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29080049c0 con 0x55802a206550 2024-05-21T15:34:43.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.764+0000 7f2923219740 1 -- 172.21.15.38:0/3712079310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a206550 msgr2=0x55802a5d7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.764+0000 7f2923219740 1 --2- 172.21.15.38:0/3712079310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a206550 0x55802a5d7a70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2908005ab0 tx=0x7f2908000bc0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.764+0000 7f2923219740 1 -- 172.21.15.38:0/3712079310 shutdown_connections 2024-05-21T15:34:43.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.764+0000 7f2923219740 1 --2- 172.21.15.38:0/3712079310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a22f410 0x55802a5ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.764+0000 7f2923219740 1 --2- 172.21.15.38:0/3712079310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a20e730 0x55802a5d7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.764+0000 7f2923219740 1 --2- 172.21.15.38:0/3712079310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a206550 0x55802a5d7a70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.764+0000 7f2923219740 1 -- 172.21.15.38:0/3712079310 >> 172.21.15.38:0/3712079310 conn(0x55802a223560 msgr2=0x55802a5d3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f2923219740 1 -- 172.21.15.38:0/3712079310 shutdown_connections 2024-05-21T15:34:43.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f2923219740 1 -- 172.21.15.38:0/3712079310 wait complete. 2024-05-21T15:34:43.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f2923219740 1 -- start start 2024-05-21T15:34:43.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f2923219740 1 Processor -- start 2024-05-21T15:34:43.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f2923219740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a206550 0x55802a5dd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f2923219740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a20e730 0x55802a5ddbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f2923219740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a22f410 0x55802a5deae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f2923219740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f29141ab960 con 0x55802a22f410 2024-05-21T15:34:43.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f2923219740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29141ab290 con 0x55802a206550 2024-05-21T15:34:43.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f2923219740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29141a84d0 con 0x55802a20e730 2024-05-21T15:34:43.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f2923219740 10 client.0 _ll_register_callbacks cb 0x7f2908005ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:43.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f291982d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a22f410 0x55802a5deae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f291982d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a22f410 0x55802a5deae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60166/0 (socket says 172.21.15.38:60166) 2024-05-21T15:34:43.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f291982d640 1 -- 172.21.15.38:0/4249446574 learned_addr learned my addr 172.21.15.38:0/4249446574 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:43.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f291a02e640 1 --2- 172.21.15.38:0/4249446574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a20e730 0x55802a5ddbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.765+0000 7f291982d640 1 --2- 172.21.15.38:0/4249446574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a206550 0x55802a5dd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f291982d640 1 -- 172.21.15.38:0/4249446574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a20e730 msgr2=0x55802a5ddbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f291982d640 1 --2- 172.21.15.38:0/4249446574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a20e730 0x55802a5ddbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f291982d640 1 -- 172.21.15.38:0/4249446574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a206550 msgr2=0x55802a5dd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f291982d640 1 --2- 172.21.15.38:0/4249446574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a206550 0x55802a5dd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f291982d640 1 -- 172.21.15.38:0/4249446574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55802a5df0a0 con 0x55802a22f410 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f291a02e640 1 --2- 172.21.15.38:0/4249446574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a20e730 0x55802a5ddbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f291982d640 1 --2- 172.21.15.38:0/4249446574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a22f410 0x55802a5deae0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2908031e20 tx=0x7f2908009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c909e50a2d3507f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f28f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f28f9ffb640 1 -- 172.21.15.38:0/4249446574 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908035070 con 0x55802a22f410 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f28f9ffb640 1 -- 172.21.15.38:0/4249446574 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f29080053e0 con 0x55802a22f410 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f2923219740 1 -- 172.21.15.38:0/4249446574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2908005030 con 0x55802a22f410 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f28f9ffb640 1 -- 172.21.15.38:0/4249446574 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2908004760 con 0x55802a22f410 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f2923219740 10 client.15501 fetch_fsmap learned FSMap version 27 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f2923219740 1 -- 172.21.15.38:0/4249446574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55802a64ed70 con 0x55802a22f410 2024-05-21T15:34:43.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.766+0000 7f28f9ffb640 1 -- 172.21.15.38:0/4249446574 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2908005030 con 0x55802a22f410 2024-05-21T15:34:43.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.767+0000 7f28f9ffb640 1 -- 172.21.15.38:0/4249446574 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2908060050 con 0x55802a22f410 2024-05-21T15:34:43.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.767+0000 7f2923219740 10 client.15501 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:34:43.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.767+0000 7f2923219740 10 client.15501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:34:43.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.767+0000 7f2923219740 20 client.15501 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:43.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.767+0000 7f2923219740 1 --2- 172.21.15.38:0/4249446574 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55802a1c3630 0x55802a6513d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.767+0000 7f2923219740 4 client.15501 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:34:43.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.767+0000 7f2923219740 1 -- 172.21.15.38:0/4249446574 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55802a53b410 con 0x55802a1c3630 2024-05-21T15:34:43.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.767+0000 7f291a02e640 1 --2- 172.21.15.38:0/4249446574 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55802a1c3630 0x55802a6513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.767+0000 7f28f9ffb640 10 client.15501 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:43.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.767+0000 7f291a02e640 1 --2- 172.21.15.38:0/4249446574 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55802a1c3630 0x55802a6513d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd8f186284db4a32 server_cookie=b2202d7af495da75 in_seq=0 out_seq=0 2024-05-21T15:34:43.774 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:34:43.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f28f9ffb640 1 -- 172.21.15.38:0/4249446574 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55802a53b410 con 0x55802a1c3630 2024-05-21T15:34:43.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f28f9ffb640 10 client.15501 handle_command_reply: tid=0 2024-05-21T15:34:43.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f2923219740 1 client.15501 shutdown 2024-05-21T15:34:43.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f2923219740 1 -- 172.21.15.38:0/4249446574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a22f410 msgr2=0x55802a5deae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f2923219740 1 --2- 172.21.15.38:0/4249446574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a22f410 0x55802a5deae0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2908031e20 tx=0x7f2908009260 comp rx=0 tx=0).stop 2024-05-21T15:34:43.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f2923219740 1 -- 172.21.15.38:0/4249446574 shutdown_connections 2024-05-21T15:34:43.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f2923219740 1 --2- 172.21.15.38:0/4249446574 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55802a1c3630 0x55802a6513d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f2923219740 1 --2- 172.21.15.38:0/4249446574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55802a22f410 0x55802a5deae0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f28f9ffb640 0 client.15501 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:43.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f2923219740 1 --2- 172.21.15.38:0/4249446574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55802a20e730 0x55802a5ddbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f2923219740 1 --2- 172.21.15.38:0/4249446574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55802a206550 0x55802a5dd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f2923219740 1 -- 172.21.15.38:0/4249446574 >> 172.21.15.38:0/4249446574 conn(0x55802a223560 msgr2=0x55802a5d7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f2923219740 1 -- 172.21.15.38:0/4249446574 shutdown_connections 2024-05-21T15:34:43.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f2923219740 1 -- 172.21.15.38:0/4249446574 wait complete. 2024-05-21T15:34:43.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.769+0000 7f2923219740 20 client.15501 trim_cache size 0 max 16384 2024-05-21T15:34:43.776 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:34:43.776 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-05-21T15:34:43.776 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "8f981338-957a-40e5-b340-68b75df02100", 2024-05-21T15:34:43.776 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-05-21T15:34:43.777 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:34:43.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.770+0000 7f291c292640 1 -- 172.21.15.38:0/1850459841 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28fc0a2b90 msgr2=0x7f28fc0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.770+0000 7f291c292640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28fc0a2b90 0x7f28fc0a50a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f290c09b500 tx=0x7f290c0661f0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.770+0000 7f291c292640 1 -- 172.21.15.38:0/1850459841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141078c0 msgr2=0x7f29141a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.770+0000 7f291c292640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141078c0 0x7f29141a0740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f290c00c970 tx=0x7f290c00ce30 comp rx=0 tx=0).stop 2024-05-21T15:34:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.770+0000 7f291c292640 1 -- 172.21.15.38:0/1850459841 shutdown_connections 2024-05-21T15:34:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.770+0000 7f291c292640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28fc0a2b90 0x7f28fc0a50a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.770+0000 7f291c292640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29141078c0 0x7f29141a0740 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.770+0000 7f291c292640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2914102bc0 0x7f29141997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.770+0000 7f291c292640 1 --2- 172.21.15.38:0/1850459841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2914101980 0x7f2914199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.770+0000 7f291c292640 1 -- 172.21.15.38:0/1850459841 >> 172.21.15.38:0/1850459841 conn(0x7f29140fcb70 msgr2=0x7f2914108cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.770+0000 7f291c292640 1 -- 172.21.15.38:0/1850459841 shutdown_connections 2024-05-21T15:34:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.770+0000 7f291c292640 1 -- 172.21.15.38:0/1850459841 wait complete. 2024-05-21T15:34:43.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:34:43.781 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:34:43.837 INFO:tasks.workunit.client.0.smithi038.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-21T15:34:43.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.942+0000 7f6cb0037640 1 -- 172.21.15.38:0/158732515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ca8101990 msgr2=0x7f6ca8102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.942+0000 7f6cb0037640 1 --2- 172.21.15.38:0/158732515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ca8101990 0x7f6ca8102690 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009900 tx=0x7f6ca002a410 comp rx=0 tx=0).stop 2024-05-21T15:34:43.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.942+0000 7f6cb0037640 1 -- 172.21.15.38:0/158732515 shutdown_connections 2024-05-21T15:34:43.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.942+0000 7f6cb0037640 1 --2- 172.21.15.38:0/158732515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ca81078d0 0x7f6ca8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.942+0000 7f6cb0037640 1 --2- 172.21.15.38:0/158732515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ca8102bd0 0x7f6ca8102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.942+0000 7f6cb0037640 1 --2- 172.21.15.38:0/158732515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ca8101990 0x7f6ca8102690 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.942+0000 7f6cb0037640 1 -- 172.21.15.38:0/158732515 >> 172.21.15.38:0/158732515 conn(0x7f6ca80fcb80 msgr2=0x7f6ca80ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.942+0000 7f6cb0037640 1 -- 172.21.15.38:0/158732515 shutdown_connections 2024-05-21T15:34:43.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.942+0000 7f6cb0037640 1 -- 172.21.15.38:0/158732515 wait complete. 2024-05-21T15:34:43.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6cb0037640 1 Processor -- start 2024-05-21T15:34:43.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6cb0037640 1 -- start start 2024-05-21T15:34:43.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6cb0037640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ca8101990 0x7f6ca80747f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6cb0037640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ca8102bd0 0x7f6ca819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6cb0037640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ca81078d0 0x7f6ca81a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6cb0037640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca810d5e0 con 0x7f6ca8101990 2024-05-21T15:34:43.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6cb0037640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca810d460 con 0x7f6ca81078d0 2024-05-21T15:34:43.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6cb0037640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca810d760 con 0x7f6ca8102bd0 2024-05-21T15:34:43.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6caddd3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ca8101990 0x7f6ca80747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6caddd3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ca8101990 0x7f6ca80747f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60198/0 (socket says 172.21.15.38:60198) 2024-05-21T15:34:43.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6caddd3640 1 -- 172.21.15.38:0/1204584650 learned_addr learned my addr 172.21.15.38:0/1204584650 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:43.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6cad5d2640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ca8102bd0 0x7f6ca819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6cae5d4640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ca81078d0 0x7f6ca81a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6caddd3640 1 -- 172.21.15.38:0/1204584650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ca8102bd0 msgr2=0x7f6ca819e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6caddd3640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ca8102bd0 0x7f6ca819e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6caddd3640 1 -- 172.21.15.38:0/1204584650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ca81078d0 msgr2=0x7f6ca81a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.943+0000 7f6caddd3640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ca81078d0 0x7f6ca81a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.944+0000 7f6caddd3640 1 -- 172.21.15.38:0/1204584650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca81a0dc0 con 0x7f6ca8101990 2024-05-21T15:34:43.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.944+0000 7f6cad5d2640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ca8102bd0 0x7f6ca819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:43.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.944+0000 7f6cae5d4640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ca81078d0 0x7f6ca81a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:43.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.944+0000 7f6caddd3640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ca8101990 0x7f6ca80747f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0006010 tx=0x7f6ca0031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.944+0000 7f6ca6ffd640 1 -- 172.21.15.38:0/1204584650 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca003f030 con 0x7f6ca8101990 2024-05-21T15:34:43.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.944+0000 7f6ca6ffd640 1 -- 172.21.15.38:0/1204584650 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ca0003ad0 con 0x7f6ca8101990 2024-05-21T15:34:43.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.944+0000 7f6ca6ffd640 1 -- 172.21.15.38:0/1204584650 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0035070 con 0x7f6ca8101990 2024-05-21T15:34:43.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.944+0000 7f6cb0037640 1 -- 172.21.15.38:0/1204584650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca81a1050 con 0x7f6ca8101990 2024-05-21T15:34:43.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.944+0000 7f6cb0037640 1 -- 172.21.15.38:0/1204584650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca81a1590 con 0x7f6ca8101990 2024-05-21T15:34:43.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.947+0000 7f6cb6fb4740 1 Processor -- start 2024-05-21T15:34:43.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.947+0000 7f6cb6fb4740 1 -- start start 2024-05-21T15:34:43.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cb6fb4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556133585390 0x556133924bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cb6fb4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 0x556133925130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cb6fb4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561335646b0 0x55613392c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cb6fb4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556133889d00 con 0x5561335646b0 2024-05-21T15:34:43.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cb6fb4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5561338314c0 con 0x556133585390 2024-05-21T15:34:43.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cb6fb4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556133885cc0 con 0x55613355c4d0 2024-05-21T15:34:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6ca6ffd640 1 -- 172.21.15.38:0/1204584650 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6ca0042030 con 0x7f6ca8101990 2024-05-21T15:34:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cad5d2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561335646b0 0x55613392c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cad5d2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561335646b0 0x55613392c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60212/0 (socket says 172.21.15.38:60212) 2024-05-21T15:34:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cad5d2640 1 -- 172.21.15.38:0/3316063266 learned_addr learned my addr 172.21.15.38:0/3316063266 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cae5d4640 1 --2- 172.21.15.38:0/3316063266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556133585390 0x556133924bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cad5d2640 1 -- 172.21.15.38:0/3316063266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 msgr2=0x556133925130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6caddd3640 1 --2- 172.21.15.38:0/3316063266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 0x556133925130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cad5d2640 1 --2- 172.21.15.38:0/3316063266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 0x556133925130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cad5d2640 1 -- 172.21.15.38:0/3316063266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556133585390 msgr2=0x556133924bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cad5d2640 1 --2- 172.21.15.38:0/3316063266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556133585390 0x556133924bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.948+0000 7f6cad5d2640 1 -- 172.21.15.38:0/3316063266 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561336677d0 con 0x5561335646b0 2024-05-21T15:34:43.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.949+0000 7f6cad5d2640 1 --2- 172.21.15.38:0/3316063266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561335646b0 0x55613392c030 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00bd30 tx=0x7f6c9c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1a0d271243c7f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.949+0000 7f6c8ffff640 1 -- 172.21.15.38:0/3316063266 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c015070 con 0x5561335646b0 2024-05-21T15:34:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.949+0000 7f6ca6ffd640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6c900a2c10 0x7f6c900a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.949+0000 7f6c8ffff640 1 -- 172.21.15.38:0/3316063266 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c9c00be20 con 0x5561335646b0 2024-05-21T15:34:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.949+0000 7f6c8ffff640 1 -- 172.21.15.38:0/3316063266 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c0116a0 con 0x5561335646b0 2024-05-21T15:34:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.949+0000 7f6cae5d4640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6c900a2c10 0x7f6c900a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.949+0000 7f6ca6ffd640 1 -- 172.21.15.38:0/1204584650 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f6ca00ef690 con 0x7f6ca8101990 2024-05-21T15:34:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.950+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/3316063266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561335646b0 msgr2=0x55613392c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.950+0000 7f6cae5d4640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6c900a2c10 0x7f6c900a5120 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x556133925d80 tx=0x7f6c98009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.950+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/3316063266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561335646b0 0x55613392c030 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00bd30 tx=0x7f6c9c00f730 comp rx=0 tx=0).stop 2024-05-21T15:34:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.950+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/3316063266 shutdown_connections 2024-05-21T15:34:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.950+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/3316063266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561335646b0 0x55613392c030 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.950+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/3316063266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 0x556133925130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.950+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/3316063266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556133585390 0x556133924bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.950+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/3316063266 >> 172.21.15.38:0/3316063266 conn(0x5561335794e0 msgr2=0x556133920540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.950+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/3316063266 shutdown_connections 2024-05-21T15:34:43.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.950+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/3316063266 wait complete. 2024-05-21T15:34:43.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.950+0000 7f6cb6fb4740 1 -- start start 2024-05-21T15:34:43.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.950+0000 7f6cb6fb4740 1 Processor -- start 2024-05-21T15:34:43.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cb6fb4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55613355c4d0 0x55613392eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cb6fb4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561335646b0 0x55613392f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cb6fb4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556133585390 0x556133935fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cb6fb4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55613392ca80 con 0x556133585390 2024-05-21T15:34:43.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cb6fb4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556133844b90 con 0x55613355c4d0 2024-05-21T15:34:43.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cb6fb4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55613392cc00 con 0x5561335646b0 2024-05-21T15:34:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cb6fb4740 10 client.0 _ll_register_callbacks cb 0x556133880da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cae5d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556133585390 0x556133935fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cae5d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556133585390 0x556133935fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60214/0 (socket says 172.21.15.38:60214) 2024-05-21T15:34:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cae5d4640 1 -- 172.21.15.38:0/2032742879 learned_addr learned my addr 172.21.15.38:0/2032742879 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6caddd3640 1 --2- 172.21.15.38:0/2032742879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55613355c4d0 0x55613392eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cad5d2640 1 --2- 172.21.15.38:0/2032742879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561335646b0 0x55613392f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cae5d4640 1 -- 172.21.15.38:0/2032742879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561335646b0 msgr2=0x55613392f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cae5d4640 1 --2- 172.21.15.38:0/2032742879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561335646b0 0x55613392f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cae5d4640 1 -- 172.21.15.38:0/2032742879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55613355c4d0 msgr2=0x55613392eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cae5d4640 1 --2- 172.21.15.38:0/2032742879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55613355c4d0 0x55613392eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cae5d4640 1 -- 172.21.15.38:0/2032742879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55613359d3f0 con 0x556133585390 2024-05-21T15:34:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6caddd3640 1 --2- 172.21.15.38:0/2032742879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55613355c4d0 0x55613392eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:34:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.951+0000 7f6cae5d4640 1 --2- 172.21.15.38:0/2032742879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556133585390 0x556133935fa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6c98010a30 tx=0x7f6c98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed5b2e284516751 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6c8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:43.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6c8d7fa640 1 -- 172.21.15.38:0/2032742879 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98003b70 con 0x556133585390 2024-05-21T15:34:43.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6c8d7fa640 1 -- 172.21.15.38:0/2032742879 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f6c98003d10 con 0x556133585390 2024-05-21T15:34:43.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/2032742879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c9c00be20 con 0x556133585390 2024-05-21T15:34:43.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6c8d7fa640 1 -- 172.21.15.38:0/2032742879 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c9801d6f0 con 0x556133585390 2024-05-21T15:34:43.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6cb6fb4740 10 client.15513 fetch_fsmap learned FSMap version 27 2024-05-21T15:34:43.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/2032742879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55613355d0f0 con 0x556133585390 2024-05-21T15:34:43.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6c8d7fa640 1 -- 172.21.15.38:0/2032742879 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c9c00be20 con 0x556133585390 2024-05-21T15:34:43.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6c8d7fa640 1 -- 172.21.15.38:0/2032742879 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6c98028430 con 0x556133585390 2024-05-21T15:34:43.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6cb6fb4740 10 client.15513 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:34:43.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6cb6fb4740 10 client.15513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:34:43.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6cb6fb4740 20 client.15513 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:43.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/2032742879 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556133543f60 0x55613393c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6cb6fb4740 4 client.15513 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:34:43.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.952+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/2032742879 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556133897430 con 0x556133543f60 2024-05-21T15:34:43.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.953+0000 7f6caddd3640 1 --2- 172.21.15.38:0/2032742879 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556133543f60 0x55613393c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.953+0000 7f6c8d7fa640 10 client.15513 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:43.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.953+0000 7f6caddd3640 1 --2- 172.21.15.38:0/2032742879 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556133543f60 0x55613393c910 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15df97debd595b8d server_cookie=838a1515aeb5e7f in_seq=0 out_seq=0 2024-05-21T15:34:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6c8d7fa640 1 -- 172.21.15.38:0/2032742879 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556133897430 con 0x556133543f60 2024-05-21T15:34:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6c8d7fa640 10 client.15513 handle_command_reply: tid=0 2024-05-21T15:34:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6cb6fb4740 1 client.15513 shutdown 2024-05-21T15:34:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/2032742879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556133585390 msgr2=0x556133935fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/2032742879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556133585390 0x556133935fa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6c98010a30 tx=0x7f6c98004a40 comp rx=0 tx=0).stop 2024-05-21T15:34:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/2032742879 shutdown_connections 2024-05-21T15:34:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/2032742879 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556133543f60 0x55613393c910 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/2032742879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556133585390 0x556133935fa0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/2032742879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561335646b0 0x55613392f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6c8d7fa640 0 client.15513 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/2032742879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55613355c4d0 0x55613392eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/2032742879 >> 172.21.15.38:0/2032742879 conn(0x5561335794e0 msgr2=0x55613391f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/2032742879 shutdown_connections 2024-05-21T15:34:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/2032742879 wait complete. 2024-05-21T15:34:43.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.954+0000 7f6cb6fb4740 20 client.15513 trim_cache size 0 max 16384 2024-05-21T15:34:43.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.972+0000 7f6cb6fb4740 1 Processor -- start 2024-05-21T15:34:43.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.972+0000 7f6cb6fb4740 1 -- start start 2024-05-21T15:34:43.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.972+0000 7f6cb6fb4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 0x556133929540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.972+0000 7f6cb6fb4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561335646b0 0x556133935ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.972+0000 7f6cb6fb4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556133585390 0x55613392d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.972+0000 7f6cb6fb4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81aa370 con 0x556133585390 2024-05-21T15:34:43.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.972+0000 7f6cb6fb4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81abeb0 con 0x5561335646b0 2024-05-21T15:34:43.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.972+0000 7f6cb6fb4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81aaa40 con 0x55613355c4d0 2024-05-21T15:34:43.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.973+0000 7f6cae5d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556133585390 0x55613392d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.973+0000 7f6cae5d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556133585390 0x55613392d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60218/0 (socket says 172.21.15.38:60218) 2024-05-21T15:34:43.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.973+0000 7f6cae5d4640 1 -- 172.21.15.38:0/1010178933 learned_addr learned my addr 172.21.15.38:0/1010178933 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:43.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.973+0000 7f6cad5d2640 1 --2- 172.21.15.38:0/1010178933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561335646b0 0x556133935ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.973+0000 7f6caddd3640 1 --2- 172.21.15.38:0/1010178933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 0x556133929540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.973+0000 7f6cae5d4640 1 -- 172.21.15.38:0/1010178933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 msgr2=0x556133929540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.973+0000 7f6cae5d4640 1 --2- 172.21.15.38:0/1010178933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 0x556133929540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.973+0000 7f6cae5d4640 1 -- 172.21.15.38:0/1010178933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561335646b0 msgr2=0x556133935ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.973+0000 7f6cae5d4640 1 --2- 172.21.15.38:0/1010178933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561335646b0 0x556133935ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.973+0000 7f6cae5d4640 1 -- 172.21.15.38:0/1010178933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556133927fa0 con 0x556133585390 2024-05-21T15:34:43.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.973+0000 7f6cae5d4640 1 --2- 172.21.15.38:0/1010178933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556133585390 0x55613392d860 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6c98012e30 tx=0x7f6c98010fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6451499e98680da server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.973+0000 7f6c8f7fe640 1 -- 172.21.15.38:0/1010178933 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98028630 con 0x556133585390 2024-05-21T15:34:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.973+0000 7f6c8f7fe640 1 -- 172.21.15.38:0/1010178933 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c98003e50 con 0x556133585390 2024-05-21T15:34:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.973+0000 7f6c8f7fe640 1 -- 172.21.15.38:0/1010178933 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98004040 con 0x556133585390 2024-05-21T15:34:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.974+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/1010178933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556133585390 msgr2=0x55613392d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.974+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/1010178933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556133585390 0x55613392d860 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6c98012e30 tx=0x7f6c98010fb0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.974+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/1010178933 shutdown_connections 2024-05-21T15:34:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.974+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/1010178933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556133585390 0x55613392d860 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.974+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/1010178933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561335646b0 0x556133935ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.974+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/1010178933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 0x556133929540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.974+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/1010178933 >> 172.21.15.38:0/1010178933 conn(0x5561335794e0 msgr2=0x55613392d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.974+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/1010178933 shutdown_connections 2024-05-21T15:34:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.974+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/1010178933 wait complete. 2024-05-21T15:34:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.974+0000 7f6cb6fb4740 1 -- start start 2024-05-21T15:34:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cb6fb4740 1 Processor -- start 2024-05-21T15:34:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cb6fb4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 0x5561339331a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cb6fb4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561335646b0 0x5561339336e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cb6fb4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556133585390 0x5561339345c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cb6fb4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81ab7e0 con 0x5561335646b0 2024-05-21T15:34:43.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cb6fb4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81ab110 con 0x556133585390 2024-05-21T15:34:43.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cb6fb4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81a8350 con 0x55613355c4d0 2024-05-21T15:34:43.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cb6fb4740 10 client.0 _ll_register_callbacks cb 0x7f6c98009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:43.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cad5d2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561335646b0 0x5561339336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cad5d2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561335646b0 0x5561339336e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60224/0 (socket says 172.21.15.38:60224) 2024-05-21T15:34:43.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cad5d2640 1 -- 172.21.15.38:0/1310969951 learned_addr learned my addr 172.21.15.38:0/1310969951 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:43.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6caddd3640 1 --2- 172.21.15.38:0/1310969951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 0x5561339331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cae5d4640 1 --2- 172.21.15.38:0/1310969951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556133585390 0x5561339345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cad5d2640 1 -- 172.21.15.38:0/1310969951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 msgr2=0x5561339331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cad5d2640 1 --2- 172.21.15.38:0/1310969951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 0x5561339331a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.975+0000 7f6cad5d2640 1 -- 172.21.15.38:0/1310969951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556133585390 msgr2=0x5561339345c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6cad5d2640 1 --2- 172.21.15.38:0/1310969951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556133585390 0x5561339345c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6cad5d2640 1 -- 172.21.15.38:0/1310969951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556133936400 con 0x5561335646b0 2024-05-21T15:34:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6cae5d4640 1 --2- 172.21.15.38:0/1310969951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556133585390 0x5561339345c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:34:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6cad5d2640 1 --2- 172.21.15.38:0/1310969951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561335646b0 0x5561339336e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00bd30 tx=0x7f6c9c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0208a303021d770 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6c8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6c8d7fa640 1 -- 172.21.15.38:0/1310969951 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c015070 con 0x5561335646b0 2024-05-21T15:34:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6c8d7fa640 1 -- 172.21.15.38:0/1310969951 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f6c9c004680 con 0x5561335646b0 2024-05-21T15:34:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/1310969951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c98003e50 con 0x5561335646b0 2024-05-21T15:34:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6c8d7fa640 1 -- 172.21.15.38:0/1310969951 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c9c004880 con 0x5561335646b0 2024-05-21T15:34:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6cb6fb4740 10 client.15519 fetch_fsmap learned FSMap version 27 2024-05-21T15:34:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/1310969951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556133935c70 con 0x5561335646b0 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6c8d7fa640 1 -- 172.21.15.38:0/1310969951 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c98003e50 con 0x5561335646b0 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6c8d7fa640 1 -- 172.21.15.38:0/1310969951 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6c9c026ba0 con 0x5561335646b0 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6cb6fb4740 10 client.15519 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6cb6fb4740 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6cb6fb4740 20 client.15519 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/1310969951 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5561335193a0 0x5561339a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6cb6fb4740 4 client.15519 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.976+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/1310969951 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556133891780 con 0x5561335193a0 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.977+0000 7f6caddd3640 1 --2- 172.21.15.38:0/1310969951 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5561335193a0 0x5561339a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.977+0000 7f6caddd3640 1 --2- 172.21.15.38:0/1310969951 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5561335193a0 0x5561339a6cd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53691093dfdbe2f0 server_cookie=2bdc4521e50d121e in_seq=0 out_seq=0 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.977+0000 7f6c8d7fa640 10 client.15519 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.977+0000 7f6c8d7fa640 1 -- 172.21.15.38:0/1310969951 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556133891780 con 0x5561335193a0 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.977+0000 7f6c8d7fa640 10 client.15519 handle_command_reply: tid=0 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.977+0000 7f6cb6fb4740 1 client.15519 shutdown 2024-05-21T15:34:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.977+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/1310969951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561335646b0 msgr2=0x5561339336e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.977+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/1310969951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561335646b0 0x5561339336e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00bd30 tx=0x7f6c9c005270 comp rx=0 tx=0).stop 2024-05-21T15:34:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/1310969951 shutdown_connections 2024-05-21T15:34:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/1310969951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556133585390 0x5561339345c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/1310969951 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5561335193a0 0x5561339a6cd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/1310969951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5561335646b0 0x5561339336e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb6fb4740 1 --2- 172.21.15.38:0/1310969951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55613355c4d0 0x5561339331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6c8d7fa640 0 client.15519 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/1310969951 >> 172.21.15.38:0/1310969951 conn(0x5561335794e0 msgr2=0x55613391eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/1310969951 shutdown_connections 2024-05-21T15:34:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb6fb4740 1 -- 172.21.15.38:0/1310969951 wait complete. 2024-05-21T15:34:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb6fb4740 20 client.15519 trim_cache size 0 max 16384 2024-05-21T15:34:43.985 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:34:43.986 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:34:43.986 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-05-21T15:34:43.986 INFO:teuthology.orchestra.run.smithi038.stdout: "8f981338-957a-40e5-b340-68b75df02100": { 2024-05-21T15:34:43.986 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-05-21T15:34:43.986 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "8f981338-957a-40e5-b340-68b75df02100", 2024-05-21T15:34:43.986 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-05-21T15:34:43.986 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:34:43.986 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:34:43.986 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:34:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb0037640 1 -- 172.21.15.38:0/1204584650 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6c900a2c10 msgr2=0x7f6c900a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb0037640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6c900a2c10 0x7f6c900a5120 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x556133925d80 tx=0x7f6c98009500 comp rx=0 tx=0).stop 2024-05-21T15:34:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb0037640 1 -- 172.21.15.38:0/1204584650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ca8101990 msgr2=0x7f6ca80747f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb0037640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ca8101990 0x7f6ca80747f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0006010 tx=0x7f6ca0031b60 comp rx=0 tx=0).stop 2024-05-21T15:34:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb0037640 1 -- 172.21.15.38:0/1204584650 shutdown_connections 2024-05-21T15:34:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb0037640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ca81078d0 0x7f6ca81a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb0037640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ca8102bd0 0x7f6ca819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb0037640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6c900a2c10 0x7f6c900a5120 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb0037640 1 --2- 172.21.15.38:0/1204584650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ca8101990 0x7f6ca80747f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb0037640 1 -- 172.21.15.38:0/1204584650 >> 172.21.15.38:0/1204584650 conn(0x7f6ca80fcb80 msgr2=0x7f6ca81038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb0037640 1 -- 172.21.15.38:0/1204584650 shutdown_connections 2024-05-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:34:43.978+0000 7f6cb0037640 1 -- 172.21.15.38:0/1204584650 wait complete. 2024-05-21T15:34:43.991 INFO:tasks.cephfs.filesystem:scrub status for tag:8f981338-957a-40e5-b340-68b75df02100 - {'path': '/', 'tag': '8f981338-957a-40e5-b340-68b75df02100', 'options': 'recursive,force'} 2024-05-21T15:34:44.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:44 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4153065470' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:44.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:44 smithi053 ceph-mon[36496]: scrub queued for path: / 2024-05-21T15:34:44.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:44 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:34:44.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:44 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:34:44.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:44 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:34:44.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:44 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:34:44.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:44 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4153065470' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:44.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:44 smithi045 ceph-mon[36409]: scrub queued for path: / 2024-05-21T15:34:44.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:44 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:34:44.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:44 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:34:44.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:44 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:34:44.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:44 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:34:44.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:44 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4153065470' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:34:44.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:44 smithi038 ceph-mon[31995]: scrub queued for path: / 2024-05-21T15:34:44.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:44 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:34:44.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:44 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:34:44.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:44 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:34:44.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:44 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:34:45.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:45 smithi053 ceph-mon[36496]: pgmap v361: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-21T15:34:45.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:45 smithi045 ceph-mon[36409]: pgmap v361: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-21T15:34:45.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:45 smithi038 ceph-mon[31995]: pgmap v361: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-21T15:34:47.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:47 smithi053 ceph-mon[36496]: pgmap v362: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-21T15:34:47.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:47 smithi045 ceph-mon[36409]: pgmap v362: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-21T15:34:47.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:47 smithi038 ceph-mon[31995]: pgmap v362: 97 pgs: 97 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-21T15:34:49.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:49 smithi038 ceph-mon[31995]: pgmap v363: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-21T15:34:49.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:49 smithi053 ceph-mon[36496]: pgmap v363: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-21T15:34:49.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:49 smithi045 ceph-mon[36409]: pgmap v363: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-21T15:34:51.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:51 smithi053 ceph-mon[36496]: pgmap v364: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-21T15:34:51.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:51 smithi045 ceph-mon[36409]: pgmap v364: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-21T15:34:51.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:51 smithi038 ceph-mon[31995]: pgmap v364: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-21T15:34:53.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:53 smithi053 ceph-mon[36496]: pgmap v365: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-21T15:34:53.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:53 smithi045 ceph-mon[36409]: pgmap v365: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-21T15:34:53.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:53 smithi038 ceph-mon[31995]: pgmap v365: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-21T15:34:54.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:54 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:34:54.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:54 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:34:54.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:54 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:34:55.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:55 smithi053 ceph-mon[36496]: pgmap v366: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-21T15:34:55.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:55 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:55 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:55 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:55 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:55 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.706 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:55 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:55 smithi045 ceph-mon[36409]: pgmap v366: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-21T15:34:55.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:55 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:55 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:55 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:55 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:55 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:55 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:55 smithi038 ceph-mon[31995]: pgmap v366: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-21T15:34:55.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:55 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:55 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:55 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:55 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:55 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:55.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:55 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:56.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:34:56.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:34:56.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:56.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:34:56.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:34:56.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.770+0000 7ff4b566c640 1 -- 172.21.15.38:0/768137910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b0102c30 msgr2=0x7ff4b0103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.770+0000 7ff4b566c640 1 --2- 172.21.15.38:0/768137910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b0102c30 0x7ff4b0103040 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009900 tx=0x7ff4a402a410 comp rx=0 tx=0).stop 2024-05-21T15:34:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.770+0000 7ff4b566c640 1 -- 172.21.15.38:0/768137910 shutdown_connections 2024-05-21T15:34:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.770+0000 7ff4b566c640 1 --2- 172.21.15.38:0/768137910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4b0107930 0x7ff4b0109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.770+0000 7ff4b566c640 1 --2- 172.21.15.38:0/768137910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b0102c30 0x7ff4b0103040 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.770+0000 7ff4b566c640 1 --2- 172.21.15.38:0/768137910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b01019f0 0x7ff4b01026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.770+0000 7ff4b566c640 1 -- 172.21.15.38:0/768137910 >> 172.21.15.38:0/768137910 conn(0x7ff4b00fcbe0 msgr2=0x7ff4b00ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.770+0000 7ff4b566c640 1 -- 172.21.15.38:0/768137910 shutdown_connections 2024-05-21T15:34:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.771+0000 7ff4b566c640 1 -- 172.21.15.38:0/768137910 wait complete. 2024-05-21T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.771+0000 7ff4b566c640 1 Processor -- start 2024-05-21T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.771+0000 7ff4b566c640 1 -- start start 2024-05-21T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4b566c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4b01019f0 0x7ff4b0199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4b566c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b0102c30 0x7ff4b0199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4b566c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b0107930 0x7ff4b01a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4b566c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b010d640 con 0x7ff4b0107930 2024-05-21T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4b566c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b010d4c0 con 0x7ff4b01019f0 2024-05-21T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4b566c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b010d7c0 con 0x7ff4b0102c30 2024-05-21T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b0107930 0x7ff4b01a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b0107930 0x7ff4b01a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39896/0 (socket says 172.21.15.38:39896) 2024-05-21T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4af7fe640 1 -- 172.21.15.38:0/3669949636 learned_addr learned my addr 172.21.15.38:0/3669949636 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4aeffd640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4b01019f0 0x7ff4b0199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b0102c30 0x7ff4b0199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4af7fe640 1 -- 172.21.15.38:0/3669949636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b0102c30 msgr2=0x7ff4b0199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4af7fe640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b0102c30 0x7ff4b0199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4af7fe640 1 -- 172.21.15.38:0/3669949636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4b01019f0 msgr2=0x7ff4b0199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4af7fe640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4b01019f0 0x7ff4b0199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4af7fe640 1 -- 172.21.15.38:0/3669949636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b01a0d70 con 0x7ff4b0107930 2024-05-21T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4aeffd640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4b01019f0 0x7ff4b0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.772+0000 7ff4af7fe640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b0107930 0x7ff4b01a0700 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800d750 tx=0x7ff4a800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.773+0000 7ff497fff640 1 -- 172.21.15.38:0/3669949636 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8017070 con 0x7ff4b0107930 2024-05-21T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.773+0000 7ff497fff640 1 -- 172.21.15.38:0/3669949636 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a8005500 con 0x7ff4b0107930 2024-05-21T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.773+0000 7ff4b566c640 1 -- 172.21.15.38:0/3669949636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b01a1060 con 0x7ff4b0107930 2024-05-21T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.773+0000 7ff497fff640 1 -- 172.21.15.38:0/3669949636 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8004d90 con 0x7ff4b0107930 2024-05-21T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.773+0000 7ff4b566c640 1 -- 172.21.15.38:0/3669949636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b01a1560 con 0x7ff4b0107930 2024-05-21T15:34:56.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:34:56.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:34:56.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.775+0000 7ff497fff640 1 -- 172.21.15.38:0/3669949636 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff4a8004f30 con 0x7ff4b0107930 2024-05-21T15:34:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff4bc5ef740 1 Processor -- start 2024-05-21T15:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff4bc5ef740 1 -- start start 2024-05-21T15:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff4bc5ef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf98698270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff4bc5ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982cb2f0 0x55bf986987f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff4bc5ef740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982d34d0 0x55bf9869f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff4bc5ef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bf985f8340 con 0x55bf982f41b0 2024-05-21T15:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff4bc5ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bf985a0290 con 0x55bf982cb2f0 2024-05-21T15:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff4bc5ef740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bf985f4300 con 0x55bf982d34d0 2024-05-21T15:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff4ae7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf98698270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff4ae7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf98698270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39912/0 (socket says 172.21.15.38:39912) 2024-05-21T15:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/23210206 learned_addr learned my addr 172.21.15.38:0/23210206 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff4aeffd640 1 --2- 172.21.15.38:0/23210206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982cb2f0 0x55bf986987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff497fff640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff4980a2c30 0x7ff4980a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/23210206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982d34d0 0x55bf9869f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.776+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/23210206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982d34d0 msgr2=0x55bf9869f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.777+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/23210206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982d34d0 0x55bf9869f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.777+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/23210206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982cb2f0 msgr2=0x55bf986987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.777+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/23210206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982cb2f0 0x55bf986987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.777+0000 7ff4af7fe640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff4980a2c30 0x7ff4980a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.777+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/23210206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf985c48e0 con 0x55bf982f41b0 2024-05-21T15:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.777+0000 7ff497fff640 1 -- 172.21.15.38:0/3669949636 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff4a80ceab0 con 0x7ff4b0107930 2024-05-21T15:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.777+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/23210206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf98698270 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff4a402a930 tx=0x7ff4a4005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e5f3ad542e0912 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.777+0000 7ff4957fa640 1 -- 172.21.15.38:0/23210206 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4035050 con 0x55bf982f41b0 2024-05-21T15:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.777+0000 7ff4957fa640 1 -- 172.21.15.38:0/23210206 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a4031850 con 0x55bf982f41b0 2024-05-21T15:34:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.777+0000 7ff4957fa640 1 -- 172.21.15.38:0/23210206 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40046c0 con 0x55bf982f41b0 2024-05-21T15:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.778+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/23210206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 msgr2=0x55bf98698270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.778+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/23210206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf98698270 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff4a402a930 tx=0x7ff4a4005d70 comp rx=0 tx=0).stop 2024-05-21T15:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.778+0000 7ff4af7fe640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff4980a2c30 0x7ff4980a5140 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8097720 tx=0x7ff4a8065e90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.778+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/23210206 shutdown_connections 2024-05-21T15:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.778+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/23210206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982d34d0 0x55bf9869f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.778+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/23210206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982cb2f0 0x55bf986987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.778+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/23210206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf98698270 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.778+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/23210206 >> 172.21.15.38:0/23210206 conn(0x55bf982e8300 msgr2=0x55bf98693c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.778+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/23210206 shutdown_connections 2024-05-21T15:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.778+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/23210206 wait complete. 2024-05-21T15:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.778+0000 7ff4bc5ef740 1 -- start start 2024-05-21T15:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.778+0000 7ff4bc5ef740 1 Processor -- start 2024-05-21T15:34:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4bc5ef740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982cb2f0 0x55bf986a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4bc5ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982d34d0 0x55bf986a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4bc5ef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf986a9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4bc5ef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bf986a00e0 con 0x55bf982f41b0 2024-05-21T15:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4bc5ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bf985b3790 con 0x55bf982d34d0 2024-05-21T15:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4bc5ef740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bf986a0260 con 0x55bf982cb2f0 2024-05-21T15:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4bc5ef740 10 client.0 _ll_register_callbacks cb 0x55bf985eea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4ae7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf986a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4ae7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf986a9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39922/0 (socket says 172.21.15.38:39922) 2024-05-21T15:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/917486133 learned_addr learned my addr 172.21.15.38:0/917486133 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4aeffd640 1 --2- 172.21.15.38:0/917486133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982d34d0 0x55bf986a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/917486133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982cb2f0 0x55bf986a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/917486133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982cb2f0 msgr2=0x55bf986a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/917486133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982cb2f0 0x55bf986a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/917486133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982d34d0 msgr2=0x55bf986a24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/917486133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982d34d0 0x55bf986a24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/917486133 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf9830c210 con 0x55bf982f41b0 2024-05-21T15:34:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/917486133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf986a9420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4031a80 tx=0x7ff4a403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ef0bb3ca16e698 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.779+0000 7ff492ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.780+0000 7ff492ff5640 1 -- 172.21.15.38:0/917486133 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4051060 con 0x55bf982f41b0 2024-05-21T15:34:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.780+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/917486133 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a4031850 con 0x55bf982f41b0 2024-05-21T15:34:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.780+0000 7ff492ff5640 1 -- 172.21.15.38:0/917486133 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff4a4005530 con 0x55bf982f41b0 2024-05-21T15:34:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.780+0000 7ff492ff5640 1 -- 172.21.15.38:0/917486133 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4a4035070 con 0x55bf982f41b0 2024-05-21T15:34:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.780+0000 7ff4bc5ef740 10 client.15531 fetch_fsmap learned FSMap version 27 2024-05-21T15:34:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.780+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/917486133 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf983d8620 con 0x55bf982f41b0 2024-05-21T15:34:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.780+0000 7ff492ff5640 1 -- 172.21.15.38:0/917486133 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a4031850 con 0x55bf982f41b0 2024-05-21T15:34:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.780+0000 7ff492ff5640 1 -- 172.21.15.38:0/917486133 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff4a40038c0 con 0x55bf982f41b0 2024-05-21T15:34:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.780+0000 7ff4bc5ef740 10 client.15531 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:34:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.780+0000 7ff4bc5ef740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:34:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.780+0000 7ff4bc5ef740 20 client.15531 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.780+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/917486133 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf982b2d80 0x55bf986af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.780+0000 7ff4bc5ef740 4 client.15531 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:34:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.780+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/917486133 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf98605a70 con 0x55bf982b2d80 2024-05-21T15:34:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.781+0000 7ff4aeffd640 1 --2- 172.21.15.38:0/917486133 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf982b2d80 0x55bf986af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.781+0000 7ff492ff5640 10 client.15531 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.781+0000 7ff4aeffd640 1 --2- 172.21.15.38:0/917486133 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf982b2d80 0x55bf986af4e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82e1c9b65f7ec267 server_cookie=d95342a3ed2e9ed1 in_seq=0 out_seq=0 2024-05-21T15:34:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff492ff5640 1 -- 172.21.15.38:0/917486133 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bf98605a70 con 0x55bf982b2d80 2024-05-21T15:34:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff492ff5640 10 client.15531 handle_command_reply: tid=0 2024-05-21T15:34:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff4bc5ef740 1 client.15531 shutdown 2024-05-21T15:34:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/917486133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 msgr2=0x55bf986a9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/917486133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf986a9420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4031a80 tx=0x7ff4a403f040 comp rx=0 tx=0).stop 2024-05-21T15:34:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/917486133 shutdown_connections 2024-05-21T15:34:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/917486133 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf982b2d80 0x55bf986af4e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/917486133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf986a9420 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/917486133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982d34d0 0x55bf986a24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/917486133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982cb2f0 0x55bf986a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff492ff5640 0 client.15531 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/917486133 >> 172.21.15.38:0/917486133 conn(0x55bf982e8300 msgr2=0x55bf98696bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/917486133 shutdown_connections 2024-05-21T15:34:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/917486133 wait complete. 2024-05-21T15:34:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.782+0000 7ff4bc5ef740 20 client.15531 trim_cache size 0 max 16384 2024-05-21T15:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4bc5ef740 1 Processor -- start 2024-05-21T15:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4bc5ef740 1 -- start start 2024-05-21T15:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4bc5ef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982cb2f0 0x55bf98693c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4bc5ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982d34d0 0x55bf986a9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4bc5ef740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982f41b0 0x55bf986a9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4bc5ef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01a8080 con 0x55bf982cb2f0 2024-05-21T15:34:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4bc5ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01a9bc0 con 0x55bf982d34d0 2024-05-21T15:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4bc5ef740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01a8750 con 0x55bf982f41b0 2024-05-21T15:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4ae7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982cb2f0 0x55bf98693c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4ae7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982cb2f0 0x55bf98693c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39932/0 (socket says 172.21.15.38:39932) 2024-05-21T15:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/3438783122 learned_addr learned my addr 172.21.15.38:0/3438783122 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4aeffd640 1 --2- 172.21.15.38:0/3438783122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982d34d0 0x55bf986a9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/3438783122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982f41b0 msgr2=0x55bf986a9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/3438783122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982f41b0 0x55bf986a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/3438783122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982d34d0 msgr2=0x55bf986a9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/3438783122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982d34d0 0x55bf986a9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/3438783122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf98697e20 con 0x55bf982cb2f0 2024-05-21T15:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.798+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/3438783122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982cb2f0 0x55bf98693c40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff4a403a610 tx=0x7ff4a4005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcdf211d468fa8b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.799+0000 7ff494ff9640 1 -- 172.21.15.38:0/3438783122 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4051040 con 0x55bf982cb2f0 2024-05-21T15:34:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.799+0000 7ff494ff9640 1 -- 172.21.15.38:0/3438783122 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a4004df0 con 0x55bf982cb2f0 2024-05-21T15:34:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.799+0000 7ff494ff9640 1 -- 172.21.15.38:0/3438783122 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4003910 con 0x55bf982cb2f0 2024-05-21T15:34:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.799+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/3438783122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982cb2f0 msgr2=0x55bf98693c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.799+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/3438783122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982cb2f0 0x55bf98693c40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff4a403a610 tx=0x7ff4a4005d70 comp rx=0 tx=0).stop 2024-05-21T15:34:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.799+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/3438783122 shutdown_connections 2024-05-21T15:34:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.799+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/3438783122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982f41b0 0x55bf986a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.799+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/3438783122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982d34d0 0x55bf986a9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.799+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/3438783122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982cb2f0 0x55bf98693c40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.799+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/3438783122 >> 172.21.15.38:0/3438783122 conn(0x55bf982e8300 msgr2=0x55bf98693530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/3438783122 shutdown_connections 2024-05-21T15:34:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/3438783122 wait complete. 2024-05-21T15:34:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4bc5ef740 1 -- start start 2024-05-21T15:34:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4bc5ef740 1 Processor -- start 2024-05-21T15:34:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4bc5ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982cb2f0 0x55bf986a7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4bc5ef740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982d34d0 0x55bf986a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4bc5ef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf986a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4bc5ef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01a94f0 con 0x55bf982f41b0 2024-05-21T15:34:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4bc5ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01a8e20 con 0x55bf982cb2f0 2024-05-21T15:34:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4bc5ef740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01a6060 con 0x55bf982d34d0 2024-05-21T15:34:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4bc5ef740 10 client.0 _ll_register_callbacks cb 0x7ff4a00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:34:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4ae7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf986a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4ae7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf986a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39940/0 (socket says 172.21.15.38:39940) 2024-05-21T15:34:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/3941128489 learned_addr learned my addr 172.21.15.38:0/3941128489 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:34:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/3941128489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982cb2f0 0x55bf986a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.800+0000 7ff4aeffd640 1 --2- 172.21.15.38:0/3941128489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982d34d0 0x55bf986a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.801+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/3941128489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982d34d0 msgr2=0x55bf986a75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.801+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/3941128489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982d34d0 0x55bf986a75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.801+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/3941128489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982cb2f0 msgr2=0x55bf986a7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:34:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.801+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/3941128489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982cb2f0 0x55bf986a7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.801+0000 7ff4ae7fc640 1 -- 172.21.15.38:0/3941128489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf986a8a90 con 0x55bf982f41b0 2024-05-21T15:34:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.801+0000 7ff4ae7fc640 1 --2- 172.21.15.38:0/3941128489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf986a84d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff4a404f040 tx=0x7ff4a403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff817a0a9ce4c067 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:34:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.801+0000 7ff492ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:34:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.801+0000 7ff492ff5640 1 -- 172.21.15.38:0/3941128489 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4051070 con 0x55bf982f41b0 2024-05-21T15:34:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.801+0000 7ff492ff5640 1 -- 172.21.15.38:0/3941128489 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff4a4005460 con 0x55bf982f41b0 2024-05-21T15:34:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.801+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/3941128489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a4004df0 con 0x55bf982f41b0 2024-05-21T15:34:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.801+0000 7ff492ff5640 1 -- 172.21.15.38:0/3941128489 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4a4031a80 con 0x55bf982f41b0 2024-05-21T15:34:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.801+0000 7ff4bc5ef740 10 client.15537 fetch_fsmap learned FSMap version 27 2024-05-21T15:34:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.801+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/3941128489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf986a9d20 con 0x55bf982f41b0 2024-05-21T15:34:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.801+0000 7ff492ff5640 1 -- 172.21.15.38:0/3941128489 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a4004df0 con 0x55bf982f41b0 2024-05-21T15:34:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.802+0000 7ff492ff5640 1 -- 172.21.15.38:0/3941128489 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff4a40041c0 con 0x55bf982f41b0 2024-05-21T15:34:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.802+0000 7ff4bc5ef740 10 client.15537 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:34:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.802+0000 7ff4bc5ef740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:34:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.802+0000 7ff4bc5ef740 20 client.15537 populate_metadata read hostname 'smithi038' 2024-05-21T15:34:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.802+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/3941128489 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf982881c0 0x55bf98719d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:34:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.802+0000 7ff4bc5ef740 4 client.15537 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:34:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.802+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/3941128489 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf985ffdc0 con 0x55bf982881c0 2024-05-21T15:34:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.802+0000 7ff4aeffd640 1 --2- 172.21.15.38:0/3941128489 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf982881c0 0x55bf98719d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:34:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.802+0000 7ff4aeffd640 1 --2- 172.21.15.38:0/3941128489 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf982881c0 0x55bf98719d60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=272ef6565ba1e259 server_cookie=936259bdee5c8da5 in_seq=0 out_seq=0 2024-05-21T15:34:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.802+0000 7ff492ff5640 10 client.15537 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.816+0000 7ff492ff5640 1 -- 172.21.15.38:0/3941128489 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf985ffdc0 con 0x55bf982881c0 2024-05-21T15:34:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.816+0000 7ff492ff5640 10 client.15537 handle_command_reply: tid=0 2024-05-21T15:34:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.816+0000 7ff4bc5ef740 1 client.15537 shutdown 2024-05-21T15:34:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/3941128489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 msgr2=0x55bf986a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/3941128489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf986a84d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff4a404f040 tx=0x7ff4a403f040 comp rx=0 tx=0).stop 2024-05-21T15:34:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/3941128489 shutdown_connections 2024-05-21T15:34:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/3941128489 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf982881c0 0x55bf98719d60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/3941128489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf982f41b0 0x55bf986a84d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/3941128489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf982d34d0 0x55bf986a75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff492ff5640 0 client.15537 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:34:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff4bc5ef740 1 --2- 172.21.15.38:0/3941128489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf982cb2f0 0x55bf986a7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/3941128489 >> 172.21.15.38:0/3941128489 conn(0x55bf982e8300 msgr2=0x55bf9868dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/3941128489 shutdown_connections 2024-05-21T15:34:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff4bc5ef740 1 -- 172.21.15.38:0/3941128489 wait complete. 2024-05-21T15:34:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff4bc5ef740 20 client.15537 trim_cache size 0 max 16384 2024-05-21T15:34:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:34:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:34:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:34:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:34:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff4b566c640 1 -- 172.21.15.38:0/3669949636 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff4980a2c30 msgr2=0x7ff4980a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff4b566c640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff4980a2c30 0x7ff4980a5140 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8097720 tx=0x7ff4a8065e90 comp rx=0 tx=0).stop 2024-05-21T15:34:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.817+0000 7ff4b566c640 1 -- 172.21.15.38:0/3669949636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b0107930 msgr2=0x7ff4b01a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:34:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.818+0000 7ff4b566c640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b0107930 0x7ff4b01a0700 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800d750 tx=0x7ff4a800dc10 comp rx=0 tx=0).stop 2024-05-21T15:34:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.818+0000 7ff4b566c640 1 -- 172.21.15.38:0/3669949636 shutdown_connections 2024-05-21T15:34:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.818+0000 7ff4b566c640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff4980a2c30 0x7ff4980a5140 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.818+0000 7ff4b566c640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4b0107930 0x7ff4b01a0700 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.818+0000 7ff4b566c640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b0102c30 0x7ff4b0199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.818+0000 7ff4b566c640 1 --2- 172.21.15.38:0/3669949636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4b01019f0 0x7ff4b0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:34:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.818+0000 7ff4b566c640 1 -- 172.21.15.38:0/3669949636 >> 172.21.15.38:0/3669949636 conn(0x7ff4b00fcbe0 msgr2=0x7ff4b006a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:34:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.818+0000 7ff4b566c640 1 -- 172.21.15.38:0/3669949636 shutdown_connections 2024-05-21T15:34:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:34:56.818+0000 7ff4b566c640 1 -- 172.21.15.38:0/3669949636 wait complete. 2024-05-21T15:34:58.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:57 smithi053 ceph-mon[36496]: pgmap v367: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-21T15:34:58.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:57 smithi045 ceph-mon[36409]: pgmap v367: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-21T15:34:58.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:57 smithi038 ceph-mon[31995]: pgmap v367: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-21T15:35:00.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:34:59 smithi053 ceph-mon[36496]: pgmap v368: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-05-21T15:35:00.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:34:59 smithi045 ceph-mon[36409]: pgmap v368: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-05-21T15:35:00.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:34:59 smithi038 ceph-mon[31995]: pgmap v368: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-05-21T15:35:02.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:01 smithi053 ceph-mon[36496]: pgmap v369: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-05-21T15:35:02.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:01 smithi045 ceph-mon[36409]: pgmap v369: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-05-21T15:35:02.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:01 smithi038 ceph-mon[31995]: pgmap v369: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-05-21T15:35:04.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:03 smithi053 ceph-mon[36496]: pgmap v370: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-21T15:35:04.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:03 smithi045 ceph-mon[36409]: pgmap v370: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-21T15:35:04.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:03 smithi038 ceph-mon[31995]: pgmap v370: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-21T15:35:06.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:05 smithi053 ceph-mon[36496]: pgmap v371: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-05-21T15:35:06.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:05 smithi045 ceph-mon[36409]: pgmap v371: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-05-21T15:35:06.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:05 smithi038 ceph-mon[31995]: pgmap v371: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-05-21T15:35:08.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:07 smithi053 ceph-mon[36496]: pgmap v372: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-05-21T15:35:08.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:07 smithi045 ceph-mon[36409]: pgmap v372: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-05-21T15:35:08.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:07 smithi038 ceph-mon[31995]: pgmap v372: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-05-21T15:35:09.934 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.932+0000 7f8f3d8bc640 1 -- 172.21.15.38:0/1288384940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38108870 msgr2=0x7f8f3810ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:09.934 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.932+0000 7f8f3d8bc640 1 --2- 172.21.15.38:0/1288384940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38108870 0x7f8f3810ac90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009ca0 tx=0x7f8f2802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.934 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.932+0000 7f8f3d8bc640 1 -- 172.21.15.38:0/1288384940 shutdown_connections 2024-05-21T15:35:09.934 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.932+0000 7f8f3d8bc640 1 --2- 172.21.15.38:0/1288384940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38108870 0x7f8f3810ac90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.934 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.932+0000 7f8f3d8bc640 1 --2- 172.21.15.38:0/1288384940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f38105f10 0x7f8f38108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.934 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.932+0000 7f8f3d8bc640 1 --2- 172.21.15.38:0/1288384940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f381035b0 0x7f8f381059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.934 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.932+0000 7f8f3d8bc640 1 -- 172.21.15.38:0/1288384940 >> 172.21.15.38:0/1288384940 conn(0x7f8f380fcbb0 msgr2=0x7f8f380ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:09.934 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.932+0000 7f8f3d8bc640 1 -- 172.21.15.38:0/1288384940 shutdown_connections 2024-05-21T15:35:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.932+0000 7f8f3d8bc640 1 -- 172.21.15.38:0/1288384940 wait complete. 2024-05-21T15:35:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.933+0000 7f8f3d8bc640 1 Processor -- start 2024-05-21T15:35:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.933+0000 7f8f3d8bc640 1 -- start start 2024-05-21T15:35:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.933+0000 7f8f3d8bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f381035b0 0x7f8f381991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.933+0000 7f8f3d8bc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f38105f10 0x7f8f38199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.933+0000 7f8f3d8bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38108870 0x7f8f381a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.933+0000 7f8f3d8bc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3810e790 con 0x7f8f38108870 2024-05-21T15:35:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.933+0000 7f8f3d8bc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3810e610 con 0x7f8f38105f10 2024-05-21T15:35:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.933+0000 7f8f3d8bc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3810e910 con 0x7f8f381035b0 2024-05-21T15:35:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.933+0000 7f8f377fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38108870 0x7f8f381a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.933+0000 7f8f377fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38108870 0x7f8f381a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42504/0 (socket says 172.21.15.38:42504) 2024-05-21T15:35:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.933+0000 7f8f377fe640 1 -- 172.21.15.38:0/2170332828 learned_addr learned my addr 172.21.15.38:0/2170332828 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.933+0000 7f8f36ffd640 1 --2- 172.21.15.38:0/2170332828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f381035b0 0x7f8f381991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.934+0000 7f8f367fc640 1 --2- 172.21.15.38:0/2170332828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f38105f10 0x7f8f38199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.934+0000 7f8f377fe640 1 -- 172.21.15.38:0/2170332828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f381035b0 msgr2=0x7f8f381991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.934+0000 7f8f377fe640 1 --2- 172.21.15.38:0/2170332828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f381035b0 0x7f8f381991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.934+0000 7f8f377fe640 1 -- 172.21.15.38:0/2170332828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f38105f10 msgr2=0x7f8f38199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.934+0000 7f8f377fe640 1 --2- 172.21.15.38:0/2170332828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f38105f10 0x7f8f38199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.934+0000 7f8f377fe640 1 -- 172.21.15.38:0/2170332828 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f381a0d10 con 0x7f8f38108870 2024-05-21T15:35:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.934+0000 7f8f377fe640 1 --2- 172.21.15.38:0/2170332828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38108870 0x7f8f381a06a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009430 tx=0x7f8f280319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.934+0000 7f8f3c8ba640 1 -- 172.21.15.38:0/2170332828 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28003b70 con 0x7f8f38108870 2024-05-21T15:35:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.934+0000 7f8f3c8ba640 1 -- 172.21.15.38:0/2170332828 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f28003d10 con 0x7f8f38108870 2024-05-21T15:35:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.934+0000 7f8f3d8bc640 1 -- 172.21.15.38:0/2170332828 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f381a0fa0 con 0x7f8f38108870 2024-05-21T15:35:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.934+0000 7f8f3c8ba640 1 -- 172.21.15.38:0/2170332828 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2802fc50 con 0x7f8f38108870 2024-05-21T15:35:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.934+0000 7f8f3d8bc640 1 -- 172.21.15.38:0/2170332828 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f381a1400 con 0x7f8f38108870 2024-05-21T15:35:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.937+0000 7f8f44837740 1 Processor -- start 2024-05-21T15:35:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.937+0000 7f8f3c8ba640 1 -- 172.21.15.38:0/2170332828 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8f28004040 con 0x7f8f38108870 2024-05-21T15:35:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.937+0000 7f8f44837740 1 -- start start 2024-05-21T15:35:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.937+0000 7f8f44837740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccf41b0 0x56514d093e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.937+0000 7f8f44837740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514cccb2f0 0x56514d0943e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.937+0000 7f8f44837740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccd34d0 0x56514d09b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.937+0000 7f8f44837740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56514cff8340 con 0x56514cccb2f0 2024-05-21T15:35:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.937+0000 7f8f44837740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56514cfa0290 con 0x56514ccd34d0 2024-05-21T15:35:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.937+0000 7f8f44837740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56514cff4300 con 0x56514ccf41b0 2024-05-21T15:35:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f36ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514cccb2f0 0x56514d0943e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f36ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514cccb2f0 0x56514d0943e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42514/0 (socket says 172.21.15.38:42514) 2024-05-21T15:35:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f36ffd640 1 -- 172.21.15.38:0/36158767 learned_addr learned my addr 172.21.15.38:0/36158767 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f367fc640 1 --2- 172.21.15.38:0/36158767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccf41b0 0x56514d093e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f36ffd640 1 -- 172.21.15.38:0/36158767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccf41b0 msgr2=0x56514d093e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f36ffd640 1 --2- 172.21.15.38:0/36158767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccf41b0 0x56514d093e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f367fc640 1 --2- 172.21.15.38:0/36158767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccd34d0 0x56514d09b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f36ffd640 1 -- 172.21.15.38:0/36158767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccd34d0 msgr2=0x56514d09b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f36ffd640 1 --2- 172.21.15.38:0/36158767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccd34d0 0x56514d09b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f36ffd640 1 -- 172.21.15.38:0/36158767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56514cfc48e0 con 0x56514cccb2f0 2024-05-21T15:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f367fc640 1 --2- 172.21.15.38:0/36158767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccd34d0 0x56514d09b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f3c8ba640 1 --2- 172.21.15.38:0/2170332828 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f240a2c30 0x7f8f240a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f36ffd640 1 --2- 172.21.15.38:0/36158767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514cccb2f0 0x56514d0943e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000b740 tx=0x7f8f3000bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bc7ea5db4f5f43 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f367fc640 1 --2- 172.21.15.38:0/2170332828 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f240a2c30 0x7f8f240a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f3c8ba640 1 -- 172.21.15.38:0/2170332828 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8f280f0750 con 0x7f8f38108870 2024-05-21T15:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f2dffb640 1 -- 172.21.15.38:0/36158767 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30003dd0 con 0x56514cccb2f0 2024-05-21T15:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f2dffb640 1 -- 172.21.15.38:0/36158767 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f30015070 con 0x56514cccb2f0 2024-05-21T15:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.938+0000 7f8f2dffb640 1 -- 172.21.15.38:0/36158767 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30004e00 con 0x56514cccb2f0 2024-05-21T15:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.939+0000 7f8f44837740 1 -- 172.21.15.38:0/36158767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514cccb2f0 msgr2=0x56514d0943e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.939+0000 7f8f44837740 1 --2- 172.21.15.38:0/36158767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514cccb2f0 0x56514d0943e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000b740 tx=0x7f8f3000bc00 comp rx=0 tx=0).stop 2024-05-21T15:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.939+0000 7f8f367fc640 1 --2- 172.21.15.38:0/2170332828 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f240a2c30 0x7f8f240a5140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x56514d095030 tx=0x7f8f200046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.939+0000 7f8f44837740 1 -- 172.21.15.38:0/36158767 shutdown_connections 2024-05-21T15:35:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.939+0000 7f8f44837740 1 --2- 172.21.15.38:0/36158767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccd34d0 0x56514d09b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.939+0000 7f8f44837740 1 --2- 172.21.15.38:0/36158767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514cccb2f0 0x56514d0943e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.939+0000 7f8f44837740 1 --2- 172.21.15.38:0/36158767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccf41b0 0x56514d093e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.939+0000 7f8f44837740 1 -- 172.21.15.38:0/36158767 >> 172.21.15.38:0/36158767 conn(0x56514cce8300 msgr2=0x56514d08f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.940+0000 7f8f44837740 1 -- 172.21.15.38:0/36158767 shutdown_connections 2024-05-21T15:35:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.940+0000 7f8f44837740 1 -- 172.21.15.38:0/36158767 wait complete. 2024-05-21T15:35:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.940+0000 7f8f44837740 1 -- start start 2024-05-21T15:35:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.941+0000 7f8f44837740 1 Processor -- start 2024-05-21T15:35:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.941+0000 7f8f44837740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514cccb2f0 0x56514d09dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.941+0000 7f8f44837740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccd34d0 0x56514d09e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.941+0000 7f8f44837740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514ccf41b0 0x56514d0a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.941+0000 7f8f44837740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56514d09bd30 con 0x56514ccf41b0 2024-05-21T15:35:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.941+0000 7f8f44837740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56514cfb3790 con 0x56514cccb2f0 2024-05-21T15:35:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.941+0000 7f8f44837740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56514d09beb0 con 0x56514ccd34d0 2024-05-21T15:35:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.941+0000 7f8f44837740 10 client.0 _ll_register_callbacks cb 0x56514cfeea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.941+0000 7f8f377fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514ccf41b0 0x56514d0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.941+0000 7f8f377fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514ccf41b0 0x56514d0a5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42530/0 (socket says 172.21.15.38:42530) 2024-05-21T15:35:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.941+0000 7f8f377fe640 1 -- 172.21.15.38:0/1513383312 learned_addr learned my addr 172.21.15.38:0/1513383312 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.941+0000 7f8f367fc640 1 --2- 172.21.15.38:0/1513383312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccd34d0 0x56514d09e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.941+0000 7f8f36ffd640 1 --2- 172.21.15.38:0/1513383312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514cccb2f0 0x56514d09dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.942+0000 7f8f377fe640 1 -- 172.21.15.38:0/1513383312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccd34d0 msgr2=0x56514d09e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.942+0000 7f8f377fe640 1 --2- 172.21.15.38:0/1513383312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccd34d0 0x56514d09e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.942+0000 7f8f377fe640 1 -- 172.21.15.38:0/1513383312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514cccb2f0 msgr2=0x56514d09dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.942+0000 7f8f377fe640 1 --2- 172.21.15.38:0/1513383312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514cccb2f0 0x56514d09dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.942+0000 7f8f377fe640 1 -- 172.21.15.38:0/1513383312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56514cd0c210 con 0x56514ccf41b0 2024-05-21T15:35:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.942+0000 7f8f377fe640 1 --2- 172.21.15.38:0/1513383312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514ccf41b0 0x56514d0a5060 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8f280c56a0 tx=0x7f8f280bcdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b134faf357200d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.942+0000 7f8f16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.947+0000 7f8f16ffd640 1 -- 172.21.15.38:0/1513383312 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f280bd660 con 0x56514ccf41b0 2024-05-21T15:35:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.947+0000 7f8f16ffd640 1 -- 172.21.15.38:0/1513383312 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8f280b95d0 con 0x56514ccf41b0 2024-05-21T15:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.947+0000 7f8f44837740 1 -- 172.21.15.38:0/1513383312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f30015070 con 0x56514ccf41b0 2024-05-21T15:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.947+0000 7f8f16ffd640 1 -- 172.21.15.38:0/1513383312 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8f280bcf60 con 0x56514ccf41b0 2024-05-21T15:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.947+0000 7f8f44837740 10 client.15549 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.947+0000 7f8f44837740 1 -- 172.21.15.38:0/1513383312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56514cdd8620 con 0x56514ccf41b0 2024-05-21T15:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.947+0000 7f8f16ffd640 1 -- 172.21.15.38:0/1513383312 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f30015070 con 0x56514ccf41b0 2024-05-21T15:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.947+0000 7f8f16ffd640 1 -- 172.21.15.38:0/1513383312 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8f280db050 con 0x56514ccf41b0 2024-05-21T15:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.948+0000 7f8f44837740 10 client.15549 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.948+0000 7f8f44837740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.948+0000 7f8f44837740 20 client.15549 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.948+0000 7f8f44837740 1 --2- 172.21.15.38:0/1513383312 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56514ccb2d80 0x56514d0ab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.948+0000 7f8f44837740 4 client.15549 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.948+0000 7f8f44837740 1 -- 172.21.15.38:0/1513383312 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56514d005a70 con 0x56514ccb2d80 2024-05-21T15:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.948+0000 7f8f36ffd640 1 --2- 172.21.15.38:0/1513383312 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56514ccb2d80 0x56514d0ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.948+0000 7f8f36ffd640 1 --2- 172.21.15.38:0/1513383312 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56514ccb2d80 0x56514d0ab590 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879c8ea950440e7f server_cookie=88a9e560717f071f in_seq=0 out_seq=0 2024-05-21T15:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.948+0000 7f8f16ffd640 10 client.15549 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.949+0000 7f8f16ffd640 1 -- 172.21.15.38:0/1513383312 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56514d005a70 con 0x56514ccb2d80 2024-05-21T15:35:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.949+0000 7f8f16ffd640 10 client.15549 handle_command_reply: tid=0 2024-05-21T15:35:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.949+0000 7f8f44837740 1 client.15549 shutdown 2024-05-21T15:35:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.949+0000 7f8f44837740 1 -- 172.21.15.38:0/1513383312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514ccf41b0 msgr2=0x56514d0a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.949+0000 7f8f44837740 1 --2- 172.21.15.38:0/1513383312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514ccf41b0 0x56514d0a5060 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8f280c56a0 tx=0x7f8f280bcdf0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.950+0000 7f8f44837740 1 -- 172.21.15.38:0/1513383312 shutdown_connections 2024-05-21T15:35:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.950+0000 7f8f44837740 1 --2- 172.21.15.38:0/1513383312 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56514ccb2d80 0x56514d0ab590 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.950+0000 7f8f44837740 1 --2- 172.21.15.38:0/1513383312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514ccf41b0 0x56514d0a5060 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.950+0000 7f8f44837740 1 --2- 172.21.15.38:0/1513383312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccd34d0 0x56514d09e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.950+0000 7f8f16ffd640 0 client.15549 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.950+0000 7f8f44837740 1 --2- 172.21.15.38:0/1513383312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514cccb2f0 0x56514d09dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.950+0000 7f8f44837740 1 -- 172.21.15.38:0/1513383312 >> 172.21.15.38:0/1513383312 conn(0x56514cce8300 msgr2=0x56514d0927a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.950+0000 7f8f44837740 1 -- 172.21.15.38:0/1513383312 shutdown_connections 2024-05-21T15:35:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.950+0000 7f8f44837740 1 -- 172.21.15.38:0/1513383312 wait complete. 2024-05-21T15:35:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.950+0000 7f8f44837740 20 client.15549 trim_cache size 0 max 16384 2024-05-21T15:35:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.965+0000 7f8f44837740 1 Processor -- start 2024-05-21T15:35:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.965+0000 7f8f44837740 1 -- start start 2024-05-21T15:35:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.965+0000 7f8f44837740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514cccb2f0 0x56514d08f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.965+0000 7f8f44837740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccd34d0 0x56514d0a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.965+0000 7f8f44837740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccf41b0 0x56514d08f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f44837740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8f381aa2a0 con 0x56514cccb2f0 2024-05-21T15:35:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f44837740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f381abde0 con 0x56514ccd34d0 2024-05-21T15:35:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f44837740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8f381aa970 con 0x56514ccf41b0 2024-05-21T15:35:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f36ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514cccb2f0 0x56514d08f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f36ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514cccb2f0 0x56514d08f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42536/0 (socket says 172.21.15.38:42536) 2024-05-21T15:35:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f36ffd640 1 -- 172.21.15.38:0/3199475726 learned_addr learned my addr 172.21.15.38:0/3199475726 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f377fe640 1 --2- 172.21.15.38:0/3199475726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccf41b0 0x56514d08f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f367fc640 1 --2- 172.21.15.38:0/3199475726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccd34d0 0x56514d0a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f36ffd640 1 -- 172.21.15.38:0/3199475726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccf41b0 msgr2=0x56514d08f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f36ffd640 1 --2- 172.21.15.38:0/3199475726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccf41b0 0x56514d08f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f36ffd640 1 -- 172.21.15.38:0/3199475726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccd34d0 msgr2=0x56514d0a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f36ffd640 1 --2- 172.21.15.38:0/3199475726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccd34d0 0x56514d0a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f36ffd640 1 -- 172.21.15.38:0/3199475726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56514d095060 con 0x56514cccb2f0 2024-05-21T15:35:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f367fc640 1 --2- 172.21.15.38:0/3199475726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccd34d0 0x56514d0a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f36ffd640 1 --2- 172.21.15.38:0/3199475726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514cccb2f0 0x56514d08f250 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x56514d095240 tx=0x7f8f30005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78ade145dcc1d02 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f2d7fa640 1 -- 172.21.15.38:0/3199475726 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3002a240 con 0x56514cccb2f0 2024-05-21T15:35:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f2d7fa640 1 -- 172.21.15.38:0/3199475726 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f30015070 con 0x56514cccb2f0 2024-05-21T15:35:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.966+0000 7f8f2d7fa640 1 -- 172.21.15.38:0/3199475726 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30011b20 con 0x56514cccb2f0 2024-05-21T15:35:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.967+0000 7f8f44837740 1 -- 172.21.15.38:0/3199475726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514cccb2f0 msgr2=0x56514d08f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.967+0000 7f8f44837740 1 --2- 172.21.15.38:0/3199475726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514cccb2f0 0x56514d08f250 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x56514d095240 tx=0x7f8f30005c50 comp rx=0 tx=0).stop 2024-05-21T15:35:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.967+0000 7f8f44837740 1 -- 172.21.15.38:0/3199475726 shutdown_connections 2024-05-21T15:35:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.967+0000 7f8f44837740 1 --2- 172.21.15.38:0/3199475726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514ccf41b0 0x56514d08f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.967+0000 7f8f44837740 1 --2- 172.21.15.38:0/3199475726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccd34d0 0x56514d0a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.967+0000 7f8f44837740 1 --2- 172.21.15.38:0/3199475726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514cccb2f0 0x56514d08f250 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.967+0000 7f8f44837740 1 -- 172.21.15.38:0/3199475726 >> 172.21.15.38:0/3199475726 conn(0x56514cce8300 msgr2=0x56514d097fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.967+0000 7f8f44837740 1 -- 172.21.15.38:0/3199475726 shutdown_connections 2024-05-21T15:35:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.967+0000 7f8f44837740 1 -- 172.21.15.38:0/3199475726 wait complete. 2024-05-21T15:35:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.967+0000 7f8f44837740 1 -- start start 2024-05-21T15:35:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f44837740 1 Processor -- start 2024-05-21T15:35:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f44837740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514cccb2f0 0x56514d0a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f44837740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514ccd34d0 0x56514d0a2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f44837740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccf41b0 0x56514d0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f44837740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8f381ab710 con 0x56514ccd34d0 2024-05-21T15:35:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f44837740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f381ab040 con 0x56514ccf41b0 2024-05-21T15:35:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f44837740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8f381a8280 con 0x56514cccb2f0 2024-05-21T15:35:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f44837740 10 client.0 _ll_register_callbacks cb 0x7f8f28003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f367fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514ccd34d0 0x56514d0a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f367fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514ccd34d0 0x56514d0a2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42548/0 (socket says 172.21.15.38:42548) 2024-05-21T15:35:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f367fc640 1 -- 172.21.15.38:0/1022418097 learned_addr learned my addr 172.21.15.38:0/1022418097 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f36ffd640 1 --2- 172.21.15.38:0/1022418097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514cccb2f0 0x56514d0a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f377fe640 1 --2- 172.21.15.38:0/1022418097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccf41b0 0x56514d0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f367fc640 1 -- 172.21.15.38:0/1022418097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514cccb2f0 msgr2=0x56514d0a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f367fc640 1 --2- 172.21.15.38:0/1022418097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514cccb2f0 0x56514d0a2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f367fc640 1 -- 172.21.15.38:0/1022418097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccf41b0 msgr2=0x56514d0a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f367fc640 1 --2- 172.21.15.38:0/1022418097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccf41b0 0x56514d0a37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f367fc640 1 -- 172.21.15.38:0/1022418097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56514d0a3d80 con 0x56514ccd34d0 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.968+0000 7f8f36ffd640 1 --2- 172.21.15.38:0/1022418097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514cccb2f0 0x56514d0a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.969+0000 7f8f367fc640 1 --2- 172.21.15.38:0/1022418097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514ccd34d0 0x56514d0a2890 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8f2001fd60 tx=0x7f8f20013050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7d7d87478469a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.969+0000 7f8f16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.969+0000 7f8f16ffd640 1 -- 172.21.15.38:0/1022418097 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20005220 con 0x56514ccd34d0 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.969+0000 7f8f16ffd640 1 -- 172.21.15.38:0/1022418097 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8f200053c0 con 0x56514ccd34d0 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.969+0000 7f8f44837740 1 -- 172.21.15.38:0/1022418097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f30015070 con 0x56514ccd34d0 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.969+0000 7f8f16ffd640 1 -- 172.21.15.38:0/1022418097 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8f200055c0 con 0x56514ccd34d0 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.969+0000 7f8f44837740 10 client.15555 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.969+0000 7f8f44837740 1 -- 172.21.15.38:0/1022418097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56514d0a50b0 con 0x56514ccd34d0 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.969+0000 7f8f16ffd640 1 -- 172.21.15.38:0/1022418097 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f30015070 con 0x56514ccd34d0 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.969+0000 7f8f16ffd640 1 -- 172.21.15.38:0/1022418097 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8f20032070 con 0x56514ccd34d0 2024-05-21T15:35:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.969+0000 7f8f44837740 10 client.15555 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.969+0000 7f8f44837740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.969+0000 7f8f44837740 20 client.15555 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.969+0000 7f8f44837740 1 --2- 172.21.15.38:0/1022418097 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56514cc881c0 0x56514d115f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.970+0000 7f8f44837740 4 client.15555 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:35:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.970+0000 7f8f44837740 1 -- 172.21.15.38:0/1022418097 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56514cfffdc0 con 0x56514cc881c0 2024-05-21T15:35:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.970+0000 7f8f36ffd640 1 --2- 172.21.15.38:0/1022418097 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56514cc881c0 0x56514d115f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.970+0000 7f8f16ffd640 10 client.15555 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.970+0000 7f8f36ffd640 1 --2- 172.21.15.38:0/1022418097 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56514cc881c0 0x56514d115f30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7500019a52d4466b server_cookie=a5f8ea41d9d5f2de in_seq=0 out_seq=0 2024-05-21T15:35:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.977+0000 7f8f16ffd640 1 -- 172.21.15.38:0/1022418097 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56514cfffdc0 con 0x56514cc881c0 2024-05-21T15:35:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.977+0000 7f8f16ffd640 10 client.15555 handle_command_reply: tid=0 2024-05-21T15:35:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.977+0000 7f8f44837740 1 client.15555 shutdown 2024-05-21T15:35:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.977+0000 7f8f44837740 1 -- 172.21.15.38:0/1022418097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514ccd34d0 msgr2=0x56514d0a2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.977+0000 7f8f44837740 1 --2- 172.21.15.38:0/1022418097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514ccd34d0 0x56514d0a2890 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8f2001fd60 tx=0x7f8f20013050 comp rx=0 tx=0).stop 2024-05-21T15:35:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.977+0000 7f8f44837740 1 -- 172.21.15.38:0/1022418097 shutdown_connections 2024-05-21T15:35:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.977+0000 7f8f44837740 1 --2- 172.21.15.38:0/1022418097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56514ccf41b0 0x56514d0a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.977+0000 7f8f44837740 1 --2- 172.21.15.38:0/1022418097 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56514cc881c0 0x56514d115f30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.977+0000 7f8f44837740 1 --2- 172.21.15.38:0/1022418097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56514ccd34d0 0x56514d0a2890 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.977+0000 7f8f44837740 1 --2- 172.21.15.38:0/1022418097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56514cccb2f0 0x56514d0a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.977+0000 7f8f16ffd640 0 client.15555 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.977+0000 7f8f44837740 1 -- 172.21.15.38:0/1022418097 >> 172.21.15.38:0/1022418097 conn(0x56514cce8300 msgr2=0x56514d092690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.977+0000 7f8f44837740 1 -- 172.21.15.38:0/1022418097 shutdown_connections 2024-05-21T15:35:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.978+0000 7f8f44837740 1 -- 172.21.15.38:0/1022418097 wait complete. 2024-05-21T15:35:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.978+0000 7f8f44837740 20 client.15555 trim_cache size 0 max 16384 2024-05-21T15:35:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:35:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:35:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:35:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:35:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.978+0000 7f8f3d8bc640 1 -- 172.21.15.38:0/2170332828 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f240a2c30 msgr2=0x7f8f240a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.978+0000 7f8f3d8bc640 1 --2- 172.21.15.38:0/2170332828 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f240a2c30 0x7f8f240a5140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x56514d095030 tx=0x7f8f200046c0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.978+0000 7f8f3d8bc640 1 -- 172.21.15.38:0/2170332828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38108870 msgr2=0x7f8f381a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.978+0000 7f8f3d8bc640 1 --2- 172.21.15.38:0/2170332828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38108870 0x7f8f381a06a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009430 tx=0x7f8f280319f0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.978+0000 7f8f3d8bc640 1 -- 172.21.15.38:0/2170332828 shutdown_connections 2024-05-21T15:35:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.978+0000 7f8f3d8bc640 1 --2- 172.21.15.38:0/2170332828 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8f240a2c30 0x7f8f240a5140 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.978+0000 7f8f3d8bc640 1 --2- 172.21.15.38:0/2170332828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f38108870 0x7f8f381a06a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.978+0000 7f8f3d8bc640 1 --2- 172.21.15.38:0/2170332828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f38105f10 0x7f8f38199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.978+0000 7f8f3d8bc640 1 --2- 172.21.15.38:0/2170332828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f381035b0 0x7f8f381991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.978+0000 7f8f3d8bc640 1 -- 172.21.15.38:0/2170332828 >> 172.21.15.38:0/2170332828 conn(0x7f8f380fcbb0 msgr2=0x7f8f381043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.978+0000 7f8f3d8bc640 1 -- 172.21.15.38:0/2170332828 shutdown_connections 2024-05-21T15:35:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:09.978+0000 7f8f3d8bc640 1 -- 172.21.15.38:0/2170332828 wait complete. 2024-05-21T15:35:10.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:09 smithi053 ceph-mon[36496]: pgmap v373: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-05-21T15:35:10.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:09 smithi045 ceph-mon[36409]: pgmap v373: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-05-21T15:35:10.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:09 smithi038 ceph-mon[31995]: pgmap v373: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-05-21T15:35:12.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:11 smithi053 ceph-mon[36496]: pgmap v374: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-21T15:35:12.262 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:11 smithi045 ceph-mon[36409]: pgmap v374: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-21T15:35:12.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:11 smithi038 ceph-mon[31995]: pgmap v374: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-21T15:35:13.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:35:13.992 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:35:14.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.128+0000 7fd31c557640 1 -- 172.21.15.38:0/3662290660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3141078c0 msgr2=0x7fd314109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.128+0000 7fd31c557640 1 --2- 172.21.15.38:0/3662290660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3141078c0 0x7fd314109ce0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009ca0 tx=0x7fd30c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.128+0000 7fd31c557640 1 -- 172.21.15.38:0/3662290660 shutdown_connections 2024-05-21T15:35:14.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.128+0000 7fd31c557640 1 --2- 172.21.15.38:0/3662290660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3141078c0 0x7fd314109ce0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.128+0000 7fd31c557640 1 --2- 172.21.15.38:0/3662290660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd314102bc0 0x7fd314102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.128+0000 7fd31c557640 1 --2- 172.21.15.38:0/3662290660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd314101980 0x7fd314102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.128+0000 7fd31c557640 1 -- 172.21.15.38:0/3662290660 >> 172.21.15.38:0/3662290660 conn(0x7fd3140fcb70 msgr2=0x7fd3140ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.128+0000 7fd31c557640 1 -- 172.21.15.38:0/3662290660 shutdown_connections 2024-05-21T15:35:14.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.128+0000 7fd31c557640 1 -- 172.21.15.38:0/3662290660 wait complete. 2024-05-21T15:35:14.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.129+0000 7fd31c557640 1 Processor -- start 2024-05-21T15:35:14.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.129+0000 7fd31c557640 1 -- start start 2024-05-21T15:35:14.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.129+0000 7fd31c557640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd314101980 0x7fd3141980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.129+0000 7fd31c557640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd314102bc0 0x7fd314198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.129+0000 7fd31c557640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3141078c0 0x7fd31419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.129+0000 7fd31c557640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd31410d5e0 con 0x7fd314102bc0 2024-05-21T15:35:14.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.129+0000 7fd31c557640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd31410d460 con 0x7fd3141078c0 2024-05-21T15:35:14.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.129+0000 7fd31c557640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd31410d760 con 0x7fd314101980 2024-05-21T15:35:14.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.129+0000 7fd319af2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd314102bc0 0x7fd314198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.129+0000 7fd319af2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd314102bc0 0x7fd314198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42568/0 (socket says 172.21.15.38:42568) 2024-05-21T15:35:14.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd319af2640 1 -- 172.21.15.38:0/3863793916 learned_addr learned my addr 172.21.15.38:0/3863793916 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd31aaf4640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3141078c0 0x7fd31419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd31a2f3640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd314101980 0x7fd3141980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd319af2640 1 -- 172.21.15.38:0/3863793916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd314101980 msgr2=0x7fd3141980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd319af2640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd314101980 0x7fd3141980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd319af2640 1 -- 172.21.15.38:0/3863793916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3141078c0 msgr2=0x7fd31419f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd319af2640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3141078c0 0x7fd31419f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd319af2640 1 -- 172.21.15.38:0/3863793916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3140690b0 con 0x7fd314102bc0 2024-05-21T15:35:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd31aaf4640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3141078c0 0x7fd31419f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd319af2640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd314102bc0 0x7fd314198600 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd30400d950 tx=0x7fd30400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd30b7fe640 1 -- 172.21.15.38:0/3863793916 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304017030 con 0x7fd314102bc0 2024-05-21T15:35:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd30b7fe640 1 -- 172.21.15.38:0/3863793916 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd304004020 con 0x7fd314102bc0 2024-05-21T15:35:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd31c557640 1 -- 172.21.15.38:0/3863793916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3140693a0 con 0x7fd314102bc0 2024-05-21T15:35:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd30b7fe640 1 -- 172.21.15.38:0/3863793916 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3040047a0 con 0x7fd314102bc0 2024-05-21T15:35:14.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.130+0000 7fd31c557640 1 -- 172.21.15.38:0/3863793916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3140698f0 con 0x7fd314102bc0 2024-05-21T15:35:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.134+0000 7fd323400740 1 Processor -- start 2024-05-21T15:35:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.134+0000 7fd323400740 1 -- start start 2024-05-21T15:35:14.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.134+0000 7fd30b7fe640 1 -- 172.21.15.38:0/3863793916 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd3040041d0 con 0x7fd314102bc0 2024-05-21T15:35:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.134+0000 7fd323400740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 0x55f90fcffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.134+0000 7fd323400740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f9374d0 0x55f90fd00130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.134+0000 7fd323400740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 0x55f90fd07030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.134+0000 7fd323400740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f90fc64d00 con 0x55f90f9374d0 2024-05-21T15:35:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.134+0000 7fd323400740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f90fc0c4c0 con 0x55f90f960390 2024-05-21T15:35:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.134+0000 7fd323400740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f90fc60cc0 con 0x55f90f93f6b0 2024-05-21T15:35:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.134+0000 7fd31a2f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f9374d0 0x55f90fd00130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.134+0000 7fd31a2f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f9374d0 0x55f90fd00130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42570/0 (socket says 172.21.15.38:42570) 2024-05-21T15:35:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.134+0000 7fd31a2f3640 1 -- 172.21.15.38:0/1285201120 learned_addr learned my addr 172.21.15.38:0/1285201120 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.135+0000 7fd319af2640 1 --2- 172.21.15.38:0/1285201120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 0x55f90fd07030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.135+0000 7fd31aaf4640 1 --2- 172.21.15.38:0/1285201120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 0x55f90fcffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.135+0000 7fd31a2f3640 1 -- 172.21.15.38:0/1285201120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 msgr2=0x55f90fd07030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.135+0000 7fd31a2f3640 1 --2- 172.21.15.38:0/1285201120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 0x55f90fd07030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.135+0000 7fd31a2f3640 1 -- 172.21.15.38:0/1285201120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 msgr2=0x55f90fcffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.135+0000 7fd31a2f3640 1 --2- 172.21.15.38:0/1285201120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 0x55f90fcffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.135+0000 7fd31a2f3640 1 -- 172.21.15.38:0/1285201120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f90fa427d0 con 0x55f90f9374d0 2024-05-21T15:35:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.135+0000 7fd30b7fe640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3000a2b90 0x7fd3000a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.135+0000 7fd31a2f3640 1 --2- 172.21.15.38:0/1285201120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f9374d0 0x55f90fd00130 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd31000bc50 tx=0x7fd31000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a404593ba2edf321 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.135+0000 7fd31aaf4640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3000a2b90 0x7fd3000a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.135+0000 7fd30b7fe640 1 -- 172.21.15.38:0/3863793916 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fd3040ceb30 con 0x7fd314102bc0 2024-05-21T15:35:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.135+0000 7fd308bfe640 1 -- 172.21.15.38:0/1285201120 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310015070 con 0x55f90f9374d0 2024-05-21T15:35:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.135+0000 7fd308bfe640 1 -- 172.21.15.38:0/1285201120 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3100035f0 con 0x55f90f9374d0 2024-05-21T15:35:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.135+0000 7fd308bfe640 1 -- 172.21.15.38:0/1285201120 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3100116a0 con 0x55f90f9374d0 2024-05-21T15:35:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.136+0000 7fd323400740 1 -- 172.21.15.38:0/1285201120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f9374d0 msgr2=0x55f90fd00130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.136+0000 7fd323400740 1 --2- 172.21.15.38:0/1285201120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f9374d0 0x55f90fd00130 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd31000bc50 tx=0x7fd31000f730 comp rx=0 tx=0).stop 2024-05-21T15:35:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.136+0000 7fd31aaf4640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3000a2b90 0x7fd3000a50a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd30c031c10 tx=0x7fd30c009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.136+0000 7fd323400740 1 -- 172.21.15.38:0/1285201120 shutdown_connections 2024-05-21T15:35:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.136+0000 7fd323400740 1 --2- 172.21.15.38:0/1285201120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 0x55f90fd07030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.136+0000 7fd323400740 1 --2- 172.21.15.38:0/1285201120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f9374d0 0x55f90fd00130 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.136+0000 7fd323400740 1 --2- 172.21.15.38:0/1285201120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 0x55f90fcffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.136+0000 7fd323400740 1 -- 172.21.15.38:0/1285201120 >> 172.21.15.38:0/1285201120 conn(0x55f90f9544e0 msgr2=0x55f90fcfb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.136+0000 7fd323400740 1 -- 172.21.15.38:0/1285201120 shutdown_connections 2024-05-21T15:35:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.136+0000 7fd323400740 1 -- 172.21.15.38:0/1285201120 wait complete. 2024-05-21T15:35:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.137+0000 7fd323400740 1 -- start start 2024-05-21T15:35:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.137+0000 7fd323400740 1 Processor -- start 2024-05-21T15:35:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.137+0000 7fd323400740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f9374d0 0x55f90fd09b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.137+0000 7fd323400740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 0x55f90fd0a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.137+0000 7fd323400740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f960390 0x55f90fd10f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.137+0000 7fd323400740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f90fd07a80 con 0x55f90f960390 2024-05-21T15:35:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.137+0000 7fd323400740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f90fc1fb90 con 0x55f90f9374d0 2024-05-21T15:35:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.137+0000 7fd323400740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f90fd07c00 con 0x55f90f93f6b0 2024-05-21T15:35:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.137+0000 7fd323400740 10 client.0 _ll_register_callbacks cb 0x55f90fc5bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.137+0000 7fd31aaf4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f960390 0x55f90fd10f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.137+0000 7fd31aaf4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f960390 0x55f90fd10f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42572/0 (socket says 172.21.15.38:42572) 2024-05-21T15:35:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.137+0000 7fd31aaf4640 1 -- 172.21.15.38:0/2354075003 learned_addr learned my addr 172.21.15.38:0/2354075003 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.137+0000 7fd319af2640 1 --2- 172.21.15.38:0/2354075003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 0x55f90fd0a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.137+0000 7fd31a2f3640 1 --2- 172.21.15.38:0/2354075003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f9374d0 0x55f90fd09b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd31aaf4640 1 -- 172.21.15.38:0/2354075003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 msgr2=0x55f90fd0a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd31aaf4640 1 --2- 172.21.15.38:0/2354075003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 0x55f90fd0a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd31aaf4640 1 -- 172.21.15.38:0/2354075003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f9374d0 msgr2=0x55f90fd09b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd31aaf4640 1 --2- 172.21.15.38:0/2354075003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f9374d0 0x55f90fd09b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd31aaf4640 1 -- 172.21.15.38:0/2354075003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f90f9783f0 con 0x55f90f960390 2024-05-21T15:35:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd31a2f3640 1 --2- 172.21.15.38:0/2354075003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f9374d0 0x55f90fd09b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd31aaf4640 1 --2- 172.21.15.38:0/2354075003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f960390 0x55f90fd10f70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd30c02f540 tx=0x7fd30c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4e5b21114befae server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd2fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd2fdffb640 1 -- 172.21.15.38:0/2354075003 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c003e00 con 0x55f90f960390 2024-05-21T15:35:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd2fdffb640 1 -- 172.21.15.38:0/2354075003 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fd30c004590 con 0x55f90f960390 2024-05-21T15:35:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd323400740 1 -- 172.21.15.38:0/2354075003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3100035f0 con 0x55f90f960390 2024-05-21T15:35:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd2fdffb640 1 -- 172.21.15.38:0/2354075003 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd30c004790 con 0x55f90f960390 2024-05-21T15:35:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd323400740 10 client.15567 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd323400740 1 -- 172.21.15.38:0/2354075003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f90f9380f0 con 0x55f90f960390 2024-05-21T15:35:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.138+0000 7fd2fdffb640 1 -- 172.21.15.38:0/2354075003 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3100035f0 con 0x55f90f960390 2024-05-21T15:35:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.139+0000 7fd2fdffb640 1 -- 172.21.15.38:0/2354075003 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd30c030650 con 0x55f90f960390 2024-05-21T15:35:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.139+0000 7fd323400740 10 client.15567 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.139+0000 7fd323400740 10 client.15567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.139+0000 7fd323400740 20 client.15567 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.139+0000 7fd323400740 1 --2- 172.21.15.38:0/2354075003 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f90f91ef60 0x55f90fd17830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.139+0000 7fd323400740 4 client.15567 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.139+0000 7fd323400740 1 -- 172.21.15.38:0/2354075003 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f90fc72430 con 0x55f90f91ef60 2024-05-21T15:35:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.139+0000 7fd31a2f3640 1 --2- 172.21.15.38:0/2354075003 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f90f91ef60 0x55f90fd17830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.140+0000 7fd31a2f3640 1 --2- 172.21.15.38:0/2354075003 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f90f91ef60 0x55f90fd17830 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bf96f751a246b59 server_cookie=f27ca095de36f83a in_seq=0 out_seq=0 2024-05-21T15:35:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.140+0000 7fd2fdffb640 10 client.15567 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.140+0000 7fd2fdffb640 1 -- 172.21.15.38:0/2354075003 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f90fc72430 con 0x55f90f91ef60 2024-05-21T15:35:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.140+0000 7fd2fdffb640 10 client.15567 handle_command_reply: tid=0 2024-05-21T15:35:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.140+0000 7fd323400740 1 client.15567 shutdown 2024-05-21T15:35:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.140+0000 7fd323400740 1 -- 172.21.15.38:0/2354075003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f960390 msgr2=0x55f90fd10f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.140+0000 7fd323400740 1 --2- 172.21.15.38:0/2354075003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f960390 0x55f90fd10f70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd30c02f540 tx=0x7fd30c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.141+0000 7fd323400740 1 -- 172.21.15.38:0/2354075003 shutdown_connections 2024-05-21T15:35:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.141+0000 7fd323400740 1 --2- 172.21.15.38:0/2354075003 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f90f91ef60 0x55f90fd17830 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.141+0000 7fd323400740 1 --2- 172.21.15.38:0/2354075003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f960390 0x55f90fd10f70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.141+0000 7fd323400740 1 --2- 172.21.15.38:0/2354075003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 0x55f90fd0a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.141+0000 7fd323400740 1 --2- 172.21.15.38:0/2354075003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f9374d0 0x55f90fd09b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.141+0000 7fd2fdffb640 0 client.15567 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.141+0000 7fd323400740 1 -- 172.21.15.38:0/2354075003 >> 172.21.15.38:0/2354075003 conn(0x55f90f9544e0 msgr2=0x55f90fcfa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.141+0000 7fd323400740 1 -- 172.21.15.38:0/2354075003 shutdown_connections 2024-05-21T15:35:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.141+0000 7fd323400740 1 -- 172.21.15.38:0/2354075003 wait complete. 2024-05-21T15:35:14.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.141+0000 7fd323400740 20 client.15567 trim_cache size 0 max 16384 2024-05-21T15:35:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.159+0000 7fd323400740 1 Processor -- start 2024-05-21T15:35:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.159+0000 7fd323400740 1 -- start start 2024-05-21T15:35:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.159+0000 7fd323400740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f9374d0 0x55f90fd045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.159+0000 7fd323400740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 0x55f90fd10e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.159+0000 7fd323400740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 0x55f90fd09b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.159+0000 7fd323400740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141aa3e0 con 0x55f90f9374d0 2024-05-21T15:35:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.159+0000 7fd323400740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141abf20 con 0x55f90f960390 2024-05-21T15:35:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.159+0000 7fd323400740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141aaab0 con 0x55f90f93f6b0 2024-05-21T15:35:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.159+0000 7fd31a2f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f9374d0 0x55f90fd045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.159+0000 7fd31a2f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f9374d0 0x55f90fd045e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42574/0 (socket says 172.21.15.38:42574) 2024-05-21T15:35:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.159+0000 7fd31a2f3640 1 -- 172.21.15.38:0/3964840745 learned_addr learned my addr 172.21.15.38:0/3964840745 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.160+0000 7fd31aaf4640 1 --2- 172.21.15.38:0/3964840745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 0x55f90fd09b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.160+0000 7fd319af2640 1 --2- 172.21.15.38:0/3964840745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 0x55f90fd10e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.160+0000 7fd31a2f3640 1 -- 172.21.15.38:0/3964840745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 msgr2=0x55f90fd10e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.160+0000 7fd31a2f3640 1 --2- 172.21.15.38:0/3964840745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 0x55f90fd10e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.160+0000 7fd31a2f3640 1 -- 172.21.15.38:0/3964840745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 msgr2=0x55f90fd09b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.160+0000 7fd31a2f3640 1 --2- 172.21.15.38:0/3964840745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 0x55f90fd09b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.160+0000 7fd31a2f3640 1 -- 172.21.15.38:0/3964840745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f90fcfc0d0 con 0x55f90f9374d0 2024-05-21T15:35:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.160+0000 7fd319af2640 1 --2- 172.21.15.38:0/3964840745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 0x55f90fd10e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.160+0000 7fd31a2f3640 1 --2- 172.21.15.38:0/3964840745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f9374d0 0x55f90fd045e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55f90fd00f90 tx=0x7fd310005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96e501face6a6b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.160+0000 7fd2fffff640 1 -- 172.21.15.38:0/3964840745 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310015050 con 0x55f90f9374d0 2024-05-21T15:35:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.160+0000 7fd2fffff640 1 -- 172.21.15.38:0/3964840745 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd310026e50 con 0x55f90f9374d0 2024-05-21T15:35:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.160+0000 7fd2fffff640 1 -- 172.21.15.38:0/3964840745 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310025440 con 0x55f90f9374d0 2024-05-21T15:35:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.161+0000 7fd323400740 1 -- 172.21.15.38:0/3964840745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f9374d0 msgr2=0x55f90fd045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.161+0000 7fd323400740 1 --2- 172.21.15.38:0/3964840745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f9374d0 0x55f90fd045e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55f90fd00f90 tx=0x7fd310005730 comp rx=0 tx=0).stop 2024-05-21T15:35:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.161+0000 7fd323400740 1 -- 172.21.15.38:0/3964840745 shutdown_connections 2024-05-21T15:35:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.161+0000 7fd323400740 1 --2- 172.21.15.38:0/3964840745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 0x55f90fd09b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.161+0000 7fd323400740 1 --2- 172.21.15.38:0/3964840745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f93f6b0 0x55f90fd10e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.161+0000 7fd323400740 1 --2- 172.21.15.38:0/3964840745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f9374d0 0x55f90fd045e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.161+0000 7fd323400740 1 -- 172.21.15.38:0/3964840745 >> 172.21.15.38:0/3964840745 conn(0x55f90f9544e0 msgr2=0x55f90fcfacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.161+0000 7fd323400740 1 -- 172.21.15.38:0/3964840745 shutdown_connections 2024-05-21T15:35:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.161+0000 7fd323400740 1 -- 172.21.15.38:0/3964840745 wait complete. 2024-05-21T15:35:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.161+0000 7fd323400740 1 -- start start 2024-05-21T15:35:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.161+0000 7fd323400740 1 Processor -- start 2024-05-21T15:35:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd323400740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f9374d0 0x55f90fd0e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd323400740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f93f6b0 0x55f90fd0ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd323400740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 0x55f90fd0faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd323400740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141ab850 con 0x55f90f93f6b0 2024-05-21T15:35:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd323400740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141ab180 con 0x55f90f960390 2024-05-21T15:35:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd323400740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141a83c0 con 0x55f90f9374d0 2024-05-21T15:35:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd323400740 10 client.0 _ll_register_callbacks cb 0x7fd30c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd319af2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f93f6b0 0x55f90fd0ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd319af2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f93f6b0 0x55f90fd0ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42590/0 (socket says 172.21.15.38:42590) 2024-05-21T15:35:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd319af2640 1 -- 172.21.15.38:0/2173046845 learned_addr learned my addr 172.21.15.38:0/2173046845 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd31a2f3640 1 --2- 172.21.15.38:0/2173046845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f9374d0 0x55f90fd0e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd31aaf4640 1 --2- 172.21.15.38:0/2173046845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 0x55f90fd0faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd319af2640 1 -- 172.21.15.38:0/2173046845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f9374d0 msgr2=0x55f90fd0e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd319af2640 1 --2- 172.21.15.38:0/2173046845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f9374d0 0x55f90fd0e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd319af2640 1 -- 172.21.15.38:0/2173046845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 msgr2=0x55f90fd0faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd319af2640 1 --2- 172.21.15.38:0/2173046845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 0x55f90fd0faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd319af2640 1 -- 172.21.15.38:0/2173046845 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f90fd100b0 con 0x55f90f93f6b0 2024-05-21T15:35:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd31aaf4640 1 --2- 172.21.15.38:0/2173046845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 0x55f90fd0faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd319af2640 1 --2- 172.21.15.38:0/2173046845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f93f6b0 0x55f90fd0ebc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd30409b2d0 tx=0x7fd304097700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616f41e618e36aac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd2fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd2fdffb640 1 -- 172.21.15.38:0/2173046845 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30409ba40 con 0x55f90f93f6b0 2024-05-21T15:35:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd2fdffb640 1 -- 172.21.15.38:0/2173046845 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fd30409a790 con 0x55f90f93f6b0 2024-05-21T15:35:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.162+0000 7fd323400740 1 -- 172.21.15.38:0/2173046845 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd310026e50 con 0x55f90f93f6b0 2024-05-21T15:35:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.163+0000 7fd2fdffb640 1 -- 172.21.15.38:0/2173046845 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd30409b340 con 0x55f90f93f6b0 2024-05-21T15:35:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.163+0000 7fd323400740 10 client.15573 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.163+0000 7fd323400740 1 -- 172.21.15.38:0/2173046845 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f90fd7fba0 con 0x55f90f93f6b0 2024-05-21T15:35:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.163+0000 7fd2fdffb640 1 -- 172.21.15.38:0/2173046845 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd310026e50 con 0x55f90f93f6b0 2024-05-21T15:35:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.163+0000 7fd2fdffb640 1 -- 172.21.15.38:0/2173046845 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd3040bb0b0 con 0x55f90f93f6b0 2024-05-21T15:35:14.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.163+0000 7fd323400740 10 client.15573 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.163+0000 7fd323400740 10 client.15573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.163+0000 7fd323400740 20 client.15573 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.163+0000 7fd323400740 1 --2- 172.21.15.38:0/2173046845 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f90f8f43a0 0x55f90fd82480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.163+0000 7fd323400740 4 client.15573 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:35:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.163+0000 7fd323400740 1 -- 172.21.15.38:0/2173046845 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f90fc6c780 con 0x55f90f8f43a0 2024-05-21T15:35:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.163+0000 7fd31a2f3640 1 --2- 172.21.15.38:0/2173046845 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f90f8f43a0 0x55f90fd82480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd2fdffb640 10 client.15573 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd31a2f3640 1 --2- 172.21.15.38:0/2173046845 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f90f8f43a0 0x55f90fd82480 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99639c7dcfe21f4e server_cookie=9dee8be0b88aff95 in_seq=0 out_seq=0 2024-05-21T15:35:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd2fdffb640 1 -- 172.21.15.38:0/2173046845 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f90fc6c780 con 0x55f90f8f43a0 2024-05-21T15:35:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd2fdffb640 10 client.15573 handle_command_reply: tid=0 2024-05-21T15:35:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd323400740 1 client.15573 shutdown 2024-05-21T15:35:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd323400740 1 -- 172.21.15.38:0/2173046845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f93f6b0 msgr2=0x55f90fd0ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd323400740 1 --2- 172.21.15.38:0/2173046845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f93f6b0 0x55f90fd0ebc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd30409b2d0 tx=0x7fd304097700 comp rx=0 tx=0).stop 2024-05-21T15:35:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd323400740 1 -- 172.21.15.38:0/2173046845 shutdown_connections 2024-05-21T15:35:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd323400740 1 --2- 172.21.15.38:0/2173046845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f90f960390 0x55f90fd0faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd323400740 1 --2- 172.21.15.38:0/2173046845 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f90f8f43a0 0x55f90fd82480 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd323400740 1 --2- 172.21.15.38:0/2173046845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f90f93f6b0 0x55f90fd0ebc0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd323400740 1 --2- 172.21.15.38:0/2173046845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f90f9374d0 0x55f90fd0e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd2fdffb640 0 client.15573 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd323400740 1 -- 172.21.15.38:0/2173046845 >> 172.21.15.38:0/2173046845 conn(0x55f90f9544e0 msgr2=0x55f90fd1bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd323400740 1 -- 172.21.15.38:0/2173046845 shutdown_connections 2024-05-21T15:35:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd323400740 1 -- 172.21.15.38:0/2173046845 wait complete. 2024-05-21T15:35:14.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.164+0000 7fd323400740 20 client.15573 trim_cache size 0 max 16384 2024-05-21T15:35:14.172 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:35:14.172 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-05-21T15:35:14.172 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-05-21T15:35:14.172 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:35:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.165+0000 7fd31c557640 1 -- 172.21.15.38:0/3863793916 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3000a2b90 msgr2=0x7fd3000a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.165+0000 7fd31c557640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3000a2b90 0x7fd3000a50a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd30c031c10 tx=0x7fd30c009360 comp rx=0 tx=0).stop 2024-05-21T15:35:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.165+0000 7fd31c557640 1 -- 172.21.15.38:0/3863793916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd314102bc0 msgr2=0x7fd314198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.165+0000 7fd31c557640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd314102bc0 0x7fd314198600 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd30400d950 tx=0x7fd30400de10 comp rx=0 tx=0).stop 2024-05-21T15:35:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.165+0000 7fd31c557640 1 -- 172.21.15.38:0/3863793916 shutdown_connections 2024-05-21T15:35:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.165+0000 7fd31c557640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3141078c0 0x7fd31419f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.165+0000 7fd31c557640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3000a2b90 0x7fd3000a50a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.165+0000 7fd31c557640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd314102bc0 0x7fd314198600 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.165+0000 7fd31c557640 1 --2- 172.21.15.38:0/3863793916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd314101980 0x7fd3141980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.165+0000 7fd31c557640 1 -- 172.21.15.38:0/3863793916 >> 172.21.15.38:0/3863793916 conn(0x7fd3140fcb70 msgr2=0x7fd3141038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.165+0000 7fd31c557640 1 -- 172.21.15.38:0/3863793916 shutdown_connections 2024-05-21T15:35:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.165+0000 7fd31c557640 1 -- 172.21.15.38:0/3863793916 wait complete. 2024-05-21T15:35:14.175 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:35:14.176 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:35:14.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:13 smithi053 ceph-mon[36496]: pgmap v375: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-21T15:35:14.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:13 smithi045 ceph-mon[36409]: pgmap v375: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-21T15:35:14.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:13 smithi038 ceph-mon[31995]: pgmap v375: 97 pgs: 97 active+clean; 656 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-21T15:35:14.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.293+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/1127422265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cec105f00 msgr2=0x7f6cec108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.293+0000 7f6cf3aa0640 1 --2- 172.21.15.38:0/1127422265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cec105f00 0x7f6cec108320 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009900 tx=0x7f6cdc02a410 comp rx=0 tx=0).stop 2024-05-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.293+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/1127422265 shutdown_connections 2024-05-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.293+0000 7f6cf3aa0640 1 --2- 172.21.15.38:0/1127422265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cec108860 0x7f6cec10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.293+0000 7f6cf3aa0640 1 --2- 172.21.15.38:0/1127422265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cec105f00 0x7f6cec108320 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.293+0000 7f6cf3aa0640 1 --2- 172.21.15.38:0/1127422265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cec1035a0 0x7f6cec1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.293+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/1127422265 >> 172.21.15.38:0/1127422265 conn(0x7f6cec0fcba0 msgr2=0x7f6cec0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.293+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/1127422265 shutdown_connections 2024-05-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.293+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/1127422265 wait complete. 2024-05-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.293+0000 7f6cf3aa0640 1 Processor -- start 2024-05-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.293+0000 7f6cf3aa0640 1 -- start start 2024-05-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf3aa0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cec1035a0 0x7f6cec1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf3aa0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cec105f00 0x7f6cec1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf3aa0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cec108860 0x7f6cec1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf3aa0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec10e730 con 0x7f6cec1035a0 2024-05-21T15:35:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf3aa0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec10e5b0 con 0x7f6cec108860 2024-05-21T15:35:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf3aa0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec10e8b0 con 0x7f6cec105f00 2024-05-21T15:35:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf183c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cec1035a0 0x7f6cec1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf183c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cec1035a0 0x7f6cec1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42612/0 (socket says 172.21.15.38:42612) 2024-05-21T15:35:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf183c640 1 -- 172.21.15.38:0/2612853294 learned_addr learned my addr 172.21.15.38:0/2612853294 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf203d640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cec108860 0x7f6cec1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf103b640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cec105f00 0x7f6cec1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf183c640 1 -- 172.21.15.38:0/2612853294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cec105f00 msgr2=0x7f6cec1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf183c640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cec105f00 0x7f6cec1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf183c640 1 -- 172.21.15.38:0/2612853294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cec108860 msgr2=0x7f6cec1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf183c640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cec108860 0x7f6cec1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf183c640 1 -- 172.21.15.38:0/2612853294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cec1a0cf0 con 0x7f6cec1035a0 2024-05-21T15:35:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf203d640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cec108860 0x7f6cec1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6cf183c640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cec1035a0 0x7f6cec1991b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800d770 tx=0x7f6ce800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.294+0000 7f6ce2ffd640 1 -- 172.21.15.38:0/2612853294 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8017070 con 0x7f6cec1035a0 2024-05-21T15:35:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.295+0000 7f6ce2ffd640 1 -- 172.21.15.38:0/2612853294 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce8009c30 con 0x7f6cec1035a0 2024-05-21T15:35:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.295+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/2612853294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cec1a0fe0 con 0x7f6cec1035a0 2024-05-21T15:35:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.295+0000 7f6ce2ffd640 1 -- 172.21.15.38:0/2612853294 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce800f620 con 0x7f6cec1035a0 2024-05-21T15:35:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.295+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/2612853294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cec1a1440 con 0x7f6cec1035a0 2024-05-21T15:35:14.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.296+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/2612853294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cec0691c0 con 0x7f6cec1035a0 2024-05-21T15:35:14.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.297+0000 7f6ce2ffd640 1 -- 172.21.15.38:0/2612853294 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6ce801a070 con 0x7f6cec1035a0 2024-05-21T15:35:14.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.298+0000 7f6ce2ffd640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6cd80a2c30 0x7f6cd80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.298+0000 7f6cf103b640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6cd80a2c30 0x7f6cd80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.298+0000 7f6ce2ffd640 1 -- 172.21.15.38:0/2612853294 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f6ce80cec40 con 0x7f6cec1035a0 2024-05-21T15:35:14.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.298+0000 7f6cf103b640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6cd80a2c30 0x7f6cd80a5140 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc000c00 tx=0x7f6cdc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.301+0000 7f6ce2ffd640 1 -- 172.21.15.38:0/2612853294 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6ce8009de0 con 0x7f6cec1035a0 2024-05-21T15:35:14.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.525+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/2612853294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cec02d320 con 0x7f6cec1035a0 2024-05-21T15:35:14.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.526+0000 7f6ce2ffd640 1 -- 172.21.15.38:0/2612853294 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f6ce809bb50 con 0x7f6cec1035a0 2024-05-21T15:35:14.526 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:35:14.526 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:34:36.440701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24551,24523,24602,24550,24515]},"id":1}]} 2024-05-21T15:35:14.526 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-05-21T15:35:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.528+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/2612853294 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6cd80a2c30 msgr2=0x7f6cd80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.528+0000 7f6cf3aa0640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6cd80a2c30 0x7f6cd80a5140 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc000c00 tx=0x7f6cdc031400 comp rx=0 tx=0).stop 2024-05-21T15:35:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.528+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/2612853294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cec1035a0 msgr2=0x7f6cec1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.528+0000 7f6cf3aa0640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cec1035a0 0x7f6cec1991b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800d770 tx=0x7f6ce800dc30 comp rx=0 tx=0).stop 2024-05-21T15:35:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.528+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/2612853294 shutdown_connections 2024-05-21T15:35:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.528+0000 7f6cf3aa0640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cec108860 0x7f6cec1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.528+0000 7f6cf3aa0640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cec105f00 0x7f6cec1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.528+0000 7f6cf3aa0640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6cd80a2c30 0x7f6cd80a5140 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.528+0000 7f6cf3aa0640 1 --2- 172.21.15.38:0/2612853294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cec1035a0 0x7f6cec1991b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.528+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/2612853294 >> 172.21.15.38:0/2612853294 conn(0x7f6cec0fcba0 msgr2=0x7f6cec104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.528+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/2612853294 shutdown_connections 2024-05-21T15:35:14.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.528+0000 7f6cf3aa0640 1 -- 172.21.15.38:0/2612853294 wait complete. 2024-05-21T15:35:14.539 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:35:14.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.657+0000 7fddb0c0d640 1 -- 172.21.15.38:0/3239233033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac105ba0 msgr2=0x7fddac108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.657+0000 7fddb0c0d640 1 --2- 172.21.15.38:0/3239233033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac105ba0 0x7fddac108080 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdd94009900 tx=0x7fdd9402a410 comp rx=0 tx=0).stop 2024-05-21T15:35:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.657+0000 7fddb0c0d640 1 -- 172.21.15.38:0/3239233033 shutdown_connections 2024-05-21T15:35:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.657+0000 7fddb0c0d640 1 --2- 172.21.15.38:0/3239233033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddac108710 0x7fddac10ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.657+0000 7fddb0c0d640 1 --2- 172.21.15.38:0/3239233033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac105ba0 0x7fddac108080 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.657+0000 7fddb0c0d640 1 --2- 172.21.15.38:0/3239233033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddac1031b0 0x7fddac1055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.657+0000 7fddb0c0d640 1 -- 172.21.15.38:0/3239233033 >> 172.21.15.38:0/3239233033 conn(0x7fddac0fcb70 msgr2=0x7fddac0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.657+0000 7fddb0c0d640 1 -- 172.21.15.38:0/3239233033 shutdown_connections 2024-05-21T15:35:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.657+0000 7fddb0c0d640 1 -- 172.21.15.38:0/3239233033 wait complete. 2024-05-21T15:35:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.658+0000 7fddb0c0d640 1 Processor -- start 2024-05-21T15:35:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.658+0000 7fddb0c0d640 1 -- start start 2024-05-21T15:35:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.658+0000 7fddb0c0d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac1031b0 0x7fddac199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.658+0000 7fddb0c0d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddac105ba0 0x7fddac1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.658+0000 7fddb0c0d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddac108710 0x7fddac1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.658+0000 7fddb0c0d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fddac10e6b0 con 0x7fddac1031b0 2024-05-21T15:35:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.658+0000 7fddb0c0d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fddac10e530 con 0x7fddac105ba0 2024-05-21T15:35:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.658+0000 7fddb0c0d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fddac10e830 con 0x7fddac108710 2024-05-21T15:35:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.658+0000 7fddaa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac1031b0 0x7fddac199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.658+0000 7fddaa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac1031b0 0x7fddac199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42636/0 (socket says 172.21.15.38:42636) 2024-05-21T15:35:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.658+0000 7fddaa59c640 1 -- 172.21.15.38:0/2787312953 learned_addr learned my addr 172.21.15.38:0/2787312953 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.658+0000 7fddaad9d640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddac108710 0x7fddac1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.658+0000 7fdda9d9b640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddac105ba0 0x7fddac1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.659+0000 7fddaa59c640 1 -- 172.21.15.38:0/2787312953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddac108710 msgr2=0x7fddac1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.659+0000 7fddaa59c640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddac108710 0x7fddac1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.659+0000 7fddaa59c640 1 -- 172.21.15.38:0/2787312953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddac105ba0 msgr2=0x7fddac1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.659+0000 7fddaa59c640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddac105ba0 0x7fddac1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.659+0000 7fddaa59c640 1 -- 172.21.15.38:0/2787312953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddac1a0dc0 con 0x7fddac1031b0 2024-05-21T15:35:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.659+0000 7fddaad9d640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddac108710 0x7fddac1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.659+0000 7fddaa59c640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac1031b0 0x7fddac199280 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdda400e9c0 tx=0x7fdda400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.659+0000 7fdda37fe640 1 -- 172.21.15.38:0/2787312953 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda400cd80 con 0x7fddac1031b0 2024-05-21T15:35:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.659+0000 7fdda37fe640 1 -- 172.21.15.38:0/2787312953 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda40035f0 con 0x7fddac1031b0 2024-05-21T15:35:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.659+0000 7fddb0c0d640 1 -- 172.21.15.38:0/2787312953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddac1a10b0 con 0x7fddac1031b0 2024-05-21T15:35:14.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.659+0000 7fdda37fe640 1 -- 172.21.15.38:0/2787312953 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4010690 con 0x7fddac1031b0 2024-05-21T15:35:14.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.659+0000 7fddb0c0d640 1 -- 172.21.15.38:0/2787312953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddac1a1600 con 0x7fddac1031b0 2024-05-21T15:35:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.661+0000 7fdda37fe640 1 -- 172.21.15.38:0/2787312953 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdda40037a0 con 0x7fddac1031b0 2024-05-21T15:35:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.662+0000 7fddb7b8e740 1 Processor -- start 2024-05-21T15:35:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.662+0000 7fddb7b8e740 1 -- start start 2024-05-21T15:35:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.662+0000 7fddb7b8e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c92390 0x55b1a8031b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.662+0000 7fddb7b8e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c694d0 0x55b1a8032080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.662+0000 7fddb7b8e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c716b0 0x55b1a8038f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.662+0000 7fddb7b8e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b1a7f96c40 con 0x55b1a7c694d0 2024-05-21T15:35:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.662+0000 7fddb7b8e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b1a7f3e580 con 0x55b1a7c716b0 2024-05-21T15:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.662+0000 7fddb7b8e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b1a7f92c00 con 0x55b1a7c92390 2024-05-21T15:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.662+0000 7fdda37fe640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdd980a2b90 0x7fdd980a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.662+0000 7fddaa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c694d0 0x55b1a8032080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.662+0000 7fddaa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c694d0 0x55b1a8032080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42652/0 (socket says 172.21.15.38:42652) 2024-05-21T15:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.662+0000 7fddaa59c640 1 -- 172.21.15.38:0/3710241191 learned_addr learned my addr 172.21.15.38:0/3710241191 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.662+0000 7fddaad9d640 1 --2- 172.21.15.38:0/3710241191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c92390 0x55b1a8031b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.663+0000 7fdda9d9b640 1 --2- 172.21.15.38:0/3710241191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c716b0 0x55b1a8038f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.663+0000 7fddaad9d640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdd980a2b90 0x7fdd980a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.663+0000 7fdda37fe640 1 -- 172.21.15.38:0/2787312953 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdda40cf600 con 0x7fddac1031b0 2024-05-21T15:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.663+0000 7fddaa59c640 1 -- 172.21.15.38:0/3710241191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c92390 msgr2=0x55b1a8031b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.663+0000 7fddaa59c640 1 --2- 172.21.15.38:0/3710241191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c92390 0x55b1a8031b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.663+0000 7fddaa59c640 1 -- 172.21.15.38:0/3710241191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c716b0 msgr2=0x55b1a8038f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.663+0000 7fddaa59c640 1 --2- 172.21.15.38:0/3710241191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c716b0 0x55b1a8038f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.663+0000 7fddaa59c640 1 -- 172.21.15.38:0/3710241191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a7caa3f0 con 0x55b1a7c694d0 2024-05-21T15:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.663+0000 7fddaad9d640 1 --2- 172.21.15.38:0/3710241191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c92390 0x55b1a8031b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.663+0000 7fddaa59c640 1 --2- 172.21.15.38:0/3710241191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c694d0 0x55b1a8032080 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdda40cd050 tx=0x7fdda409bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c2afc06c3965c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.663+0000 7fddaad9d640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdd980a2b90 0x7fdd980a50a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55b1a8032cd0 tx=0x7fdd9c0094a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.663+0000 7fdda0ff9640 1 -- 172.21.15.38:0/3710241191 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda409be10 con 0x55b1a7c694d0 2024-05-21T15:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.663+0000 7fdda0ff9640 1 -- 172.21.15.38:0/3710241191 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda40a01c0 con 0x55b1a7c694d0 2024-05-21T15:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.663+0000 7fdda0ff9640 1 -- 172.21.15.38:0/3710241191 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda409b260 con 0x55b1a7c694d0 2024-05-21T15:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.664+0000 7fddb7b8e740 1 -- 172.21.15.38:0/3710241191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c694d0 msgr2=0x55b1a8032080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.664+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/3710241191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c694d0 0x55b1a8032080 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdda40cd050 tx=0x7fdda409bca0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.664+0000 7fddb7b8e740 1 -- 172.21.15.38:0/3710241191 shutdown_connections 2024-05-21T15:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.664+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/3710241191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c716b0 0x55b1a8038f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.664+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/3710241191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c694d0 0x55b1a8032080 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.664+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/3710241191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c92390 0x55b1a8031b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.664+0000 7fddb7b8e740 1 -- 172.21.15.38:0/3710241191 >> 172.21.15.38:0/3710241191 conn(0x55b1a7c864e0 msgr2=0x55b1a802d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.664+0000 7fddb7b8e740 1 -- 172.21.15.38:0/3710241191 shutdown_connections 2024-05-21T15:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.664+0000 7fddb7b8e740 1 -- 172.21.15.38:0/3710241191 wait complete. 2024-05-21T15:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddb7b8e740 1 -- start start 2024-05-21T15:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddb7b8e740 1 Processor -- start 2024-05-21T15:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddb7b8e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c694d0 0x55b1a803b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddb7b8e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c716b0 0x55b1a803bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddb7b8e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c92390 0x55b1a8042d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddb7b8e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b1a8039910 con 0x55b1a7c92390 2024-05-21T15:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddb7b8e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b1a7f51c40 con 0x55b1a7c694d0 2024-05-21T15:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddb7b8e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b1a8039a90 con 0x55b1a7c716b0 2024-05-21T15:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddb7b8e740 10 client.0 _ll_register_callbacks cb 0x55b1a7ca9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddaad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c92390 0x55b1a8042d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddaad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c92390 0x55b1a8042d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42660/0 (socket says 172.21.15.38:42660) 2024-05-21T15:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddaad9d640 1 -- 172.21.15.38:0/1126178071 learned_addr learned my addr 172.21.15.38:0/1126178071 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fdda9d9b640 1 --2- 172.21.15.38:0/1126178071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c716b0 0x55b1a803bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddaa59c640 1 --2- 172.21.15.38:0/1126178071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c694d0 0x55b1a803b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddaad9d640 1 -- 172.21.15.38:0/1126178071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c716b0 msgr2=0x55b1a803bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddaad9d640 1 --2- 172.21.15.38:0/1126178071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c716b0 0x55b1a803bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddaad9d640 1 -- 172.21.15.38:0/1126178071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c694d0 msgr2=0x55b1a803b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddaad9d640 1 --2- 172.21.15.38:0/1126178071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c694d0 0x55b1a803b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.665+0000 7fddaad9d640 1 -- 172.21.15.38:0/1126178071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a7d747d0 con 0x55b1a7c92390 2024-05-21T15:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fddaad9d640 1 --2- 172.21.15.38:0/1126178071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c92390 0x55b1a8042d30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00de90 tx=0x7fdd9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5191abdf1f1025ba server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fdd91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fdd91ffb640 1 -- 172.21.15.38:0/1126178071 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c025030 con 0x55b1a7c92390 2024-05-21T15:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fdd91ffb640 1 -- 172.21.15.38:0/1126178071 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdd9c00fd80 con 0x55b1a7c92390 2024-05-21T15:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fddb7b8e740 1 -- 172.21.15.38:0/1126178071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdda40a01c0 con 0x55b1a7c92390 2024-05-21T15:35:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fdd91ffb640 1 -- 172.21.15.38:0/1126178071 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd9c01b040 con 0x55b1a7c92390 2024-05-21T15:35:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fddb7b8e740 10 client.15591 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fddb7b8e740 1 -- 172.21.15.38:0/1126178071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1a80460c0 con 0x55b1a7c92390 2024-05-21T15:35:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fdd91ffb640 1 -- 172.21.15.38:0/1126178071 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda40a01c0 con 0x55b1a7c92390 2024-05-21T15:35:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fdd91ffb640 1 -- 172.21.15.38:0/1126178071 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdd9c0057d0 con 0x55b1a7c92390 2024-05-21T15:35:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fddb7b8e740 10 client.15591 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fddb7b8e740 10 client.15591 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:35:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fddb7b8e740 20 client.15591 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/1126178071 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b1a7c50f60 0x55b1a8049890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fddb7b8e740 4 client.15591 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.666+0000 7fddb7b8e740 1 -- 172.21.15.38:0/1126178071 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1a7fa4370 con 0x55b1a7c50f60 2024-05-21T15:35:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.667+0000 7fddaa59c640 1 --2- 172.21.15.38:0/1126178071 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b1a7c50f60 0x55b1a8049890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.667+0000 7fddaa59c640 1 --2- 172.21.15.38:0/1126178071 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b1a7c50f60 0x55b1a8049890 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7e1bd72533262af1 server_cookie=c66da376765a3fc8 in_seq=0 out_seq=0 2024-05-21T15:35:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.667+0000 7fdd91ffb640 10 client.15591 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.668+0000 7fdd91ffb640 1 -- 172.21.15.38:0/1126178071 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b1a7fa4370 con 0x55b1a7c50f60 2024-05-21T15:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.668+0000 7fdd91ffb640 10 client.15591 handle_command_reply: tid=0 2024-05-21T15:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.668+0000 7fddb7b8e740 1 client.15591 shutdown 2024-05-21T15:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.668+0000 7fddb7b8e740 1 -- 172.21.15.38:0/1126178071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c92390 msgr2=0x55b1a8042d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.668+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/1126178071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c92390 0x55b1a8042d30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00de90 tx=0x7fdd9c004a40 comp rx=0 tx=0).stop 2024-05-21T15:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.668+0000 7fddb7b8e740 1 -- 172.21.15.38:0/1126178071 shutdown_connections 2024-05-21T15:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.668+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/1126178071 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b1a7c50f60 0x55b1a8049890 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.668+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/1126178071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c92390 0x55b1a8042d30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.668+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/1126178071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c716b0 0x55b1a803bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.668+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/1126178071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c694d0 0x55b1a803b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.668+0000 7fdd91ffb640 0 client.15591 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.668+0000 7fddb7b8e740 1 -- 172.21.15.38:0/1126178071 >> 172.21.15.38:0/1126178071 conn(0x55b1a7c864e0 msgr2=0x55b1a802c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.668+0000 7fddb7b8e740 1 -- 172.21.15.38:0/1126178071 shutdown_connections 2024-05-21T15:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.668+0000 7fddb7b8e740 1 -- 172.21.15.38:0/1126178071 wait complete. 2024-05-21T15:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.669+0000 7fddb7b8e740 20 client.15591 trim_cache size 0 max 16384 2024-05-21T15:35:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.682+0000 7fddb7b8e740 1 Processor -- start 2024-05-21T15:35:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.682+0000 7fddb7b8e740 1 -- start start 2024-05-21T15:35:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.682+0000 7fddb7b8e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c50f60 0x55b1a8035d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.682+0000 7fddb7b8e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c694d0 0x55b1a8036280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.682+0000 7fddb7b8e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c716b0 0x55b1a8039e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.682+0000 7fddb7b8e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fddac1aa4d0 con 0x55b1a7c694d0 2024-05-21T15:35:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.682+0000 7fddb7b8e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fddac1ac010 con 0x55b1a7c716b0 2024-05-21T15:35:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.682+0000 7fddb7b8e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fddac1aaba0 con 0x55b1a7c50f60 2024-05-21T15:35:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fdda9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c694d0 0x55b1a8036280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fdda9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c694d0 0x55b1a8036280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42666/0 (socket says 172.21.15.38:42666) 2024-05-21T15:35:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fdda9d9b640 1 -- 172.21.15.38:0/2977460537 learned_addr learned my addr 172.21.15.38:0/2977460537 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fddaa59c640 1 --2- 172.21.15.38:0/2977460537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c50f60 0x55b1a8035d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fddaad9d640 1 --2- 172.21.15.38:0/2977460537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c716b0 0x55b1a8039e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fdda9d9b640 1 -- 172.21.15.38:0/2977460537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c50f60 msgr2=0x55b1a8035d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fdda9d9b640 1 --2- 172.21.15.38:0/2977460537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c50f60 0x55b1a8035d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fdda9d9b640 1 -- 172.21.15.38:0/2977460537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c716b0 msgr2=0x55b1a8039e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fdda9d9b640 1 --2- 172.21.15.38:0/2977460537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c716b0 0x55b1a8039e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fdda9d9b640 1 -- 172.21.15.38:0/2977460537 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a8028d20 con 0x55b1a7c694d0 2024-05-21T15:35:14.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fddaad9d640 1 --2- 172.21.15.38:0/2977460537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c716b0 0x55b1a8039e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:14.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fdda9d9b640 1 --2- 172.21.15.38:0/2977460537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c694d0 0x55b1a8036280 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdd94031440 tx=0x7fdd94032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633eec46e31a0dec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fdd93fff640 1 -- 172.21.15.38:0/2977460537 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94035040 con 0x55b1a7c694d0 2024-05-21T15:35:14.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fdd93fff640 1 -- 172.21.15.38:0/2977460537 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd940318b0 con 0x55b1a7c694d0 2024-05-21T15:35:14.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.683+0000 7fdd93fff640 1 -- 172.21.15.38:0/2977460537 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd940046e0 con 0x55b1a7c694d0 2024-05-21T15:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.684+0000 7fddb7b8e740 1 -- 172.21.15.38:0/2977460537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c694d0 msgr2=0x55b1a8036280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.684+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/2977460537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c694d0 0x55b1a8036280 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdd94031440 tx=0x7fdd94032e70 comp rx=0 tx=0).stop 2024-05-21T15:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.684+0000 7fddb7b8e740 1 -- 172.21.15.38:0/2977460537 shutdown_connections 2024-05-21T15:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.684+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/2977460537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c716b0 0x55b1a8039e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.684+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/2977460537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c694d0 0x55b1a8036280 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.684+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/2977460537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c50f60 0x55b1a8035d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.684+0000 7fddb7b8e740 1 -- 172.21.15.38:0/2977460537 >> 172.21.15.38:0/2977460537 conn(0x55b1a7c864e0 msgr2=0x55b1a802cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.684+0000 7fddb7b8e740 1 -- 172.21.15.38:0/2977460537 shutdown_connections 2024-05-21T15:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.684+0000 7fddb7b8e740 1 -- 172.21.15.38:0/2977460537 wait complete. 2024-05-21T15:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.684+0000 7fddb7b8e740 1 -- start start 2024-05-21T15:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddb7b8e740 1 Processor -- start 2024-05-21T15:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddb7b8e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c50f60 0x55b1a803efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddb7b8e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c694d0 0x55b1a803f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddb7b8e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c716b0 0x55b1a80afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddb7b8e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fddac1ab940 con 0x55b1a7c716b0 2024-05-21T15:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddb7b8e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fddac1ab270 con 0x55b1a7c694d0 2024-05-21T15:35:14.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddb7b8e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fddac1a84b0 con 0x55b1a7c50f60 2024-05-21T15:35:14.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddb7b8e740 10 client.0 _ll_register_callbacks cb 0x7fdd9c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:14.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddaad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c716b0 0x55b1a80afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddaad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c716b0 0x55b1a80afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42674/0 (socket says 172.21.15.38:42674) 2024-05-21T15:35:14.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddaad9d640 1 -- 172.21.15.38:0/2396732814 learned_addr learned my addr 172.21.15.38:0/2396732814 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddaa59c640 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c50f60 0x55b1a803efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fdda9d9b640 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c694d0 0x55b1a803f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddaad9d640 1 -- 172.21.15.38:0/2396732814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c50f60 msgr2=0x55b1a803efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddaad9d640 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c50f60 0x55b1a803efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddaad9d640 1 -- 172.21.15.38:0/2396732814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c694d0 msgr2=0x55b1a803f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddaad9d640 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c694d0 0x55b1a803f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddaad9d640 1 -- 172.21.15.38:0/2396732814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a80424d0 con 0x55b1a7c716b0 2024-05-21T15:35:14.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddaa59c640 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c50f60 0x55b1a803efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:14.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fdda9d9b640 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c694d0 0x55b1a803f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:14.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fddaad9d640 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c716b0 0x55b1a80afc30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c012a50 tx=0x7fdd9c010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4e1d9b1dbdeec5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.685+0000 7fdd91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:14.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.686+0000 7fdd91ffb640 1 -- 172.21.15.38:0/2396732814 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c025070 con 0x55b1a7c716b0 2024-05-21T15:35:14.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.686+0000 7fddb7b8e740 1 -- 172.21.15.38:0/2396732814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd940318b0 con 0x55b1a7c716b0 2024-05-21T15:35:14.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.686+0000 7fdd91ffb640 1 -- 172.21.15.38:0/2396732814 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdd9c004cb0 con 0x55b1a7c716b0 2024-05-21T15:35:14.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.686+0000 7fdd91ffb640 1 -- 172.21.15.38:0/2396732814 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd9c005040 con 0x55b1a7c716b0 2024-05-21T15:35:14.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.686+0000 7fddb7b8e740 10 client.15597 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:14.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.686+0000 7fddb7b8e740 1 -- 172.21.15.38:0/2396732814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1a80b1090 con 0x55b1a7c716b0 2024-05-21T15:35:14.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.686+0000 7fdd91ffb640 1 -- 172.21.15.38:0/2396732814 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd940318b0 con 0x55b1a7c716b0 2024-05-21T15:35:14.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.686+0000 7fdd91ffb640 1 -- 172.21.15.38:0/2396732814 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdd9c03c740 con 0x55b1a7c716b0 2024-05-21T15:35:14.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.686+0000 7fddb7b8e740 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:14.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.686+0000 7fddb7b8e740 10 client.15597 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:35:14.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.686+0000 7fddb7b8e740 20 client.15597 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:14.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.686+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b1a7c92390 0x55b1a80b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.686+0000 7fddb7b8e740 4 client.15597 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:35:14.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.686+0000 7fddb7b8e740 1 -- 172.21.15.38:0/2396732814 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1a7f9e6c0 con 0x55b1a7c92390 2024-05-21T15:35:14.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.687+0000 7fddaa59c640 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b1a7c92390 0x55b1a80b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.687+0000 7fdd91ffb640 10 client.15597 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:35:14.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.687+0000 7fddaa59c640 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b1a7c92390 0x55b1a80b3ab0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=184ff2aff0506a4 server_cookie=9ffe04fbbbfcd480 in_seq=0 out_seq=0 2024-05-21T15:35:14.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.687+0000 7fdd91ffb640 1 -- 172.21.15.38:0/2396732814 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1a7f9e6c0 con 0x55b1a7c92390 2024-05-21T15:35:14.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.687+0000 7fdd91ffb640 10 client.15597 handle_command_reply: tid=0 2024-05-21T15:35:14.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.687+0000 7fddb7b8e740 1 client.15597 shutdown 2024-05-21T15:35:14.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.687+0000 7fddb7b8e740 1 -- 172.21.15.38:0/2396732814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c716b0 msgr2=0x55b1a80afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.687+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c716b0 0x55b1a80afc30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c012a50 tx=0x7fdd9c010ef0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.687+0000 7fddb7b8e740 1 -- 172.21.15.38:0/2396732814 shutdown_connections 2024-05-21T15:35:14.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.687+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b1a7c92390 0x55b1a80b3ab0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1a7c716b0 0x55b1a80afc30 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1a7c694d0 0x55b1a803f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb7b8e740 1 --2- 172.21.15.38:0/2396732814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1a7c50f60 0x55b1a803efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fdd91ffb640 0 client.15597 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:35:14.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb7b8e740 1 -- 172.21.15.38:0/2396732814 >> 172.21.15.38:0/2396732814 conn(0x55b1a7c864e0 msgr2=0x55b1a8039e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb7b8e740 1 -- 172.21.15.38:0/2396732814 shutdown_connections 2024-05-21T15:35:14.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb7b8e740 1 -- 172.21.15.38:0/2396732814 wait complete. 2024-05-21T15:35:14.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb7b8e740 20 client.15597 trim_cache size 0 max 16384 2024-05-21T15:35:14.693 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:35:14.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb0c0d640 1 -- 172.21.15.38:0/2787312953 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdd980a2b90 msgr2=0x7fdd980a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb0c0d640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdd980a2b90 0x7fdd980a50a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55b1a8032cd0 tx=0x7fdd9c0094a0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb0c0d640 1 -- 172.21.15.38:0/2787312953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac1031b0 msgr2=0x7fddac199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb0c0d640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac1031b0 0x7fddac199280 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdda400e9c0 tx=0x7fdda400ee80 comp rx=0 tx=0).stop 2024-05-21T15:35:14.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb0c0d640 1 -- 172.21.15.38:0/2787312953 shutdown_connections 2024-05-21T15:35:14.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb0c0d640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddac108710 0x7fddac1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb0c0d640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddac105ba0 0x7fddac1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb0c0d640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdd980a2b90 0x7fdd980a50a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb0c0d640 1 --2- 172.21.15.38:0/2787312953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac1031b0 0x7fddac199280 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb0c0d640 1 -- 172.21.15.38:0/2787312953 >> 172.21.15.38:0/2787312953 conn(0x7fddac0fcb70 msgr2=0x7fddac103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb0c0d640 1 -- 172.21.15.38:0/2787312953 shutdown_connections 2024-05-21T15:35:14.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.688+0000 7fddb0c0d640 1 -- 172.21.15.38:0/2787312953 wait complete. 2024-05-21T15:35:14.699 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:35:14.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.816+0000 7f3d6ac13640 1 -- 172.21.15.38:0/1862519139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d64102bd0 msgr2=0x7f3d64102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.816+0000 7f3d6ac13640 1 --2- 172.21.15.38:0/1862519139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d64102bd0 0x7f3d64102fe0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3d50009900 tx=0x7f3d5002a410 comp rx=0 tx=0).stop 2024-05-21T15:35:14.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.816+0000 7f3d6ac13640 1 -- 172.21.15.38:0/1862519139 shutdown_connections 2024-05-21T15:35:14.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.816+0000 7f3d6ac13640 1 --2- 172.21.15.38:0/1862519139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d641078d0 0x7f3d64109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.816+0000 7f3d6ac13640 1 --2- 172.21.15.38:0/1862519139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d64102bd0 0x7f3d64102fe0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.816+0000 7f3d6ac13640 1 --2- 172.21.15.38:0/1862519139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d64101990 0x7f3d64102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.816+0000 7f3d6ac13640 1 -- 172.21.15.38:0/1862519139 >> 172.21.15.38:0/1862519139 conn(0x7f3d640fcb80 msgr2=0x7f3d640ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.816+0000 7f3d6ac13640 1 -- 172.21.15.38:0/1862519139 shutdown_connections 2024-05-21T15:35:14.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.817+0000 7f3d6ac13640 1 -- 172.21.15.38:0/1862519139 wait complete. 2024-05-21T15:35:14.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.817+0000 7f3d6ac13640 1 Processor -- start 2024-05-21T15:35:14.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.817+0000 7f3d6ac13640 1 -- start start 2024-05-21T15:35:14.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.817+0000 7f3d6ac13640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d64101990 0x7f3d64198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.817+0000 7f3d6ac13640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d64102bd0 0x7f3d64198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.817+0000 7f3d6ac13640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d641078d0 0x7f3d6419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.817+0000 7f3d6ac13640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6410d5e0 con 0x7f3d64102bd0 2024-05-21T15:35:14.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.817+0000 7f3d6ac13640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6410d460 con 0x7f3d641078d0 2024-05-21T15:35:14.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.817+0000 7f3d6ac13640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6410d760 con 0x7f3d64101990 2024-05-21T15:35:14.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d63fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d64102bd0 0x7f3d64198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d63fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d64102bd0 0x7f3d64198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40102/0 (socket says 172.21.15.38:40102) 2024-05-21T15:35:14.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d63fff640 1 -- 172.21.15.38:0/1159719271 learned_addr learned my addr 172.21.15.38:0/1159719271 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d691b0640 1 --2- 172.21.15.38:0/1159719271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d641078d0 0x7f3d6419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d689af640 1 --2- 172.21.15.38:0/1159719271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d64101990 0x7f3d64198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d63fff640 1 -- 172.21.15.38:0/1159719271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d64101990 msgr2=0x7f3d64198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d63fff640 1 --2- 172.21.15.38:0/1159719271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d64101990 0x7f3d64198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d63fff640 1 -- 172.21.15.38:0/1159719271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d641078d0 msgr2=0x7f3d6419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d63fff640 1 --2- 172.21.15.38:0/1159719271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d641078d0 0x7f3d6419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d63fff640 1 -- 172.21.15.38:0/1159719271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6419fc90 con 0x7f3d64102bd0 2024-05-21T15:35:14.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d63fff640 1 --2- 172.21.15.38:0/1159719271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d64102bd0 0x7f3d64198690 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006010 tx=0x7f3d50004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d61ffb640 1 -- 172.21.15.38:0/1159719271 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5003f030 con 0x7f3d64102bd0 2024-05-21T15:35:14.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d6ac13640 1 -- 172.21.15.38:0/1159719271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6419ff20 con 0x7f3d64102bd0 2024-05-21T15:35:14.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d61ffb640 1 -- 172.21.15.38:0/1159719271 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d50003ce0 con 0x7f3d64102bd0 2024-05-21T15:35:14.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.818+0000 7f3d61ffb640 1 -- 172.21.15.38:0/1159719271 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50035070 con 0x7f3d64102bd0 2024-05-21T15:35:14.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.819+0000 7f3d6ac13640 1 -- 172.21.15.38:0/1159719271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d641a0380 con 0x7f3d64102bd0 2024-05-21T15:35:14.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.821+0000 7f3d61ffb640 1 -- 172.21.15.38:0/1159719271 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3d50042030 con 0x7f3d64102bd0 2024-05-21T15:35:14.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.821+0000 7f3d71b8b740 1 Processor -- start 2024-05-21T15:35:14.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.821+0000 7f3d71b8b740 1 -- start start 2024-05-21T15:35:14.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d71b8b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03dbd390 0x55cc04163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d71b8b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03d944d0 0x55cc04165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d71b8b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d9c6b0 0x55cc041682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d71b8b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cc040c1c40 con 0x55cc03d944d0 2024-05-21T15:35:14.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d71b8b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cc04069580 con 0x55cc03d9c6b0 2024-05-21T15:35:14.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d71b8b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cc040bdc00 con 0x55cc03dbd390 2024-05-21T15:35:14.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d689af640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03d944d0 0x55cc04165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d689af640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03d944d0 0x55cc04165b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40112/0 (socket says 172.21.15.38:40112) 2024-05-21T15:35:14.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d689af640 1 -- 172.21.15.38:0/2353571602 learned_addr learned my addr 172.21.15.38:0/2353571602 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d691b0640 1 --2- 172.21.15.38:0/2353571602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03dbd390 0x55cc04163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d63fff640 1 --2- 172.21.15.38:0/2353571602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d9c6b0 0x55cc041682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d61ffb640 1 --2- 172.21.15.38:0/1159719271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3d540a2b90 0x7f3d540a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d689af640 1 -- 172.21.15.38:0/2353571602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03dbd390 msgr2=0x55cc04163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d689af640 1 --2- 172.21.15.38:0/2353571602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03dbd390 0x55cc04163620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d689af640 1 -- 172.21.15.38:0/2353571602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d9c6b0 msgr2=0x55cc041682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d689af640 1 --2- 172.21.15.38:0/2353571602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d9c6b0 0x55cc041682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d689af640 1 -- 172.21.15.38:0/2353571602 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc03dd53f0 con 0x55cc03d944d0 2024-05-21T15:35:14.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d691b0640 1 --2- 172.21.15.38:0/2353571602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03dbd390 0x55cc04163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:14.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d63fff640 1 --2- 172.21.15.38:0/2353571602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d9c6b0 0x55cc041682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:14.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d61ffb640 1 -- 172.21.15.38:0/1159719271 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f3d500efa20 con 0x7f3d64102bd0 2024-05-21T15:35:14.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d691b0640 1 --2- 172.21.15.38:0/1159719271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3d540a2b90 0x7f3d540a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.822+0000 7f3d689af640 1 --2- 172.21.15.38:0/2353571602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03d944d0 0x55cc04165b70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800bc50 tx=0x7f3d5800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e5a4d3afbc40d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.823+0000 7f3d4effd640 1 -- 172.21.15.38:0/2353571602 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58015070 con 0x55cc03d944d0 2024-05-21T15:35:14.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.823+0000 7f3d4effd640 1 -- 172.21.15.38:0/2353571602 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d580035f0 con 0x55cc03d944d0 2024-05-21T15:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.823+0000 7f3d4effd640 1 -- 172.21.15.38:0/2353571602 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d580116a0 con 0x55cc03d944d0 2024-05-21T15:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.823+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2353571602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03d944d0 msgr2=0x55cc04165b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.823+0000 7f3d691b0640 1 --2- 172.21.15.38:0/1159719271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3d540a2b90 0x7f3d540a50a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009fd0 tx=0x7f3d5c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.823+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2353571602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03d944d0 0x55cc04165b70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800bc50 tx=0x7f3d5800f730 comp rx=0 tx=0).stop 2024-05-21T15:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.823+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2353571602 shutdown_connections 2024-05-21T15:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.823+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2353571602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d9c6b0 0x55cc041682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.823+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2353571602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03d944d0 0x55cc04165b70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.823+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2353571602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03dbd390 0x55cc04163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.823+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2353571602 >> 172.21.15.38:0/2353571602 conn(0x55cc03db14e0 msgr2=0x55cc04157ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.823+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2353571602 shutdown_connections 2024-05-21T15:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.824+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2353571602 wait complete. 2024-05-21T15:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.824+0000 7f3d71b8b740 1 -- start start 2024-05-21T15:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.824+0000 7f3d71b8b740 1 Processor -- start 2024-05-21T15:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.824+0000 7f3d71b8b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 0x55cc0416acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.824+0000 7f3d71b8b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03d9c6b0 0x55cc0416b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.824+0000 7f3d71b8b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03dbd390 0x55cc04172190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.824+0000 7f3d71b8b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cc04168df0 con 0x55cc03d9c6b0 2024-05-21T15:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.824+0000 7f3d71b8b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cc0407cc40 con 0x55cc03d944d0 2024-05-21T15:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.824+0000 7f3d71b8b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cc04168f70 con 0x55cc03dbd390 2024-05-21T15:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d71b8b740 10 client.0 _ll_register_callbacks cb 0x55cc03dd4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d63fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03d9c6b0 0x55cc0416b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d63fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03d9c6b0 0x55cc0416b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40122/0 (socket says 172.21.15.38:40122) 2024-05-21T15:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d63fff640 1 -- 172.21.15.38:0/2945219168 learned_addr learned my addr 172.21.15.38:0/2945219168 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d689af640 1 --2- 172.21.15.38:0/2945219168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 0x55cc0416acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d691b0640 1 --2- 172.21.15.38:0/2945219168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03dbd390 0x55cc04172190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d63fff640 1 -- 172.21.15.38:0/2945219168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03dbd390 msgr2=0x55cc04172190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d63fff640 1 --2- 172.21.15.38:0/2945219168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03dbd390 0x55cc04172190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d63fff640 1 -- 172.21.15.38:0/2945219168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 msgr2=0x55cc0416acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d63fff640 1 --2- 172.21.15.38:0/2945219168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 0x55cc0416acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d63fff640 1 -- 172.21.15.38:0/2945219168 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc03e9f7d0 con 0x55cc03d9c6b0 2024-05-21T15:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d63fff640 1 --2- 172.21.15.38:0/2945219168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03d9c6b0 0x55cc0416b230 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3d500b8c00 tx=0x7f3d50009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69384eec3ae8926 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d43fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d43fff640 1 -- 172.21.15.38:0/2945219168 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50049050 con 0x55cc03d9c6b0 2024-05-21T15:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d43fff640 1 -- 172.21.15.38:0/2945219168 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f3d5003b460 con 0x55cc03d9c6b0 2024-05-21T15:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2945219168 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d580035f0 con 0x55cc03d9c6b0 2024-05-21T15:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d43fff640 1 -- 172.21.15.38:0/2945219168 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d500bbd70 con 0x55cc03d9c6b0 2024-05-21T15:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d71b8b740 10 client.15609 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.825+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2945219168 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cc04175450 con 0x55cc03d9c6b0 2024-05-21T15:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.826+0000 7f3d43fff640 1 -- 172.21.15.38:0/2945219168 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d580035f0 con 0x55cc03d9c6b0 2024-05-21T15:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.826+0000 7f3d43fff640 1 -- 172.21.15.38:0/2945219168 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3d500bc490 con 0x55cc03d9c6b0 2024-05-21T15:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.826+0000 7f3d71b8b740 10 client.15609 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.826+0000 7f3d71b8b740 10 client.15609 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.826+0000 7f3d71b8b740 20 client.15609 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.826+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2945219168 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55cc03d7bf60 0x55cc04178b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.826+0000 7f3d71b8b740 4 client.15609 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.826+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2945219168 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc040cf370 con 0x55cc03d7bf60 2024-05-21T15:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.826+0000 7f3d689af640 1 --2- 172.21.15.38:0/2945219168 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55cc03d7bf60 0x55cc04178b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.827+0000 7f3d43fff640 10 client.15609 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.827+0000 7f3d689af640 1 --2- 172.21.15.38:0/2945219168 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55cc03d7bf60 0x55cc04178b60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2603774ec30e527b server_cookie=329c54fa20ad357a in_seq=0 out_seq=0 2024-05-21T15:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.827+0000 7f3d43fff640 1 -- 172.21.15.38:0/2945219168 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cc040cf370 con 0x55cc03d7bf60 2024-05-21T15:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.827+0000 7f3d43fff640 10 client.15609 handle_command_reply: tid=0 2024-05-21T15:35:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.827+0000 7f3d71b8b740 1 client.15609 shutdown 2024-05-21T15:35:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.828+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2945219168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03d9c6b0 msgr2=0x55cc0416b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.828+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2945219168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03d9c6b0 0x55cc0416b230 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3d500b8c00 tx=0x7f3d50009620 comp rx=0 tx=0).stop 2024-05-21T15:35:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.828+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2945219168 shutdown_connections 2024-05-21T15:35:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.828+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2945219168 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55cc03d7bf60 0x55cc04178b60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.828+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2945219168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03dbd390 0x55cc04172190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.828+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2945219168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03d9c6b0 0x55cc0416b230 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.828+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2945219168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 0x55cc0416acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.828+0000 7f3d43fff640 0 client.15609 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:35:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.828+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2945219168 >> 172.21.15.38:0/2945219168 conn(0x55cc03db14e0 msgr2=0x55cc04157570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.828+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2945219168 shutdown_connections 2024-05-21T15:35:14.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.828+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2945219168 wait complete. 2024-05-21T15:35:14.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.828+0000 7f3d71b8b740 20 client.15609 trim_cache size 0 max 16384 2024-05-21T15:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.841+0000 7f3d71b8b740 1 Processor -- start 2024-05-21T15:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.841+0000 7f3d71b8b740 1 -- start start 2024-05-21T15:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d71b8b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 0x55cc041699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d71b8b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03d9c6b0 0x55cc04161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d71b8b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03dbd390 0x55cc0417d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d71b8b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641a8230 con 0x55cc03dbd390 2024-05-21T15:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d71b8b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641a9d70 con 0x55cc03d944d0 2024-05-21T15:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d71b8b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641a8900 con 0x55cc03d9c6b0 2024-05-21T15:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d691b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03dbd390 0x55cc0417d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d691b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03dbd390 0x55cc0417d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40132/0 (socket says 172.21.15.38:40132) 2024-05-21T15:35:14.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d691b0640 1 -- 172.21.15.38:0/2752950655 learned_addr learned my addr 172.21.15.38:0/2752950655 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d689af640 1 --2- 172.21.15.38:0/2752950655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 0x55cc041699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d63fff640 1 --2- 172.21.15.38:0/2752950655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03d9c6b0 0x55cc04161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d691b0640 1 -- 172.21.15.38:0/2752950655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03d9c6b0 msgr2=0x55cc04161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d691b0640 1 --2- 172.21.15.38:0/2752950655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03d9c6b0 0x55cc04161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d691b0640 1 -- 172.21.15.38:0/2752950655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 msgr2=0x55cc041699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d691b0640 1 --2- 172.21.15.38:0/2752950655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 0x55cc041699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d691b0640 1 -- 172.21.15.38:0/2752950655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc04153d20 con 0x55cc03dbd390 2024-05-21T15:35:14.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.842+0000 7f3d691b0640 1 --2- 172.21.15.38:0/2752950655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03dbd390 0x55cc0417d320 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c011630 tx=0x7f3d5c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2416404fd183c7b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.843+0000 7f3d4e7fc640 1 -- 172.21.15.38:0/2752950655 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c004d20 con 0x55cc03dbd390 2024-05-21T15:35:14.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.843+0000 7f3d4e7fc640 1 -- 172.21.15.38:0/2752950655 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d5c020a50 con 0x55cc03dbd390 2024-05-21T15:35:14.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.843+0000 7f3d4e7fc640 1 -- 172.21.15.38:0/2752950655 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c005720 con 0x55cc03dbd390 2024-05-21T15:35:14.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.843+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2752950655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03dbd390 msgr2=0x55cc0417d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.843+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2752950655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03dbd390 0x55cc0417d320 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c011630 tx=0x7f3d5c016000 comp rx=0 tx=0).stop 2024-05-21T15:35:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.843+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2752950655 shutdown_connections 2024-05-21T15:35:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.843+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2752950655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03dbd390 0x55cc0417d320 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.843+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2752950655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03d9c6b0 0x55cc04161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.843+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2752950655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 0x55cc041699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.843+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2752950655 >> 172.21.15.38:0/2752950655 conn(0x55cc03db14e0 msgr2=0x55cc041692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.843+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2752950655 shutdown_connections 2024-05-21T15:35:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.843+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2752950655 wait complete. 2024-05-21T15:35:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.844+0000 7f3d71b8b740 1 -- start start 2024-05-21T15:35:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.844+0000 7f3d71b8b740 1 Processor -- start 2024-05-21T15:35:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.844+0000 7f3d71b8b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 0x55cc0416eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.844+0000 7f3d71b8b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03d9c6b0 0x55cc0416f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.844+0000 7f3d71b8b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03dbd390 0x55cc0416ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.844+0000 7f3d71b8b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641a96a0 con 0x55cc03dbd390 2024-05-21T15:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.844+0000 7f3d71b8b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641a8fd0 con 0x55cc03d944d0 2024-05-21T15:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.844+0000 7f3d71b8b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641a6210 con 0x55cc03d9c6b0 2024-05-21T15:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.844+0000 7f3d71b8b740 10 client.0 _ll_register_callbacks cb 0x7f3d50004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.844+0000 7f3d691b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03dbd390 0x55cc0416ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.844+0000 7f3d691b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03dbd390 0x55cc0416ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40146/0 (socket says 172.21.15.38:40146) 2024-05-21T15:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.844+0000 7f3d691b0640 1 -- 172.21.15.38:0/2318913518 learned_addr learned my addr 172.21.15.38:0/2318913518 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.844+0000 7f3d689af640 1 --2- 172.21.15.38:0/2318913518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 0x55cc0416eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d63fff640 1 --2- 172.21.15.38:0/2318913518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03d9c6b0 0x55cc0416f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d689af640 1 -- 172.21.15.38:0/2318913518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03d9c6b0 msgr2=0x55cc0416f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d689af640 1 --2- 172.21.15.38:0/2318913518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03d9c6b0 0x55cc0416f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d689af640 1 -- 172.21.15.38:0/2318913518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03dbd390 msgr2=0x55cc0416ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d689af640 1 --2- 172.21.15.38:0/2318913518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03dbd390 0x55cc0416ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d689af640 1 -- 172.21.15.38:0/2318913518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc04170520 con 0x55cc03d944d0 2024-05-21T15:35:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d691b0640 1 --2- 172.21.15.38:0/2318913518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03dbd390 0x55cc0416ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d689af640 1 --2- 172.21.15.38:0/2318913518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 0x55cc0416eb60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800fc30 tx=0x7f3d58005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a518df107358a2cf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d43fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:35:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d43fff640 1 -- 172.21.15.38:0/2318913518 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5802b070 con 0x55cc03d944d0 2024-05-21T15:35:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d43fff640 1 -- 172.21.15.38:0/2318913518 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f3d58004150 con 0x55cc03d944d0 2024-05-21T15:35:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2318913518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d5c020a50 con 0x55cc03d944d0 2024-05-21T15:35:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d43fff640 1 -- 172.21.15.38:0/2318913518 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d58037050 con 0x55cc03d944d0 2024-05-21T15:35:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d71b8b740 10 client.25337 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.845+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2318913518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cc04171810 con 0x55cc03d944d0 2024-05-21T15:35:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.846+0000 7f3d43fff640 1 -- 172.21.15.38:0/2318913518 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d5c020a50 con 0x55cc03d944d0 2024-05-21T15:35:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.846+0000 7f3d43fff640 1 -- 172.21.15.38:0/2318913518 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3d58027550 con 0x55cc03d944d0 2024-05-21T15:35:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.846+0000 7f3d71b8b740 10 client.25337 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:14.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.846+0000 7f3d71b8b740 10 client.25337 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:35:14.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.846+0000 7f3d71b8b740 20 client.25337 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:14.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.846+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2318913518 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55cc03d513a0 0x55cc041e30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.846+0000 7f3d71b8b740 4 client.25337 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:35:14.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.846+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2318913518 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc040c96c0 con 0x55cc03d513a0 2024-05-21T15:35:14.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.846+0000 7f3d63fff640 1 --2- 172.21.15.38:0/2318913518 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55cc03d513a0 0x55cc041e30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d43fff640 10 client.25337 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:35:14.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d63fff640 1 --2- 172.21.15.38:0/2318913518 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55cc03d513a0 0x55cc041e30e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=54308d762e6da310 server_cookie=75478c55838011c2 in_seq=0 out_seq=0 2024-05-21T15:35:14.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d43fff640 1 -- 172.21.15.38:0/2318913518 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc040c96c0 con 0x55cc03d513a0 2024-05-21T15:35:14.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d43fff640 10 client.25337 handle_command_reply: tid=0 2024-05-21T15:35:14.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d71b8b740 1 client.25337 shutdown 2024-05-21T15:35:14.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2318913518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 msgr2=0x55cc0416eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2318913518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 0x55cc0416eb60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800fc30 tx=0x7f3d58005720 comp rx=0 tx=0).stop 2024-05-21T15:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2318913518 shutdown_connections 2024-05-21T15:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2318913518 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55cc03d513a0 0x55cc041e30e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2318913518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc03dbd390 0x55cc0416ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2318913518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc03d9c6b0 0x55cc0416f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d71b8b740 1 --2- 172.21.15.38:0/2318913518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc03d944d0 0x55cc0416eb60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d43fff640 0 client.25337 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2318913518 >> 172.21.15.38:0/2318913518 conn(0x55cc03db14e0 msgr2=0x55cc04160410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2318913518 shutdown_connections 2024-05-21T15:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d71b8b740 1 -- 172.21.15.38:0/2318913518 wait complete. 2024-05-21T15:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.847+0000 7f3d71b8b740 20 client.25337 trim_cache size 0 max 16384 2024-05-21T15:35:14.852 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.848+0000 7f3d6ac13640 1 -- 172.21.15.38:0/1159719271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3d540a2b90 msgr2=0x7f3d540a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.848+0000 7f3d6ac13640 1 --2- 172.21.15.38:0/1159719271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3d540a2b90 0x7f3d540a50a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009fd0 tx=0x7f3d5c004a40 comp rx=0 tx=0).stop 2024-05-21T15:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.848+0000 7f3d6ac13640 1 -- 172.21.15.38:0/1159719271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d64102bd0 msgr2=0x7f3d64198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.848+0000 7f3d6ac13640 1 --2- 172.21.15.38:0/1159719271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d64102bd0 0x7f3d64198690 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006010 tx=0x7f3d50004360 comp rx=0 tx=0).stop 2024-05-21T15:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.848+0000 7f3d6ac13640 1 -- 172.21.15.38:0/1159719271 shutdown_connections 2024-05-21T15:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.848+0000 7f3d6ac13640 1 --2- 172.21.15.38:0/1159719271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d641078d0 0x7f3d6419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.848+0000 7f3d6ac13640 1 --2- 172.21.15.38:0/1159719271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3d540a2b90 0x7f3d540a50a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.848+0000 7f3d6ac13640 1 --2- 172.21.15.38:0/1159719271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d64102bd0 0x7f3d64198690 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.848+0000 7f3d6ac13640 1 --2- 172.21.15.38:0/1159719271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d64101990 0x7f3d64198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.848+0000 7f3d6ac13640 1 -- 172.21.15.38:0/1159719271 >> 172.21.15.38:0/1159719271 conn(0x7f3d640fcb80 msgr2=0x7f3d641038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.848+0000 7f3d6ac13640 1 -- 172.21.15.38:0/1159719271 shutdown_connections 2024-05-21T15:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.848+0000 7f3d6ac13640 1 -- 172.21.15.38:0/1159719271 wait complete. 2024-05-21T15:35:14.858 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.976+0000 7f038f86c640 1 -- 172.21.15.38:0/3395012282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0388101980 msgr2=0x7f0388102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.976+0000 7f038f86c640 1 --2- 172.21.15.38:0/3395012282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0388101980 0x7f0388102680 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0384009900 tx=0x7f038402a410 comp rx=0 tx=0).stop 2024-05-21T15:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.976+0000 7f038f86c640 1 -- 172.21.15.38:0/3395012282 shutdown_connections 2024-05-21T15:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.976+0000 7f038f86c640 1 --2- 172.21.15.38:0/3395012282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03881078c0 0x7f0388109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.976+0000 7f038f86c640 1 --2- 172.21.15.38:0/3395012282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0388102bc0 0x7f0388102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.976+0000 7f038f86c640 1 --2- 172.21.15.38:0/3395012282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0388101980 0x7f0388102680 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.976+0000 7f038f86c640 1 -- 172.21.15.38:0/3395012282 >> 172.21.15.38:0/3395012282 conn(0x7f03880fcb70 msgr2=0x7f03880ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.976+0000 7f038f86c640 1 -- 172.21.15.38:0/3395012282 shutdown_connections 2024-05-21T15:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.976+0000 7f038f86c640 1 -- 172.21.15.38:0/3395012282 wait complete. 2024-05-21T15:35:14.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.977+0000 7f038f86c640 1 Processor -- start 2024-05-21T15:35:14.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.977+0000 7f038f86c640 1 -- start start 2024-05-21T15:35:14.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.977+0000 7f038f86c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0388101980 0x7f0388198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.977+0000 7f038f86c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0388102bc0 0x7f03881986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.977+0000 7f038f86c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03881078c0 0x7f038819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.977+0000 7f038f86c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f038810d5b0 con 0x7f03881078c0 2024-05-21T15:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.977+0000 7f038f86c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f038810d430 con 0x7f0388102bc0 2024-05-21T15:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.977+0000 7f038f86c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f038810d730 con 0x7f0388101980 2024-05-21T15:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.977+0000 7f038de09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03881078c0 0x7f038819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.977+0000 7f038de09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03881078c0 0x7f038819f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40162/0 (socket says 172.21.15.38:40162) 2024-05-21T15:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.977+0000 7f038de09640 1 -- 172.21.15.38:0/2425456391 learned_addr learned my addr 172.21.15.38:0/2425456391 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.977+0000 7f038ce07640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0388102bc0 0x7f03881986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.977+0000 7f038d608640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0388101980 0x7f0388198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.978+0000 7f038de09640 1 -- 172.21.15.38:0/2425456391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0388101980 msgr2=0x7f0388198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.978+0000 7f038de09640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0388101980 0x7f0388198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.978+0000 7f038de09640 1 -- 172.21.15.38:0/2425456391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0388102bc0 msgr2=0x7f03881986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.978+0000 7f038de09640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0388102bc0 0x7f03881986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.978+0000 7f038de09640 1 -- 172.21.15.38:0/2425456391 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038819fcc0 con 0x7f03881078c0 2024-05-21T15:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.978+0000 7f038d608640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0388101980 0x7f0388198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:14.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.978+0000 7f038de09640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03881078c0 0x7f038819f650 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0380009fd0 tx=0x7f0380007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.978+0000 7f037e7fc640 1 -- 172.21.15.38:0/2425456391 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038000de10 con 0x7f03881078c0 2024-05-21T15:35:14.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.978+0000 7f037e7fc640 1 -- 172.21.15.38:0/2425456391 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0380003e50 con 0x7f03881078c0 2024-05-21T15:35:14.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.978+0000 7f038f86c640 1 -- 172.21.15.38:0/2425456391 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0388069010 con 0x7f03881078c0 2024-05-21T15:35:14.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.978+0000 7f037e7fc640 1 -- 172.21.15.38:0/2425456391 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380018420 con 0x7f03881078c0 2024-05-21T15:35:14.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.978+0000 7f038f86c640 1 -- 172.21.15.38:0/2425456391 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0388069560 con 0x7f03881078c0 2024-05-21T15:35:14.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f037e7fc640 1 -- 172.21.15.38:0/2425456391 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0380003970 con 0x7f03881078c0 2024-05-21T15:35:14.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f03967ed740 1 Processor -- start 2024-05-21T15:35:14.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f03967ed740 1 -- start start 2024-05-21T15:35:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f03967ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe2f390 0x55d8901d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f03967ed740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe064d0 0x55d8901d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f03967ed740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 0x55d8901da390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f03967ed740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d890133c40 con 0x55d88fe064d0 2024-05-21T15:35:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f03967ed740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d8900db580 con 0x55d88fe0e6b0 2024-05-21T15:35:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f03967ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d89012fc00 con 0x55d88fe2f390 2024-05-21T15:35:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f038d608640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe064d0 0x55d8901d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f038d608640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe064d0 0x55d8901d3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40164/0 (socket says 172.21.15.38:40164) 2024-05-21T15:35:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f038d608640 1 -- 172.21.15.38:0/1219747565 learned_addr learned my addr 172.21.15.38:0/1219747565 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f038ce07640 1 --2- 172.21.15.38:0/1219747565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe2f390 0x55d8901d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f038ce07640 1 --2- 172.21.15.38:0/1219747565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 0x55d8901da390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f038d608640 1 -- 172.21.15.38:0/1219747565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe2f390 msgr2=0x55d8901d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f038d608640 1 --2- 172.21.15.38:0/1219747565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe2f390 0x55d8901d2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f038d608640 1 -- 172.21.15.38:0/1219747565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 msgr2=0x55d8901da390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f038d608640 1 --2- 172.21.15.38:0/1219747565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 0x55d8901da390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f038d608640 1 -- 172.21.15.38:0/1219747565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88fe473f0 con 0x55d88fe064d0 2024-05-21T15:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.981+0000 7f038ce07640 1 --2- 172.21.15.38:0/1219747565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 0x55d8901da390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.982+0000 7f037e7fc640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f03740a2b90 0x7f03740a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.982+0000 7f038d608640 1 --2- 172.21.15.38:0/1219747565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe064d0 0x55d8901d3490 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0384031550 tx=0x7f0384009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02af20e864ab94c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.982+0000 7f038ce07640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f03740a2b90 0x7f03740a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.982+0000 7f037e7fc640 1 -- 172.21.15.38:0/2425456391 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f03800d0790 con 0x7f03881078c0 2024-05-21T15:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.982+0000 7f03737fe640 1 -- 172.21.15.38:0/1219747565 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038403f070 con 0x55d88fe064d0 2024-05-21T15:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.982+0000 7f03737fe640 1 -- 172.21.15.38:0/1219747565 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03840318b0 con 0x55d88fe064d0 2024-05-21T15:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.982+0000 7f03737fe640 1 -- 172.21.15.38:0/1219747565 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384035070 con 0x55d88fe064d0 2024-05-21T15:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.983+0000 7f03967ed740 1 -- 172.21.15.38:0/1219747565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe064d0 msgr2=0x55d8901d3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.983+0000 7f03967ed740 1 --2- 172.21.15.38:0/1219747565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe064d0 0x55d8901d3490 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0384031550 tx=0x7f0384009680 comp rx=0 tx=0).stop 2024-05-21T15:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.983+0000 7f038ce07640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f03740a2b90 0x7f03740a50a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55d8901d40e0 tx=0x7f037800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.983+0000 7f03967ed740 1 -- 172.21.15.38:0/1219747565 shutdown_connections 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.983+0000 7f03967ed740 1 --2- 172.21.15.38:0/1219747565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 0x55d8901da390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.983+0000 7f03967ed740 1 --2- 172.21.15.38:0/1219747565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe064d0 0x55d8901d3490 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.983+0000 7f03967ed740 1 --2- 172.21.15.38:0/1219747565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe2f390 0x55d8901d2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.983+0000 7f03967ed740 1 -- 172.21.15.38:0/1219747565 >> 172.21.15.38:0/1219747565 conn(0x55d88fe234e0 msgr2=0x55d8901ca490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.983+0000 7f03967ed740 1 -- 172.21.15.38:0/1219747565 shutdown_connections 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.983+0000 7f03967ed740 1 -- 172.21.15.38:0/1219747565 wait complete. 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.983+0000 7f03967ed740 1 -- start start 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.983+0000 7f03967ed740 1 Processor -- start 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f03967ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe064d0 0x55d8901dcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f03967ed740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 0x55d8901dd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f03967ed740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe2f390 0x55d8901e41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f03967ed740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d8901dad80 con 0x55d88fe2f390 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f03967ed740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d8900eec40 con 0x55d88fe0e6b0 2024-05-21T15:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f03967ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d8901daf00 con 0x55d88fe064d0 2024-05-21T15:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f03967ed740 10 client.0 _ll_register_callbacks cb 0x55d88fe46be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f038de09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe2f390 0x55d8901e41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f038de09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe2f390 0x55d8901e41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40180/0 (socket says 172.21.15.38:40180) 2024-05-21T15:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f038de09640 1 -- 172.21.15.38:0/2346827933 learned_addr learned my addr 172.21.15.38:0/2346827933 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f038ce07640 1 --2- 172.21.15.38:0/2346827933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 0x55d8901dd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f038d608640 1 --2- 172.21.15.38:0/2346827933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe064d0 0x55d8901dcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f038de09640 1 -- 172.21.15.38:0/2346827933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe064d0 msgr2=0x55d8901dcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f038de09640 1 --2- 172.21.15.38:0/2346827933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe064d0 0x55d8901dcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f038de09640 1 -- 172.21.15.38:0/2346827933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 msgr2=0x55d8901dd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f038de09640 1 --2- 172.21.15.38:0/2346827933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 0x55d8901dd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f038de09640 1 -- 172.21.15.38:0/2346827933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88ff117d0 con 0x55d88fe2f390 2024-05-21T15:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f038de09640 1 --2- 172.21.15.38:0/2346827933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe2f390 0x55d8901e41e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0380099400 tx=0x7f038009cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c144281bbe3d012e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.984+0000 7f0370ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.985+0000 7f0370ff9640 1 -- 172.21.15.38:0/2346827933 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038009cfa0 con 0x55d88fe2f390 2024-05-21T15:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.985+0000 7f0370ff9640 1 -- 172.21.15.38:0/2346827933 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f038009b2a0 con 0x55d88fe2f390 2024-05-21T15:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.985+0000 7f03967ed740 1 -- 172.21.15.38:0/2346827933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03840318b0 con 0x55d88fe2f390 2024-05-21T15:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.985+0000 7f0370ff9640 1 -- 172.21.15.38:0/2346827933 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03800a1350 con 0x55d88fe2f390 2024-05-21T15:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.985+0000 7f03967ed740 10 client.15627 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.985+0000 7f03967ed740 1 -- 172.21.15.38:0/2346827933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8901e7570 con 0x55d88fe2f390 2024-05-21T15:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.985+0000 7f0370ff9640 1 -- 172.21.15.38:0/2346827933 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03840318b0 con 0x55d88fe2f390 2024-05-21T15:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.985+0000 7f0370ff9640 1 -- 172.21.15.38:0/2346827933 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f03800ba050 con 0x55d88fe2f390 2024-05-21T15:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.985+0000 7f03967ed740 10 client.15627 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.985+0000 7f03967ed740 10 client.15627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.985+0000 7f03967ed740 20 client.15627 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.985+0000 7f03967ed740 1 --2- 172.21.15.38:0/2346827933 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55d88fdedf60 0x55d8901eac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.985+0000 7f03967ed740 4 client.15627 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.985+0000 7f03967ed740 1 -- 172.21.15.38:0/2346827933 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d890141370 con 0x55d88fdedf60 2024-05-21T15:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.986+0000 7f038d608640 1 --2- 172.21.15.38:0/2346827933 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55d88fdedf60 0x55d8901eac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.986+0000 7f038d608640 1 --2- 172.21.15.38:0/2346827933 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55d88fdedf60 0x55d8901eac90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c30c2f9b9dfbfd7 server_cookie=a6e2cd034c7a843d in_seq=0 out_seq=0 2024-05-21T15:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.986+0000 7f0370ff9640 10 client.15627 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f0370ff9640 1 -- 172.21.15.38:0/2346827933 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d890141370 con 0x55d88fdedf60 2024-05-21T15:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f0370ff9640 10 client.15627 handle_command_reply: tid=0 2024-05-21T15:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f03967ed740 1 client.15627 shutdown 2024-05-21T15:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f03967ed740 1 -- 172.21.15.38:0/2346827933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe2f390 msgr2=0x55d8901e41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f03967ed740 1 --2- 172.21.15.38:0/2346827933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe2f390 0x55d8901e41e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0380099400 tx=0x7f038009cb60 comp rx=0 tx=0).stop 2024-05-21T15:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f03967ed740 1 -- 172.21.15.38:0/2346827933 shutdown_connections 2024-05-21T15:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f03967ed740 1 --2- 172.21.15.38:0/2346827933 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55d88fdedf60 0x55d8901eac90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f03967ed740 1 --2- 172.21.15.38:0/2346827933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe2f390 0x55d8901e41e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f0370ff9640 0 client.15627 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f03967ed740 1 --2- 172.21.15.38:0/2346827933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 0x55d8901dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f03967ed740 1 --2- 172.21.15.38:0/2346827933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe064d0 0x55d8901dcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f03967ed740 1 -- 172.21.15.38:0/2346827933 >> 172.21.15.38:0/2346827933 conn(0x55d88fe234e0 msgr2=0x55d8901c9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f03967ed740 1 -- 172.21.15.38:0/2346827933 shutdown_connections 2024-05-21T15:35:14.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f03967ed740 1 -- 172.21.15.38:0/2346827933 wait complete. 2024-05-21T15:35:14.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:14.987+0000 7f03967ed740 20 client.15627 trim_cache size 0 max 16384 2024-05-21T15:35:15.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f03967ed740 1 Processor -- start 2024-05-21T15:35:15.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f03967ed740 1 -- start start 2024-05-21T15:35:15.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f03967ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe064d0 0x55d8901db360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f03967ed740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 0x55d8901e4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f03967ed740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe2f390 0x55d8901db8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f03967ed740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f03881a80d0 con 0x55d88fe2f390 2024-05-21T15:35:15.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f03967ed740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03881a9c10 con 0x55d88fe0e6b0 2024-05-21T15:35:15.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f03967ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f03881a87a0 con 0x55d88fe064d0 2024-05-21T15:35:15.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f038de09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe2f390 0x55d8901db8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f038de09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe2f390 0x55d8901db8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40190/0 (socket says 172.21.15.38:40190) 2024-05-21T15:35:15.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f038de09640 1 -- 172.21.15.38:0/3619974006 learned_addr learned my addr 172.21.15.38:0/3619974006 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:15.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f038ce07640 1 --2- 172.21.15.38:0/3619974006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 0x55d8901e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f038d608640 1 --2- 172.21.15.38:0/3619974006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe064d0 0x55d8901db360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f038de09640 1 -- 172.21.15.38:0/3619974006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe064d0 msgr2=0x55d8901db360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f038de09640 1 --2- 172.21.15.38:0/3619974006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe064d0 0x55d8901db360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f038de09640 1 -- 172.21.15.38:0/3619974006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 msgr2=0x55d8901e4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.001+0000 7f038de09640 1 --2- 172.21.15.38:0/3619974006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 0x55d8901e4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.002+0000 7f038de09640 1 -- 172.21.15.38:0/3619974006 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8901c5d20 con 0x55d88fe2f390 2024-05-21T15:35:15.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.002+0000 7f038d608640 1 --2- 172.21.15.38:0/3619974006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe064d0 0x55d8901db360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:15.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.002+0000 7f038ce07640 1 --2- 172.21.15.38:0/3619974006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 0x55d8901e4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:15.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.002+0000 7f038de09640 1 --2- 172.21.15.38:0/3619974006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe2f390 0x55d8901db8a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f03800ce5b0 tx=0x7f0380007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707f48b026fece83 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:15.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.002+0000 7f0372ffd640 1 -- 172.21.15.38:0/3619974006 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038009c6e0 con 0x55d88fe2f390 2024-05-21T15:35:15.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.002+0000 7f0372ffd640 1 -- 172.21.15.38:0/3619974006 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0380080020 con 0x55d88fe2f390 2024-05-21T15:35:15.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.002+0000 7f0372ffd640 1 -- 172.21.15.38:0/3619974006 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03800bbbe0 con 0x55d88fe2f390 2024-05-21T15:35:15.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.003+0000 7f03967ed740 1 -- 172.21.15.38:0/3619974006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe2f390 msgr2=0x55d8901db8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.003+0000 7f03967ed740 1 --2- 172.21.15.38:0/3619974006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe2f390 0x55d8901db8a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f03800ce5b0 tx=0x7f0380007f40 comp rx=0 tx=0).stop 2024-05-21T15:35:15.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.003+0000 7f03967ed740 1 -- 172.21.15.38:0/3619974006 shutdown_connections 2024-05-21T15:35:15.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.003+0000 7f03967ed740 1 --2- 172.21.15.38:0/3619974006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe2f390 0x55d8901db8a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.003+0000 7f03967ed740 1 --2- 172.21.15.38:0/3619974006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe0e6b0 0x55d8901e4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.003+0000 7f03967ed740 1 --2- 172.21.15.38:0/3619974006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe064d0 0x55d8901db360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.003+0000 7f03967ed740 1 -- 172.21.15.38:0/3619974006 >> 172.21.15.38:0/3619974006 conn(0x55d88fe234e0 msgr2=0x55d8901d7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.003+0000 7f03967ed740 1 -- 172.21.15.38:0/3619974006 shutdown_connections 2024-05-21T15:35:15.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.003+0000 7f03967ed740 1 -- 172.21.15.38:0/3619974006 wait complete. 2024-05-21T15:35:15.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.003+0000 7f03967ed740 1 -- start start 2024-05-21T15:35:15.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.003+0000 7f03967ed740 1 Processor -- start 2024-05-21T15:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f03967ed740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe064d0 0x55d8901e1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f03967ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe0e6b0 0x55d8901e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f03967ed740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe2f390 0x55d8901e2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f03967ed740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f03881a9540 con 0x55d88fe064d0 2024-05-21T15:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f03967ed740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03881a8e70 con 0x55d88fe2f390 2024-05-21T15:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f03967ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f03881a60b0 con 0x55d88fe0e6b0 2024-05-21T15:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f03967ed740 10 client.0 _ll_register_callbacks cb 0x7f0380004240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f038d608640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe064d0 0x55d8901e1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f038d608640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe064d0 0x55d8901e1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40206/0 (socket says 172.21.15.38:40206) 2024-05-21T15:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f038d608640 1 -- 172.21.15.38:0/3627045610 learned_addr learned my addr 172.21.15.38:0/3627045610 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f038ce07640 1 --2- 172.21.15.38:0/3627045610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe0e6b0 0x55d8901e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f038de09640 1 --2- 172.21.15.38:0/3627045610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe2f390 0x55d8901e2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f038d608640 1 -- 172.21.15.38:0/3627045610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe0e6b0 msgr2=0x55d8901e1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f038d608640 1 --2- 172.21.15.38:0/3627045610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe0e6b0 0x55d8901e1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f038d608640 1 -- 172.21.15.38:0/3627045610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe2f390 msgr2=0x55d8901e2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f038d608640 1 --2- 172.21.15.38:0/3627045610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe2f390 0x55d8901e2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f038d608640 1 -- 172.21.15.38:0/3627045610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8901e30e0 con 0x55d88fe064d0 2024-05-21T15:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f038de09640 1 --2- 172.21.15.38:0/3627045610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe2f390 0x55d8901e2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f038d608640 1 --2- 172.21.15.38:0/3627045610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe064d0 0x55d8901e1700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0384005940 tx=0x7f0384032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb62bc0bb105e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f0370ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f0370ff9640 1 -- 172.21.15.38:0/3627045610 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384053060 con 0x55d88fe064d0 2024-05-21T15:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f0370ff9640 1 -- 172.21.15.38:0/3627045610 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f0384004230 con 0x55d88fe064d0 2024-05-21T15:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.004+0000 7f03967ed740 1 -- 172.21.15.38:0/3627045610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0380080020 con 0x55d88fe064d0 2024-05-21T15:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.005+0000 7f0370ff9640 1 -- 172.21.15.38:0/3627045610 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0384035070 con 0x55d88fe064d0 2024-05-21T15:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.005+0000 7f03967ed740 10 client.15633 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.005+0000 7f03967ed740 1 -- 172.21.15.38:0/3627045610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8901e4310 con 0x55d88fe064d0 2024-05-21T15:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.005+0000 7f0370ff9640 1 -- 172.21.15.38:0/3627045610 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0380080020 con 0x55d88fe064d0 2024-05-21T15:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.005+0000 7f0370ff9640 1 -- 172.21.15.38:0/3627045610 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f038404e5a0 con 0x55d88fe064d0 2024-05-21T15:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.005+0000 7f03967ed740 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.005+0000 7f03967ed740 10 client.15633 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.005+0000 7f03967ed740 20 client.15633 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.005+0000 7f03967ed740 1 --2- 172.21.15.38:0/3627045610 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55d88fdc33a0 0x55d890255290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.005+0000 7f03967ed740 4 client.15633 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.005+0000 7f03967ed740 1 -- 172.21.15.38:0/3627045610 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d89013b6c0 con 0x55d88fdc33a0 2024-05-21T15:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.005+0000 7f038ce07640 1 --2- 172.21.15.38:0/3627045610 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55d88fdc33a0 0x55d890255290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f038ce07640 1 --2- 172.21.15.38:0/3627045610 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55d88fdc33a0 0x55d890255290 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc98fe630e69e608 server_cookie=c61b124b9e334805 in_seq=0 out_seq=0 2024-05-21T15:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f0370ff9640 10 client.15633 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f0370ff9640 1 -- 172.21.15.38:0/3627045610 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d89013b6c0 con 0x55d88fdc33a0 2024-05-21T15:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f0370ff9640 10 client.15633 handle_command_reply: tid=0 2024-05-21T15:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f03967ed740 1 client.15633 shutdown 2024-05-21T15:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f03967ed740 1 -- 172.21.15.38:0/3627045610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe064d0 msgr2=0x55d8901e1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f03967ed740 1 --2- 172.21.15.38:0/3627045610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe064d0 0x55d8901e1700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0384005940 tx=0x7f0384032eb0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f03967ed740 1 -- 172.21.15.38:0/3627045610 shutdown_connections 2024-05-21T15:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f03967ed740 1 --2- 172.21.15.38:0/3627045610 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55d88fdc33a0 0x55d890255290 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f03967ed740 1 --2- 172.21.15.38:0/3627045610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d88fe2f390 0x55d8901e2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f0370ff9640 0 client.15633 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f03967ed740 1 --2- 172.21.15.38:0/3627045610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d88fe0e6b0 0x55d8901e1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f03967ed740 1 --2- 172.21.15.38:0/3627045610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d88fe064d0 0x55d8901e1700 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f03967ed740 1 -- 172.21.15.38:0/3627045610 >> 172.21.15.38:0/3627045610 conn(0x55d88fe234e0 msgr2=0x55d8901db260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f03967ed740 1 -- 172.21.15.38:0/3627045610 shutdown_connections 2024-05-21T15:35:15.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f03967ed740 1 -- 172.21.15.38:0/3627045610 wait complete. 2024-05-21T15:35:15.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.006+0000 7f03967ed740 20 client.15633 trim_cache size 0 max 16384 2024-05-21T15:35:15.013 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:35:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.007+0000 7f038f86c640 1 -- 172.21.15.38:0/2425456391 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f03740a2b90 msgr2=0x7f03740a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.007+0000 7f038f86c640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f03740a2b90 0x7f03740a50a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55d8901d40e0 tx=0x7f037800b390 comp rx=0 tx=0).stop 2024-05-21T15:35:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.007+0000 7f038f86c640 1 -- 172.21.15.38:0/2425456391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03881078c0 msgr2=0x7f038819f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.007+0000 7f038f86c640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03881078c0 0x7f038819f650 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0380009fd0 tx=0x7f0380007590 comp rx=0 tx=0).stop 2024-05-21T15:35:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.007+0000 7f038f86c640 1 -- 172.21.15.38:0/2425456391 shutdown_connections 2024-05-21T15:35:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.007+0000 7f038f86c640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f03740a2b90 0x7f03740a50a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.007+0000 7f038f86c640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03881078c0 0x7f038819f650 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.007+0000 7f038f86c640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0388102bc0 0x7f03881986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.007+0000 7f038f86c640 1 --2- 172.21.15.38:0/2425456391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0388101980 0x7f0388198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.007+0000 7f038f86c640 1 -- 172.21.15.38:0/2425456391 >> 172.21.15.38:0/2425456391 conn(0x7f03880fcb70 msgr2=0x7f03881038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.007+0000 7f038f86c640 1 -- 172.21.15.38:0/2425456391 shutdown_connections 2024-05-21T15:35:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.007+0000 7f038f86c640 1 -- 172.21.15.38:0/2425456391 wait complete. 2024-05-21T15:35:15.017 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:35:15.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.134+0000 7fa112102640 1 -- 172.21.15.38:0/1432482736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10c101ef0 msgr2=0x7fa10c102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.134+0000 7fa112102640 1 --2- 172.21.15.38:0/1432482736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10c101ef0 0x7fa10c102300 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009900 tx=0x7fa0fc02a410 comp rx=0 tx=0).stop 2024-05-21T15:35:15.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.135+0000 7fa112102640 1 -- 172.21.15.38:0/1432482736 shutdown_connections 2024-05-21T15:35:15.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.135+0000 7fa112102640 1 --2- 172.21.15.38:0/1432482736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c103410 0x7fa10c109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.135+0000 7fa112102640 1 --2- 172.21.15.38:0/1432482736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa10c1028d0 0x7fa10c102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.135+0000 7fa112102640 1 --2- 172.21.15.38:0/1432482736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10c101ef0 0x7fa10c102300 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.135+0000 7fa112102640 1 -- 172.21.15.38:0/1432482736 >> 172.21.15.38:0/1432482736 conn(0x7fa10c0fc330 msgr2=0x7fa10c0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.135+0000 7fa112102640 1 -- 172.21.15.38:0/1432482736 shutdown_connections 2024-05-21T15:35:15.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.135+0000 7fa112102640 1 -- 172.21.15.38:0/1432482736 wait complete. 2024-05-21T15:35:15.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.135+0000 7fa112102640 1 Processor -- start 2024-05-21T15:35:15.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.135+0000 7fa112102640 1 -- start start 2024-05-21T15:35:15.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa112102640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c101ef0 0x7fa10c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa112102640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa10c1028d0 0x7fa10c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa112102640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10c103410 0x7fa10c1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa112102640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c10d640 con 0x7fa10c103410 2024-05-21T15:35:15.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa112102640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c10d4c0 con 0x7fa10c1028d0 2024-05-21T15:35:15.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa112102640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c10d7c0 con 0x7fa10c101ef0 2024-05-21T15:35:15.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa10bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10c103410 0x7fa10c1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa10bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10c103410 0x7fa10c1a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40222/0 (socket says 172.21.15.38:40222) 2024-05-21T15:35:15.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa10bfff640 1 -- 172.21.15.38:0/1590148918 learned_addr learned my addr 172.21.15.38:0/1590148918 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:15.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa10affd640 1 --2- 172.21.15.38:0/1590148918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa10c1028d0 0x7fa10c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa10b7fe640 1 --2- 172.21.15.38:0/1590148918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c101ef0 0x7fa10c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa10bfff640 1 -- 172.21.15.38:0/1590148918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c101ef0 msgr2=0x7fa10c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa10bfff640 1 --2- 172.21.15.38:0/1590148918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c101ef0 0x7fa10c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa10bfff640 1 -- 172.21.15.38:0/1590148918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa10c1028d0 msgr2=0x7fa10c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa10bfff640 1 --2- 172.21.15.38:0/1590148918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa10c1028d0 0x7fa10c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa10bfff640 1 -- 172.21.15.38:0/1590148918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10c1a0da0 con 0x7fa10c103410 2024-05-21T15:35:15.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa10bfff640 1 --2- 172.21.15.38:0/1590148918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10c103410 0x7fa10c1a0730 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa10400eaa0 tx=0x7fa10400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:15.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.136+0000 7fa108ff9640 1 -- 172.21.15.38:0/1590148918 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10400cdb0 con 0x7fa10c103410 2024-05-21T15:35:15.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.137+0000 7fa108ff9640 1 -- 172.21.15.38:0/1590148918 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa104005020 con 0x7fa10c103410 2024-05-21T15:35:15.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.137+0000 7fa108ff9640 1 -- 172.21.15.38:0/1590148918 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1040116b0 con 0x7fa10c103410 2024-05-21T15:35:15.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.137+0000 7fa112102640 1 -- 172.21.15.38:0/1590148918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10c1a1090 con 0x7fa10c103410 2024-05-21T15:35:15.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.137+0000 7fa112102640 1 -- 172.21.15.38:0/1590148918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa10c1a1630 con 0x7fa10c103410 2024-05-21T15:35:15.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.139+0000 7fa108ff9640 1 -- 172.21.15.38:0/1590148918 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa104005560 con 0x7fa10c103410 2024-05-21T15:35:15.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.139+0000 7fa118fa9740 1 Processor -- start 2024-05-21T15:35:15.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.139+0000 7fa118fa9740 1 -- start start 2024-05-21T15:35:15.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa118fa9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 0x561422507b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa118fa9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56142213f4d0 0x561422508080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa118fa9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614221476b0 0x56142250ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa118fa9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56142246cc40 con 0x56142213f4d0 2024-05-21T15:35:15.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa118fa9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561422414580 con 0x561422168390 2024-05-21T15:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa118fa9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561422468c00 con 0x5614221476b0 2024-05-21T15:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa10b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56142213f4d0 0x561422508080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa10b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56142213f4d0 0x561422508080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40226/0 (socket says 172.21.15.38:40226) 2024-05-21T15:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa10b7fe640 1 -- 172.21.15.38:0/928453030 learned_addr learned my addr 172.21.15.38:0/928453030 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa10affd640 1 --2- 172.21.15.38:0/928453030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614221476b0 0x56142250ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa10affd640 1 --2- 172.21.15.38:0/928453030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 0x561422507b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa10b7fe640 1 -- 172.21.15.38:0/928453030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614221476b0 msgr2=0x56142250ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa10b7fe640 1 --2- 172.21.15.38:0/928453030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614221476b0 0x56142250ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa10b7fe640 1 -- 172.21.15.38:0/928453030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 msgr2=0x561422507b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa10b7fe640 1 --2- 172.21.15.38:0/928453030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 0x561422507b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa108ff9640 1 --2- 172.21.15.38:0/1590148918 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0f40a2c10 0x7fa0f40a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa10b7fe640 1 -- 172.21.15.38:0/928453030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614221803f0 con 0x56142213f4d0 2024-05-21T15:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa108ff9640 1 -- 172.21.15.38:0/1590148918 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa1040cf1c0 con 0x7fa10c103410 2024-05-21T15:35:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa10affd640 1 --2- 172.21.15.38:0/1590148918 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0f40a2c10 0x7fa0f40a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.140+0000 7fa10b7fe640 1 --2- 172.21.15.38:0/928453030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56142213f4d0 0x561422508080 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc006010 tx=0x7fa0fc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd8f1777a46a458 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.141+0000 7fa0f1ffb640 1 -- 172.21.15.38:0/928453030 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc035030 con 0x56142213f4d0 2024-05-21T15:35:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.141+0000 7fa0f1ffb640 1 -- 172.21.15.38:0/928453030 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0fc005520 con 0x56142213f4d0 2024-05-21T15:35:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.141+0000 7fa0f1ffb640 1 -- 172.21.15.38:0/928453030 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0316b0 con 0x56142213f4d0 2024-05-21T15:35:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.141+0000 7fa118fa9740 1 -- 172.21.15.38:0/928453030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56142213f4d0 msgr2=0x561422508080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.141+0000 7fa10affd640 1 --2- 172.21.15.38:0/1590148918 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0f40a2c10 0x7fa0f40a5120 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x561422508cd0 tx=0x7fa10000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:15.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.141+0000 7fa118fa9740 1 --2- 172.21.15.38:0/928453030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56142213f4d0 0x561422508080 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc006010 tx=0x7fa0fc0040c0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.141+0000 7fa118fa9740 1 -- 172.21.15.38:0/928453030 shutdown_connections 2024-05-21T15:35:15.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.141+0000 7fa118fa9740 1 --2- 172.21.15.38:0/928453030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5614221476b0 0x56142250ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.141+0000 7fa118fa9740 1 --2- 172.21.15.38:0/928453030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56142213f4d0 0x561422508080 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.141+0000 7fa118fa9740 1 --2- 172.21.15.38:0/928453030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 0x561422507b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.141+0000 7fa118fa9740 1 -- 172.21.15.38:0/928453030 >> 172.21.15.38:0/928453030 conn(0x56142215c4e0 msgr2=0x561422503490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.142+0000 7fa118fa9740 1 -- 172.21.15.38:0/928453030 shutdown_connections 2024-05-21T15:35:15.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.142+0000 7fa118fa9740 1 -- 172.21.15.38:0/928453030 wait complete. 2024-05-21T15:35:15.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.142+0000 7fa118fa9740 1 -- start start 2024-05-21T15:35:15.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.142+0000 7fa118fa9740 1 Processor -- start 2024-05-21T15:35:15.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.142+0000 7fa118fa9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56142213f4d0 0x561422511900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.142+0000 7fa118fa9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614221476b0 0x561422511e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.142+0000 7fa118fa9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 0x561422518da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.142+0000 7fa118fa9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56142250f970 con 0x5614221476b0 2024-05-21T15:35:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.142+0000 7fa118fa9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561422427c40 con 0x561422168390 2024-05-21T15:35:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.142+0000 7fa118fa9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56142250faf0 con 0x56142213f4d0 2024-05-21T15:35:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa118fa9740 10 client.0 _ll_register_callbacks cb 0x56142217fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa10affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614221476b0 0x561422511e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa10affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614221476b0 0x561422511e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40238/0 (socket says 172.21.15.38:40238) 2024-05-21T15:35:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa10affd640 1 -- 172.21.15.38:0/639156297 learned_addr learned my addr 172.21.15.38:0/639156297 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa10b7fe640 1 --2- 172.21.15.38:0/639156297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56142213f4d0 0x561422511900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa10bfff640 1 --2- 172.21.15.38:0/639156297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 0x561422518da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa10affd640 1 -- 172.21.15.38:0/639156297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56142213f4d0 msgr2=0x561422511900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa10affd640 1 --2- 172.21.15.38:0/639156297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56142213f4d0 0x561422511900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa10affd640 1 -- 172.21.15.38:0/639156297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 msgr2=0x561422518da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa10affd640 1 --2- 172.21.15.38:0/639156297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 0x561422518da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa10affd640 1 -- 172.21.15.38:0/639156297 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56142224a7d0 con 0x5614221476b0 2024-05-21T15:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa10bfff640 1 --2- 172.21.15.38:0/639156297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 0x561422518da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa10affd640 1 --2- 172.21.15.38:0/639156297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614221476b0 0x561422511e40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa100012930 tx=0x7fa1000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e20e7078bbd4777 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa0eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa0eaffd640 1 -- 172.21.15.38:0/639156297 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10002d060 con 0x5614221476b0 2024-05-21T15:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa0eaffd640 1 -- 172.21.15.38:0/639156297 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa100020b10 con 0x5614221476b0 2024-05-21T15:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.143+0000 7fa118fa9740 1 -- 172.21.15.38:0/639156297 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0fc005520 con 0x5614221476b0 2024-05-21T15:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.144+0000 7fa0eaffd640 1 -- 172.21.15.38:0/639156297 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa100020d10 con 0x5614221476b0 2024-05-21T15:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.144+0000 7fa118fa9740 10 client.15645 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.144+0000 7fa118fa9740 1 -- 172.21.15.38:0/639156297 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56142251c130 con 0x5614221476b0 2024-05-21T15:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.144+0000 7fa0eaffd640 1 -- 172.21.15.38:0/639156297 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0fc005520 con 0x5614221476b0 2024-05-21T15:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.144+0000 7fa0eaffd640 1 -- 172.21.15.38:0/639156297 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa10001c040 con 0x5614221476b0 2024-05-21T15:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.144+0000 7fa118fa9740 10 client.15645 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.144+0000 7fa118fa9740 10 client.15645 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.144+0000 7fa118fa9740 20 client.15645 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.144+0000 7fa118fa9740 1 --2- 172.21.15.38:0/639156297 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x561422126f60 0x56142251f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.144+0000 7fa118fa9740 4 client.15645 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.144+0000 7fa118fa9740 1 -- 172.21.15.38:0/639156297 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56142247a370 con 0x561422126f60 2024-05-21T15:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.145+0000 7fa10b7fe640 1 --2- 172.21.15.38:0/639156297 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x561422126f60 0x56142251f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.145+0000 7fa0eaffd640 10 client.15645 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.145+0000 7fa10b7fe640 1 --2- 172.21.15.38:0/639156297 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x561422126f60 0x56142251f950 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7c21ec253dde9f0 server_cookie=9f67d8eeb8097124 in_seq=0 out_seq=0 2024-05-21T15:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa0eaffd640 1 -- 172.21.15.38:0/639156297 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56142247a370 con 0x561422126f60 2024-05-21T15:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa0eaffd640 10 client.15645 handle_command_reply: tid=0 2024-05-21T15:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa118fa9740 1 client.15645 shutdown 2024-05-21T15:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa118fa9740 1 -- 172.21.15.38:0/639156297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614221476b0 msgr2=0x561422511e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa118fa9740 1 --2- 172.21.15.38:0/639156297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614221476b0 0x561422511e40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa100012930 tx=0x7fa1000046c0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa118fa9740 1 -- 172.21.15.38:0/639156297 shutdown_connections 2024-05-21T15:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa118fa9740 1 --2- 172.21.15.38:0/639156297 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x561422126f60 0x56142251f950 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa118fa9740 1 --2- 172.21.15.38:0/639156297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 0x561422518da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa0eaffd640 0 client.15645 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa118fa9740 1 --2- 172.21.15.38:0/639156297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614221476b0 0x561422511e40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa118fa9740 1 --2- 172.21.15.38:0/639156297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56142213f4d0 0x561422511900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa118fa9740 1 -- 172.21.15.38:0/639156297 >> 172.21.15.38:0/639156297 conn(0x56142215c4e0 msgr2=0x561422502570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa118fa9740 1 -- 172.21.15.38:0/639156297 shutdown_connections 2024-05-21T15:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa118fa9740 1 -- 172.21.15.38:0/639156297 wait complete. 2024-05-21T15:35:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.146+0000 7fa118fa9740 20 client.15645 trim_cache size 0 max 16384 2024-05-21T15:35:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa118fa9740 1 Processor -- start 2024-05-21T15:35:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa118fa9740 1 -- start start 2024-05-21T15:35:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa118fa9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56142213f4d0 0x56142250fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa118fa9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614221476b0 0x561422518cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa118fa9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561422168390 0x561422510430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa118fa9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1aa390 con 0x56142213f4d0 2024-05-21T15:35:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa118fa9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1abed0 con 0x5614221476b0 2024-05-21T15:35:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa118fa9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1aaa60 con 0x561422168390 2024-05-21T15:35:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa10b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56142213f4d0 0x56142250fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa10b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56142213f4d0 0x56142250fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40240/0 (socket says 172.21.15.38:40240) 2024-05-21T15:35:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa10b7fe640 1 -- 172.21.15.38:0/1991192911 learned_addr learned my addr 172.21.15.38:0/1991192911 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa10affd640 1 --2- 172.21.15.38:0/1991192911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614221476b0 0x561422518cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa10bfff640 1 --2- 172.21.15.38:0/1991192911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561422168390 0x561422510430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa10b7fe640 1 -- 172.21.15.38:0/1991192911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561422168390 msgr2=0x561422510430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa10b7fe640 1 --2- 172.21.15.38:0/1991192911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561422168390 0x561422510430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa10b7fe640 1 -- 172.21.15.38:0/1991192911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614221476b0 msgr2=0x561422518cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa10b7fe640 1 --2- 172.21.15.38:0/1991192911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614221476b0 0x561422518cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.160+0000 7fa10b7fe640 1 -- 172.21.15.38:0/1991192911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142251d5f0 con 0x56142213f4d0 2024-05-21T15:35:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.161+0000 7fa10b7fe640 1 --2- 172.21.15.38:0/1991192911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56142213f4d0 0x56142250fef0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc02aa00 tx=0x7fa0fc0314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acbd11ceaeea34c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.161+0000 7fa0f17fa640 1 -- 172.21.15.38:0/1991192911 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc035040 con 0x56142213f4d0 2024-05-21T15:35:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.161+0000 7fa0f17fa640 1 -- 172.21.15.38:0/1991192911 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0fc031b90 con 0x56142213f4d0 2024-05-21T15:35:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.161+0000 7fa0f17fa640 1 -- 172.21.15.38:0/1991192911 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc003a60 con 0x56142213f4d0 2024-05-21T15:35:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 -- 172.21.15.38:0/1991192911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56142213f4d0 msgr2=0x56142250fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 --2- 172.21.15.38:0/1991192911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56142213f4d0 0x56142250fef0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc02aa00 tx=0x7fa0fc0314e0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 -- 172.21.15.38:0/1991192911 shutdown_connections 2024-05-21T15:35:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 --2- 172.21.15.38:0/1991192911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561422168390 0x561422510430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 --2- 172.21.15.38:0/1991192911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614221476b0 0x561422518cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 --2- 172.21.15.38:0/1991192911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56142213f4d0 0x56142250fef0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 -- 172.21.15.38:0/1991192911 >> 172.21.15.38:0/1991192911 conn(0x56142215c4e0 msgr2=0x56142250bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 -- 172.21.15.38:0/1991192911 shutdown_connections 2024-05-21T15:35:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 -- 172.21.15.38:0/1991192911 wait complete. 2024-05-21T15:35:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 -- start start 2024-05-21T15:35:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 Processor -- start 2024-05-21T15:35:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56142213f4d0 0x561422516040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614221476b0 0x561422516580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 0x5614225174b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1ab800 con 0x5614221476b0 2024-05-21T15:35:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1ab130 con 0x561422168390 2024-05-21T15:35:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.162+0000 7fa118fa9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1a8370 con 0x56142213f4d0 2024-05-21T15:35:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa118fa9740 10 client.0 _ll_register_callbacks cb 0x7fa1000125e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa10affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614221476b0 0x561422516580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa10affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614221476b0 0x561422516580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40242/0 (socket says 172.21.15.38:40242) 2024-05-21T15:35:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa10affd640 1 -- 172.21.15.38:0/1625564759 learned_addr learned my addr 172.21.15.38:0/1625564759 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa10bfff640 1 --2- 172.21.15.38:0/1625564759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 0x5614225174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa10b7fe640 1 --2- 172.21.15.38:0/1625564759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56142213f4d0 0x561422516040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa10affd640 1 -- 172.21.15.38:0/1625564759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56142213f4d0 msgr2=0x561422516040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa10affd640 1 --2- 172.21.15.38:0/1625564759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56142213f4d0 0x561422516040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa10affd640 1 -- 172.21.15.38:0/1625564759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 msgr2=0x5614225174b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa10affd640 1 --2- 172.21.15.38:0/1625564759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 0x5614225174b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa10affd640 1 -- 172.21.15.38:0/1625564759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561422517a70 con 0x5614221476b0 2024-05-21T15:35:15.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa10b7fe640 1 --2- 172.21.15.38:0/1625564759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56142213f4d0 0x561422516040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:15.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa10affd640 1 --2- 172.21.15.38:0/1625564759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614221476b0 0x561422516580 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa10000b690 tx=0x7fa100033b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54b40c349e7381b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:15.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa0eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:15.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa0eaffd640 1 -- 172.21.15.38:0/1625564759 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100005250 con 0x5614221476b0 2024-05-21T15:35:15.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa0eaffd640 1 -- 172.21.15.38:0/1625564759 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa1000053f0 con 0x5614221476b0 2024-05-21T15:35:15.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa118fa9740 1 -- 172.21.15.38:0/1625564759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0fc031b90 con 0x5614221476b0 2024-05-21T15:35:15.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.163+0000 7fa0eaffd640 1 -- 172.21.15.38:0/1625564759 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1000055f0 con 0x5614221476b0 2024-05-21T15:35:15.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.164+0000 7fa118fa9740 10 client.15651 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:15.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.164+0000 7fa118fa9740 1 -- 172.21.15.38:0/1625564759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561422518cf0 con 0x5614221476b0 2024-05-21T15:35:15.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.164+0000 7fa0eaffd640 1 -- 172.21.15.38:0/1625564759 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0fc031b90 con 0x5614221476b0 2024-05-21T15:35:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.164+0000 7fa0eaffd640 1 -- 172.21.15.38:0/1625564759 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa10003a070 con 0x5614221476b0 2024-05-21T15:35:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.164+0000 7fa118fa9740 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.164+0000 7fa118fa9740 10 client.15651 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:35:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.164+0000 7fa118fa9740 20 client.15651 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.164+0000 7fa118fa9740 1 --2- 172.21.15.38:0/1625564759 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x5614220fc3a0 0x56142258a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.164+0000 7fa118fa9740 4 client.15651 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:35:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.164+0000 7fa118fa9740 1 -- 172.21.15.38:0/1625564759 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614224746c0 con 0x5614220fc3a0 2024-05-21T15:35:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.164+0000 7fa10b7fe640 1 --2- 172.21.15.38:0/1625564759 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x5614220fc3a0 0x56142258a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa0eaffd640 10 client.15651 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:35:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa10b7fe640 1 --2- 172.21.15.38:0/1625564759 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x5614220fc3a0 0x56142258a220 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e61a2365e3b7a21 server_cookie=e295f3f83b34283e in_seq=0 out_seq=0 2024-05-21T15:35:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa0eaffd640 1 -- 172.21.15.38:0/1625564759 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614224746c0 con 0x5614220fc3a0 2024-05-21T15:35:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa0eaffd640 10 client.15651 handle_command_reply: tid=0 2024-05-21T15:35:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa118fa9740 1 client.15651 shutdown 2024-05-21T15:35:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa118fa9740 1 -- 172.21.15.38:0/1625564759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614221476b0 msgr2=0x561422516580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa118fa9740 1 --2- 172.21.15.38:0/1625564759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614221476b0 0x561422516580 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa10000b690 tx=0x7fa100033b50 comp rx=0 tx=0).stop 2024-05-21T15:35:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa118fa9740 1 -- 172.21.15.38:0/1625564759 shutdown_connections 2024-05-21T15:35:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa118fa9740 1 --2- 172.21.15.38:0/1625564759 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x5614220fc3a0 0x56142258a220 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa118fa9740 1 --2- 172.21.15.38:0/1625564759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561422168390 0x5614225174b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa0eaffd640 0 client.15651 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:35:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa118fa9740 1 --2- 172.21.15.38:0/1625564759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5614221476b0 0x561422516580 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa118fa9740 1 --2- 172.21.15.38:0/1625564759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56142213f4d0 0x561422516040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa118fa9740 1 -- 172.21.15.38:0/1625564759 >> 172.21.15.38:0/1625564759 conn(0x56142215c4e0 msgr2=0x561422502ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa118fa9740 1 -- 172.21.15.38:0/1625564759 shutdown_connections 2024-05-21T15:35:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa118fa9740 1 -- 172.21.15.38:0/1625564759 wait complete. 2024-05-21T15:35:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.165+0000 7fa118fa9740 20 client.15651 trim_cache size 0 max 16384 2024-05-21T15:35:15.179 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:35:15.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.166+0000 7fa112102640 1 -- 172.21.15.38:0/1590148918 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0f40a2c10 msgr2=0x7fa0f40a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.166+0000 7fa112102640 1 --2- 172.21.15.38:0/1590148918 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0f40a2c10 0x7fa0f40a5120 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x561422508cd0 tx=0x7fa10000b410 comp rx=0 tx=0).stop 2024-05-21T15:35:15.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.166+0000 7fa112102640 1 -- 172.21.15.38:0/1590148918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10c103410 msgr2=0x7fa10c1a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.166+0000 7fa112102640 1 --2- 172.21.15.38:0/1590148918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10c103410 0x7fa10c1a0730 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa10400eaa0 tx=0x7fa10400ef60 comp rx=0 tx=0).stop 2024-05-21T15:35:15.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.166+0000 7fa112102640 1 -- 172.21.15.38:0/1590148918 shutdown_connections 2024-05-21T15:35:15.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.166+0000 7fa112102640 1 --2- 172.21.15.38:0/1590148918 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0f40a2c10 0x7fa0f40a5120 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.166+0000 7fa112102640 1 --2- 172.21.15.38:0/1590148918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa10c103410 0x7fa10c1a0730 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.166+0000 7fa112102640 1 --2- 172.21.15.38:0/1590148918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa10c1028d0 0x7fa10c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.166+0000 7fa112102640 1 --2- 172.21.15.38:0/1590148918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10c101ef0 0x7fa10c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.166+0000 7fa112102640 1 -- 172.21.15.38:0/1590148918 >> 172.21.15.38:0/1590148918 conn(0x7fa10c0fc330 msgr2=0x7fa10c0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.166+0000 7fa112102640 1 -- 172.21.15.38:0/1590148918 shutdown_connections 2024-05-21T15:35:15.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.166+0000 7fa112102640 1 -- 172.21.15.38:0/1590148918 wait complete. 2024-05-21T15:35:15.181 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:14 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2612853294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:35:15.182 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:35:15.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:14 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2612853294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:35:15.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.300+0000 7f9926873640 1 -- 172.21.15.38:0/670515172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9920101980 msgr2=0x7f9920102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.300+0000 7f9926873640 1 --2- 172.21.15.38:0/670515172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9920101980 0x7f9920102680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9910009900 tx=0x7f991002a410 comp rx=0 tx=0).stop 2024-05-21T15:35:15.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.300+0000 7f9926873640 1 -- 172.21.15.38:0/670515172 shutdown_connections 2024-05-21T15:35:15.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.300+0000 7f9926873640 1 --2- 172.21.15.38:0/670515172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99201078c0 0x7f9920109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.300+0000 7f9926873640 1 --2- 172.21.15.38:0/670515172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920102bc0 0x7f9920102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.300+0000 7f9926873640 1 --2- 172.21.15.38:0/670515172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9920101980 0x7f9920102680 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.300+0000 7f9926873640 1 -- 172.21.15.38:0/670515172 >> 172.21.15.38:0/670515172 conn(0x7f99200fcb70 msgr2=0x7f99200ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.300+0000 7f9926873640 1 -- 172.21.15.38:0/670515172 shutdown_connections 2024-05-21T15:35:15.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.300+0000 7f9926873640 1 -- 172.21.15.38:0/670515172 wait complete. 2024-05-21T15:35:15.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.301+0000 7f9926873640 1 Processor -- start 2024-05-21T15:35:15.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.301+0000 7f9926873640 1 -- start start 2024-05-21T15:35:15.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.301+0000 7f9926873640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9920101980 0x7f9920069a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.301+0000 7f9926873640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920102bc0 0x7f992019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.301+0000 7f9926873640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99201078c0 0x7f99201a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.301+0000 7f9926873640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f992010d600 con 0x7f9920101980 2024-05-21T15:35:15.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.301+0000 7f9926873640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f992010d480 con 0x7f9920102bc0 2024-05-21T15:35:15.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.301+0000 7f9926873640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f992010d780 con 0x7f99201078c0 2024-05-21T15:35:15.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.301+0000 7f991ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9920101980 0x7f9920069a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.301+0000 7f991ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9920101980 0x7f9920069a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40268/0 (socket says 172.21.15.38:40268) 2024-05-21T15:35:15.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.301+0000 7f991ffff640 1 -- 172.21.15.38:0/1664880139 learned_addr learned my addr 172.21.15.38:0/1664880139 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:15.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.301+0000 7f991f7fe640 1 --2- 172.21.15.38:0/1664880139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920102bc0 0x7f992019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.302+0000 7f991ffff640 1 -- 172.21.15.38:0/1664880139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99201078c0 msgr2=0x7f99201a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.302+0000 7f991ffff640 1 --2- 172.21.15.38:0/1664880139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99201078c0 0x7f99201a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.302+0000 7f991ffff640 1 -- 172.21.15.38:0/1664880139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920102bc0 msgr2=0x7f992019e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.302+0000 7f991ffff640 1 --2- 172.21.15.38:0/1664880139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920102bc0 0x7f992019e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.302+0000 7f991ffff640 1 -- 172.21.15.38:0/1664880139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99201a0dc0 con 0x7f9920101980 2024-05-21T15:35:15.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.302+0000 7f991ffff640 1 --2- 172.21.15.38:0/1664880139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9920101980 0x7f9920069a20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9910031440 tx=0x7f9910032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:15.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.302+0000 7f991d7fa640 1 -- 172.21.15.38:0/1664880139 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991003f030 con 0x7f9920101980 2024-05-21T15:35:15.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.302+0000 7f9926873640 1 -- 172.21.15.38:0/1664880139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99201a1050 con 0x7f9920101980 2024-05-21T15:35:15.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.302+0000 7f991d7fa640 1 -- 172.21.15.38:0/1664880139 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9910003ce0 con 0x7f9920101980 2024-05-21T15:35:15.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.302+0000 7f991d7fa640 1 -- 172.21.15.38:0/1664880139 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910035070 con 0x7f9920101980 2024-05-21T15:35:15.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.302+0000 7f9926873640 1 -- 172.21.15.38:0/1664880139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99201a1540 con 0x7f9920101980 2024-05-21T15:35:15.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.304+0000 7f991d7fa640 1 -- 172.21.15.38:0/1664880139 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9910042030 con 0x7f9920101980 2024-05-21T15:35:15.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f992d7f6740 1 Processor -- start 2024-05-21T15:35:15.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f992d7f6740 1 -- start start 2024-05-21T15:35:15.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f992d7f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209ca390 0x562120d69b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f992d7f6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d6a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f992d7f6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209a96b0 0x562120d70f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f992d7f6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562120ccec40 con 0x5621209a14d0 2024-05-21T15:35:15.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f992d7f6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562120c76580 con 0x5621209a96b0 2024-05-21T15:35:15.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f992d7f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562120ccac00 con 0x5621209ca390 2024-05-21T15:35:15.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f991ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d6a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f991ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d6a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40272/0 (socket says 172.21.15.38:40272) 2024-05-21T15:35:15.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f991ffff640 1 -- 172.21.15.38:0/3427912970 learned_addr learned my addr 172.21.15.38:0/3427912970 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:15.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f9924e10640 1 --2- 172.21.15.38:0/3427912970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209ca390 0x562120d69b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f991f7fe640 1 --2- 172.21.15.38:0/3427912970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209a96b0 0x562120d70f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f991d7fa640 1 --2- 172.21.15.38:0/1664880139 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f99080a2b90 0x7f99080a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f991ffff640 1 -- 172.21.15.38:0/3427912970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209ca390 msgr2=0x562120d69b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f991ffff640 1 --2- 172.21.15.38:0/3427912970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209ca390 0x562120d69b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f991ffff640 1 -- 172.21.15.38:0/3427912970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209a96b0 msgr2=0x562120d70f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f991ffff640 1 --2- 172.21.15.38:0/3427912970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209a96b0 0x562120d70f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f991ffff640 1 -- 172.21.15.38:0/3427912970 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621209e23f0 con 0x5621209a14d0 2024-05-21T15:35:15.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.305+0000 7f991f7fe640 1 --2- 172.21.15.38:0/3427912970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209a96b0 0x562120d70f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:15.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.306+0000 7f9924e10640 1 --2- 172.21.15.38:0/1664880139 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f99080a2b90 0x7f99080a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.306+0000 7f991d7fa640 1 -- 172.21.15.38:0/1664880139 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f99100ef650 con 0x7f9920101980 2024-05-21T15:35:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.306+0000 7f991ffff640 1 --2- 172.21.15.38:0/3427912970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d6a080 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f991009c3d0 tx=0x7f99100869d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05159ff98948467 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.306+0000 7f99067fc640 1 -- 172.21.15.38:0/3427912970 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991009ac70 con 0x5621209a14d0 2024-05-21T15:35:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.306+0000 7f99067fc640 1 -- 172.21.15.38:0/3427912970 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f991009b570 con 0x5621209a14d0 2024-05-21T15:35:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.306+0000 7f99067fc640 1 -- 172.21.15.38:0/3427912970 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910086ee0 con 0x5621209a14d0 2024-05-21T15:35:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.307+0000 7f992d7f6740 1 -- 172.21.15.38:0/3427912970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 msgr2=0x562120d6a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.307+0000 7f9924e10640 1 --2- 172.21.15.38:0/1664880139 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f99080a2b90 0x7f99080a50a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9918009910 tx=0x7f99180098a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.307+0000 7f992d7f6740 1 --2- 172.21.15.38:0/3427912970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d6a080 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f991009c3d0 tx=0x7f99100869d0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.307+0000 7f992d7f6740 1 -- 172.21.15.38:0/3427912970 shutdown_connections 2024-05-21T15:35:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.307+0000 7f992d7f6740 1 --2- 172.21.15.38:0/3427912970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209a96b0 0x562120d70f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.307+0000 7f992d7f6740 1 --2- 172.21.15.38:0/3427912970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d6a080 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.307+0000 7f992d7f6740 1 --2- 172.21.15.38:0/3427912970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209ca390 0x562120d69b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.307+0000 7f992d7f6740 1 -- 172.21.15.38:0/3427912970 >> 172.21.15.38:0/3427912970 conn(0x5621209be4e0 msgr2=0x562120d65490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.307+0000 7f992d7f6740 1 -- 172.21.15.38:0/3427912970 shutdown_connections 2024-05-21T15:35:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.307+0000 7f992d7f6740 1 -- 172.21.15.38:0/3427912970 wait complete. 2024-05-21T15:35:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.307+0000 7f992d7f6740 1 -- start start 2024-05-21T15:35:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.307+0000 7f992d7f6740 1 Processor -- start 2024-05-21T15:35:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f992d7f6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f992d7f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209a96b0 0x562120d73b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f992d7f6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209ca390 0x562120d7aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f992d7f6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562120d71790 con 0x5621209a14d0 2024-05-21T15:35:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f992d7f6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562120c89c40 con 0x5621209ca390 2024-05-21T15:35:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f992d7f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562120d71910 con 0x5621209a96b0 2024-05-21T15:35:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f992d7f6740 10 client.0 _ll_register_callbacks cb 0x5621209e1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f991ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f991ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40284/0 (socket says 172.21.15.38:40284) 2024-05-21T15:35:15.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f991ffff640 1 -- 172.21.15.38:0/1903888795 learned_addr learned my addr 172.21.15.38:0/1903888795 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f991f7fe640 1 --2- 172.21.15.38:0/1903888795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209a96b0 0x562120d73b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f9924e10640 1 --2- 172.21.15.38:0/1903888795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209ca390 0x562120d7aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f991ffff640 1 -- 172.21.15.38:0/1903888795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209a96b0 msgr2=0x562120d73b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f991ffff640 1 --2- 172.21.15.38:0/1903888795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209a96b0 0x562120d73b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f991ffff640 1 -- 172.21.15.38:0/1903888795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209ca390 msgr2=0x562120d7aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f991ffff640 1 --2- 172.21.15.38:0/1903888795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209ca390 0x562120d7aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f991ffff640 1 -- 172.21.15.38:0/1903888795 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562120aac7d0 con 0x5621209a14d0 2024-05-21T15:35:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f991f7fe640 1 --2- 172.21.15.38:0/1903888795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209a96b0 0x562120d73b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f991ffff640 1 --2- 172.21.15.38:0/1903888795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d735f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f99100bbd90 tx=0x7f99100314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6bb0de5ca6d1e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f98ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.308+0000 7f98ff7fe640 1 -- 172.21.15.38:0/1903888795 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910049030 con 0x5621209a14d0 2024-05-21T15:35:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.309+0000 7f98ff7fe640 1 -- 172.21.15.38:0/1903888795 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f99100ba100 con 0x5621209a14d0 2024-05-21T15:35:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.309+0000 7f992d7f6740 1 -- 172.21.15.38:0/1903888795 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562120cdc370 con 0x5621209a14d0 2024-05-21T15:35:15.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.309+0000 7f98ff7fe640 1 -- 172.21.15.38:0/1903888795 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f991009ac70 con 0x5621209a14d0 2024-05-21T15:35:15.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.309+0000 7f992d7f6740 10 client.15663 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:15.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.309+0000 7f992d7f6740 1 -- 172.21.15.38:0/1903888795 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562120d71790 con 0x5621209a14d0 2024-05-21T15:35:15.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.309+0000 7f98ff7fe640 1 -- 172.21.15.38:0/1903888795 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562120cdc370 con 0x5621209a14d0 2024-05-21T15:35:15.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.309+0000 7f98ff7fe640 1 -- 172.21.15.38:0/1903888795 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f991008e020 con 0x5621209a14d0 2024-05-21T15:35:15.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.309+0000 7f992d7f6740 10 client.15663 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:15.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.309+0000 7f992d7f6740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:15.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.309+0000 7f992d7f6740 20 client.15663 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:15.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.309+0000 7f992d7f6740 1 --2- 172.21.15.38:0/1903888795 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562120988f60 0x562120d81250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.309+0000 7f992d7f6740 4 client.15663 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:15.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.309+0000 7f992d7f6740 1 -- 172.21.15.38:0/1903888795 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562120cd66c0 con 0x562120988f60 2024-05-21T15:35:15.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.309+0000 7f991f7fe640 1 --2- 172.21.15.38:0/1903888795 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562120988f60 0x562120d81250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.310+0000 7f98ff7fe640 10 client.15663 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:15.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.310+0000 7f991f7fe640 1 --2- 172.21.15.38:0/1903888795 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562120988f60 0x562120d81250 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20cd44af057c0c88 server_cookie=cdf23ec67e2b6b8 in_seq=0 out_seq=0 2024-05-21T15:35:15.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.310+0000 7f98ff7fe640 1 -- 172.21.15.38:0/1903888795 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562120cd66c0 con 0x562120988f60 2024-05-21T15:35:15.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.311+0000 7f98ff7fe640 10 client.15663 handle_command_reply: tid=0 2024-05-21T15:35:15.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.311+0000 7f992d7f6740 1 client.15663 shutdown 2024-05-21T15:35:15.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.311+0000 7f992d7f6740 1 -- 172.21.15.38:0/1903888795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 msgr2=0x562120d735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.311+0000 7f992d7f6740 1 --2- 172.21.15.38:0/1903888795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d735f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f99100bbd90 tx=0x7f99100314e0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.311+0000 7f992d7f6740 1 -- 172.21.15.38:0/1903888795 shutdown_connections 2024-05-21T15:35:15.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.311+0000 7f992d7f6740 1 --2- 172.21.15.38:0/1903888795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209ca390 0x562120d7aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.311+0000 7f992d7f6740 1 --2- 172.21.15.38:0/1903888795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209a96b0 0x562120d73b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.311+0000 7f992d7f6740 1 --2- 172.21.15.38:0/1903888795 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562120988f60 0x562120d81250 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.311+0000 7f992d7f6740 1 --2- 172.21.15.38:0/1903888795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d735f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.311+0000 7f992d7f6740 1 -- 172.21.15.38:0/1903888795 >> 172.21.15.38:0/1903888795 conn(0x5621209be4e0 msgr2=0x562120d64570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.311+0000 7f98ff7fe640 0 client.15663 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:15.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.311+0000 7f992d7f6740 1 -- 172.21.15.38:0/1903888795 shutdown_connections 2024-05-21T15:35:15.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.311+0000 7f992d7f6740 1 -- 172.21.15.38:0/1903888795 wait complete. 2024-05-21T15:35:15.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.311+0000 7f992d7f6740 20 client.15663 trim_cache size 0 max 16384 2024-05-21T15:35:15.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f992d7f6740 1 Processor -- start 2024-05-21T15:35:15.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f992d7f6740 1 -- start start 2024-05-21T15:35:15.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f992d7f6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d6e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f992d7f6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209a96b0 0x562120d7aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f992d7f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209ca390 0x562120d64250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f992d7f6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f99201aa470 con 0x5621209a14d0 2024-05-21T15:35:15.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f992d7f6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99201abfb0 con 0x5621209a96b0 2024-05-21T15:35:15.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f992d7f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f99201aab40 con 0x5621209ca390 2024-05-21T15:35:15.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f991ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d6e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f991ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d6e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40294/0 (socket says 172.21.15.38:40294) 2024-05-21T15:35:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f991ffff640 1 -- 172.21.15.38:0/124189036 learned_addr learned my addr 172.21.15.38:0/124189036 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f991f7fe640 1 --2- 172.21.15.38:0/124189036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209a96b0 0x562120d7aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f991ffff640 1 -- 172.21.15.38:0/124189036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209ca390 msgr2=0x562120d64250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f991ffff640 1 --2- 172.21.15.38:0/124189036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209ca390 0x562120d64250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f991ffff640 1 -- 172.21.15.38:0/124189036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209a96b0 msgr2=0x562120d7aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f991ffff640 1 --2- 172.21.15.38:0/124189036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209a96b0 0x562120d7aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f991ffff640 1 -- 172.21.15.38:0/124189036 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562120d60d20 con 0x5621209a14d0 2024-05-21T15:35:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.325+0000 7f991f7fe640 1 --2- 172.21.15.38:0/124189036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209a96b0 0x562120d7aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.326+0000 7f991ffff640 1 --2- 172.21.15.38:0/124189036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d6e370 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f99100bbb00 tx=0x7f9910009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0f577da238970f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:15.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.326+0000 7f9905ffb640 1 -- 172.21.15.38:0/124189036 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910049070 con 0x5621209a14d0 2024-05-21T15:35:15.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.326+0000 7f9905ffb640 1 -- 172.21.15.38:0/124189036 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99100bc500 con 0x5621209a14d0 2024-05-21T15:35:15.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.326+0000 7f9905ffb640 1 -- 172.21.15.38:0/124189036 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99100ba1d0 con 0x5621209a14d0 2024-05-21T15:35:15.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.326+0000 7f992d7f6740 1 -- 172.21.15.38:0/124189036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 msgr2=0x562120d6e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.326+0000 7f992d7f6740 1 --2- 172.21.15.38:0/124189036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d6e370 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f99100bbb00 tx=0x7f9910009260 comp rx=0 tx=0).stop 2024-05-21T15:35:15.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.326+0000 7f992d7f6740 1 -- 172.21.15.38:0/124189036 shutdown_connections 2024-05-21T15:35:15.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.326+0000 7f992d7f6740 1 --2- 172.21.15.38:0/124189036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209ca390 0x562120d64250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.326+0000 7f992d7f6740 1 --2- 172.21.15.38:0/124189036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209a96b0 0x562120d7aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.326+0000 7f992d7f6740 1 --2- 172.21.15.38:0/124189036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a14d0 0x562120d6e370 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.326+0000 7f992d7f6740 1 -- 172.21.15.38:0/124189036 >> 172.21.15.38:0/124189036 conn(0x5621209be4e0 msgr2=0x562120d63b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.326+0000 7f992d7f6740 1 -- 172.21.15.38:0/124189036 shutdown_connections 2024-05-21T15:35:15.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.327+0000 7f992d7f6740 1 -- 172.21.15.38:0/124189036 wait complete. 2024-05-21T15:35:15.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.327+0000 7f992d7f6740 1 -- start start 2024-05-21T15:35:15.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.327+0000 7f992d7f6740 1 Processor -- start 2024-05-21T15:35:15.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.327+0000 7f992d7f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209a14d0 0x562120d77d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.327+0000 7f992d7f6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a96b0 0x562120d78280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.327+0000 7f992d7f6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209ca390 0x562120d79160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.327+0000 7f992d7f6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f99201ab8e0 con 0x5621209a96b0 2024-05-21T15:35:15.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.327+0000 7f992d7f6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99201ab210 con 0x5621209ca390 2024-05-21T15:35:15.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.327+0000 7f992d7f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f99201a8450 con 0x5621209a14d0 2024-05-21T15:35:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.327+0000 7f992d7f6740 10 client.0 _ll_register_callbacks cb 0x7f99140095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.327+0000 7f991f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a96b0 0x562120d78280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.327+0000 7f991f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a96b0 0x562120d78280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40304/0 (socket says 172.21.15.38:40304) 2024-05-21T15:35:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.327+0000 7f991f7fe640 1 -- 172.21.15.38:0/8197725 learned_addr learned my addr 172.21.15.38:0/8197725 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f991ffff640 1 --2- 172.21.15.38:0/8197725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209a14d0 0x562120d77d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f991f7fe640 1 -- 172.21.15.38:0/8197725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209a14d0 msgr2=0x562120d77d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f991f7fe640 1 --2- 172.21.15.38:0/8197725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209a14d0 0x562120d77d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f9924e10640 1 --2- 172.21.15.38:0/8197725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209ca390 0x562120d79160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f991f7fe640 1 -- 172.21.15.38:0/8197725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209ca390 msgr2=0x562120d79160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f991f7fe640 1 --2- 172.21.15.38:0/8197725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209ca390 0x562120d79160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f991f7fe640 1 -- 172.21.15.38:0/8197725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562120d79720 con 0x5621209a96b0 2024-05-21T15:35:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f991ffff640 1 --2- 172.21.15.38:0/8197725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209a14d0 0x562120d77d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f991f7fe640 1 --2- 172.21.15.38:0/8197725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a96b0 0x562120d78280 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9914009cb0 tx=0x7f9914012970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902a3231c3e477e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f98ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f98ff7fe640 1 -- 172.21.15.38:0/8197725 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99140035f0 con 0x5621209a96b0 2024-05-21T15:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f98ff7fe640 1 -- 172.21.15.38:0/8197725 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9914003790 con 0x5621209a96b0 2024-05-21T15:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f992d7f6740 1 -- 172.21.15.38:0/8197725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99100bc500 con 0x5621209a96b0 2024-05-21T15:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f98ff7fe640 1 -- 172.21.15.38:0/8197725 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9914030050 con 0x5621209a96b0 2024-05-21T15:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f992d7f6740 10 client.15669 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.328+0000 7f992d7f6740 1 -- 172.21.15.38:0/8197725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562120d7aa10 con 0x5621209a96b0 2024-05-21T15:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.329+0000 7f98ff7fe640 1 -- 172.21.15.38:0/8197725 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99100bc500 con 0x5621209a96b0 2024-05-21T15:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.329+0000 7f98ff7fe640 1 -- 172.21.15.38:0/8197725 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f991401fb80 con 0x5621209a96b0 2024-05-21T15:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.329+0000 7f992d7f6740 10 client.15669 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.329+0000 7f992d7f6740 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.329+0000 7f992d7f6740 20 client.15669 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.329+0000 7f992d7f6740 1 --2- 172.21.15.38:0/8197725 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56212095e3a0 0x562120deba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.329+0000 7f992d7f6740 4 client.15669 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.329+0000 7f992d7f6740 1 -- 172.21.15.38:0/8197725 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562120c53820 con 0x56212095e3a0 2024-05-21T15:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.329+0000 7f991ffff640 1 --2- 172.21.15.38:0/8197725 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56212095e3a0 0x562120deba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f98ff7fe640 10 client.15669 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f991ffff640 1 --2- 172.21.15.38:0/8197725 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56212095e3a0 0x562120deba40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1c05db2318fde8f server_cookie=2d6d833cabd096e2 in_seq=0 out_seq=0 2024-05-21T15:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f98ff7fe640 1 -- 172.21.15.38:0/8197725 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562120c53820 con 0x56212095e3a0 2024-05-21T15:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f98ff7fe640 10 client.15669 handle_command_reply: tid=0 2024-05-21T15:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f992d7f6740 1 client.15669 shutdown 2024-05-21T15:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f992d7f6740 1 -- 172.21.15.38:0/8197725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a96b0 msgr2=0x562120d78280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f992d7f6740 1 --2- 172.21.15.38:0/8197725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a96b0 0x562120d78280 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9914009cb0 tx=0x7f9914012970 comp rx=0 tx=0).stop 2024-05-21T15:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f992d7f6740 1 -- 172.21.15.38:0/8197725 shutdown_connections 2024-05-21T15:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f992d7f6740 1 --2- 172.21.15.38:0/8197725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621209ca390 0x562120d79160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f992d7f6740 1 --2- 172.21.15.38:0/8197725 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56212095e3a0 0x562120deba40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f992d7f6740 1 --2- 172.21.15.38:0/8197725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5621209a96b0 0x562120d78280 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f992d7f6740 1 --2- 172.21.15.38:0/8197725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5621209a14d0 0x562120d77d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f98ff7fe640 0 client.15669 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f992d7f6740 1 -- 172.21.15.38:0/8197725 >> 172.21.15.38:0/8197725 conn(0x5621209be4e0 msgr2=0x562120d64570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f992d7f6740 1 -- 172.21.15.38:0/8197725 shutdown_connections 2024-05-21T15:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f992d7f6740 1 -- 172.21.15.38:0/8197725 wait complete. 2024-05-21T15:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.330+0000 7f992d7f6740 20 client.15669 trim_cache size 0 max 16384 2024-05-21T15:35:15.336 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.331+0000 7f9926873640 1 -- 172.21.15.38:0/1664880139 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f99080a2b90 msgr2=0x7f99080a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.331+0000 7f9926873640 1 --2- 172.21.15.38:0/1664880139 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f99080a2b90 0x7f99080a50a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9918009910 tx=0x7f99180098a0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.331+0000 7f9926873640 1 -- 172.21.15.38:0/1664880139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9920101980 msgr2=0x7f9920069a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.331+0000 7f9926873640 1 --2- 172.21.15.38:0/1664880139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9920101980 0x7f9920069a20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9910031440 tx=0x7f9910032ce0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.331+0000 7f9926873640 1 -- 172.21.15.38:0/1664880139 shutdown_connections 2024-05-21T15:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.331+0000 7f9926873640 1 --2- 172.21.15.38:0/1664880139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99201078c0 0x7f99201a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.331+0000 7f9926873640 1 --2- 172.21.15.38:0/1664880139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920102bc0 0x7f992019e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.331+0000 7f9926873640 1 --2- 172.21.15.38:0/1664880139 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f99080a2b90 0x7f99080a50a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.331+0000 7f9926873640 1 --2- 172.21.15.38:0/1664880139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9920101980 0x7f9920069a20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.331+0000 7f9926873640 1 -- 172.21.15.38:0/1664880139 >> 172.21.15.38:0/1664880139 conn(0x7f99200fcb70 msgr2=0x7f99201038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.331+0000 7f9926873640 1 -- 172.21.15.38:0/1664880139 shutdown_connections 2024-05-21T15:35:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:15.331+0000 7f9926873640 1 -- 172.21.15.38:0/1664880139 wait complete. 2024-05-21T15:35:15.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:14 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2612853294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:35:16.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:15 smithi045 ceph-mon[36409]: pgmap v376: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T15:35:16.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:15 smithi038 ceph-mon[31995]: pgmap v376: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T15:35:16.342 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:35:16.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:35:16.343 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:35:16.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:15 smithi053 ceph-mon[36496]: pgmap v376: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T15:35:16.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.478+0000 7fdc866e2640 1 -- 172.21.15.38:0/3219791375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc80101990 msgr2=0x7fdc80102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.478+0000 7fdc866e2640 1 --2- 172.21.15.38:0/3219791375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc80101990 0x7fdc80102690 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009900 tx=0x7fdc7002a410 comp rx=0 tx=0).stop 2024-05-21T15:35:16.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.478+0000 7fdc866e2640 1 -- 172.21.15.38:0/3219791375 shutdown_connections 2024-05-21T15:35:16.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.478+0000 7fdc866e2640 1 --2- 172.21.15.38:0/3219791375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc801078d0 0x7fdc80109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.478+0000 7fdc866e2640 1 --2- 172.21.15.38:0/3219791375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc80102bd0 0x7fdc80102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.478+0000 7fdc866e2640 1 --2- 172.21.15.38:0/3219791375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc80101990 0x7fdc80102690 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.478+0000 7fdc866e2640 1 -- 172.21.15.38:0/3219791375 >> 172.21.15.38:0/3219791375 conn(0x7fdc800fcba0 msgr2=0x7fdc800ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:16.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.478+0000 7fdc866e2640 1 -- 172.21.15.38:0/3219791375 shutdown_connections 2024-05-21T15:35:16.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.478+0000 7fdc866e2640 1 -- 172.21.15.38:0/3219791375 wait complete. 2024-05-21T15:35:16.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.478+0000 7fdc866e2640 1 Processor -- start 2024-05-21T15:35:16.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.478+0000 7fdc866e2640 1 -- start start 2024-05-21T15:35:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc866e2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc80101990 0x7fdc801992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc866e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc80102bd0 0x7fdc801997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc866e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc801078d0 0x7fdc801a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc866e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8010d5e0 con 0x7fdc80102bd0 2024-05-21T15:35:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc866e2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8010d460 con 0x7fdc80101990 2024-05-21T15:35:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc866e2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8010d760 con 0x7fdc801078d0 2024-05-21T15:35:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc80102bd0 0x7fdc801997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc80102bd0 0x7fdc801997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40318/0 (socket says 172.21.15.38:40318) 2024-05-21T15:35:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc7f7fe640 1 -- 172.21.15.38:0/612554180 learned_addr learned my addr 172.21.15.38:0/612554180 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc7ffff640 1 --2- 172.21.15.38:0/612554180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc80101990 0x7fdc801992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc84c7f640 1 --2- 172.21.15.38:0/612554180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc801078d0 0x7fdc801a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc7f7fe640 1 -- 172.21.15.38:0/612554180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc801078d0 msgr2=0x7fdc801a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc7f7fe640 1 --2- 172.21.15.38:0/612554180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc801078d0 0x7fdc801a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc7f7fe640 1 -- 172.21.15.38:0/612554180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc80101990 msgr2=0x7fdc801992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc7f7fe640 1 --2- 172.21.15.38:0/612554180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc80101990 0x7fdc801992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.479+0000 7fdc7f7fe640 1 -- 172.21.15.38:0/612554180 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc801a0de0 con 0x7fdc80102bd0 2024-05-21T15:35:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.480+0000 7fdc7f7fe640 1 --2- 172.21.15.38:0/612554180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc80102bd0 0x7fdc801997e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400e9f0 tx=0x7fdc7400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.480+0000 7fdc7d7fa640 1 -- 172.21.15.38:0/612554180 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7400cd80 con 0x7fdc80102bd0 2024-05-21T15:35:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.480+0000 7fdc7d7fa640 1 -- 172.21.15.38:0/612554180 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc74005500 con 0x7fdc80102bd0 2024-05-21T15:35:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.480+0000 7fdc866e2640 1 -- 172.21.15.38:0/612554180 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc801a10d0 con 0x7fdc80102bd0 2024-05-21T15:35:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.480+0000 7fdc7d7fa640 1 -- 172.21.15.38:0/612554180 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74010690 con 0x7fdc80102bd0 2024-05-21T15:35:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.480+0000 7fdc866e2640 1 -- 172.21.15.38:0/612554180 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc801a15d0 con 0x7fdc80102bd0 2024-05-21T15:35:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc8d585740 1 Processor -- start 2024-05-21T15:35:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc8d585740 1 -- start start 2024-05-21T15:35:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc7d7fa640 1 -- 172.21.15.38:0/612554180 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdc74010830 con 0x7fdc80102bd0 2024-05-21T15:35:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc8d585740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e83bc410 0x55b1e87608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc8d585740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e8393550 0x55b1e8760e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc8d585740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e839b730 0x55b1e8767d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc8d585740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b1e86c0990 con 0x55b1e83bc410 2024-05-21T15:35:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc8d585740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b1e86685b0 con 0x55b1e8393550 2024-05-21T15:35:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc8d585740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b1e86bc950 con 0x55b1e839b730 2024-05-21T15:35:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc84c7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e83bc410 0x55b1e87608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc84c7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e83bc410 0x55b1e87608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40320/0 (socket says 172.21.15.38:40320) 2024-05-21T15:35:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc84c7f640 1 -- 172.21.15.38:0/3027355598 learned_addr learned my addr 172.21.15.38:0/3027355598 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc7ffff640 1 --2- 172.21.15.38:0/3027355598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e8393550 0x55b1e8760e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc7f7fe640 1 --2- 172.21.15.38:0/3027355598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e839b730 0x55b1e8767d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc84c7f640 1 -- 172.21.15.38:0/3027355598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e839b730 msgr2=0x55b1e8767d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.483+0000 7fdc84c7f640 1 --2- 172.21.15.38:0/3027355598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e839b730 0x55b1e8767d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.484+0000 7fdc84c7f640 1 -- 172.21.15.38:0/3027355598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e8393550 msgr2=0x55b1e8760e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.484+0000 7fdc84c7f640 1 --2- 172.21.15.38:0/3027355598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e8393550 0x55b1e8760e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.484+0000 7fdc84c7f640 1 -- 172.21.15.38:0/3027355598 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1e849c7e0 con 0x55b1e83bc410 2024-05-21T15:35:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.484+0000 7fdc7f7fe640 1 --2- 172.21.15.38:0/3027355598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e839b730 0x55b1e8767d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.484+0000 7fdc7ffff640 1 --2- 172.21.15.38:0/3027355598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e8393550 0x55b1e8760e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.484+0000 7fdc7d7fa640 1 --2- 172.21.15.38:0/612554180 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdc680a2b90 0x7fdc680a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.484+0000 7fdc84c7f640 1 --2- 172.21.15.38:0/3027355598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e83bc410 0x55b1e87608b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdc78010370 tx=0x7fdc78010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325c853766ea9f80 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.484+0000 7fdc7ffff640 1 --2- 172.21.15.38:0/612554180 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdc680a2b90 0x7fdc680a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.484+0000 7fdc667fc640 1 -- 172.21.15.38:0/3027355598 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78016070 con 0x55b1e83bc410 2024-05-21T15:35:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.484+0000 7fdc7d7fa640 1 -- 172.21.15.38:0/612554180 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdc740cf4c0 con 0x7fdc80102bd0 2024-05-21T15:35:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.484+0000 7fdc667fc640 1 -- 172.21.15.38:0/3027355598 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc78004d20 con 0x55b1e83bc410 2024-05-21T15:35:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.484+0000 7fdc667fc640 1 -- 172.21.15.38:0/3027355598 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78005420 con 0x55b1e83bc410 2024-05-21T15:35:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.485+0000 7fdc8d585740 1 -- 172.21.15.38:0/3027355598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e83bc410 msgr2=0x55b1e87608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.485+0000 7fdc7ffff640 1 --2- 172.21.15.38:0/612554180 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdc680a2b90 0x7fdc680a50a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdc70005ab0 tx=0x7fdc70031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.485+0000 7fdc8d585740 1 --2- 172.21.15.38:0/3027355598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e83bc410 0x55b1e87608b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdc78010370 tx=0x7fdc78010830 comp rx=0 tx=0).stop 2024-05-21T15:35:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.485+0000 7fdc8d585740 1 -- 172.21.15.38:0/3027355598 shutdown_connections 2024-05-21T15:35:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.485+0000 7fdc8d585740 1 --2- 172.21.15.38:0/3027355598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e839b730 0x55b1e8767d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.485+0000 7fdc8d585740 1 --2- 172.21.15.38:0/3027355598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e8393550 0x55b1e8760e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.485+0000 7fdc8d585740 1 --2- 172.21.15.38:0/3027355598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e83bc410 0x55b1e87608b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.485+0000 7fdc8d585740 1 -- 172.21.15.38:0/3027355598 >> 172.21.15.38:0/3027355598 conn(0x55b1e83b0560 msgr2=0x55b1e8757e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.485+0000 7fdc8d585740 1 -- 172.21.15.38:0/3027355598 shutdown_connections 2024-05-21T15:35:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.485+0000 7fdc8d585740 1 -- 172.21.15.38:0/3027355598 wait complete. 2024-05-21T15:35:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.485+0000 7fdc8d585740 1 -- start start 2024-05-21T15:35:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc8d585740 1 Processor -- start 2024-05-21T15:35:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc8d585740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e8393550 0x55b1e876a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc8d585740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e839b730 0x55b1e876ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc8d585740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e83bc410 0x55b1e8771d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc8d585740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b1e8768780 con 0x55b1e839b730 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc8d585740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b1e867bb40 con 0x55b1e83bc410 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc8d585740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b1e8768900 con 0x55b1e8393550 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc8d585740 10 client.0 _ll_register_callbacks cb 0x55b1e86700f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e839b730 0x55b1e876ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e839b730 0x55b1e876ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40330/0 (socket says 172.21.15.38:40330) 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc7f7fe640 1 -- 172.21.15.38:0/3632785691 learned_addr learned my addr 172.21.15.38:0/3632785691 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc7ffff640 1 --2- 172.21.15.38:0/3632785691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e8393550 0x55b1e876a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc84c7f640 1 --2- 172.21.15.38:0/3632785691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e83bc410 0x55b1e8771d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc7f7fe640 1 -- 172.21.15.38:0/3632785691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e8393550 msgr2=0x55b1e876a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc7f7fe640 1 --2- 172.21.15.38:0/3632785691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e8393550 0x55b1e876a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc7f7fe640 1 -- 172.21.15.38:0/3632785691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e83bc410 msgr2=0x55b1e8771d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc7f7fe640 1 --2- 172.21.15.38:0/3632785691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e83bc410 0x55b1e8771d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.486+0000 7fdc7f7fe640 1 -- 172.21.15.38:0/3632785691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1e84fcf70 con 0x55b1e839b730 2024-05-21T15:35:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc7f7fe640 1 --2- 172.21.15.38:0/3632785691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e839b730 0x55b1e876ade0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdc74098130 tx=0x7fdc7409b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fab99fd1768f6a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc5f7fe640 1 -- 172.21.15.38:0/3632785691 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc740a0080 con 0x55b1e839b730 2024-05-21T15:35:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc5f7fe640 1 -- 172.21.15.38:0/3632785691 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdc740a44b0 con 0x55b1e839b730 2024-05-21T15:35:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc8d585740 1 -- 172.21.15.38:0/3632785691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc78004d20 con 0x55b1e839b730 2024-05-21T15:35:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc5f7fe640 1 -- 172.21.15.38:0/3632785691 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc740a4940 con 0x55b1e839b730 2024-05-21T15:35:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc8d585740 10 client.15681 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc8d585740 1 -- 172.21.15.38:0/3632785691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1e8394170 con 0x55b1e839b730 2024-05-21T15:35:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc5f7fe640 1 -- 172.21.15.38:0/3632785691 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc78004d20 con 0x55b1e839b730 2024-05-21T15:35:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc5f7fe640 1 -- 172.21.15.38:0/3632785691 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdc740a4cb0 con 0x55b1e839b730 2024-05-21T15:35:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc8d585740 10 client.15681 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc8d585740 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc8d585740 20 client.15681 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc8d585740 1 --2- 172.21.15.38:0/3632785691 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b1e837b1f0 0x55b1e8778090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc8d585740 4 client.15681 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.487+0000 7fdc8d585740 1 -- 172.21.15.38:0/3632785691 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1e86ce0c0 con 0x55b1e837b1f0 2024-05-21T15:35:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.488+0000 7fdc7ffff640 1 --2- 172.21.15.38:0/3632785691 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b1e837b1f0 0x55b1e8778090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.488+0000 7fdc7ffff640 1 --2- 172.21.15.38:0/3632785691 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b1e837b1f0 0x55b1e8778090 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93157c0702afd170 server_cookie=d94b6372e3b2079f in_seq=0 out_seq=0 2024-05-21T15:35:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.488+0000 7fdc5f7fe640 10 client.15681 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.489+0000 7fdc5f7fe640 1 -- 172.21.15.38:0/3632785691 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b1e86ce0c0 con 0x55b1e837b1f0 2024-05-21T15:35:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.489+0000 7fdc5f7fe640 10 client.15681 handle_command_reply: tid=0 2024-05-21T15:35:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.489+0000 7fdc8d585740 1 client.15681 shutdown 2024-05-21T15:35:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.489+0000 7fdc8d585740 1 -- 172.21.15.38:0/3632785691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e839b730 msgr2=0x55b1e876ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.489+0000 7fdc8d585740 1 --2- 172.21.15.38:0/3632785691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e839b730 0x55b1e876ade0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdc74098130 tx=0x7fdc7409b630 comp rx=0 tx=0).stop 2024-05-21T15:35:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.489+0000 7fdc8d585740 1 -- 172.21.15.38:0/3632785691 shutdown_connections 2024-05-21T15:35:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.489+0000 7fdc8d585740 1 --2- 172.21.15.38:0/3632785691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e83bc410 0x55b1e8771d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.489+0000 7fdc8d585740 1 --2- 172.21.15.38:0/3632785691 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b1e837b1f0 0x55b1e8778090 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.489+0000 7fdc8d585740 1 --2- 172.21.15.38:0/3632785691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e839b730 0x55b1e876ade0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.489+0000 7fdc8d585740 1 --2- 172.21.15.38:0/3632785691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e8393550 0x55b1e876a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.489+0000 7fdc5f7fe640 0 client.15681 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.489+0000 7fdc8d585740 1 -- 172.21.15.38:0/3632785691 >> 172.21.15.38:0/3632785691 conn(0x55b1e83b0560 msgr2=0x55b1e8756e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.489+0000 7fdc8d585740 1 -- 172.21.15.38:0/3632785691 shutdown_connections 2024-05-21T15:35:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.489+0000 7fdc8d585740 1 -- 172.21.15.38:0/3632785691 wait complete. 2024-05-21T15:35:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.490+0000 7fdc8d585740 20 client.15681 trim_cache size 0 max 16384 2024-05-21T15:35:16.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc8d585740 1 Processor -- start 2024-05-21T15:35:16.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc8d585740 1 -- start start 2024-05-21T15:35:16.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc8d585740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e8393550 0x55b1e87695d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc8d585740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e839b730 0x55b1e87563d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc8d585740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e83bc410 0x55b1e8756910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc8d585740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdc801a81d0 con 0x55b1e839b730 2024-05-21T15:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc8d585740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc801a9d10 con 0x55b1e8393550 2024-05-21T15:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc8d585740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdc801a88a0 con 0x55b1e83bc410 2024-05-21T15:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e839b730 0x55b1e87563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e839b730 0x55b1e87563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40336/0 (socket says 172.21.15.38:40336) 2024-05-21T15:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc7f7fe640 1 -- 172.21.15.38:0/2532917429 learned_addr learned my addr 172.21.15.38:0/2532917429 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc84c7f640 1 --2- 172.21.15.38:0/2532917429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e83bc410 0x55b1e8756910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc7ffff640 1 --2- 172.21.15.38:0/2532917429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e8393550 0x55b1e87695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc7f7fe640 1 -- 172.21.15.38:0/2532917429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e83bc410 msgr2=0x55b1e8756910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc7f7fe640 1 --2- 172.21.15.38:0/2532917429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e83bc410 0x55b1e8756910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc7f7fe640 1 -- 172.21.15.38:0/2532917429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e8393550 msgr2=0x55b1e87695d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc7f7fe640 1 --2- 172.21.15.38:0/2532917429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e8393550 0x55b1e87695d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc7f7fe640 1 -- 172.21.15.38:0/2532917429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1e87680e0 con 0x55b1e839b730 2024-05-21T15:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.508+0000 7fdc7ffff640 1 --2- 172.21.15.38:0/2532917429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e8393550 0x55b1e87695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.509+0000 7fdc7f7fe640 1 --2- 172.21.15.38:0/2532917429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e839b730 0x55b1e87563d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdc7409b9d0 tx=0x7fdc7409be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513a7d46e904e571 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.509+0000 7fdc65ffb640 1 -- 172.21.15.38:0/2532917429 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc740a0080 con 0x55b1e839b730 2024-05-21T15:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.509+0000 7fdc65ffb640 1 -- 172.21.15.38:0/2532917429 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc740cf110 con 0x55b1e839b730 2024-05-21T15:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.509+0000 7fdc65ffb640 1 -- 172.21.15.38:0/2532917429 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74097a20 con 0x55b1e839b730 2024-05-21T15:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.510+0000 7fdc8d585740 1 -- 172.21.15.38:0/2532917429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e839b730 msgr2=0x55b1e87563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.510+0000 7fdc8d585740 1 --2- 172.21.15.38:0/2532917429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e839b730 0x55b1e87563d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdc7409b9d0 tx=0x7fdc7409be50 comp rx=0 tx=0).stop 2024-05-21T15:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.510+0000 7fdc8d585740 1 -- 172.21.15.38:0/2532917429 shutdown_connections 2024-05-21T15:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.510+0000 7fdc8d585740 1 --2- 172.21.15.38:0/2532917429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e83bc410 0x55b1e8756910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.510+0000 7fdc8d585740 1 --2- 172.21.15.38:0/2532917429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e839b730 0x55b1e87563d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.510+0000 7fdc8d585740 1 --2- 172.21.15.38:0/2532917429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e8393550 0x55b1e87695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.510+0000 7fdc8d585740 1 -- 172.21.15.38:0/2532917429 >> 172.21.15.38:0/2532917429 conn(0x55b1e83b0560 msgr2=0x55b1e8768ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:16.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.510+0000 7fdc8d585740 1 -- 172.21.15.38:0/2532917429 shutdown_connections 2024-05-21T15:35:16.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.510+0000 7fdc8d585740 1 -- 172.21.15.38:0/2532917429 wait complete. 2024-05-21T15:35:16.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.510+0000 7fdc8d585740 1 -- start start 2024-05-21T15:35:16.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.510+0000 7fdc8d585740 1 Processor -- start 2024-05-21T15:35:16.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc8d585740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e8393550 0x55b1e876e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc8d585740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e839b730 0x55b1e876ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc8d585740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e83bc410 0x55b1e876fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc8d585740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdc801a9640 con 0x55b1e83bc410 2024-05-21T15:35:16.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc8d585740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc801a8f70 con 0x55b1e839b730 2024-05-21T15:35:16.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc8d585740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdc801a61b0 con 0x55b1e8393550 2024-05-21T15:35:16.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc8d585740 10 client.0 _ll_register_callbacks cb 0x7fdc7400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:16.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc84c7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e83bc410 0x55b1e876fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc84c7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e83bc410 0x55b1e876fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40344/0 (socket says 172.21.15.38:40344) 2024-05-21T15:35:16.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc84c7f640 1 -- 172.21.15.38:0/1548983521 learned_addr learned my addr 172.21.15.38:0/1548983521 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:16.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc7ffff640 1 --2- 172.21.15.38:0/1548983521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e8393550 0x55b1e876e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc7f7fe640 1 --2- 172.21.15.38:0/1548983521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e839b730 0x55b1e876ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc84c7f640 1 -- 172.21.15.38:0/1548983521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e8393550 msgr2=0x55b1e876e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc84c7f640 1 --2- 172.21.15.38:0/1548983521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e8393550 0x55b1e876e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc84c7f640 1 -- 172.21.15.38:0/1548983521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e839b730 msgr2=0x55b1e876ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc84c7f640 1 --2- 172.21.15.38:0/1548983521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e839b730 0x55b1e876ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc84c7f640 1 -- 172.21.15.38:0/1548983521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1e87701c0 con 0x55b1e83bc410 2024-05-21T15:35:16.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc84c7f640 1 --2- 172.21.15.38:0/1548983521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e83bc410 0x55b1e876fc00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdc78010370 tx=0x7fdc78003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1420af968d06b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:16.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:16.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc5f7fe640 1 -- 172.21.15.38:0/1548983521 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7802b060 con 0x55b1e83bc410 2024-05-21T15:35:16.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc5f7fe640 1 -- 172.21.15.38:0/1548983521 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdc780052b0 con 0x55b1e83bc410 2024-05-21T15:35:16.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.511+0000 7fdc8d585740 1 -- 172.21.15.38:0/1548983521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc740cf110 con 0x55b1e83bc410 2024-05-21T15:35:16.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.512+0000 7fdc5f7fe640 1 -- 172.21.15.38:0/1548983521 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc78016070 con 0x55b1e83bc410 2024-05-21T15:35:16.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.512+0000 7fdc8d585740 10 client.15687 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:16.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.512+0000 7fdc8d585740 1 -- 172.21.15.38:0/1548983521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1e8771420 con 0x55b1e83bc410 2024-05-21T15:35:16.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.512+0000 7fdc5f7fe640 1 -- 172.21.15.38:0/1548983521 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc740cf110 con 0x55b1e83bc410 2024-05-21T15:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.512+0000 7fdc5f7fe640 1 -- 172.21.15.38:0/1548983521 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdc7800f040 con 0x55b1e83bc410 2024-05-21T15:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.512+0000 7fdc8d585740 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.512+0000 7fdc8d585740 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.512+0000 7fdc8d585740 20 client.15687 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.512+0000 7fdc8d585740 1 --2- 172.21.15.38:0/1548983521 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b1e8350630 0x55b1e87e2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.512+0000 7fdc8d585740 4 client.15687 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.512+0000 7fdc8d585740 1 -- 172.21.15.38:0/1548983521 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b1e86c8410 con 0x55b1e8350630 2024-05-21T15:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.512+0000 7fdc7ffff640 1 --2- 172.21.15.38:0/1548983521 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b1e8350630 0x55b1e87e2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.513+0000 7fdc5f7fe640 10 client.15687 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.513+0000 7fdc7ffff640 1 --2- 172.21.15.38:0/1548983521 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b1e8350630 0x55b1e87e2660 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2dd6cb9e2a70e5e server_cookie=d6b44a825241bdf2 in_seq=0 out_seq=0 2024-05-21T15:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc5f7fe640 1 -- 172.21.15.38:0/1548983521 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b1e86c8410 con 0x55b1e8350630 2024-05-21T15:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc5f7fe640 10 client.15687 handle_command_reply: tid=0 2024-05-21T15:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc8d585740 1 client.15687 shutdown 2024-05-21T15:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc8d585740 1 -- 172.21.15.38:0/1548983521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e83bc410 msgr2=0x55b1e876fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc8d585740 1 --2- 172.21.15.38:0/1548983521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e83bc410 0x55b1e876fc00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdc78010370 tx=0x7fdc78003ba0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc8d585740 1 -- 172.21.15.38:0/1548983521 shutdown_connections 2024-05-21T15:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc8d585740 1 --2- 172.21.15.38:0/1548983521 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b1e8350630 0x55b1e87e2660 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc8d585740 1 --2- 172.21.15.38:0/1548983521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b1e83bc410 0x55b1e876fc00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc8d585740 1 --2- 172.21.15.38:0/1548983521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1e839b730 0x55b1e876ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc8d585740 1 --2- 172.21.15.38:0/1548983521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b1e8393550 0x55b1e876e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc8d585740 1 -- 172.21.15.38:0/1548983521 >> 172.21.15.38:0/1548983521 conn(0x55b1e83b0560 msgr2=0x55b1e87574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc5f7fe640 0 client.15687 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc8d585740 1 -- 172.21.15.38:0/1548983521 shutdown_connections 2024-05-21T15:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc8d585740 1 -- 172.21.15.38:0/1548983521 wait complete. 2024-05-21T15:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.514+0000 7fdc8d585740 20 client.15687 trim_cache size 0 max 16384 2024-05-21T15:35:16.520 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:35:16.520 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-05-21T15:35:16.520 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "c03c90b1-8d25-4614-a944-33ed999d0535", 2024-05-21T15:35:16.520 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-05-21T15:35:16.520 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:35:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.515+0000 7fdc866e2640 1 -- 172.21.15.38:0/612554180 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdc680a2b90 msgr2=0x7fdc680a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.515+0000 7fdc866e2640 1 --2- 172.21.15.38:0/612554180 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdc680a2b90 0x7fdc680a50a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdc70005ab0 tx=0x7fdc70031400 comp rx=0 tx=0).stop 2024-05-21T15:35:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.515+0000 7fdc866e2640 1 -- 172.21.15.38:0/612554180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc80102bd0 msgr2=0x7fdc801997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.515+0000 7fdc866e2640 1 --2- 172.21.15.38:0/612554180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc80102bd0 0x7fdc801997e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400e9f0 tx=0x7fdc7400edf0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.515+0000 7fdc866e2640 1 -- 172.21.15.38:0/612554180 shutdown_connections 2024-05-21T15:35:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.515+0000 7fdc866e2640 1 --2- 172.21.15.38:0/612554180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc801078d0 0x7fdc801a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.515+0000 7fdc866e2640 1 --2- 172.21.15.38:0/612554180 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdc680a2b90 0x7fdc680a50a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.515+0000 7fdc866e2640 1 --2- 172.21.15.38:0/612554180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc80102bd0 0x7fdc801997e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.515+0000 7fdc866e2640 1 --2- 172.21.15.38:0/612554180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc80101990 0x7fdc801992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.515+0000 7fdc866e2640 1 -- 172.21.15.38:0/612554180 >> 172.21.15.38:0/612554180 conn(0x7fdc800fcba0 msgr2=0x7fdc80108b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.515+0000 7fdc866e2640 1 -- 172.21.15.38:0/612554180 shutdown_connections 2024-05-21T15:35:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.515+0000 7fdc866e2640 1 -- 172.21.15.38:0/612554180 wait complete. 2024-05-21T15:35:16.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:35:16.526 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:35:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.644+0000 7fa4eb691640 1 -- 172.21.15.38:0/841727226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4e4101980 msgr2=0x7fa4e4102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.644+0000 7fa4eb691640 1 --2- 172.21.15.38:0/841727226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4e4101980 0x7fa4e4102680 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009900 tx=0x7fa4e002a410 comp rx=0 tx=0).stop 2024-05-21T15:35:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.644+0000 7fa4eb691640 1 -- 172.21.15.38:0/841727226 shutdown_connections 2024-05-21T15:35:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.644+0000 7fa4eb691640 1 --2- 172.21.15.38:0/841727226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4e41078c0 0x7fa4e4109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.644+0000 7fa4eb691640 1 --2- 172.21.15.38:0/841727226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e4102bc0 0x7fa4e4102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.644+0000 7fa4eb691640 1 --2- 172.21.15.38:0/841727226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4e4101980 0x7fa4e4102680 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.644+0000 7fa4eb691640 1 -- 172.21.15.38:0/841727226 >> 172.21.15.38:0/841727226 conn(0x7fa4e40fcb70 msgr2=0x7fa4e40ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.644+0000 7fa4eb691640 1 -- 172.21.15.38:0/841727226 shutdown_connections 2024-05-21T15:35:16.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.644+0000 7fa4eb691640 1 -- 172.21.15.38:0/841727226 wait complete. 2024-05-21T15:35:16.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.645+0000 7fa4eb691640 1 Processor -- start 2024-05-21T15:35:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.645+0000 7fa4eb691640 1 -- start start 2024-05-21T15:35:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.645+0000 7fa4eb691640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4e4101980 0x7fa4e41992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.645+0000 7fa4eb691640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e4102bc0 0x7fa4e4199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.645+0000 7fa4eb691640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4e41078c0 0x7fa4e41a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.645+0000 7fa4eb691640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e410d620 con 0x7fa4e41078c0 2024-05-21T15:35:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.645+0000 7fa4eb691640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e410d4a0 con 0x7fa4e4101980 2024-05-21T15:35:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.645+0000 7fa4eb691640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e410d7a0 con 0x7fa4e4102bc0 2024-05-21T15:35:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.645+0000 7fa4e9c2e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4e41078c0 0x7fa4e41a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.645+0000 7fa4e9c2e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4e41078c0 0x7fa4e41a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40360/0 (socket says 172.21.15.38:40360) 2024-05-21T15:35:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.645+0000 7fa4e9c2e640 1 -- 172.21.15.38:0/480149519 learned_addr learned my addr 172.21.15.38:0/480149519 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.645+0000 7fa4e942d640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4e4101980 0x7fa4e41992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.645+0000 7fa4e8c2c640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e4102bc0 0x7fa4e4199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.646+0000 7fa4e9c2e640 1 -- 172.21.15.38:0/480149519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e4102bc0 msgr2=0x7fa4e4199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.646+0000 7fa4e9c2e640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e4102bc0 0x7fa4e4199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.646+0000 7fa4e9c2e640 1 -- 172.21.15.38:0/480149519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4e4101980 msgr2=0x7fa4e41992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.646+0000 7fa4e9c2e640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4e4101980 0x7fa4e41992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.646+0000 7fa4e9c2e640 1 -- 172.21.15.38:0/480149519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e41a0d50 con 0x7fa4e41078c0 2024-05-21T15:35:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.646+0000 7fa4e8c2c640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e4102bc0 0x7fa4e4199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.646+0000 7fa4e9c2e640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4e41078c0 0x7fa4e41a0790 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00cce0 tx=0x7fa4dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.646+0000 7fa4da7fc640 1 -- 172.21.15.38:0/480149519 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00f450 con 0x7fa4e41078c0 2024-05-21T15:35:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.646+0000 7fa4da7fc640 1 -- 172.21.15.38:0/480149519 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4dc00fad0 con 0x7fa4e41078c0 2024-05-21T15:35:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.646+0000 7fa4da7fc640 1 -- 172.21.15.38:0/480149519 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc018c10 con 0x7fa4e41078c0 2024-05-21T15:35:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.646+0000 7fa4eb691640 1 -- 172.21.15.38:0/480149519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e41a0fe0 con 0x7fa4e41078c0 2024-05-21T15:35:16.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.646+0000 7fa4eb691640 1 -- 172.21.15.38:0/480149519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e41a1450 con 0x7fa4e41078c0 2024-05-21T15:35:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.649+0000 7fa4f260f740 1 Processor -- start 2024-05-21T15:35:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.649+0000 7fa4da7fc640 1 -- 172.21.15.38:0/480149519 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa4dc00f5f0 con 0x7fa4e41078c0 2024-05-21T15:35:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.649+0000 7fa4f260f740 1 -- start start 2024-05-21T15:35:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4f260f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf081abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4f260f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 0x55dbf081b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4f260f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf0822030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4f260f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dbf077fd00 con 0x55dbf047b390 2024-05-21T15:35:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4f260f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dbf07274c0 con 0x55dbf045a6b0 2024-05-21T15:35:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4f260f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55dbf077bcc0 con 0x55dbf04524d0 2024-05-21T15:35:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4e8c2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf081abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4e8c2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf081abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40366/0 (socket says 172.21.15.38:40366) 2024-05-21T15:35:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4e8c2c640 1 -- 172.21.15.38:0/1006509925 learned_addr learned my addr 172.21.15.38:0/1006509925 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4e942d640 1 --2- 172.21.15.38:0/1006509925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 0x55dbf081b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4e8c2c640 1 --2- 172.21.15.38:0/1006509925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf0822030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4e8c2c640 1 -- 172.21.15.38:0/1006509925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 msgr2=0x55dbf081b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4e8c2c640 1 --2- 172.21.15.38:0/1006509925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 0x55dbf081b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4e8c2c640 1 -- 172.21.15.38:0/1006509925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 msgr2=0x55dbf0822030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4e8c2c640 1 --2- 172.21.15.38:0/1006509925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf0822030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4e8c2c640 1 -- 172.21.15.38:0/1006509925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf055d7d0 con 0x55dbf047b390 2024-05-21T15:35:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4da7fc640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa4d00a2c10 0x7fa4d00a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4e8c2c640 1 --2- 172.21.15.38:0/1006509925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf081abb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40155f0 tx=0x7fa4d4009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3dc86459677ca65 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4e942d640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa4d00a2c10 0x7fa4d00a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.650+0000 7fa4da7fc640 1 -- 172.21.15.38:0/480149519 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa4dc0cf700 con 0x7fa4e41078c0 2024-05-21T15:35:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.651+0000 7fa4cf7fe640 1 -- 172.21.15.38:0/1006509925 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40134c0 con 0x55dbf047b390 2024-05-21T15:35:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.651+0000 7fa4cf7fe640 1 -- 172.21.15.38:0/1006509925 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d401b070 con 0x55dbf047b390 2024-05-21T15:35:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.651+0000 7fa4cf7fe640 1 -- 172.21.15.38:0/1006509925 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4012890 con 0x55dbf047b390 2024-05-21T15:35:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.652+0000 7fa4f260f740 1 -- 172.21.15.38:0/1006509925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 msgr2=0x55dbf081abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.652+0000 7fa4e942d640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa4d00a2c10 0x7fa4d00a5120 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa4e002aa00 tx=0x7fa4e00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:16.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.652+0000 7fa4f260f740 1 --2- 172.21.15.38:0/1006509925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf081abb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40155f0 tx=0x7fa4d4009660 comp rx=0 tx=0).stop 2024-05-21T15:35:16.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.652+0000 7fa4f260f740 1 -- 172.21.15.38:0/1006509925 shutdown_connections 2024-05-21T15:35:16.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.652+0000 7fa4f260f740 1 --2- 172.21.15.38:0/1006509925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf0822030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.652+0000 7fa4f260f740 1 --2- 172.21.15.38:0/1006509925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 0x55dbf081b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.652+0000 7fa4f260f740 1 --2- 172.21.15.38:0/1006509925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf081abb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.652+0000 7fa4f260f740 1 -- 172.21.15.38:0/1006509925 >> 172.21.15.38:0/1006509925 conn(0x55dbf046f4e0 msgr2=0x55dbf0816540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:16.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.652+0000 7fa4f260f740 1 -- 172.21.15.38:0/1006509925 shutdown_connections 2024-05-21T15:35:16.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.652+0000 7fa4f260f740 1 -- 172.21.15.38:0/1006509925 wait complete. 2024-05-21T15:35:16.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.652+0000 7fa4f260f740 1 -- start start 2024-05-21T15:35:16.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.652+0000 7fa4f260f740 1 Processor -- start 2024-05-21T15:35:16.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4f260f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 0x55dbf0824ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4f260f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf0825020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4f260f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf082bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4f260f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dbf0822a20 con 0x55dbf047b390 2024-05-21T15:35:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4f260f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dbf073ab90 con 0x55dbf045a6b0 2024-05-21T15:35:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4f260f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55dbf0822ba0 con 0x55dbf04524d0 2024-05-21T15:35:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4f260f740 10 client.0 _ll_register_callbacks cb 0x55dbf0776da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4e9c2e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf082bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4e9c2e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf082bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40376/0 (socket says 172.21.15.38:40376) 2024-05-21T15:35:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4e9c2e640 1 -- 172.21.15.38:0/3947169212 learned_addr learned my addr 172.21.15.38:0/3947169212 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4e8c2c640 1 --2- 172.21.15.38:0/3947169212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf0825020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4e942d640 1 --2- 172.21.15.38:0/3947169212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 0x55dbf0824ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4e9c2e640 1 -- 172.21.15.38:0/3947169212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 msgr2=0x55dbf0824ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4e9c2e640 1 --2- 172.21.15.38:0/3947169212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 0x55dbf0824ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4e9c2e640 1 -- 172.21.15.38:0/3947169212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 msgr2=0x55dbf0825020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4e9c2e640 1 --2- 172.21.15.38:0/3947169212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf0825020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4e9c2e640 1 -- 172.21.15.38:0/3947169212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf04933f0 con 0x55dbf047b390 2024-05-21T15:35:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4e942d640 1 --2- 172.21.15.38:0/3947169212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 0x55dbf0824ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4e9c2e640 1 --2- 172.21.15.38:0/3947169212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf082bf80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc09bea0 tx=0x7fa4dc09b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c42d485be56c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4ccff9640 1 -- 172.21.15.38:0/3947169212 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc0a02c0 con 0x55dbf047b390 2024-05-21T15:35:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4f260f740 1 -- 172.21.15.38:0/3947169212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d401b070 con 0x55dbf047b390 2024-05-21T15:35:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.653+0000 7fa4ccff9640 1 -- 172.21.15.38:0/3947169212 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa4dc0a46f0 con 0x55dbf047b390 2024-05-21T15:35:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.654+0000 7fa4ccff9640 1 -- 172.21.15.38:0/3947169212 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4dc0a4b80 con 0x55dbf047b390 2024-05-21T15:35:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.654+0000 7fa4f260f740 10 client.15699 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.654+0000 7fa4f260f740 1 -- 172.21.15.38:0/3947169212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dbf04530f0 con 0x55dbf047b390 2024-05-21T15:35:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.654+0000 7fa4ccff9640 1 -- 172.21.15.38:0/3947169212 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d401b070 con 0x55dbf047b390 2024-05-21T15:35:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.654+0000 7fa4ccff9640 1 -- 172.21.15.38:0/3947169212 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa4dc09b360 con 0x55dbf047b390 2024-05-21T15:35:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.654+0000 7fa4f260f740 10 client.15699 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.654+0000 7fa4f260f740 10 client.15699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.654+0000 7fa4f260f740 20 client.15699 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.654+0000 7fa4f260f740 1 --2- 172.21.15.38:0/3947169212 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55dbf0439f60 0x55dbf0832880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.654+0000 7fa4f260f740 4 client.15699 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.654+0000 7fa4f260f740 1 -- 172.21.15.38:0/3947169212 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbf078d430 con 0x55dbf0439f60 2024-05-21T15:35:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.654+0000 7fa4e942d640 1 --2- 172.21.15.38:0/3947169212 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55dbf0439f60 0x55dbf0832880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.655+0000 7fa4ccff9640 10 client.15699 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.655+0000 7fa4e942d640 1 --2- 172.21.15.38:0/3947169212 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55dbf0439f60 0x55dbf0832880 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ecd84087234534 server_cookie=99fe066e80bad0f1 in_seq=0 out_seq=0 2024-05-21T15:35:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.655+0000 7fa4ccff9640 1 -- 172.21.15.38:0/3947169212 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dbf078d430 con 0x55dbf0439f60 2024-05-21T15:35:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.655+0000 7fa4ccff9640 10 client.15699 handle_command_reply: tid=0 2024-05-21T15:35:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.656+0000 7fa4f260f740 1 client.15699 shutdown 2024-05-21T15:35:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.656+0000 7fa4f260f740 1 -- 172.21.15.38:0/3947169212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 msgr2=0x55dbf082bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.656+0000 7fa4f260f740 1 --2- 172.21.15.38:0/3947169212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf082bf80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc09bea0 tx=0x7fa4dc09b870 comp rx=0 tx=0).stop 2024-05-21T15:35:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.656+0000 7fa4f260f740 1 -- 172.21.15.38:0/3947169212 shutdown_connections 2024-05-21T15:35:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.656+0000 7fa4f260f740 1 --2- 172.21.15.38:0/3947169212 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55dbf0439f60 0x55dbf0832880 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.656+0000 7fa4f260f740 1 --2- 172.21.15.38:0/3947169212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf082bf80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.656+0000 7fa4f260f740 1 --2- 172.21.15.38:0/3947169212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf0825020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.656+0000 7fa4ccff9640 0 client.15699 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.656+0000 7fa4f260f740 1 --2- 172.21.15.38:0/3947169212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 0x55dbf0824ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.656+0000 7fa4f260f740 1 -- 172.21.15.38:0/3947169212 >> 172.21.15.38:0/3947169212 conn(0x55dbf046f4e0 msgr2=0x55dbf08155c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.656+0000 7fa4f260f740 1 -- 172.21.15.38:0/3947169212 shutdown_connections 2024-05-21T15:35:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.656+0000 7fa4f260f740 1 -- 172.21.15.38:0/3947169212 wait complete. 2024-05-21T15:35:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.656+0000 7fa4f260f740 20 client.15699 trim_cache size 0 max 16384 2024-05-21T15:35:16.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.674+0000 7fa4f260f740 1 Processor -- start 2024-05-21T15:35:16.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.674+0000 7fa4f260f740 1 -- start start 2024-05-21T15:35:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.674+0000 7fa4f260f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf04524d0 0x55dbf081f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.674+0000 7fa4f260f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf082bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.674+0000 7fa4f260f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf047b390 0x55dbf0824ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.674+0000 7fa4f260f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41aa2a0 con 0x55dbf04524d0 2024-05-21T15:35:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.674+0000 7fa4f260f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41abde0 con 0x55dbf045a6b0 2024-05-21T15:35:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.674+0000 7fa4f260f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41aa970 con 0x55dbf047b390 2024-05-21T15:35:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.674+0000 7fa4e942d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf04524d0 0x55dbf081f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.674+0000 7fa4e942d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf04524d0 0x55dbf081f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40390/0 (socket says 172.21.15.38:40390) 2024-05-21T15:35:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.674+0000 7fa4e942d640 1 -- 172.21.15.38:0/4133389448 learned_addr learned my addr 172.21.15.38:0/4133389448 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.675+0000 7fa4e8c2c640 1 --2- 172.21.15.38:0/4133389448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf082bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.675+0000 7fa4e942d640 1 -- 172.21.15.38:0/4133389448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf047b390 msgr2=0x55dbf0824ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.675+0000 7fa4e9c2e640 1 --2- 172.21.15.38:0/4133389448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf047b390 0x55dbf0824ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.675+0000 7fa4e942d640 1 --2- 172.21.15.38:0/4133389448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf047b390 0x55dbf0824ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.675+0000 7fa4e942d640 1 -- 172.21.15.38:0/4133389448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 msgr2=0x55dbf082bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.675+0000 7fa4e942d640 1 --2- 172.21.15.38:0/4133389448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf082bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.675+0000 7fa4e942d640 1 -- 172.21.15.38:0/4133389448 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf0825c20 con 0x55dbf04524d0 2024-05-21T15:35:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.675+0000 7fa4e8c2c640 1 --2- 172.21.15.38:0/4133389448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf082bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.675+0000 7fa4e942d640 1 --2- 172.21.15.38:0/4133389448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf04524d0 0x55dbf081f5b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0005f70 tx=0x7fa4e003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4756f6ab9f5e5fd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.675+0000 7fa4ceffd640 1 -- 172.21.15.38:0/4133389448 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0035040 con 0x55dbf04524d0 2024-05-21T15:35:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.675+0000 7fa4ceffd640 1 -- 172.21.15.38:0/4133389448 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e00326c0 con 0x55dbf04524d0 2024-05-21T15:35:16.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.675+0000 7fa4ceffd640 1 -- 172.21.15.38:0/4133389448 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00524b0 con 0x55dbf04524d0 2024-05-21T15:35:16.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.676+0000 7fa4f260f740 1 -- 172.21.15.38:0/4133389448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf04524d0 msgr2=0x55dbf081f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.676+0000 7fa4f260f740 1 --2- 172.21.15.38:0/4133389448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf04524d0 0x55dbf081f5b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0005f70 tx=0x7fa4e003f000 comp rx=0 tx=0).stop 2024-05-21T15:35:16.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.676+0000 7fa4f260f740 1 -- 172.21.15.38:0/4133389448 shutdown_connections 2024-05-21T15:35:16.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.676+0000 7fa4f260f740 1 --2- 172.21.15.38:0/4133389448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf047b390 0x55dbf0824ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.676+0000 7fa4f260f740 1 --2- 172.21.15.38:0/4133389448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf082bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.676+0000 7fa4f260f740 1 --2- 172.21.15.38:0/4133389448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf04524d0 0x55dbf081f5b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.676+0000 7fa4f260f740 1 -- 172.21.15.38:0/4133389448 >> 172.21.15.38:0/4133389448 conn(0x55dbf046f4e0 msgr2=0x55dbf0815f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:16.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.676+0000 7fa4f260f740 1 -- 172.21.15.38:0/4133389448 shutdown_connections 2024-05-21T15:35:16.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.676+0000 7fa4f260f740 1 -- 172.21.15.38:0/4133389448 wait complete. 2024-05-21T15:35:16.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.676+0000 7fa4f260f740 1 -- start start 2024-05-21T15:35:16.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.676+0000 7fa4f260f740 1 Processor -- start 2024-05-21T15:35:16.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4f260f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 0x55dbf08294a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4f260f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf08299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4f260f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf082a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4f260f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41ab710 con 0x55dbf047b390 2024-05-21T15:35:16.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4f260f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41ab040 con 0x55dbf045a6b0 2024-05-21T15:35:16.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4f260f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e41a8280 con 0x55dbf04524d0 2024-05-21T15:35:16.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4f260f740 10 client.0 _ll_register_callbacks cb 0x7fa4dc0077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:16.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4e9c2e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf082a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4e9c2e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf082a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40396/0 (socket says 172.21.15.38:40396) 2024-05-21T15:35:16.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4e9c2e640 1 -- 172.21.15.38:0/2795877672 learned_addr learned my addr 172.21.15.38:0/2795877672 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:16.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4e8c2c640 1 --2- 172.21.15.38:0/2795877672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf08299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4e942d640 1 --2- 172.21.15.38:0/2795877672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 0x55dbf08294a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4e9c2e640 1 -- 172.21.15.38:0/2795877672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 msgr2=0x55dbf08294a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4e9c2e640 1 --2- 172.21.15.38:0/2795877672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 0x55dbf08294a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4e9c2e640 1 -- 172.21.15.38:0/2795877672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 msgr2=0x55dbf08299e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:16.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4e9c2e640 1 --2- 172.21.15.38:0/2795877672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf08299e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4e9c2e640 1 -- 172.21.15.38:0/2795877672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf082aed0 con 0x55dbf047b390 2024-05-21T15:35:16.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4e9c2e640 1 --2- 172.21.15.38:0/2795877672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf082a910 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0a4650 tx=0x7fa4dc018510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd69419d9d18251 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:16.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.677+0000 7fa4ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:16.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.678+0000 7fa4ccff9640 1 -- 172.21.15.38:0/2795877672 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc0981b0 con 0x55dbf047b390 2024-05-21T15:35:16.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.678+0000 7fa4ccff9640 1 -- 172.21.15.38:0/2795877672 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa4dc09b360 con 0x55dbf047b390 2024-05-21T15:35:16.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.678+0000 7fa4f260f740 1 -- 172.21.15.38:0/2795877672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4e00326c0 con 0x55dbf047b390 2024-05-21T15:35:16.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.678+0000 7fa4ccff9640 1 -- 172.21.15.38:0/2795877672 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4dc0a6e60 con 0x55dbf047b390 2024-05-21T15:35:16.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.678+0000 7fa4f260f740 10 client.15705 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:16.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.678+0000 7fa4f260f740 1 -- 172.21.15.38:0/2795877672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dbf082c0d0 con 0x55dbf047b390 2024-05-21T15:35:16.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.678+0000 7fa4ccff9640 1 -- 172.21.15.38:0/2795877672 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e00326c0 con 0x55dbf047b390 2024-05-21T15:35:16.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.678+0000 7fa4ccff9640 1 -- 172.21.15.38:0/2795877672 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa4dc0bab50 con 0x55dbf047b390 2024-05-21T15:35:16.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.678+0000 7fa4f260f740 10 client.15705 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:16.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.678+0000 7fa4f260f740 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.678+0000 7fa4f260f740 20 client.15705 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.678+0000 7fa4f260f740 1 --2- 172.21.15.38:0/2795877672 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55dbf040f3a0 0x55dbf089d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.678+0000 7fa4f260f740 4 client.15705 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:35:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.678+0000 7fa4f260f740 1 -- 172.21.15.38:0/2795877672 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dbf0787780 con 0x55dbf040f3a0 2024-05-21T15:35:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4e942d640 1 --2- 172.21.15.38:0/2795877672 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55dbf040f3a0 0x55dbf089d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4e942d640 1 --2- 172.21.15.38:0/2795877672 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55dbf040f3a0 0x55dbf089d0e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bb04e823924dddf server_cookie=84818c9398647099 in_seq=0 out_seq=0 2024-05-21T15:35:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4ccff9640 10 client.15705 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4ccff9640 1 -- 172.21.15.38:0/2795877672 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dbf0787780 con 0x55dbf040f3a0 2024-05-21T15:35:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4ccff9640 10 client.15705 handle_command_reply: tid=0 2024-05-21T15:35:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4f260f740 1 client.15705 shutdown 2024-05-21T15:35:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4f260f740 1 -- 172.21.15.38:0/2795877672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 msgr2=0x55dbf082a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4f260f740 1 --2- 172.21.15.38:0/2795877672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf082a910 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0a4650 tx=0x7fa4dc018510 comp rx=0 tx=0).stop 2024-05-21T15:35:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4f260f740 1 -- 172.21.15.38:0/2795877672 shutdown_connections 2024-05-21T15:35:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4f260f740 1 --2- 172.21.15.38:0/2795877672 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55dbf040f3a0 0x55dbf089d0e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4f260f740 1 --2- 172.21.15.38:0/2795877672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dbf047b390 0x55dbf082a910 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4f260f740 1 --2- 172.21.15.38:0/2795877672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbf045a6b0 0x55dbf08299e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4ccff9640 0 client.15705 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:16.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4f260f740 1 --2- 172.21.15.38:0/2795877672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dbf04524d0 0x55dbf08294a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.679+0000 7fa4f260f740 1 -- 172.21.15.38:0/2795877672 >> 172.21.15.38:0/2795877672 conn(0x55dbf046f4e0 msgr2=0x55dbf0837030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:16.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4f260f740 1 -- 172.21.15.38:0/2795877672 shutdown_connections 2024-05-21T15:35:16.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4f260f740 1 -- 172.21.15.38:0/2795877672 wait complete. 2024-05-21T15:35:16.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4f260f740 20 client.15705 trim_cache size 0 max 16384 2024-05-21T15:35:16.685 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:35:16.685 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:35:16.685 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-05-21T15:35:16.685 INFO:teuthology.orchestra.run.smithi038.stdout: "c03c90b1-8d25-4614-a944-33ed999d0535": { 2024-05-21T15:35:16.686 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-05-21T15:35:16.686 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "c03c90b1-8d25-4614-a944-33ed999d0535", 2024-05-21T15:35:16.686 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-05-21T15:35:16.686 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:35:16.686 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:35:16.686 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:35:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4eb691640 1 -- 172.21.15.38:0/480149519 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa4d00a2c10 msgr2=0x7fa4d00a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4eb691640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa4d00a2c10 0x7fa4d00a5120 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa4e002aa00 tx=0x7fa4e00315b0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4eb691640 1 -- 172.21.15.38:0/480149519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4e41078c0 msgr2=0x7fa4e41a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4eb691640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4e41078c0 0x7fa4e41a0790 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00cce0 tx=0x7fa4dc00a620 comp rx=0 tx=0).stop 2024-05-21T15:35:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4eb691640 1 -- 172.21.15.38:0/480149519 shutdown_connections 2024-05-21T15:35:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4eb691640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa4d00a2c10 0x7fa4d00a5120 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4eb691640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4e41078c0 0x7fa4e41a0790 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4eb691640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4e4102bc0 0x7fa4e4199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4eb691640 1 --2- 172.21.15.38:0/480149519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4e4101980 0x7fa4e41992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4eb691640 1 -- 172.21.15.38:0/480149519 >> 172.21.15.38:0/480149519 conn(0x7fa4e40fcb70 msgr2=0x7fa4e4108b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4eb691640 1 -- 172.21.15.38:0/480149519 shutdown_connections 2024-05-21T15:35:16.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:16.680+0000 7fa4eb691640 1 -- 172.21.15.38:0/480149519 wait complete. 2024-05-21T15:35:16.691 INFO:tasks.cephfs.filesystem:scrub status for tag:c03c90b1-8d25-4614-a944-33ed999d0535 - {'path': '/', 'tag': 'c03c90b1-8d25-4614-a944-33ed999d0535', 'options': 'recursive,force'} 2024-05-21T15:35:18.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:17 smithi045 ceph-mon[36409]: pgmap v377: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:35:18.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:17 smithi045 ceph-mon[36409]: scrub queued for path: / 2024-05-21T15:35:18.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:17 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:35:18.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:17 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:35:18.261 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:17 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:35:18.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:17 smithi038 ceph-mon[31995]: pgmap v377: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:35:18.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:17 smithi038 ceph-mon[31995]: scrub queued for path: / 2024-05-21T15:35:18.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:17 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:35:18.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:17 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:35:18.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:17 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:35:18.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:17 smithi053 ceph-mon[36496]: pgmap v377: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:35:18.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:17 smithi053 ceph-mon[36496]: scrub queued for path: / 2024-05-21T15:35:18.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:17 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:35:18.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:17 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:35:18.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:17 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:35:20.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:19 smithi045 ceph-mon[36409]: pgmap v378: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-21T15:35:20.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:19 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:35:20.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:19 smithi038 ceph-mon[31995]: pgmap v378: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-21T15:35:20.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:19 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:35:20.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:19 smithi053 ceph-mon[36496]: pgmap v378: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-21T15:35:20.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:19 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:35:22.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:21 smithi045 ceph-mon[36409]: pgmap v379: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:35:22.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:21 smithi038 ceph-mon[31995]: pgmap v379: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:35:22.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:21 smithi053 ceph-mon[36496]: pgmap v379: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.093+0000 7fb982338640 1 -- 172.21.15.38:0/2847709277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c108a20 msgr2=0x7fb97c10ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.093+0000 7fb982338640 1 --2- 172.21.15.38:0/2847709277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c108a20 0x7fb97c10ae40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb974009ca0 tx=0x7fb97402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.093+0000 7fb982338640 1 -- 172.21.15.38:0/2847709277 shutdown_connections 2024-05-21T15:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.093+0000 7fb982338640 1 --2- 172.21.15.38:0/2847709277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c108a20 0x7fb97c10ae40 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.093+0000 7fb982338640 1 --2- 172.21.15.38:0/2847709277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb97c103d20 0x7fb97c104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.093+0000 7fb982338640 1 --2- 172.21.15.38:0/2847709277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb97c102ae0 0x7fb97c1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.093+0000 7fb982338640 1 -- 172.21.15.38:0/2847709277 >> 172.21.15.38:0/2847709277 conn(0x7fb97c0fdcf0 msgr2=0x7fb97c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.093+0000 7fb982338640 1 -- 172.21.15.38:0/2847709277 shutdown_connections 2024-05-21T15:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.093+0000 7fb982338640 1 -- 172.21.15.38:0/2847709277 wait complete. 2024-05-21T15:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.094+0000 7fb982338640 1 Processor -- start 2024-05-21T15:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.094+0000 7fb982338640 1 -- start start 2024-05-21T15:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.094+0000 7fb982338640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb97c102ae0 0x7fb97c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.094+0000 7fb982338640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c103d20 0x7fb97c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.094+0000 7fb982338640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb97c108a20 0x7fb97c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.094+0000 7fb982338640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c10e760 con 0x7fb97c103d20 2024-05-21T15:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.094+0000 7fb982338640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c10e5e0 con 0x7fb97c108a20 2024-05-21T15:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.094+0000 7fb982338640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c10e8e0 con 0x7fb97c102ae0 2024-05-21T15:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.094+0000 7fb97b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c103d20 0x7fb97c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.094+0000 7fb97b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c103d20 0x7fb97c1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40418/0 (socket says 172.21.15.38:40418) 2024-05-21T15:35:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.094+0000 7fb97b7fe640 1 -- 172.21.15.38:0/3218120551 learned_addr learned my addr 172.21.15.38:0/3218120551 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.095+0000 7fb97bfff640 1 --2- 172.21.15.38:0/3218120551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb97c102ae0 0x7fb97c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.095+0000 7fb9808d5640 1 --2- 172.21.15.38:0/3218120551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb97c108a20 0x7fb97c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.095+0000 7fb97b7fe640 1 -- 172.21.15.38:0/3218120551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb97c102ae0 msgr2=0x7fb97c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.095+0000 7fb97b7fe640 1 --2- 172.21.15.38:0/3218120551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb97c102ae0 0x7fb97c199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.095+0000 7fb97b7fe640 1 -- 172.21.15.38:0/3218120551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb97c108a20 msgr2=0x7fb97c1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.095+0000 7fb97b7fe640 1 --2- 172.21.15.38:0/3218120551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb97c108a20 0x7fb97c1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.095+0000 7fb97b7fe640 1 -- 172.21.15.38:0/3218120551 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c1a0c00 con 0x7fb97c103d20 2024-05-21T15:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.095+0000 7fb97b7fe640 1 --2- 172.21.15.38:0/3218120551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c103d20 0x7fb97c1996b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb97000d770 tx=0x7fb97000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.095+0000 7fb9797fa640 1 -- 172.21.15.38:0/3218120551 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970017070 con 0x7fb97c103d20 2024-05-21T15:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.095+0000 7fb9797fa640 1 -- 172.21.15.38:0/3218120551 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb970005500 con 0x7fb97c103d20 2024-05-21T15:35:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.095+0000 7fb982338640 1 -- 172.21.15.38:0/3218120551 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97c1a0ef0 con 0x7fb97c103d20 2024-05-21T15:35:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.095+0000 7fb9797fa640 1 -- 172.21.15.38:0/3218120551 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970004e00 con 0x7fb97c103d20 2024-05-21T15:35:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.095+0000 7fb982338640 1 -- 172.21.15.38:0/3218120551 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97c1a1360 con 0x7fb97c103d20 2024-05-21T15:35:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.098+0000 7fb9797fa640 1 -- 172.21.15.38:0/3218120551 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb970004fa0 con 0x7fb97c103d20 2024-05-21T15:35:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.098+0000 7fb9891e7740 1 Processor -- start 2024-05-21T15:35:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.098+0000 7fb9891e7740 1 -- start start 2024-05-21T15:35:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.098+0000 7fb9891e7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad3c1b0 0x5607db0e0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.098+0000 7fb9891e7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad132f0 0x5607db0e07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.098+0000 7fb9891e7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad1b4d0 0x5607db0e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb9891e7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5607db040340 con 0x5607dad132f0 2024-05-21T15:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb9891e7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5607dafe8290 con 0x5607dad3c1b0 2024-05-21T15:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb9891e7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5607db03c300 con 0x5607dad1b4d0 2024-05-21T15:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb97bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad132f0 0x5607db0e07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb97bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad132f0 0x5607db0e07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40432/0 (socket says 172.21.15.38:40432) 2024-05-21T15:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb97bfff640 1 -- 172.21.15.38:0/304561278 learned_addr learned my addr 172.21.15.38:0/304561278 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb9797fa640 1 --2- 172.21.15.38:0/3218120551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9640a2c30 0x7fb9640a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb97b7fe640 1 --2- 172.21.15.38:0/304561278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad1b4d0 0x5607db0e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb9808d5640 1 --2- 172.21.15.38:0/304561278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad3c1b0 0x5607db0e0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb9808d5640 1 --2- 172.21.15.38:0/3218120551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9640a2c30 0x7fb9640a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb97bfff640 1 -- 172.21.15.38:0/304561278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad1b4d0 msgr2=0x5607db0e76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb97bfff640 1 --2- 172.21.15.38:0/304561278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad1b4d0 0x5607db0e76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb97bfff640 1 -- 172.21.15.38:0/304561278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad3c1b0 msgr2=0x5607db0e0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb97bfff640 1 --2- 172.21.15.38:0/304561278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad3c1b0 0x5607db0e0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb9797fa640 1 -- 172.21.15.38:0/3218120551 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fb9700ceb10 con 0x7fb97c103d20 2024-05-21T15:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb97bfff640 1 -- 172.21.15.38:0/304561278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607db00c8e0 con 0x5607dad132f0 2024-05-21T15:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb9808d5640 1 --2- 172.21.15.38:0/304561278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad3c1b0 0x5607db0e0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.099+0000 7fb97bfff640 1 --2- 172.21.15.38:0/304561278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad132f0 0x5607db0e07f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00ac40 tx=0x7fb96c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ee52eab353fe3f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.100+0000 7fb9808d5640 1 --2- 172.21.15.38:0/3218120551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9640a2c30 0x7fb9640a5140 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5607db0e1440 tx=0x7fb9740095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.100+0000 7fb9627fc640 1 -- 172.21.15.38:0/304561278 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c005220 con 0x5607dad132f0 2024-05-21T15:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.100+0000 7fb9627fc640 1 -- 172.21.15.38:0/304561278 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb96c004c80 con 0x5607dad132f0 2024-05-21T15:35:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.100+0000 7fb9627fc640 1 -- 172.21.15.38:0/304561278 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0188b0 con 0x5607dad132f0 2024-05-21T15:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.100+0000 7fb9891e7740 1 -- 172.21.15.38:0/304561278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad132f0 msgr2=0x5607db0e07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.100+0000 7fb9891e7740 1 --2- 172.21.15.38:0/304561278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad132f0 0x5607db0e07f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00ac40 tx=0x7fb96c00f790 comp rx=0 tx=0).stop 2024-05-21T15:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.101+0000 7fb9891e7740 1 -- 172.21.15.38:0/304561278 shutdown_connections 2024-05-21T15:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.101+0000 7fb9891e7740 1 --2- 172.21.15.38:0/304561278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad1b4d0 0x5607db0e76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.101+0000 7fb9891e7740 1 --2- 172.21.15.38:0/304561278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad132f0 0x5607db0e07f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.101+0000 7fb9891e7740 1 --2- 172.21.15.38:0/304561278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad3c1b0 0x5607db0e0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.101+0000 7fb9891e7740 1 -- 172.21.15.38:0/304561278 >> 172.21.15.38:0/304561278 conn(0x5607dad30300 msgr2=0x5607db0dbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.101+0000 7fb9891e7740 1 -- 172.21.15.38:0/304561278 shutdown_connections 2024-05-21T15:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.101+0000 7fb9891e7740 1 -- 172.21.15.38:0/304561278 wait complete. 2024-05-21T15:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.101+0000 7fb9891e7740 1 -- start start 2024-05-21T15:35:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.101+0000 7fb9891e7740 1 Processor -- start 2024-05-21T15:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb9891e7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad132f0 0x5607db0e9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb9891e7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad1b4d0 0x5607db0ea520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb9891e7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad3c1b0 0x5607db0f1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb9891e7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5607db0e8140 con 0x5607dad132f0 2024-05-21T15:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb9891e7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5607daffb790 con 0x5607dad3c1b0 2024-05-21T15:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb9891e7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5607db0e82c0 con 0x5607dad1b4d0 2024-05-21T15:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb9891e7740 10 client.0 _ll_register_callbacks cb 0x5607db036a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb97bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad132f0 0x5607db0e9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb97bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad132f0 0x5607db0e9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40444/0 (socket says 172.21.15.38:40444) 2024-05-21T15:35:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb97bfff640 1 -- 172.21.15.38:0/3845870886 learned_addr learned my addr 172.21.15.38:0/3845870886 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb97b7fe640 1 --2- 172.21.15.38:0/3845870886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad1b4d0 0x5607db0ea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb9808d5640 1 --2- 172.21.15.38:0/3845870886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad3c1b0 0x5607db0f1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb97bfff640 1 -- 172.21.15.38:0/3845870886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad1b4d0 msgr2=0x5607db0ea520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb97bfff640 1 --2- 172.21.15.38:0/3845870886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad1b4d0 0x5607db0ea520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb97bfff640 1 -- 172.21.15.38:0/3845870886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad3c1b0 msgr2=0x5607db0f1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb97bfff640 1 --2- 172.21.15.38:0/3845870886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad3c1b0 0x5607db0f1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb97bfff640 1 -- 172.21.15.38:0/3845870886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607dad54210 con 0x5607dad132f0 2024-05-21T15:35:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb9808d5640 1 --2- 172.21.15.38:0/3845870886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad3c1b0 0x5607db0f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb97bfff640 1 --2- 172.21.15.38:0/3845870886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad132f0 0x5607db0e9fe0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb96c01a040 tx=0x7fb96c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259a6ec1655170a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.102+0000 7fb95b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.103+0000 7fb95b7fe640 1 -- 172.21.15.38:0/3845870886 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c027020 con 0x5607dad132f0 2024-05-21T15:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.103+0000 7fb95b7fe640 1 -- 172.21.15.38:0/3845870886 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fb96c005210 con 0x5607dad132f0 2024-05-21T15:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.103+0000 7fb9891e7740 1 -- 172.21.15.38:0/3845870886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb96c004c80 con 0x5607dad132f0 2024-05-21T15:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.103+0000 7fb95b7fe640 1 -- 172.21.15.38:0/3845870886 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb96c032050 con 0x5607dad132f0 2024-05-21T15:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.103+0000 7fb9891e7740 10 client.15717 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.103+0000 7fb9891e7740 1 -- 172.21.15.38:0/3845870886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5607dae20620 con 0x5607dad132f0 2024-05-21T15:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.103+0000 7fb95b7fe640 1 -- 172.21.15.38:0/3845870886 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb96c004c80 con 0x5607dad132f0 2024-05-21T15:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.103+0000 7fb95b7fe640 1 -- 172.21.15.38:0/3845870886 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb96c003ad0 con 0x5607dad132f0 2024-05-21T15:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.103+0000 7fb9891e7740 10 client.15717 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.103+0000 7fb9891e7740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.103+0000 7fb9891e7740 20 client.15717 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.103+0000 7fb9891e7740 1 --2- 172.21.15.38:0/3845870886 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5607dacfad80 0x5607db0f7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.103+0000 7fb9891e7740 4 client.15717 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.103+0000 7fb9891e7740 1 -- 172.21.15.38:0/3845870886 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607db04da70 con 0x5607dacfad80 2024-05-21T15:35:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.104+0000 7fb97b7fe640 1 --2- 172.21.15.38:0/3845870886 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5607dacfad80 0x5607db0f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.104+0000 7fb95b7fe640 10 client.15717 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.104+0000 7fb97b7fe640 1 --2- 172.21.15.38:0/3845870886 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5607dacfad80 0x5607db0f7700 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b63c03396319fc9 server_cookie=a055ed91d16ad236 in_seq=0 out_seq=0 2024-05-21T15:35:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.105+0000 7fb95b7fe640 1 -- 172.21.15.38:0/3845870886 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5607db04da70 con 0x5607dacfad80 2024-05-21T15:35:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.105+0000 7fb95b7fe640 10 client.15717 handle_command_reply: tid=0 2024-05-21T15:35:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.105+0000 7fb9891e7740 1 client.15717 shutdown 2024-05-21T15:35:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.105+0000 7fb9891e7740 1 -- 172.21.15.38:0/3845870886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad132f0 msgr2=0x5607db0e9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.105+0000 7fb9891e7740 1 --2- 172.21.15.38:0/3845870886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad132f0 0x5607db0e9fe0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb96c01a040 tx=0x7fb96c004270 comp rx=0 tx=0).stop 2024-05-21T15:35:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.105+0000 7fb9891e7740 1 -- 172.21.15.38:0/3845870886 shutdown_connections 2024-05-21T15:35:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.105+0000 7fb9891e7740 1 --2- 172.21.15.38:0/3845870886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad3c1b0 0x5607db0f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.105+0000 7fb9891e7740 1 --2- 172.21.15.38:0/3845870886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad1b4d0 0x5607db0ea520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.105+0000 7fb9891e7740 1 --2- 172.21.15.38:0/3845870886 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5607dacfad80 0x5607db0f7700 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.105+0000 7fb9891e7740 1 --2- 172.21.15.38:0/3845870886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad132f0 0x5607db0e9fe0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.105+0000 7fb9891e7740 1 -- 172.21.15.38:0/3845870886 >> 172.21.15.38:0/3845870886 conn(0x5607dad30300 msgr2=0x5607db0debb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.105+0000 7fb95b7fe640 0 client.15717 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.106+0000 7fb9891e7740 1 -- 172.21.15.38:0/3845870886 shutdown_connections 2024-05-21T15:35:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.106+0000 7fb9891e7740 1 -- 172.21.15.38:0/3845870886 wait complete. 2024-05-21T15:35:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.106+0000 7fb9891e7740 20 client.15717 trim_cache size 0 max 16384 2024-05-21T15:35:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb9891e7740 1 Processor -- start 2024-05-21T15:35:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb9891e7740 1 -- start start 2024-05-21T15:35:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb9891e7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad132f0 0x5607db0ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb9891e7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad1b4d0 0x5607db0f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb9891e7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad3c1b0 0x5607db0f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb9891e7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1a7fd0 con 0x5607dad1b4d0 2024-05-21T15:35:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb9891e7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1a9b10 con 0x5607dad132f0 2024-05-21T15:35:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb9891e7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1a86a0 con 0x5607dad3c1b0 2024-05-21T15:35:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb97b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad1b4d0 0x5607db0f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb97b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad1b4d0 0x5607db0f13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40458/0 (socket says 172.21.15.38:40458) 2024-05-21T15:35:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb97b7fe640 1 -- 172.21.15.38:0/1390046088 learned_addr learned my addr 172.21.15.38:0/1390046088 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb97b7fe640 1 -- 172.21.15.38:0/1390046088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad3c1b0 msgr2=0x5607db0f18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb9808d5640 1 --2- 172.21.15.38:0/1390046088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad3c1b0 0x5607db0f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb97b7fe640 1 --2- 172.21.15.38:0/1390046088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad3c1b0 0x5607db0f18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb97b7fe640 1 -- 172.21.15.38:0/1390046088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad132f0 msgr2=0x5607db0ea2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.121+0000 7fb97b7fe640 1 --2- 172.21.15.38:0/1390046088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad132f0 0x5607db0ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.122+0000 7fb97b7fe640 1 -- 172.21.15.38:0/1390046088 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607db0f1e20 con 0x5607dad1b4d0 2024-05-21T15:35:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.122+0000 7fb9808d5640 1 --2- 172.21.15.38:0/1390046088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad3c1b0 0x5607db0f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.122+0000 7fb97b7fe640 1 --2- 172.21.15.38:0/1390046088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad1b4d0 0x5607db0f13a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb970097780 tx=0x7fb9700bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d863b87014380ed3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.122+0000 7fb961ffb640 1 -- 172.21.15.38:0/1390046088 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970020040 con 0x5607dad1b4d0 2024-05-21T15:35:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.122+0000 7fb961ffb640 1 -- 172.21.15.38:0/1390046088 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9700993c0 con 0x5607dad1b4d0 2024-05-21T15:35:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.122+0000 7fb961ffb640 1 -- 172.21.15.38:0/1390046088 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970097990 con 0x5607dad1b4d0 2024-05-21T15:35:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.123+0000 7fb9891e7740 1 -- 172.21.15.38:0/1390046088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad1b4d0 msgr2=0x5607db0f13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.123+0000 7fb9891e7740 1 --2- 172.21.15.38:0/1390046088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad1b4d0 0x5607db0f13a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb970097780 tx=0x7fb9700bb040 comp rx=0 tx=0).stop 2024-05-21T15:35:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.123+0000 7fb9891e7740 1 -- 172.21.15.38:0/1390046088 shutdown_connections 2024-05-21T15:35:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.123+0000 7fb9891e7740 1 --2- 172.21.15.38:0/1390046088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad3c1b0 0x5607db0f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.123+0000 7fb9891e7740 1 --2- 172.21.15.38:0/1390046088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad1b4d0 0x5607db0f13a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.123+0000 7fb9891e7740 1 --2- 172.21.15.38:0/1390046088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad132f0 0x5607db0ea2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.123+0000 7fb9891e7740 1 -- 172.21.15.38:0/1390046088 >> 172.21.15.38:0/1390046088 conn(0x5607dad30300 msgr2=0x5607db0e9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.123+0000 7fb9891e7740 1 -- 172.21.15.38:0/1390046088 shutdown_connections 2024-05-21T15:35:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.123+0000 7fb9891e7740 1 -- 172.21.15.38:0/1390046088 wait complete. 2024-05-21T15:35:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.123+0000 7fb9891e7740 1 -- start start 2024-05-21T15:35:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.123+0000 7fb9891e7740 1 Processor -- start 2024-05-21T15:35:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9891e7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad132f0 0x5607db0eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9891e7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad1b4d0 0x5607db0ef000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9891e7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad3c1b0 0x5607db0eff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9891e7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1a9440 con 0x5607dad3c1b0 2024-05-21T15:35:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9891e7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1a8d70 con 0x5607dad1b4d0 2024-05-21T15:35:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9891e7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1a5fb0 con 0x5607dad132f0 2024-05-21T15:35:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9891e7740 10 client.0 _ll_register_callbacks cb 0x7fb97009b170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9808d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad3c1b0 0x5607db0eff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9808d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad3c1b0 0x5607db0eff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40462/0 (socket says 172.21.15.38:40462) 2024-05-21T15:35:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9808d5640 1 -- 172.21.15.38:0/834807668 learned_addr learned my addr 172.21.15.38:0/834807668 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb97bfff640 1 --2- 172.21.15.38:0/834807668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad132f0 0x5607db0eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb97b7fe640 1 --2- 172.21.15.38:0/834807668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad1b4d0 0x5607db0ef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9808d5640 1 -- 172.21.15.38:0/834807668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad132f0 msgr2=0x5607db0eeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9808d5640 1 --2- 172.21.15.38:0/834807668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad132f0 0x5607db0eeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9808d5640 1 -- 172.21.15.38:0/834807668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad1b4d0 msgr2=0x5607db0ef000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9808d5640 1 --2- 172.21.15.38:0/834807668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad1b4d0 0x5607db0ef000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9808d5640 1 -- 172.21.15.38:0/834807668 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607db0f04f0 con 0x5607dad3c1b0 2024-05-21T15:35:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb9808d5640 1 --2- 172.21.15.38:0/834807668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad3c1b0 0x5607db0eff30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb974035000 tx=0x7fb97402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59cf5acd11a5915 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.124+0000 7fb95b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.125+0000 7fb95b7fe640 1 -- 172.21.15.38:0/834807668 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974043070 con 0x5607dad3c1b0 2024-05-21T15:35:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.125+0000 7fb9891e7740 1 -- 172.21.15.38:0/834807668 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9700993c0 con 0x5607dad3c1b0 2024-05-21T15:35:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.125+0000 7fb95b7fe640 1 -- 172.21.15.38:0/834807668 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fb9740045d0 con 0x5607dad3c1b0 2024-05-21T15:35:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.125+0000 7fb95b7fe640 1 -- 172.21.15.38:0/834807668 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb9740047d0 con 0x5607dad3c1b0 2024-05-21T15:35:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.125+0000 7fb9891e7740 10 client.15723 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.125+0000 7fb9891e7740 1 -- 172.21.15.38:0/834807668 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5607db0f17b0 con 0x5607dad3c1b0 2024-05-21T15:35:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.125+0000 7fb95b7fe640 1 -- 172.21.15.38:0/834807668 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9700993c0 con 0x5607dad3c1b0 2024-05-21T15:35:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.125+0000 7fb95b7fe640 1 -- 172.21.15.38:0/834807668 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb9740306e0 con 0x5607dad3c1b0 2024-05-21T15:35:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.125+0000 7fb9891e7740 10 client.15723 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.125+0000 7fb9891e7740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.125+0000 7fb9891e7740 20 client.15723 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.125+0000 7fb9891e7740 1 --2- 172.21.15.38:0/834807668 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5607dacd01c0 0x5607db161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.125+0000 7fb9891e7740 4 client.15723 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:35:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.125+0000 7fb9891e7740 1 -- 172.21.15.38:0/834807668 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607db047dc0 con 0x5607dacd01c0 2024-05-21T15:35:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.126+0000 7fb97bfff640 1 --2- 172.21.15.38:0/834807668 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5607dacd01c0 0x5607db161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.126+0000 7fb97bfff640 1 --2- 172.21.15.38:0/834807668 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5607dacd01c0 0x5607db161d00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bbf874bc799c08e server_cookie=c37bdb416302fccb in_seq=0 out_seq=0 2024-05-21T15:35:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.126+0000 7fb95b7fe640 10 client.15723 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb95b7fe640 1 -- 172.21.15.38:0/834807668 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5607db047dc0 con 0x5607dacd01c0 2024-05-21T15:35:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb95b7fe640 10 client.15723 handle_command_reply: tid=0 2024-05-21T15:35:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb9891e7740 1 client.15723 shutdown 2024-05-21T15:35:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb9891e7740 1 -- 172.21.15.38:0/834807668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad3c1b0 msgr2=0x5607db0eff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb9891e7740 1 --2- 172.21.15.38:0/834807668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad3c1b0 0x5607db0eff30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb974035000 tx=0x7fb97402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb9891e7740 1 -- 172.21.15.38:0/834807668 shutdown_connections 2024-05-21T15:35:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb9891e7740 1 --2- 172.21.15.38:0/834807668 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5607dacd01c0 0x5607db161d00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb9891e7740 1 --2- 172.21.15.38:0/834807668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5607dad3c1b0 0x5607db0eff30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb9891e7740 1 --2- 172.21.15.38:0/834807668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607dad1b4d0 0x5607db0ef000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb95b7fe640 0 client.15723 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb9891e7740 1 --2- 172.21.15.38:0/834807668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607dad132f0 0x5607db0eeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb9891e7740 1 -- 172.21.15.38:0/834807668 >> 172.21.15.38:0/834807668 conn(0x5607dad30300 msgr2=0x5607db0deaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb9891e7740 1 -- 172.21.15.38:0/834807668 shutdown_connections 2024-05-21T15:35:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb9891e7740 1 -- 172.21.15.38:0/834807668 wait complete. 2024-05-21T15:35:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.134+0000 7fb9891e7740 20 client.15723 trim_cache size 0 max 16384 2024-05-21T15:35:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:35:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:35:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:35:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:35:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.135+0000 7fb982338640 1 -- 172.21.15.38:0/3218120551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9640a2c30 msgr2=0x7fb9640a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.135+0000 7fb982338640 1 --2- 172.21.15.38:0/3218120551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9640a2c30 0x7fb9640a5140 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5607db0e1440 tx=0x7fb9740095c0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.135+0000 7fb982338640 1 -- 172.21.15.38:0/3218120551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c103d20 msgr2=0x7fb97c1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.135+0000 7fb982338640 1 --2- 172.21.15.38:0/3218120551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c103d20 0x7fb97c1996b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb97000d770 tx=0x7fb97000dc30 comp rx=0 tx=0).stop 2024-05-21T15:35:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.135+0000 7fb982338640 1 -- 172.21.15.38:0/3218120551 shutdown_connections 2024-05-21T15:35:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.135+0000 7fb982338640 1 --2- 172.21.15.38:0/3218120551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb97c108a20 0x7fb97c1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.135+0000 7fb982338640 1 --2- 172.21.15.38:0/3218120551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb9640a2c30 0x7fb9640a5140 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.135+0000 7fb982338640 1 --2- 172.21.15.38:0/3218120551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c103d20 0x7fb97c1996b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.135+0000 7fb982338640 1 --2- 172.21.15.38:0/3218120551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb97c102ae0 0x7fb97c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.135+0000 7fb982338640 1 -- 172.21.15.38:0/3218120551 >> 172.21.15.38:0/3218120551 conn(0x7fb97c0fdcf0 msgr2=0x7fb97c100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.135+0000 7fb982338640 1 -- 172.21.15.38:0/3218120551 shutdown_connections 2024-05-21T15:35:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:23.135+0000 7fb982338640 1 -- 172.21.15.38:0/3218120551 wait complete. 2024-05-21T15:35:24.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:23 smithi045 ceph-mon[36409]: pgmap v380: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-21T15:35:24.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:23 smithi038 ceph-mon[31995]: pgmap v380: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-21T15:35:24.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:23 smithi053 ceph-mon[36496]: pgmap v380: 97 pgs: 97 active+clean; 657 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-21T15:35:26.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:26 smithi045 ceph-mon[36409]: pgmap v381: 97 pgs: 97 active+clean; 666 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-21T15:35:26.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:26 smithi038 ceph-mon[31995]: pgmap v381: 97 pgs: 97 active+clean; 666 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-21T15:35:26.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:26 smithi053 ceph-mon[36496]: pgmap v381: 97 pgs: 97 active+clean; 666 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-21T15:35:28.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:28 smithi045 ceph-mon[36409]: pgmap v382: 97 pgs: 97 active+clean; 666 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-21T15:35:28.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:28 smithi038 ceph-mon[31995]: pgmap v382: 97 pgs: 97 active+clean; 666 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-21T15:35:28.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:28 smithi053 ceph-mon[36496]: pgmap v382: 97 pgs: 97 active+clean; 666 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-21T15:35:29.458 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:29 smithi053 ceph-mon[36496]: pgmap v383: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-21T15:35:29.510 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:29 smithi045 ceph-mon[36409]: pgmap v383: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-21T15:35:29.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:29 smithi038 ceph-mon[31995]: pgmap v383: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-21T15:35:31.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:31 smithi053 ceph-mon[36496]: pgmap v384: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-21T15:35:31.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:31 smithi045 ceph-mon[36409]: pgmap v384: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-21T15:35:31.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:31 smithi038 ceph-mon[31995]: pgmap v384: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-21T15:35:33.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:33 smithi053 ceph-mon[36496]: pgmap v385: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-21T15:35:33.761 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:33 smithi045 ceph-mon[36409]: pgmap v385: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-21T15:35:33.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:33 smithi038 ceph-mon[31995]: pgmap v385: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-21T15:35:35.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:35 smithi053 ceph-mon[36496]: pgmap v386: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-21T15:35:35.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:35 smithi045 ceph-mon[36409]: pgmap v386: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-21T15:35:35.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:35 smithi038 ceph-mon[31995]: pgmap v386: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-21T15:35:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.239+0000 7faed6cde640 1 -- 172.21.15.38:0/362860995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faed01019d0 msgr2=0x7faed01026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.239+0000 7faed6cde640 1 --2- 172.21.15.38:0/362860995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faed01019d0 0x7faed01026d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7faec4009900 tx=0x7faec402a410 comp rx=0 tx=0).stop 2024-05-21T15:35:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.240+0000 7faed6cde640 1 -- 172.21.15.38:0/362860995 shutdown_connections 2024-05-21T15:35:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.240+0000 7faed6cde640 1 --2- 172.21.15.38:0/362860995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faed0107910 0x7faed0109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.240+0000 7faed6cde640 1 --2- 172.21.15.38:0/362860995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0102c10 0x7faed0103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.240+0000 7faed6cde640 1 --2- 172.21.15.38:0/362860995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faed01019d0 0x7faed01026d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.240+0000 7faed6cde640 1 -- 172.21.15.38:0/362860995 >> 172.21.15.38:0/362860995 conn(0x7faed00fcbc0 msgr2=0x7faed00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.240+0000 7faed6cde640 1 -- 172.21.15.38:0/362860995 shutdown_connections 2024-05-21T15:35:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.240+0000 7faed6cde640 1 -- 172.21.15.38:0/362860995 wait complete. 2024-05-21T15:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.240+0000 7faed6cde640 1 Processor -- start 2024-05-21T15:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faed6cde640 1 -- start start 2024-05-21T15:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faed6cde640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed01019d0 0x7faed0069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faed6cde640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faed0102c10 0x7faed019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faed6cde640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faed0107910 0x7faed01a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faed6cde640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faed010d620 con 0x7faed0102c10 2024-05-21T15:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faed6cde640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faed010d4a0 con 0x7faed0107910 2024-05-21T15:35:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faed6cde640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faed010d7a0 con 0x7faed01019d0 2024-05-21T15:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faecffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faed0102c10 0x7faed019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faecffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faed0102c10 0x7faed019e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44752/0 (socket says 172.21.15.38:44752) 2024-05-21T15:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faecffff640 1 -- 172.21.15.38:0/3032332017 learned_addr learned my addr 172.21.15.38:0/3032332017 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faed4a7a640 1 --2- 172.21.15.38:0/3032332017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed01019d0 0x7faed0069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faed527b640 1 --2- 172.21.15.38:0/3032332017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faed0107910 0x7faed01a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faecffff640 1 -- 172.21.15.38:0/3032332017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed01019d0 msgr2=0x7faed0069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faecffff640 1 --2- 172.21.15.38:0/3032332017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed01019d0 0x7faed0069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faecffff640 1 -- 172.21.15.38:0/3032332017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faed0107910 msgr2=0x7faed01a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faecffff640 1 --2- 172.21.15.38:0/3032332017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faed0107910 0x7faed01a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.241+0000 7faecffff640 1 -- 172.21.15.38:0/3032332017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed01a0d80 con 0x7faed0102c10 2024-05-21T15:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.242+0000 7faecffff640 1 --2- 172.21.15.38:0/3032332017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faed0102c10 0x7faed019e010 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7faebc00e9f0 tx=0x7faebc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.242+0000 7faecdffb640 1 -- 172.21.15.38:0/3032332017 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc00cd80 con 0x7faed0102c10 2024-05-21T15:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.242+0000 7faecdffb640 1 -- 172.21.15.38:0/3032332017 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faebc005500 con 0x7faed0102c10 2024-05-21T15:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.242+0000 7faed6cde640 1 -- 172.21.15.38:0/3032332017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed01a1070 con 0x7faed0102c10 2024-05-21T15:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.242+0000 7faecdffb640 1 -- 172.21.15.38:0/3032332017 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc010690 con 0x7faed0102c10 2024-05-21T15:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.242+0000 7faed6cde640 1 -- 172.21.15.38:0/3032332017 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed01a15c0 con 0x7faed0102c10 2024-05-21T15:35:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.245+0000 7faecdffb640 1 -- 172.21.15.38:0/3032332017 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faebc0046c0 con 0x7faed0102c10 2024-05-21T15:35:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.245+0000 7faeddb7f740 1 Processor -- start 2024-05-21T15:35:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.245+0000 7faeddb7f740 1 -- start start 2024-05-21T15:35:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.245+0000 7faeddb7f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31ab1b0 0x561fd354ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.245+0000 7faeddb7f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31822f0 0x561fd354b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.245+0000 7faeddb7f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd35522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.245+0000 7faeddb7f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561fd34af340 con 0x561fd318a4d0 2024-05-21T15:35:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.245+0000 7faeddb7f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561fd3457290 con 0x561fd31822f0 2024-05-21T15:35:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.245+0000 7faeddb7f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561fd34ab300 con 0x561fd31ab1b0 2024-05-21T15:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.245+0000 7faecffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd35522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.245+0000 7faecffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd35522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44758/0 (socket says 172.21.15.38:44758) 2024-05-21T15:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.245+0000 7faecffff640 1 -- 172.21.15.38:0/2741998929 learned_addr learned my addr 172.21.15.38:0/2741998929 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.246+0000 7faed527b640 1 --2- 172.21.15.38:0/2741998929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31ab1b0 0x561fd354ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.246+0000 7faed4a7a640 1 --2- 172.21.15.38:0/2741998929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31822f0 0x561fd354b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.246+0000 7faecffff640 1 -- 172.21.15.38:0/2741998929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31ab1b0 msgr2=0x561fd354ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.246+0000 7faecffff640 1 --2- 172.21.15.38:0/2741998929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31ab1b0 0x561fd354ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.246+0000 7faecffff640 1 -- 172.21.15.38:0/2741998929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31822f0 msgr2=0x561fd354b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.246+0000 7faecffff640 1 --2- 172.21.15.38:0/2741998929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31822f0 0x561fd354b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.246+0000 7faecdffb640 1 --2- 172.21.15.38:0/3032332017 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faec00a2cb0 0x7faec00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.246+0000 7faecffff640 1 -- 172.21.15.38:0/2741998929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd347b8e0 con 0x561fd318a4d0 2024-05-21T15:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.246+0000 7faed527b640 1 --2- 172.21.15.38:0/3032332017 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faec00a2cb0 0x7faec00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.246+0000 7faecffff640 1 --2- 172.21.15.38:0/2741998929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd35522e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7faebc0d54a0 tx=0x7faebc097f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1855358fa30a863d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.246+0000 7faecdffb640 1 -- 172.21.15.38:0/3032332017 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7faebc0cf4c0 con 0x7faed0102c10 2024-05-21T15:35:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.246+0000 7faebaffd640 1 -- 172.21.15.38:0/2741998929 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc09baa0 con 0x561fd318a4d0 2024-05-21T15:35:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.247+0000 7faebaffd640 1 -- 172.21.15.38:0/2741998929 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faebc0a2870 con 0x561fd318a4d0 2024-05-21T15:35:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.247+0000 7faebaffd640 1 -- 172.21.15.38:0/2741998929 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc09b630 con 0x561fd318a4d0 2024-05-21T15:35:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.248+0000 7faeddb7f740 1 -- 172.21.15.38:0/2741998929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 msgr2=0x561fd35522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.248+0000 7faed527b640 1 --2- 172.21.15.38:0/3032332017 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faec00a2cb0 0x7faec00a51c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x561fd354c030 tx=0x7faec8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.248+0000 7faeddb7f740 1 --2- 172.21.15.38:0/2741998929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd35522e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7faebc0d54a0 tx=0x7faebc097f70 comp rx=0 tx=0).stop 2024-05-21T15:35:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.248+0000 7faeddb7f740 1 -- 172.21.15.38:0/2741998929 shutdown_connections 2024-05-21T15:35:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.248+0000 7faeddb7f740 1 --2- 172.21.15.38:0/2741998929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd35522e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.248+0000 7faeddb7f740 1 --2- 172.21.15.38:0/2741998929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31822f0 0x561fd354b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.248+0000 7faeddb7f740 1 --2- 172.21.15.38:0/2741998929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31ab1b0 0x561fd354ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.248+0000 7faeddb7f740 1 -- 172.21.15.38:0/2741998929 >> 172.21.15.38:0/2741998929 conn(0x561fd319f300 msgr2=0x561fd35467f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.248+0000 7faeddb7f740 1 -- 172.21.15.38:0/2741998929 shutdown_connections 2024-05-21T15:35:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.248+0000 7faeddb7f740 1 -- 172.21.15.38:0/2741998929 wait complete. 2024-05-21T15:35:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.248+0000 7faeddb7f740 1 -- start start 2024-05-21T15:35:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.248+0000 7faeddb7f740 1 Processor -- start 2024-05-21T15:35:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faeddb7f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31822f0 0x561fd3554a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faeddb7f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd3554fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faeddb7f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31ab1b0 0x561fd355bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faeddb7f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561fd3552c70 con 0x561fd318a4d0 2024-05-21T15:35:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faeddb7f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561fd346a790 con 0x561fd31ab1b0 2024-05-21T15:35:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faeddb7f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561fd3552df0 con 0x561fd31822f0 2024-05-21T15:35:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faeddb7f740 10 client.0 _ll_register_callbacks cb 0x561fd34a5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faecffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd3554fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faecffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd3554fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44774/0 (socket says 172.21.15.38:44774) 2024-05-21T15:35:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faecffff640 1 -- 172.21.15.38:0/2396153419 learned_addr learned my addr 172.21.15.38:0/2396153419 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faed4a7a640 1 --2- 172.21.15.38:0/2396153419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31822f0 0x561fd3554a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faed527b640 1 --2- 172.21.15.38:0/2396153419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31ab1b0 0x561fd355bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faecffff640 1 -- 172.21.15.38:0/2396153419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31822f0 msgr2=0x561fd3554a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faecffff640 1 --2- 172.21.15.38:0/2396153419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31822f0 0x561fd3554a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faecffff640 1 -- 172.21.15.38:0/2396153419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31ab1b0 msgr2=0x561fd355bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faecffff640 1 --2- 172.21.15.38:0/2396153419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31ab1b0 0x561fd355bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faecffff640 1 -- 172.21.15.38:0/2396153419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd31c3210 con 0x561fd318a4d0 2024-05-21T15:35:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faecffff640 1 --2- 172.21.15.38:0/2396153419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd3554fc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7faebc0d5260 tx=0x7faebc0d2df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7a349aa43f9934 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.249+0000 7faeaffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.250+0000 7faeaffff640 1 -- 172.21.15.38:0/2396153419 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc09b810 con 0x561fd318a4d0 2024-05-21T15:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.250+0000 7faeaffff640 1 -- 172.21.15.38:0/2396153419 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7faebc0cf0f0 con 0x561fd318a4d0 2024-05-21T15:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.250+0000 7faeddb7f740 1 -- 172.21.15.38:0/2396153419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561fd34bca70 con 0x561fd318a4d0 2024-05-21T15:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.250+0000 7faeaffff640 1 -- 172.21.15.38:0/2396153419 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faebc09b630 con 0x561fd318a4d0 2024-05-21T15:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.250+0000 7faeddb7f740 10 client.15735 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.250+0000 7faeddb7f740 1 -- 172.21.15.38:0/2396153419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561fd328f620 con 0x561fd318a4d0 2024-05-21T15:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.250+0000 7faeaffff640 1 -- 172.21.15.38:0/2396153419 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561fd34bca70 con 0x561fd318a4d0 2024-05-21T15:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.250+0000 7faeaffff640 1 -- 172.21.15.38:0/2396153419 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faebc0a44b0 con 0x561fd318a4d0 2024-05-21T15:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.250+0000 7faeddb7f740 10 client.15735 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.250+0000 7faeddb7f740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.250+0000 7faeddb7f740 20 client.15735 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.251+0000 7faeddb7f740 1 --2- 172.21.15.38:0/2396153419 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561fd3169d80 0x561fd3561fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.251+0000 7faeddb7f740 4 client.15735 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.251+0000 7faeddb7f740 1 -- 172.21.15.38:0/2396153419 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fd34b6dc0 con 0x561fd3169d80 2024-05-21T15:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.251+0000 7faed4a7a640 1 --2- 172.21.15.38:0/2396153419 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561fd3169d80 0x561fd3561fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.251+0000 7faed4a7a640 1 --2- 172.21.15.38:0/2396153419 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561fd3169d80 0x561fd3561fe0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a00c12a4710a00c3 server_cookie=be0f4dd3454289c8 in_seq=0 out_seq=0 2024-05-21T15:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.251+0000 7faeaffff640 10 client.15735 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.252+0000 7faeaffff640 1 -- 172.21.15.38:0/2396153419 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561fd34b6dc0 con 0x561fd3169d80 2024-05-21T15:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.252+0000 7faeaffff640 10 client.15735 handle_command_reply: tid=0 2024-05-21T15:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.252+0000 7faeddb7f740 1 client.15735 shutdown 2024-05-21T15:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.252+0000 7faeddb7f740 1 -- 172.21.15.38:0/2396153419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 msgr2=0x561fd3554fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.252+0000 7faeddb7f740 1 --2- 172.21.15.38:0/2396153419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd3554fc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7faebc0d5260 tx=0x7faebc0d2df0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.252+0000 7faeddb7f740 1 -- 172.21.15.38:0/2396153419 shutdown_connections 2024-05-21T15:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.252+0000 7faeddb7f740 1 --2- 172.21.15.38:0/2396153419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31ab1b0 0x561fd355bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.252+0000 7faeddb7f740 1 --2- 172.21.15.38:0/2396153419 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561fd3169d80 0x561fd3561fe0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.252+0000 7faeddb7f740 1 --2- 172.21.15.38:0/2396153419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd3554fc0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.252+0000 7faeddb7f740 1 --2- 172.21.15.38:0/2396153419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31822f0 0x561fd3554a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.252+0000 7faeaffff640 0 client.15735 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.252+0000 7faeddb7f740 1 -- 172.21.15.38:0/2396153419 >> 172.21.15.38:0/2396153419 conn(0x561fd319f300 msgr2=0x561fd3545810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.252+0000 7faeddb7f740 1 -- 172.21.15.38:0/2396153419 shutdown_connections 2024-05-21T15:35:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.252+0000 7faeddb7f740 1 -- 172.21.15.38:0/2396153419 wait complete. 2024-05-21T15:35:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.253+0000 7faeddb7f740 20 client.15735 trim_cache size 0 max 16384 2024-05-21T15:35:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.269+0000 7faeddb7f740 1 Processor -- start 2024-05-21T15:35:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.269+0000 7faeddb7f740 1 -- start start 2024-05-21T15:35:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.269+0000 7faeddb7f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31822f0 0x561fd354f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.269+0000 7faeddb7f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd35538a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.269+0000 7faeddb7f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31ab1b0 0x561fd3554650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.269+0000 7faeddb7f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faed01aa420 con 0x561fd318a4d0 2024-05-21T15:35:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.269+0000 7faeddb7f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faed01abf60 con 0x561fd31ab1b0 2024-05-21T15:35:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.269+0000 7faeddb7f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faed01aaaf0 con 0x561fd31822f0 2024-05-21T15:35:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.269+0000 7faecffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd35538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.269+0000 7faecffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd35538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44790/0 (socket says 172.21.15.38:44790) 2024-05-21T15:35:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.269+0000 7faecffff640 1 -- 172.21.15.38:0/4086636045 learned_addr learned my addr 172.21.15.38:0/4086636045 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.270+0000 7faed4a7a640 1 --2- 172.21.15.38:0/4086636045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31822f0 0x561fd354f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.270+0000 7faed527b640 1 --2- 172.21.15.38:0/4086636045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31ab1b0 0x561fd3554650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.270+0000 7faecffff640 1 -- 172.21.15.38:0/4086636045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31822f0 msgr2=0x561fd354f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.270+0000 7faecffff640 1 --2- 172.21.15.38:0/4086636045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31822f0 0x561fd354f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.270+0000 7faecffff640 1 -- 172.21.15.38:0/4086636045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31ab1b0 msgr2=0x561fd3554650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.270+0000 7faecffff640 1 --2- 172.21.15.38:0/4086636045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31ab1b0 0x561fd3554650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.270+0000 7faecffff640 1 -- 172.21.15.38:0/4086636045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd3562350 con 0x561fd318a4d0 2024-05-21T15:35:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.270+0000 7faed4a7a640 1 --2- 172.21.15.38:0/4086636045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31822f0 0x561fd354f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.270+0000 7faed527b640 1 --2- 172.21.15.38:0/4086636045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31ab1b0 0x561fd3554650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.270+0000 7faecffff640 1 --2- 172.21.15.38:0/4086636045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd35538a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faebc0d54a0 tx=0x7faebc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40327bac63e312d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.270+0000 7faeba7fc640 1 -- 172.21.15.38:0/4086636045 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc09baa0 con 0x561fd318a4d0 2024-05-21T15:35:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.270+0000 7faeba7fc640 1 -- 172.21.15.38:0/4086636045 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faebc09c3d0 con 0x561fd318a4d0 2024-05-21T15:35:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.270+0000 7faeba7fc640 1 -- 172.21.15.38:0/4086636045 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc0a44b0 con 0x561fd318a4d0 2024-05-21T15:35:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.271+0000 7faeddb7f740 1 -- 172.21.15.38:0/4086636045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 msgr2=0x561fd35538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.271+0000 7faeddb7f740 1 --2- 172.21.15.38:0/4086636045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd35538a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faebc0d54a0 tx=0x7faebc004a40 comp rx=0 tx=0).stop 2024-05-21T15:35:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.271+0000 7faeddb7f740 1 -- 172.21.15.38:0/4086636045 shutdown_connections 2024-05-21T15:35:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.271+0000 7faeddb7f740 1 --2- 172.21.15.38:0/4086636045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31ab1b0 0x561fd3554650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.271+0000 7faeddb7f740 1 --2- 172.21.15.38:0/4086636045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd318a4d0 0x561fd35538a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.271+0000 7faeddb7f740 1 --2- 172.21.15.38:0/4086636045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd31822f0 0x561fd354f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.271+0000 7faeddb7f740 1 -- 172.21.15.38:0/4086636045 >> 172.21.15.38:0/4086636045 conn(0x561fd319f300 msgr2=0x561fd3553190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.271+0000 7faeddb7f740 1 -- 172.21.15.38:0/4086636045 shutdown_connections 2024-05-21T15:35:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.271+0000 7faeddb7f740 1 -- 172.21.15.38:0/4086636045 wait complete. 2024-05-21T15:35:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.272+0000 7faeddb7f740 1 -- start start 2024-05-21T15:35:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.272+0000 7faeddb7f740 1 Processor -- start 2024-05-21T15:35:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.272+0000 7faeddb7f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31822f0 0x561fd3558320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.272+0000 7faeddb7f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd318a4d0 0x561fd3558860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.272+0000 7faeddb7f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd31ab1b0 0x561fd35c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.272+0000 7faeddb7f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faed01ab890 con 0x561fd31ab1b0 2024-05-21T15:35:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.272+0000 7faeddb7f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faed01ab1c0 con 0x561fd31822f0 2024-05-21T15:35:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.272+0000 7faeddb7f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faed01a8400 con 0x561fd318a4d0 2024-05-21T15:35:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.272+0000 7faeddb7f740 10 client.0 _ll_register_callbacks cb 0x7faebc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.272+0000 7faed527b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd31ab1b0 0x561fd35c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.272+0000 7faed527b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd31ab1b0 0x561fd35c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44796/0 (socket says 172.21.15.38:44796) 2024-05-21T15:35:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.272+0000 7faed527b640 1 -- 172.21.15.38:0/877885823 learned_addr learned my addr 172.21.15.38:0/877885823 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.272+0000 7faecffff640 1 --2- 172.21.15.38:0/877885823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd318a4d0 0x561fd3558860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.273+0000 7faed4a7a640 1 --2- 172.21.15.38:0/877885823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31822f0 0x561fd3558320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.273+0000 7faed527b640 1 -- 172.21.15.38:0/877885823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd318a4d0 msgr2=0x561fd3558860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.273+0000 7faed527b640 1 --2- 172.21.15.38:0/877885823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd318a4d0 0x561fd3558860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.273+0000 7faed527b640 1 -- 172.21.15.38:0/877885823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31822f0 msgr2=0x561fd3558320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.273+0000 7faed527b640 1 --2- 172.21.15.38:0/877885823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31822f0 0x561fd3558320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.273+0000 7faed527b640 1 -- 172.21.15.38:0/877885823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd355b820 con 0x561fd31ab1b0 2024-05-21T15:35:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.273+0000 7faed527b640 1 --2- 172.21.15.38:0/877885823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd31ab1b0 0x561fd35c82d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7faec8010c30 tx=0x7faec8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a50bf396d320fc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.273+0000 7faeaffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.273+0000 7faeaffff640 1 -- 172.21.15.38:0/877885823 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec802d060 con 0x561fd31ab1b0 2024-05-21T15:35:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.273+0000 7faeaffff640 1 -- 172.21.15.38:0/877885823 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7faec8028430 con 0x561fd31ab1b0 2024-05-21T15:35:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.273+0000 7faeddb7f740 1 -- 172.21.15.38:0/877885823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faebc09c3d0 con 0x561fd31ab1b0 2024-05-21T15:35:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.273+0000 7faeaffff640 1 -- 172.21.15.38:0/877885823 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faec8028670 con 0x561fd31ab1b0 2024-05-21T15:35:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.273+0000 7faeddb7f740 10 client.15741 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.273+0000 7faeddb7f740 1 -- 172.21.15.38:0/877885823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561fd35c9850 con 0x561fd31ab1b0 2024-05-21T15:35:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.274+0000 7faeaffff640 1 -- 172.21.15.38:0/877885823 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faebc09c3d0 con 0x561fd31ab1b0 2024-05-21T15:35:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.274+0000 7faeaffff640 1 -- 172.21.15.38:0/877885823 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faec8027070 con 0x561fd31ab1b0 2024-05-21T15:35:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.274+0000 7faeddb7f740 10 client.15741 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.274+0000 7faeddb7f740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.274+0000 7faeddb7f740 20 client.15741 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.274+0000 7faeddb7f740 1 --2- 172.21.15.38:0/877885823 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561fd313f1c0 0x561fd35cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.274+0000 7faeddb7f740 4 client.15741 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:35:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.274+0000 7faeddb7f740 1 -- 172.21.15.38:0/877885823 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561fd354b7e0 con 0x561fd313f1c0 2024-05-21T15:35:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.274+0000 7faed4a7a640 1 --2- 172.21.15.38:0/877885823 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561fd313f1c0 0x561fd35cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.275+0000 7faeaffff640 10 client.15741 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.275+0000 7faed4a7a640 1 --2- 172.21.15.38:0/877885823 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561fd313f1c0 0x561fd35cc270 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55cb67c662e7f17b server_cookie=27ee7a9513aa5a54 in_seq=0 out_seq=0 2024-05-21T15:35:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeaffff640 1 -- 172.21.15.38:0/877885823 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561fd354b7e0 con 0x561fd313f1c0 2024-05-21T15:35:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeaffff640 10 client.15741 handle_command_reply: tid=0 2024-05-21T15:35:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeddb7f740 1 client.15741 shutdown 2024-05-21T15:35:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeddb7f740 1 -- 172.21.15.38:0/877885823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd31ab1b0 msgr2=0x561fd35c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeddb7f740 1 --2- 172.21.15.38:0/877885823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd31ab1b0 0x561fd35c82d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7faec8010c30 tx=0x7faec8004ab0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeddb7f740 1 -- 172.21.15.38:0/877885823 shutdown_connections 2024-05-21T15:35:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeddb7f740 1 --2- 172.21.15.38:0/877885823 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561fd313f1c0 0x561fd35cc270 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeddb7f740 1 --2- 172.21.15.38:0/877885823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561fd31ab1b0 0x561fd35c82d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeddb7f740 1 --2- 172.21.15.38:0/877885823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561fd318a4d0 0x561fd3558860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeaffff640 0 client.15741 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeddb7f740 1 --2- 172.21.15.38:0/877885823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561fd31822f0 0x561fd3558320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeddb7f740 1 -- 172.21.15.38:0/877885823 >> 172.21.15.38:0/877885823 conn(0x561fd319f300 msgr2=0x561fd35545c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeddb7f740 1 -- 172.21.15.38:0/877885823 shutdown_connections 2024-05-21T15:35:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeddb7f740 1 -- 172.21.15.38:0/877885823 wait complete. 2024-05-21T15:35:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.279+0000 7faeddb7f740 20 client.15741 trim_cache size 0 max 16384 2024-05-21T15:35:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:35:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:35:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:35:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:35:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.280+0000 7faed6cde640 1 -- 172.21.15.38:0/3032332017 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faec00a2cb0 msgr2=0x7faec00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.280+0000 7faed6cde640 1 --2- 172.21.15.38:0/3032332017 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faec00a2cb0 0x7faec00a51c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x561fd354c030 tx=0x7faec8009500 comp rx=0 tx=0).stop 2024-05-21T15:35:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.280+0000 7faed6cde640 1 -- 172.21.15.38:0/3032332017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faed0102c10 msgr2=0x7faed019e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.280+0000 7faed6cde640 1 --2- 172.21.15.38:0/3032332017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faed0102c10 0x7faed019e010 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7faebc00e9f0 tx=0x7faebc00edf0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.280+0000 7faed6cde640 1 -- 172.21.15.38:0/3032332017 shutdown_connections 2024-05-21T15:35:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.280+0000 7faed6cde640 1 --2- 172.21.15.38:0/3032332017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faed0107910 0x7faed01a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.280+0000 7faed6cde640 1 --2- 172.21.15.38:0/3032332017 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faec00a2cb0 0x7faec00a51c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.280+0000 7faed6cde640 1 --2- 172.21.15.38:0/3032332017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faed0102c10 0x7faed019e010 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.280+0000 7faed6cde640 1 --2- 172.21.15.38:0/3032332017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed01019d0 0x7faed0069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.280+0000 7faed6cde640 1 -- 172.21.15.38:0/3032332017 >> 172.21.15.38:0/3032332017 conn(0x7faed00fcbc0 msgr2=0x7faed01038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.280+0000 7faed6cde640 1 -- 172.21.15.38:0/3032332017 shutdown_connections 2024-05-21T15:35:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:36.280+0000 7faed6cde640 1 -- 172.21.15.38:0/3032332017 wait complete. 2024-05-21T15:35:37.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:37 smithi053 ceph-mon[36496]: pgmap v387: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-21T15:35:37.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:37 smithi045 ceph-mon[36409]: pgmap v387: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-21T15:35:37.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:37 smithi038 ceph-mon[31995]: pgmap v387: 97 pgs: 97 active+clean; 667 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-21T15:35:39.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:39 smithi053 ceph-mon[36496]: pgmap v388: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 13 op/s 2024-05-21T15:35:39.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:39 smithi045 ceph-mon[36409]: pgmap v388: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 13 op/s 2024-05-21T15:35:39.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:39 smithi038 ceph-mon[31995]: pgmap v388: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 13 op/s 2024-05-21T15:35:41.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:41 smithi053 ceph-mon[36496]: pgmap v389: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:35:41.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:41 smithi045 ceph-mon[36409]: pgmap v389: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:35:41.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:41 smithi038 ceph-mon[31995]: pgmap v389: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:35:43.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:43 smithi038 ceph-mon[31995]: pgmap v390: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:35:43.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:43 smithi053 ceph-mon[36496]: pgmap v390: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:35:43.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:43 smithi045 ceph-mon[36409]: pgmap v390: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:35:45.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:45 smithi053 ceph-mon[36496]: pgmap v391: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-21T15:35:45.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:45 smithi045 ceph-mon[36409]: pgmap v391: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-21T15:35:45.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:45 smithi038 ceph-mon[31995]: pgmap v391: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-21T15:35:46.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:35:46.692 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:35:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.876+0000 7f368557e640 1 -- 172.21.15.38:0/2683174990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3680102340 msgr2=0x7f3680102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:46.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.876+0000 7f368557e640 1 --2- 172.21.15.38:0/2683174990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3680102340 0x7f3680102750 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3674009900 tx=0x7f367402e820 comp rx=0 tx=0).stop 2024-05-21T15:35:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.876+0000 7f368557e640 1 -- 172.21.15.38:0/2683174990 shutdown_connections 2024-05-21T15:35:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.876+0000 7f368557e640 1 --2- 172.21.15.38:0/2683174990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3680107040 0x7f3680109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.876+0000 7f368557e640 1 --2- 172.21.15.38:0/2683174990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3680102340 0x7f3680102750 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.876+0000 7f368557e640 1 --2- 172.21.15.38:0/2683174990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680101100 0x7f3680101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.876+0000 7f368557e640 1 -- 172.21.15.38:0/2683174990 >> 172.21.15.38:0/2683174990 conn(0x7f36800fc2f0 msgr2=0x7f36800fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.876+0000 7f368557e640 1 -- 172.21.15.38:0/2683174990 shutdown_connections 2024-05-21T15:35:46.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.876+0000 7f368557e640 1 -- 172.21.15.38:0/2683174990 wait complete. 2024-05-21T15:35:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.877+0000 7f368557e640 1 Processor -- start 2024-05-21T15:35:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.877+0000 7f368557e640 1 -- start start 2024-05-21T15:35:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f368557e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3680101100 0x7f36801970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f368557e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3680102340 0x7f36801975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f368557e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107040 0x7f368019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f368557e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f368010c530 con 0x7f3680102340 2024-05-21T15:35:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f368557e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f368010c3b0 con 0x7f3680101100 2024-05-21T15:35:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f368557e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f368010c6b0 con 0x7f3680107040 2024-05-21T15:35:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f367e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3680102340 0x7f36801975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f367e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3680102340 0x7f36801975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49270/0 (socket says 172.21.15.38:49270) 2024-05-21T15:35:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f367e7fc640 1 -- 172.21.15.38:0/576710677 learned_addr learned my addr 172.21.15.38:0/576710677 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f367effd640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3680101100 0x7f36801970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f367f7fe640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107040 0x7f368019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f367e7fc640 1 -- 172.21.15.38:0/576710677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107040 msgr2=0x7f368019e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f367e7fc640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107040 0x7f368019e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f367e7fc640 1 -- 172.21.15.38:0/576710677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3680101100 msgr2=0x7f36801970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f367e7fc640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3680101100 0x7f36801970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f367e7fc640 1 -- 172.21.15.38:0/576710677 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368019eb40 con 0x7f3680102340 2024-05-21T15:35:46.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f367f7fe640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107040 0x7f368019e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:46.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.878+0000 7f367effd640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3680101100 0x7f36801970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:46.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.879+0000 7f367e7fc640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3680102340 0x7f36801975f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3674005ab0 tx=0x7f36740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:46.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.879+0000 7f3667fff640 1 -- 172.21.15.38:0/576710677 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674045020 con 0x7f3680102340 2024-05-21T15:35:46.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.879+0000 7f3667fff640 1 -- 172.21.15.38:0/576710677 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3674003ce0 con 0x7f3680102340 2024-05-21T15:35:46.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.879+0000 7f368557e640 1 -- 172.21.15.38:0/576710677 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368019edd0 con 0x7f3680102340 2024-05-21T15:35:46.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.879+0000 7f3667fff640 1 -- 172.21.15.38:0/576710677 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674040a20 con 0x7f3680102340 2024-05-21T15:35:46.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.880+0000 7f368557e640 1 -- 172.21.15.38:0/576710677 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368019f240 con 0x7f3680102340 2024-05-21T15:35:46.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.883+0000 7f3667fff640 1 -- 172.21.15.38:0/576710677 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3674048020 con 0x7f3680102340 2024-05-21T15:35:46.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.884+0000 7f368c42b740 1 Processor -- start 2024-05-21T15:35:46.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.884+0000 7f368c42b740 1 -- start start 2024-05-21T15:35:46.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.884+0000 7f368c42b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d597390 0x55697d936bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.884+0000 7f368c42b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d56e4d0 0x55697d937130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.884+0000 7f368c42b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d5766b0 0x55697d93e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.884+0000 7f368c42b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55697d89bd00 con 0x55697d5766b0 2024-05-21T15:35:46.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.884+0000 7f368c42b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55697d8434c0 con 0x55697d56e4d0 2024-05-21T15:35:46.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.884+0000 7f368c42b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55697d897cc0 con 0x55697d597390 2024-05-21T15:35:46.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.884+0000 7f367e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d5766b0 0x55697d93e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.884+0000 7f3667fff640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f36680a2b90 0x7f36680a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.884+0000 7f367e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d5766b0 0x55697d93e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49284/0 (socket says 172.21.15.38:49284) 2024-05-21T15:35:46.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.884+0000 7f367e7fc640 1 -- 172.21.15.38:0/1905311628 learned_addr learned my addr 172.21.15.38:0/1905311628 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:46.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.884+0000 7f367effd640 1 --2- 172.21.15.38:0/1905311628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d56e4d0 0x55697d937130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.885+0000 7f367f7fe640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f36680a2b90 0x7f36680a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.885+0000 7f3667fff640 1 -- 172.21.15.38:0/576710677 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f36740f2d30 con 0x7f3680102340 2024-05-21T15:35:46.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.885+0000 7f367e7fc640 1 -- 172.21.15.38:0/1905311628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d597390 msgr2=0x55697d936bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:46.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.885+0000 7f367e7fc640 1 --2- 172.21.15.38:0/1905311628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d597390 0x55697d936bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.885+0000 7f367e7fc640 1 -- 172.21.15.38:0/1905311628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d56e4d0 msgr2=0x55697d937130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:46.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.885+0000 7f367e7fc640 1 --2- 172.21.15.38:0/1905311628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d56e4d0 0x55697d937130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.885+0000 7f367e7fc640 1 -- 172.21.15.38:0/1905311628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55697d6797d0 con 0x55697d5766b0 2024-05-21T15:35:46.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.885+0000 7f367effd640 1 --2- 172.21.15.38:0/1905311628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d56e4d0 0x55697d937130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:46.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.885+0000 7f367e7fc640 1 --2- 172.21.15.38:0/1905311628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d5766b0 0x55697d93e030 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f367404f000 tx=0x7f367404b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67320b9f706bacb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:46.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.885+0000 7f36657fa640 1 -- 172.21.15.38:0/1905311628 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36740bf5f0 con 0x55697d5766b0 2024-05-21T15:35:46.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.886+0000 7f36657fa640 1 -- 172.21.15.38:0/1905311628 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36740df040 con 0x55697d5766b0 2024-05-21T15:35:46.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.886+0000 7f36657fa640 1 -- 172.21.15.38:0/1905311628 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674003800 con 0x55697d5766b0 2024-05-21T15:35:46.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.887+0000 7f368c42b740 1 -- 172.21.15.38:0/1905311628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d5766b0 msgr2=0x55697d93e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:46.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.887+0000 7f368c42b740 1 --2- 172.21.15.38:0/1905311628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d5766b0 0x55697d93e030 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f367404f000 tx=0x7f367404b000 comp rx=0 tx=0).stop 2024-05-21T15:35:46.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.887+0000 7f367f7fe640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f36680a2b90 0x7f36680a50a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55697d937d80 tx=0x7f36780072a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:46.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.887+0000 7f368c42b740 1 -- 172.21.15.38:0/1905311628 shutdown_connections 2024-05-21T15:35:46.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.887+0000 7f368c42b740 1 --2- 172.21.15.38:0/1905311628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d5766b0 0x55697d93e030 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.887+0000 7f368c42b740 1 --2- 172.21.15.38:0/1905311628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d56e4d0 0x55697d937130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.887+0000 7f368c42b740 1 --2- 172.21.15.38:0/1905311628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d597390 0x55697d936bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.887+0000 7f368c42b740 1 -- 172.21.15.38:0/1905311628 >> 172.21.15.38:0/1905311628 conn(0x55697d58b4e0 msgr2=0x55697d932540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:46.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.887+0000 7f368c42b740 1 -- 172.21.15.38:0/1905311628 shutdown_connections 2024-05-21T15:35:46.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.888+0000 7f368c42b740 1 -- 172.21.15.38:0/1905311628 wait complete. 2024-05-21T15:35:46.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.888+0000 7f368c42b740 1 -- start start 2024-05-21T15:35:46.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.888+0000 7f368c42b740 1 Processor -- start 2024-05-21T15:35:46.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.888+0000 7f368c42b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d56e4d0 0x55697d940460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f368c42b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 0x55697d9409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f368c42b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d597390 0x55697d947900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f368c42b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55697d89bd00 con 0x55697d597390 2024-05-21T15:35:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f367effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d56e4d0 0x55697d940460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f368c42b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55697d856b90 con 0x55697d5766b0 2024-05-21T15:35:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f368c42b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55697d93e840 con 0x55697d56e4d0 2024-05-21T15:35:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f367effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d56e4d0 0x55697d940460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41024/0 (socket says 172.21.15.38:41024) 2024-05-21T15:35:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f367effd640 1 -- 172.21.15.38:0/3820065644 learned_addr learned my addr 172.21.15.38:0/3820065644 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f367f7fe640 1 --2- 172.21.15.38:0/3820065644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d597390 0x55697d947900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f367e7fc640 1 --2- 172.21.15.38:0/3820065644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 0x55697d9409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f368c42b740 10 client.0 _ll_register_callbacks cb 0x55697d892da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f367effd640 1 -- 172.21.15.38:0/3820065644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 msgr2=0x55697d9409a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f367effd640 1 --2- 172.21.15.38:0/3820065644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 0x55697d9409a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f367effd640 1 -- 172.21.15.38:0/3820065644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d597390 msgr2=0x55697d947900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f367effd640 1 --2- 172.21.15.38:0/3820065644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d597390 0x55697d947900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f367effd640 1 -- 172.21.15.38:0/3820065644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55697d5af3f0 con 0x55697d56e4d0 2024-05-21T15:35:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.889+0000 7f367f7fe640 1 --2- 172.21.15.38:0/3820065644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d597390 0x55697d947900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.890+0000 7f367effd640 1 --2- 172.21.15.38:0/3820065644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d56e4d0 0x55697d940460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3670005fd0 tx=0x7f3670003d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d1826132478102b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.890+0000 7f3662ff5640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:35:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.890+0000 7f3662ff5640 1 -- 172.21.15.38:0/3820065644 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367000ab70 con 0x55697d56e4d0 2024-05-21T15:35:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.890+0000 7f368c42b740 1 -- 172.21.15.38:0/3820065644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36740df040 con 0x55697d56e4d0 2024-05-21T15:35:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.890+0000 7f3662ff5640 1 -- 172.21.15.38:0/3820065644 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f367000ad10 con 0x55697d56e4d0 2024-05-21T15:35:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.890+0000 7f3662ff5640 1 -- 172.21.15.38:0/3820065644 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3670005020 con 0x55697d56e4d0 2024-05-21T15:35:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.890+0000 7f368c42b740 10 client.25498 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.890+0000 7f368c42b740 1 -- 172.21.15.38:0/3820065644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55697d56f0f0 con 0x55697d56e4d0 2024-05-21T15:35:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.890+0000 7f3662ff5640 1 -- 172.21.15.38:0/3820065644 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36740df040 con 0x55697d56e4d0 2024-05-21T15:35:46.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.890+0000 7f3662ff5640 1 -- 172.21.15.38:0/3820065644 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f367000b040 con 0x55697d56e4d0 2024-05-21T15:35:46.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.891+0000 7f368c42b740 10 client.25498 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:46.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.891+0000 7f368c42b740 10 client.25498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:46.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.891+0000 7f368c42b740 20 client.25498 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:46.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.891+0000 7f368c42b740 1 --2- 172.21.15.38:0/3820065644 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55697d555f60 0x55697d94e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.891+0000 7f368c42b740 4 client.25498 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:46.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.891+0000 7f368c42b740 1 -- 172.21.15.38:0/3820065644 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55697d8a9430 con 0x55697d555f60 2024-05-21T15:35:46.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.891+0000 7f367e7fc640 1 --2- 172.21.15.38:0/3820065644 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55697d555f60 0x55697d94e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.891+0000 7f3662ff5640 10 client.25498 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:46.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.891+0000 7f367e7fc640 1 --2- 172.21.15.38:0/3820065644 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55697d555f60 0x55697d94e180 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b80451ed672f0119 server_cookie=64058c2cc056f934 in_seq=0 out_seq=0 2024-05-21T15:35:46.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.892+0000 7f3662ff5640 1 -- 172.21.15.38:0/3820065644 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55697d8a9430 con 0x55697d555f60 2024-05-21T15:35:46.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.892+0000 7f3662ff5640 10 client.25498 handle_command_reply: tid=0 2024-05-21T15:35:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.892+0000 7f368c42b740 1 client.25498 shutdown 2024-05-21T15:35:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.892+0000 7f368c42b740 1 -- 172.21.15.38:0/3820065644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d56e4d0 msgr2=0x55697d940460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.892+0000 7f368c42b740 1 --2- 172.21.15.38:0/3820065644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d56e4d0 0x55697d940460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3670005fd0 tx=0x7f3670003d00 comp rx=0 tx=0).stop 2024-05-21T15:35:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.893+0000 7f368c42b740 1 -- 172.21.15.38:0/3820065644 shutdown_connections 2024-05-21T15:35:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.893+0000 7f368c42b740 1 --2- 172.21.15.38:0/3820065644 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55697d555f60 0x55697d94e180 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.893+0000 7f368c42b740 1 --2- 172.21.15.38:0/3820065644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d597390 0x55697d947900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.893+0000 7f3662ff5640 0 client.25498 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.893+0000 7f368c42b740 1 --2- 172.21.15.38:0/3820065644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 0x55697d9409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.893+0000 7f368c42b740 1 --2- 172.21.15.38:0/3820065644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d56e4d0 0x55697d940460 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.893+0000 7f368c42b740 1 -- 172.21.15.38:0/3820065644 >> 172.21.15.38:0/3820065644 conn(0x55697d58b4e0 msgr2=0x55697d9315c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.893+0000 7f368c42b740 1 -- 172.21.15.38:0/3820065644 shutdown_connections 2024-05-21T15:35:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.893+0000 7f368c42b740 1 -- 172.21.15.38:0/3820065644 wait complete. 2024-05-21T15:35:46.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.893+0000 7f368c42b740 20 client.25498 trim_cache size 0 max 16384 2024-05-21T15:35:46.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.929+0000 7f368c42b740 1 Processor -- start 2024-05-21T15:35:46.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.929+0000 7f368c42b740 1 -- start start 2024-05-21T15:35:46.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.929+0000 7f368c42b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d555f60 0x55697d935a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.929+0000 7f368c42b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d56e4d0 0x55697d935f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f368c42b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 0x55697d947820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f368c42b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f36801a8160 con 0x55697d56e4d0 2024-05-21T15:35:46.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f368c42b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f36801a9ca0 con 0x55697d5766b0 2024-05-21T15:35:46.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f368c42b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36801a8830 con 0x55697d555f60 2024-05-21T15:35:46.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f367e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d56e4d0 0x55697d935f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f367e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d56e4d0 0x55697d935f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49296/0 (socket says 172.21.15.38:49296) 2024-05-21T15:35:46.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f367e7fc640 1 -- 172.21.15.38:0/1446401608 learned_addr learned my addr 172.21.15.38:0/1446401608 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:46.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f367effd640 1 --2- 172.21.15.38:0/1446401608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d555f60 0x55697d935a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f367f7fe640 1 --2- 172.21.15.38:0/1446401608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 0x55697d947820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f367effd640 1 -- 172.21.15.38:0/1446401608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 msgr2=0x55697d947820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:46.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f367effd640 1 --2- 172.21.15.38:0/1446401608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 0x55697d947820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f367effd640 1 -- 172.21.15.38:0/1446401608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d56e4d0 msgr2=0x55697d935f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:46.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f367effd640 1 --2- 172.21.15.38:0/1446401608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d56e4d0 0x55697d935f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f367effd640 1 -- 172.21.15.38:0/1446401608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55697d947d60 con 0x55697d555f60 2024-05-21T15:35:46.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.930+0000 7f367e7fc640 1 --2- 172.21.15.38:0/1446401608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d56e4d0 0x55697d935f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:46.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.931+0000 7f367effd640 1 --2- 172.21.15.38:0/1446401608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d555f60 0x55697d935a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f367000dfd0 tx=0x7f3670004420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea6bb29ddf8c3e44 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.931+0000 7f3664ff9640 1 -- 172.21.15.38:0/1446401608 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367000ab70 con 0x55697d555f60 2024-05-21T15:35:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.931+0000 7f3664ff9640 1 -- 172.21.15.38:0/1446401608 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f367000ad10 con 0x55697d555f60 2024-05-21T15:35:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.931+0000 7f3664ff9640 1 -- 172.21.15.38:0/1446401608 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36700048a0 con 0x55697d555f60 2024-05-21T15:35:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.932+0000 7f368c42b740 1 -- 172.21.15.38:0/1446401608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d555f60 msgr2=0x55697d935a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:46.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.932+0000 7f368c42b740 1 --2- 172.21.15.38:0/1446401608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d555f60 0x55697d935a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f367000dfd0 tx=0x7f3670004420 comp rx=0 tx=0).stop 2024-05-21T15:35:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.932+0000 7f368c42b740 1 -- 172.21.15.38:0/1446401608 shutdown_connections 2024-05-21T15:35:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.932+0000 7f368c42b740 1 --2- 172.21.15.38:0/1446401608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 0x55697d947820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.932+0000 7f368c42b740 1 --2- 172.21.15.38:0/1446401608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d56e4d0 0x55697d935f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.932+0000 7f368c42b740 1 --2- 172.21.15.38:0/1446401608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d555f60 0x55697d935a50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.932+0000 7f368c42b740 1 -- 172.21.15.38:0/1446401608 >> 172.21.15.38:0/1446401608 conn(0x55697d58b4e0 msgr2=0x55697d935340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.932+0000 7f368c42b740 1 -- 172.21.15.38:0/1446401608 shutdown_connections 2024-05-21T15:35:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.932+0000 7f368c42b740 1 -- 172.21.15.38:0/1446401608 wait complete. 2024-05-21T15:35:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.933+0000 7f368c42b740 1 -- start start 2024-05-21T15:35:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.933+0000 7f368c42b740 1 Processor -- start 2024-05-21T15:35:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.933+0000 7f368c42b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d555f60 0x55697d944700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.933+0000 7f368c42b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d56e4d0 0x55697d944c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.933+0000 7f367e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d56e4d0 0x55697d944c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.933+0000 7f367e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d56e4d0 0x55697d944c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49310/0 (socket says 172.21.15.38:49310) 2024-05-21T15:35:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.934+0000 7f367e7fc640 1 -- 172.21.15.38:0/1079373000 learned_addr learned my addr 172.21.15.38:0/1079373000 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.934+0000 7f368c42b740 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 0x55697d9b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.934+0000 7f368c42b740 1 -- 172.21.15.38:0/1079373000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f36801a95d0 con 0x55697d56e4d0 2024-05-21T15:35:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.934+0000 7f367effd640 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d555f60 0x55697d944700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.934+0000 7f368c42b740 1 -- 172.21.15.38:0/1079373000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f36801a8f00 con 0x55697d5766b0 2024-05-21T15:35:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.934+0000 7f368c42b740 1 -- 172.21.15.38:0/1079373000 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36801a6140 con 0x55697d555f60 2024-05-21T15:35:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.934+0000 7f367f7fe640 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 0x55697d9b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.934+0000 7f367e7fc640 1 -- 172.21.15.38:0/1079373000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d555f60 msgr2=0x55697d944700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.934+0000 7f367e7fc640 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d555f60 0x55697d944700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.934+0000 7f367e7fc640 1 -- 172.21.15.38:0/1079373000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 msgr2=0x55697d9b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.934+0000 7f367e7fc640 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 0x55697d9b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.934+0000 7f367e7fc640 1 -- 172.21.15.38:0/1079373000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55697d9b4a60 con 0x55697d56e4d0 2024-05-21T15:35:46.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.934+0000 7f367f7fe640 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 0x55697d9b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.935+0000 7f367e7fc640 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d56e4d0 0x55697d944c40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f36740f0830 tx=0x7f3674000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a69cbe445ed477c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.935+0000 7f3662ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.935+0000 7f3662ff5640 1 -- 172.21.15.38:0/1079373000 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36740dd060 con 0x55697d56e4d0 2024-05-21T15:35:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.935+0000 7f3662ff5640 1 -- 172.21.15.38:0/1079373000 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f3674040450 con 0x55697d56e4d0 2024-05-21T15:35:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.935+0000 7f368c42b740 10 client.0 _ll_register_callbacks cb 0x7f367000db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.935+0000 7f368c42b740 1 -- 172.21.15.38:0/1079373000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f367000ad10 con 0x55697d56e4d0 2024-05-21T15:35:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.935+0000 7f3662ff5640 1 -- 172.21.15.38:0/1079373000 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36740bf5f0 con 0x55697d56e4d0 2024-05-21T15:35:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.935+0000 7f368c42b740 10 client.15759 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:46.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.935+0000 7f368c42b740 1 -- 172.21.15.38:0/1079373000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55697d9b59c0 con 0x55697d56e4d0 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.936+0000 7f3662ff5640 1 -- 172.21.15.38:0/1079373000 <== mon.0 v2:172.21.15.38:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3674003800 con 0x55697d56e4d0 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.936+0000 7f368c42b740 10 client.15759 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.936+0000 7f368c42b740 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.936+0000 7f368c42b740 20 client.15759 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.936+0000 7f368c42b740 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55697d597390 0x55697d9b8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.936+0000 7f368c42b740 4 client.15759 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.936+0000 7f368c42b740 1 -- 172.21.15.38:0/1079373000 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f367000ad10 con 0x55697d597390 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.936+0000 7f367effd640 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55697d597390 0x55697d9b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.937+0000 7f367effd640 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55697d597390 0x55697d9b8140 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f6be7854cacac29 server_cookie=641017f728ec0148 in_seq=0 out_seq=0 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.937+0000 7f3662ff5640 10 client.15759 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.937+0000 7f3662ff5640 1 -- 172.21.15.38:0/1079373000 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f367000ad10 con 0x55697d597390 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.937+0000 7f3662ff5640 10 client.15759 handle_command_reply: tid=0 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.937+0000 7f368c42b740 1 client.15759 shutdown 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.937+0000 7f368c42b740 1 -- 172.21.15.38:0/1079373000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d56e4d0 msgr2=0x55697d944c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:46.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.937+0000 7f368c42b740 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d56e4d0 0x55697d944c40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f36740f0830 tx=0x7f3674000bc0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.937+0000 7f368c42b740 1 -- 172.21.15.38:0/1079373000 shutdown_connections 2024-05-21T15:35:46.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.937+0000 7f368c42b740 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55697d5766b0 0x55697d9b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.937+0000 7f368c42b740 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55697d597390 0x55697d9b8140 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.937+0000 7f368c42b740 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55697d56e4d0 0x55697d944c40 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.937+0000 7f3662ff5640 0 client.15759 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:46.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.937+0000 7f368c42b740 1 --2- 172.21.15.38:0/1079373000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55697d555f60 0x55697d944700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.937+0000 7f368c42b740 1 -- 172.21.15.38:0/1079373000 >> 172.21.15.38:0/1079373000 conn(0x55697d58b4e0 msgr2=0x55697d935d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:46.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.938+0000 7f368c42b740 1 -- 172.21.15.38:0/1079373000 shutdown_connections 2024-05-21T15:35:46.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.938+0000 7f368c42b740 1 -- 172.21.15.38:0/1079373000 wait complete. 2024-05-21T15:35:46.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.938+0000 7f368c42b740 20 client.15759 trim_cache size 0 max 16384 2024-05-21T15:35:46.941 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:35:46.942 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-05-21T15:35:46.942 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-05-21T15:35:46.942 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:35:46.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.938+0000 7f368557e640 1 -- 172.21.15.38:0/576710677 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f36680a2b90 msgr2=0x7f36680a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.938+0000 7f368557e640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f36680a2b90 0x7f36680a50a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55697d937d80 tx=0x7f36780072a0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.938+0000 7f368557e640 1 -- 172.21.15.38:0/576710677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3680102340 msgr2=0x7f36801975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.938+0000 7f368557e640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3680102340 0x7f36801975f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3674005ab0 tx=0x7f36740042f0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.938+0000 7f368557e640 1 -- 172.21.15.38:0/576710677 shutdown_connections 2024-05-21T15:35:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.938+0000 7f368557e640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3680107040 0x7f368019e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.938+0000 7f368557e640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f36680a2b90 0x7f36680a50a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.938+0000 7f368557e640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3680102340 0x7f36801975f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.938+0000 7f368557e640 1 --2- 172.21.15.38:0/576710677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3680101100 0x7f36801970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.938+0000 7f368557e640 1 -- 172.21.15.38:0/576710677 >> 172.21.15.38:0/576710677 conn(0x7f36800fc2f0 msgr2=0x7f3680103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.939+0000 7f368557e640 1 -- 172.21.15.38:0/576710677 shutdown_connections 2024-05-21T15:35:46.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:46.939+0000 7f368557e640 1 -- 172.21.15.38:0/576710677 wait complete. 2024-05-21T15:35:46.949 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:35:46.950 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:35:47.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.069+0000 7f89fa931640 1 -- 172.21.15.38:0/3395346247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f4102f90 msgr2=0x7f89f41033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.069+0000 7f89fa931640 1 --2- 172.21.15.38:0/3395346247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f4102f90 0x7f89f41033a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009900 tx=0x7f89e402a410 comp rx=0 tx=0).stop 2024-05-21T15:35:47.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.070+0000 7f89fa931640 1 -- 172.21.15.38:0/3395346247 shutdown_connections 2024-05-21T15:35:47.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.070+0000 7f89fa931640 1 --2- 172.21.15.38:0/3395346247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f410a9a0 0x7f89f410ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.070+0000 7f89fa931640 1 --2- 172.21.15.38:0/3395346247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89f4107e70 0x7f89f410a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.070+0000 7f89fa931640 1 --2- 172.21.15.38:0/3395346247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f4102f90 0x7f89f41033a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.070+0000 7f89fa931640 1 -- 172.21.15.38:0/3395346247 >> 172.21.15.38:0/3395346247 conn(0x7f89f40fc330 msgr2=0x7f89f40fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.070+0000 7f89fa931640 1 -- 172.21.15.38:0/3395346247 shutdown_connections 2024-05-21T15:35:47.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.070+0000 7f89fa931640 1 -- 172.21.15.38:0/3395346247 wait complete. 2024-05-21T15:35:47.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.071+0000 7f89fa931640 1 Processor -- start 2024-05-21T15:35:47.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.071+0000 7f89fa931640 1 -- start start 2024-05-21T15:35:47.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.071+0000 7f89fa931640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f4107e70 0x7f89f419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.071+0000 7f89fa931640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f410a9a0 0x7f89f419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.071+0000 7f89fa931640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89f41a0960 0x7f89f41a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.071+0000 7f89fa931640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f89f4110900 con 0x7f89f410a9a0 2024-05-21T15:35:47.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.071+0000 7f89fa931640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f89f4110780 con 0x7f89f41a0960 2024-05-21T15:35:47.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.071+0000 7f89f37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f410a9a0 0x7f89f419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.071+0000 7f89fa931640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f89f4110a80 con 0x7f89f4107e70 2024-05-21T15:35:47.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.071+0000 7f89f37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f410a9a0 0x7f89f419ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49318/0 (socket says 172.21.15.38:49318) 2024-05-21T15:35:47.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.071+0000 7f89f37fe640 1 -- 172.21.15.38:0/2320266310 learned_addr learned my addr 172.21.15.38:0/2320266310 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.071+0000 7f89f8ece640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89f41a0960 0x7f89f41a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.071+0000 7f89f3fff640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f4107e70 0x7f89f419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.072+0000 7f89f37fe640 1 -- 172.21.15.38:0/2320266310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f4107e70 msgr2=0x7f89f419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.072+0000 7f89f37fe640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f4107e70 0x7f89f419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.072+0000 7f89f37fe640 1 -- 172.21.15.38:0/2320266310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89f41a0960 msgr2=0x7f89f41a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.072+0000 7f89f37fe640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89f41a0960 0x7f89f41a2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.072+0000 7f89f37fe640 1 -- 172.21.15.38:0/2320266310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f41a34b0 con 0x7f89f410a9a0 2024-05-21T15:35:47.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.072+0000 7f89f8ece640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89f41a0960 0x7f89f41a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:47.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.072+0000 7f89f3fff640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f4107e70 0x7f89f419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:47.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.072+0000 7f89f37fe640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f410a9a0 0x7f89f419ba60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f89e800e9f0 tx=0x7f89e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.072+0000 7f89f17fa640 1 -- 172.21.15.38:0/2320266310 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e800cd80 con 0x7f89f410a9a0 2024-05-21T15:35:47.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.072+0000 7f89f17fa640 1 -- 172.21.15.38:0/2320266310 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89e8010910 con 0x7f89f410a9a0 2024-05-21T15:35:47.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.072+0000 7f89f17fa640 1 -- 172.21.15.38:0/2320266310 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8018990 con 0x7f89f410a9a0 2024-05-21T15:35:47.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.072+0000 7f89fa931640 1 -- 172.21.15.38:0/2320266310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f41a37a0 con 0x7f89f410a9a0 2024-05-21T15:35:47.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.073+0000 7f89fa931640 1 -- 172.21.15.38:0/2320266310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f41a3c70 con 0x7f89f410a9a0 2024-05-21T15:35:47.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.074+0000 7f89fa931640 1 -- 172.21.15.38:0/2320266310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f40691c0 con 0x7f89f410a9a0 2024-05-21T15:35:47.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.077+0000 7f89f17fa640 1 -- 172.21.15.38:0/2320266310 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f89e8010ac0 con 0x7f89f410a9a0 2024-05-21T15:35:47.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.078+0000 7f89f17fa640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f89dc0a2cb0 0x7f89dc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.079+0000 7f89f17fa640 1 -- 172.21.15.38:0/2320266310 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f89e80cfbd0 con 0x7f89f410a9a0 2024-05-21T15:35:47.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.079+0000 7f89f3fff640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f89dc0a2cb0 0x7f89dc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.080+0000 7f89f3fff640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f89dc0a2cb0 0x7f89dc0a51c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f89e4008000 tx=0x7f89e4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.081+0000 7f89f17fa640 1 -- 172.21.15.38:0/2320266310 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f89e809cae0 con 0x7f89f410a9a0 2024-05-21T15:35:47.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.308+0000 7f89fa931640 1 -- 172.21.15.38:0/2320266310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89f40fde50 con 0x7f89f410a9a0 2024-05-21T15:35:47.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.309+0000 7f89f17fa640 1 -- 172.21.15.38:0/2320266310 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f89e809c3e0 con 0x7f89f410a9a0 2024-05-21T15:35:47.308 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:35:47.308 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:34:36.440701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24551,24523,24602,24550,24515]},"id":1}]} 2024-05-21T15:35:47.309 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-05-21T15:35:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.312+0000 7f89fa931640 1 -- 172.21.15.38:0/2320266310 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f89dc0a2cb0 msgr2=0x7f89dc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.312+0000 7f89fa931640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f89dc0a2cb0 0x7f89dc0a51c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f89e4008000 tx=0x7f89e4031400 comp rx=0 tx=0).stop 2024-05-21T15:35:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.312+0000 7f89fa931640 1 -- 172.21.15.38:0/2320266310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f410a9a0 msgr2=0x7f89f419ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.312+0000 7f89fa931640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f410a9a0 0x7f89f419ba60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f89e800e9f0 tx=0x7f89e800eeb0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.312+0000 7f89fa931640 1 -- 172.21.15.38:0/2320266310 shutdown_connections 2024-05-21T15:35:47.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.312+0000 7f89fa931640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89f41a0960 0x7f89f41a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.312+0000 7f89fa931640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f89dc0a2cb0 0x7f89dc0a51c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.312+0000 7f89fa931640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89f410a9a0 0x7f89f419ba60 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.312+0000 7f89fa931640 1 --2- 172.21.15.38:0/2320266310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89f4107e70 0x7f89f419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.312+0000 7f89fa931640 1 -- 172.21.15.38:0/2320266310 >> 172.21.15.38:0/2320266310 conn(0x7f89f40fc330 msgr2=0x7f89f4108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.312+0000 7f89fa931640 1 -- 172.21.15.38:0/2320266310 shutdown_connections 2024-05-21T15:35:47.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.312+0000 7f89fa931640 1 -- 172.21.15.38:0/2320266310 wait complete. 2024-05-21T15:35:47.327 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:35:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.449+0000 7f64ea1d5640 1 -- 172.21.15.38:0/405314082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e4100dc0 msgr2=0x7f64e41011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.449+0000 7f64ea1d5640 1 --2- 172.21.15.38:0/405314082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e4100dc0 0x7f64e41011d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009900 tx=0x7f64d402a410 comp rx=0 tx=0).stop 2024-05-21T15:35:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.449+0000 7f64ea1d5640 1 -- 172.21.15.38:0/405314082 shutdown_connections 2024-05-21T15:35:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.449+0000 7f64ea1d5640 1 --2- 172.21.15.38:0/405314082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e4108710 0x7f64e410ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.449+0000 7f64ea1d5640 1 --2- 172.21.15.38:0/405314082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e4105ba0 0x7f64e4108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.449+0000 7f64ea1d5640 1 --2- 172.21.15.38:0/405314082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e4100dc0 0x7f64e41011d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.449+0000 7f64ea1d5640 1 -- 172.21.15.38:0/405314082 >> 172.21.15.38:0/405314082 conn(0x7f64e40fc2f0 msgr2=0x7f64e40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.449+0000 7f64ea1d5640 1 -- 172.21.15.38:0/405314082 shutdown_connections 2024-05-21T15:35:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.449+0000 7f64ea1d5640 1 -- 172.21.15.38:0/405314082 wait complete. 2024-05-21T15:35:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.450+0000 7f64ea1d5640 1 Processor -- start 2024-05-21T15:35:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.450+0000 7f64ea1d5640 1 -- start start 2024-05-21T15:35:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64ea1d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e4100dc0 0x7f64e41992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64ea1d5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e4105ba0 0x7f64e41997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64ea1d5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e4108710 0x7f64e41a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64ea1d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f64e410e710 con 0x7f64e4100dc0 2024-05-21T15:35:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64ea1d5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f64e410e590 con 0x7f64e4105ba0 2024-05-21T15:35:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64ea1d5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f64e410e890 con 0x7f64e4108710 2024-05-21T15:35:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64e37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e4100dc0 0x7f64e41992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64e37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e4100dc0 0x7f64e41992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49332/0 (socket says 172.21.15.38:49332) 2024-05-21T15:35:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64e37fe640 1 -- 172.21.15.38:0/3057409095 learned_addr learned my addr 172.21.15.38:0/3057409095 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e4105ba0 0x7f64e41997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64e3fff640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e4108710 0x7f64e41a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64e37fe640 1 -- 172.21.15.38:0/3057409095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e4108710 msgr2=0x7f64e41a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64e37fe640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e4108710 0x7f64e41a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64e37fe640 1 -- 172.21.15.38:0/3057409095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e4105ba0 msgr2=0x7f64e41997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64e37fe640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e4105ba0 0x7f64e41997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.451+0000 7f64e37fe640 1 -- 172.21.15.38:0/3057409095 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e41a0df0 con 0x7f64e4100dc0 2024-05-21T15:35:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.452+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e4105ba0 0x7f64e41997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.452+0000 7f64e37fe640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e4100dc0 0x7f64e41992b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f64d4031440 tx=0x7f64d4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.452+0000 7f64e0ff9640 1 -- 172.21.15.38:0/3057409095 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d403f030 con 0x7f64e4100dc0 2024-05-21T15:35:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.452+0000 7f64ea1d5640 1 -- 172.21.15.38:0/3057409095 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e41a1080 con 0x7f64e4100dc0 2024-05-21T15:35:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.452+0000 7f64e0ff9640 1 -- 172.21.15.38:0/3057409095 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d4003ce0 con 0x7f64e4100dc0 2024-05-21T15:35:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.452+0000 7f64e0ff9640 1 -- 172.21.15.38:0/3057409095 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4035070 con 0x7f64e4100dc0 2024-05-21T15:35:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.452+0000 7f64ea1d5640 1 -- 172.21.15.38:0/3057409095 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e41a15c0 con 0x7f64e4100dc0 2024-05-21T15:35:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.456+0000 7f64f107e740 1 Processor -- start 2024-05-21T15:35:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.456+0000 7f64f107e740 1 -- start start 2024-05-21T15:35:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.456+0000 7f64f107e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019777390 0x55b019b16b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.456+0000 7f64f107e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b01974e4d0 0x55b019b17080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.456+0000 7f64f107e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b1df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.456+0000 7f64e0ff9640 1 -- 172.21.15.38:0/3057409095 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f64d4042030 con 0x7f64e4100dc0 2024-05-21T15:35:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.456+0000 7f64f107e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b019a7bc40 con 0x55b0197566b0 2024-05-21T15:35:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.456+0000 7f64f107e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b019a23580 con 0x55b01974e4d0 2024-05-21T15:35:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.456+0000 7f64f107e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b019a77c00 con 0x55b019777390 2024-05-21T15:35:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.456+0000 7f64e2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b1df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.456+0000 7f64e2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b1df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49334/0 (socket says 172.21.15.38:49334) 2024-05-21T15:35:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.456+0000 7f64e2ffd640 1 -- 172.21.15.38:0/2763192944 learned_addr learned my addr 172.21.15.38:0/2763192944 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.457+0000 7f64e3fff640 1 --2- 172.21.15.38:0/2763192944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019777390 0x55b019b16b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.457+0000 7f64e2ffd640 1 -- 172.21.15.38:0/2763192944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019777390 msgr2=0x55b019b16b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.457+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/2763192944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019777390 0x55b019b16b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.457+0000 7f64e2ffd640 1 -- 172.21.15.38:0/2763192944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b01974e4d0 msgr2=0x55b019b17080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:35:47.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.457+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/2763192944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b01974e4d0 0x55b019b17080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.457+0000 7f64e2ffd640 1 -- 172.21.15.38:0/2763192944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b01978f3f0 con 0x55b0197566b0 2024-05-21T15:35:47.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.457+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/2763192944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b1df80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009c20 tx=0x7f64d8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922e17cfd787028 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.458+0000 7f64c9ffb640 1 -- 172.21.15.38:0/2763192944 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8004030 con 0x55b0197566b0 2024-05-21T15:35:47.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.458+0000 7f64c9ffb640 1 -- 172.21.15.38:0/2763192944 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d80041d0 con 0x55b0197566b0 2024-05-21T15:35:47.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.458+0000 7f64e0ff9640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f64cc0a2b90 0x7f64cc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.458+0000 7f64c9ffb640 1 -- 172.21.15.38:0/2763192944 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d80126b0 con 0x55b0197566b0 2024-05-21T15:35:47.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.458+0000 7f64e3fff640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f64cc0a2b90 0x7f64cc0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.458+0000 7f64e0ff9640 1 -- 172.21.15.38:0/3057409095 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f64d40ef4b0 con 0x7f64e4100dc0 2024-05-21T15:35:47.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.459+0000 7f64e3fff640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f64cc0a2b90 0x7f64cc0a50a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55b019b17cd0 tx=0x7f64dc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.459+0000 7f64f107e740 1 -- 172.21.15.38:0/2763192944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 msgr2=0x55b019b1df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.459+0000 7f64f107e740 1 --2- 172.21.15.38:0/2763192944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b1df80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009c20 tx=0x7f64d8010740 comp rx=0 tx=0).stop 2024-05-21T15:35:47.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.459+0000 7f64f107e740 1 -- 172.21.15.38:0/2763192944 shutdown_connections 2024-05-21T15:35:47.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.459+0000 7f64f107e740 1 --2- 172.21.15.38:0/2763192944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b1df80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.459+0000 7f64f107e740 1 --2- 172.21.15.38:0/2763192944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b01974e4d0 0x55b019b17080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.459+0000 7f64f107e740 1 --2- 172.21.15.38:0/2763192944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019777390 0x55b019b16b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.459+0000 7f64f107e740 1 -- 172.21.15.38:0/2763192944 >> 172.21.15.38:0/2763192944 conn(0x55b01976b4e0 msgr2=0x55b019b12490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.459+0000 7f64f107e740 1 -- 172.21.15.38:0/2763192944 shutdown_connections 2024-05-21T15:35:47.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.460+0000 7f64f107e740 1 -- 172.21.15.38:0/2763192944 wait complete. 2024-05-21T15:35:47.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.460+0000 7f64f107e740 1 -- start start 2024-05-21T15:35:47.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.460+0000 7f64f107e740 1 Processor -- start 2024-05-21T15:35:47.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.461+0000 7f64f107e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b01974e4d0 0x55b019b209b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.461+0000 7f64f107e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b20ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.461+0000 7f64e2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b20ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.461+0000 7f64e2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b20ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49342/0 (socket says 172.21.15.38:49342) 2024-05-21T15:35:47.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.461+0000 7f64e37fe640 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b01974e4d0 0x55b019b209b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64e2ffd640 1 -- 172.21.15.38:0/1012208771 learned_addr learned my addr 172.21.15.38:0/1012208771 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64f107e740 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b019777390 0x55b019b27e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64f107e740 1 -- 172.21.15.38:0/1012208771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b019b1ea70 con 0x55b0197566b0 2024-05-21T15:35:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64f107e740 1 -- 172.21.15.38:0/1012208771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b019a36c40 con 0x55b019777390 2024-05-21T15:35:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64f107e740 1 -- 172.21.15.38:0/1012208771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b019b1ebf0 con 0x55b01974e4d0 2024-05-21T15:35:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64f107e740 10 client.0 _ll_register_callbacks cb 0x55b01978ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64e3fff640 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b019777390 0x55b019b27e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64e37fe640 1 -- 172.21.15.38:0/1012208771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b019777390 msgr2=0x55b019b27e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64e37fe640 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b019777390 0x55b019b27e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64e37fe640 1 -- 172.21.15.38:0/1012208771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 msgr2=0x55b019b20ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64e37fe640 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b20ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64e37fe640 1 -- 172.21.15.38:0/1012208771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0198597d0 con 0x55b01974e4d0 2024-05-21T15:35:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b20ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64e37fe640 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b01974e4d0 0x55b019b209b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55b019b17ee0 tx=0x7f64d402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=babc4d9b5627ce2f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:35:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64f107e740 1 -- 172.21.15.38:0/1012208771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64d80041d0 con 0x55b01974e4d0 2024-05-21T15:35:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.462+0000 7f64c2ffd640 1 -- 172.21.15.38:0/1012208771 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4049040 con 0x55b01974e4d0 2024-05-21T15:35:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.463+0000 7f64c2ffd640 1 -- 172.21.15.38:0/1012208771 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f64d40ef110 con 0x55b01974e4d0 2024-05-21T15:35:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.463+0000 7f64c2ffd640 1 -- 172.21.15.38:0/1012208771 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f64d40bc3c0 con 0x55b01974e4d0 2024-05-21T15:35:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.463+0000 7f64f107e740 10 client.25519 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.463+0000 7f64f107e740 1 -- 172.21.15.38:0/1012208771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55b019b2afb0 con 0x55b01974e4d0 2024-05-21T15:35:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.463+0000 7f64c2ffd640 1 -- 172.21.15.38:0/1012208771 <== mon.1 v2:172.21.15.53:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f64d40bbf20 con 0x55b01974e4d0 2024-05-21T15:35:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.464+0000 7f64f107e740 10 client.25519 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.464+0000 7f64f107e740 10 client.25519 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:35:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.464+0000 7f64f107e740 20 client.25519 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.464+0000 7f64f107e740 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b019735f60 0x55b019b2e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.464+0000 7f64f107e740 4 client.25519 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.464+0000 7f64f107e740 1 -- 172.21.15.38:0/1012208771 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b019a89370 con 0x55b019735f60 2024-05-21T15:35:47.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.464+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b019735f60 0x55b019b2e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.465+0000 7f64c2ffd640 10 client.25519 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:35:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.465+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b019735f60 0x55b019b2e6a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5c36279cddee2f91 server_cookie=a5367f86a5056709 in_seq=0 out_seq=0 2024-05-21T15:35:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.465+0000 7f64c2ffd640 1 -- 172.21.15.38:0/1012208771 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b019a89370 con 0x55b019735f60 2024-05-21T15:35:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.465+0000 7f64c2ffd640 10 client.25519 handle_command_reply: tid=0 2024-05-21T15:35:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.465+0000 7f64f107e740 1 client.25519 shutdown 2024-05-21T15:35:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.466+0000 7f64f107e740 1 -- 172.21.15.38:0/1012208771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b01974e4d0 msgr2=0x55b019b209b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.466+0000 7f64f107e740 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b01974e4d0 0x55b019b209b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55b019b17ee0 tx=0x7f64d402a410 comp rx=0 tx=0).stop 2024-05-21T15:35:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.466+0000 7f64f107e740 1 -- 172.21.15.38:0/1012208771 shutdown_connections 2024-05-21T15:35:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.466+0000 7f64f107e740 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b019735f60 0x55b019b2e6a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.466+0000 7f64c2ffd640 0 client.25519 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:35:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.466+0000 7f64f107e740 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b019777390 0x55b019b27e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.466+0000 7f64f107e740 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b20ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.466+0000 7f64f107e740 1 --2- 172.21.15.38:0/1012208771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b01974e4d0 0x55b019b209b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.466+0000 7f64f107e740 1 -- 172.21.15.38:0/1012208771 >> 172.21.15.38:0/1012208771 conn(0x55b01976b4e0 msgr2=0x55b019b11570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.466+0000 7f64f107e740 1 -- 172.21.15.38:0/1012208771 shutdown_connections 2024-05-21T15:35:47.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.466+0000 7f64f107e740 1 -- 172.21.15.38:0/1012208771 wait complete. 2024-05-21T15:35:47.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.466+0000 7f64f107e740 20 client.25519 trim_cache size 0 max 16384 2024-05-21T15:35:47.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.493+0000 7f64f107e740 1 Processor -- start 2024-05-21T15:35:47.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.493+0000 7f64f107e740 1 -- start start 2024-05-21T15:35:47.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.493+0000 7f64f107e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019735f60 0x55b019b1f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.493+0000 7f64f107e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b01974e4d0 0x55b019b27de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.493+0000 7f64f107e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b1f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.493+0000 7f64f107e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41aa2d0 con 0x55b0197566b0 2024-05-21T15:35:47.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.493+0000 7f64f107e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41abe10 con 0x55b01974e4d0 2024-05-21T15:35:47.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.493+0000 7f64f107e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41aa9a0 con 0x55b019735f60 2024-05-21T15:35:47.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.494+0000 7f64e3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b1f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.494+0000 7f64e3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b1f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49346/0 (socket says 172.21.15.38:49346) 2024-05-21T15:35:47.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.494+0000 7f64e3fff640 1 -- 172.21.15.38:0/62025700 learned_addr learned my addr 172.21.15.38:0/62025700 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.494+0000 7f64e37fe640 1 --2- 172.21.15.38:0/62025700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019735f60 0x55b019b1f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.494+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/62025700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b01974e4d0 0x55b019b27de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.494+0000 7f64e2ffd640 1 -- 172.21.15.38:0/62025700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019735f60 msgr2=0x55b019b1f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.494+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/62025700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019735f60 0x55b019b1f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.494+0000 7f64e2ffd640 1 -- 172.21.15.38:0/62025700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 msgr2=0x55b019b1f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.494+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/62025700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b1f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.494+0000 7f64e2ffd640 1 -- 172.21.15.38:0/62025700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b019b21060 con 0x55b01974e4d0 2024-05-21T15:35:47.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.494+0000 7f64e3fff640 1 --2- 172.21.15.38:0/62025700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b1f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:47.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.494+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/62025700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b01974e4d0 0x55b019b27de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55b019b2ee10 tx=0x7f64d8005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df72308805dc9ef server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.495+0000 7f64c97fa640 1 -- 172.21.15.38:0/62025700 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d802e020 con 0x55b01974e4d0 2024-05-21T15:35:47.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.495+0000 7f64c97fa640 1 -- 172.21.15.38:0/62025700 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d80053f0 con 0x55b01974e4d0 2024-05-21T15:35:47.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.495+0000 7f64c97fa640 1 -- 172.21.15.38:0/62025700 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8012870 con 0x55b01974e4d0 2024-05-21T15:35:47.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.496+0000 7f64f107e740 1 -- 172.21.15.38:0/62025700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b01974e4d0 msgr2=0x55b019b27de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.496+0000 7f64f107e740 1 --2- 172.21.15.38:0/62025700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b01974e4d0 0x55b019b27de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55b019b2ee10 tx=0x7f64d8005d90 comp rx=0 tx=0).stop 2024-05-21T15:35:47.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.496+0000 7f64f107e740 1 -- 172.21.15.38:0/62025700 shutdown_connections 2024-05-21T15:35:47.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.496+0000 7f64f107e740 1 --2- 172.21.15.38:0/62025700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0197566b0 0x55b019b1f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.496+0000 7f64f107e740 1 --2- 172.21.15.38:0/62025700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b01974e4d0 0x55b019b27de0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.496+0000 7f64f107e740 1 --2- 172.21.15.38:0/62025700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019735f60 0x55b019b1f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.496+0000 7f64f107e740 1 -- 172.21.15.38:0/62025700 >> 172.21.15.38:0/62025700 conn(0x55b01976b4e0 msgr2=0x55b019b20950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.496+0000 7f64f107e740 1 -- 172.21.15.38:0/62025700 shutdown_connections 2024-05-21T15:35:47.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.496+0000 7f64f107e740 1 -- 172.21.15.38:0/62025700 wait complete. 2024-05-21T15:35:47.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.496+0000 7f64f107e740 1 -- start start 2024-05-21T15:35:47.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.497+0000 7f64f107e740 1 Processor -- start 2024-05-21T15:35:47.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.497+0000 7f64f107e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019735f60 0x55b019b23160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.497+0000 7f64f107e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b01974e4d0 0x55b019b236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.497+0000 7f64f107e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0197566b0 0x55b019b265e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.497+0000 7f64f107e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41ab740 con 0x55b01974e4d0 2024-05-21T15:35:47.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.497+0000 7f64f107e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41ab070 con 0x55b0197566b0 2024-05-21T15:35:47.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.497+0000 7f64f107e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41a82b0 con 0x55b019735f60 2024-05-21T15:35:47.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.497+0000 7f64f107e740 10 client.0 _ll_register_callbacks cb 0x7f64d4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:47.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.497+0000 7f64e2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b01974e4d0 0x55b019b236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.497+0000 7f64e2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b01974e4d0 0x55b019b236a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49360/0 (socket says 172.21.15.38:49360) 2024-05-21T15:35:47.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.497+0000 7f64e2ffd640 1 -- 172.21.15.38:0/813498290 learned_addr learned my addr 172.21.15.38:0/813498290 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.497+0000 7f64e3fff640 1 --2- 172.21.15.38:0/813498290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0197566b0 0x55b019b265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.497+0000 7f64e37fe640 1 --2- 172.21.15.38:0/813498290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019735f60 0x55b019b23160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.498+0000 7f64e2ffd640 1 -- 172.21.15.38:0/813498290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019735f60 msgr2=0x55b019b23160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.498+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/813498290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019735f60 0x55b019b23160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.498+0000 7f64e2ffd640 1 -- 172.21.15.38:0/813498290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0197566b0 msgr2=0x55b019b265e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.498+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/813498290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0197566b0 0x55b019b265e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.498+0000 7f64e2ffd640 1 -- 172.21.15.38:0/813498290 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b019b26ba0 con 0x55b01974e4d0 2024-05-21T15:35:47.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.498+0000 7f64e37fe640 1 --2- 172.21.15.38:0/813498290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019735f60 0x55b019b23160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:47.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.498+0000 7f64e2ffd640 1 --2- 172.21.15.38:0/813498290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b01974e4d0 0x55b019b236a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f64d8010c40 tx=0x7f64d8004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db355609a891a39 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.498+0000 7f64c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:47.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.498+0000 7f64c2ffd640 1 -- 172.21.15.38:0/813498290 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d803b020 con 0x55b01974e4d0 2024-05-21T15:35:47.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.498+0000 7f64c2ffd640 1 -- 172.21.15.38:0/813498290 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f64d80037f0 con 0x55b01974e4d0 2024-05-21T15:35:47.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.498+0000 7f64f107e740 1 -- 172.21.15.38:0/813498290 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64d80053f0 con 0x55b01974e4d0 2024-05-21T15:35:47.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.498+0000 7f64c2ffd640 1 -- 172.21.15.38:0/813498290 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f64d8046050 con 0x55b01974e4d0 2024-05-21T15:35:47.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.498+0000 7f64f107e740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:47.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.498+0000 7f64f107e740 1 -- 172.21.15.38:0/813498290 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b019b27e20 con 0x55b01974e4d0 2024-05-21T15:35:47.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.499+0000 7f64c2ffd640 1 -- 172.21.15.38:0/813498290 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d80053f0 con 0x55b01974e4d0 2024-05-21T15:35:47.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.499+0000 7f64c2ffd640 1 -- 172.21.15.38:0/813498290 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f64d800f040 con 0x55b01974e4d0 2024-05-21T15:35:47.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.499+0000 7f64f107e740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:47.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.499+0000 7f64f107e740 10 client.15783 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:35:47.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.499+0000 7f64f107e740 20 client.15783 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:47.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.499+0000 7f64f107e740 1 --2- 172.21.15.38:0/813498290 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b019777390 0x55b019b9d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.499+0000 7f64f107e740 4 client.15783 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:35:47.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.499+0000 7f64f107e740 1 -- 172.21.15.38:0/813498290 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b019a836c0 con 0x55b019777390 2024-05-21T15:35:47.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.499+0000 7f64e37fe640 1 --2- 172.21.15.38:0/813498290 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b019777390 0x55b019b9d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64c2ffd640 10 client.15783 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:35:47.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64e37fe640 1 --2- 172.21.15.38:0/813498290 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b019777390 0x55b019b9d080 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e7e2a5c5825bbb95 server_cookie=2660654e6bb51edb in_seq=0 out_seq=0 2024-05-21T15:35:47.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64c2ffd640 1 -- 172.21.15.38:0/813498290 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b019a836c0 con 0x55b019777390 2024-05-21T15:35:47.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64c2ffd640 10 client.15783 handle_command_reply: tid=0 2024-05-21T15:35:47.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64f107e740 1 client.15783 shutdown 2024-05-21T15:35:47.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64f107e740 1 -- 172.21.15.38:0/813498290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b01974e4d0 msgr2=0x55b019b236a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64f107e740 1 --2- 172.21.15.38:0/813498290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b01974e4d0 0x55b019b236a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f64d8010c40 tx=0x7f64d8004000 comp rx=0 tx=0).stop 2024-05-21T15:35:47.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64f107e740 1 -- 172.21.15.38:0/813498290 shutdown_connections 2024-05-21T15:35:47.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64f107e740 1 --2- 172.21.15.38:0/813498290 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b019777390 0x55b019b9d080 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64f107e740 1 --2- 172.21.15.38:0/813498290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0197566b0 0x55b019b265e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64f107e740 1 --2- 172.21.15.38:0/813498290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b01974e4d0 0x55b019b236a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64f107e740 1 --2- 172.21.15.38:0/813498290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b019735f60 0x55b019b23160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64f107e740 1 -- 172.21.15.38:0/813498290 >> 172.21.15.38:0/813498290 conn(0x55b01976b4e0 msgr2=0x55b019b1acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64c2ffd640 0 client.15783 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:35:47.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64f107e740 1 -- 172.21.15.38:0/813498290 shutdown_connections 2024-05-21T15:35:47.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.500+0000 7f64f107e740 1 -- 172.21.15.38:0/813498290 wait complete. 2024-05-21T15:35:47.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.501+0000 7f64f107e740 20 client.15783 trim_cache size 0 max 16384 2024-05-21T15:35:47.505 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:35:47.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.501+0000 7f64ea1d5640 1 -- 172.21.15.38:0/3057409095 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f64cc0a2b90 msgr2=0x7f64cc0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.501+0000 7f64ea1d5640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f64cc0a2b90 0x7f64cc0a50a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55b019b17cd0 tx=0x7f64dc009500 comp rx=0 tx=0).stop 2024-05-21T15:35:47.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.501+0000 7f64ea1d5640 1 -- 172.21.15.38:0/3057409095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e4100dc0 msgr2=0x7f64e41992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.501+0000 7f64ea1d5640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e4100dc0 0x7f64e41992b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f64d4031440 tx=0x7f64d4032ce0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.501+0000 7f64ea1d5640 1 -- 172.21.15.38:0/3057409095 shutdown_connections 2024-05-21T15:35:47.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.501+0000 7f64ea1d5640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e4108710 0x7f64e41a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.501+0000 7f64ea1d5640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e4105ba0 0x7f64e41997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.501+0000 7f64ea1d5640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f64cc0a2b90 0x7f64cc0a50a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.501+0000 7f64ea1d5640 1 --2- 172.21.15.38:0/3057409095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64e4100dc0 0x7f64e41992b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.501+0000 7f64ea1d5640 1 -- 172.21.15.38:0/3057409095 >> 172.21.15.38:0/3057409095 conn(0x7f64e40fc2f0 msgr2=0x7f64e4106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.501+0000 7f64ea1d5640 1 -- 172.21.15.38:0/3057409095 shutdown_connections 2024-05-21T15:35:47.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.501+0000 7f64ea1d5640 1 -- 172.21.15.38:0/3057409095 wait complete. 2024-05-21T15:35:47.512 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:35:47.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.641+0000 7f51f67ce640 1 -- 172.21.15.38:0/1506395929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51f0108840 msgr2=0x7f51f010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.641+0000 7f51f67ce640 1 --2- 172.21.15.38:0/1506395929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51f0108840 0x7f51f010ac60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009ca0 tx=0x7f51e802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.641+0000 7f51f67ce640 1 -- 172.21.15.38:0/1506395929 shutdown_connections 2024-05-21T15:35:47.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.641+0000 7f51f67ce640 1 --2- 172.21.15.38:0/1506395929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51f0108840 0x7f51f010ac60 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.641+0000 7f51f67ce640 1 --2- 172.21.15.38:0/1506395929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51f0105ee0 0x7f51f0108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.641+0000 7f51f67ce640 1 --2- 172.21.15.38:0/1506395929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0103580 0x7f51f01059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.641+0000 7f51f67ce640 1 -- 172.21.15.38:0/1506395929 >> 172.21.15.38:0/1506395929 conn(0x7f51f00fcb60 msgr2=0x7f51f00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.641+0000 7f51f67ce640 1 -- 172.21.15.38:0/1506395929 shutdown_connections 2024-05-21T15:35:47.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.641+0000 7f51f67ce640 1 -- 172.21.15.38:0/1506395929 wait complete. 2024-05-21T15:35:47.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.642+0000 7f51f67ce640 1 Processor -- start 2024-05-21T15:35:47.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.642+0000 7f51f67ce640 1 -- start start 2024-05-21T15:35:47.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.642+0000 7f51f67ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51f0103580 0x7f51f01029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.642+0000 7f51f67ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0105ee0 0x7f51f0101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.642+0000 7f51f67ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51f0108840 0x7f51f01015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.642+0000 7f51f67ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f51f010e6f0 con 0x7f51f0108840 2024-05-21T15:35:47.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.642+0000 7f51f67ce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51f010e570 con 0x7f51f0103580 2024-05-21T15:35:47.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.642+0000 7f51f67ce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f51f010e870 con 0x7f51f0105ee0 2024-05-21T15:35:47.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.643+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0105ee0 0x7f51f0101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.643+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0105ee0 0x7f51f0101070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41130/0 (socket says 172.21.15.38:41130) 2024-05-21T15:35:47.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.643+0000 7f51e7fff640 1 -- 172.21.15.38:0/1517963608 learned_addr learned my addr 172.21.15.38:0/1517963608 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.643+0000 7f51f4d6b640 1 --2- 172.21.15.38:0/1517963608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51f0108840 0x7f51f01015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.643+0000 7f51effff640 1 --2- 172.21.15.38:0/1517963608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51f0103580 0x7f51f01029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.643+0000 7f51e7fff640 1 -- 172.21.15.38:0/1517963608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51f0103580 msgr2=0x7f51f01029d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.643+0000 7f51e7fff640 1 --2- 172.21.15.38:0/1517963608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51f0103580 0x7f51f01029d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.643+0000 7f51e7fff640 1 -- 172.21.15.38:0/1517963608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51f0108840 msgr2=0x7f51f01015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.643+0000 7f51e7fff640 1 --2- 172.21.15.38:0/1517963608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51f0108840 0x7f51f01015b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.643+0000 7f51e7fff640 1 -- 172.21.15.38:0/1517963608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f0101ce0 con 0x7f51f0105ee0 2024-05-21T15:35:47.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.643+0000 7f51e7fff640 1 --2- 172.21.15.38:0/1517963608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0105ee0 0x7f51f0101070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f51d800ea60 tx=0x7f51d800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.644+0000 7f51edffb640 1 -- 172.21.15.38:0/1517963608 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d800cde0 con 0x7f51f0105ee0 2024-05-21T15:35:47.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.644+0000 7f51f67ce640 1 -- 172.21.15.38:0/1517963608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f01a32d0 con 0x7f51f0105ee0 2024-05-21T15:35:47.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.644+0000 7f51edffb640 1 -- 172.21.15.38:0/1517963608 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51d80035f0 con 0x7f51f0105ee0 2024-05-21T15:35:47.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.644+0000 7f51f67ce640 1 -- 172.21.15.38:0/1517963608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f01a37c0 con 0x7f51f0105ee0 2024-05-21T15:35:47.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.644+0000 7f51edffb640 1 -- 172.21.15.38:0/1517963608 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8010670 con 0x7f51f0105ee0 2024-05-21T15:35:47.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.647+0000 7f51fd676740 1 Processor -- start 2024-05-21T15:35:47.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.647+0000 7f51fd676740 1 -- start start 2024-05-21T15:35:47.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51edffb640 1 -- 172.21.15.38:0/1517963608 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f51d80037a0 con 0x7f51f0105ee0 2024-05-21T15:35:47.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51fd676740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b3f390 0x560789edeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51fd676740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b164d0 0x560789edf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51fd676740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b1e6b0 0x560789ee5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51fd676740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560789e43c40 con 0x560789b164d0 2024-05-21T15:35:47.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51fd676740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560789deb580 con 0x560789b3f390 2024-05-21T15:35:47.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51fd676740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560789e3fc00 con 0x560789b1e6b0 2024-05-21T15:35:47.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b164d0 0x560789edf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b164d0 0x560789edf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49384/0 (socket says 172.21.15.38:49384) 2024-05-21T15:35:47.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51effff640 1 -- 172.21.15.38:0/543611271 learned_addr learned my addr 172.21.15.38:0/543611271 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51f4d6b640 1 --2- 172.21.15.38:0/543611271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b3f390 0x560789edeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51e7fff640 1 --2- 172.21.15.38:0/543611271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b1e6b0 0x560789ee5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51effff640 1 -- 172.21.15.38:0/543611271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b1e6b0 msgr2=0x560789ee5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51effff640 1 --2- 172.21.15.38:0/543611271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b1e6b0 0x560789ee5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51effff640 1 -- 172.21.15.38:0/543611271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b3f390 msgr2=0x560789edeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51effff640 1 --2- 172.21.15.38:0/543611271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b3f390 0x560789edeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.648+0000 7f51effff640 1 -- 172.21.15.38:0/543611271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560789b573f0 con 0x560789b164d0 2024-05-21T15:35:47.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.649+0000 7f51e7fff640 1 --2- 172.21.15.38:0/543611271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b1e6b0 0x560789ee5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:47.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.649+0000 7f51f4d6b640 1 --2- 172.21.15.38:0/543611271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b3f390 0x560789edeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:47.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.649+0000 7f51effff640 1 --2- 172.21.15.38:0/543611271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b164d0 0x560789edf080 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009c90 tx=0x7f51e0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e729b04227427854 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.649+0000 7f51edffb640 1 --2- 172.21.15.38:0/1517963608 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51dc0a2b90 0x7f51dc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.649+0000 7f51f4d6b640 1 --2- 172.21.15.38:0/1517963608 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51dc0a2b90 0x7f51dc0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.649+0000 7f51edffb640 1 -- 172.21.15.38:0/1517963608 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f51d80cf2f0 con 0x7f51f0105ee0 2024-05-21T15:35:47.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.649+0000 7f51e67fc640 1 -- 172.21.15.38:0/543611271 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e001e070 con 0x560789b164d0 2024-05-21T15:35:47.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.649+0000 7f51e67fc640 1 -- 172.21.15.38:0/543611271 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51e0019b10 con 0x560789b164d0 2024-05-21T15:35:47.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.649+0000 7f51e67fc640 1 -- 172.21.15.38:0/543611271 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e000eaf0 con 0x560789b164d0 2024-05-21T15:35:47.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.650+0000 7f51fd676740 1 -- 172.21.15.38:0/543611271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b164d0 msgr2=0x560789edf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.650+0000 7f51f4d6b640 1 --2- 172.21.15.38:0/1517963608 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51dc0a2b90 0x7f51dc0a50a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f51e8005cc0 tx=0x7f51e803d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.650+0000 7f51fd676740 1 --2- 172.21.15.38:0/543611271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b164d0 0x560789edf080 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009c90 tx=0x7f51e0011810 comp rx=0 tx=0).stop 2024-05-21T15:35:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.650+0000 7f51fd676740 1 -- 172.21.15.38:0/543611271 shutdown_connections 2024-05-21T15:35:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.650+0000 7f51fd676740 1 --2- 172.21.15.38:0/543611271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b1e6b0 0x560789ee5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.650+0000 7f51fd676740 1 --2- 172.21.15.38:0/543611271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b164d0 0x560789edf080 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.650+0000 7f51fd676740 1 --2- 172.21.15.38:0/543611271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b3f390 0x560789edeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.650+0000 7f51fd676740 1 -- 172.21.15.38:0/543611271 >> 172.21.15.38:0/543611271 conn(0x560789b334e0 msgr2=0x560789eda490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.650+0000 7f51fd676740 1 -- 172.21.15.38:0/543611271 shutdown_connections 2024-05-21T15:35:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.650+0000 7f51fd676740 1 -- 172.21.15.38:0/543611271 wait complete. 2024-05-21T15:35:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.651+0000 7f51fd676740 1 -- start start 2024-05-21T15:35:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.651+0000 7f51fd676740 1 Processor -- start 2024-05-21T15:35:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.651+0000 7f51fd676740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b164d0 0x560789ee8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.651+0000 7f51fd676740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b1e6b0 0x560789ee8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.651+0000 7f51fd676740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b3f390 0x560789eefe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.651+0000 7f51fd676740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560789ee69d0 con 0x560789b164d0 2024-05-21T15:35:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.651+0000 7f51fd676740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560789dfec40 con 0x560789b1e6b0 2024-05-21T15:35:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.651+0000 7f51fd676740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560789ee6b50 con 0x560789b3f390 2024-05-21T15:35:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.651+0000 7f51effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b164d0 0x560789ee8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.651+0000 7f51effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b164d0 0x560789ee8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49388/0 (socket says 172.21.15.38:49388) 2024-05-21T15:35:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51effff640 1 -- 172.21.15.38:0/3757961815 learned_addr learned my addr 172.21.15.38:0/3757961815 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51fd676740 10 client.0 _ll_register_callbacks cb 0x560789b56be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51f4d6b640 1 --2- 172.21.15.38:0/3757961815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b3f390 0x560789eefe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51e7fff640 1 --2- 172.21.15.38:0/3757961815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b1e6b0 0x560789ee8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51effff640 1 -- 172.21.15.38:0/3757961815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b3f390 msgr2=0x560789eefe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51effff640 1 --2- 172.21.15.38:0/3757961815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b3f390 0x560789eefe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51effff640 1 -- 172.21.15.38:0/3757961815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b1e6b0 msgr2=0x560789ee8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51effff640 1 --2- 172.21.15.38:0/3757961815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b1e6b0 0x560789ee8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51effff640 1 -- 172.21.15.38:0/3757961815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560789c217d0 con 0x560789b164d0 2024-05-21T15:35:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51e7fff640 1 --2- 172.21.15.38:0/3757961815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b1e6b0 0x560789ee8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51effff640 1 --2- 172.21.15.38:0/3757961815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b164d0 0x560789ee8990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009460 tx=0x7f51e0019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef91a74ac8f5c7f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51cf7fe640 1 -- 172.21.15.38:0/3757961815 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e001e040 con 0x560789b164d0 2024-05-21T15:35:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51cf7fe640 1 -- 172.21.15.38:0/3757961815 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f51e0004030 con 0x560789b164d0 2024-05-21T15:35:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.652+0000 7f51fd676740 1 -- 172.21.15.38:0/3757961815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51e0019b10 con 0x560789b164d0 2024-05-21T15:35:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.653+0000 7f51cf7fe640 1 -- 172.21.15.38:0/3757961815 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f51e0004230 con 0x560789b164d0 2024-05-21T15:35:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.653+0000 7f51fd676740 10 client.15795 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.653+0000 7f51fd676740 1 -- 172.21.15.38:0/3757961815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560789ef3040 con 0x560789b164d0 2024-05-21T15:35:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.653+0000 7f51cf7fe640 1 -- 172.21.15.38:0/3757961815 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51e0019b10 con 0x560789b164d0 2024-05-21T15:35:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.653+0000 7f51cf7fe640 1 -- 172.21.15.38:0/3757961815 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f51e000e950 con 0x560789b164d0 2024-05-21T15:35:47.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.653+0000 7f51fd676740 10 client.15795 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:47.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.653+0000 7f51fd676740 10 client.15795 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:35:47.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.653+0000 7f51fd676740 20 client.15795 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:47.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.653+0000 7f51fd676740 1 --2- 172.21.15.38:0/3757961815 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x560789afdf60 0x560789ef6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.653+0000 7f51fd676740 4 client.15795 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:47.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.653+0000 7f51fd676740 1 -- 172.21.15.38:0/3757961815 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560789e51370 con 0x560789afdf60 2024-05-21T15:35:47.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.654+0000 7f51e7fff640 1 --2- 172.21.15.38:0/3757961815 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x560789afdf60 0x560789ef6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.654+0000 7f51e7fff640 1 --2- 172.21.15.38:0/3757961815 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x560789afdf60 0x560789ef6480 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15d4c3d52fb73443 server_cookie=7773ac7d38b5d78d in_seq=0 out_seq=0 2024-05-21T15:35:47.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.654+0000 7f51cf7fe640 10 client.15795 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:35:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51cf7fe640 1 -- 172.21.15.38:0/3757961815 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560789e51370 con 0x560789afdf60 2024-05-21T15:35:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51cf7fe640 10 client.15795 handle_command_reply: tid=0 2024-05-21T15:35:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51fd676740 1 client.15795 shutdown 2024-05-21T15:35:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51fd676740 1 -- 172.21.15.38:0/3757961815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b164d0 msgr2=0x560789ee8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51fd676740 1 --2- 172.21.15.38:0/3757961815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b164d0 0x560789ee8990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009460 tx=0x7f51e0019660 comp rx=0 tx=0).stop 2024-05-21T15:35:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51fd676740 1 -- 172.21.15.38:0/3757961815 shutdown_connections 2024-05-21T15:35:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51fd676740 1 --2- 172.21.15.38:0/3757961815 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x560789afdf60 0x560789ef6480 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51fd676740 1 --2- 172.21.15.38:0/3757961815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b3f390 0x560789eefe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51fd676740 1 --2- 172.21.15.38:0/3757961815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b1e6b0 0x560789ee8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51fd676740 1 --2- 172.21.15.38:0/3757961815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b164d0 0x560789ee8990 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51fd676740 1 -- 172.21.15.38:0/3757961815 >> 172.21.15.38:0/3757961815 conn(0x560789b334e0 msgr2=0x560789ed9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51cf7fe640 0 client.15795 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:35:47.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51fd676740 1 -- 172.21.15.38:0/3757961815 shutdown_connections 2024-05-21T15:35:47.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51fd676740 1 -- 172.21.15.38:0/3757961815 wait complete. 2024-05-21T15:35:47.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.655+0000 7f51fd676740 20 client.15795 trim_cache size 0 max 16384 2024-05-21T15:35:47.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.669+0000 7f51fd676740 1 Processor -- start 2024-05-21T15:35:47.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.669+0000 7f51fd676740 1 -- start start 2024-05-21T15:35:47.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.669+0000 7f51fd676740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b164d0 0x560789ee8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.669+0000 7f51fd676740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b1e6b0 0x560789ee8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51fd676740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b3f390 0x560789edd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51fd676740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f51f01ac6c0 con 0x560789b3f390 2024-05-21T15:35:47.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51fd676740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51f01ae200 con 0x560789b1e6b0 2024-05-21T15:35:47.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51fd676740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f51f01acd90 con 0x560789b164d0 2024-05-21T15:35:47.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51f4d6b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b3f390 0x560789edd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51f4d6b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b3f390 0x560789edd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49390/0 (socket says 172.21.15.38:49390) 2024-05-21T15:35:47.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b1e6b0 0x560789ee8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51f4d6b640 1 -- 172.21.15.38:0/1568671519 learned_addr learned my addr 172.21.15.38:0/1568671519 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51effff640 1 --2- 172.21.15.38:0/1568671519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b164d0 0x560789ee8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51f4d6b640 1 -- 172.21.15.38:0/1568671519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b164d0 msgr2=0x560789ee8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51f4d6b640 1 --2- 172.21.15.38:0/1568671519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b164d0 0x560789ee8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51f4d6b640 1 -- 172.21.15.38:0/1568671519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b1e6b0 msgr2=0x560789ee8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51f4d6b640 1 --2- 172.21.15.38:0/1568671519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b1e6b0 0x560789ee8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51f4d6b640 1 -- 172.21.15.38:0/1568671519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560789eda5d0 con 0x560789b3f390 2024-05-21T15:35:47.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51e7fff640 1 --2- 172.21.15.38:0/1568671519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b1e6b0 0x560789ee8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:47.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.670+0000 7f51f4d6b640 1 --2- 172.21.15.38:0/1568671519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b3f390 0x560789edd330 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f51e8035000 tx=0x7f51e8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369675f445daf178 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.671+0000 7f51e5ffb640 1 -- 172.21.15.38:0/1568671519 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e802fa90 con 0x560789b3f390 2024-05-21T15:35:47.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.671+0000 7f51e5ffb640 1 -- 172.21.15.38:0/1568671519 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51e8056070 con 0x560789b3f390 2024-05-21T15:35:47.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.671+0000 7f51e5ffb640 1 -- 172.21.15.38:0/1568671519 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8046700 con 0x560789b3f390 2024-05-21T15:35:47.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.671+0000 7f51fd676740 1 -- 172.21.15.38:0/1568671519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b3f390 msgr2=0x560789edd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.671+0000 7f51fd676740 1 --2- 172.21.15.38:0/1568671519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b3f390 0x560789edd330 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f51e8035000 tx=0x7f51e8041000 comp rx=0 tx=0).stop 2024-05-21T15:35:47.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.672+0000 7f51fd676740 1 -- 172.21.15.38:0/1568671519 shutdown_connections 2024-05-21T15:35:47.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.672+0000 7f51fd676740 1 --2- 172.21.15.38:0/1568671519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b3f390 0x560789edd330 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.672+0000 7f51fd676740 1 --2- 172.21.15.38:0/1568671519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b1e6b0 0x560789ee8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.672+0000 7f51fd676740 1 --2- 172.21.15.38:0/1568671519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b164d0 0x560789ee8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.672+0000 7f51fd676740 1 -- 172.21.15.38:0/1568671519 >> 172.21.15.38:0/1568671519 conn(0x560789b334e0 msgr2=0x560789ee1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.672+0000 7f51fd676740 1 -- 172.21.15.38:0/1568671519 shutdown_connections 2024-05-21T15:35:47.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.672+0000 7f51fd676740 1 -- 172.21.15.38:0/1568671519 wait complete. 2024-05-21T15:35:47.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.672+0000 7f51fd676740 1 -- start start 2024-05-21T15:35:47.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.672+0000 7f51fd676740 1 Processor -- start 2024-05-21T15:35:47.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.672+0000 7f51fd676740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b164d0 0x560789eeecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51fd676740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b1e6b0 0x560789eef230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51fd676740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b3f390 0x560789eeb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51fd676740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f51f01adb30 con 0x560789b3f390 2024-05-21T15:35:47.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51fd676740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51f01ad460 con 0x560789b164d0 2024-05-21T15:35:47.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51fd676740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f51f01aa6a0 con 0x560789b1e6b0 2024-05-21T15:35:47.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51fd676740 10 client.0 _ll_register_callbacks cb 0x7f51e001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:47.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51f4d6b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b3f390 0x560789eeb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b1e6b0 0x560789eef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51effff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b164d0 0x560789eeecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51f4d6b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b3f390 0x560789eeb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49400/0 (socket says 172.21.15.38:49400) 2024-05-21T15:35:47.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b1e6b0 0x560789eef230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41166/0 (socket says 172.21.15.38:41166) 2024-05-21T15:35:47.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51effff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b164d0 0x560789eeecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:58620/0 (socket says 172.21.15.38:58620) 2024-05-21T15:35:47.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51f4d6b640 1 -- 172.21.15.38:0/1571355217 learned_addr learned my addr 172.21.15.38:0/1571355217 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51f4d6b640 1 -- 172.21.15.38:0/1571355217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b1e6b0 msgr2=0x560789eef230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51f4d6b640 1 --2- 172.21.15.38:0/1571355217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b1e6b0 0x560789eef230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51f4d6b640 1 -- 172.21.15.38:0/1571355217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b164d0 msgr2=0x560789eeecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51f4d6b640 1 --2- 172.21.15.38:0/1571355217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b164d0 0x560789eeecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.673+0000 7f51f4d6b640 1 -- 172.21.15.38:0/1571355217 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560789eebf80 con 0x560789b3f390 2024-05-21T15:35:47.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.674+0000 7f51f4d6b640 1 --2- 172.21.15.38:0/1571355217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b3f390 0x560789eeb910 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009ca0 tx=0x7f51e802ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5618efc5a5e8be80 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.674+0000 7f51cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:47.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.674+0000 7f51cf7fe640 1 -- 172.21.15.38:0/1571355217 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e80309d0 con 0x560789b3f390 2024-05-21T15:35:47.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.674+0000 7f51fd676740 1 -- 172.21.15.38:0/1571355217 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51e8056070 con 0x560789b3f390 2024-05-21T15:35:47.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.674+0000 7f51cf7fe640 1 -- 172.21.15.38:0/1571355217 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f51e8046480 con 0x560789b3f390 2024-05-21T15:35:47.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.674+0000 7f51cf7fe640 1 -- 172.21.15.38:0/1571355217 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f51e8046680 con 0x560789b3f390 2024-05-21T15:35:47.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.674+0000 7f51fd676740 10 client.15801 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:47.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.674+0000 7f51fd676740 1 -- 172.21.15.38:0/1571355217 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560789f5d360 con 0x560789b3f390 2024-05-21T15:35:47.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.674+0000 7f51cf7fe640 1 -- 172.21.15.38:0/1571355217 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51e8056070 con 0x560789b3f390 2024-05-21T15:35:47.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.675+0000 7f51cf7fe640 1 -- 172.21.15.38:0/1571355217 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f51e805a860 con 0x560789b3f390 2024-05-21T15:35:47.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.675+0000 7f51fd676740 10 client.15801 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:47.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.675+0000 7f51fd676740 10 client.15801 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:35:47.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.675+0000 7f51fd676740 20 client.15801 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:47.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.675+0000 7f51fd676740 1 --2- 172.21.15.38:0/1571355217 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x560789ad33a0 0x560789f5fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.675+0000 7f51fd676740 4 client.15801 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:35:47.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.675+0000 7f51fd676740 1 -- 172.21.15.38:0/1571355217 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560789e4b6c0 con 0x560789ad33a0 2024-05-21T15:35:47.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.675+0000 7f51effff640 1 --2- 172.21.15.38:0/1571355217 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x560789ad33a0 0x560789f5fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.675+0000 7f51cf7fe640 10 client.15801 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:35:47.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.675+0000 7f51effff640 1 --2- 172.21.15.38:0/1571355217 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x560789ad33a0 0x560789f5fce0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2d89b467211f8cc8 server_cookie=e2b5628859ddcdfe in_seq=0 out_seq=0 2024-05-21T15:35:47.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51cf7fe640 1 -- 172.21.15.38:0/1571355217 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560789e4b6c0 con 0x560789ad33a0 2024-05-21T15:35:47.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51cf7fe640 10 client.15801 handle_command_reply: tid=0 2024-05-21T15:35:47.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51fd676740 1 client.15801 shutdown 2024-05-21T15:35:47.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51fd676740 1 -- 172.21.15.38:0/1571355217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b3f390 msgr2=0x560789eeb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51fd676740 1 --2- 172.21.15.38:0/1571355217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b3f390 0x560789eeb910 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009ca0 tx=0x7f51e802ff50 comp rx=0 tx=0).stop 2024-05-21T15:35:47.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51fd676740 1 -- 172.21.15.38:0/1571355217 shutdown_connections 2024-05-21T15:35:47.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51fd676740 1 --2- 172.21.15.38:0/1571355217 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x560789ad33a0 0x560789f5fce0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51fd676740 1 --2- 172.21.15.38:0/1571355217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560789b3f390 0x560789eeb910 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51cf7fe640 0 client.15801 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:35:47.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51fd676740 1 --2- 172.21.15.38:0/1571355217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560789b1e6b0 0x560789eef230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51fd676740 1 --2- 172.21.15.38:0/1571355217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560789b164d0 0x560789eeecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51fd676740 1 -- 172.21.15.38:0/1571355217 >> 172.21.15.38:0/1571355217 conn(0x560789b334e0 msgr2=0x560789ed8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51fd676740 1 -- 172.21.15.38:0/1571355217 shutdown_connections 2024-05-21T15:35:47.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51fd676740 1 -- 172.21.15.38:0/1571355217 wait complete. 2024-05-21T15:35:47.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.676+0000 7f51fd676740 20 client.15801 trim_cache size 0 max 16384 2024-05-21T15:35:47.681 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:35:47.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.677+0000 7f51f67ce640 1 -- 172.21.15.38:0/1517963608 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51dc0a2b90 msgr2=0x7f51dc0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.677+0000 7f51f67ce640 1 --2- 172.21.15.38:0/1517963608 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51dc0a2b90 0x7f51dc0a50a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f51e8005cc0 tx=0x7f51e803d040 comp rx=0 tx=0).stop 2024-05-21T15:35:47.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.677+0000 7f51f67ce640 1 -- 172.21.15.38:0/1517963608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0105ee0 msgr2=0x7f51f0101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.677+0000 7f51f67ce640 1 --2- 172.21.15.38:0/1517963608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0105ee0 0x7f51f0101070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f51d800ea60 tx=0x7f51d800ef20 comp rx=0 tx=0).stop 2024-05-21T15:35:47.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.677+0000 7f51f67ce640 1 -- 172.21.15.38:0/1517963608 shutdown_connections 2024-05-21T15:35:47.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.677+0000 7f51f67ce640 1 --2- 172.21.15.38:0/1517963608 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f51dc0a2b90 0x7f51dc0a50a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.677+0000 7f51f67ce640 1 --2- 172.21.15.38:0/1517963608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51f0108840 0x7f51f01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.677+0000 7f51f67ce640 1 --2- 172.21.15.38:0/1517963608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0105ee0 0x7f51f0101070 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.677+0000 7f51f67ce640 1 --2- 172.21.15.38:0/1517963608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51f0103580 0x7f51f01029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.677+0000 7f51f67ce640 1 -- 172.21.15.38:0/1517963608 >> 172.21.15.38:0/1517963608 conn(0x7f51f00fcb60 msgr2=0x7f51f0104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.677+0000 7f51f67ce640 1 -- 172.21.15.38:0/1517963608 shutdown_connections 2024-05-21T15:35:47.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.677+0000 7f51f67ce640 1 -- 172.21.15.38:0/1517963608 wait complete. 2024-05-21T15:35:47.687 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:47 smithi038 ceph-mon[31995]: pgmap v392: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:35:47.687 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:47 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2320266310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:35:47.688 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:35:47.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:47 smithi053 ceph-mon[36496]: pgmap v392: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:35:47.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:47 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2320266310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:35:47.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:47 smithi045 ceph-mon[36409]: pgmap v392: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:35:47.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:47 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2320266310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:35:47.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.810+0000 7f2e304d8640 1 -- 172.21.15.38:0/3092664195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e28102bc0 msgr2=0x7f2e28102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.810+0000 7f2e304d8640 1 --2- 172.21.15.38:0/3092664195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e28102bc0 0x7f2e28102fd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009900 tx=0x7f2e1802a410 comp rx=0 tx=0).stop 2024-05-21T15:35:47.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.810+0000 7f2e304d8640 1 -- 172.21.15.38:0/3092664195 shutdown_connections 2024-05-21T15:35:47.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.810+0000 7f2e304d8640 1 --2- 172.21.15.38:0/3092664195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e281078c0 0x7f2e28109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.810+0000 7f2e304d8640 1 --2- 172.21.15.38:0/3092664195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e28102bc0 0x7f2e28102fd0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.810+0000 7f2e304d8640 1 --2- 172.21.15.38:0/3092664195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e28101980 0x7f2e28102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.810+0000 7f2e304d8640 1 -- 172.21.15.38:0/3092664195 >> 172.21.15.38:0/3092664195 conn(0x7f2e280fcb70 msgr2=0x7f2e280ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.810+0000 7f2e304d8640 1 -- 172.21.15.38:0/3092664195 shutdown_connections 2024-05-21T15:35:47.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.810+0000 7f2e304d8640 1 -- 172.21.15.38:0/3092664195 wait complete. 2024-05-21T15:35:47.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.811+0000 7f2e304d8640 1 Processor -- start 2024-05-21T15:35:47.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.811+0000 7f2e304d8640 1 -- start start 2024-05-21T15:35:47.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.811+0000 7f2e304d8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e28101980 0x7f2e280fea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.811+0000 7f2e304d8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e28102bc0 0x7f2e280fd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.811+0000 7f2e304d8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e281078c0 0x7f2e280fd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.811+0000 7f2e304d8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2810d5d0 con 0x7f2e28102bc0 2024-05-21T15:35:47.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.811+0000 7f2e304d8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2810d450 con 0x7f2e28101980 2024-05-21T15:35:47.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.811+0000 7f2e304d8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2810d750 con 0x7f2e281078c0 2024-05-21T15:35:47.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.811+0000 7f2e2da73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e28102bc0 0x7f2e280fd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.811+0000 7f2e2da73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e28102bc0 0x7f2e280fd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49430/0 (socket says 172.21.15.38:49430) 2024-05-21T15:35:47.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.811+0000 7f2e2da73640 1 -- 172.21.15.38:0/3037256814 learned_addr learned my addr 172.21.15.38:0/3037256814 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.811+0000 7f2e2ea75640 1 --2- 172.21.15.38:0/3037256814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e281078c0 0x7f2e280fd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.811+0000 7f2e2e274640 1 --2- 172.21.15.38:0/3037256814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e28101980 0x7f2e280fea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.812+0000 7f2e2da73640 1 -- 172.21.15.38:0/3037256814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e281078c0 msgr2=0x7f2e280fd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.812+0000 7f2e2da73640 1 --2- 172.21.15.38:0/3037256814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e281078c0 0x7f2e280fd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.812+0000 7f2e2da73640 1 -- 172.21.15.38:0/3037256814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e28101980 msgr2=0x7f2e280fea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.812+0000 7f2e2da73640 1 --2- 172.21.15.38:0/3037256814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e28101980 0x7f2e280fea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.812+0000 7f2e2da73640 1 -- 172.21.15.38:0/3037256814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e280fdd10 con 0x7f2e28102bc0 2024-05-21T15:35:47.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.812+0000 7f2e2da73640 1 --2- 172.21.15.38:0/3037256814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e28102bc0 0x7f2e280fd0a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2e18006010 tx=0x7f2e18004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.812+0000 7f2e1f7fe640 1 -- 172.21.15.38:0/3037256814 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1803f030 con 0x7f2e28102bc0 2024-05-21T15:35:47.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.812+0000 7f2e304d8640 1 -- 172.21.15.38:0/3037256814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e281a3330 con 0x7f2e28102bc0 2024-05-21T15:35:47.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.812+0000 7f2e304d8640 1 -- 172.21.15.38:0/3037256814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e281a3790 con 0x7f2e28102bc0 2024-05-21T15:35:47.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.812+0000 7f2e1f7fe640 1 -- 172.21.15.38:0/3037256814 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e18003ce0 con 0x7f2e28102bc0 2024-05-21T15:35:47.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.812+0000 7f2e1f7fe640 1 -- 172.21.15.38:0/3037256814 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18035070 con 0x7f2e28102bc0 2024-05-21T15:35:47.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.815+0000 7f2e1f7fe640 1 -- 172.21.15.38:0/3037256814 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2e18042030 con 0x7f2e28102bc0 2024-05-21T15:35:47.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.816+0000 7f2e1f7fe640 1 --2- 172.21.15.38:0/3037256814 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2e140a2c10 0x7f2e140a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.816+0000 7f2e1f7fe640 1 -- 172.21.15.38:0/3037256814 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2e180efa60 con 0x7f2e28102bc0 2024-05-21T15:35:47.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.816+0000 7f2e2e274640 1 --2- 172.21.15.38:0/3037256814 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2e140a2c10 0x7f2e140a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.816+0000 7f2e37378740 1 Processor -- start 2024-05-21T15:35:47.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.816+0000 7f2e37378740 1 -- start start 2024-05-21T15:35:47.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e2e274640 1 --2- 172.21.15.38:0/3037256814 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2e140a2c10 0x7f2e140a5120 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2e24005d00 tx=0x7f2e24005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e37378740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9caf8390 0x55aa9ce97b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e37378740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cacf4d0 0x55aa9ce98080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e37378740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cad76b0 0x55aa9ce9ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e37378740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55aa9cdfcc40 con 0x55aa9cacf4d0 2024-05-21T15:35:47.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e37378740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aa9cda4580 con 0x55aa9caf8390 2024-05-21T15:35:47.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e37378740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55aa9cdf8c00 con 0x55aa9cad76b0 2024-05-21T15:35:47.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e2da73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cacf4d0 0x55aa9ce98080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e2e274640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9caf8390 0x55aa9ce97b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e2da73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cacf4d0 0x55aa9ce98080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49434/0 (socket says 172.21.15.38:49434) 2024-05-21T15:35:47.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e2da73640 1 -- 172.21.15.38:0/772693878 learned_addr learned my addr 172.21.15.38:0/772693878 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e2ea75640 1 --2- 172.21.15.38:0/772693878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cad76b0 0x55aa9ce9ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e2da73640 1 -- 172.21.15.38:0/772693878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cad76b0 msgr2=0x55aa9ce9ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e2da73640 1 --2- 172.21.15.38:0/772693878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cad76b0 0x55aa9ce9ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e2da73640 1 -- 172.21.15.38:0/772693878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9caf8390 msgr2=0x55aa9ce97b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e2da73640 1 --2- 172.21.15.38:0/772693878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9caf8390 0x55aa9ce97b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e2da73640 1 -- 172.21.15.38:0/772693878 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa9cb103f0 con 0x55aa9cacf4d0 2024-05-21T15:35:47.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.817+0000 7f2e2e274640 1 --2- 172.21.15.38:0/772693878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9caf8390 0x55aa9ce97b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:47.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.818+0000 7f2e2da73640 1 --2- 172.21.15.38:0/772693878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cacf4d0 0x55aa9ce98080 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2e180bc200 tx=0x7f2e180bbdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2717fa2111eddbf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.818+0000 7f2e1cbfe640 1 -- 172.21.15.38:0/772693878 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18049070 con 0x55aa9cacf4d0 2024-05-21T15:35:47.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.818+0000 7f2e1cbfe640 1 -- 172.21.15.38:0/772693878 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e180bc270 con 0x55aa9cacf4d0 2024-05-21T15:35:47.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.818+0000 7f2e1cbfe640 1 -- 172.21.15.38:0/772693878 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1803b460 con 0x55aa9cacf4d0 2024-05-21T15:35:47.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.819+0000 7f2e37378740 1 -- 172.21.15.38:0/772693878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cacf4d0 msgr2=0x55aa9ce98080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.819+0000 7f2e37378740 1 --2- 172.21.15.38:0/772693878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cacf4d0 0x55aa9ce98080 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2e180bc200 tx=0x7f2e180bbdb0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.819+0000 7f2e37378740 1 -- 172.21.15.38:0/772693878 shutdown_connections 2024-05-21T15:35:47.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.819+0000 7f2e37378740 1 --2- 172.21.15.38:0/772693878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cad76b0 0x55aa9ce9ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.819+0000 7f2e37378740 1 --2- 172.21.15.38:0/772693878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cacf4d0 0x55aa9ce98080 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.819+0000 7f2e37378740 1 --2- 172.21.15.38:0/772693878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9caf8390 0x55aa9ce97b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.820+0000 7f2e37378740 1 -- 172.21.15.38:0/772693878 >> 172.21.15.38:0/772693878 conn(0x55aa9caec4e0 msgr2=0x55aa9ce93490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.820+0000 7f2e37378740 1 -- 172.21.15.38:0/772693878 shutdown_connections 2024-05-21T15:35:47.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.820+0000 7f2e37378740 1 -- 172.21.15.38:0/772693878 wait complete. 2024-05-21T15:35:47.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.820+0000 7f2e37378740 1 -- start start 2024-05-21T15:35:47.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.820+0000 7f2e37378740 1 Processor -- start 2024-05-21T15:35:47.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e37378740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cacf4d0 0x55aa9cea1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e37378740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cad76b0 0x55aa9cea1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e37378740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9caf8390 0x55aa9cea8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e37378740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55aa9ce9f790 con 0x55aa9cad76b0 2024-05-21T15:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e37378740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aa9cdb7c40 con 0x55aa9caf8390 2024-05-21T15:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e37378740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55aa9ce9f910 con 0x55aa9cacf4d0 2024-05-21T15:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e2e274640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cacf4d0 0x55aa9cea1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e2e274640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cacf4d0 0x55aa9cea1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41196/0 (socket says 172.21.15.38:41196) 2024-05-21T15:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e2e274640 1 -- 172.21.15.38:0/2958166736 learned_addr learned my addr 172.21.15.38:0/2958166736 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e2da73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cad76b0 0x55aa9cea1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e2ea75640 1 --2- 172.21.15.38:0/2958166736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9caf8390 0x55aa9cea8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e2e274640 1 -- 172.21.15.38:0/2958166736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9caf8390 msgr2=0x55aa9cea8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e2e274640 1 --2- 172.21.15.38:0/2958166736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9caf8390 0x55aa9cea8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e2e274640 1 -- 172.21.15.38:0/2958166736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cad76b0 msgr2=0x55aa9cea1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e2e274640 1 --2- 172.21.15.38:0/2958166736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cad76b0 0x55aa9cea1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e37378740 10 client.0 _ll_register_callbacks cb 0x55aa9cb0fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e2e274640 1 -- 172.21.15.38:0/2958166736 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa9cbda7d0 con 0x55aa9cacf4d0 2024-05-21T15:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e37378740 1 -- 172.21.15.38:0/2958166736 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e180bc270 con 0x55aa9cacf4d0 2024-05-21T15:35:47.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e2da73640 1 --2- 172.21.15.38:0/2958166736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cad76b0 0x55aa9cea1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:47.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e2e274640 1 --2- 172.21.15.38:0/2958166736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cacf4d0 0x55aa9cea1630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2e240116c0 tx=0x7f2e240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98aaa485f38688e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.821+0000 7f2e11ffb640 10 client.25549 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:35:47.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.822+0000 7f2e11ffb640 1 -- 172.21.15.38:0/2958166736 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24023070 con 0x55aa9cacf4d0 2024-05-21T15:35:47.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.822+0000 7f2e11ffb640 1 -- 172.21.15.38:0/2958166736 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2e24010960 con 0x55aa9cacf4d0 2024-05-21T15:35:47.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.822+0000 7f2e11ffb640 1 -- 172.21.15.38:0/2958166736 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e24010b60 con 0x55aa9cacf4d0 2024-05-21T15:35:47.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.822+0000 7f2e37378740 10 client.25549 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:47.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.822+0000 7f2e37378740 1 -- 172.21.15.38:0/2958166736 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55aa9ceabd90 con 0x55aa9cacf4d0 2024-05-21T15:35:47.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.822+0000 7f2e11ffb640 1 -- 172.21.15.38:0/2958166736 <== mon.1 v2:172.21.15.53:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2e2402d020 con 0x55aa9cacf4d0 2024-05-21T15:35:47.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.822+0000 7f2e37378740 10 client.25549 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:47.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.822+0000 7f2e37378740 10 client.25549 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:35:47.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.822+0000 7f2e37378740 20 client.25549 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:47.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.823+0000 7f2e37378740 1 --2- 172.21.15.38:0/2958166736 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55aa9cab6f60 0x55aa9ceaf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.823+0000 7f2e37378740 4 client.25549 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:47.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.823+0000 7f2e37378740 1 -- 172.21.15.38:0/2958166736 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa9ce0a370 con 0x55aa9cab6f60 2024-05-21T15:35:47.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.823+0000 7f2e2da73640 1 --2- 172.21.15.38:0/2958166736 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55aa9cab6f60 0x55aa9ceaf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.823+0000 7f2e2da73640 1 --2- 172.21.15.38:0/2958166736 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55aa9cab6f60 0x55aa9ceaf3f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=527126cbbb90f622 server_cookie=7a7cb02eb634c079 in_seq=0 out_seq=0 2024-05-21T15:35:47.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.823+0000 7f2e11ffb640 10 client.25549 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:35:47.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e11ffb640 1 -- 172.21.15.38:0/2958166736 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa9ce0a370 con 0x55aa9cab6f60 2024-05-21T15:35:47.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e11ffb640 10 client.25549 handle_command_reply: tid=0 2024-05-21T15:35:47.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e37378740 1 client.25549 shutdown 2024-05-21T15:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e37378740 1 -- 172.21.15.38:0/2958166736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cacf4d0 msgr2=0x55aa9cea1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e37378740 1 --2- 172.21.15.38:0/2958166736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cacf4d0 0x55aa9cea1630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2e240116c0 tx=0x7f2e240046c0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e37378740 1 -- 172.21.15.38:0/2958166736 shutdown_connections 2024-05-21T15:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e37378740 1 --2- 172.21.15.38:0/2958166736 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55aa9cab6f60 0x55aa9ceaf3f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e11ffb640 0 client.25549 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e37378740 1 --2- 172.21.15.38:0/2958166736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9caf8390 0x55aa9cea8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e37378740 1 --2- 172.21.15.38:0/2958166736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cad76b0 0x55aa9cea1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e37378740 1 --2- 172.21.15.38:0/2958166736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cacf4d0 0x55aa9cea1630 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e37378740 1 -- 172.21.15.38:0/2958166736 >> 172.21.15.38:0/2958166736 conn(0x55aa9caec4e0 msgr2=0x55aa9ce92570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e37378740 1 -- 172.21.15.38:0/2958166736 shutdown_connections 2024-05-21T15:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e37378740 1 -- 172.21.15.38:0/2958166736 wait complete. 2024-05-21T15:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.824+0000 7f2e37378740 20 client.25549 trim_cache size 0 max 16384 2024-05-21T15:35:47.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.851+0000 7f2e37378740 1 Processor -- start 2024-05-21T15:35:47.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.852+0000 7f2e37378740 1 -- start start 2024-05-21T15:35:47.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.852+0000 7f2e37378740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cab6f60 0x55aa9cea1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.852+0000 7f2e37378740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cacf4d0 0x55aa9ce9c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.852+0000 7f2e2e274640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cab6f60 0x55aa9cea1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.852+0000 7f2e2e274640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cab6f60 0x55aa9cea1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41206/0 (socket says 172.21.15.38:41206) 2024-05-21T15:35:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.852+0000 7f2e2e274640 1 -- 172.21.15.38:0/2471107623 learned_addr learned my addr 172.21.15.38:0/2471107623 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.852+0000 7f2e37378740 1 --2- 172.21.15.38:0/2471107623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9cad76b0 0x55aa9cea8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.852+0000 7f2e37378740 1 -- 172.21.15.38:0/2471107623 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2e281a8190 con 0x55aa9cacf4d0 2024-05-21T15:35:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.852+0000 7f2e37378740 1 -- 172.21.15.38:0/2471107623 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e281a9cd0 con 0x55aa9cad76b0 2024-05-21T15:35:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.852+0000 7f2e37378740 1 -- 172.21.15.38:0/2471107623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2e281a8860 con 0x55aa9cab6f60 2024-05-21T15:35:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.852+0000 7f2e2da73640 1 --2- 172.21.15.38:0/2471107623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cacf4d0 0x55aa9ce9c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.853+0000 7f2e2e274640 1 -- 172.21.15.38:0/2471107623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9cad76b0 msgr2=0x55aa9cea8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.853+0000 7f2e2e274640 1 --2- 172.21.15.38:0/2471107623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9cad76b0 0x55aa9cea8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.853+0000 7f2e2e274640 1 -- 172.21.15.38:0/2471107623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cacf4d0 msgr2=0x55aa9ce9c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.853+0000 7f2e2e274640 1 --2- 172.21.15.38:0/2471107623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cacf4d0 0x55aa9ce9c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.853+0000 7f2e2e274640 1 -- 172.21.15.38:0/2471107623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa9ce8ed20 con 0x55aa9cab6f60 2024-05-21T15:35:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.853+0000 7f2e2da73640 1 --2- 172.21.15.38:0/2471107623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cacf4d0 0x55aa9ce9c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.853+0000 7f2e2e274640 1 --2- 172.21.15.38:0/2471107623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cab6f60 0x55aa9cea1c30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2e24011fd0 tx=0x7f2e24020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c1423586be9e621 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.853+0000 7f2e13fff640 1 -- 172.21.15.38:0/2471107623 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2403d070 con 0x55aa9cab6f60 2024-05-21T15:35:47.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.853+0000 7f2e13fff640 1 -- 172.21.15.38:0/2471107623 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e180bc270 con 0x55aa9cab6f60 2024-05-21T15:35:47.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.853+0000 7f2e13fff640 1 -- 172.21.15.38:0/2471107623 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24023070 con 0x55aa9cab6f60 2024-05-21T15:35:47.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.855+0000 7f2e37378740 1 -- 172.21.15.38:0/2471107623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cab6f60 msgr2=0x55aa9cea1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.855+0000 7f2e37378740 1 --2- 172.21.15.38:0/2471107623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cab6f60 0x55aa9cea1c30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2e24011fd0 tx=0x7f2e24020000 comp rx=0 tx=0).stop 2024-05-21T15:35:47.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.855+0000 7f2e37378740 1 -- 172.21.15.38:0/2471107623 shutdown_connections 2024-05-21T15:35:47.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.855+0000 7f2e37378740 1 --2- 172.21.15.38:0/2471107623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9cad76b0 0x55aa9cea8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.855+0000 7f2e37378740 1 --2- 172.21.15.38:0/2471107623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cacf4d0 0x55aa9ce9c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.855+0000 7f2e37378740 1 --2- 172.21.15.38:0/2471107623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cab6f60 0x55aa9cea1c30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.855+0000 7f2e37378740 1 -- 172.21.15.38:0/2471107623 >> 172.21.15.38:0/2471107623 conn(0x55aa9caec4e0 msgr2=0x55aa9ce91b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.855+0000 7f2e37378740 1 -- 172.21.15.38:0/2471107623 shutdown_connections 2024-05-21T15:35:47.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.855+0000 7f2e37378740 1 -- 172.21.15.38:0/2471107623 wait complete. 2024-05-21T15:35:47.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.855+0000 7f2e37378740 1 -- start start 2024-05-21T15:35:47.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.855+0000 7f2e37378740 1 Processor -- start 2024-05-21T15:35:47.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e37378740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cab6f60 0x55aa9cea5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e37378740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9cacf4d0 0x55aa9cea62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e37378740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cad76b0 0x55aa9cea71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e37378740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2e281a9600 con 0x55aa9cad76b0 2024-05-21T15:35:47.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e37378740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e281a8f30 con 0x55aa9cacf4d0 2024-05-21T15:35:47.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e37378740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2e281a6170 con 0x55aa9cab6f60 2024-05-21T15:35:47.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e37378740 10 client.0 _ll_register_callbacks cb 0x7f2e2400be20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:47.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e2ea75640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cad76b0 0x55aa9cea71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e2ea75640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cad76b0 0x55aa9cea71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49464/0 (socket says 172.21.15.38:49464) 2024-05-21T15:35:47.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e2ea75640 1 -- 172.21.15.38:0/2361081999 learned_addr learned my addr 172.21.15.38:0/2361081999 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:47.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e2e274640 1 --2- 172.21.15.38:0/2361081999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cab6f60 0x55aa9cea5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e2da73640 1 --2- 172.21.15.38:0/2361081999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9cacf4d0 0x55aa9cea62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e2ea75640 1 -- 172.21.15.38:0/2361081999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cab6f60 msgr2=0x55aa9cea5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e2ea75640 1 --2- 172.21.15.38:0/2361081999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cab6f60 0x55aa9cea5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e2ea75640 1 -- 172.21.15.38:0/2361081999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9cacf4d0 msgr2=0x55aa9cea62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:47.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e2ea75640 1 --2- 172.21.15.38:0/2361081999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9cacf4d0 0x55aa9cea62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e2ea75640 1 -- 172.21.15.38:0/2361081999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa9cea7790 con 0x55aa9cad76b0 2024-05-21T15:35:47.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.856+0000 7f2e2e274640 1 --2- 172.21.15.38:0/2361081999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cab6f60 0x55aa9cea5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:47.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.857+0000 7f2e2ea75640 1 --2- 172.21.15.38:0/2361081999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cad76b0 0x55aa9cea71d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009f60 tx=0x7f2e20009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba1bd7a40b9e6b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:47.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.857+0000 7f2e11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:47.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.857+0000 7f2e11ffb640 1 -- 172.21.15.38:0/2361081999 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20003d80 con 0x55aa9cad76b0 2024-05-21T15:35:47.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.857+0000 7f2e11ffb640 1 -- 172.21.15.38:0/2361081999 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2e20004050 con 0x55aa9cad76b0 2024-05-21T15:35:47.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.857+0000 7f2e37378740 1 -- 172.21.15.38:0/2361081999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e180bc270 con 0x55aa9cad76b0 2024-05-21T15:35:47.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.857+0000 7f2e11ffb640 1 -- 172.21.15.38:0/2361081999 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e20004250 con 0x55aa9cad76b0 2024-05-21T15:35:47.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.857+0000 7f2e37378740 10 client.15819 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:47.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.857+0000 7f2e37378740 1 -- 172.21.15.38:0/2361081999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa9cea8ae0 con 0x55aa9cad76b0 2024-05-21T15:35:47.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.857+0000 7f2e11ffb640 1 -- 172.21.15.38:0/2361081999 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e180bc270 con 0x55aa9cad76b0 2024-05-21T15:35:47.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.858+0000 7f2e11ffb640 1 -- 172.21.15.38:0/2361081999 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2e20019020 con 0x55aa9cad76b0 2024-05-21T15:35:47.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.858+0000 7f2e37378740 10 client.15819 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:47.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.858+0000 7f2e37378740 10 client.15819 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:35:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.858+0000 7f2e37378740 20 client.15819 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.858+0000 7f2e37378740 1 --2- 172.21.15.38:0/2361081999 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55aa9caf8390 0x55aa9cf2acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.858+0000 7f2e37378740 4 client.15819 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:35:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.858+0000 7f2e37378740 1 -- 172.21.15.38:0/2361081999 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa9ce046c0 con 0x55aa9caf8390 2024-05-21T15:35:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.858+0000 7f2e2e274640 1 --2- 172.21.15.38:0/2361081999 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55aa9caf8390 0x55aa9cf2acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.858+0000 7f2e2e274640 1 --2- 172.21.15.38:0/2361081999 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55aa9caf8390 0x55aa9cf2acb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e41f05bcd0eada1e server_cookie=3a0263c4fdbf64bf in_seq=0 out_seq=0 2024-05-21T15:35:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.858+0000 7f2e11ffb640 10 client.15819 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:35:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e11ffb640 1 -- 172.21.15.38:0/2361081999 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa9ce046c0 con 0x55aa9caf8390 2024-05-21T15:35:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e11ffb640 10 client.15819 handle_command_reply: tid=0 2024-05-21T15:35:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e37378740 1 client.15819 shutdown 2024-05-21T15:35:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e37378740 1 -- 172.21.15.38:0/2361081999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cad76b0 msgr2=0x55aa9cea71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e37378740 1 --2- 172.21.15.38:0/2361081999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cad76b0 0x55aa9cea71d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009f60 tx=0x7f2e20009f90 comp rx=0 tx=0).stop 2024-05-21T15:35:47.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e37378740 1 -- 172.21.15.38:0/2361081999 shutdown_connections 2024-05-21T15:35:47.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e37378740 1 --2- 172.21.15.38:0/2361081999 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55aa9caf8390 0x55aa9cf2acb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e37378740 1 --2- 172.21.15.38:0/2361081999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55aa9cad76b0 0x55aa9cea71d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e37378740 1 --2- 172.21.15.38:0/2361081999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa9cacf4d0 0x55aa9cea62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e37378740 1 --2- 172.21.15.38:0/2361081999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55aa9cab6f60 0x55aa9cea5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e37378740 1 -- 172.21.15.38:0/2361081999 >> 172.21.15.38:0/2361081999 conn(0x55aa9caec4e0 msgr2=0x55aa9ce92570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e11ffb640 0 client.15819 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:35:47.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e37378740 1 -- 172.21.15.38:0/2361081999 shutdown_connections 2024-05-21T15:35:47.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e37378740 1 -- 172.21.15.38:0/2361081999 wait complete. 2024-05-21T15:35:47.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.859+0000 7f2e37378740 20 client.15819 trim_cache size 0 max 16384 2024-05-21T15:35:47.861 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:35:47.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.860+0000 7f2e304d8640 1 -- 172.21.15.38:0/3037256814 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2e140a2c10 msgr2=0x7f2e140a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.860+0000 7f2e304d8640 1 --2- 172.21.15.38:0/3037256814 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2e140a2c10 0x7f2e140a5120 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2e24005d00 tx=0x7f2e24005c50 comp rx=0 tx=0).stop 2024-05-21T15:35:47.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.860+0000 7f2e304d8640 1 -- 172.21.15.38:0/3037256814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e28102bc0 msgr2=0x7f2e280fd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:47.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.860+0000 7f2e304d8640 1 --2- 172.21.15.38:0/3037256814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e28102bc0 0x7f2e280fd0a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2e18006010 tx=0x7f2e18004360 comp rx=0 tx=0).stop 2024-05-21T15:35:47.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.860+0000 7f2e304d8640 1 -- 172.21.15.38:0/3037256814 shutdown_connections 2024-05-21T15:35:47.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.860+0000 7f2e304d8640 1 --2- 172.21.15.38:0/3037256814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e281078c0 0x7f2e280fd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.860+0000 7f2e304d8640 1 --2- 172.21.15.38:0/3037256814 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2e140a2c10 0x7f2e140a5120 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.860+0000 7f2e304d8640 1 --2- 172.21.15.38:0/3037256814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e28102bc0 0x7f2e280fd0a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.860+0000 7f2e304d8640 1 --2- 172.21.15.38:0/3037256814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e28101980 0x7f2e280fea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:47.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.860+0000 7f2e304d8640 1 -- 172.21.15.38:0/3037256814 >> 172.21.15.38:0/3037256814 conn(0x7f2e280fcb70 msgr2=0x7f2e28108d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:47.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.860+0000 7f2e304d8640 1 -- 172.21.15.38:0/3037256814 shutdown_connections 2024-05-21T15:35:47.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:47.860+0000 7f2e304d8640 1 -- 172.21.15.38:0/3037256814 wait complete. 2024-05-21T15:35:47.873 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:35:48.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.015+0000 7fcca66c3640 1 -- 172.21.15.38:0/680563143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca0102bc0 msgr2=0x7fcca0102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.015+0000 7fcca66c3640 1 --2- 172.21.15.38:0/680563143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca0102bc0 0x7fcca0102fd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009900 tx=0x7fcc9402a410 comp rx=0 tx=0).stop 2024-05-21T15:35:48.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.015+0000 7fcca66c3640 1 -- 172.21.15.38:0/680563143 shutdown_connections 2024-05-21T15:35:48.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.015+0000 7fcca66c3640 1 --2- 172.21.15.38:0/680563143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca01078c0 0x7fcca0109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.015+0000 7fcca66c3640 1 --2- 172.21.15.38:0/680563143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca0102bc0 0x7fcca0102fd0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.015+0000 7fcca66c3640 1 --2- 172.21.15.38:0/680563143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcca0101980 0x7fcca0102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.015+0000 7fcca66c3640 1 -- 172.21.15.38:0/680563143 >> 172.21.15.38:0/680563143 conn(0x7fcca00fcb70 msgr2=0x7fcca00ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:48.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.015+0000 7fcca66c3640 1 -- 172.21.15.38:0/680563143 shutdown_connections 2024-05-21T15:35:48.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.015+0000 7fcca66c3640 1 -- 172.21.15.38:0/680563143 wait complete. 2024-05-21T15:35:48.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.015+0000 7fcca66c3640 1 Processor -- start 2024-05-21T15:35:48.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.016+0000 7fcca66c3640 1 -- start start 2024-05-21T15:35:48.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.016+0000 7fcca66c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca0101980 0x7fcca0199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.016+0000 7fcca66c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca0102bc0 0x7fcca0199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.016+0000 7fcca66c3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcca01078c0 0x7fcca01a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.016+0000 7fcca66c3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcca010d5d0 con 0x7fcca0102bc0 2024-05-21T15:35:48.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.016+0000 7fcca66c3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcca010d450 con 0x7fcca0101980 2024-05-21T15:35:48.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.016+0000 7fcca66c3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcca010d750 con 0x7fcca01078c0 2024-05-21T15:35:48.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.016+0000 7fcc9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca0102bc0 0x7fcca0199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.016+0000 7fcca4c60640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcca01078c0 0x7fcca01a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.016+0000 7fcc9ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca0101980 0x7fcca0199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.016+0000 7fcc9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca0102bc0 0x7fcca0199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49484/0 (socket says 172.21.15.38:49484) 2024-05-21T15:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.016+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/2448413913 learned_addr learned my addr 172.21.15.38:0/2448413913 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.016+0000 7fcca4c60640 1 --2- 172.21.15.38:0/2448413913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcca01078c0 0x7fcca01a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41220/0 (socket says 172.21.15.38:41220) 2024-05-21T15:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.017+0000 7fcca4c60640 1 -- 172.21.15.38:0/2448413913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca0101980 msgr2=0x7fcca0199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.017+0000 7fcca4c60640 1 --2- 172.21.15.38:0/2448413913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca0101980 0x7fcca0199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.017+0000 7fcca4c60640 1 -- 172.21.15.38:0/2448413913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca0102bc0 msgr2=0x7fcca0199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.017+0000 7fcca4c60640 1 --2- 172.21.15.38:0/2448413913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca0102bc0 0x7fcca0199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.017+0000 7fcca4c60640 1 -- 172.21.15.38:0/2448413913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca01a0d80 con 0x7fcca01078c0 2024-05-21T15:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.017+0000 7fcca4c60640 1 --2- 172.21.15.38:0/2448413913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcca01078c0 0x7fcca01a0710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800e940 tx=0x7fcc9800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.017+0000 7fcc9d7fa640 1 -- 172.21.15.38:0/2448413913 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98004d20 con 0x7fcca01078c0 2024-05-21T15:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.017+0000 7fcca66c3640 1 -- 172.21.15.38:0/2448413913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca01a1070 con 0x7fcca01078c0 2024-05-21T15:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.017+0000 7fcc9d7fa640 1 -- 172.21.15.38:0/2448413913 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc98005020 con 0x7fcca01078c0 2024-05-21T15:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.017+0000 7fcca66c3640 1 -- 172.21.15.38:0/2448413913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca01a15c0 con 0x7fcca01078c0 2024-05-21T15:35:48.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.017+0000 7fcc9d7fa640 1 -- 172.21.15.38:0/2448413913 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc980116b0 con 0x7fcca01078c0 2024-05-21T15:35:48.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.020+0000 7fcc9d7fa640 1 -- 172.21.15.38:0/2448413913 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcc98011850 con 0x7fcca01078c0 2024-05-21T15:35:48.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.020+0000 7fccad566740 1 Processor -- start 2024-05-21T15:35:48.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.020+0000 7fccad566740 1 -- start start 2024-05-21T15:35:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.021+0000 7fccad566740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda91390 0x5640fde34f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.021+0000 7fccad566740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde35490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.021+0000 7fccad566740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda706b0 0x5640fde3c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.021+0000 7fccad566740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5640fdd95c40 con 0x5640fda91390 2024-05-21T15:35:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.021+0000 7fccad566740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5640fdd3d580 con 0x5640fda706b0 2024-05-21T15:35:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.021+0000 7fccad566740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5640fdd91c00 con 0x5640fda684d0 2024-05-21T15:35:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.021+0000 7fcc9d7fa640 1 --2- 172.21.15.38:0/2448413913 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcc880a2c10 0x7fcc880a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.021+0000 7fcc9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda91390 0x5640fde34f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.021+0000 7fcc9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda91390 0x5640fde34f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49492/0 (socket says 172.21.15.38:49492) 2024-05-21T15:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.021+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/3434949689 learned_addr learned my addr 172.21.15.38:0/3434949689 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.021+0000 7fcc9d7fa640 1 -- 172.21.15.38:0/2448413913 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fcc980cefc0 con 0x7fcca01078c0 2024-05-21T15:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.022+0000 7fcc9ffff640 1 --2- 172.21.15.38:0/3434949689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde35490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.022+0000 7fcca4c60640 1 --2- 172.21.15.38:0/2448413913 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcc880a2c10 0x7fcc880a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.022+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/3434949689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 msgr2=0x5640fde35490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.022+0000 7fcc9f7fe640 1 --2- 172.21.15.38:0/3434949689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde35490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.022+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/3434949689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda706b0 msgr2=0x5640fde3c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.022+0000 7fcc9f7fe640 1 --2- 172.21.15.38:0/3434949689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda706b0 0x5640fde3c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.022+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/3434949689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640fdaa93f0 con 0x5640fda91390 2024-05-21T15:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.022+0000 7fcc9f7fe640 1 --2- 172.21.15.38:0/3434949689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda91390 0x5640fde34f10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcc94000c00 tx=0x7fcc94032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2414380f468889a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.022+0000 7fcca4c60640 1 --2- 172.21.15.38:0/2448413913 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcc880a2c10 0x7fcc880a5120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcc98098250 tx=0x7fcc9800ef30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.022+0000 7fcc867fc640 1 -- 172.21.15.38:0/3434949689 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94040070 con 0x5640fda91390 2024-05-21T15:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.022+0000 7fcc867fc640 1 -- 172.21.15.38:0/3434949689 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc940318b0 con 0x5640fda91390 2024-05-21T15:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.023+0000 7fcc867fc640 1 -- 172.21.15.38:0/3434949689 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94035040 con 0x5640fda91390 2024-05-21T15:35:48.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.023+0000 7fccad566740 1 -- 172.21.15.38:0/3434949689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda91390 msgr2=0x5640fde34f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.023+0000 7fccad566740 1 --2- 172.21.15.38:0/3434949689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda91390 0x5640fde34f10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcc94000c00 tx=0x7fcc94032d00 comp rx=0 tx=0).stop 2024-05-21T15:35:48.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.024+0000 7fccad566740 1 -- 172.21.15.38:0/3434949689 shutdown_connections 2024-05-21T15:35:48.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.024+0000 7fccad566740 1 --2- 172.21.15.38:0/3434949689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda706b0 0x5640fde3c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.024+0000 7fccad566740 1 --2- 172.21.15.38:0/3434949689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde35490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.024+0000 7fccad566740 1 --2- 172.21.15.38:0/3434949689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda91390 0x5640fde34f10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.024+0000 7fccad566740 1 -- 172.21.15.38:0/3434949689 >> 172.21.15.38:0/3434949689 conn(0x5640fda854e0 msgr2=0x5640fde2c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:48.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.024+0000 7fccad566740 1 -- 172.21.15.38:0/3434949689 shutdown_connections 2024-05-21T15:35:48.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.024+0000 7fccad566740 1 -- 172.21.15.38:0/3434949689 wait complete. 2024-05-21T15:35:48.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.024+0000 7fccad566740 1 -- start start 2024-05-21T15:35:48.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.024+0000 7fccad566740 1 Processor -- start 2024-05-21T15:35:48.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fccad566740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde3ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fccad566740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda706b0 0x5640fde3f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fccad566740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda91390 0x5640fde461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fccad566740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5640fde3cd80 con 0x5640fda706b0 2024-05-21T15:35:48.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fccad566740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5640fdd50c40 con 0x5640fda91390 2024-05-21T15:35:48.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fccad566740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5640fde3cf00 con 0x5640fda684d0 2024-05-21T15:35:48.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fccad566740 10 client.0 _ll_register_callbacks cb 0x5640fdaa8be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:48.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fcc9f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde3ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fcc9ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda706b0 0x5640fde3f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fcc9f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde3ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41228/0 (socket says 172.21.15.38:41228) 2024-05-21T15:35:48.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/2558376582 learned_addr learned my addr 172.21.15.38:0/2558376582 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:48.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fcc9f7fe640 1 --2- 172.21.15.38:0/2558376582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda91390 0x5640fde461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/2558376582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda91390 msgr2=0x5640fde461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fcc9f7fe640 1 --2- 172.21.15.38:0/2558376582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda91390 0x5640fde461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/2558376582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda706b0 msgr2=0x5640fde3f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fcc9f7fe640 1 --2- 172.21.15.38:0/2558376582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda706b0 0x5640fde3f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/2558376582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640fdb737d0 con 0x5640fda684d0 2024-05-21T15:35:48.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fcc9f7fe640 1 --2- 172.21.15.38:0/2558376582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde3ed30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009a20 tx=0x7fcc9402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9747bd8af0bd51e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:48.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.025+0000 7fcc7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:35:48.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.026+0000 7fcc7f7fe640 1 -- 172.21.15.38:0/2558376582 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94040040 con 0x5640fda684d0 2024-05-21T15:35:48.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.026+0000 7fccad566740 1 -- 172.21.15.38:0/2558376582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc940318b0 con 0x5640fda684d0 2024-05-21T15:35:48.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.026+0000 7fcc7f7fe640 1 -- 172.21.15.38:0/2558376582 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fcc94004930 con 0x5640fda684d0 2024-05-21T15:35:48.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.026+0000 7fcc7f7fe640 1 -- 172.21.15.38:0/2558376582 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc94004b30 con 0x5640fda684d0 2024-05-21T15:35:48.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.026+0000 7fccad566740 10 client.25567 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:48.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.026+0000 7fccad566740 1 -- 172.21.15.38:0/2558376582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5640fde3cdb0 con 0x5640fda684d0 2024-05-21T15:35:48.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.026+0000 7fcc7f7fe640 1 -- 172.21.15.38:0/2558376582 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc940318b0 con 0x5640fda684d0 2024-05-21T15:35:48.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.026+0000 7fcc7f7fe640 1 -- 172.21.15.38:0/2558376582 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcc9403c440 con 0x5640fda684d0 2024-05-21T15:35:48.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.027+0000 7fccad566740 10 client.25567 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:48.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.027+0000 7fccad566740 10 client.25567 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:35:48.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.027+0000 7fccad566740 20 client.25567 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:48.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.027+0000 7fccad566740 1 --2- 172.21.15.38:0/2558376582 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x5640fda4ff60 0x5640fde4c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.027+0000 7fccad566740 4 client.25567 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:48.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.027+0000 7fccad566740 1 -- 172.21.15.38:0/2558376582 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640fdda3370 con 0x5640fda4ff60 2024-05-21T15:35:48.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.027+0000 7fcc9ffff640 1 --2- 172.21.15.38:0/2558376582 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x5640fda4ff60 0x5640fde4c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.027+0000 7fcc7f7fe640 10 client.25567 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:35:48.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.027+0000 7fcc9ffff640 1 --2- 172.21.15.38:0/2558376582 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x5640fda4ff60 0x5640fde4c6d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=773d0588317a3583 server_cookie=43cd491825bfb515 in_seq=0 out_seq=0 2024-05-21T15:35:48.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.028+0000 7fcc7f7fe640 1 -- 172.21.15.38:0/2558376582 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5640fdda3370 con 0x5640fda4ff60 2024-05-21T15:35:48.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.028+0000 7fcc7f7fe640 10 client.25567 handle_command_reply: tid=0 2024-05-21T15:35:48.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.028+0000 7fccad566740 1 client.25567 shutdown 2024-05-21T15:35:48.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.028+0000 7fccad566740 1 -- 172.21.15.38:0/2558376582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 msgr2=0x5640fde3ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.028+0000 7fccad566740 1 --2- 172.21.15.38:0/2558376582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde3ed30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009a20 tx=0x7fcc9402a410 comp rx=0 tx=0).stop 2024-05-21T15:35:48.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.028+0000 7fccad566740 1 -- 172.21.15.38:0/2558376582 shutdown_connections 2024-05-21T15:35:48.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.028+0000 7fccad566740 1 --2- 172.21.15.38:0/2558376582 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x5640fda4ff60 0x5640fde4c6d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.029+0000 7fcc7f7fe640 0 client.25567 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:35:48.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.029+0000 7fccad566740 1 --2- 172.21.15.38:0/2558376582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda91390 0x5640fde461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.029+0000 7fccad566740 1 --2- 172.21.15.38:0/2558376582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda706b0 0x5640fde3f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.029+0000 7fccad566740 1 --2- 172.21.15.38:0/2558376582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde3ed30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.029+0000 7fccad566740 1 -- 172.21.15.38:0/2558376582 >> 172.21.15.38:0/2558376582 conn(0x5640fda854e0 msgr2=0x5640fde2b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:48.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.029+0000 7fccad566740 1 -- 172.21.15.38:0/2558376582 shutdown_connections 2024-05-21T15:35:48.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.029+0000 7fccad566740 1 -- 172.21.15.38:0/2558376582 wait complete. 2024-05-21T15:35:48.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.029+0000 7fccad566740 20 client.25567 trim_cache size 0 max 16384 2024-05-21T15:35:48.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.043+0000 7fccad566740 1 Processor -- start 2024-05-21T15:35:48.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.043+0000 7fccad566740 1 -- start start 2024-05-21T15:35:48.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.043+0000 7fccad566740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda4ff60 0x5640fde2b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.043+0000 7fccad566740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde46160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.043+0000 7fccad566740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda706b0 0x5640fde50e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.043+0000 7fccad566740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c00e5b0 con 0x5640fda4ff60 2024-05-21T15:35:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.043+0000 7fccad566740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c0100f0 con 0x5640fda706b0 2024-05-21T15:35:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.043+0000 7fccad566740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c00ec80 con 0x5640fda684d0 2024-05-21T15:35:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.043+0000 7fcc9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda4ff60 0x5640fde2b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.043+0000 7fcc9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda4ff60 0x5640fde2b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49512/0 (socket says 172.21.15.38:49512) 2024-05-21T15:35:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.043+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/704057285 learned_addr learned my addr 172.21.15.38:0/704057285 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.043+0000 7fcc9ffff640 1 --2- 172.21.15.38:0/704057285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde46160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.044+0000 7fcc9f7fe640 1 --2- 172.21.15.38:0/704057285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda706b0 0x5640fde50e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.044+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/704057285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 msgr2=0x5640fde46160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.044+0000 7fcc9f7fe640 1 --2- 172.21.15.38:0/704057285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde46160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.044+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/704057285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda706b0 msgr2=0x5640fde50e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.044+0000 7fcc9f7fe640 1 --2- 172.21.15.38:0/704057285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda706b0 0x5640fde50e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.044+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/704057285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640fde38d00 con 0x5640fda4ff60 2024-05-21T15:35:48.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.044+0000 7fcc9f7fe640 1 --2- 172.21.15.38:0/704057285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda4ff60 0x5640fde2b250 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcc940317c0 tx=0x7fcc94005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fee3bb7436de49a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:48.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.044+0000 7fcc85ffb640 1 -- 172.21.15.38:0/704057285 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94061070 con 0x5640fda4ff60 2024-05-21T15:35:48.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.044+0000 7fcc85ffb640 1 -- 172.21.15.38:0/704057285 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc94005180 con 0x5640fda4ff60 2024-05-21T15:35:48.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.044+0000 7fcc85ffb640 1 -- 172.21.15.38:0/704057285 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94035030 con 0x5640fda4ff60 2024-05-21T15:35:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.046+0000 7fccad566740 1 -- 172.21.15.38:0/704057285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda4ff60 msgr2=0x5640fde2b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.046+0000 7fccad566740 1 --2- 172.21.15.38:0/704057285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda4ff60 0x5640fde2b250 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcc940317c0 tx=0x7fcc94005480 comp rx=0 tx=0).stop 2024-05-21T15:35:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.046+0000 7fccad566740 1 -- 172.21.15.38:0/704057285 shutdown_connections 2024-05-21T15:35:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.046+0000 7fccad566740 1 --2- 172.21.15.38:0/704057285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda706b0 0x5640fde50e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.046+0000 7fccad566740 1 --2- 172.21.15.38:0/704057285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde46160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.046+0000 7fccad566740 1 --2- 172.21.15.38:0/704057285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda4ff60 0x5640fde2b250 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.046+0000 7fccad566740 1 -- 172.21.15.38:0/704057285 >> 172.21.15.38:0/704057285 conn(0x5640fda854e0 msgr2=0x5640fde2ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.046+0000 7fccad566740 1 -- 172.21.15.38:0/704057285 shutdown_connections 2024-05-21T15:35:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.046+0000 7fccad566740 1 -- 172.21.15.38:0/704057285 wait complete. 2024-05-21T15:35:48.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.046+0000 7fccad566740 1 -- start start 2024-05-21T15:35:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.047+0000 7fccad566740 1 Processor -- start 2024-05-21T15:35:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.047+0000 7fccad566740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda4ff60 0x5640fde45070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.047+0000 7fcc9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda4ff60 0x5640fde45070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.047+0000 7fcc9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda4ff60 0x5640fde45070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49522/0 (socket says 172.21.15.38:49522) 2024-05-21T15:35:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.047+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/2405465400 learned_addr learned my addr 172.21.15.38:0/2405465400 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.047+0000 7fccad566740 1 --2- 172.21.15.38:0/2405465400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde455b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.047+0000 7fccad566740 1 --2- 172.21.15.38:0/2405465400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda706b0 0x5640fdeb2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.047+0000 7fccad566740 1 -- 172.21.15.38:0/2405465400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c00fa20 con 0x5640fda4ff60 2024-05-21T15:35:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.047+0000 7fccad566740 1 -- 172.21.15.38:0/2405465400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c00f350 con 0x5640fda706b0 2024-05-21T15:35:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.047+0000 7fccad566740 1 -- 172.21.15.38:0/2405465400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c00c590 con 0x5640fda684d0 2024-05-21T15:35:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.048+0000 7fcc9ffff640 1 --2- 172.21.15.38:0/2405465400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.048+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/2405465400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 msgr2=0x5640fde455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.048+0000 7fcc9f7fe640 1 --2- 172.21.15.38:0/2405465400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde455b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.048+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/2405465400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda706b0 msgr2=0x5640fdeb2a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:35:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.048+0000 7fcc9f7fe640 1 --2- 172.21.15.38:0/2405465400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda706b0 0x5640fdeb2a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.048+0000 7fcc9f7fe640 1 -- 172.21.15.38:0/2405465400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640fdeb2f70 con 0x5640fda4ff60 2024-05-21T15:35:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.048+0000 7fccad566740 10 client.0 _ll_register_callbacks cb 0x7fcc94032c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.048+0000 7fccad566740 1 -- 172.21.15.38:0/2405465400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc94005180 con 0x5640fda4ff60 2024-05-21T15:35:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.048+0000 7fcc9f7fe640 1 --2- 172.21.15.38:0/2405465400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda4ff60 0x5640fde45070 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcc94036040 tx=0x7fcc9402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d7259bf4726858 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.048+0000 7fcc7f7fe640 10 client.15837 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.049+0000 7fcc7f7fe640 1 -- 172.21.15.38:0/2405465400 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94061070 con 0x5640fda4ff60 2024-05-21T15:35:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.049+0000 7fcc7f7fe640 1 -- 172.21.15.38:0/2405465400 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fcc9403c440 con 0x5640fda4ff60 2024-05-21T15:35:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.049+0000 7fcc7f7fe640 1 -- 172.21.15.38:0/2405465400 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc9403ce60 con 0x5640fda4ff60 2024-05-21T15:35:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.049+0000 7fccad566740 10 client.15837 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:48.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.049+0000 7fccad566740 1 -- 172.21.15.38:0/2405465400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5640fdeb3f70 con 0x5640fda4ff60 2024-05-21T15:35:48.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.049+0000 7fcc7f7fe640 1 -- 172.21.15.38:0/2405465400 <== mon.0 v2:172.21.15.38:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcc94004cd0 con 0x5640fda4ff60 2024-05-21T15:35:48.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.049+0000 7fccad566740 10 client.15837 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:48.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.049+0000 7fccad566740 10 client.15837 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:35:48.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.049+0000 7fccad566740 20 client.15837 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.049+0000 7fccad566740 1 --2- 172.21.15.38:0/2405465400 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x5640fda91390 0x5640fdeb65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.049+0000 7fccad566740 4 client.15837 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:35:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.049+0000 7fccad566740 1 -- 172.21.15.38:0/2405465400 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640fdd9d6c0 con 0x5640fda91390 2024-05-21T15:35:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.050+0000 7fcc9ffff640 1 --2- 172.21.15.38:0/2405465400 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x5640fda91390 0x5640fdeb65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.050+0000 7fcc9ffff640 1 --2- 172.21.15.38:0/2405465400 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x5640fda91390 0x5640fdeb65d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c06bf1badc00780 server_cookie=9af1b0e67765809b in_seq=0 out_seq=0 2024-05-21T15:35:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.050+0000 7fcc7f7fe640 10 client.15837 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:35:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fcc7f7fe640 1 -- 172.21.15.38:0/2405465400 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640fdd9d6c0 con 0x5640fda91390 2024-05-21T15:35:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fcc7f7fe640 10 client.15837 handle_command_reply: tid=0 2024-05-21T15:35:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fccad566740 1 client.15837 shutdown 2024-05-21T15:35:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fccad566740 1 -- 172.21.15.38:0/2405465400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda4ff60 msgr2=0x5640fde45070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fccad566740 1 --2- 172.21.15.38:0/2405465400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda4ff60 0x5640fde45070 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcc94036040 tx=0x7fcc9402a410 comp rx=0 tx=0).stop 2024-05-21T15:35:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fccad566740 1 -- 172.21.15.38:0/2405465400 shutdown_connections 2024-05-21T15:35:48.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fccad566740 1 --2- 172.21.15.38:0/2405465400 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x5640fda91390 0x5640fdeb65d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fccad566740 1 --2- 172.21.15.38:0/2405465400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640fda706b0 0x5640fdeb2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fccad566740 1 --2- 172.21.15.38:0/2405465400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640fda684d0 0x5640fde455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fccad566740 1 --2- 172.21.15.38:0/2405465400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5640fda4ff60 0x5640fde45070 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fccad566740 1 -- 172.21.15.38:0/2405465400 >> 172.21.15.38:0/2405465400 conn(0x5640fda854e0 msgr2=0x5640fde2b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:48.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fcc7f7fe640 0 client.15837 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:35:48.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fccad566740 1 -- 172.21.15.38:0/2405465400 shutdown_connections 2024-05-21T15:35:48.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fccad566740 1 -- 172.21.15.38:0/2405465400 wait complete. 2024-05-21T15:35:48.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.051+0000 7fccad566740 20 client.15837 trim_cache size 0 max 16384 2024-05-21T15:35:48.054 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:35:48.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.052+0000 7fcca66c3640 1 -- 172.21.15.38:0/2448413913 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcc880a2c10 msgr2=0x7fcc880a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.052+0000 7fcca66c3640 1 --2- 172.21.15.38:0/2448413913 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcc880a2c10 0x7fcc880a5120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcc98098250 tx=0x7fcc9800ef30 comp rx=0 tx=0).stop 2024-05-21T15:35:48.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.052+0000 7fcca66c3640 1 -- 172.21.15.38:0/2448413913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcca01078c0 msgr2=0x7fcca01a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.052+0000 7fcca66c3640 1 --2- 172.21.15.38:0/2448413913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcca01078c0 0x7fcca01a0710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800e940 tx=0x7fcc9800ee00 comp rx=0 tx=0).stop 2024-05-21T15:35:48.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.052+0000 7fcca66c3640 1 -- 172.21.15.38:0/2448413913 shutdown_connections 2024-05-21T15:35:48.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.052+0000 7fcca66c3640 1 --2- 172.21.15.38:0/2448413913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcca01078c0 0x7fcca01a0710 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.052+0000 7fcca66c3640 1 --2- 172.21.15.38:0/2448413913 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcc880a2c10 0x7fcc880a5120 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.052+0000 7fcca66c3640 1 --2- 172.21.15.38:0/2448413913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcca0102bc0 0x7fcca0199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.052+0000 7fcca66c3640 1 --2- 172.21.15.38:0/2448413913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcca0101980 0x7fcca0199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.052+0000 7fcca66c3640 1 -- 172.21.15.38:0/2448413913 >> 172.21.15.38:0/2448413913 conn(0x7fcca00fcb70 msgr2=0x7fcca0108d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:48.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.052+0000 7fcca66c3640 1 -- 172.21.15.38:0/2448413913 shutdown_connections 2024-05-21T15:35:48.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.052+0000 7fcca66c3640 1 -- 172.21.15.38:0/2448413913 wait complete. 2024-05-21T15:35:48.067 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:35:48.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.234+0000 7fa2a1577640 1 -- 172.21.15.38:0/2284856330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa29c10a960 msgr2=0x7fa29c10ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.234+0000 7fa2a1577640 1 --2- 172.21.15.38:0/2284856330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa29c10a960 0x7fa29c10ce40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa294009ca0 tx=0x7fa29402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.234+0000 7fa2a1577640 1 -- 172.21.15.38:0/2284856330 shutdown_connections 2024-05-21T15:35:48.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.234+0000 7fa2a1577640 1 --2- 172.21.15.38:0/2284856330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa29c10a960 0x7fa29c10ce40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.234+0000 7fa2a1577640 1 --2- 172.21.15.38:0/2284856330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa29c107e30 0x7fa29c10a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.234+0000 7fa2a1577640 1 --2- 172.21.15.38:0/2284856330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa29c102f50 0x7fa29c103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.234+0000 7fa2a1577640 1 -- 172.21.15.38:0/2284856330 >> 172.21.15.38:0/2284856330 conn(0x7fa29c0fc2f0 msgr2=0x7fa29c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.234+0000 7fa2a1577640 1 -- 172.21.15.38:0/2284856330 shutdown_connections 2024-05-21T15:35:48.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.234+0000 7fa2a1577640 1 -- 172.21.15.38:0/2284856330 wait complete. 2024-05-21T15:35:48.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.234+0000 7fa2a1577640 1 Processor -- start 2024-05-21T15:35:48.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.235+0000 7fa2a1577640 1 -- start start 2024-05-21T15:35:48.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.235+0000 7fa2a1577640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa29c102f50 0x7fa29c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.235+0000 7fa2a1577640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa29c107e30 0x7fa29c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.235+0000 7fa2a1577640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa29c10a960 0x7fa29c1a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.235+0000 7fa2a1577640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c110900 con 0x7fa29c107e30 2024-05-21T15:35:48.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.235+0000 7fa2a1577640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c110780 con 0x7fa29c10a960 2024-05-21T15:35:48.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.235+0000 7fa2a1577640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c110a80 con 0x7fa29c102f50 2024-05-21T15:35:48.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.235+0000 7fa29a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa29c107e30 0x7fa29c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.235+0000 7fa29a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa29c107e30 0x7fa29c19b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49546/0 (socket says 172.21.15.38:49546) 2024-05-21T15:35:48.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.235+0000 7fa29a7fc640 1 -- 172.21.15.38:0/1906537100 learned_addr learned my addr 172.21.15.38:0/1906537100 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.235+0000 7fa29b7fe640 1 --2- 172.21.15.38:0/1906537100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa29c10a960 0x7fa29c1a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.235+0000 7fa29affd640 1 --2- 172.21.15.38:0/1906537100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa29c102f50 0x7fa29c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.236+0000 7fa29a7fc640 1 -- 172.21.15.38:0/1906537100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa29c102f50 msgr2=0x7fa29c19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.236+0000 7fa29a7fc640 1 --2- 172.21.15.38:0/1906537100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa29c102f50 0x7fa29c19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.236+0000 7fa29a7fc640 1 -- 172.21.15.38:0/1906537100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa29c10a960 msgr2=0x7fa29c1a28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.236+0000 7fa29a7fc640 1 --2- 172.21.15.38:0/1906537100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa29c10a960 0x7fa29c1a28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.236+0000 7fa29a7fc640 1 -- 172.21.15.38:0/1906537100 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29c1a2f50 con 0x7fa29c107e30 2024-05-21T15:35:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.236+0000 7fa29a7fc640 1 --2- 172.21.15.38:0/1906537100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa29c107e30 0x7fa29c19b950 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa29000ea60 tx=0x7fa29000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.236+0000 7fa283fff640 1 -- 172.21.15.38:0/1906537100 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29000cde0 con 0x7fa29c107e30 2024-05-21T15:35:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.236+0000 7fa2a1577640 1 -- 172.21.15.38:0/1906537100 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa29c1a3240 con 0x7fa29c107e30 2024-05-21T15:35:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.236+0000 7fa283fff640 1 -- 172.21.15.38:0/1906537100 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2900035f0 con 0x7fa29c107e30 2024-05-21T15:35:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.236+0000 7fa283fff640 1 -- 172.21.15.38:0/1906537100 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290010670 con 0x7fa29c107e30 2024-05-21T15:35:48.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.237+0000 7fa2a1577640 1 -- 172.21.15.38:0/1906537100 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa29c1a3740 con 0x7fa29c107e30 2024-05-21T15:35:48.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.240+0000 7fa283fff640 1 -- 172.21.15.38:0/1906537100 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa2900037a0 con 0x7fa29c107e30 2024-05-21T15:35:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.240+0000 7fa2a8424740 1 Processor -- start 2024-05-21T15:35:48.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.240+0000 7fa2a8424740 1 -- start start 2024-05-21T15:35:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa2a8424740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2cf390 0x55bb7c66eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa2a8424740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 0x55bb7c66f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa2a8424740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c675f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa2a8424740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c5d3c40 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa2a8424740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c57b580 con 0x55bb7c2a64d0 2024-05-21T15:35:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa2a8424740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c5cfc00 con 0x55bb7c2cf390 2024-05-21T15:35:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa29a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c675f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa29affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 0x55bb7c66f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa29affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 0x55bb7c66f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:58762/0 (socket says 172.21.15.38:58762) 2024-05-21T15:35:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa29affd640 1 -- 172.21.15.38:0/593154981 learned_addr learned my addr 172.21.15.38:0/593154981 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa283fff640 1 --2- 172.21.15.38:0/1906537100 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2840a2b90 0x7fa2840a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa29b7fe640 1 --2- 172.21.15.38:0/593154981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2cf390 0x55bb7c66eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa29b7fe640 1 --2- 172.21.15.38:0/1906537100 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2840a2b90 0x7fa2840a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa29affd640 1 -- 172.21.15.38:0/593154981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2cf390 msgr2=0x55bb7c66eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa29affd640 1 --2- 172.21.15.38:0/593154981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2cf390 0x55bb7c66eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.241+0000 7fa29affd640 1 -- 172.21.15.38:0/593154981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 msgr2=0x55bb7c675f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.242+0000 7fa283fff640 1 -- 172.21.15.38:0/1906537100 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa2900cf380 con 0x7fa29c107e30 2024-05-21T15:35:48.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.242+0000 7fa29affd640 1 --2- 172.21.15.38:0/593154981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c675f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.242+0000 7fa29affd640 1 -- 172.21.15.38:0/593154981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7c2e73f0 con 0x55bb7c2a64d0 2024-05-21T15:35:48.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.242+0000 7fa29a7fc640 1 --2- 172.21.15.38:0/593154981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c675f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:48.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.242+0000 7fa29b7fe640 1 --2- 172.21.15.38:0/1906537100 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2840a2b90 0x7fa2840a50a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55bb7c66fcd0 tx=0x7fa294005c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:48.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.242+0000 7fa29affd640 1 --2- 172.21.15.38:0/593154981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 0x55bb7c66f080 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009c60 tx=0x7fa28c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e37d018439ecb7a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:48.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.242+0000 7fa2817fa640 1 -- 172.21.15.38:0/593154981 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c004230 con 0x55bb7c2a64d0 2024-05-21T15:35:48.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.243+0000 7fa2817fa640 1 -- 172.21.15.38:0/593154981 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa28c009e10 con 0x55bb7c2a64d0 2024-05-21T15:35:48.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.243+0000 7fa2817fa640 1 -- 172.21.15.38:0/593154981 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c019880 con 0x55bb7c2a64d0 2024-05-21T15:35:48.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.243+0000 7fa2a8424740 1 -- 172.21.15.38:0/593154981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 msgr2=0x55bb7c66f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.243+0000 7fa2a8424740 1 --2- 172.21.15.38:0/593154981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 0x55bb7c66f080 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009c60 tx=0x7fa28c011810 comp rx=0 tx=0).stop 2024-05-21T15:35:48.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.243+0000 7fa2a8424740 1 -- 172.21.15.38:0/593154981 shutdown_connections 2024-05-21T15:35:48.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.243+0000 7fa2a8424740 1 --2- 172.21.15.38:0/593154981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c675f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.243+0000 7fa2a8424740 1 --2- 172.21.15.38:0/593154981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 0x55bb7c66f080 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.243+0000 7fa2a8424740 1 --2- 172.21.15.38:0/593154981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2cf390 0x55bb7c66eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.243+0000 7fa2a8424740 1 -- 172.21.15.38:0/593154981 >> 172.21.15.38:0/593154981 conn(0x55bb7c2c34e0 msgr2=0x55bb7c66a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:48.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.244+0000 7fa2a8424740 1 -- 172.21.15.38:0/593154981 shutdown_connections 2024-05-21T15:35:48.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.244+0000 7fa2a8424740 1 -- 172.21.15.38:0/593154981 wait complete. 2024-05-21T15:35:48.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.244+0000 7fa2a8424740 1 -- start start 2024-05-21T15:35:48.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.244+0000 7fa2a8424740 1 Processor -- start 2024-05-21T15:35:48.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.244+0000 7fa2a8424740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 0x55bb7c678980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.244+0000 7fa2a8424740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c678ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.244+0000 7fa2a8424740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2cf390 0x55bb7c67fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.244+0000 7fa2a8424740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c6769d0 con 0x55bb7c2cf390 2024-05-21T15:35:48.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.244+0000 7fa2a8424740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c58ec40 con 0x55bb7c2a64d0 2024-05-21T15:35:48.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.244+0000 7fa2a8424740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c676b50 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.245+0000 7fa2a8424740 10 client.0 _ll_register_callbacks cb 0x55bb7c2e6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:48.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.245+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2cf390 0x55bb7c67fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.245+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2cf390 0x55bb7c67fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49562/0 (socket says 172.21.15.38:49562) 2024-05-21T15:35:48.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.245+0000 7fa29b7fe640 1 -- 172.21.15.38:0/1977406243 learned_addr learned my addr 172.21.15.38:0/1977406243 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:48.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.245+0000 7fa29a7fc640 1 --2- 172.21.15.38:0/1977406243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c678ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.245+0000 7fa29affd640 1 --2- 172.21.15.38:0/1977406243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 0x55bb7c678980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.245+0000 7fa29a7fc640 1 -- 172.21.15.38:0/1977406243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 msgr2=0x55bb7c678980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.245+0000 7fa29a7fc640 1 --2- 172.21.15.38:0/1977406243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 0x55bb7c678980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.245+0000 7fa29a7fc640 1 -- 172.21.15.38:0/1977406243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2cf390 msgr2=0x55bb7c67fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.245+0000 7fa29a7fc640 1 --2- 172.21.15.38:0/1977406243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2cf390 0x55bb7c67fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.245+0000 7fa29a7fc640 1 -- 172.21.15.38:0/1977406243 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7c3b17d0 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.245+0000 7fa29b7fe640 1 --2- 172.21.15.38:0/1977406243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2cf390 0x55bb7c67fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:48.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.245+0000 7fa29a7fc640 1 --2- 172.21.15.38:0/1977406243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c678ec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa2900cce80 tx=0x7fa290009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18b4c27223c12798 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:48.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.245+0000 7fa27eff5640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:35:48.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.246+0000 7fa27eff5640 1 -- 172.21.15.38:0/1977406243 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29009bb90 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.246+0000 7fa27eff5640 1 -- 172.21.15.38:0/1977406243 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa290098200 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.246+0000 7fa2a8424740 1 -- 172.21.15.38:0/1977406243 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa28c009e10 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.246+0000 7fa27eff5640 1 -- 172.21.15.38:0/1977406243 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa29009ff40 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.246+0000 7fa2a8424740 10 client.25585 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:48.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.246+0000 7fa2a8424740 1 -- 172.21.15.38:0/1977406243 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bb7c683150 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.246+0000 7fa27eff5640 1 -- 172.21.15.38:0/1977406243 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa28c009e10 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.246+0000 7fa27eff5640 1 -- 172.21.15.38:0/1977406243 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa2900ce070 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.246+0000 7fa2a8424740 10 client.25585 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:48.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.246+0000 7fa2a8424740 10 client.25585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:48.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.246+0000 7fa2a8424740 20 client.25585 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:48.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.246+0000 7fa2a8424740 1 --2- 172.21.15.38:0/1977406243 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bb7c28df60 0x55bb7c6868d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.246+0000 7fa2a8424740 4 client.25585 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:48.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.246+0000 7fa2a8424740 1 -- 172.21.15.38:0/1977406243 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb7c5e1370 con 0x55bb7c28df60 2024-05-21T15:35:48.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.247+0000 7fa29affd640 1 --2- 172.21.15.38:0/1977406243 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bb7c28df60 0x55bb7c6868d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.247+0000 7fa27eff5640 10 client.25585 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:48.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.247+0000 7fa29affd640 1 --2- 172.21.15.38:0/1977406243 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bb7c28df60 0x55bb7c6868d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4683e85b62090a3b server_cookie=20b61c99a54775b9 in_seq=0 out_seq=0 2024-05-21T15:35:48.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa27eff5640 1 -- 172.21.15.38:0/1977406243 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb7c5e1370 con 0x55bb7c28df60 2024-05-21T15:35:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa27eff5640 10 client.25585 handle_command_reply: tid=0 2024-05-21T15:35:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa2a8424740 1 client.25585 shutdown 2024-05-21T15:35:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa2a8424740 1 -- 172.21.15.38:0/1977406243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2ae6b0 msgr2=0x55bb7c678ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa2a8424740 1 --2- 172.21.15.38:0/1977406243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c678ec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa2900cce80 tx=0x7fa290009550 comp rx=0 tx=0).stop 2024-05-21T15:35:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa2a8424740 1 -- 172.21.15.38:0/1977406243 shutdown_connections 2024-05-21T15:35:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa2a8424740 1 --2- 172.21.15.38:0/1977406243 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bb7c28df60 0x55bb7c6868d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa2a8424740 1 --2- 172.21.15.38:0/1977406243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2cf390 0x55bb7c67fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa27eff5640 0 client.25585 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa2a8424740 1 --2- 172.21.15.38:0/1977406243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c678ec0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa2a8424740 1 --2- 172.21.15.38:0/1977406243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 0x55bb7c678980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa2a8424740 1 -- 172.21.15.38:0/1977406243 >> 172.21.15.38:0/1977406243 conn(0x55bb7c2c34e0 msgr2=0x55bb7c669570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa2a8424740 1 -- 172.21.15.38:0/1977406243 shutdown_connections 2024-05-21T15:35:48.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa2a8424740 1 -- 172.21.15.38:0/1977406243 wait complete. 2024-05-21T15:35:48.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.248+0000 7fa2a8424740 20 client.25585 trim_cache size 0 max 16384 2024-05-21T15:35:48.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.276+0000 7fa2a8424740 1 Processor -- start 2024-05-21T15:35:48.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.276+0000 7fa2a8424740 1 -- start start 2024-05-21T15:35:48.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.276+0000 7fa2a8424740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c28df60 0x55bb7c677660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.276+0000 7fa2a8424740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 0x55bb7c67fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.276+0000 7fa2a8424740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c68b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.276+0000 7fa2a8424740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1ac590 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.276+0000 7fa2a8424740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1ae0d0 con 0x55bb7c2a64d0 2024-05-21T15:35:48.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.276+0000 7fa2a8424740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1acc60 con 0x55bb7c28df60 2024-05-21T15:35:48.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.276+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c68b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.276+0000 7fa29affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c28df60 0x55bb7c677660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.276+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c68b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49578/0 (socket says 172.21.15.38:49578) 2024-05-21T15:35:48.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.276+0000 7fa29affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c28df60 0x55bb7c677660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41284/0 (socket says 172.21.15.38:41284) 2024-05-21T15:35:48.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.276+0000 7fa29b7fe640 1 -- 172.21.15.38:0/3954106942 learned_addr learned my addr 172.21.15.38:0/3954106942 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:48.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.276+0000 7fa29a7fc640 1 --2- 172.21.15.38:0/3954106942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 0x55bb7c67fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.277+0000 7fa29b7fe640 1 -- 172.21.15.38:0/3954106942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c28df60 msgr2=0x55bb7c677660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.277+0000 7fa29b7fe640 1 --2- 172.21.15.38:0/3954106942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c28df60 0x55bb7c677660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.277+0000 7fa29b7fe640 1 -- 172.21.15.38:0/3954106942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 msgr2=0x55bb7c67fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.277+0000 7fa29b7fe640 1 --2- 172.21.15.38:0/3954106942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 0x55bb7c67fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.277+0000 7fa29b7fe640 1 -- 172.21.15.38:0/3954106942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7c665d20 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.277+0000 7fa29b7fe640 1 --2- 172.21.15.38:0/3954106942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c68b050 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa294006010 tx=0x7fa294043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af853c55ea6a4e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:48.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.277+0000 7fa280ff9640 1 -- 172.21.15.38:0/3954106942 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294031820 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.277+0000 7fa280ff9640 1 -- 172.21.15.38:0/3954106942 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa294036040 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.277+0000 7fa280ff9640 1 -- 172.21.15.38:0/3954106942 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294049820 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.278+0000 7fa2a8424740 1 -- 172.21.15.38:0/3954106942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 msgr2=0x55bb7c68b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.278+0000 7fa2a8424740 1 --2- 172.21.15.38:0/3954106942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c68b050 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa294006010 tx=0x7fa294043000 comp rx=0 tx=0).stop 2024-05-21T15:35:48.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.279+0000 7fa2a8424740 1 -- 172.21.15.38:0/3954106942 shutdown_connections 2024-05-21T15:35:48.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.279+0000 7fa2a8424740 1 --2- 172.21.15.38:0/3954106942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c68b050 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.279+0000 7fa2a8424740 1 --2- 172.21.15.38:0/3954106942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c2a64d0 0x55bb7c67fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.279+0000 7fa2a8424740 1 --2- 172.21.15.38:0/3954106942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c28df60 0x55bb7c677660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.279+0000 7fa2a8424740 1 -- 172.21.15.38:0/3954106942 >> 172.21.15.38:0/3954106942 conn(0x55bb7c2c34e0 msgr2=0x55bb7c676f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:48.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.279+0000 7fa2a8424740 1 -- 172.21.15.38:0/3954106942 shutdown_connections 2024-05-21T15:35:48.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.279+0000 7fa2a8424740 1 -- 172.21.15.38:0/3954106942 wait complete. 2024-05-21T15:35:48.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.279+0000 7fa2a8424740 1 -- start start 2024-05-21T15:35:48.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.279+0000 7fa2a8424740 1 Processor -- start 2024-05-21T15:35:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.280+0000 7fa2a8424740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c28df60 0x55bb7c67ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.280+0000 7fa2a8424740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2a64d0 0x55bb7c67d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.280+0000 7fa2a8424740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c67e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.280+0000 7fa29affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c28df60 0x55bb7c67ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.280+0000 7fa29affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c28df60 0x55bb7c67ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:58776/0 (socket says 172.21.15.38:58776) 2024-05-21T15:35:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.280+0000 7fa29a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2a64d0 0x55bb7c67d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.280+0000 7fa29a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2a64d0 0x55bb7c67d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41298/0 (socket says 172.21.15.38:41298) 2024-05-21T15:35:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.280+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c67e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.280+0000 7fa29b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c67e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49580/0 (socket says 172.21.15.38:49580) 2024-05-21T15:35:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.280+0000 7fa29affd640 1 -- 172.21.15.38:0/3875674193 learned_addr learned my addr 172.21.15.38:0/3875674193 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.280+0000 7fa2a8424740 1 -- 172.21.15.38:0/3875674193 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1ada00 con 0x55bb7c2ae6b0 2024-05-21T15:35:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.280+0000 7fa2a8424740 1 -- 172.21.15.38:0/3875674193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1ad330 con 0x55bb7c28df60 2024-05-21T15:35:48.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.280+0000 7fa2a8424740 1 -- 172.21.15.38:0/3875674193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1aa570 con 0x55bb7c2a64d0 2024-05-21T15:35:48.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.281+0000 7fa29affd640 1 -- 172.21.15.38:0/3875674193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2a64d0 msgr2=0x55bb7c67d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.281+0000 7fa29affd640 1 --2- 172.21.15.38:0/3875674193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2a64d0 0x55bb7c67d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.281+0000 7fa29affd640 1 -- 172.21.15.38:0/3875674193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 msgr2=0x55bb7c67e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.281+0000 7fa29affd640 1 --2- 172.21.15.38:0/3875674193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c67e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.281+0000 7fa29affd640 1 -- 172.21.15.38:0/3875674193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7c67e8e0 con 0x55bb7c28df60 2024-05-21T15:35:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.281+0000 7fa2a8424740 10 client.0 _ll_register_callbacks cb 0x7fa290003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.281+0000 7fa2a8424740 1 -- 172.21.15.38:0/3875674193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa294036040 con 0x55bb7c28df60 2024-05-21T15:35:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.281+0000 7fa29affd640 1 --2- 172.21.15.38:0/3875674193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c28df60 0x55bb7c67ceb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55bb7c66fee0 tx=0x7fa28c009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa5c7be51224bb22 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.281+0000 7fa27eff5640 10 client.25523 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:35:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.281+0000 7fa27eff5640 1 -- 172.21.15.38:0/3875674193 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c01f040 con 0x55bb7c28df60 2024-05-21T15:35:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.281+0000 7fa27eff5640 1 -- 172.21.15.38:0/3875674193 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa28c003910 con 0x55bb7c28df60 2024-05-21T15:35:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.282+0000 7fa27eff5640 1 -- 172.21.15.38:0/3875674193 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa28c003b70 con 0x55bb7c28df60 2024-05-21T15:35:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.282+0000 7fa2a8424740 10 client.25523 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.282+0000 7fa2a8424740 1 -- 172.21.15.38:0/3875674193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55bb7c67fa70 con 0x55bb7c28df60 2024-05-21T15:35:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.282+0000 7fa27eff5640 1 -- 172.21.15.38:0/3875674193 <== mon.2 v2:172.21.15.45:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa28c018070 con 0x55bb7c28df60 2024-05-21T15:35:48.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.282+0000 7fa2a8424740 10 client.25523 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.282+0000 7fa2a8424740 10 client.25523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.282+0000 7fa2a8424740 20 client.25523 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.282+0000 7fa2a8424740 1 --2- 172.21.15.38:0/3875674193 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bb7c2cf390 0x55bb7c6f0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.282+0000 7fa2a8424740 4 client.25523 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:35:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.282+0000 7fa2a8424740 1 -- 172.21.15.38:0/3875674193 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb7c5db6c0 con 0x55bb7c2cf390 2024-05-21T15:35:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.282+0000 7fa29a7fc640 1 --2- 172.21.15.38:0/3875674193 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bb7c2cf390 0x55bb7c6f0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.283+0000 7fa27eff5640 10 client.25523 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.283+0000 7fa29a7fc640 1 --2- 172.21.15.38:0/3875674193 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bb7c2cf390 0x55bb7c6f0c20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81234b3963b4397b server_cookie=90bf12ef1dee261a in_seq=0 out_seq=0 2024-05-21T15:35:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.283+0000 7fa27eff5640 1 -- 172.21.15.38:0/3875674193 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb7c5db6c0 con 0x55bb7c2cf390 2024-05-21T15:35:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.283+0000 7fa27eff5640 10 client.25523 handle_command_reply: tid=0 2024-05-21T15:35:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.283+0000 7fa2a8424740 1 client.25523 shutdown 2024-05-21T15:35:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.283+0000 7fa2a8424740 1 -- 172.21.15.38:0/3875674193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c28df60 msgr2=0x55bb7c67ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.283+0000 7fa2a8424740 1 --2- 172.21.15.38:0/3875674193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c28df60 0x55bb7c67ceb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55bb7c66fee0 tx=0x7fa28c009940 comp rx=0 tx=0).stop 2024-05-21T15:35:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.283+0000 7fa2a8424740 1 -- 172.21.15.38:0/3875674193 shutdown_connections 2024-05-21T15:35:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.283+0000 7fa2a8424740 1 --2- 172.21.15.38:0/3875674193 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bb7c2cf390 0x55bb7c6f0c20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.283+0000 7fa2a8424740 1 --2- 172.21.15.38:0/3875674193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bb7c2ae6b0 0x55bb7c67e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.283+0000 7fa2a8424740 1 --2- 172.21.15.38:0/3875674193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb7c2a64d0 0x55bb7c67d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.283+0000 7fa27eff5640 0 client.25523 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.283+0000 7fa2a8424740 1 --2- 172.21.15.38:0/3875674193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb7c28df60 0x55bb7c67ceb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.283+0000 7fa2a8424740 1 -- 172.21.15.38:0/3875674193 >> 172.21.15.38:0/3875674193 conn(0x55bb7c2c34e0 msgr2=0x55bb7c66d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.284+0000 7fa2a8424740 1 -- 172.21.15.38:0/3875674193 shutdown_connections 2024-05-21T15:35:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.284+0000 7fa2a8424740 1 -- 172.21.15.38:0/3875674193 wait complete. 2024-05-21T15:35:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.284+0000 7fa2a8424740 20 client.25523 trim_cache size 0 max 16384 2024-05-21T15:35:48.287 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:35:48.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.284+0000 7fa2a1577640 1 -- 172.21.15.38:0/1906537100 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2840a2b90 msgr2=0x7fa2840a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.284+0000 7fa2a1577640 1 --2- 172.21.15.38:0/1906537100 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2840a2b90 0x7fa2840a50a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55bb7c66fcd0 tx=0x7fa294005c30 comp rx=0 tx=0).stop 2024-05-21T15:35:48.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.284+0000 7fa2a1577640 1 -- 172.21.15.38:0/1906537100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa29c107e30 msgr2=0x7fa29c19b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:48.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.284+0000 7fa2a1577640 1 --2- 172.21.15.38:0/1906537100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa29c107e30 0x7fa29c19b950 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa29000ea60 tx=0x7fa29000ef20 comp rx=0 tx=0).stop 2024-05-21T15:35:48.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.285+0000 7fa2a1577640 1 -- 172.21.15.38:0/1906537100 shutdown_connections 2024-05-21T15:35:48.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.285+0000 7fa2a1577640 1 --2- 172.21.15.38:0/1906537100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa29c10a960 0x7fa29c1a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.285+0000 7fa2a1577640 1 --2- 172.21.15.38:0/1906537100 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2840a2b90 0x7fa2840a50a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.285+0000 7fa2a1577640 1 --2- 172.21.15.38:0/1906537100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa29c107e30 0x7fa29c19b950 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.285+0000 7fa2a1577640 1 --2- 172.21.15.38:0/1906537100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa29c102f50 0x7fa29c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:48.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.285+0000 7fa2a1577640 1 -- 172.21.15.38:0/1906537100 >> 172.21.15.38:0/1906537100 conn(0x7fa29c0fc2f0 msgr2=0x7fa29c108cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:48.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.285+0000 7fa2a1577640 1 -- 172.21.15.38:0/1906537100 shutdown_connections 2024-05-21T15:35:48.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:48.285+0000 7fa2a1577640 1 -- 172.21.15.38:0/1906537100 wait complete. 2024-05-21T15:35:49.299 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:35:49.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:35:49.299 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:35:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.438+0000 7f61e6c1b640 1 -- 172.21.15.38:0/345231780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e0102370 msgr2=0x7f61e0102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.438+0000 7f61e6c1b640 1 --2- 172.21.15.38:0/345231780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e0102370 0x7f61e0102780 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00d910 tx=0x7f61cc02a160 comp rx=0 tx=0).stop 2024-05-21T15:35:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.438+0000 7f61e6c1b640 1 -- 172.21.15.38:0/345231780 shutdown_connections 2024-05-21T15:35:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.438+0000 7f61e6c1b640 1 --2- 172.21.15.38:0/345231780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61e0107070 0x7f61e0109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.438+0000 7f61e6c1b640 1 --2- 172.21.15.38:0/345231780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e0102370 0x7f61e0102780 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.438+0000 7f61e6c1b640 1 --2- 172.21.15.38:0/345231780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e0101130 0x7f61e0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.438+0000 7f61e6c1b640 1 -- 172.21.15.38:0/345231780 >> 172.21.15.38:0/345231780 conn(0x7f61e00fc300 msgr2=0x7f61e00fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.438+0000 7f61e6c1b640 1 -- 172.21.15.38:0/345231780 shutdown_connections 2024-05-21T15:35:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.438+0000 7f61e6c1b640 1 -- 172.21.15.38:0/345231780 wait complete. 2024-05-21T15:35:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.439+0000 7f61e6c1b640 1 Processor -- start 2024-05-21T15:35:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.439+0000 7f61e6c1b640 1 -- start start 2024-05-21T15:35:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.439+0000 7f61e6c1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61e0101130 0x7f61e0197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.439+0000 7f61e6c1b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e0107070 0x7f61e0197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.439+0000 7f61e6c1b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e019c540 0x7f61e019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.440+0000 7f61e6c1b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f61e010c560 con 0x7f61e019c540 2024-05-21T15:35:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.440+0000 7f61e6c1b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f61e010c3e0 con 0x7f61e0101130 2024-05-21T15:35:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.440+0000 7f61e51b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e019c540 0x7f61e019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.440+0000 7f61e51b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e019c540 0x7f61e019e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49592/0 (socket says 172.21.15.38:49592) 2024-05-21T15:35:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.440+0000 7f61e51b8640 1 -- 172.21.15.38:0/1820323500 learned_addr learned my addr 172.21.15.38:0/1820323500 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.440+0000 7f61e6c1b640 1 -- 172.21.15.38:0/1820323500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f61e010c6e0 con 0x7f61e0107070 2024-05-21T15:35:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.440+0000 7f61dffff640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e0107070 0x7f61e0197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.440+0000 7f61e49b7640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61e0101130 0x7f61e0197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.440+0000 7f61dffff640 1 -- 172.21.15.38:0/1820323500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61e0101130 msgr2=0x7f61e0197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.440+0000 7f61dffff640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61e0101130 0x7f61e0197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.440+0000 7f61dffff640 1 -- 172.21.15.38:0/1820323500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e019c540 msgr2=0x7f61e019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.440+0000 7f61dffff640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e019c540 0x7f61e019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.440+0000 7f61dffff640 1 -- 172.21.15.38:0/1820323500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e019f090 con 0x7f61e0107070 2024-05-21T15:35:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.440+0000 7f61e51b8640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e019c540 0x7f61e019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.441+0000 7f61dffff640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e0107070 0x7f61e0197640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f61cc008000 tx=0x7f61cc02adc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.441+0000 7f61ddffb640 1 -- 172.21.15.38:0/1820323500 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc004030 con 0x7f61e0107070 2024-05-21T15:35:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.441+0000 7f61e6c1b640 1 -- 172.21.15.38:0/1820323500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e019f320 con 0x7f61e0107070 2024-05-21T15:35:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.441+0000 7f61ddffb640 1 -- 172.21.15.38:0/1820323500 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61cc00cbd0 con 0x7f61e0107070 2024-05-21T15:35:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.441+0000 7f61ddffb640 1 -- 172.21.15.38:0/1820323500 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc003a10 con 0x7f61e0107070 2024-05-21T15:35:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.441+0000 7f61e6c1b640 1 -- 172.21.15.38:0/1820323500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e019f7e0 con 0x7f61e0107070 2024-05-21T15:35:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.445+0000 7f61edba3740 1 Processor -- start 2024-05-21T15:35:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.445+0000 7f61ddffb640 1 -- 172.21.15.38:0/1820323500 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f61cc03a070 con 0x7f61e0107070 2024-05-21T15:35:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.445+0000 7f61edba3740 1 -- start start 2024-05-21T15:35:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.445+0000 7f61edba3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f391b0 0x5646c62d8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.445+0000 7f61edba3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f102f0 0x5646c62d93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.445+0000 7f61edba3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f184d0 0x5646c62e02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.445+0000 7f61edba3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5646c623d340 con 0x5646c5f102f0 2024-05-21T15:35:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.445+0000 7f61edba3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5646c61e5290 con 0x5646c5f184d0 2024-05-21T15:35:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.445+0000 7f61edba3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5646c6239300 con 0x5646c5f391b0 2024-05-21T15:35:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.446+0000 7f61e49b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f102f0 0x5646c62d93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.446+0000 7f61e49b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f102f0 0x5646c62d93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49604/0 (socket says 172.21.15.38:49604) 2024-05-21T15:35:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.446+0000 7f61e49b7640 1 -- 172.21.15.38:0/1861252141 learned_addr learned my addr 172.21.15.38:0/1861252141 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.446+0000 7f61dffff640 1 --2- 172.21.15.38:0/1861252141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f184d0 0x5646c62e02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.446+0000 7f61e51b8640 1 --2- 172.21.15.38:0/1861252141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f391b0 0x5646c62d8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.446+0000 7f61e49b7640 1 -- 172.21.15.38:0/1861252141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f391b0 msgr2=0x5646c62d8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.446+0000 7f61e49b7640 1 --2- 172.21.15.38:0/1861252141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f391b0 0x5646c62d8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.446+0000 7f61e49b7640 1 -- 172.21.15.38:0/1861252141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f184d0 msgr2=0x5646c62e02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.446+0000 7f61e49b7640 1 --2- 172.21.15.38:0/1861252141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f184d0 0x5646c62e02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.446+0000 7f61ddffb640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f61d00a2cb0 0x7f61d00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.446+0000 7f61e49b7640 1 -- 172.21.15.38:0/1861252141 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c62098e0 con 0x5646c5f102f0 2024-05-21T15:35:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.446+0000 7f61e51b8640 1 --2- 172.21.15.38:0/1861252141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f391b0 0x5646c62d8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.447+0000 7f61ddffb640 1 -- 172.21.15.38:0/1820323500 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f61cc0eea60 con 0x7f61e0107070 2024-05-21T15:35:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.447+0000 7f61e51b8640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f61d00a2cb0 0x7f61d00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.448+0000 7f61e49b7640 1 --2- 172.21.15.38:0/1861252141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f102f0 0x5646c62d93e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f61d400f900 tx=0x7f61d400fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1449dd4e29993902 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.448+0000 7f61e51b8640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f61d00a2cb0 0x7f61d00a51c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f61d8003fd0 tx=0x7f61d800f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.448+0000 7f61caffd640 1 -- 172.21.15.38:0/1861252141 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d401a410 con 0x5646c5f102f0 2024-05-21T15:35:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.448+0000 7f61caffd640 1 -- 172.21.15.38:0/1861252141 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d401f070 con 0x5646c5f102f0 2024-05-21T15:35:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.448+0000 7f61caffd640 1 -- 172.21.15.38:0/1861252141 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4019760 con 0x5646c5f102f0 2024-05-21T15:35:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.449+0000 7f61edba3740 1 -- 172.21.15.38:0/1861252141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f102f0 msgr2=0x5646c62d93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.449+0000 7f61edba3740 1 --2- 172.21.15.38:0/1861252141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f102f0 0x5646c62d93e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f61d400f900 tx=0x7f61d400fdc0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.450+0000 7f61edba3740 1 -- 172.21.15.38:0/1861252141 shutdown_connections 2024-05-21T15:35:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.450+0000 7f61edba3740 1 --2- 172.21.15.38:0/1861252141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f184d0 0x5646c62e02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.450+0000 7f61edba3740 1 --2- 172.21.15.38:0/1861252141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f102f0 0x5646c62d93e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.450+0000 7f61edba3740 1 --2- 172.21.15.38:0/1861252141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f391b0 0x5646c62d8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.450+0000 7f61edba3740 1 -- 172.21.15.38:0/1861252141 >> 172.21.15.38:0/1861252141 conn(0x5646c5f2d300 msgr2=0x5646c62d47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.450+0000 7f61edba3740 1 -- 172.21.15.38:0/1861252141 shutdown_connections 2024-05-21T15:35:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.450+0000 7f61edba3740 1 -- 172.21.15.38:0/1861252141 wait complete. 2024-05-21T15:35:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.451+0000 7f61edba3740 1 -- start start 2024-05-21T15:35:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.451+0000 7f61edba3740 1 Processor -- start 2024-05-21T15:35:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.452+0000 7f61edba3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f102f0 0x5646c62e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.452+0000 7f61edba3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f184d0 0x5646c62e3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.452+0000 7f61edba3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f391b0 0x5646c62ea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.452+0000 7f61e49b7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f102f0 0x5646c62e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.452+0000 7f61e49b7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f102f0 0x5646c62e2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41332/0 (socket says 172.21.15.38:41332) 2024-05-21T15:35:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.452+0000 7f61dffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f184d0 0x5646c62e3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.452+0000 7f61dffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f184d0 0x5646c62e3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:58808/0 (socket says 172.21.15.38:58808) 2024-05-21T15:35:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.452+0000 7f61e51b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f391b0 0x5646c62ea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.452+0000 7f61e51b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f391b0 0x5646c62ea0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49612/0 (socket says 172.21.15.38:49612) 2024-05-21T15:35:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.452+0000 7f61e49b7640 1 -- 172.21.15.38:0/853116137 learned_addr learned my addr 172.21.15.38:0/853116137 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.452+0000 7f61edba3740 1 -- 172.21.15.38:0/853116137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5646c62e0d30 con 0x5646c5f391b0 2024-05-21T15:35:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.452+0000 7f61edba3740 1 -- 172.21.15.38:0/853116137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5646c61f8790 con 0x5646c5f184d0 2024-05-21T15:35:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.453+0000 7f61edba3740 1 -- 172.21.15.38:0/853116137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5646c62e0eb0 con 0x5646c5f102f0 2024-05-21T15:35:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.453+0000 7f61e51b8640 1 -- 172.21.15.38:0/853116137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f102f0 msgr2=0x5646c62e2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.453+0000 7f61e51b8640 1 --2- 172.21.15.38:0/853116137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f102f0 0x5646c62e2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.453+0000 7f61e51b8640 1 -- 172.21.15.38:0/853116137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f184d0 msgr2=0x5646c62e3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.453+0000 7f61e51b8640 1 --2- 172.21.15.38:0/853116137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f184d0 0x5646c62e3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.453+0000 7f61e51b8640 1 -- 172.21.15.38:0/853116137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c5f51210 con 0x5646c5f391b0 2024-05-21T15:35:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.453+0000 7f61edba3740 10 client.0 _ll_register_callbacks cb 0x5646c6233a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.453+0000 7f61edba3740 1 -- 172.21.15.38:0/853116137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d401f070 con 0x5646c5f391b0 2024-05-21T15:35:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.454+0000 7f61e51b8640 1 --2- 172.21.15.38:0/853116137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f391b0 0x5646c62ea0f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x5646c62da030 tx=0x7f61d8010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b02dc3745867917 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.454+0000 7f61bffff640 10 client.15867 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.454+0000 7f61bffff640 1 -- 172.21.15.38:0/853116137 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d802b060 con 0x5646c5f391b0 2024-05-21T15:35:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.454+0000 7f61bffff640 1 -- 172.21.15.38:0/853116137 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f61d8017470 con 0x5646c5f391b0 2024-05-21T15:35:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.454+0000 7f61bffff640 1 -- 172.21.15.38:0/853116137 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f61d8017670 con 0x5646c5f391b0 2024-05-21T15:35:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.454+0000 7f61edba3740 10 client.15867 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.454+0000 7f61edba3740 1 -- 172.21.15.38:0/853116137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5646c601d620 con 0x5646c5f391b0 2024-05-21T15:35:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.455+0000 7f61bffff640 1 -- 172.21.15.38:0/853116137 <== mon.0 v2:172.21.15.38:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f61d800b040 con 0x5646c5f391b0 2024-05-21T15:35:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.455+0000 7f61edba3740 10 client.15867 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.455+0000 7f61edba3740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.455+0000 7f61edba3740 20 client.15867 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.455+0000 7f61edba3740 1 --2- 172.21.15.38:0/853116137 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c5ef7d80 0x5646c62f0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.455+0000 7f61edba3740 4 client.15867 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.455+0000 7f61edba3740 1 -- 172.21.15.38:0/853116137 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646c624aa70 con 0x5646c5ef7d80 2024-05-21T15:35:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.455+0000 7f61e49b7640 1 --2- 172.21.15.38:0/853116137 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c5ef7d80 0x5646c62f0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.456+0000 7f61bffff640 10 client.15867 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.456+0000 7f61e49b7640 1 --2- 172.21.15.38:0/853116137 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c5ef7d80 0x5646c62f0260 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b8b1e48a6b7809a server_cookie=bb6a8062ac7e930d in_seq=0 out_seq=0 2024-05-21T15:35:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.457+0000 7f61bffff640 1 -- 172.21.15.38:0/853116137 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646c624aa70 con 0x5646c5ef7d80 2024-05-21T15:35:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.457+0000 7f61bffff640 10 client.15867 handle_command_reply: tid=0 2024-05-21T15:35:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.457+0000 7f61edba3740 1 client.15867 shutdown 2024-05-21T15:35:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.457+0000 7f61edba3740 1 -- 172.21.15.38:0/853116137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f391b0 msgr2=0x5646c62ea0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.457+0000 7f61edba3740 1 --2- 172.21.15.38:0/853116137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f391b0 0x5646c62ea0f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x5646c62da030 tx=0x7f61d8010040 comp rx=0 tx=0).stop 2024-05-21T15:35:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.457+0000 7f61edba3740 1 -- 172.21.15.38:0/853116137 shutdown_connections 2024-05-21T15:35:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.457+0000 7f61edba3740 1 --2- 172.21.15.38:0/853116137 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c5ef7d80 0x5646c62f0260 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.457+0000 7f61edba3740 1 --2- 172.21.15.38:0/853116137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f391b0 0x5646c62ea0f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.457+0000 7f61edba3740 1 --2- 172.21.15.38:0/853116137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f184d0 0x5646c62e3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.457+0000 7f61edba3740 1 --2- 172.21.15.38:0/853116137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f102f0 0x5646c62e2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.457+0000 7f61edba3740 1 -- 172.21.15.38:0/853116137 >> 172.21.15.38:0/853116137 conn(0x5646c5f2d300 msgr2=0x5646c62d77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.457+0000 7f61bffff640 0 client.15867 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.457+0000 7f61edba3740 1 -- 172.21.15.38:0/853116137 shutdown_connections 2024-05-21T15:35:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.457+0000 7f61edba3740 1 -- 172.21.15.38:0/853116137 wait complete. 2024-05-21T15:35:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.458+0000 7f61edba3740 20 client.15867 trim_cache size 0 max 16384 2024-05-21T15:35:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.488+0000 7f61edba3740 1 Processor -- start 2024-05-21T15:35:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.488+0000 7f61edba3740 1 -- start start 2024-05-21T15:35:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f3892e51640 1 -- 172.21.15.38:0/1574484989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388c103cf0 msgr2=0x7f388c104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f3892e51640 1 --2- 172.21.15.38:0/1574484989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388c103cf0 0x7f388c104100 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3878009900 tx=0x7f387802a410 comp rx=0 tx=0).stop 2024-05-21T15:35:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f61edba3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f102f0 0x5646c62e1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f61edba3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f184d0 0x5646c62e2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f61edba3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f391b0 0x5646c62e2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f3892e51640 1 -- 172.21.15.38:0/1574484989 shutdown_connections 2024-05-21T15:35:49.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f3892e51640 1 --2- 172.21.15.38:0/1574484989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f388c1089f0 0x7f388c10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f3892e51640 1 --2- 172.21.15.38:0/1574484989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388c103cf0 0x7f388c104100 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f3892e51640 1 --2- 172.21.15.38:0/1574484989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f388c102ab0 0x7f388c1037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f61dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f184d0 0x5646c62e2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f61e49b7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f102f0 0x5646c62e1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f3892e51640 1 -- 172.21.15.38:0/1574484989 >> 172.21.15.38:0/1574484989 conn(0x7f388c0fdca0 msgr2=0x7f388c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f61e49b7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f102f0 0x5646c62e1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41342/0 (socket says 172.21.15.38:41342) 2024-05-21T15:35:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f61dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f184d0 0x5646c62e2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49626/0 (socket says 172.21.15.38:49626) 2024-05-21T15:35:49.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f3892e51640 1 -- 172.21.15.38:0/1574484989 shutdown_connections 2024-05-21T15:35:49.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f3892e51640 1 -- 172.21.15.38:0/1574484989 wait complete. 2024-05-21T15:35:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f61e51b8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f391b0 0x5646c62e2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f61e51b8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f391b0 0x5646c62e2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:58830/0 (socket says 172.21.15.38:58830) 2024-05-21T15:35:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f61edba3740 1 -- 172.21.15.38:0/2586655551 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01a8640 con 0x5646c5f184d0 2024-05-21T15:35:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f61e49b7640 1 -- 172.21.15.38:0/2586655551 learned_addr learned my addr 172.21.15.38:0/2586655551 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f61edba3740 1 -- 172.21.15.38:0/2586655551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01aa180 con 0x5646c5f391b0 2024-05-21T15:35:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.489+0000 7f61edba3740 1 -- 172.21.15.38:0/2586655551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01a8d10 con 0x5646c5f102f0 2024-05-21T15:35:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.490+0000 7f61e49b7640 1 -- 172.21.15.38:0/2586655551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f391b0 msgr2=0x5646c62e2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.490+0000 7f61e49b7640 1 --2- 172.21.15.38:0/2586655551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f391b0 0x5646c62e2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.490+0000 7f61e49b7640 1 -- 172.21.15.38:0/2586655551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f184d0 msgr2=0x5646c62e2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.490+0000 7f61e49b7640 1 --2- 172.21.15.38:0/2586655551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f184d0 0x5646c62e2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.490+0000 7f61e49b7640 1 -- 172.21.15.38:0/2586655551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c62d8a10 con 0x5646c5f102f0 2024-05-21T15:35:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.490+0000 7f61e51b8640 1 --2- 172.21.15.38:0/2586655551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f391b0 0x5646c62e2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.490+0000 7f61dffff640 1 --2- 172.21.15.38:0/2586655551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f184d0 0x5646c62e2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.490+0000 7f61e49b7640 1 --2- 172.21.15.38:0/2586655551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f102f0 0x5646c62e1960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f61d401ad80 tx=0x7f61d401ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a48de3222c5d7c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.490+0000 7f3892e51640 1 Processor -- start 2024-05-21T15:35:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.490+0000 7f61ca7fc640 1 -- 172.21.15.38:0/2586655551 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4033070 con 0x5646c5f102f0 2024-05-21T15:35:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.490+0000 7f61ca7fc640 1 -- 172.21.15.38:0/2586655551 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4005520 con 0x5646c5f102f0 2024-05-21T15:35:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.490+0000 7f61ca7fc640 1 -- 172.21.15.38:0/2586655551 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d402d340 con 0x5646c5f102f0 2024-05-21T15:35:49.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.490+0000 7f3892e51640 1 -- start start 2024-05-21T15:35:49.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.491+0000 7f3892e51640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f388c102ab0 0x7f388c199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.491+0000 7f3892e51640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f388c103cf0 0x7f388c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.491+0000 7f3892e51640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388c1089f0 0x7f388c1a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.491+0000 7f3890bed640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f388c102ab0 0x7f388c199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.491+0000 7f3890bed640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f388c102ab0 0x7f388c199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:58846/0 (socket says 172.21.15.38:58846) 2024-05-21T15:35:49.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.491+0000 7f38913ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388c1089f0 0x7f388c1a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.491+0000 7f388bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f388c103cf0 0x7f388c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.491+0000 7f388bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f388c103cf0 0x7f388c199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41348/0 (socket says 172.21.15.38:41348) 2024-05-21T15:35:49.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.491+0000 7f38913ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388c1089f0 0x7f388c1a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49638/0 (socket says 172.21.15.38:49638) 2024-05-21T15:35:49.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.491+0000 7f3890bed640 1 -- 172.21.15.38:0/2443347588 learned_addr learned my addr 172.21.15.38:0/2443347588 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.491+0000 7f3892e51640 1 -- 172.21.15.38:0/2443347588 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f388c10e780 con 0x7f388c1089f0 2024-05-21T15:35:49.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.491+0000 7f3892e51640 1 -- 172.21.15.38:0/2443347588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f388c10e600 con 0x7f388c102ab0 2024-05-21T15:35:49.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.491+0000 7f3892e51640 1 -- 172.21.15.38:0/2443347588 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f388c10e900 con 0x7f388c103cf0 2024-05-21T15:35:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f61edba3740 1 -- 172.21.15.38:0/2586655551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f102f0 msgr2=0x5646c62e1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f61edba3740 1 --2- 172.21.15.38:0/2586655551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f102f0 0x5646c62e1960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f61d401ad80 tx=0x7f61d401ac80 comp rx=0 tx=0).stop 2024-05-21T15:35:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f61edba3740 1 -- 172.21.15.38:0/2586655551 shutdown_connections 2024-05-21T15:35:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f61edba3740 1 --2- 172.21.15.38:0/2586655551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f391b0 0x5646c62e2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f61edba3740 1 --2- 172.21.15.38:0/2586655551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f184d0 0x5646c62e2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f61edba3740 1 --2- 172.21.15.38:0/2586655551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f102f0 0x5646c62e1960 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f61edba3740 1 -- 172.21.15.38:0/2586655551 >> 172.21.15.38:0/2586655551 conn(0x5646c5f2d300 msgr2=0x5646c62e1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f3890bed640 1 -- 172.21.15.38:0/2443347588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f388c103cf0 msgr2=0x7f388c199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f3890bed640 1 --2- 172.21.15.38:0/2443347588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f388c103cf0 0x7f388c199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f3890bed640 1 -- 172.21.15.38:0/2443347588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388c1089f0 msgr2=0x7f388c1a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f61edba3740 1 -- 172.21.15.38:0/2586655551 shutdown_connections 2024-05-21T15:35:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f3890bed640 1 --2- 172.21.15.38:0/2443347588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388c1089f0 0x7f388c1a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f3890bed640 1 -- 172.21.15.38:0/2443347588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c1a0d90 con 0x7f388c102ab0 2024-05-21T15:35:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f38913ee640 1 --2- 172.21.15.38:0/2443347588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388c1089f0 0x7f388c1a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f3890bed640 1 --2- 172.21.15.38:0/2443347588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f388c102ab0 0x7f388c199250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f388000e980 tx=0x7f388000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f61edba3740 1 -- 172.21.15.38:0/2586655551 wait complete. 2024-05-21T15:35:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f61edba3740 1 -- start start 2024-05-21T15:35:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f61edba3740 1 Processor -- start 2024-05-21T15:35:49.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f3889ffb640 1 -- 172.21.15.38:0/2443347588 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388000cd80 con 0x7f388c102ab0 2024-05-21T15:35:49.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.492+0000 7f3892e51640 1 -- 172.21.15.38:0/2443347588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388c1a1080 con 0x7f388c102ab0 2024-05-21T15:35:49.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.493+0000 7f3889ffb640 1 -- 172.21.15.38:0/2443347588 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3880010910 con 0x7f388c102ab0 2024-05-21T15:35:49.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.493+0000 7f3889ffb640 1 -- 172.21.15.38:0/2443347588 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38800189c0 con 0x7f388c102ab0 2024-05-21T15:35:49.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.493+0000 7f3892e51640 1 -- 172.21.15.38:0/2443347588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388c1a15f0 con 0x7f388c102ab0 2024-05-21T15:35:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.493+0000 7f61edba3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f102f0 0x5646c62e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.493+0000 7f61edba3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f184d0 0x5646c62e8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.493+0000 7f61edba3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f391b0 0x5646c635ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.493+0000 7f61edba3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01a9ab0 con 0x5646c5f102f0 2024-05-21T15:35:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.493+0000 7f61edba3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01a93e0 con 0x5646c5f391b0 2024-05-21T15:35:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.493+0000 7f61edba3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01a6620 con 0x5646c5f184d0 2024-05-21T15:35:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.493+0000 7f61edba3740 10 client.0 _ll_register_callbacks cb 0x7f61d800a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.493+0000 7f61dffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f184d0 0x5646c62e8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.493+0000 7f61e49b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f102f0 0x5646c62e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.493+0000 7f61dffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f184d0 0x5646c62e8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41360/0 (socket says 172.21.15.38:41360) 2024-05-21T15:35:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.493+0000 7f61dffff640 1 -- 172.21.15.38:0/3718773704 learned_addr learned my addr 172.21.15.38:0/3718773704 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.494+0000 7f61e51b8640 1 --2- 172.21.15.38:0/3718773704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f391b0 0x5646c635ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.494+0000 7f61dffff640 1 -- 172.21.15.38:0/3718773704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f391b0 msgr2=0x5646c635ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.494+0000 7f61dffff640 1 --2- 172.21.15.38:0/3718773704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f391b0 0x5646c635ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.494+0000 7f61dffff640 1 -- 172.21.15.38:0/3718773704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f102f0 msgr2=0x5646c62e86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.494+0000 7f61dffff640 1 --2- 172.21.15.38:0/3718773704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f102f0 0x5646c62e86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.494+0000 7f61dffff640 1 -- 172.21.15.38:0/3718773704 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c635f350 con 0x5646c5f184d0 2024-05-21T15:35:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.494+0000 7f61e49b7640 1 --2- 172.21.15.38:0/3718773704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f102f0 0x5646c62e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.495+0000 7f61dffff640 1 --2- 172.21.15.38:0/3718773704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f184d0 0x5646c62e8be0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0b7670 tx=0x7f61cc00d630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9104effd726a105 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.495+0000 7f61bffff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:35:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.495+0000 7f61bffff640 1 -- 172.21.15.38:0/3718773704 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0bad50 con 0x5646c5f184d0 2024-05-21T15:35:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.495+0000 7f61bffff640 1 -- 172.21.15.38:0/3718773704 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f61cc0db0c0 con 0x5646c5f184d0 2024-05-21T15:35:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.495+0000 7f61edba3740 1 -- 172.21.15.38:0/3718773704 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d4005520 con 0x5646c5f184d0 2024-05-21T15:35:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.495+0000 7f61bffff640 1 -- 172.21.15.38:0/3718773704 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f61cc0bab70 con 0x5646c5f184d0 2024-05-21T15:35:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.495+0000 7f61edba3740 10 client.25615 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.496+0000 7f61edba3740 1 -- 172.21.15.38:0/3718773704 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5646c63605d0 con 0x5646c5f184d0 2024-05-21T15:35:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.496+0000 7f61bffff640 1 -- 172.21.15.38:0/3718773704 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4005520 con 0x5646c5f184d0 2024-05-21T15:35:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.496+0000 7f61bffff640 1 -- 172.21.15.38:0/3718773704 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f61cc04c060 con 0x5646c5f184d0 2024-05-21T15:35:49.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.496+0000 7f3889ffb640 1 -- 172.21.15.38:0/2443347588 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f388001f020 con 0x7f388c102ab0 2024-05-21T15:35:49.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.496+0000 7f3899dd0740 1 Processor -- start 2024-05-21T15:35:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.496+0000 7f61edba3740 10 client.25615 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.496+0000 7f61edba3740 10 client.25615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.496+0000 7f61edba3740 20 client.25615 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.496+0000 7f61edba3740 1 --2- 172.21.15.38:0/3718773704 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c5ecd1c0 0x5646c6362f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.496+0000 7f61edba3740 4 client.25615 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:35:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.496+0000 7f61edba3740 1 -- 172.21.15.38:0/3718773704 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646c6244dc0 con 0x5646c5ecd1c0 2024-05-21T15:35:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.496+0000 7f61e49b7640 1 --2- 172.21.15.38:0/3718773704 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c5ecd1c0 0x5646c6362f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.496+0000 7f3899dd0740 1 -- start start 2024-05-21T15:35:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.497+0000 7f3899dd0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330492410 0x558330835880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.497+0000 7f3899dd0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558330469550 0x558330835e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.497+0000 7f3899dd0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 0x558330832530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.497+0000 7f3899dd0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558330796990 con 0x558330469550 2024-05-21T15:35:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.497+0000 7f3899dd0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55833073e5b0 con 0x558330492410 2024-05-21T15:35:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.497+0000 7f3899dd0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558330792950 con 0x558330471730 2024-05-21T15:35:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.497+0000 7f388bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 0x558330832530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.497+0000 7f38913ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330492410 0x558330835880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.497+0000 7f38913ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330492410 0x558330835880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:58864/0 (socket says 172.21.15.38:58864) 2024-05-21T15:35:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.497+0000 7f38913ee640 1 -- 172.21.15.38:0/247618368 learned_addr learned my addr 172.21.15.38:0/247618368 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.497+0000 7f61e49b7640 1 --2- 172.21.15.38:0/3718773704 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c5ecd1c0 0x5646c6362f80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=722b386e1e605726 server_cookie=e2c9daeed9650f34 in_seq=0 out_seq=0 2024-05-21T15:35:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.497+0000 7f61bffff640 10 client.25615 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:49.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.497+0000 7f3890bed640 1 --2- 172.21.15.38:0/247618368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558330469550 0x558330835e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.498+0000 7f3889ffb640 1 --2- 172.21.15.38:0/2443347588 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f387c0a2c90 0x7f387c0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.498+0000 7f38913ee640 1 -- 172.21.15.38:0/247618368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 msgr2=0x558330832530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.498+0000 7f38913ee640 1 --2- 172.21.15.38:0/247618368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 0x558330832530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.498+0000 7f38913ee640 1 -- 172.21.15.38:0/247618368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558330469550 msgr2=0x558330835e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.498+0000 7f38913ee640 1 --2- 172.21.15.38:0/247618368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558330469550 0x558330835e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.498+0000 7f38913ee640 1 -- 172.21.15.38:0/247618368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583305727e0 con 0x558330492410 2024-05-21T15:35:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.498+0000 7f3890bed640 1 --2- 172.21.15.38:0/247618368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558330469550 0x558330835e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:35:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.498+0000 7f38913ee640 1 --2- 172.21.15.38:0/2443347588 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f387c0a2c90 0x7f387c0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.498+0000 7f3889ffb640 1 -- 172.21.15.38:0/2443347588 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f38800cf0b0 con 0x7f388c102ab0 2024-05-21T15:35:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.498+0000 7f38913ee640 1 --2- 172.21.15.38:0/247618368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330492410 0x558330835880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f38840117e0 tx=0x7f3884011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b0c185de1f878c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.499+0000 7f38913ee640 1 --2- 172.21.15.38:0/2443347588 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f387c0a2c90 0x7f387c0a51a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3884022000 tx=0x7f3884023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.499+0000 7f3876ffd640 1 -- 172.21.15.38:0/247618368 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884017040 con 0x558330492410 2024-05-21T15:35:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.499+0000 7f3876ffd640 1 -- 172.21.15.38:0/247618368 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3884010bc0 con 0x558330492410 2024-05-21T15:35:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.499+0000 7f3876ffd640 1 -- 172.21.15.38:0/247618368 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884027420 con 0x558330492410 2024-05-21T15:35:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.500+0000 7f3899dd0740 1 -- 172.21.15.38:0/247618368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330492410 msgr2=0x558330835880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.500+0000 7f3899dd0740 1 --2- 172.21.15.38:0/247618368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330492410 0x558330835880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f38840117e0 tx=0x7f3884011ca0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.500+0000 7f3899dd0740 1 -- 172.21.15.38:0/247618368 shutdown_connections 2024-05-21T15:35:49.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.500+0000 7f3899dd0740 1 --2- 172.21.15.38:0/247618368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 0x558330832530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.500+0000 7f3899dd0740 1 --2- 172.21.15.38:0/247618368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558330469550 0x558330835e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.500+0000 7f3899dd0740 1 --2- 172.21.15.38:0/247618368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330492410 0x558330835880 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.501+0000 7f3899dd0740 1 -- 172.21.15.38:0/247618368 >> 172.21.15.38:0/247618368 conn(0x558330486560 msgr2=0x55833082de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.501+0000 7f3899dd0740 1 -- 172.21.15.38:0/247618368 shutdown_connections 2024-05-21T15:35:49.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.501+0000 7f3899dd0740 1 -- 172.21.15.38:0/247618368 wait complete. 2024-05-21T15:35:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.501+0000 7f61bffff640 1 -- 172.21.15.38:0/3718773704 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646c6244dc0 con 0x5646c5ecd1c0 2024-05-21T15:35:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.501+0000 7f61bffff640 10 client.25615 handle_command_reply: tid=0 2024-05-21T15:35:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.501+0000 7f61edba3740 1 client.25615 shutdown 2024-05-21T15:35:49.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.502+0000 7f3899dd0740 1 -- start start 2024-05-21T15:35:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.502+0000 7f61edba3740 1 -- 172.21.15.38:0/3718773704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f184d0 msgr2=0x5646c62e8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.502+0000 7f61edba3740 1 --2- 172.21.15.38:0/3718773704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f184d0 0x5646c62e8be0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0b7670 tx=0x7f61cc00d630 comp rx=0 tx=0).stop 2024-05-21T15:35:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.502+0000 7f61edba3740 1 -- 172.21.15.38:0/3718773704 shutdown_connections 2024-05-21T15:35:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.502+0000 7f61edba3740 1 --2- 172.21.15.38:0/3718773704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c5f391b0 0x5646c635ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.502+0000 7f61edba3740 1 --2- 172.21.15.38:0/3718773704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c5f184d0 0x5646c62e8be0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.502+0000 7f61edba3740 1 --2- 172.21.15.38:0/3718773704 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c5ecd1c0 0x5646c6362f80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.502+0000 7f61edba3740 1 --2- 172.21.15.38:0/3718773704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c5f102f0 0x5646c62e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.502+0000 7f61edba3740 1 -- 172.21.15.38:0/3718773704 >> 172.21.15.38:0/3718773704 conn(0x5646c5f2d300 msgr2=0x5646c62e2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.502+0000 7f61bffff640 0 client.25615 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:49.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.502+0000 7f3899dd0740 1 Processor -- start 2024-05-21T15:35:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.502+0000 7f61edba3740 1 -- 172.21.15.38:0/3718773704 shutdown_connections 2024-05-21T15:35:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.502+0000 7f61edba3740 1 -- 172.21.15.38:0/3718773704 wait complete. 2024-05-21T15:35:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.502+0000 7f61edba3740 20 client.25615 trim_cache size 0 max 16384 2024-05-21T15:35:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:35:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:35:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:35:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:35:49.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f3899dd0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330469550 0x55833083c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f3899dd0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558330471730 0x55833083c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f3899dd0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330492410 0x5583308414e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f3899dd0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558330751b40 con 0x558330471730 2024-05-21T15:35:49.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f3899dd0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558330836500 con 0x558330492410 2024-05-21T15:35:49.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f3899dd0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558330841a20 con 0x558330469550 2024-05-21T15:35:49.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f38913ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330492410 0x5583308414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f3899dd0740 10 client.0 _ll_register_callbacks cb 0x5583307460f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:49.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f388bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558330471730 0x55833083c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f38913ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330492410 0x5583308414e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:58876/0 (socket says 172.21.15.38:58876) 2024-05-21T15:35:49.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f38913ee640 1 -- 172.21.15.38:0/2053040966 learned_addr learned my addr 172.21.15.38:0/2053040966 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f3890bed640 1 --2- 172.21.15.38:0/2053040966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330469550 0x55833083c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f61e6c1b640 1 -- 172.21.15.38:0/1820323500 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f61d00a2cb0 msgr2=0x7f61d00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f61e6c1b640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f61d00a2cb0 0x7f61d00a51c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f61d8003fd0 tx=0x7f61d800f040 comp rx=0 tx=0).stop 2024-05-21T15:35:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f61e6c1b640 1 -- 172.21.15.38:0/1820323500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e0107070 msgr2=0x7f61e0197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f61e6c1b640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e0107070 0x7f61e0197640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f61cc008000 tx=0x7f61cc02adc0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f61e6c1b640 1 -- 172.21.15.38:0/1820323500 shutdown_connections 2024-05-21T15:35:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f61e6c1b640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f61d00a2cb0 0x7f61d00a51c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f61e6c1b640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61e019c540 0x7f61e019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f61e6c1b640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e0107070 0x7f61e0197640 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f61e6c1b640 1 --2- 172.21.15.38:0/1820323500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61e0101130 0x7f61e0197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f61e6c1b640 1 -- 172.21.15.38:0/1820323500 >> 172.21.15.38:0/1820323500 conn(0x7f61e00fc300 msgr2=0x7f61e0103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f61e6c1b640 1 -- 172.21.15.38:0/1820323500 shutdown_connections 2024-05-21T15:35:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:35:49.503+0000 7f61e6c1b640 1 -- 172.21.15.38:0/1820323500 wait complete. 2024-05-21T15:35:49.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.504+0000 7f38913ee640 1 -- 172.21.15.38:0/2053040966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330469550 msgr2=0x55833083c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.504+0000 7f38913ee640 1 --2- 172.21.15.38:0/2053040966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330469550 0x55833083c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.504+0000 7f38913ee640 1 -- 172.21.15.38:0/2053040966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558330471730 msgr2=0x55833083c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.504+0000 7f38913ee640 1 --2- 172.21.15.38:0/2053040966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558330471730 0x55833083c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.504+0000 7f38913ee640 1 -- 172.21.15.38:0/2053040966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583305d2f70 con 0x558330492410 2024-05-21T15:35:49.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.504+0000 7f3890bed640 1 --2- 172.21.15.38:0/2053040966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330469550 0x55833083c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:49.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.504+0000 7f388bfff640 1 --2- 172.21.15.38:0/2053040966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558330471730 0x55833083c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:49.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.504+0000 7f38913ee640 1 --2- 172.21.15.38:0/2053040966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330492410 0x5583308414e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3884010fd0 tx=0x7f3884010a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50e3752946b57411 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.504+0000 7f386bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:35:49.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.504+0000 7f386bfff640 1 -- 172.21.15.38:0/2053040966 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884037060 con 0x558330492410 2024-05-21T15:35:49.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.504+0000 7f386bfff640 1 -- 172.21.15.38:0/2053040966 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f3884004bf0 con 0x558330492410 2024-05-21T15:35:49.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.504+0000 7f3899dd0740 1 -- 172.21.15.38:0/2053040966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3884010bc0 con 0x558330492410 2024-05-21T15:35:49.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.505+0000 7f386bfff640 1 -- 172.21.15.38:0/2053040966 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3884004df0 con 0x558330492410 2024-05-21T15:35:49.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.505+0000 7f3899dd0740 10 client.25550 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.505+0000 7f3899dd0740 1 -- 172.21.15.38:0/2053040966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55833046a170 con 0x558330492410 2024-05-21T15:35:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.506+0000 7f386bfff640 1 -- 172.21.15.38:0/2053040966 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3884010bc0 con 0x558330492410 2024-05-21T15:35:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.506+0000 7f386bfff640 1 -- 172.21.15.38:0/2053040966 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f388401b070 con 0x558330492410 2024-05-21T15:35:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.506+0000 7f3899dd0740 10 client.25550 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.506+0000 7f3899dd0740 10 client.25550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.506+0000 7f3899dd0740 20 client.25550 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.506+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2053040966 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5583304511f0 0x55833085a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.506+0000 7f3899dd0740 4 client.25550 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.506+0000 7f3899dd0740 1 -- 172.21.15.38:0/2053040966 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583307a40c0 con 0x5583304511f0 2024-05-21T15:35:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.507+0000 7f3890bed640 1 --2- 172.21.15.38:0/2053040966 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5583304511f0 0x55833085a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.508+0000 7f3890bed640 1 --2- 172.21.15.38:0/2053040966 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5583304511f0 0x55833085a8a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc30f51cf3977f9c server_cookie=8bff5bdea3d5c4f6 in_seq=0 out_seq=0 2024-05-21T15:35:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.508+0000 7f386bfff640 10 client.25550 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.508+0000 7f386bfff640 1 -- 172.21.15.38:0/2053040966 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583307a40c0 con 0x5583304511f0 2024-05-21T15:35:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.508+0000 7f386bfff640 10 client.25550 handle_command_reply: tid=0 2024-05-21T15:35:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.508+0000 7f3899dd0740 1 client.25550 shutdown 2024-05-21T15:35:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.509+0000 7f3899dd0740 1 -- 172.21.15.38:0/2053040966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330492410 msgr2=0x5583308414e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.509+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2053040966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330492410 0x5583308414e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3884010fd0 tx=0x7f3884010a30 comp rx=0 tx=0).stop 2024-05-21T15:35:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.509+0000 7f3899dd0740 1 -- 172.21.15.38:0/2053040966 shutdown_connections 2024-05-21T15:35:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.509+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2053040966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330492410 0x5583308414e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.509+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2053040966 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5583304511f0 0x55833085a8a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.509+0000 7f386bfff640 0 client.25550 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.509+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2053040966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558330471730 0x55833083c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.509+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2053040966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330469550 0x55833083c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.509+0000 7f3899dd0740 1 -- 172.21.15.38:0/2053040966 >> 172.21.15.38:0/2053040966 conn(0x558330486560 msgr2=0x55833082ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.509+0000 7f3899dd0740 1 -- 172.21.15.38:0/2053040966 shutdown_connections 2024-05-21T15:35:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.509+0000 7f3899dd0740 1 -- 172.21.15.38:0/2053040966 wait complete. 2024-05-21T15:35:49.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.510+0000 7f3899dd0740 20 client.25550 trim_cache size 0 max 16384 2024-05-21T15:35:49.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.546+0000 7f3899dd0740 1 Processor -- start 2024-05-21T15:35:49.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.546+0000 7f3899dd0740 1 -- start start 2024-05-21T15:35:49.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3899dd0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583304511f0 0x558330837580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3899dd0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330469550 0x558330841b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3899dd0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 0x558330830c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3899dd0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f387000e600 con 0x5583304511f0 2024-05-21T15:35:49.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3899dd0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3870010140 con 0x558330469550 2024-05-21T15:35:49.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3899dd0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f387000ecd0 con 0x558330471730 2024-05-21T15:35:49.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3890bed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583304511f0 0x558330837580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3890bed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583304511f0 0x558330837580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49658/0 (socket says 172.21.15.38:49658) 2024-05-21T15:35:49.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3890bed640 1 -- 172.21.15.38:0/2753876581 learned_addr learned my addr 172.21.15.38:0/2753876581 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f388bfff640 1 --2- 172.21.15.38:0/2753876581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330469550 0x558330841b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f38913ee640 1 --2- 172.21.15.38:0/2753876581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 0x558330830c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3890bed640 1 -- 172.21.15.38:0/2753876581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 msgr2=0x558330830c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3890bed640 1 --2- 172.21.15.38:0/2753876581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 0x558330830c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3890bed640 1 -- 172.21.15.38:0/2753876581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330469550 msgr2=0x558330841b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3890bed640 1 --2- 172.21.15.38:0/2753876581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330469550 0x558330841b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3890bed640 1 -- 172.21.15.38:0/2753876581 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558330828ea0 con 0x5583304511f0 2024-05-21T15:35:49.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f38913ee640 1 --2- 172.21.15.38:0/2753876581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 0x558330830c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:49.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.547+0000 7f3890bed640 1 --2- 172.21.15.38:0/2753876581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583304511f0 0x558330837580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3880097cc0 tx=0x7f38800cf300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf86556a7ab9d20 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.548+0000 7f38767fc640 1 -- 172.21.15.38:0/2753876581 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388009fc10 con 0x5583304511f0 2024-05-21T15:35:49.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.548+0000 7f38767fc640 1 -- 172.21.15.38:0/2753876581 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38800a44d0 con 0x5583304511f0 2024-05-21T15:35:49.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.548+0000 7f38767fc640 1 -- 172.21.15.38:0/2753876581 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388009bac0 con 0x5583304511f0 2024-05-21T15:35:49.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.549+0000 7f3899dd0740 1 -- 172.21.15.38:0/2753876581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583304511f0 msgr2=0x558330837580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.549+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2753876581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583304511f0 0x558330837580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3880097cc0 tx=0x7f38800cf300 comp rx=0 tx=0).stop 2024-05-21T15:35:49.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.549+0000 7f3899dd0740 1 -- 172.21.15.38:0/2753876581 shutdown_connections 2024-05-21T15:35:49.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.549+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2753876581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 0x558330830c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.549+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2753876581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330469550 0x558330841b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.549+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2753876581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583304511f0 0x558330837580 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.549+0000 7f3899dd0740 1 -- 172.21.15.38:0/2753876581 >> 172.21.15.38:0/2753876581 conn(0x558330486560 msgr2=0x558330841400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.549+0000 7f3899dd0740 1 -- 172.21.15.38:0/2753876581 shutdown_connections 2024-05-21T15:35:49.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.549+0000 7f3899dd0740 1 -- 172.21.15.38:0/2753876581 wait complete. 2024-05-21T15:35:49.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.549+0000 7f3899dd0740 1 -- start start 2024-05-21T15:35:49.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.550+0000 7f3899dd0740 1 Processor -- start 2024-05-21T15:35:49.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.550+0000 7f3899dd0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583304511f0 0x558330868f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.550+0000 7f3899dd0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330469550 0x5583308694b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.550+0000 7f3899dd0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 0x5583308c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.550+0000 7f3899dd0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f387000fa70 con 0x5583304511f0 2024-05-21T15:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.550+0000 7f3899dd0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f387000f3a0 con 0x558330469550 2024-05-21T15:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.550+0000 7f3899dd0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f387000c5e0 con 0x558330471730 2024-05-21T15:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.550+0000 7f3899dd0740 10 client.0 _ll_register_callbacks cb 0x7f38840093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.550+0000 7f3890bed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583304511f0 0x558330868f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.550+0000 7f38913ee640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 0x5583308c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.550+0000 7f38913ee640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 0x5583308c08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41404/0 (socket says 172.21.15.38:41404) 2024-05-21T15:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.550+0000 7f388bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330469550 0x5583308694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.550+0000 7f38913ee640 1 -- 172.21.15.38:0/2158318890 learned_addr learned my addr 172.21.15.38:0/2158318890 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.550+0000 7f3890bed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583304511f0 0x558330868f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49664/0 (socket says 172.21.15.38:49664) 2024-05-21T15:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.551+0000 7f388bfff640 1 -- 172.21.15.38:0/2158318890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 msgr2=0x5583308c08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.551+0000 7f388bfff640 1 --2- 172.21.15.38:0/2158318890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 0x5583308c08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.551+0000 7f388bfff640 1 -- 172.21.15.38:0/2158318890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583304511f0 msgr2=0x558330868f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.551+0000 7f388bfff640 1 --2- 172.21.15.38:0/2158318890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583304511f0 0x558330868f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.551+0000 7f388bfff640 1 -- 172.21.15.38:0/2158318890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558330869af0 con 0x558330469550 2024-05-21T15:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.551+0000 7f388bfff640 1 --2- 172.21.15.38:0/2158318890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330469550 0x5583308694b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f387802a930 tx=0x7f38780053b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da9ac92a65b25682 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.551+0000 7f386bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.551+0000 7f386bfff640 1 -- 172.21.15.38:0/2158318890 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878035040 con 0x558330469550 2024-05-21T15:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.552+0000 7f386bfff640 1 -- 172.21.15.38:0/2158318890 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f3878004760 con 0x558330469550 2024-05-21T15:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.552+0000 7f3899dd0740 1 -- 172.21.15.38:0/2158318890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38800a44d0 con 0x558330469550 2024-05-21T15:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.552+0000 7f386bfff640 1 -- 172.21.15.38:0/2158318890 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3878004960 con 0x558330469550 2024-05-21T15:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.552+0000 7f3899dd0740 10 client.25556 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.552+0000 7f3899dd0740 1 -- 172.21.15.38:0/2158318890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5583308c1f20 con 0x558330469550 2024-05-21T15:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.552+0000 7f386bfff640 1 -- 172.21.15.38:0/2158318890 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38800a44d0 con 0x558330469550 2024-05-21T15:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.552+0000 7f386bfff640 1 -- 172.21.15.38:0/2158318890 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f38780035f0 con 0x558330469550 2024-05-21T15:35:49.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.552+0000 7f3899dd0740 10 client.25556 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:49.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.552+0000 7f3899dd0740 10 client.25556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:49.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.552+0000 7f3899dd0740 20 client.25556 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:49.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.552+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2158318890 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x558330492410 0x5583308c4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.552+0000 7f3899dd0740 4 client.25556 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:35:49.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.552+0000 7f3899dd0740 1 -- 172.21.15.38:0/2158318890 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55833079e410 con 0x558330492410 2024-05-21T15:35:49.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.553+0000 7f3890bed640 1 --2- 172.21.15.38:0/2158318890 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x558330492410 0x5583308c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.553+0000 7f386bfff640 10 client.25556 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:49.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.553+0000 7f3890bed640 1 --2- 172.21.15.38:0/2158318890 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x558330492410 0x5583308c4940 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f959365a3a9dd791 server_cookie=d5872397604a3bcf in_seq=0 out_seq=0 2024-05-21T15:35:49.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.554+0000 7f386bfff640 1 -- 172.21.15.38:0/2158318890 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55833079e410 con 0x558330492410 2024-05-21T15:35:49.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.554+0000 7f386bfff640 10 client.25556 handle_command_reply: tid=0 2024-05-21T15:35:49.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.554+0000 7f3899dd0740 1 client.25556 shutdown 2024-05-21T15:35:49.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.554+0000 7f3899dd0740 1 -- 172.21.15.38:0/2158318890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330469550 msgr2=0x5583308694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.554+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2158318890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330469550 0x5583308694b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f387802a930 tx=0x7f38780053b0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.554+0000 7f3899dd0740 1 -- 172.21.15.38:0/2158318890 shutdown_connections 2024-05-21T15:35:49.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.554+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2158318890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558330471730 0x5583308c08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.555+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2158318890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558330469550 0x5583308694b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.555+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2158318890 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x558330492410 0x5583308c4940 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.555+0000 7f386bfff640 0 client.25556 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:49.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.555+0000 7f3899dd0740 1 --2- 172.21.15.38:0/2158318890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583304511f0 0x558330868f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.555+0000 7f3899dd0740 1 -- 172.21.15.38:0/2158318890 >> 172.21.15.38:0/2158318890 conn(0x558330486560 msgr2=0x55833082c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.555+0000 7f3899dd0740 1 -- 172.21.15.38:0/2158318890 shutdown_connections 2024-05-21T15:35:49.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.555+0000 7f3899dd0740 1 -- 172.21.15.38:0/2158318890 wait complete. 2024-05-21T15:35:49.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.555+0000 7f3899dd0740 20 client.25556 trim_cache size 0 max 16384 2024-05-21T15:35:49.558 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:35:49.558 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-05-21T15:35:49.558 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "07660293-4012-43cc-b513-ff85cf6c240c", 2024-05-21T15:35:49.558 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-05-21T15:35:49.558 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:35:49.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.556+0000 7f3892e51640 1 -- 172.21.15.38:0/2443347588 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f387c0a2c90 msgr2=0x7f387c0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.556+0000 7f3892e51640 1 --2- 172.21.15.38:0/2443347588 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f387c0a2c90 0x7f387c0a51a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3884022000 tx=0x7f3884023000 comp rx=0 tx=0).stop 2024-05-21T15:35:49.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.556+0000 7f3892e51640 1 -- 172.21.15.38:0/2443347588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f388c102ab0 msgr2=0x7f388c199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.556+0000 7f3892e51640 1 --2- 172.21.15.38:0/2443347588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f388c102ab0 0x7f388c199250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f388000e980 tx=0x7f388000ee40 comp rx=0 tx=0).stop 2024-05-21T15:35:49.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.556+0000 7f3892e51640 1 -- 172.21.15.38:0/2443347588 shutdown_connections 2024-05-21T15:35:49.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.556+0000 7f3892e51640 1 --2- 172.21.15.38:0/2443347588 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f387c0a2c90 0x7f387c0a51a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.556+0000 7f3892e51640 1 --2- 172.21.15.38:0/2443347588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388c1089f0 0x7f388c1a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.556+0000 7f3892e51640 1 --2- 172.21.15.38:0/2443347588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f388c103cf0 0x7f388c199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.556+0000 7f3892e51640 1 --2- 172.21.15.38:0/2443347588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f388c102ab0 0x7f388c199250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.556+0000 7f3892e51640 1 -- 172.21.15.38:0/2443347588 >> 172.21.15.38:0/2443347588 conn(0x7f388c0fdca0 msgr2=0x7f388c1049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.556+0000 7f3892e51640 1 -- 172.21.15.38:0/2443347588 shutdown_connections 2024-05-21T15:35:49.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.556+0000 7f3892e51640 1 -- 172.21.15.38:0/2443347588 wait complete. 2024-05-21T15:35:49.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:35:49.571 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:35:49.636 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:49 smithi038 ceph-mon[31995]: pgmap v393: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T15:35:49.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:49 smithi053 ceph-mon[36496]: pgmap v393: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T15:35:49.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.733+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2433115726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7b01078c0 msgr2=0x7fe7b0109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.733+0000 7fe7b7c08640 1 --2- 172.21.15.38:0/2433115726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7b01078c0 0x7fe7b0109ce0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009ca0 tx=0x7fe7a802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.733+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2433115726 shutdown_connections 2024-05-21T15:35:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.733+0000 7fe7b7c08640 1 --2- 172.21.15.38:0/2433115726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7b01078c0 0x7fe7b0109ce0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.733+0000 7fe7b7c08640 1 --2- 172.21.15.38:0/2433115726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7b0102bc0 0x7fe7b0102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.733+0000 7fe7b7c08640 1 --2- 172.21.15.38:0/2433115726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0101980 0x7fe7b0102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.733+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2433115726 >> 172.21.15.38:0/2433115726 conn(0x7fe7b00fcb70 msgr2=0x7fe7b00ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.733+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2433115726 shutdown_connections 2024-05-21T15:35:49.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.733+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2433115726 wait complete. 2024-05-21T15:35:49.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.734+0000 7fe7b7c08640 1 Processor -- start 2024-05-21T15:35:49.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.734+0000 7fe7b7c08640 1 -- start start 2024-05-21T15:35:49.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.734+0000 7fe7b7c08640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0101980 0x7fe7b019b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.734+0000 7fe7b7c08640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7b0102bc0 0x7fe7b019b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.734+0000 7fe7b7c08640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7b01078c0 0x7fe7b01a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.734+0000 7fe7b51a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7b0102bc0 0x7fe7b019b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.734+0000 7fe7b51a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7b0102bc0 0x7fe7b019b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49686/0 (socket says 172.21.15.38:49686) 2024-05-21T15:35:49.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.734+0000 7fe7b51a3640 1 -- 172.21.15.38:0/2275497085 learned_addr learned my addr 172.21.15.38:0/2275497085 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.734+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2275497085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b010c4f0 con 0x7fe7b0102bc0 2024-05-21T15:35:49.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.734+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2275497085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b010c370 con 0x7fe7b01078c0 2024-05-21T15:35:49.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.734+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2275497085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b010c670 con 0x7fe7b0101980 2024-05-21T15:35:49.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.734+0000 7fe7b59a4640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0101980 0x7fe7b019b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.735+0000 7fe7b61a5640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7b01078c0 0x7fe7b01a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.735+0000 7fe7b51a3640 1 -- 172.21.15.38:0/2275497085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0101980 msgr2=0x7fe7b019b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.735+0000 7fe7b51a3640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0101980 0x7fe7b019b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.735+0000 7fe7b51a3640 1 -- 172.21.15.38:0/2275497085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7b01078c0 msgr2=0x7fe7b01a28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.735+0000 7fe7b51a3640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7b01078c0 0x7fe7b01a28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.735+0000 7fe7b51a3640 1 -- 172.21.15.38:0/2275497085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b01a2f50 con 0x7fe7b0102bc0 2024-05-21T15:35:49.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.735+0000 7fe7b61a5640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7b01078c0 0x7fe7b01a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:49.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.735+0000 7fe7b51a3640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7b0102bc0 0x7fe7b019b9c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000d950 tx=0x7fe7a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.735+0000 7fe7a6ffd640 1 -- 172.21.15.38:0/2275497085 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0017030 con 0x7fe7b0102bc0 2024-05-21T15:35:49.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.735+0000 7fe7a6ffd640 1 -- 172.21.15.38:0/2275497085 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7a0004020 con 0x7fe7b0102bc0 2024-05-21T15:35:49.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.735+0000 7fe7a6ffd640 1 -- 172.21.15.38:0/2275497085 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00047a0 con 0x7fe7b0102bc0 2024-05-21T15:35:49.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.735+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2275497085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b01a3240 con 0x7fe7b0102bc0 2024-05-21T15:35:49.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.736+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2275497085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b01a3740 con 0x7fe7b0102bc0 2024-05-21T15:35:49.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.738+0000 7fe7a6ffd640 1 -- 172.21.15.38:0/2275497085 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe7a00041d0 con 0x7fe7b0102bc0 2024-05-21T15:35:49.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.738+0000 7fe7beb81740 1 Processor -- start 2024-05-21T15:35:49.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.739+0000 7fe7beb81740 1 -- start start 2024-05-21T15:35:49.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.739+0000 7fe7beb81740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee9797d390 0x55ee97d1cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.739+0000 7fe7beb81740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee979544d0 0x55ee97d1d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.739+0000 7fe7beb81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9795c6b0 0x55ee97d24030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.739+0000 7fe7beb81740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ee97c81d00 con 0x55ee9797d390 2024-05-21T15:35:49.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.739+0000 7fe7beb81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ee97c294c0 con 0x55ee9795c6b0 2024-05-21T15:35:49.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.739+0000 7fe7beb81740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ee97c7dcc0 con 0x55ee979544d0 2024-05-21T15:35:49.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.739+0000 7fe7b61a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee9797d390 0x55ee97d1cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.739+0000 7fe7b61a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee9797d390 0x55ee97d1cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49688/0 (socket says 172.21.15.38:49688) 2024-05-21T15:35:49.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.739+0000 7fe7b61a5640 1 -- 172.21.15.38:0/1416645521 learned_addr learned my addr 172.21.15.38:0/1416645521 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.739+0000 7fe7b59a4640 1 --2- 172.21.15.38:0/1416645521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee979544d0 0x55ee97d1d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.739+0000 7fe7b51a3640 1 --2- 172.21.15.38:0/1416645521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9795c6b0 0x55ee97d24030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.740+0000 7fe7a6ffd640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe79c0a2b70 0x7fe79c0a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.740+0000 7fe7b59a4640 1 -- 172.21.15.38:0/1416645521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9795c6b0 msgr2=0x55ee97d24030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.740+0000 7fe7b59a4640 1 --2- 172.21.15.38:0/1416645521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9795c6b0 0x55ee97d24030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.740+0000 7fe7b59a4640 1 -- 172.21.15.38:0/1416645521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee9797d390 msgr2=0x55ee97d1cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.740+0000 7fe7b59a4640 1 --2- 172.21.15.38:0/1416645521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee9797d390 0x55ee97d1cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.740+0000 7fe7b59a4640 1 -- 172.21.15.38:0/1416645521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee97a5f7d0 con 0x55ee979544d0 2024-05-21T15:35:49.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.740+0000 7fe7a6ffd640 1 -- 172.21.15.38:0/2275497085 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe7a00cea70 con 0x7fe7b0102bc0 2024-05-21T15:35:49.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.740+0000 7fe7b59a4640 1 --2- 172.21.15.38:0/1416645521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee979544d0 0x55ee97d1d130 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0101250 tx=0x7fe7ac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=573a44dd8816a099 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.740+0000 7fe7b61a5640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe79c0a2b70 0x7fe79c0a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.740+0000 7fe79bfff640 1 -- 172.21.15.38:0/1416645521 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac004230 con 0x55ee979544d0 2024-05-21T15:35:49.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.741+0000 7fe79bfff640 1 -- 172.21.15.38:0/1416645521 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7ac009e10 con 0x55ee979544d0 2024-05-21T15:35:49.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.741+0000 7fe79bfff640 1 -- 172.21.15.38:0/1416645521 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac019790 con 0x55ee979544d0 2024-05-21T15:35:49.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.742+0000 7fe7beb81740 1 -- 172.21.15.38:0/1416645521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee979544d0 msgr2=0x55ee97d1d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.742+0000 7fe7beb81740 1 --2- 172.21.15.38:0/1416645521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee979544d0 0x55ee97d1d130 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0101250 tx=0x7fe7ac011810 comp rx=0 tx=0).stop 2024-05-21T15:35:49.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.742+0000 7fe7b61a5640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe79c0a2b70 0x7fe79c0a5080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80094e0 tx=0x7fe7a80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.742+0000 7fe7beb81740 1 -- 172.21.15.38:0/1416645521 shutdown_connections 2024-05-21T15:35:49.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.742+0000 7fe7beb81740 1 --2- 172.21.15.38:0/1416645521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9795c6b0 0x55ee97d24030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.742+0000 7fe7beb81740 1 --2- 172.21.15.38:0/1416645521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee979544d0 0x55ee97d1d130 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.742+0000 7fe7beb81740 1 --2- 172.21.15.38:0/1416645521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee9797d390 0x55ee97d1cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.742+0000 7fe7beb81740 1 -- 172.21.15.38:0/1416645521 >> 172.21.15.38:0/1416645521 conn(0x55ee979714e0 msgr2=0x55ee97d18540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.742+0000 7fe7beb81740 1 -- 172.21.15.38:0/1416645521 shutdown_connections 2024-05-21T15:35:49.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.742+0000 7fe7beb81740 1 -- 172.21.15.38:0/1416645521 wait complete. 2024-05-21T15:35:49.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.742+0000 7fe7beb81740 1 -- start start 2024-05-21T15:35:49.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.742+0000 7fe7beb81740 1 Processor -- start 2024-05-21T15:35:49.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.743+0000 7fe7beb81740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee979544d0 0x55ee97d26aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.743+0000 7fe7beb81740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9795c6b0 0x55ee97d26fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.743+0000 7fe7beb81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9797d390 0x55ee97d2df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.743+0000 7fe7beb81740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ee97d24a80 con 0x55ee979544d0 2024-05-21T15:35:49.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.743+0000 7fe7beb81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ee97c3cb90 con 0x55ee9797d390 2024-05-21T15:35:49.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.743+0000 7fe7beb81740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ee97d24c00 con 0x55ee9795c6b0 2024-05-21T15:35:49.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.743+0000 7fe7b59a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee979544d0 0x55ee97d26aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.743+0000 7fe7b59a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee979544d0 0x55ee97d26aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49694/0 (socket says 172.21.15.38:49694) 2024-05-21T15:35:49.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.743+0000 7fe7b61a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9797d390 0x55ee97d2df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.743+0000 7fe7b59a4640 1 -- 172.21.15.38:0/1616800073 learned_addr learned my addr 172.21.15.38:0/1616800073 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.743+0000 7fe7b51a3640 1 --2- 172.21.15.38:0/1616800073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9795c6b0 0x55ee97d26fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.743+0000 7fe7beb81740 10 client.0 _ll_register_callbacks cb 0x55ee97c78da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:49.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.744+0000 7fe7b61a5640 1 -- 172.21.15.38:0/1616800073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9795c6b0 msgr2=0x55ee97d26fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.744+0000 7fe7b61a5640 1 --2- 172.21.15.38:0/1616800073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9795c6b0 0x55ee97d26fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.744+0000 7fe7b61a5640 1 -- 172.21.15.38:0/1616800073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee979544d0 msgr2=0x55ee97d26aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.744+0000 7fe7b61a5640 1 --2- 172.21.15.38:0/1616800073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee979544d0 0x55ee97d26aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.744+0000 7fe7b61a5640 1 -- 172.21.15.38:0/1616800073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee979953f0 con 0x55ee9797d390 2024-05-21T15:35:49.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.744+0000 7fe7b59a4640 1 --2- 172.21.15.38:0/1616800073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee979544d0 0x55ee97d26aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:49.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.744+0000 7fe7b61a5640 1 --2- 172.21.15.38:0/1616800073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9797d390 0x55ee97d2df40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009ca0 tx=0x7fe7a802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47c106fa045f4277 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.744+0000 7fe7997fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:35:49.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.744+0000 7fe7997fa640 1 -- 172.21.15.38:0/1616800073 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8004590 con 0x55ee9797d390 2024-05-21T15:35:49.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.744+0000 7fe7997fa640 1 -- 172.21.15.38:0/1616800073 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe7a8004730 con 0x55ee9797d390 2024-05-21T15:35:49.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.744+0000 7fe7beb81740 1 -- 172.21.15.38:0/1616800073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7ac009e10 con 0x55ee9797d390 2024-05-21T15:35:49.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.745+0000 7fe7997fa640 1 -- 172.21.15.38:0/1616800073 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7a802f610 con 0x55ee9797d390 2024-05-21T15:35:49.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.745+0000 7fe7beb81740 10 client.25568 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:49.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.745+0000 7fe7beb81740 1 -- 172.21.15.38:0/1616800073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee979550f0 con 0x55ee9797d390 2024-05-21T15:35:49.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.745+0000 7fe7997fa640 1 -- 172.21.15.38:0/1616800073 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7a8036040 con 0x55ee9797d390 2024-05-21T15:35:49.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.746+0000 7fe7997fa640 1 -- 172.21.15.38:0/1616800073 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe7a8050020 con 0x55ee9797d390 2024-05-21T15:35:49.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.746+0000 7fe7beb81740 10 client.25568 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:49.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.746+0000 7fe7beb81740 10 client.25568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:49.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.746+0000 7fe7beb81740 20 client.25568 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:49.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.746+0000 7fe7beb81740 1 --2- 172.21.15.38:0/1616800073 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ee9793bf60 0x55ee97d34720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.746+0000 7fe7beb81740 4 client.25568 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:35:49.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.746+0000 7fe7beb81740 1 -- 172.21.15.38:0/1616800073 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe7ac009e10 con 0x55ee9793bf60 2024-05-21T15:35:49.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.746+0000 7fe7b59a4640 1 --2- 172.21.15.38:0/1616800073 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ee9793bf60 0x55ee97d34720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.747+0000 7fe7b59a4640 1 --2- 172.21.15.38:0/1616800073 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ee9793bf60 0x55ee97d34720 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6828c786fa202ea server_cookie=11b197eafe137ebb in_seq=0 out_seq=0 2024-05-21T15:35:49.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.747+0000 7fe7997fa640 10 client.25568 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:49.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.748+0000 7fe7997fa640 1 -- 172.21.15.38:0/1616800073 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fe7ac009e10 con 0x55ee9793bf60 2024-05-21T15:35:49.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.748+0000 7fe7997fa640 10 client.25568 handle_command_reply: tid=0 2024-05-21T15:35:49.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.748+0000 7fe7beb81740 1 client.25568 shutdown 2024-05-21T15:35:49.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.748+0000 7fe7beb81740 1 -- 172.21.15.38:0/1616800073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9797d390 msgr2=0x55ee97d2df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.748+0000 7fe7beb81740 1 --2- 172.21.15.38:0/1616800073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9797d390 0x55ee97d2df40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009ca0 tx=0x7fe7a802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.748+0000 7fe7beb81740 1 -- 172.21.15.38:0/1616800073 shutdown_connections 2024-05-21T15:35:49.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.748+0000 7fe7beb81740 1 --2- 172.21.15.38:0/1616800073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9797d390 0x55ee97d2df40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.748+0000 7fe7beb81740 1 --2- 172.21.15.38:0/1616800073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9795c6b0 0x55ee97d26fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.748+0000 7fe7beb81740 1 --2- 172.21.15.38:0/1616800073 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ee9793bf60 0x55ee97d34720 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.748+0000 7fe7beb81740 1 --2- 172.21.15.38:0/1616800073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee979544d0 0x55ee97d26aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.748+0000 7fe7997fa640 0 client.25568 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:49.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.749+0000 7fe7beb81740 1 -- 172.21.15.38:0/1616800073 >> 172.21.15.38:0/1616800073 conn(0x55ee979714e0 msgr2=0x55ee97d175c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.749+0000 7fe7beb81740 1 -- 172.21.15.38:0/1616800073 shutdown_connections 2024-05-21T15:35:49.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.749+0000 7fe7beb81740 1 -- 172.21.15.38:0/1616800073 wait complete. 2024-05-21T15:35:49.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.749+0000 7fe7beb81740 20 client.25568 trim_cache size 0 max 16384 2024-05-21T15:35:49.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:49 smithi045 ceph-mon[36409]: pgmap v393: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T15:35:49.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.785+0000 7fe7beb81740 1 Processor -- start 2024-05-21T15:35:49.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.785+0000 7fe7beb81740 1 -- start start 2024-05-21T15:35:49.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.785+0000 7fe7beb81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9793bf60 0x55ee97d260d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.785+0000 7fe7beb81740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee979544d0 0x55ee97d2de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.785+0000 7fe7beb81740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9795c6b0 0x55ee97d1ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.785+0000 7fe7beb81740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b01ac630 con 0x55ee979544d0 2024-05-21T15:35:49.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.785+0000 7fe7beb81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b01ae170 con 0x55ee9793bf60 2024-05-21T15:35:49.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.785+0000 7fe7beb81740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b01acd00 con 0x55ee9795c6b0 2024-05-21T15:35:49.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.785+0000 7fe7b51a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee979544d0 0x55ee97d2de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.785+0000 7fe7b59a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9793bf60 0x55ee97d260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.785+0000 7fe7b51a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee979544d0 0x55ee97d2de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49710/0 (socket says 172.21.15.38:49710) 2024-05-21T15:35:49.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.785+0000 7fe7b51a3640 1 -- 172.21.15.38:0/1134011009 learned_addr learned my addr 172.21.15.38:0/1134011009 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.785+0000 7fe7b61a5640 1 --2- 172.21.15.38:0/1134011009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9795c6b0 0x55ee97d1ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.786+0000 7fe7b59a4640 1 -- 172.21.15.38:0/1134011009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9795c6b0 msgr2=0x55ee97d1ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.786+0000 7fe7b59a4640 1 --2- 172.21.15.38:0/1134011009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9795c6b0 0x55ee97d1ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.786+0000 7fe7b59a4640 1 -- 172.21.15.38:0/1134011009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee979544d0 msgr2=0x55ee97d2de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.786+0000 7fe7b59a4640 1 --2- 172.21.15.38:0/1134011009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee979544d0 0x55ee97d2de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.786+0000 7fe7b59a4640 1 -- 172.21.15.38:0/1134011009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee97d2e3a0 con 0x55ee9793bf60 2024-05-21T15:35:49.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.786+0000 7fe7b61a5640 1 --2- 172.21.15.38:0/1134011009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9795c6b0 0x55ee97d1ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:49.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.786+0000 7fe7b51a3640 1 --2- 172.21.15.38:0/1134011009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee979544d0 0x55ee97d2de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:35:49.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.786+0000 7fe7b59a4640 1 --2- 172.21.15.38:0/1134011009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9793bf60 0x55ee97d260d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55ee97d34e90 tx=0x7fe7ac009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9aa00f289a318d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.786+0000 7fe79b7fe640 1 -- 172.21.15.38:0/1134011009 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac005040 con 0x55ee9793bf60 2024-05-21T15:35:49.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.786+0000 7fe79b7fe640 1 -- 172.21.15.38:0/1134011009 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7ac0036f0 con 0x55ee9793bf60 2024-05-21T15:35:49.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.786+0000 7fe79b7fe640 1 -- 172.21.15.38:0/1134011009 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac004680 con 0x55ee9793bf60 2024-05-21T15:35:49.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.787+0000 7fe7beb81740 1 -- 172.21.15.38:0/1134011009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9793bf60 msgr2=0x55ee97d260d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.787+0000 7fe7beb81740 1 --2- 172.21.15.38:0/1134011009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9793bf60 0x55ee97d260d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55ee97d34e90 tx=0x7fe7ac009310 comp rx=0 tx=0).stop 2024-05-21T15:35:49.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.787+0000 7fe7beb81740 1 -- 172.21.15.38:0/1134011009 shutdown_connections 2024-05-21T15:35:49.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.787+0000 7fe7beb81740 1 --2- 172.21.15.38:0/1134011009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9795c6b0 0x55ee97d1ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.787+0000 7fe7beb81740 1 --2- 172.21.15.38:0/1134011009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee979544d0 0x55ee97d2de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.787+0000 7fe7beb81740 1 --2- 172.21.15.38:0/1134011009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee9793bf60 0x55ee97d260d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.787+0000 7fe7beb81740 1 -- 172.21.15.38:0/1134011009 >> 172.21.15.38:0/1134011009 conn(0x55ee979714e0 msgr2=0x55ee97d1b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.787+0000 7fe7beb81740 1 -- 172.21.15.38:0/1134011009 shutdown_connections 2024-05-21T15:35:49.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.787+0000 7fe7beb81740 1 -- 172.21.15.38:0/1134011009 wait complete. 2024-05-21T15:35:49.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.788+0000 7fe7beb81740 1 -- start start 2024-05-21T15:35:49.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.788+0000 7fe7beb81740 1 Processor -- start 2024-05-21T15:35:49.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.788+0000 7fe7beb81740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9793bf60 0x55ee97d2c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.788+0000 7fe7beb81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee979544d0 0x55ee97d2cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.788+0000 7fe7beb81740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee9795c6b0 0x55ee97d9aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.788+0000 7fe7beb81740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b01adaa0 con 0x55ee9795c6b0 2024-05-21T15:35:49.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.788+0000 7fe7beb81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b01ad3d0 con 0x55ee979544d0 2024-05-21T15:35:49.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.788+0000 7fe7beb81740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b01aa610 con 0x55ee9793bf60 2024-05-21T15:35:49.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.788+0000 7fe7beb81740 10 client.0 _ll_register_callbacks cb 0x7fe7a8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:35:49.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7b61a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee9795c6b0 0x55ee97d9aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7b61a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee9795c6b0 0x55ee97d9aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49718/0 (socket says 172.21.15.38:49718) 2024-05-21T15:35:49.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7b61a5640 1 -- 172.21.15.38:0/536194915 learned_addr learned my addr 172.21.15.38:0/536194915 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:35:49.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7b51a3640 1 --2- 172.21.15.38:0/536194915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee979544d0 0x55ee97d2cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7b59a4640 1 --2- 172.21.15.38:0/536194915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9793bf60 0x55ee97d2c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7b61a5640 1 -- 172.21.15.38:0/536194915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9793bf60 msgr2=0x55ee97d2c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7b61a5640 1 --2- 172.21.15.38:0/536194915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9793bf60 0x55ee97d2c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7b61a5640 1 -- 172.21.15.38:0/536194915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee979544d0 msgr2=0x55ee97d2cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:35:49.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7b61a5640 1 --2- 172.21.15.38:0/536194915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee979544d0 0x55ee97d2cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7b61a5640 1 -- 172.21.15.38:0/536194915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee97d2d730 con 0x55ee9795c6b0 2024-05-21T15:35:49.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7b51a3640 1 --2- 172.21.15.38:0/536194915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee979544d0 0x55ee97d2cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:35:49.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7b61a5640 1 --2- 172.21.15.38:0/536194915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee9795c6b0 0x55ee97d9aa90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009ca0 tx=0x7fe7a80305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785e1e14f49b34c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:35:49.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7997fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:35:49.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7997fa640 1 -- 172.21.15.38:0/536194915 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8030ed0 con 0x55ee9795c6b0 2024-05-21T15:35:49.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7997fa640 1 -- 172.21.15.38:0/536194915 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe7a8004610 con 0x55ee9795c6b0 2024-05-21T15:35:49.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.789+0000 7fe7beb81740 1 -- 172.21.15.38:0/536194915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7ac0036f0 con 0x55ee9795c6b0 2024-05-21T15:35:49.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.790+0000 7fe7997fa640 1 -- 172.21.15.38:0/536194915 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7a8004810 con 0x55ee9795c6b0 2024-05-21T15:35:49.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.790+0000 7fe7beb81740 10 client.15906 fetch_fsmap learned FSMap version 27 2024-05-21T15:35:49.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.790+0000 7fe7beb81740 1 -- 172.21.15.38:0/536194915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee97d9c060 con 0x55ee9795c6b0 2024-05-21T15:35:49.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.790+0000 7fe7997fa640 1 -- 172.21.15.38:0/536194915 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7ac0036f0 con 0x55ee9795c6b0 2024-05-21T15:35:49.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.790+0000 7fe7997fa640 1 -- 172.21.15.38:0/536194915 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe7a8065070 con 0x55ee9795c6b0 2024-05-21T15:35:49.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.790+0000 7fe7beb81740 10 client.15906 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:35:49.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.790+0000 7fe7beb81740 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:35:49.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.790+0000 7fe7beb81740 20 client.15906 populate_metadata read hostname 'smithi038' 2024-05-21T15:35:49.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.790+0000 7fe7beb81740 1 --2- 172.21.15.38:0/536194915 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ee9797d390 0x55ee97d9eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:35:49.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.790+0000 7fe7beb81740 4 client.15906 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:35:49.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.790+0000 7fe7beb81740 1 -- 172.21.15.38:0/536194915 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee97c8f430 con 0x55ee9797d390 2024-05-21T15:35:49.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.790+0000 7fe7b59a4640 1 --2- 172.21.15.38:0/536194915 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ee9797d390 0x55ee97d9eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:35:49.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.791+0000 7fe7997fa640 10 client.15906 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:49.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.791+0000 7fe7b59a4640 1 --2- 172.21.15.38:0/536194915 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ee9797d390 0x55ee97d9eb00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f28fefb9454b0311 server_cookie=885316576c75932c in_seq=0 out_seq=0 2024-05-21T15:35:49.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.791+0000 7fe7997fa640 1 -- 172.21.15.38:0/536194915 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ee97c8f430 con 0x55ee9797d390 2024-05-21T15:35:49.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.791+0000 7fe7997fa640 10 client.15906 handle_command_reply: tid=0 2024-05-21T15:35:49.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.791+0000 7fe7beb81740 1 client.15906 shutdown 2024-05-21T15:35:49.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.792+0000 7fe7beb81740 1 -- 172.21.15.38:0/536194915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee9795c6b0 msgr2=0x55ee97d9aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.792+0000 7fe7beb81740 1 --2- 172.21.15.38:0/536194915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee9795c6b0 0x55ee97d9aa90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009ca0 tx=0x7fe7a80305c0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.792+0000 7fe7beb81740 1 -- 172.21.15.38:0/536194915 shutdown_connections 2024-05-21T15:35:49.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.792+0000 7fe7beb81740 1 --2- 172.21.15.38:0/536194915 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ee9797d390 0x55ee97d9eb00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.792+0000 7fe7beb81740 1 --2- 172.21.15.38:0/536194915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ee9795c6b0 0x55ee97d9aa90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.792+0000 7fe7beb81740 1 --2- 172.21.15.38:0/536194915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee979544d0 0x55ee97d2cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.792+0000 7fe7beb81740 1 --2- 172.21.15.38:0/536194915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee9793bf60 0x55ee97d2c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.792+0000 7fe7beb81740 1 -- 172.21.15.38:0/536194915 >> 172.21.15.38:0/536194915 conn(0x55ee979714e0 msgr2=0x55ee97d1bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.792+0000 7fe7997fa640 0 client.15906 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:35:49.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.792+0000 7fe7beb81740 1 -- 172.21.15.38:0/536194915 shutdown_connections 2024-05-21T15:35:49.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.792+0000 7fe7beb81740 1 -- 172.21.15.38:0/536194915 wait complete. 2024-05-21T15:35:49.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.792+0000 7fe7beb81740 20 client.15906 trim_cache size 0 max 16384 2024-05-21T15:35:49.796 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:35:49.796 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:35:49.796 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-05-21T15:35:49.796 INFO:teuthology.orchestra.run.smithi038.stdout: "07660293-4012-43cc-b513-ff85cf6c240c": { 2024-05-21T15:35:49.797 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-05-21T15:35:49.797 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "07660293-4012-43cc-b513-ff85cf6c240c", 2024-05-21T15:35:49.797 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-05-21T15:35:49.797 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:35:49.797 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:35:49.797 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:35:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.793+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2275497085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe79c0a2b70 msgr2=0x7fe79c0a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.793+0000 7fe7b7c08640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe79c0a2b70 0x7fe79c0a5080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80094e0 tx=0x7fe7a80095c0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.793+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2275497085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7b0102bc0 msgr2=0x7fe7b019b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:35:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.793+0000 7fe7b7c08640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7b0102bc0 0x7fe7b019b9c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000d950 tx=0x7fe7a000de10 comp rx=0 tx=0).stop 2024-05-21T15:35:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.793+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2275497085 shutdown_connections 2024-05-21T15:35:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.793+0000 7fe7b7c08640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7b01078c0 0x7fe7b01a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.793+0000 7fe7b7c08640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe79c0a2b70 0x7fe79c0a5080 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.793+0000 7fe7b7c08640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7b0102bc0 0x7fe7b019b9c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.793+0000 7fe7b7c08640 1 --2- 172.21.15.38:0/2275497085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7b0101980 0x7fe7b019b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:35:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.793+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2275497085 >> 172.21.15.38:0/2275497085 conn(0x7fe7b00fcb70 msgr2=0x7fe7b01038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:35:49.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.793+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2275497085 shutdown_connections 2024-05-21T15:35:49.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:35:49.793+0000 7fe7b7c08640 1 -- 172.21.15.38:0/2275497085 wait complete. 2024-05-21T15:35:49.804 INFO:tasks.cephfs.filesystem:scrub status for tag:07660293-4012-43cc-b513-ff85cf6c240c - {'path': '/', 'tag': '07660293-4012-43cc-b513-ff85cf6c240c', 'options': 'recursive,force'} 2024-05-21T15:35:50.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:50 smithi053 ceph-mon[36496]: scrub queued for path: / 2024-05-21T15:35:50.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:50 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:35:50.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:50 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:35:50.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:50 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:35:50.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:50 smithi045 ceph-mon[36409]: scrub queued for path: / 2024-05-21T15:35:50.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:50 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:35:50.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:50 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:35:50.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:50 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:35:50.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:50 smithi038 ceph-mon[31995]: scrub queued for path: / 2024-05-21T15:35:50.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:50 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:35:50.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:50 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:35:50.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:50 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:35:51.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:51 smithi053 ceph-mon[36496]: pgmap v394: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-21T15:35:51.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:51 smithi045 ceph-mon[36409]: pgmap v394: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-21T15:35:51.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:51 smithi038 ceph-mon[31995]: pgmap v394: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-05-21T15:35:53.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:53 smithi053 ceph-mon[36496]: pgmap v395: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-21T15:35:53.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:53 smithi045 ceph-mon[36409]: pgmap v395: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-21T15:35:53.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:53 smithi038 ceph-mon[31995]: pgmap v395: 97 pgs: 97 active+clean; 668 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-21T15:35:54.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:54 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:35:54.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:54 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:35:54.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:54 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:35:55.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:55 smithi053 ceph-mon[36496]: pgmap v396: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T15:35:55.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:55 smithi045 ceph-mon[36409]: pgmap v396: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T15:35:55.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:55 smithi038 ceph-mon[31995]: pgmap v396: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T15:35:56.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:56 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:35:56.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:56 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:35:56.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:56 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:35:57.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:57 smithi053 ceph-mon[36496]: pgmap v397: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T15:35:57.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:57.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:57.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:57.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:57.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:57.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:57.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:35:57.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:35:57.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:57 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:58.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:57 smithi045 ceph-mon[36409]: pgmap v397: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T15:35:58.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:58.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:58.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:58.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:58.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:58.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:58.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:35:58.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:35:58.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:57 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:58.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:57 smithi038 ceph-mon[31995]: pgmap v397: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T15:35:58.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:58.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:58.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:58.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:58.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:58.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:58.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:35:58.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:35:58.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:57 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:35:59.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:35:59 smithi053 ceph-mon[36496]: pgmap v398: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T15:36:00.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:35:59 smithi045 ceph-mon[36409]: pgmap v398: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T15:36:00.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:35:59 smithi038 ceph-mon[31995]: pgmap v398: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T15:36:01.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:01 smithi053 ceph-mon[36496]: pgmap v399: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T15:36:02.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:01 smithi045 ceph-mon[36409]: pgmap v399: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T15:36:02.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:01 smithi038 ceph-mon[31995]: pgmap v399: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T15:36:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.658+0000 7f61b5364640 1 -- 172.21.15.38:0/1605791947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61b0100700 msgr2=0x7f61b0100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.658+0000 7f61b5364640 1 --2- 172.21.15.38:0/1605791947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61b0100700 0x7f61b0100b70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6198009860 tx=0x7f619802a340 comp rx=0 tx=0).stop 2024-05-21T15:36:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.658+0000 7f61b5364640 1 -- 172.21.15.38:0/1605791947 shutdown_connections 2024-05-21T15:36:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.658+0000 7f61b5364640 1 --2- 172.21.15.38:0/1605791947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b0101200 0x7f61b010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.658+0000 7f61b5364640 1 --2- 172.21.15.38:0/1605791947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61b0100700 0x7f61b0100b70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.658+0000 7f61b5364640 1 --2- 172.21.15.38:0/1605791947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61b0104050 0x7f61b0104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.658+0000 7f61b5364640 1 -- 172.21.15.38:0/1605791947 >> 172.21.15.38:0/1605791947 conn(0x7f61b00fc300 msgr2=0x7f61b00fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.658+0000 7f61b5364640 1 -- 172.21.15.38:0/1605791947 shutdown_connections 2024-05-21T15:36:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.658+0000 7f61b5364640 1 -- 172.21.15.38:0/1605791947 wait complete. 2024-05-21T15:36:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.659+0000 7f61b5364640 1 Processor -- start 2024-05-21T15:36:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.659+0000 7f61b5364640 1 -- start start 2024-05-21T15:36:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61b5364640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b0100700 0x7f61b01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61b5364640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61b0101200 0x7f61b01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b0100700 0x7f61b01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b0100700 0x7f61b01992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50050/0 (socket says 172.21.15.38:50050) 2024-05-21T15:36:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61aeffd640 1 -- 172.21.15.38:0/3403030903 learned_addr learned my addr 172.21.15.38:0/3403030903 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61b5364640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61b0104050 0x7f61b01a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61b5364640 1 -- 172.21.15.38:0/3403030903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f61b010e730 con 0x7f61b0100700 2024-05-21T15:36:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61b5364640 1 -- 172.21.15.38:0/3403030903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f61b010e5b0 con 0x7f61b0104050 2024-05-21T15:36:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61b5364640 1 -- 172.21.15.38:0/3403030903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f61b010e8b0 con 0x7f61b0101200 2024-05-21T15:36:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61a7fff640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61b0101200 0x7f61b01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61af7fe640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61b0104050 0x7f61b01a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61aeffd640 1 -- 172.21.15.38:0/3403030903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61b0101200 msgr2=0x7f61b01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61aeffd640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61b0101200 0x7f61b01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61aeffd640 1 -- 172.21.15.38:0/3403030903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61b0104050 msgr2=0x7f61b01a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61aeffd640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61b0104050 0x7f61b01a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61aeffd640 1 -- 172.21.15.38:0/3403030903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b01a0d70 con 0x7f61b0100700 2024-05-21T15:36:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.660+0000 7f61a7fff640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61b0101200 0x7f61b01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:36:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.661+0000 7f61aeffd640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b0100700 0x7f61b01992a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009ea0 tx=0x7f61a0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.661+0000 7f61acff9640 1 -- 172.21.15.38:0/3403030903 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a000ccc0 con 0x7f61b0100700 2024-05-21T15:36:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.661+0000 7f61acff9640 1 -- 172.21.15.38:0/3403030903 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61a00137d0 con 0x7f61b0100700 2024-05-21T15:36:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.661+0000 7f61b5364640 1 -- 172.21.15.38:0/3403030903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b01017e0 con 0x7f61b0100700 2024-05-21T15:36:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.661+0000 7f61acff9640 1 -- 172.21.15.38:0/3403030903 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0003850 con 0x7f61b0100700 2024-05-21T15:36:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.661+0000 7f61b5364640 1 -- 172.21.15.38:0/3403030903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b0101c70 con 0x7f61b0100700 2024-05-21T15:36:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.665+0000 7f61bc205740 1 Processor -- start 2024-05-21T15:36:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.665+0000 7f61bc205740 1 -- start start 2024-05-21T15:36:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.665+0000 7f61acff9640 1 -- 172.21.15.38:0/3403030903 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f61a0013980 con 0x7f61b0100700 2024-05-21T15:36:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.666+0000 7f61bc205740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f421051b0 0x563f424ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.666+0000 7f61bc205740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.666+0000 7f61bc205740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420e44d0 0x563f424b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.666+0000 7f61aeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.666+0000 7f61aeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424adbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:38550/0 (socket says 172.21.15.38:38550) 2024-05-21T15:36:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.666+0000 7f61af7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f421051b0 0x563f424ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.666+0000 7f61af7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f421051b0 0x563f424ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50028/0 (socket says 172.21.15.38:50028) 2024-05-21T15:36:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.666+0000 7f61bc205740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563f42409340 con 0x563f420e44d0 2024-05-21T15:36:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.666+0000 7f61bc205740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563f423b1290 con 0x563f421051b0 2024-05-21T15:36:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.666+0000 7f61bc205740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563f42405300 con 0x563f420dc2f0 2024-05-21T15:36:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.666+0000 7f61aeffd640 1 -- 172.21.15.38:0/1404696361 learned_addr learned my addr 172.21.15.38:0/1404696361 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.666+0000 7f61a7fff640 1 --2- 172.21.15.38:0/1404696361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420e44d0 0x563f424b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.667+0000 7f61acff9640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f619c0a2cb0 0x7f619c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.667+0000 7f61a7fff640 1 -- 172.21.15.38:0/1404696361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 msgr2=0x563f424adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.667+0000 7f61a7fff640 1 --2- 172.21.15.38:0/1404696361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.667+0000 7f61a7fff640 1 -- 172.21.15.38:0/1404696361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f421051b0 msgr2=0x563f424ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.667+0000 7f61a7fff640 1 --2- 172.21.15.38:0/1404696361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f421051b0 0x563f424ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.667+0000 7f61a7fff640 1 -- 172.21.15.38:0/1404696361 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f423d58e0 con 0x563f420e44d0 2024-05-21T15:36:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.667+0000 7f61acff9640 1 -- 172.21.15.38:0/3403030903 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f61a00cf130 con 0x7f61b0100700 2024-05-21T15:36:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.667+0000 7f61a7fff640 1 --2- 172.21.15.38:0/1404696361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420e44d0 0x563f424b4af0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6198035040 tx=0x7f6198005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102ceec42b2ba2e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.667+0000 7f61af7fe640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f619c0a2cb0 0x7f619c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.667+0000 7f61a57fa640 1 -- 172.21.15.38:0/1404696361 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198037040 con 0x563f420e44d0 2024-05-21T15:36:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.668+0000 7f61a57fa640 1 -- 172.21.15.38:0/1404696361 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6198030d50 con 0x563f420e44d0 2024-05-21T15:36:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.668+0000 7f61a57fa640 1 -- 172.21.15.38:0/1404696361 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61980053c0 con 0x563f420e44d0 2024-05-21T15:36:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.669+0000 7f61bc205740 1 -- 172.21.15.38:0/1404696361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420e44d0 msgr2=0x563f424b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.669+0000 7f61af7fe640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f619c0a2cb0 0x7f619c0a51c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x563f424ae840 tx=0x7f61a8011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.669+0000 7f61bc205740 1 --2- 172.21.15.38:0/1404696361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420e44d0 0x563f424b4af0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6198035040 tx=0x7f6198005dc0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.669+0000 7f61bc205740 1 -- 172.21.15.38:0/1404696361 shutdown_connections 2024-05-21T15:36:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.669+0000 7f61bc205740 1 --2- 172.21.15.38:0/1404696361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420e44d0 0x563f424b4af0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.669+0000 7f61bc205740 1 --2- 172.21.15.38:0/1404696361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.669+0000 7f61bc205740 1 --2- 172.21.15.38:0/1404696361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f421051b0 0x563f424ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.669+0000 7f61bc205740 1 -- 172.21.15.38:0/1404696361 >> 172.21.15.38:0/1404696361 conn(0x563f420f9300 msgr2=0x563f424a9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.669+0000 7f61bc205740 1 -- 172.21.15.38:0/1404696361 shutdown_connections 2024-05-21T15:36:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.669+0000 7f61bc205740 1 -- 172.21.15.38:0/1404696361 wait complete. 2024-05-21T15:36:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.669+0000 7f61bc205740 1 -- start start 2024-05-21T15:36:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.670+0000 7f61bc205740 1 Processor -- start 2024-05-21T15:36:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.670+0000 7f61bc205740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.670+0000 7f61bc205740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420e44d0 0x563f424b7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.670+0000 7f61bc205740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f421051b0 0x563f424be7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.670+0000 7f61aeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.670+0000 7f61af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f421051b0 0x563f424be7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.670+0000 7f61a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420e44d0 0x563f424b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.670+0000 7f61aeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424b7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:38566/0 (socket says 172.21.15.38:38566) 2024-05-21T15:36:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.670+0000 7f61a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420e44d0 0x563f424b7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50040/0 (socket says 172.21.15.38:50040) 2024-05-21T15:36:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.670+0000 7f61af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f421051b0 0x563f424be7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50068/0 (socket says 172.21.15.38:50068) 2024-05-21T15:36:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.670+0000 7f61aeffd640 1 -- 172.21.15.38:0/2470375968 learned_addr learned my addr 172.21.15.38:0/2470375968 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.670+0000 7f61bc205740 1 -- 172.21.15.38:0/2470375968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563f424b5530 con 0x563f421051b0 2024-05-21T15:36:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.670+0000 7f61bc205740 1 -- 172.21.15.38:0/2470375968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563f423c4790 con 0x563f420e44d0 2024-05-21T15:36:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.671+0000 7f61bc205740 1 -- 172.21.15.38:0/2470375968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563f424b56b0 con 0x563f420dc2f0 2024-05-21T15:36:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.671+0000 7f61a7fff640 1 -- 172.21.15.38:0/2470375968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 msgr2=0x563f424b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.671+0000 7f61a7fff640 1 --2- 172.21.15.38:0/2470375968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.671+0000 7f61a7fff640 1 -- 172.21.15.38:0/2470375968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f421051b0 msgr2=0x563f424be7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.671+0000 7f61a7fff640 1 --2- 172.21.15.38:0/2470375968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f421051b0 0x563f424be7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.671+0000 7f61a7fff640 1 -- 172.21.15.38:0/2470375968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f4211d210 con 0x563f420e44d0 2024-05-21T15:36:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.671+0000 7f61af7fe640 1 --2- 172.21.15.38:0/2470375968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f421051b0 0x563f424be7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.671+0000 7f61bc205740 10 client.0 _ll_register_callbacks cb 0x563f423ffa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.671+0000 7f61bc205740 1 -- 172.21.15.38:0/2470375968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6198030d50 con 0x563f420e44d0 2024-05-21T15:36:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.671+0000 7f61a7fff640 1 --2- 172.21.15.38:0/2470375968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420e44d0 0x563f424b7860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f619802a930 tx=0x7f6198030f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93907fad994599 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.672+0000 7f61967fc640 10 client.25586 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.672+0000 7f61967fc640 1 -- 172.21.15.38:0/2470375968 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198049020 con 0x563f420e44d0 2024-05-21T15:36:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.672+0000 7f61967fc640 1 -- 172.21.15.38:0/2470375968 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f61980308e0 con 0x563f420e44d0 2024-05-21T15:36:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.672+0000 7f61967fc640 1 -- 172.21.15.38:0/2470375968 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6198037040 con 0x563f420e44d0 2024-05-21T15:36:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.672+0000 7f61bc205740 10 client.25586 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.672+0000 7f61bc205740 1 -- 172.21.15.38:0/2470375968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=27}) -- 0x563f421e9620 con 0x563f420e44d0 2024-05-21T15:36:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.673+0000 7f61967fc640 1 -- 172.21.15.38:0/2470375968 <== mon.2 v2:172.21.15.45:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6198005140 con 0x563f420e44d0 2024-05-21T15:36:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.673+0000 7f61bc205740 10 client.25586 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.673+0000 7f61bc205740 10 client.25586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.673+0000 7f61bc205740 20 client.25586 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.673+0000 7f61bc205740 1 --2- 172.21.15.38:0/2470375968 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x563f420c3d80 0x563f424c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.673+0000 7f61bc205740 4 client.25586 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.673+0000 7f61bc205740 1 -- 172.21.15.38:0/2470375968 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f42416a70 con 0x563f420c3d80 2024-05-21T15:36:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.673+0000 7f61aeffd640 1 --2- 172.21.15.38:0/2470375968 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x563f420c3d80 0x563f424c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.674+0000 7f61967fc640 10 client.25586 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.674+0000 7f61aeffd640 1 --2- 172.21.15.38:0/2470375968 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x563f420c3d80 0x563f424c4670 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b445b778490c7870 server_cookie=a38206c3cd0eb3eb in_seq=0 out_seq=0 2024-05-21T15:36:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61967fc640 1 -- 172.21.15.38:0/2470375968 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f42416a70 con 0x563f420c3d80 2024-05-21T15:36:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61967fc640 10 client.25586 handle_command_reply: tid=0 2024-05-21T15:36:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61bc205740 1 client.25586 shutdown 2024-05-21T15:36:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61bc205740 1 -- 172.21.15.38:0/2470375968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420e44d0 msgr2=0x563f424b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61bc205740 1 --2- 172.21.15.38:0/2470375968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420e44d0 0x563f424b7860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f619802a930 tx=0x7f6198030f80 comp rx=0 tx=0).stop 2024-05-21T15:36:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61bc205740 1 -- 172.21.15.38:0/2470375968 shutdown_connections 2024-05-21T15:36:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61bc205740 1 --2- 172.21.15.38:0/2470375968 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x563f420c3d80 0x563f424c4670 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61967fc640 0 client.25586 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61bc205740 1 --2- 172.21.15.38:0/2470375968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f421051b0 0x563f424be7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61bc205740 1 --2- 172.21.15.38:0/2470375968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420e44d0 0x563f424b7860 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61bc205740 1 --2- 172.21.15.38:0/2470375968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61bc205740 1 -- 172.21.15.38:0/2470375968 >> 172.21.15.38:0/2470375968 conn(0x563f420f9300 msgr2=0x563f4249f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61bc205740 1 -- 172.21.15.38:0/2470375968 shutdown_connections 2024-05-21T15:36:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61bc205740 1 -- 172.21.15.38:0/2470375968 wait complete. 2024-05-21T15:36:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.675+0000 7f61bc205740 20 client.25586 trim_cache size 0 max 16384 2024-05-21T15:36:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.706+0000 7f61bc205740 1 Processor -- start 2024-05-21T15:36:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.706+0000 7f61bc205740 1 -- start start 2024-05-21T15:36:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.707+0000 7f61bc205740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420c3d80 0x563f424b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.707+0000 7f61bc205740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424b7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.707+0000 7f61bc205740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420e44d0 0x563f424be6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.707+0000 7f61bc205740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01a7f80 con 0x563f420c3d80 2024-05-21T15:36:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.707+0000 7f61bc205740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01a9ac0 con 0x563f420e44d0 2024-05-21T15:36:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.707+0000 7f61bc205740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01a8650 con 0x563f420dc2f0 2024-05-21T15:36:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.707+0000 7f61aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420c3d80 0x563f424b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.707+0000 7f61aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420c3d80 0x563f424b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50084/0 (socket says 172.21.15.38:50084) 2024-05-21T15:36:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.707+0000 7f61aeffd640 1 -- 172.21.15.38:0/1524445304 learned_addr learned my addr 172.21.15.38:0/1524445304 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.707+0000 7f61a7fff640 1 --2- 172.21.15.38:0/1524445304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.707+0000 7f61af7fe640 1 --2- 172.21.15.38:0/1524445304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420e44d0 0x563f424be6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.707+0000 7f61aeffd640 1 -- 172.21.15.38:0/1524445304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 msgr2=0x563f424b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.707+0000 7f61aeffd640 1 --2- 172.21.15.38:0/1524445304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424b7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.708+0000 7f61aeffd640 1 -- 172.21.15.38:0/1524445304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420e44d0 msgr2=0x563f424be6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.708+0000 7f61aeffd640 1 --2- 172.21.15.38:0/1524445304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420e44d0 0x563f424be6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.708+0000 7f61aeffd640 1 -- 172.21.15.38:0/1524445304 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f424a9c30 con 0x563f420c3d80 2024-05-21T15:36:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.708+0000 7f61af7fe640 1 --2- 172.21.15.38:0/1524445304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420e44d0 0x563f424be6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:36:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.708+0000 7f61a7fff640 1 --2- 172.21.15.38:0/1524445304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.708+0000 7f61aeffd640 1 --2- 172.21.15.38:0/1524445304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420c3d80 0x563f424b6ef0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f61a00ccb20 tx=0x7f61a00d2c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71b45c60299d9f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.708+0000 7f61a4ff9640 1 -- 172.21.15.38:0/1524445304 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a009f6b0 con 0x563f420c3d80 2024-05-21T15:36:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.708+0000 7f61a4ff9640 1 -- 172.21.15.38:0/1524445304 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61a0013cf0 con 0x563f420c3d80 2024-05-21T15:36:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.708+0000 7f61a4ff9640 1 -- 172.21.15.38:0/1524445304 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a009de60 con 0x563f420c3d80 2024-05-21T15:36:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.710+0000 7f61bc205740 1 -- 172.21.15.38:0/1524445304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420c3d80 msgr2=0x563f424b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.710+0000 7f61bc205740 1 --2- 172.21.15.38:0/1524445304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420c3d80 0x563f424b6ef0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f61a00ccb20 tx=0x7f61a00d2c10 comp rx=0 tx=0).stop 2024-05-21T15:36:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.710+0000 7f61bc205740 1 -- 172.21.15.38:0/1524445304 shutdown_connections 2024-05-21T15:36:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.710+0000 7f61bc205740 1 --2- 172.21.15.38:0/1524445304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420e44d0 0x563f424be6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.710+0000 7f61bc205740 1 --2- 172.21.15.38:0/1524445304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420dc2f0 0x563f424b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.710+0000 7f61bc205740 1 --2- 172.21.15.38:0/1524445304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420c3d80 0x563f424b6ef0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.710+0000 7f61bc205740 1 -- 172.21.15.38:0/1524445304 >> 172.21.15.38:0/1524445304 conn(0x563f420f9300 msgr2=0x563f424b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.710+0000 7f61bc205740 1 -- 172.21.15.38:0/1524445304 shutdown_connections 2024-05-21T15:36:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.710+0000 7f61bc205740 1 -- 172.21.15.38:0/1524445304 wait complete. 2024-05-21T15:36:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.710+0000 7f61bc205740 1 -- start start 2024-05-21T15:36:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.711+0000 7f61bc205740 1 Processor -- start 2024-05-21T15:36:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.711+0000 7f61bc205740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420c3d80 0x563f424bece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.711+0000 7f61bc205740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420dc2f0 0x563f424bb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.711+0000 7f61bc205740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420e44d0 0x563f424bbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.711+0000 7f61bc205740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01a93f0 con 0x563f420dc2f0 2024-05-21T15:36:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.711+0000 7f61a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420dc2f0 0x563f424bb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.711+0000 7f61a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420dc2f0 0x563f424bb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50092/0 (socket says 172.21.15.38:50092) 2024-05-21T15:36:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.711+0000 7f61a7fff640 1 -- 172.21.15.38:0/1211166709 learned_addr learned my addr 172.21.15.38:0/1211166709 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.712+0000 7f61aeffd640 1 --2- 172.21.15.38:0/1211166709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420c3d80 0x563f424bece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.712+0000 7f61bc205740 1 -- 172.21.15.38:0/1211166709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01a8d20 con 0x563f420c3d80 2024-05-21T15:36:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.712+0000 7f61af7fe640 1 --2- 172.21.15.38:0/1211166709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420e44d0 0x563f424bbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.712+0000 7f61bc205740 1 -- 172.21.15.38:0/1211166709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f61b01a5f60 con 0x563f420e44d0 2024-05-21T15:36:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.712+0000 7f61bc205740 10 client.0 _ll_register_callbacks cb 0x7f61a80098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.712+0000 7f61a7fff640 1 -- 172.21.15.38:0/1211166709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420e44d0 msgr2=0x563f424bbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.712+0000 7f61a7fff640 1 --2- 172.21.15.38:0/1211166709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420e44d0 0x563f424bbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.712+0000 7f61a7fff640 1 -- 172.21.15.38:0/1211166709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420c3d80 msgr2=0x563f424bece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.712+0000 7f61a7fff640 1 --2- 172.21.15.38:0/1211166709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420c3d80 0x563f424bece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.712+0000 7f61a7fff640 1 -- 172.21.15.38:0/1211166709 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f4252aa10 con 0x563f420dc2f0 2024-05-21T15:36:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.712+0000 7f61af7fe640 1 --2- 172.21.15.38:0/1211166709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420e44d0 0x563f424bbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.712+0000 7f61a7fff640 1 --2- 172.21.15.38:0/1211166709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420dc2f0 0x563f424bb8d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f619802c010 tx=0x7f6198030550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280a8eef7a328c1a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.712+0000 7f61967fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.713+0000 7f61967fc640 1 -- 172.21.15.38:0/1211166709 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619805c060 con 0x563f420dc2f0 2024-05-21T15:36:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.713+0000 7f61967fc640 1 -- 172.21.15.38:0/1211166709 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f61980308e0 con 0x563f420dc2f0 2024-05-21T15:36:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.713+0000 7f61bc205740 1 -- 172.21.15.38:0/1211166709 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563f42410dc0 con 0x563f420dc2f0 2024-05-21T15:36:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.713+0000 7f61967fc640 1 -- 172.21.15.38:0/1211166709 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6198049040 con 0x563f420dc2f0 2024-05-21T15:36:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.713+0000 7f61bc205740 10 client.15924 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.713+0000 7f61bc205740 1 -- 172.21.15.38:0/1211166709 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f4252bb00 con 0x563f420dc2f0 2024-05-21T15:36:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.714+0000 7f61967fc640 1 -- 172.21.15.38:0/1211166709 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563f42410dc0 con 0x563f420dc2f0 2024-05-21T15:36:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.714+0000 7f61967fc640 1 -- 172.21.15.38:0/1211166709 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6198045ca0 con 0x563f420dc2f0 2024-05-21T15:36:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.714+0000 7f61bc205740 10 client.15924 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.714+0000 7f61bc205740 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.714+0000 7f61bc205740 20 client.15924 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.714+0000 7f61bc205740 1 --2- 172.21.15.38:0/1211166709 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x563f421051b0 0x563f4252e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.714+0000 7f61bc205740 4 client.15924 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:36:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.714+0000 7f61bc205740 1 -- 172.21.15.38:0/1211166709 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f4238e8e0 con 0x563f421051b0 2024-05-21T15:36:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.715+0000 7f61aeffd640 1 --2- 172.21.15.38:0/1211166709 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x563f421051b0 0x563f4252e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.715+0000 7f61967fc640 10 client.15924 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.715+0000 7f61aeffd640 1 --2- 172.21.15.38:0/1211166709 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x563f421051b0 0x563f4252e4c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9180574ebbcbdcd server_cookie=99ee6edf01aa92e6 in_seq=0 out_seq=0 2024-05-21T15:36:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.719+0000 7f61967fc640 1 -- 172.21.15.38:0/1211166709 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f4238e8e0 con 0x563f421051b0 2024-05-21T15:36:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.719+0000 7f61967fc640 10 client.15924 handle_command_reply: tid=0 2024-05-21T15:36:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.720+0000 7f61bc205740 1 client.15924 shutdown 2024-05-21T15:36:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.720+0000 7f61bc205740 1 -- 172.21.15.38:0/1211166709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420dc2f0 msgr2=0x563f424bb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.720+0000 7f61bc205740 1 --2- 172.21.15.38:0/1211166709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420dc2f0 0x563f424bb8d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f619802c010 tx=0x7f6198030550 comp rx=0 tx=0).stop 2024-05-21T15:36:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.720+0000 7f61bc205740 1 -- 172.21.15.38:0/1211166709 shutdown_connections 2024-05-21T15:36:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.720+0000 7f61bc205740 1 --2- 172.21.15.38:0/1211166709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563f420e44d0 0x563f424bbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.720+0000 7f61bc205740 1 --2- 172.21.15.38:0/1211166709 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x563f421051b0 0x563f4252e4c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.720+0000 7f61bc205740 1 --2- 172.21.15.38:0/1211166709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563f420dc2f0 0x563f424bb8d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.720+0000 7f61bc205740 1 --2- 172.21.15.38:0/1211166709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563f420c3d80 0x563f424bece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.720+0000 7f61bc205740 1 -- 172.21.15.38:0/1211166709 >> 172.21.15.38:0/1211166709 conn(0x563f420f9300 msgr2=0x563f424b6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.720+0000 7f61967fc640 0 client.15924 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.720+0000 7f61bc205740 1 -- 172.21.15.38:0/1211166709 shutdown_connections 2024-05-21T15:36:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.720+0000 7f61bc205740 1 -- 172.21.15.38:0/1211166709 wait complete. 2024-05-21T15:36:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.720+0000 7f61bc205740 20 client.15924 trim_cache size 0 max 16384 2024-05-21T15:36:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:36:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:36:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:36:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:36:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.721+0000 7f61b5364640 1 -- 172.21.15.38:0/3403030903 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f619c0a2cb0 msgr2=0x7f619c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.721+0000 7f61b5364640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f619c0a2cb0 0x7f619c0a51c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x563f424ae840 tx=0x7f61a8011040 comp rx=0 tx=0).stop 2024-05-21T15:36:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.721+0000 7f61b5364640 1 -- 172.21.15.38:0/3403030903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b0100700 msgr2=0x7f61b01992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.721+0000 7f61b5364640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b0100700 0x7f61b01992a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009ea0 tx=0x7f61a0009f90 comp rx=0 tx=0).stop 2024-05-21T15:36:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.721+0000 7f61b5364640 1 -- 172.21.15.38:0/3403030903 shutdown_connections 2024-05-21T15:36:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.721+0000 7f61b5364640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61b0104050 0x7f61b01a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.721+0000 7f61b5364640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61b0101200 0x7f61b01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.721+0000 7f61b5364640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f619c0a2cb0 0x7f619c0a51c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.721+0000 7f61b5364640 1 --2- 172.21.15.38:0/3403030903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61b0100700 0x7f61b01992a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.721+0000 7f61b5364640 1 -- 172.21.15.38:0/3403030903 >> 172.21.15.38:0/3403030903 conn(0x7f61b00fc300 msgr2=0x7f61b0104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.721+0000 7f61b5364640 1 -- 172.21.15.38:0/3403030903 shutdown_connections 2024-05-21T15:36:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:02.721+0000 7f61b5364640 1 -- 172.21.15.38:0/3403030903 wait complete. 2024-05-21T15:36:03.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:03 smithi053 ceph-mon[36496]: pgmap v400: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T15:36:04.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:03 smithi045 ceph-mon[36409]: pgmap v400: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T15:36:04.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:03 smithi038 ceph-mon[31995]: pgmap v400: 97 pgs: 97 active+clean; 669 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T15:36:05.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:05 smithi053 ceph-mon[36496]: pgmap v401: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-21T15:36:06.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:05 smithi045 ceph-mon[36409]: pgmap v401: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-21T15:36:06.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:05 smithi038 ceph-mon[31995]: pgmap v401: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-21T15:36:08.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:07 smithi045 ceph-mon[36409]: pgmap v402: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-21T15:36:08.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:07 smithi038 ceph-mon[31995]: pgmap v402: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-21T15:36:08.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:07 smithi053 ceph-mon[36496]: pgmap v402: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-21T15:36:10.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:09 smithi045 ceph-mon[36409]: pgmap v403: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T15:36:10.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:09 smithi038 ceph-mon[31995]: pgmap v403: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T15:36:10.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:09 smithi053 ceph-mon[36496]: pgmap v403: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T15:36:11.039 INFO:tasks.workunit.client.0.smithi038.stderr:Updating files: 92% (11392/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-21T15:36:11.330 INFO:tasks.workunit.client.0.smithi038.stderr:Note: switching to 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'. 2024-05-21T15:36:11.330 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-05-21T15:36:11.330 INFO:tasks.workunit.client.0.smithi038.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-21T15:36:11.331 INFO:tasks.workunit.client.0.smithi038.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-21T15:36:11.331 INFO:tasks.workunit.client.0.smithi038.stderr:state without impacting any branches by switching back to a branch. 2024-05-21T15:36:11.331 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-05-21T15:36:11.331 INFO:tasks.workunit.client.0.smithi038.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-21T15:36:11.331 INFO:tasks.workunit.client.0.smithi038.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-21T15:36:11.331 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-05-21T15:36:11.331 INFO:tasks.workunit.client.0.smithi038.stderr: git switch -c 2024-05-21T15:36:11.331 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-05-21T15:36:11.331 INFO:tasks.workunit.client.0.smithi038.stderr:Or undo this operation with: 2024-05-21T15:36:11.331 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-05-21T15:36:11.331 INFO:tasks.workunit.client.0.smithi038.stderr: git switch - 2024-05-21T15:36:11.331 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-05-21T15:36:11.331 INFO:tasks.workunit.client.0.smithi038.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-21T15:36:11.332 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-05-21T15:36:11.332 INFO:tasks.workunit.client.0.smithi038.stderr:HEAD is now at d2b507c724e Merge PR #56984 into wip-pdonnell-testing-20240517.203521-squid-debug 2024-05-21T15:36:11.337 DEBUG:teuthology.orchestra.run.smithi038:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-21T15:36:11.374 INFO:tasks.workunit.client.0.smithi038.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-21T15:36:11.377 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-21T15:36:11.377 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-21T15:36:12.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:11 smithi045 ceph-mon[36409]: pgmap v404: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:36:12.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:11 smithi038 ceph-mon[31995]: pgmap v404: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:36:12.090 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-21T15:36:12.160 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-21T15:36:12.200 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-21T15:36:12.202 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-21T15:36:12.202 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-21T15:36:12.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:11 smithi053 ceph-mon[36496]: pgmap v404: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:36:12.242 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-21T15:36:12.246 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:36:12.246 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-21T15:36:12.303 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-05-21T15:36:12.304 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-05-21T15:36:12.305 DEBUG:teuthology.orchestra.run.smithi038:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2b507c724ef6c8295ee0a35d185cdab167fd61d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-05-21T15:36:12.368 INFO:tasks.workunit.client.0.smithi038.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-05-21T15:36:12.368 INFO:tasks.workunit.client.0.smithi038.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-05-21T15:36:12.368 INFO:tasks.workunit.client.0.smithi038.stderr:+ trap -- retry EXIT 2024-05-21T15:36:12.369 INFO:tasks.workunit.client.0.smithi038.stderr:+ rm -rf ffsb 2024-05-21T15:36:12.369 INFO:tasks.workunit.client.0.smithi038.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-05-21T15:36:12.373 INFO:tasks.workunit.client.0.smithi038.stderr:Cloning into 'ffsb'... 2024-05-21T15:36:13.408 INFO:tasks.workunit.client.0.smithi038.stderr:+ trap - EXIT 2024-05-21T15:36:13.408 INFO:tasks.workunit.client.0.smithi038.stderr:+ cd ffsb 2024-05-21T15:36:13.408 INFO:tasks.workunit.client.0.smithi038.stderr:+ ./configure 2024-05-21T15:36:13.556 INFO:tasks.workunit.client.0.smithi038.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-21T15:36:14.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:13 smithi045 ceph-mon[36409]: pgmap v405: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:36:14.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:13 smithi038 ceph-mon[31995]: pgmap v405: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:36:14.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:13 smithi053 ceph-mon[36496]: pgmap v405: 97 pgs: 97 active+clean; 670 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:36:14.565 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether build environment is sane... yes 2024-05-21T15:36:14.577 INFO:tasks.workunit.client.0.smithi038.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-05-21T15:36:14.577 INFO:tasks.workunit.client.0.smithi038.stdout:checking for gawk... gawk 2024-05-21T15:36:14.594 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether make sets $(MAKE)... yes 2024-05-21T15:36:14.629 INFO:tasks.workunit.client.0.smithi038.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-21T15:36:14.629 INFO:tasks.workunit.client.0.smithi038.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-21T15:36:14.642 INFO:tasks.workunit.client.0.smithi038.stdout:checking for style of include used by make... GNU 2024-05-21T15:36:14.647 INFO:tasks.workunit.client.0.smithi038.stdout:checking for gcc... gcc 2024-05-21T15:36:14.707 INFO:tasks.workunit.client.0.smithi038.stdout:checking for C compiler default output file name... a.out 2024-05-21T15:36:14.707 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether the C compiler works... yes 2024-05-21T15:36:14.710 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether we are cross compiling... no 2024-05-21T15:36:14.750 INFO:tasks.workunit.client.0.smithi038.stdout:checking for suffix of executables... 2024-05-21T15:36:14.786 INFO:tasks.workunit.client.0.smithi038.stdout:checking for suffix of object files... o 2024-05-21T15:36:14.819 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether we are using the GNU C compiler... yes 2024-05-21T15:36:14.852 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether gcc accepts -g... yes 2024-05-21T15:36:14.920 INFO:tasks.workunit.client.0.smithi038.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-21T15:36:15.032 INFO:tasks.workunit.client.0.smithi038.stdout:checking dependency style of gcc... gcc3 2024-05-21T15:36:15.090 INFO:tasks.workunit.client.0.smithi038.stdout:checking how to run the C preprocessor... gcc -E 2024-05-21T15:36:15.134 INFO:tasks.workunit.client.0.smithi038.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-21T15:36:15.135 INFO:tasks.workunit.client.0.smithi038.stdout:checking for egrep... /usr/bin/grep -E 2024-05-21T15:36:15.144 INFO:tasks.workunit.client.0.smithi038.stdout:checking for AIX... no 2024-05-21T15:36:15.167 INFO:tasks.workunit.client.0.smithi038.stdout:checking for gcc... (cached) gcc 2024-05-21T15:36:15.174 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-21T15:36:15.176 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-21T15:36:15.176 INFO:tasks.workunit.client.0.smithi038.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-21T15:36:15.176 INFO:tasks.workunit.client.0.smithi038.stdout:checking dependency style of gcc... (cached) gcc3 2024-05-21T15:36:15.236 INFO:tasks.workunit.client.0.smithi038.stdout:checking for main in -lm... yes 2024-05-21T15:36:15.291 INFO:tasks.workunit.client.0.smithi038.stdout:checking for main in -lpthread... yes 2024-05-21T15:36:15.459 INFO:tasks.workunit.client.0.smithi038.stdout:checking for ANSI C header files... yes 2024-05-21T15:36:15.511 INFO:tasks.workunit.client.0.smithi038.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-05-21T15:36:15.557 INFO:tasks.workunit.client.0.smithi038.stdout:checking for sys/types.h... yes 2024-05-21T15:36:15.607 INFO:tasks.workunit.client.0.smithi038.stdout:checking for sys/stat.h... yes 2024-05-21T15:36:15.655 INFO:tasks.workunit.client.0.smithi038.stdout:checking for stdlib.h... yes 2024-05-21T15:36:15.702 INFO:tasks.workunit.client.0.smithi038.stdout:checking for string.h... yes 2024-05-21T15:36:15.752 INFO:tasks.workunit.client.0.smithi038.stdout:checking for memory.h... yes 2024-05-21T15:36:15.803 INFO:tasks.workunit.client.0.smithi038.stdout:checking for strings.h... yes 2024-05-21T15:36:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.830+0000 7f021cd38640 1 -- 172.21.15.38:0/3222827421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02181019e0 msgr2=0x7f02181026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.830+0000 7f021cd38640 1 --2- 172.21.15.38:0/3222827421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02181019e0 0x7f02181026e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0208009900 tx=0x7f020802a410 comp rx=0 tx=0).stop 2024-05-21T15:36:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.830+0000 7f021cd38640 1 -- 172.21.15.38:0/3222827421 shutdown_connections 2024-05-21T15:36:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.830+0000 7f021cd38640 1 --2- 172.21.15.38:0/3222827421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0218107920 0x7f0218109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.830+0000 7f021cd38640 1 --2- 172.21.15.38:0/3222827421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218102c20 0x7f0218103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.830+0000 7f021cd38640 1 --2- 172.21.15.38:0/3222827421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02181019e0 0x7f02181026e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.830+0000 7f021cd38640 1 -- 172.21.15.38:0/3222827421 >> 172.21.15.38:0/3222827421 conn(0x7f02180fcbd0 msgr2=0x7f02180ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.830+0000 7f021cd38640 1 -- 172.21.15.38:0/3222827421 shutdown_connections 2024-05-21T15:36:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.830+0000 7f021cd38640 1 -- 172.21.15.38:0/3222827421 wait complete. 2024-05-21T15:36:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.831+0000 7f021cd38640 1 Processor -- start 2024-05-21T15:36:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f021cd38640 1 -- start start 2024-05-21T15:36:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f021cd38640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02181019e0 0x7f02181980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f021cd38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0218102c20 0x7f0218198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f021cd38640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0218107920 0x7f021819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f021cd38640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f021810d660 con 0x7f0218102c20 2024-05-21T15:36:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f021cd38640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f021810d4e0 con 0x7f02181019e0 2024-05-21T15:36:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f021cd38640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f021810d7e0 con 0x7f0218107920 2024-05-21T15:36:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f020ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0218102c20 0x7f0218198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f020ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0218102c20 0x7f0218198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46628/0 (socket says 172.21.15.38:46628) 2024-05-21T15:36:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f020ffff640 1 -- 172.21.15.38:0/3295218619 learned_addr learned my addr 172.21.15.38:0/3295218619 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f021659c640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02181019e0 0x7f02181980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f0216d9d640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0218107920 0x7f021819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f020ffff640 1 -- 172.21.15.38:0/3295218619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0218107920 msgr2=0x7f021819f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f020ffff640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0218107920 0x7f021819f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f020ffff640 1 -- 172.21.15.38:0/3295218619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02181019e0 msgr2=0x7f02181980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f020ffff640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02181019e0 0x7f02181980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f020ffff640 1 -- 172.21.15.38:0/3295218619 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021819fc10 con 0x7f0218102c20 2024-05-21T15:36:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f021659c640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02181019e0 0x7f02181980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:36:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f0216d9d640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0218107920 0x7f021819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:36:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.832+0000 7f020ffff640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0218102c20 0x7f0218198610 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f020000e9f0 tx=0x7f020000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.833+0000 7f020f7fe640 1 -- 172.21.15.38:0/3295218619 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020000cd80 con 0x7f0218102c20 2024-05-21T15:36:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.833+0000 7f020f7fe640 1 -- 172.21.15.38:0/3295218619 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0200010910 con 0x7f0218102c20 2024-05-21T15:36:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.833+0000 7f021cd38640 1 -- 172.21.15.38:0/3295218619 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021819ff00 con 0x7f0218102c20 2024-05-21T15:36:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.833+0000 7f020f7fe640 1 -- 172.21.15.38:0/3295218619 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200018990 con 0x7f0218102c20 2024-05-21T15:36:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.833+0000 7f021cd38640 1 -- 172.21.15.38:0/3295218619 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02181a03c0 con 0x7f0218102c20 2024-05-21T15:36:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.835+0000 7f020f7fe640 1 -- 172.21.15.38:0/3295218619 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0200018b30 con 0x7f0218102c20 2024-05-21T15:36:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.835+0000 7f0223bd6740 1 Processor -- start 2024-05-21T15:36:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f0223bd6740 1 -- start start 2024-05-21T15:36:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f0223bd6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5ccae1b0 0x556d5d050070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f0223bd6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d0505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f0223bd6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc8d4d0 0x556d5d0574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f0223bd6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556d5cfb2340 con 0x556d5cc8d4d0 2024-05-21T15:36:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f0223bd6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556d5cf5a290 con 0x556d5cc852f0 2024-05-21T15:36:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f0223bd6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556d5cfae300 con 0x556d5ccae1b0 2024-05-21T15:36:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f020ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc8d4d0 0x556d5d0574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f021659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d0505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f020ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc8d4d0 0x556d5d0574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46634/0 (socket says 172.21.15.38:46634) 2024-05-21T15:36:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f020ffff640 1 -- 172.21.15.38:0/338997437 learned_addr learned my addr 172.21.15.38:0/338997437 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f0216d9d640 1 --2- 172.21.15.38:0/338997437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5ccae1b0 0x556d5d050070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f020ffff640 1 -- 172.21.15.38:0/338997437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5ccae1b0 msgr2=0x556d5d050070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f020ffff640 1 --2- 172.21.15.38:0/338997437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5ccae1b0 0x556d5d050070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f020ffff640 1 -- 172.21.15.38:0/338997437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 msgr2=0x556d5d0505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f020ffff640 1 --2- 172.21.15.38:0/338997437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d0505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f020ffff640 1 -- 172.21.15.38:0/338997437 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d5cf7e8e0 con 0x556d5cc8d4d0 2024-05-21T15:36:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.836+0000 7f021659c640 1 --2- 172.21.15.38:0/338997437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d0505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.837+0000 7f020ffff640 1 --2- 172.21.15.38:0/338997437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc8d4d0 0x556d5d0574f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f02000d5600 tx=0x7f02000d5ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab6afb454bba8ec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.837+0000 7f020f7fe640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f02040a2cb0 0x7f02040a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.837+0000 7f0216d9d640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f02040a2cb0 0x7f02040a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.837+0000 7f020cff9640 1 -- 172.21.15.38:0/338997437 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02000e2070 con 0x556d5cc8d4d0 2024-05-21T15:36:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.837+0000 7f020f7fe640 1 -- 172.21.15.38:0/3295218619 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f02000cf490 con 0x7f0218102c20 2024-05-21T15:36:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.837+0000 7f020cff9640 1 -- 172.21.15.38:0/338997437 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0200017e60 con 0x556d5cc8d4d0 2024-05-21T15:36:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.837+0000 7f020cff9640 1 -- 172.21.15.38:0/338997437 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020009b600 con 0x556d5cc8d4d0 2024-05-21T15:36:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.838+0000 7f0223bd6740 1 -- 172.21.15.38:0/338997437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc8d4d0 msgr2=0x556d5d0574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.838+0000 7f0223bd6740 1 --2- 172.21.15.38:0/338997437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc8d4d0 0x556d5d0574f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f02000d5600 tx=0x7f02000d5ac0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.838+0000 7f0216d9d640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f02040a2cb0 0x7f02040a51c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x556d5d051240 tx=0x7f0210009700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.838+0000 7f0223bd6740 1 -- 172.21.15.38:0/338997437 shutdown_connections 2024-05-21T15:36:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.838+0000 7f0223bd6740 1 --2- 172.21.15.38:0/338997437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc8d4d0 0x556d5d0574f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.838+0000 7f0223bd6740 1 --2- 172.21.15.38:0/338997437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d0505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.838+0000 7f0223bd6740 1 --2- 172.21.15.38:0/338997437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5ccae1b0 0x556d5d050070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.838+0000 7f0223bd6740 1 -- 172.21.15.38:0/338997437 >> 172.21.15.38:0/338997437 conn(0x556d5cca2300 msgr2=0x556d5d0497f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.838+0000 7f0223bd6740 1 -- 172.21.15.38:0/338997437 shutdown_connections 2024-05-21T15:36:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.838+0000 7f0223bd6740 1 -- 172.21.15.38:0/338997437 wait complete. 2024-05-21T15:36:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f0223bd6740 1 -- start start 2024-05-21T15:36:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f0223bd6740 1 Processor -- start 2024-05-21T15:36:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f0223bd6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d059da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f0223bd6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc8d4d0 0x556d5d05a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f0223bd6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5ccae1b0 0x556d5d061200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f0223bd6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556d5d057f40 con 0x556d5ccae1b0 2024-05-21T15:36:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f0223bd6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556d5cf6d790 con 0x556d5cc852f0 2024-05-21T15:36:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f0223bd6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556d5d0580c0 con 0x556d5cc8d4d0 2024-05-21T15:36:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f0223bd6740 10 client.0 _ll_register_callbacks cb 0x556d5cfa8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f0216d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5ccae1b0 0x556d5d061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f0216d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5ccae1b0 0x556d5d061200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46640/0 (socket says 172.21.15.38:46640) 2024-05-21T15:36:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f0216d9d640 1 -- 172.21.15.38:0/2354640306 learned_addr learned my addr 172.21.15.38:0/2354640306 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f021659c640 1 --2- 172.21.15.38:0/2354640306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d059da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f020ffff640 1 --2- 172.21.15.38:0/2354640306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc8d4d0 0x556d5d05a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f021659c640 1 -- 172.21.15.38:0/2354640306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc8d4d0 msgr2=0x556d5d05a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f021659c640 1 --2- 172.21.15.38:0/2354640306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc8d4d0 0x556d5d05a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f021659c640 1 -- 172.21.15.38:0/2354640306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5ccae1b0 msgr2=0x556d5d061200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f021659c640 1 --2- 172.21.15.38:0/2354640306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5ccae1b0 0x556d5d061200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f021659c640 1 -- 172.21.15.38:0/2354640306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d5ccc6210 con 0x556d5cc852f0 2024-05-21T15:36:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.839+0000 7f0216d9d640 1 --2- 172.21.15.38:0/2354640306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5ccae1b0 0x556d5d061200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.840+0000 7f021659c640 1 --2- 172.21.15.38:0/2354640306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d059da0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f020802a930 tx=0x7f02080043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce491bf30a72d16 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.840+0000 7f01fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.840+0000 7f01fdffb640 1 -- 172.21.15.38:0/2354640306 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208035040 con 0x556d5cc852f0 2024-05-21T15:36:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.840+0000 7f01fdffb640 1 -- 172.21.15.38:0/2354640306 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f0208005450 con 0x556d5cc852f0 2024-05-21T15:36:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.840+0000 7f0223bd6740 1 -- 172.21.15.38:0/2354640306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d5cfbfa70 con 0x556d5cc852f0 2024-05-21T15:36:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.840+0000 7f01fdffb640 1 -- 172.21.15.38:0/2354640306 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0208005650 con 0x556d5cc852f0 2024-05-21T15:36:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.840+0000 7f0223bd6740 10 client.25601 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.840+0000 7f0223bd6740 1 -- 172.21.15.38:0/2354640306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d5cd92620 con 0x556d5cc852f0 2024-05-21T15:36:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.840+0000 7f01fdffb640 1 -- 172.21.15.38:0/2354640306 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556d5cfbfa70 con 0x556d5cc852f0 2024-05-21T15:36:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.841+0000 7f01fdffb640 1 -- 172.21.15.38:0/2354640306 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0208004be0 con 0x556d5cc852f0 2024-05-21T15:36:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.841+0000 7f0223bd6740 10 client.25601 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.841+0000 7f0223bd6740 10 client.25601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.841+0000 7f0223bd6740 20 client.25601 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.841+0000 7f0223bd6740 1 --2- 172.21.15.38:0/2354640306 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556d5cc6cd80 0x556d5d067690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.841+0000 7f0223bd6740 4 client.25601 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.841+0000 7f0223bd6740 1 -- 172.21.15.38:0/2354640306 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d5cfb9dc0 con 0x556d5cc6cd80 2024-05-21T15:36:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.841+0000 7f020ffff640 1 --2- 172.21.15.38:0/2354640306 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556d5cc6cd80 0x556d5d067690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.841+0000 7f01fdffb640 10 client.25601 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.842+0000 7f020ffff640 1 --2- 172.21.15.38:0/2354640306 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556d5cc6cd80 0x556d5d067690 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeb61ac0b0580423 server_cookie=95d561e480c89ec9 in_seq=0 out_seq=0 2024-05-21T15:36:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.842+0000 7f01fdffb640 1 -- 172.21.15.38:0/2354640306 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556d5cfb9dc0 con 0x556d5cc6cd80 2024-05-21T15:36:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.842+0000 7f01fdffb640 10 client.25601 handle_command_reply: tid=0 2024-05-21T15:36:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.843+0000 7f0223bd6740 1 client.25601 shutdown 2024-05-21T15:36:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.843+0000 7f0223bd6740 1 -- 172.21.15.38:0/2354640306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 msgr2=0x556d5d059da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.843+0000 7f0223bd6740 1 --2- 172.21.15.38:0/2354640306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d059da0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f020802a930 tx=0x7f02080043b0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.843+0000 7f0223bd6740 1 -- 172.21.15.38:0/2354640306 shutdown_connections 2024-05-21T15:36:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.843+0000 7f0223bd6740 1 --2- 172.21.15.38:0/2354640306 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556d5cc6cd80 0x556d5d067690 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.843+0000 7f0223bd6740 1 --2- 172.21.15.38:0/2354640306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5ccae1b0 0x556d5d061200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.843+0000 7f01fdffb640 0 client.25601 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.843+0000 7f0223bd6740 1 --2- 172.21.15.38:0/2354640306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc8d4d0 0x556d5d05a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.843+0000 7f0223bd6740 1 --2- 172.21.15.38:0/2354640306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d059da0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.843+0000 7f0223bd6740 1 -- 172.21.15.38:0/2354640306 >> 172.21.15.38:0/2354640306 conn(0x556d5cca2300 msgr2=0x556d5d048810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.843+0000 7f0223bd6740 1 -- 172.21.15.38:0/2354640306 shutdown_connections 2024-05-21T15:36:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.843+0000 7f0223bd6740 1 -- 172.21.15.38:0/2354640306 wait complete. 2024-05-21T15:36:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.843+0000 7f0223bd6740 20 client.25601 trim_cache size 0 max 16384 2024-05-21T15:36:15.854 INFO:tasks.workunit.client.0.smithi038.stdout:checking for inttypes.h... yes 2024-05-21T15:36:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.858+0000 7f0223bd6740 1 Processor -- start 2024-05-21T15:36:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.858+0000 7f0223bd6740 1 -- start start 2024-05-21T15:36:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0223bd6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc6cd80 0x556d5d049870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0223bd6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d047de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0223bd6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc8d4d0 0x556d5d048320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0223bd6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02181a8240 con 0x556d5cc8d4d0 2024-05-21T15:36:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0223bd6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f02181a9d80 con 0x556d5cc852f0 2024-05-21T15:36:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0223bd6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f02181a8910 con 0x556d5cc6cd80 2024-05-21T15:36:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0216d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc8d4d0 0x556d5d048320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f020ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d047de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0216d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc8d4d0 0x556d5d048320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46644/0 (socket says 172.21.15.38:46644) 2024-05-21T15:36:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f021659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc6cd80 0x556d5d049870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0216d9d640 1 -- 172.21.15.38:0/3296193480 learned_addr learned my addr 172.21.15.38:0/3296193480 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0216d9d640 1 -- 172.21.15.38:0/3296193480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc6cd80 msgr2=0x556d5d049870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0216d9d640 1 --2- 172.21.15.38:0/3296193480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc6cd80 0x556d5d049870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0216d9d640 1 -- 172.21.15.38:0/3296193480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 msgr2=0x556d5d047de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0216d9d640 1 --2- 172.21.15.38:0/3296193480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d047de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0216d9d640 1 -- 172.21.15.38:0/3296193480 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d5d04fc20 con 0x556d5cc8d4d0 2024-05-21T15:36:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f020ffff640 1 --2- 172.21.15.38:0/3296193480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d047de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.859+0000 7f0216d9d640 1 --2- 172.21.15.38:0/3296193480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc8d4d0 0x556d5d048320 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0210005c40 tx=0x7f0210015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49333b67e036b5bf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.860+0000 7f01fffff640 1 -- 172.21.15.38:0/3296193480 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210020840 con 0x556d5cc8d4d0 2024-05-21T15:36:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.860+0000 7f01fffff640 1 -- 172.21.15.38:0/3296193480 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02100283e0 con 0x556d5cc8d4d0 2024-05-21T15:36:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.860+0000 7f01fffff640 1 -- 172.21.15.38:0/3296193480 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021000f620 con 0x556d5cc8d4d0 2024-05-21T15:36:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.861+0000 7f0223bd6740 1 -- 172.21.15.38:0/3296193480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc8d4d0 msgr2=0x556d5d048320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.861+0000 7f0223bd6740 1 --2- 172.21.15.38:0/3296193480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc8d4d0 0x556d5d048320 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0210005c40 tx=0x7f0210015000 comp rx=0 tx=0).stop 2024-05-21T15:36:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.861+0000 7f0223bd6740 1 -- 172.21.15.38:0/3296193480 shutdown_connections 2024-05-21T15:36:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.861+0000 7f0223bd6740 1 --2- 172.21.15.38:0/3296193480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc8d4d0 0x556d5d048320 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.861+0000 7f0223bd6740 1 --2- 172.21.15.38:0/3296193480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc852f0 0x556d5d047de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.861+0000 7f0223bd6740 1 --2- 172.21.15.38:0/3296193480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc6cd80 0x556d5d049870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.861+0000 7f0223bd6740 1 -- 172.21.15.38:0/3296193480 >> 172.21.15.38:0/3296193480 conn(0x556d5cca2300 msgr2=0x556d5d049160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.861+0000 7f0223bd6740 1 -- 172.21.15.38:0/3296193480 shutdown_connections 2024-05-21T15:36:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.861+0000 7f0223bd6740 1 -- 172.21.15.38:0/3296193480 wait complete. 2024-05-21T15:36:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.861+0000 7f0223bd6740 1 -- start start 2024-05-21T15:36:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.861+0000 7f0223bd6740 1 Processor -- start 2024-05-21T15:36:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f0223bd6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc6cd80 0x556d5d05dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f0223bd6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc852f0 0x556d5d05e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f0223bd6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc8d4d0 0x556d5d05f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f0223bd6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02181a96b0 con 0x556d5cc6cd80 2024-05-21T15:36:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f0223bd6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f02181a8fe0 con 0x556d5cc8d4d0 2024-05-21T15:36:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f0223bd6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f02181a6220 con 0x556d5cc852f0 2024-05-21T15:36:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f0223bd6740 10 client.0 _ll_register_callbacks cb 0x7f0208009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f021659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc6cd80 0x556d5d05dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f021659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc6cd80 0x556d5d05dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46646/0 (socket says 172.21.15.38:46646) 2024-05-21T15:36:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f021659c640 1 -- 172.21.15.38:0/2541183021 learned_addr learned my addr 172.21.15.38:0/2541183021 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f0216d9d640 1 --2- 172.21.15.38:0/2541183021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc8d4d0 0x556d5d05f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f020ffff640 1 --2- 172.21.15.38:0/2541183021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc852f0 0x556d5d05e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f021659c640 1 -- 172.21.15.38:0/2541183021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc852f0 msgr2=0x556d5d05e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f021659c640 1 --2- 172.21.15.38:0/2541183021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc852f0 0x556d5d05e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f021659c640 1 -- 172.21.15.38:0/2541183021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc8d4d0 msgr2=0x556d5d05f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f021659c640 1 --2- 172.21.15.38:0/2541183021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc8d4d0 0x556d5d05f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f021659c640 1 -- 172.21.15.38:0/2541183021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d5d05f720 con 0x556d5cc6cd80 2024-05-21T15:36:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f0216d9d640 1 --2- 172.21.15.38:0/2541183021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc8d4d0 0x556d5d05f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:36:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f021659c640 1 --2- 172.21.15.38:0/2541183021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc6cd80 0x556d5d05dcf0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0208031610 tx=0x7f0208009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912704226c6739e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f01fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f01fdffb640 1 -- 172.21.15.38:0/2541183021 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208035040 con 0x556d5cc6cd80 2024-05-21T15:36:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f01fdffb640 1 -- 172.21.15.38:0/2541183021 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f0208030d00 con 0x556d5cc6cd80 2024-05-21T15:36:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.862+0000 7f0223bd6740 1 -- 172.21.15.38:0/2541183021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02100283e0 con 0x556d5cc6cd80 2024-05-21T15:36:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.863+0000 7f01fdffb640 1 -- 172.21.15.38:0/2541183021 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f020802aa90 con 0x556d5cc6cd80 2024-05-21T15:36:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.863+0000 7f0223bd6740 10 client.15942 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.863+0000 7f0223bd6740 1 -- 172.21.15.38:0/2541183021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d5d0609b0 con 0x556d5cc6cd80 2024-05-21T15:36:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.863+0000 7f01fdffb640 1 -- 172.21.15.38:0/2541183021 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02100283e0 con 0x556d5cc6cd80 2024-05-21T15:36:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.863+0000 7f01fdffb640 1 -- 172.21.15.38:0/2541183021 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0208056030 con 0x556d5cc6cd80 2024-05-21T15:36:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.863+0000 7f0223bd6740 10 client.15942 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.863+0000 7f0223bd6740 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.863+0000 7f0223bd6740 20 client.15942 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.863+0000 7f0223bd6740 1 --2- 172.21.15.38:0/2541183021 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556d5ccae1b0 0x556d5d0d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.863+0000 7f0223bd6740 4 client.15942 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:36:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.863+0000 7f0223bd6740 1 -- 172.21.15.38:0/2541183021 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d5cf378e0 con 0x556d5ccae1b0 2024-05-21T15:36:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.863+0000 7f020ffff640 1 --2- 172.21.15.38:0/2541183021 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556d5ccae1b0 0x556d5d0d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.864+0000 7f01fdffb640 10 client.15942 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.864+0000 7f020ffff640 1 --2- 172.21.15.38:0/2541183021 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556d5ccae1b0 0x556d5d0d1f90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ad3ba7879bb8f2 server_cookie=4db6e87b0fd77ade in_seq=0 out_seq=0 2024-05-21T15:36:15.962 INFO:tasks.workunit.client.0.smithi038.stdout:checking for stdint.h... yes 2024-05-21T15:36:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.973+0000 7f01fdffb640 1 -- 172.21.15.38:0/2541183021 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d5cf378e0 con 0x556d5ccae1b0 2024-05-21T15:36:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.973+0000 7f01fdffb640 10 client.15942 handle_command_reply: tid=0 2024-05-21T15:36:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.973+0000 7f0223bd6740 1 client.15942 shutdown 2024-05-21T15:36:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.973+0000 7f0223bd6740 1 -- 172.21.15.38:0/2541183021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc6cd80 msgr2=0x556d5d05dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.973+0000 7f0223bd6740 1 --2- 172.21.15.38:0/2541183021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc6cd80 0x556d5d05dcf0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0208031610 tx=0x7f0208009260 comp rx=0 tx=0).stop 2024-05-21T15:36:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.973+0000 7f0223bd6740 1 -- 172.21.15.38:0/2541183021 shutdown_connections 2024-05-21T15:36:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.973+0000 7f0223bd6740 1 --2- 172.21.15.38:0/2541183021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d5cc8d4d0 0x556d5d05f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.973+0000 7f0223bd6740 1 --2- 172.21.15.38:0/2541183021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d5cc852f0 0x556d5d05e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.973+0000 7f0223bd6740 1 --2- 172.21.15.38:0/2541183021 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556d5ccae1b0 0x556d5d0d1f90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.973+0000 7f0223bd6740 1 --2- 172.21.15.38:0/2541183021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556d5cc6cd80 0x556d5d05dcf0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.973+0000 7f01fdffb640 0 client.15942 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.973+0000 7f0223bd6740 1 -- 172.21.15.38:0/2541183021 >> 172.21.15.38:0/2541183021 conn(0x556d5cca2300 msgr2=0x556d5d04c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f0223bd6740 1 -- 172.21.15.38:0/2541183021 shutdown_connections 2024-05-21T15:36:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f0223bd6740 1 -- 172.21.15.38:0/2541183021 wait complete. 2024-05-21T15:36:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f0223bd6740 20 client.15942 trim_cache size 0 max 16384 2024-05-21T15:36:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:36:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:36:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:36:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:36:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f021cd38640 1 -- 172.21.15.38:0/3295218619 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f02040a2cb0 msgr2=0x7f02040a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f021cd38640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f02040a2cb0 0x7f02040a51c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x556d5d051240 tx=0x7f0210009700 comp rx=0 tx=0).stop 2024-05-21T15:36:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f021cd38640 1 -- 172.21.15.38:0/3295218619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0218102c20 msgr2=0x7f0218198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f021cd38640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0218102c20 0x7f0218198610 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f020000e9f0 tx=0x7f020000eeb0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f021cd38640 1 -- 172.21.15.38:0/3295218619 shutdown_connections 2024-05-21T15:36:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f021cd38640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0218107920 0x7f021819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f021cd38640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f02040a2cb0 0x7f02040a51c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f021cd38640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0218102c20 0x7f0218198610 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f021cd38640 1 --2- 172.21.15.38:0/3295218619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02181019e0 0x7f02181980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f021cd38640 1 -- 172.21.15.38:0/3295218619 >> 172.21.15.38:0/3295218619 conn(0x7f02180fcbd0 msgr2=0x7f02180ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f021cd38640 1 -- 172.21.15.38:0/3295218619 shutdown_connections 2024-05-21T15:36:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:15.974+0000 7f021cd38640 1 -- 172.21.15.38:0/3295218619 wait complete. 2024-05-21T15:36:16.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:15 smithi045 ceph-mon[36409]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 68 KiB/s wr, 11 op/s 2024-05-21T15:36:16.014 INFO:tasks.workunit.client.0.smithi038.stdout:checking for unistd.h... yes 2024-05-21T15:36:16.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:15 smithi038 ceph-mon[31995]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 68 KiB/s wr, 11 op/s 2024-05-21T15:36:16.074 INFO:tasks.workunit.client.0.smithi038.stdout:checking pthread.h usability... yes 2024-05-21T15:36:16.098 INFO:tasks.workunit.client.0.smithi038.stdout:checking pthread.h presence... yes 2024-05-21T15:36:16.098 INFO:tasks.workunit.client.0.smithi038.stdout:checking for pthread.h... yes 2024-05-21T15:36:16.160 INFO:tasks.workunit.client.0.smithi038.stdout:checking fcntl.h usability... yes 2024-05-21T15:36:16.182 INFO:tasks.workunit.client.0.smithi038.stdout:checking fcntl.h presence... yes 2024-05-21T15:36:16.182 INFO:tasks.workunit.client.0.smithi038.stdout:checking for fcntl.h... yes 2024-05-21T15:36:16.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:15 smithi053 ceph-mon[36496]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 68 KiB/s wr, 11 op/s 2024-05-21T15:36:16.237 INFO:tasks.workunit.client.0.smithi038.stdout:checking limits.h usability... yes 2024-05-21T15:36:16.258 INFO:tasks.workunit.client.0.smithi038.stdout:checking limits.h presence... yes 2024-05-21T15:36:16.258 INFO:tasks.workunit.client.0.smithi038.stdout:checking for limits.h... yes 2024-05-21T15:36:16.273 INFO:tasks.workunit.client.0.smithi038.stdout:checking for stdint.h... (cached) yes 2024-05-21T15:36:16.323 INFO:tasks.workunit.client.0.smithi038.stdout:checking sys/time.h usability... yes 2024-05-21T15:36:16.347 INFO:tasks.workunit.client.0.smithi038.stdout:checking sys/time.h presence... yes 2024-05-21T15:36:16.347 INFO:tasks.workunit.client.0.smithi038.stdout:checking for sys/time.h... yes 2024-05-21T15:36:16.362 INFO:tasks.workunit.client.0.smithi038.stdout:checking for unistd.h... (cached) yes 2024-05-21T15:36:16.406 INFO:tasks.workunit.client.0.smithi038.stdout:checking sys/vfs.h usability... yes 2024-05-21T15:36:16.427 INFO:tasks.workunit.client.0.smithi038.stdout:checking sys/vfs.h presence... yes 2024-05-21T15:36:16.427 INFO:tasks.workunit.client.0.smithi038.stdout:checking for sys/vfs.h... yes 2024-05-21T15:36:16.477 INFO:tasks.workunit.client.0.smithi038.stdout:checking sys/limits.h usability... no 2024-05-21T15:36:16.499 INFO:tasks.workunit.client.0.smithi038.stdout:checking sys/limits.h presence... no 2024-05-21T15:36:16.499 INFO:tasks.workunit.client.0.smithi038.stdout:checking for sys/limits.h... no 2024-05-21T15:36:16.534 INFO:tasks.workunit.client.0.smithi038.stdout:checking for an ANSI C-conforming const... yes 2024-05-21T15:36:16.570 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-21T15:36:16.571 INFO:tasks.workunit.client.0.smithi038.stdout:checking for function prototypes... yes 2024-05-21T15:36:16.600 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether setvbuf arguments are reversed... no 2024-05-21T15:36:16.697 INFO:tasks.workunit.client.0.smithi038.stdout:checking for system... yes 2024-05-21T15:36:16.766 INFO:tasks.workunit.client.0.smithi038.stdout:checking for gettimeofday... yes 2024-05-21T15:36:16.835 INFO:tasks.workunit.client.0.smithi038.stdout:checking for mkdir... yes 2024-05-21T15:36:16.899 INFO:tasks.workunit.client.0.smithi038.stdout:checking for strerror... yes 2024-05-21T15:36:16.963 INFO:tasks.workunit.client.0.smithi038.stdout:checking for open64... yes 2024-05-21T15:36:17.035 INFO:tasks.workunit.client.0.smithi038.stdout:checking for stat64... yes 2024-05-21T15:36:17.100 INFO:tasks.workunit.client.0.smithi038.stdout:checking for fseeko64... yes 2024-05-21T15:36:17.164 INFO:tasks.workunit.client.0.smithi038.stdout:checking for lrand48_r... yes 2024-05-21T15:36:17.230 INFO:tasks.workunit.client.0.smithi038.stdout:checking for srand48_r... yes 2024-05-21T15:36:17.255 INFO:tasks.workunit.client.0.smithi038.stdout:configure: creating ./config.status 2024-05-21T15:36:17.356 INFO:tasks.workunit.client.0.smithi038.stdout:config.status: creating Makefile 2024-05-21T15:36:17.370 INFO:tasks.workunit.client.0.smithi038.stdout:config.status: creating config.h 2024-05-21T15:36:17.387 INFO:tasks.workunit.client.0.smithi038.stdout:config.status: executing depfiles commands 2024-05-21T15:36:17.437 INFO:tasks.workunit.client.0.smithi038.stdout:config.status: executing default commands 2024-05-21T15:36:17.484 INFO:tasks.workunit.client.0.smithi038.stderr:+ make 2024-05-21T15:36:17.490 INFO:tasks.workunit.client.0.smithi038.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-05-21T15:36:17.492 INFO:tasks.workunit.client.0.smithi038.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-05-21T15:36:17.493 INFO:tasks.workunit.client.0.smithi038.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-05-21T15:36:17.494 INFO:tasks.workunit.client.0.smithi038.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-05-21T15:36:17.494 INFO:tasks.workunit.client.0.smithi038.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-05-21T15:36:17.494 INFO:tasks.workunit.client.0.smithi038.stderr: any GNU archive site. 2024-05-21T15:36:17.495 INFO:tasks.workunit.client.0.smithi038.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-05-21T15:36:17.496 INFO:tasks.workunit.client.0.smithi038.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-05-21T15:36:17.497 INFO:tasks.workunit.client.0.smithi038.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-05-21T15:36:17.497 INFO:tasks.workunit.client.0.smithi038.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-05-21T15:36:17.497 INFO:tasks.workunit.client.0.smithi038.stderr: You might want to install the `Automake' and `Perl' packages. 2024-05-21T15:36:17.497 INFO:tasks.workunit.client.0.smithi038.stderr: Grab them from any GNU archive site. 2024-05-21T15:36:17.546 INFO:tasks.workunit.client.0.smithi038.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-05-21T15:36:17.907 INFO:tasks.workunit.client.0.smithi038.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-05-21T15:36:17.908 INFO:tasks.workunit.client.0.smithi038.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-05-21T15:36:17.908 INFO:tasks.workunit.client.0.smithi038.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-05-21T15:36:17.908 INFO:tasks.workunit.client.0.smithi038.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-05-21T15:36:18.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:17 smithi045 ceph-mon[36409]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 67 KiB/s wr, 9 op/s 2024-05-21T15:36:18.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:17 smithi038 ceph-mon[31995]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 67 KiB/s wr, 9 op/s 2024-05-21T15:36:18.028 INFO:tasks.workunit.client.0.smithi038.stdout:/bin/sh ./config.status --recheck 2024-05-21T15:36:18.086 INFO:tasks.workunit.client.0.smithi038.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-05-21T15:36:18.200 INFO:tasks.workunit.client.0.smithi038.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-21T15:36:18.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:17 smithi053 ceph-mon[36496]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 67 KiB/s wr, 9 op/s 2024-05-21T15:36:19.208 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether build environment is sane... yes 2024-05-21T15:36:19.213 INFO:tasks.workunit.client.0.smithi038.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-05-21T15:36:19.213 INFO:tasks.workunit.client.0.smithi038.stdout:checking for gawk... gawk 2024-05-21T15:36:19.230 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether make sets $(MAKE)... yes 2024-05-21T15:36:19.265 INFO:tasks.workunit.client.0.smithi038.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-21T15:36:19.265 INFO:tasks.workunit.client.0.smithi038.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-21T15:36:19.279 INFO:tasks.workunit.client.0.smithi038.stdout:checking for style of include used by make... GNU 2024-05-21T15:36:19.284 INFO:tasks.workunit.client.0.smithi038.stdout:checking for gcc... gcc 2024-05-21T15:36:19.380 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether the C compiler works... yes 2024-05-21T15:36:19.380 INFO:tasks.workunit.client.0.smithi038.stdout:checking for C compiler default output file name... a.out 2024-05-21T15:36:19.420 INFO:tasks.workunit.client.0.smithi038.stdout:checking for suffix of executables... 2024-05-21T15:36:19.459 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether we are cross compiling... no 2024-05-21T15:36:19.498 INFO:tasks.workunit.client.0.smithi038.stdout:checking for suffix of object files... o 2024-05-21T15:36:19.524 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether we are using the GNU C compiler... yes 2024-05-21T15:36:19.549 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether gcc accepts -g... yes 2024-05-21T15:36:19.587 INFO:tasks.workunit.client.0.smithi038.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-21T15:36:19.694 INFO:tasks.workunit.client.0.smithi038.stdout:checking dependency style of gcc... gcc3 2024-05-21T15:36:19.743 INFO:tasks.workunit.client.0.smithi038.stdout:checking how to run the C preprocessor... gcc -E 2024-05-21T15:36:19.795 INFO:tasks.workunit.client.0.smithi038.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-21T15:36:19.796 INFO:tasks.workunit.client.0.smithi038.stdout:checking for egrep... /usr/bin/grep -E 2024-05-21T15:36:19.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:36:19.805 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:36:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.927+0000 7fbfa1637640 1 -- 172.21.15.38:0/3022183850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c1078c0 msgr2=0x7fbf9c109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.927+0000 7fbfa1637640 1 --2- 172.21.15.38:0/3022183850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c1078c0 0x7fbf9c109ce0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009be0 tx=0x7fbf9402a6c0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.928+0000 7fbfa1637640 1 -- 172.21.15.38:0/3022183850 shutdown_connections 2024-05-21T15:36:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.928+0000 7fbfa1637640 1 --2- 172.21.15.38:0/3022183850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c1078c0 0x7fbf9c109ce0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.928+0000 7fbfa1637640 1 --2- 172.21.15.38:0/3022183850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c102bc0 0x7fbf9c102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.928+0000 7fbfa1637640 1 --2- 172.21.15.38:0/3022183850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf9c101980 0x7fbf9c102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.928+0000 7fbfa1637640 1 -- 172.21.15.38:0/3022183850 >> 172.21.15.38:0/3022183850 conn(0x7fbf9c0fcb70 msgr2=0x7fbf9c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.928+0000 7fbfa1637640 1 -- 172.21.15.38:0/3022183850 shutdown_connections 2024-05-21T15:36:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.928+0000 7fbfa1637640 1 -- 172.21.15.38:0/3022183850 wait complete. 2024-05-21T15:36:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.928+0000 7fbfa1637640 1 Processor -- start 2024-05-21T15:36:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.928+0000 7fbfa1637640 1 -- start start 2024-05-21T15:36:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.928+0000 7fbfa1637640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf9c101980 0x7fbf9c0fea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.928+0000 7fbfa1637640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c102bc0 0x7fbf9c0fd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.928+0000 7fbfa1637640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c1078c0 0x7fbf9c0fd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbfa1637640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c10d5b0 con 0x7fbf9c1078c0 2024-05-21T15:36:19.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbfa1637640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c10d430 con 0x7fbf9c102bc0 2024-05-21T15:36:19.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbfa1637640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c10d730 con 0x7fbf9c101980 2024-05-21T15:36:19.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c1078c0 0x7fbf9c0fd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbf93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c102bc0 0x7fbf9c0fd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbf93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c102bc0 0x7fbf9c0fd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53192/0 (socket says 172.21.15.38:53192) 2024-05-21T15:36:19.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c1078c0 0x7fbf9c0fd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46668/0 (socket says 172.21.15.38:46668) 2024-05-21T15:36:19.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbf93fff640 1 -- 172.21.15.38:0/3006944030 learned_addr learned my addr 172.21.15.38:0/3006944030 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:19.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbf9affd640 1 --2- 172.21.15.38:0/3006944030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf9c101980 0x7fbf9c0fea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbf93fff640 1 -- 172.21.15.38:0/3006944030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf9c101980 msgr2=0x7fbf9c0fea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbf93fff640 1 --2- 172.21.15.38:0/3006944030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf9c101980 0x7fbf9c0fea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbf93fff640 1 -- 172.21.15.38:0/3006944030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c1078c0 msgr2=0x7fbf9c0fd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbf93fff640 1 --2- 172.21.15.38:0/3006944030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c1078c0 0x7fbf9c0fd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbf93fff640 1 -- 172.21.15.38:0/3006944030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9c19d0d0 con 0x7fbf9c102bc0 2024-05-21T15:36:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbf9b7fe640 1 --2- 172.21.15.38:0/3006944030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c1078c0 0x7fbf9c0fd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.929+0000 7fbf93fff640 1 --2- 172.21.15.38:0/3006944030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c102bc0 0x7fbf9c0fd0a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400e9c0 tx=0x7fbf8400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.930+0000 7fbf98ff9640 1 -- 172.21.15.38:0/3006944030 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8400cde0 con 0x7fbf9c102bc0 2024-05-21T15:36:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.930+0000 7fbfa1637640 1 -- 172.21.15.38:0/3006944030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9c0fdc40 con 0x7fbf9c102bc0 2024-05-21T15:36:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.930+0000 7fbf98ff9640 1 -- 172.21.15.38:0/3006944030 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf840035f0 con 0x7fbf9c102bc0 2024-05-21T15:36:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.930+0000 7fbf98ff9640 1 -- 172.21.15.38:0/3006944030 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84010650 con 0x7fbf9c102bc0 2024-05-21T15:36:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.930+0000 7fbfa1637640 1 -- 172.21.15.38:0/3006944030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf9c1a7cd0 con 0x7fbf9c102bc0 2024-05-21T15:36:19.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.932+0000 7fbfa85b3740 1 Processor -- start 2024-05-21T15:36:19.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.932+0000 7fbf98ff9640 1 -- 172.21.15.38:0/3006944030 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbf840037a0 con 0x7fbf9c102bc0 2024-05-21T15:36:19.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.932+0000 7fbfa85b3740 1 -- start start 2024-05-21T15:36:19.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbfa85b3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca97390 0x55764ce36bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbfa85b3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca6e4d0 0x55764ce37130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbfa85b3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca766b0 0x55764ce3e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbfa85b3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55764cd9bd00 con 0x55764ca97390 2024-05-21T15:36:19.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbfa85b3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55764cd434c0 con 0x55764ca766b0 2024-05-21T15:36:19.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbfa85b3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55764cd97cc0 con 0x55764ca6e4d0 2024-05-21T15:36:19.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca97390 0x55764ce36bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca97390 0x55764ce36bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46682/0 (socket says 172.21.15.38:46682) 2024-05-21T15:36:19.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbf9affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca6e4d0 0x55764ce37130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbf9b7fe640 1 -- 172.21.15.38:0/2680013510 learned_addr learned my addr 172.21.15.38:0/2680013510 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:19.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbf93fff640 1 --2- 172.21.15.38:0/2680013510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca766b0 0x55764ce3e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbf9affd640 1 -- 172.21.15.38:0/2680013510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca766b0 msgr2=0x55764ce3e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:19.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbf9affd640 1 --2- 172.21.15.38:0/2680013510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca766b0 0x55764ce3e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbf9affd640 1 -- 172.21.15.38:0/2680013510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca97390 msgr2=0x55764ce36bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbf9affd640 1 --2- 172.21.15.38:0/2680013510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca97390 0x55764ce36bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbf9affd640 1 -- 172.21.15.38:0/2680013510 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55764cb797d0 con 0x55764ca6e4d0 2024-05-21T15:36:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.933+0000 7fbf93fff640 1 --2- 172.21.15.38:0/2680013510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca766b0 0x55764ce3e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.934+0000 7fbf98ff9640 1 --2- 172.21.15.38:0/3006944030 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf880a2b90 0x7fbf880a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.934+0000 7fbf9affd640 1 --2- 172.21.15.38:0/2680013510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca6e4d0 0x55764ce37130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c101250 tx=0x7fbf8c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2e319472b980c54 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.934+0000 7fbf9b7fe640 1 --2- 172.21.15.38:0/3006944030 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf880a2b90 0x7fbf880a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.934+0000 7fbf98ff9640 1 -- 172.21.15.38:0/3006944030 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbf840cf3b0 con 0x7fbf9c102bc0 2024-05-21T15:36:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.934+0000 7fbf917fa640 1 -- 172.21.15.38:0/2680013510 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c004230 con 0x55764ca6e4d0 2024-05-21T15:36:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.934+0000 7fbf917fa640 1 -- 172.21.15.38:0/2680013510 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf8c009e10 con 0x55764ca6e4d0 2024-05-21T15:36:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.934+0000 7fbf917fa640 1 -- 172.21.15.38:0/2680013510 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c019780 con 0x55764ca6e4d0 2024-05-21T15:36:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.935+0000 7fbfa85b3740 1 -- 172.21.15.38:0/2680013510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca6e4d0 msgr2=0x55764ce37130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:19.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.935+0000 7fbf9b7fe640 1 --2- 172.21.15.38:0/3006944030 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf880a2b90 0x7fbf880a50a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55764ce37d80 tx=0x7fbf94030fc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.935+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/2680013510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca6e4d0 0x55764ce37130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c101250 tx=0x7fbf8c011810 comp rx=0 tx=0).stop 2024-05-21T15:36:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.935+0000 7fbfa85b3740 1 -- 172.21.15.38:0/2680013510 shutdown_connections 2024-05-21T15:36:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.935+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/2680013510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca766b0 0x55764ce3e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.935+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/2680013510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca6e4d0 0x55764ce37130 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.935+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/2680013510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca97390 0x55764ce36bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.935+0000 7fbfa85b3740 1 -- 172.21.15.38:0/2680013510 >> 172.21.15.38:0/2680013510 conn(0x55764ca8b4e0 msgr2=0x55764ce32540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.935+0000 7fbfa85b3740 1 -- 172.21.15.38:0/2680013510 shutdown_connections 2024-05-21T15:36:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.935+0000 7fbfa85b3740 1 -- 172.21.15.38:0/2680013510 wait complete. 2024-05-21T15:36:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.935+0000 7fbfa85b3740 1 -- start start 2024-05-21T15:36:19.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbfa85b3740 1 Processor -- start 2024-05-21T15:36:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbfa85b3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca6e4d0 0x55764ce40aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbfa85b3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca766b0 0x55764ce40fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbfa85b3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca97390 0x55764ce47f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbfa85b3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55764ce3ea80 con 0x55764ca766b0 2024-05-21T15:36:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbfa85b3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55764cd56b90 con 0x55764ca6e4d0 2024-05-21T15:36:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbfa85b3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55764ce3ec00 con 0x55764ca97390 2024-05-21T15:36:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbfa85b3740 10 client.0 _ll_register_callbacks cb 0x55764cd92da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbf93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca766b0 0x55764ce40fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbf9affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca6e4d0 0x55764ce40aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbf9affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca6e4d0 0x55764ce40aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53210/0 (socket says 172.21.15.38:53210) 2024-05-21T15:36:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbf9affd640 1 -- 172.21.15.38:0/1429685793 learned_addr learned my addr 172.21.15.38:0/1429685793 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbf9b7fe640 1 --2- 172.21.15.38:0/1429685793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca97390 0x55764ce47f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbf9affd640 1 -- 172.21.15.38:0/1429685793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca97390 msgr2=0x55764ce47f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbf9affd640 1 --2- 172.21.15.38:0/1429685793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca97390 0x55764ce47f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbf9affd640 1 -- 172.21.15.38:0/1429685793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca766b0 msgr2=0x55764ce40fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbf9affd640 1 --2- 172.21.15.38:0/1429685793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca766b0 0x55764ce40fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbf9affd640 1 -- 172.21.15.38:0/1429685793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55764caaf3f0 con 0x55764ca6e4d0 2024-05-21T15:36:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.936+0000 7fbf93fff640 1 --2- 172.21.15.38:0/1429685793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca766b0 0x55764ce40fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbf9affd640 1 --2- 172.21.15.38:0/1429685793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca6e4d0 0x55764ce40aa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c01b040 tx=0x7fbf8c005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45aefeacb0e93170 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbf827fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbf827fc640 1 -- 172.21.15.38:0/1429685793 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c027070 con 0x55764ca6e4d0 2024-05-21T15:36:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbf827fc640 1 -- 172.21.15.38:0/1429685793 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbf8c0054d0 con 0x55764ca6e4d0 2024-05-21T15:36:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbfa85b3740 1 -- 172.21.15.38:0/1429685793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf8c009e10 con 0x55764ca6e4d0 2024-05-21T15:36:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbf827fc640 1 -- 172.21.15.38:0/1429685793 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf8c033050 con 0x55764ca6e4d0 2024-05-21T15:36:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbfa85b3740 10 client.25619 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbfa85b3740 1 -- 172.21.15.38:0/1429685793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55764ca6f0f0 con 0x55764ca6e4d0 2024-05-21T15:36:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbf827fc640 1 -- 172.21.15.38:0/1429685793 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf8c009e10 con 0x55764ca6e4d0 2024-05-21T15:36:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbf827fc640 1 -- 172.21.15.38:0/1429685793 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbf8c023500 con 0x55764ca6e4d0 2024-05-21T15:36:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbfa85b3740 10 client.25619 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbfa85b3740 10 client.25619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:19.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbfa85b3740 20 client.25619 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:19.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/1429685793 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55764ca55f60 0x55764ce4e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbfa85b3740 4 client.25619 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:19.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.937+0000 7fbfa85b3740 1 -- 172.21.15.38:0/1429685793 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55764cda9430 con 0x55764ca55f60 2024-05-21T15:36:19.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.938+0000 7fbf93fff640 1 --2- 172.21.15.38:0/1429685793 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55764ca55f60 0x55764ce4e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.938+0000 7fbf93fff640 1 --2- 172.21.15.38:0/1429685793 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55764ca55f60 0x55764ce4e630 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=140e3baf14a823cf server_cookie=1052c0c4909098b2 in_seq=0 out_seq=0 2024-05-21T15:36:19.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.938+0000 7fbf827fc640 10 client.25619 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:19.943 INFO:tasks.workunit.client.0.smithi038.stdout:checking for ANSI C header files... yes 2024-05-21T15:36:19.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.939+0000 7fbf827fc640 1 -- 172.21.15.38:0/1429685793 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55764cda9430 con 0x55764ca55f60 2024-05-21T15:36:19.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.939+0000 7fbf827fc640 10 client.25619 handle_command_reply: tid=0 2024-05-21T15:36:19.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.939+0000 7fbfa85b3740 1 client.25619 shutdown 2024-05-21T15:36:19.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.939+0000 7fbfa85b3740 1 -- 172.21.15.38:0/1429685793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca6e4d0 msgr2=0x55764ce40aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:19.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.939+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/1429685793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca6e4d0 0x55764ce40aa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c01b040 tx=0x7fbf8c005d90 comp rx=0 tx=0).stop 2024-05-21T15:36:19.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.939+0000 7fbfa85b3740 1 -- 172.21.15.38:0/1429685793 shutdown_connections 2024-05-21T15:36:19.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.939+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/1429685793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca97390 0x55764ce47f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.939+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/1429685793 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55764ca55f60 0x55764ce4e630 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.939+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/1429685793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca766b0 0x55764ce40fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.939+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/1429685793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca6e4d0 0x55764ce40aa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.939+0000 7fbf827fc640 0 client.25619 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:19.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.939+0000 7fbfa85b3740 1 -- 172.21.15.38:0/1429685793 >> 172.21.15.38:0/1429685793 conn(0x55764ca8b4e0 msgr2=0x55764ce315c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:19.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.939+0000 7fbfa85b3740 1 -- 172.21.15.38:0/1429685793 shutdown_connections 2024-05-21T15:36:19.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.940+0000 7fbfa85b3740 1 -- 172.21.15.38:0/1429685793 wait complete. 2024-05-21T15:36:19.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.940+0000 7fbfa85b3740 20 client.25619 trim_cache size 0 max 16384 2024-05-21T15:36:19.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.958+0000 7fbfa85b3740 1 Processor -- start 2024-05-21T15:36:19.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.958+0000 7fbfa85b3740 1 -- start start 2024-05-21T15:36:19.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.958+0000 7fbfa85b3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca6e4d0 0x55764ce410a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.958+0000 7fbfa85b3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca766b0 0x55764ce3b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.958+0000 7fbfa85b3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca97390 0x55764ce3f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.958+0000 7fbfa85b3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1b0a50 con 0x55764ca6e4d0 2024-05-21T15:36:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.958+0000 7fbfa85b3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1b2590 con 0x55764ca97390 2024-05-21T15:36:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.958+0000 7fbfa85b3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1b1120 con 0x55764ca766b0 2024-05-21T15:36:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.958+0000 7fbf9affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca6e4d0 0x55764ce410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.958+0000 7fbf9affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca6e4d0 0x55764ce410a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46702/0 (socket says 172.21.15.38:46702) 2024-05-21T15:36:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.958+0000 7fbf9affd640 1 -- 172.21.15.38:0/1602486522 learned_addr learned my addr 172.21.15.38:0/1602486522 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.959+0000 7fbf9b7fe640 1 --2- 172.21.15.38:0/1602486522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca97390 0x55764ce3f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.959+0000 7fbf93fff640 1 --2- 172.21.15.38:0/1602486522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca766b0 0x55764ce3b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.959+0000 7fbf9affd640 1 -- 172.21.15.38:0/1602486522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca766b0 msgr2=0x55764ce3b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.959+0000 7fbf9affd640 1 --2- 172.21.15.38:0/1602486522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca766b0 0x55764ce3b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.959+0000 7fbf9affd640 1 -- 172.21.15.38:0/1602486522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca97390 msgr2=0x55764ce3f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.959+0000 7fbf9affd640 1 --2- 172.21.15.38:0/1602486522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca97390 0x55764ce3f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.959+0000 7fbf9affd640 1 -- 172.21.15.38:0/1602486522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55764ce23990 con 0x55764ca6e4d0 2024-05-21T15:36:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.959+0000 7fbf9affd640 1 --2- 172.21.15.38:0/1602486522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca6e4d0 0x55764ce410a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0238d0 tx=0x7fbf8c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafb973dab21f372 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.959+0000 7fbf90ff9640 1 -- 172.21.15.38:0/1602486522 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c027070 con 0x55764ca6e4d0 2024-05-21T15:36:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.959+0000 7fbf90ff9640 1 -- 172.21.15.38:0/1602486522 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf8c030030 con 0x55764ca6e4d0 2024-05-21T15:36:19.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.959+0000 7fbf90ff9640 1 -- 172.21.15.38:0/1602486522 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c02c830 con 0x55764ca6e4d0 2024-05-21T15:36:19.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.960+0000 7fbfa85b3740 1 -- 172.21.15.38:0/1602486522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca6e4d0 msgr2=0x55764ce410a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:19.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.960+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/1602486522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca6e4d0 0x55764ce410a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0238d0 tx=0x7fbf8c0096a0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.960+0000 7fbfa85b3740 1 -- 172.21.15.38:0/1602486522 shutdown_connections 2024-05-21T15:36:19.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.960+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/1602486522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca97390 0x55764ce3f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.960+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/1602486522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca766b0 0x55764ce3b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.960+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/1602486522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca6e4d0 0x55764ce410a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.960+0000 7fbfa85b3740 1 -- 172.21.15.38:0/1602486522 >> 172.21.15.38:0/1602486522 conn(0x55764ca8b4e0 msgr2=0x55764ce3f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:19.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.960+0000 7fbfa85b3740 1 -- 172.21.15.38:0/1602486522 shutdown_connections 2024-05-21T15:36:19.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.960+0000 7fbfa85b3740 1 -- 172.21.15.38:0/1602486522 wait complete. 2024-05-21T15:36:19.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.961+0000 7fbfa85b3740 1 -- start start 2024-05-21T15:36:19.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.961+0000 7fbfa85b3740 1 Processor -- start 2024-05-21T15:36:19.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.961+0000 7fbfa85b3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca6e4d0 0x55764ce44410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.961+0000 7fbfa85b3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca766b0 0x55764ce44950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.961+0000 7fbfa85b3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca97390 0x55764ceb4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.961+0000 7fbfa85b3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1b1ec0 con 0x55764ca766b0 2024-05-21T15:36:19.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.961+0000 7fbfa85b3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1b17f0 con 0x55764ca97390 2024-05-21T15:36:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.961+0000 7fbfa85b3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1aea30 con 0x55764ca6e4d0 2024-05-21T15:36:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.961+0000 7fbfa85b3740 10 client.0 _ll_register_callbacks cb 0x7fbf84005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.961+0000 7fbf93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca766b0 0x55764ce44950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.961+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca97390 0x55764ceb4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.961+0000 7fbf93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca766b0 0x55764ce44950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46706/0 (socket says 172.21.15.38:46706) 2024-05-21T15:36:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.961+0000 7fbf93fff640 1 -- 172.21.15.38:0/3225933263 learned_addr learned my addr 172.21.15.38:0/3225933263 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.961+0000 7fbf9affd640 1 --2- 172.21.15.38:0/3225933263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca6e4d0 0x55764ce44410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.962+0000 7fbf9b7fe640 1 -- 172.21.15.38:0/3225933263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca6e4d0 msgr2=0x55764ce44410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.962+0000 7fbf9b7fe640 1 --2- 172.21.15.38:0/3225933263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca6e4d0 0x55764ce44410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.962+0000 7fbf9b7fe640 1 -- 172.21.15.38:0/3225933263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca766b0 msgr2=0x55764ce44950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.962+0000 7fbf9b7fe640 1 --2- 172.21.15.38:0/3225933263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca766b0 0x55764ce44950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.962+0000 7fbf9b7fe640 1 -- 172.21.15.38:0/3225933263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55764ce44f60 con 0x55764ca97390 2024-05-21T15:36:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.962+0000 7fbf93fff640 1 --2- 172.21.15.38:0/3225933263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca766b0 0x55764ce44950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.962+0000 7fbf9b7fe640 1 --2- 172.21.15.38:0/3225933263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca97390 0x55764ceb4990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf9402abe0 tx=0x7fbf94009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee32cf84dae4d1ea server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.962+0000 7fbf827fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.962+0000 7fbf827fc640 1 -- 172.21.15.38:0/3225933263 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94036050 con 0x55764ca97390 2024-05-21T15:36:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.962+0000 7fbfa85b3740 1 -- 172.21.15.38:0/3225933263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf8c030030 con 0x55764ca97390 2024-05-21T15:36:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.962+0000 7fbf827fc640 1 -- 172.21.15.38:0/3225933263 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbf940045d0 con 0x55764ca97390 2024-05-21T15:36:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.962+0000 7fbf827fc640 1 -- 172.21.15.38:0/3225933263 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf940047d0 con 0x55764ca97390 2024-05-21T15:36:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.962+0000 7fbfa85b3740 10 client.25625 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.962+0000 7fbfa85b3740 1 -- 172.21.15.38:0/3225933263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55764ceb5fa0 con 0x55764ca97390 2024-05-21T15:36:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.963+0000 7fbf827fc640 1 -- 172.21.15.38:0/3225933263 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf8c030030 con 0x55764ca97390 2024-05-21T15:36:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.963+0000 7fbf827fc640 1 -- 172.21.15.38:0/3225933263 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbf94055020 con 0x55764ca97390 2024-05-21T15:36:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.963+0000 7fbfa85b3740 10 client.25625 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.963+0000 7fbfa85b3740 10 client.25625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:19.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.963+0000 7fbfa85b3740 20 client.25625 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.963+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/3225933263 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55764ca2b3a0 0x55764ceb89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.963+0000 7fbfa85b3740 4 client.25625 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:36:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.963+0000 7fbfa85b3740 1 -- 172.21.15.38:0/3225933263 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55764cda3780 con 0x55764ca2b3a0 2024-05-21T15:36:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.963+0000 7fbf9affd640 1 --2- 172.21.15.38:0/3225933263 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55764ca2b3a0 0x55764ceb89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbf827fc640 10 client.25625 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbf9affd640 1 --2- 172.21.15.38:0/3225933263 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55764ca2b3a0 0x55764ceb89c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae65f18b9414f4e7 server_cookie=68d23fe7e37be4d4 in_seq=0 out_seq=0 2024-05-21T15:36:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbf827fc640 1 -- 172.21.15.38:0/3225933263 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55764cda3780 con 0x55764ca2b3a0 2024-05-21T15:36:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbf827fc640 10 client.25625 handle_command_reply: tid=0 2024-05-21T15:36:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbfa85b3740 1 client.25625 shutdown 2024-05-21T15:36:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbfa85b3740 1 -- 172.21.15.38:0/3225933263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca97390 msgr2=0x55764ceb4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/3225933263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca97390 0x55764ceb4990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf9402abe0 tx=0x7fbf94009500 comp rx=0 tx=0).stop 2024-05-21T15:36:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbfa85b3740 1 -- 172.21.15.38:0/3225933263 shutdown_connections 2024-05-21T15:36:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/3225933263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55764ca97390 0x55764ceb4990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/3225933263 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55764ca2b3a0 0x55764ceb89c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/3225933263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55764ca766b0 0x55764ce44950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbfa85b3740 1 --2- 172.21.15.38:0/3225933263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55764ca6e4d0 0x55764ce44410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbfa85b3740 1 -- 172.21.15.38:0/3225933263 >> 172.21.15.38:0/3225933263 conn(0x55764ca8b4e0 msgr2=0x55764ce40de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:19.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbf827fc640 0 client.25625 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:19.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbfa85b3740 1 -- 172.21.15.38:0/3225933263 shutdown_connections 2024-05-21T15:36:19.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbfa85b3740 1 -- 172.21.15.38:0/3225933263 wait complete. 2024-05-21T15:36:19.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.964+0000 7fbfa85b3740 20 client.25625 trim_cache size 0 max 16384 2024-05-21T15:36:19.969 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:36:19.969 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-05-21T15:36:19.970 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-05-21T15:36:19.970 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:36:19.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.965+0000 7fbfa1637640 1 -- 172.21.15.38:0/3006944030 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf880a2b90 msgr2=0x7fbf880a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.965+0000 7fbfa1637640 1 --2- 172.21.15.38:0/3006944030 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf880a2b90 0x7fbf880a50a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55764ce37d80 tx=0x7fbf94030fc0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.965+0000 7fbfa1637640 1 -- 172.21.15.38:0/3006944030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c102bc0 msgr2=0x7fbf9c0fd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.965+0000 7fbfa1637640 1 --2- 172.21.15.38:0/3006944030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c102bc0 0x7fbf9c0fd0a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400e9c0 tx=0x7fbf8400ee80 comp rx=0 tx=0).stop 2024-05-21T15:36:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.965+0000 7fbfa1637640 1 -- 172.21.15.38:0/3006944030 shutdown_connections 2024-05-21T15:36:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.965+0000 7fbfa1637640 1 --2- 172.21.15.38:0/3006944030 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf880a2b90 0x7fbf880a50a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.965+0000 7fbfa1637640 1 --2- 172.21.15.38:0/3006944030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf9c1078c0 0x7fbf9c0fd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.965+0000 7fbfa1637640 1 --2- 172.21.15.38:0/3006944030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c102bc0 0x7fbf9c0fd0a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.965+0000 7fbfa1637640 1 --2- 172.21.15.38:0/3006944030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf9c101980 0x7fbf9c0fea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.965+0000 7fbfa1637640 1 -- 172.21.15.38:0/3006944030 >> 172.21.15.38:0/3006944030 conn(0x7fbf9c0fcb70 msgr2=0x7fbf9c108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.965+0000 7fbfa1637640 1 -- 172.21.15.38:0/3006944030 shutdown_connections 2024-05-21T15:36:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:19.965+0000 7fbfa1637640 1 -- 172.21.15.38:0/3006944030 wait complete. 2024-05-21T15:36:19.976 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:36:19.976 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:36:19.976 INFO:tasks.workunit.client.0.smithi038.stdout:checking for sys/types.h... yes 2024-05-21T15:36:19.999 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:19 smithi038 ceph-mon[31995]: pgmap v408: 97 pgs: 97 active+clean; 3.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 295 KiB/s wr, 57 op/s 2024-05-21T15:36:20.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:19 smithi045 ceph-mon[36409]: pgmap v408: 97 pgs: 97 active+clean; 3.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 295 KiB/s wr, 57 op/s 2024-05-21T15:36:20.016 INFO:tasks.workunit.client.0.smithi038.stdout:checking for sys/stat.h... yes 2024-05-21T15:36:20.056 INFO:tasks.workunit.client.0.smithi038.stdout:checking for stdlib.h... yes 2024-05-21T15:36:20.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.096+0000 7f7c41421640 1 -- 172.21.15.38:0/2735181047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c3c107970 msgr2=0x7f7c3c109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.096+0000 7f7c41421640 1 --2- 172.21.15.38:0/2735181047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c3c107970 0x7f7c3c109d90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009ca0 tx=0x7f7c3402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.096+0000 7f7c41421640 1 -- 172.21.15.38:0/2735181047 shutdown_connections 2024-05-21T15:36:20.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.096+0000 7f7c41421640 1 --2- 172.21.15.38:0/2735181047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c3c107970 0x7f7c3c109d90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.096+0000 7f7c41421640 1 --2- 172.21.15.38:0/2735181047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c3c102c70 0x7f7c3c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.096+0000 7f7c41421640 1 --2- 172.21.15.38:0/2735181047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3c101a30 0x7f7c3c102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.096+0000 7f7c41421640 1 -- 172.21.15.38:0/2735181047 >> 172.21.15.38:0/2735181047 conn(0x7f7c3c0fcc40 msgr2=0x7f7c3c0ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.096+0000 7f7c41421640 1 -- 172.21.15.38:0/2735181047 shutdown_connections 2024-05-21T15:36:20.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.096+0000 7f7c41421640 1 -- 172.21.15.38:0/2735181047 wait complete. 2024-05-21T15:36:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.097+0000 7f7c41421640 1 Processor -- start 2024-05-21T15:36:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.097+0000 7f7c41421640 1 -- start start 2024-05-21T15:36:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.097+0000 7f7c41421640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c3c101a30 0x7f7c3c06ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.097+0000 7f7c41421640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3c102c70 0x7f7c3c19df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.097+0000 7f7c41421640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c3c107970 0x7f7c3c1a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.097+0000 7f7c41421640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c10d640 con 0x7f7c3c107970 2024-05-21T15:36:20.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.097+0000 7f7c41421640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c10d4c0 con 0x7f7c3c102c70 2024-05-21T15:36:20.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.097+0000 7f7c41421640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c10d7c0 con 0x7f7c3c101a30 2024-05-21T15:36:20.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c3b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c3c107970 0x7f7c3c1a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c3a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3c102c70 0x7f7c3c19df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c3b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c3c107970 0x7f7c3c1a04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46726/0 (socket says 172.21.15.38:46726) 2024-05-21T15:36:20.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c3b7fe640 1 -- 172.21.15.38:0/2128040485 learned_addr learned my addr 172.21.15.38:0/2128040485 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c3affd640 1 --2- 172.21.15.38:0/2128040485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c3c101a30 0x7f7c3c06ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c3a7fc640 1 -- 172.21.15.38:0/2128040485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c3c101a30 msgr2=0x7f7c3c06ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c3a7fc640 1 --2- 172.21.15.38:0/2128040485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c3c101a30 0x7f7c3c06ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c3a7fc640 1 -- 172.21.15.38:0/2128040485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c3c107970 msgr2=0x7f7c3c1a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c3a7fc640 1 --2- 172.21.15.38:0/2128040485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c3c107970 0x7f7c3c1a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c3a7fc640 1 -- 172.21.15.38:0/2128040485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3c1a0ce0 con 0x7f7c3c102c70 2024-05-21T15:36:20.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c3b7fe640 1 --2- 172.21.15.38:0/2128040485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c3c107970 0x7f7c3c1a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:20.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c3a7fc640 1 --2- 172.21.15.38:0/2128040485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3c102c70 0x7f7c3c19df50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000d720 tx=0x7f7c3000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c23fff640 1 -- 172.21.15.38:0/2128040485 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30017070 con 0x7f7c3c102c70 2024-05-21T15:36:20.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c23fff640 1 -- 172.21.15.38:0/2128040485 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3000de30 con 0x7f7c3c102c70 2024-05-21T15:36:20.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c41421640 1 -- 172.21.15.38:0/2128040485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3c1a0fd0 con 0x7f7c3c102c70 2024-05-21T15:36:20.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c23fff640 1 -- 172.21.15.38:0/2128040485 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c300047a0 con 0x7f7c3c102c70 2024-05-21T15:36:20.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.098+0000 7f7c41421640 1 -- 172.21.15.38:0/2128040485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c3c1a1430 con 0x7f7c3c102c70 2024-05-21T15:36:20.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.099+0000 7f7c41421640 1 -- 172.21.15.38:0/2128040485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c3c106070 con 0x7f7c3c102c70 2024-05-21T15:36:20.101 INFO:tasks.workunit.client.0.smithi038.stdout:checking for string.h... yes 2024-05-21T15:36:20.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.101+0000 7f7c23fff640 1 -- 172.21.15.38:0/2128040485 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7c300099d0 con 0x7f7c3c102c70 2024-05-21T15:36:20.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.102+0000 7f7c23fff640 1 --2- 172.21.15.38:0/2128040485 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7c240a2c30 0x7f7c240a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.102+0000 7f7c3affd640 1 --2- 172.21.15.38:0/2128040485 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7c240a2c30 0x7f7c240a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.102+0000 7f7c23fff640 1 -- 172.21.15.38:0/2128040485 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7c300cee00 con 0x7f7c3c102c70 2024-05-21T15:36:20.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.102+0000 7f7c3affd640 1 --2- 172.21.15.38:0/2128040485 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7c240a2c30 0x7f7c240a5140 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c102630 tx=0x7f7c2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.105+0000 7f7c23fff640 1 -- 172.21.15.38:0/2128040485 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7c30020020 con 0x7f7c3c102c70 2024-05-21T15:36:20.145 INFO:tasks.workunit.client.0.smithi038.stdout:checking for memory.h... yes 2024-05-21T15:36:20.191 INFO:tasks.workunit.client.0.smithi038.stdout:checking for strings.h... yes 2024-05-21T15:36:20.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:19 smithi053 ceph-mon[36496]: pgmap v408: 97 pgs: 97 active+clean; 3.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 295 KiB/s wr, 57 op/s 2024-05-21T15:36:20.238 INFO:tasks.workunit.client.0.smithi038.stdout:checking for inttypes.h... yes 2024-05-21T15:36:20.280 INFO:tasks.workunit.client.0.smithi038.stdout:checking for stdint.h... yes 2024-05-21T15:36:20.328 INFO:tasks.workunit.client.0.smithi038.stdout:checking for unistd.h... yes 2024-05-21T15:36:20.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.334+0000 7f7c41421640 1 -- 172.21.15.38:0/2128040485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c3c02d340 con 0x7f7c3c102c70 2024-05-21T15:36:20.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.335+0000 7f7c23fff640 1 -- 172.21.15.38:0/2128040485 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11914 (secure 0 0 0) 0x7f7c3009bcb0 con 0x7f7c3c102c70 2024-05-21T15:36:20.334 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:36:20.334 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:34:36.440701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24551,24523,24602,24550,24515]},"id":1}]} 2024-05-21T15:36:20.334 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-05-21T15:36:20.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.337+0000 7f7c41421640 1 -- 172.21.15.38:0/2128040485 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7c240a2c30 msgr2=0x7f7c240a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.337+0000 7f7c41421640 1 --2- 172.21.15.38:0/2128040485 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7c240a2c30 0x7f7c240a5140 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c102630 tx=0x7f7c2c005f90 comp rx=0 tx=0).stop 2024-05-21T15:36:20.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.337+0000 7f7c41421640 1 -- 172.21.15.38:0/2128040485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3c102c70 msgr2=0x7f7c3c19df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.337+0000 7f7c41421640 1 --2- 172.21.15.38:0/2128040485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3c102c70 0x7f7c3c19df50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000d720 tx=0x7f7c3000dbe0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.337+0000 7f7c41421640 1 -- 172.21.15.38:0/2128040485 shutdown_connections 2024-05-21T15:36:20.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.337+0000 7f7c41421640 1 --2- 172.21.15.38:0/2128040485 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7c240a2c30 0x7f7c240a5140 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.337+0000 7f7c41421640 1 --2- 172.21.15.38:0/2128040485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c3c107970 0x7f7c3c1a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.337+0000 7f7c41421640 1 --2- 172.21.15.38:0/2128040485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3c102c70 0x7f7c3c19df50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.337+0000 7f7c41421640 1 --2- 172.21.15.38:0/2128040485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c3c101a30 0x7f7c3c06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.337+0000 7f7c41421640 1 -- 172.21.15.38:0/2128040485 >> 172.21.15.38:0/2128040485 conn(0x7f7c3c0fcc40 msgr2=0x7f7c3c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.337+0000 7f7c41421640 1 -- 172.21.15.38:0/2128040485 shutdown_connections 2024-05-21T15:36:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.337+0000 7f7c41421640 1 -- 172.21.15.38:0/2128040485 wait complete. 2024-05-21T15:36:20.348 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:36:20.377 INFO:tasks.workunit.client.0.smithi038.stdout:checking minix/config.h usability... no 2024-05-21T15:36:20.413 INFO:tasks.workunit.client.0.smithi038.stdout:checking minix/config.h presence... no 2024-05-21T15:36:20.413 INFO:tasks.workunit.client.0.smithi038.stdout:checking for minix/config.h... no 2024-05-21T15:36:20.449 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-05-21T15:36:20.449 INFO:tasks.workunit.client.0.smithi038.stdout:checking for gcc... (cached) gcc 2024-05-21T15:36:20.486 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-21T15:36:20.486 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-21T15:36:20.486 INFO:tasks.workunit.client.0.smithi038.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-21T15:36:20.486 INFO:tasks.workunit.client.0.smithi038.stdout:checking dependency style of gcc... (cached) gcc3 2024-05-21T15:36:20.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.497+0000 7f0671d96640 1 -- 172.21.15.38:0/48065912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c1019c0 msgr2=0x7f066c1026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.497+0000 7f0671d96640 1 --2- 172.21.15.38:0/48065912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c1019c0 0x7f066c1026c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f065c009900 tx=0x7f065c02a410 comp rx=0 tx=0).stop 2024-05-21T15:36:20.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.497+0000 7f0671d96640 1 -- 172.21.15.38:0/48065912 shutdown_connections 2024-05-21T15:36:20.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.497+0000 7f0671d96640 1 --2- 172.21.15.38:0/48065912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f066c107900 0x7f066c109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.497+0000 7f0671d96640 1 --2- 172.21.15.38:0/48065912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f066c102c00 0x7f066c103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.497+0000 7f0671d96640 1 --2- 172.21.15.38:0/48065912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c1019c0 0x7f066c1026c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.497+0000 7f0671d96640 1 -- 172.21.15.38:0/48065912 >> 172.21.15.38:0/48065912 conn(0x7f066c0fcbb0 msgr2=0x7f066c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.497+0000 7f0671d96640 1 -- 172.21.15.38:0/48065912 shutdown_connections 2024-05-21T15:36:20.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.497+0000 7f0671d96640 1 -- 172.21.15.38:0/48065912 wait complete. 2024-05-21T15:36:20.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.498+0000 7f0671d96640 1 Processor -- start 2024-05-21T15:36:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.498+0000 7f0671d96640 1 -- start start 2024-05-21T15:36:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.498+0000 7f0671d96640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f066c1019c0 0x7f066c06ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.498+0000 7f0671d96640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f066c102c00 0x7f066c19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.498+0000 7f0671d96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c107900 0x7f066c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.498+0000 7f0671d96640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f066c10d5f0 con 0x7f066c107900 2024-05-21T15:36:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.498+0000 7f0671d96640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f066c10d470 con 0x7f066c102c00 2024-05-21T15:36:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.498+0000 7f0671d96640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f066c10d770 con 0x7f066c1019c0 2024-05-21T15:36:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f066bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c107900 0x7f066c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f066bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c107900 0x7f066c1a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46738/0 (socket says 172.21.15.38:46738) 2024-05-21T15:36:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f066bfff640 1 -- 172.21.15.38:0/762423919 learned_addr learned my addr 172.21.15.38:0/762423919 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f066b7fe640 1 --2- 172.21.15.38:0/762423919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f066c1019c0 0x7f066c06ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f066affd640 1 --2- 172.21.15.38:0/762423919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f066c102c00 0x7f066c19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f066bfff640 1 -- 172.21.15.38:0/762423919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f066c1019c0 msgr2=0x7f066c06ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f066bfff640 1 --2- 172.21.15.38:0/762423919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f066c1019c0 0x7f066c06ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f066bfff640 1 -- 172.21.15.38:0/762423919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f066c102c00 msgr2=0x7f066c19e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f066bfff640 1 --2- 172.21.15.38:0/762423919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f066c102c00 0x7f066c19e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f066bfff640 1 -- 172.21.15.38:0/762423919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066c1a0e20 con 0x7f066c107900 2024-05-21T15:36:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f066bfff640 1 --2- 172.21.15.38:0/762423919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c107900 0x7f066c1a0600 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f066400eaa0 tx=0x7f066400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f0668ff9640 1 -- 172.21.15.38:0/762423919 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066400cdb0 con 0x7f066c107900 2024-05-21T15:36:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f0671d96640 1 -- 172.21.15.38:0/762423919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066c1a1110 con 0x7f066c107900 2024-05-21T15:36:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f0668ff9640 1 -- 172.21.15.38:0/762423919 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0664005020 con 0x7f066c107900 2024-05-21T15:36:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f0668ff9640 1 -- 172.21.15.38:0/762423919 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06640116b0 con 0x7f066c107900 2024-05-21T15:36:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.499+0000 7f0671d96640 1 -- 172.21.15.38:0/762423919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066c1a1660 con 0x7f066c107900 2024-05-21T15:36:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.502+0000 7f0668ff9640 1 -- 172.21.15.38:0/762423919 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0664011850 con 0x7f066c107900 2024-05-21T15:36:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.502+0000 7f0678c40740 1 Processor -- start 2024-05-21T15:36:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.502+0000 7f0678c40740 1 -- start start 2024-05-21T15:36:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.502+0000 7f0678c40740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d917390 0x55dd1dcb6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f0678c40740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcb7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f0678c40740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcbdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f0678c40740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dd1dc1bc40 con 0x55dd1d917390 2024-05-21T15:36:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f0678c40740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dd1dbc3580 con 0x55dd1d8f66b0 2024-05-21T15:36:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f0678c40740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55dd1dc17c00 con 0x55dd1d8ee4d0 2024-05-21T15:36:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f066affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d917390 0x55dd1dcb6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f066affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d917390 0x55dd1dcb6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46740/0 (socket says 172.21.15.38:46740) 2024-05-21T15:36:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f066affd640 1 -- 172.21.15.38:0/1487935752 learned_addr learned my addr 172.21.15.38:0/1487935752 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f066b7fe640 1 --2- 172.21.15.38:0/1487935752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcb7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f0668ff9640 1 --2- 172.21.15.38:0/762423919 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f06580a2b90 0x7f06580a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f066affd640 1 --2- 172.21.15.38:0/1487935752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcbdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f066bfff640 1 --2- 172.21.15.38:0/762423919 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f06580a2b90 0x7f06580a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f066b7fe640 1 -- 172.21.15.38:0/1487935752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 msgr2=0x55dd1dcbdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f066b7fe640 1 --2- 172.21.15.38:0/1487935752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcbdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f066b7fe640 1 -- 172.21.15.38:0/1487935752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d917390 msgr2=0x55dd1dcb6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f066b7fe640 1 --2- 172.21.15.38:0/1487935752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d917390 0x55dd1dcb6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f066b7fe640 1 -- 172.21.15.38:0/1487935752 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd1d92f3f0 con 0x55dd1d8ee4d0 2024-05-21T15:36:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f0668ff9640 1 -- 172.21.15.38:0/762423919 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f06640cf2d0 con 0x7f066c107900 2024-05-21T15:36:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f066affd640 1 --2- 172.21.15.38:0/1487935752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d917390 0x55dd1dcb6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.503+0000 7f066b7fe640 1 --2- 172.21.15.38:0/1487935752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcb7080 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f065c02a930 tx=0x7f065c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbce3f9a142e25d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.504+0000 7f066bfff640 1 --2- 172.21.15.38:0/762423919 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f06580a2b90 0x7f06580a50a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f06640a41c0 tx=0x7f06640676c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.504+0000 7f0661ffb640 1 -- 172.21.15.38:0/1487935752 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c040070 con 0x55dd1d8ee4d0 2024-05-21T15:36:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.504+0000 7f0661ffb640 1 -- 172.21.15.38:0/1487935752 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f065c005020 con 0x55dd1d8ee4d0 2024-05-21T15:36:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.504+0000 7f0661ffb640 1 -- 172.21.15.38:0/1487935752 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c035040 con 0x55dd1d8ee4d0 2024-05-21T15:36:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.505+0000 7f0678c40740 1 -- 172.21.15.38:0/1487935752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d8ee4d0 msgr2=0x55dd1dcb7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.505+0000 7f0678c40740 1 --2- 172.21.15.38:0/1487935752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcb7080 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f065c02a930 tx=0x7f065c005e80 comp rx=0 tx=0).stop 2024-05-21T15:36:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.505+0000 7f0678c40740 1 -- 172.21.15.38:0/1487935752 shutdown_connections 2024-05-21T15:36:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.505+0000 7f0678c40740 1 --2- 172.21.15.38:0/1487935752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcbdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.505+0000 7f0678c40740 1 --2- 172.21.15.38:0/1487935752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcb7080 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.505+0000 7f0678c40740 1 --2- 172.21.15.38:0/1487935752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d917390 0x55dd1dcb6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.505+0000 7f0678c40740 1 -- 172.21.15.38:0/1487935752 >> 172.21.15.38:0/1487935752 conn(0x55dd1d90b4e0 msgr2=0x55dd1dcb2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.505+0000 7f0678c40740 1 -- 172.21.15.38:0/1487935752 shutdown_connections 2024-05-21T15:36:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.505+0000 7f0678c40740 1 -- 172.21.15.38:0/1487935752 wait complete. 2024-05-21T15:36:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.505+0000 7f0678c40740 1 -- start start 2024-05-21T15:36:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.505+0000 7f0678c40740 1 Processor -- start 2024-05-21T15:36:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f0678c40740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcc0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f0678c40740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f0678c40740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d917390 0x55dd1dcc7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f0678c40740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dd1dcbe970 con 0x55dd1d8ee4d0 2024-05-21T15:36:20.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f0678c40740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dd1dbd6c40 con 0x55dd1d8f66b0 2024-05-21T15:36:20.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f0678c40740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55dd1dcbeaf0 con 0x55dd1d917390 2024-05-21T15:36:20.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f0678c40740 10 client.0 _ll_register_callbacks cb 0x55dd1d92ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:20.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f066affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcc0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f066b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f066affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcc0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46754/0 (socket says 172.21.15.38:46754) 2024-05-21T15:36:20.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f066affd640 1 -- 172.21.15.38:0/61532049 learned_addr learned my addr 172.21.15.38:0/61532049 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f066affd640 1 --2- 172.21.15.38:0/61532049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d917390 0x55dd1dcc7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f066b7fe640 1 -- 172.21.15.38:0/61532049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d917390 msgr2=0x55dd1dcc7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f066b7fe640 1 --2- 172.21.15.38:0/61532049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d917390 0x55dd1dcc7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f066b7fe640 1 -- 172.21.15.38:0/61532049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d8ee4d0 msgr2=0x55dd1dcc0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f066b7fe640 1 --2- 172.21.15.38:0/61532049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcc0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f066b7fe640 1 -- 172.21.15.38:0/61532049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd1d9f97d0 con 0x55dd1d8f66b0 2024-05-21T15:36:20.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.506+0000 7f066affd640 1 --2- 172.21.15.38:0/61532049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d917390 0x55dd1dcc7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:36:20.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.507+0000 7f066b7fe640 1 --2- 172.21.15.38:0/61532049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc0dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f065c000c00 tx=0x7f065c004260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48930d823a6cbe9e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.507+0000 7f064affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:20.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.507+0000 7f064affd640 1 -- 172.21.15.38:0/61532049 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c040070 con 0x55dd1d8f66b0 2024-05-21T15:36:20.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.507+0000 7f064affd640 1 -- 172.21.15.38:0/61532049 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f065c031870 con 0x55dd1d8f66b0 2024-05-21T15:36:20.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.507+0000 7f0678c40740 1 -- 172.21.15.38:0/61532049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f065c005020 con 0x55dd1d8f66b0 2024-05-21T15:36:20.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.507+0000 7f064affd640 1 -- 172.21.15.38:0/61532049 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f065c035050 con 0x55dd1d8f66b0 2024-05-21T15:36:20.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.507+0000 7f0678c40740 10 client.25643 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:20.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.507+0000 7f0678c40740 1 -- 172.21.15.38:0/61532049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dd1dccaff0 con 0x55dd1d8f66b0 2024-05-21T15:36:20.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.507+0000 7f064affd640 1 -- 172.21.15.38:0/61532049 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f065c005020 con 0x55dd1d8f66b0 2024-05-21T15:36:20.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.507+0000 7f064affd640 1 -- 172.21.15.38:0/61532049 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f065c0035f0 con 0x55dd1d8f66b0 2024-05-21T15:36:20.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.507+0000 7f0678c40740 10 client.25643 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:20.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.507+0000 7f0678c40740 10 client.25643 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:36:20.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.508+0000 7f0678c40740 20 client.25643 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:20.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.508+0000 7f0678c40740 1 --2- 172.21.15.38:0/61532049 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55dd1d8d5f60 0x55dd1dcce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.508+0000 7f0678c40740 4 client.25643 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:20.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.508+0000 7f0678c40740 1 -- 172.21.15.38:0/61532049 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd1dc29370 con 0x55dd1d8d5f60 2024-05-21T15:36:20.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.508+0000 7f066affd640 1 --2- 172.21.15.38:0/61532049 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55dd1d8d5f60 0x55dd1dcce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.508+0000 7f064affd640 10 client.25643 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:36:20.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.508+0000 7f066affd640 1 --2- 172.21.15.38:0/61532049 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55dd1d8d5f60 0x55dd1dcce350 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=331d4fa4c63b2c6d server_cookie=a49b27fd7a52fdd0 in_seq=0 out_seq=0 2024-05-21T15:36:20.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.509+0000 7f064affd640 1 -- 172.21.15.38:0/61532049 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dd1dc29370 con 0x55dd1d8d5f60 2024-05-21T15:36:20.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.509+0000 7f064affd640 10 client.25643 handle_command_reply: tid=0 2024-05-21T15:36:20.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.509+0000 7f0678c40740 1 client.25643 shutdown 2024-05-21T15:36:20.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.510+0000 7f0678c40740 1 -- 172.21.15.38:0/61532049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 msgr2=0x55dd1dcc0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.510+0000 7f0678c40740 1 --2- 172.21.15.38:0/61532049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc0dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f065c000c00 tx=0x7f065c004260 comp rx=0 tx=0).stop 2024-05-21T15:36:20.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.510+0000 7f0678c40740 1 -- 172.21.15.38:0/61532049 shutdown_connections 2024-05-21T15:36:20.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.510+0000 7f0678c40740 1 --2- 172.21.15.38:0/61532049 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55dd1d8d5f60 0x55dd1dcce350 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.510+0000 7f0678c40740 1 --2- 172.21.15.38:0/61532049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d917390 0x55dd1dcc7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.510+0000 7f0678c40740 1 --2- 172.21.15.38:0/61532049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc0dd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.510+0000 7f0678c40740 1 --2- 172.21.15.38:0/61532049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcc0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.510+0000 7f064affd640 0 client.25643 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:36:20.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.510+0000 7f0678c40740 1 -- 172.21.15.38:0/61532049 >> 172.21.15.38:0/61532049 conn(0x55dd1d90b4e0 msgr2=0x55dd1dcb1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.510+0000 7f0678c40740 1 -- 172.21.15.38:0/61532049 shutdown_connections 2024-05-21T15:36:20.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.510+0000 7f0678c40740 1 -- 172.21.15.38:0/61532049 wait complete. 2024-05-21T15:36:20.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.510+0000 7f0678c40740 20 client.25643 trim_cache size 0 max 16384 2024-05-21T15:36:20.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.524+0000 7f0678c40740 1 Processor -- start 2024-05-21T15:36:20.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.524+0000 7f0678c40740 1 -- start start 2024-05-21T15:36:20.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.524+0000 7f0678c40740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcbf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.524+0000 7f0678c40740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.524+0000 7f0678c40740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d917390 0x55dd1dcd29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.524+0000 7f0678c40740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f066c1aa3b0 con 0x55dd1d917390 2024-05-21T15:36:20.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.524+0000 7f0678c40740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f066c1abef0 con 0x55dd1d8f66b0 2024-05-21T15:36:20.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.524+0000 7f0678c40740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f066c1aaa80 con 0x55dd1d8ee4d0 2024-05-21T15:36:20.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.524+0000 7f066b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.524+0000 7f066affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d917390 0x55dd1dcd29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.524+0000 7f066b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53298/0 (socket says 172.21.15.38:53298) 2024-05-21T15:36:20.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.524+0000 7f066b7fe640 1 -- 172.21.15.38:0/4100657986 learned_addr learned my addr 172.21.15.38:0/4100657986 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.524+0000 7f066affd640 1 --2- 172.21.15.38:0/4100657986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcbf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.524+0000 7f066b7fe640 1 -- 172.21.15.38:0/4100657986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d8ee4d0 msgr2=0x55dd1dcbf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.525+0000 7f066b7fe640 1 --2- 172.21.15.38:0/4100657986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcbf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.525+0000 7f066b7fe640 1 -- 172.21.15.38:0/4100657986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d917390 msgr2=0x55dd1dcd29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.525+0000 7f066b7fe640 1 --2- 172.21.15.38:0/4100657986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d917390 0x55dd1dcd29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.525+0000 7f066b7fe640 1 -- 172.21.15.38:0/4100657986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd1dcadd20 con 0x55dd1d8f66b0 2024-05-21T15:36:20.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.525+0000 7f066b7fe640 1 --2- 172.21.15.38:0/4100657986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc7c50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f065c003a40 tx=0x7f065c004590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdb9b6c7003a0e09 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.525+0000 7f06617fa640 1 -- 172.21.15.38:0/4100657986 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c040070 con 0x55dd1d8f66b0 2024-05-21T15:36:20.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.525+0000 7f06617fa640 1 -- 172.21.15.38:0/4100657986 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f065c005d50 con 0x55dd1d8f66b0 2024-05-21T15:36:20.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.525+0000 7f06617fa640 1 -- 172.21.15.38:0/4100657986 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c0053e0 con 0x55dd1d8f66b0 2024-05-21T15:36:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.526+0000 7f0678c40740 1 -- 172.21.15.38:0/4100657986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 msgr2=0x55dd1dcc7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.526+0000 7f0678c40740 1 --2- 172.21.15.38:0/4100657986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc7c50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f065c003a40 tx=0x7f065c004590 comp rx=0 tx=0).stop 2024-05-21T15:36:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.526+0000 7f0678c40740 1 -- 172.21.15.38:0/4100657986 shutdown_connections 2024-05-21T15:36:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.526+0000 7f0678c40740 1 --2- 172.21.15.38:0/4100657986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d917390 0x55dd1dcd29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.526+0000 7f0678c40740 1 --2- 172.21.15.38:0/4100657986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc7c50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.526+0000 7f0678c40740 1 --2- 172.21.15.38:0/4100657986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcbf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.526+0000 7f0678c40740 1 -- 172.21.15.38:0/4100657986 >> 172.21.15.38:0/4100657986 conn(0x55dd1d90b4e0 msgr2=0x55dd1dcbee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.526+0000 7f0678c40740 1 -- 172.21.15.38:0/4100657986 shutdown_connections 2024-05-21T15:36:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.526+0000 7f0678c40740 1 -- 172.21.15.38:0/4100657986 wait complete. 2024-05-21T15:36:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.526+0000 7f0678c40740 1 -- start start 2024-05-21T15:36:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f0678c40740 1 Processor -- start 2024-05-21T15:36:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f0678c40740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcc5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f0678c40740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f0678c40740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d917390 0x55dd1dcc63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f0678c40740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f066c1ab820 con 0x55dd1d8f66b0 2024-05-21T15:36:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f0678c40740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f066c1ab150 con 0x55dd1d8ee4d0 2024-05-21T15:36:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f0678c40740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f066c1a8390 con 0x55dd1d917390 2024-05-21T15:36:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f0678c40740 10 client.0 _ll_register_callbacks cb 0x7f065400b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f066b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f066b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46774/0 (socket says 172.21.15.38:46774) 2024-05-21T15:36:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f066b7fe640 1 -- 172.21.15.38:0/883705659 learned_addr learned my addr 172.21.15.38:0/883705659 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f066affd640 1 --2- 172.21.15.38:0/883705659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d917390 0x55dd1dcc63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f066affd640 1 --2- 172.21.15.38:0/883705659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcc5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f066b7fe640 1 -- 172.21.15.38:0/883705659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d917390 msgr2=0x55dd1dcc63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f066b7fe640 1 --2- 172.21.15.38:0/883705659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d917390 0x55dd1dcc63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f066b7fe640 1 -- 172.21.15.38:0/883705659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8ee4d0 msgr2=0x55dd1dcc5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f066b7fe640 1 --2- 172.21.15.38:0/883705659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcc5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.527+0000 7f066b7fe640 1 -- 172.21.15.38:0/883705659 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd1dcc6990 con 0x55dd1d8f66b0 2024-05-21T15:36:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f066b7fe640 1 --2- 172.21.15.38:0/883705659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc5580 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f065c003820 tx=0x7f065c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64942bb065316049 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f064affd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f064affd640 1 -- 172.21.15.38:0/883705659 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c040040 con 0x55dd1d8f66b0 2024-05-21T15:36:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f064affd640 1 -- 172.21.15.38:0/883705659 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f065c031870 con 0x55dd1d8f66b0 2024-05-21T15:36:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f0678c40740 1 -- 172.21.15.38:0/883705659 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f065c005d50 con 0x55dd1d8f66b0 2024-05-21T15:36:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f064affd640 1 -- 172.21.15.38:0/883705659 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f065c0053e0 con 0x55dd1d8f66b0 2024-05-21T15:36:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f0678c40740 10 client.15984 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f0678c40740 1 -- 172.21.15.38:0/883705659 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dd1dcc7c40 con 0x55dd1d8f66b0 2024-05-21T15:36:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f064affd640 1 -- 172.21.15.38:0/883705659 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f065c005d50 con 0x55dd1d8f66b0 2024-05-21T15:36:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f064affd640 1 -- 172.21.15.38:0/883705659 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f065c004dd0 con 0x55dd1d8f66b0 2024-05-21T15:36:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f0678c40740 10 client.15984 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f0678c40740 10 client.15984 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:36:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f0678c40740 20 client.15984 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f0678c40740 1 --2- 172.21.15.38:0/883705659 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55dd1d8ab3a0 0x55dd1dd38960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f0678c40740 4 client.15984 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:36:20.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.528+0000 7f0678c40740 1 -- 172.21.15.38:0/883705659 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd1dc236c0 con 0x55dd1d8ab3a0 2024-05-21T15:36:20.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.529+0000 7f066affd640 1 --2- 172.21.15.38:0/883705659 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55dd1d8ab3a0 0x55dd1dd38960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.529+0000 7f064affd640 10 client.15984 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:36:20.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.529+0000 7f066affd640 1 --2- 172.21.15.38:0/883705659 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55dd1d8ab3a0 0x55dd1dd38960 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=32a8628a4f1a12cc server_cookie=e65f3fdfd53b938b in_seq=0 out_seq=0 2024-05-21T15:36:20.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.529+0000 7f064affd640 1 -- 172.21.15.38:0/883705659 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd1dc236c0 con 0x55dd1d8ab3a0 2024-05-21T15:36:20.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.529+0000 7f064affd640 10 client.15984 handle_command_reply: tid=0 2024-05-21T15:36:20.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.529+0000 7f0678c40740 1 client.15984 shutdown 2024-05-21T15:36:20.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.529+0000 7f0678c40740 1 -- 172.21.15.38:0/883705659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d8f66b0 msgr2=0x55dd1dcc5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.529+0000 7f0678c40740 1 --2- 172.21.15.38:0/883705659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc5580 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f065c003820 tx=0x7f065c004610 comp rx=0 tx=0).stop 2024-05-21T15:36:20.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.529+0000 7f0678c40740 1 -- 172.21.15.38:0/883705659 shutdown_connections 2024-05-21T15:36:20.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.529+0000 7f0678c40740 1 --2- 172.21.15.38:0/883705659 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55dd1d8ab3a0 0x55dd1dd38960 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.529+0000 7f0678c40740 1 --2- 172.21.15.38:0/883705659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dd1d917390 0x55dd1dcc63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0678c40740 1 --2- 172.21.15.38:0/883705659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dd1d8f66b0 0x55dd1dcc5580 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f064affd640 0 client.15984 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:36:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0678c40740 1 --2- 172.21.15.38:0/883705659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd1d8ee4d0 0x55dd1dcc5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0678c40740 1 -- 172.21.15.38:0/883705659 >> 172.21.15.38:0/883705659 conn(0x55dd1d90b4e0 msgr2=0x55dd1dcb5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0678c40740 1 -- 172.21.15.38:0/883705659 shutdown_connections 2024-05-21T15:36:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0678c40740 1 -- 172.21.15.38:0/883705659 wait complete. 2024-05-21T15:36:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0678c40740 20 client.15984 trim_cache size 0 max 16384 2024-05-21T15:36:20.536 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:36:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0671d96640 1 -- 172.21.15.38:0/762423919 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f06580a2b90 msgr2=0x7f06580a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0671d96640 1 --2- 172.21.15.38:0/762423919 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f06580a2b90 0x7f06580a50a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f06640a41c0 tx=0x7f06640676c0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0671d96640 1 -- 172.21.15.38:0/762423919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c107900 msgr2=0x7f066c1a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0671d96640 1 --2- 172.21.15.38:0/762423919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c107900 0x7f066c1a0600 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f066400eaa0 tx=0x7f066400ef60 comp rx=0 tx=0).stop 2024-05-21T15:36:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0671d96640 1 -- 172.21.15.38:0/762423919 shutdown_connections 2024-05-21T15:36:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0671d96640 1 --2- 172.21.15.38:0/762423919 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f06580a2b90 0x7f06580a50a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0671d96640 1 --2- 172.21.15.38:0/762423919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f066c107900 0x7f066c1a0600 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0671d96640 1 --2- 172.21.15.38:0/762423919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f066c102c00 0x7f066c19e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0671d96640 1 --2- 172.21.15.38:0/762423919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f066c1019c0 0x7f066c06ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0671d96640 1 -- 172.21.15.38:0/762423919 >> 172.21.15.38:0/762423919 conn(0x7f066c0fcbb0 msgr2=0x7f066c1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0671d96640 1 -- 172.21.15.38:0/762423919 shutdown_connections 2024-05-21T15:36:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.530+0000 7f0671d96640 1 -- 172.21.15.38:0/762423919 wait complete. 2024-05-21T15:36:20.539 INFO:tasks.workunit.client.0.smithi038.stdout:checking for main in -lm... yes 2024-05-21T15:36:20.541 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:36:20.585 INFO:tasks.workunit.client.0.smithi038.stdout:checking for main in -lpthread... yes 2024-05-21T15:36:20.587 INFO:tasks.workunit.client.0.smithi038.stdout:checking for ANSI C header files... (cached) yes 2024-05-21T15:36:20.626 INFO:tasks.workunit.client.0.smithi038.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-05-21T15:36:20.695 INFO:tasks.workunit.client.0.smithi038.stdout:checking pthread.h usability... yes 2024-05-21T15:36:20.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.704+0000 7f121b2e2640 1 -- 172.21.15.38:0/2564589405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1214104020 msgr2=0x7f1214104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.704+0000 7f121b2e2640 1 --2- 172.21.15.38:0/2564589405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1214104020 0x7f1214104430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1204009be0 tx=0x7f120402a6c0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.704+0000 7f121b2e2640 1 -- 172.21.15.38:0/2564589405 shutdown_connections 2024-05-21T15:36:20.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.704+0000 7f121b2e2640 1 --2- 172.21.15.38:0/2564589405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12141011d0 0x7f121410ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.704+0000 7f121b2e2640 1 --2- 172.21.15.38:0/2564589405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12141006d0 0x7f1214100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.704+0000 7f121b2e2640 1 --2- 172.21.15.38:0/2564589405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1214104020 0x7f1214104430 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.704+0000 7f121b2e2640 1 -- 172.21.15.38:0/2564589405 >> 172.21.15.38:0/2564589405 conn(0x7f12140fc2f0 msgr2=0x7f12140fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.704+0000 7f121b2e2640 1 -- 172.21.15.38:0/2564589405 shutdown_connections 2024-05-21T15:36:20.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.704+0000 7f121b2e2640 1 -- 172.21.15.38:0/2564589405 wait complete. 2024-05-21T15:36:20.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121b2e2640 1 Processor -- start 2024-05-21T15:36:20.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121b2e2640 1 -- start start 2024-05-21T15:36:20.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121b2e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12141006d0 0x7f12141992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121b2e2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12141011d0 0x7f1214199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121b2e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1214104020 0x7f12141a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121b2e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f121410e690 con 0x7f12141006d0 2024-05-21T15:36:20.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121b2e2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f121410e510 con 0x7f12141011d0 2024-05-21T15:36:20.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121b2e2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f121410e810 con 0x7f1214104020 2024-05-21T15:36:20.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121907e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12141006d0 0x7f12141992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121907e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12141006d0 0x7f12141992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46794/0 (socket says 172.21.15.38:46794) 2024-05-21T15:36:20.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121907e640 1 -- 172.21.15.38:0/873725725 learned_addr learned my addr 172.21.15.38:0/873725725 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121887d640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12141011d0 0x7f1214199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121987f640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1214104020 0x7f12141a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121907e640 1 -- 172.21.15.38:0/873725725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1214104020 msgr2=0x7f12141a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121907e640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1214104020 0x7f12141a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121907e640 1 -- 172.21.15.38:0/873725725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12141011d0 msgr2=0x7f1214199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121907e640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12141011d0 0x7f1214199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121907e640 1 -- 172.21.15.38:0/873725725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12141a0e20 con 0x7f12141006d0 2024-05-21T15:36:20.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121987f640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1214104020 0x7f12141a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:36:20.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.705+0000 7f121887d640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12141011d0 0x7f1214199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:20.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.706+0000 7f121907e640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12141006d0 0x7f12141992e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1204034040 tx=0x7f1204030630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.706+0000 7f12127fc640 1 -- 172.21.15.38:0/873725725 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204041020 con 0x7f12141006d0 2024-05-21T15:36:20.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.706+0000 7f12127fc640 1 -- 172.21.15.38:0/873725725 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1204037040 con 0x7f12141006d0 2024-05-21T15:36:20.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.706+0000 7f121b2e2640 1 -- 172.21.15.38:0/873725725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12141a10b0 con 0x7f12141006d0 2024-05-21T15:36:20.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.706+0000 7f12127fc640 1 -- 172.21.15.38:0/873725725 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204036070 con 0x7f12141006d0 2024-05-21T15:36:20.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.706+0000 7f121b2e2640 1 -- 172.21.15.38:0/873725725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12141a15a0 con 0x7f12141006d0 2024-05-21T15:36:20.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.708+0000 7f12127fc640 1 -- 172.21.15.38:0/873725725 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1204044020 con 0x7f12141006d0 2024-05-21T15:36:20.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.708+0000 7f1222184740 1 Processor -- start 2024-05-21T15:36:20.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f1222184740 1 -- start start 2024-05-21T15:36:20.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f1222184740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 0x55d7aeef7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f1222184740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb2b4d0 0x55d7aeef8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f1222184740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb336b0 0x55d7aeeff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f1222184740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d7aee58c40 con 0x55d7aeb336b0 2024-05-21T15:36:20.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f1222184740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d7aee00580 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f1222184740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d7aee54c00 con 0x55d7aeb54390 2024-05-21T15:36:20.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f12127fc640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11fc0a2b90 0x7f11fc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f121887d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb336b0 0x55d7aeeff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f121887d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb336b0 0x55d7aeeff390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46808/0 (socket says 172.21.15.38:46808) 2024-05-21T15:36:20.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f121887d640 1 -- 172.21.15.38:0/1856717412 learned_addr learned my addr 172.21.15.38:0/1856717412 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f121987f640 1 --2- 172.21.15.38:0/1856717412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 0x55d7aeef7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f121907e640 1 --2- 172.21.15.38:0/1856717412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb2b4d0 0x55d7aeef8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f12127fc640 1 -- 172.21.15.38:0/873725725 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f12040efd60 con 0x7f12141006d0 2024-05-21T15:36:20.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f121987f640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11fc0a2b90 0x7f11fc0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f121887d640 1 -- 172.21.15.38:0/1856717412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 msgr2=0x55d7aeef7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f121887d640 1 --2- 172.21.15.38:0/1856717412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 0x55d7aeef7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f121887d640 1 -- 172.21.15.38:0/1856717412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb2b4d0 msgr2=0x55d7aeef8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f121887d640 1 --2- 172.21.15.38:0/1856717412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb2b4d0 0x55d7aeef8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f121887d640 1 -- 172.21.15.38:0/1856717412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7aeb6c3f0 con 0x55d7aeb336b0 2024-05-21T15:36:20.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f121987f640 1 --2- 172.21.15.38:0/1856717412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 0x55d7aeef7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:20.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.709+0000 7f121907e640 1 --2- 172.21.15.38:0/1856717412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb2b4d0 0x55d7aeef8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:20.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.710+0000 7f121987f640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11fc0a2b90 0x7f11fc0a50a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55d7aeef90e0 tx=0x7f120c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.710+0000 7f121887d640 1 --2- 172.21.15.38:0/1856717412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb336b0 0x55d7aeeff390 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f12080114f0 tx=0x7f12080119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a25fa02de240e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.710+0000 7f11fb7fe640 1 -- 172.21.15.38:0/1856717412 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208004020 con 0x55d7aeb336b0 2024-05-21T15:36:20.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.710+0000 7f11fb7fe640 1 -- 172.21.15.38:0/1856717412 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12080041c0 con 0x55d7aeb336b0 2024-05-21T15:36:20.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.710+0000 7f11fb7fe640 1 -- 172.21.15.38:0/1856717412 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120800f960 con 0x55d7aeb336b0 2024-05-21T15:36:20.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.711+0000 7f1222184740 1 -- 172.21.15.38:0/1856717412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb336b0 msgr2=0x55d7aeeff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.711+0000 7f1222184740 1 --2- 172.21.15.38:0/1856717412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb336b0 0x55d7aeeff390 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f12080114f0 tx=0x7f12080119b0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.711+0000 7f1222184740 1 -- 172.21.15.38:0/1856717412 shutdown_connections 2024-05-21T15:36:20.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.711+0000 7f1222184740 1 --2- 172.21.15.38:0/1856717412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb336b0 0x55d7aeeff390 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.711+0000 7f1222184740 1 --2- 172.21.15.38:0/1856717412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb2b4d0 0x55d7aeef8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.711+0000 7f1222184740 1 --2- 172.21.15.38:0/1856717412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 0x55d7aeef7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.711+0000 7f1222184740 1 -- 172.21.15.38:0/1856717412 >> 172.21.15.38:0/1856717412 conn(0x55d7aeb484e0 msgr2=0x55d7aeeef490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.711+0000 7f1222184740 1 -- 172.21.15.38:0/1856717412 shutdown_connections 2024-05-21T15:36:20.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.711+0000 7f1222184740 1 -- 172.21.15.38:0/1856717412 wait complete. 2024-05-21T15:36:20.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.711+0000 7f1222184740 1 -- start start 2024-05-21T15:36:20.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.711+0000 7f1222184740 1 Processor -- start 2024-05-21T15:36:20.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f1222184740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb2b4d0 0x55d7aef01c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f1222184740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb336b0 0x55d7aef021d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f1222184740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 0x55d7aef09130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f1222184740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d7aeeffde0 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f1222184740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d7aee13c40 con 0x55d7aeb336b0 2024-05-21T15:36:20.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f1222184740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d7aeefff60 con 0x55d7aeb54390 2024-05-21T15:36:20.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f1222184740 10 client.0 _ll_register_callbacks cb 0x55d7aeb6bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:20.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f121907e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb2b4d0 0x55d7aef01c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f121907e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb2b4d0 0x55d7aef01c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46816/0 (socket says 172.21.15.38:46816) 2024-05-21T15:36:20.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f121907e640 1 -- 172.21.15.38:0/547187115 learned_addr learned my addr 172.21.15.38:0/547187115 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f121887d640 1 --2- 172.21.15.38:0/547187115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb336b0 0x55d7aef021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f121987f640 1 --2- 172.21.15.38:0/547187115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 0x55d7aef09130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f121907e640 1 -- 172.21.15.38:0/547187115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 msgr2=0x55d7aef09130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f121907e640 1 --2- 172.21.15.38:0/547187115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 0x55d7aef09130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f121907e640 1 -- 172.21.15.38:0/547187115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb336b0 msgr2=0x55d7aef021d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f121907e640 1 --2- 172.21.15.38:0/547187115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb336b0 0x55d7aef021d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f121907e640 1 -- 172.21.15.38:0/547187115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7aec367d0 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f121907e640 1 --2- 172.21.15.38:0/547187115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb2b4d0 0x55d7aef01c90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f12040b89d0 tx=0x7f1204003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b37d61050b06af8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.712+0000 7f11f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:20.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.713+0000 7f11f8ff9640 1 -- 172.21.15.38:0/547187115 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12040bc0b0 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.713+0000 7f11f8ff9640 1 -- 172.21.15.38:0/547187115 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f12040c4d50 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.713+0000 7f1222184740 1 -- 172.21.15.38:0/547187115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12080041c0 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.713+0000 7f11f8ff9640 1 -- 172.21.15.38:0/547187115 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f12040bbed0 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.713+0000 7f1222184740 10 client.15996 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:20.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.713+0000 7f1222184740 1 -- 172.21.15.38:0/547187115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d7aef0c3f0 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.713+0000 7f11f8ff9640 1 -- 172.21.15.38:0/547187115 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12080041c0 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.713+0000 7f11f8ff9640 1 -- 172.21.15.38:0/547187115 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1204004580 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.713+0000 7f1222184740 10 client.15996 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:20.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.713+0000 7f1222184740 10 client.15996 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:36:20.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.713+0000 7f1222184740 20 client.15996 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:20.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.713+0000 7f1222184740 1 --2- 172.21.15.38:0/547187115 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55d7aeb12f60 0x55d7aef0fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.713+0000 7f1222184740 4 client.15996 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:20.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.713+0000 7f1222184740 1 -- 172.21.15.38:0/547187115 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7aee66370 con 0x55d7aeb12f60 2024-05-21T15:36:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.714+0000 7f121887d640 1 --2- 172.21.15.38:0/547187115 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55d7aeb12f60 0x55d7aef0fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.714+0000 7f11f8ff9640 10 client.15996 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:36:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.714+0000 7f121887d640 1 --2- 172.21.15.38:0/547187115 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55d7aeb12f60 0x55d7aef0fc40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ee3e6b974f1d66cf server_cookie=d5d42cc46240ea20 in_seq=0 out_seq=0 2024-05-21T15:36:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f11f8ff9640 1 -- 172.21.15.38:0/547187115 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d7aee66370 con 0x55d7aeb12f60 2024-05-21T15:36:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f11f8ff9640 10 client.15996 handle_command_reply: tid=0 2024-05-21T15:36:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f1222184740 1 client.15996 shutdown 2024-05-21T15:36:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f1222184740 1 -- 172.21.15.38:0/547187115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb2b4d0 msgr2=0x55d7aef01c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f1222184740 1 --2- 172.21.15.38:0/547187115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb2b4d0 0x55d7aef01c90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f12040b89d0 tx=0x7f1204003e40 comp rx=0 tx=0).stop 2024-05-21T15:36:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f1222184740 1 -- 172.21.15.38:0/547187115 shutdown_connections 2024-05-21T15:36:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f1222184740 1 --2- 172.21.15.38:0/547187115 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55d7aeb12f60 0x55d7aef0fc40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f1222184740 1 --2- 172.21.15.38:0/547187115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 0x55d7aef09130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f1222184740 1 --2- 172.21.15.38:0/547187115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb336b0 0x55d7aef021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f1222184740 1 --2- 172.21.15.38:0/547187115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb2b4d0 0x55d7aef01c90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f11f8ff9640 0 client.15996 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:36:20.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f1222184740 1 -- 172.21.15.38:0/547187115 >> 172.21.15.38:0/547187115 conn(0x55d7aeb484e0 msgr2=0x55d7aeeee570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f1222184740 1 -- 172.21.15.38:0/547187115 shutdown_connections 2024-05-21T15:36:20.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f1222184740 1 -- 172.21.15.38:0/547187115 wait complete. 2024-05-21T15:36:20.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.715+0000 7f1222184740 20 client.15996 trim_cache size 0 max 16384 2024-05-21T15:36:20.721 INFO:tasks.workunit.client.0.smithi038.stdout:checking pthread.h presence... yes 2024-05-21T15:36:20.721 INFO:tasks.workunit.client.0.smithi038.stdout:checking for pthread.h... yes 2024-05-21T15:36:20.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.729+0000 7f1222184740 1 Processor -- start 2024-05-21T15:36:20.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f1222184740 1 -- start start 2024-05-21T15:36:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f1222184740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb2b4d0 0x55d7aeefb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f1222184740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb336b0 0x55d7aef090c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f1222184740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 0x55d7aeef2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f1222184740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f12141a8220 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f1222184740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f12141a9d60 con 0x55d7aeb336b0 2024-05-21T15:36:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f1222184740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f12141a88f0 con 0x55d7aeb54390 2024-05-21T15:36:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f121907e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb2b4d0 0x55d7aeefb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f121887d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb336b0 0x55d7aef090c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f121907e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb2b4d0 0x55d7aeefb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46830/0 (socket says 172.21.15.38:46830) 2024-05-21T15:36:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f121907e640 1 -- 172.21.15.38:0/3748744803 learned_addr learned my addr 172.21.15.38:0/3748744803 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f121887d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb336b0 0x55d7aef090c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53346/0 (socket says 172.21.15.38:53346) 2024-05-21T15:36:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f121987f640 1 --2- 172.21.15.38:0/3748744803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 0x55d7aeef2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f121907e640 1 -- 172.21.15.38:0/3748744803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 msgr2=0x55d7aeef2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f121907e640 1 --2- 172.21.15.38:0/3748744803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 0x55d7aeef2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f121907e640 1 -- 172.21.15.38:0/3748744803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb336b0 msgr2=0x55d7aef090c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f121907e640 1 --2- 172.21.15.38:0/3748744803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb336b0 0x55d7aef090c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f121907e640 1 -- 172.21.15.38:0/3748744803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7aeeead20 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f121887d640 1 --2- 172.21.15.38:0/3748744803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb336b0 0x55d7aef090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.730+0000 7f121907e640 1 --2- 172.21.15.38:0/3748744803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb2b4d0 0x55d7aeefb790 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f12040b8f40 tx=0x7f12040ed8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7641ff18f49f49c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.731+0000 7f11faffd640 1 -- 172.21.15.38:0/3748744803 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204087070 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.731+0000 7f11faffd640 1 -- 172.21.15.38:0/3748744803 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12040bcc70 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.731+0000 7f11faffd640 1 -- 172.21.15.38:0/3748744803 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204004580 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.732+0000 7f1222184740 1 -- 172.21.15.38:0/3748744803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb2b4d0 msgr2=0x55d7aeefb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.732+0000 7f1222184740 1 --2- 172.21.15.38:0/3748744803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb2b4d0 0x55d7aeefb790 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f12040b8f40 tx=0x7f12040ed8a0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.732+0000 7f1222184740 1 -- 172.21.15.38:0/3748744803 shutdown_connections 2024-05-21T15:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.732+0000 7f1222184740 1 --2- 172.21.15.38:0/3748744803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb54390 0x55d7aeef2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.732+0000 7f1222184740 1 --2- 172.21.15.38:0/3748744803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb336b0 0x55d7aef090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.732+0000 7f1222184740 1 --2- 172.21.15.38:0/3748744803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb2b4d0 0x55d7aeefb790 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.732+0000 7f1222184740 1 -- 172.21.15.38:0/3748744803 >> 172.21.15.38:0/3748744803 conn(0x55d7aeb484e0 msgr2=0x55d7aeef2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.732+0000 7f1222184740 1 -- 172.21.15.38:0/3748744803 shutdown_connections 2024-05-21T15:36:20.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.732+0000 7f1222184740 1 -- 172.21.15.38:0/3748744803 wait complete. 2024-05-21T15:36:20.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.732+0000 7f1222184740 1 -- start start 2024-05-21T15:36:20.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.732+0000 7f1222184740 1 Processor -- start 2024-05-21T15:36:20.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.732+0000 7f1222184740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb2b4d0 0x55d7aef065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.732+0000 7f1222184740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb336b0 0x55d7aef06b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f1222184740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb54390 0x55d7aef07a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f1222184740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f12141a9690 con 0x55d7aeb54390 2024-05-21T15:36:20.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f1222184740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f12141a8fc0 con 0x55d7aeb2b4d0 2024-05-21T15:36:20.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f1222184740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f12141a6200 con 0x55d7aeb336b0 2024-05-21T15:36:20.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f1222184740 10 client.0 _ll_register_callbacks cb 0x7f12040a70b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:20.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f121987f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb54390 0x55d7aef07a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f121887d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb336b0 0x55d7aef06b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f121987f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb54390 0x55d7aef07a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46832/0 (socket says 172.21.15.38:46832) 2024-05-21T15:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f121987f640 1 -- 172.21.15.38:0/925469680 learned_addr learned my addr 172.21.15.38:0/925469680 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f121907e640 1 --2- 172.21.15.38:0/925469680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb2b4d0 0x55d7aef065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f121987f640 1 -- 172.21.15.38:0/925469680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb336b0 msgr2=0x55d7aef06b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f121987f640 1 --2- 172.21.15.38:0/925469680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb336b0 0x55d7aef06b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f121987f640 1 -- 172.21.15.38:0/925469680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb2b4d0 msgr2=0x55d7aef065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f121987f640 1 --2- 172.21.15.38:0/925469680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb2b4d0 0x55d7aef065e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f121987f640 1 -- 172.21.15.38:0/925469680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7aef08010 con 0x55d7aeb54390 2024-05-21T15:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f121987f640 1 --2- 172.21.15.38:0/925469680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb54390 0x55d7aef07a50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f120c011e60 tx=0x7f120c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fe8383ecdf4ce4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f11f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.733+0000 7f11f8ff9640 1 -- 172.21.15.38:0/925469680 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c020de0 con 0x55d7aeb54390 2024-05-21T15:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.734+0000 7f11f8ff9640 1 -- 172.21.15.38:0/925469680 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f120c004020 con 0x55d7aeb54390 2024-05-21T15:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.734+0000 7f1222184740 1 -- 172.21.15.38:0/925469680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12040bcc70 con 0x55d7aeb54390 2024-05-21T15:36:20.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.734+0000 7f11f8ff9640 1 -- 172.21.15.38:0/925469680 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f120c011e90 con 0x55d7aeb54390 2024-05-21T15:36:20.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.734+0000 7f1222184740 10 client.16002 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.734+0000 7f1222184740 1 -- 172.21.15.38:0/925469680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d7aef09300 con 0x55d7aeb54390 2024-05-21T15:36:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.734+0000 7f11f8ff9640 1 -- 172.21.15.38:0/925469680 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12040bcc70 con 0x55d7aeb54390 2024-05-21T15:36:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.734+0000 7f11f8ff9640 1 -- 172.21.15.38:0/925469680 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f120c034070 con 0x55d7aeb54390 2024-05-21T15:36:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.734+0000 7f1222184740 10 client.16002 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.734+0000 7f1222184740 10 client.16002 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:36:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.734+0000 7f1222184740 20 client.16002 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.734+0000 7f1222184740 1 --2- 172.21.15.38:0/925469680 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55d7aeae83a0 0x55d7aef7a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.734+0000 7f1222184740 4 client.16002 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:36:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.734+0000 7f1222184740 1 -- 172.21.15.38:0/925469680 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7aee606c0 con 0x55d7aeae83a0 2024-05-21T15:36:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.735+0000 7f121907e640 1 --2- 172.21.15.38:0/925469680 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55d7aeae83a0 0x55d7aef7a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.735+0000 7f11f8ff9640 10 client.16002 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:36:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.735+0000 7f121907e640 1 --2- 172.21.15.38:0/925469680 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55d7aeae83a0 0x55d7aef7a730 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ad567d7ddd3a7f6a server_cookie=48dd07d62ae8174 in_seq=0 out_seq=0 2024-05-21T15:36:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.735+0000 7f11f8ff9640 1 -- 172.21.15.38:0/925469680 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7aee606c0 con 0x55d7aeae83a0 2024-05-21T15:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.735+0000 7f11f8ff9640 10 client.16002 handle_command_reply: tid=0 2024-05-21T15:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.735+0000 7f1222184740 1 client.16002 shutdown 2024-05-21T15:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.735+0000 7f1222184740 1 -- 172.21.15.38:0/925469680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb54390 msgr2=0x55d7aef07a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.735+0000 7f1222184740 1 --2- 172.21.15.38:0/925469680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb54390 0x55d7aef07a50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f120c011e60 tx=0x7f120c0046c0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.735+0000 7f1222184740 1 -- 172.21.15.38:0/925469680 shutdown_connections 2024-05-21T15:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.735+0000 7f1222184740 1 --2- 172.21.15.38:0/925469680 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55d7aeae83a0 0x55d7aef7a730 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.735+0000 7f1222184740 1 --2- 172.21.15.38:0/925469680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d7aeb54390 0x55d7aef07a50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.735+0000 7f1222184740 1 --2- 172.21.15.38:0/925469680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7aeb336b0 0x55d7aef06b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.735+0000 7f1222184740 1 --2- 172.21.15.38:0/925469680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aeb2b4d0 0x55d7aef065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.735+0000 7f11f8ff9640 0 client.16002 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f1222184740 1 -- 172.21.15.38:0/925469680 >> 172.21.15.38:0/925469680 conn(0x55d7aeb484e0 msgr2=0x55d7aef02290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f1222184740 1 -- 172.21.15.38:0/925469680 shutdown_connections 2024-05-21T15:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f1222184740 1 -- 172.21.15.38:0/925469680 wait complete. 2024-05-21T15:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f1222184740 20 client.16002 trim_cache size 0 max 16384 2024-05-21T15:36:20.741 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:36:20.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f121b2e2640 1 -- 172.21.15.38:0/873725725 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11fc0a2b90 msgr2=0x7f11fc0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f121b2e2640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11fc0a2b90 0x7f11fc0a50a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55d7aeef90e0 tx=0x7f120c009260 comp rx=0 tx=0).stop 2024-05-21T15:36:20.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f121b2e2640 1 -- 172.21.15.38:0/873725725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12141006d0 msgr2=0x7f12141992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f121b2e2640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12141006d0 0x7f12141992e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1204034040 tx=0x7f1204030630 comp rx=0 tx=0).stop 2024-05-21T15:36:20.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f121b2e2640 1 -- 172.21.15.38:0/873725725 shutdown_connections 2024-05-21T15:36:20.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f121b2e2640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1214104020 0x7f12141a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f121b2e2640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12141011d0 0x7f1214199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f121b2e2640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11fc0a2b90 0x7f11fc0a50a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f121b2e2640 1 --2- 172.21.15.38:0/873725725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12141006d0 0x7f12141992e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f121b2e2640 1 -- 172.21.15.38:0/873725725 >> 172.21.15.38:0/873725725 conn(0x7f12140fc2f0 msgr2=0x7f1214104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f121b2e2640 1 -- 172.21.15.38:0/873725725 shutdown_connections 2024-05-21T15:36:20.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.736+0000 7f121b2e2640 1 -- 172.21.15.38:0/873725725 wait complete. 2024-05-21T15:36:20.747 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:36:20.768 INFO:tasks.workunit.client.0.smithi038.stdout:checking fcntl.h usability... yes 2024-05-21T15:36:20.786 INFO:tasks.workunit.client.0.smithi038.stdout:checking fcntl.h presence... yes 2024-05-21T15:36:20.786 INFO:tasks.workunit.client.0.smithi038.stdout:checking for fcntl.h... yes 2024-05-21T15:36:20.838 INFO:tasks.workunit.client.0.smithi038.stdout:checking limits.h usability... yes 2024-05-21T15:36:20.857 INFO:tasks.workunit.client.0.smithi038.stdout:checking limits.h presence... yes 2024-05-21T15:36:20.857 INFO:tasks.workunit.client.0.smithi038.stdout:checking for limits.h... yes 2024-05-21T15:36:20.872 INFO:tasks.workunit.client.0.smithi038.stdout:checking for stdint.h... (cached) yes 2024-05-21T15:36:20.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.888+0000 7f5889be3640 1 -- 172.21.15.38:0/2441876578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5884101980 msgr2=0x7f5884102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.888+0000 7f5889be3640 1 --2- 172.21.15.38:0/2441876578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5884101980 0x7f5884102680 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5874009900 tx=0x7f587402a410 comp rx=0 tx=0).stop 2024-05-21T15:36:20.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.888+0000 7f5889be3640 1 -- 172.21.15.38:0/2441876578 shutdown_connections 2024-05-21T15:36:20.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.888+0000 7f5889be3640 1 --2- 172.21.15.38:0/2441876578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58841078c0 0x7f5884109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.888+0000 7f5889be3640 1 --2- 172.21.15.38:0/2441876578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5884102bc0 0x7f5884102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.888+0000 7f5889be3640 1 --2- 172.21.15.38:0/2441876578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5884101980 0x7f5884102680 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.888+0000 7f5889be3640 1 -- 172.21.15.38:0/2441876578 >> 172.21.15.38:0/2441876578 conn(0x7f58840fcb70 msgr2=0x7f58840ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.888+0000 7f5889be3640 1 -- 172.21.15.38:0/2441876578 shutdown_connections 2024-05-21T15:36:20.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.888+0000 7f5889be3640 1 -- 172.21.15.38:0/2441876578 wait complete. 2024-05-21T15:36:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.889+0000 7f5889be3640 1 Processor -- start 2024-05-21T15:36:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.889+0000 7f5889be3640 1 -- start start 2024-05-21T15:36:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.889+0000 7f5889be3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5884101980 0x7f588419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.889+0000 7f5889be3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5884102bc0 0x7f588419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.889+0000 7f5889be3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58841078c0 0x7f58841a2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.889+0000 7f5889be3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f588410d5d0 con 0x7f5884102bc0 2024-05-21T15:36:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.889+0000 7f5889be3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f588410d450 con 0x7f58841078c0 2024-05-21T15:36:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.889+0000 7f5889be3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f588410d750 con 0x7f5884101980 2024-05-21T15:36:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.889+0000 7f5882ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5884102bc0 0x7f588419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.889+0000 7f5882ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5884102bc0 0x7f588419b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46858/0 (socket says 172.21.15.38:46858) 2024-05-21T15:36:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.889+0000 7f5882ffd640 1 -- 172.21.15.38:0/361929694 learned_addr learned my addr 172.21.15.38:0/361929694 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.889+0000 7f5883fff640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58841078c0 0x7f58841a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.889+0000 7f58837fe640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5884101980 0x7f588419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.889+0000 7f5883fff640 1 -- 172.21.15.38:0/361929694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5884101980 msgr2=0x7f588419b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.890+0000 7f5883fff640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5884101980 0x7f588419b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.890+0000 7f5883fff640 1 -- 172.21.15.38:0/361929694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5884102bc0 msgr2=0x7f588419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.890+0000 7f5883fff640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5884102bc0 0x7f588419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.890+0000 7f5883fff640 1 -- 172.21.15.38:0/361929694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58841a2fd0 con 0x7f58841078c0 2024-05-21T15:36:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.890+0000 7f5882ffd640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5884102bc0 0x7f588419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.890+0000 7f5883fff640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58841078c0 0x7f58841a2960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f587c009fd0 tx=0x7f587c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.890+0000 7f5880ff9640 1 -- 172.21.15.38:0/361929694 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c00ce20 con 0x7f58841078c0 2024-05-21T15:36:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.890+0000 7f5889be3640 1 -- 172.21.15.38:0/361929694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58841a32c0 con 0x7f58841078c0 2024-05-21T15:36:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.890+0000 7f5880ff9640 1 -- 172.21.15.38:0/361929694 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587c005020 con 0x7f58841078c0 2024-05-21T15:36:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.890+0000 7f5880ff9640 1 -- 172.21.15.38:0/361929694 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c010720 con 0x7f58841078c0 2024-05-21T15:36:20.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.890+0000 7f5889be3640 1 -- 172.21.15.38:0/361929694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58841a37c0 con 0x7f58841078c0 2024-05-21T15:36:20.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.893+0000 7f5890b64740 1 Processor -- start 2024-05-21T15:36:20.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.893+0000 7f5880ff9640 1 -- 172.21.15.38:0/361929694 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f587c010900 con 0x7f58841078c0 2024-05-21T15:36:20.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.893+0000 7f5890b64740 1 -- start start 2024-05-21T15:36:20.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f5890b64740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f383b390 0x5563f3bdab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f5890b64740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f38124d0 0x5563f3bdb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f5890b64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f381a6b0 0x5563f3be1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f5890b64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5563f3b3fc40 con 0x5563f381a6b0 2024-05-21T15:36:20.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f5890b64740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5563f3ae7580 con 0x5563f383b390 2024-05-21T15:36:20.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f5890b64740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5563f3b3bc00 con 0x5563f38124d0 2024-05-21T15:36:20.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f5882ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f383b390 0x5563f3bdab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f5882ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f383b390 0x5563f3bdab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53396/0 (socket says 172.21.15.38:53396) 2024-05-21T15:36:20.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f5882ffd640 1 -- 172.21.15.38:0/3470239076 learned_addr learned my addr 172.21.15.38:0/3470239076 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f58837fe640 1 --2- 172.21.15.38:0/3470239076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f38124d0 0x5563f3bdb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f5882ffd640 1 --2- 172.21.15.38:0/3470239076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f381a6b0 0x5563f3be1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f5880ff9640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f586c0a2b90 0x7f586c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f58837fe640 1 -- 172.21.15.38:0/3470239076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f383b390 msgr2=0x5563f3bdab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f58837fe640 1 --2- 172.21.15.38:0/3470239076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f383b390 0x5563f3bdab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f58837fe640 1 -- 172.21.15.38:0/3470239076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f381a6b0 msgr2=0x5563f3be1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f58837fe640 1 --2- 172.21.15.38:0/3470239076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f381a6b0 0x5563f3be1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f58837fe640 1 -- 172.21.15.38:0/3470239076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f38533f0 con 0x5563f38124d0 2024-05-21T15:36:20.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.894+0000 7f5883fff640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f586c0a2b90 0x7f586c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.895+0000 7f5880ff9640 1 -- 172.21.15.38:0/361929694 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f587c0cf780 con 0x7f58841078c0 2024-05-21T15:36:20.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.895+0000 7f58837fe640 1 --2- 172.21.15.38:0/3470239076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f38124d0 0x5563f3bdb080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f587402a930 tx=0x7f5874000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b44e6996beb590ca server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.895+0000 7f5869ffb640 1 -- 172.21.15.38:0/3470239076 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874040070 con 0x5563f38124d0 2024-05-21T15:36:20.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.895+0000 7f5869ffb640 1 -- 172.21.15.38:0/3470239076 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5874004030 con 0x5563f38124d0 2024-05-21T15:36:20.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.895+0000 7f5869ffb640 1 -- 172.21.15.38:0/3470239076 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874035040 con 0x5563f38124d0 2024-05-21T15:36:20.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.896+0000 7f5890b64740 1 -- 172.21.15.38:0/3470239076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f38124d0 msgr2=0x5563f3bdb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.896+0000 7f5883fff640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f586c0a2b90 0x7f586c0a50a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f587c09bc90 tx=0x7f587c0bc040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.896+0000 7f5890b64740 1 --2- 172.21.15.38:0/3470239076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f38124d0 0x5563f3bdb080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f587402a930 tx=0x7f5874000bc0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.896+0000 7f5890b64740 1 -- 172.21.15.38:0/3470239076 shutdown_connections 2024-05-21T15:36:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.896+0000 7f5890b64740 1 --2- 172.21.15.38:0/3470239076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f381a6b0 0x5563f3be1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.896+0000 7f5890b64740 1 --2- 172.21.15.38:0/3470239076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f38124d0 0x5563f3bdb080 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.896+0000 7f5890b64740 1 --2- 172.21.15.38:0/3470239076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f383b390 0x5563f3bdab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.896+0000 7f5890b64740 1 -- 172.21.15.38:0/3470239076 >> 172.21.15.38:0/3470239076 conn(0x5563f382f4e0 msgr2=0x5563f3bd6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.896+0000 7f5890b64740 1 -- 172.21.15.38:0/3470239076 shutdown_connections 2024-05-21T15:36:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.896+0000 7f5890b64740 1 -- 172.21.15.38:0/3470239076 wait complete. 2024-05-21T15:36:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.896+0000 7f5890b64740 1 -- start start 2024-05-21T15:36:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.896+0000 7f5890b64740 1 Processor -- start 2024-05-21T15:36:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5890b64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f38124d0 0x5563f3be4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5890b64740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f381a6b0 0x5563f3be4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5890b64740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f383b390 0x5563f3bebdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5890b64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5563f3be2970 con 0x5563f38124d0 2024-05-21T15:36:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5890b64740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5563f3afac40 con 0x5563f381a6b0 2024-05-21T15:36:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5890b64740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5563f3be2af0 con 0x5563f383b390 2024-05-21T15:36:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5890b64740 10 client.0 _ll_register_callbacks cb 0x5563f3852be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5882ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f38124d0 0x5563f3be4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5882ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f38124d0 0x5563f3be4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46886/0 (socket says 172.21.15.38:46886) 2024-05-21T15:36:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5882ffd640 1 -- 172.21.15.38:0/1820976876 learned_addr learned my addr 172.21.15.38:0/1820976876 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f58837fe640 1 --2- 172.21.15.38:0/1820976876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f381a6b0 0x5563f3be4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5882ffd640 1 --2- 172.21.15.38:0/1820976876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f383b390 0x5563f3bebdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5882ffd640 1 -- 172.21.15.38:0/1820976876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f383b390 msgr2=0x5563f3bebdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5882ffd640 1 --2- 172.21.15.38:0/1820976876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f383b390 0x5563f3bebdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5882ffd640 1 -- 172.21.15.38:0/1820976876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f381a6b0 msgr2=0x5563f3be4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5882ffd640 1 --2- 172.21.15.38:0/1820976876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f381a6b0 0x5563f3be4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5882ffd640 1 -- 172.21.15.38:0/1820976876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f391d7d0 con 0x5563f38124d0 2024-05-21T15:36:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5882ffd640 1 --2- 172.21.15.38:0/1820976876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f38124d0 0x5563f3be4930 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5878012f90 tx=0x7f5878004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f43b2f1ee374e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5862ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.897+0000 7f5862ffd640 1 -- 172.21.15.38:0/1820976876 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878005020 con 0x5563f38124d0 2024-05-21T15:36:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.898+0000 7f5890b64740 1 -- 172.21.15.38:0/1820976876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5874004030 con 0x5563f38124d0 2024-05-21T15:36:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.898+0000 7f5862ffd640 1 -- 172.21.15.38:0/1820976876 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f58780051c0 con 0x5563f38124d0 2024-05-21T15:36:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.898+0000 7f5862ffd640 1 -- 172.21.15.38:0/1820976876 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58780053c0 con 0x5563f38124d0 2024-05-21T15:36:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.898+0000 7f5890b64740 10 client.16014 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.898+0000 7f5890b64740 1 -- 172.21.15.38:0/1820976876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5563f3bef100 con 0x5563f38124d0 2024-05-21T15:36:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.898+0000 7f5862ffd640 1 -- 172.21.15.38:0/1820976876 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5874004030 con 0x5563f38124d0 2024-05-21T15:36:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.898+0000 7f5862ffd640 1 -- 172.21.15.38:0/1820976876 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5878029070 con 0x5563f38124d0 2024-05-21T15:36:20.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.898+0000 7f5890b64740 10 client.16014 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:20.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.898+0000 7f5890b64740 10 client.16014 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:36:20.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.898+0000 7f5890b64740 20 client.16014 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:20.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.898+0000 7f5890b64740 1 --2- 172.21.15.38:0/1820976876 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x5563f37f9f60 0x5563f3bf2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.898+0000 7f5890b64740 4 client.16014 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:20.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.898+0000 7f5890b64740 1 -- 172.21.15.38:0/1820976876 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563f3b4d370 con 0x5563f37f9f60 2024-05-21T15:36:20.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.899+0000 7f58837fe640 1 --2- 172.21.15.38:0/1820976876 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x5563f37f9f60 0x5563f3bf2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.899+0000 7f58837fe640 1 --2- 172.21.15.38:0/1820976876 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x5563f37f9f60 0x5563f3bf2800 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b84d4e08d549f3d7 server_cookie=ea213f773119ffb1 in_seq=0 out_seq=0 2024-05-21T15:36:20.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.899+0000 7f5862ffd640 10 client.16014 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:36:20.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5862ffd640 1 -- 172.21.15.38:0/1820976876 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5563f3b4d370 con 0x5563f37f9f60 2024-05-21T15:36:20.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5862ffd640 10 client.16014 handle_command_reply: tid=0 2024-05-21T15:36:20.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5890b64740 1 client.16014 shutdown 2024-05-21T15:36:20.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5890b64740 1 -- 172.21.15.38:0/1820976876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f38124d0 msgr2=0x5563f3be4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5890b64740 1 --2- 172.21.15.38:0/1820976876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f38124d0 0x5563f3be4930 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5878012f90 tx=0x7f5878004730 comp rx=0 tx=0).stop 2024-05-21T15:36:20.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5890b64740 1 -- 172.21.15.38:0/1820976876 shutdown_connections 2024-05-21T15:36:20.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5890b64740 1 --2- 172.21.15.38:0/1820976876 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x5563f37f9f60 0x5563f3bf2800 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5890b64740 1 --2- 172.21.15.38:0/1820976876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f383b390 0x5563f3bebdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5890b64740 1 --2- 172.21.15.38:0/1820976876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f381a6b0 0x5563f3be4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5862ffd640 0 client.16014 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:36:20.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5890b64740 1 --2- 172.21.15.38:0/1820976876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f38124d0 0x5563f3be4930 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5890b64740 1 -- 172.21.15.38:0/1820976876 >> 172.21.15.38:0/1820976876 conn(0x5563f382f4e0 msgr2=0x5563f3bd5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5890b64740 1 -- 172.21.15.38:0/1820976876 shutdown_connections 2024-05-21T15:36:20.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5890b64740 1 -- 172.21.15.38:0/1820976876 wait complete. 2024-05-21T15:36:20.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.900+0000 7f5890b64740 20 client.16014 trim_cache size 0 max 16384 2024-05-21T15:36:20.913 INFO:tasks.workunit.client.0.smithi038.stdout:checking sys/time.h usability... yes 2024-05-21T15:36:20.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.914+0000 7f5890b64740 1 Processor -- start 2024-05-21T15:36:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.914+0000 7f5890b64740 1 -- start start 2024-05-21T15:36:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.914+0000 7f5890b64740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f38124d0 0x5563f3be4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.914+0000 7f5890b64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f381a6b0 0x5563f3bebd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.914+0000 7f5890b64740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f383b390 0x5563f3be4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.914+0000 7f5890b64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f58841ac560 con 0x5563f381a6b0 2024-05-21T15:36:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.914+0000 7f5890b64740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f58841ae0a0 con 0x5563f383b390 2024-05-21T15:36:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.914+0000 7f5890b64740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f58841acc30 con 0x5563f38124d0 2024-05-21T15:36:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.914+0000 7f58837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f381a6b0 0x5563f3bebd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.915+0000 7f58837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f381a6b0 0x5563f3bebd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46892/0 (socket says 172.21.15.38:46892) 2024-05-21T15:36:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.915+0000 7f58837fe640 1 -- 172.21.15.38:0/1692274077 learned_addr learned my addr 172.21.15.38:0/1692274077 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.915+0000 7f5882ffd640 1 --2- 172.21.15.38:0/1692274077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f383b390 0x5563f3be4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.915+0000 7f5882ffd640 1 --2- 172.21.15.38:0/1692274077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f38124d0 0x5563f3be4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.915+0000 7f58837fe640 1 -- 172.21.15.38:0/1692274077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f38124d0 msgr2=0x5563f3be4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.915+0000 7f58837fe640 1 --2- 172.21.15.38:0/1692274077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f38124d0 0x5563f3be4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.915+0000 7f58837fe640 1 -- 172.21.15.38:0/1692274077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f383b390 msgr2=0x5563f3be4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.915+0000 7f58837fe640 1 --2- 172.21.15.38:0/1692274077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f383b390 0x5563f3be4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.915+0000 7f58837fe640 1 -- 172.21.15.38:0/1692274077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f3bd07e0 con 0x5563f381a6b0 2024-05-21T15:36:20.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.915+0000 7f58837fe640 1 --2- 172.21.15.38:0/1692274077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f381a6b0 0x5563f3bebd60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5563f3bdbee0 tx=0x7f5874009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b984d3d918d01336 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.915+0000 7f58697fa640 1 -- 172.21.15.38:0/1692274077 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874056070 con 0x5563f381a6b0 2024-05-21T15:36:20.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.915+0000 7f58697fa640 1 -- 172.21.15.38:0/1692274077 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5874004040 con 0x5563f381a6b0 2024-05-21T15:36:20.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.915+0000 7f58697fa640 1 -- 172.21.15.38:0/1692274077 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874040070 con 0x5563f381a6b0 2024-05-21T15:36:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.916+0000 7f5890b64740 1 -- 172.21.15.38:0/1692274077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f381a6b0 msgr2=0x5563f3bebd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.916+0000 7f5890b64740 1 --2- 172.21.15.38:0/1692274077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f381a6b0 0x5563f3bebd60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5563f3bdbee0 tx=0x7f5874009290 comp rx=0 tx=0).stop 2024-05-21T15:36:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.916+0000 7f5890b64740 1 -- 172.21.15.38:0/1692274077 shutdown_connections 2024-05-21T15:36:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.916+0000 7f5890b64740 1 --2- 172.21.15.38:0/1692274077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f383b390 0x5563f3be4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.916+0000 7f5890b64740 1 --2- 172.21.15.38:0/1692274077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f381a6b0 0x5563f3bebd60 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.916+0000 7f5890b64740 1 --2- 172.21.15.38:0/1692274077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f38124d0 0x5563f3be4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.916+0000 7f5890b64740 1 -- 172.21.15.38:0/1692274077 >> 172.21.15.38:0/1692274077 conn(0x5563f382f4e0 msgr2=0x5563f3be2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.916+0000 7f5890b64740 1 -- 172.21.15.38:0/1692274077 shutdown_connections 2024-05-21T15:36:20.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.916+0000 7f5890b64740 1 -- 172.21.15.38:0/1692274077 wait complete. 2024-05-21T15:36:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.917+0000 7f5890b64740 1 -- start start 2024-05-21T15:36:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.917+0000 7f5890b64740 1 Processor -- start 2024-05-21T15:36:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.917+0000 7f5890b64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f38124d0 0x5563f3be9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.917+0000 7f5890b64740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f381a6b0 0x5563f3be9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.917+0000 7f5890b64740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f383b390 0x5563f3beaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.917+0000 7f5890b64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f58841ad9d0 con 0x5563f38124d0 2024-05-21T15:36:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.917+0000 7f5890b64740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f58841ad300 con 0x5563f381a6b0 2024-05-21T15:36:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.917+0000 7f5890b64740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f58841aa540 con 0x5563f383b390 2024-05-21T15:36:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.917+0000 7f5890b64740 10 client.0 _ll_register_callbacks cb 0x7f5878012590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.917+0000 7f5882ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f38124d0 0x5563f3be9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.917+0000 7f5882ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f38124d0 0x5563f3be9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46904/0 (socket says 172.21.15.38:46904) 2024-05-21T15:36:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.917+0000 7f5882ffd640 1 -- 172.21.15.38:0/3613520978 learned_addr learned my addr 172.21.15.38:0/3613520978 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:20.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.917+0000 7f58837fe640 1 --2- 172.21.15.38:0/3613520978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f381a6b0 0x5563f3be9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5882ffd640 1 --2- 172.21.15.38:0/3613520978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f383b390 0x5563f3beaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5882ffd640 1 -- 172.21.15.38:0/3613520978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f383b390 msgr2=0x5563f3beaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5882ffd640 1 --2- 172.21.15.38:0/3613520978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f383b390 0x5563f3beaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5882ffd640 1 -- 172.21.15.38:0/3613520978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f381a6b0 msgr2=0x5563f3be9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5882ffd640 1 --2- 172.21.15.38:0/3613520978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f381a6b0 0x5563f3be9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5882ffd640 1 -- 172.21.15.38:0/3613520978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f3beb030 con 0x5563f38124d0 2024-05-21T15:36:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5882ffd640 1 --2- 172.21.15.38:0/3613520978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f38124d0 0x5563f3be9600 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f587800d640 tx=0x7f5878012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7375af5498beaa9a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5862ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5862ffd640 1 -- 172.21.15.38:0/3613520978 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587802f040 con 0x5563f38124d0 2024-05-21T15:36:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5862ffd640 1 -- 172.21.15.38:0/3613520978 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f587800dac0 con 0x5563f38124d0 2024-05-21T15:36:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5890b64740 1 -- 172.21.15.38:0/3613520978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5874004040 con 0x5563f38124d0 2024-05-21T15:36:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5862ffd640 1 -- 172.21.15.38:0/3613520978 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f587800dcc0 con 0x5563f38124d0 2024-05-21T15:36:20.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5890b64740 10 client.16020 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5890b64740 1 -- 172.21.15.38:0/3613520978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5563f3c5ac40 con 0x5563f38124d0 2024-05-21T15:36:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.918+0000 7f5862ffd640 1 -- 172.21.15.38:0/3613520978 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5874004040 con 0x5563f38124d0 2024-05-21T15:36:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.919+0000 7f5862ffd640 1 -- 172.21.15.38:0/3613520978 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5878029040 con 0x5563f38124d0 2024-05-21T15:36:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.919+0000 7f5890b64740 10 client.16020 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.919+0000 7f5890b64740 10 client.16020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:36:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.919+0000 7f5890b64740 20 client.16020 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.919+0000 7f5890b64740 1 --2- 172.21.15.38:0/3613520978 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x5563f37cf3a0 0x5563f3c5d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.919+0000 7f5890b64740 4 client.16020 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:36:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.919+0000 7f5890b64740 1 -- 172.21.15.38:0/3613520978 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563f3b476c0 con 0x5563f37cf3a0 2024-05-21T15:36:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.919+0000 7f58837fe640 1 --2- 172.21.15.38:0/3613520978 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x5563f37cf3a0 0x5563f3c5d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.919+0000 7f5862ffd640 10 client.16020 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:36:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.919+0000 7f58837fe640 1 --2- 172.21.15.38:0/3613520978 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x5563f37cf3a0 0x5563f3c5d610 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57fcb44fbf526ffa server_cookie=cfbeddd338b27d93 in_seq=0 out_seq=0 2024-05-21T15:36:20.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5862ffd640 1 -- 172.21.15.38:0/3613520978 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563f3b476c0 con 0x5563f37cf3a0 2024-05-21T15:36:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5862ffd640 10 client.16020 handle_command_reply: tid=0 2024-05-21T15:36:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5890b64740 1 client.16020 shutdown 2024-05-21T15:36:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5890b64740 1 -- 172.21.15.38:0/3613520978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f38124d0 msgr2=0x5563f3be9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5890b64740 1 --2- 172.21.15.38:0/3613520978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f38124d0 0x5563f3be9600 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f587800d640 tx=0x7f5878012520 comp rx=0 tx=0).stop 2024-05-21T15:36:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5890b64740 1 -- 172.21.15.38:0/3613520978 shutdown_connections 2024-05-21T15:36:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5890b64740 1 --2- 172.21.15.38:0/3613520978 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x5563f37cf3a0 0x5563f3c5d610 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5890b64740 1 --2- 172.21.15.38:0/3613520978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563f383b390 0x5563f3beaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5890b64740 1 --2- 172.21.15.38:0/3613520978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5563f381a6b0 0x5563f3be9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5890b64740 1 --2- 172.21.15.38:0/3613520978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563f38124d0 0x5563f3be9600 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5862ffd640 0 client.16020 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:36:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5890b64740 1 -- 172.21.15.38:0/3613520978 >> 172.21.15.38:0/3613520978 conn(0x5563f382f4e0 msgr2=0x5563f3be4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5890b64740 1 -- 172.21.15.38:0/3613520978 shutdown_connections 2024-05-21T15:36:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5890b64740 1 -- 172.21.15.38:0/3613520978 wait complete. 2024-05-21T15:36:20.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.920+0000 7f5890b64740 20 client.16020 trim_cache size 0 max 16384 2024-05-21T15:36:20.926 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:36:20.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.921+0000 7f5889be3640 1 -- 172.21.15.38:0/361929694 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f586c0a2b90 msgr2=0x7f586c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.921+0000 7f5889be3640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f586c0a2b90 0x7f586c0a50a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f587c09bc90 tx=0x7f587c0bc040 comp rx=0 tx=0).stop 2024-05-21T15:36:20.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.921+0000 7f5889be3640 1 -- 172.21.15.38:0/361929694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58841078c0 msgr2=0x7f58841a2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:20.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.921+0000 7f5889be3640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58841078c0 0x7f58841a2960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f587c009fd0 tx=0x7f587c00ef10 comp rx=0 tx=0).stop 2024-05-21T15:36:20.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.921+0000 7f5889be3640 1 -- 172.21.15.38:0/361929694 shutdown_connections 2024-05-21T15:36:20.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.921+0000 7f5889be3640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58841078c0 0x7f58841a2960 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.921+0000 7f5889be3640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f586c0a2b90 0x7f586c0a50a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.921+0000 7f5889be3640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5884102bc0 0x7f588419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.921+0000 7f5889be3640 1 --2- 172.21.15.38:0/361929694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5884101980 0x7f588419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:20.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.921+0000 7f5889be3640 1 -- 172.21.15.38:0/361929694 >> 172.21.15.38:0/361929694 conn(0x7f58840fcb70 msgr2=0x7f58840fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:20.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.921+0000 7f5889be3640 1 -- 172.21.15.38:0/361929694 shutdown_connections 2024-05-21T15:36:20.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:20.921+0000 7f5889be3640 1 -- 172.21.15.38:0/361929694 wait complete. 2024-05-21T15:36:20.930 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:20 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2128040485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:36:20.931 INFO:tasks.workunit.client.0.smithi038.stdout:checking sys/time.h presence... yes 2024-05-21T15:36:20.931 INFO:tasks.workunit.client.0.smithi038.stdout:checking for sys/time.h... yes 2024-05-21T15:36:20.932 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:36:20.946 INFO:tasks.workunit.client.0.smithi038.stdout:checking for unistd.h... (cached) yes 2024-05-21T15:36:20.990 INFO:tasks.workunit.client.0.smithi038.stdout:checking sys/vfs.h usability... yes 2024-05-21T15:36:21.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:20 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2128040485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:36:21.013 INFO:tasks.workunit.client.0.smithi038.stdout:checking sys/vfs.h presence... yes 2024-05-21T15:36:21.013 INFO:tasks.workunit.client.0.smithi038.stdout:checking for sys/vfs.h... yes 2024-05-21T15:36:21.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.065+0000 7fb19772a640 1 -- 172.21.15.38:0/1509891373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb190101210 msgr2=0x7fb19010ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.065+0000 7fb19772a640 1 --2- 172.21.15.38:0/1509891373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb190101210 0x7fb19010ac70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb188009ca0 tx=0x7fb18802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.065+0000 7fb19772a640 1 -- 172.21.15.38:0/1509891373 shutdown_connections 2024-05-21T15:36:21.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.065+0000 7fb19772a640 1 --2- 172.21.15.38:0/1509891373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb190101210 0x7fb19010ac70 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.065+0000 7fb19772a640 1 --2- 172.21.15.38:0/1509891373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb190100710 0x7fb190100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.065+0000 7fb19772a640 1 --2- 172.21.15.38:0/1509891373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb190104060 0x7fb190104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.065+0000 7fb19772a640 1 -- 172.21.15.38:0/1509891373 >> 172.21.15.38:0/1509891373 conn(0x7fb1900fc330 msgr2=0x7fb1900fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:21.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.065+0000 7fb19772a640 1 -- 172.21.15.38:0/1509891373 shutdown_connections 2024-05-21T15:36:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.066+0000 7fb19772a640 1 -- 172.21.15.38:0/1509891373 wait complete. 2024-05-21T15:36:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.066+0000 7fb19772a640 1 Processor -- start 2024-05-21T15:36:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.066+0000 7fb19772a640 1 -- start start 2024-05-21T15:36:21.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.066+0000 7fb19772a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb190100710 0x7fb190103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.066+0000 7fb19772a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb190101210 0x7fb1901018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.066+0000 7fb19772a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb190104060 0x7fb190101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.066+0000 7fb19772a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010e740 con 0x7fb190104060 2024-05-21T15:36:21.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.066+0000 7fb19772a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010e5c0 con 0x7fb190101210 2024-05-21T15:36:21.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.066+0000 7fb19772a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010e8c0 con 0x7fb190100710 2024-05-21T15:36:21.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.066+0000 7fb195cc7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb190104060 0x7fb190101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb195cc7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb190104060 0x7fb190101df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46914/0 (socket says 172.21.15.38:46914) 2024-05-21T15:36:21.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb195cc7640 1 -- 172.21.15.38:0/3858365803 learned_addr learned my addr 172.21.15.38:0/3858365803 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:21.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb194cc5640 1 --2- 172.21.15.38:0/3858365803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb190101210 0x7fb1901018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb1954c6640 1 --2- 172.21.15.38:0/3858365803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb190100710 0x7fb190103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb195cc7640 1 -- 172.21.15.38:0/3858365803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb190100710 msgr2=0x7fb190103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb195cc7640 1 --2- 172.21.15.38:0/3858365803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb190100710 0x7fb190103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb195cc7640 1 -- 172.21.15.38:0/3858365803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb190101210 msgr2=0x7fb1901018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb195cc7640 1 --2- 172.21.15.38:0/3858365803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb190101210 0x7fb1901018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb195cc7640 1 -- 172.21.15.38:0/3858365803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb190102520 con 0x7fb190104060 2024-05-21T15:36:21.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb195cc7640 1 --2- 172.21.15.38:0/3858365803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb190104060 0x7fb190101df0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb188005cc0 tx=0x7fb188004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:21.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb1867fc640 1 -- 172.21.15.38:0/3858365803 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188003b70 con 0x7fb190104060 2024-05-21T15:36:21.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb1867fc640 1 -- 172.21.15.38:0/3858365803 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb188003d10 con 0x7fb190104060 2024-05-21T15:36:21.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb1867fc640 1 -- 172.21.15.38:0/3858365803 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18802fbd0 con 0x7fb190104060 2024-05-21T15:36:21.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb19772a640 1 -- 172.21.15.38:0/3858365803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1901a32e0 con 0x7fb190104060 2024-05-21T15:36:21.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.067+0000 7fb19772a640 1 -- 172.21.15.38:0/3858365803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1901a3740 con 0x7fb190104060 2024-05-21T15:36:21.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.070+0000 7fb1867fc640 1 -- 172.21.15.38:0/3858365803 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb18802fd70 con 0x7fb190104060 2024-05-21T15:36:21.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.070+0000 7fb19e5cc740 1 Processor -- start 2024-05-21T15:36:21.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.070+0000 7fb19e5cc740 1 -- start start 2024-05-21T15:36:21.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.070+0000 7fb19e5cc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983ef390 0x560c9878eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.070+0000 7fb19e5cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983c64d0 0x560c9878f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.070+0000 7fb19e5cc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c98795f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.070+0000 7fb19e5cc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560c986f3c40 con 0x560c983ce6b0 2024-05-21T15:36:21.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.070+0000 7fb19e5cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560c9869b580 con 0x560c983c64d0 2024-05-21T15:36:21.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.070+0000 7fb19e5cc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560c986efc00 con 0x560c983ef390 2024-05-21T15:36:21.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.070+0000 7fb1954c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983c64d0 0x560c9878f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.070+0000 7fb194cc5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c98795f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.070+0000 7fb1954c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983c64d0 0x560c9878f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53452/0 (socket says 172.21.15.38:53452) 2024-05-21T15:36:21.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.070+0000 7fb1954c6640 1 -- 172.21.15.38:0/761045216 learned_addr learned my addr 172.21.15.38:0/761045216 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:21.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb194cc5640 1 --2- 172.21.15.38:0/761045216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983ef390 0x560c9878eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb1954c6640 1 -- 172.21.15.38:0/761045216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983ef390 msgr2=0x560c9878eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb1867fc640 1 --2- 172.21.15.38:0/3858365803 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb17c0a2b90 0x7fb17c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb1954c6640 1 --2- 172.21.15.38:0/761045216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983ef390 0x560c9878eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb1954c6640 1 -- 172.21.15.38:0/761045216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 msgr2=0x560c98795f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb1954c6640 1 --2- 172.21.15.38:0/761045216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c98795f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb1954c6640 1 -- 172.21.15.38:0/761045216 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c984073f0 con 0x560c983c64d0 2024-05-21T15:36:21.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb194cc5640 1 --2- 172.21.15.38:0/761045216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c98795f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:21.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb195cc7640 1 --2- 172.21.15.38:0/3858365803 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb17c0a2b90 0x7fb17c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb1867fc640 1 -- 172.21.15.38:0/3858365803 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fb1880f03c0 con 0x7fb190104060 2024-05-21T15:36:21.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb1954c6640 1 --2- 172.21.15.38:0/761045216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983c64d0 0x560c9878f080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00bc70 tx=0x7fb18c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c30f52b239c25606 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:21.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb17b7fe640 1 -- 172.21.15.38:0/761045216 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c01c680 con 0x560c983c64d0 2024-05-21T15:36:21.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb17b7fe640 1 -- 172.21.15.38:0/761045216 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb18c00be20 con 0x560c983c64d0 2024-05-21T15:36:21.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb195cc7640 1 --2- 172.21.15.38:0/3858365803 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb17c0a2b90 0x7fb17c0a50a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb1880ee1e0 tx=0x7fb1880e7760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:21.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.071+0000 7fb17b7fe640 1 -- 172.21.15.38:0/761045216 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c011bd0 con 0x560c983c64d0 2024-05-21T15:36:21.073 INFO:tasks.workunit.client.0.smithi038.stdout:checking sys/limits.h usability... no 2024-05-21T15:36:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.072+0000 7fb19e5cc740 1 -- 172.21.15.38:0/761045216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983c64d0 msgr2=0x560c9878f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.072+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/761045216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983c64d0 0x560c9878f080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00bc70 tx=0x7fb18c0107a0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.072+0000 7fb19e5cc740 1 -- 172.21.15.38:0/761045216 shutdown_connections 2024-05-21T15:36:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.072+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/761045216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c98795f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.072+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/761045216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983c64d0 0x560c9878f080 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.072+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/761045216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983ef390 0x560c9878eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.072+0000 7fb19e5cc740 1 -- 172.21.15.38:0/761045216 >> 172.21.15.38:0/761045216 conn(0x560c983e34e0 msgr2=0x560c9878a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.072+0000 7fb19e5cc740 1 -- 172.21.15.38:0/761045216 shutdown_connections 2024-05-21T15:36:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.072+0000 7fb19e5cc740 1 -- 172.21.15.38:0/761045216 wait complete. 2024-05-21T15:36:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb19e5cc740 1 -- start start 2024-05-21T15:36:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb19e5cc740 1 Processor -- start 2024-05-21T15:36:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb19e5cc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 0x560c98798980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb19e5cc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c98798ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb19e5cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c9879fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb19e5cc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560c987969d0 con 0x560c983ce6b0 2024-05-21T15:36:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb19e5cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560c986aec40 con 0x560c983ef390 2024-05-21T15:36:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb19e5cc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560c98796b50 con 0x560c983c64d0 2024-05-21T15:36:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb19e5cc740 10 client.0 _ll_register_callbacks cb 0x560c98406be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb1954c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c98798ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb194cc5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c9879fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb194cc5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c9879fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53466/0 (socket says 172.21.15.38:53466) 2024-05-21T15:36:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb1954c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c98798ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46936/0 (socket says 172.21.15.38:46936) 2024-05-21T15:36:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb194cc5640 1 -- 172.21.15.38:0/2425492613 learned_addr learned my addr 172.21.15.38:0/2425492613 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.073+0000 7fb194cc5640 1 --2- 172.21.15.38:0/2425492613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 0x560c98798980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.074+0000 7fb194cc5640 1 -- 172.21.15.38:0/2425492613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 msgr2=0x560c98798980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.074+0000 7fb194cc5640 1 --2- 172.21.15.38:0/2425492613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 0x560c98798980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.074+0000 7fb194cc5640 1 -- 172.21.15.38:0/2425492613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 msgr2=0x560c98798ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.074+0000 7fb194cc5640 1 --2- 172.21.15.38:0/2425492613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c98798ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.074+0000 7fb194cc5640 1 -- 172.21.15.38:0/2425492613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c984d17d0 con 0x560c983ef390 2024-05-21T15:36:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.074+0000 7fb194cc5640 1 --2- 172.21.15.38:0/2425492613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c9879fe20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb180010fc0 tx=0x7fb180004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=612e195b2e4e913b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.074+0000 7fb178ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.074+0000 7fb178ff9640 1 -- 172.21.15.38:0/2425492613 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18000e3b0 con 0x560c983ef390 2024-05-21T15:36:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.074+0000 7fb178ff9640 1 -- 172.21.15.38:0/2425492613 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fb18000e550 con 0x560c983ef390 2024-05-21T15:36:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.074+0000 7fb19e5cc740 1 -- 172.21.15.38:0/2425492613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb18c00be20 con 0x560c983ef390 2024-05-21T15:36:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.074+0000 7fb178ff9640 1 -- 172.21.15.38:0/2425492613 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb180029050 con 0x560c983ef390 2024-05-21T15:36:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.074+0000 7fb19e5cc740 10 client.25697 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.074+0000 7fb19e5cc740 1 -- 172.21.15.38:0/2425492613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560c987a3150 con 0x560c983ef390 2024-05-21T15:36:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.074+0000 7fb178ff9640 1 -- 172.21.15.38:0/2425492613 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb18c00be20 con 0x560c983ef390 2024-05-21T15:36:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.075+0000 7fb178ff9640 1 -- 172.21.15.38:0/2425492613 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb1800247e0 con 0x560c983ef390 2024-05-21T15:36:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.075+0000 7fb19e5cc740 10 client.25697 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.075+0000 7fb19e5cc740 10 client.25697 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:36:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.075+0000 7fb19e5cc740 20 client.25697 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.075+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/2425492613 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x560c983adf60 0x560c987a6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.075+0000 7fb19e5cc740 4 client.25697 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.075+0000 7fb19e5cc740 1 -- 172.21.15.38:0/2425492613 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c98701370 con 0x560c983adf60 2024-05-21T15:36:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.075+0000 7fb1954c6640 1 --2- 172.21.15.38:0/2425492613 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x560c983adf60 0x560c987a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.075+0000 7fb178ff9640 10 client.25697 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:36:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.075+0000 7fb1954c6640 1 --2- 172.21.15.38:0/2425492613 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x560c983adf60 0x560c987a6890 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f68412a4d7df46e server_cookie=8b89b1754b9e857e in_seq=0 out_seq=0 2024-05-21T15:36:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb178ff9640 1 -- 172.21.15.38:0/2425492613 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560c98701370 con 0x560c983adf60 2024-05-21T15:36:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb178ff9640 10 client.25697 handle_command_reply: tid=0 2024-05-21T15:36:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb19e5cc740 1 client.25697 shutdown 2024-05-21T15:36:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb19e5cc740 1 -- 172.21.15.38:0/2425492613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 msgr2=0x560c9879fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/2425492613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c9879fe20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb180010fc0 tx=0x7fb180004730 comp rx=0 tx=0).stop 2024-05-21T15:36:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb19e5cc740 1 -- 172.21.15.38:0/2425492613 shutdown_connections 2024-05-21T15:36:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/2425492613 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x560c983adf60 0x560c987a6890 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/2425492613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c9879fe20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/2425492613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c98798ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/2425492613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 0x560c98798980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb178ff9640 0 client.25697 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:36:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb19e5cc740 1 -- 172.21.15.38:0/2425492613 >> 172.21.15.38:0/2425492613 conn(0x560c983e34e0 msgr2=0x560c98789570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb19e5cc740 1 -- 172.21.15.38:0/2425492613 shutdown_connections 2024-05-21T15:36:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb19e5cc740 1 -- 172.21.15.38:0/2425492613 wait complete. 2024-05-21T15:36:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.076+0000 7fb19e5cc740 20 client.25697 trim_cache size 0 max 16384 2024-05-21T15:36:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.091+0000 7fb19e5cc740 1 Processor -- start 2024-05-21T15:36:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.091+0000 7fb19e5cc740 1 -- start start 2024-05-21T15:36:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.091+0000 7fb19e5cc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 0x560c98797660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb19e5cc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c9879fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb19e5cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c987ab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb19e5cc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901ac530 con 0x560c983ce6b0 2024-05-21T15:36:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb19e5cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901ae070 con 0x560c983ef390 2024-05-21T15:36:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb19e5cc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901acc00 con 0x560c983c64d0 2024-05-21T15:36:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb1954c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c9879fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb194cc5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c987ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb1954c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c9879fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46946/0 (socket says 172.21.15.38:46946) 2024-05-21T15:36:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb194cc5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c987ab040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53472/0 (socket says 172.21.15.38:53472) 2024-05-21T15:36:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb1954c6640 1 -- 172.21.15.38:0/3900533390 learned_addr learned my addr 172.21.15.38:0/3900533390 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb194cc5640 1 --2- 172.21.15.38:0/3900533390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 0x560c98797660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb1954c6640 1 -- 172.21.15.38:0/3900533390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 msgr2=0x560c98797660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb1954c6640 1 --2- 172.21.15.38:0/3900533390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 0x560c98797660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb1954c6640 1 -- 172.21.15.38:0/3900533390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 msgr2=0x560c987ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb1954c6640 1 --2- 172.21.15.38:0/3900533390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c987ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb1954c6640 1 -- 172.21.15.38:0/3900533390 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c98785d20 con 0x560c983ce6b0 2024-05-21T15:36:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb194cc5640 1 --2- 172.21.15.38:0/3900533390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c987ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb1954c6640 1 --2- 172.21.15.38:0/3900533390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c9879fdb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb18c005d00 tx=0x7fb18c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6e942f6bccef30 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb17affd640 1 -- 172.21.15.38:0/3900533390 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c02f070 con 0x560c983ce6b0 2024-05-21T15:36:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.092+0000 7fb17affd640 1 -- 172.21.15.38:0/3900533390 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb18c00be20 con 0x560c983ce6b0 2024-05-21T15:36:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.093+0000 7fb17affd640 1 -- 172.21.15.38:0/3900533390 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c004860 con 0x560c983ce6b0 2024-05-21T15:36:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.093+0000 7fb19e5cc740 1 -- 172.21.15.38:0/3900533390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 msgr2=0x560c9879fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.093+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/3900533390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c9879fdb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb18c005d00 tx=0x7fb18c01e040 comp rx=0 tx=0).stop 2024-05-21T15:36:21.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.093+0000 7fb19e5cc740 1 -- 172.21.15.38:0/3900533390 shutdown_connections 2024-05-21T15:36:21.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.093+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/3900533390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c987ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.093+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/3900533390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c9879fdb0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.093+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/3900533390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 0x560c98797660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.093+0000 7fb19e5cc740 1 -- 172.21.15.38:0/3900533390 >> 172.21.15.38:0/3900533390 conn(0x560c983e34e0 msgr2=0x560c98796f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:21.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.093+0000 7fb19e5cc740 1 -- 172.21.15.38:0/3900533390 shutdown_connections 2024-05-21T15:36:21.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.093+0000 7fb19e5cc740 1 -- 172.21.15.38:0/3900533390 wait complete. 2024-05-21T15:36:21.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.094+0000 7fb19e5cc740 1 -- start start 2024-05-21T15:36:21.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.094+0000 7fb19e5cc740 1 Processor -- start 2024-05-21T15:36:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.094+0000 7fb19e5cc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 0x560c9879cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.094+0000 7fb19e5cc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c9879d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.094+0000 7fb19e5cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c9880cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.094+0000 7fb19e5cc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901ad9a0 con 0x560c983ce6b0 2024-05-21T15:36:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.094+0000 7fb19e5cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901ad2d0 con 0x560c983ef390 2024-05-21T15:36:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.094+0000 7fb19e5cc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901aa510 con 0x560c983c64d0 2024-05-21T15:36:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.094+0000 7fb19e5cc740 10 client.0 _ll_register_callbacks cb 0x7fb180009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.094+0000 7fb1954c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c9879d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.094+0000 7fb1954c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c9879d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46954/0 (socket says 172.21.15.38:46954) 2024-05-21T15:36:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.094+0000 7fb1954c6640 1 -- 172.21.15.38:0/3327215318 learned_addr learned my addr 172.21.15.38:0/3327215318 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.094+0000 7fb194cc5640 1 --2- 172.21.15.38:0/3327215318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c9880cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.094+0000 7fb194cc5640 1 --2- 172.21.15.38:0/3327215318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 0x560c9879cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.095+0000 7fb1954c6640 1 -- 172.21.15.38:0/3327215318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 msgr2=0x560c9879cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.095+0000 7fb1954c6640 1 --2- 172.21.15.38:0/3327215318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 0x560c9879cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.095+0000 7fb1954c6640 1 -- 172.21.15.38:0/3327215318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 msgr2=0x560c9880cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.095+0000 7fb1954c6640 1 --2- 172.21.15.38:0/3327215318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c9880cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.095+0000 7fb1954c6640 1 -- 172.21.15.38:0/3327215318 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c9880d1b0 con 0x560c983ce6b0 2024-05-21T15:36:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.095+0000 7fb194cc5640 1 --2- 172.21.15.38:0/3327215318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c9880cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.095+0000 7fb1954c6640 1 --2- 172.21.15.38:0/3327215318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c9879d500 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb18c02d040 tx=0x7fb18c0117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a95e734e4d817b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.095+0000 7fb178ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.095+0000 7fb178ff9640 1 -- 172.21.15.38:0/3327215318 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c02f040 con 0x560c983ce6b0 2024-05-21T15:36:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.095+0000 7fb178ff9640 1 -- 172.21.15.38:0/3327215318 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fb18c011d60 con 0x560c983ce6b0 2024-05-21T15:36:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.095+0000 7fb19e5cc740 1 -- 172.21.15.38:0/3327215318 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb18c00be20 con 0x560c983ce6b0 2024-05-21T15:36:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.095+0000 7fb178ff9640 1 -- 172.21.15.38:0/3327215318 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb18c02b160 con 0x560c983ce6b0 2024-05-21T15:36:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.095+0000 7fb19e5cc740 10 client.16038 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.095+0000 7fb19e5cc740 1 -- 172.21.15.38:0/3327215318 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560c9880e430 con 0x560c983ce6b0 2024-05-21T15:36:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.096+0000 7fb178ff9640 1 -- 172.21.15.38:0/3327215318 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb18c00be20 con 0x560c983ce6b0 2024-05-21T15:36:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.096+0000 7fb178ff9640 1 -- 172.21.15.38:0/3327215318 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb18c01c850 con 0x560c983ce6b0 2024-05-21T15:36:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.096+0000 7fb19e5cc740 10 client.16038 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.096+0000 7fb19e5cc740 10 client.16038 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:36:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.096+0000 7fb19e5cc740 20 client.16038 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.096+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/3327215318 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x560c983833a0 0x560c98810d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.096+0000 7fb19e5cc740 4 client.16038 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:36:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.096+0000 7fb19e5cc740 1 -- 172.21.15.38:0/3327215318 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c986fb6c0 con 0x560c983833a0 2024-05-21T15:36:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.096+0000 7fb194cc5640 1 --2- 172.21.15.38:0/3327215318 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x560c983833a0 0x560c98810d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.096+0000 7fb178ff9640 10 client.16038 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:36:21.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb194cc5640 1 --2- 172.21.15.38:0/3327215318 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x560c983833a0 0x560c98810d80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29046d959f5bdb16 server_cookie=1876588ba18762d in_seq=0 out_seq=0 2024-05-21T15:36:21.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb178ff9640 1 -- 172.21.15.38:0/3327215318 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c986fb6c0 con 0x560c983833a0 2024-05-21T15:36:21.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb178ff9640 10 client.16038 handle_command_reply: tid=0 2024-05-21T15:36:21.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb19e5cc740 1 client.16038 shutdown 2024-05-21T15:36:21.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb19e5cc740 1 -- 172.21.15.38:0/3327215318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 msgr2=0x560c9879d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/3327215318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c9879d500 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb18c02d040 tx=0x7fb18c0117a0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb19e5cc740 1 -- 172.21.15.38:0/3327215318 shutdown_connections 2024-05-21T15:36:21.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/3327215318 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x560c983833a0 0x560c98810d80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/3327215318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c983ef390 0x560c9880cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb178ff9640 0 client.16038 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:36:21.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/3327215318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c983ce6b0 0x560c9879d500 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb19e5cc740 1 --2- 172.21.15.38:0/3327215318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c983c64d0 0x560c9879cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb19e5cc740 1 -- 172.21.15.38:0/3327215318 >> 172.21.15.38:0/3327215318 conn(0x560c983e34e0 msgr2=0x560c9878d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:21.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb19e5cc740 1 -- 172.21.15.38:0/3327215318 shutdown_connections 2024-05-21T15:36:21.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb19e5cc740 1 -- 172.21.15.38:0/3327215318 wait complete. 2024-05-21T15:36:21.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.097+0000 7fb19e5cc740 20 client.16038 trim_cache size 0 max 16384 2024-05-21T15:36:21.103 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:36:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.098+0000 7fb19772a640 1 -- 172.21.15.38:0/3858365803 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb17c0a2b90 msgr2=0x7fb17c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.098+0000 7fb19772a640 1 --2- 172.21.15.38:0/3858365803 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb17c0a2b90 0x7fb17c0a50a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb1880ee1e0 tx=0x7fb1880e7760 comp rx=0 tx=0).stop 2024-05-21T15:36:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.098+0000 7fb19772a640 1 -- 172.21.15.38:0/3858365803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb190104060 msgr2=0x7fb190101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.098+0000 7fb19772a640 1 --2- 172.21.15.38:0/3858365803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb190104060 0x7fb190101df0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb188005cc0 tx=0x7fb188004ab0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.098+0000 7fb19772a640 1 -- 172.21.15.38:0/3858365803 shutdown_connections 2024-05-21T15:36:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.098+0000 7fb19772a640 1 --2- 172.21.15.38:0/3858365803 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb17c0a2b90 0x7fb17c0a50a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.098+0000 7fb19772a640 1 --2- 172.21.15.38:0/3858365803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb190104060 0x7fb190101df0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.098+0000 7fb19772a640 1 --2- 172.21.15.38:0/3858365803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb190101210 0x7fb1901018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.098+0000 7fb19772a640 1 --2- 172.21.15.38:0/3858365803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb190100710 0x7fb190103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.098+0000 7fb19772a640 1 -- 172.21.15.38:0/3858365803 >> 172.21.15.38:0/3858365803 conn(0x7fb1900fc330 msgr2=0x7fb190106bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.098+0000 7fb19772a640 1 -- 172.21.15.38:0/3858365803 shutdown_connections 2024-05-21T15:36:21.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.098+0000 7fb19772a640 1 -- 172.21.15.38:0/3858365803 wait complete. 2024-05-21T15:36:21.105 INFO:tasks.workunit.client.0.smithi038.stdout:checking sys/limits.h presence... no 2024-05-21T15:36:21.105 INFO:tasks.workunit.client.0.smithi038.stdout:checking for sys/limits.h... no 2024-05-21T15:36:21.108 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:36:21.131 INFO:tasks.workunit.client.0.smithi038.stdout:checking for an ANSI C-conforming const... yes 2024-05-21T15:36:21.161 INFO:tasks.workunit.client.0.smithi038.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-21T15:36:21.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:20 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2128040485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:36:21.225 INFO:tasks.workunit.client.0.smithi038.stdout:checking for system... yes 2024-05-21T15:36:21.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.252+0000 7f87fe777640 1 -- 172.21.15.38:0/1545596519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f8101990 msgr2=0x7f87f8102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.252+0000 7f87fe777640 1 --2- 172.21.15.38:0/1545596519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f8101990 0x7f87f8102690 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009900 tx=0x7f87e802a410 comp rx=0 tx=0).stop 2024-05-21T15:36:21.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.252+0000 7f87fe777640 1 -- 172.21.15.38:0/1545596519 shutdown_connections 2024-05-21T15:36:21.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.252+0000 7f87fe777640 1 --2- 172.21.15.38:0/1545596519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f81078d0 0x7f87f8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.252+0000 7f87fe777640 1 --2- 172.21.15.38:0/1545596519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f8102bd0 0x7f87f8102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.252+0000 7f87fe777640 1 --2- 172.21.15.38:0/1545596519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f8101990 0x7f87f8102690 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.252+0000 7f87fe777640 1 -- 172.21.15.38:0/1545596519 >> 172.21.15.38:0/1545596519 conn(0x7f87f80fcb60 msgr2=0x7f87f80ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:21.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.252+0000 7f87fe777640 1 -- 172.21.15.38:0/1545596519 shutdown_connections 2024-05-21T15:36:21.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.252+0000 7f87fe777640 1 -- 172.21.15.38:0/1545596519 wait complete. 2024-05-21T15:36:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87fe777640 1 Processor -- start 2024-05-21T15:36:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87fe777640 1 -- start start 2024-05-21T15:36:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87fe777640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f8101990 0x7f87f8199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87fe777640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f8102bd0 0x7f87f8199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87fe777640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f81078d0 0x7f87f81a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87fe777640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f87f810e710 con 0x7f87f8101990 2024-05-21T15:36:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87fe777640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87f810e590 con 0x7f87f81078d0 2024-05-21T15:36:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87fe777640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f87f810e890 con 0x7f87f8102bd0 2024-05-21T15:36:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f8101990 0x7f87f8199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f8101990 0x7f87f8199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46980/0 (socket says 172.21.15.38:46980) 2024-05-21T15:36:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87f7fff640 1 -- 172.21.15.38:0/882311384 learned_addr learned my addr 172.21.15.38:0/882311384 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87fcd14640 1 --2- 172.21.15.38:0/882311384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f81078d0 0x7f87f81a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87effff640 1 --2- 172.21.15.38:0/882311384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f8102bd0 0x7f87f8199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87f7fff640 1 -- 172.21.15.38:0/882311384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f8102bd0 msgr2=0x7f87f8199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87f7fff640 1 --2- 172.21.15.38:0/882311384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f8102bd0 0x7f87f8199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87f7fff640 1 -- 172.21.15.38:0/882311384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f81078d0 msgr2=0x7f87f81a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87f7fff640 1 --2- 172.21.15.38:0/882311384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f81078d0 0x7f87f81a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.253+0000 7f87f7fff640 1 -- 172.21.15.38:0/882311384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f81a0e40 con 0x7f87f8101990 2024-05-21T15:36:21.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.254+0000 7f87f7fff640 1 --2- 172.21.15.38:0/882311384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f8101990 0x7f87f8199300 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f87e8005ab0 tx=0x7f87e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:21.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.254+0000 7f87f5ffb640 1 -- 172.21.15.38:0/882311384 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e803f030 con 0x7f87f8101990 2024-05-21T15:36:21.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.254+0000 7f87f5ffb640 1 -- 172.21.15.38:0/882311384 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e8003ad0 con 0x7f87f8101990 2024-05-21T15:36:21.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.254+0000 7f87fe777640 1 -- 172.21.15.38:0/882311384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f81a10d0 con 0x7f87f8101990 2024-05-21T15:36:21.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.254+0000 7f87f5ffb640 1 -- 172.21.15.38:0/882311384 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8035070 con 0x7f87f8101990 2024-05-21T15:36:21.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.254+0000 7f87fe777640 1 -- 172.21.15.38:0/882311384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f81a15c0 con 0x7f87f8101990 2024-05-21T15:36:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.256+0000 7f87f5ffb640 1 -- 172.21.15.38:0/882311384 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87e8042030 con 0x7f87f8101990 2024-05-21T15:36:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.257+0000 7f8805629740 1 Processor -- start 2024-05-21T15:36:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.257+0000 7f8805629740 1 -- start start 2024-05-21T15:36:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.257+0000 7f8805629740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c9f390 0x560756042f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.257+0000 7f8805629740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c764d0 0x560756043490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.257+0000 7f8805629740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c7e6b0 0x56075604a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.257+0000 7f8805629740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560755fa3c40 con 0x560755c9f390 2024-05-21T15:36:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.257+0000 7f8805629740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560755f4b580 con 0x560755c7e6b0 2024-05-21T15:36:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.257+0000 7f8805629740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560755f9fc00 con 0x560755c764d0 2024-05-21T15:36:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.257+0000 7f87fcd14640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c9f390 0x560756042f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.257+0000 7f87effff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c7e6b0 0x56075604a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.257+0000 7f87fcd14640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c9f390 0x560756042f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46982/0 (socket says 172.21.15.38:46982) 2024-05-21T15:36:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.257+0000 7f87fcd14640 1 -- 172.21.15.38:0/1386749444 learned_addr learned my addr 172.21.15.38:0/1386749444 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.257+0000 7f87f5ffb640 1 --2- 172.21.15.38:0/882311384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f87e40a2b90 0x7f87e40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.257+0000 7f87f7fff640 1 --2- 172.21.15.38:0/1386749444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c764d0 0x560756043490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.258+0000 7f87effff640 1 -- 172.21.15.38:0/1386749444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c764d0 msgr2=0x560756043490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.258+0000 7f87effff640 1 --2- 172.21.15.38:0/1386749444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c764d0 0x560756043490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.258+0000 7f87effff640 1 -- 172.21.15.38:0/1386749444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c9f390 msgr2=0x560756042f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.258+0000 7f87effff640 1 --2- 172.21.15.38:0/1386749444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c9f390 0x560756042f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.258+0000 7f87effff640 1 -- 172.21.15.38:0/1386749444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560755cb73f0 con 0x560755c7e6b0 2024-05-21T15:36:21.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.258+0000 7f87fcd14640 1 --2- 172.21.15.38:0/882311384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f87e40a2b90 0x7f87e40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.258+0000 7f87f5ffb640 1 -- 172.21.15.38:0/882311384 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f87e80ef640 con 0x7f87f8101990 2024-05-21T15:36:21.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.258+0000 7f87effff640 1 --2- 172.21.15.38:0/1386749444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c7e6b0 0x56075604a390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009bd0 tx=0x7f87e0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d25212f1cb6eb82e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:21.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.258+0000 7f87ee7fc640 1 -- 172.21.15.38:0/1386749444 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0004230 con 0x560755c7e6b0 2024-05-21T15:36:21.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.258+0000 7f87ee7fc640 1 -- 172.21.15.38:0/1386749444 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e0009e40 con 0x560755c7e6b0 2024-05-21T15:36:21.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.258+0000 7f87ee7fc640 1 -- 172.21.15.38:0/1386749444 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0004d70 con 0x560755c7e6b0 2024-05-21T15:36:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.259+0000 7f8805629740 1 -- 172.21.15.38:0/1386749444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c7e6b0 msgr2=0x56075604a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.259+0000 7f87fcd14640 1 --2- 172.21.15.38:0/882311384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f87e40a2b90 0x7f87e40a50a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5607560440e0 tx=0x7f87f0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.259+0000 7f8805629740 1 --2- 172.21.15.38:0/1386749444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c7e6b0 0x56075604a390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009bd0 tx=0x7f87e0011810 comp rx=0 tx=0).stop 2024-05-21T15:36:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.259+0000 7f8805629740 1 -- 172.21.15.38:0/1386749444 shutdown_connections 2024-05-21T15:36:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.259+0000 7f8805629740 1 --2- 172.21.15.38:0/1386749444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c7e6b0 0x56075604a390 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.259+0000 7f8805629740 1 --2- 172.21.15.38:0/1386749444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c764d0 0x560756043490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.259+0000 7f8805629740 1 --2- 172.21.15.38:0/1386749444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c9f390 0x560756042f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.259+0000 7f8805629740 1 -- 172.21.15.38:0/1386749444 >> 172.21.15.38:0/1386749444 conn(0x560755c934e0 msgr2=0x56075603a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.259+0000 7f8805629740 1 -- 172.21.15.38:0/1386749444 shutdown_connections 2024-05-21T15:36:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.259+0000 7f8805629740 1 -- 172.21.15.38:0/1386749444 wait complete. 2024-05-21T15:36:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.260+0000 7f8805629740 1 -- start start 2024-05-21T15:36:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.260+0000 7f8805629740 1 Processor -- start 2024-05-21T15:36:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.260+0000 7f8805629740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c764d0 0x56075604cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.260+0000 7f8805629740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c7e6b0 0x56075604d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.260+0000 7f8805629740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c9f390 0x560756054180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.260+0000 7f8805629740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56075604ade0 con 0x560755c9f390 2024-05-21T15:36:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.260+0000 7f8805629740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560755f5ec40 con 0x560755c7e6b0 2024-05-21T15:36:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.260+0000 7f8805629740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56075604af60 con 0x560755c764d0 2024-05-21T15:36:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.260+0000 7f8805629740 10 client.0 _ll_register_callbacks cb 0x560755cb6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.260+0000 7f87fcd14640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c9f390 0x560756054180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.260+0000 7f87fcd14640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c9f390 0x560756054180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46994/0 (socket says 172.21.15.38:46994) 2024-05-21T15:36:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.260+0000 7f87fcd14640 1 -- 172.21.15.38:0/2513816924 learned_addr learned my addr 172.21.15.38:0/2513816924 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.260+0000 7f87effff640 1 --2- 172.21.15.38:0/2513816924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c7e6b0 0x56075604d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.260+0000 7f87f7fff640 1 --2- 172.21.15.38:0/2513816924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c764d0 0x56075604cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.261+0000 7f87fcd14640 1 -- 172.21.15.38:0/2513816924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c764d0 msgr2=0x56075604cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.261+0000 7f87fcd14640 1 --2- 172.21.15.38:0/2513816924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c764d0 0x56075604cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.261+0000 7f87fcd14640 1 -- 172.21.15.38:0/2513816924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c7e6b0 msgr2=0x56075604d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.261+0000 7f87fcd14640 1 --2- 172.21.15.38:0/2513816924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c7e6b0 0x56075604d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.261+0000 7f87fcd14640 1 -- 172.21.15.38:0/2513816924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560755d817d0 con 0x560755c9f390 2024-05-21T15:36:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.261+0000 7f87fcd14640 1 --2- 172.21.15.38:0/2513816924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c9f390 0x560756054180 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f87f0010ae0 tx=0x7f87f0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be915ff77a2cab2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.261+0000 7f87d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.261+0000 7f87d77fe640 1 -- 172.21.15.38:0/2513816924 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f002e060 con 0x560755c9f390 2024-05-21T15:36:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.261+0000 7f8805629740 1 -- 172.21.15.38:0/2513816924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560755fb1370 con 0x560755c9f390 2024-05-21T15:36:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.261+0000 7f87d77fe640 1 -- 172.21.15.38:0/2513816924 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f87f001dd80 con 0x560755c9f390 2024-05-21T15:36:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.261+0000 7f87d77fe640 1 -- 172.21.15.38:0/2513816924 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f87f0028400 con 0x560755c9f390 2024-05-21T15:36:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.261+0000 7f8805629740 10 client.16050 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.261+0000 7f8805629740 1 -- 172.21.15.38:0/2513816924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560756057510 con 0x560755c9f390 2024-05-21T15:36:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.262+0000 7f87d77fe640 1 -- 172.21.15.38:0/2513816924 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560755fb1370 con 0x560755c9f390 2024-05-21T15:36:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.262+0000 7f87d77fe640 1 -- 172.21.15.38:0/2513816924 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f87f0032070 con 0x560755c9f390 2024-05-21T15:36:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.262+0000 7f8805629740 10 client.16050 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.262+0000 7f8805629740 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.262+0000 7f8805629740 20 client.16050 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.262+0000 7f8805629740 1 --2- 172.21.15.38:0/2513816924 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560755c5df60 0x56075605ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.262+0000 7f8805629740 4 client.16050 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:21.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.262+0000 7f8805629740 1 -- 172.21.15.38:0/2513816924 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560755fab6c0 con 0x560755c5df60 2024-05-21T15:36:21.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.262+0000 7f87f7fff640 1 --2- 172.21.15.38:0/2513816924 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560755c5df60 0x56075605ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.263+0000 7f87f7fff640 1 --2- 172.21.15.38:0/2513816924 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560755c5df60 0x56075605ad70 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c297003584e973d server_cookie=442fff4b45847938 in_seq=0 out_seq=0 2024-05-21T15:36:21.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.263+0000 7f87d77fe640 10 client.16050 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:21.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.263+0000 7f87d77fe640 1 -- 172.21.15.38:0/2513816924 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560755fab6c0 con 0x560755c5df60 2024-05-21T15:36:21.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.264+0000 7f87d77fe640 10 client.16050 handle_command_reply: tid=0 2024-05-21T15:36:21.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.264+0000 7f8805629740 1 client.16050 shutdown 2024-05-21T15:36:21.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.264+0000 7f8805629740 1 -- 172.21.15.38:0/2513816924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c9f390 msgr2=0x560756054180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.264+0000 7f8805629740 1 --2- 172.21.15.38:0/2513816924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c9f390 0x560756054180 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f87f0010ae0 tx=0x7f87f0004ab0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.264+0000 7f8805629740 1 -- 172.21.15.38:0/2513816924 shutdown_connections 2024-05-21T15:36:21.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.264+0000 7f8805629740 1 --2- 172.21.15.38:0/2513816924 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560755c5df60 0x56075605ad70 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.264+0000 7f8805629740 1 --2- 172.21.15.38:0/2513816924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c9f390 0x560756054180 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.264+0000 7f87d77fe640 0 client.16050 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:21.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.264+0000 7f8805629740 1 --2- 172.21.15.38:0/2513816924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c7e6b0 0x56075604d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.264+0000 7f8805629740 1 --2- 172.21.15.38:0/2513816924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c764d0 0x56075604cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.264+0000 7f8805629740 1 -- 172.21.15.38:0/2513816924 >> 172.21.15.38:0/2513816924 conn(0x560755c934e0 msgr2=0x560756039570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.264+0000 7f8805629740 1 -- 172.21.15.38:0/2513816924 shutdown_connections 2024-05-21T15:36:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.264+0000 7f8805629740 1 -- 172.21.15.38:0/2513816924 wait complete. 2024-05-21T15:36:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.264+0000 7f8805629740 20 client.16050 trim_cache size 0 max 16384 2024-05-21T15:36:21.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.278+0000 7f8805629740 1 Processor -- start 2024-05-21T15:36:21.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.278+0000 7f8805629740 1 -- start start 2024-05-21T15:36:21.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.278+0000 7f8805629740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c934e0 0x56075604b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.278+0000 7f8805629740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c5df60 0x5607560540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.278+0000 7f8805629740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c764d0 0x56075604b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.278+0000 7f8805629740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81a81a0 con 0x560755c934e0 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.278+0000 7f8805629740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81a9ce0 con 0x560755c764d0 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.278+0000 7f8805629740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81a8870 con 0x560755c5df60 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.278+0000 7f87f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c934e0 0x56075604b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.278+0000 7f87f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c934e0 0x56075604b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47010/0 (socket says 172.21.15.38:47010) 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.278+0000 7f87f7fff640 1 -- 172.21.15.38:0/4193206378 learned_addr learned my addr 172.21.15.38:0/4193206378 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.278+0000 7f87fcd14640 1 --2- 172.21.15.38:0/4193206378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c764d0 0x56075604b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.278+0000 7f87effff640 1 --2- 172.21.15.38:0/4193206378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c5df60 0x5607560540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.279+0000 7f87fcd14640 1 -- 172.21.15.38:0/4193206378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c5df60 msgr2=0x5607560540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.279+0000 7f87fcd14640 1 --2- 172.21.15.38:0/4193206378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c5df60 0x5607560540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.279+0000 7f87fcd14640 1 -- 172.21.15.38:0/4193206378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c934e0 msgr2=0x56075604b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.279+0000 7f87fcd14640 1 --2- 172.21.15.38:0/4193206378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c934e0 0x56075604b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.279+0000 7f87fcd14640 1 -- 172.21.15.38:0/4193206378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560756035d20 con 0x560755c764d0 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.279+0000 7f87f7fff640 1 --2- 172.21.15.38:0/4193206378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c934e0 0x56075604b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:21.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.279+0000 7f87fcd14640 1 --2- 172.21.15.38:0/4193206378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c764d0 0x56075604b800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f87f0033990 tx=0x7f87f0010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a675f4297a0c886f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:21.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.279+0000 7f87edffb640 1 -- 172.21.15.38:0/4193206378 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0038030 con 0x560755c764d0 2024-05-21T15:36:21.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.279+0000 7f87edffb640 1 -- 172.21.15.38:0/4193206378 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f00333e0 con 0x560755c764d0 2024-05-21T15:36:21.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.279+0000 7f87edffb640 1 -- 172.21.15.38:0/4193206378 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0028910 con 0x560755c764d0 2024-05-21T15:36:21.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.280+0000 7f8805629740 1 -- 172.21.15.38:0/4193206378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c764d0 msgr2=0x56075604b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.280+0000 7f8805629740 1 --2- 172.21.15.38:0/4193206378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c764d0 0x56075604b800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f87f0033990 tx=0x7f87f0010740 comp rx=0 tx=0).stop 2024-05-21T15:36:21.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.280+0000 7f8805629740 1 -- 172.21.15.38:0/4193206378 shutdown_connections 2024-05-21T15:36:21.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.280+0000 7f8805629740 1 --2- 172.21.15.38:0/4193206378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c764d0 0x56075604b800 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.280+0000 7f8805629740 1 --2- 172.21.15.38:0/4193206378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c5df60 0x5607560540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.280+0000 7f8805629740 1 --2- 172.21.15.38:0/4193206378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c934e0 0x56075604b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.280+0000 7f8805629740 1 -- 172.21.15.38:0/4193206378 >> 172.21.15.38:0/4193206378 conn(0x560756046a60 msgr2=0x5607560471b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:21.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.280+0000 7f8805629740 1 -- 172.21.15.38:0/4193206378 shutdown_connections 2024-05-21T15:36:21.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.280+0000 7f8805629740 1 -- 172.21.15.38:0/4193206378 wait complete. 2024-05-21T15:36:21.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.281+0000 7f8805629740 1 -- start start 2024-05-21T15:36:21.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.281+0000 7f8805629740 1 Processor -- start 2024-05-21T15:36:21.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.281+0000 7f8805629740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c5df60 0x560756051540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.281+0000 7f8805629740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c764d0 0x560756051a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.281+0000 7f8805629740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c934e0 0x5607560529b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.281+0000 7f8805629740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81a9610 con 0x560755c764d0 2024-05-21T15:36:21.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.281+0000 7f8805629740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81a8f40 con 0x560755c934e0 2024-05-21T15:36:21.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.281+0000 7f8805629740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81a6180 con 0x560755c5df60 2024-05-21T15:36:21.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.281+0000 7f8805629740 10 client.0 _ll_register_callbacks cb 0x7f87f0009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:21.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.281+0000 7f87effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c764d0 0x560756051a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.281+0000 7f87effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c764d0 0x560756051a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47024/0 (socket says 172.21.15.38:47024) 2024-05-21T15:36:21.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.281+0000 7f87effff640 1 -- 172.21.15.38:0/454610311 learned_addr learned my addr 172.21.15.38:0/454610311 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:21.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.281+0000 7f87f7fff640 1 --2- 172.21.15.38:0/454610311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c5df60 0x560756051540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.281+0000 7f87fcd14640 1 --2- 172.21.15.38:0/454610311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c934e0 0x5607560529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.282+0000 7f87effff640 1 -- 172.21.15.38:0/454610311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c5df60 msgr2=0x560756051540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.282+0000 7f87effff640 1 --2- 172.21.15.38:0/454610311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c5df60 0x560756051540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.282+0000 7f87effff640 1 -- 172.21.15.38:0/454610311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c934e0 msgr2=0x5607560529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:21.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.282+0000 7f87effff640 1 --2- 172.21.15.38:0/454610311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c934e0 0x5607560529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.282+0000 7f87effff640 1 -- 172.21.15.38:0/454610311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560756052f70 con 0x560755c764d0 2024-05-21T15:36:21.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.282+0000 7f87effff640 1 --2- 172.21.15.38:0/454610311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c764d0 0x560756051a80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f87e00058c0 tx=0x7f87e0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86277a2cfaf0ad29 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:21.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.282+0000 7f87d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:21.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.282+0000 7f87d77fe640 1 -- 172.21.15.38:0/454610311 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e001e040 con 0x560755c764d0 2024-05-21T15:36:21.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.282+0000 7f8805629740 1 -- 172.21.15.38:0/454610311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f00333e0 con 0x560755c764d0 2024-05-21T15:36:21.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.282+0000 7f87d77fe640 1 -- 172.21.15.38:0/454610311 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f87e00037f0 con 0x560755c764d0 2024-05-21T15:36:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.282+0000 7f87d77fe640 1 -- 172.21.15.38:0/454610311 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f87e00039f0 con 0x560755c764d0 2024-05-21T15:36:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.282+0000 7f8805629740 10 client.16056 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.282+0000 7f8805629740 1 -- 172.21.15.38:0/454610311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560756054220 con 0x560755c764d0 2024-05-21T15:36:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.282+0000 7f87d77fe640 1 -- 172.21.15.38:0/454610311 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f00333e0 con 0x560755c764d0 2024-05-21T15:36:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.283+0000 7f87d77fe640 1 -- 172.21.15.38:0/454610311 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f87e0018070 con 0x560755c764d0 2024-05-21T15:36:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.283+0000 7f8805629740 10 client.16056 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.283+0000 7f8805629740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.283+0000 7f8805629740 20 client.16056 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.283+0000 7f8805629740 1 --2- 172.21.15.38:0/454610311 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560755c7e6b0 0x5607560c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.283+0000 7f8805629740 4 client.16056 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:36:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.283+0000 7f8805629740 1 -- 172.21.15.38:0/454610311 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560755f28820 con 0x560755c7e6b0 2024-05-21T15:36:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.283+0000 7f87f7fff640 1 --2- 172.21.15.38:0/454610311 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560755c7e6b0 0x5607560c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.283+0000 7f87d77fe640 10 client.16056 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.283+0000 7f87f7fff640 1 --2- 172.21.15.38:0/454610311 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560755c7e6b0 0x5607560c54d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9523226996c65384 server_cookie=5fc7fc6f05fd6f94 in_seq=0 out_seq=0 2024-05-21T15:36:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.284+0000 7f87d77fe640 1 -- 172.21.15.38:0/454610311 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560755f28820 con 0x560755c7e6b0 2024-05-21T15:36:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.284+0000 7f87d77fe640 10 client.16056 handle_command_reply: tid=0 2024-05-21T15:36:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.284+0000 7f8805629740 1 client.16056 shutdown 2024-05-21T15:36:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.284+0000 7f8805629740 1 -- 172.21.15.38:0/454610311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c764d0 msgr2=0x560756051a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.284+0000 7f8805629740 1 --2- 172.21.15.38:0/454610311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c764d0 0x560756051a80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f87e00058c0 tx=0x7f87e0004610 comp rx=0 tx=0).stop 2024-05-21T15:36:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.284+0000 7f8805629740 1 -- 172.21.15.38:0/454610311 shutdown_connections 2024-05-21T15:36:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.284+0000 7f8805629740 1 --2- 172.21.15.38:0/454610311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560755c934e0 0x5607560529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.284+0000 7f8805629740 1 --2- 172.21.15.38:0/454610311 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560755c7e6b0 0x5607560c54d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.284+0000 7f8805629740 1 --2- 172.21.15.38:0/454610311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560755c764d0 0x560756051a80 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.284+0000 7f8805629740 1 --2- 172.21.15.38:0/454610311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560755c5df60 0x560756051540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.284+0000 7f87d77fe640 0 client.16056 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.284+0000 7f8805629740 1 -- 172.21.15.38:0/454610311 >> 172.21.15.38:0/454610311 conn(0x560756046a60 msgr2=0x56075604b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f8805629740 1 -- 172.21.15.38:0/454610311 shutdown_connections 2024-05-21T15:36:21.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f8805629740 1 -- 172.21.15.38:0/454610311 wait complete. 2024-05-21T15:36:21.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f8805629740 20 client.16056 trim_cache size 0 max 16384 2024-05-21T15:36:21.289 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:36:21.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f87fe777640 1 -- 172.21.15.38:0/882311384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f87e40a2b90 msgr2=0x7f87e40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f87fe777640 1 --2- 172.21.15.38:0/882311384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f87e40a2b90 0x7f87e40a50a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5607560440e0 tx=0x7f87f0004a40 comp rx=0 tx=0).stop 2024-05-21T15:36:21.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f87fe777640 1 -- 172.21.15.38:0/882311384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f8101990 msgr2=0x7f87f8199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:21.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f87fe777640 1 --2- 172.21.15.38:0/882311384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f8101990 0x7f87f8199300 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f87e8005ab0 tx=0x7f87e8004270 comp rx=0 tx=0).stop 2024-05-21T15:36:21.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f87fe777640 1 -- 172.21.15.38:0/882311384 shutdown_connections 2024-05-21T15:36:21.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f87fe777640 1 --2- 172.21.15.38:0/882311384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87f81078d0 0x7f87f81a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f87fe777640 1 --2- 172.21.15.38:0/882311384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f8102bd0 0x7f87f8199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f87fe777640 1 --2- 172.21.15.38:0/882311384 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f87e40a2b90 0x7f87e40a50a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f87fe777640 1 --2- 172.21.15.38:0/882311384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87f8101990 0x7f87f8199300 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:21.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f87fe777640 1 -- 172.21.15.38:0/882311384 >> 172.21.15.38:0/882311384 conn(0x7f87f80fcb60 msgr2=0x7f87f81038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:21.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f87fe777640 1 -- 172.21.15.38:0/882311384 shutdown_connections 2024-05-21T15:36:21.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:21.285+0000 7f87fe777640 1 -- 172.21.15.38:0/882311384 wait complete. 2024-05-21T15:36:21.291 INFO:tasks.workunit.client.0.smithi038.stdout:checking for gettimeofday... yes 2024-05-21T15:36:21.346 INFO:tasks.workunit.client.0.smithi038.stdout:checking for mkdir... yes 2024-05-21T15:36:21.408 INFO:tasks.workunit.client.0.smithi038.stdout:checking for strerror... yes 2024-05-21T15:36:21.466 INFO:tasks.workunit.client.0.smithi038.stdout:checking for open64... yes 2024-05-21T15:36:21.524 INFO:tasks.workunit.client.0.smithi038.stdout:checking for stat64... yes 2024-05-21T15:36:21.581 INFO:tasks.workunit.client.0.smithi038.stdout:checking for fseeko64... yes 2024-05-21T15:36:21.639 INFO:tasks.workunit.client.0.smithi038.stdout:checking for lrand48_r... yes 2024-05-21T15:36:21.697 INFO:tasks.workunit.client.0.smithi038.stdout:checking for srand48_r... yes 2024-05-21T15:36:21.722 INFO:tasks.workunit.client.0.smithi038.stdout:configure: creating ./config.status 2024-05-21T15:36:21.786 INFO:tasks.workunit.client.0.smithi038.stdout: /bin/sh ./config.status 2024-05-21T15:36:21.836 INFO:tasks.workunit.client.0.smithi038.stdout:config.status: creating Makefile 2024-05-21T15:36:21.848 INFO:tasks.workunit.client.0.smithi038.stdout:config.status: creating config.h 2024-05-21T15:36:21.871 INFO:tasks.workunit.client.0.smithi038.stdout:config.status: executing depfiles commands 2024-05-21T15:36:21.882 INFO:tasks.workunit.client.0.smithi038.stdout:config.status: executing default commands 2024-05-21T15:36:21.910 INFO:tasks.workunit.client.0.smithi038.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-05-21T15:36:22.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:21 smithi045 ceph-mon[36409]: pgmap v409: 97 pgs: 97 active+clean; 4.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 333 KiB/s wr, 89 op/s 2024-05-21T15:36:22.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:21 smithi038 ceph-mon[31995]: pgmap v409: 97 pgs: 97 active+clean; 4.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 333 KiB/s wr, 89 op/s 2024-05-21T15:36:22.065 INFO:tasks.workunit.client.0.smithi038.stdout:rm -f stamp-h1 2024-05-21T15:36:22.066 INFO:tasks.workunit.client.0.smithi038.stdout:touch config.h.in 2024-05-21T15:36:22.068 INFO:tasks.workunit.client.0.smithi038.stdout:cd . && /bin/sh ./config.status config.h 2024-05-21T15:36:22.111 INFO:tasks.workunit.client.0.smithi038.stdout:config.status: creating config.h 2024-05-21T15:36:22.139 INFO:tasks.workunit.client.0.smithi038.stdout:make all-am 2024-05-21T15:36:22.145 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-05-21T15:36:22.146 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-05-21T15:36:22.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:21 smithi053 ceph-mon[36496]: pgmap v409: 97 pgs: 97 active+clean; 4.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 333 KiB/s wr, 89 op/s 2024-05-21T15:36:22.297 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:36:22.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:36:22.297 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:36:22.320 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-05-21T15:36:22.323 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-05-21T15:36:22.409 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-05-21T15:36:22.412 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-05-21T15:36:22.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.420+0000 7f6e7c33e640 1 -- 172.21.15.38:0/3594646124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e741012e0 msgr2=0x7f6e74108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.420+0000 7f6e7c33e640 1 --2- 172.21.15.38:0/3594646124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e741012e0 0x7f6e74108070 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009900 tx=0x7f6e6402a410 comp rx=0 tx=0).stop 2024-05-21T15:36:22.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.420+0000 7f6e7c33e640 1 -- 172.21.15.38:0/3594646124 shutdown_connections 2024-05-21T15:36:22.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.420+0000 7f6e7c33e640 1 --2- 172.21.15.38:0/3594646124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e74108700 0x7f6e7410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.420+0000 7f6e7c33e640 1 --2- 172.21.15.38:0/3594646124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e741012e0 0x7f6e74108070 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.420+0000 7f6e7c33e640 1 --2- 172.21.15.38:0/3594646124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e74100de0 0x7f6e74105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.420+0000 7f6e7c33e640 1 -- 172.21.15.38:0/3594646124 >> 172.21.15.38:0/3594646124 conn(0x7f6e740fc2f0 msgr2=0x7f6e740fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:22.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.420+0000 7f6e7c33e640 1 -- 172.21.15.38:0/3594646124 shutdown_connections 2024-05-21T15:36:22.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.420+0000 7f6e7c33e640 1 -- 172.21.15.38:0/3594646124 wait complete. 2024-05-21T15:36:22.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.421+0000 7f6e7c33e640 1 Processor -- start 2024-05-21T15:36:22.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.421+0000 7f6e7c33e640 1 -- start start 2024-05-21T15:36:22.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.421+0000 7f6e7c33e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e74100de0 0x7f6e74199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.421+0000 7f6e7c33e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e741012e0 0x7f6e741997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.421+0000 7f6e7c33e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e74108700 0x7f6e741a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.421+0000 7f6e7c33e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7410e6c0 con 0x7f6e741012e0 2024-05-21T15:36:22.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.421+0000 7f6e7c33e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7410e540 con 0x7f6e74100de0 2024-05-21T15:36:22.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.421+0000 7f6e7c33e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7410e840 con 0x7f6e74108700 2024-05-21T15:36:22.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.421+0000 7f6e798d9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e741012e0 0x7f6e741997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e798d9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e741012e0 0x7f6e741997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47040/0 (socket says 172.21.15.38:47040) 2024-05-21T15:36:22.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e798d9640 1 -- 172.21.15.38:0/4222668653 learned_addr learned my addr 172.21.15.38:0/4222668653 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:22.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e7a0da640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e74100de0 0x7f6e74199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e7a8db640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e74108700 0x7f6e741a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e798d9640 1 -- 172.21.15.38:0/4222668653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e74108700 msgr2=0x7f6e741a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e798d9640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e74108700 0x7f6e741a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e798d9640 1 -- 172.21.15.38:0/4222668653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e74100de0 msgr2=0x7f6e74199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e798d9640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e74100de0 0x7f6e74199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e798d9640 1 -- 172.21.15.38:0/4222668653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e741a0da0 con 0x7f6e741012e0 2024-05-21T15:36:22.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e7a8db640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e74108700 0x7f6e741a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:36:22.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e7a0da640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e74100de0 0x7f6e74199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:22.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e798d9640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e741012e0 0x7f6e741997a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6e64006010 tx=0x7f6e640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:22.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e6b7fe640 1 -- 172.21.15.38:0/4222668653 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6403f030 con 0x7f6e741012e0 2024-05-21T15:36:22.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e6b7fe640 1 -- 172.21.15.38:0/4222668653 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e64003ce0 con 0x7f6e741012e0 2024-05-21T15:36:22.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e7c33e640 1 -- 172.21.15.38:0/4222668653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e741a1030 con 0x7f6e741012e0 2024-05-21T15:36:22.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e6b7fe640 1 -- 172.21.15.38:0/4222668653 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64035070 con 0x7f6e741012e0 2024-05-21T15:36:22.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.422+0000 7f6e7c33e640 1 -- 172.21.15.38:0/4222668653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e741a1520 con 0x7f6e741012e0 2024-05-21T15:36:22.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.425+0000 7f6e831df740 1 Processor -- start 2024-05-21T15:36:22.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.425+0000 7f6e6b7fe640 1 -- 172.21.15.38:0/4222668653 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6e64042030 con 0x7f6e741012e0 2024-05-21T15:36:22.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.425+0000 7f6e831df740 1 -- start start 2024-05-21T15:36:22.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.425+0000 7f6e831df740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f399410 0x555a4f7394a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.425+0000 7f6e831df740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f370550 0x555a4f739a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.425+0000 7f6e831df740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f378730 0x555a4f740920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e831df740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555a4f69d990 con 0x555a4f370550 2024-05-21T15:36:22.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e831df740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555a4f6455b0 con 0x555a4f399410 2024-05-21T15:36:22.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e831df740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555a4f699950 con 0x555a4f378730 2024-05-21T15:36:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e7a0da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f370550 0x555a4f739a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e7a0da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f370550 0x555a4f739a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47044/0 (socket says 172.21.15.38:47044) 2024-05-21T15:36:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e7a0da640 1 -- 172.21.15.38:0/1008707417 learned_addr learned my addr 172.21.15.38:0/1008707417 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e7a8db640 1 --2- 172.21.15.38:0/1008707417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f399410 0x555a4f7394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e798d9640 1 --2- 172.21.15.38:0/1008707417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f378730 0x555a4f740920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e6b7fe640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e600a2b90 0x7f6e600a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e7a0da640 1 -- 172.21.15.38:0/1008707417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f378730 msgr2=0x555a4f740920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e7a0da640 1 --2- 172.21.15.38:0/1008707417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f378730 0x555a4f740920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e7a0da640 1 -- 172.21.15.38:0/1008707417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f399410 msgr2=0x555a4f7394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e7a0da640 1 --2- 172.21.15.38:0/1008707417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f399410 0x555a4f7394a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e7a0da640 1 -- 172.21.15.38:0/1008707417 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a4f4797e0 con 0x555a4f370550 2024-05-21T15:36:22.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e7a8db640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e600a2b90 0x7f6e600a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e6b7fe640 1 -- 172.21.15.38:0/4222668653 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f6e640ef8c0 con 0x7f6e741012e0 2024-05-21T15:36:22.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.426+0000 7f6e7a0da640 1 --2- 172.21.15.38:0/1008707417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f370550 0x555a4f739a20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6e700115f0 tx=0x7f6e70011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a116adcf2f1454 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:22.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.427+0000 7f6e68bfe640 1 -- 172.21.15.38:0/1008707417 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7001e070 con 0x555a4f370550 2024-05-21T15:36:22.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.427+0000 7f6e68bfe640 1 -- 172.21.15.38:0/1008707417 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e7000fb20 con 0x555a4f370550 2024-05-21T15:36:22.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.427+0000 7f6e68bfe640 1 -- 172.21.15.38:0/1008707417 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70019c60 con 0x555a4f370550 2024-05-21T15:36:22.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e831df740 1 -- 172.21.15.38:0/1008707417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f370550 msgr2=0x555a4f739a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e831df740 1 --2- 172.21.15.38:0/1008707417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f370550 0x555a4f739a20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6e700115f0 tx=0x7f6e70011ab0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e7a8db640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e600a2b90 0x7f6e600a50a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x555a4f73a670 tx=0x7f6e6c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:22.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e831df740 1 -- 172.21.15.38:0/1008707417 shutdown_connections 2024-05-21T15:36:22.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e831df740 1 --2- 172.21.15.38:0/1008707417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f378730 0x555a4f740920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e831df740 1 --2- 172.21.15.38:0/1008707417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f370550 0x555a4f739a20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e831df740 1 --2- 172.21.15.38:0/1008707417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f399410 0x555a4f7394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e831df740 1 -- 172.21.15.38:0/1008707417 >> 172.21.15.38:0/1008707417 conn(0x555a4f38d560 msgr2=0x555a4f734e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:22.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e831df740 1 -- 172.21.15.38:0/1008707417 shutdown_connections 2024-05-21T15:36:22.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e831df740 1 -- 172.21.15.38:0/1008707417 wait complete. 2024-05-21T15:36:22.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e831df740 1 -- start start 2024-05-21T15:36:22.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e831df740 1 Processor -- start 2024-05-21T15:36:22.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e831df740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f370550 0x555a4f743450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e831df740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f378730 0x555a4f743990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.428+0000 7f6e831df740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f399410 0x555a4f74a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e831df740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555a4f741370 con 0x555a4f399410 2024-05-21T15:36:22.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e831df740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555a4f658b40 con 0x555a4f370550 2024-05-21T15:36:22.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e831df740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555a4f7414f0 con 0x555a4f378730 2024-05-21T15:36:22.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e831df740 10 client.0 _ll_register_callbacks cb 0x555a4f64d0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:22.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e7a8db640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f399410 0x555a4f74a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e7a0da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f370550 0x555a4f743450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e7a0da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f370550 0x555a4f743450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53588/0 (socket says 172.21.15.38:53588) 2024-05-21T15:36:22.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e7a8db640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f399410 0x555a4f74a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47056/0 (socket says 172.21.15.38:47056) 2024-05-21T15:36:22.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e7a0da640 1 -- 172.21.15.38:0/572844518 learned_addr learned my addr 172.21.15.38:0/572844518 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:22.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e798d9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f378730 0x555a4f743990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e7a0da640 1 -- 172.21.15.38:0/572844518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f378730 msgr2=0x555a4f743990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e7a0da640 1 --2- 172.21.15.38:0/572844518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f378730 0x555a4f743990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e7a0da640 1 -- 172.21.15.38:0/572844518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f399410 msgr2=0x555a4f74a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e7a0da640 1 --2- 172.21.15.38:0/572844518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f399410 0x555a4f74a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e7a0da640 1 -- 172.21.15.38:0/572844518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a4f4d9f70 con 0x555a4f370550 2024-05-21T15:36:22.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e7a8db640 1 --2- 172.21.15.38:0/572844518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f399410 0x555a4f74a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:22.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e7a0da640 1 --2- 172.21.15.38:0/572844518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f370550 0x555a4f743450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6e70005890 tx=0x7f6e7000f8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc86d6d17ef084e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:22.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:22.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e5dffb640 1 -- 172.21.15.38:0/572844518 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7001e040 con 0x555a4f370550 2024-05-21T15:36:22.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e5dffb640 1 -- 172.21.15.38:0/572844518 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f6e700199e0 con 0x555a4f370550 2024-05-21T15:36:22.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.429+0000 7f6e831df740 1 -- 172.21.15.38:0/572844518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e7000fb20 con 0x555a4f370550 2024-05-21T15:36:22.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.430+0000 7f6e5dffb640 1 -- 172.21.15.38:0/572844518 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6e70019410 con 0x555a4f370550 2024-05-21T15:36:22.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.430+0000 7f6e831df740 10 client.25733 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:22.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.430+0000 7f6e831df740 1 -- 172.21.15.38:0/572844518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a4f371170 con 0x555a4f370550 2024-05-21T15:36:22.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.430+0000 7f6e5dffb640 1 -- 172.21.15.38:0/572844518 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e7000fb20 con 0x555a4f370550 2024-05-21T15:36:22.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.430+0000 7f6e5dffb640 1 -- 172.21.15.38:0/572844518 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6e7002a030 con 0x555a4f370550 2024-05-21T15:36:22.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.430+0000 7f6e831df740 10 client.25733 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.430+0000 7f6e831df740 10 client.25733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.430+0000 7f6e831df740 20 client.25733 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.430+0000 7f6e831df740 1 --2- 172.21.15.38:0/572844518 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555a4f3581f0 0x555a4f750b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.430+0000 7f6e831df740 4 client.25733 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.430+0000 7f6e831df740 1 -- 172.21.15.38:0/572844518 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a4f6ab0c0 con 0x555a4f3581f0 2024-05-21T15:36:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.430+0000 7f6e798d9640 1 --2- 172.21.15.38:0/572844518 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555a4f3581f0 0x555a4f750b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.431+0000 7f6e5dffb640 10 client.25733 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.431+0000 7f6e798d9640 1 --2- 172.21.15.38:0/572844518 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555a4f3581f0 0x555a4f750b50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19224aa4501c1c7c server_cookie=9ba32cc87d17f6a5 in_seq=0 out_seq=0 2024-05-21T15:36:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.431+0000 7f6e5dffb640 1 -- 172.21.15.38:0/572844518 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555a4f6ab0c0 con 0x555a4f3581f0 2024-05-21T15:36:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.431+0000 7f6e5dffb640 10 client.25733 handle_command_reply: tid=0 2024-05-21T15:36:22.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.432+0000 7f6e831df740 1 client.25733 shutdown 2024-05-21T15:36:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.432+0000 7f6e831df740 1 -- 172.21.15.38:0/572844518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f370550 msgr2=0x555a4f743450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.432+0000 7f6e831df740 1 --2- 172.21.15.38:0/572844518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f370550 0x555a4f743450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6e70005890 tx=0x7f6e7000f8f0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.432+0000 7f6e831df740 1 -- 172.21.15.38:0/572844518 shutdown_connections 2024-05-21T15:36:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.432+0000 7f6e831df740 1 --2- 172.21.15.38:0/572844518 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555a4f3581f0 0x555a4f750b50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.432+0000 7f6e831df740 1 --2- 172.21.15.38:0/572844518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f399410 0x555a4f74a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.432+0000 7f6e5dffb640 0 client.25733 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.432+0000 7f6e831df740 1 --2- 172.21.15.38:0/572844518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f378730 0x555a4f743990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.432+0000 7f6e831df740 1 --2- 172.21.15.38:0/572844518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f370550 0x555a4f743450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.432+0000 7f6e831df740 1 -- 172.21.15.38:0/572844518 >> 172.21.15.38:0/572844518 conn(0x555a4f38d560 msgr2=0x555a4f733e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.432+0000 7f6e831df740 1 -- 172.21.15.38:0/572844518 shutdown_connections 2024-05-21T15:36:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.432+0000 7f6e831df740 1 -- 172.21.15.38:0/572844518 wait complete. 2024-05-21T15:36:22.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.432+0000 7f6e831df740 20 client.25733 trim_cache size 0 max 16384 2024-05-21T15:36:22.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.450+0000 7f6e831df740 1 Processor -- start 2024-05-21T15:36:22.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e831df740 1 -- start start 2024-05-21T15:36:22.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e831df740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f3581f0 0x555a4f734750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e831df740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f370550 0x555a4f734c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e831df740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f378730 0x555a4f743020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e831df740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e741aa400 con 0x555a4f3581f0 2024-05-21T15:36:22.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e831df740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e741abf40 con 0x555a4f378730 2024-05-21T15:36:22.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e831df740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6e741aaad0 con 0x555a4f370550 2024-05-21T15:36:22.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e7a0da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f3581f0 0x555a4f734750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e7a0da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f3581f0 0x555a4f734750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47066/0 (socket says 172.21.15.38:47066) 2024-05-21T15:36:22.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e7a0da640 1 -- 172.21.15.38:0/171450518 learned_addr learned my addr 172.21.15.38:0/171450518 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:22.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e7a8db640 1 --2- 172.21.15.38:0/171450518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f378730 0x555a4f743020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e798d9640 1 --2- 172.21.15.38:0/171450518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f370550 0x555a4f734c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e7a0da640 1 -- 172.21.15.38:0/171450518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f370550 msgr2=0x555a4f734c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e7a0da640 1 --2- 172.21.15.38:0/171450518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f370550 0x555a4f734c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e7a0da640 1 -- 172.21.15.38:0/171450518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f378730 msgr2=0x555a4f743020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e7a0da640 1 --2- 172.21.15.38:0/171450518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f378730 0x555a4f743020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.451+0000 7f6e7a0da640 1 -- 172.21.15.38:0/171450518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a4f74b8c0 con 0x555a4f3581f0 2024-05-21T15:36:22.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.452+0000 7f6e7a0da640 1 --2- 172.21.15.38:0/171450518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f3581f0 0x555a4f734750 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6e700057c0 tx=0x7f6e70011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806e7ebdaacdea8f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:22.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.452+0000 7f6e5ffff640 1 -- 172.21.15.38:0/171450518 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7001e030 con 0x555a4f3581f0 2024-05-21T15:36:22.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.452+0000 7f6e5ffff640 1 -- 172.21.15.38:0/171450518 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e70004520 con 0x555a4f3581f0 2024-05-21T15:36:22.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.452+0000 7f6e5ffff640 1 -- 172.21.15.38:0/171450518 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70019d10 con 0x555a4f3581f0 2024-05-21T15:36:22.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 -- 172.21.15.38:0/171450518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f3581f0 msgr2=0x555a4f734750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 --2- 172.21.15.38:0/171450518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f3581f0 0x555a4f734750 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6e700057c0 tx=0x7f6e70011ab0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 -- 172.21.15.38:0/171450518 shutdown_connections 2024-05-21T15:36:22.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 --2- 172.21.15.38:0/171450518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f378730 0x555a4f743020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 --2- 172.21.15.38:0/171450518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f370550 0x555a4f734c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 --2- 172.21.15.38:0/171450518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f3581f0 0x555a4f734750 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 -- 172.21.15.38:0/171450518 >> 172.21.15.38:0/171450518 conn(0x555a4f38d560 msgr2=0x555a4f741a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 -- 172.21.15.38:0/171450518 shutdown_connections 2024-05-21T15:36:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 -- 172.21.15.38:0/171450518 wait complete. 2024-05-21T15:36:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 -- start start 2024-05-21T15:36:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 Processor -- start 2024-05-21T15:36:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f3581f0 0x555a4f746fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f370550 0x555a4f747500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f378730 0x555a4f748430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e741ab870 con 0x555a4f378730 2024-05-21T15:36:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e741ab1a0 con 0x555a4f3581f0 2024-05-21T15:36:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.453+0000 7f6e831df740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6e741a83e0 con 0x555a4f370550 2024-05-21T15:36:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e831df740 10 client.0 _ll_register_callbacks cb 0x7f6e7001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:22.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e7a8db640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f378730 0x555a4f748430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e7a0da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f3581f0 0x555a4f746fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e7a8db640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f378730 0x555a4f748430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47082/0 (socket says 172.21.15.38:47082) 2024-05-21T15:36:22.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e7a0da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f3581f0 0x555a4f746fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53614/0 (socket says 172.21.15.38:53614) 2024-05-21T15:36:22.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e7a8db640 1 -- 172.21.15.38:0/248826218 learned_addr learned my addr 172.21.15.38:0/248826218 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:22.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e798d9640 1 --2- 172.21.15.38:0/248826218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f370550 0x555a4f747500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e7a8db640 1 -- 172.21.15.38:0/248826218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f370550 msgr2=0x555a4f747500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e7a8db640 1 --2- 172.21.15.38:0/248826218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f370550 0x555a4f747500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e7a8db640 1 -- 172.21.15.38:0/248826218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f3581f0 msgr2=0x555a4f746fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e7a8db640 1 --2- 172.21.15.38:0/248826218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f3581f0 0x555a4f746fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e7a8db640 1 -- 172.21.15.38:0/248826218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a4f7489f0 con 0x555a4f378730 2024-05-21T15:36:22.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e7a0da640 1 --2- 172.21.15.38:0/248826218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f3581f0 0x555a4f746fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:22.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e7a8db640 1 --2- 172.21.15.38:0/248826218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f378730 0x555a4f748430 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00fdb0 tx=0x7f6e6c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2678cc60efbce04 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:22.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:22.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e5dffb640 1 -- 172.21.15.38:0/248826218 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c02c020 con 0x555a4f378730 2024-05-21T15:36:22.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e5dffb640 1 -- 172.21.15.38:0/248826218 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f6e6c01c0d0 con 0x555a4f378730 2024-05-21T15:36:22.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.454+0000 7f6e831df740 1 -- 172.21.15.38:0/248826218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e70004520 con 0x555a4f378730 2024-05-21T15:36:22.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.455+0000 7f6e5dffb640 1 -- 172.21.15.38:0/248826218 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6e6c037050 con 0x555a4f378730 2024-05-21T15:36:22.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.455+0000 7f6e831df740 10 client.16074 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:22.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.455+0000 7f6e831df740 1 -- 172.21.15.38:0/248826218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a4f749d80 con 0x555a4f378730 2024-05-21T15:36:22.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.455+0000 7f6e5dffb640 1 -- 172.21.15.38:0/248826218 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e70004520 con 0x555a4f378730 2024-05-21T15:36:22.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.455+0000 7f6e5dffb640 1 -- 172.21.15.38:0/248826218 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6e6c01a720 con 0x555a4f378730 2024-05-21T15:36:22.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.455+0000 7f6e831df740 10 client.16074 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:22.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.455+0000 7f6e831df740 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:22.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.455+0000 7f6e831df740 20 client.16074 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:22.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.455+0000 7f6e831df740 1 --2- 172.21.15.38:0/248826218 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555a4f399410 0x555a4f7bb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.455+0000 7f6e831df740 4 client.16074 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:36:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.455+0000 7f6e831df740 1 -- 172.21.15.38:0/248826218 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555a4f6a5410 con 0x555a4f399410 2024-05-21T15:36:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.455+0000 7f6e7a0da640 1 --2- 172.21.15.38:0/248826218 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555a4f399410 0x555a4f7bb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.456+0000 7f6e7a0da640 1 --2- 172.21.15.38:0/248826218 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555a4f399410 0x555a4f7bb310 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ee86d345f11d008 server_cookie=ad7bffa899afaaf in_seq=0 out_seq=0 2024-05-21T15:36:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.456+0000 7f6e5dffb640 10 client.16074 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e5dffb640 1 -- 172.21.15.38:0/248826218 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555a4f6a5410 con 0x555a4f399410 2024-05-21T15:36:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e5dffb640 10 client.16074 handle_command_reply: tid=0 2024-05-21T15:36:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e831df740 1 client.16074 shutdown 2024-05-21T15:36:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e831df740 1 -- 172.21.15.38:0/248826218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f378730 msgr2=0x555a4f748430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e831df740 1 --2- 172.21.15.38:0/248826218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f378730 0x555a4f748430 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00fdb0 tx=0x7f6e6c004ab0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e831df740 1 -- 172.21.15.38:0/248826218 shutdown_connections 2024-05-21T15:36:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e831df740 1 --2- 172.21.15.38:0/248826218 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555a4f399410 0x555a4f7bb310 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e831df740 1 --2- 172.21.15.38:0/248826218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555a4f378730 0x555a4f748430 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e831df740 1 --2- 172.21.15.38:0/248826218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a4f370550 0x555a4f747500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e5dffb640 0 client.16074 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:22.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e831df740 1 --2- 172.21.15.38:0/248826218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a4f3581f0 0x555a4f746fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e831df740 1 -- 172.21.15.38:0/248826218 >> 172.21.15.38:0/248826218 conn(0x555a4f38d560 msgr2=0x555a4f734750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:22.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e831df740 1 -- 172.21.15.38:0/248826218 shutdown_connections 2024-05-21T15:36:22.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e831df740 1 -- 172.21.15.38:0/248826218 wait complete. 2024-05-21T15:36:22.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.457+0000 7f6e831df740 20 client.16074 trim_cache size 0 max 16384 2024-05-21T15:36:22.461 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:36:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-05-21T15:36:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "b5c5d903-ff57-4f8b-9cc0-eb595688726e", 2024-05-21T15:36:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-05-21T15:36:22.462 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:36:22.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.458+0000 7f6e7c33e640 1 -- 172.21.15.38:0/4222668653 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e600a2b90 msgr2=0x7f6e600a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.458+0000 7f6e7c33e640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e600a2b90 0x7f6e600a50a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x555a4f73a670 tx=0x7f6e6c004a40 comp rx=0 tx=0).stop 2024-05-21T15:36:22.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.458+0000 7f6e7c33e640 1 -- 172.21.15.38:0/4222668653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e741012e0 msgr2=0x7f6e741997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.458+0000 7f6e7c33e640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e741012e0 0x7f6e741997a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6e64006010 tx=0x7f6e640042f0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.458+0000 7f6e7c33e640 1 -- 172.21.15.38:0/4222668653 shutdown_connections 2024-05-21T15:36:22.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.458+0000 7f6e7c33e640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e74108700 0x7f6e741a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.458+0000 7f6e7c33e640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e600a2b90 0x7f6e600a50a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.458+0000 7f6e7c33e640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e741012e0 0x7f6e741997a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.458+0000 7f6e7c33e640 1 --2- 172.21.15.38:0/4222668653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e74100de0 0x7f6e74199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.458+0000 7f6e7c33e640 1 -- 172.21.15.38:0/4222668653 >> 172.21.15.38:0/4222668653 conn(0x7f6e740fc2f0 msgr2=0x7f6e74104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:22.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.458+0000 7f6e7c33e640 1 -- 172.21.15.38:0/4222668653 shutdown_connections 2024-05-21T15:36:22.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.458+0000 7f6e7c33e640 1 -- 172.21.15.38:0/4222668653 wait complete. 2024-05-21T15:36:22.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:36:22.470 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:36:22.502 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-05-21T15:36:22.505 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-05-21T15:36:22.597 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-05-21T15:36:22.600 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-05-21T15:36:22.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.622+0000 7f54e7b1d640 1 -- 172.21.15.38:0/3210928950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54e01078f0 msgr2=0x7f54e0109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.622+0000 7f54e7b1d640 1 --2- 172.21.15.38:0/3210928950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54e01078f0 0x7f54e0109d10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009ca0 tx=0x7f54d802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.622+0000 7f54e7b1d640 1 -- 172.21.15.38:0/3210928950 shutdown_connections 2024-05-21T15:36:22.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.622+0000 7f54e7b1d640 1 --2- 172.21.15.38:0/3210928950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54e01078f0 0x7f54e0109d10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.622+0000 7f54e7b1d640 1 --2- 172.21.15.38:0/3210928950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e0102bf0 0x7f54e0103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.622+0000 7f54e7b1d640 1 --2- 172.21.15.38:0/3210928950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e01019b0 0x7f54e01026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.622+0000 7f54e7b1d640 1 -- 172.21.15.38:0/3210928950 >> 172.21.15.38:0/3210928950 conn(0x7f54e00fcbe0 msgr2=0x7f54e00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:22.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.622+0000 7f54e7b1d640 1 -- 172.21.15.38:0/3210928950 shutdown_connections 2024-05-21T15:36:22.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.622+0000 7f54e7b1d640 1 -- 172.21.15.38:0/3210928950 wait complete. 2024-05-21T15:36:22.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.623+0000 7f54e7b1d640 1 Processor -- start 2024-05-21T15:36:22.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.623+0000 7f54e7b1d640 1 -- start start 2024-05-21T15:36:22.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.623+0000 7f54e7b1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e01019b0 0x7f54e00747f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.623+0000 7f54e7b1d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54e0102bf0 0x7f54e019dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.623+0000 7f54e7b1d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e01078f0 0x7f54e01a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.623+0000 7f54e7b1d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f54e010d630 con 0x7f54e0102bf0 2024-05-21T15:36:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.623+0000 7f54e7b1d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f54e010d4b0 con 0x7f54e01078f0 2024-05-21T15:36:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.623+0000 7f54e7b1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f54e010d7b0 con 0x7f54e01019b0 2024-05-21T15:36:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.623+0000 7f54e50b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54e0102bf0 0x7f54e019dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54e50b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54e0102bf0 0x7f54e019dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47096/0 (socket says 172.21.15.38:47096) 2024-05-21T15:36:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54e50b8640 1 -- 172.21.15.38:0/545582213 learned_addr learned my addr 172.21.15.38:0/545582213 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54e58b9640 1 --2- 172.21.15.38:0/545582213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e01019b0 0x7f54e00747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54e60ba640 1 --2- 172.21.15.38:0/545582213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e01078f0 0x7f54e01a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54e50b8640 1 -- 172.21.15.38:0/545582213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e01019b0 msgr2=0x7f54e00747f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54e50b8640 1 --2- 172.21.15.38:0/545582213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e01019b0 0x7f54e00747f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54e50b8640 1 -- 172.21.15.38:0/545582213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e01078f0 msgr2=0x7f54e01a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54e50b8640 1 --2- 172.21.15.38:0/545582213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e01078f0 0x7f54e01a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54e50b8640 1 -- 172.21.15.38:0/545582213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e01a0d30 con 0x7f54e0102bf0 2024-05-21T15:36:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54e50b8640 1 --2- 172.21.15.38:0/545582213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54e0102bf0 0x7f54e019dfc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f54d000e9f0 tx=0x7f54d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54d6ffd640 1 -- 172.21.15.38:0/545582213 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d000cd80 con 0x7f54e0102bf0 2024-05-21T15:36:22.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54d6ffd640 1 -- 172.21.15.38:0/545582213 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d0005500 con 0x7f54e0102bf0 2024-05-21T15:36:22.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54e7b1d640 1 -- 172.21.15.38:0/545582213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e01a0fc0 con 0x7f54e0102bf0 2024-05-21T15:36:22.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54d6ffd640 1 -- 172.21.15.38:0/545582213 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0010690 con 0x7f54e0102bf0 2024-05-21T15:36:22.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.624+0000 7f54e7b1d640 1 -- 172.21.15.38:0/545582213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e01a1480 con 0x7f54e0102bf0 2024-05-21T15:36:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.627+0000 7f54ee9bd740 1 Processor -- start 2024-05-21T15:36:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.627+0000 7f54ee9bd740 1 -- start start 2024-05-21T15:36:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.627+0000 7f54d6ffd640 1 -- 172.21.15.38:0/545582213 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f54d00046c0 con 0x7f54e0102bf0 2024-05-21T15:36:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.627+0000 7f54ee9bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8d5390 0x55fb5cc78fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.627+0000 7f54ee9bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc79540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.627+0000 7f54ee9bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc80440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.627+0000 7f54ee9bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fb5cbd9d00 con 0x55fb5c8b46b0 2024-05-21T15:36:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.627+0000 7f54ee9bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fb5cb814c0 con 0x55fb5c8d5390 2024-05-21T15:36:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.627+0000 7f54ee9bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55fb5cbd5cc0 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.628+0000 7f54e50b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc80440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.628+0000 7f54e50b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc80440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47098/0 (socket says 172.21.15.38:47098) 2024-05-21T15:36:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.628+0000 7f54e50b8640 1 -- 172.21.15.38:0/4274536578 learned_addr learned my addr 172.21.15.38:0/4274536578 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.628+0000 7f54e60ba640 1 --2- 172.21.15.38:0/4274536578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8d5390 0x55fb5cc78fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.628+0000 7f54e58b9640 1 --2- 172.21.15.38:0/4274536578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc79540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.628+0000 7f54e50b8640 1 -- 172.21.15.38:0/4274536578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8ac4d0 msgr2=0x55fb5cc79540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.628+0000 7f54e50b8640 1 --2- 172.21.15.38:0/4274536578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc79540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.628+0000 7f54e50b8640 1 -- 172.21.15.38:0/4274536578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8d5390 msgr2=0x55fb5cc78fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.628+0000 7f54e50b8640 1 --2- 172.21.15.38:0/4274536578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8d5390 0x55fb5cc78fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.628+0000 7f54e50b8640 1 -- 172.21.15.38:0/4274536578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5c9b77d0 con 0x55fb5c8b46b0 2024-05-21T15:36:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.628+0000 7f54e50b8640 1 --2- 172.21.15.38:0/4274536578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc80440 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f54d00d5450 tx=0x7f54d00d5540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685d77337f0582af server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.628+0000 7f54d6ffd640 1 --2- 172.21.15.38:0/545582213 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f54cc0a2b90 0x7f54cc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.628+0000 7f54e60ba640 1 --2- 172.21.15.38:0/545582213 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f54cc0a2b90 0x7f54cc0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.628+0000 7f54cbfff640 1 -- 172.21.15.38:0/4274536578 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d00e3070 con 0x55fb5c8b46b0 2024-05-21T15:36:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.629+0000 7f54cbfff640 1 -- 172.21.15.38:0/4274536578 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d009c380 con 0x55fb5c8b46b0 2024-05-21T15:36:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.629+0000 7f54cbfff640 1 -- 172.21.15.38:0/4274536578 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d009ba50 con 0x55fb5c8b46b0 2024-05-21T15:36:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.629+0000 7f54d6ffd640 1 -- 172.21.15.38:0/545582213 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f54d00cf470 con 0x7f54e0102bf0 2024-05-21T15:36:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.630+0000 7f54ee9bd740 1 -- 172.21.15.38:0/4274536578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 msgr2=0x55fb5cc80440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.630+0000 7f54e60ba640 1 --2- 172.21.15.38:0/545582213 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f54cc0a2b90 0x7f54cc0a50a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55fb5cc7a190 tx=0x7f54d803d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.630+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/4274536578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc80440 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f54d00d5450 tx=0x7f54d00d5540 comp rx=0 tx=0).stop 2024-05-21T15:36:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.630+0000 7f54ee9bd740 1 -- 172.21.15.38:0/4274536578 shutdown_connections 2024-05-21T15:36:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.630+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/4274536578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc80440 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.630+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/4274536578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc79540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.630+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/4274536578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8d5390 0x55fb5cc78fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.630+0000 7f54ee9bd740 1 -- 172.21.15.38:0/4274536578 >> 172.21.15.38:0/4274536578 conn(0x55fb5c8c94e0 msgr2=0x55fb5cc70540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.630+0000 7f54ee9bd740 1 -- 172.21.15.38:0/4274536578 shutdown_connections 2024-05-21T15:36:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.630+0000 7f54ee9bd740 1 -- 172.21.15.38:0/4274536578 wait complete. 2024-05-21T15:36:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.630+0000 7f54ee9bd740 1 -- start start 2024-05-21T15:36:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.630+0000 7f54ee9bd740 1 Processor -- start 2024-05-21T15:36:22.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54ee9bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc82f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54ee9bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc83440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54ee9bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8d5390 0x55fb5cc8a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54ee9bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fb5cc80e90 con 0x55fb5c8b46b0 2024-05-21T15:36:22.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54ee9bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fb5cb94b90 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54ee9bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55fb5cc81010 con 0x55fb5c8d5390 2024-05-21T15:36:22.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54ee9bd740 10 client.0 _ll_register_callbacks cb 0x55fb5cbd0da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:22.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54e50b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc83440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54e58b9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc82f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54e50b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc83440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47106/0 (socket says 172.21.15.38:47106) 2024-05-21T15:36:22.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54e50b8640 1 -- 172.21.15.38:0/217161281 learned_addr learned my addr 172.21.15.38:0/217161281 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54e60ba640 1 --2- 172.21.15.38:0/217161281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8d5390 0x55fb5cc8a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54e58b9640 1 -- 172.21.15.38:0/217161281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8d5390 msgr2=0x55fb5cc8a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54e58b9640 1 --2- 172.21.15.38:0/217161281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8d5390 0x55fb5cc8a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54e58b9640 1 -- 172.21.15.38:0/217161281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 msgr2=0x55fb5cc83440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54e58b9640 1 --2- 172.21.15.38:0/217161281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc83440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54e58b9640 1 -- 172.21.15.38:0/217161281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5c8ed3f0 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54e58b9640 1 --2- 172.21.15.38:0/217161281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc82f00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55fb5cc7a3a0 tx=0x7f54dc012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bba8d825a590a18 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.631+0000 7f54c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54c97fa640 1 -- 172.21.15.38:0/217161281 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc004050 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54ee9bd740 1 -- 172.21.15.38:0/217161281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54d009c380 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54c97fa640 1 -- 172.21.15.38:0/217161281 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f54dc0041f0 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54c97fa640 1 -- 172.21.15.38:0/217161281 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54dc0043f0 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54ee9bd740 10 client.25751 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54ee9bd740 1 -- 172.21.15.38:0/217161281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fb5c8ad0f0 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54c97fa640 1 -- 172.21.15.38:0/217161281 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d009c380 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54c97fa640 1 -- 172.21.15.38:0/217161281 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f54dc003b80 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54ee9bd740 10 client.25751 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:22.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54ee9bd740 10 client.25751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:22.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54ee9bd740 20 client.25751 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:22.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/217161281 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fb5c893f60 0x55fb5cc90c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54ee9bd740 4 client.25751 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:22.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54ee9bd740 1 -- 172.21.15.38:0/217161281 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb5cbe7430 con 0x55fb5c893f60 2024-05-21T15:36:22.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.632+0000 7f54e50b8640 1 --2- 172.21.15.38:0/217161281 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fb5c893f60 0x55fb5cc90c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.633+0000 7f54c97fa640 10 client.25751 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:22.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.633+0000 7f54e50b8640 1 --2- 172.21.15.38:0/217161281 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fb5c893f60 0x55fb5cc90c20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa727005d4a4a7a server_cookie=a7cbed75db7bd34 in_seq=0 out_seq=0 2024-05-21T15:36:22.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.633+0000 7f54c97fa640 1 -- 172.21.15.38:0/217161281 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fb5cbe7430 con 0x55fb5c893f60 2024-05-21T15:36:22.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.634+0000 7f54c97fa640 10 client.25751 handle_command_reply: tid=0 2024-05-21T15:36:22.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.634+0000 7f54ee9bd740 1 client.25751 shutdown 2024-05-21T15:36:22.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.634+0000 7f54ee9bd740 1 -- 172.21.15.38:0/217161281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8ac4d0 msgr2=0x55fb5cc82f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.634+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/217161281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc82f00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55fb5cc7a3a0 tx=0x7f54dc012b60 comp rx=0 tx=0).stop 2024-05-21T15:36:22.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.634+0000 7f54ee9bd740 1 -- 172.21.15.38:0/217161281 shutdown_connections 2024-05-21T15:36:22.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.634+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/217161281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8d5390 0x55fb5cc8a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.634+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/217161281 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fb5c893f60 0x55fb5cc90c20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.634+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/217161281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc83440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.634+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/217161281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc82f00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.634+0000 7f54c97fa640 0 client.25751 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:22.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.634+0000 7f54ee9bd740 1 -- 172.21.15.38:0/217161281 >> 172.21.15.38:0/217161281 conn(0x55fb5c8c94e0 msgr2=0x55fb5cc6f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:22.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.634+0000 7f54ee9bd740 1 -- 172.21.15.38:0/217161281 shutdown_connections 2024-05-21T15:36:22.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.634+0000 7f54ee9bd740 1 -- 172.21.15.38:0/217161281 wait complete. 2024-05-21T15:36:22.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.634+0000 7f54ee9bd740 20 client.25751 trim_cache size 0 max 16384 2024-05-21T15:36:22.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.652+0000 7f54ee9bd740 1 Processor -- start 2024-05-21T15:36:22.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.652+0000 7f54ee9bd740 1 -- start start 2024-05-21T15:36:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.652+0000 7f54ee9bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc81b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.652+0000 7f54ee9bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc8a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.652+0000 7f54ee9bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8d5390 0x55fb5cc831e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.652+0000 7f54ee9bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01a8280 con 0x55fb5c8b46b0 2024-05-21T15:36:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.652+0000 7f54ee9bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01a9dc0 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.652+0000 7f54ee9bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01a8950 con 0x55fb5c8d5390 2024-05-21T15:36:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54e50b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc8a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54e50b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc8a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47122/0 (socket says 172.21.15.38:47122) 2024-05-21T15:36:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54e50b8640 1 -- 172.21.15.38:0/3919766919 learned_addr learned my addr 172.21.15.38:0/3919766919 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54e60ba640 1 --2- 172.21.15.38:0/3919766919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8d5390 0x55fb5cc831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54e58b9640 1 --2- 172.21.15.38:0/3919766919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc81b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54e50b8640 1 -- 172.21.15.38:0/3919766919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8d5390 msgr2=0x55fb5cc831e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54e50b8640 1 --2- 172.21.15.38:0/3919766919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8d5390 0x55fb5cc831e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54e50b8640 1 -- 172.21.15.38:0/3919766919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8ac4d0 msgr2=0x55fb5cc81b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54e50b8640 1 --2- 172.21.15.38:0/3919766919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc81b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54e50b8640 1 -- 172.21.15.38:0/3919766919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5cc79cb0 con 0x55fb5c8b46b0 2024-05-21T15:36:22.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54e60ba640 1 --2- 172.21.15.38:0/3919766919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8d5390 0x55fb5cc831e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:22.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54e50b8640 1 --2- 172.21.15.38:0/3919766919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc8a250 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f54d00980e0 tx=0x7f54d0004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef56ba1b9051a616 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:22.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54cb7fe640 1 -- 172.21.15.38:0/3919766919 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0048b00 con 0x55fb5c8b46b0 2024-05-21T15:36:22.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54cb7fe640 1 -- 172.21.15.38:0/3919766919 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d00d2d20 con 0x55fb5c8b46b0 2024-05-21T15:36:22.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.653+0000 7f54cb7fe640 1 -- 172.21.15.38:0/3919766919 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0048d40 con 0x55fb5c8b46b0 2024-05-21T15:36:22.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.654+0000 7f54ee9bd740 1 -- 172.21.15.38:0/3919766919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 msgr2=0x55fb5cc8a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.654+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/3919766919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc8a250 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f54d00980e0 tx=0x7f54d0004ae0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.654+0000 7f54ee9bd740 1 -- 172.21.15.38:0/3919766919 shutdown_connections 2024-05-21T15:36:22.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.654+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/3919766919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8d5390 0x55fb5cc831e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.654+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/3919766919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc8a250 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.654+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/3919766919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc81b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.654+0000 7f54ee9bd740 1 -- 172.21.15.38:0/3919766919 >> 172.21.15.38:0/3919766919 conn(0x55fb5c8c94e0 msgr2=0x55fb5cc82ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:22.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.654+0000 7f54ee9bd740 1 -- 172.21.15.38:0/3919766919 shutdown_connections 2024-05-21T15:36:22.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.654+0000 7f54ee9bd740 1 -- 172.21.15.38:0/3919766919 wait complete. 2024-05-21T15:36:22.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.655+0000 7f54ee9bd740 1 -- start start 2024-05-21T15:36:22.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.655+0000 7f54ee9bd740 1 Processor -- start 2024-05-21T15:36:22.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.655+0000 7f54ee9bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc87810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.655+0000 7f54ee9bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc87d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.655+0000 7f54ee9bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8d5390 0x55fb5cc88c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.655+0000 7f54ee9bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01a96f0 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.655+0000 7f54ee9bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01a9020 con 0x55fb5c8d5390 2024-05-21T15:36:22.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.655+0000 7f54ee9bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01a6260 con 0x55fb5c8b46b0 2024-05-21T15:36:22.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.655+0000 7f54ee9bd740 10 client.0 _ll_register_callbacks cb 0x7f54dc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:22.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.655+0000 7f54e58b9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc87810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.655+0000 7f54e58b9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc87810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47132/0 (socket says 172.21.15.38:47132) 2024-05-21T15:36:22.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.655+0000 7f54e58b9640 1 -- 172.21.15.38:0/3684191024 learned_addr learned my addr 172.21.15.38:0/3684191024 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:22.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.655+0000 7f54e60ba640 1 --2- 172.21.15.38:0/3684191024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8d5390 0x55fb5cc88c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54e50b8640 1 --2- 172.21.15.38:0/3684191024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc87d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54e58b9640 1 -- 172.21.15.38:0/3684191024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8b46b0 msgr2=0x55fb5cc87d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54e58b9640 1 --2- 172.21.15.38:0/3684191024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc87d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54e58b9640 1 -- 172.21.15.38:0/3684191024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8d5390 msgr2=0x55fb5cc88c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:22.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54e58b9640 1 --2- 172.21.15.38:0/3684191024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8d5390 0x55fb5cc88c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54e58b9640 1 -- 172.21.15.38:0/3684191024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5cc891d0 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54e50b8640 1 --2- 172.21.15.38:0/3684191024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc87d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:36:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54e58b9640 1 --2- 172.21.15.38:0/3684191024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc87810 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55fb5cc6be60 tx=0x7f54dc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caeb72e2a3dc7d9b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54c97fa640 1 -- 172.21.15.38:0/3684191024 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc005670 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54c97fa640 1 -- 172.21.15.38:0/3684191024 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f54dc00f790 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54ee9bd740 1 -- 172.21.15.38:0/3684191024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54d00d2d20 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54c97fa640 1 -- 172.21.15.38:0/3684191024 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54dc00f9f0 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54ee9bd740 10 client.16092 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.656+0000 7f54ee9bd740 1 -- 172.21.15.38:0/3684191024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fb5cc8a500 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.657+0000 7f54c97fa640 1 -- 172.21.15.38:0/3684191024 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d00d2d20 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.657+0000 7f54c97fa640 1 -- 172.21.15.38:0/3684191024 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f54dc038030 con 0x55fb5c8ac4d0 2024-05-21T15:36:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.657+0000 7f54ee9bd740 10 client.16092 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:22.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.657+0000 7f54ee9bd740 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.657+0000 7f54ee9bd740 20 client.16092 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.657+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/3684191024 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fb5c8693a0 0x55fb5ccfb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.657+0000 7f54ee9bd740 4 client.16092 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:36:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.657+0000 7f54ee9bd740 1 -- 172.21.15.38:0/3684191024 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fb5cbe1780 con 0x55fb5c8693a0 2024-05-21T15:36:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.657+0000 7f54e50b8640 1 --2- 172.21.15.38:0/3684191024 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fb5c8693a0 0x55fb5ccfb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.657+0000 7f54c97fa640 10 client.16092 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.657+0000 7f54e50b8640 1 --2- 172.21.15.38:0/3684191024 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fb5c8693a0 0x55fb5ccfb770 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cec24f5532e0770 server_cookie=e84b80b548b393bb in_seq=0 out_seq=0 2024-05-21T15:36:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54c97fa640 1 -- 172.21.15.38:0/3684191024 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fb5cbe1780 con 0x55fb5c8693a0 2024-05-21T15:36:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54c97fa640 10 client.16092 handle_command_reply: tid=0 2024-05-21T15:36:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54ee9bd740 1 client.16092 shutdown 2024-05-21T15:36:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54ee9bd740 1 -- 172.21.15.38:0/3684191024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8ac4d0 msgr2=0x55fb5cc87810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/3684191024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc87810 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55fb5cc6be60 tx=0x7f54dc005270 comp rx=0 tx=0).stop 2024-05-21T15:36:22.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54ee9bd740 1 -- 172.21.15.38:0/3684191024 shutdown_connections 2024-05-21T15:36:22.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/3684191024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb5c8d5390 0x55fb5cc88c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/3684191024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fb5c8b46b0 0x55fb5cc87d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/3684191024 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fb5c8693a0 0x55fb5ccfb770 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54ee9bd740 1 --2- 172.21.15.38:0/3684191024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fb5c8ac4d0 0x55fb5cc87810 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54c97fa640 0 client.16092 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:22.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54ee9bd740 1 -- 172.21.15.38:0/3684191024 >> 172.21.15.38:0/3684191024 conn(0x55fb5c8c94e0 msgr2=0x55fb5cc83470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:22.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54ee9bd740 1 -- 172.21.15.38:0/3684191024 shutdown_connections 2024-05-21T15:36:22.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54ee9bd740 1 -- 172.21.15.38:0/3684191024 wait complete. 2024-05-21T15:36:22.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.658+0000 7f54ee9bd740 20 client.16092 trim_cache size 0 max 16384 2024-05-21T15:36:22.663 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:36:22.663 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:36:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-05-21T15:36:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: "b5c5d903-ff57-4f8b-9cc0-eb595688726e": { 2024-05-21T15:36:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-05-21T15:36:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "b5c5d903-ff57-4f8b-9cc0-eb595688726e", 2024-05-21T15:36:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-05-21T15:36:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:36:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:36:22.664 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:36:22.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.659+0000 7f54e7b1d640 1 -- 172.21.15.38:0/545582213 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f54cc0a2b90 msgr2=0x7f54cc0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.659+0000 7f54e7b1d640 1 --2- 172.21.15.38:0/545582213 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f54cc0a2b90 0x7f54cc0a50a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55fb5cc7a190 tx=0x7f54d803d040 comp rx=0 tx=0).stop 2024-05-21T15:36:22.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.659+0000 7f54e7b1d640 1 -- 172.21.15.38:0/545582213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54e0102bf0 msgr2=0x7f54e019dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:22.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.659+0000 7f54e7b1d640 1 --2- 172.21.15.38:0/545582213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54e0102bf0 0x7f54e019dfc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f54d000e9f0 tx=0x7f54d000edf0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.659+0000 7f54e7b1d640 1 -- 172.21.15.38:0/545582213 shutdown_connections 2024-05-21T15:36:22.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.659+0000 7f54e7b1d640 1 --2- 172.21.15.38:0/545582213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e01078f0 0x7f54e01a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.659+0000 7f54e7b1d640 1 --2- 172.21.15.38:0/545582213 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f54cc0a2b90 0x7f54cc0a50a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.659+0000 7f54e7b1d640 1 --2- 172.21.15.38:0/545582213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54e0102bf0 0x7f54e019dfc0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.659+0000 7f54e7b1d640 1 --2- 172.21.15.38:0/545582213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e01019b0 0x7f54e00747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.659+0000 7f54e7b1d640 1 -- 172.21.15.38:0/545582213 >> 172.21.15.38:0/545582213 conn(0x7f54e00fcbe0 msgr2=0x7f54e01038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.659+0000 7f54e7b1d640 1 -- 172.21.15.38:0/545582213 shutdown_connections 2024-05-21T15:36:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:22.659+0000 7f54e7b1d640 1 -- 172.21.15.38:0/545582213 wait complete. 2024-05-21T15:36:22.669 INFO:tasks.cephfs.filesystem:scrub status for tag:b5c5d903-ff57-4f8b-9cc0-eb595688726e - {'path': '/', 'tag': 'b5c5d903-ff57-4f8b-9cc0-eb595688726e', 'options': 'recursive,force'} 2024-05-21T15:36:22.728 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-05-21T15:36:22.733 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-05-21T15:36:22.783 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-05-21T15:36:22.786 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-05-21T15:36:22.829 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-05-21T15:36:22.832 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-05-21T15:36:22.879 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-05-21T15:36:22.882 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-05-21T15:36:23.104 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-05-21T15:36:23.107 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-05-21T15:36:23.238 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-05-21T15:36:23.241 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-05-21T15:36:23.392 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-05-21T15:36:23.397 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-05-21T15:36:23.469 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-05-21T15:36:23.474 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-05-21T15:36:23.548 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-05-21T15:36:23.553 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-05-21T15:36:23.659 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-05-21T15:36:23.665 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-05-21T15:36:23.693 INFO:tasks.workunit.client.0.smithi038.stderr:In file included from parser.h:22, 2024-05-21T15:36:23.693 INFO:tasks.workunit.client.0.smithi038.stderr: from parser.c:25: 2024-05-21T15:36:23.693 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c: In function ‘init_filesys’: 2024-05-21T15:36:23.693 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:36:23.693 INFO:tasks.workunit.client.0.smithi038.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:36:23.693 INFO:tasks.workunit.client.0.smithi038.stderr: | ^ 2024-05-21T15:36:23.693 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:36:23.693 INFO:tasks.workunit.client.0.smithi038.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:36:23.693 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~ 2024-05-21T15:36:23.693 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:36:23.694 INFO:tasks.workunit.client.0.smithi038.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:36:23.694 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:36:23.694 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:36:23.694 INFO:tasks.workunit.client.0.smithi038.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-21T15:36:23.694 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~ 2024-05-21T15:36:23.694 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:36:23.694 INFO:tasks.workunit.client.0.smithi038.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-05-21T15:36:23.694 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:36:23.694 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:36:23.694 INFO:tasks.workunit.client.0.smithi038.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:36:23.694 INFO:tasks.workunit.client.0.smithi038.stderr: | ^ 2024-05-21T15:36:23.694 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:36:23.695 INFO:tasks.workunit.client.0.smithi038.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:36:23.695 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~ 2024-05-21T15:36:23.695 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:36:23.695 INFO:tasks.workunit.client.0.smithi038.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:36:23.695 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:36:23.695 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:36:23.695 INFO:tasks.workunit.client.0.smithi038.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-21T15:36:23.695 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~ 2024-05-21T15:36:23.695 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:36:23.695 INFO:tasks.workunit.client.0.smithi038.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-05-21T15:36:23.695 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:36:23.695 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:36:23.695 INFO:tasks.workunit.client.0.smithi038.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:36:23.696 INFO:tasks.workunit.client.0.smithi038.stderr: | ^ 2024-05-21T15:36:23.696 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:36:23.696 INFO:tasks.workunit.client.0.smithi038.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:36:23.696 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~ 2024-05-21T15:36:23.696 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:36:23.696 INFO:tasks.workunit.client.0.smithi038.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:36:23.696 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:36:23.696 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:36:23.696 INFO:tasks.workunit.client.0.smithi038.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-21T15:36:23.696 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~ 2024-05-21T15:36:23.696 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:36:23.696 INFO:tasks.workunit.client.0.smithi038.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-05-21T15:36:23.696 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:36:23.697 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:36:23.697 INFO:tasks.workunit.client.0.smithi038.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:36:23.697 INFO:tasks.workunit.client.0.smithi038.stderr: | ^ 2024-05-21T15:36:23.697 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:36:23.697 INFO:tasks.workunit.client.0.smithi038.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:36:23.697 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~ 2024-05-21T15:36:23.697 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:36:23.697 INFO:tasks.workunit.client.0.smithi038.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:36:23.697 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:36:23.697 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:36:23.697 INFO:tasks.workunit.client.0.smithi038.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-21T15:36:23.697 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~ 2024-05-21T15:36:23.697 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:36:23.697 INFO:tasks.workunit.client.0.smithi038.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-05-21T15:36:23.698 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:36:23.698 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c: In function ‘init_tg_stats’: 2024-05-21T15:36:23.698 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:36:23.698 INFO:tasks.workunit.client.0.smithi038.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:36:23.698 INFO:tasks.workunit.client.0.smithi038.stderr: | ^ 2024-05-21T15:36:23.698 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:36:23.698 INFO:tasks.workunit.client.0.smithi038.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:36:23.698 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~ 2024-05-21T15:36:23.698 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:36:23.698 INFO:tasks.workunit.client.0.smithi038.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:36:23.698 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:36:23.698 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:36:23.699 INFO:tasks.workunit.client.0.smithi038.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-21T15:36:23.699 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~ 2024-05-21T15:36:23.699 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:36:23.699 INFO:tasks.workunit.client.0.smithi038.stderr: 971 | list_for_each_entry(tmp_list, 2024-05-21T15:36:23.699 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:36:23.699 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:36:23.699 INFO:tasks.workunit.client.0.smithi038.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:36:23.699 INFO:tasks.workunit.client.0.smithi038.stderr: | ^ 2024-05-21T15:36:23.699 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:36:23.699 INFO:tasks.workunit.client.0.smithi038.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:36:23.699 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~ 2024-05-21T15:36:23.699 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:36:23.699 INFO:tasks.workunit.client.0.smithi038.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:36:23.700 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:36:23.700 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:36:23.700 INFO:tasks.workunit.client.0.smithi038.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-21T15:36:23.700 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~ 2024-05-21T15:36:23.700 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:36:23.700 INFO:tasks.workunit.client.0.smithi038.stderr: 971 | list_for_each_entry(tmp_list, 2024-05-21T15:36:23.700 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:36:23.700 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:36:23.700 INFO:tasks.workunit.client.0.smithi038.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:36:23.700 INFO:tasks.workunit.client.0.smithi038.stderr: | ^ 2024-05-21T15:36:23.700 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:36:23.700 INFO:tasks.workunit.client.0.smithi038.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:36:23.700 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~ 2024-05-21T15:36:23.701 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:36:23.701 INFO:tasks.workunit.client.0.smithi038.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:36:23.701 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:36:23.701 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:36:23.701 INFO:tasks.workunit.client.0.smithi038.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-21T15:36:23.701 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~ 2024-05-21T15:36:23.701 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:36:23.701 INFO:tasks.workunit.client.0.smithi038.stderr: 980 | list_for_each_entry(tmp_list, 2024-05-21T15:36:23.701 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:36:23.701 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:36:23.701 INFO:tasks.workunit.client.0.smithi038.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:36:23.701 INFO:tasks.workunit.client.0.smithi038.stderr: | ^ 2024-05-21T15:36:23.701 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:36:23.701 INFO:tasks.workunit.client.0.smithi038.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:36:23.702 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~ 2024-05-21T15:36:23.702 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:36:23.702 INFO:tasks.workunit.client.0.smithi038.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:36:23.702 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:36:23.702 INFO:tasks.workunit.client.0.smithi038.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:36:23.702 INFO:tasks.workunit.client.0.smithi038.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-21T15:36:23.702 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~ 2024-05-21T15:36:23.702 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:36:23.702 INFO:tasks.workunit.client.0.smithi038.stderr: 980 | list_for_each_entry(tmp_list, 2024-05-21T15:36:23.702 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:36:23.739 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c: In function ‘get_optstr’: 2024-05-21T15:36:23.740 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-05-21T15:36:23.740 INFO:tasks.workunit.client.0.smithi038.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-05-21T15:36:23.740 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:36:23.740 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c:200:14: note: source object declared here 2024-05-21T15:36:23.740 INFO:tasks.workunit.client.0.smithi038.stderr: 200 | char temp[BUFSIZE]; 2024-05-21T15:36:23.740 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~ 2024-05-21T15:36:23.746 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c: In function ‘size64_convert’: 2024-05-21T15:36:23.746 INFO:tasks.workunit.client.0.smithi038.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-05-21T15:36:23.746 INFO:tasks.workunit.client.0.smithi038.stderr: 119 | memcpy(unit, "\0", 3); 2024-05-21T15:36:23.746 INFO:tasks.workunit.client.0.smithi038.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-05-21T15:36:24.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:23 smithi045 ceph-mon[36409]: pgmap v410: 97 pgs: 97 active+clean; 5.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 463 KiB/s wr, 130 op/s 2024-05-21T15:36:24.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:23 smithi045 ceph-mon[36409]: scrub queued for path: / 2024-05-21T15:36:24.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:23 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:36:24.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:23 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:36:24.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:23 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:36:24.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:23 smithi038 ceph-mon[31995]: pgmap v410: 97 pgs: 97 active+clean; 5.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 463 KiB/s wr, 130 op/s 2024-05-21T15:36:24.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:23 smithi038 ceph-mon[31995]: scrub queued for path: / 2024-05-21T15:36:24.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:23 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:36:24.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:23 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:36:24.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:23 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:36:24.046 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-05-21T15:36:24.052 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-05-21T15:36:24.114 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-05-21T15:36:24.121 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-05-21T15:36:24.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:23 smithi053 ceph-mon[36496]: pgmap v410: 97 pgs: 97 active+clean; 5.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 463 KiB/s wr, 130 op/s 2024-05-21T15:36:24.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:23 smithi053 ceph-mon[36496]: scrub queued for path: / 2024-05-21T15:36:24.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:23 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:36:24.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:23 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:36:24.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:23 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:36:24.213 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-05-21T15:36:24.216 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-05-21T15:36:24.247 INFO:tasks.workunit.client.0.smithi038.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-05-21T15:36:24.252 INFO:tasks.workunit.client.0.smithi038.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-05-21T15:36:24.287 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-05-21T15:36:24.287 INFO:tasks.workunit.client.0.smithi038.stderr:+ cd .. 2024-05-21T15:36:24.287 INFO:tasks.workunit.client.0.smithi038.stderr:+ mkdir tmp 2024-05-21T15:36:24.289 INFO:tasks.workunit.client.0.smithi038.stderr:+ cd tmp 2024-05-21T15:36:24.289 INFO:tasks.workunit.client.0.smithi038.stderr:+ for f in $mydir/*.ffsb 2024-05-21T15:36:24.289 INFO:tasks.workunit.client.0.smithi038.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-05-21T15:36:24.289 INFO:tasks.workunit.client.0.smithi038.stdout:FFSB version 6.0-RC2 started 2024-05-21T15:36:24.289 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:36:24.293 INFO:tasks.workunit.client.0.smithi038.stdout:benchmark time = 300 2024-05-21T15:36:24.293 INFO:tasks.workunit.client.0.smithi038.stdout:ThreadGroup 0 2024-05-21T15:36:24.293 INFO:tasks.workunit.client.0.smithi038.stdout:================ 2024-05-21T15:36:24.293 INFO:tasks.workunit.client.0.smithi038.stdout: num_threads = 32 2024-05-21T15:36:24.293 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:36:24.293 INFO:tasks.workunit.client.0.smithi038.stdout: read_random = off 2024-05-21T15:36:24.293 INFO:tasks.workunit.client.0.smithi038.stdout: read_size = 0 (0B) 2024-05-21T15:36:24.293 INFO:tasks.workunit.client.0.smithi038.stdout: read_blocksize = 0 (0B) 2024-05-21T15:36:24.293 INFO:tasks.workunit.client.0.smithi038.stdout: read_skip = off 2024-05-21T15:36:24.294 INFO:tasks.workunit.client.0.smithi038.stdout: read_skipsize = 0 (0B) 2024-05-21T15:36:24.294 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:36:24.294 INFO:tasks.workunit.client.0.smithi038.stdout: write_random = on 2024-05-21T15:36:24.294 INFO:tasks.workunit.client.0.smithi038.stdout: write_size = 5242880 (5MB) 2024-05-21T15:36:24.294 INFO:tasks.workunit.client.0.smithi038.stdout: fsync_file = 0 2024-05-21T15:36:24.294 INFO:tasks.workunit.client.0.smithi038.stdout: write_blocksize = 4096 (4KB) 2024-05-21T15:36:24.294 INFO:tasks.workunit.client.0.smithi038.stdout: wait time = 0 2024-05-21T15:36:24.294 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:36:24.294 INFO:tasks.workunit.client.0.smithi038.stdout: op weights 2024-05-21T15:36:24.294 INFO:tasks.workunit.client.0.smithi038.stdout: read = 0 (0.00%) 2024-05-21T15:36:24.294 INFO:tasks.workunit.client.0.smithi038.stdout: readall = 0 (0.00%) 2024-05-21T15:36:24.294 INFO:tasks.workunit.client.0.smithi038.stdout: write = 1 (100.00%) 2024-05-21T15:36:24.295 INFO:tasks.workunit.client.0.smithi038.stdout: create = 0 (0.00%) 2024-05-21T15:36:24.295 INFO:tasks.workunit.client.0.smithi038.stdout: append = 0 (0.00%) 2024-05-21T15:36:24.295 INFO:tasks.workunit.client.0.smithi038.stdout: delete = 0 (0.00%) 2024-05-21T15:36:24.295 INFO:tasks.workunit.client.0.smithi038.stdout: metaop = 0 (0.00%) 2024-05-21T15:36:24.295 INFO:tasks.workunit.client.0.smithi038.stdout: createdir = 0 (0.00%) 2024-05-21T15:36:24.295 INFO:tasks.workunit.client.0.smithi038.stdout: stat = 0 (0.00%) 2024-05-21T15:36:24.295 INFO:tasks.workunit.client.0.smithi038.stdout: writeall = 0 (0.00%) 2024-05-21T15:36:24.295 INFO:tasks.workunit.client.0.smithi038.stdout: writeall_fsync = 0 (0.00%) 2024-05-21T15:36:24.295 INFO:tasks.workunit.client.0.smithi038.stdout: open_close = 0 (0.00%) 2024-05-21T15:36:24.295 INFO:tasks.workunit.client.0.smithi038.stdout: write_fsync = 0 (0.00%) 2024-05-21T15:36:24.295 INFO:tasks.workunit.client.0.smithi038.stdout: create_fsync = 0 (0.00%) 2024-05-21T15:36:24.295 INFO:tasks.workunit.client.0.smithi038.stdout: append_fsync = 0 (0.00%) 2024-05-21T15:36:24.295 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:36:24.296 INFO:tasks.workunit.client.0.smithi038.stdout:FileSystem . 2024-05-21T15:36:24.296 INFO:tasks.workunit.client.0.smithi038.stdout:========== 2024-05-21T15:36:24.296 INFO:tasks.workunit.client.0.smithi038.stdout: num_dirs = 0 2024-05-21T15:36:24.296 INFO:tasks.workunit.client.0.smithi038.stdout: starting files = 128 2024-05-21T15:36:24.296 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:36:24.296 INFO:tasks.workunit.client.0.smithi038.stdout: min file size = 104857600 (100MB) 2024-05-21T15:36:24.296 INFO:tasks.workunit.client.0.smithi038.stdout: max file size = 104857600 (100MB) 2024-05-21T15:36:24.296 INFO:tasks.workunit.client.0.smithi038.stdout: directio = off 2024-05-21T15:36:24.296 INFO:tasks.workunit.client.0.smithi038.stdout: alignedio = on 2024-05-21T15:36:24.296 INFO:tasks.workunit.client.0.smithi038.stdout: bufferedio = off 2024-05-21T15:36:24.296 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:36:24.296 INFO:tasks.workunit.client.0.smithi038.stdout: aging is off 2024-05-21T15:36:24.296 INFO:tasks.workunit.client.0.smithi038.stdout: current utilization = 0.00% 2024-05-21T15:36:24.297 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:36:24.297 INFO:tasks.workunit.client.0.smithi038.stdout:checking existing fs: . 2024-05-21T15:36:24.297 INFO:tasks.workunit.client.0.smithi038.stdout:recreating new fileset 2024-05-21T15:36:24.297 INFO:tasks.workunit.client.0.smithi038.stderr:opendir: No such file or directory 2024-05-21T15:36:25.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:24 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:36:25.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:24 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:36:25.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:24 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:36:26.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:25 smithi053 ceph-mon[36496]: pgmap v411: 97 pgs: 97 active+clean; 5.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 542 KiB/s wr, 145 op/s 2024-05-21T15:36:26.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:25 smithi045 ceph-mon[36409]: pgmap v411: 97 pgs: 97 active+clean; 5.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 542 KiB/s wr, 145 op/s 2024-05-21T15:36:26.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:25 smithi038 ceph-mon[31995]: pgmap v411: 97 pgs: 97 active+clean; 5.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 542 KiB/s wr, 145 op/s 2024-05-21T15:36:28.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:28 smithi038 ceph-mon[31995]: pgmap v412: 97 pgs: 97 active+clean; 5.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 475 KiB/s wr, 151 op/s 2024-05-21T15:36:28.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:28 smithi053 ceph-mon[36496]: pgmap v412: 97 pgs: 97 active+clean; 5.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 475 KiB/s wr, 151 op/s 2024-05-21T15:36:28.510 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:28 smithi045 ceph-mon[36409]: pgmap v412: 97 pgs: 97 active+clean; 5.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 475 KiB/s wr, 151 op/s 2024-05-21T15:36:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.146+0000 7fbf4d3ea640 1 -- 172.21.15.38:0/2459633560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf4810a300 msgr2=0x7fbf4810a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.146+0000 7fbf4d3ea640 1 --2- 172.21.15.38:0/2459633560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf4810a300 0x7fbf4810a710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009900 tx=0x7fbf3802a440 comp rx=0 tx=0).stop 2024-05-21T15:36:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.146+0000 7fbf4d3ea640 1 -- 172.21.15.38:0/2459633560 shutdown_connections 2024-05-21T15:36:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.146+0000 7fbf4d3ea640 1 --2- 172.21.15.38:0/2459633560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf4810f020 0x7fbf4810f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.146+0000 7fbf4d3ea640 1 --2- 172.21.15.38:0/2459633560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf4810a300 0x7fbf4810a710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.146+0000 7fbf4d3ea640 1 --2- 172.21.15.38:0/2459633560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf481093d0 0x7fbf48109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.147+0000 7fbf4d3ea640 1 -- 172.21.15.38:0/2459633560 >> 172.21.15.38:0/2459633560 conn(0x7fbf4806f4c0 msgr2=0x7fbf48071970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.147+0000 7fbf4d3ea640 1 -- 172.21.15.38:0/2459633560 shutdown_connections 2024-05-21T15:36:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.147+0000 7fbf4d3ea640 1 -- 172.21.15.38:0/2459633560 wait complete. 2024-05-21T15:36:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.148+0000 7fbf4d3ea640 1 Processor -- start 2024-05-21T15:36:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.148+0000 7fbf4d3ea640 1 -- start start 2024-05-21T15:36:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.148+0000 7fbf4d3ea640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf481093d0 0x7fbf481a8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.148+0000 7fbf4d3ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf4810a300 0x7fbf481a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.148+0000 7fbf4d3ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf4810f020 0x7fbf481af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.148+0000 7fbf4d3ea640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4811d670 con 0x7fbf4810a300 2024-05-21T15:36:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.148+0000 7fbf4d3ea640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4811d4f0 con 0x7fbf4810f020 2024-05-21T15:36:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.148+0000 7fbf4d3ea640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4811d7f0 con 0x7fbf481093d0 2024-05-21T15:36:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.149+0000 7fbf467fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf4810a300 0x7fbf481a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.149+0000 7fbf467fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf4810a300 0x7fbf481a86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60486/0 (socket says 172.21.15.38:60486) 2024-05-21T15:36:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.149+0000 7fbf467fc640 1 -- 172.21.15.38:0/2258905111 learned_addr learned my addr 172.21.15.38:0/2258905111 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.149+0000 7fbf46ffd640 1 --2- 172.21.15.38:0/2258905111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf481093d0 0x7fbf481a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.149+0000 7fbf477fe640 1 --2- 172.21.15.38:0/2258905111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf4810f020 0x7fbf481af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.149+0000 7fbf467fc640 1 -- 172.21.15.38:0/2258905111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf481093d0 msgr2=0x7fbf481a8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.149+0000 7fbf467fc640 1 --2- 172.21.15.38:0/2258905111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf481093d0 0x7fbf481a8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.149+0000 7fbf467fc640 1 -- 172.21.15.38:0/2258905111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf4810f020 msgr2=0x7fbf481af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.149+0000 7fbf467fc640 1 --2- 172.21.15.38:0/2258905111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf4810f020 0x7fbf481af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.149+0000 7fbf467fc640 1 -- 172.21.15.38:0/2258905111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf481afcc0 con 0x7fbf4810a300 2024-05-21T15:36:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.150+0000 7fbf467fc640 1 --2- 172.21.15.38:0/2258905111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf4810a300 0x7fbf481a86c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbf3802a960 tx=0x7fbf38032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.151+0000 7fbf2ffff640 1 -- 172.21.15.38:0/2258905111 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3803f030 con 0x7fbf4810a300 2024-05-21T15:36:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.151+0000 7fbf2ffff640 1 -- 172.21.15.38:0/2258905111 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf38003ce0 con 0x7fbf4810a300 2024-05-21T15:36:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.151+0000 7fbf2ffff640 1 -- 172.21.15.38:0/2258905111 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38035070 con 0x7fbf4810a300 2024-05-21T15:36:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.151+0000 7fbf4d3ea640 1 -- 172.21.15.38:0/2258905111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf481aff50 con 0x7fbf4810a300 2024-05-21T15:36:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.151+0000 7fbf4d3ea640 1 -- 172.21.15.38:0/2258905111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf481b0300 con 0x7fbf4810a300 2024-05-21T15:36:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.155+0000 7fbf54369740 1 Processor -- start 2024-05-21T15:36:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.155+0000 7fbf54369740 1 -- start start 2024-05-21T15:36:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.156+0000 7fbf54369740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f5e1b0 0x560d412fde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.156+0000 7fbf54369740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f352f0 0x560d412fe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.156+0000 7fbf54369740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f3d4d0 0x560d413052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.156+0000 7fbf54369740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560d41262340 con 0x560d40f352f0 2024-05-21T15:36:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.156+0000 7fbf54369740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560d4120a290 con 0x560d40f3d4d0 2024-05-21T15:36:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.156+0000 7fbf54369740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560d4125e300 con 0x560d40f5e1b0 2024-05-21T15:36:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.156+0000 7fbf2ffff640 1 -- 172.21.15.38:0/2258905111 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbf38042030 con 0x7fbf4810a300 2024-05-21T15:36:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.156+0000 7fbf46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f352f0 0x560d412fe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.156+0000 7fbf46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f352f0 0x560d412fe3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60500/0 (socket says 172.21.15.38:60500) 2024-05-21T15:36:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.156+0000 7fbf46ffd640 1 -- 172.21.15.38:0/3197141517 learned_addr learned my addr 172.21.15.38:0/3197141517 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.157+0000 7fbf467fc640 1 --2- 172.21.15.38:0/3197141517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f3d4d0 0x560d413052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.157+0000 7fbf477fe640 1 --2- 172.21.15.38:0/3197141517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f5e1b0 0x560d412fde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.157+0000 7fbf2ffff640 1 --2- 172.21.15.38:0/2258905111 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf300a2da0 0x7fbf300a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.157+0000 7fbf46ffd640 1 -- 172.21.15.38:0/3197141517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f5e1b0 msgr2=0x560d412fde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.158+0000 7fbf46ffd640 1 --2- 172.21.15.38:0/3197141517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f5e1b0 0x560d412fde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.158+0000 7fbf2ffff640 1 -- 172.21.15.38:0/2258905111 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbf380ef690 con 0x7fbf4810a300 2024-05-21T15:36:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.158+0000 7fbf46ffd640 1 -- 172.21.15.38:0/3197141517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f3d4d0 msgr2=0x560d413052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.158+0000 7fbf46ffd640 1 --2- 172.21.15.38:0/3197141517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f3d4d0 0x560d413052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.158+0000 7fbf46ffd640 1 -- 172.21.15.38:0/3197141517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d4122e8e0 con 0x560d40f352f0 2024-05-21T15:36:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.158+0000 7fbf467fc640 1 --2- 172.21.15.38:0/3197141517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f3d4d0 0x560d413052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:36:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.158+0000 7fbf477fe640 1 --2- 172.21.15.38:0/3197141517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f5e1b0 0x560d412fde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:36:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.158+0000 7fbf477fe640 1 --2- 172.21.15.38:0/2258905111 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf300a2da0 0x7fbf300a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.158+0000 7fbf477fe640 1 --2- 172.21.15.38:0/2258905111 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf300a2da0 0x7fbf300a52b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x560d412ff030 tx=0x7fbf4000c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.159+0000 7fbf46ffd640 1 --2- 172.21.15.38:0/3197141517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f352f0 0x560d412fe3e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009bf0 tx=0x7fbf3c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fb1e94bb3e22ce server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.159+0000 7fbf2d7fa640 1 -- 172.21.15.38:0/3197141517 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c004030 con 0x560d40f352f0 2024-05-21T15:36:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.159+0000 7fbf2d7fa640 1 -- 172.21.15.38:0/3197141517 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf3c0041d0 con 0x560d40f352f0 2024-05-21T15:36:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.159+0000 7fbf2d7fa640 1 -- 172.21.15.38:0/3197141517 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c0126b0 con 0x560d40f352f0 2024-05-21T15:36:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.162+0000 7fbf54369740 1 -- 172.21.15.38:0/3197141517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f352f0 msgr2=0x560d412fe3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.162+0000 7fbf54369740 1 --2- 172.21.15.38:0/3197141517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f352f0 0x560d412fe3e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009bf0 tx=0x7fbf3c0117b0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.163+0000 7fbf54369740 1 -- 172.21.15.38:0/3197141517 shutdown_connections 2024-05-21T15:36:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.163+0000 7fbf54369740 1 --2- 172.21.15.38:0/3197141517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f3d4d0 0x560d413052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.163+0000 7fbf54369740 1 --2- 172.21.15.38:0/3197141517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f352f0 0x560d412fe3e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.163+0000 7fbf54369740 1 --2- 172.21.15.38:0/3197141517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f5e1b0 0x560d412fde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.163+0000 7fbf54369740 1 -- 172.21.15.38:0/3197141517 >> 172.21.15.38:0/3197141517 conn(0x560d40f52300 msgr2=0x560d412f97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.163+0000 7fbf54369740 1 -- 172.21.15.38:0/3197141517 shutdown_connections 2024-05-21T15:36:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.163+0000 7fbf54369740 1 -- 172.21.15.38:0/3197141517 wait complete. 2024-05-21T15:36:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.164+0000 7fbf54369740 1 -- start start 2024-05-21T15:36:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.164+0000 7fbf54369740 1 Processor -- start 2024-05-21T15:36:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.165+0000 7fbf54369740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f352f0 0x560d41307b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.165+0000 7fbf54369740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f3d4d0 0x560d413080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.165+0000 7fbf54369740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f5e1b0 0x560d4130f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.165+0000 7fbf54369740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560d41305d30 con 0x560d40f352f0 2024-05-21T15:36:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.165+0000 7fbf54369740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560d4121d790 con 0x560d40f5e1b0 2024-05-21T15:36:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.165+0000 7fbf46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f352f0 0x560d41307b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.165+0000 7fbf54369740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560d41305eb0 con 0x560d40f3d4d0 2024-05-21T15:36:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.165+0000 7fbf46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f352f0 0x560d41307b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60516/0 (socket says 172.21.15.38:60516) 2024-05-21T15:36:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.165+0000 7fbf46ffd640 1 -- 172.21.15.38:0/1366515351 learned_addr learned my addr 172.21.15.38:0/1366515351 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.165+0000 7fbf54369740 10 client.0 _ll_register_callbacks cb 0x560d41258a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.165+0000 7fbf467fc640 1 --2- 172.21.15.38:0/1366515351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f3d4d0 0x560d413080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.165+0000 7fbf477fe640 1 --2- 172.21.15.38:0/1366515351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f5e1b0 0x560d4130f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.166+0000 7fbf46ffd640 1 -- 172.21.15.38:0/1366515351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f3d4d0 msgr2=0x560d413080a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.166+0000 7fbf46ffd640 1 --2- 172.21.15.38:0/1366515351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f3d4d0 0x560d413080a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.166+0000 7fbf46ffd640 1 -- 172.21.15.38:0/1366515351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f5e1b0 msgr2=0x560d4130f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.166+0000 7fbf46ffd640 1 --2- 172.21.15.38:0/1366515351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f5e1b0 0x560d4130f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.166+0000 7fbf46ffd640 1 -- 172.21.15.38:0/1366515351 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d40f76210 con 0x560d40f352f0 2024-05-21T15:36:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.166+0000 7fbf477fe640 1 --2- 172.21.15.38:0/1366515351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f5e1b0 0x560d4130f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:36:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.166+0000 7fbf46ffd640 1 --2- 172.21.15.38:0/1366515351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f352f0 0x560d41307b60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009ff0 tx=0x7fbf3c009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115bca76fd24009 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.166+0000 7fbf2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.167+0000 7fbf2aff5640 1 -- 172.21.15.38:0/1366515351 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c028020 con 0x560d40f352f0 2024-05-21T15:36:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.167+0000 7fbf2aff5640 1 -- 172.21.15.38:0/1366515351 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbf3c0037f0 con 0x560d40f352f0 2024-05-21T15:36:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.167+0000 7fbf54369740 1 -- 172.21.15.38:0/1366515351 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf3c0041d0 con 0x560d40f352f0 2024-05-21T15:36:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.167+0000 7fbf2aff5640 1 -- 172.21.15.38:0/1366515351 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf3c01f040 con 0x560d40f352f0 2024-05-21T15:36:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.167+0000 7fbf54369740 10 client.16104 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.167+0000 7fbf54369740 1 -- 172.21.15.38:0/1366515351 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560d41042620 con 0x560d40f352f0 2024-05-21T15:36:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.168+0000 7fbf2aff5640 1 -- 172.21.15.38:0/1366515351 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf3c0041d0 con 0x560d40f352f0 2024-05-21T15:36:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.168+0000 7fbf2aff5640 1 -- 172.21.15.38:0/1366515351 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbf3c024410 con 0x560d40f352f0 2024-05-21T15:36:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.168+0000 7fbf54369740 10 client.16104 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.168+0000 7fbf54369740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.168+0000 7fbf54369740 20 client.16104 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.168+0000 7fbf54369740 1 --2- 172.21.15.38:0/1366515351 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560d40f1cd80 0x560d41315160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.168+0000 7fbf54369740 4 client.16104 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.168+0000 7fbf54369740 1 -- 172.21.15.38:0/1366515351 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d4126fa70 con 0x560d40f1cd80 2024-05-21T15:36:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.170+0000 7fbf467fc640 1 --2- 172.21.15.38:0/1366515351 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560d40f1cd80 0x560d41315160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.172+0000 7fbf2aff5640 10 client.16104 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.172+0000 7fbf467fc640 1 --2- 172.21.15.38:0/1366515351 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560d40f1cd80 0x560d41315160 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f3961d003c8d40 server_cookie=c754e23e8e52be85 in_seq=0 out_seq=0 2024-05-21T15:36:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.174+0000 7fbf2aff5640 1 -- 172.21.15.38:0/1366515351 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560d4126fa70 con 0x560d40f1cd80 2024-05-21T15:36:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.174+0000 7fbf2aff5640 10 client.16104 handle_command_reply: tid=0 2024-05-21T15:36:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.174+0000 7fbf54369740 1 client.16104 shutdown 2024-05-21T15:36:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.174+0000 7fbf54369740 1 -- 172.21.15.38:0/1366515351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f352f0 msgr2=0x560d41307b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.174+0000 7fbf54369740 1 --2- 172.21.15.38:0/1366515351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f352f0 0x560d41307b60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009ff0 tx=0x7fbf3c009890 comp rx=0 tx=0).stop 2024-05-21T15:36:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.174+0000 7fbf54369740 1 -- 172.21.15.38:0/1366515351 shutdown_connections 2024-05-21T15:36:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.174+0000 7fbf54369740 1 --2- 172.21.15.38:0/1366515351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f5e1b0 0x560d4130f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.174+0000 7fbf54369740 1 --2- 172.21.15.38:0/1366515351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f3d4d0 0x560d413080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.174+0000 7fbf54369740 1 --2- 172.21.15.38:0/1366515351 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560d40f1cd80 0x560d41315160 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.175+0000 7fbf2aff5640 0 client.16104 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.175+0000 7fbf54369740 1 --2- 172.21.15.38:0/1366515351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f352f0 0x560d41307b60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.175+0000 7fbf54369740 1 -- 172.21.15.38:0/1366515351 >> 172.21.15.38:0/1366515351 conn(0x560d40f52300 msgr2=0x560d412fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.175+0000 7fbf54369740 1 -- 172.21.15.38:0/1366515351 shutdown_connections 2024-05-21T15:36:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.175+0000 7fbf54369740 1 -- 172.21.15.38:0/1366515351 wait complete. 2024-05-21T15:36:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.175+0000 7fbf54369740 20 client.16104 trim_cache size 0 max 16384 2024-05-21T15:36:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.213+0000 7fbf54369740 1 Processor -- start 2024-05-21T15:36:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.214+0000 7fbf54369740 1 -- start start 2024-05-21T15:36:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.214+0000 7fbf54369740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f1cd80 0x560d412f9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.214+0000 7fbf54369740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f352f0 0x560d4130ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.214+0000 7fbf54369740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f3d4d0 0x560d4130f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.215+0000 7fbf54369740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf481b9250 con 0x560d40f1cd80 2024-05-21T15:36:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.215+0000 7fbf54369740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf481bad90 con 0x560d40f352f0 2024-05-21T15:36:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.215+0000 7fbf54369740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf481b9920 con 0x560d40f3d4d0 2024-05-21T15:36:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.215+0000 7fbf467fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f352f0 0x560d4130ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.217+0000 7fbf46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f1cd80 0x560d412f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.217+0000 7fbf46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f1cd80 0x560d412f9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60520/0 (socket says 172.21.15.38:60520) 2024-05-21T15:36:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.217+0000 7fbf46ffd640 1 -- 172.21.15.38:0/3313958093 learned_addr learned my addr 172.21.15.38:0/3313958093 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.217+0000 7fbf477fe640 1 --2- 172.21.15.38:0/3313958093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f3d4d0 0x560d4130f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.219+0000 7fbf46ffd640 1 -- 172.21.15.38:0/3313958093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f3d4d0 msgr2=0x560d4130f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.219+0000 7fbf46ffd640 1 --2- 172.21.15.38:0/3313958093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f3d4d0 0x560d4130f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.219+0000 7fbf46ffd640 1 -- 172.21.15.38:0/3313958093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f352f0 msgr2=0x560d4130ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.219+0000 7fbf46ffd640 1 --2- 172.21.15.38:0/3313958093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f352f0 0x560d4130ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.219+0000 7fbf46ffd640 1 -- 172.21.15.38:0/3313958093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d412f4a60 con 0x560d40f1cd80 2024-05-21T15:36:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.219+0000 7fbf46ffd640 1 --2- 172.21.15.38:0/3313958093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f1cd80 0x560d412f9960 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009690 tx=0x7fbf3c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9cbd5f9436df684 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.220+0000 7fbf2cff9640 1 -- 172.21.15.38:0/3313958093 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c028040 con 0x560d40f1cd80 2024-05-21T15:36:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.220+0000 7fbf2cff9640 1 -- 172.21.15.38:0/3313958093 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf3c024ca0 con 0x560d40f1cd80 2024-05-21T15:36:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.220+0000 7fbf2cff9640 1 -- 172.21.15.38:0/3313958093 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c012450 con 0x560d40f1cd80 2024-05-21T15:36:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.222+0000 7fbf54369740 1 -- 172.21.15.38:0/3313958093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f1cd80 msgr2=0x560d412f9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.222+0000 7fbf54369740 1 --2- 172.21.15.38:0/3313958093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f1cd80 0x560d412f9960 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009690 tx=0x7fbf3c005fd0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.222+0000 7fbf54369740 1 -- 172.21.15.38:0/3313958093 shutdown_connections 2024-05-21T15:36:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.222+0000 7fbf54369740 1 --2- 172.21.15.38:0/3313958093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f3d4d0 0x560d4130f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.222+0000 7fbf54369740 1 --2- 172.21.15.38:0/3313958093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f352f0 0x560d4130ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.222+0000 7fbf54369740 1 --2- 172.21.15.38:0/3313958093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f1cd80 0x560d412f9960 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.222+0000 7fbf54369740 1 -- 172.21.15.38:0/3313958093 >> 172.21.15.38:0/3313958093 conn(0x560d40f52300 msgr2=0x560d412f9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.222+0000 7fbf54369740 1 -- 172.21.15.38:0/3313958093 shutdown_connections 2024-05-21T15:36:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.222+0000 7fbf54369740 1 -- 172.21.15.38:0/3313958093 wait complete. 2024-05-21T15:36:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.223+0000 7fbf54369740 1 -- start start 2024-05-21T15:36:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.223+0000 7fbf54369740 1 Processor -- start 2024-05-21T15:36:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.223+0000 7fbf54369740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f1cd80 0x560d4130c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.223+0000 7fbf54369740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f352f0 0x560d4130cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.223+0000 7fbf54369740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f3d4d0 0x560d4137b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.223+0000 7fbf54369740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf481ba6c0 con 0x560d40f3d4d0 2024-05-21T15:36:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.223+0000 7fbf54369740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf481b9ff0 con 0x560d40f1cd80 2024-05-21T15:36:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.223+0000 7fbf54369740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf481b7230 con 0x560d40f352f0 2024-05-21T15:36:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.224+0000 7fbf54369740 10 client.0 _ll_register_callbacks cb 0x7fbf3c009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.224+0000 7fbf467fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f352f0 0x560d4130cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.224+0000 7fbf477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f3d4d0 0x560d4137b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.224+0000 7fbf477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f3d4d0 0x560d4137b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60532/0 (socket says 172.21.15.38:60532) 2024-05-21T15:36:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.224+0000 7fbf477fe640 1 -- 172.21.15.38:0/2503169567 learned_addr learned my addr 172.21.15.38:0/2503169567 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.224+0000 7fbf467fc640 1 --2- 172.21.15.38:0/2503169567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f352f0 0x560d4130cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:57066/0 (socket says 172.21.15.38:57066) 2024-05-21T15:36:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.224+0000 7fbf46ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f1cd80 0x560d4130c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.225+0000 7fbf46ffd640 1 -- 172.21.15.38:0/2503169567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f352f0 msgr2=0x560d4130cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.225+0000 7fbf46ffd640 1 --2- 172.21.15.38:0/2503169567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f352f0 0x560d4130cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.225+0000 7fbf46ffd640 1 -- 172.21.15.38:0/2503169567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f3d4d0 msgr2=0x560d4137b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.225+0000 7fbf46ffd640 1 --2- 172.21.15.38:0/2503169567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f3d4d0 0x560d4137b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.225+0000 7fbf46ffd640 1 -- 172.21.15.38:0/2503169567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d4138c9c0 con 0x560d40f1cd80 2024-05-21T15:36:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.226+0000 7fbf477fe640 1 --2- 172.21.15.38:0/2503169567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f3d4d0 0x560d4137b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.226+0000 7fbf46ffd640 1 --2- 172.21.15.38:0/2503169567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f1cd80 0x560d4130c6b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x560d413154a0 tx=0x7fbf3c024650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45212872f349d7c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.227+0000 7fbf2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.228+0000 7fbf2aff5640 1 -- 172.21.15.38:0/2503169567 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c041070 con 0x560d40f1cd80 2024-05-21T15:36:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.229+0000 7fbf2aff5640 1 -- 172.21.15.38:0/2503169567 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbf3c003800 con 0x560d40f1cd80 2024-05-21T15:36:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.229+0000 7fbf54369740 1 -- 172.21.15.38:0/2503169567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf3c024ca0 con 0x560d40f1cd80 2024-05-21T15:36:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.230+0000 7fbf2aff5640 1 -- 172.21.15.38:0/2503169567 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf3c04b050 con 0x560d40f1cd80 2024-05-21T15:36:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.231+0000 7fbf54369740 10 client.25775 fetch_fsmap learned FSMap version 27 2024-05-21T15:36:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.231+0000 7fbf54369740 1 -- 172.21.15.38:0/2503169567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560d4138db00 con 0x560d40f1cd80 2024-05-21T15:36:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.237+0000 7fbf2aff5640 1 -- 172.21.15.38:0/2503169567 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf3c024ca0 con 0x560d40f1cd80 2024-05-21T15:36:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.237+0000 7fbf2aff5640 1 -- 172.21.15.38:0/2503169567 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbf3c00f480 con 0x560d40f1cd80 2024-05-21T15:36:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.237+0000 7fbf54369740 10 client.25775 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:36:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.237+0000 7fbf54369740 10 client.25775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.237+0000 7fbf54369740 20 client.25775 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.237+0000 7fbf54369740 1 --2- 172.21.15.38:0/2503169567 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560d40f5e1b0 0x560d41390450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.237+0000 7fbf54369740 4 client.25775 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:36:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.237+0000 7fbf54369740 1 -- 172.21.15.38:0/2503169567 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d41269dc0 con 0x560d40f5e1b0 2024-05-21T15:36:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.244+0000 7fbf467fc640 1 --2- 172.21.15.38:0/2503169567 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560d40f5e1b0 0x560d41390450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.252+0000 7fbf467fc640 1 --2- 172.21.15.38:0/2503169567 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560d40f5e1b0 0x560d41390450 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b94427876df9055 server_cookie=63d7576f8dff0363 in_seq=0 out_seq=0 2024-05-21T15:36:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.252+0000 7fbf2aff5640 10 client.25775 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.313+0000 7fbf2aff5640 1 -- 172.21.15.38:0/2503169567 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d41269dc0 con 0x560d40f5e1b0 2024-05-21T15:36:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.313+0000 7fbf2aff5640 10 client.25775 handle_command_reply: tid=0 2024-05-21T15:36:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.313+0000 7fbf54369740 1 client.25775 shutdown 2024-05-21T15:36:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.313+0000 7fbf54369740 1 -- 172.21.15.38:0/2503169567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f1cd80 msgr2=0x560d4130c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:29.312 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.313+0000 7fbf54369740 1 --2- 172.21.15.38:0/2503169567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f1cd80 0x560d4130c6b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x560d413154a0 tx=0x7fbf3c024650 comp rx=0 tx=0).stop 2024-05-21T15:36:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.322+0000 7fbf54369740 1 -- 172.21.15.38:0/2503169567 shutdown_connections 2024-05-21T15:36:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.322+0000 7fbf54369740 1 --2- 172.21.15.38:0/2503169567 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560d40f5e1b0 0x560d41390450 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.322+0000 7fbf54369740 1 --2- 172.21.15.38:0/2503169567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560d40f3d4d0 0x560d4137b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.322+0000 7fbf54369740 1 --2- 172.21.15.38:0/2503169567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d40f352f0 0x560d4130cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.322+0000 7fbf54369740 1 --2- 172.21.15.38:0/2503169567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560d40f1cd80 0x560d4130c6b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.322+0000 7fbf2aff5640 0 client.25775 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.322+0000 7fbf54369740 1 -- 172.21.15.38:0/2503169567 >> 172.21.15.38:0/2503169567 conn(0x560d40f52300 msgr2=0x560d413076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.322+0000 7fbf54369740 1 -- 172.21.15.38:0/2503169567 shutdown_connections 2024-05-21T15:36:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.323+0000 7fbf54369740 1 -- 172.21.15.38:0/2503169567 wait complete. 2024-05-21T15:36:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.323+0000 7fbf54369740 20 client.25775 trim_cache size 0 max 16384 2024-05-21T15:36:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:36:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:36:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:36:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:36:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.323+0000 7fbf4d3ea640 1 -- 172.21.15.38:0/2258905111 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf300a2da0 msgr2=0x7fbf300a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.323+0000 7fbf4d3ea640 1 --2- 172.21.15.38:0/2258905111 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf300a2da0 0x7fbf300a52b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x560d412ff030 tx=0x7fbf4000c040 comp rx=0 tx=0).stop 2024-05-21T15:36:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.323+0000 7fbf4d3ea640 1 -- 172.21.15.38:0/2258905111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf4810a300 msgr2=0x7fbf481a86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.323+0000 7fbf4d3ea640 1 --2- 172.21.15.38:0/2258905111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf4810a300 0x7fbf481a86c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbf3802a960 tx=0x7fbf38032c90 comp rx=0 tx=0).stop 2024-05-21T15:36:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.323+0000 7fbf4d3ea640 1 -- 172.21.15.38:0/2258905111 shutdown_connections 2024-05-21T15:36:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.323+0000 7fbf4d3ea640 1 --2- 172.21.15.38:0/2258905111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf4810f020 0x7fbf481af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.323+0000 7fbf4d3ea640 1 --2- 172.21.15.38:0/2258905111 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf300a2da0 0x7fbf300a52b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.323+0000 7fbf4d3ea640 1 --2- 172.21.15.38:0/2258905111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf4810a300 0x7fbf481a86c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.323+0000 7fbf4d3ea640 1 --2- 172.21.15.38:0/2258905111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf481093d0 0x7fbf481a8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.323+0000 7fbf4d3ea640 1 -- 172.21.15.38:0/2258905111 >> 172.21.15.38:0/2258905111 conn(0x7fbf4806f4c0 msgr2=0x7fbf4810b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.324+0000 7fbf4d3ea640 1 -- 172.21.15.38:0/2258905111 shutdown_connections 2024-05-21T15:36:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:29.324+0000 7fbf4d3ea640 1 -- 172.21.15.38:0/2258905111 wait complete. 2024-05-21T15:36:29.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:29 smithi053 ceph-mon[36496]: pgmap v413: 97 pgs: 97 active+clean; 412 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 34 MiB/s wr, 210 op/s 2024-05-21T15:36:29.510 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:29 smithi045 ceph-mon[36409]: pgmap v413: 97 pgs: 97 active+clean; 412 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 34 MiB/s wr, 210 op/s 2024-05-21T15:36:29.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:29 smithi038 ceph-mon[31995]: pgmap v413: 97 pgs: 97 active+clean; 412 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 34 MiB/s wr, 210 op/s 2024-05-21T15:36:31.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:31 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:36:31.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:31 smithi053 ceph-mon[36496]: pgmap v414: 97 pgs: 97 active+clean; 993 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 83 MiB/s wr, 187 op/s 2024-05-21T15:36:31.510 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:31 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:36:31.510 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:31 smithi045 ceph-mon[36409]: pgmap v414: 97 pgs: 97 active+clean; 993 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 83 MiB/s wr, 187 op/s 2024-05-21T15:36:31.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:31 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:36:31.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:31 smithi038 ceph-mon[31995]: pgmap v414: 97 pgs: 97 active+clean; 993 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 83 MiB/s wr, 187 op/s 2024-05-21T15:36:33.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:33 smithi053 ceph-mon[36496]: pgmap v415: 97 pgs: 97 active+clean; 3.4 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 291 MiB/s wr, 287 op/s 2024-05-21T15:36:33.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:33 smithi045 ceph-mon[36409]: pgmap v415: 97 pgs: 97 active+clean; 3.4 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 291 MiB/s wr, 287 op/s 2024-05-21T15:36:33.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:33 smithi038 ceph-mon[31995]: pgmap v415: 97 pgs: 97 active+clean; 3.4 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 291 MiB/s wr, 287 op/s 2024-05-21T15:36:35.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:35 smithi053 ceph-mon[36496]: pgmap v416: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 390 MiB/s wr, 278 op/s 2024-05-21T15:36:35.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:35 smithi045 ceph-mon[36409]: pgmap v416: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 390 MiB/s wr, 278 op/s 2024-05-21T15:36:35.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:35 smithi038 ceph-mon[31995]: pgmap v416: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 390 MiB/s wr, 278 op/s 2024-05-21T15:36:37.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:37 smithi053 ceph-mon[36496]: pgmap v417: 97 pgs: 97 active+clean; 6.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 541 MiB/s wr, 331 op/s 2024-05-21T15:36:37.718 INFO:tasks.workunit.client.0.smithi038.stdout:fs setup took 13 secs 2024-05-21T15:36:37.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:37 smithi045 ceph-mon[36409]: pgmap v417: 97 pgs: 97 active+clean; 6.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 541 MiB/s wr, 331 op/s 2024-05-21T15:36:37.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:37 smithi038 ceph-mon[31995]: pgmap v417: 97 pgs: 97 active+clean; 6.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 541 MiB/s wr, 331 op/s 2024-05-21T15:36:39.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:39 smithi053 ceph-mon[36496]: pgmap v418: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 695 MiB/s wr, 371 op/s 2024-05-21T15:36:39.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:39 smithi045 ceph-mon[36409]: pgmap v418: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 695 MiB/s wr, 371 op/s 2024-05-21T15:36:39.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:39 smithi038 ceph-mon[31995]: pgmap v418: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 695 MiB/s wr, 371 op/s 2024-05-21T15:36:41.391 INFO:tasks.workunit.client.0.smithi038.stdout:Syncing()...3 sec 2024-05-21T15:36:41.392 INFO:tasks.workunit.client.0.smithi038.stdout:Starting Actual Benchmark At: Tue May 21 15:36:41 2024 2024-05-21T15:36:41.392 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:36:41.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:41 smithi053 ceph-mon[36496]: pgmap v419: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 737 MiB/s wr, 331 op/s 2024-05-21T15:36:41.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:41 smithi045 ceph-mon[36409]: pgmap v419: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 737 MiB/s wr, 331 op/s 2024-05-21T15:36:41.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:41 smithi038 ceph-mon[31995]: pgmap v419: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 737 MiB/s wr, 331 op/s 2024-05-21T15:36:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.523+0000 7f956959a640 1 -- 172.21.15.38:0/2853302363 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c0319e0 con 0x7f956c1093d0 2024-05-21T15:36:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.524+0000 7f95708e9640 1 -- 172.21.15.38:0/2853302363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c1093d0 msgr2=0x7f956c109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.524+0000 7f95708e9640 1 --2- 172.21.15.38:0/2853302363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c1093d0 0x7f956c109dc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f955c009900 tx=0x7f955c02a440 comp rx=0 tx=0).stop 2024-05-21T15:36:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.524+0000 7f95708e9640 1 -- 172.21.15.38:0/2853302363 shutdown_connections 2024-05-21T15:36:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.524+0000 7f95708e9640 1 --2- 172.21.15.38:0/2853302363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f956c10f020 0x7f956c10f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.524+0000 7f95708e9640 1 --2- 172.21.15.38:0/2853302363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f956c10a300 0x7f956c10a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.524+0000 7f95708e9640 1 --2- 172.21.15.38:0/2853302363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c1093d0 0x7f956c109dc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.524+0000 7f95708e9640 1 -- 172.21.15.38:0/2853302363 >> 172.21.15.38:0/2853302363 conn(0x7f956c06f4c0 msgr2=0x7f956c071970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.525+0000 7f95708e9640 1 -- 172.21.15.38:0/2853302363 shutdown_connections 2024-05-21T15:36:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.528+0000 7f95708e9640 1 -- 172.21.15.38:0/2853302363 wait complete. 2024-05-21T15:36:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.529+0000 7f95708e9640 1 Processor -- start 2024-05-21T15:36:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.529+0000 7f95708e9640 1 -- start start 2024-05-21T15:36:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.529+0000 7f95708e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f956c1093d0 0x7f956c118e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.529+0000 7f95708e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f956c10a300 0x7f956c119370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.529+0000 7f95708e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c10f020 0x7f956c11a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.529+0000 7f95708e9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f956c111040 con 0x7f956c1093d0 2024-05-21T15:36:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.529+0000 7f95708e9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f956c110ec0 con 0x7f956c10f020 2024-05-21T15:36:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.529+0000 7f95708e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f956c1111c0 con 0x7f956c10a300 2024-05-21T15:36:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.530+0000 7f956ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c10f020 0x7f956c11a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.530+0000 7f956ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c10f020 0x7f956c11a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:44048/0 (socket says 172.21.15.38:44048) 2024-05-21T15:36:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.530+0000 7f956ad9d640 1 -- 172.21.15.38:0/1724974865 learned_addr learned my addr 172.21.15.38:0/1724974865 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.530+0000 7f956ad9d640 1 -- 172.21.15.38:0/1724974865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f956c10a300 msgr2=0x7f956c119370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:36:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.530+0000 7f956ad9d640 1 --2- 172.21.15.38:0/1724974865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f956c10a300 0x7f956c119370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.530+0000 7f956ad9d640 1 -- 172.21.15.38:0/1724974865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f956c1093d0 msgr2=0x7f956c118e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:36:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.530+0000 7f956ad9d640 1 --2- 172.21.15.38:0/1724974865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f956c1093d0 0x7f956c118e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.530+0000 7f956ad9d640 1 -- 172.21.15.38:0/1724974865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956c11a940 con 0x7f956c10f020 2024-05-21T15:36:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.531+0000 7f956ad9d640 1 --2- 172.21.15.38:0/1724974865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c10f020 0x7f956c11a2d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f956400eb10 tx=0x7f956400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.532+0000 7f955b7fe640 1 -- 172.21.15.38:0/1724974865 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564004b20 con 0x7f956c10f020 2024-05-21T15:36:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.532+0000 7f955b7fe640 1 -- 172.21.15.38:0/1724974865 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9564004cc0 con 0x7f956c10f020 2024-05-21T15:36:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.532+0000 7f955b7fe640 1 -- 172.21.15.38:0/1724974865 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564010790 con 0x7f956c10f020 2024-05-21T15:36:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.533+0000 7f95708e9640 1 -- 172.21.15.38:0/1724974865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956c11ac30 con 0x7f956c10f020 2024-05-21T15:36:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.533+0000 7f95708e9640 1 -- 172.21.15.38:0/1724974865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956c11b130 con 0x7f956c10f020 2024-05-21T15:36:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.541+0000 7f955b7fe640 1 -- 172.21.15.38:0/1724974865 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9564005020 con 0x7f956c10f020 2024-05-21T15:36:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.543+0000 7f9577794740 1 Processor -- start 2024-05-21T15:36:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.543+0000 7f9577794740 1 -- start start 2024-05-21T15:36:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.544+0000 7f9577794740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4eb51b0 0x55bfa5254e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.544+0000 7f9577794740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e8c2f0 0x55bfa52553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.544+0000 7f9577794740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 0x55bfa525c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.544+0000 7f9577794740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bfa51b9340 con 0x55bfa4e944d0 2024-05-21T15:36:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.544+0000 7f9577794740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bfa5161290 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.544+0000 7f9577794740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bfa51b5300 con 0x55bfa4eb51b0 2024-05-21T15:36:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.544+0000 7f9569d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 0x55bfa525c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.544+0000 7f9569d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 0x55bfa525c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44868/0 (socket says 172.21.15.38:44868) 2024-05-21T15:36:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.544+0000 7f9569d9b640 1 -- 172.21.15.38:0/3337762822 learned_addr learned my addr 172.21.15.38:0/3337762822 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.544+0000 7f9569d9b640 1 --2- 172.21.15.38:0/3337762822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4eb51b0 0x55bfa5254e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.544+0000 7f955b7fe640 1 --2- 172.21.15.38:0/1724974865 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f95540a2e70 0x7f95540a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.545+0000 7f9569d9b640 1 -- 172.21.15.38:0/3337762822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e8c2f0 msgr2=0x55bfa52553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.545+0000 7f9569d9b640 1 --2- 172.21.15.38:0/3337762822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e8c2f0 0x55bfa52553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.545+0000 7f9569d9b640 1 -- 172.21.15.38:0/3337762822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 msgr2=0x55bfa525c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.545+0000 7f9569d9b640 1 --2- 172.21.15.38:0/3337762822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 0x55bfa525c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.545+0000 7f9569d9b640 1 -- 172.21.15.38:0/3337762822 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa51858e0 con 0x55bfa4eb51b0 2024-05-21T15:36:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.545+0000 7f9569d9b640 1 --2- 172.21.15.38:0/3337762822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4eb51b0 0x55bfa5254e60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9560035730 tx=0x7f9560005a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa3e68be45a9a2a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.546+0000 7f955b7fe640 1 -- 172.21.15.38:0/1724974865 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f95640cf6d0 con 0x7f956c10f020 2024-05-21T15:36:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.546+0000 7f956ad9d640 1 --2- 172.21.15.38:0/1724974865 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f95540a2e70 0x7f95540a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.546+0000 7f956ad9d640 1 --2- 172.21.15.38:0/1724974865 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f95540a2e70 0x7f95540a5380 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f956409be10 tx=0x7f9564009200 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.547+0000 7f9558ff9640 1 -- 172.21.15.38:0/3337762822 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560035d70 con 0x55bfa4eb51b0 2024-05-21T15:36:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.547+0000 7f9558ff9640 1 -- 172.21.15.38:0/3337762822 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9560030e20 con 0x55bfa4eb51b0 2024-05-21T15:36:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.547+0000 7f9558ff9640 1 -- 172.21.15.38:0/3337762822 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560033790 con 0x55bfa4eb51b0 2024-05-21T15:36:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.549+0000 7f9577794740 1 -- 172.21.15.38:0/3337762822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4eb51b0 msgr2=0x55bfa5254e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.549+0000 7f9577794740 1 --2- 172.21.15.38:0/3337762822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4eb51b0 0x55bfa5254e60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9560035730 tx=0x7f9560005a60 comp rx=0 tx=0).stop 2024-05-21T15:36:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.550+0000 7f9577794740 1 -- 172.21.15.38:0/3337762822 shutdown_connections 2024-05-21T15:36:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.550+0000 7f9577794740 1 --2- 172.21.15.38:0/3337762822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 0x55bfa525c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.550+0000 7f9577794740 1 --2- 172.21.15.38:0/3337762822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e8c2f0 0x55bfa52553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.550+0000 7f9577794740 1 --2- 172.21.15.38:0/3337762822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4eb51b0 0x55bfa5254e60 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9560035730 tx=0x7f9560005a60 comp rx=0 tx=0).stop 2024-05-21T15:36:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.550+0000 7f9577794740 1 -- 172.21.15.38:0/3337762822 >> 172.21.15.38:0/3337762822 conn(0x55bfa4ea9300 msgr2=0x55bfa52507f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.550+0000 7f9577794740 1 -- 172.21.15.38:0/3337762822 shutdown_connections 2024-05-21T15:36:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.550+0000 7f9577794740 1 -- 172.21.15.38:0/3337762822 wait complete. 2024-05-21T15:36:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.551+0000 7f9577794740 1 -- start start 2024-05-21T15:36:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.551+0000 7f9577794740 1 Processor -- start 2024-05-21T15:36:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.551+0000 7f9577794740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e8c2f0 0x55bfa525edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.551+0000 7f9577794740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e944d0 0x55bfa525f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.551+0000 7f9577794740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e73d80 0x55bfa5266290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.551+0000 7f9577794740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bfa525cd30 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.551+0000 7f9577794740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bfa5174790 con 0x55bfa4e944d0 2024-05-21T15:36:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.551+0000 7f9577794740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bfa525ceb0 con 0x55bfa4e73d80 2024-05-21T15:36:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.552+0000 7f9577794740 10 client.0 _ll_register_callbacks cb 0x55bfa51afa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.552+0000 7f9569d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e73d80 0x55bfa5266290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.552+0000 7f956a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e944d0 0x55bfa525f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.552+0000 7f9569d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e8c2f0 0x55bfa525edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.552+0000 7f9569d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e8c2f0 0x55bfa525edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44874/0 (socket says 172.21.15.38:44874) 2024-05-21T15:36:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.552+0000 7f9569d9b640 1 -- 172.21.15.38:0/2087053711 learned_addr learned my addr 172.21.15.38:0/2087053711 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.553+0000 7f9569d9b640 1 -- 172.21.15.38:0/2087053711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e73d80 msgr2=0x55bfa5266290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.553+0000 7f9569d9b640 1 --2- 172.21.15.38:0/2087053711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e73d80 0x55bfa5266290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.553+0000 7f9569d9b640 1 -- 172.21.15.38:0/2087053711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e944d0 msgr2=0x55bfa525f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.553+0000 7f9569d9b640 1 --2- 172.21.15.38:0/2087053711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e944d0 0x55bfa525f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.553+0000 7f9569d9b640 1 -- 172.21.15.38:0/2087053711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa4ecd210 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.555+0000 7f9569d9b640 1 --2- 172.21.15.38:0/2087053711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e8c2f0 0x55bfa525edf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9560038040 tx=0x7f9560000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4367819973409d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.555+0000 7f9551ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.555+0000 7f9551ffb640 1 -- 172.21.15.38:0/2087053711 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560004660 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.555+0000 7f9551ffb640 1 -- 172.21.15.38:0/2087053711 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9560035020 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.556+0000 7f9577794740 1 -- 172.21.15.38:0/2087053711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9560030e20 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.556+0000 7f9551ffb640 1 -- 172.21.15.38:0/2087053711 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9560035220 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.557+0000 7f9577794740 10 client.16119 fetch_fsmap learned FSMap version 28 2024-05-21T15:36:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.557+0000 7f9577794740 1 -- 172.21.15.38:0/2087053711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bfa4f99620 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.558+0000 7f9551ffb640 1 -- 172.21.15.38:0/2087053711 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9560030e20 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.558+0000 7f9551ffb640 1 -- 172.21.15.38:0/2087053711 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f9560030040 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.558+0000 7f9577794740 10 client.16119 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T15:36:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.558+0000 7f9577794740 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.558+0000 7f9577794740 20 client.16119 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.558+0000 7f9577794740 1 --2- 172.21.15.38:0/2087053711 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bfa4e491c0 0x55bfa526c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.558+0000 7f9577794740 4 client.16119 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.558+0000 7f9577794740 1 -- 172.21.15.38:0/2087053711 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfa51c6a70 con 0x55bfa4e491c0 2024-05-21T15:36:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.560+0000 7f956a59c640 1 --2- 172.21.15.38:0/2087053711 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bfa4e491c0 0x55bfa526c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.562+0000 7f956a59c640 1 --2- 172.21.15.38:0/2087053711 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bfa4e491c0 0x55bfa526c470 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e6d5c4b81d813a8 server_cookie=5e387de6a4c30105 in_seq=0 out_seq=0 2024-05-21T15:36:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.563+0000 7f9551ffb640 10 client.16119 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.567+0000 7f9551ffb640 1 -- 172.21.15.38:0/2087053711 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bfa51c6a70 con 0x55bfa4e491c0 2024-05-21T15:36:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.567+0000 7f9551ffb640 10 client.16119 handle_command_reply: tid=0 2024-05-21T15:36:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.567+0000 7f9577794740 1 client.16119 shutdown 2024-05-21T15:36:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.568+0000 7f9577794740 1 -- 172.21.15.38:0/2087053711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e8c2f0 msgr2=0x55bfa525edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.568+0000 7f9577794740 1 --2- 172.21.15.38:0/2087053711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e8c2f0 0x55bfa525edf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9560038040 tx=0x7f9560000bc0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.569+0000 7f9577794740 1 -- 172.21.15.38:0/2087053711 shutdown_connections 2024-05-21T15:36:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.569+0000 7f9577794740 1 --2- 172.21.15.38:0/2087053711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e73d80 0x55bfa5266290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.569+0000 7f9577794740 1 --2- 172.21.15.38:0/2087053711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e944d0 0x55bfa525f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.569+0000 7f9577794740 1 --2- 172.21.15.38:0/2087053711 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bfa4e491c0 0x55bfa526c470 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.569+0000 7f9577794740 1 --2- 172.21.15.38:0/2087053711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e8c2f0 0x55bfa525edf0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.569+0000 7f9577794740 1 -- 172.21.15.38:0/2087053711 >> 172.21.15.38:0/2087053711 conn(0x55bfa4ea9300 msgr2=0x55bfa524f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.571+0000 7f9551ffb640 0 client.16119 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.572+0000 7f9577794740 1 -- 172.21.15.38:0/2087053711 shutdown_connections 2024-05-21T15:36:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.573+0000 7f9577794740 1 -- 172.21.15.38:0/2087053711 wait complete. 2024-05-21T15:36:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.573+0000 7f9577794740 20 client.16119 trim_cache size 0 max 16384 2024-05-21T15:36:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.607+0000 7f9577794740 1 Processor -- start 2024-05-21T15:36:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.608+0000 7f9577794740 1 -- start start 2024-05-21T15:36:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.608+0000 7f9577794740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e73d80 0x55bfa52591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.608+0000 7f9577794740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e8c2f0 0x55bfa5259730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.608+0000 7f9577794740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 0x55bfa525d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.608+0000 7f9577794740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f956c07c9b0 con 0x55bfa4e944d0 2024-05-21T15:36:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.608+0000 7f9577794740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f956c07e520 con 0x55bfa4e73d80 2024-05-21T15:36:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.608+0000 7f9577794740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f956c07d080 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.610+0000 7f956a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e8c2f0 0x55bfa5259730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.610+0000 7f956a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e8c2f0 0x55bfa5259730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:54566/0 (socket says 172.21.15.38:54566) 2024-05-21T15:36:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.610+0000 7f956a59c640 1 -- 172.21.15.38:0/1012922503 learned_addr learned my addr 172.21.15.38:0/1012922503 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.611+0000 7f9569d9b640 1 --2- 172.21.15.38:0/1012922503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 0x55bfa525d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.611+0000 7f9569d9b640 1 --2- 172.21.15.38:0/1012922503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e73d80 0x55bfa52591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.612+0000 7f956a59c640 1 -- 172.21.15.38:0/1012922503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e73d80 msgr2=0x55bfa52591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.612+0000 7f956a59c640 1 --2- 172.21.15.38:0/1012922503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e73d80 0x55bfa52591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.612+0000 7f956a59c640 1 -- 172.21.15.38:0/1012922503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 msgr2=0x55bfa525d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.612+0000 7f956a59c640 1 --2- 172.21.15.38:0/1012922503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 0x55bfa525d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.612+0000 7f956a59c640 1 -- 172.21.15.38:0/1012922503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa5254a10 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.613+0000 7f956a59c640 1 --2- 172.21.15.38:0/1012922503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e8c2f0 0x55bfa5259730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f955c009850 tx=0x7f955c004380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f68ff2e9eeedc4b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.613+0000 7f9553fff640 1 -- 172.21.15.38:0/1012922503 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c04b070 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.613+0000 7f9553fff640 1 -- 172.21.15.38:0/1012922503 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f955c004470 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.615+0000 7f9577794740 1 -- 172.21.15.38:0/1012922503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e8c2f0 msgr2=0x55bfa5259730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.615+0000 7f9577794740 1 --2- 172.21.15.38:0/1012922503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e8c2f0 0x55bfa5259730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f955c009850 tx=0x7f955c004380 comp rx=0 tx=0).stop 2024-05-21T15:36:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.616+0000 7f9577794740 1 -- 172.21.15.38:0/1012922503 shutdown_connections 2024-05-21T15:36:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.616+0000 7f9577794740 1 --2- 172.21.15.38:0/1012922503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 0x55bfa525d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.616+0000 7f9577794740 1 --2- 172.21.15.38:0/1012922503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e8c2f0 0x55bfa5259730 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.616+0000 7f9577794740 1 --2- 172.21.15.38:0/1012922503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e73d80 0x55bfa52591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.616+0000 7f9577794740 1 -- 172.21.15.38:0/1012922503 >> 172.21.15.38:0/1012922503 conn(0x55bfa4ea9300 msgr2=0x55bfa5254de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.617+0000 7f9577794740 1 -- 172.21.15.38:0/1012922503 shutdown_connections 2024-05-21T15:36:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.617+0000 7f9577794740 1 -- 172.21.15.38:0/1012922503 wait complete. 2024-05-21T15:36:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.617+0000 7f9577794740 1 -- start start 2024-05-21T15:36:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.617+0000 7f9577794740 1 Processor -- start 2024-05-21T15:36:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.618+0000 7f9577794740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e73d80 0x55bfa5262cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.618+0000 7f9577794740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e8c2f0 0x55bfa5263210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.618+0000 7f9577794740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 0x55bfa5264140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.618+0000 7f9577794740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f956c07de20 con 0x55bfa4e944d0 2024-05-21T15:36:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.618+0000 7f9577794740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f956c07d750 con 0x55bfa4e73d80 2024-05-21T15:36:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.618+0000 7f9577794740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f956c07a9c0 con 0x55bfa4e8c2f0 2024-05-21T15:36:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.618+0000 7f9577794740 10 client.0 _ll_register_callbacks cb 0x7f95600307d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.618+0000 7f9569d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e73d80 0x55bfa5262cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.618+0000 7f9569d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e73d80 0x55bfa5262cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:44090/0 (socket says 172.21.15.38:44090) 2024-05-21T15:36:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.618+0000 7f9569d9b640 1 -- 172.21.15.38:0/1560584671 learned_addr learned my addr 172.21.15.38:0/1560584671 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.618+0000 7f956a59c640 1 --2- 172.21.15.38:0/1560584671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e8c2f0 0x55bfa5263210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.619+0000 7f9569d9b640 1 --2- 172.21.15.38:0/1560584671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 0x55bfa5264140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.619+0000 7f9569d9b640 1 -- 172.21.15.38:0/1560584671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e8c2f0 msgr2=0x55bfa5263210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.619+0000 7f9569d9b640 1 --2- 172.21.15.38:0/1560584671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e8c2f0 0x55bfa5263210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.619+0000 7f9569d9b640 1 -- 172.21.15.38:0/1560584671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 msgr2=0x55bfa5264140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.619+0000 7f9569d9b640 1 --2- 172.21.15.38:0/1560584671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 0x55bfa5264140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.619+0000 7f9569d9b640 1 -- 172.21.15.38:0/1560584671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa5264700 con 0x55bfa4e73d80 2024-05-21T15:36:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.619+0000 7f9569d9b640 1 --2- 172.21.15.38:0/1560584671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e73d80 0x55bfa5262cd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9560031010 tx=0x7f9560033f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b2dadda9b2c645f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.619+0000 7f9551ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.620+0000 7f9551ffb640 1 -- 172.21.15.38:0/1560584671 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560058040 con 0x55bfa4e73d80 2024-05-21T15:36:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.620+0000 7f9577794740 1 -- 172.21.15.38:0/1560584671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f955c004470 con 0x55bfa4e73d80 2024-05-21T15:36:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.620+0000 7f9551ffb640 1 -- 172.21.15.38:0/1560584671 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9560052b00 con 0x55bfa4e73d80 2024-05-21T15:36:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.621+0000 7f9551ffb640 1 -- 172.21.15.38:0/1560584671 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9560052d00 con 0x55bfa4e73d80 2024-05-21T15:36:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.621+0000 7f9577794740 10 client.25790 fetch_fsmap learned FSMap version 28 2024-05-21T15:36:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.621+0000 7f9577794740 1 -- 172.21.15.38:0/1560584671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bfa5265990 con 0x55bfa4e73d80 2024-05-21T15:36:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.623+0000 7f9551ffb640 1 -- 172.21.15.38:0/1560584671 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f955c004470 con 0x55bfa4e73d80 2024-05-21T15:36:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.624+0000 7f9551ffb640 1 -- 172.21.15.38:0/1560584671 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f9560004ae0 con 0x55bfa4e73d80 2024-05-21T15:36:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.624+0000 7f9577794740 10 client.25790 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T15:36:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.624+0000 7f9577794740 10 client.25790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.624+0000 7f9577794740 20 client.25790 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.624+0000 7f9577794740 1 --2- 172.21.15.38:0/1560584671 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bfa52d4a30 0x55bfa52d6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.624+0000 7f9577794740 4 client.25790 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:36:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.624+0000 7f9577794740 1 -- 172.21.15.38:0/1560584671 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bfa51c0dc0 con 0x55bfa52d4a30 2024-05-21T15:36:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.628+0000 7f956a59c640 1 --2- 172.21.15.38:0/1560584671 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bfa52d4a30 0x55bfa52d6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.631+0000 7f956a59c640 1 --2- 172.21.15.38:0/1560584671 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bfa52d4a30 0x55bfa52d6e50 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eac01bf229d8ace2 server_cookie=bb3bd8ea3f4f8088 in_seq=0 out_seq=0 2024-05-21T15:36:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:42.631+0000 7f9551ffb640 10 client.25790 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:43.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:43 smithi053 ceph-mon[36496]: pgmap v420: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 866 MiB/s wr, 380 op/s 2024-05-21T15:36:43.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:43 smithi045 ceph-mon[36409]: pgmap v420: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 866 MiB/s wr, 380 op/s 2024-05-21T15:36:43.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:43 smithi038 ceph-mon[31995]: pgmap v420: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 866 MiB/s wr, 380 op/s 2024-05-21T15:36:44.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:44 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:36:44.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:44 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:36:44.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:44 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:36:45.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:45 smithi053 ceph-mon[36496]: pgmap v421: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 742 MiB/s wr, 1.40k op/s 2024-05-21T15:36:45.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:45 smithi045 ceph-mon[36409]: pgmap v421: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 742 MiB/s wr, 1.40k op/s 2024-05-21T15:36:45.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:45 smithi038 ceph-mon[31995]: pgmap v421: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 742 MiB/s wr, 1.40k op/s 2024-05-21T15:36:47.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:47 smithi053 ceph-mon[36496]: pgmap v422: 97 pgs: 97 active+clean; 13 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 707 MiB/s wr, 5.47k op/s 2024-05-21T15:36:47.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:47 smithi045 ceph-mon[36409]: pgmap v422: 97 pgs: 97 active+clean; 13 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 707 MiB/s wr, 5.47k op/s 2024-05-21T15:36:47.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:47 smithi038 ceph-mon[31995]: pgmap v422: 97 pgs: 97 active+clean; 13 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 707 MiB/s wr, 5.47k op/s 2024-05-21T15:36:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9551ffb640 1 -- 172.21.15.38:0/1560584671 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bfa51c0dc0 con 0x55bfa52d4a30 2024-05-21T15:36:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9551ffb640 10 client.25790 handle_command_reply: tid=0 2024-05-21T15:36:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9577794740 1 client.25790 shutdown 2024-05-21T15:36:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9577794740 1 -- 172.21.15.38:0/1560584671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e73d80 msgr2=0x55bfa5262cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9577794740 1 --2- 172.21.15.38:0/1560584671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e73d80 0x55bfa5262cd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9560031010 tx=0x7f9560033f40 comp rx=0 tx=0).stop 2024-05-21T15:36:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9577794740 1 -- 172.21.15.38:0/1560584671 shutdown_connections 2024-05-21T15:36:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9577794740 1 --2- 172.21.15.38:0/1560584671 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bfa52d4a30 0x55bfa52d6e50 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9577794740 1 --2- 172.21.15.38:0/1560584671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bfa4e944d0 0x55bfa5264140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9577794740 1 --2- 172.21.15.38:0/1560584671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfa4e8c2f0 0x55bfa5263210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9577794740 1 --2- 172.21.15.38:0/1560584671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfa4e73d80 0x55bfa5262cd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9577794740 1 -- 172.21.15.38:0/1560584671 >> 172.21.15.38:0/1560584671 conn(0x55bfa4ea9300 msgr2=0x55bfa5254de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9551ffb640 0 client.25790 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9577794740 1 -- 172.21.15.38:0/1560584671 shutdown_connections 2024-05-21T15:36:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9577794740 1 -- 172.21.15.38:0/1560584671 wait complete. 2024-05-21T15:36:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.778+0000 7f9577794740 20 client.25790 trim_cache size 0 max 16384 2024-05-21T15:36:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:36:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:36:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:36:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:36:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.779+0000 7f95708e9640 1 -- 172.21.15.38:0/1724974865 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f95540a2e70 msgr2=0x7f95540a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.779+0000 7f95708e9640 1 --2- 172.21.15.38:0/1724974865 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f95540a2e70 0x7f95540a5380 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f956409be10 tx=0x7f9564009200 comp rx=0 tx=0).stop 2024-05-21T15:36:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.779+0000 7f95708e9640 1 -- 172.21.15.38:0/1724974865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c10f020 msgr2=0x7f956c11a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.779+0000 7f95708e9640 1 --2- 172.21.15.38:0/1724974865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c10f020 0x7f956c11a2d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f956400eb10 tx=0x7f956400c2d0 comp rx=0 tx=0).stop 2024-05-21T15:36:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.780+0000 7f95708e9640 1 -- 172.21.15.38:0/1724974865 shutdown_connections 2024-05-21T15:36:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.780+0000 7f95708e9640 1 --2- 172.21.15.38:0/1724974865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956c10f020 0x7f956c11a2d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.780+0000 7f95708e9640 1 --2- 172.21.15.38:0/1724974865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f956c10a300 0x7f956c119370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.780+0000 7f95708e9640 1 --2- 172.21.15.38:0/1724974865 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f95540a2e70 0x7f95540a5380 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.780+0000 7f95708e9640 1 --2- 172.21.15.38:0/1724974865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f956c1093d0 0x7f956c118e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.780+0000 7f95708e9640 1 -- 172.21.15.38:0/1724974865 >> 172.21.15.38:0/1724974865 conn(0x7f956c06f4c0 msgr2=0x7f956c10d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.780+0000 7f95708e9640 1 -- 172.21.15.38:0/1724974865 shutdown_connections 2024-05-21T15:36:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:36:47.780+0000 7f95708e9640 1 -- 172.21.15.38:0/1724974865 wait complete. 2024-05-21T15:36:49.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:49 smithi053 ceph-mon[36496]: pgmap v423: 97 pgs: 97 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 579 MiB/s wr, 9.05k op/s 2024-05-21T15:36:49.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:49 smithi045 ceph-mon[36409]: pgmap v423: 97 pgs: 97 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 579 MiB/s wr, 9.05k op/s 2024-05-21T15:36:49.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:49 smithi038 ceph-mon[31995]: pgmap v423: 97 pgs: 97 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 579 MiB/s wr, 9.05k op/s 2024-05-21T15:36:51.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:51 smithi053 ceph-mon[36496]: pgmap v424: 97 pgs: 97 active+clean; 13 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 462 MiB/s wr, 14.35k op/s 2024-05-21T15:36:51.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:51 smithi045 ceph-mon[36409]: pgmap v424: 97 pgs: 97 active+clean; 13 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 462 MiB/s wr, 14.35k op/s 2024-05-21T15:36:51.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:51 smithi038 ceph-mon[31995]: pgmap v424: 97 pgs: 97 active+clean; 13 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 462 MiB/s wr, 14.35k op/s 2024-05-21T15:36:52.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:36:52.671 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:36:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.966+0000 7f2c0fc84640 1 -- 172.21.15.38:0/3284732110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0810f410 msgr2=0x7f2c08111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.966+0000 7f2c0fc84640 1 --2- 172.21.15.38:0/3284732110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0810f410 0x7f2c08111830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009ca0 tx=0x7f2c0002a8f0 comp rx=0 tx=0).stop 2024-05-21T15:36:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.967+0000 7f2c0fc84640 1 -- 172.21.15.38:0/3284732110 shutdown_connections 2024-05-21T15:36:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.967+0000 7f2c0fc84640 1 --2- 172.21.15.38:0/3284732110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0810f410 0x7f2c08111830 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.967+0000 7f2c0fc84640 1 --2- 172.21.15.38:0/3284732110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0810a710 0x7f2c0810ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.967+0000 7f2c0fc84640 1 --2- 172.21.15.38:0/3284732110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c081094d0 0x7f2c0810a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.967+0000 7f2c0fc84640 1 -- 172.21.15.38:0/3284732110 >> 172.21.15.38:0/3284732110 conn(0x7f2c0806f4a0 msgr2=0x7f2c08071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:52.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.973+0000 7f2c0fc84640 1 -- 172.21.15.38:0/3284732110 shutdown_connections 2024-05-21T15:36:52.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.978+0000 7f2c0fc84640 1 -- 172.21.15.38:0/3284732110 wait complete. 2024-05-21T15:36:52.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.979+0000 7f2c0fc84640 1 Processor -- start 2024-05-21T15:36:52.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.979+0000 7f2c0fc84640 1 -- start start 2024-05-21T15:36:52.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.980+0000 7f2c0fc84640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c081094d0 0x7f2c0819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:52.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.980+0000 7f2c0fc84640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0810a710 0x7f2c0819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:52.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.980+0000 7f2c0fc84640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0810f410 0x7f2c081a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:52.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.980+0000 7f2c0fc84640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2c08114d60 con 0x7f2c0810a710 2024-05-21T15:36:52.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.981+0000 7f2c0fc84640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2c08114be0 con 0x7f2c081094d0 2024-05-21T15:36:52.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.981+0000 7f2c0e221640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0810f410 0x7f2c081a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:52.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.981+0000 7f2c0e221640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0810f410 0x7f2c081a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:52010/0 (socket says 172.21.15.38:52010) 2024-05-21T15:36:52.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.981+0000 7f2c0e221640 1 -- 172.21.15.38:0/4271909304 learned_addr learned my addr 172.21.15.38:0/4271909304 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:52.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.981+0000 7f2c0fc84640 1 -- 172.21.15.38:0/4271909304 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c08114ee0 con 0x7f2c0810f410 2024-05-21T15:36:52.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.984+0000 7f2c0da20640 1 --2- 172.21.15.38:0/4271909304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c081094d0 0x7f2c0819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.985+0000 7f2c0da20640 1 -- 172.21.15.38:0/4271909304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0810f410 msgr2=0x7f2c081a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.985+0000 7f2c0da20640 1 --2- 172.21.15.38:0/4271909304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0810f410 0x7f2c081a6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.985+0000 7f2c0da20640 1 -- 172.21.15.38:0/4271909304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0810a710 msgr2=0x7f2c0819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.985+0000 7f2c0da20640 1 --2- 172.21.15.38:0/4271909304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0810a710 0x7f2c0819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.985+0000 7f2c0da20640 1 -- 172.21.15.38:0/4271909304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c081a7390 con 0x7f2c081094d0 2024-05-21T15:36:52.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.990+0000 7f2c0da20640 1 --2- 172.21.15.38:0/4271909304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c081094d0 0x7f2c0819f8a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2c08108da0 tx=0x7f2c04007cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:52.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.991+0000 7f2bfeffd640 1 -- 172.21.15.38:0/4271909304 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04017030 con 0x7f2c081094d0 2024-05-21T15:36:52.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.991+0000 7f2bfeffd640 1 -- 172.21.15.38:0/4271909304 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c0400d6a0 con 0x7f2c081094d0 2024-05-21T15:36:52.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.991+0000 7f2bfeffd640 1 -- 172.21.15.38:0/4271909304 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04003920 con 0x7f2c081094d0 2024-05-21T15:36:52.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.994+0000 7f2c0fc84640 1 -- 172.21.15.38:0/4271909304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c081a7680 con 0x7f2c081094d0 2024-05-21T15:36:52.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:52.994+0000 7f2c0fc84640 1 -- 172.21.15.38:0/4271909304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c081a7bd0 con 0x7f2c081094d0 2024-05-21T15:36:52.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.000+0000 7f2bfeffd640 1 -- 172.21.15.38:0/4271909304 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2c0401a070 con 0x7f2c081094d0 2024-05-21T15:36:53.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.002+0000 7f2bfeffd640 1 --2- 172.21.15.38:0/4271909304 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2bf40a71b0 0x7f2bf40a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.002+0000 7f2bfeffd640 1 -- 172.21.15.38:0/4271909304 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2c040ce2f0 con 0x7f2c081094d0 2024-05-21T15:36:53.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.003+0000 7f2c16c0a740 1 Processor -- start 2024-05-21T15:36:53.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.004+0000 7f2c16c0a740 1 -- start start 2024-05-21T15:36:53.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.005+0000 7f2c0d21f640 1 --2- 172.21.15.38:0/4271909304 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2bf40a71b0 0x7f2bf40a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.005+0000 7f2c16c0a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108db390 0x55ac10c7abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.005+0000 7f2c16c0a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108b24d0 0x55ac10c7b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.005+0000 7f2c16c0a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108ba6b0 0x55ac10c82030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.005+0000 7f2c16c0a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ac10bdfd00 con 0x55ac108b24d0 2024-05-21T15:36:53.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.005+0000 7f2c16c0a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ac10b874c0 con 0x55ac108db390 2024-05-21T15:36:53.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.005+0000 7f2c16c0a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ac10bdbcc0 con 0x55ac108ba6b0 2024-05-21T15:36:53.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.007+0000 7f2c0d21f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108b24d0 0x55ac10c7b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.007+0000 7f2c0d21f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108b24d0 0x55ac10c7b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49320/0 (socket says 172.21.15.38:49320) 2024-05-21T15:36:53.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.007+0000 7f2c0d21f640 1 -- 172.21.15.38:0/3443136286 learned_addr learned my addr 172.21.15.38:0/3443136286 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:53.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.007+0000 7f2c0e221640 1 --2- 172.21.15.38:0/3443136286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108ba6b0 0x55ac10c82030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.007+0000 7f2c0d21f640 1 --2- 172.21.15.38:0/4271909304 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2bf40a71b0 0x7f2bf40a96c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2c081a08f0 tx=0x7f2bf8009b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:53.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.007+0000 7f2c0d21f640 1 -- 172.21.15.38:0/3443136286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108ba6b0 msgr2=0x55ac10c82030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:53.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.007+0000 7f2c0d21f640 1 --2- 172.21.15.38:0/3443136286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108ba6b0 0x55ac10c82030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.007+0000 7f2c0d21f640 1 -- 172.21.15.38:0/3443136286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108db390 msgr2=0x55ac10c7abb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:36:53.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.007+0000 7f2c0d21f640 1 --2- 172.21.15.38:0/3443136286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108db390 0x55ac10c7abb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.007+0000 7f2c0d21f640 1 -- 172.21.15.38:0/3443136286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac109bd7d0 con 0x55ac108b24d0 2024-05-21T15:36:53.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.008+0000 7f2c0d21f640 1 --2- 172.21.15.38:0/3443136286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108b24d0 0x55ac10c7b130 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800ba30 tx=0x7f2bf8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e2007aaf9578d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:53.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.008+0000 7f2bf3fff640 1 -- 172.21.15.38:0/3443136286 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8023070 con 0x55ac108b24d0 2024-05-21T15:36:53.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.009+0000 7f2bf3fff640 1 -- 172.21.15.38:0/3443136286 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf8016ce0 con 0x55ac108b24d0 2024-05-21T15:36:53.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.009+0000 7f2bf3fff640 1 -- 172.21.15.38:0/3443136286 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8023070 con 0x55ac108b24d0 2024-05-21T15:36:53.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.010+0000 7f2c16c0a740 1 -- 172.21.15.38:0/3443136286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108b24d0 msgr2=0x55ac10c7b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:53.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.010+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/3443136286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108b24d0 0x55ac10c7b130 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800ba30 tx=0x7f2bf8016000 comp rx=0 tx=0).stop 2024-05-21T15:36:53.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.010+0000 7f2c16c0a740 1 -- 172.21.15.38:0/3443136286 shutdown_connections 2024-05-21T15:36:53.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.010+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/3443136286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108ba6b0 0x55ac10c82030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.010+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/3443136286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108b24d0 0x55ac10c7b130 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.010+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/3443136286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108db390 0x55ac10c7abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.010+0000 7f2c16c0a740 1 -- 172.21.15.38:0/3443136286 >> 172.21.15.38:0/3443136286 conn(0x55ac108cf4e0 msgr2=0x55ac10c76540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:53.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.014+0000 7f2c16c0a740 1 -- 172.21.15.38:0/3443136286 shutdown_connections 2024-05-21T15:36:53.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.015+0000 7f2c16c0a740 1 -- 172.21.15.38:0/3443136286 wait complete. 2024-05-21T15:36:53.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.015+0000 7f2c16c0a740 1 -- start start 2024-05-21T15:36:53.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.015+0000 7f2c16c0a740 1 Processor -- start 2024-05-21T15:36:53.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.017+0000 7f2c16c0a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108b24d0 0x55ac10c8d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.017+0000 7f2c16c0a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108ba6b0 0x55ac10c8d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.017+0000 7f2c16c0a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108db390 0x55ac10c94810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.017+0000 7f2c16c0a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ac10c82a80 con 0x55ac108ba6b0 2024-05-21T15:36:53.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.017+0000 7f2c16c0a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ac10b9ab90 con 0x55ac108db390 2024-05-21T15:36:53.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.017+0000 7f2c16c0a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ac10c82c00 con 0x55ac108b24d0 2024-05-21T15:36:53.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.017+0000 7f2c16c0a740 10 client.0 _ll_register_callbacks cb 0x55ac10bd6da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:53.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.020+0000 7f2c0e221640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108db390 0x55ac10c94810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.020+0000 7f2c0e221640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108db390 0x55ac10c94810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46676/0 (socket says 172.21.15.38:46676) 2024-05-21T15:36:53.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.020+0000 7f2c0e221640 1 -- 172.21.15.38:0/2600190695 learned_addr learned my addr 172.21.15.38:0/2600190695 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:53.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.020+0000 7f2c0da20640 1 --2- 172.21.15.38:0/2600190695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108b24d0 0x55ac10c8d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.020+0000 7f2c0d21f640 1 --2- 172.21.15.38:0/2600190695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108ba6b0 0x55ac10c8d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.020+0000 7f2c0e221640 1 -- 172.21.15.38:0/2600190695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108b24d0 msgr2=0x55ac10c8d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:53.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.020+0000 7f2c0e221640 1 --2- 172.21.15.38:0/2600190695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108b24d0 0x55ac10c8d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.020+0000 7f2c0e221640 1 -- 172.21.15.38:0/2600190695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108ba6b0 msgr2=0x55ac10c8d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:53.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.020+0000 7f2c0e221640 1 --2- 172.21.15.38:0/2600190695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108ba6b0 0x55ac10c8d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.020+0000 7f2c0e221640 1 -- 172.21.15.38:0/2600190695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac108f33f0 con 0x55ac108db390 2024-05-21T15:36:53.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.021+0000 7f2c0d21f640 1 --2- 172.21.15.38:0/2600190695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108ba6b0 0x55ac10c8d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:36:53.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.021+0000 7f2c0e221640 1 --2- 172.21.15.38:0/2600190695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108db390 0x55ac10c94810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2c000318b0 tx=0x7f2c000318e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9003c854909d8531 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:53.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.021+0000 7f2bf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:53.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.023+0000 7f2bf17fa640 1 -- 172.21.15.38:0/2600190695 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00004040 con 0x55ac108db390 2024-05-21T15:36:53.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.023+0000 7f2c16c0a740 1 -- 172.21.15.38:0/2600190695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bf8016ce0 con 0x55ac108db390 2024-05-21T15:36:53.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.023+0000 7f2bf17fa640 1 -- 172.21.15.38:0/2600190695 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2c000041e0 con 0x55ac108db390 2024-05-21T15:36:53.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.025+0000 7f2bf17fa640 1 -- 172.21.15.38:0/2600190695 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c000043e0 con 0x55ac108db390 2024-05-21T15:36:53.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.025+0000 7f2c16c0a740 10 client.25799 fetch_fsmap learned FSMap version 29 2024-05-21T15:36:53.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.025+0000 7f2c16c0a740 1 -- 172.21.15.38:0/2600190695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac108b30f0 con 0x55ac108db390 2024-05-21T15:36:53.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.026+0000 7f2bf17fa640 1 -- 172.21.15.38:0/2600190695 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf8016ce0 con 0x55ac108db390 2024-05-21T15:36:53.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.027+0000 7f2bf17fa640 1 -- 172.21.15.38:0/2600190695 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2c000305f0 con 0x55ac108db390 2024-05-21T15:36:53.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.027+0000 7f2c16c0a740 10 client.25799 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:36:53.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.027+0000 7f2c16c0a740 10 client.25799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:53.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.027+0000 7f2c16c0a740 20 client.25799 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:53.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.027+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/2600190695 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ac10899f60 0x55ac10c89820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.027+0000 7f2c16c0a740 4 client.25799 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:53.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.027+0000 7f2c16c0a740 1 -- 172.21.15.38:0/2600190695 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac10bed430 con 0x55ac10899f60 2024-05-21T15:36:53.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.031+0000 7f2c0da20640 1 --2- 172.21.15.38:0/2600190695 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ac10899f60 0x55ac10c89820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.041+0000 7f2c0da20640 1 --2- 172.21.15.38:0/2600190695 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ac10899f60 0x55ac10c89820 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e6be3f400dd0df server_cookie=fb722f3c9d31b2b1 in_seq=0 out_seq=0 2024-05-21T15:36:53.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.042+0000 7f2bf17fa640 10 client.25799 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:53.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.049+0000 7f2bf17fa640 1 -- 172.21.15.38:0/2600190695 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac10bed430 con 0x55ac10899f60 2024-05-21T15:36:53.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.049+0000 7f2bf17fa640 10 client.25799 handle_command_reply: tid=0 2024-05-21T15:36:53.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.050+0000 7f2c16c0a740 1 client.25799 shutdown 2024-05-21T15:36:53.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.052+0000 7f2c16c0a740 1 -- 172.21.15.38:0/2600190695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108db390 msgr2=0x55ac10c94810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:53.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.052+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/2600190695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108db390 0x55ac10c94810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2c000318b0 tx=0x7f2c000318e0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.055+0000 7f2c16c0a740 1 -- 172.21.15.38:0/2600190695 shutdown_connections 2024-05-21T15:36:53.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.055+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/2600190695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108db390 0x55ac10c94810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.055+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/2600190695 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ac10899f60 0x55ac10c89820 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.055+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/2600190695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108ba6b0 0x55ac10c8d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.055+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/2600190695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108b24d0 0x55ac10c8d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.055+0000 7f2c16c0a740 1 -- 172.21.15.38:0/2600190695 >> 172.21.15.38:0/2600190695 conn(0x55ac108cf4e0 msgr2=0x55ac10c755c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.057+0000 7f2bf17fa640 0 client.25799 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.057+0000 7f2c16c0a740 1 -- 172.21.15.38:0/2600190695 shutdown_connections 2024-05-21T15:36:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.057+0000 7f2c16c0a740 1 -- 172.21.15.38:0/2600190695 wait complete. 2024-05-21T15:36:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.058+0000 7f2c16c0a740 20 client.25799 trim_cache size 0 max 16384 2024-05-21T15:36:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.107+0000 7f2c16c0a740 1 Processor -- start 2024-05-21T15:36:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.107+0000 7f2c16c0a740 1 -- start start 2024-05-21T15:36:53.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.108+0000 7f2c16c0a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108b24d0 0x55ac10c7f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.108+0000 7f2c16c0a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108ba6b0 0x55ac10c8cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.108+0000 7f2c16c0a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108db390 0x55ac10c94730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.108+0000 7f2c16c0a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081b0900 con 0x55ac108b24d0 2024-05-21T15:36:53.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.108+0000 7f2c16c0a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081b24a0 con 0x55ac108ba6b0 2024-05-21T15:36:53.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.108+0000 7f2c16c0a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081b0fd0 con 0x55ac108db390 2024-05-21T15:36:53.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.109+0000 7f2c0da20640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108b24d0 0x55ac10c7f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.109+0000 7f2c0da20640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108b24d0 0x55ac10c7f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49340/0 (socket says 172.21.15.38:49340) 2024-05-21T15:36:53.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.109+0000 7f2c0da20640 1 -- 172.21.15.38:0/1849492302 learned_addr learned my addr 172.21.15.38:0/1849492302 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:53.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.109+0000 7f2c0e221640 1 --2- 172.21.15.38:0/1849492302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108db390 0x55ac10c94730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.109+0000 7f2c0d21f640 1 --2- 172.21.15.38:0/1849492302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108ba6b0 0x55ac10c8cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.110+0000 7f2c0e221640 1 -- 172.21.15.38:0/1849492302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108ba6b0 msgr2=0x55ac10c8cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.110+0000 7f2c0e221640 1 --2- 172.21.15.38:0/1849492302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108ba6b0 0x55ac10c8cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.110+0000 7f2c0e221640 1 -- 172.21.15.38:0/1849492302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108b24d0 msgr2=0x55ac10c7f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.110+0000 7f2c0e221640 1 --2- 172.21.15.38:0/1849492302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108b24d0 0x55ac10c7f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.110+0000 7f2c0e221640 1 -- 172.21.15.38:0/1849492302 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac10c7bdb0 con 0x55ac108db390 2024-05-21T15:36:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.110+0000 7f2c0e221640 1 --2- 172.21.15.38:0/1849492302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108db390 0x55ac10c94730 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2c000309f0 tx=0x7f2c00030e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae481be8c7230c50 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.111+0000 7f2c0da20640 1 --2- 172.21.15.38:0/1849492302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108b24d0 0x55ac10c7f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.111+0000 7f2bf37fe640 1 -- 172.21.15.38:0/1849492302 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c000307f0 con 0x55ac108db390 2024-05-21T15:36:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.111+0000 7f2bf37fe640 1 -- 172.21.15.38:0/1849492302 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c00036040 con 0x55ac108db390 2024-05-21T15:36:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.112+0000 7f2bf37fe640 1 -- 172.21.15.38:0/1849492302 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0002f8a0 con 0x55ac108db390 2024-05-21T15:36:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.113+0000 7f2c16c0a740 1 -- 172.21.15.38:0/1849492302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108db390 msgr2=0x55ac10c94730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.113+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/1849492302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108db390 0x55ac10c94730 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2c000309f0 tx=0x7f2c00030e60 comp rx=0 tx=0).stop 2024-05-21T15:36:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.114+0000 7f2c16c0a740 1 -- 172.21.15.38:0/1849492302 shutdown_connections 2024-05-21T15:36:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.114+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/1849492302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108db390 0x55ac10c94730 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.114+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/1849492302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108ba6b0 0x55ac10c8cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.114+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/1849492302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108b24d0 0x55ac10c7f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.114+0000 7f2c16c0a740 1 -- 172.21.15.38:0/1849492302 >> 172.21.15.38:0/1849492302 conn(0x55ac108cf4e0 msgr2=0x55ac10c8c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:53.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.115+0000 7f2c16c0a740 1 -- 172.21.15.38:0/1849492302 shutdown_connections 2024-05-21T15:36:53.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.115+0000 7f2c16c0a740 1 -- 172.21.15.38:0/1849492302 wait complete. 2024-05-21T15:36:53.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.116+0000 7f2c16c0a740 1 -- start start 2024-05-21T15:36:53.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.116+0000 7f2c16c0a740 1 Processor -- start 2024-05-21T15:36:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.117+0000 7f2c16c0a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108b24d0 0x55ac10c916a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.117+0000 7f2c16c0a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108ba6b0 0x55ac10c91be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.117+0000 7f2c16c0a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108db390 0x55ac10d09920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.117+0000 7f2c16c0a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081b1da0 con 0x55ac108db390 2024-05-21T15:36:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.117+0000 7f2c16c0a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081b16a0 con 0x55ac108ba6b0 2024-05-21T15:36:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.117+0000 7f2c16c0a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081ae990 con 0x55ac108b24d0 2024-05-21T15:36:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.117+0000 7f2c16c0a740 10 client.0 _ll_register_callbacks cb 0x7f2c000093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.117+0000 7f2c0d21f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108ba6b0 0x55ac10c91be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.117+0000 7f2c0d21f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108ba6b0 0x55ac10c91be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46698/0 (socket says 172.21.15.38:46698) 2024-05-21T15:36:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.117+0000 7f2c0d21f640 1 -- 172.21.15.38:0/3483739700 learned_addr learned my addr 172.21.15.38:0/3483739700 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:53.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.118+0000 7f2c0d21f640 1 -- 172.21.15.38:0/3483739700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108b24d0 msgr2=0x55ac10c916a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:53.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.118+0000 7f2c0d21f640 1 --2- 172.21.15.38:0/3483739700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108b24d0 0x55ac10c916a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.118+0000 7f2c0d21f640 1 -- 172.21.15.38:0/3483739700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108db390 msgr2=0x55ac10d09920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:53.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.118+0000 7f2c0d21f640 1 --2- 172.21.15.38:0/3483739700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108db390 0x55ac10d09920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.118+0000 7f2c0d21f640 1 -- 172.21.15.38:0/3483739700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac10d09e60 con 0x55ac108ba6b0 2024-05-21T15:36:53.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.119+0000 7f2c0d21f640 1 --2- 172.21.15.38:0/3483739700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108ba6b0 0x55ac10c91be0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800b550 tx=0x7f2bf8016f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe8c84f9fcae3581 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:53.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.119+0000 7f2bf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:53.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.119+0000 7f2bf17fa640 1 -- 172.21.15.38:0/3483739700 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8031070 con 0x55ac108ba6b0 2024-05-21T15:36:53.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.119+0000 7f2bf17fa640 1 -- 172.21.15.38:0/3483739700 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2bf800b6c0 con 0x55ac108ba6b0 2024-05-21T15:36:53.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.120+0000 7f2c16c0a740 1 -- 172.21.15.38:0/3483739700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c00036040 con 0x55ac108ba6b0 2024-05-21T15:36:53.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.124+0000 7f2bf17fa640 1 -- 172.21.15.38:0/3483739700 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2bf803b050 con 0x55ac108ba6b0 2024-05-21T15:36:53.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.124+0000 7f2c16c0a740 10 client.25805 fetch_fsmap learned FSMap version 29 2024-05-21T15:36:53.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.124+0000 7f2c16c0a740 1 -- 172.21.15.38:0/3483739700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac10d0b0d0 con 0x55ac108ba6b0 2024-05-21T15:36:53.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.132+0000 7f2bf17fa640 1 -- 172.21.15.38:0/3483739700 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c00036040 con 0x55ac108ba6b0 2024-05-21T15:36:53.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.132+0000 7f2bf17fa640 1 -- 172.21.15.38:0/3483739700 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2bf8026040 con 0x55ac108ba6b0 2024-05-21T15:36:53.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.133+0000 7f2c16c0a740 10 client.25805 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:36:53.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.133+0000 7f2c16c0a740 10 client.25805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:53.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.133+0000 7f2c16c0a740 20 client.25805 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:53.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.133+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/3483739700 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ac1086f3a0 0x55ac10d0da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.133+0000 7f2c16c0a740 4 client.25805 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:36:53.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.133+0000 7f2c16c0a740 1 -- 172.21.15.38:0/3483739700 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ac10be7780 con 0x55ac1086f3a0 2024-05-21T15:36:53.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.137+0000 7f2c0da20640 1 --2- 172.21.15.38:0/3483739700 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ac1086f3a0 0x55ac10d0da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.141+0000 7f2c0da20640 1 --2- 172.21.15.38:0/3483739700 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ac1086f3a0 0x55ac10d0da80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0813867254dc5a server_cookie=9212b1984a4ac0f6 in_seq=0 out_seq=0 2024-05-21T15:36:53.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.141+0000 7f2bf17fa640 10 client.25805 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:53.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.150+0000 7f2bf17fa640 1 -- 172.21.15.38:0/3483739700 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ac10be7780 con 0x55ac1086f3a0 2024-05-21T15:36:53.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.150+0000 7f2bf17fa640 10 client.25805 handle_command_reply: tid=0 2024-05-21T15:36:53.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.151+0000 7f2c16c0a740 1 client.25805 shutdown 2024-05-21T15:36:53.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.151+0000 7f2c16c0a740 1 -- 172.21.15.38:0/3483739700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108ba6b0 msgr2=0x55ac10c91be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:53.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.151+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/3483739700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108ba6b0 0x55ac10c91be0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800b550 tx=0x7f2bf8016f90 comp rx=0 tx=0).stop 2024-05-21T15:36:53.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.154+0000 7f2c16c0a740 1 -- 172.21.15.38:0/3483739700 shutdown_connections 2024-05-21T15:36:53.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.154+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/3483739700 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ac1086f3a0 0x55ac10d0da80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.154+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/3483739700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ac108db390 0x55ac10d09920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.154+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/3483739700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac108ba6b0 0x55ac10c91be0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.154+0000 7f2c16c0a740 1 --2- 172.21.15.38:0/3483739700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ac108b24d0 0x55ac10c916a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.154+0000 7f2c16c0a740 1 -- 172.21.15.38:0/3483739700 >> 172.21.15.38:0/3483739700 conn(0x55ac108cf4e0 msgr2=0x55ac10c8d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:53.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.154+0000 7f2bf17fa640 0 client.25805 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:53.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.159+0000 7f2c16c0a740 1 -- 172.21.15.38:0/3483739700 shutdown_connections 2024-05-21T15:36:53.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.160+0000 7f2c16c0a740 1 -- 172.21.15.38:0/3483739700 wait complete. 2024-05-21T15:36:53.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.160+0000 7f2c16c0a740 20 client.25805 trim_cache size 0 max 16384 2024-05-21T15:36:53.160 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:36:53.160 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-05-21T15:36:53.160 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-05-21T15:36:53.160 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:36:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.164+0000 7f2c0fc84640 1 -- 172.21.15.38:0/4271909304 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2bf40a71b0 msgr2=0x7f2bf40a96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.164+0000 7f2c0fc84640 1 --2- 172.21.15.38:0/4271909304 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2bf40a71b0 0x7f2bf40a96c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2c081a08f0 tx=0x7f2bf8009b70 comp rx=0 tx=0).stop 2024-05-21T15:36:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.164+0000 7f2c0fc84640 1 -- 172.21.15.38:0/4271909304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c081094d0 msgr2=0x7f2c0819f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.164+0000 7f2c0fc84640 1 --2- 172.21.15.38:0/4271909304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c081094d0 0x7f2c0819f8a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2c08108da0 tx=0x7f2c04007cc0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.166+0000 7f2c0fc84640 1 -- 172.21.15.38:0/4271909304 shutdown_connections 2024-05-21T15:36:53.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.166+0000 7f2c0fc84640 1 --2- 172.21.15.38:0/4271909304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0810f410 0x7f2c081a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.166+0000 7f2c0fc84640 1 --2- 172.21.15.38:0/4271909304 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2bf40a71b0 0x7f2bf40a96c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.166+0000 7f2c0fc84640 1 --2- 172.21.15.38:0/4271909304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0810a710 0x7f2c0819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.166+0000 7f2c0fc84640 1 --2- 172.21.15.38:0/4271909304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c081094d0 0x7f2c0819f8a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.166+0000 7f2c0fc84640 1 -- 172.21.15.38:0/4271909304 >> 172.21.15.38:0/4271909304 conn(0x7f2c0806f4a0 msgr2=0x7f2c0810b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:53.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.168+0000 7f2c0fc84640 1 -- 172.21.15.38:0/4271909304 shutdown_connections 2024-05-21T15:36:53.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.169+0000 7f2c0fc84640 1 -- 172.21.15.38:0/4271909304 wait complete. 2024-05-21T15:36:53.193 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:36:53.193 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:36:53.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.444+0000 7f72c4456640 1 -- 172.21.15.38:0/1740573527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72bc1093d0 msgr2=0x7f72bc109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:53.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.444+0000 7f72c4456640 1 --2- 172.21.15.38:0/1740573527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72bc1093d0 0x7f72bc109dc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009ca0 tx=0x7f72b402a8d0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.449+0000 7f72c4456640 1 -- 172.21.15.38:0/1740573527 shutdown_connections 2024-05-21T15:36:53.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.449+0000 7f72c4456640 1 --2- 172.21.15.38:0/1740573527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc10f020 0x7f72bc10f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.449+0000 7f72c4456640 1 --2- 172.21.15.38:0/1740573527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72bc10a300 0x7f72bc10a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.449+0000 7f72c4456640 1 --2- 172.21.15.38:0/1740573527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72bc1093d0 0x7f72bc109dc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.449+0000 7f72c4456640 1 -- 172.21.15.38:0/1740573527 >> 172.21.15.38:0/1740573527 conn(0x7f72bc06f4e0 msgr2=0x7f72bc071970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:53.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.458+0000 7f72c4456640 1 -- 172.21.15.38:0/1740573527 shutdown_connections 2024-05-21T15:36:53.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.459+0000 7f72c4456640 1 -- 172.21.15.38:0/1740573527 wait complete. 2024-05-21T15:36:53.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.460+0000 7f72c4456640 1 Processor -- start 2024-05-21T15:36:53.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.462+0000 7f72c4456640 1 -- start start 2024-05-21T15:36:53.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.462+0000 7f72c4456640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72bc1093d0 0x7f72bc117f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.462+0000 7f72c4456640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc10a300 0x7f72bc118440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.462+0000 7f72c4456640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72bc10f020 0x7f72bc1193a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.462+0000 7f72c4456640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc11d620 con 0x7f72bc10f020 2024-05-21T15:36:53.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.462+0000 7f72c4456640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc11d4a0 con 0x7f72bc10a300 2024-05-21T15:36:53.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.462+0000 7f72c4456640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc11d7a0 con 0x7f72bc1093d0 2024-05-21T15:36:53.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.464+0000 7f72c29f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72bc10f020 0x7f72bc1193a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.464+0000 7f72c29f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72bc10f020 0x7f72bc1193a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49368/0 (socket says 172.21.15.38:49368) 2024-05-21T15:36:53.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.464+0000 7f72c29f3640 1 -- 172.21.15.38:0/775932706 learned_addr learned my addr 172.21.15.38:0/775932706 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:53.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.465+0000 7f72c19f1640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc10a300 0x7f72bc118440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.467+0000 7f72c21f2640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72bc1093d0 0x7f72bc117f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.467+0000 7f72c29f3640 1 -- 172.21.15.38:0/775932706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72bc1093d0 msgr2=0x7f72bc117f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:53.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.469+0000 7f72c29f3640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72bc1093d0 0x7f72bc117f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.469+0000 7f72c29f3640 1 -- 172.21.15.38:0/775932706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc10a300 msgr2=0x7f72bc118440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:53.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.469+0000 7f72c29f3640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc10a300 0x7f72bc118440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.469+0000 7f72c29f3640 1 -- 172.21.15.38:0/775932706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc119a10 con 0x7f72bc10f020 2024-05-21T15:36:53.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.470+0000 7f72c19f1640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc10a300 0x7f72bc118440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:53.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.470+0000 7f72c21f2640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72bc1093d0 0x7f72bc117f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:36:53.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.471+0000 7f72c29f3640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72bc10f020 0x7f72bc1193a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f72b800c970 tx=0x7f72b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:53.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.472+0000 7f72b37fe640 1 -- 172.21.15.38:0/775932706 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8007bb0 con 0x7f72bc10f020 2024-05-21T15:36:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.473+0000 7f72c4456640 1 -- 172.21.15.38:0/775932706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72bc119d00 con 0x7f72bc10f020 2024-05-21T15:36:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.473+0000 7f72c4456640 1 -- 172.21.15.38:0/775932706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72bc11a250 con 0x7f72bc10f020 2024-05-21T15:36:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.473+0000 7f72b37fe640 1 -- 172.21.15.38:0/775932706 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b8007d50 con 0x7f72bc10f020 2024-05-21T15:36:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.473+0000 7f72b37fe640 1 -- 172.21.15.38:0/775932706 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b800f450 con 0x7f72bc10f020 2024-05-21T15:36:53.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.479+0000 7f72b37fe640 1 -- 172.21.15.38:0/775932706 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f72b8004f60 con 0x7f72bc10f020 2024-05-21T15:36:53.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.480+0000 7f72b37fe640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f72a80a2e70 0x7f72a80a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:53.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.480+0000 7f72b37fe640 1 -- 172.21.15.38:0/775932706 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f72b80cec20 con 0x7f72bc10f020 2024-05-21T15:36:53.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.481+0000 7f72c21f2640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f72a80a2e70 0x7f72a80a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:53.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.481+0000 7f72c4456640 1 -- 172.21.15.38:0/775932706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a00052c0 con 0x7f72bc10f020 2024-05-21T15:36:53.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.492+0000 7f72c21f2640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f72a80a2e70 0x7f72a80a5380 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f72b4006010 tx=0x7f72b4000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:53.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.493+0000 7f72b37fe640 1 -- 172.21.15.38:0/775932706 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f72b8014030 con 0x7f72bc10f020 2024-05-21T15:36:53.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:53 smithi038 ceph-mon[31995]: pgmap v425: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 437 MiB/s wr, 21.25k op/s 2024-05-21T15:36:53.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:53 smithi053 ceph-mon[36496]: pgmap v425: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 437 MiB/s wr, 21.25k op/s 2024-05-21T15:36:53.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:53 smithi045 ceph-mon[36409]: pgmap v425: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 437 MiB/s wr, 21.25k op/s 2024-05-21T15:36:53.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.976+0000 7f72c4456640 1 -- 172.21.15.38:0/775932706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72a00055b0 con 0x7f72bc10f020 2024-05-21T15:36:53.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.978+0000 7f72b37fe640 1 -- 172.21.15.38:0/775932706 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11916 (secure 0 0 0) 0x7f72b809bad0 con 0x7f72bc10f020 2024-05-21T15:36:53.978 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:36:53.978 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:36:42.399643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24523,24551,24602,24515,24550]},"id":1}]} 2024-05-21T15:36:53.978 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 29 2024-05-21T15:36:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.989+0000 7f72c4456640 1 -- 172.21.15.38:0/775932706 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f72a80a2e70 msgr2=0x7f72a80a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.989+0000 7f72c4456640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f72a80a2e70 0x7f72a80a5380 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f72b4006010 tx=0x7f72b4000f40 comp rx=0 tx=0).stop 2024-05-21T15:36:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.989+0000 7f72c4456640 1 -- 172.21.15.38:0/775932706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72bc10f020 msgr2=0x7f72bc1193a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.989+0000 7f72c4456640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72bc10f020 0x7f72bc1193a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f72b800c970 tx=0x7f72b800ce30 comp rx=0 tx=0).stop 2024-05-21T15:36:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.989+0000 7f72c4456640 1 -- 172.21.15.38:0/775932706 shutdown_connections 2024-05-21T15:36:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.989+0000 7f72c4456640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f72a80a2e70 0x7f72a80a5380 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.989+0000 7f72c4456640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72bc10f020 0x7f72bc1193a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.989+0000 7f72c4456640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72bc10a300 0x7f72bc118440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.989+0000 7f72c4456640 1 --2- 172.21.15.38:0/775932706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72bc1093d0 0x7f72bc117f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:53.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.989+0000 7f72c4456640 1 -- 172.21.15.38:0/775932706 >> 172.21.15.38:0/775932706 conn(0x7f72bc06f4e0 msgr2=0x7f72bc071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:53.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.989+0000 7f72c4456640 1 -- 172.21.15.38:0/775932706 shutdown_connections 2024-05-21T15:36:53.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:53.990+0000 7f72c4456640 1 -- 172.21.15.38:0/775932706 wait complete. 2024-05-21T15:36:54.011 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:36:54.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.245+0000 7fcf4b185640 1 -- 172.21.15.38:0/1156756757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44105f00 msgr2=0x7fcf44106310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.245+0000 7fcf4b185640 1 --2- 172.21.15.38:0/1156756757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44105f00 0x7fcf44106310 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009900 tx=0x7fcf2c02a400 comp rx=0 tx=0).stop 2024-05-21T15:36:54.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.245+0000 7fcf4b185640 1 -- 172.21.15.38:0/1156756757 shutdown_connections 2024-05-21T15:36:54.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.245+0000 7fcf4b185640 1 --2- 172.21.15.38:0/1156756757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf4410ac00 0x7fcf4410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.245+0000 7fcf4b185640 1 --2- 172.21.15.38:0/1156756757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44105f00 0x7fcf44106310 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.245+0000 7fcf4b185640 1 --2- 172.21.15.38:0/1156756757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf44104cc0 0x7fcf441059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.245+0000 7fcf4b185640 1 -- 172.21.15.38:0/1156756757 >> 172.21.15.38:0/1156756757 conn(0x7fcf440ffed0 msgr2=0x7fcf44102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:54.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.246+0000 7fcf4b185640 1 -- 172.21.15.38:0/1156756757 shutdown_connections 2024-05-21T15:36:54.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.246+0000 7fcf4b185640 1 -- 172.21.15.38:0/1156756757 wait complete. 2024-05-21T15:36:54.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.247+0000 7fcf4b185640 1 Processor -- start 2024-05-21T15:36:54.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.247+0000 7fcf4b185640 1 -- start start 2024-05-21T15:36:54.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.247+0000 7fcf4b185640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44104cc0 0x7fcf4419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.247+0000 7fcf4b185640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf44105f00 0x7fcf4419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.247+0000 7fcf4b185640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf4410ac00 0x7fcf441a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.247+0000 7fcf4b185640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44110910 con 0x7fcf44104cc0 2024-05-21T15:36:54.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.247+0000 7fcf4b185640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44110790 con 0x7fcf4410ac00 2024-05-21T15:36:54.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.247+0000 7fcf4b185640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44110a90 con 0x7fcf44105f00 2024-05-21T15:36:54.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.247+0000 7fcf49722640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf4410ac00 0x7fcf441a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.247+0000 7fcf49722640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf4410ac00 0x7fcf441a6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46736/0 (socket says 172.21.15.38:46736) 2024-05-21T15:36:54.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.247+0000 7fcf49722640 1 -- 172.21.15.38:0/588021525 learned_addr learned my addr 172.21.15.38:0/588021525 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:54.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.247+0000 7fcf43fff640 1 --2- 172.21.15.38:0/588021525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf44105f00 0x7fcf4419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.248+0000 7fcf49722640 1 -- 172.21.15.38:0/588021525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf44105f00 msgr2=0x7fcf4419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.248+0000 7fcf49722640 1 --2- 172.21.15.38:0/588021525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf44105f00 0x7fcf4419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.248+0000 7fcf49722640 1 -- 172.21.15.38:0/588021525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44104cc0 msgr2=0x7fcf4419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.248+0000 7fcf49722640 1 --2- 172.21.15.38:0/588021525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44104cc0 0x7fcf4419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.248+0000 7fcf49722640 1 -- 172.21.15.38:0/588021525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf441a7360 con 0x7fcf4410ac00 2024-05-21T15:36:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.248+0000 7fcf49722640 1 --2- 172.21.15.38:0/588021525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf4410ac00 0x7fcf441a6cf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009fd0 tx=0x7fcf3400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.249+0000 7fcf41ffb640 1 -- 172.21.15.38:0/588021525 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34017070 con 0x7fcf4410ac00 2024-05-21T15:36:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.249+0000 7fcf41ffb640 1 -- 172.21.15.38:0/588021525 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf34003e50 con 0x7fcf4410ac00 2024-05-21T15:36:54.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.249+0000 7fcf41ffb640 1 -- 172.21.15.38:0/588021525 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf340106c0 con 0x7fcf4410ac00 2024-05-21T15:36:54.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.249+0000 7fcf4b185640 1 -- 172.21.15.38:0/588021525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf441a7650 con 0x7fcf4410ac00 2024-05-21T15:36:54.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.250+0000 7fcf4b185640 1 -- 172.21.15.38:0/588021525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf441a7b50 con 0x7fcf4410ac00 2024-05-21T15:36:54.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.255+0000 7fcf52035740 1 Processor -- start 2024-05-21T15:36:54.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.255+0000 7fcf52035740 1 -- start start 2024-05-21T15:36:54.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.255+0000 7fcf52035740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164bf390 0x55b51685eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.255+0000 7fcf52035740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164964d0 0x55b51685f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.255+0000 7fcf52035740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b51649e6b0 0x55b516865f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.255+0000 7fcf52035740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b5167c3c40 con 0x55b5164964d0 2024-05-21T15:36:54.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.255+0000 7fcf52035740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b51676b580 con 0x55b51649e6b0 2024-05-21T15:36:54.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.255+0000 7fcf52035740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b5167bfc00 con 0x55b5164bf390 2024-05-21T15:36:54.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.256+0000 7fcf48f21640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164964d0 0x55b51685f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.256+0000 7fcf48f21640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164964d0 0x55b51685f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49396/0 (socket says 172.21.15.38:49396) 2024-05-21T15:36:54.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.256+0000 7fcf48f21640 1 -- 172.21.15.38:0/1901544455 learned_addr learned my addr 172.21.15.38:0/1901544455 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:54.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.256+0000 7fcf48f21640 1 -- 172.21.15.38:0/1901544455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164bf390 msgr2=0x55b51685eb00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:36:54.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.256+0000 7fcf48f21640 1 --2- 172.21.15.38:0/1901544455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164bf390 0x55b51685eb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.256+0000 7fcf48f21640 1 -- 172.21.15.38:0/1901544455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b51649e6b0 msgr2=0x55b516865f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:36:54.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.256+0000 7fcf48f21640 1 --2- 172.21.15.38:0/1901544455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b51649e6b0 0x55b516865f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.256+0000 7fcf48f21640 1 -- 172.21.15.38:0/1901544455 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5164d73f0 con 0x55b5164964d0 2024-05-21T15:36:54.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.257+0000 7fcf48f21640 1 --2- 172.21.15.38:0/1901544455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164964d0 0x55b51685f080 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00baf0 tx=0x7fcf3c00d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e23fd986162b33 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:54.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.258+0000 7fcf41ffb640 1 -- 172.21.15.38:0/588021525 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcf34010860 con 0x7fcf4410ac00 2024-05-21T15:36:54.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.258+0000 7fcf3affd640 1 -- 172.21.15.38:0/1901544455 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c016070 con 0x55b5164964d0 2024-05-21T15:36:54.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.258+0000 7fcf3affd640 1 -- 172.21.15.38:0/1901544455 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3c00dd10 con 0x55b5164964d0 2024-05-21T15:36:54.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.258+0000 7fcf3affd640 1 -- 172.21.15.38:0/1901544455 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00f880 con 0x55b5164964d0 2024-05-21T15:36:54.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.259+0000 7fcf41ffb640 1 --2- 172.21.15.38:0/588021525 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcf300a2d60 0x7fcf300a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.260+0000 7fcf52035740 1 -- 172.21.15.38:0/1901544455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164964d0 msgr2=0x55b51685f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.260+0000 7fcf52035740 1 --2- 172.21.15.38:0/1901544455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164964d0 0x55b51685f080 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00baf0 tx=0x7fcf3c00d2e0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.260+0000 7fcf41ffb640 1 -- 172.21.15.38:0/588021525 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fcf340cf310 con 0x7fcf4410ac00 2024-05-21T15:36:54.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.261+0000 7fcf43fff640 1 --2- 172.21.15.38:0/588021525 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcf300a2d60 0x7fcf300a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.261+0000 7fcf52035740 1 -- 172.21.15.38:0/1901544455 shutdown_connections 2024-05-21T15:36:54.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.261+0000 7fcf52035740 1 --2- 172.21.15.38:0/1901544455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b51649e6b0 0x55b516865f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.261+0000 7fcf52035740 1 --2- 172.21.15.38:0/1901544455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164964d0 0x55b51685f080 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.261+0000 7fcf52035740 1 --2- 172.21.15.38:0/1901544455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164bf390 0x55b51685eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.261+0000 7fcf52035740 1 -- 172.21.15.38:0/1901544455 >> 172.21.15.38:0/1901544455 conn(0x55b5164b34e0 msgr2=0x55b51685a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:54.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.264+0000 7fcf52035740 1 -- 172.21.15.38:0/1901544455 shutdown_connections 2024-05-21T15:36:54.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.264+0000 7fcf43fff640 1 --2- 172.21.15.38:0/588021525 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcf300a2d60 0x7fcf300a5270 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c000c00 tx=0x7fcf2c031660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:54.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.264+0000 7fcf52035740 1 -- 172.21.15.38:0/1901544455 wait complete. 2024-05-21T15:36:54.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.265+0000 7fcf52035740 1 -- start start 2024-05-21T15:36:54.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.265+0000 7fcf52035740 1 Processor -- start 2024-05-21T15:36:54.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.265+0000 7fcf52035740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164964d0 0x55b516868940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.265+0000 7fcf52035740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b51649e6b0 0x55b516868e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.266+0000 7fcf52035740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5164bf390 0x55b51686fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.266+0000 7fcf52035740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b5168669d0 con 0x55b51649e6b0 2024-05-21T15:36:54.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.266+0000 7fcf52035740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b51677ec40 con 0x55b5164bf390 2024-05-21T15:36:54.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.266+0000 7fcf52035740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b516866b50 con 0x55b5164964d0 2024-05-21T15:36:54.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.266+0000 7fcf52035740 10 client.0 _ll_register_callbacks cb 0x55b5164d6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:54.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.266+0000 7fcf49722640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5164bf390 0x55b51686fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.266+0000 7fcf48f21640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164964d0 0x55b516868940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.266+0000 7fcf43fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b51649e6b0 0x55b516868e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.266+0000 7fcf49722640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5164bf390 0x55b51686fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46740/0 (socket says 172.21.15.38:46740) 2024-05-21T15:36:54.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.266+0000 7fcf49722640 1 -- 172.21.15.38:0/2838195783 learned_addr learned my addr 172.21.15.38:0/2838195783 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:54.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.267+0000 7fcf43fff640 1 -- 172.21.15.38:0/2838195783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164964d0 msgr2=0x55b516868940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:54.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.267+0000 7fcf43fff640 1 --2- 172.21.15.38:0/2838195783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164964d0 0x55b516868940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.267+0000 7fcf43fff640 1 -- 172.21.15.38:0/2838195783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5164bf390 msgr2=0x55b51686fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:54.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.267+0000 7fcf43fff640 1 --2- 172.21.15.38:0/2838195783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5164bf390 0x55b51686fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.267+0000 7fcf43fff640 1 -- 172.21.15.38:0/2838195783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5165a17d0 con 0x55b51649e6b0 2024-05-21T15:36:54.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.267+0000 7fcf43fff640 1 --2- 172.21.15.38:0/2838195783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b51649e6b0 0x55b516868e80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009230 tx=0x7fcf2c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdda6f96486af21 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:54.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.269+0000 7fcf23fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:54.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.269+0000 7fcf23fff640 1 -- 172.21.15.38:0/2838195783 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c04e020 con 0x55b51649e6b0 2024-05-21T15:36:54.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.269+0000 7fcf23fff640 1 -- 172.21.15.38:0/2838195783 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fcf2c0317a0 con 0x55b51649e6b0 2024-05-21T15:36:54.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.269+0000 7fcf52035740 1 -- 172.21.15.38:0/2838195783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf3c00dd10 con 0x55b51649e6b0 2024-05-21T15:36:54.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.269+0000 7fcf23fff640 1 -- 172.21.15.38:0/2838195783 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf2c059050 con 0x55b51649e6b0 2024-05-21T15:36:54.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.269+0000 7fcf52035740 10 client.16146 fetch_fsmap learned FSMap version 29 2024-05-21T15:36:54.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.269+0000 7fcf52035740 1 -- 172.21.15.38:0/2838195783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b516884150 con 0x55b51649e6b0 2024-05-21T15:36:54.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.270+0000 7fcf23fff640 1 -- 172.21.15.38:0/2838195783 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3c00dd10 con 0x55b51649e6b0 2024-05-21T15:36:54.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.270+0000 7fcf23fff640 1 -- 172.21.15.38:0/2838195783 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcf2c03a040 con 0x55b51649e6b0 2024-05-21T15:36:54.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.270+0000 7fcf52035740 10 client.16146 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:36:54.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.270+0000 7fcf52035740 10 client.16146 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:36:54.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.270+0000 7fcf52035740 20 client.16146 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:54.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.270+0000 7fcf52035740 1 --2- 172.21.15.38:0/2838195783 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b51647df60 0x55b516887e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.270+0000 7fcf52035740 4 client.16146 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:54.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.270+0000 7fcf52035740 1 -- 172.21.15.38:0/2838195783 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5167d1370 con 0x55b51647df60 2024-05-21T15:36:54.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.271+0000 7fcf48f21640 1 --2- 172.21.15.38:0/2838195783 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b51647df60 0x55b516887e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.275+0000 7fcf48f21640 1 --2- 172.21.15.38:0/2838195783 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b51647df60 0x55b516887e10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=95d8dd1449d3caad server_cookie=70084df4a25100 in_seq=0 out_seq=0 2024-05-21T15:36:54.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.275+0000 7fcf23fff640 10 client.16146 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:36:54.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.276+0000 7fcf23fff640 1 -- 172.21.15.38:0/2838195783 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b5167d1370 con 0x55b51647df60 2024-05-21T15:36:54.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.277+0000 7fcf23fff640 10 client.16146 handle_command_reply: tid=0 2024-05-21T15:36:54.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.279+0000 7fcf52035740 1 client.16146 shutdown 2024-05-21T15:36:54.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.279+0000 7fcf52035740 1 -- 172.21.15.38:0/2838195783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b51649e6b0 msgr2=0x55b516868e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.279+0000 7fcf52035740 1 --2- 172.21.15.38:0/2838195783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b51649e6b0 0x55b516868e80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009230 tx=0x7fcf2c009260 comp rx=0 tx=0).stop 2024-05-21T15:36:54.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.279+0000 7fcf52035740 1 -- 172.21.15.38:0/2838195783 shutdown_connections 2024-05-21T15:36:54.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.279+0000 7fcf52035740 1 --2- 172.21.15.38:0/2838195783 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b51647df60 0x55b516887e10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.279+0000 7fcf52035740 1 --2- 172.21.15.38:0/2838195783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5164bf390 0x55b51686fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.279+0000 7fcf52035740 1 --2- 172.21.15.38:0/2838195783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b51649e6b0 0x55b516868e80 secure :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009230 tx=0x7fcf2c009260 comp rx=0 tx=0).stop 2024-05-21T15:36:54.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.279+0000 7fcf52035740 1 --2- 172.21.15.38:0/2838195783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164964d0 0x55b516868940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.279+0000 7fcf52035740 1 -- 172.21.15.38:0/2838195783 >> 172.21.15.38:0/2838195783 conn(0x55b5164b34e0 msgr2=0x55b516859570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:54.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.281+0000 7fcf23fff640 0 client.16146 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:36:54.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.282+0000 7fcf52035740 1 -- 172.21.15.38:0/2838195783 shutdown_connections 2024-05-21T15:36:54.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.282+0000 7fcf52035740 1 -- 172.21.15.38:0/2838195783 wait complete. 2024-05-21T15:36:54.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.282+0000 7fcf52035740 20 client.16146 trim_cache size 0 max 16384 2024-05-21T15:36:54.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.314+0000 7fcf52035740 1 Processor -- start 2024-05-21T15:36:54.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.316+0000 7fcf52035740 1 -- start start 2024-05-21T15:36:54.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.316+0000 7fcf52035740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164964d0 0x55b516862cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.316+0000 7fcf52035740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5164bf390 0x55b51686fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.316+0000 7fcf52035740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164533a0 0x55b516863210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.316+0000 7fcf52035740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2400e630 con 0x55b5164533a0 2024-05-21T15:36:54.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.316+0000 7fcf52035740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcf240101d0 con 0x55b5164bf390 2024-05-21T15:36:54.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.316+0000 7fcf52035740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2400ed00 con 0x55b5164964d0 2024-05-21T15:36:54.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.317+0000 7fcf48f21640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164964d0 0x55b516862cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.317+0000 7fcf49722640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164533a0 0x55b516863210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.317+0000 7fcf48f21640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164964d0 0x55b516862cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:52104/0 (socket says 172.21.15.38:52104) 2024-05-21T15:36:54.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.317+0000 7fcf49722640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164533a0 0x55b516863210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49412/0 (socket says 172.21.15.38:49412) 2024-05-21T15:36:54.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.317+0000 7fcf48f21640 1 -- 172.21.15.38:0/2189365422 learned_addr learned my addr 172.21.15.38:0/2189365422 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:54.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.318+0000 7fcf48f21640 1 -- 172.21.15.38:0/2189365422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5164bf390 msgr2=0x55b51686fda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:36:54.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.318+0000 7fcf48f21640 1 --2- 172.21.15.38:0/2189365422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5164bf390 0x55b51686fda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.318+0000 7fcf48f21640 1 -- 172.21.15.38:0/2189365422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164533a0 msgr2=0x55b516863210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:54.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.318+0000 7fcf48f21640 1 --2- 172.21.15.38:0/2189365422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164533a0 0x55b516863210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.318+0000 7fcf48f21640 1 -- 172.21.15.38:0/2189365422 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b516855d20 con 0x55b5164964d0 2024-05-21T15:36:54.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.319+0000 7fcf48f21640 1 --2- 172.21.15.38:0/2189365422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164964d0 0x55b516862cd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00d7e0 tx=0x7fcf3c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad12a1620ffa8b0c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:54.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.321+0000 7fcf3a7fc640 1 -- 172.21.15.38:0/2189365422 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c016040 con 0x55b5164964d0 2024-05-21T15:36:54.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.321+0000 7fcf3a7fc640 1 -- 172.21.15.38:0/2189365422 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3c024a90 con 0x55b5164964d0 2024-05-21T15:36:54.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.322+0000 7fcf52035740 1 -- 172.21.15.38:0/2189365422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164964d0 msgr2=0x55b516862cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.322+0000 7fcf52035740 1 --2- 172.21.15.38:0/2189365422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164964d0 0x55b516862cd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00d7e0 tx=0x7fcf3c0046c0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.322+0000 7fcf52035740 1 -- 172.21.15.38:0/2189365422 shutdown_connections 2024-05-21T15:36:54.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.322+0000 7fcf52035740 1 --2- 172.21.15.38:0/2189365422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164533a0 0x55b516863210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.322+0000 7fcf52035740 1 --2- 172.21.15.38:0/2189365422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5164bf390 0x55b51686fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.322+0000 7fcf52035740 1 --2- 172.21.15.38:0/2189365422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164964d0 0x55b516862cd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.323+0000 7fcf52035740 1 -- 172.21.15.38:0/2189365422 >> 172.21.15.38:0/2189365422 conn(0x55b5164b34e0 msgr2=0x55b516866fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:54.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.328+0000 7fcf52035740 1 -- 172.21.15.38:0/2189365422 shutdown_connections 2024-05-21T15:36:54.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.329+0000 7fcf52035740 1 -- 172.21.15.38:0/2189365422 wait complete. 2024-05-21T15:36:54.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.329+0000 7fcf52035740 1 -- start start 2024-05-21T15:36:54.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.330+0000 7fcf52035740 1 Processor -- start 2024-05-21T15:36:54.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.330+0000 7fcf52035740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5164533a0 0x55b51686b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.330+0000 7fcf52035740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164964d0 0x55b51686b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.330+0000 7fcf52035740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164bf390 0x55b51686e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.330+0000 7fcf52035740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2400fad0 con 0x55b5164964d0 2024-05-21T15:36:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.330+0000 7fcf52035740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2400f3d0 con 0x55b5164533a0 2024-05-21T15:36:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.330+0000 7fcf52035740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2400c610 con 0x55b5164bf390 2024-05-21T15:36:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.330+0000 7fcf52035740 10 client.0 _ll_register_callbacks cb 0x7fcf2c041070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.332+0000 7fcf49722640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164bf390 0x55b51686e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.332+0000 7fcf49722640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164bf390 0x55b51686e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:52110/0 (socket says 172.21.15.38:52110) 2024-05-21T15:36:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.332+0000 7fcf49722640 1 -- 172.21.15.38:0/3825347722 learned_addr learned my addr 172.21.15.38:0/3825347722 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.332+0000 7fcf49722640 1 -- 172.21.15.38:0/3825347722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5164533a0 msgr2=0x55b51686b190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:36:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.332+0000 7fcf49722640 1 --2- 172.21.15.38:0/3825347722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5164533a0 0x55b51686b190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.332+0000 7fcf49722640 1 -- 172.21.15.38:0/3825347722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164964d0 msgr2=0x55b51686b6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:36:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.332+0000 7fcf49722640 1 --2- 172.21.15.38:0/3825347722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164964d0 0x55b51686b6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.332+0000 7fcf49722640 1 -- 172.21.15.38:0/3825347722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b51686ebd0 con 0x55b5164bf390 2024-05-21T15:36:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.334+0000 7fcf49722640 1 --2- 172.21.15.38:0/3825347722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164bf390 0x55b51686e610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400b760 tx=0x7fcf3409ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b72ff54bfe659510 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.334+0000 7fcf23fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:36:54.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.335+0000 7fcf23fff640 1 -- 172.21.15.38:0/3825347722 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34020040 con 0x55b5164bf390 2024-05-21T15:36:54.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.335+0000 7fcf52035740 1 -- 172.21.15.38:0/3825347722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b5167cb6c0 con 0x55b5164bf390 2024-05-21T15:36:54.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.335+0000 7fcf23fff640 1 -- 172.21.15.38:0/3825347722 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fcf34010b50 con 0x55b5164bf390 2024-05-21T15:36:54.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.335+0000 7fcf23fff640 1 -- 172.21.15.38:0/3825347722 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf34010d50 con 0x55b5164bf390 2024-05-21T15:36:54.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.335+0000 7fcf52035740 10 client.25873 fetch_fsmap learned FSMap version 29 2024-05-21T15:36:54.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.335+0000 7fcf52035740 1 -- 172.21.15.38:0/3825347722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b51686ff80 con 0x55b5164bf390 2024-05-21T15:36:54.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.336+0000 7fcf23fff640 1 -- 172.21.15.38:0/3825347722 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b5167cb6c0 con 0x55b5164bf390 2024-05-21T15:36:54.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.336+0000 7fcf23fff640 1 -- 172.21.15.38:0/3825347722 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcf3409ba70 con 0x55b5164bf390 2024-05-21T15:36:54.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.336+0000 7fcf52035740 10 client.25873 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:36:54.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.336+0000 7fcf52035740 10 client.25873 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:36:54.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.336+0000 7fcf52035740 20 client.25873 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:54.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.336+0000 7fcf52035740 1 --2- 172.21.15.38:0/3825347722 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b5169018c0 0x55b516903ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.336+0000 7fcf52035740 4 client.25873 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:36:54.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.336+0000 7fcf52035740 1 -- 172.21.15.38:0/3825347722 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b516748820 con 0x55b5169018c0 2024-05-21T15:36:54.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.339+0000 7fcf48f21640 1 --2- 172.21.15.38:0/3825347722 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b5169018c0 0x55b516903ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.341+0000 7fcf48f21640 1 --2- 172.21.15.38:0/3825347722 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b5169018c0 0x55b516903ce0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1bebaadbaa8e55e3 server_cookie=94a21916e9b469a7 in_seq=0 out_seq=0 2024-05-21T15:36:54.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.341+0000 7fcf23fff640 10 client.25873 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:36:54.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.348+0000 7fcf23fff640 1 -- 172.21.15.38:0/3825347722 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b516748820 con 0x55b5169018c0 2024-05-21T15:36:54.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.348+0000 7fcf23fff640 10 client.25873 handle_command_reply: tid=0 2024-05-21T15:36:54.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.349+0000 7fcf52035740 1 client.25873 shutdown 2024-05-21T15:36:54.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.349+0000 7fcf52035740 1 -- 172.21.15.38:0/3825347722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164bf390 msgr2=0x55b51686e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.349+0000 7fcf52035740 1 --2- 172.21.15.38:0/3825347722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164bf390 0x55b51686e610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400b760 tx=0x7fcf3409ce50 comp rx=0 tx=0).stop 2024-05-21T15:36:54.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.351+0000 7fcf52035740 1 -- 172.21.15.38:0/3825347722 shutdown_connections 2024-05-21T15:36:54.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.351+0000 7fcf52035740 1 --2- 172.21.15.38:0/3825347722 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b5169018c0 0x55b516903ce0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.351+0000 7fcf52035740 1 --2- 172.21.15.38:0/3825347722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5164bf390 0x55b51686e610 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.351+0000 7fcf52035740 1 --2- 172.21.15.38:0/3825347722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5164964d0 0x55b51686b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.351+0000 7fcf52035740 1 --2- 172.21.15.38:0/3825347722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5164533a0 0x55b51686b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.351+0000 7fcf52035740 1 -- 172.21.15.38:0/3825347722 >> 172.21.15.38:0/3825347722 conn(0x55b5164b34e0 msgr2=0x55b5168688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:54.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.351+0000 7fcf23fff640 0 client.25873 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:36:54.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.351+0000 7fcf52035740 1 -- 172.21.15.38:0/3825347722 shutdown_connections 2024-05-21T15:36:54.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.352+0000 7fcf52035740 1 -- 172.21.15.38:0/3825347722 wait complete. 2024-05-21T15:36:54.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.353+0000 7fcf52035740 20 client.25873 trim_cache size 0 max 16384 2024-05-21T15:36:54.355 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:36:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.359+0000 7fcf4b185640 1 -- 172.21.15.38:0/588021525 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcf300a2d60 msgr2=0x7fcf300a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.359+0000 7fcf4b185640 1 --2- 172.21.15.38:0/588021525 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcf300a2d60 0x7fcf300a5270 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c000c00 tx=0x7fcf2c031660 comp rx=0 tx=0).stop 2024-05-21T15:36:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.359+0000 7fcf4b185640 1 -- 172.21.15.38:0/588021525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf4410ac00 msgr2=0x7fcf441a6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.359+0000 7fcf4b185640 1 --2- 172.21.15.38:0/588021525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf4410ac00 0x7fcf441a6cf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009fd0 tx=0x7fcf3400b260 comp rx=0 tx=0).stop 2024-05-21T15:36:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.359+0000 7fcf4b185640 1 -- 172.21.15.38:0/588021525 shutdown_connections 2024-05-21T15:36:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.359+0000 7fcf4b185640 1 --2- 172.21.15.38:0/588021525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf4410ac00 0x7fcf441a6cf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.359+0000 7fcf4b185640 1 --2- 172.21.15.38:0/588021525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf44105f00 0x7fcf4419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.359+0000 7fcf4b185640 1 --2- 172.21.15.38:0/588021525 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcf300a2d60 0x7fcf300a5270 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.359+0000 7fcf4b185640 1 --2- 172.21.15.38:0/588021525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf44104cc0 0x7fcf4419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.359+0000 7fcf4b185640 1 -- 172.21.15.38:0/588021525 >> 172.21.15.38:0/588021525 conn(0x7fcf440ffed0 msgr2=0x7fcf44106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:54.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.361+0000 7fcf4b185640 1 -- 172.21.15.38:0/588021525 shutdown_connections 2024-05-21T15:36:54.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.362+0000 7fcf4b185640 1 -- 172.21.15.38:0/588021525 wait complete. 2024-05-21T15:36:54.381 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:36:54.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.564+0000 7fca553db640 1 -- 172.21.15.38:0/2136248180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca480a5940 msgr2=0x7fca480a6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.564+0000 7fca553db640 1 --2- 172.21.15.38:0/2136248180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca480a5940 0x7fca480a6500 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fca44009900 tx=0x7fca4402a420 comp rx=0 tx=0).stop 2024-05-21T15:36:54.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.568+0000 7fca553db640 1 -- 172.21.15.38:0/2136248180 shutdown_connections 2024-05-21T15:36:54.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.568+0000 7fca553db640 1 --2- 172.21.15.38:0/2136248180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca480ab760 0x7fca480abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.568+0000 7fca553db640 1 --2- 172.21.15.38:0/2136248180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca480a6a40 0x7fca480a6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.568+0000 7fca553db640 1 --2- 172.21.15.38:0/2136248180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca480a5940 0x7fca480a6500 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.568+0000 7fca553db640 1 -- 172.21.15.38:0/2136248180 >> 172.21.15.38:0/2136248180 conn(0x7fca480a09b0 msgr2=0x7fca480a2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:54.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.571+0000 7fca553db640 1 -- 172.21.15.38:0/2136248180 shutdown_connections 2024-05-21T15:36:54.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.573+0000 7fca553db640 1 -- 172.21.15.38:0/2136248180 wait complete. 2024-05-21T15:36:54.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.574+0000 7fca553db640 1 Processor -- start 2024-05-21T15:36:54.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.576+0000 7fca553db640 1 -- start start 2024-05-21T15:36:54.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.577+0000 7fca553db640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca480a5940 0x7fca480b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.577+0000 7fca553db640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca480a6a40 0x7fca480b7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.577+0000 7fca553db640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca480ab760 0x7fca480b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.577+0000 7fca553db640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fca480adac0 con 0x7fca480a5940 2024-05-21T15:36:54.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.577+0000 7fca553db640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fca480ad940 con 0x7fca480a6a40 2024-05-21T15:36:54.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.577+0000 7fca553db640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fca480adc40 con 0x7fca480ab760 2024-05-21T15:36:54.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.577+0000 7fca54bda640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca480ab760 0x7fca480b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.577+0000 7fca54bda640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca480ab760 0x7fca480b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:52118/0 (socket says 172.21.15.38:52118) 2024-05-21T15:36:54.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.577+0000 7fca54bda640 1 -- 172.21.15.38:0/103543251 learned_addr learned my addr 172.21.15.38:0/103543251 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:54.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.578+0000 7fca54bda640 1 -- 172.21.15.38:0/103543251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca480a6a40 msgr2=0x7fca480b7350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:36:54.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.578+0000 7fca54bda640 1 --2- 172.21.15.38:0/103543251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca480a6a40 0x7fca480b7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.578+0000 7fca54bda640 1 -- 172.21.15.38:0/103543251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca480a5940 msgr2=0x7fca480b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.578+0000 7fca54bda640 1 --2- 172.21.15.38:0/103543251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca480a5940 0x7fca480b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.578+0000 7fca54bda640 1 -- 172.21.15.38:0/103543251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca480b5d90 con 0x7fca480ab760 2024-05-21T15:36:54.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.579+0000 7fca54bda640 1 --2- 172.21.15.38:0/103543251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca480ab760 0x7fca480b5570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca50050990 tx=0x7fca50070570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:54.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.580+0000 7fca4d7fa640 1 -- 172.21.15.38:0/103543251 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50076620 con 0x7fca480ab760 2024-05-21T15:36:54.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.580+0000 7fca553db640 1 -- 172.21.15.38:0/103543251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca480b6080 con 0x7fca480ab760 2024-05-21T15:36:54.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.581+0000 7fca553db640 1 -- 172.21.15.38:0/103543251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca48155290 con 0x7fca480ab760 2024-05-21T15:36:54.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.581+0000 7fca4d7fa640 1 -- 172.21.15.38:0/103543251 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5007b070 con 0x7fca480ab760 2024-05-21T15:36:54.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.581+0000 7fca4d7fa640 1 -- 172.21.15.38:0/103543251 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50075b40 con 0x7fca480ab760 2024-05-21T15:36:54.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.586+0000 7fca4d7fa640 1 -- 172.21.15.38:0/103543251 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca500767c0 con 0x7fca480ab760 2024-05-21T15:36:54.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.591+0000 7fca4d7fa640 1 --2- 172.21.15.38:0/103543251 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fca400a2c10 0x7fca400a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.591+0000 7fca4d7fa640 1 -- 172.21.15.38:0/103543251 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fca50132380 con 0x7fca480ab760 2024-05-21T15:36:54.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.595+0000 7fca5d5c2740 1 Processor -- start 2024-05-21T15:36:54.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.595+0000 7fca5d5c2740 1 -- start start 2024-05-21T15:36:54.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.595+0000 7fca5d5c2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a909c0390 0x563a90d5fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.595+0000 7fca5d5c2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909974d0 0x563a90d602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.595+0000 7fca5d5c2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a9099f6b0 0x563a90d671f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.595+0000 7fca5d5c2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563a90cc4c40 con 0x563a9099f6b0 2024-05-21T15:36:54.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.595+0000 7fca5d5c2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563a90c6c580 con 0x563a909c0390 2024-05-21T15:36:54.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.595+0000 7fca5d5c2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563a90cc0c00 con 0x563a909974d0 2024-05-21T15:36:54.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.596+0000 7fca4ffff640 1 --2- 172.21.15.38:0/103543251 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fca400a2c10 0x7fca400a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.596+0000 7fca4ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a909c0390 0x563a90d5fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.596+0000 7fca4ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a909c0390 0x563a90d5fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46770/0 (socket says 172.21.15.38:46770) 2024-05-21T15:36:54.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.596+0000 7fca4ffff640 1 -- 172.21.15.38:0/168103651 learned_addr learned my addr 172.21.15.38:0/168103651 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:54.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.597+0000 7fca54bda640 1 --2- 172.21.15.38:0/168103651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a9099f6b0 0x563a90d671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.597+0000 7fca4f7fe640 1 --2- 172.21.15.38:0/168103651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909974d0 0x563a90d602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.597+0000 7fca4ffff640 1 --2- 172.21.15.38:0/103543251 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fca400a2c10 0x7fca400a5120 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fca4402aa10 tx=0x7fca440043a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:54.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.598+0000 7fca4ffff640 1 -- 172.21.15.38:0/168103651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909974d0 msgr2=0x563a90d602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:54.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.598+0000 7fca4ffff640 1 --2- 172.21.15.38:0/168103651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909974d0 0x563a90d602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.598+0000 7fca4ffff640 1 -- 172.21.15.38:0/168103651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a9099f6b0 msgr2=0x563a90d671f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:54.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.598+0000 7fca4ffff640 1 --2- 172.21.15.38:0/168103651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a9099f6b0 0x563a90d671f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.598+0000 7fca4ffff640 1 -- 172.21.15.38:0/168103651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a909d83f0 con 0x563a909c0390 2024-05-21T15:36:54.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.598+0000 7fca54bda640 1 --2- 172.21.15.38:0/168103651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a9099f6b0 0x563a90d671f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:54.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.599+0000 7fca4ffff640 1 --2- 172.21.15.38:0/168103651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a909c0390 0x563a90d5fd70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fca44005f70 tx=0x7fca44045000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f408687cd7f3f7e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:54.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.600+0000 7fca3a7fc640 1 -- 172.21.15.38:0/168103651 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4404e070 con 0x563a909c0390 2024-05-21T15:36:54.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.601+0000 7fca3a7fc640 1 -- 172.21.15.38:0/168103651 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca44003de0 con 0x563a909c0390 2024-05-21T15:36:54.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.602+0000 7fca5d5c2740 1 -- 172.21.15.38:0/168103651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a909c0390 msgr2=0x563a90d5fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.602+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/168103651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a909c0390 0x563a90d5fd70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fca44005f70 tx=0x7fca44045000 comp rx=0 tx=0).stop 2024-05-21T15:36:54.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.602+0000 7fca5d5c2740 1 -- 172.21.15.38:0/168103651 shutdown_connections 2024-05-21T15:36:54.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.602+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/168103651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a9099f6b0 0x563a90d671f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.602+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/168103651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909974d0 0x563a90d602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.602+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/168103651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a909c0390 0x563a90d5fd70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.602+0000 7fca5d5c2740 1 -- 172.21.15.38:0/168103651 >> 172.21.15.38:0/168103651 conn(0x563a909b44e0 msgr2=0x563a90d5b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:54.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.606+0000 7fca5d5c2740 1 -- 172.21.15.38:0/168103651 shutdown_connections 2024-05-21T15:36:54.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.606+0000 7fca5d5c2740 1 -- 172.21.15.38:0/168103651 wait complete. 2024-05-21T15:36:54.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.607+0000 7fca5d5c2740 1 -- start start 2024-05-21T15:36:54.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.608+0000 7fca5d5c2740 1 Processor -- start 2024-05-21T15:36:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.608+0000 7fca5d5c2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909974d0 0x563a90d69c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.608+0000 7fca5d5c2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a9099f6b0 0x563a90d6a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.608+0000 7fca5d5c2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909c0390 0x563a90d710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.608+0000 7fca5d5c2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563a90d67c30 con 0x563a909974d0 2024-05-21T15:36:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.608+0000 7fca5d5c2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563a90c7fc40 con 0x563a9099f6b0 2024-05-21T15:36:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.608+0000 7fca5d5c2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563a90d67db0 con 0x563a909c0390 2024-05-21T15:36:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.608+0000 7fca5d5c2740 10 client.0 _ll_register_callbacks cb 0x563a909d7be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.609+0000 7fca4f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a9099f6b0 0x563a90d6a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.609+0000 7fca4f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a9099f6b0 0x563a90d6a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46774/0 (socket says 172.21.15.38:46774) 2024-05-21T15:36:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.609+0000 7fca4f7fe640 1 -- 172.21.15.38:0/1236023998 learned_addr learned my addr 172.21.15.38:0/1236023998 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.609+0000 7fca54bda640 1 --2- 172.21.15.38:0/1236023998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909c0390 0x563a90d710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.609+0000 7fca4ffff640 1 --2- 172.21.15.38:0/1236023998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909974d0 0x563a90d69c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.610+0000 7fca4f7fe640 1 -- 172.21.15.38:0/1236023998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909c0390 msgr2=0x563a90d710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.610+0000 7fca4f7fe640 1 --2- 172.21.15.38:0/1236023998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909c0390 0x563a90d710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.610+0000 7fca4f7fe640 1 -- 172.21.15.38:0/1236023998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909974d0 msgr2=0x563a90d69c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.610+0000 7fca4f7fe640 1 --2- 172.21.15.38:0/1236023998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909974d0 0x563a90d69c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.611+0000 7fca4f7fe640 1 -- 172.21.15.38:0/1236023998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a90aa27d0 con 0x563a9099f6b0 2024-05-21T15:36:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.613+0000 7fca4f7fe640 1 --2- 172.21.15.38:0/1236023998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a9099f6b0 0x563a90d6a140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fca3c003870 tx=0x7fca3c003c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2ae4828f5512f09 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.615+0000 7fca2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.615+0000 7fca2f7fe640 1 -- 172.21.15.38:0/1236023998 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c003e60 con 0x563a9099f6b0 2024-05-21T15:36:54.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.615+0000 7fca2f7fe640 1 -- 172.21.15.38:0/1236023998 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fca3c01a470 con 0x563a9099f6b0 2024-05-21T15:36:54.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.615+0000 7fca5d5c2740 1 -- 172.21.15.38:0/1236023998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca44003de0 con 0x563a9099f6b0 2024-05-21T15:36:54.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.625+0000 7fca2f7fe640 1 -- 172.21.15.38:0/1236023998 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca3c01a670 con 0x563a9099f6b0 2024-05-21T15:36:54.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.625+0000 7fca5d5c2740 10 client.25826 fetch_fsmap learned FSMap version 30 2024-05-21T15:36:54.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.625+0000 7fca5d5c2740 1 -- 172.21.15.38:0/1236023998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a90d74460 con 0x563a9099f6b0 2024-05-21T15:36:54.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.628+0000 7fca2f7fe640 1 -- 172.21.15.38:0/1236023998 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca44003de0 con 0x563a9099f6b0 2024-05-21T15:36:54.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.628+0000 7fca2f7fe640 1 -- 172.21.15.38:0/1236023998 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fca3c011b40 con 0x563a9099f6b0 2024-05-21T15:36:54.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.629+0000 7fca5d5c2740 10 client.25826 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:36:54.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.629+0000 7fca5d5c2740 10 client.25826 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:36:54.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.629+0000 7fca5d5c2740 20 client.25826 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:54.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.629+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/1236023998 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x563a9097ef60 0x563a90d77b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.629+0000 7fca5d5c2740 4 client.25826 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:54.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.629+0000 7fca5d5c2740 1 -- 172.21.15.38:0/1236023998 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a90cd2370 con 0x563a9097ef60 2024-05-21T15:36:54.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.630+0000 7fca4ffff640 1 --2- 172.21.15.38:0/1236023998 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x563a9097ef60 0x563a90d77b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.631+0000 7fca4ffff640 1 --2- 172.21.15.38:0/1236023998 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x563a9097ef60 0x563a90d77b80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=103dd5b2563a6f69 server_cookie=17171a5f7f2cd78 in_seq=0 out_seq=0 2024-05-21T15:36:54.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.631+0000 7fca2f7fe640 10 client.25826 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:36:54.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.633+0000 7fca2f7fe640 1 -- 172.21.15.38:0/1236023998 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563a90cd2370 con 0x563a9097ef60 2024-05-21T15:36:54.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.633+0000 7fca2f7fe640 10 client.25826 handle_command_reply: tid=0 2024-05-21T15:36:54.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.633+0000 7fca5d5c2740 1 client.25826 shutdown 2024-05-21T15:36:54.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.637+0000 7fca5d5c2740 1 -- 172.21.15.38:0/1236023998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a9099f6b0 msgr2=0x563a90d6a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.637+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/1236023998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a9099f6b0 0x563a90d6a140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fca3c003870 tx=0x7fca3c003c70 comp rx=0 tx=0).stop 2024-05-21T15:36:54.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.641+0000 7fca5d5c2740 1 -- 172.21.15.38:0/1236023998 shutdown_connections 2024-05-21T15:36:54.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.641+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/1236023998 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x563a9097ef60 0x563a90d77b80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.641+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/1236023998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909c0390 0x563a90d710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.641+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/1236023998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a9099f6b0 0x563a90d6a140 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.641+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/1236023998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909974d0 0x563a90d69c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.641+0000 7fca2f7fe640 0 client.25826 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:36:54.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.641+0000 7fca5d5c2740 1 -- 172.21.15.38:0/1236023998 >> 172.21.15.38:0/1236023998 conn(0x563a909b44e0 msgr2=0x563a90d5a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:54.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.641+0000 7fca5d5c2740 1 -- 172.21.15.38:0/1236023998 shutdown_connections 2024-05-21T15:36:54.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.642+0000 7fca5d5c2740 1 -- 172.21.15.38:0/1236023998 wait complete. 2024-05-21T15:36:54.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.644+0000 7fca5d5c2740 20 client.25826 trim_cache size 0 max 16384 2024-05-21T15:36:54.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.674+0000 7fca5d5c2740 1 Processor -- start 2024-05-21T15:36:54.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.674+0000 7fca5d5c2740 1 -- start start 2024-05-21T15:36:54.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.675+0000 7fca5d5c2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909974d0 0x563a90d63f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.675+0000 7fca5d5c2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a9099f6b0 0x563a90d644d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.675+0000 7fca5d5c2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909c0390 0x563a90d70fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.675+0000 7fca5d5c2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fca3400e710 con 0x563a909974d0 2024-05-21T15:36:54.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.675+0000 7fca5d5c2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fca34010310 con 0x563a9099f6b0 2024-05-21T15:36:54.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.675+0000 7fca5d5c2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fca3400ede0 con 0x563a909c0390 2024-05-21T15:36:54.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.676+0000 7fca4ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909974d0 0x563a90d63f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.676+0000 7fca4ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909974d0 0x563a90d63f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49464/0 (socket says 172.21.15.38:49464) 2024-05-21T15:36:54.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.676+0000 7fca4ffff640 1 -- 172.21.15.38:0/1441194743 learned_addr learned my addr 172.21.15.38:0/1441194743 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:54.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.676+0000 7fca54bda640 1 --2- 172.21.15.38:0/1441194743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909c0390 0x563a90d70fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.676+0000 7fca4f7fe640 1 --2- 172.21.15.38:0/1441194743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a9099f6b0 0x563a90d644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.676+0000 7fca54bda640 1 -- 172.21.15.38:0/1441194743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a9099f6b0 msgr2=0x563a90d644d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:54.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.676+0000 7fca54bda640 1 --2- 172.21.15.38:0/1441194743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a9099f6b0 0x563a90d644d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.676+0000 7fca54bda640 1 -- 172.21.15.38:0/1441194743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909974d0 msgr2=0x563a90d63f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:54.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.676+0000 7fca54bda640 1 --2- 172.21.15.38:0/1441194743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909974d0 0x563a90d63f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.677+0000 7fca54bda640 1 -- 172.21.15.38:0/1441194743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a90d71500 con 0x563a909c0390 2024-05-21T15:36:54.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.677+0000 7fca54bda640 1 --2- 172.21.15.38:0/1441194743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909c0390 0x563a90d70fc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca500fe830 tx=0x7fca50078040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11de9c96147a5957 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:54.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.677+0000 7fca39ffb640 1 -- 172.21.15.38:0/1441194743 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5011c070 con 0x563a909c0390 2024-05-21T15:36:54.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.677+0000 7fca39ffb640 1 -- 172.21.15.38:0/1441194743 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca50102ee0 con 0x563a909c0390 2024-05-21T15:36:54.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.677+0000 7fca39ffb640 1 -- 172.21.15.38:0/1441194743 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50117030 con 0x563a909c0390 2024-05-21T15:36:54.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.678+0000 7fca5d5c2740 1 -- 172.21.15.38:0/1441194743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909c0390 msgr2=0x563a90d70fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.678+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/1441194743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909c0390 0x563a90d70fc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca500fe830 tx=0x7fca50078040 comp rx=0 tx=0).stop 2024-05-21T15:36:54.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.679+0000 7fca5d5c2740 1 -- 172.21.15.38:0/1441194743 shutdown_connections 2024-05-21T15:36:54.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.679+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/1441194743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a909c0390 0x563a90d70fc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.679+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/1441194743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a9099f6b0 0x563a90d644d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.679+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/1441194743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909974d0 0x563a90d63f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.679+0000 7fca5d5c2740 1 -- 172.21.15.38:0/1441194743 >> 172.21.15.38:0/1441194743 conn(0x563a909b44e0 msgr2=0x563a90d68230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:54.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.681+0000 7fca5d5c2740 1 -- 172.21.15.38:0/1441194743 shutdown_connections 2024-05-21T15:36:54.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.682+0000 7fca5d5c2740 1 -- 172.21.15.38:0/1441194743 wait complete. 2024-05-21T15:36:54.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.683+0000 7fca5d5c2740 1 -- start start 2024-05-21T15:36:54.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.684+0000 7fca5d5c2740 1 Processor -- start 2024-05-21T15:36:54.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.684+0000 7fca5d5c2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a909974d0 0x563a90d6dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.685+0000 7fca5d5c2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a9099f6b0 0x563a90d6e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.685+0000 7fca5d5c2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909c0390 0x563a90dddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.685+0000 7fca4ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a909974d0 0x563a90d6dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.685+0000 7fca4ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a909974d0 0x563a90d6dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46798/0 (socket says 172.21.15.38:46798) 2024-05-21T15:36:54.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.685+0000 7fca4f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a9099f6b0 0x563a90d6e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.685+0000 7fca4f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a9099f6b0 0x563a90d6e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:52166/0 (socket says 172.21.15.38:52166) 2024-05-21T15:36:54.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.686+0000 7fca4ffff640 1 -- 172.21.15.38:0/3905509466 learned_addr learned my addr 172.21.15.38:0/3905509466 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:54.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.686+0000 7fca54bda640 1 --2- 172.21.15.38:0/3905509466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909c0390 0x563a90dddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.686+0000 7fca5d5c2740 1 -- 172.21.15.38:0/3905509466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fca3400fbb0 con 0x563a909c0390 2024-05-21T15:36:54.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.686+0000 7fca5d5c2740 1 -- 172.21.15.38:0/3905509466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fca3400f4b0 con 0x563a909974d0 2024-05-21T15:36:54.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.686+0000 7fca5d5c2740 1 -- 172.21.15.38:0/3905509466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fca3400c6f0 con 0x563a9099f6b0 2024-05-21T15:36:54.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.686+0000 7fca5d5c2740 10 client.0 _ll_register_callbacks cb 0x7fca3c0040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:54.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.687+0000 7fca54bda640 1 -- 172.21.15.38:0/3905509466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a9099f6b0 msgr2=0x563a90d6e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:54.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.687+0000 7fca54bda640 1 --2- 172.21.15.38:0/3905509466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a9099f6b0 0x563a90d6e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.687+0000 7fca54bda640 1 -- 172.21.15.38:0/3905509466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a909974d0 msgr2=0x563a90d6dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:54.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.687+0000 7fca54bda640 1 --2- 172.21.15.38:0/3905509466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a909974d0 0x563a90d6dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.687+0000 7fca54bda640 1 -- 172.21.15.38:0/3905509466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a90d6e8c0 con 0x563a909c0390 2024-05-21T15:36:54.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.688+0000 7fca54bda640 1 --2- 172.21.15.38:0/3905509466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909c0390 0x563a90dddf20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fca500fd080 tx=0x7fca50067cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb325c8182208be server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:54.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.689+0000 7fca2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:54.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.689+0000 7fca2f7fe640 1 -- 172.21.15.38:0/3905509466 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca500e2020 con 0x563a909c0390 2024-05-21T15:36:54.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.689+0000 7fca2f7fe640 1 -- 172.21.15.38:0/3905509466 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fca50076ab0 con 0x563a909c0390 2024-05-21T15:36:54.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.690+0000 7fca5d5c2740 1 -- 172.21.15.38:0/3905509466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca50102ee0 con 0x563a909c0390 2024-05-21T15:36:54.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.690+0000 7fca2f7fe640 1 -- 172.21.15.38:0/3905509466 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca5011c070 con 0x563a909c0390 2024-05-21T15:36:54.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.690+0000 7fca5d5c2740 10 client.16161 fetch_fsmap learned FSMap version 30 2024-05-21T15:36:54.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.690+0000 7fca5d5c2740 1 -- 172.21.15.38:0/3905509466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a90ddf430 con 0x563a909c0390 2024-05-21T15:36:54.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.691+0000 7fca2f7fe640 1 -- 172.21.15.38:0/3905509466 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5011d2b0 con 0x563a909c0390 2024-05-21T15:36:54.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.691+0000 7fca2f7fe640 1 -- 172.21.15.38:0/3905509466 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fca500fdff0 con 0x563a909c0390 2024-05-21T15:36:54.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.691+0000 7fca5d5c2740 10 client.16161 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:36:54.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.692+0000 7fca5d5c2740 10 client.16161 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:36:54.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.692+0000 7fca5d5c2740 20 client.16161 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:54.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.692+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/3905509466 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x563a909543a0 0x563a90de1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.692+0000 7fca5d5c2740 4 client.16161 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:36:54.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.692+0000 7fca5d5c2740 1 -- 172.21.15.38:0/3905509466 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fca50102ee0 con 0x563a909543a0 2024-05-21T15:36:54.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.695+0000 7fca4ffff640 1 --2- 172.21.15.38:0/3905509466 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x563a909543a0 0x563a90de1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.697+0000 7fca2f7fe640 10 client.16161 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:36:54.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.697+0000 7fca4ffff640 1 --2- 172.21.15.38:0/3905509466 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x563a909543a0 0x563a90de1d30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a05c31cb0823f75c server_cookie=f046050e48d7aa4a in_seq=0 out_seq=0 2024-05-21T15:36:54.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.699+0000 7fca2f7fe640 1 -- 172.21.15.38:0/3905509466 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fca50102ee0 con 0x563a909543a0 2024-05-21T15:36:54.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.699+0000 7fca2f7fe640 10 client.16161 handle_command_reply: tid=0 2024-05-21T15:36:54.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.700+0000 7fca5d5c2740 1 client.16161 shutdown 2024-05-21T15:36:54.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:54 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/775932706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:36:54.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.706+0000 7fca5d5c2740 1 -- 172.21.15.38:0/3905509466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909c0390 msgr2=0x563a90dddf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.706+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/3905509466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909c0390 0x563a90dddf20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fca500fd080 tx=0x7fca50067cf0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.706+0000 7fca5d5c2740 1 -- 172.21.15.38:0/3905509466 shutdown_connections 2024-05-21T15:36:54.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.706+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/3905509466 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x563a909543a0 0x563a90de1d30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.706+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/3905509466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563a909c0390 0x563a90dddf20 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.706+0000 7fca2f7fe640 0 client.16161 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:36:54.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.707+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/3905509466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a9099f6b0 0x563a90d6e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.707+0000 7fca5d5c2740 1 --2- 172.21.15.38:0/3905509466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a909974d0 0x563a90d6dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.707+0000 7fca5d5c2740 1 -- 172.21.15.38:0/3905509466 >> 172.21.15.38:0/3905509466 conn(0x563a909b44e0 msgr2=0x563a90d68230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:54.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.709+0000 7fca5d5c2740 1 -- 172.21.15.38:0/3905509466 shutdown_connections 2024-05-21T15:36:54.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.709+0000 7fca5d5c2740 1 -- 172.21.15.38:0/3905509466 wait complete. 2024-05-21T15:36:54.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.710+0000 7fca5d5c2740 20 client.16161 trim_cache size 0 max 16384 2024-05-21T15:36:54.712 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:36:54.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.714+0000 7fca553db640 1 -- 172.21.15.38:0/103543251 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fca400a2c10 msgr2=0x7fca400a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.714+0000 7fca553db640 1 --2- 172.21.15.38:0/103543251 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fca400a2c10 0x7fca400a5120 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fca4402aa10 tx=0x7fca440043a0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.714+0000 7fca553db640 1 -- 172.21.15.38:0/103543251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca480ab760 msgr2=0x7fca480b5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.714+0000 7fca553db640 1 --2- 172.21.15.38:0/103543251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca480ab760 0x7fca480b5570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca50050990 tx=0x7fca50070570 comp rx=0 tx=0).stop 2024-05-21T15:36:54.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.716+0000 7fca553db640 1 -- 172.21.15.38:0/103543251 shutdown_connections 2024-05-21T15:36:54.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.716+0000 7fca553db640 1 --2- 172.21.15.38:0/103543251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca480ab760 0x7fca480b5570 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.716+0000 7fca553db640 1 --2- 172.21.15.38:0/103543251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca480a6a40 0x7fca480b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.716+0000 7fca553db640 1 --2- 172.21.15.38:0/103543251 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fca400a2c10 0x7fca400a5120 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.716+0000 7fca553db640 1 --2- 172.21.15.38:0/103543251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca480a5940 0x7fca480b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.716+0000 7fca553db640 1 -- 172.21.15.38:0/103543251 >> 172.21.15.38:0/103543251 conn(0x7fca480a09b0 msgr2=0x7fca480a2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:54.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.717+0000 7fca553db640 1 -- 172.21.15.38:0/103543251 shutdown_connections 2024-05-21T15:36:54.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.717+0000 7fca553db640 1 -- 172.21.15.38:0/103543251 wait complete. 2024-05-21T15:36:54.738 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:54 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/775932706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:36:54.739 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:36:54.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:54 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/775932706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:36:54.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.976+0000 7f94970a2640 1 -- 172.21.15.38:0/2615588107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9490106630 msgr2=0x7f9490106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.976+0000 7f94970a2640 1 --2- 172.21.15.38:0/2615588107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9490106630 0x7f9490106a40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9488009ca0 tx=0x7f948802a8f0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.976+0000 7f94970a2640 1 -- 172.21.15.38:0/2615588107 shutdown_connections 2024-05-21T15:36:54.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.976+0000 7f94970a2640 1 --2- 172.21.15.38:0/2615588107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9490106630 0x7f9490106a40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.976+0000 7f94970a2640 1 --2- 172.21.15.38:0/2615588107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9490103cd0 0x7f94901060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.976+0000 7f94970a2640 1 --2- 172.21.15.38:0/2615588107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9490101370 0x7f9490103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.976+0000 7f94970a2640 1 -- 172.21.15.38:0/2615588107 >> 172.21.15.38:0/2615588107 conn(0x7f94900fcb80 msgr2=0x7f94900ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:54.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.979+0000 7f94970a2640 1 -- 172.21.15.38:0/2615588107 shutdown_connections 2024-05-21T15:36:54.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.980+0000 7f94970a2640 1 -- 172.21.15.38:0/2615588107 wait complete. 2024-05-21T15:36:54.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.981+0000 7f94970a2640 1 Processor -- start 2024-05-21T15:36:54.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.982+0000 7f94970a2640 1 -- start start 2024-05-21T15:36:54.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.982+0000 7f94970a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9490101370 0x7f949019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.982+0000 7f94970a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9490103cd0 0x7f949019ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.982+0000 7f94970a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9490106630 0x7f94901a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.982+0000 7f94970a2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9490114e50 con 0x7f9490103cd0 2024-05-21T15:36:54.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.982+0000 7f94970a2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9490114cd0 con 0x7f9490106630 2024-05-21T15:36:54.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.982+0000 7f94970a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9490114fd0 con 0x7f9490101370 2024-05-21T15:36:54.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.982+0000 7f949563f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9490106630 0x7f94901a6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.982+0000 7f948ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9490103cd0 0x7f949019ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.983+0000 7f948ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9490103cd0 0x7f949019ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44528/0 (socket says 172.21.15.38:44528) 2024-05-21T15:36:54.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.983+0000 7f948ffff640 1 -- 172.21.15.38:0/3834547048 learned_addr learned my addr 172.21.15.38:0/3834547048 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:54.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.983+0000 7f9494e3e640 1 --2- 172.21.15.38:0/3834547048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9490101370 0x7f949019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.983+0000 7f948ffff640 1 -- 172.21.15.38:0/3834547048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9490101370 msgr2=0x7f949019f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.983+0000 7f948ffff640 1 --2- 172.21.15.38:0/3834547048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9490101370 0x7f949019f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.983+0000 7f948ffff640 1 -- 172.21.15.38:0/3834547048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9490106630 msgr2=0x7f94901a6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:54.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.983+0000 7f948ffff640 1 --2- 172.21.15.38:0/3834547048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9490106630 0x7f94901a6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.983+0000 7f948ffff640 1 -- 172.21.15.38:0/3834547048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949006b6a0 con 0x7f9490103cd0 2024-05-21T15:36:54.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.983+0000 7f949563f640 1 --2- 172.21.15.38:0/3834547048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9490106630 0x7f94901a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:36:54.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.984+0000 7f948ffff640 1 --2- 172.21.15.38:0/3834547048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9490103cd0 0x7f949019ff00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f948400b450 tx=0x7f948400b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:54.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.986+0000 7f948dffb640 1 -- 172.21.15.38:0/3834547048 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484015020 con 0x7f9490103cd0 2024-05-21T15:36:54.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.986+0000 7f948dffb640 1 -- 172.21.15.38:0/3834547048 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9484011d10 con 0x7f9490103cd0 2024-05-21T15:36:54.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.986+0000 7f948dffb640 1 -- 172.21.15.38:0/3834547048 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948400f2e0 con 0x7f9490103cd0 2024-05-21T15:36:54.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.987+0000 7f94970a2640 1 -- 172.21.15.38:0/3834547048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f949006b990 con 0x7f9490103cd0 2024-05-21T15:36:54.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.987+0000 7f94970a2640 1 -- 172.21.15.38:0/3834547048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f949006bf10 con 0x7f9490103cd0 2024-05-21T15:36:54.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.994+0000 7f949e028740 1 Processor -- start 2024-05-21T15:36:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.997+0000 7f949e028740 1 -- start start 2024-05-21T15:36:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.998+0000 7f949e028740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab98266b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.998+0000 7f949e028740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98267080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.998+0000 7f949e028740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab9826df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.998+0000 7f949e028740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ab981cbc40 con 0x55ab97e9e4d0 2024-05-21T15:36:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.998+0000 7f949e028740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ab98173580 con 0x55ab97ec7390 2024-05-21T15:36:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.998+0000 7f949e028740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ab981c7c00 con 0x55ab97ea66b0 2024-05-21T15:36:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.998+0000 7f949563f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab98266b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.999+0000 7f949563f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab98266b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53086/0 (socket says 172.21.15.38:53086) 2024-05-21T15:36:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.999+0000 7f949563f640 1 -- 172.21.15.38:0/54341294 learned_addr learned my addr 172.21.15.38:0/54341294 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.999+0000 7f9494e3e640 1 --2- 172.21.15.38:0/54341294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98267080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.999+0000 7f949563f640 1 -- 172.21.15.38:0/54341294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 msgr2=0x55ab9826df80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:36:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.999+0000 7f949563f640 1 --2- 172.21.15.38:0/54341294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab9826df80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.999+0000 7f949563f640 1 -- 172.21.15.38:0/54341294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 msgr2=0x55ab98267080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.999+0000 7f949563f640 1 --2- 172.21.15.38:0/54341294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98267080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:54.999+0000 7f949563f640 1 -- 172.21.15.38:0/54341294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab97edf3f0 con 0x55ab97ec7390 2024-05-21T15:36:54.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.000+0000 7f948dffb640 1 -- 172.21.15.38:0/3834547048 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9484018020 con 0x7f9490103cd0 2024-05-21T15:36:54.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.000+0000 7f949563f640 1 --2- 172.21.15.38:0/54341294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab98266b00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9488035000 tx=0x7f94880306a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81db3ce9398d2718 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:54.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.001+0000 7f948dffb640 1 --2- 172.21.15.38:0/3834547048 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f94780a2ce0 0x7f94780a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.001+0000 7f948dffb640 1 -- 172.21.15.38:0/3834547048 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f94840ce840 con 0x7f9490103cd0 2024-05-21T15:36:55.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.002+0000 7f9494e3e640 1 --2- 172.21.15.38:0/3834547048 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f94780a2ce0 0x7f94780a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.003+0000 7f9476ffd640 1 -- 172.21.15.38:0/54341294 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488031680 con 0x55ab97ec7390 2024-05-21T15:36:55.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.003+0000 7f9476ffd640 1 -- 172.21.15.38:0/54341294 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9488036040 con 0x55ab97ec7390 2024-05-21T15:36:55.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.003+0000 7f9476ffd640 1 -- 172.21.15.38:0/54341294 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94880042a0 con 0x55ab97ec7390 2024-05-21T15:36:55.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.004+0000 7f949e028740 1 -- 172.21.15.38:0/54341294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 msgr2=0x55ab98266b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.004+0000 7f949e028740 1 --2- 172.21.15.38:0/54341294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab98266b00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9488035000 tx=0x7f94880306a0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.004+0000 7f9494e3e640 1 --2- 172.21.15.38:0/3834547048 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f94780a2ce0 0x7f94780a51f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55ab98267ee0 tx=0x7f9480009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.005+0000 7f949e028740 1 -- 172.21.15.38:0/54341294 shutdown_connections 2024-05-21T15:36:55.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.005+0000 7f949e028740 1 --2- 172.21.15.38:0/54341294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab9826df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.005+0000 7f949e028740 1 --2- 172.21.15.38:0/54341294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98267080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.005+0000 7f949e028740 1 --2- 172.21.15.38:0/54341294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab98266b00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.005+0000 7f949e028740 1 -- 172.21.15.38:0/54341294 >> 172.21.15.38:0/54341294 conn(0x55ab97ebb4e0 msgr2=0x55ab98262490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.006+0000 7f949e028740 1 -- 172.21.15.38:0/54341294 shutdown_connections 2024-05-21T15:36:55.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.006+0000 7f949e028740 1 -- 172.21.15.38:0/54341294 wait complete. 2024-05-21T15:36:55.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.006+0000 7f949e028740 1 -- start start 2024-05-21T15:36:55.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.007+0000 7f949e028740 1 Processor -- start 2024-05-21T15:36:55.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.007+0000 7f949e028740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98281ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.007+0000 7f949e028740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab98282000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.007+0000 7f949e028740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab98288f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.007+0000 7f949e028740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ab9827f980 con 0x55ab97e9e4d0 2024-05-21T15:36:55.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.007+0000 7f949e028740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ab98186c40 con 0x55ab97ec7390 2024-05-21T15:36:55.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.007+0000 7f949e028740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ab9827fb00 con 0x55ab97ea66b0 2024-05-21T15:36:55.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.007+0000 7f949e028740 10 client.0 _ll_register_callbacks cb 0x55ab97edebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:55.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.009+0000 7f9494e3e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98281ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.009+0000 7f9494e3e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98281ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44548/0 (socket says 172.21.15.38:44548) 2024-05-21T15:36:55.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.010+0000 7f9494e3e640 1 -- 172.21.15.38:0/441865608 learned_addr learned my addr 172.21.15.38:0/441865608 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:55.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.010+0000 7f949563f640 1 --2- 172.21.15.38:0/441865608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab98288f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.010+0000 7f948ffff640 1 --2- 172.21.15.38:0/441865608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab98282000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.013+0000 7f9494e3e640 1 -- 172.21.15.38:0/441865608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 msgr2=0x55ab98282000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.013+0000 7f9494e3e640 1 --2- 172.21.15.38:0/441865608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab98282000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.013+0000 7f9494e3e640 1 -- 172.21.15.38:0/441865608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 msgr2=0x55ab98288f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.013+0000 7f9494e3e640 1 --2- 172.21.15.38:0/441865608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab98288f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.013+0000 7f9494e3e640 1 -- 172.21.15.38:0/441865608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab97fa97d0 con 0x55ab97e9e4d0 2024-05-21T15:36:55.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.015+0000 7f9494e3e640 1 --2- 172.21.15.38:0/441865608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98281ac0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f94800094c0 tx=0x7f94800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea1f812dcc73568 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.015+0000 7f948ffff640 1 --2- 172.21.15.38:0/441865608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab98282000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:36:55.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.016+0000 7f946ffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:55.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.018+0000 7f946ffff640 1 -- 172.21.15.38:0/441865608 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480026020 con 0x55ab97e9e4d0 2024-05-21T15:36:55.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.019+0000 7f949e028740 1 -- 172.21.15.38:0/441865608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9488036040 con 0x55ab97e9e4d0 2024-05-21T15:36:55.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.019+0000 7f946ffff640 1 -- 172.21.15.38:0/441865608 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f94800223f0 con 0x55ab97e9e4d0 2024-05-21T15:36:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.021+0000 7f946ffff640 1 -- 172.21.15.38:0/441865608 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f948001e040 con 0x55ab97e9e4d0 2024-05-21T15:36:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.021+0000 7f949e028740 10 client.16170 fetch_fsmap learned FSMap version 30 2024-05-21T15:36:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.021+0000 7f949e028740 1 -- 172.21.15.38:0/441865608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ab9828c2f0 con 0x55ab97e9e4d0 2024-05-21T15:36:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.022+0000 7f946ffff640 1 -- 172.21.15.38:0/441865608 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9488036040 con 0x55ab97e9e4d0 2024-05-21T15:36:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.022+0000 7f946ffff640 1 -- 172.21.15.38:0/441865608 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9480021030 con 0x55ab97e9e4d0 2024-05-21T15:36:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.022+0000 7f949e028740 10 client.16170 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:36:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.022+0000 7f949e028740 10 client.16170 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:36:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.022+0000 7f949e028740 20 client.16170 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.022+0000 7f949e028740 1 --2- 172.21.15.38:0/441865608 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55ab97e85f60 0x55ab9828fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.022+0000 7f949e028740 4 client.16170 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.022+0000 7f949e028740 1 -- 172.21.15.38:0/441865608 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab981d9370 con 0x55ab97e85f60 2024-05-21T15:36:55.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.025+0000 7f948ffff640 1 --2- 172.21.15.38:0/441865608 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55ab97e85f60 0x55ab9828fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.035+0000 7f948ffff640 1 --2- 172.21.15.38:0/441865608 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55ab97e85f60 0x55ab9828fa60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1670b77a2a7ba22 server_cookie=88386de0c08bfbaa in_seq=0 out_seq=0 2024-05-21T15:36:55.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.035+0000 7f946ffff640 10 client.16170 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:36:55.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.038+0000 7f946ffff640 1 -- 172.21.15.38:0/441865608 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ab981d9370 con 0x55ab97e85f60 2024-05-21T15:36:55.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.038+0000 7f946ffff640 10 client.16170 handle_command_reply: tid=0 2024-05-21T15:36:55.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.040+0000 7f949e028740 1 client.16170 shutdown 2024-05-21T15:36:55.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.041+0000 7f949e028740 1 -- 172.21.15.38:0/441865608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 msgr2=0x55ab98281ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.041+0000 7f949e028740 1 --2- 172.21.15.38:0/441865608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98281ac0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f94800094c0 tx=0x7f94800046c0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.043+0000 7f949e028740 1 -- 172.21.15.38:0/441865608 shutdown_connections 2024-05-21T15:36:55.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.043+0000 7f949e028740 1 --2- 172.21.15.38:0/441865608 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55ab97e85f60 0x55ab9828fa60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.043+0000 7f949e028740 1 --2- 172.21.15.38:0/441865608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab98288f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.043+0000 7f949e028740 1 --2- 172.21.15.38:0/441865608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab98282000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.043+0000 7f949e028740 1 --2- 172.21.15.38:0/441865608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98281ac0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.043+0000 7f949e028740 1 -- 172.21.15.38:0/441865608 >> 172.21.15.38:0/441865608 conn(0x55ab97ebb4e0 msgr2=0x55ab98261570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.043+0000 7f946ffff640 0 client.16170 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:36:55.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.047+0000 7f949e028740 1 -- 172.21.15.38:0/441865608 shutdown_connections 2024-05-21T15:36:55.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.047+0000 7f949e028740 1 -- 172.21.15.38:0/441865608 wait complete. 2024-05-21T15:36:55.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.049+0000 7f949e028740 20 client.16170 trim_cache size 0 max 16384 2024-05-21T15:36:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.080+0000 7f949e028740 1 Processor -- start 2024-05-21T15:36:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.081+0000 7f949e028740 1 -- start start 2024-05-21T15:36:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.081+0000 7f949e028740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97e9e4d0 0x55ab98281da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.081+0000 7f949e028740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab98288ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.081+0000 7f949e028740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97ec7390 0x55ab98294210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.081+0000 7f949e028740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94901087d0 con 0x55ab97ec7390 2024-05-21T15:36:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.081+0000 7f949e028740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f949010a3d0 con 0x55ab97e9e4d0 2024-05-21T15:36:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.081+0000 7f949e028740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9490108ea0 con 0x55ab97ea66b0 2024-05-21T15:36:55.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.082+0000 7f949563f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97ec7390 0x55ab98294210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.082+0000 7f949563f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97ec7390 0x55ab98294210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44556/0 (socket says 172.21.15.38:44556) 2024-05-21T15:36:55.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.082+0000 7f949563f640 1 -- 172.21.15.38:0/642734023 learned_addr learned my addr 172.21.15.38:0/642734023 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:55.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.082+0000 7f9494e3e640 1 --2- 172.21.15.38:0/642734023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97e9e4d0 0x55ab98281da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.083+0000 7f949563f640 1 -- 172.21.15.38:0/642734023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 msgr2=0x55ab98288ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.083+0000 7f949563f640 1 --2- 172.21.15.38:0/642734023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab98288ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.083+0000 7f949563f640 1 -- 172.21.15.38:0/642734023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97e9e4d0 msgr2=0x55ab98281da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.083+0000 7f949563f640 1 --2- 172.21.15.38:0/642734023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97e9e4d0 0x55ab98281da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.083+0000 7f949563f640 1 -- 172.21.15.38:0/642734023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab9827fed0 con 0x55ab97ec7390 2024-05-21T15:36:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.083+0000 7f949563f640 1 --2- 172.21.15.38:0/642734023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97ec7390 0x55ab98294210 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9488009ca0 tx=0x7f9488005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd22e74ca8766485 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.084+0000 7f94767fc640 1 -- 172.21.15.38:0/642734023 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488009780 con 0x55ab97ec7390 2024-05-21T15:36:55.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.085+0000 7f94767fc640 1 -- 172.21.15.38:0/642734023 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9488052070 con 0x55ab97ec7390 2024-05-21T15:36:55.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.086+0000 7f949e028740 1 -- 172.21.15.38:0/642734023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97ec7390 msgr2=0x55ab98294210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.086+0000 7f949e028740 1 --2- 172.21.15.38:0/642734023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97ec7390 0x55ab98294210 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9488009ca0 tx=0x7f9488005c70 comp rx=0 tx=0).stop 2024-05-21T15:36:55.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.087+0000 7f949e028740 1 -- 172.21.15.38:0/642734023 shutdown_connections 2024-05-21T15:36:55.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.087+0000 7f949e028740 1 --2- 172.21.15.38:0/642734023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97ec7390 0x55ab98294210 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.087+0000 7f949e028740 1 --2- 172.21.15.38:0/642734023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab98288ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.087+0000 7f949e028740 1 --2- 172.21.15.38:0/642734023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97e9e4d0 0x55ab98281da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.087+0000 7f949e028740 1 -- 172.21.15.38:0/642734023 >> 172.21.15.38:0/642734023 conn(0x55ab97ebb4e0 msgr2=0x55ab98281690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.087+0000 7f949e028740 1 -- 172.21.15.38:0/642734023 shutdown_connections 2024-05-21T15:36:55.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.087+0000 7f949e028740 1 -- 172.21.15.38:0/642734023 wait complete. 2024-05-21T15:36:55.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.088+0000 7f949e028740 1 -- start start 2024-05-21T15:36:55.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.088+0000 7f949e028740 1 Processor -- start 2024-05-21T15:36:55.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.089+0000 7f949e028740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98287820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.089+0000 7f949e028740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab98287d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.089+0000 7f949e028740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab982882a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.089+0000 7f949e028740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9490109c70 con 0x55ab97e9e4d0 2024-05-21T15:36:55.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.089+0000 7f949e028740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9490109570 con 0x55ab97ec7390 2024-05-21T15:36:55.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.089+0000 7f949e028740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9490072de0 con 0x55ab97ea66b0 2024-05-21T15:36:55.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.089+0000 7f949e028740 10 client.0 _ll_register_callbacks cb 0x7f94800097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.090+0000 7f949563f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab982882a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.090+0000 7f949563f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab982882a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53120/0 (socket says 172.21.15.38:53120) 2024-05-21T15:36:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.090+0000 7f949563f640 1 -- 172.21.15.38:0/50281730 learned_addr learned my addr 172.21.15.38:0/50281730 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.090+0000 7f9494e3e640 1 --2- 172.21.15.38:0/50281730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98287820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.090+0000 7f948ffff640 1 --2- 172.21.15.38:0/50281730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab98287d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.091+0000 7f9494e3e640 1 -- 172.21.15.38:0/50281730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 msgr2=0x55ab98287d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.091+0000 7f9494e3e640 1 --2- 172.21.15.38:0/50281730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab98287d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.091+0000 7f9494e3e640 1 -- 172.21.15.38:0/50281730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 msgr2=0x55ab982882a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.091+0000 7f9494e3e640 1 --2- 172.21.15.38:0/50281730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab982882a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.091+0000 7f9494e3e640 1 -- 172.21.15.38:0/50281730 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab982f5e70 con 0x55ab97e9e4d0 2024-05-21T15:36:55.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.091+0000 7f9494e3e640 1 --2- 172.21.15.38:0/50281730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98287820 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9480009490 tx=0x7f948002ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15757e1153ae0d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.091+0000 7f946ffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:55.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.092+0000 7f946ffff640 1 -- 172.21.15.38:0/50281730 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480026040 con 0x55ab97e9e4d0 2024-05-21T15:36:55.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.092+0000 7f949e028740 1 -- 172.21.15.38:0/50281730 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9488052070 con 0x55ab97e9e4d0 2024-05-21T15:36:55.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.092+0000 7f946ffff640 1 -- 172.21.15.38:0/50281730 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9480005dd0 con 0x55ab97e9e4d0 2024-05-21T15:36:55.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.092+0000 7f946ffff640 1 -- 172.21.15.38:0/50281730 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9480006040 con 0x55ab97e9e4d0 2024-05-21T15:36:55.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.092+0000 7f949e028740 10 client.16176 fetch_fsmap learned FSMap version 30 2024-05-21T15:36:55.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.092+0000 7f949e028740 1 -- 172.21.15.38:0/50281730 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ab982f71c0 con 0x55ab97e9e4d0 2024-05-21T15:36:55.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.093+0000 7f946ffff640 1 -- 172.21.15.38:0/50281730 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9488052070 con 0x55ab97e9e4d0 2024-05-21T15:36:55.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.093+0000 7f946ffff640 1 -- 172.21.15.38:0/50281730 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f948003c030 con 0x55ab97e9e4d0 2024-05-21T15:36:55.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.094+0000 7f949e028740 10 client.16176 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:36:55.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.094+0000 7f949e028740 10 client.16176 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:36:55.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.094+0000 7f949e028740 20 client.16176 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:55.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.094+0000 7f949e028740 1 --2- 172.21.15.38:0/50281730 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55ab97e5b3a0 0x55ab982f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.094+0000 7f949e028740 4 client.16176 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:36:55.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.094+0000 7f949e028740 1 -- 172.21.15.38:0/50281730 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab981d36c0 con 0x55ab97e5b3a0 2024-05-21T15:36:55.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.096+0000 7f948ffff640 1 --2- 172.21.15.38:0/50281730 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55ab97e5b3a0 0x55ab982f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.098+0000 7f948ffff640 1 --2- 172.21.15.38:0/50281730 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55ab97e5b3a0 0x55ab982f9c00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1fe4a4883849cb53 server_cookie=646cad0ba3f89714 in_seq=0 out_seq=0 2024-05-21T15:36:55.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.098+0000 7f946ffff640 10 client.16176 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:36:55.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.098+0000 7f946ffff640 1 -- 172.21.15.38:0/50281730 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab981d36c0 con 0x55ab97e5b3a0 2024-05-21T15:36:55.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.098+0000 7f946ffff640 10 client.16176 handle_command_reply: tid=0 2024-05-21T15:36:55.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.099+0000 7f949e028740 1 client.16176 shutdown 2024-05-21T15:36:55.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.099+0000 7f949e028740 1 -- 172.21.15.38:0/50281730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 msgr2=0x55ab98287820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.099+0000 7f949e028740 1 --2- 172.21.15.38:0/50281730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98287820 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9480009490 tx=0x7f948002ad80 comp rx=0 tx=0).stop 2024-05-21T15:36:55.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.101+0000 7f949e028740 1 -- 172.21.15.38:0/50281730 shutdown_connections 2024-05-21T15:36:55.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.101+0000 7f949e028740 1 --2- 172.21.15.38:0/50281730 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55ab97e5b3a0 0x55ab982f9c00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.101+0000 7f949e028740 1 --2- 172.21.15.38:0/50281730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab97ec7390 0x55ab982882a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.101+0000 7f949e028740 1 --2- 172.21.15.38:0/50281730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab97ea66b0 0x55ab98287d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.101+0000 7f949e028740 1 --2- 172.21.15.38:0/50281730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab97e9e4d0 0x55ab98287820 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.101+0000 7f949e028740 1 -- 172.21.15.38:0/50281730 >> 172.21.15.38:0/50281730 conn(0x55ab97ebb4e0 msgr2=0x55ab982814e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.101+0000 7f946ffff640 0 client.16176 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:36:55.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.104+0000 7f949e028740 1 -- 172.21.15.38:0/50281730 shutdown_connections 2024-05-21T15:36:55.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.105+0000 7f949e028740 1 -- 172.21.15.38:0/50281730 wait complete. 2024-05-21T15:36:55.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.106+0000 7f949e028740 20 client.16176 trim_cache size 0 max 16384 2024-05-21T15:36:55.109 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:36:55.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.113+0000 7f94970a2640 1 -- 172.21.15.38:0/3834547048 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f94780a2ce0 msgr2=0x7f94780a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.113+0000 7f94970a2640 1 --2- 172.21.15.38:0/3834547048 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f94780a2ce0 0x7f94780a51f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55ab98267ee0 tx=0x7f9480009110 comp rx=0 tx=0).stop 2024-05-21T15:36:55.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.113+0000 7f94970a2640 1 -- 172.21.15.38:0/3834547048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9490103cd0 msgr2=0x7f949019ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.113+0000 7f94970a2640 1 --2- 172.21.15.38:0/3834547048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9490103cd0 0x7f949019ff00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f948400b450 tx=0x7f948400b850 comp rx=0 tx=0).stop 2024-05-21T15:36:55.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.118+0000 7f94970a2640 1 -- 172.21.15.38:0/3834547048 shutdown_connections 2024-05-21T15:36:55.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.118+0000 7f94970a2640 1 --2- 172.21.15.38:0/3834547048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9490106630 0x7f94901a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.118+0000 7f94970a2640 1 --2- 172.21.15.38:0/3834547048 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f94780a2ce0 0x7f94780a51f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.118+0000 7f94970a2640 1 --2- 172.21.15.38:0/3834547048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9490103cd0 0x7f949019ff00 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.118+0000 7f94970a2640 1 --2- 172.21.15.38:0/3834547048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9490101370 0x7f949019f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.118+0000 7f94970a2640 1 -- 172.21.15.38:0/3834547048 >> 172.21.15.38:0/3834547048 conn(0x7f94900fcb80 msgr2=0x7f94901021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.121+0000 7f94970a2640 1 -- 172.21.15.38:0/3834547048 shutdown_connections 2024-05-21T15:36:55.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.121+0000 7f94970a2640 1 -- 172.21.15.38:0/3834547048 wait complete. 2024-05-21T15:36:55.141 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:36:55.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.366+0000 7f7ec943d640 1 -- 172.21.15.38:0/939468755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ec4106620 msgr2=0x7f7ec4106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.366+0000 7f7ec943d640 1 --2- 172.21.15.38:0/939468755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ec4106620 0x7f7ec4106a30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009ca0 tx=0x7f7ebc02a8f0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.368+0000 7f7ec943d640 1 -- 172.21.15.38:0/939468755 shutdown_connections 2024-05-21T15:36:55.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.368+0000 7f7ec943d640 1 --2- 172.21.15.38:0/939468755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ec4106620 0x7f7ec4106a30 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.368+0000 7f7ec943d640 1 --2- 172.21.15.38:0/939468755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ec4103cc0 0x7f7ec41060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.368+0000 7f7ec943d640 1 --2- 172.21.15.38:0/939468755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec4101360 0x7f7ec4103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.368+0000 7f7ec943d640 1 -- 172.21.15.38:0/939468755 >> 172.21.15.38:0/939468755 conn(0x7f7ec40fcb70 msgr2=0x7f7ec40ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.372+0000 7f7ec943d640 1 -- 172.21.15.38:0/939468755 shutdown_connections 2024-05-21T15:36:55.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.373+0000 7f7ec943d640 1 -- 172.21.15.38:0/939468755 wait complete. 2024-05-21T15:36:55.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.374+0000 7f7ec943d640 1 Processor -- start 2024-05-21T15:36:55.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.375+0000 7f7ec943d640 1 -- start start 2024-05-21T15:36:55.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.375+0000 7f7ec943d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec4101360 0x7f7ec410cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.375+0000 7f7ec943d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ec4103cc0 0x7f7ec410d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.375+0000 7f7ec943d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ec4106620 0x7f7ec410e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.375+0000 7f7ec943d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec4114df0 con 0x7f7ec4106620 2024-05-21T15:36:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.376+0000 7f7ec943d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec4114c70 con 0x7f7ec4103cc0 2024-05-21T15:36:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.376+0000 7f7ec943d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec4114f70 con 0x7f7ec4101360 2024-05-21T15:36:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.376+0000 7f7ec37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ec4106620 0x7f7ec410e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.376+0000 7f7ec37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ec4106620 0x7f7ec410e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44580/0 (socket says 172.21.15.38:44580) 2024-05-21T15:36:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.376+0000 7f7ec37fe640 1 -- 172.21.15.38:0/2248685463 learned_addr learned my addr 172.21.15.38:0/2248685463 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.376+0000 7f7ec27fc640 1 --2- 172.21.15.38:0/2248685463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ec4103cc0 0x7f7ec410d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.376+0000 7f7ec37fe640 1 -- 172.21.15.38:0/2248685463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec4101360 msgr2=0x7f7ec410cbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:36:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.376+0000 7f7ec37fe640 1 --2- 172.21.15.38:0/2248685463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec4101360 0x7f7ec410cbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.376+0000 7f7ec37fe640 1 -- 172.21.15.38:0/2248685463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ec4103cc0 msgr2=0x7f7ec410d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.376+0000 7f7ec37fe640 1 --2- 172.21.15.38:0/2248685463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ec4103cc0 0x7f7ec410d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.376+0000 7f7ec37fe640 1 -- 172.21.15.38:0/2248685463 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec410e6f0 con 0x7f7ec4106620 2024-05-21T15:36:55.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.378+0000 7f7ec37fe640 1 --2- 172.21.15.38:0/2248685463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ec4106620 0x7f7ec410e080 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc031a50 tx=0x7f7ebc032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.380+0000 7f7eabfff640 1 -- 172.21.15.38:0/2248685463 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc030d70 con 0x7f7ec4106620 2024-05-21T15:36:55.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.380+0000 7f7eabfff640 1 -- 172.21.15.38:0/2248685463 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc02d070 con 0x7f7ec4106620 2024-05-21T15:36:55.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.380+0000 7f7ec943d640 1 -- 172.21.15.38:0/2248685463 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec410e980 con 0x7f7ec4106620 2024-05-21T15:36:55.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.380+0000 7f7ec943d640 1 -- 172.21.15.38:0/2248685463 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec410ef50 con 0x7f7ec4106620 2024-05-21T15:36:55.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.382+0000 7f7eabfff640 1 -- 172.21.15.38:0/2248685463 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc03b410 con 0x7f7ec4106620 2024-05-21T15:36:55.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.386+0000 7f7ed03be740 1 Processor -- start 2024-05-21T15:36:55.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.389+0000 7f7eabfff640 1 -- 172.21.15.38:0/2248685463 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ebc004040 con 0x7f7ec4106620 2024-05-21T15:36:55.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.389+0000 7f7ed03be740 1 -- start start 2024-05-21T15:36:55.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.391+0000 7f7ed03be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.391+0000 7f7ed03be740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e0d84d0 0x55693e4a1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.391+0000 7f7ed03be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0e06b0 0x55693e4a7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.391+0000 7f7ed03be740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55693e405c40 con 0x55693e0d84d0 2024-05-21T15:36:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.391+0000 7f7ed03be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55693e3ad580 con 0x55693e101390 2024-05-21T15:36:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.391+0000 7f7ed03be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55693e401c00 con 0x55693e0e06b0 2024-05-21T15:36:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.392+0000 7f7ec27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0e06b0 0x55693e4a7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.392+0000 7f7ec27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0e06b0 0x55693e4a7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:35062/0 (socket says 172.21.15.38:35062) 2024-05-21T15:36:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.392+0000 7f7ec27fc640 1 -- 172.21.15.38:0/1949885455 learned_addr learned my addr 172.21.15.38:0/1949885455 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.392+0000 7f7ec2ffd640 1 --2- 172.21.15.38:0/1949885455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e0d84d0 0x55693e4a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.393+0000 7f7ec27fc640 1 -- 172.21.15.38:0/1949885455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 msgr2=0x55693e4a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.393+0000 7f7ec27fc640 1 --2- 172.21.15.38:0/1949885455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.393+0000 7f7ec27fc640 1 -- 172.21.15.38:0/1949885455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e0d84d0 msgr2=0x55693e4a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.393+0000 7f7ec27fc640 1 --2- 172.21.15.38:0/1949885455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e0d84d0 0x55693e4a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.393+0000 7f7ec27fc640 1 -- 172.21.15.38:0/1949885455 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55693e1193f0 con 0x55693e0e06b0 2024-05-21T15:36:55.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.393+0000 7f7ec27fc640 1 --2- 172.21.15.38:0/1949885455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0e06b0 0x55693e4a7f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009a60 tx=0x7f7eb8009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c5f5f2981694c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.394+0000 7f7eabfff640 1 --2- 172.21.15.38:0/2248685463 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7eac0a2d10 0x7f7eac0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.394+0000 7f7eabfff640 1 -- 172.21.15.38:0/2248685463 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7ebc0f07f0 con 0x7f7ec4106620 2024-05-21T15:36:55.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.395+0000 7f7ea97fa640 1 -- 172.21.15.38:0/1949885455 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8010bc0 con 0x55693e0e06b0 2024-05-21T15:36:55.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.395+0000 7f7ea97fa640 1 -- 172.21.15.38:0/1949885455 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb8019070 con 0x55693e0e06b0 2024-05-21T15:36:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.395+0000 7f7ea97fa640 1 -- 172.21.15.38:0/1949885455 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb80136c0 con 0x55693e0e06b0 2024-05-21T15:36:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.396+0000 7f7ec2ffd640 1 --2- 172.21.15.38:0/2248685463 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7eac0a2d10 0x7f7eac0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.397+0000 7f7ed03be740 1 -- 172.21.15.38:0/1949885455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0e06b0 msgr2=0x55693e4a7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.397+0000 7f7ed03be740 1 --2- 172.21.15.38:0/1949885455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0e06b0 0x55693e4a7f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009a60 tx=0x7f7eb8009f20 comp rx=0 tx=0).stop 2024-05-21T15:36:55.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.400+0000 7f7ed03be740 1 -- 172.21.15.38:0/1949885455 shutdown_connections 2024-05-21T15:36:55.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.400+0000 7f7ed03be740 1 --2- 172.21.15.38:0/1949885455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0e06b0 0x55693e4a7f80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.400+0000 7f7ed03be740 1 --2- 172.21.15.38:0/1949885455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e0d84d0 0x55693e4a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.400+0000 7f7ed03be740 1 --2- 172.21.15.38:0/1949885455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.400+0000 7f7ed03be740 1 -- 172.21.15.38:0/1949885455 >> 172.21.15.38:0/1949885455 conn(0x55693e0f54e0 msgr2=0x55693e49c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.401+0000 7f7ec2ffd640 1 --2- 172.21.15.38:0/2248685463 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7eac0a2d10 0x7f7eac0a5220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55693e49d2c0 tx=0x7f7eb4009060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.402+0000 7f7ed03be740 1 -- 172.21.15.38:0/1949885455 shutdown_connections 2024-05-21T15:36:55.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.402+0000 7f7ed03be740 1 -- 172.21.15.38:0/1949885455 wait complete. 2024-05-21T15:36:55.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.403+0000 7f7ed03be740 1 -- start start 2024-05-21T15:36:55.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.403+0000 7f7ed03be740 1 Processor -- start 2024-05-21T15:36:55.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.404+0000 7f7ed03be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0d84d0 0x55693e4aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.404+0000 7f7ed03be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e0e06b0 0x55693e4aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.404+0000 7f7ed03be740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e101390 0x55693e4b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.404+0000 7f7ed03be740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55693e4a89d0 con 0x55693e101390 2024-05-21T15:36:55.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.404+0000 7f7ed03be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55693e3c0c40 con 0x55693e0e06b0 2024-05-21T15:36:55.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.404+0000 7f7ed03be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55693e4a8b50 con 0x55693e0d84d0 2024-05-21T15:36:55.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.404+0000 7f7ed03be740 10 client.0 _ll_register_callbacks cb 0x55693e118be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:55.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.404+0000 7f7ec27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e0e06b0 0x55693e4aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.404+0000 7f7ec27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e0e06b0 0x55693e4aaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53156/0 (socket says 172.21.15.38:53156) 2024-05-21T15:36:55.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.404+0000 7f7ec27fc640 1 -- 172.21.15.38:0/1959859595 learned_addr learned my addr 172.21.15.38:0/1959859595 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:55.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.405+0000 7f7ec27fc640 1 -- 172.21.15.38:0/1959859595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0d84d0 msgr2=0x55693e4aaa40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:36:55.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.405+0000 7f7ec27fc640 1 --2- 172.21.15.38:0/1959859595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0d84d0 0x55693e4aaa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.405+0000 7f7ec27fc640 1 -- 172.21.15.38:0/1959859595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e101390 msgr2=0x55693e4b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.405+0000 7f7ec27fc640 1 --2- 172.21.15.38:0/1959859595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e101390 0x55693e4b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.405+0000 7f7ec27fc640 1 -- 172.21.15.38:0/1959859595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55693e1e37d0 con 0x55693e0e06b0 2024-05-21T15:36:55.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.406+0000 7f7ec27fc640 1 --2- 172.21.15.38:0/1959859595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e0e06b0 0x55693e4aaf80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8000ea0 tx=0x7f7eb80146a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62aaf74181f3aae3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.406+0000 7f7ea6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:55.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.408+0000 7f7ea6ff5640 1 -- 172.21.15.38:0/1959859595 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8025600 con 0x55693e0e06b0 2024-05-21T15:36:55.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.408+0000 7f7ed03be740 1 -- 172.21.15.38:0/1959859595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eb8019070 con 0x55693e0e06b0 2024-05-21T15:36:55.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.408+0000 7f7ea6ff5640 1 -- 172.21.15.38:0/1959859595 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7eb801d3f0 con 0x55693e0e06b0 2024-05-21T15:36:55.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.411+0000 7f7ea6ff5640 1 -- 172.21.15.38:0/1959859595 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7eb8009700 con 0x55693e0e06b0 2024-05-21T15:36:55.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.413+0000 7f7ed03be740 10 client.25853 fetch_fsmap learned FSMap version 30 2024-05-21T15:36:55.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.413+0000 7f7ed03be740 1 -- 172.21.15.38:0/1959859595 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55693e4b51d0 con 0x55693e0e06b0 2024-05-21T15:36:55.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.415+0000 7f7ea6ff5640 1 -- 172.21.15.38:0/1959859595 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb8019070 con 0x55693e0e06b0 2024-05-21T15:36:55.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.415+0000 7f7ea6ff5640 1 -- 172.21.15.38:0/1959859595 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7eb801dae0 con 0x55693e0e06b0 2024-05-21T15:36:55.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.421+0000 7f7ed03be740 10 client.25853 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:36:55.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.421+0000 7f7ed03be740 10 client.25853 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:36:55.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.421+0000 7f7ed03be740 20 client.25853 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:55.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.421+0000 7f7ed03be740 1 --2- 172.21.15.38:0/1959859595 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55693e0bff60 0x55693e4b8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.421+0000 7f7ed03be740 4 client.25853 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:55.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.421+0000 7f7ed03be740 1 -- 172.21.15.38:0/1959859595 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55693e413370 con 0x55693e0bff60 2024-05-21T15:36:55.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.422+0000 7f7ec2ffd640 1 --2- 172.21.15.38:0/1959859595 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55693e0bff60 0x55693e4b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.424+0000 7f7ec2ffd640 1 --2- 172.21.15.38:0/1959859595 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55693e0bff60 0x55693e4b8590 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f30939bd793a74b5 server_cookie=7a13dae7f2c3c487 in_seq=0 out_seq=0 2024-05-21T15:36:55.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.424+0000 7f7ea6ff5640 10 client.25853 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:36:55.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.426+0000 7f7ea6ff5640 1 -- 172.21.15.38:0/1959859595 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55693e413370 con 0x55693e0bff60 2024-05-21T15:36:55.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.426+0000 7f7ea6ff5640 10 client.25853 handle_command_reply: tid=0 2024-05-21T15:36:55.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.426+0000 7f7ed03be740 1 client.25853 shutdown 2024-05-21T15:36:55.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.426+0000 7f7ed03be740 1 -- 172.21.15.38:0/1959859595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e0e06b0 msgr2=0x55693e4aaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.426+0000 7f7ed03be740 1 --2- 172.21.15.38:0/1959859595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e0e06b0 0x55693e4aaf80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8000ea0 tx=0x7f7eb80146a0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.426+0000 7f7ed03be740 1 -- 172.21.15.38:0/1959859595 shutdown_connections 2024-05-21T15:36:55.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.426+0000 7f7ed03be740 1 --2- 172.21.15.38:0/1959859595 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55693e0bff60 0x55693e4b8590 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.426+0000 7f7ed03be740 1 --2- 172.21.15.38:0/1959859595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e101390 0x55693e4b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.426+0000 7f7ed03be740 1 --2- 172.21.15.38:0/1959859595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e0e06b0 0x55693e4aaf80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.426+0000 7f7ed03be740 1 --2- 172.21.15.38:0/1959859595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0d84d0 0x55693e4aaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.426+0000 7f7ed03be740 1 -- 172.21.15.38:0/1959859595 >> 172.21.15.38:0/1959859595 conn(0x55693e0f54e0 msgr2=0x55693e49b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.426+0000 7f7ea6ff5640 0 client.25853 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:36:55.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.427+0000 7f7ed03be740 1 -- 172.21.15.38:0/1959859595 shutdown_connections 2024-05-21T15:36:55.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.427+0000 7f7ed03be740 1 -- 172.21.15.38:0/1959859595 wait complete. 2024-05-21T15:36:55.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.428+0000 7f7ed03be740 20 client.25853 trim_cache size 0 max 16384 2024-05-21T15:36:55.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.461+0000 7f7ed03be740 1 Processor -- start 2024-05-21T15:36:55.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.464+0000 7f7ed03be740 1 -- start start 2024-05-21T15:36:55.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.464+0000 7f7ed03be740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e0d84d0 0x55693e4a9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.464+0000 7f7ed03be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0e06b0 0x55693e4b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.464+0000 7f7ed03be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.464+0000 7f7ed03be740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb000e6d0 con 0x55693e0d84d0 2024-05-21T15:36:55.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.464+0000 7f7ed03be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb00102d0 con 0x55693e101390 2024-05-21T15:36:55.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.464+0000 7f7ed03be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb000eda0 con 0x55693e0e06b0 2024-05-21T15:36:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.465+0000 7f7ec37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.465+0000 7f7ec27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0e06b0 0x55693e4b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.465+0000 7f7ec37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4a3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53172/0 (socket says 172.21.15.38:53172) 2024-05-21T15:36:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.465+0000 7f7ec37fe640 1 -- 172.21.15.38:0/3157201192 learned_addr learned my addr 172.21.15.38:0/3157201192 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.465+0000 7f7ec2ffd640 1 --2- 172.21.15.38:0/3157201192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e0d84d0 0x55693e4a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.466+0000 7f7ec37fe640 1 -- 172.21.15.38:0/3157201192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0e06b0 msgr2=0x55693e4b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.466+0000 7f7ec37fe640 1 --2- 172.21.15.38:0/3157201192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0e06b0 0x55693e4b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.466+0000 7f7ec37fe640 1 -- 172.21.15.38:0/3157201192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e0d84d0 msgr2=0x55693e4a9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.466+0000 7f7ec37fe640 1 --2- 172.21.15.38:0/3157201192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e0d84d0 0x55693e4a9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.466+0000 7f7ec37fe640 1 -- 172.21.15.38:0/3157201192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55693e497d20 con 0x55693e101390 2024-05-21T15:36:55.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.467+0000 7f7ec37fe640 1 --2- 172.21.15.38:0/3157201192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4a3b70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0c56e0 tx=0x7f7ebc0b92d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5a586c935d5e0db server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.467+0000 7f7ea8ff9640 1 -- 172.21.15.38:0/3157201192 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc0bc990 con 0x55693e101390 2024-05-21T15:36:55.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.468+0000 7f7ea8ff9640 1 -- 172.21.15.38:0/3157201192 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc036030 con 0x55693e101390 2024-05-21T15:36:55.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.468+0000 7f7ea8ff9640 1 -- 172.21.15.38:0/3157201192 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc0c10d0 con 0x55693e101390 2024-05-21T15:36:55.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.470+0000 7f7ed03be740 1 -- 172.21.15.38:0/3157201192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 msgr2=0x55693e4a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.470+0000 7f7ed03be740 1 --2- 172.21.15.38:0/3157201192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4a3b70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0c56e0 tx=0x7f7ebc0b92d0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.471+0000 7f7ed03be740 1 -- 172.21.15.38:0/3157201192 shutdown_connections 2024-05-21T15:36:55.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.471+0000 7f7ed03be740 1 --2- 172.21.15.38:0/3157201192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4a3b70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.471+0000 7f7ed03be740 1 --2- 172.21.15.38:0/3157201192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0e06b0 0x55693e4b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.471+0000 7f7ed03be740 1 --2- 172.21.15.38:0/3157201192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e0d84d0 0x55693e4a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.471+0000 7f7ed03be740 1 -- 172.21.15.38:0/3157201192 >> 172.21.15.38:0/3157201192 conn(0x55693e0f54e0 msgr2=0x55693e4a8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.473+0000 7f7ed03be740 1 -- 172.21.15.38:0/3157201192 shutdown_connections 2024-05-21T15:36:55.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.473+0000 7f7ed03be740 1 -- 172.21.15.38:0/3157201192 wait complete. 2024-05-21T15:36:55.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.474+0000 7f7ed03be740 1 -- start start 2024-05-21T15:36:55.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.474+0000 7f7ed03be740 1 Processor -- start 2024-05-21T15:36:55.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.474+0000 7f7ed03be740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0d84d0 0x55693e4af310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.474+0000 7f7ed03be740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e0e06b0 0x55693e4af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.474+0000 7f7ed03be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4b0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.474+0000 7f7ed03be740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb000fb70 con 0x55693e0e06b0 2024-05-21T15:36:55.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.474+0000 7f7ed03be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb000f470 con 0x55693e101390 2024-05-21T15:36:55.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.474+0000 7f7ed03be740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb000c6b0 con 0x55693e0d84d0 2024-05-21T15:36:55.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.474+0000 7f7ed03be740 10 client.0 _ll_register_callbacks cb 0x7f7eb4006e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:55.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.476+0000 7f7ec37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.477+0000 7f7ec37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53184/0 (socket says 172.21.15.38:53184) 2024-05-21T15:36:55.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.477+0000 7f7ec37fe640 1 -- 172.21.15.38:0/3396509815 learned_addr learned my addr 172.21.15.38:0/3396509815 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:55.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.478+0000 7f7ec37fe640 1 -- 172.21.15.38:0/3396509815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0d84d0 msgr2=0x55693e4af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.478+0000 7f7ec37fe640 1 --2- 172.21.15.38:0/3396509815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0d84d0 0x55693e4af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.478+0000 7f7ec37fe640 1 -- 172.21.15.38:0/3396509815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e0e06b0 msgr2=0x55693e4af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.478+0000 7f7ec37fe640 1 --2- 172.21.15.38:0/3396509815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e0e06b0 0x55693e4af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.478+0000 7f7ec37fe640 1 -- 172.21.15.38:0/3396509815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55693e4b2340 con 0x55693e101390 2024-05-21T15:36:55.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.481+0000 7f7ec37fe640 1 --2- 172.21.15.38:0/3396509815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4b0780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0b9490 tx=0x7f7ebc0c5ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c0f4d0b51027da server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.481+0000 7f7ea6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:55.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.484+0000 7f7ea6ff5640 1 -- 172.21.15.38:0/3396509815 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc0c5e60 con 0x55693e101390 2024-05-21T15:36:55.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.484+0000 7f7ea6ff5640 1 -- 172.21.15.38:0/3396509815 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7ebc0b96a0 con 0x55693e101390 2024-05-21T15:36:55.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.484+0000 7f7ed03be740 1 -- 172.21.15.38:0/3396509815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ebc036030 con 0x55693e101390 2024-05-21T15:36:55.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.485+0000 7f7ea6ff5640 1 -- 172.21.15.38:0/3396509815 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ebc0bc990 con 0x55693e101390 2024-05-21T15:36:55.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.487+0000 7f7ed03be740 10 client.25859 fetch_fsmap learned FSMap version 30 2024-05-21T15:36:55.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.487+0000 7f7ed03be740 1 -- 172.21.15.38:0/3396509815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55693e4b1e60 con 0x55693e101390 2024-05-21T15:36:55.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.488+0000 7f7ea6ff5640 1 -- 172.21.15.38:0/3396509815 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc036030 con 0x55693e101390 2024-05-21T15:36:55.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.488+0000 7f7ea6ff5640 1 -- 172.21.15.38:0/3396509815 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7ebc0dbbe0 con 0x55693e101390 2024-05-21T15:36:55.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.488+0000 7f7ed03be740 10 client.25859 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:36:55.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.488+0000 7f7ed03be740 10 client.25859 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:36:55.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.488+0000 7f7ed03be740 20 client.25859 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:55.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.488+0000 7f7ed03be740 1 --2- 172.21.15.38:0/3396509815 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55693e0953a0 0x55693e522d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.488+0000 7f7ed03be740 4 client.25859 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:36:55.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.488+0000 7f7ed03be740 1 -- 172.21.15.38:0/3396509815 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55693e40d6c0 con 0x55693e0953a0 2024-05-21T15:36:55.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.490+0000 7f7ec2ffd640 1 --2- 172.21.15.38:0/3396509815 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55693e0953a0 0x55693e522d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.493+0000 7f7ec2ffd640 1 --2- 172.21.15.38:0/3396509815 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55693e0953a0 0x55693e522d80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c89b3ebdedfebb2f server_cookie=9bc9b02ebd7765ee in_seq=0 out_seq=0 2024-05-21T15:36:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.494+0000 7f7ea6ff5640 10 client.25859 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:36:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.494+0000 7f7ea6ff5640 1 -- 172.21.15.38:0/3396509815 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55693e40d6c0 con 0x55693e0953a0 2024-05-21T15:36:55.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.495+0000 7f7ea6ff5640 10 client.25859 handle_command_reply: tid=0 2024-05-21T15:36:55.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.496+0000 7f7ed03be740 1 client.25859 shutdown 2024-05-21T15:36:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.498+0000 7f7ed03be740 1 -- 172.21.15.38:0/3396509815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 msgr2=0x55693e4b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.498+0000 7f7ed03be740 1 --2- 172.21.15.38:0/3396509815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4b0780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0b9490 tx=0x7f7ebc0c5ca0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.500+0000 7f7ed03be740 1 -- 172.21.15.38:0/3396509815 shutdown_connections 2024-05-21T15:36:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.500+0000 7f7ed03be740 1 --2- 172.21.15.38:0/3396509815 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55693e0953a0 0x55693e522d80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.500+0000 7f7ed03be740 1 --2- 172.21.15.38:0/3396509815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55693e101390 0x55693e4b0780 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.500+0000 7f7ed03be740 1 --2- 172.21.15.38:0/3396509815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55693e0e06b0 0x55693e4af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.500+0000 7f7ed03be740 1 --2- 172.21.15.38:0/3396509815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55693e0d84d0 0x55693e4af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.500+0000 7f7ed03be740 1 -- 172.21.15.38:0/3396509815 >> 172.21.15.38:0/3396509815 conn(0x55693e0f54e0 msgr2=0x55693e4a3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.501+0000 7f7ea6ff5640 0 client.25859 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:36:55.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.504+0000 7f7ed03be740 1 -- 172.21.15.38:0/3396509815 shutdown_connections 2024-05-21T15:36:55.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.505+0000 7f7ed03be740 1 -- 172.21.15.38:0/3396509815 wait complete. 2024-05-21T15:36:55.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.505+0000 7f7ed03be740 20 client.25859 trim_cache size 0 max 16384 2024-05-21T15:36:55.504 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:36:55.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.507+0000 7f7ec943d640 1 -- 172.21.15.38:0/2248685463 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7eac0a2d10 msgr2=0x7f7eac0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.507+0000 7f7ec943d640 1 --2- 172.21.15.38:0/2248685463 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7eac0a2d10 0x7f7eac0a5220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55693e49d2c0 tx=0x7f7eb4009060 comp rx=0 tx=0).stop 2024-05-21T15:36:55.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.507+0000 7f7ec943d640 1 -- 172.21.15.38:0/2248685463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ec4106620 msgr2=0x7f7ec410e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.507+0000 7f7ec943d640 1 --2- 172.21.15.38:0/2248685463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ec4106620 0x7f7ec410e080 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc031a50 tx=0x7f7ebc032f70 comp rx=0 tx=0).stop 2024-05-21T15:36:55.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.507+0000 7f7ec943d640 1 -- 172.21.15.38:0/2248685463 shutdown_connections 2024-05-21T15:36:55.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.507+0000 7f7ec943d640 1 --2- 172.21.15.38:0/2248685463 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7eac0a2d10 0x7f7eac0a5220 secure :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55693e49d2c0 tx=0x7f7eb4009060 comp rx=0 tx=0).stop 2024-05-21T15:36:55.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.507+0000 7f7ec943d640 1 --2- 172.21.15.38:0/2248685463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ec4106620 0x7f7ec410e080 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.507+0000 7f7ec943d640 1 --2- 172.21.15.38:0/2248685463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ec4103cc0 0x7f7ec410d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.507+0000 7f7ec943d640 1 --2- 172.21.15.38:0/2248685463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec4101360 0x7f7ec410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.507+0000 7f7ec943d640 1 -- 172.21.15.38:0/2248685463 >> 172.21.15.38:0/2248685463 conn(0x7f7ec40fcb70 msgr2=0x7f7ec4102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.509+0000 7f7ec943d640 1 -- 172.21.15.38:0/2248685463 shutdown_connections 2024-05-21T15:36:55.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.510+0000 7f7ec943d640 1 -- 172.21.15.38:0/2248685463 wait complete. 2024-05-21T15:36:55.530 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:36:55.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:55 smithi053 ceph-mon[36496]: pgmap v426: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 300 MiB/s wr, 25.97k op/s 2024-05-21T15:36:55.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:55 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:36:55.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.756+0000 7f7a67909640 1 -- 172.21.15.38:0/3132131626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a60100cc0 msgr2=0x7f7a601019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.756+0000 7f7a67909640 1 --2- 172.21.15.38:0/3132131626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a60100cc0 0x7f7a601019c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009ca0 tx=0x7f7a5802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.756+0000 7f7a67909640 1 -- 172.21.15.38:0/3132131626 shutdown_connections 2024-05-21T15:36:55.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.756+0000 7f7a67909640 1 --2- 172.21.15.38:0/3132131626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60106c00 0x7f7a60109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.756+0000 7f7a67909640 1 --2- 172.21.15.38:0/3132131626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a60101f00 0x7f7a60102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.756+0000 7f7a67909640 1 --2- 172.21.15.38:0/3132131626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a60100cc0 0x7f7a601019c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.756+0000 7f7a67909640 1 -- 172.21.15.38:0/3132131626 >> 172.21.15.38:0/3132131626 conn(0x7f7a600fbed0 msgr2=0x7f7a600fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.756+0000 7f7a67909640 1 -- 172.21.15.38:0/3132131626 shutdown_connections 2024-05-21T15:36:55.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.757+0000 7f7a67909640 1 -- 172.21.15.38:0/3132131626 wait complete. 2024-05-21T15:36:55.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.758+0000 7f7a67909640 1 Processor -- start 2024-05-21T15:36:55.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.758+0000 7f7a67909640 1 -- start start 2024-05-21T15:36:55.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.759+0000 7f7a67909640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a60100cc0 0x7f7a60112390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.759+0000 7f7a67909640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a60101f00 0x7f7a601128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.759+0000 7f7a67909640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60106c00 0x7f7a60115840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.759+0000 7f7a64ea4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a60101f00 0x7f7a601128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.759+0000 7f7a64ea4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a60101f00 0x7f7a601128d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44644/0 (socket says 172.21.15.38:44644) 2024-05-21T15:36:55.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.759+0000 7f7a64ea4640 1 -- 172.21.15.38:0/2907090374 learned_addr learned my addr 172.21.15.38:0/2907090374 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:55.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.759+0000 7f7a656a5640 1 --2- 172.21.15.38:0/2907090374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a60100cc0 0x7f7a60112390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.760+0000 7f7a67909640 1 -- 172.21.15.38:0/2907090374 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6010b6c0 con 0x7f7a60101f00 2024-05-21T15:36:55.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.760+0000 7f7a67909640 1 -- 172.21.15.38:0/2907090374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6010b540 con 0x7f7a60106c00 2024-05-21T15:36:55.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.760+0000 7f7a67909640 1 -- 172.21.15.38:0/2907090374 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6010b840 con 0x7f7a60100cc0 2024-05-21T15:36:55.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:55 smithi045 ceph-mon[36409]: pgmap v426: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 300 MiB/s wr, 25.97k op/s 2024-05-21T15:36:55.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:55 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:36:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.760+0000 7f7a65ea6640 1 --2- 172.21.15.38:0/2907090374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60106c00 0x7f7a60115840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.762+0000 7f7a64ea4640 1 -- 172.21.15.38:0/2907090374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a60100cc0 msgr2=0x7f7a60112390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.762+0000 7f7a64ea4640 1 --2- 172.21.15.38:0/2907090374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a60100cc0 0x7f7a60112390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.762+0000 7f7a64ea4640 1 -- 172.21.15.38:0/2907090374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60106c00 msgr2=0x7f7a60115840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.762+0000 7f7a64ea4640 1 --2- 172.21.15.38:0/2907090374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60106c00 0x7f7a60115840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.762+0000 7f7a64ea4640 1 -- 172.21.15.38:0/2907090374 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a60115eb0 con 0x7f7a60101f00 2024-05-21T15:36:55.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.762+0000 7f7a64ea4640 1 --2- 172.21.15.38:0/2907090374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a60101f00 0x7f7a601128d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d950 tx=0x7f7a5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.764+0000 7f7a5e7fc640 1 -- 172.21.15.38:0/2907090374 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50017030 con 0x7f7a60101f00 2024-05-21T15:36:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.764+0000 7f7a5e7fc640 1 -- 172.21.15.38:0/2907090374 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a50004020 con 0x7f7a60101f00 2024-05-21T15:36:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.764+0000 7f7a5e7fc640 1 -- 172.21.15.38:0/2907090374 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50003d60 con 0x7f7a60101f00 2024-05-21T15:36:55.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.764+0000 7f7a67909640 1 -- 172.21.15.38:0/2907090374 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a601161a0 con 0x7f7a60101f00 2024-05-21T15:36:55.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.765+0000 7f7a67909640 1 -- 172.21.15.38:0/2907090374 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a601166f0 con 0x7f7a60101f00 2024-05-21T15:36:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.770+0000 7f7a6e7b1740 1 Processor -- start 2024-05-21T15:36:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.770+0000 7f7a6e7b1740 1 -- start start 2024-05-21T15:36:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.770+0000 7f7a6e7b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 0x5646c1e9db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.770+0000 7f7a6e7b1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1ac44d0 0x5646c1e9e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.770+0000 7f7a6e7b1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1acc6b0 0x5646c1ea4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.770+0000 7f7a6e7b1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5646c1df1c40 con 0x5646c1ac44d0 2024-05-21T15:36:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.770+0000 7f7a6e7b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5646c1d99580 con 0x5646c1aed390 2024-05-21T15:36:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.770+0000 7f7a6e7b1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5646c1dedc00 con 0x5646c1acc6b0 2024-05-21T15:36:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.771+0000 7f7a656a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1ac44d0 0x5646c1e9e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.771+0000 7f7a656a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1ac44d0 0x5646c1e9e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44660/0 (socket says 172.21.15.38:44660) 2024-05-21T15:36:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.771+0000 7f7a656a5640 1 -- 172.21.15.38:0/4225070960 learned_addr learned my addr 172.21.15.38:0/4225070960 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.772+0000 7f7a656a5640 1 -- 172.21.15.38:0/4225070960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1acc6b0 msgr2=0x5646c1ea4f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:36:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.772+0000 7f7a656a5640 1 --2- 172.21.15.38:0/4225070960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1acc6b0 0x5646c1ea4f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.772+0000 7f7a656a5640 1 -- 172.21.15.38:0/4225070960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 msgr2=0x5646c1e9db10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:36:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.772+0000 7f7a656a5640 1 --2- 172.21.15.38:0/4225070960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 0x5646c1e9db10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.772+0000 7f7a656a5640 1 -- 172.21.15.38:0/4225070960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c1b053f0 con 0x5646c1ac44d0 2024-05-21T15:36:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.772+0000 7f7a656a5640 1 --2- 172.21.15.38:0/4225070960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1ac44d0 0x5646c1e9e090 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009ca0 tx=0x7f7a58005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755af54077cca6c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.773+0000 7f7a477fe640 1 -- 172.21.15.38:0/4225070960 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58031710 con 0x5646c1ac44d0 2024-05-21T15:36:55.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.773+0000 7f7a5e7fc640 1 -- 172.21.15.38:0/2907090374 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7a5001a070 con 0x7f7a60101f00 2024-05-21T15:36:55.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.774+0000 7f7a477fe640 1 -- 172.21.15.38:0/4225070960 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a58036030 con 0x5646c1ac44d0 2024-05-21T15:36:55.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.775+0000 7f7a5e7fc640 1 --2- 172.21.15.38:0/2907090374 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7a480a2d60 0x7f7a480a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.775+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/4225070960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1ac44d0 msgr2=0x5646c1e9e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.775+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/4225070960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1ac44d0 0x5646c1e9e090 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009ca0 tx=0x7f7a58005c80 comp rx=0 tx=0).stop 2024-05-21T15:36:55.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:55 smithi038 ceph-mon[31995]: pgmap v426: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 300 MiB/s wr, 25.97k op/s 2024-05-21T15:36:55.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:55 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:36:55.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.775+0000 7f7a5e7fc640 1 -- 172.21.15.38:0/2907090374 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7a500ceaa0 con 0x7f7a60101f00 2024-05-21T15:36:55.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.776+0000 7f7a65ea6640 1 --2- 172.21.15.38:0/2907090374 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7a480a2d60 0x7f7a480a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.776+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/4225070960 shutdown_connections 2024-05-21T15:36:55.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.776+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/4225070960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1acc6b0 0x5646c1ea4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.776+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/4225070960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1ac44d0 0x5646c1e9e090 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.776+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/4225070960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 0x5646c1e9db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.777+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/4225070960 >> 172.21.15.38:0/4225070960 conn(0x5646c1ae14e0 msgr2=0x5646c1e88490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.779+0000 7f7a65ea6640 1 --2- 172.21.15.38:0/2907090374 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7a480a2d60 0x7f7a480a5270 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5646c1e9ece0 tx=0x7f7a5400b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.787+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/4225070960 shutdown_connections 2024-05-21T15:36:55.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.788+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/4225070960 wait complete. 2024-05-21T15:36:55.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.788+0000 7f7a6e7b1740 1 -- start start 2024-05-21T15:36:55.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.788+0000 7f7a6e7b1740 1 Processor -- start 2024-05-21T15:36:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.789+0000 7f7a6e7b1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1ac44d0 0x5646c1e8e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.789+0000 7f7a6e7b1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1acc6b0 0x5646c1e8ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.789+0000 7f7a6e7b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 0x5646c1e95cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.789+0000 7f7a6e7b1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5646c1e8c7a0 con 0x5646c1ac44d0 2024-05-21T15:36:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.789+0000 7f7a6e7b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5646c1dacc40 con 0x5646c1aed390 2024-05-21T15:36:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.789+0000 7f7a6e7b1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5646c1e8c920 con 0x5646c1acc6b0 2024-05-21T15:36:55.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.789+0000 7f7a6e7b1740 10 client.0 _ll_register_callbacks cb 0x5646c1b04be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:55.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.790+0000 7f7a65ea6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 0x5646c1e95cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.790+0000 7f7a65ea6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 0x5646c1e95cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53202/0 (socket says 172.21.15.38:53202) 2024-05-21T15:36:55.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.790+0000 7f7a65ea6640 1 -- 172.21.15.38:0/1297062948 learned_addr learned my addr 172.21.15.38:0/1297062948 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:55.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.790+0000 7f7a656a5640 1 --2- 172.21.15.38:0/1297062948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1ac44d0 0x5646c1e8e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.791+0000 7f7a656a5640 1 -- 172.21.15.38:0/1297062948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1acc6b0 msgr2=0x5646c1e8ed60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:36:55.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.791+0000 7f7a656a5640 1 --2- 172.21.15.38:0/1297062948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1acc6b0 0x5646c1e8ed60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.791+0000 7f7a656a5640 1 -- 172.21.15.38:0/1297062948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 msgr2=0x5646c1e95cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.791+0000 7f7a656a5640 1 --2- 172.21.15.38:0/1297062948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 0x5646c1e95cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.791+0000 7f7a656a5640 1 -- 172.21.15.38:0/1297062948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c1bcf7d0 con 0x5646c1ac44d0 2024-05-21T15:36:55.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.792+0000 7f7a656a5640 1 --2- 172.21.15.38:0/1297062948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1ac44d0 0x5646c1e8e820 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7a58004af0 tx=0x7f7a5802b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912ed8d3f08189bf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.792+0000 7f7a44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:55.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.792+0000 7f7a44ff9640 1 -- 172.21.15.38:0/1297062948 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58074020 con 0x5646c1ac44d0 2024-05-21T15:36:55.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.792+0000 7f7a44ff9640 1 -- 172.21.15.38:0/1297062948 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7a580309f0 con 0x5646c1ac44d0 2024-05-21T15:36:55.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.793+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1297062948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a58036030 con 0x5646c1ac44d0 2024-05-21T15:36:55.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.794+0000 7f7a44ff9640 1 -- 172.21.15.38:0/1297062948 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a58009360 con 0x5646c1ac44d0 2024-05-21T15:36:55.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.795+0000 7f7a6e7b1740 10 client.16200 fetch_fsmap learned FSMap version 30 2024-05-21T15:36:55.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.795+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1297062948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5646c1e8c7a0 con 0x5646c1ac44d0 2024-05-21T15:36:55.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.798+0000 7f7a44ff9640 1 -- 172.21.15.38:0/1297062948 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a58036030 con 0x5646c1ac44d0 2024-05-21T15:36:55.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.799+0000 7f7a44ff9640 1 -- 172.21.15.38:0/1297062948 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7a58065070 con 0x5646c1ac44d0 2024-05-21T15:36:55.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.800+0000 7f7a6e7b1740 10 client.16200 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:36:55.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.800+0000 7f7a6e7b1740 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:55.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.800+0000 7f7a6e7b1740 20 client.16200 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:55.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.800+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1297062948 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c1aabf60 0x5646c1e9c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.800+0000 7f7a6e7b1740 4 client.16200 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:55.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.800+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1297062948 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646c1dff370 con 0x5646c1aabf60 2024-05-21T15:36:55.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.801+0000 7f7a64ea4640 1 --2- 172.21.15.38:0/1297062948 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c1aabf60 0x5646c1e9c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.811+0000 7f7a64ea4640 1 --2- 172.21.15.38:0/1297062948 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c1aabf60 0x5646c1e9c480 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9cab0cbcd12ff45 server_cookie=610fe6bf31b1718 in_seq=0 out_seq=0 2024-05-21T15:36:55.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.812+0000 7f7a44ff9640 10 client.16200 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:55.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.817+0000 7f7a44ff9640 1 -- 172.21.15.38:0/1297062948 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646c1dff370 con 0x5646c1aabf60 2024-05-21T15:36:55.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.817+0000 7f7a44ff9640 10 client.16200 handle_command_reply: tid=0 2024-05-21T15:36:55.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.817+0000 7f7a6e7b1740 1 client.16200 shutdown 2024-05-21T15:36:55.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.820+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1297062948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1ac44d0 msgr2=0x5646c1e8e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.820+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1297062948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1ac44d0 0x5646c1e8e820 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7a58004af0 tx=0x7f7a5802b040 comp rx=0 tx=0).stop 2024-05-21T15:36:55.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.820+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1297062948 shutdown_connections 2024-05-21T15:36:55.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.820+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1297062948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 0x5646c1e95cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.820+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1297062948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1acc6b0 0x5646c1e8ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.820+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1297062948 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c1aabf60 0x5646c1e9c480 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.820+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1297062948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1ac44d0 0x5646c1e8e820 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.820+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1297062948 >> 172.21.15.38:0/1297062948 conn(0x5646c1ae14e0 msgr2=0x5646c1e87570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.822+0000 7f7a44ff9640 0 client.16200 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:55.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.823+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1297062948 shutdown_connections 2024-05-21T15:36:55.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.824+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1297062948 wait complete. 2024-05-21T15:36:55.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.826+0000 7f7a6e7b1740 20 client.16200 trim_cache size 0 max 16384 2024-05-21T15:36:55.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.856+0000 7f7a6e7b1740 1 Processor -- start 2024-05-21T15:36:55.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.857+0000 7f7a6e7b1740 1 -- start start 2024-05-21T15:36:55.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.857+0000 7f7a6e7b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1ac44d0 0x5646c1e9d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.858+0000 7f7a6e7b1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1acc6b0 0x5646c1e8de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.858+0000 7f7a6e7b1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1aed390 0x5646c1e8e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.858+0000 7f7a6e7b1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c00e4b0 con 0x5646c1aed390 2024-05-21T15:36:55.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.858+0000 7f7a6e7b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c0100b0 con 0x5646c1ac44d0 2024-05-21T15:36:55.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.858+0000 7f7a6e7b1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c00eb80 con 0x5646c1acc6b0 2024-05-21T15:36:55.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.858+0000 7f7a656a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1ac44d0 0x5646c1e9d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.858+0000 7f7a656a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1ac44d0 0x5646c1e9d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53216/0 (socket says 172.21.15.38:53216) 2024-05-21T15:36:55.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.858+0000 7f7a656a5640 1 -- 172.21.15.38:0/1492940285 learned_addr learned my addr 172.21.15.38:0/1492940285 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:55.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.859+0000 7f7a656a5640 1 -- 172.21.15.38:0/1492940285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1acc6b0 msgr2=0x5646c1e8de00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:36:55.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.859+0000 7f7a656a5640 1 --2- 172.21.15.38:0/1492940285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1acc6b0 0x5646c1e8de00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.859+0000 7f7a656a5640 1 -- 172.21.15.38:0/1492940285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1aed390 msgr2=0x5646c1e8e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.859+0000 7f7a656a5640 1 --2- 172.21.15.38:0/1492940285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1aed390 0x5646c1e8e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.859+0000 7f7a656a5640 1 -- 172.21.15.38:0/1492940285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c1aac190 con 0x5646c1ac44d0 2024-05-21T15:36:55.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.860+0000 7f7a656a5640 1 --2- 172.21.15.38:0/1492940285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1ac44d0 0x5646c1e9d360 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7a58002230 tx=0x7f7a58004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1817f12337513921 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.861+0000 7f7a46ffd640 1 -- 172.21.15.38:0/1492940285 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58004320 con 0x5646c1ac44d0 2024-05-21T15:36:55.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.861+0000 7f7a46ffd640 1 -- 172.21.15.38:0/1492940285 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a58036070 con 0x5646c1ac44d0 2024-05-21T15:36:55.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.861+0000 7f7a46ffd640 1 -- 172.21.15.38:0/1492940285 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58003bd0 con 0x5646c1ac44d0 2024-05-21T15:36:55.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.862+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1492940285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1ac44d0 msgr2=0x5646c1e9d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.862+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1492940285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1ac44d0 0x5646c1e9d360 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7a58002230 tx=0x7f7a58004530 comp rx=0 tx=0).stop 2024-05-21T15:36:55.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.867+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1492940285 shutdown_connections 2024-05-21T15:36:55.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.867+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1492940285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1aed390 0x5646c1e8e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.867+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1492940285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1acc6b0 0x5646c1e8de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.867+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1492940285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1ac44d0 0x5646c1e9d360 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.867+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1492940285 >> 172.21.15.38:0/1492940285 conn(0x5646c1ae14e0 msgr2=0x5646c1e9cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.867+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1492940285 shutdown_connections 2024-05-21T15:36:55.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.867+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1492940285 wait complete. 2024-05-21T15:36:55.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.868+0000 7f7a6e7b1740 1 -- start start 2024-05-21T15:36:55.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.868+0000 7f7a6e7b1740 1 Processor -- start 2024-05-21T15:36:55.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.869+0000 7f7a6e7b1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1ac44d0 0x5646c1ea8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.869+0000 7f7a6e7b1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1acc6b0 0x5646c1ea9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.869+0000 7f7a6e7b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 0x5646c1eaa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.869+0000 7f7a6e7b1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c00f950 con 0x5646c1acc6b0 2024-05-21T15:36:55.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.869+0000 7f7a6e7b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c00f250 con 0x5646c1aed390 2024-05-21T15:36:55.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.869+0000 7f7a6e7b1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c00c540 con 0x5646c1ac44d0 2024-05-21T15:36:55.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.869+0000 7f7a6e7b1740 10 client.0 _ll_register_callbacks cb 0x7f7a58031cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:55.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.869+0000 7f7a64ea4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1acc6b0 0x5646c1ea9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.869+0000 7f7a64ea4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1acc6b0 0x5646c1ea9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44688/0 (socket says 172.21.15.38:44688) 2024-05-21T15:36:55.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.869+0000 7f7a64ea4640 1 -- 172.21.15.38:0/1045599531 learned_addr learned my addr 172.21.15.38:0/1045599531 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:55.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.870+0000 7f7a656a5640 1 --2- 172.21.15.38:0/1045599531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1ac44d0 0x5646c1ea8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.870+0000 7f7a65ea6640 1 --2- 172.21.15.38:0/1045599531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 0x5646c1eaa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.871+0000 7f7a64ea4640 1 -- 172.21.15.38:0/1045599531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1ac44d0 msgr2=0x5646c1ea8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.871+0000 7f7a64ea4640 1 --2- 172.21.15.38:0/1045599531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1ac44d0 0x5646c1ea8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.871+0000 7f7a64ea4640 1 -- 172.21.15.38:0/1045599531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 msgr2=0x5646c1eaa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:55.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.871+0000 7f7a64ea4640 1 --2- 172.21.15.38:0/1045599531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 0x5646c1eaa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.871+0000 7f7a64ea4640 1 -- 172.21.15.38:0/1045599531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c1eaa820 con 0x5646c1acc6b0 2024-05-21T15:36:55.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.874+0000 7f7a64ea4640 1 --2- 172.21.15.38:0/1045599531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1acc6b0 0x5646c1ea9330 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7a50005fd0 tx=0x7f7a5009bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fe4e094772cb14 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:55.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.874+0000 7f7a44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:55.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.877+0000 7f7a44ff9640 1 -- 172.21.15.38:0/1045599531 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a500a3ec0 con 0x5646c1acc6b0 2024-05-21T15:36:55.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.877+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1045599531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a58036070 con 0x5646c1acc6b0 2024-05-21T15:36:55.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.877+0000 7f7a44ff9640 1 -- 172.21.15.38:0/1045599531 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7a500035f0 con 0x5646c1acc6b0 2024-05-21T15:36:55.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.878+0000 7f7a44ff9640 1 -- 172.21.15.38:0/1045599531 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a500b4120 con 0x5646c1acc6b0 2024-05-21T15:36:55.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.878+0000 7f7a6e7b1740 10 client.16203 fetch_fsmap learned FSMap version 30 2024-05-21T15:36:55.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.878+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1045599531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5646c1f0bae0 con 0x5646c1acc6b0 2024-05-21T15:36:55.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.880+0000 7f7a44ff9640 1 -- 172.21.15.38:0/1045599531 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a58036070 con 0x5646c1acc6b0 2024-05-21T15:36:55.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.880+0000 7f7a44ff9640 1 -- 172.21.15.38:0/1045599531 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7a50024060 con 0x5646c1acc6b0 2024-05-21T15:36:55.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.880+0000 7f7a6e7b1740 10 client.16203 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:36:55.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.880+0000 7f7a6e7b1740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:55.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.880+0000 7f7a6e7b1740 20 client.16203 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:55.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.880+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1045599531 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c1a813a0 0x5646c1f0e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:55.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.880+0000 7f7a6e7b1740 4 client.16203 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:36:55.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.880+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1045599531 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646c1df96c0 con 0x5646c1a813a0 2024-05-21T15:36:55.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.882+0000 7f7a656a5640 1 --2- 172.21.15.38:0/1045599531 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c1a813a0 0x5646c1f0e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:55.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.885+0000 7f7a656a5640 1 --2- 172.21.15.38:0/1045599531 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c1a813a0 0x5646c1f0e5e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5401f1b3477de27 server_cookie=237d83457c752a4b in_seq=0 out_seq=0 2024-05-21T15:36:55.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.886+0000 7f7a44ff9640 10 client.16203 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.888+0000 7f7a44ff9640 1 -- 172.21.15.38:0/1045599531 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646c1df96c0 con 0x5646c1a813a0 2024-05-21T15:36:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.888+0000 7f7a44ff9640 10 client.16203 handle_command_reply: tid=0 2024-05-21T15:36:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.888+0000 7f7a6e7b1740 1 client.16203 shutdown 2024-05-21T15:36:55.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.890+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1045599531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1acc6b0 msgr2=0x5646c1ea9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.890+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1045599531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1acc6b0 0x5646c1ea9330 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7a50005fd0 tx=0x7f7a5009bf60 comp rx=0 tx=0).stop 2024-05-21T15:36:55.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.890+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1045599531 shutdown_connections 2024-05-21T15:36:55.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.890+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1045599531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646c1aed390 0x5646c1eaa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.890+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1045599531 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5646c1a813a0 0x5646c1f0e5e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.890+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1045599531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646c1acc6b0 0x5646c1ea9330 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.890+0000 7f7a6e7b1740 1 --2- 172.21.15.38:0/1045599531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646c1ac44d0 0x5646c1ea8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.891+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1045599531 >> 172.21.15.38:0/1045599531 conn(0x5646c1ae14e0 msgr2=0x5646c1e8b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.891+0000 7f7a44ff9640 0 client.16203 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:55.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.891+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1045599531 shutdown_connections 2024-05-21T15:36:55.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.893+0000 7f7a6e7b1740 1 -- 172.21.15.38:0/1045599531 wait complete. 2024-05-21T15:36:55.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.894+0000 7f7a6e7b1740 20 client.16203 trim_cache size 0 max 16384 2024-05-21T15:36:55.893 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:36:55.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.896+0000 7f7a67909640 1 -- 172.21.15.38:0/2907090374 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7a480a2d60 msgr2=0x7f7a480a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.896+0000 7f7a67909640 1 --2- 172.21.15.38:0/2907090374 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7a480a2d60 0x7f7a480a5270 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5646c1e9ece0 tx=0x7f7a5400b2e0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.896+0000 7f7a67909640 1 -- 172.21.15.38:0/2907090374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a60101f00 msgr2=0x7f7a601128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:55.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.896+0000 7f7a67909640 1 --2- 172.21.15.38:0/2907090374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a60101f00 0x7f7a601128d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d950 tx=0x7f7a5000de10 comp rx=0 tx=0).stop 2024-05-21T15:36:55.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.896+0000 7f7a67909640 1 -- 172.21.15.38:0/2907090374 shutdown_connections 2024-05-21T15:36:55.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.896+0000 7f7a67909640 1 --2- 172.21.15.38:0/2907090374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60106c00 0x7f7a60115840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.896+0000 7f7a67909640 1 --2- 172.21.15.38:0/2907090374 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7a480a2d60 0x7f7a480a5270 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.896+0000 7f7a67909640 1 --2- 172.21.15.38:0/2907090374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a60101f00 0x7f7a601128d0 secure :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d950 tx=0x7f7a5000de10 comp rx=0 tx=0).stop 2024-05-21T15:36:55.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.896+0000 7f7a67909640 1 --2- 172.21.15.38:0/2907090374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a60100cc0 0x7f7a60112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:55.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.896+0000 7f7a67909640 1 -- 172.21.15.38:0/2907090374 >> 172.21.15.38:0/2907090374 conn(0x7f7a600fbed0 msgr2=0x7f7a600fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:55.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.897+0000 7f7a67909640 1 -- 172.21.15.38:0/2907090374 shutdown_connections 2024-05-21T15:36:55.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:55.897+0000 7f7a67909640 1 -- 172.21.15.38:0/2907090374 wait complete. 2024-05-21T15:36:56.920 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:36:56.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:36:56.921 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:36:57.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.155+0000 7f8615ffb640 1 -- 172.21.15.38:0/1746424334 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608003850 con 0x7f86181076e0 2024-05-21T15:36:57.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.156+0000 7f8617fff640 1 -- 172.21.15.38:0/1746424334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86181076e0 msgr2=0x7f8618109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.156+0000 7f8617fff640 1 --2- 172.21.15.38:0/1746424334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86181076e0 0x7f8618109b00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f860800fb10 tx=0x7f8608032990 comp rx=0 tx=0).stop 2024-05-21T15:36:57.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.156+0000 7f8617fff640 1 -- 172.21.15.38:0/1746424334 shutdown_connections 2024-05-21T15:36:57.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.156+0000 7f8617fff640 1 --2- 172.21.15.38:0/1746424334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86181076e0 0x7f8618109b00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.156+0000 7f8617fff640 1 --2- 172.21.15.38:0/1746424334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86181029e0 0x7f8618102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.156+0000 7f8617fff640 1 --2- 172.21.15.38:0/1746424334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86181017a0 0x7f86181024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.156+0000 7f8617fff640 1 -- 172.21.15.38:0/1746424334 >> 172.21.15.38:0/1746424334 conn(0x7f86180fc9d0 msgr2=0x7f86180fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:57.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.158+0000 7f8617fff640 1 -- 172.21.15.38:0/1746424334 shutdown_connections 2024-05-21T15:36:57.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.159+0000 7f8617fff640 1 -- 172.21.15.38:0/1746424334 wait complete. 2024-05-21T15:36:57.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.160+0000 7f8617fff640 1 Processor -- start 2024-05-21T15:36:57.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.161+0000 7f8617fff640 1 -- start start 2024-05-21T15:36:57.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.161+0000 7f8617fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86181017a0 0x7f8618071820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.161+0000 7f8617fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86181029e0 0x7f8618071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.161+0000 7f8617fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86181076e0 0x7f8618072cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.161+0000 7f8617fff640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f861806baa0 con 0x7f86181017a0 2024-05-21T15:36:57.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.161+0000 7f8617fff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f861806b920 con 0x7f86181029e0 2024-05-21T15:36:57.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.161+0000 7f8617fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f861806bc20 con 0x7f86181076e0 2024-05-21T15:36:57.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.162+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86181017a0 0x7f8618071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.162+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86181017a0 0x7f8618071820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44708/0 (socket says 172.21.15.38:44708) 2024-05-21T15:36:57.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.162+0000 7f8616ffd640 1 -- 172.21.15.38:0/6102752 learned_addr learned my addr 172.21.15.38:0/6102752 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:57.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.162+0000 7f86177fe640 1 --2- 172.21.15.38:0/6102752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86181076e0 0x7f8618072cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.162+0000 7f8616ffd640 1 -- 172.21.15.38:0/6102752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86181076e0 msgr2=0x7f8618072cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.162+0000 7f8616ffd640 1 --2- 172.21.15.38:0/6102752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86181076e0 0x7f8618072cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.162+0000 7f8616ffd640 1 -- 172.21.15.38:0/6102752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86181029e0 msgr2=0x7f8618071d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:36:57.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.163+0000 7f8616ffd640 1 --2- 172.21.15.38:0/6102752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86181029e0 0x7f8618071d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.163+0000 7f8616ffd640 1 -- 172.21.15.38:0/6102752 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8618073330 con 0x7f86181017a0 2024-05-21T15:36:57.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.165+0000 7f8616ffd640 1 --2- 172.21.15.38:0/6102752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86181017a0 0x7f8618071820 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f860c00c970 tx=0x7f860c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:57.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.167+0000 7f85fffff640 1 -- 172.21.15.38:0/6102752 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c004220 con 0x7f86181017a0 2024-05-21T15:36:57.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.167+0000 7f85fffff640 1 -- 172.21.15.38:0/6102752 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f860c0043c0 con 0x7f86181017a0 2024-05-21T15:36:57.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.167+0000 7f85fffff640 1 -- 172.21.15.38:0/6102752 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c00f450 con 0x7f86181017a0 2024-05-21T15:36:57.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.167+0000 7f8617fff640 1 -- 172.21.15.38:0/6102752 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8618073620 con 0x7f86181017a0 2024-05-21T15:36:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.168+0000 7f8617fff640 1 -- 172.21.15.38:0/6102752 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8618073b20 con 0x7f86181017a0 2024-05-21T15:36:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.172+0000 7f86241f0740 1 Processor -- start 2024-05-21T15:36:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.172+0000 7f86241f0740 1 -- start start 2024-05-21T15:36:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.172+0000 7f86241f0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950279410 0x55d9506196b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.172+0000 7f86241f0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950250550 0x55d950619c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.172+0000 7f86241f0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950258730 0x55d950620b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.172+0000 7f86241f0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d95057d990 con 0x55d950279410 2024-05-21T15:36:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.172+0000 7f86241f0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d9505255b0 con 0x55d950258730 2024-05-21T15:36:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.172+0000 7f86241f0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d950579950 con 0x55d950250550 2024-05-21T15:36:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.173+0000 7f86167fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950258730 0x55d950620b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.175+0000 7f86177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950279410 0x55d9506196b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.175+0000 7f86177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950279410 0x55d9506196b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44722/0 (socket says 172.21.15.38:44722) 2024-05-21T15:36:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.175+0000 7f86177fe640 1 -- 172.21.15.38:0/3734049487 learned_addr learned my addr 172.21.15.38:0/3734049487 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.175+0000 7f86167fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950258730 0x55d950620b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53250/0 (socket says 172.21.15.38:53250) 2024-05-21T15:36:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.175+0000 7f86167fc640 1 -- 172.21.15.38:0/3734049487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950250550 msgr2=0x55d950619c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.175+0000 7f86167fc640 1 --2- 172.21.15.38:0/3734049487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950250550 0x55d950619c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.175+0000 7f86167fc640 1 -- 172.21.15.38:0/3734049487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950279410 msgr2=0x55d9506196b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.175+0000 7f86167fc640 1 --2- 172.21.15.38:0/3734049487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950279410 0x55d9506196b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.175+0000 7f86167fc640 1 -- 172.21.15.38:0/3734049487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9503597e0 con 0x55d950258730 2024-05-21T15:36:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.176+0000 7f86167fc640 1 --2- 172.21.15.38:0/3734049487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950258730 0x55d950620b30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8610005fd0 tx=0x7f8610009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f13950900ecec332 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.176+0000 7f85fd7fa640 1 -- 172.21.15.38:0/3734049487 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610018070 con 0x55d950258730 2024-05-21T15:36:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.176+0000 7f85fd7fa640 1 -- 172.21.15.38:0/3734049487 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8610003e50 con 0x55d950258730 2024-05-21T15:36:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.177+0000 7f85fd7fa640 1 -- 172.21.15.38:0/3734049487 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86100047e0 con 0x55d950258730 2024-05-21T15:36:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.178+0000 7f86241f0740 1 -- 172.21.15.38:0/3734049487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950258730 msgr2=0x55d950620b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.178+0000 7f86241f0740 1 --2- 172.21.15.38:0/3734049487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950258730 0x55d950620b30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8610005fd0 tx=0x7f8610009260 comp rx=0 tx=0).stop 2024-05-21T15:36:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.179+0000 7f86241f0740 1 -- 172.21.15.38:0/3734049487 shutdown_connections 2024-05-21T15:36:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.179+0000 7f86241f0740 1 --2- 172.21.15.38:0/3734049487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950258730 0x55d950620b30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.179+0000 7f86241f0740 1 --2- 172.21.15.38:0/3734049487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950250550 0x55d950619c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.179+0000 7f86241f0740 1 --2- 172.21.15.38:0/3734049487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950279410 0x55d9506196b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.179+0000 7f86241f0740 1 -- 172.21.15.38:0/3734049487 >> 172.21.15.38:0/3734049487 conn(0x55d95026d560 msgr2=0x55d950615040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:57.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.180+0000 7f85fffff640 1 -- 172.21.15.38:0/6102752 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f860c0079b0 con 0x7f86181017a0 2024-05-21T15:36:57.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.181+0000 7f85fffff640 1 --2- 172.21.15.38:0/6102752 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86000a2d90 0x7f86000a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.181+0000 7f85fffff640 1 -- 172.21.15.38:0/6102752 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f860c0d0ff0 con 0x7f86181017a0 2024-05-21T15:36:57.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.182+0000 7f86167fc640 1 --2- 172.21.15.38:0/6102752 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86000a2d90 0x7f86000a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.183+0000 7f86241f0740 1 -- 172.21.15.38:0/3734049487 shutdown_connections 2024-05-21T15:36:57.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.183+0000 7f86241f0740 1 -- 172.21.15.38:0/3734049487 wait complete. 2024-05-21T15:36:57.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.184+0000 7f86167fc640 1 --2- 172.21.15.38:0/6102752 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86000a2d90 0x7f86000a52a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8610004280 tx=0x7f8610005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:57.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.184+0000 7f86241f0740 1 -- start start 2024-05-21T15:36:57.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.184+0000 7f86241f0740 1 Processor -- start 2024-05-21T15:36:57.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.185+0000 7f86241f0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950250550 0x55d9506235a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.185+0000 7f86241f0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950258730 0x55d950623ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.185+0000 7f86241f0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950279410 0x55d95062aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.185+0000 7f86241f0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d950621620 con 0x55d950250550 2024-05-21T15:36:57.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.185+0000 7f86241f0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d950538b40 con 0x55d950279410 2024-05-21T15:36:57.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.185+0000 7f86241f0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d9506217a0 con 0x55d950258730 2024-05-21T15:36:57.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.185+0000 7f86241f0740 10 client.0 _ll_register_callbacks cb 0x55d95052d0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:57.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.186+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950250550 0x55d9506235a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.186+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950250550 0x55d9506235a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44732/0 (socket says 172.21.15.38:44732) 2024-05-21T15:36:57.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.186+0000 7f8616ffd640 1 -- 172.21.15.38:0/2444607394 learned_addr learned my addr 172.21.15.38:0/2444607394 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:57.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.187+0000 7f86167fc640 1 --2- 172.21.15.38:0/2444607394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950258730 0x55d950623ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.187+0000 7f8616ffd640 1 -- 172.21.15.38:0/2444607394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950258730 msgr2=0x55d950623ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.187+0000 7f8616ffd640 1 --2- 172.21.15.38:0/2444607394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950258730 0x55d950623ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.187+0000 7f8616ffd640 1 -- 172.21.15.38:0/2444607394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950279410 msgr2=0x55d95062aa40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:36:57.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.187+0000 7f8616ffd640 1 --2- 172.21.15.38:0/2444607394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950279410 0x55d95062aa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.187+0000 7f8616ffd640 1 -- 172.21.15.38:0/2444607394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9503b9f70 con 0x55d950250550 2024-05-21T15:36:57.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.188+0000 7f8616ffd640 1 --2- 172.21.15.38:0/2444607394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950250550 0x55d9506235a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f860c00afd0 tx=0x7f860c09dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532d01f81308e97 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:57.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.188+0000 7f85faff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:57.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.188+0000 7f85faff5640 1 -- 172.21.15.38:0/2444607394 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c0b5070 con 0x55d950250550 2024-05-21T15:36:57.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.188+0000 7f86241f0740 1 -- 172.21.15.38:0/2444607394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8610003e50 con 0x55d950250550 2024-05-21T15:36:57.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.189+0000 7f85faff5640 1 -- 172.21.15.38:0/2444607394 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f860c003610 con 0x55d950250550 2024-05-21T15:36:57.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.192+0000 7f85faff5640 1 -- 172.21.15.38:0/2444607394 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f860c0a6470 con 0x55d950250550 2024-05-21T15:36:57.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.192+0000 7f86241f0740 10 client.16215 fetch_fsmap learned FSMap version 30 2024-05-21T15:36:57.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.192+0000 7f86241f0740 1 -- 172.21.15.38:0/2444607394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d950251170 con 0x55d950250550 2024-05-21T15:36:57.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.193+0000 7f85faff5640 1 -- 172.21.15.38:0/2444607394 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8610003e50 con 0x55d950250550 2024-05-21T15:36:57.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.194+0000 7f85faff5640 1 -- 172.21.15.38:0/2444607394 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f860c020060 con 0x55d950250550 2024-05-21T15:36:57.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.194+0000 7f86241f0740 10 client.16215 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:36:57.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.194+0000 7f86241f0740 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:57.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.194+0000 7f86241f0740 20 client.16215 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:57.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.194+0000 7f86241f0740 1 --2- 172.21.15.38:0/2444607394 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d9502381f0 0x55d950631460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.194+0000 7f86241f0740 4 client.16215 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:57.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.194+0000 7f86241f0740 1 -- 172.21.15.38:0/2444607394 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d95058b0c0 con 0x55d9502381f0 2024-05-21T15:36:57.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.194+0000 7f86167fc640 1 --2- 172.21.15.38:0/2444607394 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d9502381f0 0x55d950631460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.196+0000 7f86167fc640 1 --2- 172.21.15.38:0/2444607394 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d9502381f0 0x55d950631460 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcdd62d321943eae server_cookie=bce34956e84017a in_seq=0 out_seq=0 2024-05-21T15:36:57.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.196+0000 7f85faff5640 10 client.16215 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.197+0000 7f85faff5640 1 -- 172.21.15.38:0/2444607394 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d95058b0c0 con 0x55d9502381f0 2024-05-21T15:36:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.198+0000 7f85faff5640 10 client.16215 handle_command_reply: tid=0 2024-05-21T15:36:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.198+0000 7f86241f0740 1 client.16215 shutdown 2024-05-21T15:36:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.199+0000 7f86241f0740 1 -- 172.21.15.38:0/2444607394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950250550 msgr2=0x55d9506235a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.199+0000 7f86241f0740 1 --2- 172.21.15.38:0/2444607394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950250550 0x55d9506235a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f860c00afd0 tx=0x7f860c09dfc0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.199+0000 7f86241f0740 1 -- 172.21.15.38:0/2444607394 shutdown_connections 2024-05-21T15:36:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.199+0000 7f86241f0740 1 --2- 172.21.15.38:0/2444607394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950279410 0x55d95062aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.199+0000 7f86241f0740 1 --2- 172.21.15.38:0/2444607394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950258730 0x55d950623ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.199+0000 7f86241f0740 1 --2- 172.21.15.38:0/2444607394 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d9502381f0 0x55d950631460 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.199+0000 7f86241f0740 1 --2- 172.21.15.38:0/2444607394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950250550 0x55d9506235a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.199+0000 7f86241f0740 1 -- 172.21.15.38:0/2444607394 >> 172.21.15.38:0/2444607394 conn(0x55d95026d560 msgr2=0x55d9506142e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.199+0000 7f85faff5640 0 client.16215 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:57.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.199+0000 7f86241f0740 1 -- 172.21.15.38:0/2444607394 shutdown_connections 2024-05-21T15:36:57.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.199+0000 7f86241f0740 1 -- 172.21.15.38:0/2444607394 wait complete. 2024-05-21T15:36:57.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.200+0000 7f86241f0740 20 client.16215 trim_cache size 0 max 16384 2024-05-21T15:36:57.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.239+0000 7f86241f0740 1 Processor -- start 2024-05-21T15:36:57.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.239+0000 7f86241f0740 1 -- start start 2024-05-21T15:36:57.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.240+0000 7f86241f0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950250550 0x55d95061cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.240+0000 7f86241f0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950258730 0x55d95062a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.240+0000 7f86241f0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950279410 0x55d95062af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.240+0000 7f86241f0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86181b07a0 con 0x55d950250550 2024-05-21T15:36:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.240+0000 7f86167fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950258730 0x55d95062a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.241+0000 7f86167fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950258730 0x55d95062a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53254/0 (socket says 172.21.15.38:53254) 2024-05-21T15:36:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.241+0000 7f86167fc640 1 -- 172.21.15.38:0/559291917 learned_addr learned my addr 172.21.15.38:0/559291917 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.241+0000 7f86177fe640 1 --2- 172.21.15.38:0/559291917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950279410 0x55d95062af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.241+0000 7f86241f0740 1 -- 172.21.15.38:0/559291917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f86181b23a0 con 0x55d950258730 2024-05-21T15:36:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.241+0000 7f86241f0740 1 -- 172.21.15.38:0/559291917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f86181b0e70 con 0x55d950279410 2024-05-21T15:36:57.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.243+0000 7f8616ffd640 1 --2- 172.21.15.38:0/559291917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950250550 0x55d95061cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.246+0000 7f8616ffd640 1 -- 172.21.15.38:0/559291917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950279410 msgr2=0x55d95062af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.246+0000 7f8616ffd640 1 --2- 172.21.15.38:0/559291917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950279410 0x55d95062af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.246+0000 7f8616ffd640 1 -- 172.21.15.38:0/559291917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950258730 msgr2=0x55d95062a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.246+0000 7f8616ffd640 1 --2- 172.21.15.38:0/559291917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950258730 0x55d95062a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.246+0000 7f8616ffd640 1 -- 172.21.15.38:0/559291917 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9506150f0 con 0x55d950250550 2024-05-21T15:36:57.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.246+0000 7f86167fc640 1 --2- 172.21.15.38:0/559291917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950258730 0x55d95062a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.247+0000 7f8616ffd640 1 --2- 172.21.15.38:0/559291917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950250550 0x55d95061cfd0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f860c09d500 tx=0x7f860c09bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d13dc7eb1af5ef server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:57.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.248+0000 7f85fcff9640 1 -- 172.21.15.38:0/559291917 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c0b5070 con 0x55d950250550 2024-05-21T15:36:57.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.249+0000 7f85fcff9640 1 -- 172.21.15.38:0/559291917 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f860c007ca0 con 0x55d950250550 2024-05-21T15:36:57.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.249+0000 7f85fcff9640 1 -- 172.21.15.38:0/559291917 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c09ed90 con 0x55d950250550 2024-05-21T15:36:57.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.251+0000 7f86241f0740 1 -- 172.21.15.38:0/559291917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950250550 msgr2=0x55d95061cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.251+0000 7f86241f0740 1 --2- 172.21.15.38:0/559291917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950250550 0x55d95061cfd0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f860c09d500 tx=0x7f860c09bd10 comp rx=0 tx=0).stop 2024-05-21T15:36:57.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.253+0000 7f86241f0740 1 -- 172.21.15.38:0/559291917 shutdown_connections 2024-05-21T15:36:57.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.253+0000 7f86241f0740 1 --2- 172.21.15.38:0/559291917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950279410 0x55d95062af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.253+0000 7f86241f0740 1 --2- 172.21.15.38:0/559291917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950258730 0x55d95062a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.253+0000 7f86241f0740 1 --2- 172.21.15.38:0/559291917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950250550 0x55d95061cfd0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.253+0000 7f86241f0740 1 -- 172.21.15.38:0/559291917 >> 172.21.15.38:0/559291917 conn(0x55d95026d560 msgr2=0x55d95061c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:57.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.253+0000 7f86241f0740 1 -- 172.21.15.38:0/559291917 shutdown_connections 2024-05-21T15:36:57.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.253+0000 7f86241f0740 1 -- 172.21.15.38:0/559291917 wait complete. 2024-05-21T15:36:57.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.254+0000 7f86241f0740 1 -- start start 2024-05-21T15:36:57.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.254+0000 7f86241f0740 1 Processor -- start 2024-05-21T15:36:57.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.257+0000 7f86241f0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950250550 0x55d950627e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.257+0000 7f86241f0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950258730 0x55d950628350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.257+0000 7f86241f0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950279410 0x55d950697800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.257+0000 7f86241f0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86181b1c40 con 0x55d950258730 2024-05-21T15:36:57.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.257+0000 7f86241f0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f86181b1540 con 0x55d950279410 2024-05-21T15:36:57.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.257+0000 7f86241f0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f86181ae780 con 0x55d950250550 2024-05-21T15:36:57.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.257+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950250550 0x55d950627e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.257+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950250550 0x55d950627e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:35166/0 (socket says 172.21.15.38:35166) 2024-05-21T15:36:57.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.257+0000 7f8616ffd640 1 -- 172.21.15.38:0/2889256472 learned_addr learned my addr 172.21.15.38:0/2889256472 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:57.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.257+0000 7f86241f0740 10 client.0 _ll_register_callbacks cb 0x7f860c00a520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:57.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.257+0000 7f86167fc640 1 --2- 172.21.15.38:0/2889256472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950258730 0x55d950628350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.258+0000 7f86177fe640 1 --2- 172.21.15.38:0/2889256472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950279410 0x55d950697800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.258+0000 7f86167fc640 1 -- 172.21.15.38:0/2889256472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950250550 msgr2=0x55d950627e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.258+0000 7f86167fc640 1 --2- 172.21.15.38:0/2889256472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950250550 0x55d950627e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.258+0000 7f86167fc640 1 -- 172.21.15.38:0/2889256472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950279410 msgr2=0x55d950697800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.258+0000 7f86167fc640 1 --2- 172.21.15.38:0/2889256472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950279410 0x55d950697800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.258+0000 7f86167fc640 1 -- 172.21.15.38:0/2889256472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9506a8d50 con 0x55d950258730 2024-05-21T15:36:57.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.258+0000 7f8616ffd640 1 --2- 172.21.15.38:0/2889256472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950250550 0x55d950627e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:57.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.259+0000 7f86167fc640 1 --2- 172.21.15.38:0/2889256472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950258730 0x55d950628350 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f861000d010 tx=0x7f861000d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a07a63f307fa93 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:57.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.259+0000 7f85faff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:36:57.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.259+0000 7f85faff5640 1 -- 172.21.15.38:0/2889256472 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610018070 con 0x55d950258730 2024-05-21T15:36:57.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.259+0000 7f86241f0740 1 -- 172.21.15.38:0/2889256472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f860c007ca0 con 0x55d950258730 2024-05-21T15:36:57.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.260+0000 7f85faff5640 1 -- 172.21.15.38:0/2889256472 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8610033640 con 0x55d950258730 2024-05-21T15:36:57.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.260+0000 7f85faff5640 1 -- 172.21.15.38:0/2889256472 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f86100044b0 con 0x55d950258730 2024-05-21T15:36:57.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.260+0000 7f86241f0740 10 client.16221 fetch_fsmap learned FSMap version 30 2024-05-21T15:36:57.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.260+0000 7f86241f0740 1 -- 172.21.15.38:0/2889256472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d9506a9fb0 con 0x55d950258730 2024-05-21T15:36:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.262+0000 7f85faff5640 1 -- 172.21.15.38:0/2889256472 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8610037070 con 0x55d950258730 2024-05-21T15:36:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.262+0000 7f85faff5640 1 -- 172.21.15.38:0/2889256472 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8610033ce0 con 0x55d950258730 2024-05-21T15:36:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.262+0000 7f86241f0740 10 client.16221 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:36:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.262+0000 7f86241f0740 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.262+0000 7f86241f0740 20 client.16221 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.262+0000 7f86241f0740 1 --2- 172.21.15.38:0/2889256472 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d95020d630 0x55d9506ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.262+0000 7f86241f0740 4 client.16221 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:36:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.262+0000 7f86241f0740 1 -- 172.21.15.38:0/2889256472 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f860c007ca0 con 0x55d95020d630 2024-05-21T15:36:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.262+0000 7f8616ffd640 1 --2- 172.21.15.38:0/2889256472 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d95020d630 0x55d9506ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.263+0000 7f85faff5640 10 client.16221 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:57.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.263+0000 7f8616ffd640 1 --2- 172.21.15.38:0/2889256472 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d95020d630 0x55d9506ac9c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34d9d99bc154749b server_cookie=ed44f34f5e73c8ff in_seq=0 out_seq=0 2024-05-21T15:36:57.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.265+0000 7f85faff5640 1 -- 172.21.15.38:0/2889256472 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f860c007ca0 con 0x55d95020d630 2024-05-21T15:36:57.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.265+0000 7f85faff5640 10 client.16221 handle_command_reply: tid=0 2024-05-21T15:36:57.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.265+0000 7f86241f0740 1 client.16221 shutdown 2024-05-21T15:36:57.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.270+0000 7f86241f0740 1 -- 172.21.15.38:0/2889256472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950258730 msgr2=0x55d950628350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.270+0000 7f86241f0740 1 --2- 172.21.15.38:0/2889256472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950258730 0x55d950628350 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f861000d010 tx=0x7f861000d040 comp rx=0 tx=0).stop 2024-05-21T15:36:57.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.274+0000 7f86241f0740 1 -- 172.21.15.38:0/2889256472 shutdown_connections 2024-05-21T15:36:57.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.274+0000 7f86241f0740 1 --2- 172.21.15.38:0/2889256472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d950279410 0x55d950697800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.274+0000 7f86241f0740 1 --2- 172.21.15.38:0/2889256472 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d95020d630 0x55d9506ac9c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.274+0000 7f86241f0740 1 --2- 172.21.15.38:0/2889256472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d950258730 0x55d950628350 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.274+0000 7f85faff5640 0 client.16221 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:57.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.274+0000 7f86241f0740 1 --2- 172.21.15.38:0/2889256472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d950250550 0x55d950627e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.274+0000 7f86241f0740 1 -- 172.21.15.38:0/2889256472 >> 172.21.15.38:0/2889256472 conn(0x55d95026d560 msgr2=0x55d950617e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:57.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.281+0000 7f86241f0740 1 -- 172.21.15.38:0/2889256472 shutdown_connections 2024-05-21T15:36:57.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.283+0000 7f86241f0740 1 -- 172.21.15.38:0/2889256472 wait complete. 2024-05-21T15:36:57.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.284+0000 7f86241f0740 20 client.16221 trim_cache size 0 max 16384 2024-05-21T15:36:57.285 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:36:57.285 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-05-21T15:36:57.285 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "45bd4e48-5f67-41ff-96a1-69bab23490b7", 2024-05-21T15:36:57.285 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-05-21T15:36:57.286 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:36:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.290+0000 7f8617fff640 1 -- 172.21.15.38:0/6102752 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86000a2d90 msgr2=0x7f86000a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.290+0000 7f8617fff640 1 --2- 172.21.15.38:0/6102752 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86000a2d90 0x7f86000a52a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8610004280 tx=0x7f8610005bd0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.290+0000 7f8617fff640 1 -- 172.21.15.38:0/6102752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86181017a0 msgr2=0x7f8618071820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.290+0000 7f8617fff640 1 --2- 172.21.15.38:0/6102752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86181017a0 0x7f8618071820 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f860c00c970 tx=0x7f860c00ce30 comp rx=0 tx=0).stop 2024-05-21T15:36:57.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.292+0000 7f8617fff640 1 -- 172.21.15.38:0/6102752 shutdown_connections 2024-05-21T15:36:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.292+0000 7f8617fff640 1 --2- 172.21.15.38:0/6102752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86181076e0 0x7f8618072cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.292+0000 7f8617fff640 1 --2- 172.21.15.38:0/6102752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86181029e0 0x7f8618071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.292+0000 7f8617fff640 1 --2- 172.21.15.38:0/6102752 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86000a2d90 0x7f86000a52a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.292+0000 7f8617fff640 1 --2- 172.21.15.38:0/6102752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86181017a0 0x7f8618071820 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.292+0000 7f8617fff640 1 -- 172.21.15.38:0/6102752 >> 172.21.15.38:0/6102752 conn(0x7f86180fc9d0 msgr2=0x7f86181036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.293+0000 7f8617fff640 1 -- 172.21.15.38:0/6102752 shutdown_connections 2024-05-21T15:36:57.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.293+0000 7f8617fff640 1 -- 172.21.15.38:0/6102752 wait complete. 2024-05-21T15:36:57.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:36:57.316 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:36:57.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.559+0000 7f8edfb57640 1 -- 172.21.15.38:0/3684663732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed810afc0 msgr2=0x7f8ed810b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.559+0000 7f8edfb57640 1 --2- 172.21.15.38:0/3684663732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed810afc0 0x7f8ed810b3d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009ca0 tx=0x7f8ed002a8f0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.563+0000 7f8edfb57640 1 -- 172.21.15.38:0/3684663732 shutdown_connections 2024-05-21T15:36:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.563+0000 7f8edfb57640 1 --2- 172.21.15.38:0/3684663732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed810afc0 0x7f8ed810b3d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.563+0000 7f8edfb57640 1 --2- 172.21.15.38:0/3684663732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ed810a2c0 0x7f8ed810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.563+0000 7f8edfb57640 1 --2- 172.21.15.38:0/3684663732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed81093d0 0x7f8ed8109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.563+0000 7f8edfb57640 1 -- 172.21.15.38:0/3684663732 >> 172.21.15.38:0/3684663732 conn(0x7f8ed806f4a0 msgr2=0x7f8ed8071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:57.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.563+0000 7f8edfb57640 1 -- 172.21.15.38:0/3684663732 shutdown_connections 2024-05-21T15:36:57.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.563+0000 7f8edfb57640 1 -- 172.21.15.38:0/3684663732 wait complete. 2024-05-21T15:36:57.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.565+0000 7f8edfb57640 1 Processor -- start 2024-05-21T15:36:57.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.567+0000 7f8edfb57640 1 -- start start 2024-05-21T15:36:57.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.568+0000 7f8edfb57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ed81093d0 0x7f8ed81a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.568+0000 7f8edfb57640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed810a2c0 0x7f8ed81a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.568+0000 7f8edfb57640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed810afc0 0x7f8ed81ab340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.568+0000 7f8edfb57640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81192a0 con 0x7f8ed81093d0 2024-05-21T15:36:57.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.569+0000 7f8edfb57640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed8119120 con 0x7f8ed810a2c0 2024-05-21T15:36:57.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.569+0000 7f8edfb57640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed8119420 con 0x7f8ed810afc0 2024-05-21T15:36:57.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.569+0000 7f8ede0f4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed810afc0 0x7f8ed81ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.569+0000 7f8edd0f2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed810a2c0 0x7f8ed81a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.569+0000 7f8ede0f4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed810afc0 0x7f8ed81ab340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:35182/0 (socket says 172.21.15.38:35182) 2024-05-21T15:36:57.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.569+0000 7f8ede0f4640 1 -- 172.21.15.38:0/185875501 learned_addr learned my addr 172.21.15.38:0/185875501 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:57.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.570+0000 7f8edd8f3640 1 --2- 172.21.15.38:0/185875501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ed81093d0 0x7f8ed81a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.571+0000 7f8ede0f4640 1 -- 172.21.15.38:0/185875501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed810a2c0 msgr2=0x7f8ed81a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.571+0000 7f8ede0f4640 1 --2- 172.21.15.38:0/185875501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed810a2c0 0x7f8ed81a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.571+0000 7f8ede0f4640 1 -- 172.21.15.38:0/185875501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ed81093d0 msgr2=0x7f8ed81a3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.571+0000 7f8ede0f4640 1 --2- 172.21.15.38:0/185875501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ed81093d0 0x7f8ed81a3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.571+0000 7f8ede0f4640 1 -- 172.21.15.38:0/185875501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed81ab9b0 con 0x7f8ed810afc0 2024-05-21T15:36:57.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.571+0000 7f8edd0f2640 1 --2- 172.21.15.38:0/185875501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed810a2c0 0x7f8ed81a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:36:57.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.572+0000 7f8ede0f4640 1 --2- 172.21.15.38:0/185875501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed810afc0 0x7f8ed81ab340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0033040 tx=0x7f8ed0003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:57.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.573+0000 7f8eceffd640 1 -- 172.21.15.38:0/185875501 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0004040 con 0x7f8ed810afc0 2024-05-21T15:36:57.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.573+0000 7f8eceffd640 1 -- 172.21.15.38:0/185875501 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ed002d070 con 0x7f8ed810afc0 2024-05-21T15:36:57.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.573+0000 7f8eceffd640 1 -- 172.21.15.38:0/185875501 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed002fbf0 con 0x7f8ed810afc0 2024-05-21T15:36:57.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.573+0000 7f8edfb57640 1 -- 172.21.15.38:0/185875501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed81abc40 con 0x7f8ed810afc0 2024-05-21T15:36:57.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.574+0000 7f8edfb57640 1 -- 172.21.15.38:0/185875501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed81ac140 con 0x7f8ed810afc0 2024-05-21T15:36:57.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.581+0000 7f8eceffd640 1 -- 172.21.15.38:0/185875501 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8ed00305f0 con 0x7f8ed810afc0 2024-05-21T15:36:57.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.581+0000 7f8ee6a03740 1 Processor -- start 2024-05-21T15:36:57.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.581+0000 7f8ee6a03740 1 -- start start 2024-05-21T15:36:57.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.582+0000 7f8ee6a03740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f7471390 0x5582f7810bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.582+0000 7f8ee6a03740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74484d0 0x5582f7811130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.582+0000 7f8ee6a03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74506b0 0x5582f7818030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.582+0000 7f8ee6a03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5582f7775d00 con 0x5582f74506b0 2024-05-21T15:36:57.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.582+0000 7f8ee6a03740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5582f771d4c0 con 0x5582f7471390 2024-05-21T15:36:57.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.582+0000 7f8ee6a03740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5582f7771cc0 con 0x5582f74484d0 2024-05-21T15:36:57.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.582+0000 7f8edd0f2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74506b0 0x5582f7818030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.582+0000 7f8edd0f2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74506b0 0x5582f7818030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44768/0 (socket says 172.21.15.38:44768) 2024-05-21T15:36:57.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.582+0000 7f8edd0f2640 1 -- 172.21.15.38:0/3601117730 learned_addr learned my addr 172.21.15.38:0/3601117730 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:57.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.583+0000 7f8edd8f3640 1 --2- 172.21.15.38:0/3601117730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74484d0 0x5582f7811130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.584+0000 7f8edd0f2640 1 --2- 172.21.15.38:0/3601117730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f7471390 0x5582f7810bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.584+0000 7f8edd0f2640 1 -- 172.21.15.38:0/3601117730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74484d0 msgr2=0x5582f7811130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.584+0000 7f8edd0f2640 1 --2- 172.21.15.38:0/3601117730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74484d0 0x5582f7811130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.584+0000 7f8edd0f2640 1 -- 172.21.15.38:0/3601117730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74506b0 msgr2=0x5582f7818030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.585+0000 7f8edd0f2640 1 --2- 172.21.15.38:0/3601117730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74506b0 0x5582f7818030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.585+0000 7f8edd0f2640 1 -- 172.21.15.38:0/3601117730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f75537d0 con 0x5582f7471390 2024-05-21T15:36:57.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.585+0000 7f8edd0f2640 1 --2- 172.21.15.38:0/3601117730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f7471390 0x5582f7810bb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4037aa0 tx=0x7f8ed40090e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7240dff43984158 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:57.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.586+0000 7f8eceffd640 1 --2- 172.21.15.38:0/185875501 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8ec40a2d90 0x7f8ec40a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.586+0000 7f8eceffd640 1 -- 172.21.15.38:0/185875501 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8ed00f07d0 con 0x7f8ed810afc0 2024-05-21T15:36:57.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.587+0000 7f8ec3fff640 1 -- 172.21.15.38:0/3601117730 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4039970 con 0x5582f7471390 2024-05-21T15:36:57.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.587+0000 7f8ec3fff640 1 -- 172.21.15.38:0/3601117730 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ed403e070 con 0x5582f7471390 2024-05-21T15:36:57.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.589+0000 7f8ee6a03740 1 -- 172.21.15.38:0/3601117730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f7471390 msgr2=0x5582f7810bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.589+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/3601117730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f7471390 0x5582f7810bb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4037aa0 tx=0x7f8ed40090e0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.590+0000 7f8edd8f3640 1 --2- 172.21.15.38:0/185875501 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8ec40a2d90 0x7f8ec40a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.590+0000 7f8ee6a03740 1 -- 172.21.15.38:0/3601117730 shutdown_connections 2024-05-21T15:36:57.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.590+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/3601117730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74506b0 0x5582f7818030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.590+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/3601117730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74484d0 0x5582f7811130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.590+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/3601117730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f7471390 0x5582f7810bb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.590+0000 7f8ee6a03740 1 -- 172.21.15.38:0/3601117730 >> 172.21.15.38:0/3601117730 conn(0x5582f74654e0 msgr2=0x5582f780c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:57.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.591+0000 7f8ee6a03740 1 -- 172.21.15.38:0/3601117730 shutdown_connections 2024-05-21T15:36:57.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.593+0000 7f8ee6a03740 1 -- 172.21.15.38:0/3601117730 wait complete. 2024-05-21T15:36:57.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.594+0000 7f8ee6a03740 1 -- start start 2024-05-21T15:36:57.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.594+0000 7f8ee6a03740 1 Processor -- start 2024-05-21T15:36:57.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.595+0000 7f8ee6a03740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f74484d0 0x5582f78232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.595+0000 7f8ee6a03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74506b0 0x5582f7823800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.595+0000 7f8ee6a03740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f7471390 0x5582f782a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.595+0000 7f8ee6a03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5582f7818a70 con 0x5582f74506b0 2024-05-21T15:36:57.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.595+0000 7f8ee6a03740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5582f7730b90 con 0x5582f74484d0 2024-05-21T15:36:57.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.595+0000 7f8ee6a03740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5582f7818bf0 con 0x5582f7471390 2024-05-21T15:36:57.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.595+0000 7f8ee6a03740 10 client.0 _ll_register_callbacks cb 0x5582f776cda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:57.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.596+0000 7f8edd8f3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f74484d0 0x5582f78232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.596+0000 7f8edd8f3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f74484d0 0x5582f78232c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53296/0 (socket says 172.21.15.38:53296) 2024-05-21T15:36:57.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.596+0000 7f8edd8f3640 1 -- 172.21.15.38:0/2802072833 learned_addr learned my addr 172.21.15.38:0/2802072833 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:57.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.596+0000 7f8ede0f4640 1 --2- 172.21.15.38:0/2802072833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f7471390 0x5582f782a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.596+0000 7f8edd0f2640 1 --2- 172.21.15.38:0/2802072833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74506b0 0x5582f7823800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.597+0000 7f8edd8f3640 1 -- 172.21.15.38:0/2802072833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f7471390 msgr2=0x5582f782a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.597+0000 7f8edd8f3640 1 --2- 172.21.15.38:0/2802072833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f7471390 0x5582f782a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.597+0000 7f8edd8f3640 1 -- 172.21.15.38:0/2802072833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74506b0 msgr2=0x5582f7823800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.597+0000 7f8edd8f3640 1 --2- 172.21.15.38:0/2802072833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74506b0 0x5582f7823800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.597+0000 7f8edd8f3640 1 -- 172.21.15.38:0/2802072833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f74893f0 con 0x5582f74484d0 2024-05-21T15:36:57.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.598+0000 7f8edd8f3640 1 --2- 172.21.15.38:0/185875501 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8ec40a2d90 0x7f8ec40a52a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x5582f780d2d0 tx=0x7f8ec8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:57.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.600+0000 7f8edd8f3640 1 --2- 172.21.15.38:0/2802072833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f74484d0 0x5582f78232c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8011fd0 tx=0x7f8ec80046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=954ce98383bc2f33 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:57.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.601+0000 7f8ec17fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:36:57.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.601+0000 7f8ec17fa640 1 -- 172.21.15.38:0/2802072833 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8024030 con 0x5582f74484d0 2024-05-21T15:36:57.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.601+0000 7f8ec17fa640 1 -- 172.21.15.38:0/2802072833 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8ec800f050 con 0x5582f74484d0 2024-05-21T15:36:57.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.602+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2802072833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed403e070 con 0x5582f74484d0 2024-05-21T15:36:57.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.604+0000 7f8ec17fa640 1 -- 172.21.15.38:0/2802072833 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ec801a070 con 0x5582f74484d0 2024-05-21T15:36:57.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.606+0000 7f8ee6a03740 10 client.25892 fetch_fsmap learned FSMap version 30 2024-05-21T15:36:57.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.606+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2802072833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5582f74490f0 con 0x5582f74484d0 2024-05-21T15:36:57.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.608+0000 7f8ec17fa640 1 -- 172.21.15.38:0/2802072833 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ed403e070 con 0x5582f74484d0 2024-05-21T15:36:57.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.608+0000 7f8ec17fa640 1 -- 172.21.15.38:0/2802072833 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8ec8029900 con 0x5582f74484d0 2024-05-21T15:36:57.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.608+0000 7f8ee6a03740 10 client.25892 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:36:57.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.608+0000 7f8ee6a03740 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:57.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.608+0000 7f8ee6a03740 20 client.25892 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:57.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.608+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/2802072833 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5582f742ff60 0x5582f78316d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.608+0000 7f8ee6a03740 4 client.25892 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:36:57.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.608+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2802072833 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582f7783430 con 0x5582f742ff60 2024-05-21T15:36:57.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.611+0000 7f8edd0f2640 1 --2- 172.21.15.38:0/2802072833 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5582f742ff60 0x5582f78316d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.615+0000 7f8edd0f2640 1 --2- 172.21.15.38:0/2802072833 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5582f742ff60 0x5582f78316d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969c64dd90d4635 server_cookie=77fa8e85655d4667 in_seq=0 out_seq=0 2024-05-21T15:36:57.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.615+0000 7f8ec17fa640 10 client.25892 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:57.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.616+0000 7f8ec17fa640 1 -- 172.21.15.38:0/2802072833 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5582f7783430 con 0x5582f742ff60 2024-05-21T15:36:57.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.616+0000 7f8ec17fa640 10 client.25892 handle_command_reply: tid=0 2024-05-21T15:36:57.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.616+0000 7f8ee6a03740 1 client.25892 shutdown 2024-05-21T15:36:57.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.617+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2802072833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f74484d0 msgr2=0x5582f78232c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.617+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/2802072833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f74484d0 0x5582f78232c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8011fd0 tx=0x7f8ec80046e0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.620+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2802072833 shutdown_connections 2024-05-21T15:36:57.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.620+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/2802072833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f7471390 0x5582f782a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.620+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/2802072833 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5582f742ff60 0x5582f78316d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.620+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/2802072833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74506b0 0x5582f7823800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.620+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/2802072833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f74484d0 0x5582f78232c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.620+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2802072833 >> 172.21.15.38:0/2802072833 conn(0x5582f74654e0 msgr2=0x5582f780b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:57.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.620+0000 7f8ec17fa640 0 client.25892 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:57.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.620+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2802072833 shutdown_connections 2024-05-21T15:36:57.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.621+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2802072833 wait complete. 2024-05-21T15:36:57.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.621+0000 7f8ee6a03740 20 client.25892 trim_cache size 0 max 16384 2024-05-21T15:36:57.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.667+0000 7f8ee6a03740 1 Processor -- start 2024-05-21T15:36:57.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.668+0000 7f8ee6a03740 1 -- start start 2024-05-21T15:36:57.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.668+0000 7f8ee6a03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74484d0 0x5582f7819070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.668+0000 7f8ee6a03740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74506b0 0x5582f782a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.668+0000 7f8ee6a03740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f7471390 0x5582f78195b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.669+0000 7f8ee6a03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc00e5f0 con 0x5582f74484d0 2024-05-21T15:36:57.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.669+0000 7f8ee6a03740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc0101f0 con 0x5582f7471390 2024-05-21T15:36:57.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.669+0000 7f8ee6a03740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc00ecc0 con 0x5582f74506b0 2024-05-21T15:36:57.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.669+0000 7f8ede0f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f7471390 0x5582f78195b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.669+0000 7f8ede0f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f7471390 0x5582f78195b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53306/0 (socket says 172.21.15.38:53306) 2024-05-21T15:36:57.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.669+0000 7f8ede0f4640 1 -- 172.21.15.38:0/3753955943 learned_addr learned my addr 172.21.15.38:0/3753955943 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:57.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.669+0000 7f8edd0f2640 1 --2- 172.21.15.38:0/3753955943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74506b0 0x5582f782a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.670+0000 7f8ede0f4640 1 -- 172.21.15.38:0/3753955943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74506b0 msgr2=0x5582f782a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.670+0000 7f8ede0f4640 1 --2- 172.21.15.38:0/3753955943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74506b0 0x5582f782a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.670+0000 7f8ede0f4640 1 -- 172.21.15.38:0/3753955943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74484d0 msgr2=0x5582f7819070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.670+0000 7f8ede0f4640 1 --2- 172.21.15.38:0/3753955943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74484d0 0x5582f7819070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.670+0000 7f8ede0f4640 1 -- 172.21.15.38:0/3753955943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f7823ec0 con 0x5582f7471390 2024-05-21T15:36:57.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.670+0000 7f8ede0f4640 1 --2- 172.21.15.38:0/3753955943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f7471390 0x5582f78195b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00bcf10 tx=0x7f8ed00095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ce3ef3d7d510628 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:57.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.671+0000 7f8ec37fe640 1 -- 172.21.15.38:0/3753955943 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed00d5070 con 0x5582f7471390 2024-05-21T15:36:57.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.671+0000 7f8ec37fe640 1 -- 172.21.15.38:0/3753955943 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ed00d7030 con 0x5582f7471390 2024-05-21T15:36:57.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.671+0000 7f8ec37fe640 1 -- 172.21.15.38:0/3753955943 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed002fd90 con 0x5582f7471390 2024-05-21T15:36:57.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.673+0000 7f8ee6a03740 1 -- 172.21.15.38:0/3753955943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f7471390 msgr2=0x5582f78195b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.673+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/3753955943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f7471390 0x5582f78195b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00bcf10 tx=0x7f8ed00095c0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.674+0000 7f8ee6a03740 1 -- 172.21.15.38:0/3753955943 shutdown_connections 2024-05-21T15:36:57.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.674+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/3753955943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f7471390 0x5582f78195b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.674+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/3753955943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74506b0 0x5582f782a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.674+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/3753955943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f74484d0 0x5582f7819070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.674+0000 7f8ee6a03740 1 -- 172.21.15.38:0/3753955943 >> 172.21.15.38:0/3753955943 conn(0x5582f74654e0 msgr2=0x5582f7814140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:57.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.675+0000 7f8ee6a03740 1 -- 172.21.15.38:0/3753955943 shutdown_connections 2024-05-21T15:36:57.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.675+0000 7f8ee6a03740 1 -- 172.21.15.38:0/3753955943 wait complete. 2024-05-21T15:36:57.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.676+0000 7f8ee6a03740 1 -- start start 2024-05-21T15:36:57.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.676+0000 7f8ee6a03740 1 Processor -- start 2024-05-21T15:36:57.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.677+0000 7f8ee6a03740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f74484d0 0x5582f7829fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.677+0000 7f8ee6a03740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74506b0 0x5582f782a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.678+0000 7f8edd8f3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f74484d0 0x5582f7829fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.678+0000 7f8edd8f3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f74484d0 0x5582f7829fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53320/0 (socket says 172.21.15.38:53320) 2024-05-21T15:36:57.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.678+0000 7f8edd8f3640 1 -- 172.21.15.38:0/2121790383 learned_addr learned my addr 172.21.15.38:0/2121790383 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:36:57.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.678+0000 7f8edd0f2640 1 --2- 172.21.15.38:0/2121790383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74506b0 0x5582f782a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.678+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/2121790383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f7471390 0x5582f7826b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.678+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2121790383 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc00fa90 con 0x5582f7471390 2024-05-21T15:36:57.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.678+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2121790383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc00f390 con 0x5582f74484d0 2024-05-21T15:36:57.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.678+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2121790383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc00c5d0 con 0x5582f74506b0 2024-05-21T15:36:57.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.679+0000 7f8ee6a03740 10 client.0 _ll_register_callbacks cb 0x7f8ec8009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:36:57.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.679+0000 7f8edd0f2640 1 -- 172.21.15.38:0/2121790383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f74484d0 msgr2=0x5582f7829fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.679+0000 7f8edd0f2640 1 --2- 172.21.15.38:0/2121790383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f74484d0 0x5582f7829fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.679+0000 7f8edd0f2640 1 -- 172.21.15.38:0/2121790383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f7471390 msgr2=0x5582f7826b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:36:57.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.679+0000 7f8edd0f2640 1 --2- 172.21.15.38:0/2121790383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f7471390 0x5582f7826b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.679+0000 7f8edd0f2640 1 -- 172.21.15.38:0/2121790383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f7827280 con 0x5582f74506b0 2024-05-21T15:36:57.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.681+0000 7f8edd0f2640 1 --2- 172.21.15.38:0/2121790383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74506b0 0x5582f782a500 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4039880 tx=0x7f8ed4006e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60dc359daf18c3a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:36:57.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.681+0000 7f8ec17fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:36:57.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.682+0000 7f8ec17fa640 1 -- 172.21.15.38:0/2121790383 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed404f020 con 0x5582f74506b0 2024-05-21T15:36:57.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.682+0000 7f8ec17fa640 1 -- 172.21.15.38:0/2121790383 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8ed4035cb0 con 0x5582f74506b0 2024-05-21T15:36:57.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.682+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2121790383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed00d7030 con 0x5582f74506b0 2024-05-21T15:36:57.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.683+0000 7f8ec17fa640 1 -- 172.21.15.38:0/2121790383 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ed4035350 con 0x5582f74506b0 2024-05-21T15:36:57.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.683+0000 7f8ee6a03740 10 client.25939 fetch_fsmap learned FSMap version 30 2024-05-21T15:36:57.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.683+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2121790383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5582f7898620 con 0x5582f74506b0 2024-05-21T15:36:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.684+0000 7f8ec17fa640 1 -- 172.21.15.38:0/2121790383 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ed00d7030 con 0x5582f74506b0 2024-05-21T15:36:57.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.686+0000 7f8ec17fa640 1 -- 172.21.15.38:0/2121790383 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8ed4039970 con 0x5582f74506b0 2024-05-21T15:36:57.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.686+0000 7f8ee6a03740 10 client.25939 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:36:57.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.686+0000 7f8ee6a03740 10 client.25939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:36:57.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.686+0000 7f8ee6a03740 20 client.25939 populate_metadata read hostname 'smithi038' 2024-05-21T15:36:57.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.686+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/2121790383 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5582f74053a0 0x5582f789ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:36:57.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.686+0000 7f8ee6a03740 4 client.25939 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:36:57.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.686+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2121790383 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582f777d780 con 0x5582f74053a0 2024-05-21T15:36:57.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.686+0000 7f8edd8f3640 1 --2- 172.21.15.38:0/2121790383 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5582f74053a0 0x5582f789ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:36:57.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.694+0000 7f8edd8f3640 1 --2- 172.21.15.38:0/2121790383 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5582f74053a0 0x5582f789ad60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bba347154a955a60 server_cookie=c906466451d8e398 in_seq=0 out_seq=0 2024-05-21T15:36:57.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.694+0000 7f8ec17fa640 10 client.25939 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:57.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.695+0000 7f8ec17fa640 1 -- 172.21.15.38:0/2121790383 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5582f777d780 con 0x5582f74053a0 2024-05-21T15:36:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.695+0000 7f8ec17fa640 10 client.25939 handle_command_reply: tid=0 2024-05-21T15:36:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.695+0000 7f8ee6a03740 1 client.25939 shutdown 2024-05-21T15:36:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.696+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2121790383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74506b0 msgr2=0x5582f782a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.696+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/2121790383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74506b0 0x5582f782a500 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4039880 tx=0x7f8ed4006e50 comp rx=0 tx=0).stop 2024-05-21T15:36:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.696+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2121790383 shutdown_connections 2024-05-21T15:36:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.696+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/2121790383 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5582f74053a0 0x5582f789ad60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.696+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/2121790383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5582f7471390 0x5582f7826b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.696+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/2121790383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5582f74506b0 0x5582f782a500 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.696+0000 7f8ee6a03740 1 --2- 172.21.15.38:0/2121790383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582f74484d0 0x5582f7829fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.696+0000 7f8ec17fa640 0 client.25939 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:36:57.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.696+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2121790383 >> 172.21.15.38:0/2121790383 conn(0x5582f74654e0 msgr2=0x5582f7819070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:57.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.698+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2121790383 shutdown_connections 2024-05-21T15:36:57.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.700+0000 7f8ee6a03740 1 -- 172.21.15.38:0/2121790383 wait complete. 2024-05-21T15:36:57.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.702+0000 7f8ee6a03740 20 client.25939 trim_cache size 0 max 16384 2024-05-21T15:36:57.704 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:36:57.704 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "scrub active (120 inodes in the stack)", 2024-05-21T15:36:57.704 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-05-21T15:36:57.704 INFO:teuthology.orchestra.run.smithi038.stdout: "45bd4e48-5f67-41ff-96a1-69bab23490b7": { 2024-05-21T15:36:57.704 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-05-21T15:36:57.704 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "45bd4e48-5f67-41ff-96a1-69bab23490b7", 2024-05-21T15:36:57.704 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-05-21T15:36:57.704 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:36:57.704 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:36:57.704 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:36:57.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:57 smithi053 ceph-mon[36496]: pgmap v427: 97 pgs: 97 active+clean; 13 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 262 MiB/s wr, 30.02k op/s 2024-05-21T15:36:57.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:57 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:36:57.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.706+0000 7f8edfb57640 1 -- 172.21.15.38:0/185875501 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8ec40a2d90 msgr2=0x7f8ec40a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.706+0000 7f8edfb57640 1 --2- 172.21.15.38:0/185875501 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8ec40a2d90 0x7f8ec40a52a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x5582f780d2d0 tx=0x7f8ec8008040 comp rx=0 tx=0).stop 2024-05-21T15:36:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.706+0000 7f8edfb57640 1 -- 172.21.15.38:0/185875501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed810afc0 msgr2=0x7f8ed81ab340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:36:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.706+0000 7f8edfb57640 1 --2- 172.21.15.38:0/185875501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed810afc0 0x7f8ed81ab340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0033040 tx=0x7f8ed0003ba0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.706+0000 7f8edfb57640 1 -- 172.21.15.38:0/185875501 shutdown_connections 2024-05-21T15:36:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.706+0000 7f8edfb57640 1 --2- 172.21.15.38:0/185875501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ed810afc0 0x7f8ed81ab340 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.706+0000 7f8edfb57640 1 --2- 172.21.15.38:0/185875501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed810a2c0 0x7f8ed81a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.706+0000 7f8edfb57640 1 --2- 172.21.15.38:0/185875501 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8ec40a2d90 0x7f8ec40a52a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.706+0000 7f8edfb57640 1 --2- 172.21.15.38:0/185875501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ed81093d0 0x7f8ed81a3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:36:57.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.706+0000 7f8edfb57640 1 -- 172.21.15.38:0/185875501 >> 172.21.15.38:0/185875501 conn(0x7f8ed806f4a0 msgr2=0x7f8ed8112b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:36:57.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.708+0000 7f8edfb57640 1 -- 172.21.15.38:0/185875501 shutdown_connections 2024-05-21T15:36:57.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:36:57.708+0000 7f8edfb57640 1 -- 172.21.15.38:0/185875501 wait complete. 2024-05-21T15:36:57.727 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:57 smithi038 ceph-mon[31995]: pgmap v427: 97 pgs: 97 active+clean; 13 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 262 MiB/s wr, 30.02k op/s 2024-05-21T15:36:57.727 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:57 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:36:57.730 INFO:tasks.cephfs.filesystem:scrub status for tag:45bd4e48-5f67-41ff-96a1-69bab23490b7 - {'path': '/', 'tag': '45bd4e48-5f67-41ff-96a1-69bab23490b7', 'options': 'recursive,force'} 2024-05-21T15:36:57.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:57 smithi045 ceph-mon[36409]: pgmap v427: 97 pgs: 97 active+clean; 13 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 262 MiB/s wr, 30.02k op/s 2024-05-21T15:36:57.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:57 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:36:58.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:58 smithi045 ceph-mon[36409]: scrub queued for path: / 2024-05-21T15:36:58.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:58 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:36:58.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:58 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:36:58.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:58 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:36:58.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:58 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:36:58.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:58 smithi038 ceph-mon[31995]: scrub queued for path: / 2024-05-21T15:36:58.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:58 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:36:58.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:58 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:36:58.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:58 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:36:58.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:58 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:36:58.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:58 smithi053 ceph-mon[36496]: scrub queued for path: / 2024-05-21T15:36:58.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:58 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:36:58.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:58 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:36:58.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:58 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:36:58.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:58 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:36:59.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:59 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:36:59.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:59 smithi045 ceph-mon[36409]: pgmap v428: 97 pgs: 97 active+clean; 13 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 223 MiB/s wr, 28.85k op/s 2024-05-21T15:36:59.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:59 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:36:59.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:59 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:36:59.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:59 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:36:59.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:36:59 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:36:59.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:59 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:36:59.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:59 smithi038 ceph-mon[31995]: pgmap v428: 97 pgs: 97 active+clean; 13 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 223 MiB/s wr, 28.85k op/s 2024-05-21T15:36:59.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:59 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:36:59.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:59 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:36:59.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:59 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:36:59.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:36:59 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:36:59.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:59 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:36:59.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:59 smithi053 ceph-mon[36496]: pgmap v428: 97 pgs: 97 active+clean; 13 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 223 MiB/s wr, 28.85k op/s 2024-05-21T15:36:59.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:59 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:36:59.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:59 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:36:59.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:59 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:36:59.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:36:59 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.964+0000 7fb70e559640 1 -- 172.21.15.38:0/2368854769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708100d30 msgr2=0x7fb708101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.964+0000 7fb70e559640 1 --2- 172.21.15.38:0/2368854769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708100d30 0x7fb708101180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009900 tx=0x7fb6fc02a410 comp rx=0 tx=0).stop 2024-05-21T15:37:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.965+0000 7fb70e559640 1 -- 172.21.15.38:0/2368854769 shutdown_connections 2024-05-21T15:37:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.965+0000 7fb70e559640 1 --2- 172.21.15.38:0/2368854769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7081068e0 0x7fb708106cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.965+0000 7fb70e559640 1 --2- 172.21.15.38:0/2368854769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb708101770 0x7fb708101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.965+0000 7fb70e559640 1 --2- 172.21.15.38:0/2368854769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708100d30 0x7fb708101180 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009900 tx=0x7fb6fc02a410 comp rx=0 tx=0).stop 2024-05-21T15:37:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.965+0000 7fb70e559640 1 -- 172.21.15.38:0/2368854769 >> 172.21.15.38:0/2368854769 conn(0x7fb7080fc300 msgr2=0x7fb7080fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.965+0000 7fb70e559640 1 -- 172.21.15.38:0/2368854769 shutdown_connections 2024-05-21T15:37:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.965+0000 7fb70e559640 1 -- 172.21.15.38:0/2368854769 wait complete. 2024-05-21T15:37:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.966+0000 7fb70e559640 1 Processor -- start 2024-05-21T15:37:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.968+0000 7fb70e559640 1 -- start start 2024-05-21T15:37:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.969+0000 7fb70e559640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708101770 0x7fb70819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.969+0000 7fb70e559640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7081068e0 0x7fb70819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.969+0000 7fb707fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708101770 0x7fb70819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.969+0000 7fb707fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708101770 0x7fb70819f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53326/0 (socket says 172.21.15.38:53326) 2024-05-21T15:37:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.969+0000 7fb70e559640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7081a4d40 0x7fb7081a7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.969+0000 7fb70e559640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb708114d50 con 0x7fb7081068e0 2024-05-21T15:37:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.969+0000 7fb70e559640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb708114bd0 con 0x7fb708101770 2024-05-21T15:37:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.970+0000 7fb70e559640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb708114ed0 con 0x7fb7081a4d40 2024-05-21T15:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.970+0000 7fb707fff640 1 -- 172.21.15.38:0/1447938177 learned_addr learned my addr 172.21.15.38:0/1447938177 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.970+0000 7fb7077fe640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7081068e0 0x7fb70819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.970+0000 7fb70caf6640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7081a4d40 0x7fb7081a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.970+0000 7fb707fff640 1 -- 172.21.15.38:0/1447938177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7081a4d40 msgr2=0x7fb7081a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.970+0000 7fb707fff640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7081a4d40 0x7fb7081a7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.970+0000 7fb707fff640 1 -- 172.21.15.38:0/1447938177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7081068e0 msgr2=0x7fb70819fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.970+0000 7fb707fff640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7081068e0 0x7fb70819fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.970+0000 7fb707fff640 1 -- 172.21.15.38:0/1447938177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7081a7890 con 0x7fb708101770 2024-05-21T15:37:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.970+0000 7fb70caf6640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7081a4d40 0x7fb7081a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.971+0000 7fb707fff640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708101770 0x7fb70819f880 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc033040 tx=0x7fb6fc005440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.971+0000 7fb7057fa640 1 -- 172.21.15.38:0/1447938177 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc040030 con 0x7fb708101770 2024-05-21T15:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.972+0000 7fb70e559640 1 -- 172.21.15.38:0/1447938177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7081a7b20 con 0x7fb708101770 2024-05-21T15:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.972+0000 7fb70e559640 1 -- 172.21.15.38:0/1447938177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7081a7fe0 con 0x7fb708101770 2024-05-21T15:37:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.972+0000 7fb7057fa640 1 -- 172.21.15.38:0/1447938177 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc03cb30 con 0x7fb708101770 2024-05-21T15:37:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.972+0000 7fb7057fa640 1 -- 172.21.15.38:0/1447938177 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc031b10 con 0x7fb708101770 2024-05-21T15:37:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.978+0000 7fb7153fd740 1 Processor -- start 2024-05-21T15:37:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.979+0000 7fb7153fd740 1 -- start start 2024-05-21T15:37:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.980+0000 7fb7057fa640 1 -- 172.21.15.38:0/1447938177 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb6fc043030 con 0x7fb708101770 2024-05-21T15:37:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.980+0000 7fb7153fd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f323301b0 0x555f326cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.980+0000 7fb7153fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323072f0 0x555f326d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.980+0000 7fb7153fd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f3230f4d0 0x555f326d72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.980+0000 7fb7153fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555f32634340 con 0x555f323072f0 2024-05-21T15:37:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.980+0000 7fb7153fd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555f325dc290 con 0x555f3230f4d0 2024-05-21T15:37:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.980+0000 7fb7153fd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555f32630300 con 0x555f323301b0 2024-05-21T15:37:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.980+0000 7fb70caf6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f323301b0 0x555f326cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.980+0000 7fb70caf6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f323301b0 0x555f326cfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:35250/0 (socket says 172.21.15.38:35250) 2024-05-21T15:37:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.981+0000 7fb70caf6640 1 -- 172.21.15.38:0/3625370687 learned_addr learned my addr 172.21.15.38:0/3625370687 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.981+0000 7fb7057fa640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb6f00a2cb0 0x7fb6f00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.982+0000 7fb707fff640 1 --2- 172.21.15.38:0/3625370687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323072f0 0x555f326d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.982+0000 7fb70caf6640 1 -- 172.21.15.38:0/3625370687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f3230f4d0 msgr2=0x555f326d72e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:37:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.982+0000 7fb70caf6640 1 --2- 172.21.15.38:0/3625370687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f3230f4d0 0x555f326d72e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.982+0000 7fb70caf6640 1 -- 172.21.15.38:0/3625370687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323072f0 msgr2=0x555f326d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.982+0000 7fb70caf6640 1 --2- 172.21.15.38:0/3625370687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323072f0 0x555f326d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.982+0000 7fb70caf6640 1 -- 172.21.15.38:0/3625370687 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f326008e0 con 0x555f323301b0 2024-05-21T15:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.982+0000 7fb70caf6640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb6f00a2cb0 0x7fb6f00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.982+0000 7fb7057fa640 1 -- 172.21.15.38:0/1447938177 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fb6fc0ef640 con 0x7fb708101770 2024-05-21T15:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.982+0000 7fb70caf6640 1 --2- 172.21.15.38:0/3625370687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f323301b0 0x555f326cfe60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb70000f4b0 tx=0x7fb70000f970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f6b4ceddd635edc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.982+0000 7fb707fff640 1 --2- 172.21.15.38:0/3625370687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323072f0 0x555f326d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.983+0000 7fb6ee7fc640 1 -- 172.21.15.38:0/3625370687 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700018070 con 0x555f323301b0 2024-05-21T15:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.983+0000 7fb6ee7fc640 1 -- 172.21.15.38:0/3625370687 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb700021070 con 0x555f323301b0 2024-05-21T15:37:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.984+0000 7fb7153fd740 1 -- 172.21.15.38:0/3625370687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f323301b0 msgr2=0x555f326cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.984+0000 7fb7153fd740 1 --2- 172.21.15.38:0/3625370687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f323301b0 0x555f326cfe60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb70000f4b0 tx=0x7fb70000f970 comp rx=0 tx=0).stop 2024-05-21T15:37:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.984+0000 7fb6ee7fc640 1 -- 172.21.15.38:0/3625370687 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70001b610 con 0x555f323301b0 2024-05-21T15:37:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.985+0000 7fb70caf6640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb6f00a2cb0 0x7fb6f00a51c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb700020000 tx=0x7fb70001bda0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.985+0000 7fb7153fd740 1 -- 172.21.15.38:0/3625370687 shutdown_connections 2024-05-21T15:37:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.985+0000 7fb7153fd740 1 --2- 172.21.15.38:0/3625370687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f3230f4d0 0x555f326d72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.985+0000 7fb7153fd740 1 --2- 172.21.15.38:0/3625370687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323072f0 0x555f326d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.985+0000 7fb7153fd740 1 --2- 172.21.15.38:0/3625370687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f323301b0 0x555f326cfe60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.985+0000 7fb7153fd740 1 -- 172.21.15.38:0/3625370687 >> 172.21.15.38:0/3625370687 conn(0x555f32324300 msgr2=0x555f326cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.985+0000 7fb7153fd740 1 -- 172.21.15.38:0/3625370687 shutdown_connections 2024-05-21T15:37:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.985+0000 7fb7153fd740 1 -- 172.21.15.38:0/3625370687 wait complete. 2024-05-21T15:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.985+0000 7fb7153fd740 1 -- start start 2024-05-21T15:37:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.986+0000 7fb7153fd740 1 Processor -- start 2024-05-21T15:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.986+0000 7fb7153fd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f323072f0 0x555f326eac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.986+0000 7fb7153fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f3230f4d0 0x555f326eb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.986+0000 7fb7153fd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323301b0 0x555f326f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.986+0000 7fb7153fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555f326e8d90 con 0x555f3230f4d0 2024-05-21T15:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.986+0000 7fb7153fd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555f325ef790 con 0x555f323301b0 2024-05-21T15:37:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.986+0000 7fb7153fd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555f326e8f10 con 0x555f323072f0 2024-05-21T15:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.986+0000 7fb7153fd740 10 client.0 _ll_register_callbacks cb 0x555f3262aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.987+0000 7fb707fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f323072f0 0x555f326eac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.987+0000 7fb707fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f323072f0 0x555f326eac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:35262/0 (socket says 172.21.15.38:35262) 2024-05-21T15:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.987+0000 7fb707fff640 1 -- 172.21.15.38:0/1488172081 learned_addr learned my addr 172.21.15.38:0/1488172081 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.987+0000 7fb7077fe640 1 --2- 172.21.15.38:0/1488172081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f3230f4d0 0x555f326eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.987+0000 7fb70caf6640 1 --2- 172.21.15.38:0/1488172081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323301b0 0x555f326f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.988+0000 7fb7077fe640 1 -- 172.21.15.38:0/1488172081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f323072f0 msgr2=0x555f326eac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.988+0000 7fb7077fe640 1 --2- 172.21.15.38:0/1488172081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f323072f0 0x555f326eac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.988+0000 7fb7077fe640 1 -- 172.21.15.38:0/1488172081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323301b0 msgr2=0x555f326f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.988+0000 7fb7077fe640 1 --2- 172.21.15.38:0/1488172081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323301b0 0x555f326f20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.988+0000 7fb7077fe640 1 -- 172.21.15.38:0/1488172081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f32348210 con 0x555f3230f4d0 2024-05-21T15:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.988+0000 7fb7077fe640 1 --2- 172.21.15.38:0/1488172081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f3230f4d0 0x555f326eb150 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009fd0 tx=0x7fb6f8011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d6b55f56e3bc0b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.988+0000 7fb6e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.989+0000 7fb6e77fe640 1 -- 172.21.15.38:0/1488172081 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8004440 con 0x555f3230f4d0 2024-05-21T15:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.990+0000 7fb6e77fe640 1 -- 172.21.15.38:0/1488172081 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fb6f80035f0 con 0x555f3230f4d0 2024-05-21T15:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.991+0000 7fb7153fd740 1 -- 172.21.15.38:0/1488172081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb700021070 con 0x555f3230f4d0 2024-05-21T15:37:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.992+0000 7fb6e77fe640 1 -- 172.21.15.38:0/1488172081 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb6f80037f0 con 0x555f3230f4d0 2024-05-21T15:37:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.992+0000 7fb7153fd740 10 client.16239 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.992+0000 7fb7153fd740 1 -- 172.21.15.38:0/1488172081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555f32414620 con 0x555f3230f4d0 2024-05-21T15:37:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.992+0000 7fb6e77fe640 1 -- 172.21.15.38:0/1488172081 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb700021070 con 0x555f3230f4d0 2024-05-21T15:37:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.992+0000 7fb6e77fe640 1 -- 172.21.15.38:0/1488172081 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fb6f801bc40 con 0x555f3230f4d0 2024-05-21T15:37:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.993+0000 7fb7153fd740 10 client.16239 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.993+0000 7fb7153fd740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.993+0000 7fb7153fd740 20 client.16239 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.993+0000 7fb7153fd740 1 --2- 172.21.15.38:0/1488172081 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555f322eed80 0x555f326f8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.993+0000 7fb7153fd740 4 client.16239 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:37:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.993+0000 7fb7153fd740 1 -- 172.21.15.38:0/1488172081 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f32641a70 con 0x555f322eed80 2024-05-21T15:37:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.994+0000 7fb707fff640 1 --2- 172.21.15.38:0/1488172081 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555f322eed80 0x555f326f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.997+0000 7fb707fff640 1 --2- 172.21.15.38:0/1488172081 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555f322eed80 0x555f326f8620 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8bfa715b9e9fd2e server_cookie=ea946637bf37b54c in_seq=0 out_seq=0 2024-05-21T15:37:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.997+0000 7fb6e77fe640 10 client.16239 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.999+0000 7fb6e77fe640 1 -- 172.21.15.38:0/1488172081 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555f32641a70 con 0x555f322eed80 2024-05-21T15:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.999+0000 7fb6e77fe640 10 client.16239 handle_command_reply: tid=0 2024-05-21T15:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:00.999+0000 7fb7153fd740 1 client.16239 shutdown 2024-05-21T15:37:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.000+0000 7fb7153fd740 1 -- 172.21.15.38:0/1488172081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f3230f4d0 msgr2=0x555f326eb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.000+0000 7fb7153fd740 1 --2- 172.21.15.38:0/1488172081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f3230f4d0 0x555f326eb150 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009fd0 tx=0x7fb6f8011800 comp rx=0 tx=0).stop 2024-05-21T15:37:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.000+0000 7fb7153fd740 1 -- 172.21.15.38:0/1488172081 shutdown_connections 2024-05-21T15:37:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.000+0000 7fb7153fd740 1 --2- 172.21.15.38:0/1488172081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323301b0 0x555f326f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.000+0000 7fb7153fd740 1 --2- 172.21.15.38:0/1488172081 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555f322eed80 0x555f326f8620 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.000+0000 7fb7153fd740 1 --2- 172.21.15.38:0/1488172081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f3230f4d0 0x555f326eb150 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.000+0000 7fb7153fd740 1 --2- 172.21.15.38:0/1488172081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f323072f0 0x555f326eac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.000+0000 7fb7153fd740 1 -- 172.21.15.38:0/1488172081 >> 172.21.15.38:0/1488172081 conn(0x555f32324300 msgr2=0x555f326ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.000+0000 7fb6e77fe640 0 client.16239 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.000+0000 7fb7153fd740 1 -- 172.21.15.38:0/1488172081 shutdown_connections 2024-05-21T15:37:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.000+0000 7fb7153fd740 1 -- 172.21.15.38:0/1488172081 wait complete. 2024-05-21T15:37:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.000+0000 7fb7153fd740 20 client.16239 trim_cache size 0 max 16384 2024-05-21T15:37:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.038+0000 7fb7153fd740 1 Processor -- start 2024-05-21T15:37:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.042+0000 7fb7153fd740 1 -- start start 2024-05-21T15:37:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.042+0000 7fb7153fd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323072f0 0x555f326cb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.042+0000 7fb7153fd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f3230f4d0 0x555f326f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.042+0000 7fb7153fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323301b0 0x555f326f2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.042+0000 7fb7153fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f400e6c0 con 0x555f323301b0 2024-05-21T15:37:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.042+0000 7fb7153fd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f40102f0 con 0x555f323072f0 2024-05-21T15:37:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.042+0000 7fb7153fd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f400ed90 con 0x555f3230f4d0 2024-05-21T15:37:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.043+0000 7fb707fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323072f0 0x555f326cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.043+0000 7fb707fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323072f0 0x555f326cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53358/0 (socket says 172.21.15.38:53358) 2024-05-21T15:37:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.043+0000 7fb707fff640 1 -- 172.21.15.38:0/1028305630 learned_addr learned my addr 172.21.15.38:0/1028305630 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.043+0000 7fb7077fe640 1 --2- 172.21.15.38:0/1028305630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f3230f4d0 0x555f326f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.043+0000 7fb70caf6640 1 --2- 172.21.15.38:0/1028305630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323301b0 0x555f326f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.044+0000 7fb707fff640 1 -- 172.21.15.38:0/1028305630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f3230f4d0 msgr2=0x555f326f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.044+0000 7fb707fff640 1 --2- 172.21.15.38:0/1028305630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f3230f4d0 0x555f326f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.044+0000 7fb707fff640 1 -- 172.21.15.38:0/1028305630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323301b0 msgr2=0x555f326f2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.044+0000 7fb707fff640 1 --2- 172.21.15.38:0/1028305630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323301b0 0x555f326f2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.044+0000 7fb707fff640 1 -- 172.21.15.38:0/1028305630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f326cfa10 con 0x555f323072f0 2024-05-21T15:37:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.046+0000 7fb707fff640 1 --2- 172.21.15.38:0/1028305630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323072f0 0x555f326cb870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009680 tx=0x7fb6fc0ecf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab535bd8ae1b1302 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.047+0000 7fb6edffb640 1 -- 172.21.15.38:0/1028305630 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0c01a0 con 0x555f323072f0 2024-05-21T15:37:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.047+0000 7fb6edffb640 1 -- 172.21.15.38:0/1028305630 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc0c4a60 con 0x555f323072f0 2024-05-21T15:37:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.047+0000 7fb6edffb640 1 -- 172.21.15.38:0/1028305630 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0b8460 con 0x555f323072f0 2024-05-21T15:37:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.049+0000 7fb7153fd740 1 -- 172.21.15.38:0/1028305630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323072f0 msgr2=0x555f326cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.049+0000 7fb7153fd740 1 --2- 172.21.15.38:0/1028305630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323072f0 0x555f326cb870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009680 tx=0x7fb6fc0ecf60 comp rx=0 tx=0).stop 2024-05-21T15:37:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.057+0000 7fb7153fd740 1 -- 172.21.15.38:0/1028305630 shutdown_connections 2024-05-21T15:37:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.057+0000 7fb7153fd740 1 --2- 172.21.15.38:0/1028305630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323301b0 0x555f326f2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.057+0000 7fb7153fd740 1 --2- 172.21.15.38:0/1028305630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f3230f4d0 0x555f326f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.057+0000 7fb7153fd740 1 --2- 172.21.15.38:0/1028305630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323072f0 0x555f326cb870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.057+0000 7fb7153fd740 1 -- 172.21.15.38:0/1028305630 >> 172.21.15.38:0/1028305630 conn(0x555f32324300 msgr2=0x555f326cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.059+0000 7fb7153fd740 1 -- 172.21.15.38:0/1028305630 shutdown_connections 2024-05-21T15:37:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.059+0000 7fb7153fd740 1 -- 172.21.15.38:0/1028305630 wait complete. 2024-05-21T15:37:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.060+0000 7fb7153fd740 1 -- start start 2024-05-21T15:37:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.060+0000 7fb7153fd740 1 Processor -- start 2024-05-21T15:37:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.061+0000 7fb7153fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323072f0 0x555f326ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.061+0000 7fb7153fd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f3230f4d0 0x555f326efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.061+0000 7fb7153fd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323301b0 0x555f3275e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.061+0000 7fb7153fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f400fb60 con 0x555f323072f0 2024-05-21T15:37:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.061+0000 7fb7153fd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f400f460 con 0x555f323301b0 2024-05-21T15:37:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.061+0000 7fb7153fd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f400c6a0 con 0x555f3230f4d0 2024-05-21T15:37:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.061+0000 7fb7153fd740 10 client.0 _ll_register_callbacks cb 0x7fb6f8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.061+0000 7fb70caf6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323301b0 0x555f3275e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.061+0000 7fb70caf6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323301b0 0x555f3275e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53370/0 (socket says 172.21.15.38:53370) 2024-05-21T15:37:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.061+0000 7fb7077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f3230f4d0 0x555f326efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.061+0000 7fb70caf6640 1 -- 172.21.15.38:0/676593512 learned_addr learned my addr 172.21.15.38:0/676593512 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.061+0000 7fb707fff640 1 --2- 172.21.15.38:0/676593512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323072f0 0x555f326ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.062+0000 7fb707fff640 1 -- 172.21.15.38:0/676593512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f3230f4d0 msgr2=0x555f326efcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.062+0000 7fb707fff640 1 --2- 172.21.15.38:0/676593512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f3230f4d0 0x555f326efcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.062+0000 7fb707fff640 1 -- 172.21.15.38:0/676593512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323301b0 msgr2=0x555f3275e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.062+0000 7fb707fff640 1 --2- 172.21.15.38:0/676593512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323301b0 0x555f3275e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.063+0000 7fb707fff640 1 -- 172.21.15.38:0/676593512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f3275ef00 con 0x555f323072f0 2024-05-21T15:37:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.064+0000 7fb707fff640 1 --2- 172.21.15.38:0/676593512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323072f0 0x555f326ef790 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc005ab0 tx=0x7fb6fc005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4007f218132f8b37 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.065+0000 7fb6e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:37:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.065+0000 7fb6e77fe640 1 -- 172.21.15.38:0/676593512 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0b9e90 con 0x555f323072f0 2024-05-21T15:37:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.065+0000 7fb6e77fe640 1 -- 172.21.15.38:0/676593512 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fb6fc0b8460 con 0x555f323072f0 2024-05-21T15:37:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.066+0000 7fb7153fd740 1 -- 172.21.15.38:0/676593512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6fc0c4a60 con 0x555f323072f0 2024-05-21T15:37:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.067+0000 7fb6e77fe640 1 -- 172.21.15.38:0/676593512 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb6fc0d9070 con 0x555f323072f0 2024-05-21T15:37:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.067+0000 7fb7153fd740 10 client.16245 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.067+0000 7fb7153fd740 1 -- 172.21.15.38:0/676593512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555f32760060 con 0x555f323072f0 2024-05-21T15:37:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.068+0000 7fb6e77fe640 1 -- 172.21.15.38:0/676593512 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc0c4a60 con 0x555f323072f0 2024-05-21T15:37:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.068+0000 7fb6e77fe640 1 -- 172.21.15.38:0/676593512 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fb6fc0c46e0 con 0x555f323072f0 2024-05-21T15:37:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.068+0000 7fb7153fd740 10 client.16245 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.068+0000 7fb7153fd740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.068+0000 7fb7153fd740 20 client.16245 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.068+0000 7fb7153fd740 1 --2- 172.21.15.38:0/676593512 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555f322c41c0 0x555f32762a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.068+0000 7fb7153fd740 4 client.16245 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:37:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.068+0000 7fb7153fd740 1 -- 172.21.15.38:0/676593512 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f3263bdc0 con 0x555f322c41c0 2024-05-21T15:37:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.070+0000 7fb7077fe640 1 --2- 172.21.15.38:0/676593512 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555f322c41c0 0x555f32762a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.074+0000 7fb7077fe640 1 --2- 172.21.15.38:0/676593512 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555f322c41c0 0x555f32762a90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7475774b4601de07 server_cookie=b3568f4e225d1bb1 in_seq=0 out_seq=0 2024-05-21T15:37:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.074+0000 7fb6e77fe640 10 client.16245 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:01.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:00 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:37:01.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:00 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:37:01.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:00 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:37:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.733+0000 7fb6e77fe640 1 -- 172.21.15.38:0/676593512 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555f3263bdc0 con 0x555f322c41c0 2024-05-21T15:37:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.733+0000 7fb6e77fe640 10 client.16245 handle_command_reply: tid=0 2024-05-21T15:37:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.733+0000 7fb7153fd740 1 client.16245 shutdown 2024-05-21T15:37:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.733+0000 7fb7153fd740 1 -- 172.21.15.38:0/676593512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323072f0 msgr2=0x555f326ef790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.733+0000 7fb7153fd740 1 --2- 172.21.15.38:0/676593512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323072f0 0x555f326ef790 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc005ab0 tx=0x7fb6fc005730 comp rx=0 tx=0).stop 2024-05-21T15:37:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.733+0000 7fb7153fd740 1 -- 172.21.15.38:0/676593512 shutdown_connections 2024-05-21T15:37:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.733+0000 7fb7153fd740 1 --2- 172.21.15.38:0/676593512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f323301b0 0x555f3275e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.733+0000 7fb7153fd740 1 --2- 172.21.15.38:0/676593512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f3230f4d0 0x555f326efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.733+0000 7fb7153fd740 1 --2- 172.21.15.38:0/676593512 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555f322c41c0 0x555f32762a90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.733+0000 7fb7153fd740 1 --2- 172.21.15.38:0/676593512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f323072f0 0x555f326ef790 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.733+0000 7fb7153fd740 1 -- 172.21.15.38:0/676593512 >> 172.21.15.38:0/676593512 conn(0x555f32324300 msgr2=0x555f326ea7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.733+0000 7fb6e77fe640 0 client.16245 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.733+0000 7fb7153fd740 1 -- 172.21.15.38:0/676593512 shutdown_connections 2024-05-21T15:37:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.734+0000 7fb7153fd740 1 -- 172.21.15.38:0/676593512 wait complete. 2024-05-21T15:37:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.734+0000 7fb7153fd740 20 client.16245 trim_cache size 0 max 16384 2024-05-21T15:37:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:37:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:37:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:37:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:37:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.736+0000 7fb70e559640 1 -- 172.21.15.38:0/1447938177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb6f00a2cb0 msgr2=0x7fb6f00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.736+0000 7fb70e559640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb6f00a2cb0 0x7fb6f00a51c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb700020000 tx=0x7fb70001bda0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.736+0000 7fb70e559640 1 -- 172.21.15.38:0/1447938177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708101770 msgr2=0x7fb70819f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.736+0000 7fb70e559640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708101770 0x7fb70819f880 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc033040 tx=0x7fb6fc005440 comp rx=0 tx=0).stop 2024-05-21T15:37:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.736+0000 7fb70e559640 1 -- 172.21.15.38:0/1447938177 shutdown_connections 2024-05-21T15:37:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.737+0000 7fb70e559640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7081a4d40 0x7fb7081a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.737+0000 7fb70e559640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb6f00a2cb0 0x7fb6f00a51c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.737+0000 7fb70e559640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7081068e0 0x7fb70819fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.737+0000 7fb70e559640 1 --2- 172.21.15.38:0/1447938177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708101770 0x7fb70819f880 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.737+0000 7fb70e559640 1 -- 172.21.15.38:0/1447938177 >> 172.21.15.38:0/1447938177 conn(0x7fb7080fc300 msgr2=0x7fb7081028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.737+0000 7fb70e559640 1 -- 172.21.15.38:0/1447938177 shutdown_connections 2024-05-21T15:37:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:01.738+0000 7fb70e559640 1 -- 172.21.15.38:0/1447938177 wait complete. 2024-05-21T15:37:02.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:01 smithi038 ceph-mon[31995]: pgmap v429: 97 pgs: 97 active+clean; 13 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 241 MiB/s wr, 29.66k op/s 2024-05-21T15:37:02.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:01 smithi053 ceph-mon[36496]: pgmap v429: 97 pgs: 97 active+clean; 13 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 241 MiB/s wr, 29.66k op/s 2024-05-21T15:37:02.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:01 smithi045 ceph-mon[36409]: pgmap v429: 97 pgs: 97 active+clean; 13 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 241 MiB/s wr, 29.66k op/s 2024-05-21T15:37:04.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:03 smithi053 ceph-mon[36496]: pgmap v430: 97 pgs: 97 active+clean; 13 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 250 MiB/s wr, 28.98k op/s 2024-05-21T15:37:04.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:03 smithi045 ceph-mon[36409]: pgmap v430: 97 pgs: 97 active+clean; 13 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 250 MiB/s wr, 28.98k op/s 2024-05-21T15:37:04.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:03 smithi038 ceph-mon[31995]: pgmap v430: 97 pgs: 97 active+clean; 13 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 250 MiB/s wr, 28.98k op/s 2024-05-21T15:37:06.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:05 smithi053 ceph-mon[36496]: pgmap v431: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 255 MiB/s wr, 27.57k op/s 2024-05-21T15:37:06.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:05 smithi045 ceph-mon[36409]: pgmap v431: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 255 MiB/s wr, 27.57k op/s 2024-05-21T15:37:06.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:05 smithi038 ceph-mon[31995]: pgmap v431: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 255 MiB/s wr, 27.57k op/s 2024-05-21T15:37:08.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:07 smithi053 ceph-mon[36496]: pgmap v432: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 243 MiB/s wr, 25.43k op/s 2024-05-21T15:37:08.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:07 smithi045 ceph-mon[36409]: pgmap v432: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 243 MiB/s wr, 25.43k op/s 2024-05-21T15:37:08.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:07 smithi038 ceph-mon[31995]: pgmap v432: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 243 MiB/s wr, 25.43k op/s 2024-05-21T15:37:10.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:09 smithi053 ceph-mon[36496]: pgmap v433: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 243 MiB/s wr, 24.30k op/s 2024-05-21T15:37:10.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:09 smithi045 ceph-mon[36409]: pgmap v433: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 243 MiB/s wr, 24.30k op/s 2024-05-21T15:37:10.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:09 smithi038 ceph-mon[31995]: pgmap v433: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 243 MiB/s wr, 24.30k op/s 2024-05-21T15:37:12.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:11 smithi053 ceph-mon[36496]: pgmap v434: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 279 MiB/s wr, 27.15k op/s 2024-05-21T15:37:12.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:11 smithi045 ceph-mon[36409]: pgmap v434: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 279 MiB/s wr, 27.15k op/s 2024-05-21T15:37:12.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:11 smithi038 ceph-mon[31995]: pgmap v434: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 279 MiB/s wr, 27.15k op/s 2024-05-21T15:37:14.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:13 smithi053 ceph-mon[36496]: pgmap v435: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 279 MiB/s wr, 26.83k op/s 2024-05-21T15:37:14.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:13 smithi045 ceph-mon[36409]: pgmap v435: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 279 MiB/s wr, 26.83k op/s 2024-05-21T15:37:14.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:13 smithi038 ceph-mon[31995]: pgmap v435: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 279 MiB/s wr, 26.83k op/s 2024-05-21T15:37:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.944+0000 7f272d2c3640 1 -- 172.21.15.38:0/2896586831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f272810d390 msgr2=0x7f272810f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.944+0000 7f272d2c3640 1 --2- 172.21.15.38:0/2896586831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f272810d390 0x7f272810f870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2718009900 tx=0x7f271802a420 comp rx=0 tx=0).stop 2024-05-21T15:37:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.946+0000 7f272d2c3640 1 -- 172.21.15.38:0/2896586831 shutdown_connections 2024-05-21T15:37:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.946+0000 7f272d2c3640 1 --2- 172.21.15.38:0/2896586831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f272810ff00 0x7f2728112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.946+0000 7f272d2c3640 1 --2- 172.21.15.38:0/2896586831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f272810d390 0x7f272810f870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.946+0000 7f272d2c3640 1 --2- 172.21.15.38:0/2896586831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f272810a9a0 0x7f272810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.946+0000 7f272d2c3640 1 -- 172.21.15.38:0/2896586831 >> 172.21.15.38:0/2896586831 conn(0x7f272806f4c0 msgr2=0x7f2728071970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.948+0000 7f272d2c3640 1 -- 172.21.15.38:0/2896586831 shutdown_connections 2024-05-21T15:37:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.948+0000 7f272d2c3640 1 -- 172.21.15.38:0/2896586831 wait complete. 2024-05-21T15:37:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.950+0000 7f272d2c3640 1 Processor -- start 2024-05-21T15:37:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.950+0000 7f272d2c3640 1 -- start start 2024-05-21T15:37:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.950+0000 7f272d2c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f272810a9a0 0x7f272811ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.950+0000 7f272d2c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f272810d390 0x7f272811b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.950+0000 7f272d2c3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f272810ff00 0x7f272811e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.950+0000 7f2726ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f272810a9a0 0x7f272811ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.950+0000 7f2726ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f272810a9a0 0x7f272811ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33628/0 (socket says 172.21.15.38:33628) 2024-05-21T15:37:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.950+0000 7f272d2c3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2728114e80 con 0x7f272810a9a0 2024-05-21T15:37:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.950+0000 7f272d2c3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2728114d00 con 0x7f272810d390 2024-05-21T15:37:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.950+0000 7f272d2c3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2728115000 con 0x7f272810ff00 2024-05-21T15:37:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.951+0000 7f2726ffd640 1 -- 172.21.15.38:0/1366618637 learned_addr learned my addr 172.21.15.38:0/1366618637 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.952+0000 7f27267fc640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f272810d390 0x7f272811b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.952+0000 7f27277fe640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f272810ff00 0x7f272811e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.953+0000 7f2726ffd640 1 -- 172.21.15.38:0/1366618637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f272810ff00 msgr2=0x7f272811e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.953+0000 7f2726ffd640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f272810ff00 0x7f272811e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.953+0000 7f2726ffd640 1 -- 172.21.15.38:0/1366618637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f272810d390 msgr2=0x7f272811b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.953+0000 7f2726ffd640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f272810d390 0x7f272811b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.953+0000 7f2726ffd640 1 -- 172.21.15.38:0/1366618637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272811e760 con 0x7f272810a9a0 2024-05-21T15:37:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.954+0000 7f27267fc640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f272810d390 0x7f272811b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:37:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.956+0000 7f2726ffd640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f272810a9a0 0x7f272811ac40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f271c009fd0 tx=0x7f271c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.957+0000 7f270ffff640 1 -- 172.21.15.38:0/1366618637 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c011830 con 0x7f272810a9a0 2024-05-21T15:37:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.957+0000 7f272d2c3640 1 -- 172.21.15.38:0/1366618637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272811ea50 con 0x7f272810a9a0 2024-05-21T15:37:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.957+0000 7f272d2c3640 1 -- 172.21.15.38:0/1366618637 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f272811f020 con 0x7f272810a9a0 2024-05-21T15:37:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.957+0000 7f270ffff640 1 -- 172.21.15.38:0/1366618637 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f271c016070 con 0x7f272810a9a0 2024-05-21T15:37:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.957+0000 7f270ffff640 1 -- 172.21.15.38:0/1366618637 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c00f2e0 con 0x7f272810a9a0 2024-05-21T15:37:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.961+0000 7f2734163740 1 Processor -- start 2024-05-21T15:37:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.962+0000 7f2734163740 1 -- start start 2024-05-21T15:37:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.962+0000 7f2734163740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fe51b0 0x55b468385170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.962+0000 7f2734163740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 0x55b4683856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.962+0000 7f2734163740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fc44d0 0x55b46838c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.962+0000 7f2734163740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b4682e9340 con 0x55b467fe51b0 2024-05-21T15:37:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.962+0000 7f2734163740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b468291290 con 0x55b467fc44d0 2024-05-21T15:37:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.962+0000 7f2734163740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b4682e5300 con 0x55b467fbc2f0 2024-05-21T15:37:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.962+0000 7f27277fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fe51b0 0x55b468385170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.962+0000 7f27277fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fe51b0 0x55b468385170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33644/0 (socket says 172.21.15.38:33644) 2024-05-21T15:37:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.962+0000 7f27277fe640 1 -- 172.21.15.38:0/3818950121 learned_addr learned my addr 172.21.15.38:0/3818950121 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.963+0000 7f27277fe640 1 -- 172.21.15.38:0/3818950121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 msgr2=0x55b4683856f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:37:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.963+0000 7f27277fe640 1 --2- 172.21.15.38:0/3818950121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 0x55b4683856f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.963+0000 7f27277fe640 1 -- 172.21.15.38:0/3818950121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fc44d0 msgr2=0x55b46838c5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:37:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.963+0000 7f27277fe640 1 --2- 172.21.15.38:0/3818950121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fc44d0 0x55b46838c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.963+0000 7f27277fe640 1 -- 172.21.15.38:0/3818950121 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4682b58e0 con 0x55b467fe51b0 2024-05-21T15:37:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.963+0000 7f27277fe640 1 --2- 172.21.15.38:0/3818950121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fe51b0 0x55b468385170 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f272000bf10 tx=0x7f272000a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffca930099e2294 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.965+0000 7f270cff9640 1 -- 172.21.15.38:0/3818950121 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720036770 con 0x55b467fe51b0 2024-05-21T15:37:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.965+0000 7f270cff9640 1 -- 172.21.15.38:0/3818950121 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f272003c070 con 0x55b467fe51b0 2024-05-21T15:37:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.965+0000 7f270cff9640 1 -- 172.21.15.38:0/3818950121 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720037cd0 con 0x55b467fe51b0 2024-05-21T15:37:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.966+0000 7f270ffff640 1 -- 172.21.15.38:0/1366618637 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f271c0119d0 con 0x7f272810a9a0 2024-05-21T15:37:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.967+0000 7f2734163740 1 -- 172.21.15.38:0/3818950121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fe51b0 msgr2=0x55b468385170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.967+0000 7f2734163740 1 --2- 172.21.15.38:0/3818950121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fe51b0 0x55b468385170 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f272000bf10 tx=0x7f272000a680 comp rx=0 tx=0).stop 2024-05-21T15:37:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.967+0000 7f270ffff640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27100a2d80 0x7f27100a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.967+0000 7f270ffff640 1 -- 172.21.15.38:0/1366618637 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f271c0ce770 con 0x7f272810a9a0 2024-05-21T15:37:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.968+0000 7f2734163740 1 -- 172.21.15.38:0/3818950121 shutdown_connections 2024-05-21T15:37:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.968+0000 7f2734163740 1 --2- 172.21.15.38:0/3818950121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fc44d0 0x55b46838c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.968+0000 7f2734163740 1 --2- 172.21.15.38:0/3818950121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 0x55b4683856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.968+0000 7f2734163740 1 --2- 172.21.15.38:0/3818950121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fe51b0 0x55b468385170 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.968+0000 7f2734163740 1 -- 172.21.15.38:0/3818950121 >> 172.21.15.38:0/3818950121 conn(0x55b467fd9300 msgr2=0x55b468380b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.968+0000 7f27277fe640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27100a2d80 0x7f27100a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.969+0000 7f2734163740 1 -- 172.21.15.38:0/3818950121 shutdown_connections 2024-05-21T15:37:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.970+0000 7f2734163740 1 -- 172.21.15.38:0/3818950121 wait complete. 2024-05-21T15:37:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.970+0000 7f27277fe640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27100a2d80 0x7f27100a5290 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f272000b010 tx=0x7f27200377d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.971+0000 7f2734163740 1 -- start start 2024-05-21T15:37:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.971+0000 7f2734163740 1 Processor -- start 2024-05-21T15:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.971+0000 7f2734163740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 0x55b46838ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.972+0000 7f2734163740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fe51b0 0x55b46838f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.972+0000 7f2734163740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fa3d80 0x55b468396150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.972+0000 7f2734163740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b46838cf50 con 0x55b467fa3d80 2024-05-21T15:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.972+0000 7f2726ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 0x55b46838ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.972+0000 7f2726ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 0x55b46838ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:47838/0 (socket says 172.21.15.38:47838) 2024-05-21T15:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.972+0000 7f2734163740 1 -- 172.21.15.38:0/1245805852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b4682a4790 con 0x55b467fe51b0 2024-05-21T15:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.972+0000 7f2734163740 1 -- 172.21.15.38:0/1245805852 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b46838d0d0 con 0x55b467fbc2f0 2024-05-21T15:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.972+0000 7f2734163740 10 client.0 _ll_register_callbacks cb 0x55b4682dfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.972+0000 7f2726ffd640 1 -- 172.21.15.38:0/1245805852 learned_addr learned my addr 172.21.15.38:0/1245805852 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.973+0000 7f27267fc640 1 --2- 172.21.15.38:0/1245805852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fe51b0 0x55b46838f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.973+0000 7f27277fe640 1 --2- 172.21.15.38:0/1245805852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fa3d80 0x55b468396150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.974+0000 7f27267fc640 1 -- 172.21.15.38:0/1245805852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 msgr2=0x55b46838ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.975+0000 7f27267fc640 1 --2- 172.21.15.38:0/1245805852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 0x55b46838ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.975+0000 7f27267fc640 1 -- 172.21.15.38:0/1245805852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fa3d80 msgr2=0x55b468396150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.975+0000 7f27267fc640 1 --2- 172.21.15.38:0/1245805852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fa3d80 0x55b468396150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.975+0000 7f27267fc640 1 -- 172.21.15.38:0/1245805852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b467ffd210 con 0x55b467fe51b0 2024-05-21T15:37:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.975+0000 7f27277fe640 1 --2- 172.21.15.38:0/1245805852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fa3d80 0x55b468396150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:37:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.977+0000 7f27267fc640 1 --2- 172.21.15.38:0/1245805852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fe51b0 0x55b46838f1f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f271802aa10 tx=0x7f2718005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31503ff751eed9ed server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.978+0000 7f270aff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:37:14.980 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.981+0000 7f270aff5640 1 -- 172.21.15.38:0/1245805852 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718035030 con 0x55b467fe51b0 2024-05-21T15:37:14.980 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.981+0000 7f270aff5640 1 -- 172.21.15.38:0/1245805852 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f27180055f0 con 0x55b467fe51b0 2024-05-21T15:37:14.980 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.982+0000 7f2734163740 1 -- 172.21.15.38:0/1245805852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f272003c070 con 0x55b467fe51b0 2024-05-21T15:37:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.984+0000 7f270aff5640 1 -- 172.21.15.38:0/1245805852 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2718035030 con 0x55b467fe51b0 2024-05-21T15:37:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.988+0000 7f2734163740 10 client.25922 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.988+0000 7f2734163740 1 -- 172.21.15.38:0/1245805852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b4680c9620 con 0x55b467fe51b0 2024-05-21T15:37:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.991+0000 7f270aff5640 1 -- 172.21.15.38:0/1245805852 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f272003c070 con 0x55b467fe51b0 2024-05-21T15:37:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.991+0000 7f270aff5640 1 -- 172.21.15.38:0/1245805852 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f2718031a40 con 0x55b467fe51b0 2024-05-21T15:37:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.992+0000 7f2734163740 10 client.25922 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.992+0000 7f2734163740 10 client.25922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.992+0000 7f2734163740 20 client.25922 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.992+0000 7f2734163740 1 --2- 172.21.15.38:0/1245805852 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b467f791c0 0x55b46839c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.992+0000 7f2734163740 4 client.25922 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:37:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.992+0000 7f2734163740 1 -- 172.21.15.38:0/1245805852 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4682f6a70 con 0x55b467f791c0 2024-05-21T15:37:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.993+0000 7f2726ffd640 1 --2- 172.21.15.38:0/1245805852 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b467f791c0 0x55b46839c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.998+0000 7f2726ffd640 1 --2- 172.21.15.38:0/1245805852 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b467f791c0 0x55b46839c4d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85bf370e8274aa6d server_cookie=5592416993c1435e in_seq=0 out_seq=0 2024-05-21T15:37:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:14.998+0000 7f270aff5640 10 client.25922 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.000+0000 7f270aff5640 1 -- 172.21.15.38:0/1245805852 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b4682f6a70 con 0x55b467f791c0 2024-05-21T15:37:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.000+0000 7f270aff5640 10 client.25922 handle_command_reply: tid=0 2024-05-21T15:37:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.004+0000 7f2734163740 1 client.25922 shutdown 2024-05-21T15:37:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.007+0000 7f2734163740 1 -- 172.21.15.38:0/1245805852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fe51b0 msgr2=0x55b46838f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.007+0000 7f2734163740 1 --2- 172.21.15.38:0/1245805852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fe51b0 0x55b46838f1f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f271802aa10 tx=0x7f2718005270 comp rx=0 tx=0).stop 2024-05-21T15:37:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.012+0000 7f2734163740 1 -- 172.21.15.38:0/1245805852 shutdown_connections 2024-05-21T15:37:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.012+0000 7f2734163740 1 --2- 172.21.15.38:0/1245805852 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b467f791c0 0x55b46839c4d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.012+0000 7f2734163740 1 --2- 172.21.15.38:0/1245805852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fa3d80 0x55b468396150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.012+0000 7f2734163740 1 --2- 172.21.15.38:0/1245805852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fe51b0 0x55b46838f1f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.012+0000 7f2734163740 1 --2- 172.21.15.38:0/1245805852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 0x55b46838ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.012+0000 7f2734163740 1 -- 172.21.15.38:0/1245805852 >> 172.21.15.38:0/1245805852 conn(0x55b467fd9300 msgr2=0x55b46837fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.012+0000 7f270aff5640 0 client.25922 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.013+0000 7f2734163740 1 -- 172.21.15.38:0/1245805852 shutdown_connections 2024-05-21T15:37:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.013+0000 7f2734163740 1 -- 172.21.15.38:0/1245805852 wait complete. 2024-05-21T15:37:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.013+0000 7f2734163740 20 client.25922 trim_cache size 0 max 16384 2024-05-21T15:37:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.052+0000 7f2734163740 1 Processor -- start 2024-05-21T15:37:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.052+0000 7f2734163740 1 -- start start 2024-05-21T15:37:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.053+0000 7f2734163740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fa3d80 0x55b468380560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.053+0000 7f2734163740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 0x55b468380aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.053+0000 7f2734163740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fe51b0 0x55b46837f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.053+0000 7f2734163740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f272807bcd0 con 0x55b467fe51b0 2024-05-21T15:37:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.053+0000 7f2734163740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f272807d900 con 0x55b467fa3d80 2024-05-21T15:37:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.053+0000 7f2734163740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f272807c3a0 con 0x55b467fbc2f0 2024-05-21T15:37:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.055+0000 7f2726ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fa3d80 0x55b468380560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.055+0000 7f2726ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fa3d80 0x55b468380560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:49270/0 (socket says 172.21.15.38:49270) 2024-05-21T15:37:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.055+0000 7f2726ffd640 1 -- 172.21.15.38:0/2942157163 learned_addr learned my addr 172.21.15.38:0/2942157163 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.055+0000 7f27267fc640 1 --2- 172.21.15.38:0/2942157163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 0x55b468380aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.055+0000 7f2726ffd640 1 -- 172.21.15.38:0/2942157163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 msgr2=0x55b468380aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.055+0000 7f2726ffd640 1 --2- 172.21.15.38:0/2942157163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 0x55b468380aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.055+0000 7f2726ffd640 1 -- 172.21.15.38:0/2942157163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fe51b0 msgr2=0x55b46837f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.055+0000 7f2726ffd640 1 --2- 172.21.15.38:0/2942157163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fe51b0 0x55b46837f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.055+0000 7f2726ffd640 1 -- 172.21.15.38:0/2942157163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b468398d10 con 0x55b467fa3d80 2024-05-21T15:37:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.055+0000 7f2726ffd640 1 --2- 172.21.15.38:0/2942157163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fa3d80 0x55b468380560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f271c09a9f0 tx=0x7f271c010910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d68e115f37f850ad server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.056+0000 7f270c7f8640 1 -- 172.21.15.38:0/2942157163 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c09af20 con 0x55b467fa3d80 2024-05-21T15:37:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.056+0000 7f270c7f8640 1 -- 172.21.15.38:0/2942157163 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f271c0b6070 con 0x55b467fa3d80 2024-05-21T15:37:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.057+0000 7f2734163740 1 -- 172.21.15.38:0/2942157163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fa3d80 msgr2=0x55b468380560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.057+0000 7f270c7f8640 1 -- 172.21.15.38:0/2942157163 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c011cc0 con 0x55b467fa3d80 2024-05-21T15:37:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.057+0000 7f2734163740 1 --2- 172.21.15.38:0/2942157163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fa3d80 0x55b468380560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f271c09a9f0 tx=0x7f271c010910 comp rx=0 tx=0).stop 2024-05-21T15:37:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.059+0000 7f2734163740 1 -- 172.21.15.38:0/2942157163 shutdown_connections 2024-05-21T15:37:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.059+0000 7f2734163740 1 --2- 172.21.15.38:0/2942157163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fe51b0 0x55b46837f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.059+0000 7f2734163740 1 --2- 172.21.15.38:0/2942157163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fbc2f0 0x55b468380aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.059+0000 7f2734163740 1 --2- 172.21.15.38:0/2942157163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fa3d80 0x55b468380560 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.059+0000 7f2734163740 1 -- 172.21.15.38:0/2942157163 >> 172.21.15.38:0/2942157163 conn(0x55b467fd9300 msgr2=0x55b468389330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.063+0000 7f2734163740 1 -- 172.21.15.38:0/2942157163 shutdown_connections 2024-05-21T15:37:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.063+0000 7f2734163740 1 -- 172.21.15.38:0/2942157163 wait complete. 2024-05-21T15:37:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.063+0000 7f2734163740 1 -- start start 2024-05-21T15:37:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.064+0000 7f2734163740 1 Processor -- start 2024-05-21T15:37:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.064+0000 7f2734163740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fa3d80 0x55b468392530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.064+0000 7f2734163740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fbc2f0 0x55b468392a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.064+0000 7f2734163740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fe51b0 0x55b468402870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.064+0000 7f2734163740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f272807d170 con 0x55b467fa3d80 2024-05-21T15:37:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.064+0000 7f2734163740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f272807ca70 con 0x55b467fbc2f0 2024-05-21T15:37:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.064+0000 7f2734163740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2728079c70 con 0x55b467fe51b0 2024-05-21T15:37:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.064+0000 7f2734163740 10 client.0 _ll_register_callbacks cb 0x7f2718009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.065+0000 7f27267fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fbc2f0 0x55b468392a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.065+0000 7f27267fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fbc2f0 0x55b468392a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:49276/0 (socket says 172.21.15.38:49276) 2024-05-21T15:37:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.065+0000 7f27267fc640 1 -- 172.21.15.38:0/1782892314 learned_addr learned my addr 172.21.15.38:0/1782892314 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.065+0000 7f2726ffd640 1 --2- 172.21.15.38:0/1782892314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fa3d80 0x55b468392530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.065+0000 7f27277fe640 1 --2- 172.21.15.38:0/1782892314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fe51b0 0x55b468402870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.066+0000 7f27267fc640 1 -- 172.21.15.38:0/1782892314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fe51b0 msgr2=0x55b468402870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.066+0000 7f27267fc640 1 --2- 172.21.15.38:0/1782892314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fe51b0 0x55b468402870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.066+0000 7f27267fc640 1 -- 172.21.15.38:0/1782892314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fa3d80 msgr2=0x55b468392530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.066+0000 7f27267fc640 1 --2- 172.21.15.38:0/1782892314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fa3d80 0x55b468392530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.066+0000 7f27267fc640 1 -- 172.21.15.38:0/1782892314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b468395a30 con 0x55b467fbc2f0 2024-05-21T15:37:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.066+0000 7f27267fc640 1 --2- 172.21.15.38:0/1782892314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fbc2f0 0x55b468392a70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2718006010 tx=0x7f271802a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15cac92e91914af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.066+0000 7f270a7f4640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:37:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.068+0000 7f270a7f4640 1 -- 172.21.15.38:0/1782892314 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718035030 con 0x55b467fbc2f0 2024-05-21T15:37:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.068+0000 7f2734163740 1 -- 172.21.15.38:0/1782892314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f271c0b6070 con 0x55b467fbc2f0 2024-05-21T15:37:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.068+0000 7f270a7f4640 1 -- 172.21.15.38:0/1782892314 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f27180393e0 con 0x55b467fbc2f0 2024-05-21T15:37:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.069+0000 7f270a7f4640 1 -- 172.21.15.38:0/1782892314 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2718009520 con 0x55b467fbc2f0 2024-05-21T15:37:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.070+0000 7f2734163740 10 client.25928 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.070+0000 7f2734163740 1 -- 172.21.15.38:0/1782892314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b468403d60 con 0x55b467fbc2f0 2024-05-21T15:37:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.072+0000 7f270a7f4640 1 -- 172.21.15.38:0/1782892314 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f271c0b6070 con 0x55b467fbc2f0 2024-05-21T15:37:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.072+0000 7f270a7f4640 1 -- 172.21.15.38:0/1782892314 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f2718031a40 con 0x55b467fbc2f0 2024-05-21T15:37:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.072+0000 7f2734163740 10 client.25928 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.072+0000 7f2734163740 10 client.25928 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.072+0000 7f2734163740 20 client.25928 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.072+0000 7f2734163740 1 --2- 172.21.15.38:0/1782892314 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b4684047d0 0x55b468406bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.072+0000 7f2734163740 4 client.25928 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:37:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.072+0000 7f2734163740 1 -- 172.21.15.38:0/1782892314 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b4682f0dc0 con 0x55b4684047d0 2024-05-21T15:37:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.073+0000 7f2726ffd640 1 --2- 172.21.15.38:0/1782892314 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b4684047d0 0x55b468406bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.079+0000 7f2726ffd640 1 --2- 172.21.15.38:0/1782892314 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b4684047d0 0x55b468406bf0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=431fa8d01a8a8793 server_cookie=a16ea9d2cae68120 in_seq=0 out_seq=0 2024-05-21T15:37:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:15.079+0000 7f270a7f4640 10 client.25928 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:15.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:14 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:15.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:14 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:37:15.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:14 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:15.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:14 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:37:15.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:14 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:15.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:14 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:37:16.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:15 smithi053 ceph-mon[36496]: pgmap v436: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 269 MiB/s wr, 26.03k op/s 2024-05-21T15:37:16.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:15 smithi045 ceph-mon[36409]: pgmap v436: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 269 MiB/s wr, 26.03k op/s 2024-05-21T15:37:16.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:15 smithi038 ceph-mon[31995]: pgmap v436: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 269 MiB/s wr, 26.03k op/s 2024-05-21T15:37:17.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:16 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:17.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:16 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:17.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:16 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:17.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:16 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:17.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:16 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:17.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:16 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:17.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:16 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:17.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:16 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:17.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:16 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:18.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:17 smithi053 ceph-mon[36496]: pgmap v437: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 235 MiB/s wr, 22.63k op/s 2024-05-21T15:37:18.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:17 smithi045 ceph-mon[36409]: pgmap v437: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 235 MiB/s wr, 22.63k op/s 2024-05-21T15:37:18.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:17 smithi038 ceph-mon[31995]: pgmap v437: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 235 MiB/s wr, 22.63k op/s 2024-05-21T15:37:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.212+0000 7f270a7f4640 1 -- 172.21.15.38:0/1782892314 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b4682f0dc0 con 0x55b4684047d0 2024-05-21T15:37:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.212+0000 7f270a7f4640 10 client.25928 handle_command_reply: tid=0 2024-05-21T15:37:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.212+0000 7f2734163740 1 client.25928 shutdown 2024-05-21T15:37:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.212+0000 7f2734163740 1 -- 172.21.15.38:0/1782892314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fbc2f0 msgr2=0x55b468392a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.212+0000 7f2734163740 1 --2- 172.21.15.38:0/1782892314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fbc2f0 0x55b468392a70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2718006010 tx=0x7f271802a420 comp rx=0 tx=0).stop 2024-05-21T15:37:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.213+0000 7f2734163740 1 -- 172.21.15.38:0/1782892314 shutdown_connections 2024-05-21T15:37:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.213+0000 7f2734163740 1 --2- 172.21.15.38:0/1782892314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b467fe51b0 0x55b468402870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.213+0000 7f2734163740 1 --2- 172.21.15.38:0/1782892314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b467fbc2f0 0x55b468392a70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.213+0000 7f2734163740 1 --2- 172.21.15.38:0/1782892314 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b4684047d0 0x55b468406bf0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.213+0000 7f2734163740 1 --2- 172.21.15.38:0/1782892314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b467fa3d80 0x55b468392530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.213+0000 7f2734163740 1 -- 172.21.15.38:0/1782892314 >> 172.21.15.38:0/1782892314 conn(0x55b467fd9300 msgr2=0x55b4683839a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.213+0000 7f270a7f4640 0 client.25928 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.213+0000 7f2734163740 1 -- 172.21.15.38:0/1782892314 shutdown_connections 2024-05-21T15:37:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.213+0000 7f2734163740 1 -- 172.21.15.38:0/1782892314 wait complete. 2024-05-21T15:37:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.213+0000 7f2734163740 20 client.25928 trim_cache size 0 max 16384 2024-05-21T15:37:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:37:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:37:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:37:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:37:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.214+0000 7f270dffb640 1 -- 172.21.15.38:0/1366618637 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27100a2d80 msgr2=0x7f27100a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.214+0000 7f270dffb640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27100a2d80 0x7f27100a5290 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f272000b010 tx=0x7f27200377d0 comp rx=0 tx=0).stop 2024-05-21T15:37:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.214+0000 7f270dffb640 1 -- 172.21.15.38:0/1366618637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f272810a9a0 msgr2=0x7f272811ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.214+0000 7f270dffb640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f272810a9a0 0x7f272811ac40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f271c009fd0 tx=0x7f271c00b8e0 comp rx=0 tx=0).stop 2024-05-21T15:37:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.214+0000 7f270dffb640 1 -- 172.21.15.38:0/1366618637 shutdown_connections 2024-05-21T15:37:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.214+0000 7f270dffb640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f272810ff00 0x7f272811e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.214+0000 7f270dffb640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f272810d390 0x7f272811b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.214+0000 7f270dffb640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f27100a2d80 0x7f27100a5290 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.214+0000 7f270dffb640 1 --2- 172.21.15.38:0/1366618637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f272810a9a0 0x7f272811ac40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.214+0000 7f270dffb640 1 -- 172.21.15.38:0/1366618637 >> 172.21.15.38:0/1366618637 conn(0x7f272806f4c0 msgr2=0x7f272810b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.215+0000 7f270dffb640 1 -- 172.21.15.38:0/1366618637 shutdown_connections 2024-05-21T15:37:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:19.215+0000 7f270dffb640 1 -- 172.21.15.38:0/1366618637 wait complete. 2024-05-21T15:37:20.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:19 smithi053 ceph-mon[36496]: pgmap v438: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 248 MiB/s wr, 24.54k op/s 2024-05-21T15:37:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:19 smithi045 ceph-mon[36409]: pgmap v438: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 248 MiB/s wr, 24.54k op/s 2024-05-21T15:37:20.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:19 smithi038 ceph-mon[31995]: pgmap v438: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 248 MiB/s wr, 24.54k op/s 2024-05-21T15:37:22.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:21 smithi053 ceph-mon[36496]: pgmap v439: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 246 MiB/s wr, 24.97k op/s 2024-05-21T15:37:22.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:21 smithi045 ceph-mon[36409]: pgmap v439: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 246 MiB/s wr, 24.97k op/s 2024-05-21T15:37:22.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:21 smithi038 ceph-mon[31995]: pgmap v439: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 246 MiB/s wr, 24.97k op/s 2024-05-21T15:37:24.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:23 smithi053 ceph-mon[36496]: pgmap v440: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 213 MiB/s wr, 22.27k op/s 2024-05-21T15:37:24.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:23 smithi045 ceph-mon[36409]: pgmap v440: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 213 MiB/s wr, 22.27k op/s 2024-05-21T15:37:24.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:23 smithi038 ceph-mon[31995]: pgmap v440: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 213 MiB/s wr, 22.27k op/s 2024-05-21T15:37:25.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:24 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:37:25.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:24 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:37:25.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:24 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:37:25.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:24 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:37:25.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:24 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:37:25.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:24 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:37:25.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:24 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:37:25.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:24 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:37:25.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:24 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:37:25.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:24 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:37:25.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:24 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:37:25.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:24 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:37:25.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:24 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:37:25.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:24 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:37:25.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:24 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:37:26.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:25 smithi053 ceph-mon[36496]: pgmap v441: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 205 MiB/s wr, 21.47k op/s 2024-05-21T15:37:26.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:25 smithi045 ceph-mon[36409]: pgmap v441: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 205 MiB/s wr, 21.47k op/s 2024-05-21T15:37:26.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:25 smithi038 ceph-mon[31995]: pgmap v441: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 205 MiB/s wr, 21.47k op/s 2024-05-21T15:37:27.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:37:27.731 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:37:27.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.892+0000 7f304f2ca640 1 -- 172.21.15.38:0/1664194903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3048107900 msgr2=0x7f3048109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:27.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.892+0000 7f304f2ca640 1 --2- 172.21.15.38:0/1664194903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3048107900 0x7f3048109d20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3040009ca0 tx=0x7f304002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.892+0000 7f304f2ca640 1 -- 172.21.15.38:0/1664194903 shutdown_connections 2024-05-21T15:37:27.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.892+0000 7f304f2ca640 1 --2- 172.21.15.38:0/1664194903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3048107900 0x7f3048109d20 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.892+0000 7f304f2ca640 1 --2- 172.21.15.38:0/1664194903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3048102c00 0x7f3048103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.892+0000 7f304f2ca640 1 --2- 172.21.15.38:0/1664194903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30481019c0 0x7f30481026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.892+0000 7f304f2ca640 1 -- 172.21.15.38:0/1664194903 >> 172.21.15.38:0/1664194903 conn(0x7f30480fcbb0 msgr2=0x7f30480ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:27.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.892+0000 7f304f2ca640 1 -- 172.21.15.38:0/1664194903 shutdown_connections 2024-05-21T15:37:27.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.892+0000 7f304f2ca640 1 -- 172.21.15.38:0/1664194903 wait complete. 2024-05-21T15:37:27.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.893+0000 7f304f2ca640 1 Processor -- start 2024-05-21T15:37:27.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.893+0000 7f304f2ca640 1 -- start start 2024-05-21T15:37:27.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.893+0000 7f304f2ca640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30481019c0 0x7f30481991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.893+0000 7f304f2ca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3048102c00 0x7f3048199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.893+0000 7f304f2ca640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3048107900 0x7f30481a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.893+0000 7f304f2ca640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f304810d640 con 0x7f3048107900 2024-05-21T15:37:27.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.893+0000 7f304f2ca640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f304810d4c0 con 0x7f30481019c0 2024-05-21T15:37:27.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.893+0000 7f304f2ca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f304810d7c0 con 0x7f3048102c00 2024-05-21T15:37:27.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f304d867640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3048107900 0x7f30481a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f304d867640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3048107900 0x7f30481a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40680/0 (socket says 172.21.15.38:40680) 2024-05-21T15:37:27.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f304d867640 1 -- 172.21.15.38:0/3408936630 learned_addr learned my addr 172.21.15.38:0/3408936630 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:27.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f304c865640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3048102c00 0x7f3048199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f304d066640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30481019c0 0x7f30481991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f304d867640 1 -- 172.21.15.38:0/3408936630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3048102c00 msgr2=0x7f3048199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:27.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f304d867640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3048102c00 0x7f3048199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f304d867640 1 -- 172.21.15.38:0/3408936630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30481019c0 msgr2=0x7f30481991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:27.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f304d867640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30481019c0 0x7f30481991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f304d867640 1 -- 172.21.15.38:0/3408936630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304810e110 con 0x7f3048107900 2024-05-21T15:37:27.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f304d066640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30481019c0 0x7f30481991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:37:27.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f304d867640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3048107900 0x7f30481a06c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3040005e30 tx=0x7f3040031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:27.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f303e7fc640 1 -- 172.21.15.38:0/3408936630 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040003b70 con 0x7f3048107900 2024-05-21T15:37:27.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f303e7fc640 1 -- 172.21.15.38:0/3408936630 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3040003d10 con 0x7f3048107900 2024-05-21T15:37:27.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f304f2ca640 1 -- 172.21.15.38:0/3408936630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304810e3a0 con 0x7f3048107900 2024-05-21T15:37:27.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f303e7fc640 1 -- 172.21.15.38:0/3408936630 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304002fb70 con 0x7f3048107900 2024-05-21T15:37:27.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.894+0000 7f304f2ca640 1 -- 172.21.15.38:0/3408936630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304810e8f0 con 0x7f3048107900 2024-05-21T15:37:27.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.898+0000 7f303e7fc640 1 -- 172.21.15.38:0/3408936630 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f304002fd10 con 0x7f3048107900 2024-05-21T15:37:27.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.899+0000 7f3056173740 1 Processor -- start 2024-05-21T15:37:27.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.900+0000 7f3056173740 1 -- start start 2024-05-21T15:37:27.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.900+0000 7f303e7fc640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f30340a2b90 0x7f30340a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.900+0000 7f3056173740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55ea9390 0x561f562513c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.900+0000 7f3056173740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f56251940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.900+0000 7f3056173740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 0x561f56258840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.900+0000 7f303e7fc640 1 -- 172.21.15.38:0/3408936630 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f30400f0450 con 0x7f3048107900 2024-05-21T15:37:27.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.900+0000 7f304c865640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f30340a2b90 0x7f30340a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.900+0000 7f3056173740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561f561add00 con 0x561f55ea9390 2024-05-21T15:37:27.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.900+0000 7f3056173740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561f561554c0 con 0x561f55e886b0 2024-05-21T15:37:27.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.900+0000 7f3056173740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561f561a9cc0 con 0x561f55e804d0 2024-05-21T15:37:27.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.900+0000 7f304c865640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f56251940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.900+0000 7f304c865640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f56251940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46396/0 (socket says 172.21.15.38:46396) 2024-05-21T15:37:27.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.900+0000 7f304c865640 1 -- 172.21.15.38:0/3142024124 learned_addr learned my addr 172.21.15.38:0/3142024124 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.901+0000 7f304d867640 1 --2- 172.21.15.38:0/3142024124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 0x561f56258840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.901+0000 7f304d066640 1 --2- 172.21.15.38:0/3142024124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55ea9390 0x561f562513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.901+0000 7f304c865640 1 -- 172.21.15.38:0/3142024124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 msgr2=0x561f56258840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.901+0000 7f304c865640 1 --2- 172.21.15.38:0/3142024124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 0x561f56258840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.901+0000 7f304c865640 1 -- 172.21.15.38:0/3142024124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55ea9390 msgr2=0x561f562513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.901+0000 7f304c865640 1 --2- 172.21.15.38:0/3142024124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55ea9390 0x561f562513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.901+0000 7f304c865640 1 -- 172.21.15.38:0/3142024124 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f55f8b7d0 con 0x561f55e804d0 2024-05-21T15:37:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.901+0000 7f304d066640 1 --2- 172.21.15.38:0/3142024124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55ea9390 0x561f562513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:37:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.901+0000 7f304c865640 1 --2- 172.21.15.38:0/3142024124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f56251940 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3038013a20 tx=0x7f3038013ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd7f67129895a620 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.901+0000 7f304c865640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f30340a2b90 0x7f30340a50a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f303800f850 tx=0x7f3038011000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.901+0000 7f30337fe640 1 -- 172.21.15.38:0/3142024124 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3038013240 con 0x561f55e804d0 2024-05-21T15:37:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.901+0000 7f30337fe640 1 -- 172.21.15.38:0/3142024124 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3038010d10 con 0x561f55e804d0 2024-05-21T15:37:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.901+0000 7f30337fe640 1 -- 172.21.15.38:0/3142024124 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303801f8e0 con 0x561f55e804d0 2024-05-21T15:37:27.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.902+0000 7f3056173740 1 -- 172.21.15.38:0/3142024124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 msgr2=0x561f56251940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:27.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.902+0000 7f3056173740 1 --2- 172.21.15.38:0/3142024124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f56251940 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3038013a20 tx=0x7f3038013ee0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.902+0000 7f3056173740 1 -- 172.21.15.38:0/3142024124 shutdown_connections 2024-05-21T15:37:27.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.902+0000 7f3056173740 1 --2- 172.21.15.38:0/3142024124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 0x561f56258840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.902+0000 7f3056173740 1 --2- 172.21.15.38:0/3142024124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f56251940 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.902+0000 7f3056173740 1 --2- 172.21.15.38:0/3142024124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55ea9390 0x561f562513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.902+0000 7f3056173740 1 -- 172.21.15.38:0/3142024124 >> 172.21.15.38:0/3142024124 conn(0x561f55e9d4e0 msgr2=0x561f56244540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:27.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.903+0000 7f3056173740 1 -- 172.21.15.38:0/3142024124 shutdown_connections 2024-05-21T15:37:27.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.903+0000 7f3056173740 1 -- 172.21.15.38:0/3142024124 wait complete. 2024-05-21T15:37:27.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.903+0000 7f3056173740 1 -- start start 2024-05-21T15:37:27.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.904+0000 7f3056173740 1 Processor -- start 2024-05-21T15:37:27.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.904+0000 7f3056173740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f5624ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.904+0000 7f3056173740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 0x561f5624b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.904+0000 7f3056173740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55ea9390 0x561f56250270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.904+0000 7f3056173740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561f56248b90 con 0x561f55ea9390 2024-05-21T15:37:27.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.904+0000 7f3056173740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561f56168b90 con 0x561f55e886b0 2024-05-21T15:37:27.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.904+0000 7f3056173740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561f56248d10 con 0x561f55e804d0 2024-05-21T15:37:27.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.904+0000 7f3056173740 10 client.0 _ll_register_callbacks cb 0x561f561a4da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:27.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.904+0000 7f304d867640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55ea9390 0x561f56250270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.904+0000 7f304d867640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55ea9390 0x561f56250270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40704/0 (socket says 172.21.15.38:40704) 2024-05-21T15:37:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.904+0000 7f304d867640 1 -- 172.21.15.38:0/2776204739 learned_addr learned my addr 172.21.15.38:0/2776204739 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.904+0000 7f304c865640 1 --2- 172.21.15.38:0/2776204739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 0x561f5624b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.904+0000 7f304d066640 1 --2- 172.21.15.38:0/2776204739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f5624ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.905+0000 7f304c865640 1 -- 172.21.15.38:0/2776204739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 msgr2=0x561f5624ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.905+0000 7f304c865640 1 --2- 172.21.15.38:0/2776204739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f5624ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.905+0000 7f304c865640 1 -- 172.21.15.38:0/2776204739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55ea9390 msgr2=0x561f56250270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.905+0000 7f304c865640 1 --2- 172.21.15.38:0/2776204739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55ea9390 0x561f56250270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.905+0000 7f304c865640 1 -- 172.21.15.38:0/2776204739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f55ec13f0 con 0x561f55e886b0 2024-05-21T15:37:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.905+0000 7f304d066640 1 --2- 172.21.15.38:0/2776204739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f5624ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:37:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.905+0000 7f304c865640 1 --2- 172.21.15.38:0/2776204739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 0x561f5624b320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3038005d70 tx=0x7f3038013240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72de6d1d06231f74 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.905+0000 7f3030ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:37:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.906+0000 7f3030ff9640 1 -- 172.21.15.38:0/2776204739 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3038010830 con 0x561f55e886b0 2024-05-21T15:37:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.906+0000 7f3030ff9640 1 -- 172.21.15.38:0/2776204739 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f30380109d0 con 0x561f55e886b0 2024-05-21T15:37:27.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.906+0000 7f3056173740 1 -- 172.21.15.38:0/2776204739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3038010d10 con 0x561f55e886b0 2024-05-21T15:37:27.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.906+0000 7f3030ff9640 1 -- 172.21.15.38:0/2776204739 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3038005040 con 0x561f55e886b0 2024-05-21T15:37:27.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.906+0000 7f3056173740 10 client.25940 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:27.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.906+0000 7f3056173740 1 -- 172.21.15.38:0/2776204739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561f55e810f0 con 0x561f55e886b0 2024-05-21T15:37:27.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.906+0000 7f3030ff9640 1 -- 172.21.15.38:0/2776204739 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30380056c0 con 0x561f55e886b0 2024-05-21T15:37:27.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.907+0000 7f3030ff9640 1 -- 172.21.15.38:0/2776204739 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f303801fcc0 con 0x561f55e886b0 2024-05-21T15:37:27.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.907+0000 7f3056173740 10 client.25940 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:27.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.907+0000 7f3056173740 10 client.25940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:27.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.907+0000 7f3056173740 20 client.25940 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:27.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.907+0000 7f3056173740 1 --2- 172.21.15.38:0/2776204739 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561f55e67f60 0x561f56260630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.907+0000 7f3056173740 4 client.25940 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:37:27.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.907+0000 7f3056173740 1 -- 172.21.15.38:0/2776204739 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3038010d10 con 0x561f55e67f60 2024-05-21T15:37:27.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.907+0000 7f304d066640 1 --2- 172.21.15.38:0/2776204739 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561f55e67f60 0x561f56260630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.908+0000 7f3030ff9640 10 client.25940 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.908+0000 7f304d066640 1 --2- 172.21.15.38:0/2776204739 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561f55e67f60 0x561f56260630 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a291ff27b614b7 server_cookie=33a3a693ab8e250 in_seq=0 out_seq=0 2024-05-21T15:37:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.908+0000 7f3030ff9640 1 -- 172.21.15.38:0/2776204739 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f3038010d10 con 0x561f55e67f60 2024-05-21T15:37:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.908+0000 7f3030ff9640 10 client.25940 handle_command_reply: tid=0 2024-05-21T15:37:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.909+0000 7f3056173740 1 client.25940 shutdown 2024-05-21T15:37:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.909+0000 7f3056173740 1 -- 172.21.15.38:0/2776204739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 msgr2=0x561f5624b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.909+0000 7f3056173740 1 --2- 172.21.15.38:0/2776204739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 0x561f5624b320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3038005d70 tx=0x7f3038013240 comp rx=0 tx=0).stop 2024-05-21T15:37:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.909+0000 7f3056173740 1 -- 172.21.15.38:0/2776204739 shutdown_connections 2024-05-21T15:37:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.909+0000 7f3056173740 1 --2- 172.21.15.38:0/2776204739 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561f55e67f60 0x561f56260630 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.909+0000 7f3056173740 1 --2- 172.21.15.38:0/2776204739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55ea9390 0x561f56250270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.909+0000 7f3056173740 1 --2- 172.21.15.38:0/2776204739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 0x561f5624b320 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.909+0000 7f3056173740 1 --2- 172.21.15.38:0/2776204739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f5624ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.909+0000 7f3030ff9640 0 client.25940 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:27.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.909+0000 7f3056173740 1 -- 172.21.15.38:0/2776204739 >> 172.21.15.38:0/2776204739 conn(0x561f55e9d4e0 msgr2=0x561f562435c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.909+0000 7f3056173740 1 -- 172.21.15.38:0/2776204739 shutdown_connections 2024-05-21T15:37:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.909+0000 7f3056173740 1 -- 172.21.15.38:0/2776204739 wait complete. 2024-05-21T15:37:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.909+0000 7f3056173740 20 client.25940 trim_cache size 0 max 16384 2024-05-21T15:37:27.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.935+0000 7f3056173740 1 Processor -- start 2024-05-21T15:37:27.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.935+0000 7f3056173740 1 -- start start 2024-05-21T15:37:27.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f3056173740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f5624a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f3056173740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55e886b0 0x561f56255f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f3056173740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55ea9390 0x561f56264e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f3056173740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3048072b10 con 0x561f55e886b0 2024-05-21T15:37:27.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f3056173740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f30481a15c0 con 0x561f55ea9390 2024-05-21T15:37:27.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f3056173740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f30480731e0 con 0x561f55e804d0 2024-05-21T15:37:27.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f304c865640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55e886b0 0x561f56255f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f304c865640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55e886b0 0x561f56255f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40712/0 (socket says 172.21.15.38:40712) 2024-05-21T15:37:27.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f304c865640 1 -- 172.21.15.38:0/4203470094 learned_addr learned my addr 172.21.15.38:0/4203470094 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:27.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f304d867640 1 --2- 172.21.15.38:0/4203470094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55ea9390 0x561f56264e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f304d066640 1 --2- 172.21.15.38:0/4203470094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f5624a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f304c865640 1 -- 172.21.15.38:0/4203470094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 msgr2=0x561f5624a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:27.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f304c865640 1 --2- 172.21.15.38:0/4203470094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f5624a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f304c865640 1 -- 172.21.15.38:0/4203470094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55ea9390 msgr2=0x561f56264e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:27.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f304c865640 1 --2- 172.21.15.38:0/4203470094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55ea9390 0x561f56264e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.936+0000 7f304c865640 1 -- 172.21.15.38:0/4203470094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f56250e60 con 0x561f55e886b0 2024-05-21T15:37:27.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.937+0000 7f304c865640 1 --2- 172.21.15.38:0/4203470094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55e886b0 0x561f56255f50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3038004760 tx=0x7f3038013ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4095e23c5e196230 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:27.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.937+0000 7f3032ffd640 1 -- 172.21.15.38:0/4203470094 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30380047e0 con 0x561f55e886b0 2024-05-21T15:37:27.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.937+0000 7f3032ffd640 1 -- 172.21.15.38:0/4203470094 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3038004980 con 0x561f55e886b0 2024-05-21T15:37:27.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.937+0000 7f3032ffd640 1 -- 172.21.15.38:0/4203470094 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3038010a60 con 0x561f55e886b0 2024-05-21T15:37:27.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.939+0000 7f3056173740 1 -- 172.21.15.38:0/4203470094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55e886b0 msgr2=0x561f56255f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:27.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.939+0000 7f3056173740 1 --2- 172.21.15.38:0/4203470094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55e886b0 0x561f56255f50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3038004760 tx=0x7f3038013ee0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.939+0000 7f3056173740 1 -- 172.21.15.38:0/4203470094 shutdown_connections 2024-05-21T15:37:27.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.939+0000 7f3056173740 1 --2- 172.21.15.38:0/4203470094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55ea9390 0x561f56264e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.939+0000 7f3056173740 1 --2- 172.21.15.38:0/4203470094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55e886b0 0x561f56255f50 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.939+0000 7f3056173740 1 --2- 172.21.15.38:0/4203470094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55e804d0 0x561f5624a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.939+0000 7f3056173740 1 -- 172.21.15.38:0/4203470094 >> 172.21.15.38:0/4203470094 conn(0x561f55e9d4e0 msgr2=0x561f56247d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:27.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.939+0000 7f3056173740 1 -- 172.21.15.38:0/4203470094 shutdown_connections 2024-05-21T15:37:27.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.939+0000 7f3056173740 1 -- 172.21.15.38:0/4203470094 wait complete. 2024-05-21T15:37:27.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.939+0000 7f3056173740 1 -- start start 2024-05-21T15:37:27.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.940+0000 7f3056173740 1 Processor -- start 2024-05-21T15:37:27.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.940+0000 7f3056173740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55e804d0 0x561f562711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.940+0000 7f3056173740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 0x561f56271710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.940+0000 7f304d066640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55e804d0 0x561f562711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.940+0000 7f304d066640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55e804d0 0x561f562711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40724/0 (socket says 172.21.15.38:40724) 2024-05-21T15:37:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.940+0000 7f304d066640 1 -- 172.21.15.38:0/960441966 learned_addr learned my addr 172.21.15.38:0/960441966 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.940+0000 7f3056173740 1 --2- 172.21.15.38:0/960441966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55ea9390 0x561f562c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.940+0000 7f304c865640 1 --2- 172.21.15.38:0/960441966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 0x561f56271710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.940+0000 7f3056173740 1 -- 172.21.15.38:0/960441966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30481a0e30 con 0x561f55e804d0 2024-05-21T15:37:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.940+0000 7f3056173740 1 -- 172.21.15.38:0/960441966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f30480738b0 con 0x561f55e886b0 2024-05-21T15:37:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.941+0000 7f3056173740 1 -- 172.21.15.38:0/960441966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3048070af0 con 0x561f55ea9390 2024-05-21T15:37:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.941+0000 7f304d867640 1 --2- 172.21.15.38:0/960441966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55ea9390 0x561f562c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.941+0000 7f3056173740 10 client.0 _ll_register_callbacks cb 0x7f3040004900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.941+0000 7f304c865640 1 -- 172.21.15.38:0/960441966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55ea9390 msgr2=0x561f562c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.941+0000 7f304c865640 1 --2- 172.21.15.38:0/960441966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55ea9390 0x561f562c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.941+0000 7f304c865640 1 -- 172.21.15.38:0/960441966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55e804d0 msgr2=0x561f562711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.941+0000 7f304c865640 1 --2- 172.21.15.38:0/960441966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55e804d0 0x561f562711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.941+0000 7f304c865640 1 -- 172.21.15.38:0/960441966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f562c6ea0 con 0x561f55e886b0 2024-05-21T15:37:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.941+0000 7f3056173740 1 -- 172.21.15.38:0/960441966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3038004980 con 0x561f55e886b0 2024-05-21T15:37:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.941+0000 7f304c865640 1 --2- 172.21.15.38:0/960441966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 0x561f56271710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f303803d040 tx=0x7f3038010dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b59cb549b030c58 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.941+0000 7f3030ff9640 10 client.25946 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:37:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.942+0000 7f3030ff9640 1 -- 172.21.15.38:0/960441966 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3038013870 con 0x561f55e886b0 2024-05-21T15:37:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.942+0000 7f3030ff9640 1 -- 172.21.15.38:0/960441966 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f3038009040 con 0x561f55e886b0 2024-05-21T15:37:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.942+0000 7f3030ff9640 1 -- 172.21.15.38:0/960441966 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3038010a10 con 0x561f55e886b0 2024-05-21T15:37:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.942+0000 7f3056173740 10 client.25946 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.942+0000 7f3056173740 1 -- 172.21.15.38:0/960441966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=31}) -- 0x561f562c7fb0 con 0x561f55e886b0 2024-05-21T15:37:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.942+0000 7f3030ff9640 1 -- 172.21.15.38:0/960441966 <== mon.2 v2:172.21.15.45:3300/0 4 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f3038013240 con 0x561f55e886b0 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.942+0000 7f3056173740 10 client.25946 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.942+0000 7f3056173740 10 client.25946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.942+0000 7f3056173740 20 client.25946 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.942+0000 7f3056173740 1 --2- 172.21.15.38:0/960441966 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561f55e3d3a0 0x561f562ca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.942+0000 7f3056173740 4 client.25946 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.942+0000 7f3056173740 1 -- 172.21.15.38:0/960441966 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f561bb430 con 0x561f55e3d3a0 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.942+0000 7f304d066640 1 --2- 172.21.15.38:0/960441966 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561f55e3d3a0 0x561f562ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3030ff9640 10 client.25946 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f304d066640 1 --2- 172.21.15.38:0/960441966 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561f55e3d3a0 0x561f562ca920 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13b1c9c42ba3f482 server_cookie=ef7532d0b0e52987 in_seq=0 out_seq=0 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3030ff9640 1 -- 172.21.15.38:0/960441966 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561f561bb430 con 0x561f55e3d3a0 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3030ff9640 10 client.25946 handle_command_reply: tid=0 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3056173740 1 client.25946 shutdown 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3056173740 1 -- 172.21.15.38:0/960441966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 msgr2=0x561f56271710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3056173740 1 --2- 172.21.15.38:0/960441966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 0x561f56271710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f303803d040 tx=0x7f3038010dc0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3056173740 1 -- 172.21.15.38:0/960441966 shutdown_connections 2024-05-21T15:37:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3056173740 1 --2- 172.21.15.38:0/960441966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f55ea9390 0x561f562c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3056173740 1 --2- 172.21.15.38:0/960441966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f55e886b0 0x561f56271710 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3056173740 1 --2- 172.21.15.38:0/960441966 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561f55e3d3a0 0x561f562ca920 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3056173740 1 --2- 172.21.15.38:0/960441966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f55e804d0 0x561f562711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3030ff9640 0 client.25946 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3056173740 1 -- 172.21.15.38:0/960441966 >> 172.21.15.38:0/960441966 conn(0x561f55e9d4e0 msgr2=0x561f56247ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3056173740 1 -- 172.21.15.38:0/960441966 shutdown_connections 2024-05-21T15:37:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3056173740 1 -- 172.21.15.38:0/960441966 wait complete. 2024-05-21T15:37:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.943+0000 7f3056173740 20 client.25946 trim_cache size 0 max 16384 2024-05-21T15:37:27.948 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:37:27.948 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-05-21T15:37:27.949 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-05-21T15:37:27.949 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:37:27.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.944+0000 7f304f2ca640 1 -- 172.21.15.38:0/3408936630 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f30340a2b90 msgr2=0x7f30340a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:27.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.944+0000 7f304f2ca640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f30340a2b90 0x7f30340a50a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f303800f850 tx=0x7f3038011000 comp rx=0 tx=0).stop 2024-05-21T15:37:27.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.944+0000 7f304f2ca640 1 -- 172.21.15.38:0/3408936630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3048107900 msgr2=0x7f30481a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:27.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.944+0000 7f304f2ca640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3048107900 0x7f30481a06c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3040005e30 tx=0x7f3040031ee0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.944+0000 7f304f2ca640 1 -- 172.21.15.38:0/3408936630 shutdown_connections 2024-05-21T15:37:27.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.944+0000 7f304f2ca640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f30340a2b90 0x7f30340a50a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.944+0000 7f304f2ca640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3048107900 0x7f30481a06c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.944+0000 7f304f2ca640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3048102c00 0x7f3048199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.944+0000 7f304f2ca640 1 --2- 172.21.15.38:0/3408936630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30481019c0 0x7f30481991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:27.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.944+0000 7f304f2ca640 1 -- 172.21.15.38:0/3408936630 >> 172.21.15.38:0/3408936630 conn(0x7f30480fcbb0 msgr2=0x7f30480fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:27.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.944+0000 7f304f2ca640 1 -- 172.21.15.38:0/3408936630 shutdown_connections 2024-05-21T15:37:27.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:27.944+0000 7f304f2ca640 1 -- 172.21.15.38:0/3408936630 wait complete. 2024-05-21T15:37:27.959 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:37:27.959 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:37:28.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:27 smithi038 ceph-mon[31995]: pgmap v442: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 200 MiB/s wr, 21.21k op/s 2024-05-21T15:37:28.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.126+0000 7fc26b7fe640 1 -- 172.21.15.38:0/2387854205 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2600319b0 con 0x7fc26c1093d0 2024-05-21T15:37:28.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.127+0000 7fc272e56640 1 -- 172.21.15.38:0/2387854205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c1093d0 msgr2=0x7fc26c109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:28.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.127+0000 7fc272e56640 1 --2- 172.21.15.38:0/2387854205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c1093d0 0x7fc26c109dc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc260009900 tx=0x7fc26002a440 comp rx=0 tx=0).stop 2024-05-21T15:37:28.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.129+0000 7fc272e56640 1 -- 172.21.15.38:0/2387854205 shutdown_connections 2024-05-21T15:37:28.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.129+0000 7fc272e56640 1 --2- 172.21.15.38:0/2387854205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10f020 0x7fc26c10f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.129+0000 7fc272e56640 1 --2- 172.21.15.38:0/2387854205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc26c10a300 0x7fc26c10a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.129+0000 7fc272e56640 1 --2- 172.21.15.38:0/2387854205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c1093d0 0x7fc26c109dc0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.129+0000 7fc272e56640 1 -- 172.21.15.38:0/2387854205 >> 172.21.15.38:0/2387854205 conn(0x7fc26c06f4e0 msgr2=0x7fc26c071970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:28.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.132+0000 7fc272e56640 1 -- 172.21.15.38:0/2387854205 shutdown_connections 2024-05-21T15:37:28.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.133+0000 7fc272e56640 1 -- 172.21.15.38:0/2387854205 wait complete. 2024-05-21T15:37:28.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.134+0000 7fc272e56640 1 Processor -- start 2024-05-21T15:37:28.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.135+0000 7fc272e56640 1 -- start start 2024-05-21T15:37:28.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.135+0000 7fc272e56640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc26c1093d0 0x7fc26c115f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.135+0000 7fc272e56640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10a300 0x7fc26c1164a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.135+0000 7fc272e56640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c10f020 0x7fc26c117400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.135+0000 7fc272e56640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c11d720 con 0x7fc26c10f020 2024-05-21T15:37:28.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.136+0000 7fc272e56640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c11d5a0 con 0x7fc26c10a300 2024-05-21T15:37:28.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.136+0000 7fc272e56640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c11d8a0 con 0x7fc26c1093d0 2024-05-21T15:37:28.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.136+0000 7fc26bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10a300 0x7fc26c1164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:28.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.136+0000 7fc270bf2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc26c1093d0 0x7fc26c115f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:28.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.136+0000 7fc270bf2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc26c1093d0 0x7fc26c115f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46446/0 (socket says 172.21.15.38:46446) 2024-05-21T15:37:28.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.136+0000 7fc26bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10a300 0x7fc26c1164a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45132/0 (socket says 172.21.15.38:45132) 2024-05-21T15:37:28.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.136+0000 7fc26bfff640 1 -- 172.21.15.38:0/2985341561 learned_addr learned my addr 172.21.15.38:0/2985341561 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:28.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.137+0000 7fc2713f3640 1 --2- 172.21.15.38:0/2985341561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c10f020 0x7fc26c117400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:28.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.137+0000 7fc270bf2640 1 -- 172.21.15.38:0/2985341561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10a300 msgr2=0x7fc26c1164a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:28.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.137+0000 7fc270bf2640 1 --2- 172.21.15.38:0/2985341561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10a300 0x7fc26c1164a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.137+0000 7fc270bf2640 1 -- 172.21.15.38:0/2985341561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c10f020 msgr2=0x7fc26c117400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:28.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.137+0000 7fc270bf2640 1 --2- 172.21.15.38:0/2985341561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c10f020 0x7fc26c117400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.137+0000 7fc270bf2640 1 -- 172.21.15.38:0/2985341561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c117a70 con 0x7fc26c1093d0 2024-05-21T15:37:28.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.137+0000 7fc270bf2640 1 --2- 172.21.15.38:0/2985341561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc26c1093d0 0x7fc26c115f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc26002a960 tx=0x7fc260032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:28.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.138+0000 7fc269ffb640 1 -- 172.21.15.38:0/2985341561 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26003f070 con 0x7fc26c1093d0 2024-05-21T15:37:28.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.138+0000 7fc269ffb640 1 -- 172.21.15.38:0/2985341561 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc26003b460 con 0x7fc26c1093d0 2024-05-21T15:37:28.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.138+0000 7fc269ffb640 1 -- 172.21.15.38:0/2985341561 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260035070 con 0x7fc26c1093d0 2024-05-21T15:37:28.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.138+0000 7fc272e56640 1 -- 172.21.15.38:0/2985341561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26c117d00 con 0x7fc26c1093d0 2024-05-21T15:37:28.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.138+0000 7fc272e56640 1 -- 172.21.15.38:0/2985341561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26c1181f0 con 0x7fc26c1093d0 2024-05-21T15:37:28.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.140+0000 7fc272e56640 1 -- 172.21.15.38:0/2985341561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2580052c0 con 0x7fc26c1093d0 2024-05-21T15:37:28.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.144+0000 7fc269ffb640 1 -- 172.21.15.38:0/2985341561 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc260003a00 con 0x7fc26c1093d0 2024-05-21T15:37:28.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.146+0000 7fc269ffb640 1 --2- 172.21.15.38:0/2985341561 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc2540a2cb0 0x7fc2540a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.146+0000 7fc269ffb640 1 -- 172.21.15.38:0/2985341561 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fc2600efcb0 con 0x7fc26c1093d0 2024-05-21T15:37:28.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.148+0000 7fc26bfff640 1 --2- 172.21.15.38:0/2985341561 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc2540a2cb0 0x7fc2540a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:28.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.149+0000 7fc26bfff640 1 --2- 172.21.15.38:0/2985341561 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc2540a2cb0 0x7fc2540a51c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc26c117000 tx=0x7fc26400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:28.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.152+0000 7fc269ffb640 1 -- 172.21.15.38:0/2985341561 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc260048030 con 0x7fc26c1093d0 2024-05-21T15:37:28.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:27 smithi053 ceph-mon[36496]: pgmap v442: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 200 MiB/s wr, 21.21k op/s 2024-05-21T15:37:28.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:27 smithi045 ceph-mon[36409]: pgmap v442: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 200 MiB/s wr, 21.21k op/s 2024-05-21T15:37:28.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.627+0000 7fc272e56640 1 -- 172.21.15.38:0/2985341561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2580055b0 con 0x7fc26c1093d0 2024-05-21T15:37:28.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.629+0000 7fc269ffb640 1 -- 172.21.15.38:0/2985341561 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11922 (secure 0 0 0) 0x7fc2600bcb60 con 0x7fc26c1093d0 2024-05-21T15:37:28.628 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:37:28.628 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:36:59.077696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24551,24523,24602,24550,24515]},"id":1}]} 2024-05-21T15:37:28.629 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 31 2024-05-21T15:37:28.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.633+0000 7fc272e56640 1 -- 172.21.15.38:0/2985341561 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc2540a2cb0 msgr2=0x7fc2540a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:28.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.633+0000 7fc272e56640 1 --2- 172.21.15.38:0/2985341561 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc2540a2cb0 0x7fc2540a51c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc26c117000 tx=0x7fc26400b040 comp rx=0 tx=0).stop 2024-05-21T15:37:28.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.633+0000 7fc272e56640 1 -- 172.21.15.38:0/2985341561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc26c1093d0 msgr2=0x7fc26c115f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:28.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.633+0000 7fc272e56640 1 --2- 172.21.15.38:0/2985341561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc26c1093d0 0x7fc26c115f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc26002a960 tx=0x7fc260032ce0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.633+0000 7fc272e56640 1 -- 172.21.15.38:0/2985341561 shutdown_connections 2024-05-21T15:37:28.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.633+0000 7fc272e56640 1 --2- 172.21.15.38:0/2985341561 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc2540a2cb0 0x7fc2540a51c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.633+0000 7fc272e56640 1 --2- 172.21.15.38:0/2985341561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc26c10f020 0x7fc26c117400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.633+0000 7fc272e56640 1 --2- 172.21.15.38:0/2985341561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10a300 0x7fc26c1164a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.633+0000 7fc272e56640 1 --2- 172.21.15.38:0/2985341561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc26c1093d0 0x7fc26c115f60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.633+0000 7fc272e56640 1 -- 172.21.15.38:0/2985341561 >> 172.21.15.38:0/2985341561 conn(0x7fc26c06f4e0 msgr2=0x7fc26c10b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:28.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.634+0000 7fc272e56640 1 -- 172.21.15.38:0/2985341561 shutdown_connections 2024-05-21T15:37:28.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.635+0000 7fc272e56640 1 -- 172.21.15.38:0/2985341561 wait complete. 2024-05-21T15:37:28.655 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:37:28.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.895+0000 7f66dcc56640 1 -- 172.21.15.38:0/3273831450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66d8073f40 msgr2=0x7f66d8074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:28.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.895+0000 7f66dcc56640 1 --2- 172.21.15.38:0/3273831450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66d8073f40 0x7f66d8074350 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009ca0 tx=0x7f66d002a8f0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.897+0000 7f66dcc56640 1 -- 172.21.15.38:0/3273831450 shutdown_connections 2024-05-21T15:37:28.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.897+0000 7f66dcc56640 1 --2- 172.21.15.38:0/3273831450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d810f1f0 0x7f66d8111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.897+0000 7f66dcc56640 1 --2- 172.21.15.38:0/3273831450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66d8074890 0x7f66d810ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.897+0000 7f66dcc56640 1 --2- 172.21.15.38:0/3273831450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66d8073f40 0x7f66d8074350 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.897+0000 7f66dcc56640 1 -- 172.21.15.38:0/3273831450 >> 172.21.15.38:0/3273831450 conn(0x7f66d80fdc80 msgr2=0x7f66d8100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:28.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.899+0000 7f66dcc56640 1 -- 172.21.15.38:0/3273831450 shutdown_connections 2024-05-21T15:37:28.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.900+0000 7f66dcc56640 1 -- 172.21.15.38:0/3273831450 wait complete. 2024-05-21T15:37:28.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.901+0000 7f66dcc56640 1 Processor -- start 2024-05-21T15:37:28.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.901+0000 7f66dcc56640 1 -- start start 2024-05-21T15:37:28.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.901+0000 7f66dcc56640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66d8073f40 0x7f66d810e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.902+0000 7f66dcc56640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d8074890 0x7f66d810d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.902+0000 7f66dcc56640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66d810f1f0 0x7f66d810d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.902+0000 7f66d5d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d8074890 0x7f66d810d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:28.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.902+0000 7f66d5d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d8074890 0x7f66d810d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46470/0 (socket says 172.21.15.38:46470) 2024-05-21T15:37:28.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.902+0000 7f66d5d9b640 1 -- 172.21.15.38:0/4244220881 learned_addr learned my addr 172.21.15.38:0/4244220881 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:28.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.902+0000 7f66d659c640 1 --2- 172.21.15.38:0/4244220881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66d8073f40 0x7f66d810e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:28.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.902+0000 7f66dcc56640 1 -- 172.21.15.38:0/4244220881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f66d8106e50 con 0x7f66d810f1f0 2024-05-21T15:37:28.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.903+0000 7f66dcc56640 1 -- 172.21.15.38:0/4244220881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f66d8106cd0 con 0x7f66d8073f40 2024-05-21T15:37:28.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.903+0000 7f66dcc56640 1 -- 172.21.15.38:0/4244220881 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f66d8106fd0 con 0x7f66d8074890 2024-05-21T15:37:28.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.904+0000 7f66d5d9b640 1 -- 172.21.15.38:0/4244220881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66d8073f40 msgr2=0x7f66d810e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:28.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.904+0000 7f66d5d9b640 1 --2- 172.21.15.38:0/4244220881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66d8073f40 0x7f66d810e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.904+0000 7f66d5d9b640 1 -- 172.21.15.38:0/4244220881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66d810f1f0 msgr2=0x7f66d810d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:28.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.904+0000 7f66d5d9b640 1 --2- 172.21.15.38:0/4244220881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66d810f1f0 0x7f66d810d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.904+0000 7f66d5d9b640 1 -- 172.21.15.38:0/4244220881 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d810dbd0 con 0x7f66d8074890 2024-05-21T15:37:28.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.905+0000 7f66d5d9b640 1 --2- 172.21.15.38:0/4244220881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d8074890 0x7f66d810d020 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00b450 tx=0x7f66cc00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:28.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.906+0000 7f66c77fe640 1 -- 172.21.15.38:0/4244220881 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc015020 con 0x7f66d8074890 2024-05-21T15:37:28.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.906+0000 7f66c77fe640 1 -- 172.21.15.38:0/4244220881 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66cc011d70 con 0x7f66d8074890 2024-05-21T15:37:28.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.906+0000 7f66c77fe640 1 -- 172.21.15.38:0/4244220881 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc015020 con 0x7f66d8074890 2024-05-21T15:37:28.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.906+0000 7f66dcc56640 1 -- 172.21.15.38:0/4244220881 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d80715a0 con 0x7f66d8074890 2024-05-21T15:37:28.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.906+0000 7f66dcc56640 1 -- 172.21.15.38:0/4244220881 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d8071b20 con 0x7f66d8074890 2024-05-21T15:37:28.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.912+0000 7f66e3bd4740 1 Processor -- start 2024-05-21T15:37:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.913+0000 7f66e3bd4740 1 -- start start 2024-05-21T15:37:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.913+0000 7f66e3bd4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1c2390 0x55b0ef561b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.913+0000 7f66e3bd4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1994d0 0x55b0ef562080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.913+0000 7f66e3bd4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1a16b0 0x55b0ef568f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.913+0000 7f66e3bd4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef4c6c40 con 0x55b0ef1994d0 2024-05-21T15:37:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.913+0000 7f66e3bd4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef46e580 con 0x55b0ef1c2390 2024-05-21T15:37:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.913+0000 7f66e3bd4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef4c2c00 con 0x55b0ef1a16b0 2024-05-21T15:37:28.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.914+0000 7f66d659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1994d0 0x55b0ef562080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:28.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.914+0000 7f66c77fe640 1 -- 172.21.15.38:0/4244220881 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f66cc018020 con 0x7f66d8074890 2024-05-21T15:37:28.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.914+0000 7f66d659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1994d0 0x55b0ef562080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40772/0 (socket says 172.21.15.38:40772) 2024-05-21T15:37:28.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.914+0000 7f66d659c640 1 -- 172.21.15.38:0/62097672 learned_addr learned my addr 172.21.15.38:0/62097672 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:28.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.915+0000 7f66d6d9d640 1 --2- 172.21.15.38:0/62097672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1c2390 0x55b0ef561b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:28.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.917+0000 7f66d6d9d640 1 -- 172.21.15.38:0/62097672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1a16b0 msgr2=0x55b0ef568f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:37:28.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.917+0000 7f66d6d9d640 1 --2- 172.21.15.38:0/62097672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1a16b0 0x55b0ef568f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.917+0000 7f66d6d9d640 1 -- 172.21.15.38:0/62097672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1994d0 msgr2=0x55b0ef562080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:28.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.918+0000 7f66d6d9d640 1 --2- 172.21.15.38:0/62097672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1994d0 0x55b0ef562080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.918+0000 7f66d6d9d640 1 -- 172.21.15.38:0/62097672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ef1da3f0 con 0x55b0ef1c2390 2024-05-21T15:37:28.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.919+0000 7f66d659c640 1 --2- 172.21.15.38:0/62097672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1994d0 0x55b0ef562080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:37:28.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.919+0000 7f66c77fe640 1 --2- 172.21.15.38:0/4244220881 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f66c00a2fd0 0x7f66c00a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.920+0000 7f66d6d9d640 1 --2- 172.21.15.38:0/62097672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1c2390 0x55b0ef561b00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009670 tx=0x7f66c8009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e515d83665fb343 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:28.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.920+0000 7f66c77fe640 1 -- 172.21.15.38:0/4244220881 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f66cc011690 con 0x7f66d8074890 2024-05-21T15:37:28.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.921+0000 7f66d659c640 1 --2- 172.21.15.38:0/4244220881 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f66c00a2fd0 0x7f66c00a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:28.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.922+0000 7f66c4ff9640 1 -- 172.21.15.38:0/62097672 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8015020 con 0x55b0ef1c2390 2024-05-21T15:37:28.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.922+0000 7f66d659c640 1 --2- 172.21.15.38:0/4244220881 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f66c00a2fd0 0x7f66c00a54e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55b0ef562ee0 tx=0x7f66d002adf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:28.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.922+0000 7f66c4ff9640 1 -- 172.21.15.38:0/62097672 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66c800bda0 con 0x55b0ef1c2390 2024-05-21T15:37:28.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.922+0000 7f66c4ff9640 1 -- 172.21.15.38:0/62097672 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8010380 con 0x55b0ef1c2390 2024-05-21T15:37:28.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.924+0000 7f66e3bd4740 1 -- 172.21.15.38:0/62097672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1c2390 msgr2=0x55b0ef561b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:28.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.924+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/62097672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1c2390 0x55b0ef561b00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009670 tx=0x7f66c8009b30 comp rx=0 tx=0).stop 2024-05-21T15:37:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.927+0000 7f66e3bd4740 1 -- 172.21.15.38:0/62097672 shutdown_connections 2024-05-21T15:37:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.927+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/62097672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1a16b0 0x55b0ef568f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.927+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/62097672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1994d0 0x55b0ef562080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.927+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/62097672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1c2390 0x55b0ef561b00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.927+0000 7f66e3bd4740 1 -- 172.21.15.38:0/62097672 >> 172.21.15.38:0/62097672 conn(0x55b0ef1b64e0 msgr2=0x55b0ef55d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:28.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.928+0000 7f66e3bd4740 1 -- 172.21.15.38:0/62097672 shutdown_connections 2024-05-21T15:37:28.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.928+0000 7f66e3bd4740 1 -- 172.21.15.38:0/62097672 wait complete. 2024-05-21T15:37:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.928+0000 7f66e3bd4740 1 -- start start 2024-05-21T15:37:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.929+0000 7f66e3bd4740 1 Processor -- start 2024-05-21T15:37:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.929+0000 7f66e3bd4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1994d0 0x55b0ef56ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.929+0000 7f66e3bd4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1a16b0 0x55b0ef56bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.929+0000 7f66e3bd4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1c2390 0x55b0ef572f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.929+0000 7f66e3bd4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef5699d0 con 0x55b0ef1a16b0 2024-05-21T15:37:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.929+0000 7f66e3bd4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef481c40 con 0x55b0ef1994d0 2024-05-21T15:37:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.929+0000 7f66e3bd4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef569b50 con 0x55b0ef1c2390 2024-05-21T15:37:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.929+0000 7f66e3bd4740 10 client.0 _ll_register_callbacks cb 0x55b0ef1d9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.930+0000 7f66d6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1c2390 0x55b0ef572f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:28.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.930+0000 7f66d6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1c2390 0x55b0ef572f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46482/0 (socket says 172.21.15.38:46482) 2024-05-21T15:37:28.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.930+0000 7f66d6d9d640 1 -- 172.21.15.38:0/4038658648 learned_addr learned my addr 172.21.15.38:0/4038658648 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:28.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.930+0000 7f66d5d9b640 1 --2- 172.21.15.38:0/4038658648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1a16b0 0x55b0ef56bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:28.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.931+0000 7f66d6d9d640 1 -- 172.21.15.38:0/4038658648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1994d0 msgr2=0x55b0ef56ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:28.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.931+0000 7f66d6d9d640 1 --2- 172.21.15.38:0/4038658648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1994d0 0x55b0ef56ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.931+0000 7f66d6d9d640 1 -- 172.21.15.38:0/4038658648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1a16b0 msgr2=0x55b0ef56bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:28.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.931+0000 7f66d6d9d640 1 --2- 172.21.15.38:0/4038658648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1a16b0 0x55b0ef56bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.931+0000 7f66d6d9d640 1 -- 172.21.15.38:0/4038658648 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ef2a47d0 con 0x55b0ef1c2390 2024-05-21T15:37:28.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.932+0000 7f66d6d9d640 1 --2- 172.21.15.38:0/4038658648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1c2390 0x55b0ef572f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f66c80114a0 tx=0x7f66c80114d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60eee3344352f4b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:28.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.932+0000 7f66bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:37:28.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.939+0000 7f66bdffb640 1 -- 172.21.15.38:0/4038658648 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8020070 con 0x55b0ef1c2390 2024-05-21T15:37:28.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.939+0000 7f66e3bd4740 1 -- 172.21.15.38:0/4038658648 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66c800bda0 con 0x55b0ef1c2390 2024-05-21T15:37:28.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.940+0000 7f66bdffb640 1 -- 172.21.15.38:0/4038658648 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f66c800bad0 con 0x55b0ef1c2390 2024-05-21T15:37:28.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.949+0000 7f66bdffb640 1 -- 172.21.15.38:0/4038658648 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f66c8004020 con 0x55b0ef1c2390 2024-05-21T15:37:28.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.949+0000 7f66e3bd4740 10 client.25996 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:28.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.949+0000 7f66e3bd4740 1 -- 172.21.15.38:0/4038658648 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b0ef598240 con 0x55b0ef1c2390 2024-05-21T15:37:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.952+0000 7f66bdffb640 1 -- 172.21.15.38:0/4038658648 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66c800bda0 con 0x55b0ef1c2390 2024-05-21T15:37:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.952+0000 7f66bdffb640 1 -- 172.21.15.38:0/4038658648 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f66c801a420 con 0x55b0ef1c2390 2024-05-21T15:37:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.952+0000 7f66e3bd4740 10 client.25996 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.952+0000 7f66e3bd4740 10 client.25996 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:37:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.952+0000 7f66e3bd4740 20 client.25996 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.952+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/4038658648 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b0ef180f60 0x55b0ef59be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.952+0000 7f66e3bd4740 4 client.25996 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:37:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.952+0000 7f66e3bd4740 1 -- 172.21.15.38:0/4038658648 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0ef4d4370 con 0x55b0ef180f60 2024-05-21T15:37:28.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.955+0000 7f66d659c640 1 --2- 172.21.15.38:0/4038658648 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b0ef180f60 0x55b0ef59be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:28.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.956+0000 7f66d659c640 1 --2- 172.21.15.38:0/4038658648 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b0ef180f60 0x55b0ef59be10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=db0666229620e046 server_cookie=7fe6ae5983e83f75 in_seq=0 out_seq=0 2024-05-21T15:37:28.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.956+0000 7f66bdffb640 10 client.25996 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:37:28.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.958+0000 7f66bdffb640 1 -- 172.21.15.38:0/4038658648 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b0ef4d4370 con 0x55b0ef180f60 2024-05-21T15:37:28.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.958+0000 7f66bdffb640 10 client.25996 handle_command_reply: tid=0 2024-05-21T15:37:28.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.959+0000 7f66e3bd4740 1 client.25996 shutdown 2024-05-21T15:37:28.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.962+0000 7f66e3bd4740 1 -- 172.21.15.38:0/4038658648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1c2390 msgr2=0x55b0ef572f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:28.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.962+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/4038658648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1c2390 0x55b0ef572f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f66c80114a0 tx=0x7f66c80114d0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.962+0000 7f66e3bd4740 1 -- 172.21.15.38:0/4038658648 shutdown_connections 2024-05-21T15:37:28.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.962+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/4038658648 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b0ef180f60 0x55b0ef59be10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.962+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/4038658648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1c2390 0x55b0ef572f20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.962+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/4038658648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1a16b0 0x55b0ef56bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.962+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/4038658648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1994d0 0x55b0ef56ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.962+0000 7f66e3bd4740 1 -- 172.21.15.38:0/4038658648 >> 172.21.15.38:0/4038658648 conn(0x55b0ef1b64e0 msgr2=0x55b0ef55c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:28.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.962+0000 7f66bdffb640 0 client.25996 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:37:28.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.965+0000 7f66e3bd4740 1 -- 172.21.15.38:0/4038658648 shutdown_connections 2024-05-21T15:37:28.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.965+0000 7f66e3bd4740 1 -- 172.21.15.38:0/4038658648 wait complete. 2024-05-21T15:37:28.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.966+0000 7f66e3bd4740 20 client.25996 trim_cache size 0 max 16384 2024-05-21T15:37:28.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.998+0000 7f66e3bd4740 1 Processor -- start 2024-05-21T15:37:28.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.998+0000 7f66e3bd4740 1 -- start start 2024-05-21T15:37:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.999+0000 7f66e3bd4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1994d0 0x55b0ef55d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.999+0000 7f66e3bd4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1a16b0 0x55b0ef572e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.999+0000 7f66e3bd4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1c2390 0x55b0ef560330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.999+0000 7f66e3bd4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f66b800e600 con 0x55b0ef1c2390 2024-05-21T15:37:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.999+0000 7f66e3bd4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8010230 con 0x55b0ef1994d0 2024-05-21T15:37:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:28.999+0000 7f66e3bd4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f66b800ecd0 con 0x55b0ef1a16b0 2024-05-21T15:37:28.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.000+0000 7f66d6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1c2390 0x55b0ef560330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:28.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.000+0000 7f66d6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1c2390 0x55b0ef560330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40792/0 (socket says 172.21.15.38:40792) 2024-05-21T15:37:28.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.000+0000 7f66d6d9d640 1 -- 172.21.15.38:0/1435721735 learned_addr learned my addr 172.21.15.38:0/1435721735 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:28.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.000+0000 7f66d6d9d640 1 -- 172.21.15.38:0/1435721735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1a16b0 msgr2=0x55b0ef572e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:28.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.000+0000 7f66d6d9d640 1 --2- 172.21.15.38:0/1435721735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1a16b0 0x55b0ef572e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:28.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.000+0000 7f66d6d9d640 1 -- 172.21.15.38:0/1435721735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1994d0 msgr2=0x55b0ef55d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:28.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.000+0000 7f66d6d9d640 1 --2- 172.21.15.38:0/1435721735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1994d0 0x55b0ef55d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.000+0000 7f66d6d9d640 1 -- 172.21.15.38:0/1435721735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ef573380 con 0x55b0ef1c2390 2024-05-21T15:37:29.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.001+0000 7f66d6d9d640 1 --2- 172.21.15.38:0/1435721735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1c2390 0x55b0ef560330 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f66c8000c00 tx=0x7f66c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69e5e2273f639b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.002+0000 7f66bffff640 1 -- 172.21.15.38:0/1435721735 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8035070 con 0x55b0ef1c2390 2024-05-21T15:37:29.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.002+0000 7f66bffff640 1 -- 172.21.15.38:0/1435721735 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66c801aab0 con 0x55b0ef1c2390 2024-05-21T15:37:29.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.002+0000 7f66bffff640 1 -- 172.21.15.38:0/1435721735 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8020040 con 0x55b0ef1c2390 2024-05-21T15:37:29.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.003+0000 7f66e3bd4740 1 -- 172.21.15.38:0/1435721735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1c2390 msgr2=0x55b0ef560330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.003+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/1435721735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1c2390 0x55b0ef560330 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f66c8000c00 tx=0x7f66c8004750 comp rx=0 tx=0).stop 2024-05-21T15:37:29.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.008+0000 7f66e3bd4740 1 -- 172.21.15.38:0/1435721735 shutdown_connections 2024-05-21T15:37:29.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.008+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/1435721735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1c2390 0x55b0ef560330 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.008+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/1435721735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1a16b0 0x55b0ef572e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.008+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/1435721735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1994d0 0x55b0ef55d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.008+0000 7f66e3bd4740 1 -- 172.21.15.38:0/1435721735 >> 172.21.15.38:0/1435721735 conn(0x55b0ef1b64e0 msgr2=0x55b0ef55ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.011+0000 7f66e3bd4740 1 -- 172.21.15.38:0/1435721735 shutdown_connections 2024-05-21T15:37:29.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.011+0000 7f66e3bd4740 1 -- 172.21.15.38:0/1435721735 wait complete. 2024-05-21T15:37:29.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.011+0000 7f66e3bd4740 1 -- start start 2024-05-21T15:37:29.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.011+0000 7f66e3bd4740 1 Processor -- start 2024-05-21T15:37:29.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.013+0000 7f66e3bd4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1994d0 0x55b0ef572c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.013+0000 7f66e3bd4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1a16b0 0x55b0ef56f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.013+0000 7f66e3bd4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1c2390 0x55b0ef56fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.013+0000 7f66e3bd4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f66b800faa0 con 0x55b0ef1c2390 2024-05-21T15:37:29.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.013+0000 7f66e3bd4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f66b800f3a0 con 0x55b0ef1a16b0 2024-05-21T15:37:29.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.013+0000 7f66e3bd4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f66b800c5e0 con 0x55b0ef1994d0 2024-05-21T15:37:29.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.013+0000 7f66e3bd4740 10 client.0 _ll_register_callbacks cb 0x7f66c8007f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:29.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.013+0000 7f66d659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1994d0 0x55b0ef572c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.013+0000 7f66d659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1994d0 0x55b0ef572c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46500/0 (socket says 172.21.15.38:46500) 2024-05-21T15:37:29.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.013+0000 7f66d659c640 1 -- 172.21.15.38:0/1911842743 learned_addr learned my addr 172.21.15.38:0/1911842743 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:29.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.014+0000 7f66d6d9d640 1 --2- 172.21.15.38:0/1911842743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1c2390 0x55b0ef56fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.014+0000 7f66d5d9b640 1 --2- 172.21.15.38:0/1911842743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1a16b0 0x55b0ef56f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.014+0000 7f66d659c640 1 -- 172.21.15.38:0/1911842743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1a16b0 msgr2=0x55b0ef56f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.014+0000 7f66d659c640 1 --2- 172.21.15.38:0/1911842743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1a16b0 0x55b0ef56f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.014+0000 7f66d659c640 1 -- 172.21.15.38:0/1911842743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1c2390 msgr2=0x55b0ef56fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.014+0000 7f66d659c640 1 --2- 172.21.15.38:0/1911842743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1c2390 0x55b0ef56fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.014+0000 7f66d659c640 1 -- 172.21.15.38:0/1911842743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ef6021b0 con 0x55b0ef1994d0 2024-05-21T15:37:29.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.014+0000 7f66d659c640 1 --2- 172.21.15.38:0/1911842743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1994d0 0x55b0ef572c30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f66d002aee0 tx=0x7f66d00318e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4e3003c92c0616f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.015+0000 7f66bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:37:29.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.015+0000 7f66bdffb640 1 -- 172.21.15.38:0/1911842743 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0057070 con 0x55b0ef1994d0 2024-05-21T15:37:29.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.015+0000 7f66e3bd4740 1 -- 172.21.15.38:0/1911842743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66c801aab0 con 0x55b0ef1994d0 2024-05-21T15:37:29.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.015+0000 7f66bdffb640 1 -- 172.21.15.38:0/1911842743 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f66d0003bf0 con 0x55b0ef1994d0 2024-05-21T15:37:29.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.016+0000 7f66bdffb640 1 -- 172.21.15.38:0/1911842743 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f66d0063050 con 0x55b0ef1994d0 2024-05-21T15:37:29.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.016+0000 7f66e3bd4740 10 client.25999 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:29.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.016+0000 7f66e3bd4740 1 -- 172.21.15.38:0/1911842743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b0ef603380 con 0x55b0ef1994d0 2024-05-21T15:37:29.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.017+0000 7f66bdffb640 1 -- 172.21.15.38:0/1911842743 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66c801aab0 con 0x55b0ef1994d0 2024-05-21T15:37:29.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.017+0000 7f66bdffb640 1 -- 172.21.15.38:0/1911842743 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f66d0004340 con 0x55b0ef1994d0 2024-05-21T15:37:29.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.017+0000 7f66e3bd4740 10 client.25999 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:29.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.017+0000 7f66e3bd4740 10 client.25999 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:37:29.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.017+0000 7f66e3bd4740 20 client.25999 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:29.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.017+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/1911842743 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b0ef1563a0 0x55b0ef605d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.017+0000 7f66e3bd4740 4 client.25999 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:37:29.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.017+0000 7f66e3bd4740 1 -- 172.21.15.38:0/1911842743 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0ef4ce6c0 con 0x55b0ef1563a0 2024-05-21T15:37:29.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.020+0000 7f66d5d9b640 1 --2- 172.21.15.38:0/1911842743 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b0ef1563a0 0x55b0ef605d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.024+0000 7f66d5d9b640 1 --2- 172.21.15.38:0/1911842743 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b0ef1563a0 0x55b0ef605d70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=21a795d1fc1c6a86 server_cookie=1c73fdb19459ca4b in_seq=0 out_seq=0 2024-05-21T15:37:29.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.024+0000 7f66bdffb640 10 client.25999 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:37:29.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.024+0000 7f66bdffb640 1 -- 172.21.15.38:0/1911842743 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0ef4ce6c0 con 0x55b0ef1563a0 2024-05-21T15:37:29.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.024+0000 7f66bdffb640 10 client.25999 handle_command_reply: tid=0 2024-05-21T15:37:29.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.024+0000 7f66e3bd4740 1 client.25999 shutdown 2024-05-21T15:37:29.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.026+0000 7f66e3bd4740 1 -- 172.21.15.38:0/1911842743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1994d0 msgr2=0x55b0ef572c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.026+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/1911842743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1994d0 0x55b0ef572c30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f66d002aee0 tx=0x7f66d00318e0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.028+0000 7f66e3bd4740 1 -- 172.21.15.38:0/1911842743 shutdown_connections 2024-05-21T15:37:29.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.028+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/1911842743 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55b0ef1563a0 0x55b0ef605d70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.028+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/1911842743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0ef1c2390 0x55b0ef56fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.028+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/1911842743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0ef1a16b0 0x55b0ef56f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.028+0000 7f66e3bd4740 1 --2- 172.21.15.38:0/1911842743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b0ef1994d0 0x55b0ef572c30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.028+0000 7f66e3bd4740 1 -- 172.21.15.38:0/1911842743 >> 172.21.15.38:0/1911842743 conn(0x55b0ef1b64e0 msgr2=0x55b0ef55bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.028+0000 7f66bdffb640 0 client.25999 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:37:29.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.029+0000 7f66e3bd4740 1 -- 172.21.15.38:0/1911842743 shutdown_connections 2024-05-21T15:37:29.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.029+0000 7f66e3bd4740 1 -- 172.21.15.38:0/1911842743 wait complete. 2024-05-21T15:37:29.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.029+0000 7f66e3bd4740 20 client.25999 trim_cache size 0 max 16384 2024-05-21T15:37:29.031 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:37:29.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.033+0000 7f66dcc56640 1 -- 172.21.15.38:0/4244220881 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f66c00a2fd0 msgr2=0x7f66c00a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.033+0000 7f66dcc56640 1 --2- 172.21.15.38:0/4244220881 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f66c00a2fd0 0x7f66c00a54e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55b0ef562ee0 tx=0x7f66d002adf0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.034+0000 7f66dcc56640 1 -- 172.21.15.38:0/4244220881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d8074890 msgr2=0x7f66d810d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.034+0000 7f66dcc56640 1 --2- 172.21.15.38:0/4244220881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d8074890 0x7f66d810d020 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00b450 tx=0x7f66cc00b910 comp rx=0 tx=0).stop 2024-05-21T15:37:29.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.035+0000 7f66dcc56640 1 -- 172.21.15.38:0/4244220881 shutdown_connections 2024-05-21T15:37:29.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.035+0000 7f66dcc56640 1 --2- 172.21.15.38:0/4244220881 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f66c00a2fd0 0x7f66c00a54e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.035+0000 7f66dcc56640 1 --2- 172.21.15.38:0/4244220881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66d810f1f0 0x7f66d810d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.035+0000 7f66dcc56640 1 --2- 172.21.15.38:0/4244220881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d8074890 0x7f66d810d020 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.035+0000 7f66dcc56640 1 --2- 172.21.15.38:0/4244220881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66d8073f40 0x7f66d810e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.035+0000 7f66dcc56640 1 -- 172.21.15.38:0/4244220881 >> 172.21.15.38:0/4244220881 conn(0x7f66d80fdc80 msgr2=0x7f66d80feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.035+0000 7f66dcc56640 1 -- 172.21.15.38:0/4244220881 shutdown_connections 2024-05-21T15:37:29.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.035+0000 7f66dcc56640 1 -- 172.21.15.38:0/4244220881 wait complete. 2024-05-21T15:37:29.055 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:37:29.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:28 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2985341561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:37:29.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.207+0000 7f210effd640 1 -- 172.21.15.38:0/2352875405 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100030920 con 0x7f211010efe0 2024-05-21T15:37:29.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.208+0000 7f21164ca640 1 -- 172.21.15.38:0/2352875405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211010efe0 msgr2=0x7f211010f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.208+0000 7f21164ca640 1 --2- 172.21.15.38:0/2352875405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211010efe0 0x7f211010f3f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2100009860 tx=0x7f210002a360 comp rx=0 tx=0).stop 2024-05-21T15:37:29.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.208+0000 7f21164ca640 1 -- 172.21.15.38:0/2352875405 shutdown_connections 2024-05-21T15:37:29.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.208+0000 7f21164ca640 1 --2- 172.21.15.38:0/2352875405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211010efe0 0x7f211010f3f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.208+0000 7f21164ca640 1 --2- 172.21.15.38:0/2352875405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211010a2c0 0x7f211010a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.208+0000 7f21164ca640 1 --2- 172.21.15.38:0/2352875405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21101093d0 0x7f2110109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.208+0000 7f21164ca640 1 -- 172.21.15.38:0/2352875405 >> 172.21.15.38:0/2352875405 conn(0x7f211006f4a0 msgr2=0x7f2110071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.213+0000 7f21164ca640 1 -- 172.21.15.38:0/2352875405 shutdown_connections 2024-05-21T15:37:29.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.213+0000 7f21164ca640 1 -- 172.21.15.38:0/2352875405 wait complete. 2024-05-21T15:37:29.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.214+0000 7f21164ca640 1 Processor -- start 2024-05-21T15:37:29.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.214+0000 7f21164ca640 1 -- start start 2024-05-21T15:37:29.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.215+0000 7f21164ca640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21101093d0 0x7f21101a8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.215+0000 7f21164ca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211010a2c0 0x7f21101a8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.215+0000 7f21164ca640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211010efe0 0x7f21101af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.215+0000 7f21164ca640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f211011d640 con 0x7f21101093d0 2024-05-21T15:37:29.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.215+0000 7f21164ca640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f211011d4c0 con 0x7f211010efe0 2024-05-21T15:37:29.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.215+0000 7f21164ca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f211011d7c0 con 0x7f211010a2c0 2024-05-21T15:37:29.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.215+0000 7f210ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21101093d0 0x7f21101a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.215+0000 7f210ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21101093d0 0x7f21101a8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40824/0 (socket says 172.21.15.38:40824) 2024-05-21T15:37:29.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.215+0000 7f210ffff640 1 -- 172.21.15.38:0/2508552429 learned_addr learned my addr 172.21.15.38:0/2508552429 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:29.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.216+0000 7f210ffff640 1 -- 172.21.15.38:0/2508552429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211010a2c0 msgr2=0x7f21101a8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.216+0000 7f210ffff640 1 --2- 172.21.15.38:0/2508552429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211010a2c0 0x7f21101a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.216+0000 7f210ffff640 1 -- 172.21.15.38:0/2508552429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211010efe0 msgr2=0x7f21101af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.216+0000 7f210ffff640 1 --2- 172.21.15.38:0/2508552429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211010efe0 0x7f21101af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.216+0000 7f210ffff640 1 -- 172.21.15.38:0/2508552429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21101afc40 con 0x7f21101093d0 2024-05-21T15:37:29.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.217+0000 7f210ffff640 1 --2- 172.21.15.38:0/2508552429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21101093d0 0x7f21101a8100 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f210800b9c0 tx=0x7f210800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.217+0000 7f210d7fa640 1 -- 172.21.15.38:0/2508552429 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2108019070 con 0x7f21101093d0 2024-05-21T15:37:29.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.217+0000 7f210d7fa640 1 -- 172.21.15.38:0/2508552429 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210800cba0 con 0x7f21101093d0 2024-05-21T15:37:29.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.217+0000 7f210d7fa640 1 -- 172.21.15.38:0/2508552429 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21080126b0 con 0x7f21101093d0 2024-05-21T15:37:29.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.217+0000 7f21164ca640 1 -- 172.21.15.38:0/2508552429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21101aff30 con 0x7f21101093d0 2024-05-21T15:37:29.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.217+0000 7f21164ca640 1 -- 172.21.15.38:0/2508552429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21101b0440 con 0x7f21101093d0 2024-05-21T15:37:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.226+0000 7f211d36f740 1 Processor -- start 2024-05-21T15:37:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.226+0000 7f210d7fa640 1 -- 172.21.15.38:0/2508552429 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f210801c070 con 0x7f21101093d0 2024-05-21T15:37:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.226+0000 7f211d36f740 1 -- start start 2024-05-21T15:37:29.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.228+0000 7f211d36f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 0x564ceba7eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.228+0000 7f211d36f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba7f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.228+0000 7f211d36f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6be6b0 0x564ceba85f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.228+0000 7f211d36f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564ceb9e3c40 con 0x564ceb6df390 2024-05-21T15:37:29.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.228+0000 7f211d36f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564ceb98b580 con 0x564ceb6b64d0 2024-05-21T15:37:29.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.228+0000 7f211d36f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564ceb9dfc00 con 0x564ceb6be6b0 2024-05-21T15:37:29.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.228+0000 7f210ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba7f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.228+0000 7f210ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba7f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45210/0 (socket says 172.21.15.38:45210) 2024-05-21T15:37:29.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.228+0000 7f210ffff640 1 -- 172.21.15.38:0/394406401 learned_addr learned my addr 172.21.15.38:0/394406401 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:29.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.228+0000 7f210f7fe640 1 --2- 172.21.15.38:0/394406401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6be6b0 0x564ceba85f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.229+0000 7f2114a67640 1 --2- 172.21.15.38:0/394406401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 0x564ceba7eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.229+0000 7f210f7fe640 1 -- 172.21.15.38:0/394406401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 msgr2=0x564ceba7f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.229+0000 7f210f7fe640 1 --2- 172.21.15.38:0/394406401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba7f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.229+0000 7f210f7fe640 1 -- 172.21.15.38:0/394406401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 msgr2=0x564ceba7eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.229+0000 7f210f7fe640 1 --2- 172.21.15.38:0/394406401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 0x564ceba7eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.229+0000 7f210f7fe640 1 -- 172.21.15.38:0/394406401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ceb6f73f0 con 0x564ceb6be6b0 2024-05-21T15:37:29.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.229+0000 7f210d7fa640 1 --2- 172.21.15.38:0/2508552429 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f20fc0a2c90 0x7f20fc0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.229+0000 7f210f7fe640 1 --2- 172.21.15.38:0/394406401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6be6b0 0x564ceba85f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f20f8005fd0 tx=0x7f20f8009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92aee1bce41359a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.230+0000 7f210d7fa640 1 -- 172.21.15.38:0/2508552429 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f21080ce660 con 0x7f21101093d0 2024-05-21T15:37:29.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.230+0000 7f21067fc640 1 -- 172.21.15.38:0/394406401 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f800ec90 con 0x564ceb6be6b0 2024-05-21T15:37:29.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.230+0000 7f21067fc640 1 -- 172.21.15.38:0/394406401 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20f800ee30 con 0x564ceb6be6b0 2024-05-21T15:37:29.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.230+0000 7f21067fc640 1 -- 172.21.15.38:0/394406401 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8004b80 con 0x564ceb6be6b0 2024-05-21T15:37:29.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.231+0000 7f211d36f740 1 -- 172.21.15.38:0/394406401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6be6b0 msgr2=0x564ceba85f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.231+0000 7f211d36f740 1 --2- 172.21.15.38:0/394406401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6be6b0 0x564ceba85f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f20f8005fd0 tx=0x7f20f8009f80 comp rx=0 tx=0).stop 2024-05-21T15:37:29.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.232+0000 7f2114a67640 1 --2- 172.21.15.38:0/2508552429 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f20fc0a2c90 0x7f20fc0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.232+0000 7f211d36f740 1 -- 172.21.15.38:0/394406401 shutdown_connections 2024-05-21T15:37:29.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.232+0000 7f211d36f740 1 --2- 172.21.15.38:0/394406401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6be6b0 0x564ceba85f80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.232+0000 7f211d36f740 1 --2- 172.21.15.38:0/394406401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba7f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.232+0000 7f211d36f740 1 --2- 172.21.15.38:0/394406401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 0x564ceba7eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.232+0000 7f211d36f740 1 -- 172.21.15.38:0/394406401 >> 172.21.15.38:0/394406401 conn(0x564ceb6d34e0 msgr2=0x564ceba7a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.232+0000 7f211d36f740 1 -- 172.21.15.38:0/394406401 shutdown_connections 2024-05-21T15:37:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.233+0000 7f211d36f740 1 -- 172.21.15.38:0/394406401 wait complete. 2024-05-21T15:37:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.234+0000 7f211d36f740 1 -- start start 2024-05-21T15:37:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.235+0000 7f211d36f740 1 Processor -- start 2024-05-21T15:37:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.236+0000 7f2114a67640 1 --2- 172.21.15.38:0/2508552429 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f20fc0a2c90 0x7f20fc0a51a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2100009830 tx=0x7f2100030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.237+0000 7f211d36f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.237+0000 7f211d36f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6be6b0 0x564ceba88bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.237+0000 7f211d36f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6df390 0x564ceba8fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.237+0000 7f211d36f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564ceba869d0 con 0x564ceb6be6b0 2024-05-21T15:37:29.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.237+0000 7f211d36f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564ceb99ec40 con 0x564ceb6b64d0 2024-05-21T15:37:29.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.237+0000 7f211d36f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564ceba86b50 con 0x564ceb6df390 2024-05-21T15:37:29.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.238+0000 7f210ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.238+0000 7f210ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba886b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45212/0 (socket says 172.21.15.38:45212) 2024-05-21T15:37:29.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.238+0000 7f210ffff640 1 -- 172.21.15.38:0/2035348828 learned_addr learned my addr 172.21.15.38:0/2035348828 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:29.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.237+0000 7f211d36f740 10 client.0 _ll_register_callbacks cb 0x564ceb6f6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:29.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.238+0000 7f2114a67640 1 --2- 172.21.15.38:0/2035348828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6df390 0x564ceba8fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.238+0000 7f210f7fe640 1 --2- 172.21.15.38:0/2035348828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6be6b0 0x564ceba88bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.241+0000 7f210f7fe640 1 -- 172.21.15.38:0/2035348828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6df390 msgr2=0x564ceba8fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.241+0000 7f210f7fe640 1 --2- 172.21.15.38:0/2035348828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6df390 0x564ceba8fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.241+0000 7f210f7fe640 1 -- 172.21.15.38:0/2035348828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 msgr2=0x564ceba886b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.241+0000 7f210f7fe640 1 --2- 172.21.15.38:0/2035348828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba886b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.241+0000 7f210f7fe640 1 -- 172.21.15.38:0/2035348828 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ceb7c17d0 con 0x564ceb6be6b0 2024-05-21T15:37:29.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.242+0000 7f210f7fe640 1 --2- 172.21.15.38:0/2035348828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6be6b0 0x564ceba88bf0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f20f8005fd0 tx=0x7f20f8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7f67aa78e3065a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.242+0000 7f20ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:37:29.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.243+0000 7f20ef7fe640 1 -- 172.21.15.38:0/2035348828 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8004450 con 0x564ceb6be6b0 2024-05-21T15:37:29.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.243+0000 7f20ef7fe640 1 -- 172.21.15.38:0/2035348828 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f20f8004880 con 0x564ceb6be6b0 2024-05-21T15:37:29.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.243+0000 7f211d36f740 1 -- 172.21.15.38:0/2035348828 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20f800ee30 con 0x564ceb6be6b0 2024-05-21T15:37:29.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.244+0000 7f20ef7fe640 1 -- 172.21.15.38:0/2035348828 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f20f8004a80 con 0x564ceb6be6b0 2024-05-21T15:37:29.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.244+0000 7f211d36f740 10 client.16302 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:29.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.244+0000 7f211d36f740 1 -- 172.21.15.38:0/2035348828 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564ceba92e20 con 0x564ceb6be6b0 2024-05-21T15:37:29.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.245+0000 7f20ef7fe640 1 -- 172.21.15.38:0/2035348828 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20f800ee30 con 0x564ceb6be6b0 2024-05-21T15:37:29.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.246+0000 7f20ef7fe640 1 -- 172.21.15.38:0/2035348828 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f20f8018470 con 0x564ceb6be6b0 2024-05-21T15:37:29.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.246+0000 7f211d36f740 10 client.16302 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:29.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.246+0000 7f211d36f740 10 client.16302 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:37:29.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.246+0000 7f211d36f740 20 client.16302 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:29.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.246+0000 7f211d36f740 1 --2- 172.21.15.38:0/2035348828 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564ceb69df60 0x564ceba96850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.246+0000 7f211d36f740 4 client.16302 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:37:29.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.246+0000 7f211d36f740 1 -- 172.21.15.38:0/2035348828 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ceb9f1370 con 0x564ceb69df60 2024-05-21T15:37:29.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.246+0000 7f210ffff640 1 --2- 172.21.15.38:0/2035348828 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564ceb69df60 0x564ceba96850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.250+0000 7f210ffff640 1 --2- 172.21.15.38:0/2035348828 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564ceb69df60 0x564ceba96850 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6aadece3568099ef server_cookie=146d3c63cd7af783 in_seq=0 out_seq=0 2024-05-21T15:37:29.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.250+0000 7f20ef7fe640 10 client.16302 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:37:29.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.252+0000 7f20ef7fe640 1 -- 172.21.15.38:0/2035348828 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ceb9f1370 con 0x564ceb69df60 2024-05-21T15:37:29.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.252+0000 7f20ef7fe640 10 client.16302 handle_command_reply: tid=0 2024-05-21T15:37:29.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.252+0000 7f211d36f740 1 client.16302 shutdown 2024-05-21T15:37:29.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.253+0000 7f211d36f740 1 -- 172.21.15.38:0/2035348828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6be6b0 msgr2=0x564ceba88bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.253+0000 7f211d36f740 1 --2- 172.21.15.38:0/2035348828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6be6b0 0x564ceba88bf0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f20f8005fd0 tx=0x7f20f8009260 comp rx=0 tx=0).stop 2024-05-21T15:37:29.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.253+0000 7f211d36f740 1 -- 172.21.15.38:0/2035348828 shutdown_connections 2024-05-21T15:37:29.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.253+0000 7f211d36f740 1 --2- 172.21.15.38:0/2035348828 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564ceb69df60 0x564ceba96850 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.253+0000 7f211d36f740 1 --2- 172.21.15.38:0/2035348828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6df390 0x564ceba8fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.253+0000 7f211d36f740 1 --2- 172.21.15.38:0/2035348828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6be6b0 0x564ceba88bf0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.253+0000 7f211d36f740 1 --2- 172.21.15.38:0/2035348828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.253+0000 7f211d36f740 1 -- 172.21.15.38:0/2035348828 >> 172.21.15.38:0/2035348828 conn(0x564ceb6d34e0 msgr2=0x564ceba79570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.253+0000 7f20ef7fe640 0 client.16302 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:37:29.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.253+0000 7f211d36f740 1 -- 172.21.15.38:0/2035348828 shutdown_connections 2024-05-21T15:37:29.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.253+0000 7f211d36f740 1 -- 172.21.15.38:0/2035348828 wait complete. 2024-05-21T15:37:29.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.253+0000 7f211d36f740 20 client.16302 trim_cache size 0 max 16384 2024-05-21T15:37:29.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:28 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2985341561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:37:29.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:28 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2985341561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:37:29.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.285+0000 7f211d36f740 1 Processor -- start 2024-05-21T15:37:29.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.287+0000 7f211d36f740 1 -- start start 2024-05-21T15:37:29.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.288+0000 7f211d36f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba88050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.288+0000 7f211d36f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6be6b0 0x564ceba8fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.288+0000 7f211d36f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 0x564ceba79ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.288+0000 7f210ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba88050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.288+0000 7f210ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba88050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45224/0 (socket says 172.21.15.38:45224) 2024-05-21T15:37:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.289+0000 7f210ffff640 1 -- 172.21.15.38:0/143995127 learned_addr learned my addr 172.21.15.38:0/143995127 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.289+0000 7f211d36f740 1 -- 172.21.15.38:0/143995127 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20f000e680 con 0x564ceb6df390 2024-05-21T15:37:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.289+0000 7f211d36f740 1 -- 172.21.15.38:0/143995127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20f00102b0 con 0x564ceb6b64d0 2024-05-21T15:37:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.289+0000 7f211d36f740 1 -- 172.21.15.38:0/143995127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f20f000ed50 con 0x564ceb6be6b0 2024-05-21T15:37:29.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.290+0000 7f210f7fe640 1 --2- 172.21.15.38:0/143995127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6be6b0 0x564ceba8fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.291+0000 7f210ffff640 1 -- 172.21.15.38:0/143995127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6be6b0 msgr2=0x564ceba8fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.291+0000 7f210ffff640 1 --2- 172.21.15.38:0/143995127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6be6b0 0x564ceba8fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.291+0000 7f210ffff640 1 -- 172.21.15.38:0/143995127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 msgr2=0x564ceba79ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.291+0000 7f210ffff640 1 --2- 172.21.15.38:0/143995127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 0x564ceba79ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.291+0000 7f210ffff640 1 -- 172.21.15.38:0/143995127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ceba8ffb0 con 0x564ceb6b64d0 2024-05-21T15:37:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.293+0000 7f210ffff640 1 --2- 172.21.15.38:0/143995127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba88050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2108000f80 tx=0x7f210800e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c05cb1fa09e1b612 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.295+0000 7f2105ffb640 1 -- 172.21.15.38:0/143995127 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210809b510 con 0x564ceb6b64d0 2024-05-21T15:37:29.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.296+0000 7f2105ffb640 1 -- 172.21.15.38:0/143995127 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210809ae10 con 0x564ceb6b64d0 2024-05-21T15:37:29.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.296+0000 7f2105ffb640 1 -- 172.21.15.38:0/143995127 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210800cd50 con 0x564ceb6b64d0 2024-05-21T15:37:29.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.298+0000 7f211d36f740 1 -- 172.21.15.38:0/143995127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 msgr2=0x564ceba88050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.298+0000 7f211d36f740 1 --2- 172.21.15.38:0/143995127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba88050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2108000f80 tx=0x7f210800e040 comp rx=0 tx=0).stop 2024-05-21T15:37:29.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.299+0000 7f211d36f740 1 -- 172.21.15.38:0/143995127 shutdown_connections 2024-05-21T15:37:29.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.299+0000 7f211d36f740 1 --2- 172.21.15.38:0/143995127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 0x564ceba79ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.300+0000 7f211d36f740 1 --2- 172.21.15.38:0/143995127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6be6b0 0x564ceba8fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.300+0000 7f211d36f740 1 --2- 172.21.15.38:0/143995127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6b64d0 0x564ceba88050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.300+0000 7f211d36f740 1 -- 172.21.15.38:0/143995127 >> 172.21.15.38:0/143995127 conn(0x564ceb6d34e0 msgr2=0x564ceba87940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.301+0000 7f211d36f740 1 -- 172.21.15.38:0/143995127 shutdown_connections 2024-05-21T15:37:29.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.301+0000 7f211d36f740 1 -- 172.21.15.38:0/143995127 wait complete. 2024-05-21T15:37:29.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.302+0000 7f211d36f740 1 -- start start 2024-05-21T15:37:29.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.302+0000 7f211d36f740 1 Processor -- start 2024-05-21T15:37:29.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.304+0000 7f211d36f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6b64d0 0x564ceba8a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.304+0000 7f211d36f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6be6b0 0x564ceba8f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.304+0000 7f211d36f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 0x564ceba8a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.305+0000 7f211d36f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20f000fb20 con 0x564ceb6df390 2024-05-21T15:37:29.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.305+0000 7f211d36f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20f000f420 con 0x564ceb6be6b0 2024-05-21T15:37:29.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.305+0000 7f211d36f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f20f000c660 con 0x564ceb6b64d0 2024-05-21T15:37:29.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.305+0000 7f211d36f740 10 client.0 _ll_register_callbacks cb 0x7f20f8005550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:29.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.308+0000 7f210ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6b64d0 0x564ceba8a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.308+0000 7f210ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6b64d0 0x564ceba8a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46566/0 (socket says 172.21.15.38:46566) 2024-05-21T15:37:29.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.308+0000 7f210ffff640 1 -- 172.21.15.38:0/3812420231 learned_addr learned my addr 172.21.15.38:0/3812420231 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:29.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.308+0000 7f2114a67640 1 --2- 172.21.15.38:0/3812420231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 0x564ceba8a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.308+0000 7f210f7fe640 1 --2- 172.21.15.38:0/3812420231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6be6b0 0x564ceba8f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.310+0000 7f210ffff640 1 -- 172.21.15.38:0/3812420231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6be6b0 msgr2=0x564ceba8f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.310+0000 7f210ffff640 1 --2- 172.21.15.38:0/3812420231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6be6b0 0x564ceba8f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.310+0000 7f210ffff640 1 -- 172.21.15.38:0/3812420231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 msgr2=0x564ceba8a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.310+0000 7f210ffff640 1 --2- 172.21.15.38:0/3812420231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 0x564ceba8a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.311+0000 7f210ffff640 1 -- 172.21.15.38:0/3812420231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ceba8ade0 con 0x564ceb6b64d0 2024-05-21T15:37:29.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.311+0000 7f2114a67640 1 --2- 172.21.15.38:0/3812420231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 0x564ceba8a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:37:29.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.312+0000 7f210ffff640 1 --2- 172.21.15.38:0/3812420231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6b64d0 0x564ceba8a2e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f210800c6a0 tx=0x7f21080cc250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f216c943a3978278 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.312+0000 7f20ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:37:29.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.314+0000 7f20ef7fe640 1 -- 172.21.15.38:0/3812420231 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210809b510 con 0x564ceb6b64d0 2024-05-21T15:37:29.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.314+0000 7f20ef7fe640 1 -- 172.21.15.38:0/3812420231 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f210809b070 con 0x564ceb6b64d0 2024-05-21T15:37:29.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.315+0000 7f211d36f740 1 -- 172.21.15.38:0/3812420231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f210809ae10 con 0x564ceb6b64d0 2024-05-21T15:37:29.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.317+0000 7f20ef7fe640 1 -- 172.21.15.38:0/3812420231 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f210809b270 con 0x564ceb6b64d0 2024-05-21T15:37:29.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.318+0000 7f211d36f740 10 client.26011 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:29.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.318+0000 7f211d36f740 1 -- 172.21.15.38:0/3812420231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564cebafdc70 con 0x564ceb6b64d0 2024-05-21T15:37:29.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.321+0000 7f20ef7fe640 1 -- 172.21.15.38:0/3812420231 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21080a35f0 con 0x564ceb6b64d0 2024-05-21T15:37:29.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.323+0000 7f20ef7fe640 1 -- 172.21.15.38:0/3812420231 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f21080ce0d0 con 0x564ceb6b64d0 2024-05-21T15:37:29.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.323+0000 7f211d36f740 10 client.26011 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:29.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.323+0000 7f211d36f740 10 client.26011 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:37:29.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.323+0000 7f211d36f740 20 client.26011 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:29.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.323+0000 7f211d36f740 1 --2- 172.21.15.38:0/3812420231 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564ceb6733a0 0x564cebb004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.323+0000 7f211d36f740 4 client.26011 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:37:29.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.323+0000 7f211d36f740 1 -- 172.21.15.38:0/3812420231 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f210809ae10 con 0x564ceb6733a0 2024-05-21T15:37:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.328+0000 7f210f7fe640 1 --2- 172.21.15.38:0/3812420231 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564ceb6733a0 0x564cebb004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.334+0000 7f210f7fe640 1 --2- 172.21.15.38:0/3812420231 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564ceb6733a0 0x564cebb004a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7689ddb0d077ba0 server_cookie=2072743cf2913391 in_seq=0 out_seq=0 2024-05-21T15:37:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.334+0000 7f20ef7fe640 10 client.26011 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:37:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.335+0000 7f20ef7fe640 1 -- 172.21.15.38:0/3812420231 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f210809ae10 con 0x564ceb6733a0 2024-05-21T15:37:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.335+0000 7f20ef7fe640 10 client.26011 handle_command_reply: tid=0 2024-05-21T15:37:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.336+0000 7f211d36f740 1 client.26011 shutdown 2024-05-21T15:37:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.340+0000 7f211d36f740 1 -- 172.21.15.38:0/3812420231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6b64d0 msgr2=0x564ceba8a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.340+0000 7f211d36f740 1 --2- 172.21.15.38:0/3812420231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6b64d0 0x564ceba8a2e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f210800c6a0 tx=0x7f21080cc250 comp rx=0 tx=0).stop 2024-05-21T15:37:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.345+0000 7f211d36f740 1 -- 172.21.15.38:0/3812420231 shutdown_connections 2024-05-21T15:37:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.345+0000 7f211d36f740 1 --2- 172.21.15.38:0/3812420231 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564ceb6733a0 0x564cebb004a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.345+0000 7f211d36f740 1 --2- 172.21.15.38:0/3812420231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564ceb6df390 0x564ceba8a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.345+0000 7f211d36f740 1 --2- 172.21.15.38:0/3812420231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ceb6be6b0 0x564ceba8f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.345+0000 7f211d36f740 1 --2- 172.21.15.38:0/3812420231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ceb6b64d0 0x564ceba8a2e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.345+0000 7f211d36f740 1 -- 172.21.15.38:0/3812420231 >> 172.21.15.38:0/3812420231 conn(0x564ceb6d34e0 msgr2=0x564ceba79ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.347+0000 7f20ef7fe640 0 client.26011 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:37:29.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.351+0000 7f211d36f740 1 -- 172.21.15.38:0/3812420231 shutdown_connections 2024-05-21T15:37:29.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.351+0000 7f211d36f740 1 -- 172.21.15.38:0/3812420231 wait complete. 2024-05-21T15:37:29.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.351+0000 7f211d36f740 20 client.26011 trim_cache size 0 max 16384 2024-05-21T15:37:29.351 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:37:29.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.354+0000 7f21164ca640 1 -- 172.21.15.38:0/2508552429 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f20fc0a2c90 msgr2=0x7f20fc0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.354+0000 7f21164ca640 1 --2- 172.21.15.38:0/2508552429 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f20fc0a2c90 0x7f20fc0a51a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2100009830 tx=0x7f2100030450 comp rx=0 tx=0).stop 2024-05-21T15:37:29.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.354+0000 7f21164ca640 1 -- 172.21.15.38:0/2508552429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21101093d0 msgr2=0x7f21101a8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.354+0000 7f21164ca640 1 --2- 172.21.15.38:0/2508552429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21101093d0 0x7f21101a8100 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f210800b9c0 tx=0x7f210800be80 comp rx=0 tx=0).stop 2024-05-21T15:37:29.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.356+0000 7f21164ca640 1 -- 172.21.15.38:0/2508552429 shutdown_connections 2024-05-21T15:37:29.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.356+0000 7f21164ca640 1 --2- 172.21.15.38:0/2508552429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211010efe0 0x7f21101af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.356+0000 7f21164ca640 1 --2- 172.21.15.38:0/2508552429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211010a2c0 0x7f21101a8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.356+0000 7f21164ca640 1 --2- 172.21.15.38:0/2508552429 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f20fc0a2c90 0x7f20fc0a51a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.356+0000 7f21164ca640 1 --2- 172.21.15.38:0/2508552429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21101093d0 0x7f21101a8100 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.356+0000 7f21164ca640 1 -- 172.21.15.38:0/2508552429 >> 172.21.15.38:0/2508552429 conn(0x7f211006f4a0 msgr2=0x7f211010afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.356+0000 7f21164ca640 1 -- 172.21.15.38:0/2508552429 shutdown_connections 2024-05-21T15:37:29.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.357+0000 7f21164ca640 1 -- 172.21.15.38:0/2508552429 wait complete. 2024-05-21T15:37:29.377 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:37:29.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.594+0000 7f10deb02640 1 -- 172.21.15.38:0/1197448601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d8101980 msgr2=0x7f10d8102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.594+0000 7f10deb02640 1 --2- 172.21.15.38:0/1197448601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d8101980 0x7f10d8102680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009ca0 tx=0x7f10d002a8d0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.595+0000 7f10deb02640 1 -- 172.21.15.38:0/1197448601 shutdown_connections 2024-05-21T15:37:29.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.595+0000 7f10deb02640 1 --2- 172.21.15.38:0/1197448601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10d81078c0 0x7f10d8109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.595+0000 7f10deb02640 1 --2- 172.21.15.38:0/1197448601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d8102bc0 0x7f10d8102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.595+0000 7f10deb02640 1 --2- 172.21.15.38:0/1197448601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d8101980 0x7f10d8102680 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.595+0000 7f10deb02640 1 -- 172.21.15.38:0/1197448601 >> 172.21.15.38:0/1197448601 conn(0x7f10d80fcb70 msgr2=0x7f10d80ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.595+0000 7f10deb02640 1 -- 172.21.15.38:0/1197448601 shutdown_connections 2024-05-21T15:37:29.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.595+0000 7f10deb02640 1 -- 172.21.15.38:0/1197448601 wait complete. 2024-05-21T15:37:29.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.596+0000 7f10deb02640 1 Processor -- start 2024-05-21T15:37:29.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.596+0000 7f10deb02640 1 -- start start 2024-05-21T15:37:29.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.596+0000 7f10deb02640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d8101980 0x7f10d8070990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.596+0000 7f10deb02640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d8102bc0 0x7f10d8070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.596+0000 7f10deb02640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10d81078c0 0x7f10d8071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.596+0000 7f10deb02640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81109c0 con 0x7f10d81078c0 2024-05-21T15:37:29.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.596+0000 7f10deb02640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f10d8110840 con 0x7f10d8101980 2024-05-21T15:37:29.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.596+0000 7f10deb02640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f10d8110b40 con 0x7f10d8102bc0 2024-05-21T15:37:29.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.597+0000 7f10d7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d8102bc0 0x7f10d8070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.597+0000 7f10d7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d8102bc0 0x7f10d8070f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46574/0 (socket says 172.21.15.38:46574) 2024-05-21T15:37:29.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.597+0000 7f10d7fff640 1 -- 172.21.15.38:0/3360961501 learned_addr learned my addr 172.21.15.38:0/3360961501 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:29.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.597+0000 7f10dd09f640 1 --2- 172.21.15.38:0/3360961501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10d81078c0 0x7f10d8071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.597+0000 7f10d7fff640 1 -- 172.21.15.38:0/3360961501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d8101980 msgr2=0x7f10d8070990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:37:29.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.597+0000 7f10d7fff640 1 --2- 172.21.15.38:0/3360961501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d8101980 0x7f10d8070990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.597+0000 7f10d7fff640 1 -- 172.21.15.38:0/3360961501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10d81078c0 msgr2=0x7f10d8071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.597+0000 7f10d7fff640 1 --2- 172.21.15.38:0/3360961501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10d81078c0 0x7f10d8071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.597+0000 7f10d7fff640 1 -- 172.21.15.38:0/3360961501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d8072520 con 0x7f10d8102bc0 2024-05-21T15:37:29.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.598+0000 7f10d7fff640 1 --2- 172.21.15.38:0/3360961501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d8102bc0 0x7f10d8070f10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00c890 tx=0x7f10cc00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.599+0000 7f10d5ffb640 1 -- 172.21.15.38:0/3360961501 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc004220 con 0x7f10d8102bc0 2024-05-21T15:37:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.599+0000 7f10deb02640 1 -- 172.21.15.38:0/3360961501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d8072810 con 0x7f10d8102bc0 2024-05-21T15:37:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.599+0000 7f10deb02640 1 -- 172.21.15.38:0/3360961501 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d8072d60 con 0x7f10d8102bc0 2024-05-21T15:37:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.599+0000 7f10d5ffb640 1 -- 172.21.15.38:0/3360961501 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10cc0043c0 con 0x7f10d8102bc0 2024-05-21T15:37:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.599+0000 7f10d5ffb640 1 -- 172.21.15.38:0/3360961501 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc004b10 con 0x7f10d8102bc0 2024-05-21T15:37:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.603+0000 7f10e59a9740 1 Processor -- start 2024-05-21T15:37:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.604+0000 7f10e59a9740 1 -- start start 2024-05-21T15:37:29.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.607+0000 7f10e59a9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d50d390 0x555f0d8b5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.608+0000 7f10dd09f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d50d390 0x555f0d8b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.608+0000 7f10dd09f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d50d390 0x555f0d8b5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45240/0 (socket says 172.21.15.38:45240) 2024-05-21T15:37:29.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.612+0000 7f10e59a9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4e44d0 0x555f0d8b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.612+0000 7f10e59a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4ec6b0 0x555f0d8bc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.612+0000 7f10e59a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555f0d811c40 con 0x555f0d4ec6b0 2024-05-21T15:37:29.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.612+0000 7f10e59a9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555f0d7b9580 con 0x555f0d50d390 2024-05-21T15:37:29.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.612+0000 7f10e59a9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555f0d80dc00 con 0x555f0d4e44d0 2024-05-21T15:37:29.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.612+0000 7f10dd09f640 1 -- 172.21.15.38:0/3085923256 learned_addr learned my addr 172.21.15.38:0/3085923256 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:29.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.614+0000 7f10d7fff640 1 --2- 172.21.15.38:0/3085923256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4ec6b0 0x555f0d8bc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.614+0000 7f10d5ffb640 1 -- 172.21.15.38:0/3360961501 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f10cc004cb0 con 0x7f10d8102bc0 2024-05-21T15:37:29.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.615+0000 7f10dc89e640 1 --2- 172.21.15.38:0/3085923256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4e44d0 0x555f0d8b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.620+0000 7f10dd09f640 1 -- 172.21.15.38:0/3085923256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4e44d0 msgr2=0x555f0d8b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.620+0000 7f10dd09f640 1 --2- 172.21.15.38:0/3085923256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4e44d0 0x555f0d8b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.620+0000 7f10dd09f640 1 -- 172.21.15.38:0/3085923256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4ec6b0 msgr2=0x555f0d8bc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.620+0000 7f10dd09f640 1 --2- 172.21.15.38:0/3085923256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4ec6b0 0x555f0d8bc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.620+0000 7f10dd09f640 1 -- 172.21.15.38:0/3085923256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f0d5253f0 con 0x555f0d50d390 2024-05-21T15:37:29.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.620+0000 7f10dc89e640 1 --2- 172.21.15.38:0/3085923256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4e44d0 0x555f0d8b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:37:29.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.623+0000 7f10dd09f640 1 --2- 172.21.15.38:0/3085923256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d50d390 0x555f0d8b5310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f10c800b7a0 tx=0x7f10c800bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef5943315d338875 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.623+0000 7f10d5ffb640 1 --2- 172.21.15.38:0/3360961501 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10c00a2dd0 0x7f10c00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.624+0000 7f10d5ffb640 1 -- 172.21.15.38:0/3360961501 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f10cc0cecf0 con 0x7f10d8102bc0 2024-05-21T15:37:29.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.625+0000 7f10dc89e640 1 --2- 172.21.15.38:0/3360961501 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10c00a2dd0 0x7f10c00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.626+0000 7f10beffd640 1 -- 172.21.15.38:0/3085923256 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c800d950 con 0x555f0d50d390 2024-05-21T15:37:29.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.626+0000 7f10beffd640 1 -- 172.21.15.38:0/3085923256 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10c8012420 con 0x555f0d50d390 2024-05-21T15:37:29.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.626+0000 7f10beffd640 1 -- 172.21.15.38:0/3085923256 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8014650 con 0x555f0d50d390 2024-05-21T15:37:29.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.629+0000 7f10e59a9740 1 -- 172.21.15.38:0/3085923256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d50d390 msgr2=0x555f0d8b5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.629+0000 7f10e59a9740 1 --2- 172.21.15.38:0/3085923256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d50d390 0x555f0d8b5310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f10c800b7a0 tx=0x7f10c800bc60 comp rx=0 tx=0).stop 2024-05-21T15:37:29.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.630+0000 7f10e59a9740 1 -- 172.21.15.38:0/3085923256 shutdown_connections 2024-05-21T15:37:29.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.630+0000 7f10e59a9740 1 --2- 172.21.15.38:0/3085923256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4ec6b0 0x555f0d8bc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.630+0000 7f10e59a9740 1 --2- 172.21.15.38:0/3085923256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4e44d0 0x555f0d8b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.630+0000 7f10e59a9740 1 --2- 172.21.15.38:0/3085923256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d50d390 0x555f0d8b5310 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.630+0000 7f10e59a9740 1 -- 172.21.15.38:0/3085923256 >> 172.21.15.38:0/3085923256 conn(0x555f0d5014e0 msgr2=0x555f0d8a8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.634+0000 7f10dc89e640 1 --2- 172.21.15.38:0/3360961501 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10c00a2dd0 0x7f10c00a52e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f10d0006010 tx=0x7f10d0000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.637+0000 7f10e59a9740 1 -- 172.21.15.38:0/3085923256 shutdown_connections 2024-05-21T15:37:29.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.638+0000 7f10e59a9740 1 -- 172.21.15.38:0/3085923256 wait complete. 2024-05-21T15:37:29.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.638+0000 7f10e59a9740 1 -- start start 2024-05-21T15:37:29.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.638+0000 7f10e59a9740 1 Processor -- start 2024-05-21T15:37:29.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.639+0000 7f10e59a9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4e44d0 0x555f0d8acb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.639+0000 7f10e59a9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d4ec6b0 0x555f0d8ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.639+0000 7f10e59a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d50d390 0x555f0d8b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.639+0000 7f10e59a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555f0d8bd1e0 con 0x555f0d50d390 2024-05-21T15:37:29.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.639+0000 7f10e59a9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555f0d7ccc40 con 0x555f0d4ec6b0 2024-05-21T15:37:29.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.639+0000 7f10e59a9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555f0d8bd360 con 0x555f0d4e44d0 2024-05-21T15:37:29.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.639+0000 7f10e59a9740 10 client.0 _ll_register_callbacks cb 0x555f0d524be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:29.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.641+0000 7f10dd09f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d50d390 0x555f0d8b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.642+0000 7f10dd09f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d50d390 0x555f0d8b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40850/0 (socket says 172.21.15.38:40850) 2024-05-21T15:37:29.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.642+0000 7f10dd09f640 1 -- 172.21.15.38:0/3929441426 learned_addr learned my addr 172.21.15.38:0/3929441426 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.643+0000 7f10dd09f640 1 -- 172.21.15.38:0/3929441426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4e44d0 msgr2=0x555f0d8acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.643+0000 7f10dd09f640 1 --2- 172.21.15.38:0/3929441426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4e44d0 0x555f0d8acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.643+0000 7f10dd09f640 1 -- 172.21.15.38:0/3929441426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d4ec6b0 msgr2=0x555f0d8ad080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:37:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.643+0000 7f10dd09f640 1 --2- 172.21.15.38:0/3929441426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d4ec6b0 0x555f0d8ad080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.643+0000 7f10dd09f640 1 -- 172.21.15.38:0/3929441426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f0d5ef7d0 con 0x555f0d50d390 2024-05-21T15:37:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.643+0000 7f10dd09f640 1 --2- 172.21.15.38:0/3929441426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d50d390 0x555f0d8b3fe0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f10c8006f40 tx=0x7f10c800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cabb0f3052ed47c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.648+0000 7f10b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:37:29.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.648+0000 7f10b7fff640 1 -- 172.21.15.38:0/3929441426 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8021070 con 0x555f0d50d390 2024-05-21T15:37:29.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.648+0000 7f10b7fff640 1 -- 172.21.15.38:0/3929441426 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f10c80127d0 con 0x555f0d50d390 2024-05-21T15:37:29.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.648+0000 7f10e59a9740 1 -- 172.21.15.38:0/3929441426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10c8012420 con 0x555f0d50d390 2024-05-21T15:37:29.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.649+0000 7f10b7fff640 1 -- 172.21.15.38:0/3929441426 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f10c802b050 con 0x555f0d50d390 2024-05-21T15:37:29.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.649+0000 7f10e59a9740 10 client.16314 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:29.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.649+0000 7f10e59a9740 1 -- 172.21.15.38:0/3929441426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555f0d8c06d0 con 0x555f0d50d390 2024-05-21T15:37:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.650+0000 7f10b7fff640 1 -- 172.21.15.38:0/3929441426 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10c8012420 con 0x555f0d50d390 2024-05-21T15:37:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.651+0000 7f10b7fff640 1 -- 172.21.15.38:0/3929441426 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f10c8013070 con 0x555f0d50d390 2024-05-21T15:37:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.652+0000 7f10e59a9740 10 client.16314 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.652+0000 7f10e59a9740 10 client.16314 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:37:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.652+0000 7f10e59a9740 20 client.16314 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.652+0000 7f10e59a9740 1 --2- 172.21.15.38:0/3929441426 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x555f0d4cbf60 0x555f0d8c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.652+0000 7f10e59a9740 4 client.16314 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:37:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.652+0000 7f10e59a9740 1 -- 172.21.15.38:0/3929441426 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f0d81f370 con 0x555f0d4cbf60 2024-05-21T15:37:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.653+0000 7f10dc89e640 1 --2- 172.21.15.38:0/3929441426 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x555f0d4cbf60 0x555f0d8c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.655+0000 7f10dc89e640 1 --2- 172.21.15.38:0/3929441426 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x555f0d4cbf60 0x555f0d8c41f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcf659672ef89d70 server_cookie=718618afef66ef91 in_seq=0 out_seq=0 2024-05-21T15:37:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.656+0000 7f10b7fff640 10 client.16314 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:37:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.657+0000 7f10b7fff640 1 -- 172.21.15.38:0/3929441426 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555f0d81f370 con 0x555f0d4cbf60 2024-05-21T15:37:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.657+0000 7f10b7fff640 10 client.16314 handle_command_reply: tid=0 2024-05-21T15:37:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.657+0000 7f10e59a9740 1 client.16314 shutdown 2024-05-21T15:37:29.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.659+0000 7f10e59a9740 1 -- 172.21.15.38:0/3929441426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d50d390 msgr2=0x555f0d8b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.659+0000 7f10e59a9740 1 --2- 172.21.15.38:0/3929441426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d50d390 0x555f0d8b3fe0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f10c8006f40 tx=0x7f10c800db80 comp rx=0 tx=0).stop 2024-05-21T15:37:29.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.660+0000 7f10e59a9740 1 -- 172.21.15.38:0/3929441426 shutdown_connections 2024-05-21T15:37:29.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.660+0000 7f10e59a9740 1 --2- 172.21.15.38:0/3929441426 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x555f0d4cbf60 0x555f0d8c41f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.660+0000 7f10e59a9740 1 --2- 172.21.15.38:0/3929441426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d50d390 0x555f0d8b3fe0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.660+0000 7f10e59a9740 1 --2- 172.21.15.38:0/3929441426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d4ec6b0 0x555f0d8ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.660+0000 7f10e59a9740 1 --2- 172.21.15.38:0/3929441426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4e44d0 0x555f0d8acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.660+0000 7f10e59a9740 1 -- 172.21.15.38:0/3929441426 >> 172.21.15.38:0/3929441426 conn(0x555f0d5014e0 msgr2=0x555f0d8a7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.660+0000 7f10b7fff640 0 client.16314 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:37:29.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.665+0000 7f10e59a9740 1 -- 172.21.15.38:0/3929441426 shutdown_connections 2024-05-21T15:37:29.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.665+0000 7f10e59a9740 1 -- 172.21.15.38:0/3929441426 wait complete. 2024-05-21T15:37:29.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.665+0000 7f10e59a9740 20 client.16314 trim_cache size 0 max 16384 2024-05-21T15:37:29.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.700+0000 7f10e59a9740 1 Processor -- start 2024-05-21T15:37:29.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.700+0000 7f10e59a9740 1 -- start start 2024-05-21T15:37:29.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.701+0000 7f10e59a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4e44d0 0x555f0d8ad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.701+0000 7f10e59a9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d4ec6b0 0x555f0d8b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.701+0000 7f10e59a9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d50d390 0x555f0d8bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.701+0000 7f10e59a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f10c400e4f0 con 0x555f0d4e44d0 2024-05-21T15:37:29.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.701+0000 7f10e59a9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f10c4010120 con 0x555f0d4ec6b0 2024-05-21T15:37:29.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.701+0000 7f10e59a9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f10c400ebc0 con 0x555f0d50d390 2024-05-21T15:37:29.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.701+0000 7f10dc89e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4e44d0 0x555f0d8ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.701+0000 7f10dc89e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4e44d0 0x555f0d8ad250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40858/0 (socket says 172.21.15.38:40858) 2024-05-21T15:37:29.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.701+0000 7f10dc89e640 1 -- 172.21.15.38:0/3590543169 learned_addr learned my addr 172.21.15.38:0/3590543169 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:29.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.701+0000 7f10d7fff640 1 --2- 172.21.15.38:0/3590543169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d4ec6b0 0x555f0d8b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.702+0000 7f10dd09f640 1 --2- 172.21.15.38:0/3590543169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d50d390 0x555f0d8bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.702+0000 7f10dc89e640 1 -- 172.21.15.38:0/3590543169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d50d390 msgr2=0x555f0d8bd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.702+0000 7f10dc89e640 1 --2- 172.21.15.38:0/3590543169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d50d390 0x555f0d8bd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.702+0000 7f10dc89e640 1 -- 172.21.15.38:0/3590543169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d4ec6b0 msgr2=0x555f0d8b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.702+0000 7f10dc89e640 1 --2- 172.21.15.38:0/3590543169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d4ec6b0 0x555f0d8b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.702+0000 7f10dc89e640 1 -- 172.21.15.38:0/3590543169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f0d8ac8e0 con 0x555f0d4e44d0 2024-05-21T15:37:29.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.703+0000 7f10dc89e640 1 --2- 172.21.15.38:0/3590543169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4e44d0 0x555f0d8ad250 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009ca0 tx=0x7f10d003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac80690d92e30489 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.703+0000 7f10be7fc640 1 -- 172.21.15.38:0/3590543169 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d004f530 con 0x555f0d4e44d0 2024-05-21T15:37:29.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.703+0000 7f10be7fc640 1 -- 172.21.15.38:0/3590543169 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d0036070 con 0x555f0d4e44d0 2024-05-21T15:37:29.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.704+0000 7f10be7fc640 1 -- 172.21.15.38:0/3590543169 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d002f620 con 0x555f0d4e44d0 2024-05-21T15:37:29.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.706+0000 7f10e59a9740 1 -- 172.21.15.38:0/3590543169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4e44d0 msgr2=0x555f0d8ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.706+0000 7f10e59a9740 1 --2- 172.21.15.38:0/3590543169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4e44d0 0x555f0d8ad250 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009ca0 tx=0x7f10d003e000 comp rx=0 tx=0).stop 2024-05-21T15:37:29.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.706+0000 7f10e59a9740 1 -- 172.21.15.38:0/3590543169 shutdown_connections 2024-05-21T15:37:29.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.706+0000 7f10e59a9740 1 --2- 172.21.15.38:0/3590543169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d50d390 0x555f0d8bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.706+0000 7f10e59a9740 1 --2- 172.21.15.38:0/3590543169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d4ec6b0 0x555f0d8b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.706+0000 7f10e59a9740 1 --2- 172.21.15.38:0/3590543169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4e44d0 0x555f0d8ad250 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.706+0000 7f10e59a9740 1 -- 172.21.15.38:0/3590543169 >> 172.21.15.38:0/3590543169 conn(0x555f0d5014e0 msgr2=0x555f0d8acb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.710+0000 7f10e59a9740 1 -- 172.21.15.38:0/3590543169 shutdown_connections 2024-05-21T15:37:29.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.710+0000 7f10e59a9740 1 -- 172.21.15.38:0/3590543169 wait complete. 2024-05-21T15:37:29.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.710+0000 7f10e59a9740 1 -- start start 2024-05-21T15:37:29.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.710+0000 7f10e59a9740 1 Processor -- start 2024-05-21T15:37:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.711+0000 7f10e59a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4e44d0 0x555f0d8b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.711+0000 7f10e59a9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4ec6b0 0x555f0d8b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.712+0000 7f10e59a9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d50d390 0x555f0d94c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.712+0000 7f10d7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4ec6b0 0x555f0d8b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.712+0000 7f10d7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4ec6b0 0x555f0d8b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46612/0 (socket says 172.21.15.38:46612) 2024-05-21T15:37:29.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.712+0000 7f10d7fff640 1 -- 172.21.15.38:0/1574837401 learned_addr learned my addr 172.21.15.38:0/1574837401 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:29.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.712+0000 7f10dd09f640 1 --2- 172.21.15.38:0/1574837401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d50d390 0x555f0d94c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.712+0000 7f10e59a9740 1 -- 172.21.15.38:0/1574837401 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f10c400f990 con 0x555f0d4e44d0 2024-05-21T15:37:29.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.712+0000 7f10e59a9740 1 -- 172.21.15.38:0/1574837401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f10c400f290 con 0x555f0d50d390 2024-05-21T15:37:29.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.712+0000 7f10e59a9740 1 -- 172.21.15.38:0/1574837401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f10c400c580 con 0x555f0d4ec6b0 2024-05-21T15:37:29.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.713+0000 7f10e59a9740 10 client.0 _ll_register_callbacks cb 0x7f10cc0d2640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:29.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.713+0000 7f10d7fff640 1 -- 172.21.15.38:0/1574837401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d50d390 msgr2=0x555f0d94c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.713+0000 7f10d7fff640 1 --2- 172.21.15.38:0/1574837401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d50d390 0x555f0d94c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.713+0000 7f10d7fff640 1 -- 172.21.15.38:0/1574837401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4e44d0 msgr2=0x555f0d8b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.713+0000 7f10d7fff640 1 --2- 172.21.15.38:0/1574837401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4e44d0 0x555f0d8b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.713+0000 7f10d7fff640 1 -- 172.21.15.38:0/1574837401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f0d8b41e0 con 0x555f0d4ec6b0 2024-05-21T15:37:29.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.713+0000 7f10d7fff640 1 --2- 172.21.15.38:0/1574837401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4ec6b0 0x555f0d8b1220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f10cc007590 tx=0x7f10cc0d2360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd862f1fb2ea72f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.714+0000 7f10e59a9740 1 -- 172.21.15.38:0/1574837401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d0036070 con 0x555f0d4ec6b0 2024-05-21T15:37:29.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.714+0000 7f10b7fff640 10 client.26023 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:37:29.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.714+0000 7f10b7fff640 1 -- 172.21.15.38:0/1574837401 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc01d070 con 0x555f0d4ec6b0 2024-05-21T15:37:29.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.715+0000 7f10b7fff640 1 -- 172.21.15.38:0/1574837401 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f10cc004fa0 con 0x555f0d4ec6b0 2024-05-21T15:37:29.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.715+0000 7f10b7fff640 1 -- 172.21.15.38:0/1574837401 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f10cc09afe0 con 0x555f0d4ec6b0 2024-05-21T15:37:29.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.718+0000 7f10e59a9740 10 client.26023 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:29.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.718+0000 7f10e59a9740 1 -- 172.21.15.38:0/1574837401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=31}) -- 0x555f0d94dd60 con 0x555f0d4ec6b0 2024-05-21T15:37:29.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.719+0000 7f10b7fff640 1 -- 172.21.15.38:0/1574837401 <== mon.1 v2:172.21.15.53:3300/0 4 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f10cc0b4230 con 0x555f0d4ec6b0 2024-05-21T15:37:29.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.719+0000 7f10e59a9740 10 client.26023 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:29.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.719+0000 7f10e59a9740 10 client.26023 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:37:29.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.719+0000 7f10e59a9740 20 client.26023 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:29.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.719+0000 7f10e59a9740 1 --2- 172.21.15.38:0/1574837401 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x555f0d4a13a0 0x555f0d9506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.719+0000 7f10e59a9740 4 client.26023 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:37:29.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.719+0000 7f10e59a9740 1 -- 172.21.15.38:0/1574837401 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f10d0036070 con 0x555f0d4a13a0 2024-05-21T15:37:29.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.721+0000 7f10dc89e640 1 --2- 172.21.15.38:0/1574837401 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x555f0d4a13a0 0x555f0d9506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.722+0000 7f10dc89e640 1 --2- 172.21.15.38:0/1574837401 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x555f0d4a13a0 0x555f0d9506b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d4b29298dcad441 server_cookie=80a3d6d4864d266c in_seq=0 out_seq=0 2024-05-21T15:37:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.722+0000 7f10b7fff640 10 client.26023 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:37:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.723+0000 7f10b7fff640 1 -- 172.21.15.38:0/1574837401 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f10d0036070 con 0x555f0d4a13a0 2024-05-21T15:37:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.723+0000 7f10b7fff640 10 client.26023 handle_command_reply: tid=0 2024-05-21T15:37:29.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.724+0000 7f10e59a9740 1 client.26023 shutdown 2024-05-21T15:37:29.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.729+0000 7f10e59a9740 1 -- 172.21.15.38:0/1574837401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4ec6b0 msgr2=0x555f0d8b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.729+0000 7f10e59a9740 1 --2- 172.21.15.38:0/1574837401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4ec6b0 0x555f0d8b1220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f10cc007590 tx=0x7f10cc0d2360 comp rx=0 tx=0).stop 2024-05-21T15:37:29.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.732+0000 7f10e59a9740 1 -- 172.21.15.38:0/1574837401 shutdown_connections 2024-05-21T15:37:29.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.732+0000 7f10e59a9740 1 --2- 172.21.15.38:0/1574837401 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x555f0d4a13a0 0x555f0d9506b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.732+0000 7f10e59a9740 1 --2- 172.21.15.38:0/1574837401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555f0d50d390 0x555f0d94c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.732+0000 7f10e59a9740 1 --2- 172.21.15.38:0/1574837401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f0d4ec6b0 0x555f0d8b1220 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.732+0000 7f10b7fff640 0 client.26023 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:37:29.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.732+0000 7f10e59a9740 1 --2- 172.21.15.38:0/1574837401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555f0d4e44d0 0x555f0d8b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.732+0000 7f10e59a9740 1 -- 172.21.15.38:0/1574837401 >> 172.21.15.38:0/1574837401 conn(0x555f0d5014e0 msgr2=0x555f0d8bd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.735+0000 7f10e59a9740 1 -- 172.21.15.38:0/1574837401 shutdown_connections 2024-05-21T15:37:29.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.735+0000 7f10e59a9740 1 -- 172.21.15.38:0/1574837401 wait complete. 2024-05-21T15:37:29.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.735+0000 7f10e59a9740 20 client.26023 trim_cache size 0 max 16384 2024-05-21T15:37:29.739 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:37:29.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.744+0000 7f10deb02640 1 -- 172.21.15.38:0/3360961501 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10c00a2dd0 msgr2=0x7f10c00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.744+0000 7f10deb02640 1 --2- 172.21.15.38:0/3360961501 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10c00a2dd0 0x7f10c00a52e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f10d0006010 tx=0x7f10d0000f40 comp rx=0 tx=0).stop 2024-05-21T15:37:29.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.744+0000 7f10deb02640 1 -- 172.21.15.38:0/3360961501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d8102bc0 msgr2=0x7f10d8070f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.744+0000 7f10deb02640 1 --2- 172.21.15.38:0/3360961501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d8102bc0 0x7f10d8070f10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00c890 tx=0x7f10cc00cd50 comp rx=0 tx=0).stop 2024-05-21T15:37:29.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.744+0000 7f10deb02640 1 -- 172.21.15.38:0/3360961501 shutdown_connections 2024-05-21T15:37:29.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.744+0000 7f10deb02640 1 --2- 172.21.15.38:0/3360961501 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f10c00a2dd0 0x7f10c00a52e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.744+0000 7f10deb02640 1 --2- 172.21.15.38:0/3360961501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10d81078c0 0x7f10d8071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.744+0000 7f10deb02640 1 --2- 172.21.15.38:0/3360961501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10d8102bc0 0x7f10d8070f10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.744+0000 7f10deb02640 1 --2- 172.21.15.38:0/3360961501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d8101980 0x7f10d8070990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.744+0000 7f10deb02640 1 -- 172.21.15.38:0/3360961501 >> 172.21.15.38:0/3360961501 conn(0x7f10d80fcb70 msgr2=0x7f10d81038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.745+0000 7f10deb02640 1 -- 172.21.15.38:0/3360961501 shutdown_connections 2024-05-21T15:37:29.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.745+0000 7f10deb02640 1 -- 172.21.15.38:0/3360961501 wait complete. 2024-05-21T15:37:29.764 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:37:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.970+0000 7f1ced59a640 1 -- 172.21.15.38:0/3654714595 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce80307c0 con 0x7f1cf01093d0 2024-05-21T15:37:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.970+0000 7f1cf4ae9640 1 -- 172.21.15.38:0/3654714595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf01093d0 msgr2=0x7f1cf0109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.970+0000 7f1cf4ae9640 1 --2- 172.21.15.38:0/3654714595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf01093d0 0x7f1cf0109d80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009ca0 tx=0x7f1ce802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.971+0000 7f1cf4ae9640 1 -- 172.21.15.38:0/3654714595 shutdown_connections 2024-05-21T15:37:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.971+0000 7f1cf4ae9640 1 --2- 172.21.15.38:0/3654714595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf010efe0 0x7f1cf010f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.971+0000 7f1cf4ae9640 1 --2- 172.21.15.38:0/3654714595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf010a2c0 0x7f1cf010a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.971+0000 7f1cf4ae9640 1 --2- 172.21.15.38:0/3654714595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf01093d0 0x7f1cf0109d80 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.971+0000 7f1cf4ae9640 1 -- 172.21.15.38:0/3654714595 >> 172.21.15.38:0/3654714595 conn(0x7f1cf006f4a0 msgr2=0x7f1cf0071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.971+0000 7f1cf4ae9640 1 -- 172.21.15.38:0/3654714595 shutdown_connections 2024-05-21T15:37:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.971+0000 7f1cf4ae9640 1 -- 172.21.15.38:0/3654714595 wait complete. 2024-05-21T15:37:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.972+0000 7f1cf4ae9640 1 Processor -- start 2024-05-21T15:37:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.972+0000 7f1cf4ae9640 1 -- start start 2024-05-21T15:37:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.972+0000 7f1cf4ae9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf01093d0 0x7f1cf011a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.972+0000 7f1cf4ae9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf010a2c0 0x7f1cf011ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.972+0000 7f1cf4ae9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf010efe0 0x7f1cf0118c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.972+0000 7f1cf4ae9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf0111040 con 0x7f1cf010a2c0 2024-05-21T15:37:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.972+0000 7f1cf4ae9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf0110ec0 con 0x7f1cf010efe0 2024-05-21T15:37:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.972+0000 7f1cf4ae9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01111c0 con 0x7f1cf01093d0 2024-05-21T15:37:29.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.973+0000 7f1ceed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf010efe0 0x7f1cf0118c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.973+0000 7f1ceed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf010efe0 0x7f1cf0118c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45274/0 (socket says 172.21.15.38:45274) 2024-05-21T15:37:29.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.973+0000 7f1ceed9d640 1 -- 172.21.15.38:0/1662596516 learned_addr learned my addr 172.21.15.38:0/1662596516 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:29.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.973+0000 7f1cee59c640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf01093d0 0x7f1cf011a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.974+0000 7f1cee59c640 1 -- 172.21.15.38:0/1662596516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf010efe0 msgr2=0x7f1cf0118c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.974+0000 7f1cee59c640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf010efe0 0x7f1cf0118c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.974+0000 7f1cee59c640 1 -- 172.21.15.38:0/1662596516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf010a2c0 msgr2=0x7f1cf011ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.975+0000 7f1cedd9b640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf010a2c0 0x7f1cf011ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.975+0000 7f1cee59c640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf010a2c0 0x7f1cf011ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.975+0000 7f1cee59c640 1 -- 172.21.15.38:0/1662596516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf0119300 con 0x7f1cf01093d0 2024-05-21T15:37:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.975+0000 7f1cee59c640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf01093d0 0x7f1cf011a5c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0108c10 tx=0x7f1ce8003f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.975+0000 7f1cedd9b640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf010a2c0 0x7f1cf011ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:37:29.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.978+0000 7f1cdf7fe640 1 -- 172.21.15.38:0/1662596516 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8004450 con 0x7f1cf01093d0 2024-05-21T15:37:29.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.978+0000 7f1cdf7fe640 1 -- 172.21.15.38:0/1662596516 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce8040070 con 0x7f1cf01093d0 2024-05-21T15:37:29.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.978+0000 7f1cdf7fe640 1 -- 172.21.15.38:0/1662596516 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8030ec0 con 0x7f1cf01093d0 2024-05-21T15:37:29.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.979+0000 7f1cf4ae9640 1 -- 172.21.15.38:0/1662596516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf0119590 con 0x7f1cf01093d0 2024-05-21T15:37:29.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.979+0000 7f1cf4ae9640 1 -- 172.21.15.38:0/1662596516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf0119a90 con 0x7f1cf01093d0 2024-05-21T15:37:29.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.984+0000 7f1cfb990740 1 Processor -- start 2024-05-21T15:37:29.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.984+0000 7f1cfb990740 1 -- start start 2024-05-21T15:37:29.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.985+0000 7f1cfb990740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 0x557e6e076b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.985+0000 7f1cfb990740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e077080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.985+0000 7f1cfb990740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 0x557e6e07df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.985+0000 7f1cfb990740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557e6dfdbc40 con 0x557e6dcb66b0 2024-05-21T15:37:29.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.985+0000 7f1cfb990740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557e6df83580 con 0x557e6dcae4d0 2024-05-21T15:37:29.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.985+0000 7f1cfb990740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557e6dfd7c00 con 0x557e6dcd7390 2024-05-21T15:37:29.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.985+0000 7f1cedd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 0x557e6e07df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.986+0000 7f1cedd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 0x557e6e07df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40884/0 (socket says 172.21.15.38:40884) 2024-05-21T15:37:29.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.986+0000 7f1cedd9b640 1 -- 172.21.15.38:0/4153943013 learned_addr learned my addr 172.21.15.38:0/4153943013 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:29.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.986+0000 7f1ceed9d640 1 --2- 172.21.15.38:0/4153943013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 0x557e6e076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.986+0000 7f1cedd9b640 1 -- 172.21.15.38:0/4153943013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 msgr2=0x557e6e076b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.986+0000 7f1cedd9b640 1 --2- 172.21.15.38:0/4153943013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 0x557e6e076b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.986+0000 7f1cedd9b640 1 -- 172.21.15.38:0/4153943013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 msgr2=0x557e6e077080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:37:29.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.986+0000 7f1cedd9b640 1 --2- 172.21.15.38:0/4153943013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e077080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.986+0000 7f1cedd9b640 1 -- 172.21.15.38:0/4153943013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6dcef3f0 con 0x557e6dcb66b0 2024-05-21T15:37:29.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.986+0000 7f1ceed9d640 1 --2- 172.21.15.38:0/4153943013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 0x557e6e076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:37:29.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.987+0000 7f1cdf7fe640 1 -- 172.21.15.38:0/1662596516 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1ce80305c0 con 0x7f1cf01093d0 2024-05-21T15:37:29.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.988+0000 7f1cedd9b640 1 --2- 172.21.15.38:0/4153943013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 0x557e6e07df80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009c20 tx=0x7f1ce4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baaa7ac7323740c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.988+0000 7f1cdcff9640 1 -- 172.21.15.38:0/4153943013 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4004030 con 0x557e6dcb66b0 2024-05-21T15:37:29.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.988+0000 7f1cdcff9640 1 -- 172.21.15.38:0/4153943013 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce40041d0 con 0x557e6dcb66b0 2024-05-21T15:37:29.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.989+0000 7f1cdcff9640 1 -- 172.21.15.38:0/4153943013 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4012670 con 0x557e6dcb66b0 2024-05-21T15:37:29.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.989+0000 7f1cfb990740 1 -- 172.21.15.38:0/4153943013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 msgr2=0x557e6e07df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:29.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.989+0000 7f1cfb990740 1 --2- 172.21.15.38:0/4153943013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 0x557e6e07df80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009c20 tx=0x7f1ce4010740 comp rx=0 tx=0).stop 2024-05-21T15:37:29.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.990+0000 7f1cdf7fe640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1cd80a2f00 0x7f1cd80a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.990+0000 7f1cdf7fe640 1 -- 172.21.15.38:0/1662596516 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f1ce80f05a0 con 0x7f1cf01093d0 2024-05-21T15:37:29.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.992+0000 7f1ceed9d640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1cd80a2f00 0x7f1cd80a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.992+0000 7f1cfb990740 1 -- 172.21.15.38:0/4153943013 shutdown_connections 2024-05-21T15:37:29.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.992+0000 7f1cfb990740 1 --2- 172.21.15.38:0/4153943013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 0x557e6e07df80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.992+0000 7f1cfb990740 1 --2- 172.21.15.38:0/4153943013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e077080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.992+0000 7f1cfb990740 1 --2- 172.21.15.38:0/4153943013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 0x557e6e076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.992+0000 7f1cfb990740 1 -- 172.21.15.38:0/4153943013 >> 172.21.15.38:0/4153943013 conn(0x557e6dccb4e0 msgr2=0x557e6e072490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:29.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.993+0000 7f1cfb990740 1 -- 172.21.15.38:0/4153943013 shutdown_connections 2024-05-21T15:37:29.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.993+0000 7f1cfb990740 1 -- 172.21.15.38:0/4153943013 wait complete. 2024-05-21T15:37:29.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.993+0000 7f1cfb990740 1 -- start start 2024-05-21T15:37:29.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.994+0000 7f1cfb990740 1 Processor -- start 2024-05-21T15:37:29.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.994+0000 7f1cfb990740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcae4d0 0x557e6e080a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.994+0000 7f1cfb990740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcb66b0 0x557e6e080fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.994+0000 7f1cfb990740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcd7390 0x557e6e087f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:29.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.995+0000 7f1cfb990740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557e6e07ea20 con 0x557e6dcae4d0 2024-05-21T15:37:29.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.995+0000 7f1cfb990740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557e6df96c40 con 0x557e6dcd7390 2024-05-21T15:37:29.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.995+0000 7f1cfb990740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557e6e07eba0 con 0x557e6dcb66b0 2024-05-21T15:37:29.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.995+0000 7f1cfb990740 10 client.0 _ll_register_callbacks cb 0x557e6dceebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:29.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.996+0000 7f1cedd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcb66b0 0x557e6e080fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.996+0000 7f1cee59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcae4d0 0x557e6e080a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.996+0000 7f1cee59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcae4d0 0x557e6e080a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40892/0 (socket says 172.21.15.38:40892) 2024-05-21T15:37:29.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.996+0000 7f1cee59c640 1 -- 172.21.15.38:0/920056818 learned_addr learned my addr 172.21.15.38:0/920056818 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:29.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.996+0000 7f1ceed9d640 1 --2- 172.21.15.38:0/920056818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcd7390 0x557e6e087f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:29.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.997+0000 7f1ceed9d640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1cd80a2f00 0x7f1cd80a5410 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009aa0 tx=0x7f1ce0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.997+0000 7f1cee59c640 1 -- 172.21.15.38:0/920056818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcb66b0 msgr2=0x557e6e080fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.997+0000 7f1cee59c640 1 --2- 172.21.15.38:0/920056818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcb66b0 0x557e6e080fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.997+0000 7f1cee59c640 1 -- 172.21.15.38:0/920056818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcd7390 msgr2=0x557e6e087f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:29.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.997+0000 7f1cee59c640 1 --2- 172.21.15.38:0/920056818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcd7390 0x557e6e087f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:29.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.998+0000 7f1cee59c640 1 -- 172.21.15.38:0/920056818 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6ddb97d0 con 0x557e6dcae4d0 2024-05-21T15:37:29.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.998+0000 7f1cedd9b640 1 --2- 172.21.15.38:0/920056818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcb66b0 0x557e6e080fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:37:29.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.998+0000 7f1cee59c640 1 --2- 172.21.15.38:0/920056818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcae4d0 0x557e6e080a80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80c5490 tx=0x7f1ce8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c434f85bd65a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:29.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.998+0000 7f1cd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:37:29.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.999+0000 7f1cd5ffb640 1 -- 172.21.15.38:0/920056818 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce80f0310 con 0x557e6dcae4d0 2024-05-21T15:37:29.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:29.999+0000 7f1cd5ffb640 1 -- 172.21.15.38:0/920056818 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f1ce80c5530 con 0x557e6dcae4d0 2024-05-21T15:37:29.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.000+0000 7f1cfb990740 1 -- 172.21.15.38:0/920056818 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce40041d0 con 0x557e6dcae4d0 2024-05-21T15:37:29.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.000+0000 7f1cd5ffb640 1 -- 172.21.15.38:0/920056818 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1ce80da050 con 0x557e6dcae4d0 2024-05-21T15:37:29.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.000+0000 7f1cfb990740 10 client.16326 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:29.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.000+0000 7f1cfb990740 1 -- 172.21.15.38:0/920056818 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557e6e07ea20 con 0x557e6dcae4d0 2024-05-21T15:37:30.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.002+0000 7f1cd5ffb640 1 -- 172.21.15.38:0/920056818 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce40041d0 con 0x557e6dcae4d0 2024-05-21T15:37:30.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.002+0000 7f1cd5ffb640 1 -- 172.21.15.38:0/920056818 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f1ce804c020 con 0x557e6dcae4d0 2024-05-21T15:37:30.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.003+0000 7f1cfb990740 10 client.16326 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:30.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.003+0000 7f1cfb990740 10 client.16326 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:37:30.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.003+0000 7f1cfb990740 20 client.16326 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:30.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.003+0000 7f1cfb990740 1 --2- 172.21.15.38:0/920056818 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x557e6dc95f60 0x557e6e08e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.003+0000 7f1cfb990740 4 client.16326 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:37:30.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.003+0000 7f1cfb990740 1 -- 172.21.15.38:0/920056818 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e6dfe9370 con 0x557e6dc95f60 2024-05-21T15:37:30.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.006+0000 7f1cedd9b640 1 --2- 172.21.15.38:0/920056818 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x557e6dc95f60 0x557e6e08e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.010+0000 7f1cedd9b640 1 --2- 172.21.15.38:0/920056818 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x557e6dc95f60 0x557e6e08e9a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5d9d6fc69ff1865 server_cookie=aa2899a8f660a195 in_seq=0 out_seq=0 2024-05-21T15:37:30.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.010+0000 7f1cd5ffb640 10 client.16326 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:37:30.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.011+0000 7f1cd5ffb640 1 -- 172.21.15.38:0/920056818 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e6dfe9370 con 0x557e6dc95f60 2024-05-21T15:37:30.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.011+0000 7f1cd5ffb640 10 client.16326 handle_command_reply: tid=0 2024-05-21T15:37:30.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.012+0000 7f1cfb990740 1 client.16326 shutdown 2024-05-21T15:37:30.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.023+0000 7f1cfb990740 1 -- 172.21.15.38:0/920056818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcae4d0 msgr2=0x557e6e080a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:30.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.023+0000 7f1cfb990740 1 --2- 172.21.15.38:0/920056818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcae4d0 0x557e6e080a80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80c5490 tx=0x7f1ce8003f90 comp rx=0 tx=0).stop 2024-05-21T15:37:30.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.023+0000 7f1cfb990740 1 -- 172.21.15.38:0/920056818 shutdown_connections 2024-05-21T15:37:30.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.023+0000 7f1cfb990740 1 --2- 172.21.15.38:0/920056818 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x557e6dc95f60 0x557e6e08e9a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.023+0000 7f1cfb990740 1 --2- 172.21.15.38:0/920056818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcd7390 0x557e6e087f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.023+0000 7f1cfb990740 1 --2- 172.21.15.38:0/920056818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcb66b0 0x557e6e080fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.023+0000 7f1cfb990740 1 --2- 172.21.15.38:0/920056818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcae4d0 0x557e6e080a80 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.023+0000 7f1cfb990740 1 -- 172.21.15.38:0/920056818 >> 172.21.15.38:0/920056818 conn(0x557e6dccb4e0 msgr2=0x557e6e071570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:30.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.024+0000 7f1cd5ffb640 0 client.16326 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:37:30.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.024+0000 7f1cfb990740 1 -- 172.21.15.38:0/920056818 shutdown_connections 2024-05-21T15:37:30.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:29 smithi038 ceph-mon[31995]: pgmap v443: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 203 MiB/s wr, 21.93k op/s 2024-05-21T15:37:30.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.026+0000 7f1cfb990740 1 -- 172.21.15.38:0/920056818 wait complete. 2024-05-21T15:37:30.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.027+0000 7f1cfb990740 20 client.16326 trim_cache size 0 max 16384 2024-05-21T15:37:30.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.056+0000 7f1cfb990740 1 Processor -- start 2024-05-21T15:37:30.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.058+0000 7f1cfb990740 1 -- start start 2024-05-21T15:37:30.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.059+0000 7f1cfb990740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e0723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.059+0000 7f1cfb990740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 0x557e6e087eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.059+0000 7f1cee59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e0723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.059+0000 7f1cee59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e0723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45294/0 (socket says 172.21.15.38:45294) 2024-05-21T15:37:30.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.059+0000 7f1cee59c640 1 -- 172.21.15.38:0/1389468578 learned_addr learned my addr 172.21.15.38:0/1389468578 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:30.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.059+0000 7f1cfb990740 1 --2- 172.21.15.38:0/1389468578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 0x557e6e080650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.060+0000 7f1cfb990740 1 -- 172.21.15.38:0/1389468578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01c19f0 con 0x557e6dcb66b0 2024-05-21T15:37:30.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.060+0000 7f1cfb990740 1 -- 172.21.15.38:0/1389468578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01c3620 con 0x557e6dcae4d0 2024-05-21T15:37:30.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.060+0000 7f1cfb990740 1 -- 172.21.15.38:0/1389468578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01c20c0 con 0x557e6dcd7390 2024-05-21T15:37:30.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.060+0000 7f1ceed9d640 1 --2- 172.21.15.38:0/1389468578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 0x557e6e080650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.061+0000 7f1cee59c640 1 -- 172.21.15.38:0/1389468578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 msgr2=0x557e6e080650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:30.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.061+0000 7f1cee59c640 1 --2- 172.21.15.38:0/1389468578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 0x557e6e080650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.061+0000 7f1cee59c640 1 -- 172.21.15.38:0/1389468578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 msgr2=0x557e6e087eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:30.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.061+0000 7f1cee59c640 1 --2- 172.21.15.38:0/1389468578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 0x557e6e087eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.061+0000 7f1cee59c640 1 -- 172.21.15.38:0/1389468578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6e081af0 con 0x557e6dcae4d0 2024-05-21T15:37:30.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.062+0000 7f1cee59c640 1 --2- 172.21.15.38:0/1389468578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e0723b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80b9690 tx=0x7f1ce8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=550a21824414d50d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:30.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.062+0000 7f1cd7fff640 1 -- 172.21.15.38:0/1389468578 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce80bd450 con 0x557e6dcae4d0 2024-05-21T15:37:30.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.062+0000 7f1cd7fff640 1 -- 172.21.15.38:0/1389468578 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce8036030 con 0x557e6dcae4d0 2024-05-21T15:37:30.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.063+0000 7f1cfb990740 1 -- 172.21.15.38:0/1389468578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 msgr2=0x557e6e0723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:30.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.063+0000 7f1cfb990740 1 --2- 172.21.15.38:0/1389468578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e0723b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80b9690 tx=0x7f1ce8005fd0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.065+0000 7f1cfb990740 1 -- 172.21.15.38:0/1389468578 shutdown_connections 2024-05-21T15:37:30.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.065+0000 7f1cfb990740 1 --2- 172.21.15.38:0/1389468578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 0x557e6e080650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.065+0000 7f1cfb990740 1 --2- 172.21.15.38:0/1389468578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 0x557e6e087eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.065+0000 7f1cfb990740 1 --2- 172.21.15.38:0/1389468578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e0723b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.065+0000 7f1cfb990740 1 -- 172.21.15.38:0/1389468578 >> 172.21.15.38:0/1389468578 conn(0x557e6dccb4e0 msgr2=0x557e6e071ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:30.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.067+0000 7f1cfb990740 1 -- 172.21.15.38:0/1389468578 shutdown_connections 2024-05-21T15:37:30.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.067+0000 7f1cfb990740 1 -- 172.21.15.38:0/1389468578 wait complete. 2024-05-21T15:37:30.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.068+0000 7f1cfb990740 1 -- start start 2024-05-21T15:37:30.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.068+0000 7f1cfb990740 1 Processor -- start 2024-05-21T15:37:30.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.069+0000 7f1cfb990740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e084d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.069+0000 7f1cfb990740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 0x557e6e0852a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.069+0000 7f1cee59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e084d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.069+0000 7f1cee59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e084d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45306/0 (socket says 172.21.15.38:45306) 2024-05-21T15:37:30.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.069+0000 7f1cee59c640 1 -- 172.21.15.38:0/55690105 learned_addr learned my addr 172.21.15.38:0/55690105 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:30.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.069+0000 7f1cfb990740 1 --2- 172.21.15.38:0/55690105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 0x557e6e0f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.070+0000 7f1cfb990740 1 -- 172.21.15.38:0/55690105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01c2e90 con 0x557e6dcb66b0 2024-05-21T15:37:30.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.070+0000 7f1cfb990740 1 -- 172.21.15.38:0/55690105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01c2790 con 0x557e6dcae4d0 2024-05-21T15:37:30.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.070+0000 7f1cfb990740 1 -- 172.21.15.38:0/55690105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01bf9d0 con 0x557e6dcd7390 2024-05-21T15:37:30.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.070+0000 7f1ceed9d640 1 --2- 172.21.15.38:0/55690105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 0x557e6e0f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.070+0000 7f1cfb990740 10 client.0 _ll_register_callbacks cb 0x7f1ce8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:30.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.071+0000 7f1cee59c640 1 -- 172.21.15.38:0/55690105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 msgr2=0x557e6e0f4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:30.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.071+0000 7f1cee59c640 1 --2- 172.21.15.38:0/55690105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 0x557e6e0f4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.071+0000 7f1cee59c640 1 -- 172.21.15.38:0/55690105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 msgr2=0x557e6e0852a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:30.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.071+0000 7f1cee59c640 1 --2- 172.21.15.38:0/55690105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 0x557e6e0852a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.071+0000 7f1cee59c640 1 -- 172.21.15.38:0/55690105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6e085910 con 0x557e6dcae4d0 2024-05-21T15:37:30.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.071+0000 7f1cfb990740 1 -- 172.21.15.38:0/55690105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557e6dfe36c0 con 0x557e6dcae4d0 2024-05-21T15:37:30.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.072+0000 7f1cee59c640 1 --2- 172.21.15.38:0/55690105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e084d60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8031dc0 tx=0x7f1ce80b8ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b37c633a88be63b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:30.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.072+0000 7f1cd5ffb640 10 client.26000 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:37:30.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.072+0000 7f1cd5ffb640 1 -- 172.21.15.38:0/55690105 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8004d40 con 0x557e6dcae4d0 2024-05-21T15:37:30.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.072+0000 7f1cd5ffb640 1 -- 172.21.15.38:0/55690105 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f1ce80c5530 con 0x557e6dcae4d0 2024-05-21T15:37:30.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.072+0000 7f1cd5ffb640 1 -- 172.21.15.38:0/55690105 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1ce80bd450 con 0x557e6dcae4d0 2024-05-21T15:37:30.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.072+0000 7f1cfb990740 10 client.26000 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:30.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.073+0000 7f1cfb990740 1 -- 172.21.15.38:0/55690105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=31}) -- 0x557e6e0f6250 con 0x557e6dcae4d0 2024-05-21T15:37:30.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.073+0000 7f1cd5ffb640 1 -- 172.21.15.38:0/55690105 <== mon.2 v2:172.21.15.45:3300/0 4 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f1ce804c070 con 0x557e6dcae4d0 2024-05-21T15:37:30.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.073+0000 7f1cfb990740 10 client.26000 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:30.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.073+0000 7f1cfb990740 10 client.26000 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:37:30.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.073+0000 7f1cfb990740 20 client.26000 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:30.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.074+0000 7f1cfb990740 1 --2- 172.21.15.38:0/55690105 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x557e6dc6b3a0 0x557e6e0f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.074+0000 7f1cfb990740 4 client.26000 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:37:30.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.074+0000 7f1cfb990740 1 -- 172.21.15.38:0/55690105 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e6df60820 con 0x557e6dc6b3a0 2024-05-21T15:37:30.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.074+0000 7f1cedd9b640 1 --2- 172.21.15.38:0/55690105 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x557e6dc6b3a0 0x557e6e0f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.076+0000 7f1cedd9b640 1 --2- 172.21.15.38:0/55690105 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x557e6dc6b3a0 0x557e6e0f8aa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b289e2622accebc server_cookie=3ee81eb8f6d2504d in_seq=0 out_seq=0 2024-05-21T15:37:30.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.076+0000 7f1cd5ffb640 10 client.26000 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:37:30.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.076+0000 7f1cd5ffb640 1 -- 172.21.15.38:0/55690105 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e6df60820 con 0x557e6dc6b3a0 2024-05-21T15:37:30.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.076+0000 7f1cd5ffb640 10 client.26000 handle_command_reply: tid=0 2024-05-21T15:37:30.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.077+0000 7f1cfb990740 1 client.26000 shutdown 2024-05-21T15:37:30.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.078+0000 7f1cfb990740 1 -- 172.21.15.38:0/55690105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 msgr2=0x557e6e084d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:30.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.078+0000 7f1cfb990740 1 --2- 172.21.15.38:0/55690105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e084d60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8031dc0 tx=0x7f1ce80b8ca0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.078+0000 7f1cfb990740 1 -- 172.21.15.38:0/55690105 shutdown_connections 2024-05-21T15:37:30.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.078+0000 7f1cfb990740 1 --2- 172.21.15.38:0/55690105 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x557e6dc6b3a0 0x557e6e0f8aa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.078+0000 7f1cfb990740 1 --2- 172.21.15.38:0/55690105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e6dcd7390 0x557e6e0f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.078+0000 7f1cfb990740 1 --2- 172.21.15.38:0/55690105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557e6dcb66b0 0x557e6e0852a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.078+0000 7f1cfb990740 1 --2- 172.21.15.38:0/55690105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557e6dcae4d0 0x557e6e084d60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.078+0000 7f1cfb990740 1 -- 172.21.15.38:0/55690105 >> 172.21.15.38:0/55690105 conn(0x557e6dccb4e0 msgr2=0x557e6e080630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.078+0000 7f1cd5ffb640 0 client.26000 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:37:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.079+0000 7f1cfb990740 1 -- 172.21.15.38:0/55690105 shutdown_connections 2024-05-21T15:37:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.079+0000 7f1cfb990740 1 -- 172.21.15.38:0/55690105 wait complete. 2024-05-21T15:37:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.080+0000 7f1cfb990740 20 client.26000 trim_cache size 0 max 16384 2024-05-21T15:37:30.078 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:37:30.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.081+0000 7f1cf4ae9640 1 -- 172.21.15.38:0/1662596516 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1cd80a2f00 msgr2=0x7f1cd80a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:30.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.081+0000 7f1cf4ae9640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1cd80a2f00 0x7f1cd80a5410 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009aa0 tx=0x7f1ce0005f90 comp rx=0 tx=0).stop 2024-05-21T15:37:30.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.081+0000 7f1cf4ae9640 1 -- 172.21.15.38:0/1662596516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf01093d0 msgr2=0x7f1cf011a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:30.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.081+0000 7f1cf4ae9640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf01093d0 0x7f1cf011a5c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0108c10 tx=0x7f1ce8003f20 comp rx=0 tx=0).stop 2024-05-21T15:37:30.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.081+0000 7f1cf4ae9640 1 -- 172.21.15.38:0/1662596516 shutdown_connections 2024-05-21T15:37:30.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.081+0000 7f1cf4ae9640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf010efe0 0x7f1cf0118c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.082+0000 7f1cf4ae9640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1cd80a2f00 0x7f1cd80a5410 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.082+0000 7f1cf4ae9640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1cf010a2c0 0x7f1cf011ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.082+0000 7f1cf4ae9640 1 --2- 172.21.15.38:0/1662596516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cf01093d0 0x7f1cf011a5c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.082+0000 7f1cf4ae9640 1 -- 172.21.15.38:0/1662596516 >> 172.21.15.38:0/1662596516 conn(0x7f1cf006f4a0 msgr2=0x7f1cf010cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:30.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.082+0000 7f1cf4ae9640 1 -- 172.21.15.38:0/1662596516 shutdown_connections 2024-05-21T15:37:30.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.082+0000 7f1cf4ae9640 1 -- 172.21.15.38:0/1662596516 wait complete. 2024-05-21T15:37:30.103 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:37:30.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:29 smithi053 ceph-mon[36496]: pgmap v443: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 203 MiB/s wr, 21.93k op/s 2024-05-21T15:37:30.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:29 smithi045 ceph-mon[36409]: pgmap v443: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 203 MiB/s wr, 21.93k op/s 2024-05-21T15:37:30.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.334+0000 7f43c785a640 1 -- 172.21.15.38:0/1692491856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c01078c0 msgr2=0x7f43c0109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:30.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.334+0000 7f43c785a640 1 --2- 172.21.15.38:0/1692491856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c01078c0 0x7f43c0109ce0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009ca0 tx=0x7f43b802e9c0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.335+0000 7f43c785a640 1 -- 172.21.15.38:0/1692491856 shutdown_connections 2024-05-21T15:37:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.335+0000 7f43c785a640 1 --2- 172.21.15.38:0/1692491856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c01078c0 0x7f43c0109ce0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.335+0000 7f43c785a640 1 --2- 172.21.15.38:0/1692491856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43c0102bc0 0x7f43c0102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.335+0000 7f43c785a640 1 --2- 172.21.15.38:0/1692491856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c0101980 0x7f43c0102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.335+0000 7f43c785a640 1 -- 172.21.15.38:0/1692491856 >> 172.21.15.38:0/1692491856 conn(0x7f43c00fcb70 msgr2=0x7f43c00ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.341+0000 7f43c785a640 1 -- 172.21.15.38:0/1692491856 shutdown_connections 2024-05-21T15:37:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.344+0000 7f43c785a640 1 -- 172.21.15.38:0/1692491856 wait complete. 2024-05-21T15:37:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.345+0000 7f43c785a640 1 Processor -- start 2024-05-21T15:37:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.345+0000 7f43c785a640 1 -- start start 2024-05-21T15:37:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.345+0000 7f43c785a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c0101980 0x7f43c0110800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.345+0000 7f43c785a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43c0102bc0 0x7f43c0110d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.345+0000 7f43c785a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c01078c0 0x7f43c0111ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.346+0000 7f43c4df5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43c0102bc0 0x7f43c0110d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.346+0000 7f43c4df5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43c0102bc0 0x7f43c0110d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40926/0 (socket says 172.21.15.38:40926) 2024-05-21T15:37:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.346+0000 7f43c4df5640 1 -- 172.21.15.38:0/1858460717 learned_addr learned my addr 172.21.15.38:0/1858460717 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.346+0000 7f43c785a640 1 -- 172.21.15.38:0/1858460717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0114d00 con 0x7f43c0102bc0 2024-05-21T15:37:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.346+0000 7f43c785a640 1 -- 172.21.15.38:0/1858460717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0114b80 con 0x7f43c0101980 2024-05-21T15:37:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.346+0000 7f43c785a640 1 -- 172.21.15.38:0/1858460717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0114e80 con 0x7f43c01078c0 2024-05-21T15:37:30.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.346+0000 7f43c55f6640 1 --2- 172.21.15.38:0/1858460717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c0101980 0x7f43c0110800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.347+0000 7f43c4df5640 1 -- 172.21.15.38:0/1858460717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c01078c0 msgr2=0x7f43c0111ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T15:37:30.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.347+0000 7f43c4df5640 1 --2- 172.21.15.38:0/1858460717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c01078c0 0x7f43c0111ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.347+0000 7f43c4df5640 1 -- 172.21.15.38:0/1858460717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c0101980 msgr2=0x7f43c0110800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:30.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.347+0000 7f43c4df5640 1 --2- 172.21.15.38:0/1858460717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c0101980 0x7f43c0110800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.347+0000 7f43c4df5640 1 -- 172.21.15.38:0/1858460717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c0112310 con 0x7f43c0102bc0 2024-05-21T15:37:30.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.350+0000 7f43c4df5640 1 --2- 172.21.15.38:0/1858460717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43c0102bc0 0x7f43c0110d40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f43b000d950 tx=0x7f43b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:30.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.351+0000 7f43b67fc640 1 -- 172.21.15.38:0/1858460717 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0017070 con 0x7f43c0102bc0 2024-05-21T15:37:30.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.351+0000 7f43c785a640 1 -- 172.21.15.38:0/1858460717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c0112600 con 0x7f43c0102bc0 2024-05-21T15:37:30.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.351+0000 7f43c785a640 1 -- 172.21.15.38:0/1858460717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c0112a90 con 0x7f43c0102bc0 2024-05-21T15:37:30.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.351+0000 7f43b67fc640 1 -- 172.21.15.38:0/1858460717 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b0009c30 con 0x7f43c0102bc0 2024-05-21T15:37:30.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.351+0000 7f43b67fc640 1 -- 172.21.15.38:0/1858460717 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b000f620 con 0x7f43c0102bc0 2024-05-21T15:37:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.357+0000 7f43b67fc640 1 -- 172.21.15.38:0/1858460717 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f43b000f7c0 con 0x7f43c0102bc0 2024-05-21T15:37:30.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.360+0000 7f43ce7dd740 1 Processor -- start 2024-05-21T15:37:30.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.360+0000 7f43b67fc640 1 --2- 172.21.15.38:0/1858460717 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f43ac0a2d10 0x7f43ac0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.361+0000 7f43b67fc640 1 -- 172.21.15.38:0/1858460717 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f43b00cfcd0 con 0x7f43c0102bc0 2024-05-21T15:37:30.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.362+0000 7f43c5df7640 1 --2- 172.21.15.38:0/1858460717 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f43ac0a2d10 0x7f43ac0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.363+0000 7f43ce7dd740 1 -- start start 2024-05-21T15:37:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.363+0000 7f43ce7dd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x56490307cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.363+0000 7f43ce7dd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cb44d0 0x56490307d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.363+0000 7f43ce7dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cbc6b0 0x564903083f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.363+0000 7f43ce7dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564902fe1c40 con 0x564902cbc6b0 2024-05-21T15:37:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.363+0000 7f43ce7dd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564902f89580 con 0x564902cb44d0 2024-05-21T15:37:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.363+0000 7f43ce7dd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564902fddc00 con 0x564902cdd390 2024-05-21T15:37:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.364+0000 7f43c5df7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cbc6b0 0x564903083f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.364+0000 7f43c5df7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cbc6b0 0x564903083f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40940/0 (socket says 172.21.15.38:40940) 2024-05-21T15:37:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.364+0000 7f43c5df7640 1 -- 172.21.15.38:0/3818485981 learned_addr learned my addr 172.21.15.38:0/3818485981 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.365+0000 7f43c55f6640 1 --2- 172.21.15.38:0/3818485981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x56490307cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.365+0000 7f43c5df7640 1 -- 172.21.15.38:0/3818485981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 msgr2=0x56490307cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:30.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.365+0000 7f43c5df7640 1 --2- 172.21.15.38:0/3818485981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x56490307cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.365+0000 7f43c5df7640 1 -- 172.21.15.38:0/3818485981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cb44d0 msgr2=0x56490307d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:30.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.365+0000 7f43c5df7640 1 --2- 172.21.15.38:0/3818485981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cb44d0 0x56490307d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.365+0000 7f43c5df7640 1 -- 172.21.15.38:0/3818485981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564902cf53f0 con 0x564902cbc6b0 2024-05-21T15:37:30.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.365+0000 7f43c55f6640 1 --2- 172.21.15.38:0/3818485981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x56490307cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:37:30.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.366+0000 7f43c5df7640 1 --2- 172.21.15.38:0/3818485981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cbc6b0 0x564903083f80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f43b80094e0 tx=0x7f43b8032ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1984ef8c67408ee server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:30.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.367+0000 7f43c5df7640 1 --2- 172.21.15.38:0/1858460717 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f43ac0a2d10 0x7f43ac0a5220 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f43b8005e30 tx=0x7f43b800c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:30.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.367+0000 7f43ab7fe640 1 -- 172.21.15.38:0/3818485981 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b804d070 con 0x564902cbc6b0 2024-05-21T15:37:30.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.367+0000 7f43ab7fe640 1 -- 172.21.15.38:0/3818485981 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b8003e50 con 0x564902cbc6b0 2024-05-21T15:37:30.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.367+0000 7f43ab7fe640 1 -- 172.21.15.38:0/3818485981 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8037070 con 0x564902cbc6b0 2024-05-21T15:37:30.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.368+0000 7f43ce7dd740 1 -- 172.21.15.38:0/3818485981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cbc6b0 msgr2=0x564903083f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:30.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.368+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/3818485981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cbc6b0 0x564903083f80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f43b80094e0 tx=0x7f43b8032ae0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.368+0000 7f43ce7dd740 1 -- 172.21.15.38:0/3818485981 shutdown_connections 2024-05-21T15:37:30.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.368+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/3818485981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cbc6b0 0x564903083f80 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.368+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/3818485981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cb44d0 0x56490307d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.368+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/3818485981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x56490307cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.368+0000 7f43ce7dd740 1 -- 172.21.15.38:0/3818485981 >> 172.21.15.38:0/3818485981 conn(0x564902cd14e0 msgr2=0x564903078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:30.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.372+0000 7f43ce7dd740 1 -- 172.21.15.38:0/3818485981 shutdown_connections 2024-05-21T15:37:30.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.372+0000 7f43ce7dd740 1 -- 172.21.15.38:0/3818485981 wait complete. 2024-05-21T15:37:30.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.372+0000 7f43ce7dd740 1 -- start start 2024-05-21T15:37:30.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.373+0000 7f43ce7dd740 1 Processor -- start 2024-05-21T15:37:30.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.373+0000 7f43ce7dd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cb44d0 0x564903086a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.373+0000 7f43ce7dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cbc6b0 0x564903086f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.374+0000 7f43c55f6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cb44d0 0x564903086a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.374+0000 7f43c55f6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cb44d0 0x564903086a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46726/0 (socket says 172.21.15.38:46726) 2024-05-21T15:37:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.374+0000 7f43c55f6640 1 -- 172.21.15.38:0/4135795710 learned_addr learned my addr 172.21.15.38:0/4135795710 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.374+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/4135795710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cdd390 0x56490308dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.374+0000 7f43ce7dd740 1 -- 172.21.15.38:0/4135795710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564903084970 con 0x564902cbc6b0 2024-05-21T15:37:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.374+0000 7f43ce7dd740 1 -- 172.21.15.38:0/4135795710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564902f9cc40 con 0x564902cdd390 2024-05-21T15:37:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.374+0000 7f43ce7dd740 1 -- 172.21.15.38:0/4135795710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564903084af0 con 0x564902cb44d0 2024-05-21T15:37:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.374+0000 7f43ce7dd740 10 client.0 _ll_register_callbacks cb 0x564902cf4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.375+0000 7f43c55f6640 1 -- 172.21.15.38:0/4135795710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cdd390 msgr2=0x56490308dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.375+0000 7f43c55f6640 1 --2- 172.21.15.38:0/4135795710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cdd390 0x56490308dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.375+0000 7f43c55f6640 1 -- 172.21.15.38:0/4135795710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cbc6b0 msgr2=0x564903086f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.375+0000 7f43c55f6640 1 --2- 172.21.15.38:0/4135795710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cbc6b0 0x564903086f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.375+0000 7f43c55f6640 1 -- 172.21.15.38:0/4135795710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564902dbf7d0 con 0x564902cb44d0 2024-05-21T15:37:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.376+0000 7f43c55f6640 1 --2- 172.21.15.38:0/4135795710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cb44d0 0x564903086a20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00b5f0 tx=0x7f43bc00b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca774c6f129c4541 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.376+0000 7f43a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:37:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.376+0000 7f43a8ff9640 1 -- 172.21.15.38:0/4135795710 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc005220 con 0x564902cb44d0 2024-05-21T15:37:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.377+0000 7f43ce7dd740 1 -- 172.21.15.38:0/4135795710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43b8003e50 con 0x564902cb44d0 2024-05-21T15:37:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.377+0000 7f43a8ff9640 1 -- 172.21.15.38:0/4135795710 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f43bc0053c0 con 0x564902cb44d0 2024-05-21T15:37:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.377+0000 7f43a8ff9640 1 -- 172.21.15.38:0/4135795710 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f43bc0055c0 con 0x564902cb44d0 2024-05-21T15:37:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.378+0000 7f43ce7dd740 10 client.26047 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.378+0000 7f43ce7dd740 1 -- 172.21.15.38:0/4135795710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5649030911f0 con 0x564902cb44d0 2024-05-21T15:37:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.379+0000 7f43a8ff9640 1 -- 172.21.15.38:0/4135795710 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b8003e50 con 0x564902cb44d0 2024-05-21T15:37:30.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.379+0000 7f43a8ff9640 1 -- 172.21.15.38:0/4135795710 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f43bc01f070 con 0x564902cb44d0 2024-05-21T15:37:30.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.381+0000 7f43ce7dd740 10 client.26047 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:30.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.381+0000 7f43ce7dd740 10 client.26047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.381+0000 7f43ce7dd740 20 client.26047 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.381+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/4135795710 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564902c9bf60 0x564903094a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.381+0000 7f43ce7dd740 4 client.26047 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:37:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.381+0000 7f43ce7dd740 1 -- 172.21.15.38:0/4135795710 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564902fef370 con 0x564902c9bf60 2024-05-21T15:37:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.382+0000 7f43c4df5640 1 --2- 172.21.15.38:0/4135795710 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564902c9bf60 0x564903094a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.383+0000 7f43c4df5640 1 --2- 172.21.15.38:0/4135795710 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564902c9bf60 0x564903094a20 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13723fb0959e101e server_cookie=b3640fd38f0892d6 in_seq=0 out_seq=0 2024-05-21T15:37:30.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.383+0000 7f43a8ff9640 10 client.26047 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.385+0000 7f43a8ff9640 1 -- 172.21.15.38:0/4135795710 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564902fef370 con 0x564902c9bf60 2024-05-21T15:37:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.385+0000 7f43a8ff9640 10 client.26047 handle_command_reply: tid=0 2024-05-21T15:37:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.385+0000 7f43ce7dd740 1 client.26047 shutdown 2024-05-21T15:37:30.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.386+0000 7f43ce7dd740 1 -- 172.21.15.38:0/4135795710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cb44d0 msgr2=0x564903086a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:30.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.386+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/4135795710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cb44d0 0x564903086a20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00b5f0 tx=0x7f43bc00b9f0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.386+0000 7f43ce7dd740 1 -- 172.21.15.38:0/4135795710 shutdown_connections 2024-05-21T15:37:30.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.386+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/4135795710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cdd390 0x56490308dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.386+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/4135795710 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564902c9bf60 0x564903094a20 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.386+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/4135795710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cbc6b0 0x564903086f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.386+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/4135795710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cb44d0 0x564903086a20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.386+0000 7f43ce7dd740 1 -- 172.21.15.38:0/4135795710 >> 172.21.15.38:0/4135795710 conn(0x564902cd14e0 msgr2=0x564903077570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:30.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.386+0000 7f43a8ff9640 0 client.26047 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:30.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.387+0000 7f43ce7dd740 1 -- 172.21.15.38:0/4135795710 shutdown_connections 2024-05-21T15:37:30.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.387+0000 7f43ce7dd740 1 -- 172.21.15.38:0/4135795710 wait complete. 2024-05-21T15:37:30.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.388+0000 7f43ce7dd740 20 client.26047 trim_cache size 0 max 16384 2024-05-21T15:37:30.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.421+0000 7f43ce7dd740 1 Processor -- start 2024-05-21T15:37:30.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.423+0000 7f43ce7dd740 1 -- start start 2024-05-21T15:37:30.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.423+0000 7f43ce7dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cb44d0 0x564903081460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.423+0000 7f43ce7dd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cbc6b0 0x56490308de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.423+0000 7f43ce7dd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x564903086d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.424+0000 7f43ce7dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f43a400e680 con 0x564902cb44d0 2024-05-21T15:37:30.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.424+0000 7f43c5df7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x564903086d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.424+0000 7f43c5df7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x564903086d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46734/0 (socket says 172.21.15.38:46734) 2024-05-21T15:37:30.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.424+0000 7f43c5df7640 1 -- 172.21.15.38:0/1402964760 learned_addr learned my addr 172.21.15.38:0/1402964760 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:30.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.424+0000 7f43ce7dd740 1 -- 172.21.15.38:0/1402964760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43a40102b0 con 0x564902cbc6b0 2024-05-21T15:37:30.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.424+0000 7f43ce7dd740 1 -- 172.21.15.38:0/1402964760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f43a400ed50 con 0x564902cdd390 2024-05-21T15:37:30.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.424+0000 7f43c4df5640 1 --2- 172.21.15.38:0/1402964760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cbc6b0 0x56490308de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.424+0000 7f43c55f6640 1 --2- 172.21.15.38:0/1402964760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cb44d0 0x564903081460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.426+0000 7f43c55f6640 1 -- 172.21.15.38:0/1402964760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 msgr2=0x564903086d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:30.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.426+0000 7f43c55f6640 1 --2- 172.21.15.38:0/1402964760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x564903086d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.426+0000 7f43c55f6640 1 -- 172.21.15.38:0/1402964760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cbc6b0 msgr2=0x56490308de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:30.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.426+0000 7f43c55f6640 1 --2- 172.21.15.38:0/1402964760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cbc6b0 0x56490308de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.426+0000 7f43c55f6640 1 -- 172.21.15.38:0/1402964760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564903078900 con 0x564902cb44d0 2024-05-21T15:37:30.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.427+0000 7f43c55f6640 1 --2- 172.21.15.38:0/1402964760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cb44d0 0x564903081460 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f43bc016450 tx=0x7f43bc0166d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39445fb28abcbf23 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:30.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.429+0000 7f43aaffd640 1 -- 172.21.15.38:0/1402964760 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc025070 con 0x564902cb44d0 2024-05-21T15:37:30.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.429+0000 7f43aaffd640 1 -- 172.21.15.38:0/1402964760 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43bc016d20 con 0x564902cb44d0 2024-05-21T15:37:30.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.429+0000 7f43aaffd640 1 -- 172.21.15.38:0/1402964760 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc021b80 con 0x564902cb44d0 2024-05-21T15:37:30.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.430+0000 7f43ce7dd740 1 -- 172.21.15.38:0/1402964760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cb44d0 msgr2=0x564903081460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:30.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.430+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/1402964760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cb44d0 0x564903081460 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f43bc016450 tx=0x7f43bc0166d0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.433+0000 7f43ce7dd740 1 -- 172.21.15.38:0/1402964760 shutdown_connections 2024-05-21T15:37:30.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.433+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/1402964760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x564903086d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.433+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/1402964760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cbc6b0 0x56490308de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.433+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/1402964760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cb44d0 0x564903081460 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.433+0000 7f43ce7dd740 1 -- 172.21.15.38:0/1402964760 >> 172.21.15.38:0/1402964760 conn(0x564902cd14e0 msgr2=0x5649030865f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:30.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.435+0000 7f43ce7dd740 1 -- 172.21.15.38:0/1402964760 shutdown_connections 2024-05-21T15:37:30.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.435+0000 7f43ce7dd740 1 -- 172.21.15.38:0/1402964760 wait complete. 2024-05-21T15:37:30.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.436+0000 7f43ce7dd740 1 -- start start 2024-05-21T15:37:30.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.436+0000 7f43ce7dd740 1 Processor -- start 2024-05-21T15:37:30.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.437+0000 7f43ce7dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cb44d0 0x56490308adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.437+0000 7f43ce7dd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cbc6b0 0x56490308b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.437+0000 7f43ce7dd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x5649030fadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.437+0000 7f43ce7dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f43a400fb20 con 0x564902cb44d0 2024-05-21T15:37:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.437+0000 7f43ce7dd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43a400f420 con 0x564902cbc6b0 2024-05-21T15:37:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.437+0000 7f43ce7dd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f43a400c660 con 0x564902cdd390 2024-05-21T15:37:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.437+0000 7f43ce7dd740 10 client.0 _ll_register_callbacks cb 0x7f43bc005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.437+0000 7f43c5df7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x5649030fadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.437+0000 7f43c5df7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x5649030fadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46736/0 (socket says 172.21.15.38:46736) 2024-05-21T15:37:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.437+0000 7f43c5df7640 1 -- 172.21.15.38:0/2569290077 learned_addr learned my addr 172.21.15.38:0/2569290077 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.438+0000 7f43c4df5640 1 --2- 172.21.15.38:0/2569290077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cbc6b0 0x56490308b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.438+0000 7f43c55f6640 1 --2- 172.21.15.38:0/2569290077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cb44d0 0x56490308adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.438+0000 7f43c5df7640 1 -- 172.21.15.38:0/2569290077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cbc6b0 msgr2=0x56490308b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.438+0000 7f43c5df7640 1 --2- 172.21.15.38:0/2569290077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cbc6b0 0x56490308b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.438+0000 7f43c5df7640 1 -- 172.21.15.38:0/2569290077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cb44d0 msgr2=0x56490308adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:30.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.438+0000 7f43c5df7640 1 --2- 172.21.15.38:0/2569290077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cb44d0 0x56490308adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.438+0000 7f43c5df7640 1 -- 172.21.15.38:0/2569290077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649030fb300 con 0x564902cdd390 2024-05-21T15:37:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.439+0000 7f43c5df7640 1 --2- 172.21.15.38:0/2569290077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x5649030fadc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43b802efb0 tx=0x7f43b800f5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=900e3b856c94e466 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.439+0000 7f43a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:37:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.439+0000 7f43a8ff9640 1 -- 172.21.15.38:0/2569290077 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8055070 con 0x564902cdd390 2024-05-21T15:37:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.439+0000 7f43ce7dd740 1 -- 172.21.15.38:0/2569290077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43bc016d20 con 0x564902cdd390 2024-05-21T15:37:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.439+0000 7f43a8ff9640 1 -- 172.21.15.38:0/2569290077 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f43b8032cd0 con 0x564902cdd390 2024-05-21T15:37:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.440+0000 7f43a8ff9640 1 -- 172.21.15.38:0/2569290077 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f43b8037050 con 0x564902cdd390 2024-05-21T15:37:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.442+0000 7f43ce7dd740 10 client.26053 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.442+0000 7f43ce7dd740 1 -- 172.21.15.38:0/2569290077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5649030fc520 con 0x564902cdd390 2024-05-21T15:37:30.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.445+0000 7f43a8ff9640 1 -- 172.21.15.38:0/2569290077 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43bc016d20 con 0x564902cdd390 2024-05-21T15:37:30.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.445+0000 7f43a8ff9640 1 -- 172.21.15.38:0/2569290077 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f43b8058030 con 0x564902cdd390 2024-05-21T15:37:30.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.445+0000 7f43ce7dd740 10 client.26053 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:30.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.445+0000 7f43ce7dd740 10 client.26053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:30.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.445+0000 7f43ce7dd740 20 client.26053 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:30.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.445+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/2569290077 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564902c713a0 0x5649030fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:30.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.445+0000 7f43ce7dd740 4 client.26053 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:37:30.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.445+0000 7f43ce7dd740 1 -- 172.21.15.38:0/2569290077 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564902fe96c0 con 0x564902c713a0 2024-05-21T15:37:30.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.446+0000 7f43c55f6640 1 --2- 172.21.15.38:0/2569290077 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564902c713a0 0x5649030fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:30.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.450+0000 7f43c55f6640 1 --2- 172.21.15.38:0/2569290077 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564902c713a0 0x5649030fefa0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfd7adfd126479e0 server_cookie=b50f2ed61b5dd2c5 in_seq=0 out_seq=0 2024-05-21T15:37:30.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.451+0000 7f43a8ff9640 10 client.26053 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.453+0000 7f43a8ff9640 1 -- 172.21.15.38:0/2569290077 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564902fe96c0 con 0x564902c713a0 2024-05-21T15:37:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.453+0000 7f43a8ff9640 10 client.26053 handle_command_reply: tid=0 2024-05-21T15:37:30.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.453+0000 7f43ce7dd740 1 client.26053 shutdown 2024-05-21T15:37:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.458+0000 7f43ce7dd740 1 -- 172.21.15.38:0/2569290077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 msgr2=0x5649030fadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.458+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/2569290077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x5649030fadc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43b802efb0 tx=0x7f43b800f5d0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.459+0000 7f43ce7dd740 1 -- 172.21.15.38:0/2569290077 shutdown_connections 2024-05-21T15:37:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.459+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/2569290077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564902cdd390 0x5649030fadc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.459+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/2569290077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564902cbc6b0 0x56490308b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.459+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/2569290077 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564902c713a0 0x5649030fefa0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.459+0000 7f43ce7dd740 1 --2- 172.21.15.38:0/2569290077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902cb44d0 0x56490308adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.459+0000 7f43ce7dd740 1 -- 172.21.15.38:0/2569290077 >> 172.21.15.38:0/2569290077 conn(0x564902cd14e0 msgr2=0x564903077570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.459+0000 7f43a8ff9640 0 client.26053 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.462+0000 7f43ce7dd740 1 -- 172.21.15.38:0/2569290077 shutdown_connections 2024-05-21T15:37:30.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.463+0000 7f43ce7dd740 1 -- 172.21.15.38:0/2569290077 wait complete. 2024-05-21T15:37:30.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.465+0000 7f43ce7dd740 20 client.26053 trim_cache size 0 max 16384 2024-05-21T15:37:30.465 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:37:30.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.469+0000 7f43c785a640 1 -- 172.21.15.38:0/1858460717 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f43ac0a2d10 msgr2=0x7f43ac0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:30.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.469+0000 7f43c785a640 1 --2- 172.21.15.38:0/1858460717 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f43ac0a2d10 0x7f43ac0a5220 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f43b8005e30 tx=0x7f43b800c000 comp rx=0 tx=0).stop 2024-05-21T15:37:30.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.470+0000 7f43c785a640 1 -- 172.21.15.38:0/1858460717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43c0102bc0 msgr2=0x7f43c0110d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:30.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.470+0000 7f43c785a640 1 --2- 172.21.15.38:0/1858460717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43c0102bc0 0x7f43c0110d40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f43b000d950 tx=0x7f43b000de10 comp rx=0 tx=0).stop 2024-05-21T15:37:30.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.471+0000 7f43c785a640 1 -- 172.21.15.38:0/1858460717 shutdown_connections 2024-05-21T15:37:30.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.471+0000 7f43c785a640 1 --2- 172.21.15.38:0/1858460717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c01078c0 0x7f43c0111ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.471+0000 7f43c785a640 1 --2- 172.21.15.38:0/1858460717 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f43ac0a2d10 0x7f43ac0a5220 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.471+0000 7f43c785a640 1 --2- 172.21.15.38:0/1858460717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43c0102bc0 0x7f43c0110d40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.471+0000 7f43c785a640 1 --2- 172.21.15.38:0/1858460717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c0101980 0x7f43c0110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:30.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.471+0000 7f43c785a640 1 -- 172.21.15.38:0/1858460717 >> 172.21.15.38:0/1858460717 conn(0x7f43c00fcb70 msgr2=0x7f43c01038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:30.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.471+0000 7f43c785a640 1 -- 172.21.15.38:0/1858460717 shutdown_connections 2024-05-21T15:37:30.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:30.471+0000 7f43c785a640 1 -- 172.21.15.38:0/1858460717 wait complete. 2024-05-21T15:37:31.493 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:37:31.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:37:31.493 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:37:31.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.717+0000 7f14edd21640 1 -- 172.21.15.38:0/400686566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e8107900 msgr2=0x7f14e8109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:31.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.717+0000 7f14edd21640 1 --2- 172.21.15.38:0/400686566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e8107900 0x7f14e8109d20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009ca0 tx=0x7f14e0032de0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.718+0000 7f14edd21640 1 -- 172.21.15.38:0/400686566 shutdown_connections 2024-05-21T15:37:31.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.718+0000 7f14edd21640 1 --2- 172.21.15.38:0/400686566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e8107900 0x7f14e8109d20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.718+0000 7f14edd21640 1 --2- 172.21.15.38:0/400686566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14e8102c00 0x7f14e8103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.718+0000 7f14edd21640 1 --2- 172.21.15.38:0/400686566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e81019c0 0x7f14e81026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.718+0000 7f14edd21640 1 -- 172.21.15.38:0/400686566 >> 172.21.15.38:0/400686566 conn(0x7f14e80fcbb0 msgr2=0x7f14e80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:31.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.718+0000 7f14edd21640 1 -- 172.21.15.38:0/400686566 shutdown_connections 2024-05-21T15:37:31.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.718+0000 7f14edd21640 1 -- 172.21.15.38:0/400686566 wait complete. 2024-05-21T15:37:31.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.719+0000 7f14edd21640 1 Processor -- start 2024-05-21T15:37:31.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.719+0000 7f14edd21640 1 -- start start 2024-05-21T15:37:31.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.720+0000 7f14edd21640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e81019c0 0x7f14e8071830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.720+0000 7f14edd21640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14e8102c00 0x7f14e8071d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.720+0000 7f14edd21640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e8107900 0x7f14e8072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.720+0000 7f14edd21640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f14e806bae0 con 0x7f14e8102c00 2024-05-21T15:37:31.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.720+0000 7f14edd21640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14e806b960 con 0x7f14e81019c0 2024-05-21T15:37:31.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.720+0000 7f14edd21640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f14e806bc60 con 0x7f14e8107900 2024-05-21T15:37:31.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.721+0000 7f14e77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e81019c0 0x7f14e8071830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.721+0000 7f14e77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e81019c0 0x7f14e8071830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45406/0 (socket says 172.21.15.38:45406) 2024-05-21T15:37:31.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.721+0000 7f14e77fe640 1 -- 172.21.15.38:0/4249402551 learned_addr learned my addr 172.21.15.38:0/4249402551 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:31.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.721+0000 7f14e7fff640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e8107900 0x7f14e8072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.721+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14e8102c00 0x7f14e8071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.722+0000 7f14e7fff640 1 -- 172.21.15.38:0/4249402551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e81019c0 msgr2=0x7f14e8071830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:31.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.722+0000 7f14e7fff640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e81019c0 0x7f14e8071830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.722+0000 7f14e7fff640 1 -- 172.21.15.38:0/4249402551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14e8102c00 msgr2=0x7f14e8071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:31.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.722+0000 7f14e7fff640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14e8102c00 0x7f14e8071d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.722+0000 7f14e7fff640 1 -- 172.21.15.38:0/4249402551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e8073340 con 0x7f14e8107900 2024-05-21T15:37:31.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.722+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14e8102c00 0x7f14e8071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:37:31.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.722+0000 7f14e7fff640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e8107900 0x7f14e8072cd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009bf0 tx=0x7f14e0039610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:31.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.724+0000 7f14e4ff9640 1 -- 172.21.15.38:0/4249402551 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0040020 con 0x7f14e8107900 2024-05-21T15:37:31.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.724+0000 7f14e4ff9640 1 -- 172.21.15.38:0/4249402551 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e00183a0 con 0x7f14e8107900 2024-05-21T15:37:31.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.724+0000 7f14e4ff9640 1 -- 172.21.15.38:0/4249402551 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e003a8c0 con 0x7f14e8107900 2024-05-21T15:37:31.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.724+0000 7f14edd21640 1 -- 172.21.15.38:0/4249402551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e80735d0 con 0x7f14e8107900 2024-05-21T15:37:31.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.724+0000 7f14edd21640 1 -- 172.21.15.38:0/4249402551 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e8073af0 con 0x7f14e8107900 2024-05-21T15:37:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.734+0000 7f14f4bc9740 1 Processor -- start 2024-05-21T15:37:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.735+0000 7f14e4ff9640 1 -- 172.21.15.38:0/4249402551 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f14e0043020 con 0x7f14e8107900 2024-05-21T15:37:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.737+0000 7f14f4bc9740 1 -- start start 2024-05-21T15:37:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.737+0000 7f14f4bc9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b58570410 0x555b589104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.737+0000 7f14f4bc9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b58910a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.737+0000 7f14f4bc9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5854f730 0x555b58917920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.737+0000 7f14f4bc9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555b58874990 con 0x555b5854f730 2024-05-21T15:37:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.737+0000 7f14f4bc9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555b5881c5b0 con 0x555b58570410 2024-05-21T15:37:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.737+0000 7f14f4bc9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555b58870950 con 0x555b58547550 2024-05-21T15:37:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.738+0000 7f14e77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b58910a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.738+0000 7f14e6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b58570410 0x555b589104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.738+0000 7f14e6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b58570410 0x555b589104a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45408/0 (socket says 172.21.15.38:45408) 2024-05-21T15:37:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.738+0000 7f14e6ffd640 1 -- 172.21.15.38:0/3174720743 learned_addr learned my addr 172.21.15.38:0/3174720743 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.738+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/3174720743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5854f730 0x555b58917920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.739+0000 7f14e6ffd640 1 -- 172.21.15.38:0/3174720743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 msgr2=0x555b58910a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.739+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/3174720743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b58910a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.739+0000 7f14e6ffd640 1 -- 172.21.15.38:0/3174720743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5854f730 msgr2=0x555b58917920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.739+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/3174720743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5854f730 0x555b58917920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.739+0000 7f14e6ffd640 1 -- 172.21.15.38:0/3174720743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b586507e0 con 0x555b58570410 2024-05-21T15:37:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.739+0000 7f14e4ff9640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f14d00a2cd0 0x7f14d00a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.739+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/3174720743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b58570410 0x555b589104a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f14d800b400 tx=0x7f14d8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=436cd4ab2e4a4630 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.739+0000 7f14e4ff9640 1 -- 172.21.15.38:0/4249402551 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f14e00ef9d0 con 0x7f14e8107900 2024-05-21T15:37:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.740+0000 7f14cdffb640 1 -- 172.21.15.38:0/3174720743 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80077c0 con 0x555b58570410 2024-05-21T15:37:31.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.740+0000 7f14cdffb640 1 -- 172.21.15.38:0/3174720743 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d8020070 con 0x555b58570410 2024-05-21T15:37:31.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.740+0000 7f14cdffb640 1 -- 172.21.15.38:0/3174720743 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d801bbf0 con 0x555b58570410 2024-05-21T15:37:31.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.742+0000 7f14f4bc9740 1 -- 172.21.15.38:0/3174720743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b58570410 msgr2=0x555b589104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:31.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.742+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/3174720743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b58570410 0x555b589104a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f14d800b400 tx=0x7f14d8005f90 comp rx=0 tx=0).stop 2024-05-21T15:37:31.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.742+0000 7f14e77fe640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f14d00a2cd0 0x7f14d00a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.743+0000 7f14e77fe640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f14d00a2cd0 0x7f14d00a51e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x555b58911880 tx=0x7f14dc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:31.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.744+0000 7f14f4bc9740 1 -- 172.21.15.38:0/3174720743 shutdown_connections 2024-05-21T15:37:31.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.744+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/3174720743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5854f730 0x555b58917920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.744+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/3174720743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b58910a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.744+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/3174720743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b58570410 0x555b589104a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.744+0000 7f14f4bc9740 1 -- 172.21.15.38:0/3174720743 >> 172.21.15.38:0/3174720743 conn(0x555b58564560 msgr2=0x555b5890be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:31.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.746+0000 7f14f4bc9740 1 -- 172.21.15.38:0/3174720743 shutdown_connections 2024-05-21T15:37:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.746+0000 7f14f4bc9740 1 -- 172.21.15.38:0/3174720743 wait complete. 2024-05-21T15:37:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.746+0000 7f14f4bc9740 1 -- start start 2024-05-21T15:37:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.747+0000 7f14f4bc9740 1 Processor -- start 2024-05-21T15:37:31.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.747+0000 7f14f4bc9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b5891a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.747+0000 7f14f4bc9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b5854f730 0x555b5891a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.747+0000 7f14f4bc9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b58570410 0x555b58921790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.748+0000 7f14f4bc9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555b58918310 con 0x555b58570410 2024-05-21T15:37:31.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.748+0000 7f14f4bc9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555b5882fb40 con 0x555b5854f730 2024-05-21T15:37:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.748+0000 7f14f4bc9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555b58918490 con 0x555b58547550 2024-05-21T15:37:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.748+0000 7f14f4bc9740 10 client.0 _ll_register_callbacks cb 0x555b588240f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.748+0000 7f14e77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b5891a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.748+0000 7f14e77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b5891a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46778/0 (socket says 172.21.15.38:46778) 2024-05-21T15:37:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.748+0000 7f14e77fe640 1 -- 172.21.15.38:0/1346374133 learned_addr learned my addr 172.21.15.38:0/1346374133 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.748+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/1346374133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b5854f730 0x555b5891a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.748+0000 7f14e7fff640 1 --2- 172.21.15.38:0/1346374133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b58570410 0x555b58921790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.749+0000 7f14e77fe640 1 -- 172.21.15.38:0/1346374133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b5854f730 msgr2=0x555b5891a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.749+0000 7f14e77fe640 1 --2- 172.21.15.38:0/1346374133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b5854f730 0x555b5891a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.749+0000 7f14e77fe640 1 -- 172.21.15.38:0/1346374133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b58570410 msgr2=0x555b58921790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.749+0000 7f14e77fe640 1 --2- 172.21.15.38:0/1346374133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b58570410 0x555b58921790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.749+0000 7f14e77fe640 1 -- 172.21.15.38:0/1346374133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b586b0f70 con 0x555b58547550 2024-05-21T15:37:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.750+0000 7f14e77fe640 1 --2- 172.21.15.38:0/1346374133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b5891a2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0149f0 tx=0x7f14dc004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7de5d78301ba1e40 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.751+0000 7f14c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:37:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.751+0000 7f14c6ffd640 1 -- 172.21.15.38:0/1346374133 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc026020 con 0x555b58547550 2024-05-21T15:37:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.751+0000 7f14c6ffd640 1 -- 172.21.15.38:0/1346374133 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f14dc0223f0 con 0x555b58547550 2024-05-21T15:37:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.753+0000 7f14f4bc9740 1 -- 172.21.15.38:0/1346374133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d8020070 con 0x555b58547550 2024-05-21T15:37:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.753+0000 7f14c6ffd640 1 -- 172.21.15.38:0/1346374133 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f14dc01e040 con 0x555b58547550 2024-05-21T15:37:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.753+0000 7f14f4bc9740 10 client.26065 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.754+0000 7f14f4bc9740 1 -- 172.21.15.38:0/1346374133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555b58548170 con 0x555b58547550 2024-05-21T15:37:31.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.756+0000 7f14c6ffd640 1 -- 172.21.15.38:0/1346374133 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14dc022ad0 con 0x555b58547550 2024-05-21T15:37:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.757+0000 7f14c6ffd640 1 -- 172.21.15.38:0/1346374133 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f14dc021030 con 0x555b58547550 2024-05-21T15:37:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.758+0000 7f14f4bc9740 10 client.26065 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.758+0000 7f14f4bc9740 10 client.26065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.758+0000 7f14f4bc9740 20 client.26065 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.758+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/1346374133 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555b5852f1f0 0x555b58927d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.758+0000 7f14f4bc9740 4 client.26065 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:37:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.758+0000 7f14f4bc9740 1 -- 172.21.15.38:0/1346374133 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f14d8020070 con 0x555b5852f1f0 2024-05-21T15:37:31.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.758+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/1346374133 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555b5852f1f0 0x555b58927d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.759+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/1346374133 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555b5852f1f0 0x555b58927d10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f2e40430e54ec9 server_cookie=33fe622133c03ad1 in_seq=0 out_seq=0 2024-05-21T15:37:31.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.759+0000 7f14c6ffd640 10 client.26065 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:31.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.762+0000 7f14c6ffd640 1 -- 172.21.15.38:0/1346374133 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f14d8020070 con 0x555b5852f1f0 2024-05-21T15:37:31.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.762+0000 7f14c6ffd640 10 client.26065 handle_command_reply: tid=0 2024-05-21T15:37:31.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.762+0000 7f14f4bc9740 1 client.26065 shutdown 2024-05-21T15:37:31.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.764+0000 7f14f4bc9740 1 -- 172.21.15.38:0/1346374133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 msgr2=0x555b5891a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.764+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/1346374133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b5891a2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0149f0 tx=0x7f14dc004770 comp rx=0 tx=0).stop 2024-05-21T15:37:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.764+0000 7f14f4bc9740 1 -- 172.21.15.38:0/1346374133 shutdown_connections 2024-05-21T15:37:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.764+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/1346374133 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555b5852f1f0 0x555b58927d10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.764+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/1346374133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b58570410 0x555b58921790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.764+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/1346374133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b5854f730 0x555b5891a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.765+0000 7f14c6ffd640 0 client.26065 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.765+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/1346374133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b5891a2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.765+0000 7f14f4bc9740 1 -- 172.21.15.38:0/1346374133 >> 172.21.15.38:0/1346374133 conn(0x555b58564560 msgr2=0x555b5890ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:31.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.765+0000 7f14f4bc9740 1 -- 172.21.15.38:0/1346374133 shutdown_connections 2024-05-21T15:37:31.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.765+0000 7f14f4bc9740 1 -- 172.21.15.38:0/1346374133 wait complete. 2024-05-21T15:37:31.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.766+0000 7f14f4bc9740 20 client.26065 trim_cache size 0 max 16384 2024-05-21T15:37:31.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.813+0000 7f14f4bc9740 1 Processor -- start 2024-05-21T15:37:31.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.814+0000 7f14f4bc9740 1 -- start start 2024-05-21T15:37:31.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.814+0000 7f14f4bc9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5852f1f0 0x555b589192f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.815+0000 7f14f4bc9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b58919830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.815+0000 7f14f4bc9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b5854f730 0x555b58921720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.815+0000 7f14f4bc9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f14d400e530 con 0x555b5852f1f0 2024-05-21T15:37:31.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.815+0000 7f14f4bc9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4010160 con 0x555b5854f730 2024-05-21T15:37:31.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.815+0000 7f14f4bc9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f14d400ec00 con 0x555b58547550 2024-05-21T15:37:31.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.815+0000 7f14e6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b58919830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.815+0000 7f14e6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b58919830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46782/0 (socket says 172.21.15.38:46782) 2024-05-21T15:37:31.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.815+0000 7f14e6ffd640 1 -- 172.21.15.38:0/105898592 learned_addr learned my addr 172.21.15.38:0/105898592 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:31.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.816+0000 7f14e77fe640 1 --2- 172.21.15.38:0/105898592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5852f1f0 0x555b589192f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.816+0000 7f14e6ffd640 1 -- 172.21.15.38:0/105898592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b5854f730 msgr2=0x555b58921720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:37:31.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.816+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/105898592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b5854f730 0x555b58921720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.816+0000 7f14e6ffd640 1 -- 172.21.15.38:0/105898592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5852f1f0 msgr2=0x555b589192f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:31.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.816+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/105898592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5852f1f0 0x555b589192f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.816+0000 7f14e6ffd640 1 -- 172.21.15.38:0/105898592 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b589116a0 con 0x555b58547550 2024-05-21T15:37:31.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.817+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/105898592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b58919830 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14d801f000 tx=0x7f14d800b950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5ccb84b2f7e5df0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.819+0000 7f14cd7fa640 1 -- 172.21.15.38:0/105898592 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8031070 con 0x555b58547550 2024-05-21T15:37:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.819+0000 7f14cd7fa640 1 -- 172.21.15.38:0/105898592 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d8020040 con 0x555b58547550 2024-05-21T15:37:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.819+0000 7f14cd7fa640 1 -- 172.21.15.38:0/105898592 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d801bd90 con 0x555b58547550 2024-05-21T15:37:31.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.820+0000 7f14f4bc9740 1 -- 172.21.15.38:0/105898592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 msgr2=0x555b58919830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:31.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.820+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/105898592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b58919830 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14d801f000 tx=0x7f14d800b950 comp rx=0 tx=0).stop 2024-05-21T15:37:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.823+0000 7f14f4bc9740 1 -- 172.21.15.38:0/105898592 shutdown_connections 2024-05-21T15:37:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.823+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/105898592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b5854f730 0x555b58921720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.823+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/105898592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b58547550 0x555b58919830 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14d801f000 tx=0x7f14d800b950 comp rx=0 tx=0).stop 2024-05-21T15:37:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.823+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/105898592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5852f1f0 0x555b589192f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.823+0000 7f14f4bc9740 1 -- 172.21.15.38:0/105898592 >> 172.21.15.38:0/105898592 conn(0x555b58564560 msgr2=0x555b5890f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:31.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.827+0000 7f14f4bc9740 1 -- 172.21.15.38:0/105898592 shutdown_connections 2024-05-21T15:37:31.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.830+0000 7f14f4bc9740 1 -- 172.21.15.38:0/105898592 wait complete. 2024-05-21T15:37:31.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.831+0000 7f14f4bc9740 1 -- start start 2024-05-21T15:37:31.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.831+0000 7f14f4bc9740 1 Processor -- start 2024-05-21T15:37:31.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.833+0000 7f14f4bc9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5852f1f0 0x555b589214d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.833+0000 7f14f4bc9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b5854f730 0x555b58921a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.833+0000 7f14f4bc9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b58570410 0x555b5898e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.833+0000 7f14f4bc9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f14d400f9d0 con 0x555b5852f1f0 2024-05-21T15:37:31.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.833+0000 7f14f4bc9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14d400f2d0 con 0x555b58570410 2024-05-21T15:37:31.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.833+0000 7f14f4bc9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f14d400c5c0 con 0x555b5854f730 2024-05-21T15:37:31.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.833+0000 7f14f4bc9740 10 client.0 _ll_register_callbacks cb 0x7f14dc014670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:31.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.834+0000 7f14e7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b58570410 0x555b5898e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.834+0000 7f14e7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b58570410 0x555b5898e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45420/0 (socket says 172.21.15.38:45420) 2024-05-21T15:37:31.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.834+0000 7f14e7fff640 1 -- 172.21.15.38:0/2663188909 learned_addr learned my addr 172.21.15.38:0/2663188909 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:31.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.836+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/2663188909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b5854f730 0x555b58921a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.836+0000 7f14e77fe640 1 --2- 172.21.15.38:0/2663188909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5852f1f0 0x555b589214d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.838+0000 7f14e77fe640 1 -- 172.21.15.38:0/2663188909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b5854f730 msgr2=0x555b58921a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.838+0000 7f14e77fe640 1 --2- 172.21.15.38:0/2663188909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b5854f730 0x555b58921a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.838+0000 7f14e77fe640 1 -- 172.21.15.38:0/2663188909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b58570410 msgr2=0x555b5898e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.838+0000 7f14e77fe640 1 --2- 172.21.15.38:0/2663188909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b58570410 0x555b5898e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.838+0000 7f14e77fe640 1 -- 172.21.15.38:0/2663188909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b5898e9a0 con 0x555b5852f1f0 2024-05-21T15:37:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.838+0000 7f14e77fe640 1 --2- 172.21.15.38:0/2663188909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5852f1f0 0x555b589214d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f14dc016040 tx=0x7f14dc02afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb0715b8b035779 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.838+0000 7f14c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:37:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.839+0000 7f14c6ffd640 1 -- 172.21.15.38:0/2663188909 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc02a7b0 con 0x555b5852f1f0 2024-05-21T15:37:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.839+0000 7f14f4bc9740 1 -- 172.21.15.38:0/2663188909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d8020040 con 0x555b5852f1f0 2024-05-21T15:37:31.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.839+0000 7f14c6ffd640 1 -- 172.21.15.38:0/2663188909 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f14dc02a950 con 0x555b5852f1f0 2024-05-21T15:37:31.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.842+0000 7f14c6ffd640 1 -- 172.21.15.38:0/2663188909 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f14dc0036b0 con 0x555b5852f1f0 2024-05-21T15:37:31.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.844+0000 7f14f4bc9740 10 client.16359 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:31.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.844+0000 7f14f4bc9740 1 -- 172.21.15.38:0/2663188909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555b5898fcd0 con 0x555b5852f1f0 2024-05-21T15:37:31.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.845+0000 7f14c6ffd640 1 -- 172.21.15.38:0/2663188909 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d8020040 con 0x555b5852f1f0 2024-05-21T15:37:31.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.845+0000 7f14c6ffd640 1 -- 172.21.15.38:0/2663188909 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f14dc021040 con 0x555b5852f1f0 2024-05-21T15:37:31.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.845+0000 7f14f4bc9740 10 client.16359 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:31.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.845+0000 7f14f4bc9740 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:31.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.845+0000 7f14f4bc9740 20 client.16359 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:31.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.845+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/2663188909 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555b58504630 0x555b58992720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:31.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.845+0000 7f14f4bc9740 4 client.16359 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:37:31.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.845+0000 7f14f4bc9740 1 -- 172.21.15.38:0/2663188909 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555b588820c0 con 0x555b58504630 2024-05-21T15:37:31.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.845+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/2663188909 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555b58504630 0x555b58992720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:31.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.848+0000 7f14e6ffd640 1 --2- 172.21.15.38:0/2663188909 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555b58504630 0x555b58992720 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=741dc3c34bc9605d server_cookie=3a8342bd95190298 in_seq=0 out_seq=0 2024-05-21T15:37:31.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.848+0000 7f14c6ffd640 10 client.16359 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:31.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.850+0000 7f14c6ffd640 1 -- 172.21.15.38:0/2663188909 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555b588820c0 con 0x555b58504630 2024-05-21T15:37:31.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.850+0000 7f14c6ffd640 10 client.16359 handle_command_reply: tid=0 2024-05-21T15:37:31.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.850+0000 7f14f4bc9740 1 client.16359 shutdown 2024-05-21T15:37:31.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.852+0000 7f14f4bc9740 1 -- 172.21.15.38:0/2663188909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5852f1f0 msgr2=0x555b589214d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:31.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.852+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/2663188909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5852f1f0 0x555b589214d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f14dc016040 tx=0x7f14dc02afc0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.853+0000 7f14f4bc9740 1 -- 172.21.15.38:0/2663188909 shutdown_connections 2024-05-21T15:37:31.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.853+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/2663188909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555b58570410 0x555b5898e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.853+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/2663188909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b5854f730 0x555b58921a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.853+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/2663188909 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555b58504630 0x555b58992720 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.853+0000 7f14f4bc9740 1 --2- 172.21.15.38:0/2663188909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555b5852f1f0 0x555b589214d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.853+0000 7f14f4bc9740 1 -- 172.21.15.38:0/2663188909 >> 172.21.15.38:0/2663188909 conn(0x555b58564560 msgr2=0x555b589192f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:31.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.853+0000 7f14c6ffd640 0 client.16359 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:31.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.859+0000 7f14f4bc9740 1 -- 172.21.15.38:0/2663188909 shutdown_connections 2024-05-21T15:37:31.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.862+0000 7f14f4bc9740 1 -- 172.21.15.38:0/2663188909 wait complete. 2024-05-21T15:37:31.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.863+0000 7f14f4bc9740 20 client.16359 trim_cache size 0 max 16384 2024-05-21T15:37:31.865 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:37:31.865 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-05-21T15:37:31.865 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "c10e3c7e-cde3-4123-a8a8-47a3fb376fc2", 2024-05-21T15:37:31.865 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-05-21T15:37:31.865 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:37:31.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.869+0000 7f14edd21640 1 -- 172.21.15.38:0/4249402551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f14d00a2cd0 msgr2=0x7f14d00a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:31.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.869+0000 7f14edd21640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f14d00a2cd0 0x7f14d00a51e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x555b58911880 tx=0x7f14dc009190 comp rx=0 tx=0).stop 2024-05-21T15:37:31.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.870+0000 7f14edd21640 1 -- 172.21.15.38:0/4249402551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e8107900 msgr2=0x7f14e8072cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:31.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.870+0000 7f14edd21640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e8107900 0x7f14e8072cd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009bf0 tx=0x7f14e0039610 comp rx=0 tx=0).stop 2024-05-21T15:37:31.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.870+0000 7f14edd21640 1 -- 172.21.15.38:0/4249402551 shutdown_connections 2024-05-21T15:37:31.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.870+0000 7f14edd21640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e8107900 0x7f14e8072cd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.870+0000 7f14edd21640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f14d00a2cd0 0x7f14d00a51e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.871+0000 7f14edd21640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14e8102c00 0x7f14e8071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.871+0000 7f14edd21640 1 --2- 172.21.15.38:0/4249402551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e81019c0 0x7f14e8071830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:31.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.871+0000 7f14edd21640 1 -- 172.21.15.38:0/4249402551 >> 172.21.15.38:0/4249402551 conn(0x7f14e80fcbb0 msgr2=0x7f14e80fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:31.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.873+0000 7f14edd21640 1 -- 172.21.15.38:0/4249402551 shutdown_connections 2024-05-21T15:37:31.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:31.873+0000 7f14edd21640 1 -- 172.21.15.38:0/4249402551 wait complete. 2024-05-21T15:37:31.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:37:31.898 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:37:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.115+0000 7fa294702640 1 -- 172.21.15.38:0/1700950727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa28c102ae0 msgr2=0x7fa28c1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.115+0000 7fa294702640 1 --2- 172.21.15.38:0/1700950727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa28c102ae0 0x7fa28c1037e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa288009900 tx=0x7fa28802a400 comp rx=0 tx=0).stop 2024-05-21T15:37:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.115+0000 7fa294702640 1 -- 172.21.15.38:0/1700950727 shutdown_connections 2024-05-21T15:37:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.115+0000 7fa294702640 1 --2- 172.21.15.38:0/1700950727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa28c108a20 0x7fa28c10ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.115+0000 7fa294702640 1 --2- 172.21.15.38:0/1700950727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa28c103d20 0x7fa28c104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.115+0000 7fa294702640 1 --2- 172.21.15.38:0/1700950727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa28c102ae0 0x7fa28c1037e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.115+0000 7fa294702640 1 -- 172.21.15.38:0/1700950727 >> 172.21.15.38:0/1700950727 conn(0x7fa28c0fdcf0 msgr2=0x7fa28c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.116+0000 7fa294702640 1 -- 172.21.15.38:0/1700950727 shutdown_connections 2024-05-21T15:37:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.116+0000 7fa294702640 1 -- 172.21.15.38:0/1700950727 wait complete. 2024-05-21T15:37:32.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.117+0000 7fa294702640 1 Processor -- start 2024-05-21T15:37:32.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.117+0000 7fa294702640 1 -- start start 2024-05-21T15:37:32.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.117+0000 7fa294702640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa28c102ae0 0x7fa28c197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.117+0000 7fa294702640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa28c103d20 0x7fa28c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.117+0000 7fa294702640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa28c108a20 0x7fa28c1a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.117+0000 7fa294702640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c10e760 con 0x7fa28c108a20 2024-05-21T15:37:32.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.117+0000 7fa294702640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c10e5e0 con 0x7fa28c102ae0 2024-05-21T15:37:32.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.117+0000 7fa294702640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c10e8e0 con 0x7fa28c103d20 2024-05-21T15:37:32.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.117+0000 7fa292c9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa28c108a20 0x7fa28c1a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.118+0000 7fa292c9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa28c108a20 0x7fa28c1a6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41054/0 (socket says 172.21.15.38:41054) 2024-05-21T15:37:32.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.118+0000 7fa292c9f640 1 -- 172.21.15.38:0/2371727220 learned_addr learned my addr 172.21.15.38:0/2371727220 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:32.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.118+0000 7fa291c9d640 1 --2- 172.21.15.38:0/2371727220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa28c103d20 0x7fa28c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.118+0000 7fa292c9f640 1 -- 172.21.15.38:0/2371727220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa28c103d20 msgr2=0x7fa28c197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:32.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.118+0000 7fa292c9f640 1 --2- 172.21.15.38:0/2371727220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa28c103d20 0x7fa28c197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.118+0000 7fa292c9f640 1 -- 172.21.15.38:0/2371727220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa28c102ae0 msgr2=0x7fa28c197140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T15:37:32.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.118+0000 7fa292c9f640 1 --2- 172.21.15.38:0/2371727220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa28c102ae0 0x7fa28c197140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.118+0000 7fa292c9f640 1 -- 172.21.15.38:0/2371727220 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28c1a7490 con 0x7fa28c108a20 2024-05-21T15:37:32.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.119+0000 7fa292c9f640 1 --2- 172.21.15.38:0/2371727220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa28c108a20 0x7fa28c1a6df0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00d750 tx=0x7fa27c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.120+0000 7fa2837fe640 1 -- 172.21.15.38:0/2371727220 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c017070 con 0x7fa28c108a20 2024-05-21T15:37:32.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.121+0000 7fa2837fe640 1 -- 172.21.15.38:0/2371727220 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa27c005500 con 0x7fa28c108a20 2024-05-21T15:37:32.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.121+0000 7fa2837fe640 1 -- 172.21.15.38:0/2371727220 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c004d60 con 0x7fa28c108a20 2024-05-21T15:37:32.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.121+0000 7fa294702640 1 -- 172.21.15.38:0/2371727220 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28c1a7780 con 0x7fa28c108a20 2024-05-21T15:37:32.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.121+0000 7fa294702640 1 -- 172.21.15.38:0/2371727220 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa28c1a7c80 con 0x7fa28c108a20 2024-05-21T15:37:32.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.125+0000 7fa29b5ad740 1 Processor -- start 2024-05-21T15:37:32.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.127+0000 7fa2837fe640 1 -- 172.21.15.38:0/2371727220 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa27c004f00 con 0x7fa28c108a20 2024-05-21T15:37:32.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.127+0000 7fa29b5ad740 1 -- start start 2024-05-21T15:37:32.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.127+0000 7fa29b5ad740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739ab390 0x559973d4abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.128+0000 7fa29b5ad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599739824d0 0x559973d4b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.128+0000 7fa29b5ad740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55997398a6b0 0x559973d52030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.128+0000 7fa29b5ad740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559973cafd00 con 0x55997398a6b0 2024-05-21T15:37:32.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.128+0000 7fa29b5ad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559973c574c0 con 0x5599739824d0 2024-05-21T15:37:32.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.128+0000 7fa29b5ad740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559973cabcc0 con 0x5599739ab390 2024-05-21T15:37:32.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.128+0000 7fa291c9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739ab390 0x559973d4abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.128+0000 7fa291c9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739ab390 0x559973d4abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46808/0 (socket says 172.21.15.38:46808) 2024-05-21T15:37:32.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.128+0000 7fa291c9d640 1 -- 172.21.15.38:0/4285930339 learned_addr learned my addr 172.21.15.38:0/4285930339 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:32.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.128+0000 7fa29249e640 1 --2- 172.21.15.38:0/4285930339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599739824d0 0x559973d4b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.128+0000 7fa291c9d640 1 --2- 172.21.15.38:0/4285930339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55997398a6b0 0x559973d52030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.129+0000 7fa29249e640 1 -- 172.21.15.38:0/4285930339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739ab390 msgr2=0x559973d4abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:32.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.129+0000 7fa29249e640 1 --2- 172.21.15.38:0/4285930339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739ab390 0x559973d4abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.129+0000 7fa29249e640 1 -- 172.21.15.38:0/4285930339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55997398a6b0 msgr2=0x559973d52030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:32.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.129+0000 7fa29249e640 1 --2- 172.21.15.38:0/4285930339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55997398a6b0 0x559973d52030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.129+0000 7fa29249e640 1 -- 172.21.15.38:0/4285930339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559973a8d7d0 con 0x5599739824d0 2024-05-21T15:37:32.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.129+0000 7fa29249e640 1 --2- 172.21.15.38:0/4285930339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599739824d0 0x559973d4b130 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa288009230 tx=0x7fa2880314d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3398ca5719b5ba5f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:32.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.129+0000 7fa2837fe640 1 --2- 172.21.15.38:0/2371727220 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2780a2d10 0x7fa2780a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.130+0000 7fa291c9d640 1 --2- 172.21.15.38:0/2371727220 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2780a2d10 0x7fa2780a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.130+0000 7fa280ff9640 1 -- 172.21.15.38:0/4285930339 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28803e070 con 0x5599739824d0 2024-05-21T15:37:32.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.130+0000 7fa280ff9640 1 -- 172.21.15.38:0/4285930339 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa288005530 con 0x5599739824d0 2024-05-21T15:37:32.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.130+0000 7fa2837fe640 1 -- 172.21.15.38:0/2371727220 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa27c0cecd0 con 0x7fa28c108a20 2024-05-21T15:37:32.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.130+0000 7fa280ff9640 1 -- 172.21.15.38:0/4285930339 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288035040 con 0x5599739824d0 2024-05-21T15:37:32.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.131+0000 7fa29b5ad740 1 -- 172.21.15.38:0/4285930339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599739824d0 msgr2=0x559973d4b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:32.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.131+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/4285930339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599739824d0 0x559973d4b130 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa288009230 tx=0x7fa2880314d0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.132+0000 7fa29b5ad740 1 -- 172.21.15.38:0/4285930339 shutdown_connections 2024-05-21T15:37:32.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.132+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/4285930339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55997398a6b0 0x559973d52030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.132+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/4285930339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599739824d0 0x559973d4b130 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.132+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/4285930339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739ab390 0x559973d4abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.132+0000 7fa29b5ad740 1 -- 172.21.15.38:0/4285930339 >> 172.21.15.38:0/4285930339 conn(0x55997399f4e0 msgr2=0x559973d46540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.132+0000 7fa291c9d640 1 --2- 172.21.15.38:0/2371727220 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2780a2d10 0x7fa2780a5220 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa284009fd0 tx=0x7fa284009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.132+0000 7fa29b5ad740 1 -- 172.21.15.38:0/4285930339 shutdown_connections 2024-05-21T15:37:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.132+0000 7fa29b5ad740 1 -- 172.21.15.38:0/4285930339 wait complete. 2024-05-21T15:37:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.133+0000 7fa29b5ad740 1 -- start start 2024-05-21T15:37:32.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.133+0000 7fa29b5ad740 1 Processor -- start 2024-05-21T15:37:32.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.133+0000 7fa29b5ad740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739824d0 0x559973d54ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.133+0000 7fa29b5ad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997398a6b0 0x559973d55000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.134+0000 7fa29b5ad740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599739ab390 0x559973d5bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.134+0000 7fa29b5ad740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559973d52a70 con 0x5599739ab390 2024-05-21T15:37:32.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.134+0000 7fa29b5ad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559973c6ab90 con 0x55997398a6b0 2024-05-21T15:37:32.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.134+0000 7fa29b5ad740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559973d52bf0 con 0x5599739824d0 2024-05-21T15:37:32.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.134+0000 7fa29b5ad740 10 client.0 _ll_register_callbacks cb 0x559973ca6da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:32.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.134+0000 7fa292c9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599739ab390 0x559973d5bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.134+0000 7fa292c9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599739ab390 0x559973d5bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41068/0 (socket says 172.21.15.38:41068) 2024-05-21T15:37:32.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.134+0000 7fa292c9f640 1 -- 172.21.15.38:0/1370736931 learned_addr learned my addr 172.21.15.38:0/1370736931 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:32.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.134+0000 7fa29249e640 1 --2- 172.21.15.38:0/1370736931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739824d0 0x559973d54ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.135+0000 7fa292c9f640 1 -- 172.21.15.38:0/1370736931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739824d0 msgr2=0x559973d54ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:32.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.135+0000 7fa292c9f640 1 --2- 172.21.15.38:0/1370736931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739824d0 0x559973d54ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.135+0000 7fa292c9f640 1 -- 172.21.15.38:0/1370736931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997398a6b0 msgr2=0x559973d55000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:37:32.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.135+0000 7fa292c9f640 1 --2- 172.21.15.38:0/1370736931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997398a6b0 0x559973d55000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.135+0000 7fa292c9f640 1 -- 172.21.15.38:0/1370736931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599739c33f0 con 0x5599739ab390 2024-05-21T15:37:32.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.136+0000 7fa292c9f640 1 --2- 172.21.15.38:0/1370736931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599739ab390 0x559973d5bf60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009fa0 tx=0x7fa27c00df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa0fd42c625501f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:32.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.136+0000 7fa275ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:37:32.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.136+0000 7fa275ffb640 1 -- 172.21.15.38:0/1370736931 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0b4070 con 0x5599739ab390 2024-05-21T15:37:32.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.136+0000 7fa275ffb640 1 -- 172.21.15.38:0/1370736931 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa27c0051f0 con 0x5599739ab390 2024-05-21T15:37:32.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.137+0000 7fa29b5ad740 1 -- 172.21.15.38:0/1370736931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa288005530 con 0x5599739ab390 2024-05-21T15:37:32.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.137+0000 7fa275ffb640 1 -- 172.21.15.38:0/1370736931 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa27c09f910 con 0x5599739ab390 2024-05-21T15:37:32.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.137+0000 7fa29b5ad740 10 client.16371 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:32.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.137+0000 7fa29b5ad740 1 -- 172.21.15.38:0/1370736931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5599739830f0 con 0x5599739ab390 2024-05-21T15:37:32.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.138+0000 7fa275ffb640 1 -- 172.21.15.38:0/1370736931 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa288005530 con 0x5599739ab390 2024-05-21T15:37:32.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.139+0000 7fa275ffb640 1 -- 172.21.15.38:0/1370736931 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fa27c0bb360 con 0x5599739ab390 2024-05-21T15:37:32.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.139+0000 7fa29b5ad740 10 client.16371 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:32.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.139+0000 7fa29b5ad740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:32.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.139+0000 7fa29b5ad740 20 client.16371 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:32.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.139+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/1370736931 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x559973969f60 0x559973d628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.139+0000 7fa29b5ad740 4 client.16371 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:37:32.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.139+0000 7fa29b5ad740 1 -- 172.21.15.38:0/1370736931 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559973cbd430 con 0x559973969f60 2024-05-21T15:37:32.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.140+0000 7fa29249e640 1 --2- 172.21.15.38:0/1370736931 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x559973969f60 0x559973d628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.143+0000 7fa29249e640 1 --2- 172.21.15.38:0/1370736931 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x559973969f60 0x559973d628e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95093458ca751125 server_cookie=5c7f7a33f8635482 in_seq=0 out_seq=0 2024-05-21T15:37:32.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.143+0000 7fa275ffb640 10 client.16371 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:32.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.144+0000 7fa275ffb640 1 -- 172.21.15.38:0/1370736931 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559973cbd430 con 0x559973969f60 2024-05-21T15:37:32.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.144+0000 7fa275ffb640 10 client.16371 handle_command_reply: tid=0 2024-05-21T15:37:32.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.144+0000 7fa29b5ad740 1 client.16371 shutdown 2024-05-21T15:37:32.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.145+0000 7fa29b5ad740 1 -- 172.21.15.38:0/1370736931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599739ab390 msgr2=0x559973d5bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:32.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.145+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/1370736931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599739ab390 0x559973d5bf60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009fa0 tx=0x7fa27c00df50 comp rx=0 tx=0).stop 2024-05-21T15:37:32.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.145+0000 7fa29b5ad740 1 -- 172.21.15.38:0/1370736931 shutdown_connections 2024-05-21T15:37:32.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.145+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/1370736931 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x559973969f60 0x559973d628e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.145+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/1370736931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599739ab390 0x559973d5bf60 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.145+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/1370736931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997398a6b0 0x559973d55000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.145+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/1370736931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739824d0 0x559973d54ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.145+0000 7fa29b5ad740 1 -- 172.21.15.38:0/1370736931 >> 172.21.15.38:0/1370736931 conn(0x55997399f4e0 msgr2=0x559973d455c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:32.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.145+0000 7fa275ffb640 0 client.16371 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:32.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.146+0000 7fa29b5ad740 1 -- 172.21.15.38:0/1370736931 shutdown_connections 2024-05-21T15:37:32.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.146+0000 7fa29b5ad740 1 -- 172.21.15.38:0/1370736931 wait complete. 2024-05-21T15:37:32.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.147+0000 7fa29b5ad740 20 client.16371 trim_cache size 0 max 16384 2024-05-21T15:37:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.186+0000 7fa29b5ad740 1 Processor -- start 2024-05-21T15:37:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.186+0000 7fa29b5ad740 1 -- start start 2024-05-21T15:37:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.186+0000 7fa29b5ad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997399f4e0 0x559973d4f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.186+0000 7fa29b5ad740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739824d0 0x559973d5be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.186+0000 7fa29b5ad740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55997398a6b0 0x559973d54ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.186+0000 7fa29249e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997399f4e0 0x559973d4f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.186+0000 7fa29b5ad740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c1b0a40 con 0x55997398a6b0 2024-05-21T15:37:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.186+0000 7fa29b5ad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c1b2670 con 0x55997399f4e0 2024-05-21T15:37:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.186+0000 7fa29b5ad740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c1b1110 con 0x5599739824d0 2024-05-21T15:37:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.186+0000 7fa29249e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997399f4e0 0x559973d4f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45472/0 (socket says 172.21.15.38:45472) 2024-05-21T15:37:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.186+0000 7fa29249e640 1 -- 172.21.15.38:0/2273768994 learned_addr learned my addr 172.21.15.38:0/2273768994 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.187+0000 7fa292c9f640 1 --2- 172.21.15.38:0/2273768994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55997398a6b0 0x559973d54ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.187+0000 7fa292c9f640 1 -- 172.21.15.38:0/2273768994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739824d0 msgr2=0x559973d5be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.187+0000 7fa292c9f640 1 --2- 172.21.15.38:0/2273768994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739824d0 0x559973d5be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.187+0000 7fa292c9f640 1 -- 172.21.15.38:0/2273768994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997399f4e0 msgr2=0x559973d4f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.187+0000 7fa292c9f640 1 --2- 172.21.15.38:0/2273768994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997399f4e0 0x559973d4f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.187+0000 7fa292c9f640 1 -- 172.21.15.38:0/2273768994 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559973d4bdb0 con 0x55997398a6b0 2024-05-21T15:37:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.187+0000 7fa29249e640 1 --2- 172.21.15.38:0/2273768994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997399f4e0 0x559973d4f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:37:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.188+0000 7fa292c9f640 1 --2- 172.21.15.38:0/2273768994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55997398a6b0 0x559973d54ac0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa27c097240 tx=0x7fa27c099a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab13cff01724a80c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.189+0000 7fa277fff640 1 -- 172.21.15.38:0/2273768994 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0b4040 con 0x55997398a6b0 2024-05-21T15:37:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.189+0000 7fa277fff640 1 -- 172.21.15.38:0/2273768994 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa27c0a41d0 con 0x55997398a6b0 2024-05-21T15:37:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.189+0000 7fa277fff640 1 -- 172.21.15.38:0/2273768994 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0a9080 con 0x55997398a6b0 2024-05-21T15:37:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.190+0000 7fa29b5ad740 1 -- 172.21.15.38:0/2273768994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55997398a6b0 msgr2=0x559973d54ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:32.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.190+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/2273768994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55997398a6b0 0x559973d54ac0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa27c097240 tx=0x7fa27c099a70 comp rx=0 tx=0).stop 2024-05-21T15:37:32.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.194+0000 7fa29b5ad740 1 -- 172.21.15.38:0/2273768994 shutdown_connections 2024-05-21T15:37:32.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.194+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/2273768994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55997398a6b0 0x559973d54ac0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.194+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/2273768994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5599739824d0 0x559973d5be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.194+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/2273768994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997399f4e0 0x559973d4f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.194+0000 7fa29b5ad740 1 -- 172.21.15.38:0/2273768994 >> 172.21.15.38:0/2273768994 conn(0x559973d55290 msgr2=0x559973d49340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:32.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.194+0000 7fa29b5ad740 1 -- 172.21.15.38:0/2273768994 shutdown_connections 2024-05-21T15:37:32.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.194+0000 7fa29b5ad740 1 -- 172.21.15.38:0/2273768994 wait complete. 2024-05-21T15:37:32.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.195+0000 7fa29b5ad740 1 -- start start 2024-05-21T15:37:32.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.195+0000 7fa29b5ad740 1 Processor -- start 2024-05-21T15:37:32.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.196+0000 7fa29b5ad740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599739824d0 0x559973d5aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.196+0000 7fa29b5ad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997398a6b0 0x559973d5b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.196+0000 7fa29b5ad740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997399f4e0 0x559973dc8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.197+0000 7fa29b5ad740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c1b1ee0 con 0x5599739824d0 2024-05-21T15:37:32.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.197+0000 7fa29b5ad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c1b17e0 con 0x55997398a6b0 2024-05-21T15:37:32.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.197+0000 7fa29b5ad740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c1aea20 con 0x55997399f4e0 2024-05-21T15:37:32.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.197+0000 7fa292c9f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997399f4e0 0x559973dc8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.197+0000 7fa292c9f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997399f4e0 0x559973dc8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46824/0 (socket says 172.21.15.38:46824) 2024-05-21T15:37:32.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.197+0000 7fa292c9f640 1 -- 172.21.15.38:0/818819841 learned_addr learned my addr 172.21.15.38:0/818819841 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:32.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.197+0000 7fa29b5ad740 10 client.0 _ll_register_callbacks cb 0x7fa27c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:32.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.199+0000 7fa29249e640 1 --2- 172.21.15.38:0/818819841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599739824d0 0x559973d5aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.201+0000 7fa292c9f640 1 -- 172.21.15.38:0/818819841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997398a6b0 msgr2=0x559973d5b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:32.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.201+0000 7fa292c9f640 1 --2- 172.21.15.38:0/818819841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997398a6b0 0x559973d5b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.201+0000 7fa292c9f640 1 -- 172.21.15.38:0/818819841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599739824d0 msgr2=0x559973d5aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:32.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.201+0000 7fa292c9f640 1 --2- 172.21.15.38:0/818819841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599739824d0 0x559973d5aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.201+0000 7fa292c9f640 1 -- 172.21.15.38:0/818819841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559973dc9250 con 0x55997399f4e0 2024-05-21T15:37:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.202+0000 7fa292c9f640 1 --2- 172.21.15.38:0/818819841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997399f4e0 0x559973dc8c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa27c09b260 tx=0x7fa27c009e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a15f5698eee1c44c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.202+0000 7fa29249e640 1 --2- 172.21.15.38:0/818819841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599739824d0 0x559973d5aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:37:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.202+0000 7fa275ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:37:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.202+0000 7fa275ffb640 1 -- 172.21.15.38:0/818819841 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0b4040 con 0x55997399f4e0 2024-05-21T15:37:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.203+0000 7fa29b5ad740 1 -- 172.21.15.38:0/818819841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa27c0a41d0 con 0x55997399f4e0 2024-05-21T15:37:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.203+0000 7fa275ffb640 1 -- 172.21.15.38:0/818819841 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa27c097300 con 0x55997399f4e0 2024-05-21T15:37:32.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.203+0000 7fa275ffb640 1 -- 172.21.15.38:0/818819841 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa27c081050 con 0x55997399f4e0 2024-05-21T15:37:32.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.204+0000 7fa29b5ad740 10 client.26086 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:32.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.204+0000 7fa29b5ad740 1 -- 172.21.15.38:0/818819841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559973dca4e0 con 0x55997399f4e0 2024-05-21T15:37:32.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.204+0000 7fa275ffb640 1 -- 172.21.15.38:0/818819841 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa27c097500 con 0x55997399f4e0 2024-05-21T15:37:32.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.205+0000 7fa275ffb640 1 -- 172.21.15.38:0/818819841 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fa27c0a0aa0 con 0x55997399f4e0 2024-05-21T15:37:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.206+0000 7fa29b5ad740 10 client.26086 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.206+0000 7fa29b5ad740 10 client.26086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.206+0000 7fa29b5ad740 20 client.26086 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.206+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/818819841 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5599739ab390 0x559973dcce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.206+0000 7fa29b5ad740 4 client.26086 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:37:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.206+0000 7fa29b5ad740 1 -- 172.21.15.38:0/818819841 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa27c0a41d0 con 0x5599739ab390 2024-05-21T15:37:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.206+0000 7fa29249e640 1 --2- 172.21.15.38:0/818819841 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5599739ab390 0x559973dcce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:31 smithi053 ceph-mon[36496]: pgmap v444: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 191 MiB/s wr, 21.11k op/s 2024-05-21T15:37:32.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.209+0000 7fa29249e640 1 --2- 172.21.15.38:0/818819841 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5599739ab390 0x559973dcce40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12b7cc17f6919bca server_cookie=30a93696a7d98037 in_seq=0 out_seq=0 2024-05-21T15:37:32.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.209+0000 7fa275ffb640 10 client.26086 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:32.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.211+0000 7fa275ffb640 1 -- 172.21.15.38:0/818819841 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x7fa27c0a41d0 con 0x5599739ab390 2024-05-21T15:37:32.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.211+0000 7fa275ffb640 10 client.26086 handle_command_reply: tid=0 2024-05-21T15:37:32.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.211+0000 7fa29b5ad740 1 client.26086 shutdown 2024-05-21T15:37:32.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.215+0000 7fa29b5ad740 1 -- 172.21.15.38:0/818819841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997399f4e0 msgr2=0x559973dc8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:32.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.215+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/818819841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997399f4e0 0x559973dc8c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa27c09b260 tx=0x7fa27c009e70 comp rx=0 tx=0).stop 2024-05-21T15:37:32.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.215+0000 7fa29b5ad740 1 -- 172.21.15.38:0/818819841 shutdown_connections 2024-05-21T15:37:32.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.215+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/818819841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55997399f4e0 0x559973dc8c40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.215+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/818819841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55997398a6b0 0x559973d5b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.215+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/818819841 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5599739ab390 0x559973dcce40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.215+0000 7fa29b5ad740 1 --2- 172.21.15.38:0/818819841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5599739824d0 0x559973d5aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.215+0000 7fa29b5ad740 1 -- 172.21.15.38:0/818819841 >> 172.21.15.38:0/818819841 conn(0x559973d55290 msgr2=0x559973d49d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:32.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.215+0000 7fa275ffb640 0 client.26086 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:32.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.216+0000 7fa29b5ad740 1 -- 172.21.15.38:0/818819841 shutdown_connections 2024-05-21T15:37:32.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.216+0000 7fa29b5ad740 1 -- 172.21.15.38:0/818819841 wait complete. 2024-05-21T15:37:32.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.218+0000 7fa29b5ad740 20 client.26086 trim_cache size 0 max 16384 2024-05-21T15:37:32.220 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:37:32.220 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "scrub active (83 inodes in the stack)", 2024-05-21T15:37:32.220 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-05-21T15:37:32.220 INFO:teuthology.orchestra.run.smithi038.stdout: "c10e3c7e-cde3-4123-a8a8-47a3fb376fc2": { 2024-05-21T15:37:32.221 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-05-21T15:37:32.221 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "c10e3c7e-cde3-4123-a8a8-47a3fb376fc2", 2024-05-21T15:37:32.221 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-05-21T15:37:32.221 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:37:32.221 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:37:32.221 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:37:32.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.227+0000 7fa294702640 1 -- 172.21.15.38:0/2371727220 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2780a2d10 msgr2=0x7fa2780a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:32.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.227+0000 7fa294702640 1 --2- 172.21.15.38:0/2371727220 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2780a2d10 0x7fa2780a5220 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa284009fd0 tx=0x7fa284009680 comp rx=0 tx=0).stop 2024-05-21T15:37:32.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.227+0000 7fa294702640 1 -- 172.21.15.38:0/2371727220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa28c108a20 msgr2=0x7fa28c1a6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:32.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.227+0000 7fa294702640 1 --2- 172.21.15.38:0/2371727220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa28c108a20 0x7fa28c1a6df0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00d750 tx=0x7fa27c00dc10 comp rx=0 tx=0).stop 2024-05-21T15:37:32.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.231+0000 7fa294702640 1 -- 172.21.15.38:0/2371727220 shutdown_connections 2024-05-21T15:37:32.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.231+0000 7fa294702640 1 --2- 172.21.15.38:0/2371727220 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa2780a2d10 0x7fa2780a5220 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.231+0000 7fa294702640 1 --2- 172.21.15.38:0/2371727220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa28c108a20 0x7fa28c1a6df0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.231+0000 7fa294702640 1 --2- 172.21.15.38:0/2371727220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa28c103d20 0x7fa28c197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.231+0000 7fa294702640 1 --2- 172.21.15.38:0/2371727220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa28c102ae0 0x7fa28c197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.231+0000 7fa294702640 1 -- 172.21.15.38:0/2371727220 >> 172.21.15.38:0/2371727220 conn(0x7fa28c0fdcf0 msgr2=0x7fa28c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:32.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.233+0000 7fa294702640 1 -- 172.21.15.38:0/2371727220 shutdown_connections 2024-05-21T15:37:32.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:37:32.233+0000 7fa294702640 1 -- 172.21.15.38:0/2371727220 wait complete. 2024-05-21T15:37:32.257 INFO:tasks.cephfs.filesystem:scrub status for tag:c10e3c7e-cde3-4123-a8a8-47a3fb376fc2 - {'path': '/', 'tag': 'c10e3c7e-cde3-4123-a8a8-47a3fb376fc2', 'options': 'recursive,force'} 2024-05-21T15:37:32.257 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:31 smithi038 ceph-mon[31995]: pgmap v444: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 191 MiB/s wr, 21.11k op/s 2024-05-21T15:37:32.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:31 smithi045 ceph-mon[36409]: pgmap v444: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 191 MiB/s wr, 21.11k op/s 2024-05-21T15:37:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.404+0000 7ff9991a4640 1 -- 172.21.15.38:0/2699752535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff994100560 msgr2=0x7ff9941009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.404+0000 7ff9991a4640 1 --2- 172.21.15.38:0/2699752535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff994100560 0x7ff9941009d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff984009900 tx=0x7ff98402a410 comp rx=0 tx=0).stop 2024-05-21T15:37:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.405+0000 7ff9991a4640 1 -- 172.21.15.38:0/2699752535 shutdown_connections 2024-05-21T15:37:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.405+0000 7ff9991a4640 1 --2- 172.21.15.38:0/2699752535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff994101060 0x7ff99410aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.405+0000 7ff9991a4640 1 --2- 172.21.15.38:0/2699752535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff994100560 0x7ff9941009d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.405+0000 7ff9991a4640 1 --2- 172.21.15.38:0/2699752535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff994103eb0 0x7ff9941042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.405+0000 7ff9991a4640 1 -- 172.21.15.38:0/2699752535 >> 172.21.15.38:0/2699752535 conn(0x7ff9940fc1a0 msgr2=0x7ff9940fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.408+0000 7ff9991a4640 1 -- 172.21.15.38:0/2699752535 shutdown_connections 2024-05-21T15:37:32.410 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.410+0000 7ff9991a4640 1 -- 172.21.15.38:0/2699752535 wait complete. 2024-05-21T15:37:32.410 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.411+0000 7ff9991a4640 1 Processor -- start 2024-05-21T15:37:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.413+0000 7ff9991a4640 1 -- start start 2024-05-21T15:37:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.413+0000 7ff9991a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff994100560 0x7ff9941138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.413+0000 7ff9991a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff994101060 0x7ff994111fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.413+0000 7ff9991a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff994103eb0 0x7ff9941124e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.413+0000 7ff9991a4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff99410c400 con 0x7ff994101060 2024-05-21T15:37:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.413+0000 7ff9991a4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff99410c280 con 0x7ff994103eb0 2024-05-21T15:37:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.413+0000 7ff9991a4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff99410c580 con 0x7ff994100560 2024-05-21T15:37:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.415+0000 7ff9989a3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff994103eb0 0x7ff9941124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.415+0000 7ff9989a3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff994103eb0 0x7ff9941124e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45506/0 (socket says 172.21.15.38:45506) 2024-05-21T15:37:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.415+0000 7ff9989a3640 1 -- 172.21.15.38:0/1549308287 learned_addr learned my addr 172.21.15.38:0/1549308287 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.415+0000 7ff993fff640 1 --2- 172.21.15.38:0/1549308287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff994100560 0x7ff9941138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.415+0000 7ff9989a3640 1 -- 172.21.15.38:0/1549308287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff994100560 msgr2=0x7ff9941138b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.416+0000 7ff9989a3640 1 --2- 172.21.15.38:0/1549308287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff994100560 0x7ff9941138b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.416+0000 7ff9989a3640 1 -- 172.21.15.38:0/1549308287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff994101060 msgr2=0x7ff994111fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:37:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.416+0000 7ff9989a3640 1 --2- 172.21.15.38:0/1549308287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff994101060 0x7ff994111fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.416+0000 7ff9989a3640 1 -- 172.21.15.38:0/1549308287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff994112a50 con 0x7ff994103eb0 2024-05-21T15:37:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.417+0000 7ff9989a3640 1 --2- 172.21.15.38:0/1549308287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff994103eb0 0x7ff9941124e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00eb10 tx=0x7ff98c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.419+0000 7ff9917fa640 1 -- 172.21.15.38:0/1549308287 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c004b20 con 0x7ff994103eb0 2024-05-21T15:37:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.419+0000 7ff9991a4640 1 -- 172.21.15.38:0/1549308287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff994112d40 con 0x7ff994103eb0 2024-05-21T15:37:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.419+0000 7ff9991a4640 1 -- 172.21.15.38:0/1549308287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff994071890 con 0x7ff994103eb0 2024-05-21T15:37:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.419+0000 7ff9917fa640 1 -- 172.21.15.38:0/1549308287 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff98c004cc0 con 0x7ff994103eb0 2024-05-21T15:37:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.419+0000 7ff9917fa640 1 -- 172.21.15.38:0/1549308287 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c010760 con 0x7ff994103eb0 2024-05-21T15:37:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.427+0000 7ff9917fa640 1 -- 172.21.15.38:0/1549308287 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff98c010900 con 0x7ff994103eb0 2024-05-21T15:37:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.428+0000 7ff9a1389740 1 Processor -- start 2024-05-21T15:37:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.429+0000 7ff9917fa640 1 --2- 172.21.15.38:0/1549308287 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff97c0a2d90 0x7ff97c0a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.429+0000 7ff9917fa640 1 -- 172.21.15.38:0/1549308287 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff98c0cfc80 con 0x7ff994103eb0 2024-05-21T15:37:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.430+0000 7ff9a1389740 1 -- start start 2024-05-21T15:37:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.431+0000 7ff9a1389740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7507f1b0 0x55af7541f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.431+0000 7ff9a1389740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af7541f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.431+0000 7ff9a1389740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7505e4d0 0x55af754264e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.431+0000 7ff9a1389740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55af75383340 con 0x55af7507f1b0 2024-05-21T15:37:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.431+0000 7ff9a1389740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55af7532b290 con 0x55af750562f0 2024-05-21T15:37:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.431+0000 7ff9a1389740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55af7537f300 con 0x55af7505e4d0 2024-05-21T15:37:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.431+0000 7ff9937fe640 1 --2- 172.21.15.38:0/1549308287 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff97c0a2d90 0x7ff97c0a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.431+0000 7ff993fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7507f1b0 0x55af7541f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.431+0000 7ff993fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7507f1b0 0x55af7541f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41106/0 (socket says 172.21.15.38:41106) 2024-05-21T15:37:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.431+0000 7ff993fff640 1 -- 172.21.15.38:0/1546251438 learned_addr learned my addr 172.21.15.38:0/1546251438 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.431+0000 7ff9989a3640 1 --2- 172.21.15.38:0/1546251438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7505e4d0 0x55af754264e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.432+0000 7ff9989a3640 1 -- 172.21.15.38:0/1546251438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 msgr2=0x55af7541f5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:37:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.432+0000 7ff9989a3640 1 --2- 172.21.15.38:0/1546251438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af7541f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.432+0000 7ff9989a3640 1 -- 172.21.15.38:0/1546251438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7507f1b0 msgr2=0x55af7541f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.432+0000 7ff9989a3640 1 --2- 172.21.15.38:0/1546251438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7507f1b0 0x55af7541f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.432+0000 7ff9989a3640 1 -- 172.21.15.38:0/1546251438 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af7534f8e0 con 0x55af7505e4d0 2024-05-21T15:37:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.432+0000 7ff9989a3640 1 --2- 172.21.15.38:0/1546251438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7505e4d0 0x55af754264e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff98c004a40 tx=0x7ff98c005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c165981ae43d474 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.432+0000 7ff97a7fc640 1 -- 172.21.15.38:0/1546251438 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0b4020 con 0x55af7505e4d0 2024-05-21T15:37:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.432+0000 7ff97a7fc640 1 -- 172.21.15.38:0/1546251438 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff98c0a50a0 con 0x55af7505e4d0 2024-05-21T15:37:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.432+0000 7ff97a7fc640 1 -- 172.21.15.38:0/1546251438 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0a4c10 con 0x55af7505e4d0 2024-05-21T15:37:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.434+0000 7ff9a1389740 1 -- 172.21.15.38:0/1546251438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7505e4d0 msgr2=0x55af754264e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.434+0000 7ff9a1389740 1 --2- 172.21.15.38:0/1546251438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7505e4d0 0x55af754264e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff98c004a40 tx=0x7ff98c005840 comp rx=0 tx=0).stop 2024-05-21T15:37:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.434+0000 7ff9937fe640 1 --2- 172.21.15.38:0/1549308287 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff97c0a2d90 0x7ff97c0a52a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff9840096c0 tx=0x7ff984000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.436+0000 7ff9a1389740 1 -- 172.21.15.38:0/1546251438 shutdown_connections 2024-05-21T15:37:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.436+0000 7ff9a1389740 1 --2- 172.21.15.38:0/1546251438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7505e4d0 0x55af754264e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.436+0000 7ff9a1389740 1 --2- 172.21.15.38:0/1546251438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af7541f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.436+0000 7ff9a1389740 1 --2- 172.21.15.38:0/1546251438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7507f1b0 0x55af7541f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.436+0000 7ff9a1389740 1 -- 172.21.15.38:0/1546251438 >> 172.21.15.38:0/1546251438 conn(0x55af75073300 msgr2=0x55af7541a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.441+0000 7ff9a1389740 1 -- 172.21.15.38:0/1546251438 shutdown_connections 2024-05-21T15:37:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.441+0000 7ff9a1389740 1 -- 172.21.15.38:0/1546251438 wait complete. 2024-05-21T15:37:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.442+0000 7ff9a1389740 1 -- start start 2024-05-21T15:37:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.442+0000 7ff9a1389740 1 Processor -- start 2024-05-21T15:37:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.443+0000 7ff9a1389740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af75428dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.443+0000 7ff9a1389740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7505e4d0 0x55af75429300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.443+0000 7ff9a1389740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7507f1b0 0x55af75430260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.443+0000 7ff993fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af75428dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.443+0000 7ff993fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af75428dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45528/0 (socket says 172.21.15.38:45528) 2024-05-21T15:37:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.444+0000 7ff993fff640 1 -- 172.21.15.38:0/486398247 learned_addr learned my addr 172.21.15.38:0/486398247 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.444+0000 7ff9a1389740 1 -- 172.21.15.38:0/486398247 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55af75426f30 con 0x55af7505e4d0 2024-05-21T15:37:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.444+0000 7ff9989a3640 1 --2- 172.21.15.38:0/486398247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7507f1b0 0x55af75430260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.444+0000 7ff9937fe640 1 --2- 172.21.15.38:0/486398247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7505e4d0 0x55af75429300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.444+0000 7ff9a1389740 1 -- 172.21.15.38:0/486398247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55af7533e790 con 0x55af750562f0 2024-05-21T15:37:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.444+0000 7ff9a1389740 1 -- 172.21.15.38:0/486398247 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55af754270b0 con 0x55af7507f1b0 2024-05-21T15:37:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.444+0000 7ff9a1389740 10 client.0 _ll_register_callbacks cb 0x55af75379a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.445+0000 7ff9989a3640 1 -- 172.21.15.38:0/486398247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 msgr2=0x55af75428dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.445+0000 7ff9989a3640 1 --2- 172.21.15.38:0/486398247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af75428dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.445+0000 7ff9989a3640 1 -- 172.21.15.38:0/486398247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7505e4d0 msgr2=0x55af75429300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.445+0000 7ff9989a3640 1 --2- 172.21.15.38:0/486398247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7505e4d0 0x55af75429300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.445+0000 7ff9989a3640 1 -- 172.21.15.38:0/486398247 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af75097210 con 0x55af7507f1b0 2024-05-21T15:37:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.446+0000 7ff9989a3640 1 --2- 172.21.15.38:0/486398247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7507f1b0 0x55af75430260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff98c017040 tx=0x7ff98c0997e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22654eeec8ad8305 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.446+0000 7ff9737fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:37:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.447+0000 7ff9737fe640 1 -- 172.21.15.38:0/486398247 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0ac060 con 0x55af7507f1b0 2024-05-21T15:37:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.447+0000 7ff9a1389740 1 -- 172.21.15.38:0/486398247 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff98c0a50a0 con 0x55af7507f1b0 2024-05-21T15:37:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.447+0000 7ff9737fe640 1 -- 172.21.15.38:0/486398247 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff98c0cdb90 con 0x55af7507f1b0 2024-05-21T15:37:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.454+0000 7ff9737fe640 1 -- 172.21.15.38:0/486398247 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff98c0b4040 con 0x55af7507f1b0 2024-05-21T15:37:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.454+0000 7ff9a1389740 10 client.26098 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.454+0000 7ff9a1389740 1 -- 172.21.15.38:0/486398247 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55af75163620 con 0x55af7507f1b0 2024-05-21T15:37:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.457+0000 7ff9737fe640 1 -- 172.21.15.38:0/486398247 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff98c0a50a0 con 0x55af7507f1b0 2024-05-21T15:37:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.458+0000 7ff9737fe640 1 -- 172.21.15.38:0/486398247 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7ff98c0a6030 con 0x55af7507f1b0 2024-05-21T15:37:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.458+0000 7ff9a1389740 10 client.26098 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.458+0000 7ff9a1389740 10 client.26098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.458+0000 7ff9a1389740 20 client.26098 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.458+0000 7ff9a1389740 1 --2- 172.21.15.38:0/486398247 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55af7503dd80 0x55af75436680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.458+0000 7ff9a1389740 4 client.26098 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:37:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.458+0000 7ff9a1389740 1 -- 172.21.15.38:0/486398247 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af75390a70 con 0x55af7503dd80 2024-05-21T15:37:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.458+0000 7ff993fff640 1 --2- 172.21.15.38:0/486398247 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55af7503dd80 0x55af75436680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.463+0000 7ff993fff640 1 --2- 172.21.15.38:0/486398247 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55af7503dd80 0x55af75436680 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bb08e2199c642f3 server_cookie=2b2e4cfe2cc8eb03 in_seq=0 out_seq=0 2024-05-21T15:37:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.463+0000 7ff9737fe640 10 client.26098 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.465+0000 7ff9737fe640 1 -- 172.21.15.38:0/486398247 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55af75390a70 con 0x55af7503dd80 2024-05-21T15:37:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.465+0000 7ff9737fe640 10 client.26098 handle_command_reply: tid=0 2024-05-21T15:37:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.466+0000 7ff9a1389740 1 client.26098 shutdown 2024-05-21T15:37:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.468+0000 7ff9a1389740 1 -- 172.21.15.38:0/486398247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7507f1b0 msgr2=0x55af75430260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.468+0000 7ff9a1389740 1 --2- 172.21.15.38:0/486398247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7507f1b0 0x55af75430260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff98c017040 tx=0x7ff98c0997e0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.470+0000 7ff9a1389740 1 -- 172.21.15.38:0/486398247 shutdown_connections 2024-05-21T15:37:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.470+0000 7ff9a1389740 1 --2- 172.21.15.38:0/486398247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7507f1b0 0x55af75430260 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.470+0000 7ff9a1389740 1 --2- 172.21.15.38:0/486398247 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55af7503dd80 0x55af75436680 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.470+0000 7ff9a1389740 1 --2- 172.21.15.38:0/486398247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7505e4d0 0x55af75429300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.470+0000 7ff9a1389740 1 --2- 172.21.15.38:0/486398247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af75428dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.470+0000 7ff9a1389740 1 -- 172.21.15.38:0/486398247 >> 172.21.15.38:0/486398247 conn(0x55af75073300 msgr2=0x55af754198d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.470+0000 7ff9737fe640 0 client.26098 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.471+0000 7ff9a1389740 1 -- 172.21.15.38:0/486398247 shutdown_connections 2024-05-21T15:37:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.472+0000 7ff9a1389740 1 -- 172.21.15.38:0/486398247 wait complete. 2024-05-21T15:37:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.473+0000 7ff9a1389740 20 client.26098 trim_cache size 0 max 16384 2024-05-21T15:37:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.508+0000 7ff9a1389740 1 Processor -- start 2024-05-21T15:37:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.508+0000 7ff9a1389740 1 -- start start 2024-05-21T15:37:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.509+0000 7ff9a1389740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af75423210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.509+0000 7ff9a1389740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7505e4d0 0x55af75423750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.509+0000 7ff9a1389740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7507f1b0 0x55af75430180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.509+0000 7ff993fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af75423210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.509+0000 7ff993fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af75423210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45534/0 (socket says 172.21.15.38:45534) 2024-05-21T15:37:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.509+0000 7ff993fff640 1 -- 172.21.15.38:0/3496798546 learned_addr learned my addr 172.21.15.38:0/3496798546 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.509+0000 7ff9a1389740 1 -- 172.21.15.38:0/3496798546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff98000e660 con 0x55af7505e4d0 2024-05-21T15:37:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.510+0000 7ff9a1389740 1 -- 172.21.15.38:0/3496798546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff980010290 con 0x55af750562f0 2024-05-21T15:37:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.510+0000 7ff9989a3640 1 --2- 172.21.15.38:0/3496798546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7507f1b0 0x55af75430180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.510+0000 7ff9a1389740 1 -- 172.21.15.38:0/3496798546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff98000ed30 con 0x55af7507f1b0 2024-05-21T15:37:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.513+0000 7ff993fff640 1 -- 172.21.15.38:0/3496798546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7507f1b0 msgr2=0x55af75430180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.513+0000 7ff993fff640 1 --2- 172.21.15.38:0/3496798546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7507f1b0 0x55af75430180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.513+0000 7ff993fff640 1 -- 172.21.15.38:0/3496798546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7505e4d0 msgr2=0x55af75423750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.513+0000 7ff993fff640 1 --2- 172.21.15.38:0/3496798546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7505e4d0 0x55af75423750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.513+0000 7ff993fff640 1 -- 172.21.15.38:0/3496798546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af754265f0 con 0x55af750562f0 2024-05-21T15:37:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.513+0000 7ff9989a3640 1 --2- 172.21.15.38:0/3496798546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7507f1b0 0x55af75430180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:37:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.513+0000 7ff993fff640 1 --2- 172.21.15.38:0/3496798546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af75423210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c900 tx=0x7ff98800cdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ea6045c30725a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.513+0000 7ff979ffb640 1 -- 172.21.15.38:0/3496798546 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988003e10 con 0x55af750562f0 2024-05-21T15:37:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.514+0000 7ff979ffb640 1 -- 172.21.15.38:0/3496798546 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff988005500 con 0x55af750562f0 2024-05-21T15:37:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.514+0000 7ff979ffb640 1 -- 172.21.15.38:0/3496798546 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98801b6c0 con 0x55af750562f0 2024-05-21T15:37:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.516+0000 7ff9a1389740 1 -- 172.21.15.38:0/3496798546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 msgr2=0x55af75423210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.516+0000 7ff9a1389740 1 --2- 172.21.15.38:0/3496798546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af75423210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c900 tx=0x7ff98800cdc0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.516+0000 7ff9a1389740 1 -- 172.21.15.38:0/3496798546 shutdown_connections 2024-05-21T15:37:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.516+0000 7ff9a1389740 1 --2- 172.21.15.38:0/3496798546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7507f1b0 0x55af75430180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.516+0000 7ff9a1389740 1 --2- 172.21.15.38:0/3496798546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7505e4d0 0x55af75423750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.516+0000 7ff9a1389740 1 --2- 172.21.15.38:0/3496798546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af75423210 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.516+0000 7ff9a1389740 1 -- 172.21.15.38:0/3496798546 >> 172.21.15.38:0/3496798546 conn(0x55af75073300 msgr2=0x55af7541efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.517+0000 7ff9a1389740 1 -- 172.21.15.38:0/3496798546 shutdown_connections 2024-05-21T15:37:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.517+0000 7ff9a1389740 1 -- 172.21.15.38:0/3496798546 wait complete. 2024-05-21T15:37:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.518+0000 7ff9a1389740 1 -- start start 2024-05-21T15:37:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.518+0000 7ff9a1389740 1 Processor -- start 2024-05-21T15:37:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.522+0000 7ff9a1389740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af7542d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.522+0000 7ff9a1389740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7505e4d0 0x55af7542df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.522+0000 7ff9a1389740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7507f1b0 0x55af7549c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.522+0000 7ff9a1389740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff98000fb00 con 0x55af7507f1b0 2024-05-21T15:37:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.522+0000 7ff9a1389740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff98000f400 con 0x55af750562f0 2024-05-21T15:37:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.522+0000 7ff9a1389740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff98000c640 con 0x55af7505e4d0 2024-05-21T15:37:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.522+0000 7ff9a1389740 10 client.0 _ll_register_callbacks cb 0x7ff98800c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.523+0000 7ff9937fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7505e4d0 0x55af7542df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.523+0000 7ff9937fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7505e4d0 0x55af7542df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46884/0 (socket says 172.21.15.38:46884) 2024-05-21T15:37:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.523+0000 7ff9937fe640 1 -- 172.21.15.38:0/2054306782 learned_addr learned my addr 172.21.15.38:0/2054306782 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.523+0000 7ff993fff640 1 --2- 172.21.15.38:0/2054306782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af7542d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.523+0000 7ff9989a3640 1 --2- 172.21.15.38:0/2054306782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7507f1b0 0x55af7549c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.523+0000 7ff993fff640 1 -- 172.21.15.38:0/2054306782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7505e4d0 msgr2=0x55af7542df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.523+0000 7ff993fff640 1 --2- 172.21.15.38:0/2054306782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7505e4d0 0x55af7542df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.523+0000 7ff993fff640 1 -- 172.21.15.38:0/2054306782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7507f1b0 msgr2=0x55af7549c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.523+0000 7ff993fff640 1 --2- 172.21.15.38:0/2054306782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7507f1b0 0x55af7549c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.523+0000 7ff993fff640 1 -- 172.21.15.38:0/2054306782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af7549cf00 con 0x55af750562f0 2024-05-21T15:37:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.524+0000 7ff993fff640 1 --2- 172.21.15.38:0/2054306782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af7542d9c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55af75432f50 tx=0x7ff988003e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5dcd6b913ba8ace server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.525+0000 7ff9737fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:37:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.525+0000 7ff9737fe640 1 -- 172.21.15.38:0/2054306782 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98800c750 con 0x55af750562f0 2024-05-21T15:37:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.525+0000 7ff9737fe640 1 -- 172.21.15.38:0/2054306782 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff988004730 con 0x55af750562f0 2024-05-21T15:37:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.525+0000 7ff9a1389740 1 -- 172.21.15.38:0/2054306782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff988005500 con 0x55af750562f0 2024-05-21T15:37:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.527+0000 7ff9737fe640 1 -- 172.21.15.38:0/2054306782 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff988004930 con 0x55af750562f0 2024-05-21T15:37:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.527+0000 7ff9a1389740 10 client.26042 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.527+0000 7ff9a1389740 1 -- 172.21.15.38:0/2054306782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55af7549e0f0 con 0x55af750562f0 2024-05-21T15:37:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.528+0000 7ff9737fe640 1 -- 172.21.15.38:0/2054306782 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff988005500 con 0x55af750562f0 2024-05-21T15:37:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.528+0000 7ff9737fe640 1 -- 172.21.15.38:0/2054306782 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7ff988027410 con 0x55af750562f0 2024-05-21T15:37:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.528+0000 7ff9a1389740 10 client.26042 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.528+0000 7ff9a1389740 10 client.26042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.528+0000 7ff9a1389740 20 client.26042 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.528+0000 7ff9a1389740 1 --2- 172.21.15.38:0/2054306782 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55af750131c0 0x55af754a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.528+0000 7ff9a1389740 4 client.26042 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:37:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.528+0000 7ff9a1389740 1 -- 172.21.15.38:0/2054306782 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af7538adc0 con 0x55af750131c0 2024-05-21T15:37:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.533+0000 7ff9937fe640 1 --2- 172.21.15.38:0/2054306782 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55af750131c0 0x55af754a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.538+0000 7ff9937fe640 1 --2- 172.21.15.38:0/2054306782 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55af750131c0 0x55af754a0b70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99318653a01038f7 server_cookie=165dc10670e077f9 in_seq=0 out_seq=0 2024-05-21T15:37:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:32.538+0000 7ff9737fe640 10 client.26042 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.067+0000 7ff9737fe640 1 -- 172.21.15.38:0/2054306782 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af7538adc0 con 0x55af750131c0 2024-05-21T15:37:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.068+0000 7ff9737fe640 10 client.26042 handle_command_reply: tid=0 2024-05-21T15:37:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.068+0000 7ff9a1389740 1 client.26042 shutdown 2024-05-21T15:37:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.068+0000 7ff9a1389740 1 -- 172.21.15.38:0/2054306782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 msgr2=0x55af7542d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.068+0000 7ff9a1389740 1 --2- 172.21.15.38:0/2054306782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af7542d9c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55af75432f50 tx=0x7ff988003e40 comp rx=0 tx=0).stop 2024-05-21T15:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.068+0000 7ff9a1389740 1 -- 172.21.15.38:0/2054306782 shutdown_connections 2024-05-21T15:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.068+0000 7ff9a1389740 1 --2- 172.21.15.38:0/2054306782 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55af750131c0 0x55af754a0b70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.068+0000 7ff9a1389740 1 --2- 172.21.15.38:0/2054306782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af7507f1b0 0x55af7549c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.068+0000 7ff9a1389740 1 --2- 172.21.15.38:0/2054306782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af7505e4d0 0x55af7542df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.068+0000 7ff9a1389740 1 --2- 172.21.15.38:0/2054306782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af750562f0 0x55af7542d9c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.068+0000 7ff9a1389740 1 -- 172.21.15.38:0/2054306782 >> 172.21.15.38:0/2054306782 conn(0x55af75073300 msgr2=0x55af75423210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.068+0000 7ff9737fe640 0 client.26042 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.068+0000 7ff9a1389740 1 -- 172.21.15.38:0/2054306782 shutdown_connections 2024-05-21T15:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.068+0000 7ff9a1389740 1 -- 172.21.15.38:0/2054306782 wait complete. 2024-05-21T15:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.068+0000 7ff9a1389740 20 client.26042 trim_cache size 0 max 16384 2024-05-21T15:37:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:37:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:37:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:37:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:37:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.070+0000 7ff9991a4640 1 -- 172.21.15.38:0/1549308287 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff97c0a2d90 msgr2=0x7ff97c0a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.070+0000 7ff9991a4640 1 --2- 172.21.15.38:0/1549308287 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff97c0a2d90 0x7ff97c0a52a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff9840096c0 tx=0x7ff984000bc0 comp rx=0 tx=0).stop 2024-05-21T15:37:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.070+0000 7ff9991a4640 1 -- 172.21.15.38:0/1549308287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff994103eb0 msgr2=0x7ff9941124e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.070+0000 7ff9991a4640 1 --2- 172.21.15.38:0/1549308287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff994103eb0 0x7ff9941124e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00eb10 tx=0x7ff98c00c2d0 comp rx=0 tx=0).stop 2024-05-21T15:37:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.071+0000 7ff9991a4640 1 -- 172.21.15.38:0/1549308287 shutdown_connections 2024-05-21T15:37:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.071+0000 7ff9991a4640 1 --2- 172.21.15.38:0/1549308287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff994103eb0 0x7ff9941124e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.071+0000 7ff9991a4640 1 --2- 172.21.15.38:0/1549308287 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff97c0a2d90 0x7ff97c0a52a0 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff9840096c0 tx=0x7ff984000bc0 comp rx=0 tx=0).stop 2024-05-21T15:37:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.071+0000 7ff9991a4640 1 --2- 172.21.15.38:0/1549308287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff994101060 0x7ff994111fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.071+0000 7ff9991a4640 1 --2- 172.21.15.38:0/1549308287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff994100560 0x7ff9941138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.071+0000 7ff9991a4640 1 -- 172.21.15.38:0/1549308287 >> 172.21.15.38:0/1549308287 conn(0x7ff9940fc1a0 msgr2=0x7ff9940fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.071+0000 7ff9991a4640 1 -- 172.21.15.38:0/1549308287 shutdown_connections 2024-05-21T15:37:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:33.071+0000 7ff9991a4640 1 -- 172.21.15.38:0/1549308287 wait complete. 2024-05-21T15:37:33.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:32 smithi045 ceph-mon[36409]: scrub queued for path: / 2024-05-21T15:37:33.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:32 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:33.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:32 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:37:33.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:32 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:37:33.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:32 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:37:33.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:32 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:37:33.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:32 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:37:33.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:32 smithi038 ceph-mon[31995]: scrub queued for path: / 2024-05-21T15:37:33.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:32 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:33.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:32 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:37:33.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:32 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:37:33.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:32 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:37:33.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:32 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:37:33.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:32 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:37:33.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:32 smithi053 ceph-mon[36496]: scrub queued for path: / 2024-05-21T15:37:33.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:32 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:33.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:32 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:37:33.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:32 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:37:33.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:32 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:37:33.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:32 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:37:33.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:32 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:37:34.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:33 smithi045 ceph-mon[36409]: pgmap v445: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 168 MiB/s wr, 19.16k op/s 2024-05-21T15:37:34.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:33 smithi038 ceph-mon[31995]: pgmap v445: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 168 MiB/s wr, 19.16k op/s 2024-05-21T15:37:34.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:33 smithi053 ceph-mon[36496]: pgmap v445: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 168 MiB/s wr, 19.16k op/s 2024-05-21T15:37:35.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:34 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:35.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:34 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:35.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:34 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:36.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:35 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:36.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:35 smithi045 ceph-mon[36409]: pgmap v446: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 154 MiB/s wr, 18.08k op/s 2024-05-21T15:37:36.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:35 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:36.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:35 smithi038 ceph-mon[31995]: pgmap v446: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 154 MiB/s wr, 18.08k op/s 2024-05-21T15:37:36.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:35 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:37:36.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:35 smithi053 ceph-mon[36496]: pgmap v446: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 154 MiB/s wr, 18.08k op/s 2024-05-21T15:37:38.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:37 smithi045 ceph-mon[36409]: pgmap v447: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 150 MiB/s wr, 18.58k op/s 2024-05-21T15:37:38.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:37 smithi038 ceph-mon[31995]: pgmap v447: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 150 MiB/s wr, 18.58k op/s 2024-05-21T15:37:38.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:37 smithi053 ceph-mon[36496]: pgmap v447: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 150 MiB/s wr, 18.58k op/s 2024-05-21T15:37:40.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:39 smithi045 ceph-mon[36409]: pgmap v448: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 139 MiB/s wr, 17.98k op/s 2024-05-21T15:37:40.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:39 smithi038 ceph-mon[31995]: pgmap v448: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 139 MiB/s wr, 17.98k op/s 2024-05-21T15:37:40.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:39 smithi053 ceph-mon[36496]: pgmap v448: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 139 MiB/s wr, 17.98k op/s 2024-05-21T15:37:42.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:41 smithi045 ceph-mon[36409]: pgmap v449: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 156 MiB/s wr, 20.34k op/s 2024-05-21T15:37:42.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:41 smithi038 ceph-mon[31995]: pgmap v449: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 156 MiB/s wr, 20.34k op/s 2024-05-21T15:37:42.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:41 smithi053 ceph-mon[36496]: pgmap v449: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 156 MiB/s wr, 20.34k op/s 2024-05-21T15:37:44.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:44 smithi045 ceph-mon[36409]: pgmap v450: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 149 MiB/s wr, 19.71k op/s 2024-05-21T15:37:44.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:44 smithi038 ceph-mon[31995]: pgmap v450: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 149 MiB/s wr, 19.71k op/s 2024-05-21T15:37:44.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:44 smithi053 ceph-mon[36496]: pgmap v450: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 149 MiB/s wr, 19.71k op/s 2024-05-21T15:37:46.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:46 smithi045 ceph-mon[36409]: pgmap v451: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 140 MiB/s wr, 18.41k op/s 2024-05-21T15:37:46.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:46 smithi038 ceph-mon[31995]: pgmap v451: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 140 MiB/s wr, 18.41k op/s 2024-05-21T15:37:46.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:46 smithi053 ceph-mon[36496]: pgmap v451: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 140 MiB/s wr, 18.41k op/s 2024-05-21T15:37:48.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:48 smithi038 ceph-mon[31995]: pgmap v452: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 165 MiB/s wr, 21.63k op/s 2024-05-21T15:37:48.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:48 smithi053 ceph-mon[36496]: pgmap v452: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 165 MiB/s wr, 21.63k op/s 2024-05-21T15:37:48.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:48 smithi045 ceph-mon[36409]: pgmap v452: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 165 MiB/s wr, 21.63k op/s 2024-05-21T15:37:50.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:50 smithi038 ceph-mon[31995]: pgmap v453: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 161 MiB/s wr, 21.05k op/s 2024-05-21T15:37:50.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:50 smithi053 ceph-mon[36496]: pgmap v453: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 161 MiB/s wr, 21.05k op/s 2024-05-21T15:37:50.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:50 smithi045 ceph-mon[36409]: pgmap v453: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 161 MiB/s wr, 21.05k op/s 2024-05-21T15:37:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.095+0000 7f286effd640 1 -- 172.21.15.38:0/622982980 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860004230 con 0x7f28701093d0 2024-05-21T15:37:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.096+0000 7f2875b96640 1 -- 172.21.15.38:0/622982980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28701093d0 msgr2=0x7f2870109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.096+0000 7f2875b96640 1 --2- 172.21.15.38:0/622982980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28701093d0 0x7f2870109dc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2860009900 tx=0x7f286002a440 comp rx=0 tx=0).stop 2024-05-21T15:37:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.096+0000 7f2875b96640 1 -- 172.21.15.38:0/622982980 shutdown_connections 2024-05-21T15:37:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.096+0000 7f2875b96640 1 --2- 172.21.15.38:0/622982980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f287010f020 0x7f287010f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.096+0000 7f2875b96640 1 --2- 172.21.15.38:0/622982980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f287010a300 0x7f287010a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.096+0000 7f2875b96640 1 --2- 172.21.15.38:0/622982980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28701093d0 0x7f2870109dc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.096+0000 7f2875b96640 1 -- 172.21.15.38:0/622982980 >> 172.21.15.38:0/622982980 conn(0x7f287006f4c0 msgr2=0x7f2870071970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.099+0000 7f2875b96640 1 -- 172.21.15.38:0/622982980 shutdown_connections 2024-05-21T15:37:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.099+0000 7f2875b96640 1 -- 172.21.15.38:0/622982980 wait complete. 2024-05-21T15:37:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.100+0000 7f2875b96640 1 Processor -- start 2024-05-21T15:37:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.100+0000 7f2875b96640 1 -- start start 2024-05-21T15:37:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.100+0000 7f2875b96640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28701093d0 0x7f2870115f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.100+0000 7f2875b96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f287010a300 0x7f2870116500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.100+0000 7f2875b96640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f287010f020 0x7f2870117430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.101+0000 7f2867fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f287010a300 0x7f2870116500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.101+0000 7f2867fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f287010a300 0x7f2870116500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46818/0 (socket says 172.21.15.38:46818) 2024-05-21T15:37:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.101+0000 7f2875b96640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f287011d660 con 0x7f287010a300 2024-05-21T15:37:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.101+0000 7f2875b96640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f287011d4e0 con 0x7f28701093d0 2024-05-21T15:37:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.101+0000 7f2875b96640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f287011d7e0 con 0x7f287010f020 2024-05-21T15:37:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.101+0000 7f2867fff640 1 -- 172.21.15.38:0/2272265279 learned_addr learned my addr 172.21.15.38:0/2272265279 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.104+0000 7f286ffff640 1 --2- 172.21.15.38:0/2272265279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f287010f020 0x7f2870117430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.105+0000 7f286f7fe640 1 --2- 172.21.15.38:0/2272265279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28701093d0 0x7f2870115f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.105+0000 7f2867fff640 1 -- 172.21.15.38:0/2272265279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f287010f020 msgr2=0x7f2870117430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.105+0000 7f2867fff640 1 --2- 172.21.15.38:0/2272265279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f287010f020 0x7f2870117430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.105+0000 7f2867fff640 1 -- 172.21.15.38:0/2272265279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28701093d0 msgr2=0x7f2870115f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.105+0000 7f2867fff640 1 --2- 172.21.15.38:0/2272265279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28701093d0 0x7f2870115f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.105+0000 7f2867fff640 1 -- 172.21.15.38:0/2272265279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2870117aa0 con 0x7f287010a300 2024-05-21T15:37:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.106+0000 7f2867fff640 1 --2- 172.21.15.38:0/2272265279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f287010a300 0x7f2870116500 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f285800b4a0 tx=0x7f285800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.110+0000 7f286d7fa640 1 -- 172.21.15.38:0/2272265279 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858014070 con 0x7f287010a300 2024-05-21T15:37:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.110+0000 7f286d7fa640 1 -- 172.21.15.38:0/2272265279 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2858010af0 con 0x7f287010a300 2024-05-21T15:37:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.110+0000 7f286d7fa640 1 -- 172.21.15.38:0/2272265279 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858017d40 con 0x7f287010a300 2024-05-21T15:37:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.110+0000 7f2875b96640 1 -- 172.21.15.38:0/2272265279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2870117d90 con 0x7f287010a300 2024-05-21T15:37:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.111+0000 7f2875b96640 1 -- 172.21.15.38:0/2272265279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2870118260 con 0x7f287010a300 2024-05-21T15:37:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.116+0000 7f286d7fa640 1 -- 172.21.15.38:0/2272265279 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2858017070 con 0x7f287010a300 2024-05-21T15:37:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.118+0000 7f286d7fa640 1 --2- 172.21.15.38:0/2272265279 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f285c0a2e30 0x7f285c0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.118+0000 7f286d7fa640 1 -- 172.21.15.38:0/2272265279 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f28580ce290 con 0x7f287010a300 2024-05-21T15:37:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.121+0000 7f286f7fe640 1 --2- 172.21.15.38:0/2272265279 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f285c0a2e30 0x7f285c0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.123+0000 7f286f7fe640 1 --2- 172.21.15.38:0/2272265279 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f285c0a2e30 0x7f285c0a5340 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2860000c00 tx=0x7f2860005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.183+0000 7f287ca44740 1 Processor -- start 2024-05-21T15:37:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.184+0000 7f287ca44740 1 -- start start 2024-05-21T15:37:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.184+0000 7f287ca44740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 0x55d6d888fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.184+0000 7f287ca44740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84c72f0 0x55d6d88903e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.184+0000 7f287ca44740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84cf4d0 0x55d6d88972e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.184+0000 7f287ca44740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d6d87f4340 con 0x55d6d84c72f0 2024-05-21T15:37:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.184+0000 7f287ca44740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d6d879c290 con 0x55d6d84f01b0 2024-05-21T15:37:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.184+0000 7f287ca44740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d6d87f0300 con 0x55d6d84cf4d0 2024-05-21T15:37:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.185+0000 7f2867fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84c72f0 0x55d6d88903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.185+0000 7f2867fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84c72f0 0x55d6d88903e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46826/0 (socket says 172.21.15.38:46826) 2024-05-21T15:37:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.185+0000 7f2867fff640 1 -- 172.21.15.38:0/3005985441 learned_addr learned my addr 172.21.15.38:0/3005985441 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.185+0000 7f286ffff640 1 --2- 172.21.15.38:0/3005985441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84cf4d0 0x55d6d88972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.185+0000 7f286f7fe640 1 --2- 172.21.15.38:0/3005985441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 0x55d6d888fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.186+0000 7f286ffff640 1 -- 172.21.15.38:0/3005985441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 msgr2=0x55d6d888fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.186+0000 7f286ffff640 1 --2- 172.21.15.38:0/3005985441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 0x55d6d888fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.186+0000 7f286ffff640 1 -- 172.21.15.38:0/3005985441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84c72f0 msgr2=0x55d6d88903e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.186+0000 7f286ffff640 1 --2- 172.21.15.38:0/3005985441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84c72f0 0x55d6d88903e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.186+0000 7f286ffff640 1 -- 172.21.15.38:0/3005985441 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6d87c08e0 con 0x55d6d84cf4d0 2024-05-21T15:37:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.187+0000 7f286ffff640 1 --2- 172.21.15.38:0/3005985441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84cf4d0 0x55d6d88972e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2868009f00 tx=0x7f2868011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f352285360662e5c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.189+0000 7f2865ffb640 1 -- 172.21.15.38:0/3005985441 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868004d20 con 0x55d6d84cf4d0 2024-05-21T15:37:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.189+0000 7f2865ffb640 1 -- 172.21.15.38:0/3005985441 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2868004050 con 0x55d6d84cf4d0 2024-05-21T15:37:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.190+0000 7f287ca44740 1 -- 172.21.15.38:0/3005985441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84cf4d0 msgr2=0x55d6d88972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.190+0000 7f287ca44740 1 --2- 172.21.15.38:0/3005985441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84cf4d0 0x55d6d88972e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2868009f00 tx=0x7f2868011810 comp rx=0 tx=0).stop 2024-05-21T15:37:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.192+0000 7f287ca44740 1 -- 172.21.15.38:0/3005985441 shutdown_connections 2024-05-21T15:37:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.192+0000 7f287ca44740 1 --2- 172.21.15.38:0/3005985441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84cf4d0 0x55d6d88972e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.192+0000 7f287ca44740 1 --2- 172.21.15.38:0/3005985441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84c72f0 0x55d6d88903e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.193+0000 7f287ca44740 1 --2- 172.21.15.38:0/3005985441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 0x55d6d888fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.193+0000 7f287ca44740 1 -- 172.21.15.38:0/3005985441 >> 172.21.15.38:0/3005985441 conn(0x55d6d84e4300 msgr2=0x55d6d888b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.194+0000 7f287ca44740 1 -- 172.21.15.38:0/3005985441 shutdown_connections 2024-05-21T15:37:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.194+0000 7f287ca44740 1 -- 172.21.15.38:0/3005985441 wait complete. 2024-05-21T15:37:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.195+0000 7f287ca44740 1 -- start start 2024-05-21T15:37:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.195+0000 7f287ca44740 1 Processor -- start 2024-05-21T15:37:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.196+0000 7f287ca44740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d8899b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.196+0000 7f287ca44740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84cf4d0 0x55d6d889a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.196+0000 7f287ca44740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 0x55d6d88a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.197+0000 7f287ca44740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d6d8897d80 con 0x55d6d84cf4d0 2024-05-21T15:37:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.197+0000 7f287ca44740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d6d87af790 con 0x55d6d84f01b0 2024-05-21T15:37:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.197+0000 7f287ca44740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d6d8897f00 con 0x55d6d84c72f0 2024-05-21T15:37:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.197+0000 7f287ca44740 10 client.0 _ll_register_callbacks cb 0x55d6d87eaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.197+0000 7f286ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 0x55d6d88a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.197+0000 7f286ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 0x55d6d88a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46486/0 (socket says 172.21.15.38:46486) 2024-05-21T15:37:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.197+0000 7f286ffff640 1 -- 172.21.15.38:0/2944515315 learned_addr learned my addr 172.21.15.38:0/2944515315 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.197+0000 7f286f7fe640 1 --2- 172.21.15.38:0/2944515315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d8899b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.198+0000 7f286ffff640 1 -- 172.21.15.38:0/2944515315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 msgr2=0x55d6d8899b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.198+0000 7f286ffff640 1 --2- 172.21.15.38:0/2944515315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d8899b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.198+0000 7f286ffff640 1 -- 172.21.15.38:0/2944515315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84cf4d0 msgr2=0x55d6d889a080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:37:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.198+0000 7f286ffff640 1 --2- 172.21.15.38:0/2944515315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84cf4d0 0x55d6d889a080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.198+0000 7f286ffff640 1 -- 172.21.15.38:0/2944515315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6d8508210 con 0x55d6d84f01b0 2024-05-21T15:37:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.200+0000 7f286ffff640 1 --2- 172.21.15.38:0/2944515315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 0x55d6d88a0fe0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2868004890 tx=0x7f2868004bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e850124e24671f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.200+0000 7f284effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:37:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.202+0000 7f284effd640 1 -- 172.21.15.38:0/2944515315 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868029020 con 0x55d6d84f01b0 2024-05-21T15:37:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.202+0000 7f284effd640 1 -- 172.21.15.38:0/2944515315 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f28680054e0 con 0x55d6d84f01b0 2024-05-21T15:37:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.202+0000 7f287ca44740 1 -- 172.21.15.38:0/2944515315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2868004050 con 0x55d6d84f01b0 2024-05-21T15:37:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.204+0000 7f284effd640 1 -- 172.21.15.38:0/2944515315 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f28680056e0 con 0x55d6d84f01b0 2024-05-21T15:37:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.206+0000 7f287ca44740 10 client.26051 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.206+0000 7f287ca44740 1 -- 172.21.15.38:0/2944515315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d6d85d4620 con 0x55d6d84f01b0 2024-05-21T15:37:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.208+0000 7f284effd640 1 -- 172.21.15.38:0/2944515315 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2868004050 con 0x55d6d84f01b0 2024-05-21T15:37:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.208+0000 7f284effd640 1 -- 172.21.15.38:0/2944515315 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f2868022420 con 0x55d6d84f01b0 2024-05-21T15:37:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.209+0000 7f287ca44740 10 client.26051 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.209+0000 7f287ca44740 10 client.26051 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.209+0000 7f287ca44740 20 client.26051 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.209+0000 7f287ca44740 1 --2- 172.21.15.38:0/2944515315 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d6d84aed80 0x55d6d88a73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.209+0000 7f287ca44740 4 client.26051 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:37:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.209+0000 7f287ca44740 1 -- 172.21.15.38:0/2944515315 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6d8801a70 con 0x55d6d84aed80 2024-05-21T15:37:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.212+0000 7f286f7fe640 1 --2- 172.21.15.38:0/2944515315 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d6d84aed80 0x55d6d88a73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.219+0000 7f286f7fe640 1 --2- 172.21.15.38:0/2944515315 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d6d84aed80 0x55d6d88a73f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6a8948bf0c14c9c server_cookie=a010bbed55034ae2 in_seq=0 out_seq=0 2024-05-21T15:37:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.220+0000 7f284effd640 10 client.26051 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.221+0000 7f284effd640 1 -- 172.21.15.38:0/2944515315 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6d8801a70 con 0x55d6d84aed80 2024-05-21T15:37:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.221+0000 7f284effd640 10 client.26051 handle_command_reply: tid=0 2024-05-21T15:37:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.223+0000 7f287ca44740 1 client.26051 shutdown 2024-05-21T15:37:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.224+0000 7f287ca44740 1 -- 172.21.15.38:0/2944515315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 msgr2=0x55d6d88a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.224+0000 7f287ca44740 1 --2- 172.21.15.38:0/2944515315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 0x55d6d88a0fe0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2868004890 tx=0x7f2868004bf0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.224+0000 7f287ca44740 1 -- 172.21.15.38:0/2944515315 shutdown_connections 2024-05-21T15:37:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.224+0000 7f287ca44740 1 --2- 172.21.15.38:0/2944515315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 0x55d6d88a0fe0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.224+0000 7f287ca44740 1 --2- 172.21.15.38:0/2944515315 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d6d84aed80 0x55d6d88a73f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.224+0000 7f287ca44740 1 --2- 172.21.15.38:0/2944515315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84cf4d0 0x55d6d889a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.224+0000 7f287ca44740 1 --2- 172.21.15.38:0/2944515315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d8899b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.224+0000 7f287ca44740 1 -- 172.21.15.38:0/2944515315 >> 172.21.15.38:0/2944515315 conn(0x55d6d84e4300 msgr2=0x55d6d888a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.226+0000 7f284effd640 0 client.26051 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.227+0000 7f287ca44740 1 -- 172.21.15.38:0/2944515315 shutdown_connections 2024-05-21T15:37:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.227+0000 7f287ca44740 1 -- 172.21.15.38:0/2944515315 wait complete. 2024-05-21T15:37:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.228+0000 7f287ca44740 20 client.26051 trim_cache size 0 max 16384 2024-05-21T15:37:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.265+0000 7f287ca44740 1 Processor -- start 2024-05-21T15:37:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.265+0000 7f287ca44740 1 -- start start 2024-05-21T15:37:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.268+0000 7f287ca44740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d8892ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.268+0000 7f287ca44740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84cf4d0 0x55d6d8893410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.268+0000 7f287ca44740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 0x55d6d888b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.268+0000 7f287ca44740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f28701b0c00 con 0x55d6d84cf4d0 2024-05-21T15:37:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.268+0000 7f287ca44740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28701b2830 con 0x55d6d84f01b0 2024-05-21T15:37:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.269+0000 7f287ca44740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f28701b12d0 con 0x55d6d84c72f0 2024-05-21T15:37:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.270+0000 7f286f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d8892ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.270+0000 7f286f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d8892ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:44484/0 (socket says 172.21.15.38:44484) 2024-05-21T15:37:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.270+0000 7f286f7fe640 1 -- 172.21.15.38:0/1208390916 learned_addr learned my addr 172.21.15.38:0/1208390916 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.270+0000 7f2867fff640 1 --2- 172.21.15.38:0/1208390916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84cf4d0 0x55d6d8893410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.270+0000 7f286ffff640 1 --2- 172.21.15.38:0/1208390916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 0x55d6d888b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.271+0000 7f2867fff640 1 -- 172.21.15.38:0/1208390916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 msgr2=0x55d6d8892ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.271+0000 7f2867fff640 1 --2- 172.21.15.38:0/1208390916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d8892ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.271+0000 7f2867fff640 1 -- 172.21.15.38:0/1208390916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 msgr2=0x55d6d888b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.271+0000 7f2867fff640 1 --2- 172.21.15.38:0/1208390916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 0x55d6d888b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.271+0000 7f2867fff640 1 -- 172.21.15.38:0/1208390916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6d888fa10 con 0x55d6d84cf4d0 2024-05-21T15:37:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.273+0000 7f2867fff640 1 --2- 172.21.15.38:0/1208390916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84cf4d0 0x55d6d8893410 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f28580a3180 tx=0x7f28580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc6b5eda08ad6f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.274+0000 7f28657fa640 1 -- 172.21.15.38:0/1208390916 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28580b3020 con 0x55d6d84cf4d0 2024-05-21T15:37:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.274+0000 7f28657fa640 1 -- 172.21.15.38:0/1208390916 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28580a36b0 con 0x55d6d84cf4d0 2024-05-21T15:37:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.274+0000 7f28657fa640 1 -- 172.21.15.38:0/1208390916 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285801e040 con 0x55d6d84cf4d0 2024-05-21T15:37:52.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:52 smithi038 ceph-mon[31995]: pgmap v454: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 159 MiB/s wr, 21.11k op/s 2024-05-21T15:37:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.277+0000 7f287ca44740 1 -- 172.21.15.38:0/1208390916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84cf4d0 msgr2=0x55d6d8893410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.277+0000 7f287ca44740 1 --2- 172.21.15.38:0/1208390916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84cf4d0 0x55d6d8893410 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f28580a3180 tx=0x7f28580046c0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.277+0000 7f287ca44740 1 -- 172.21.15.38:0/1208390916 shutdown_connections 2024-05-21T15:37:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.277+0000 7f287ca44740 1 --2- 172.21.15.38:0/1208390916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84f01b0 0x55d6d888b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.277+0000 7f287ca44740 1 --2- 172.21.15.38:0/1208390916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84cf4d0 0x55d6d8893410 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.277+0000 7f287ca44740 1 --2- 172.21.15.38:0/1208390916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d8892ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.277+0000 7f287ca44740 1 -- 172.21.15.38:0/1208390916 >> 172.21.15.38:0/1208390916 conn(0x55d6d84e4300 msgr2=0x55d6d8893ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.280+0000 7f287ca44740 1 -- 172.21.15.38:0/1208390916 shutdown_connections 2024-05-21T15:37:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.280+0000 7f287ca44740 1 -- 172.21.15.38:0/1208390916 wait complete. 2024-05-21T15:37:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.281+0000 7f287ca44740 1 -- start start 2024-05-21T15:37:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.281+0000 7f287ca44740 1 Processor -- start 2024-05-21T15:37:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.283+0000 7f287ca44740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d88a0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.283+0000 7f287ca44740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84cf4d0 0x55d6d88a0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.283+0000 7f287ca44740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84f01b0 0x55d6d889cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.283+0000 7f287ca44740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f28701b20a0 con 0x55d6d84f01b0 2024-05-21T15:37:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.284+0000 7f287ca44740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28701b19a0 con 0x55d6d84cf4d0 2024-05-21T15:37:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.284+0000 7f287ca44740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f28701aebe0 con 0x55d6d84c72f0 2024-05-21T15:37:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.284+0000 7f286f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d88a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.284+0000 7f286f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d88a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:44494/0 (socket says 172.21.15.38:44494) 2024-05-21T15:37:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.284+0000 7f286f7fe640 1 -- 172.21.15.38:0/3958363892 learned_addr learned my addr 172.21.15.38:0/3958363892 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:37:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.284+0000 7f287ca44740 10 client.0 _ll_register_callbacks cb 0x7f2858017ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:37:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.285+0000 7f286ffff640 1 --2- 172.21.15.38:0/3958363892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84f01b0 0x55d6d889cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.285+0000 7f2867fff640 1 --2- 172.21.15.38:0/3958363892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84cf4d0 0x55d6d88a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.285+0000 7f286f7fe640 1 -- 172.21.15.38:0/3958363892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84cf4d0 msgr2=0x55d6d88a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.285+0000 7f286f7fe640 1 --2- 172.21.15.38:0/3958363892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84cf4d0 0x55d6d88a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.285+0000 7f286f7fe640 1 -- 172.21.15.38:0/3958363892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84f01b0 msgr2=0x55d6d889cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:37:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.285+0000 7f286f7fe640 1 --2- 172.21.15.38:0/3958363892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84f01b0 0x55d6d889cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.285+0000 7f286f7fe640 1 -- 172.21.15.38:0/3958363892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6d889d390 con 0x55d6d84c72f0 2024-05-21T15:37:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.286+0000 7f286f7fe640 1 --2- 172.21.15.38:0/3958363892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d88a0020 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2860032000 tx=0x7f286003f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=217028690f1a0ef3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:37:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.286+0000 7f284effd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:37:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.287+0000 7f284effd640 1 -- 172.21.15.38:0/3958363892 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860004230 con 0x55d6d84c72f0 2024-05-21T15:37:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.287+0000 7f287ca44740 1 -- 172.21.15.38:0/3958363892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28580a36b0 con 0x55d6d84c72f0 2024-05-21T15:37:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.287+0000 7f284effd640 1 -- 172.21.15.38:0/3958363892 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2860005310 con 0x55d6d84c72f0 2024-05-21T15:37:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.287+0000 7f284effd640 1 -- 172.21.15.38:0/3958363892 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2860005510 con 0x55d6d84c72f0 2024-05-21T15:37:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.287+0000 7f287ca44740 10 client.26119 fetch_fsmap learned FSMap version 31 2024-05-21T15:37:52.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.287+0000 7f287ca44740 1 -- 172.21.15.38:0/3958363892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d6d890e520 con 0x55d6d84c72f0 2024-05-21T15:37:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.288+0000 7f284effd640 1 -- 172.21.15.38:0/3958363892 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28580a36b0 con 0x55d6d84c72f0 2024-05-21T15:37:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.288+0000 7f284effd640 1 -- 172.21.15.38:0/3958363892 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f2860052ce0 con 0x55d6d84c72f0 2024-05-21T15:37:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.288+0000 7f287ca44740 10 client.26119 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:37:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.288+0000 7f287ca44740 10 client.26119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:37:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.288+0000 7f287ca44740 20 client.26119 populate_metadata read hostname 'smithi038' 2024-05-21T15:37:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.289+0000 7f287ca44740 1 --2- 172.21.15.38:0/3958363892 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d6d84841c0 0x55d6d8911030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:37:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.289+0000 7f287ca44740 4 client.26119 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:37:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.289+0000 7f287ca44740 1 -- 172.21.15.38:0/3958363892 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d6d87fbdc0 con 0x55d6d84841c0 2024-05-21T15:37:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.289+0000 7f2867fff640 1 --2- 172.21.15.38:0/3958363892 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d6d84841c0 0x55d6d8911030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:37:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.291+0000 7f284effd640 10 client.26119 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:52.292+0000 7f2867fff640 1 --2- 172.21.15.38:0/3958363892 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d6d84841c0 0x55d6d8911030 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=576e97dd5daf9ecf server_cookie=5140c71aa29d771a in_seq=0 out_seq=0 2024-05-21T15:37:52.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:52 smithi053 ceph-mon[36496]: pgmap v454: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 159 MiB/s wr, 21.11k op/s 2024-05-21T15:37:52.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:52 smithi045 ceph-mon[36409]: pgmap v454: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 159 MiB/s wr, 21.11k op/s 2024-05-21T15:37:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.396+0000 7f284effd640 1 -- 172.21.15.38:0/3958363892 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d6d87fbdc0 con 0x55d6d84841c0 2024-05-21T15:37:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.396+0000 7f284effd640 10 client.26119 handle_command_reply: tid=0 2024-05-21T15:37:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.396+0000 7f287ca44740 1 client.26119 shutdown 2024-05-21T15:37:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.397+0000 7f287ca44740 1 -- 172.21.15.38:0/3958363892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 msgr2=0x55d6d88a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.397+0000 7f287ca44740 1 --2- 172.21.15.38:0/3958363892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d88a0020 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2860032000 tx=0x7f286003f000 comp rx=0 tx=0).stop 2024-05-21T15:37:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.397+0000 7f287ca44740 1 -- 172.21.15.38:0/3958363892 shutdown_connections 2024-05-21T15:37:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.397+0000 7f287ca44740 1 --2- 172.21.15.38:0/3958363892 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d6d84841c0 0x55d6d8911030 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.397+0000 7f287ca44740 1 --2- 172.21.15.38:0/3958363892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d6d84f01b0 0x55d6d889cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.397+0000 7f287ca44740 1 --2- 172.21.15.38:0/3958363892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6d84cf4d0 0x55d6d88a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.397+0000 7f287ca44740 1 --2- 172.21.15.38:0/3958363892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d6d84c72f0 0x55d6d88a0020 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.397+0000 7f287ca44740 1 -- 172.21.15.38:0/3958363892 >> 172.21.15.38:0/3958363892 conn(0x55d6d84e4300 msgr2=0x55d6d8893ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.397+0000 7f284effd640 0 client.26119 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:37:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.397+0000 7f287ca44740 1 -- 172.21.15.38:0/3958363892 shutdown_connections 2024-05-21T15:37:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.397+0000 7f287ca44740 1 -- 172.21.15.38:0/3958363892 wait complete. 2024-05-21T15:37:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.397+0000 7f287ca44740 20 client.26119 trim_cache size 0 max 16384 2024-05-21T15:37:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:37:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:37:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:37:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:37:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.398+0000 7f2875b96640 1 -- 172.21.15.38:0/2272265279 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f285c0a2e30 msgr2=0x7f285c0a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.398+0000 7f2875b96640 1 --2- 172.21.15.38:0/2272265279 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f285c0a2e30 0x7f285c0a5340 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2860000c00 tx=0x7f2860005fd0 comp rx=0 tx=0).stop 2024-05-21T15:37:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.398+0000 7f2875b96640 1 -- 172.21.15.38:0/2272265279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f287010a300 msgr2=0x7f2870116500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:37:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.398+0000 7f2875b96640 1 --2- 172.21.15.38:0/2272265279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f287010a300 0x7f2870116500 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f285800b4a0 tx=0x7f285800b960 comp rx=0 tx=0).stop 2024-05-21T15:37:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.399+0000 7f2875b96640 1 -- 172.21.15.38:0/2272265279 shutdown_connections 2024-05-21T15:37:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.399+0000 7f2875b96640 1 --2- 172.21.15.38:0/2272265279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f287010f020 0x7f2870117430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.399+0000 7f2875b96640 1 --2- 172.21.15.38:0/2272265279 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f285c0a2e30 0x7f285c0a5340 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.399+0000 7f2875b96640 1 --2- 172.21.15.38:0/2272265279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f287010a300 0x7f2870116500 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.399+0000 7f2875b96640 1 --2- 172.21.15.38:0/2272265279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28701093d0 0x7f2870115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:37:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.399+0000 7f2875b96640 1 -- 172.21.15.38:0/2272265279 >> 172.21.15.38:0/2272265279 conn(0x7f287006f4c0 msgr2=0x7f287010b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:37:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.399+0000 7f2875b96640 1 -- 172.21.15.38:0/2272265279 shutdown_connections 2024-05-21T15:37:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:37:53.399+0000 7f2875b96640 1 -- 172.21.15.38:0/2272265279 wait complete. 2024-05-21T15:37:54.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:54 smithi053 ceph-mon[36496]: pgmap v455: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 149 MiB/s wr, 20.46k op/s 2024-05-21T15:37:54.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:54 smithi045 ceph-mon[36409]: pgmap v455: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 149 MiB/s wr, 20.46k op/s 2024-05-21T15:37:54.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:54 smithi038 ceph-mon[31995]: pgmap v455: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 149 MiB/s wr, 20.46k op/s 2024-05-21T15:37:56.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:56 smithi053 ceph-mon[36496]: pgmap v456: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 132 MiB/s wr, 18.46k op/s 2024-05-21T15:37:56.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:56 smithi045 ceph-mon[36409]: pgmap v456: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 132 MiB/s wr, 18.46k op/s 2024-05-21T15:37:56.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:56 smithi038 ceph-mon[31995]: pgmap v456: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 132 MiB/s wr, 18.46k op/s 2024-05-21T15:37:58.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:37:58 smithi053 ceph-mon[36496]: pgmap v457: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 147 MiB/s wr, 20.79k op/s 2024-05-21T15:37:58.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:37:58 smithi045 ceph-mon[36409]: pgmap v457: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 147 MiB/s wr, 20.79k op/s 2024-05-21T15:37:58.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:37:58 smithi038 ceph-mon[31995]: pgmap v457: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 147 MiB/s wr, 20.79k op/s 2024-05-21T15:38:00.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:00 smithi053 ceph-mon[36496]: pgmap v458: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 133 MiB/s wr, 19.77k op/s 2024-05-21T15:38:00.455 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:00 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:38:00.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:00 smithi045 ceph-mon[36409]: pgmap v458: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 133 MiB/s wr, 19.77k op/s 2024-05-21T15:38:00.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:00 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:38:00.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:00 smithi038 ceph-mon[31995]: pgmap v458: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 133 MiB/s wr, 19.77k op/s 2024-05-21T15:38:00.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:00 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:38:02.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:38:02.258 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:38:02.453 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:02 smithi038 ceph-mon[31995]: pgmap v459: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 121 MiB/s wr, 18.33k op/s 2024-05-21T15:38:02.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:02 smithi053 ceph-mon[36496]: pgmap v459: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 121 MiB/s wr, 18.33k op/s 2024-05-21T15:38:02.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:02 smithi045 ceph-mon[36409]: pgmap v459: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 121 MiB/s wr, 18.33k op/s 2024-05-21T15:38:02.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.689+0000 7f5c75ffb640 1 -- 172.21.15.38:0/672502508 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c680319b0 con 0x7f5c7810ee40 2024-05-21T15:38:02.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.689+0000 7f5c77fff640 1 -- 172.21.15.38:0/672502508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c7810ee40 msgr2=0x7f5c7810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:02.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.689+0000 7f5c77fff640 1 --2- 172.21.15.38:0/672502508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c7810ee40 0x7f5c7810f250 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009900 tx=0x7f5c6802a3f0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.692+0000 7f5c77fff640 1 -- 172.21.15.38:0/672502508 shutdown_connections 2024-05-21T15:38:02.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.692+0000 7f5c77fff640 1 --2- 172.21.15.38:0/672502508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c7810ee40 0x7f5c7810f250 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.692+0000 7f5c77fff640 1 --2- 172.21.15.38:0/672502508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c7810a140 0x7f5c7810a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.692+0000 7f5c77fff640 1 --2- 172.21.15.38:0/672502508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c781092e0 0x7f5c78109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.692+0000 7f5c77fff640 1 -- 172.21.15.38:0/672502508 >> 172.21.15.38:0/672502508 conn(0x7f5c7806f3e0 msgr2=0x7f5c78071800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:02.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.694+0000 7f5c77fff640 1 -- 172.21.15.38:0/672502508 shutdown_connections 2024-05-21T15:38:02.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.694+0000 7f5c77fff640 1 -- 172.21.15.38:0/672502508 wait complete. 2024-05-21T15:38:02.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.695+0000 7f5c77fff640 1 Processor -- start 2024-05-21T15:38:02.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.697+0000 7f5c77fff640 1 -- start start 2024-05-21T15:38:02.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.697+0000 7f5c77fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c781092e0 0x7f5c78115f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.697+0000 7f5c77fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c7810a140 0x7f5c781164a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.697+0000 7f5c77fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c7810ee40 0x7f5c781173b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.697+0000 7f5c77fff640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7811d3f0 con 0x7f5c781092e0 2024-05-21T15:38:02.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.697+0000 7f5c77fff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7811d270 con 0x7f5c7810ee40 2024-05-21T15:38:02.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.697+0000 7f5c77fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7811d570 con 0x7f5c7810a140 2024-05-21T15:38:02.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.698+0000 7f5c777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c7810ee40 0x7f5c781173b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:02.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.698+0000 7f5c76ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c781092e0 0x7f5c78115f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:02.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.698+0000 7f5c76ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c781092e0 0x7f5c78115f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51390/0 (socket says 172.21.15.38:51390) 2024-05-21T15:38:02.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.698+0000 7f5c76ffd640 1 -- 172.21.15.38:0/2455243410 learned_addr learned my addr 172.21.15.38:0/2455243410 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:02.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.699+0000 7f5c76ffd640 1 -- 172.21.15.38:0/2455243410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c7810a140 msgr2=0x7f5c781164a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T15:38:02.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.699+0000 7f5c76ffd640 1 --2- 172.21.15.38:0/2455243410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c7810a140 0x7f5c781164a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.699+0000 7f5c76ffd640 1 -- 172.21.15.38:0/2455243410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c7810ee40 msgr2=0x7f5c781173b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:02.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.699+0000 7f5c76ffd640 1 --2- 172.21.15.38:0/2455243410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c7810ee40 0x7f5c781173b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.699+0000 7f5c76ffd640 1 -- 172.21.15.38:0/2455243410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c78117a20 con 0x7f5c781092e0 2024-05-21T15:38:02.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.699+0000 7f5c76ffd640 1 --2- 172.21.15.38:0/2455243410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c781092e0 0x7f5c78115f60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000cce0 tx=0x7f5c7000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:02.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.699+0000 7f5c5ffff640 1 -- 172.21.15.38:0/2455243410 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70004b20 con 0x7f5c781092e0 2024-05-21T15:38:02.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.700+0000 7f5c5ffff640 1 -- 172.21.15.38:0/2455243410 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c70004cc0 con 0x7f5c781092e0 2024-05-21T15:38:02.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.700+0000 7f5c77fff640 1 -- 172.21.15.38:0/2455243410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c78117d10 con 0x7f5c781092e0 2024-05-21T15:38:02.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.700+0000 7f5c77fff640 1 -- 172.21.15.38:0/2455243410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c781182e0 con 0x7f5c781092e0 2024-05-21T15:38:02.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.700+0000 7f5c5ffff640 1 -- 172.21.15.38:0/2455243410 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000f740 con 0x7f5c781092e0 2024-05-21T15:38:02.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.710+0000 7f5c845d2740 1 Processor -- start 2024-05-21T15:38:02.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.710+0000 7f5c845d2740 1 -- start start 2024-05-21T15:38:02.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.710+0000 7f5c845d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de348db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.710+0000 7f5c845d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de349330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.710+0000 7f5c845d2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de350230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.710+0000 7f5c845d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5571de2add00 con 0x5571ddfa9390 2024-05-21T15:38:02.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.710+0000 7f5c845d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5571de2554c0 con 0x5571ddf804d0 2024-05-21T15:38:02.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.710+0000 7f5c845d2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5571de2a9cc0 con 0x5571ddf886b0 2024-05-21T15:38:02.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.710+0000 7f5c5ffff640 1 -- 172.21.15.38:0/2455243410 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5c7000f9c0 con 0x7f5c781092e0 2024-05-21T15:38:02.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.710+0000 7f5c76ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de349330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:02.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.710+0000 7f5c76ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de349330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46710/0 (socket says 172.21.15.38:46710) 2024-05-21T15:38:02.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.710+0000 7f5c76ffd640 1 -- 172.21.15.38:0/3296272262 learned_addr learned my addr 172.21.15.38:0/3296272262 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:02.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.712+0000 7f5c5ffff640 1 --2- 172.21.15.38:0/2455243410 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5c600a2d60 0x7f5c600a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.712+0000 7f5c777fe640 1 --2- 172.21.15.38:0/3296272262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de348db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:02.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.713+0000 7f5c5ffff640 1 -- 172.21.15.38:0/2455243410 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f5c700cf0e0 con 0x7f5c781092e0 2024-05-21T15:38:02.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.713+0000 7f5c767fc640 1 --2- 172.21.15.38:0/3296272262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de350230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:02.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.713+0000 7f5c76ffd640 1 -- 172.21.15.38:0/3296272262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 msgr2=0x5571de350230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:02.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.713+0000 7f5c76ffd640 1 --2- 172.21.15.38:0/3296272262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de350230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.713+0000 7f5c76ffd640 1 -- 172.21.15.38:0/3296272262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 msgr2=0x5571de348db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:02.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.713+0000 7f5c76ffd640 1 --2- 172.21.15.38:0/3296272262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de348db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.713+0000 7f5c76ffd640 1 -- 172.21.15.38:0/3296272262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571de08b7d0 con 0x5571ddf804d0 2024-05-21T15:38:02.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.713+0000 7f5c777fe640 1 --2- 172.21.15.38:0/2455243410 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5c600a2d60 0x7f5c600a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:02.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.714+0000 7f5c76ffd640 1 --2- 172.21.15.38:0/3296272262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de349330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5c70098260 tx=0x7f5c700072c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c43eb164cee90e56 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:02.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.714+0000 7f5c777fe640 1 --2- 172.21.15.38:0/2455243410 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5c600a2d60 0x7f5c600a5270 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5c68005940 tx=0x7f5c68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:02.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.716+0000 7f5c5d7fa640 1 -- 172.21.15.38:0/3296272262 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7001d050 con 0x5571ddf804d0 2024-05-21T15:38:02.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.716+0000 7f5c5d7fa640 1 -- 172.21.15.38:0/3296272262 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c7009bf90 con 0x5571ddf804d0 2024-05-21T15:38:02.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.717+0000 7f5c5d7fa640 1 -- 172.21.15.38:0/3296272262 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c700d25e0 con 0x5571ddf804d0 2024-05-21T15:38:02.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.718+0000 7f5c845d2740 1 -- 172.21.15.38:0/3296272262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 msgr2=0x5571de349330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:02.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.718+0000 7f5c845d2740 1 --2- 172.21.15.38:0/3296272262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de349330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5c70098260 tx=0x7f5c700072c0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.718+0000 7f5c845d2740 1 -- 172.21.15.38:0/3296272262 shutdown_connections 2024-05-21T15:38:02.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.718+0000 7f5c845d2740 1 --2- 172.21.15.38:0/3296272262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de350230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.718+0000 7f5c845d2740 1 --2- 172.21.15.38:0/3296272262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de349330 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.718+0000 7f5c845d2740 1 --2- 172.21.15.38:0/3296272262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de348db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.718+0000 7f5c845d2740 1 -- 172.21.15.38:0/3296272262 >> 172.21.15.38:0/3296272262 conn(0x5571ddf9d4e0 msgr2=0x5571de344740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:02.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.718+0000 7f5c845d2740 1 -- 172.21.15.38:0/3296272262 shutdown_connections 2024-05-21T15:38:02.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.718+0000 7f5c845d2740 1 -- 172.21.15.38:0/3296272262 wait complete. 2024-05-21T15:38:02.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.719+0000 7f5c845d2740 1 -- start start 2024-05-21T15:38:02.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.719+0000 7f5c845d2740 1 Processor -- start 2024-05-21T15:38:02.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.719+0000 7f5c845d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de352d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.719+0000 7f5c845d2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de353240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.719+0000 7f5c845d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de35a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.719+0000 7f5c845d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5571de350c20 con 0x5571ddfa9390 2024-05-21T15:38:02.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.719+0000 7f5c845d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5571de268b90 con 0x5571ddf804d0 2024-05-21T15:38:02.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.719+0000 7f5c845d2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5571de350da0 con 0x5571ddf886b0 2024-05-21T15:38:02.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.719+0000 7f5c845d2740 10 client.0 _ll_register_callbacks cb 0x5571de2a4da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:02.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.720+0000 7f5c777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de35a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:02.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.720+0000 7f5c777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de35a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51400/0 (socket says 172.21.15.38:51400) 2024-05-21T15:38:02.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.720+0000 7f5c777fe640 1 -- 172.21.15.38:0/605211847 learned_addr learned my addr 172.21.15.38:0/605211847 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:02.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.721+0000 7f5c76ffd640 1 --2- 172.21.15.38:0/605211847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de352d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:02.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.721+0000 7f5c777fe640 1 -- 172.21.15.38:0/605211847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 msgr2=0x5571de353240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:38:02.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.721+0000 7f5c777fe640 1 --2- 172.21.15.38:0/605211847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de353240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.721+0000 7f5c777fe640 1 -- 172.21.15.38:0/605211847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 msgr2=0x5571de352d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:02.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.721+0000 7f5c777fe640 1 --2- 172.21.15.38:0/605211847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de352d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.721+0000 7f5c777fe640 1 -- 172.21.15.38:0/605211847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ddfc13f0 con 0x5571ddfa9390 2024-05-21T15:38:02.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.721+0000 7f5c777fe640 1 --2- 172.21.15.38:0/605211847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de35a1a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009900 tx=0x7f5c6802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac849f17bbfc7af server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:02.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.722+0000 7f5c5aff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:02.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.722+0000 7f5c5aff5640 1 -- 172.21.15.38:0/605211847 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6804d030 con 0x5571ddfa9390 2024-05-21T15:38:02.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.722+0000 7f5c845d2740 1 -- 172.21.15.38:0/605211847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c7009bf90 con 0x5571ddfa9390 2024-05-21T15:38:02.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.722+0000 7f5c5aff5640 1 -- 172.21.15.38:0/605211847 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f5c68003ac0 con 0x5571ddfa9390 2024-05-21T15:38:02.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.725+0000 7f5c5aff5640 1 -- 172.21.15.38:0/605211847 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5c68035040 con 0x5571ddfa9390 2024-05-21T15:38:02.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.725+0000 7f5c845d2740 10 client.16410 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:02.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.725+0000 7f5c845d2740 1 -- 172.21.15.38:0/605211847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5571ddf810f0 con 0x5571ddfa9390 2024-05-21T15:38:02.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.727+0000 7f5c5aff5640 1 -- 172.21.15.38:0/605211847 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c7009bf90 con 0x5571ddfa9390 2024-05-21T15:38:02.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.727+0000 7f5c5aff5640 1 -- 172.21.15.38:0/605211847 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f5c6803a040 con 0x5571ddfa9390 2024-05-21T15:38:02.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.728+0000 7f5c845d2740 10 client.16410 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:02.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.728+0000 7f5c845d2740 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:02.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.728+0000 7f5c845d2740 20 client.16410 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:02.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.728+0000 7f5c845d2740 1 --2- 172.21.15.38:0/605211847 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5571ddf67f60 0x5571de360ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.728+0000 7f5c845d2740 4 client.16410 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:38:02.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.728+0000 7f5c845d2740 1 -- 172.21.15.38:0/605211847 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571de2bb430 con 0x5571ddf67f60 2024-05-21T15:38:02.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.728+0000 7f5c76ffd640 1 --2- 172.21.15.38:0/605211847 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5571ddf67f60 0x5571de360ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:02.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.730+0000 7f5c76ffd640 1 --2- 172.21.15.38:0/605211847 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5571ddf67f60 0x5571de360ff0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=540982d9a8b67c2b server_cookie=e8749accdcef2530 in_seq=0 out_seq=0 2024-05-21T15:38:02.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.730+0000 7f5c5aff5640 10 client.16410 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:02.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.731+0000 7f5c5aff5640 1 -- 172.21.15.38:0/605211847 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5571de2bb430 con 0x5571ddf67f60 2024-05-21T15:38:02.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.731+0000 7f5c5aff5640 10 client.16410 handle_command_reply: tid=0 2024-05-21T15:38:02.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.731+0000 7f5c845d2740 1 client.16410 shutdown 2024-05-21T15:38:02.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.732+0000 7f5c845d2740 1 -- 172.21.15.38:0/605211847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 msgr2=0x5571de35a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:02.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.732+0000 7f5c845d2740 1 --2- 172.21.15.38:0/605211847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de35a1a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009900 tx=0x7f5c6802a3f0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.737+0000 7f5c845d2740 1 -- 172.21.15.38:0/605211847 shutdown_connections 2024-05-21T15:38:02.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.737+0000 7f5c845d2740 1 --2- 172.21.15.38:0/605211847 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5571ddf67f60 0x5571de360ff0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.737+0000 7f5c845d2740 1 --2- 172.21.15.38:0/605211847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de35a1a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.737+0000 7f5c845d2740 1 --2- 172.21.15.38:0/605211847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de353240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.737+0000 7f5c845d2740 1 --2- 172.21.15.38:0/605211847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de352d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.737+0000 7f5c845d2740 1 -- 172.21.15.38:0/605211847 >> 172.21.15.38:0/605211847 conn(0x5571ddf9d4e0 msgr2=0x5571de343670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:02.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.737+0000 7f5c5aff5640 0 client.16410 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:02.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.738+0000 7f5c845d2740 1 -- 172.21.15.38:0/605211847 shutdown_connections 2024-05-21T15:38:02.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.739+0000 7f5c845d2740 1 -- 172.21.15.38:0/605211847 wait complete. 2024-05-21T15:38:02.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.741+0000 7f5c845d2740 20 client.16410 trim_cache size 0 max 16384 2024-05-21T15:38:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.782+0000 7f5c845d2740 1 Processor -- start 2024-05-21T15:38:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.782+0000 7f5c845d2740 1 -- start start 2024-05-21T15:38:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.782+0000 7f5c845d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de351320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.782+0000 7f5c845d2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de35a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.782+0000 7f5c845d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de351860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.782+0000 7f5c845d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6400e6d0 con 0x5571ddfa9390 2024-05-21T15:38:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.782+0000 7f5c845d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64010300 con 0x5571ddf804d0 2024-05-21T15:38:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.782+0000 7f5c845d2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6400eda0 con 0x5571ddf886b0 2024-05-21T15:38:02.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.783+0000 7f5c767fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de35a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:02.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.783+0000 7f5c767fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de35a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:58720/0 (socket says 172.21.15.38:58720) 2024-05-21T15:38:02.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.783+0000 7f5c767fc640 1 -- 172.21.15.38:0/4017210784 learned_addr learned my addr 172.21.15.38:0/4017210784 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:02.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.783+0000 7f5c767fc640 1 -- 172.21.15.38:0/4017210784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 msgr2=0x5571de351320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:38:02.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.783+0000 7f5c767fc640 1 --2- 172.21.15.38:0/4017210784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de351320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.783+0000 7f5c767fc640 1 -- 172.21.15.38:0/4017210784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 msgr2=0x5571de351860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:02.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.783+0000 7f5c767fc640 1 --2- 172.21.15.38:0/4017210784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de351860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.783+0000 7f5c767fc640 1 -- 172.21.15.38:0/4017210784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571de35a600 con 0x5571ddf886b0 2024-05-21T15:38:02.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.783+0000 7f5c767fc640 1 --2- 172.21.15.38:0/4017210784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de35a0c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00b580 tx=0x7f5c6c00ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e12c30f2b3d5f6b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:02.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.784+0000 7f5c5cff9640 1 -- 172.21.15.38:0/4017210784 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0125f0 con 0x5571ddf886b0 2024-05-21T15:38:02.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.784+0000 7f5c5cff9640 1 -- 172.21.15.38:0/4017210784 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c6c019070 con 0x5571ddf886b0 2024-05-21T15:38:02.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.784+0000 7f5c5cff9640 1 -- 172.21.15.38:0/4017210784 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c014c70 con 0x5571ddf886b0 2024-05-21T15:38:02.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.785+0000 7f5c845d2740 1 -- 172.21.15.38:0/4017210784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 msgr2=0x5571de35a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:02.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.785+0000 7f5c845d2740 1 --2- 172.21.15.38:0/4017210784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de35a0c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00b580 tx=0x7f5c6c00ba40 comp rx=0 tx=0).stop 2024-05-21T15:38:02.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.786+0000 7f5c845d2740 1 -- 172.21.15.38:0/4017210784 shutdown_connections 2024-05-21T15:38:02.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.786+0000 7f5c845d2740 1 --2- 172.21.15.38:0/4017210784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de351860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.786+0000 7f5c845d2740 1 --2- 172.21.15.38:0/4017210784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de35a0c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.786+0000 7f5c845d2740 1 --2- 172.21.15.38:0/4017210784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de351320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.786+0000 7f5c845d2740 1 -- 172.21.15.38:0/4017210784 >> 172.21.15.38:0/4017210784 conn(0x5571ddf9d4e0 msgr2=0x5571de34d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:02.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.786+0000 7f5c845d2740 1 -- 172.21.15.38:0/4017210784 shutdown_connections 2024-05-21T15:38:02.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.786+0000 7f5c845d2740 1 -- 172.21.15.38:0/4017210784 wait complete. 2024-05-21T15:38:02.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.786+0000 7f5c845d2740 1 -- start start 2024-05-21T15:38:02.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.786+0000 7f5c845d2740 1 Processor -- start 2024-05-21T15:38:02.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.787+0000 7f5c845d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de359d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.787+0000 7f5c845d2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de3583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.787+0000 7f5c845d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de358910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.787+0000 7f5c845d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6400fb70 con 0x5571ddfa9390 2024-05-21T15:38:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.787+0000 7f5c845d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6400f470 con 0x5571ddf804d0 2024-05-21T15:38:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.787+0000 7f5c845d2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6400c6b0 con 0x5571ddf886b0 2024-05-21T15:38:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.787+0000 7f5c845d2740 10 client.0 _ll_register_callbacks cb 0x7f5c68009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.787+0000 7f5c767fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de3583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.787+0000 7f5c767fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de3583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:58722/0 (socket says 172.21.15.38:58722) 2024-05-21T15:38:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.787+0000 7f5c767fc640 1 -- 172.21.15.38:0/1031747301 learned_addr learned my addr 172.21.15.38:0/1031747301 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:02.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.788+0000 7f5c777fe640 1 --2- 172.21.15.38:0/1031747301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de358910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.788+0000 7f5c767fc640 1 -- 172.21.15.38:0/1031747301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 msgr2=0x5571de359d60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:38:02.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.788+0000 7f5c767fc640 1 --2- 172.21.15.38:0/1031747301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de359d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.788+0000 7f5c767fc640 1 -- 172.21.15.38:0/1031747301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 msgr2=0x5571de358910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.788+0000 7f5c767fc640 1 --2- 172.21.15.38:0/1031747301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de358910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.788+0000 7f5c767fc640 1 -- 172.21.15.38:0/1031747301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571de358e80 con 0x5571ddf886b0 2024-05-21T15:38:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.788+0000 7f5c767fc640 1 --2- 172.21.15.38:0/1031747301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de3583d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c006aa0 tx=0x7f5c6c014c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3572b619268c78b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.789+0000 7f5c5aff5640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:38:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.789+0000 7f5c5aff5640 1 -- 172.21.15.38:0/1031747301 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c024070 con 0x5571ddf886b0 2024-05-21T15:38:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.789+0000 7f5c5aff5640 1 -- 172.21.15.38:0/1031747301 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f5c6c012a00 con 0x5571ddf886b0 2024-05-21T15:38:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.789+0000 7f5c845d2740 1 -- 172.21.15.38:0/1031747301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c6c019070 con 0x5571ddf886b0 2024-05-21T15:38:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.789+0000 7f5c5aff5640 1 -- 172.21.15.38:0/1031747301 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5c6c02e050 con 0x5571ddf886b0 2024-05-21T15:38:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.789+0000 7f5c845d2740 10 client.26128 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.789+0000 7f5c845d2740 1 -- 172.21.15.38:0/1031747301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5571de3c8160 con 0x5571ddf886b0 2024-05-21T15:38:02.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.791+0000 7f5c5aff5640 1 -- 172.21.15.38:0/1031747301 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c6c019070 con 0x5571ddf886b0 2024-05-21T15:38:02.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.791+0000 7f5c5aff5640 1 -- 172.21.15.38:0/1031747301 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f5c6c031020 con 0x5571ddf886b0 2024-05-21T15:38:02.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.791+0000 7f5c845d2740 10 client.26128 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:02.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.791+0000 7f5c845d2740 10 client.26128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:02.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.791+0000 7f5c845d2740 20 client.26128 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:02.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.791+0000 7f5c845d2740 1 --2- 172.21.15.38:0/1031747301 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5571ddf3d3a0 0x5571de3cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:02.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.791+0000 7f5c845d2740 4 client.26128 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:38:02.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.791+0000 7f5c845d2740 1 -- 172.21.15.38:0/1031747301 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5571de2b5780 con 0x5571ddf3d3a0 2024-05-21T15:38:02.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.793+0000 7f5c76ffd640 1 --2- 172.21.15.38:0/1031747301 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5571ddf3d3a0 0x5571de3cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:02.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.798+0000 7f5c76ffd640 1 --2- 172.21.15.38:0/1031747301 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5571ddf3d3a0 0x5571de3cab80 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa639f415b4e6068 server_cookie=fe9f58b524785c48 in_seq=0 out_seq=0 2024-05-21T15:38:02.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.798+0000 7f5c5aff5640 10 client.26128 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:02.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.799+0000 7f5c5aff5640 1 -- 172.21.15.38:0/1031747301 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x5571de2b5780 con 0x5571ddf3d3a0 2024-05-21T15:38:02.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.799+0000 7f5c5aff5640 10 client.26128 handle_command_reply: tid=0 2024-05-21T15:38:02.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.799+0000 7f5c845d2740 1 client.26128 shutdown 2024-05-21T15:38:02.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.800+0000 7f5c845d2740 1 -- 172.21.15.38:0/1031747301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 msgr2=0x5571de3583d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:02.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.800+0000 7f5c845d2740 1 --2- 172.21.15.38:0/1031747301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de3583d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c006aa0 tx=0x7f5c6c014c90 comp rx=0 tx=0).stop 2024-05-21T15:38:02.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.800+0000 7f5c845d2740 1 -- 172.21.15.38:0/1031747301 shutdown_connections 2024-05-21T15:38:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.800+0000 7f5c845d2740 1 --2- 172.21.15.38:0/1031747301 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5571ddf3d3a0 0x5571de3cab80 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.800+0000 7f5c845d2740 1 --2- 172.21.15.38:0/1031747301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5571ddfa9390 0x5571de358910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.800+0000 7f5c845d2740 1 --2- 172.21.15.38:0/1031747301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5571ddf886b0 0x5571de3583d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.800+0000 7f5c845d2740 1 --2- 172.21.15.38:0/1031747301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571ddf804d0 0x5571de359d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.800+0000 7f5c845d2740 1 -- 172.21.15.38:0/1031747301 >> 172.21.15.38:0/1031747301 conn(0x5571ddf9d4e0 msgr2=0x5571de347540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.800+0000 7f5c5aff5640 0 client.26128 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.801+0000 7f5c845d2740 1 -- 172.21.15.38:0/1031747301 shutdown_connections 2024-05-21T15:38:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.802+0000 7f5c845d2740 1 -- 172.21.15.38:0/1031747301 wait complete. 2024-05-21T15:38:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.802+0000 7f5c845d2740 20 client.26128 trim_cache size 0 max 16384 2024-05-21T15:38:02.801 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:38:02.801 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "scrub active (27 inodes in the stack)", 2024-05-21T15:38:02.801 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-05-21T15:38:02.801 INFO:teuthology.orchestra.run.smithi038.stdout: "c10e3c7e-cde3-4123-a8a8-47a3fb376fc2": { 2024-05-21T15:38:02.801 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-05-21T15:38:02.801 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "c10e3c7e-cde3-4123-a8a8-47a3fb376fc2", 2024-05-21T15:38:02.801 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-05-21T15:38:02.801 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:38:02.801 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:38:02.802 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:38:02.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.802+0000 7f5c77fff640 1 -- 172.21.15.38:0/2455243410 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5c600a2d60 msgr2=0x7f5c600a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:02.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.802+0000 7f5c77fff640 1 --2- 172.21.15.38:0/2455243410 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5c600a2d60 0x7f5c600a5270 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5c68005940 tx=0x7f5c68009260 comp rx=0 tx=0).stop 2024-05-21T15:38:02.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.803+0000 7f5c77fff640 1 -- 172.21.15.38:0/2455243410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c781092e0 msgr2=0x7f5c78115f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:02.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.803+0000 7f5c77fff640 1 --2- 172.21.15.38:0/2455243410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c781092e0 0x7f5c78115f60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000cce0 tx=0x7f5c7000a620 comp rx=0 tx=0).stop 2024-05-21T15:38:02.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.803+0000 7f5c77fff640 1 -- 172.21.15.38:0/2455243410 shutdown_connections 2024-05-21T15:38:02.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.803+0000 7f5c77fff640 1 --2- 172.21.15.38:0/2455243410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c7810ee40 0x7f5c781173b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.803+0000 7f5c77fff640 1 --2- 172.21.15.38:0/2455243410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c7810a140 0x7f5c781164a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.803+0000 7f5c77fff640 1 --2- 172.21.15.38:0/2455243410 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f5c600a2d60 0x7f5c600a5270 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.803+0000 7f5c77fff640 1 --2- 172.21.15.38:0/2455243410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c781092e0 0x7f5c78115f60 secure :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000cce0 tx=0x7f5c7000a620 comp rx=0 tx=0).stop 2024-05-21T15:38:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.803+0000 7f5c77fff640 1 -- 172.21.15.38:0/2455243410 >> 172.21.15.38:0/2455243410 conn(0x7f5c7806f3e0 msgr2=0x7f5c7810ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.804+0000 7f5c77fff640 1 -- 172.21.15.38:0/2455243410 shutdown_connections 2024-05-21T15:38:02.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:02.805+0000 7f5c77fff640 1 -- 172.21.15.38:0/2455243410 wait complete. 2024-05-21T15:38:02.828 INFO:tasks.cephfs.filesystem:scrub status for tag:c10e3c7e-cde3-4123-a8a8-47a3fb376fc2 - {'path': '/', 'tag': 'c10e3c7e-cde3-4123-a8a8-47a3fb376fc2', 'options': 'recursive,force'} 2024-05-21T15:38:03.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:03 smithi053 ceph-mon[36496]: pgmap v460: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 129 MiB/s wr, 19.79k op/s 2024-05-21T15:38:03.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:03 smithi045 ceph-mon[36409]: pgmap v460: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 129 MiB/s wr, 19.79k op/s 2024-05-21T15:38:03.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:03 smithi038 ceph-mon[31995]: pgmap v460: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 129 MiB/s wr, 19.79k op/s 2024-05-21T15:38:05.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:05 smithi053 ceph-mon[36496]: pgmap v461: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 108 MiB/s wr, 16.75k op/s 2024-05-21T15:38:05.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:05 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:38:05.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:05 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:38:05.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:05 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:38:05.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:05 smithi045 ceph-mon[36409]: pgmap v461: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 108 MiB/s wr, 16.75k op/s 2024-05-21T15:38:05.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:05 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:38:05.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:05 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:38:05.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:05 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:38:05.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:05 smithi038 ceph-mon[31995]: pgmap v461: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 108 MiB/s wr, 16.75k op/s 2024-05-21T15:38:05.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:05 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:38:05.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:05 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:38:05.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:05 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:38:06.661 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:06 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:06.661 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:06 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:06.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:06 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:06.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:06 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:06.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:06 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:06.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:06 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.119+0000 7fce5e87a640 1 -- 172.21.15.38:0/3859272624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5810f060 msgr2=0x7fce58111480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.119+0000 7fce5e87a640 1 --2- 172.21.15.38:0/3859272624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5810f060 0x7fce58111480 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce50009be0 tx=0x7fce5002a6d0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.119+0000 7fce5e87a640 1 -- 172.21.15.38:0/3859272624 shutdown_connections 2024-05-21T15:38:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.119+0000 7fce5e87a640 1 --2- 172.21.15.38:0/3859272624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5810f060 0x7fce58111480 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.119+0000 7fce5e87a640 1 --2- 172.21.15.38:0/3859272624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce5810a360 0x7fce5810a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.119+0000 7fce5e87a640 1 --2- 172.21.15.38:0/3859272624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce58109120 0x7fce58109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.119+0000 7fce5e87a640 1 -- 172.21.15.38:0/3859272624 >> 172.21.15.38:0/3859272624 conn(0x7fce58077d30 msgr2=0x7fce58106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.119+0000 7fce5e87a640 1 -- 172.21.15.38:0/3859272624 shutdown_connections 2024-05-21T15:38:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.119+0000 7fce5e87a640 1 -- 172.21.15.38:0/3859272624 wait complete. 2024-05-21T15:38:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce5e87a640 1 Processor -- start 2024-05-21T15:38:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce5e87a640 1 -- start start 2024-05-21T15:38:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce5e87a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce58109120 0x7fce5819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce5e87a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5810a360 0x7fce5819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce5e87a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5810f060 0x7fce581a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce5e87a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fce58114d90 con 0x7fce58109120 2024-05-21T15:38:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce5e87a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce58114c10 con 0x7fce5810a360 2024-05-21T15:38:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce5e87a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fce58114f10 con 0x7fce5810f060 2024-05-21T15:38:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce57fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce58109120 0x7fce5819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce57fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce58109120 0x7fce5819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47152/0 (socket says 172.21.15.38:47152) 2024-05-21T15:38:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce57fff640 1 -- 172.21.15.38:0/3020019508 learned_addr learned my addr 172.21.15.38:0/3020019508 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce5ce17640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5810f060 0x7fce581a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce4ffff640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5810a360 0x7fce5819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce57fff640 1 -- 172.21.15.38:0/3020019508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5810f060 msgr2=0x7fce581a6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce57fff640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5810f060 0x7fce581a6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.120+0000 7fce57fff640 1 -- 172.21.15.38:0/3020019508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5810a360 msgr2=0x7fce5819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.121+0000 7fce57fff640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5810a360 0x7fce5819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.121+0000 7fce57fff640 1 -- 172.21.15.38:0/3020019508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce581a7320 con 0x7fce58109120 2024-05-21T15:38:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.121+0000 7fce5ce17640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5810f060 0x7fce581a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:38:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.121+0000 7fce4ffff640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5810a360 0x7fce5819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:38:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.121+0000 7fce57fff640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce58109120 0x7fce5819f7e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fce4800e9f0 tx=0x7fce4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.121+0000 7fce55ffb640 1 -- 172.21.15.38:0/3020019508 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4800ccc0 con 0x7fce58109120 2024-05-21T15:38:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.121+0000 7fce5e87a640 1 -- 172.21.15.38:0/3020019508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce581a7610 con 0x7fce58109120 2024-05-21T15:38:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.121+0000 7fce55ffb640 1 -- 172.21.15.38:0/3020019508 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce480035f0 con 0x7fce58109120 2024-05-21T15:38:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.121+0000 7fce55ffb640 1 -- 172.21.15.38:0/3020019508 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4800ce60 con 0x7fce58109120 2024-05-21T15:38:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.121+0000 7fce5e87a640 1 -- 172.21.15.38:0/3020019508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce581a7bb0 con 0x7fce58109120 2024-05-21T15:38:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.123+0000 7fce55ffb640 1 -- 172.21.15.38:0/3020019508 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fce480037a0 con 0x7fce58109120 2024-05-21T15:38:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce657ff740 1 Processor -- start 2024-05-21T15:38:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce657ff740 1 -- start start 2024-05-21T15:38:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce657ff740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff65e1b0 0x564cff9fde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce657ff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff6352f0 0x564cff9fe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce657ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff63d4d0 0x564cffa052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce657ff740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564cff962340 con 0x564cff65e1b0 2024-05-21T15:38:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce657ff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564cff90a290 con 0x564cff6352f0 2024-05-21T15:38:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce657ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564cff95e300 con 0x564cff63d4d0 2024-05-21T15:38:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce5ce17640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff65e1b0 0x564cff9fde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce5ce17640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff65e1b0 0x564cff9fde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47154/0 (socket says 172.21.15.38:47154) 2024-05-21T15:38:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce5ce17640 1 -- 172.21.15.38:0/738416578 learned_addr learned my addr 172.21.15.38:0/738416578 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce57fff640 1 --2- 172.21.15.38:0/738416578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff6352f0 0x564cff9fe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce4ffff640 1 --2- 172.21.15.38:0/738416578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff63d4d0 0x564cffa052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce5ce17640 1 -- 172.21.15.38:0/738416578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff63d4d0 msgr2=0x564cffa052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce5ce17640 1 --2- 172.21.15.38:0/738416578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff63d4d0 0x564cffa052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce5ce17640 1 -- 172.21.15.38:0/738416578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff6352f0 msgr2=0x564cff9fe3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.124+0000 7fce5ce17640 1 --2- 172.21.15.38:0/738416578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff6352f0 0x564cff9fe3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.125+0000 7fce5ce17640 1 -- 172.21.15.38:0/738416578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cff92e8e0 con 0x564cff65e1b0 2024-05-21T15:38:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.125+0000 7fce55ffb640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fce440a2c30 0x7fce440a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.125+0000 7fce57fff640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fce440a2c30 0x7fce440a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.125+0000 7fce5ce17640 1 --2- 172.21.15.38:0/738416578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff65e1b0 0x564cff9fde60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fce50009be0 tx=0x7fce500092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877bc35ffec2f044 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.125+0000 7fce55ffb640 1 -- 172.21.15.38:0/3020019508 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fce480cf3b0 con 0x7fce58109120 2024-05-21T15:38:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.125+0000 7fce4e7fc640 1 -- 172.21.15.38:0/738416578 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50041030 con 0x564cff65e1b0 2024-05-21T15:38:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.125+0000 7fce4e7fc640 1 -- 172.21.15.38:0/738416578 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce50003970 con 0x564cff65e1b0 2024-05-21T15:38:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.125+0000 7fce4e7fc640 1 -- 172.21.15.38:0/738416578 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50036040 con 0x564cff65e1b0 2024-05-21T15:38:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.126+0000 7fce657ff740 1 -- 172.21.15.38:0/738416578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff65e1b0 msgr2=0x564cff9fde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.126+0000 7fce57fff640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fce440a2c30 0x7fce440a5140 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x564cff9ff240 tx=0x7fce480d2e90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.126+0000 7fce657ff740 1 --2- 172.21.15.38:0/738416578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff65e1b0 0x564cff9fde60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fce50009be0 tx=0x7fce500092f0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.126+0000 7fce657ff740 1 -- 172.21.15.38:0/738416578 shutdown_connections 2024-05-21T15:38:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.126+0000 7fce657ff740 1 --2- 172.21.15.38:0/738416578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff63d4d0 0x564cffa052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.126+0000 7fce657ff740 1 --2- 172.21.15.38:0/738416578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff6352f0 0x564cff9fe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.126+0000 7fce657ff740 1 --2- 172.21.15.38:0/738416578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff65e1b0 0x564cff9fde60 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.126+0000 7fce657ff740 1 -- 172.21.15.38:0/738416578 >> 172.21.15.38:0/738416578 conn(0x564cff652300 msgr2=0x564cff9f97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.126+0000 7fce657ff740 1 -- 172.21.15.38:0/738416578 shutdown_connections 2024-05-21T15:38:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.126+0000 7fce657ff740 1 -- 172.21.15.38:0/738416578 wait complete. 2024-05-21T15:38:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.126+0000 7fce657ff740 1 -- start start 2024-05-21T15:38:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce657ff740 1 Processor -- start 2024-05-21T15:38:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce657ff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff6352f0 0x564cffa07b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce657ff740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cffa080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce657ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff65e1b0 0x564cffa0f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce657ff740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564cffa05cd0 con 0x564cff63d4d0 2024-05-21T15:38:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce657ff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564cff91d790 con 0x564cff6352f0 2024-05-21T15:38:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce657ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564cffa05e50 con 0x564cff65e1b0 2024-05-21T15:38:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce657ff740 10 client.0 _ll_register_callbacks cb 0x564cff958a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce4ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cffa080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce4ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cffa080d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47166/0 (socket says 172.21.15.38:47166) 2024-05-21T15:38:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce4ffff640 1 -- 172.21.15.38:0/4279366260 learned_addr learned my addr 172.21.15.38:0/4279366260 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce5ce17640 1 --2- 172.21.15.38:0/4279366260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff65e1b0 0x564cffa0f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce5ce17640 1 --2- 172.21.15.38:0/4279366260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff6352f0 0x564cffa07b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce4ffff640 1 -- 172.21.15.38:0/4279366260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff65e1b0 msgr2=0x564cffa0f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce4ffff640 1 --2- 172.21.15.38:0/4279366260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff65e1b0 0x564cffa0f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce4ffff640 1 -- 172.21.15.38:0/4279366260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff6352f0 msgr2=0x564cffa07b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce4ffff640 1 --2- 172.21.15.38:0/4279366260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff6352f0 0x564cffa07b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.127+0000 7fce4ffff640 1 -- 172.21.15.38:0/4279366260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cff676210 con 0x564cff63d4d0 2024-05-21T15:38:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce4ffff640 1 --2- 172.21.15.38:0/4279366260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cffa080d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fce40012b10 tx=0x7fce4000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb47a9be4ecaaa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce377fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce377fe640 1 -- 172.21.15.38:0/4279366260 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4000fb40 con 0x564cff63d4d0 2024-05-21T15:38:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce377fe640 1 -- 172.21.15.38:0/4279366260 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fce4000fce0 con 0x564cff63d4d0 2024-05-21T15:38:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce657ff740 1 -- 172.21.15.38:0/4279366260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce50003970 con 0x564cff63d4d0 2024-05-21T15:38:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce377fe640 1 -- 172.21.15.38:0/4279366260 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fce40004050 con 0x564cff63d4d0 2024-05-21T15:38:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce657ff740 10 client.16425 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce657ff740 1 -- 172.21.15.38:0/4279366260 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564cff742620 con 0x564cff63d4d0 2024-05-21T15:38:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce377fe640 1 -- 172.21.15.38:0/4279366260 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce50003970 con 0x564cff63d4d0 2024-05-21T15:38:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce377fe640 1 -- 172.21.15.38:0/4279366260 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fce40010430 con 0x564cff63d4d0 2024-05-21T15:38:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce657ff740 10 client.16425 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce657ff740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce657ff740 20 client.16425 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce657ff740 1 --2- 172.21.15.38:0/4279366260 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564cff61cd80 0x564cffa15660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce657ff740 4 client.16425 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:38:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.128+0000 7fce657ff740 1 -- 172.21.15.38:0/4279366260 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cff96fa70 con 0x564cff61cd80 2024-05-21T15:38:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.129+0000 7fce5ce17640 1 --2- 172.21.15.38:0/4279366260 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564cff61cd80 0x564cffa15660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.129+0000 7fce377fe640 10 client.16425 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.129+0000 7fce5ce17640 1 --2- 172.21.15.38:0/4279366260 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564cff61cd80 0x564cffa15660 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da27fdae0662ce8f server_cookie=32c1bcb5cf9265f9 in_seq=0 out_seq=0 2024-05-21T15:38:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce377fe640 1 -- 172.21.15.38:0/4279366260 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564cff96fa70 con 0x564cff61cd80 2024-05-21T15:38:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce377fe640 10 client.16425 handle_command_reply: tid=0 2024-05-21T15:38:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce657ff740 1 client.16425 shutdown 2024-05-21T15:38:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce657ff740 1 -- 172.21.15.38:0/4279366260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 msgr2=0x564cffa080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce657ff740 1 --2- 172.21.15.38:0/4279366260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cffa080d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fce40012b10 tx=0x7fce4000f3c0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce657ff740 1 -- 172.21.15.38:0/4279366260 shutdown_connections 2024-05-21T15:38:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce657ff740 1 --2- 172.21.15.38:0/4279366260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff65e1b0 0x564cffa0f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce657ff740 1 --2- 172.21.15.38:0/4279366260 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564cff61cd80 0x564cffa15660 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce657ff740 1 --2- 172.21.15.38:0/4279366260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cffa080d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce657ff740 1 --2- 172.21.15.38:0/4279366260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff6352f0 0x564cffa07b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce377fe640 0 client.16425 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce657ff740 1 -- 172.21.15.38:0/4279366260 >> 172.21.15.38:0/4279366260 conn(0x564cff652300 msgr2=0x564cff9fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce657ff740 1 -- 172.21.15.38:0/4279366260 shutdown_connections 2024-05-21T15:38:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce657ff740 1 -- 172.21.15.38:0/4279366260 wait complete. 2024-05-21T15:38:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.130+0000 7fce657ff740 20 client.16425 trim_cache size 0 max 16384 2024-05-21T15:38:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.147+0000 7fce657ff740 1 Processor -- start 2024-05-21T15:38:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.147+0000 7fce657ff740 1 -- start start 2024-05-21T15:38:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce657ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff6352f0 0x564cffa02660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce657ff740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cff9fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce657ff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff65e1b0 0x564cffa0ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce657ff740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fce581b0990 con 0x564cff63d4d0 2024-05-21T15:38:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce657ff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce581b25c0 con 0x564cff65e1b0 2024-05-21T15:38:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce657ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fce581b1060 con 0x564cff6352f0 2024-05-21T15:38:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce4ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cff9fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce4ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cff9fcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47176/0 (socket says 172.21.15.38:47176) 2024-05-21T15:38:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce4ffff640 1 -- 172.21.15.38:0/786399475 learned_addr learned my addr 172.21.15.38:0/786399475 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce5ce17640 1 --2- 172.21.15.38:0/786399475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff6352f0 0x564cffa02660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce5ce17640 1 --2- 172.21.15.38:0/786399475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff65e1b0 0x564cffa0ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce4ffff640 1 -- 172.21.15.38:0/786399475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff6352f0 msgr2=0x564cffa02660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce4ffff640 1 --2- 172.21.15.38:0/786399475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff6352f0 0x564cffa02660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce4ffff640 1 -- 172.21.15.38:0/786399475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff65e1b0 msgr2=0x564cffa0ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce4ffff640 1 --2- 172.21.15.38:0/786399475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff65e1b0 0x564cffa0ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce4ffff640 1 -- 172.21.15.38:0/786399475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cff9fd2e0 con 0x564cff63d4d0 2024-05-21T15:38:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce4ffff640 1 --2- 172.21.15.38:0/786399475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cff9fcda0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fce40010880 tx=0x7fce40012830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dead820e27b87898 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce4dffb640 1 -- 172.21.15.38:0/786399475 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4002c070 con 0x564cff63d4d0 2024-05-21T15:38:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce4dffb640 1 -- 172.21.15.38:0/786399475 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce40010430 con 0x564cff63d4d0 2024-05-21T15:38:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.148+0000 7fce4dffb640 1 -- 172.21.15.38:0/786399475 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40005680 con 0x564cff63d4d0 2024-05-21T15:38:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.149+0000 7fce657ff740 1 -- 172.21.15.38:0/786399475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 msgr2=0x564cff9fcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.149+0000 7fce657ff740 1 --2- 172.21.15.38:0/786399475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cff9fcda0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fce40010880 tx=0x7fce40012830 comp rx=0 tx=0).stop 2024-05-21T15:38:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.149+0000 7fce657ff740 1 -- 172.21.15.38:0/786399475 shutdown_connections 2024-05-21T15:38:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.149+0000 7fce657ff740 1 --2- 172.21.15.38:0/786399475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff65e1b0 0x564cffa0ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.149+0000 7fce657ff740 1 --2- 172.21.15.38:0/786399475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cff9fcda0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.149+0000 7fce657ff740 1 --2- 172.21.15.38:0/786399475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff6352f0 0x564cffa02660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.149+0000 7fce657ff740 1 -- 172.21.15.38:0/786399475 >> 172.21.15.38:0/786399475 conn(0x564cff652300 msgr2=0x564cff9fc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.149+0000 7fce657ff740 1 -- 172.21.15.38:0/786399475 shutdown_connections 2024-05-21T15:38:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.149+0000 7fce657ff740 1 -- 172.21.15.38:0/786399475 wait complete. 2024-05-21T15:38:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.150+0000 7fce657ff740 1 -- start start 2024-05-21T15:38:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.150+0000 7fce657ff740 1 Processor -- start 2024-05-21T15:38:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.150+0000 7fce657ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff6352f0 0x564cffa0cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.150+0000 7fce657ff740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cffa0d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.150+0000 7fce657ff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff65e1b0 0x564cffa0df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.150+0000 7fce657ff740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fce581b1e30 con 0x564cff63d4d0 2024-05-21T15:38:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.150+0000 7fce657ff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce581b1730 con 0x564cff65e1b0 2024-05-21T15:38:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.150+0000 7fce657ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fce581ae970 con 0x564cff6352f0 2024-05-21T15:38:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.150+0000 7fce657ff740 10 client.0 _ll_register_callbacks cb 0x7fce40009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.150+0000 7fce4ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cffa0d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.150+0000 7fce4ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cffa0d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47184/0 (socket says 172.21.15.38:47184) 2024-05-21T15:38:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.150+0000 7fce4ffff640 1 -- 172.21.15.38:0/2296190087 learned_addr learned my addr 172.21.15.38:0/2296190087 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.150+0000 7fce5ce17640 1 --2- 172.21.15.38:0/2296190087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff65e1b0 0x564cffa0df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce5ce17640 1 --2- 172.21.15.38:0/2296190087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff6352f0 0x564cffa0cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce4ffff640 1 -- 172.21.15.38:0/2296190087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff6352f0 msgr2=0x564cffa0cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce4ffff640 1 --2- 172.21.15.38:0/2296190087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff6352f0 0x564cffa0cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce4ffff640 1 -- 172.21.15.38:0/2296190087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff65e1b0 msgr2=0x564cffa0df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce4ffff640 1 --2- 172.21.15.38:0/2296190087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff65e1b0 0x564cffa0df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce4ffff640 1 -- 172.21.15.38:0/2296190087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cffa0e540 con 0x564cff63d4d0 2024-05-21T15:38:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce4ffff640 1 --2- 172.21.15.38:0/2296190087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cffa0d050 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fce40006010 tx=0x7fce4000fdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7891b6776ab4ffe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce377fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce377fe640 1 -- 172.21.15.38:0/2296190087 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4002c070 con 0x564cff63d4d0 2024-05-21T15:38:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce377fe640 1 -- 172.21.15.38:0/2296190087 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fce40005610 con 0x564cff63d4d0 2024-05-21T15:38:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce657ff740 1 -- 172.21.15.38:0/2296190087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce40010430 con 0x564cff63d4d0 2024-05-21T15:38:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce377fe640 1 -- 172.21.15.38:0/2296190087 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fce40040050 con 0x564cff63d4d0 2024-05-21T15:38:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce657ff740 10 client.16431 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce657ff740 1 -- 172.21.15.38:0/2296190087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564cffa0f7d0 con 0x564cff63d4d0 2024-05-21T15:38:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.151+0000 7fce377fe640 1 -- 172.21.15.38:0/2296190087 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce40010430 con 0x564cff63d4d0 2024-05-21T15:38:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.152+0000 7fce377fe640 1 -- 172.21.15.38:0/2296190087 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fce40003cc0 con 0x564cff63d4d0 2024-05-21T15:38:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.152+0000 7fce657ff740 10 client.16431 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.152+0000 7fce657ff740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.152+0000 7fce657ff740 20 client.16431 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.152+0000 7fce657ff740 1 --2- 172.21.15.38:0/2296190087 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564cff5f21c0 0x564cffa7fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.152+0000 7fce657ff740 4 client.16431 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:38:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.152+0000 7fce657ff740 1 -- 172.21.15.38:0/2296190087 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564cff969dc0 con 0x564cff5f21c0 2024-05-21T15:38:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.152+0000 7fce5ce17640 1 --2- 172.21.15.38:0/2296190087 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564cff5f21c0 0x564cffa7fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.152+0000 7fce377fe640 10 client.16431 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.152+0000 7fce5ce17640 1 --2- 172.21.15.38:0/2296190087 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564cff5f21c0 0x564cffa7fef0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8c18ccb1301f218 server_cookie=9e2f0e29cddc03f1 in_seq=0 out_seq=0 2024-05-21T15:38:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce377fe640 1 -- 172.21.15.38:0/2296190087 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564cff969dc0 con 0x564cff5f21c0 2024-05-21T15:38:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce377fe640 10 client.16431 handle_command_reply: tid=0 2024-05-21T15:38:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce657ff740 1 client.16431 shutdown 2024-05-21T15:38:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce657ff740 1 -- 172.21.15.38:0/2296190087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 msgr2=0x564cffa0d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce657ff740 1 --2- 172.21.15.38:0/2296190087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cffa0d050 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fce40006010 tx=0x7fce4000fdf0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce657ff740 1 -- 172.21.15.38:0/2296190087 shutdown_connections 2024-05-21T15:38:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce657ff740 1 --2- 172.21.15.38:0/2296190087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cff65e1b0 0x564cffa0df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce657ff740 1 --2- 172.21.15.38:0/2296190087 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564cff5f21c0 0x564cffa7fef0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce657ff740 1 --2- 172.21.15.38:0/2296190087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564cff63d4d0 0x564cffa0d050 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce657ff740 1 --2- 172.21.15.38:0/2296190087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564cff6352f0 0x564cffa0cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce377fe640 0 client.16431 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce657ff740 1 -- 172.21.15.38:0/2296190087 >> 172.21.15.38:0/2296190087 conn(0x564cff652300 msgr2=0x564cffa19e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce657ff740 1 -- 172.21.15.38:0/2296190087 shutdown_connections 2024-05-21T15:38:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce657ff740 1 -- 172.21.15.38:0/2296190087 wait complete. 2024-05-21T15:38:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.592+0000 7fce657ff740 20 client.16431 trim_cache size 0 max 16384 2024-05-21T15:38:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:38:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:38:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:38:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:38:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.593+0000 7fce5e87a640 1 -- 172.21.15.38:0/3020019508 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fce440a2c30 msgr2=0x7fce440a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.593+0000 7fce5e87a640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fce440a2c30 0x7fce440a5140 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x564cff9ff240 tx=0x7fce480d2e90 comp rx=0 tx=0).stop 2024-05-21T15:38:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.593+0000 7fce5e87a640 1 -- 172.21.15.38:0/3020019508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce58109120 msgr2=0x7fce5819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.593+0000 7fce5e87a640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce58109120 0x7fce5819f7e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fce4800e9f0 tx=0x7fce4800edf0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.593+0000 7fce5e87a640 1 -- 172.21.15.38:0/3020019508 shutdown_connections 2024-05-21T15:38:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.593+0000 7fce5e87a640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5810f060 0x7fce581a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.593+0000 7fce5e87a640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce5810a360 0x7fce5819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.593+0000 7fce5e87a640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fce440a2c30 0x7fce440a5140 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.593+0000 7fce5e87a640 1 --2- 172.21.15.38:0/3020019508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce58109120 0x7fce5819f7e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.593+0000 7fce5e87a640 1 -- 172.21.15.38:0/3020019508 >> 172.21.15.38:0/3020019508 conn(0x7fce58077d30 msgr2=0x7fce5810b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.593+0000 7fce5e87a640 1 -- 172.21.15.38:0/3020019508 shutdown_connections 2024-05-21T15:38:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:07.593+0000 7fce5e87a640 1 -- 172.21.15.38:0/3020019508 wait complete. 2024-05-21T15:38:07.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:07 smithi053 ceph-mon[36496]: pgmap v462: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 117 MiB/s wr, 18.38k op/s 2024-05-21T15:38:07.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:07 smithi045 ceph-mon[36409]: pgmap v462: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 117 MiB/s wr, 18.38k op/s 2024-05-21T15:38:07.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:07 smithi038 ceph-mon[31995]: pgmap v462: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 117 MiB/s wr, 18.38k op/s 2024-05-21T15:38:09.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:09 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:09.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:09 smithi053 ceph-mon[36496]: pgmap v463: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 109 MiB/s wr, 17.46k op/s 2024-05-21T15:38:09.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:09 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:09.760 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:09 smithi045 ceph-mon[36409]: pgmap v463: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 109 MiB/s wr, 17.46k op/s 2024-05-21T15:38:09.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:09 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:09.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:09 smithi038 ceph-mon[31995]: pgmap v463: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 109 MiB/s wr, 17.46k op/s 2024-05-21T15:38:11.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:11 smithi053 ceph-mon[36496]: pgmap v464: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 90 MiB/s wr, 14.46k op/s 2024-05-21T15:38:11.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:11 smithi045 ceph-mon[36409]: pgmap v464: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 90 MiB/s wr, 14.46k op/s 2024-05-21T15:38:11.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:11 smithi038 ceph-mon[31995]: pgmap v464: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 90 MiB/s wr, 14.46k op/s 2024-05-21T15:38:13.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:13 smithi053 ceph-mon[36496]: pgmap v465: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 91 MiB/s wr, 14.69k op/s 2024-05-21T15:38:13.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:13 smithi045 ceph-mon[36409]: pgmap v465: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 91 MiB/s wr, 14.69k op/s 2024-05-21T15:38:13.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:13 smithi038 ceph-mon[31995]: pgmap v465: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 91 MiB/s wr, 14.69k op/s 2024-05-21T15:38:14.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:14 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:38:14.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:14 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:38:14.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:14 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:38:14.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:14 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:38:14.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:14 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:38:14.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:14 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:38:15.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:15 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:38:15.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:15 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:38:15.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:15 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:38:15.705 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:15 smithi053 ceph-mon[36496]: pgmap v466: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 81 MiB/s wr, 13.24k op/s 2024-05-21T15:38:15.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:15 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:38:15.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:15 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:38:15.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:15 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:38:15.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:15 smithi045 ceph-mon[36409]: pgmap v466: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 81 MiB/s wr, 13.24k op/s 2024-05-21T15:38:15.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:15 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:38:15.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:15 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:38:15.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:15 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:38:15.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:15 smithi038 ceph-mon[31995]: pgmap v466: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 81 MiB/s wr, 13.24k op/s 2024-05-21T15:38:17.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:17 smithi053 ceph-mon[36496]: pgmap v467: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 92 MiB/s wr, 15.13k op/s 2024-05-21T15:38:17.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:17 smithi045 ceph-mon[36409]: pgmap v467: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 92 MiB/s wr, 15.13k op/s 2024-05-21T15:38:17.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:17 smithi038 ceph-mon[31995]: pgmap v467: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 92 MiB/s wr, 15.13k op/s 2024-05-21T15:38:19.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:19 smithi053 ceph-mon[36496]: pgmap v468: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 95 MiB/s wr, 15.63k op/s 2024-05-21T15:38:19.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:19 smithi045 ceph-mon[36409]: pgmap v468: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 95 MiB/s wr, 15.63k op/s 2024-05-21T15:38:19.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:19 smithi038 ceph-mon[31995]: pgmap v468: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 95 MiB/s wr, 15.63k op/s 2024-05-21T15:38:20.766 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.767+0000 7f2557fff640 1 -- 172.21.15.38:0/3056282847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25581085a0 msgr2=0x7f255810aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:20.766 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.767+0000 7f2557fff640 1 --2- 172.21.15.38:0/3056282847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25581085a0 0x7f255810aa80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2550009ca0 tx=0x7f255002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.767 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.769+0000 7f2557fff640 1 -- 172.21.15.38:0/3056282847 shutdown_connections 2024-05-21T15:38:20.767 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.769+0000 7f2557fff640 1 --2- 172.21.15.38:0/3056282847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25581085a0 0x7f255810aa80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.767 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.769+0000 7f2557fff640 1 --2- 172.21.15.38:0/3056282847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2558105a30 0x7f2558107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.767 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.769+0000 7f2557fff640 1 --2- 172.21.15.38:0/3056282847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2558103040 0x7f2558105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.767 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.769+0000 7f2557fff640 1 -- 172.21.15.38:0/3056282847 >> 172.21.15.38:0/3056282847 conn(0x7f25580fc9e0 msgr2=0x7f25580fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:20.768 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.769+0000 7f2557fff640 1 -- 172.21.15.38:0/3056282847 shutdown_connections 2024-05-21T15:38:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.773+0000 7f2557fff640 1 -- 172.21.15.38:0/3056282847 wait complete. 2024-05-21T15:38:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.775+0000 7f2557fff640 1 Processor -- start 2024-05-21T15:38:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.776+0000 7f2557fff640 1 -- start start 2024-05-21T15:38:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.776+0000 7f2557fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2558103040 0x7f25580708f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.776+0000 7f2557fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2558105a30 0x7f2558070e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.776+0000 7f2557fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25581085a0 0x7f2558071d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.776+0000 7f2557fff640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f255810e5b0 con 0x7f25581085a0 2024-05-21T15:38:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.776+0000 7f2557fff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f255810e430 con 0x7f2558103040 2024-05-21T15:38:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.776+0000 7f2557fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f255810e730 con 0x7f2558105a30 2024-05-21T15:38:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.777+0000 7f25577fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25581085a0 0x7f2558071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.777+0000 7f25577fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25581085a0 0x7f2558071d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47664/0 (socket says 172.21.15.38:47664) 2024-05-21T15:38:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.777+0000 7f25577fe640 1 -- 172.21.15.38:0/80211062 learned_addr learned my addr 172.21.15.38:0/80211062 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.778+0000 7f2556ffd640 1 --2- 172.21.15.38:0/80211062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2558105a30 0x7f2558070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.778+0000 7f254effd640 1 --2- 172.21.15.38:0/80211062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2558103040 0x7f25580708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.779+0000 7f25577fe640 1 -- 172.21.15.38:0/80211062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2558105a30 msgr2=0x7f2558070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.779+0000 7f25577fe640 1 --2- 172.21.15.38:0/80211062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2558105a30 0x7f2558070e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.779+0000 7f25577fe640 1 -- 172.21.15.38:0/80211062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2558103040 msgr2=0x7f25580708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.779+0000 7f25577fe640 1 --2- 172.21.15.38:0/80211062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2558103040 0x7f25580708f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.779+0000 7f25577fe640 1 -- 172.21.15.38:0/80211062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2558072400 con 0x7f25581085a0 2024-05-21T15:38:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.780+0000 7f25577fe640 1 --2- 172.21.15.38:0/80211062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25581085a0 0x7f2558071d90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2550031c10 tx=0x7f25500319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.781+0000 7f255c943640 1 -- 172.21.15.38:0/80211062 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550003bf0 con 0x7f25581085a0 2024-05-21T15:38:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.781+0000 7f255c943640 1 -- 172.21.15.38:0/80211062 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2550003d90 con 0x7f25581085a0 2024-05-21T15:38:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.781+0000 7f255c943640 1 -- 172.21.15.38:0/80211062 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550003bf0 con 0x7f25581085a0 2024-05-21T15:38:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.781+0000 7f2557fff640 1 -- 172.21.15.38:0/80211062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2558072690 con 0x7f25581085a0 2024-05-21T15:38:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.782+0000 7f2557fff640 1 -- 172.21.15.38:0/80211062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2558072b90 con 0x7f25581085a0 2024-05-21T15:38:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.787+0000 7f25647f3740 1 Processor -- start 2024-05-21T15:38:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.789+0000 7f255c943640 1 -- 172.21.15.38:0/80211062 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f255002f9f0 con 0x7f25581085a0 2024-05-21T15:38:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.790+0000 7f255c943640 1 --2- 172.21.15.38:0/80211062 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f25380a3080 0x7f25380a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.791+0000 7f255c943640 1 -- 172.21.15.38:0/80211062 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f25500039f0 con 0x7f25581085a0 2024-05-21T15:38:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.791+0000 7f25647f3740 1 -- start start 2024-05-21T15:38:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.792+0000 7f25647f3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521391b0 0x560e524e3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.792+0000 7f25647f3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521102f0 0x560e524e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.792+0000 7f25647f3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521184d0 0x560e524e8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.792+0000 7f25647f3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560e5243d340 con 0x560e521391b0 2024-05-21T15:38:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.792+0000 7f25647f3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560e523e5290 con 0x560e521184d0 2024-05-21T15:38:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.792+0000 7f25647f3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560e52439300 con 0x560e521102f0 2024-05-21T15:38:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.793+0000 7f25577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521184d0 0x560e524e8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.793+0000 7f25577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521184d0 0x560e524e8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:49230/0 (socket says 172.21.15.38:49230) 2024-05-21T15:38:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.793+0000 7f25577fe640 1 -- 172.21.15.38:0/4168841466 learned_addr learned my addr 172.21.15.38:0/4168841466 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.794+0000 7f25577fe640 1 -- 172.21.15.38:0/4168841466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521102f0 msgr2=0x560e524e64b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:38:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.794+0000 7f25577fe640 1 --2- 172.21.15.38:0/4168841466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521102f0 0x560e524e64b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.794+0000 7f25577fe640 1 -- 172.21.15.38:0/4168841466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521391b0 msgr2=0x560e524e3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.794+0000 7f25577fe640 1 --2- 172.21.15.38:0/4168841466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521391b0 0x560e524e3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.794+0000 7f25577fe640 1 -- 172.21.15.38:0/4168841466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e524098e0 con 0x560e521184d0 2024-05-21T15:38:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.795+0000 7f25577fe640 1 --2- 172.21.15.38:0/4168841466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521184d0 0x560e524e8c10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2550006010 tx=0x7f25500c40f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d8225f9a1105560 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.796+0000 7f254dffb640 1 -- 172.21.15.38:0/4168841466 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25500b94f0 con 0x560e521184d0 2024-05-21T15:38:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.797+0000 7f2556ffd640 1 --2- 172.21.15.38:0/80211062 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f25380a3080 0x7f25380a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.797+0000 7f254dffb640 1 -- 172.21.15.38:0/4168841466 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2550036030 con 0x560e521184d0 2024-05-21T15:38:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.798+0000 7f25647f3740 1 -- 172.21.15.38:0/4168841466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521184d0 msgr2=0x560e524e8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.798+0000 7f25647f3740 1 --2- 172.21.15.38:0/4168841466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521184d0 0x560e524e8c10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2550006010 tx=0x7f25500c40f0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.802+0000 7f25647f3740 1 -- 172.21.15.38:0/4168841466 shutdown_connections 2024-05-21T15:38:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.802+0000 7f25647f3740 1 --2- 172.21.15.38:0/4168841466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521184d0 0x560e524e8c10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.802+0000 7f25647f3740 1 --2- 172.21.15.38:0/4168841466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521102f0 0x560e524e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.802+0000 7f25647f3740 1 --2- 172.21.15.38:0/4168841466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521391b0 0x560e524e3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.802+0000 7f25647f3740 1 -- 172.21.15.38:0/4168841466 >> 172.21.15.38:0/4168841466 conn(0x560e5212d300 msgr2=0x560e524d4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.805+0000 7f25647f3740 1 -- 172.21.15.38:0/4168841466 shutdown_connections 2024-05-21T15:38:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.806+0000 7f2556ffd640 1 --2- 172.21.15.38:0/80211062 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f25380a3080 0x7f25380a5590 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x560e524d8390 tx=0x7f2548005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.808+0000 7f25647f3740 1 -- 172.21.15.38:0/4168841466 wait complete. 2024-05-21T15:38:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.808+0000 7f25647f3740 1 -- start start 2024-05-21T15:38:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.809+0000 7f25647f3740 1 Processor -- start 2024-05-21T15:38:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.809+0000 7f25647f3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521102f0 0x560e524eb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.809+0000 7f25647f3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521184d0 0x560e524eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.809+0000 7f25647f3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521391b0 0x560e524f2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.809+0000 7f25647f3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560e523e5290 con 0x560e521184d0 2024-05-21T15:38:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.809+0000 7f25647f3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560e523f8790 con 0x560e521102f0 2024-05-21T15:38:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.809+0000 7f25647f3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560e524e94f0 con 0x560e521391b0 2024-05-21T15:38:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.809+0000 7f25647f3740 10 client.0 _ll_register_callbacks cb 0x560e52433a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.811+0000 7f254effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521102f0 0x560e524eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.811+0000 7f25577fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521391b0 0x560e524f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.811+0000 7f254effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521102f0 0x560e524eb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:49246/0 (socket says 172.21.15.38:49246) 2024-05-21T15:38:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.811+0000 7f254effd640 1 -- 172.21.15.38:0/2259372021 learned_addr learned my addr 172.21.15.38:0/2259372021 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.815+0000 7f2556ffd640 1 --2- 172.21.15.38:0/2259372021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521184d0 0x560e524eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.816+0000 7f254effd640 1 -- 172.21.15.38:0/2259372021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521391b0 msgr2=0x560e524f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.816+0000 7f254effd640 1 --2- 172.21.15.38:0/2259372021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521391b0 0x560e524f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.816+0000 7f254effd640 1 -- 172.21.15.38:0/2259372021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521184d0 msgr2=0x560e524eb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.816+0000 7f254effd640 1 --2- 172.21.15.38:0/2259372021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521184d0 0x560e524eb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.816+0000 7f254effd640 1 -- 172.21.15.38:0/2259372021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e52151210 con 0x560e521102f0 2024-05-21T15:38:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.817+0000 7f254effd640 1 --2- 172.21.15.38:0/2259372021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521102f0 0x560e524eb060 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f254000bd80 tx=0x7f254000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e6f531f03b8ade5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.817+0000 7f2546ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:38:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.819+0000 7f2546ffd640 1 -- 172.21.15.38:0/2259372021 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540015070 con 0x560e521102f0 2024-05-21T15:38:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.819+0000 7f2546ffd640 1 -- 172.21.15.38:0/2259372021 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2540011650 con 0x560e521102f0 2024-05-21T15:38:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.820+0000 7f25647f3740 1 -- 172.21.15.38:0/2259372021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2550036030 con 0x560e521102f0 2024-05-21T15:38:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.822+0000 7f2546ffd640 1 -- 172.21.15.38:0/2259372021 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2540011850 con 0x560e521102f0 2024-05-21T15:38:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.823+0000 7f25647f3740 10 client.26099 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.823+0000 7f25647f3740 1 -- 172.21.15.38:0/2259372021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560e5221d620 con 0x560e521102f0 2024-05-21T15:38:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.825+0000 7f2546ffd640 1 -- 172.21.15.38:0/2259372021 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2550036030 con 0x560e521102f0 2024-05-21T15:38:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.825+0000 7f2546ffd640 1 -- 172.21.15.38:0/2259372021 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f2540024420 con 0x560e521102f0 2024-05-21T15:38:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.826+0000 7f25647f3740 10 client.26099 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.826+0000 7f25647f3740 10 client.26099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.826+0000 7f25647f3740 20 client.26099 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.826+0000 7f25647f3740 1 --2- 172.21.15.38:0/2259372021 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e520f7d80 0x560e524de820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.826+0000 7f25647f3740 4 client.26099 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:38:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.826+0000 7f25647f3740 1 -- 172.21.15.38:0/2259372021 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e5244aa70 con 0x560e520f7d80 2024-05-21T15:38:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.827+0000 7f2556ffd640 1 --2- 172.21.15.38:0/2259372021 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e520f7d80 0x560e524de820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.829+0000 7f2556ffd640 1 --2- 172.21.15.38:0/2259372021 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e520f7d80 0x560e524de820 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bed10e2822b7954 server_cookie=9d765e11165c929f in_seq=0 out_seq=0 2024-05-21T15:38:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.829+0000 7f2546ffd640 10 client.26099 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.830+0000 7f2546ffd640 1 -- 172.21.15.38:0/2259372021 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e5244aa70 con 0x560e520f7d80 2024-05-21T15:38:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.830+0000 7f2546ffd640 10 client.26099 handle_command_reply: tid=0 2024-05-21T15:38:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.831+0000 7f25647f3740 1 client.26099 shutdown 2024-05-21T15:38:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.831+0000 7f25647f3740 1 -- 172.21.15.38:0/2259372021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521102f0 msgr2=0x560e524eb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.831+0000 7f25647f3740 1 --2- 172.21.15.38:0/2259372021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521102f0 0x560e524eb060 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f254000bd80 tx=0x7f254000f730 comp rx=0 tx=0).stop 2024-05-21T15:38:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.833+0000 7f25647f3740 1 -- 172.21.15.38:0/2259372021 shutdown_connections 2024-05-21T15:38:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.833+0000 7f25647f3740 1 --2- 172.21.15.38:0/2259372021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521391b0 0x560e524f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.833+0000 7f25647f3740 1 --2- 172.21.15.38:0/2259372021 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e520f7d80 0x560e524de820 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.833+0000 7f25647f3740 1 --2- 172.21.15.38:0/2259372021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521184d0 0x560e524eb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.833+0000 7f25647f3740 1 --2- 172.21.15.38:0/2259372021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521102f0 0x560e524eb060 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.833+0000 7f25647f3740 1 -- 172.21.15.38:0/2259372021 >> 172.21.15.38:0/2259372021 conn(0x560e5212d300 msgr2=0x560e524d38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.833+0000 7f2546ffd640 0 client.26099 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.837+0000 7f25647f3740 1 -- 172.21.15.38:0/2259372021 shutdown_connections 2024-05-21T15:38:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.838+0000 7f25647f3740 1 -- 172.21.15.38:0/2259372021 wait complete. 2024-05-21T15:38:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.839+0000 7f25647f3740 20 client.26099 trim_cache size 0 max 16384 2024-05-21T15:38:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.874+0000 7f25647f3740 1 Processor -- start 2024-05-21T15:38:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.874+0000 7f25647f3740 1 -- start start 2024-05-21T15:38:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.875+0000 7f25647f3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521102f0 0x560e524d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.875+0000 7f25647f3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521184d0 0x560e52508a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.875+0000 7f25647f3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521391b0 0x560e52508fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.875+0000 7f25647f3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f25581b08b0 con 0x560e521102f0 2024-05-21T15:38:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.875+0000 7f25647f3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f25581b24e0 con 0x560e521184d0 2024-05-21T15:38:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.875+0000 7f25647f3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f25581b0f80 con 0x560e521391b0 2024-05-21T15:38:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.875+0000 7f25577fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521391b0 0x560e52508fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.875+0000 7f25577fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521391b0 0x560e52508fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46364/0 (socket says 172.21.15.38:46364) 2024-05-21T15:38:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.875+0000 7f25577fe640 1 -- 172.21.15.38:0/2263619514 learned_addr learned my addr 172.21.15.38:0/2263619514 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.876+0000 7f254effd640 1 --2- 172.21.15.38:0/2263619514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521102f0 0x560e524d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.876+0000 7f254effd640 1 -- 172.21.15.38:0/2263619514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521391b0 msgr2=0x560e52508fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.876+0000 7f254effd640 1 --2- 172.21.15.38:0/2263619514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521391b0 0x560e52508fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.876+0000 7f254effd640 1 -- 172.21.15.38:0/2263619514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521184d0 msgr2=0x560e52508a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.876+0000 7f254effd640 1 --2- 172.21.15.38:0/2263619514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521184d0 0x560e52508a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.876+0000 7f254effd640 1 -- 172.21.15.38:0/2263619514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e524e3880 con 0x560e521102f0 2024-05-21T15:38:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.877+0000 7f254effd640 1 --2- 172.21.15.38:0/2263619514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521102f0 0x560e524d35b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2540024820 tx=0x7f2540005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3b98aa085b7a87 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.877+0000 7f254d7fa640 1 -- 172.21.15.38:0/2263619514 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540015040 con 0x560e521102f0 2024-05-21T15:38:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.877+0000 7f254d7fa640 1 -- 172.21.15.38:0/2263619514 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2540024420 con 0x560e521102f0 2024-05-21T15:38:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.877+0000 7f254d7fa640 1 -- 172.21.15.38:0/2263619514 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540004080 con 0x560e521102f0 2024-05-21T15:38:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.879+0000 7f25647f3740 1 -- 172.21.15.38:0/2263619514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521102f0 msgr2=0x560e524d35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.879+0000 7f25647f3740 1 --2- 172.21.15.38:0/2263619514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521102f0 0x560e524d35b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2540024820 tx=0x7f2540005c50 comp rx=0 tx=0).stop 2024-05-21T15:38:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.879+0000 7f25647f3740 1 -- 172.21.15.38:0/2263619514 shutdown_connections 2024-05-21T15:38:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.879+0000 7f25647f3740 1 --2- 172.21.15.38:0/2263619514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521391b0 0x560e52508fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.879+0000 7f25647f3740 1 --2- 172.21.15.38:0/2263619514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521184d0 0x560e52508a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.879+0000 7f25647f3740 1 --2- 172.21.15.38:0/2263619514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521102f0 0x560e524d35b0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.879+0000 7f25647f3740 1 -- 172.21.15.38:0/2263619514 >> 172.21.15.38:0/2263619514 conn(0x560e5212d300 msgr2=0x560e524d2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.881+0000 7f25647f3740 1 -- 172.21.15.38:0/2263619514 shutdown_connections 2024-05-21T15:38:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.881+0000 7f25647f3740 1 -- 172.21.15.38:0/2263619514 wait complete. 2024-05-21T15:38:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.882+0000 7f25647f3740 1 -- start start 2024-05-21T15:38:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.882+0000 7f25647f3740 1 Processor -- start 2024-05-21T15:38:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.882+0000 7f25647f3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521102f0 0x560e524f0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.882+0000 7f25647f3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521184d0 0x560e524f07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.882+0000 7f25647f3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521391b0 0x560e524f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.882+0000 7f25647f3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f25581b1d50 con 0x560e521102f0 2024-05-21T15:38:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.882+0000 7f25647f3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f25581b1650 con 0x560e521391b0 2024-05-21T15:38:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.882+0000 7f25647f3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f25581ae890 con 0x560e521184d0 2024-05-21T15:38:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.882+0000 7f25647f3740 10 client.0 _ll_register_callbacks cb 0x7f254000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.883+0000 7f254effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521102f0 0x560e524f0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.883+0000 7f254effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521102f0 0x560e524f0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47708/0 (socket says 172.21.15.38:47708) 2024-05-21T15:38:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.883+0000 7f254effd640 1 -- 172.21.15.38:0/2221676094 learned_addr learned my addr 172.21.15.38:0/2221676094 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.883+0000 7f2556ffd640 1 --2- 172.21.15.38:0/2221676094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521184d0 0x560e524f07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.883+0000 7f25577fe640 1 --2- 172.21.15.38:0/2221676094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521391b0 0x560e524f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.883+0000 7f25577fe640 1 -- 172.21.15.38:0/2221676094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521184d0 msgr2=0x560e524f07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.883+0000 7f25577fe640 1 --2- 172.21.15.38:0/2221676094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521184d0 0x560e524f07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.883+0000 7f25577fe640 1 -- 172.21.15.38:0/2221676094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521102f0 msgr2=0x560e524f0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.883+0000 7f25577fe640 1 --2- 172.21.15.38:0/2221676094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521102f0 0x560e524f0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.883+0000 7f25577fe640 1 -- 172.21.15.38:0/2221676094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e524f1f20 con 0x560e521391b0 2024-05-21T15:38:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.885+0000 7f25577fe640 1 --2- 172.21.15.38:0/2221676094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521391b0 0x560e524f0ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f255002f540 tx=0x7f255002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d185b32cfb59396a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.885+0000 7f2546ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:38:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.886+0000 7f2546ffd640 1 -- 172.21.15.38:0/2221676094 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25500c1030 con 0x560e521391b0 2024-05-21T15:38:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.886+0000 7f25647f3740 1 -- 172.21.15.38:0/2221676094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2540024420 con 0x560e521391b0 2024-05-21T15:38:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.886+0000 7f2546ffd640 1 -- 172.21.15.38:0/2221676094 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f25500db2b0 con 0x560e521391b0 2024-05-21T15:38:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.887+0000 7f2546ffd640 1 -- 172.21.15.38:0/2221676094 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f25500b9830 con 0x560e521391b0 2024-05-21T15:38:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.889+0000 7f25647f3740 10 client.26102 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.889+0000 7f25647f3740 1 -- 172.21.15.38:0/2221676094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560e525686d0 con 0x560e521391b0 2024-05-21T15:38:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.890+0000 7f2546ffd640 1 -- 172.21.15.38:0/2221676094 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2540024420 con 0x560e521391b0 2024-05-21T15:38:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.890+0000 7f2546ffd640 1 -- 172.21.15.38:0/2221676094 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f25500039f0 con 0x560e521391b0 2024-05-21T15:38:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.891+0000 7f25647f3740 10 client.26102 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.891+0000 7f25647f3740 10 client.26102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.891+0000 7f25647f3740 20 client.26102 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.891+0000 7f25647f3740 1 --2- 172.21.15.38:0/2221676094 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e520cd1c0 0x560e5256b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.891+0000 7f25647f3740 4 client.26102 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:38:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.891+0000 7f25647f3740 1 -- 172.21.15.38:0/2221676094 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e524cfaa0 con 0x560e520cd1c0 2024-05-21T15:38:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.892+0000 7f254effd640 1 --2- 172.21.15.38:0/2221676094 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e520cd1c0 0x560e5256b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.894+0000 7f254effd640 1 --2- 172.21.15.38:0/2221676094 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e520cd1c0 0x560e5256b0d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb5892667a889915 server_cookie=588996377fa533f5 in_seq=0 out_seq=0 2024-05-21T15:38:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:20.894+0000 7f2546ffd640 10 client.26102 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.255+0000 7f2546ffd640 1 -- 172.21.15.38:0/2221676094 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e524cfaa0 con 0x560e520cd1c0 2024-05-21T15:38:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.256+0000 7f2546ffd640 10 client.26102 handle_command_reply: tid=0 2024-05-21T15:38:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.256+0000 7f25647f3740 1 client.26102 shutdown 2024-05-21T15:38:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.256+0000 7f25647f3740 1 -- 172.21.15.38:0/2221676094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521391b0 msgr2=0x560e524f0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.256+0000 7f25647f3740 1 --2- 172.21.15.38:0/2221676094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521391b0 0x560e524f0ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f255002f540 tx=0x7f255002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:38:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.256+0000 7f25647f3740 1 -- 172.21.15.38:0/2221676094 shutdown_connections 2024-05-21T15:38:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.256+0000 7f25647f3740 1 --2- 172.21.15.38:0/2221676094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e521391b0 0x560e524f0ce0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.256+0000 7f25647f3740 1 --2- 172.21.15.38:0/2221676094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e521184d0 0x560e524f07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.256+0000 7f25647f3740 1 --2- 172.21.15.38:0/2221676094 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e520cd1c0 0x560e5256b0d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.256+0000 7f25647f3740 1 --2- 172.21.15.38:0/2221676094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e521102f0 0x560e524f0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.256+0000 7f25647f3740 1 -- 172.21.15.38:0/2221676094 >> 172.21.15.38:0/2221676094 conn(0x560e5212d300 msgr2=0x560e524d38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.256+0000 7f2546ffd640 0 client.26102 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.256+0000 7f25647f3740 1 -- 172.21.15.38:0/2221676094 shutdown_connections 2024-05-21T15:38:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.256+0000 7f25647f3740 1 -- 172.21.15.38:0/2221676094 wait complete. 2024-05-21T15:38:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.262+0000 7f25647f3740 20 client.26102 trim_cache size 0 max 16384 2024-05-21T15:38:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:38:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:38:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:38:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:38:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.263+0000 7f2557fff640 1 -- 172.21.15.38:0/80211062 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f25380a3080 msgr2=0x7f25380a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.263+0000 7f2557fff640 1 --2- 172.21.15.38:0/80211062 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f25380a3080 0x7f25380a5590 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x560e524d8390 tx=0x7f2548005f90 comp rx=0 tx=0).stop 2024-05-21T15:38:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.263+0000 7f2557fff640 1 -- 172.21.15.38:0/80211062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25581085a0 msgr2=0x7f2558071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.263+0000 7f2557fff640 1 --2- 172.21.15.38:0/80211062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25581085a0 0x7f2558071d90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2550031c10 tx=0x7f25500319f0 comp rx=0 tx=0).stop 2024-05-21T15:38:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.263+0000 7f2557fff640 1 -- 172.21.15.38:0/80211062 shutdown_connections 2024-05-21T15:38:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.263+0000 7f2557fff640 1 --2- 172.21.15.38:0/80211062 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f25380a3080 0x7f25380a5590 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.263+0000 7f2557fff640 1 --2- 172.21.15.38:0/80211062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25581085a0 0x7f2558071d90 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.263+0000 7f2557fff640 1 --2- 172.21.15.38:0/80211062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2558105a30 0x7f2558070e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.263+0000 7f2557fff640 1 --2- 172.21.15.38:0/80211062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2558103040 0x7f25580708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.263+0000 7f2557fff640 1 -- 172.21.15.38:0/80211062 >> 172.21.15.38:0/80211062 conn(0x7f25580fc9e0 msgr2=0x7f2558103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.263+0000 7f2557fff640 1 -- 172.21.15.38:0/80211062 shutdown_connections 2024-05-21T15:38:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:21.263+0000 7f2557fff640 1 -- 172.21.15.38:0/80211062 wait complete. 2024-05-21T15:38:21.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:21 smithi053 ceph-mon[36496]: pgmap v469: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 88 MiB/s wr, 14.56k op/s 2024-05-21T15:38:21.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:21 smithi045 ceph-mon[36409]: pgmap v469: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 88 MiB/s wr, 14.56k op/s 2024-05-21T15:38:21.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:21 smithi038 ceph-mon[31995]: pgmap v469: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 88 MiB/s wr, 14.56k op/s 2024-05-21T15:38:23.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:23 smithi053 ceph-mon[36496]: pgmap v470: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 95 MiB/s wr, 15.66k op/s 2024-05-21T15:38:23.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:23 smithi045 ceph-mon[36409]: pgmap v470: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 95 MiB/s wr, 15.66k op/s 2024-05-21T15:38:23.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:23 smithi038 ceph-mon[31995]: pgmap v470: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 95 MiB/s wr, 15.66k op/s 2024-05-21T15:38:25.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:25 smithi045 ceph-mon[36409]: pgmap v471: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 104 MiB/s wr, 17.37k op/s 2024-05-21T15:38:25.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:25 smithi038 ceph-mon[31995]: pgmap v471: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 104 MiB/s wr, 17.37k op/s 2024-05-21T15:38:25.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:25 smithi053 ceph-mon[36496]: pgmap v471: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 104 MiB/s wr, 17.37k op/s 2024-05-21T15:38:27.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:27 smithi045 ceph-mon[36409]: pgmap v472: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 99 MiB/s wr, 16.65k op/s 2024-05-21T15:38:27.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:27 smithi038 ceph-mon[31995]: pgmap v472: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 99 MiB/s wr, 16.65k op/s 2024-05-21T15:38:27.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:27 smithi053 ceph-mon[36496]: pgmap v472: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 99 MiB/s wr, 16.65k op/s 2024-05-21T15:38:29.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:29 smithi045 ceph-mon[36409]: pgmap v473: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 101 MiB/s wr, 17.09k op/s 2024-05-21T15:38:29.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:29 smithi038 ceph-mon[31995]: pgmap v473: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 101 MiB/s wr, 17.09k op/s 2024-05-21T15:38:29.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:29 smithi053 ceph-mon[36496]: pgmap v473: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 101 MiB/s wr, 17.09k op/s 2024-05-21T15:38:31.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:31 smithi045 ceph-mon[36409]: pgmap v474: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 89 MiB/s wr, 15.26k op/s 2024-05-21T15:38:31.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:31 smithi038 ceph-mon[31995]: pgmap v474: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 89 MiB/s wr, 15.26k op/s 2024-05-21T15:38:31.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:31 smithi053 ceph-mon[36496]: pgmap v474: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 89 MiB/s wr, 15.26k op/s 2024-05-21T15:38:32.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:38:32.829 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:38:33.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.067+0000 7f742d396640 1 -- 172.21.15.38:0/3293559180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74281094d0 msgr2=0x7f742810a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:33.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.067+0000 7f742d396640 1 --2- 172.21.15.38:0/3293559180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74281094d0 0x7f742810a1d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7418009900 tx=0x7f741802a420 comp rx=0 tx=0).stop 2024-05-21T15:38:33.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.068+0000 7f742d396640 1 -- 172.21.15.38:0/3293559180 shutdown_connections 2024-05-21T15:38:33.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.069+0000 7f742d396640 1 --2- 172.21.15.38:0/3293559180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f742810f410 0x7f7428111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.069+0000 7f742d396640 1 --2- 172.21.15.38:0/3293559180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f742810a710 0x7f742810ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.069+0000 7f742d396640 1 --2- 172.21.15.38:0/3293559180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74281094d0 0x7f742810a1d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.069+0000 7f742d396640 1 -- 172.21.15.38:0/3293559180 >> 172.21.15.38:0/3293559180 conn(0x7f742806f8e0 msgr2=0x7f7428071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:33.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.071+0000 7f742d396640 1 -- 172.21.15.38:0/3293559180 shutdown_connections 2024-05-21T15:38:33.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.072+0000 7f742d396640 1 -- 172.21.15.38:0/3293559180 wait complete. 2024-05-21T15:38:33.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.073+0000 7f742d396640 1 Processor -- start 2024-05-21T15:38:33.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.073+0000 7f742d396640 1 -- start start 2024-05-21T15:38:33.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.074+0000 7f742d396640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74281094d0 0x7f742819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.074+0000 7f742d396640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f742810a710 0x7f742819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.074+0000 7f742d396640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f742810f410 0x7f74281a6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.074+0000 7f742d396640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7428114da0 con 0x7f742810a710 2024-05-21T15:38:33.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.074+0000 7f742d396640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7428114c20 con 0x7f74281094d0 2024-05-21T15:38:33.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.074+0000 7f74277fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f742810f410 0x7f74281a6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.074+0000 7f74277fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f742810f410 0x7f74281a6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:45368/0 (socket says 172.21.15.38:45368) 2024-05-21T15:38:33.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.074+0000 7f74277fe640 1 -- 172.21.15.38:0/2229581108 learned_addr learned my addr 172.21.15.38:0/2229581108 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:33.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.075+0000 7f742d396640 1 -- 172.21.15.38:0/2229581108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7428114f20 con 0x7f742810f410 2024-05-21T15:38:33.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.076+0000 7f74277fe640 1 -- 172.21.15.38:0/2229581108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74281094d0 msgr2=0x7f742819f910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:38:33.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.076+0000 7f74277fe640 1 --2- 172.21.15.38:0/2229581108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74281094d0 0x7f742819f910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.076+0000 7f74277fe640 1 -- 172.21.15.38:0/2229581108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f742810a710 msgr2=0x7f742819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:33.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.076+0000 7f74277fe640 1 --2- 172.21.15.38:0/2229581108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f742810a710 0x7f742819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.076+0000 7f74277fe640 1 -- 172.21.15.38:0/2229581108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74281a7450 con 0x7f742810f410 2024-05-21T15:38:33.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.078+0000 7f74277fe640 1 --2- 172.21.15.38:0/2229581108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f742810f410 0x7f74281a6de0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f742000eb10 tx=0x7f742000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:33.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.084+0000 7f740ffff640 1 -- 172.21.15.38:0/2229581108 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420004b20 con 0x7f742810f410 2024-05-21T15:38:33.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.085+0000 7f740ffff640 1 -- 172.21.15.38:0/2229581108 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7420004cc0 con 0x7f742810f410 2024-05-21T15:38:33.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.085+0000 7f740ffff640 1 -- 172.21.15.38:0/2229581108 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420010760 con 0x7f742810f410 2024-05-21T15:38:33.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.086+0000 7f742d396640 1 -- 172.21.15.38:0/2229581108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74281a7740 con 0x7f742810f410 2024-05-21T15:38:33.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.086+0000 7f742d396640 1 -- 172.21.15.38:0/2229581108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74281a7c60 con 0x7f742810f410 2024-05-21T15:38:33.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.092+0000 7f740ffff640 1 -- 172.21.15.38:0/2229581108 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7420005020 con 0x7f742810f410 2024-05-21T15:38:33.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.092+0000 7f7434242740 1 Processor -- start 2024-05-21T15:38:33.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.096+0000 7f7434242740 1 -- start start 2024-05-21T15:38:33.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.101+0000 7f7434242740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd31390 0x55b7c00d93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.101+0000 7f7434242740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd084d0 0x55b7c00d9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.101+0000 7f7434242740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 0x55b7c00e0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.101+0000 7f7434242740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b7c0035d00 con 0x55b7bfd084d0 2024-05-21T15:38:33.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.101+0000 7f7434242740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b7bffdd4c0 con 0x55b7bfd106b0 2024-05-21T15:38:33.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.101+0000 7f7434242740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b7c0031cc0 con 0x55b7bfd31390 2024-05-21T15:38:33.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.102+0000 7f740ffff640 1 --2- 172.21.15.38:0/2229581108 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f74100a2d60 0x7f74100a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.102+0000 7f740ffff640 1 -- 172.21.15.38:0/2229581108 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f74200cf710 con 0x7f742810f410 2024-05-21T15:38:33.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.103+0000 7f7426ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd084d0 0x55b7c00d9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.103+0000 7f7426ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd084d0 0x55b7c00d9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42596/0 (socket says 172.21.15.38:42596) 2024-05-21T15:38:33.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.103+0000 7f7426ffd640 1 -- 172.21.15.38:0/2175648523 learned_addr learned my addr 172.21.15.38:0/2175648523 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:33.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.104+0000 7f74277fe640 1 --2- 172.21.15.38:0/2229581108 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f74100a2d60 0x7f74100a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.105+0000 7f7426ffd640 1 -- 172.21.15.38:0/2175648523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd31390 msgr2=0x55b7c00d93c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:38:33.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.105+0000 7f7426ffd640 1 --2- 172.21.15.38:0/2175648523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd31390 0x55b7c00d93c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.105+0000 7f7426ffd640 1 -- 172.21.15.38:0/2175648523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 msgr2=0x55b7c00e0840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:38:33.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.105+0000 7f7426ffd640 1 --2- 172.21.15.38:0/2175648523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 0x55b7c00e0840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.105+0000 7f7426ffd640 1 -- 172.21.15.38:0/2175648523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7bfe137d0 con 0x55b7bfd084d0 2024-05-21T15:38:33.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.106+0000 7f7426ffd640 1 --2- 172.21.15.38:0/2175648523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd084d0 0x55b7c00d9940 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7418031440 tx=0x7f7418005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398e7a1c8baca013 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:33.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.107+0000 7f740d7fa640 1 -- 172.21.15.38:0/2175648523 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741803e070 con 0x55b7bfd084d0 2024-05-21T15:38:33.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.107+0000 7f740d7fa640 1 -- 172.21.15.38:0/2175648523 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7418005530 con 0x55b7bfd084d0 2024-05-21T15:38:33.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.108+0000 7f7434242740 1 -- 172.21.15.38:0/2175648523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd084d0 msgr2=0x55b7c00d9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:33.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.108+0000 7f7434242740 1 --2- 172.21.15.38:0/2175648523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd084d0 0x55b7c00d9940 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7418031440 tx=0x7f7418005da0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.109+0000 7f74277fe640 1 --2- 172.21.15.38:0/2229581108 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f74100a2d60 0x7f74100a5270 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7420017040 tx=0x7f74200058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:33.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.110+0000 7f7434242740 1 -- 172.21.15.38:0/2175648523 shutdown_connections 2024-05-21T15:38:33.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.110+0000 7f7434242740 1 --2- 172.21.15.38:0/2175648523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 0x55b7c00e0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.110+0000 7f7434242740 1 --2- 172.21.15.38:0/2175648523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd084d0 0x55b7c00d9940 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.110+0000 7f7434242740 1 --2- 172.21.15.38:0/2175648523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd31390 0x55b7c00d93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.110+0000 7f7434242740 1 -- 172.21.15.38:0/2175648523 >> 172.21.15.38:0/2175648523 conn(0x55b7bfd254e0 msgr2=0x55b7c00cc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:33.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.112+0000 7f7434242740 1 -- 172.21.15.38:0/2175648523 shutdown_connections 2024-05-21T15:38:33.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.112+0000 7f7434242740 1 -- 172.21.15.38:0/2175648523 wait complete. 2024-05-21T15:38:33.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.113+0000 7f7434242740 1 -- start start 2024-05-21T15:38:33.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.113+0000 7f7434242740 1 Processor -- start 2024-05-21T15:38:33.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.113+0000 7f7434242740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 0x55b7c00d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.113+0000 7f7434242740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 0x55b7c00d3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.113+0000 7f7434242740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c00d8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.113+0000 7f7434242740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b7c00d0bf0 con 0x55b7bfd31390 2024-05-21T15:38:33.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.113+0000 7f7434242740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b7bfff0b90 con 0x55b7bfd106b0 2024-05-21T15:38:33.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.113+0000 7f7434242740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b7c00d0d70 con 0x55b7bfd084d0 2024-05-21T15:38:33.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.114+0000 7f7434242740 10 client.0 _ll_register_callbacks cb 0x55b7c002cda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:33.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.114+0000 7f74267fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c00d8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.114+0000 7f74267fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c00d8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42606/0 (socket says 172.21.15.38:42606) 2024-05-21T15:38:33.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.114+0000 7f74267fc640 1 -- 172.21.15.38:0/2108770717 learned_addr learned my addr 172.21.15.38:0/2108770717 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:33.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.114+0000 7f74267fc640 1 --2- 172.21.15.38:0/2108770717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 0x55b7c00d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.116+0000 7f74267fc640 1 -- 172.21.15.38:0/2108770717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 msgr2=0x55b7c00d2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.116+0000 7f74267fc640 1 --2- 172.21.15.38:0/2108770717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 0x55b7c00d2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.116+0000 7f74267fc640 1 -- 172.21.15.38:0/2108770717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 msgr2=0x55b7c00d3470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:38:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.116+0000 7f74267fc640 1 --2- 172.21.15.38:0/2108770717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 0x55b7c00d3470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.116+0000 7f74267fc640 1 -- 172.21.15.38:0/2108770717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7bfd493f0 con 0x55b7bfd31390 2024-05-21T15:38:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.116+0000 7f74267fc640 1 --2- 172.21.15.38:0/2108770717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c00d8350 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f741c015a10 tx=0x7f741c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe287e923092ef3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.116+0000 7f740aff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.117+0000 7f740aff5640 1 -- 172.21.15.38:0/2108770717 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c012040 con 0x55b7bfd31390 2024-05-21T15:38:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.117+0000 7f740aff5640 1 -- 172.21.15.38:0/2108770717 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f741c007500 con 0x55b7bfd31390 2024-05-21T15:38:33.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.117+0000 7f7434242740 1 -- 172.21.15.38:0/2108770717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b7c0043430 con 0x55b7bfd31390 2024-05-21T15:38:33.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.117+0000 7f740aff5640 1 -- 172.21.15.38:0/2108770717 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f741c001630 con 0x55b7bfd31390 2024-05-21T15:38:33.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.119+0000 7f7434242740 10 client.16452 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:33.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.119+0000 7f7434242740 1 -- 172.21.15.38:0/2108770717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b7bfd090f0 con 0x55b7bfd31390 2024-05-21T15:38:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.122+0000 7f740aff5640 1 -- 172.21.15.38:0/2108770717 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b7c0043430 con 0x55b7bfd31390 2024-05-21T15:38:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.122+0000 7f740aff5640 1 -- 172.21.15.38:0/2108770717 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f741c004770 con 0x55b7bfd31390 2024-05-21T15:38:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.122+0000 7f7434242740 10 client.16452 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.122+0000 7f7434242740 10 client.16452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.122+0000 7f7434242740 20 client.16452 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.122+0000 7f7434242740 1 --2- 172.21.15.38:0/2108770717 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b7bfceff60 0x55b7c00fa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.122+0000 7f7434242740 4 client.16452 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:38:33.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.122+0000 7f7434242740 1 -- 172.21.15.38:0/2108770717 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7c003d780 con 0x55b7bfceff60 2024-05-21T15:38:33.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.126+0000 7f7426ffd640 1 --2- 172.21.15.38:0/2108770717 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b7bfceff60 0x55b7c00fa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.130+0000 7f7426ffd640 1 --2- 172.21.15.38:0/2108770717 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b7bfceff60 0x55b7c00fa030 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca6547fd5520a63a server_cookie=866e5f17d692d8ba in_seq=0 out_seq=0 2024-05-21T15:38:33.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.131+0000 7f740aff5640 10 client.16452 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:33.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.134+0000 7f740aff5640 1 -- 172.21.15.38:0/2108770717 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b7c003d780 con 0x55b7bfceff60 2024-05-21T15:38:33.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.134+0000 7f740aff5640 10 client.16452 handle_command_reply: tid=0 2024-05-21T15:38:33.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.134+0000 7f7434242740 1 client.16452 shutdown 2024-05-21T15:38:33.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.138+0000 7f7434242740 1 -- 172.21.15.38:0/2108770717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 msgr2=0x55b7c00d8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:33.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.138+0000 7f7434242740 1 --2- 172.21.15.38:0/2108770717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c00d8350 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f741c015a10 tx=0x7f741c009da0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.146+0000 7f7434242740 1 -- 172.21.15.38:0/2108770717 shutdown_connections 2024-05-21T15:38:33.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.146+0000 7f7434242740 1 --2- 172.21.15.38:0/2108770717 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b7bfceff60 0x55b7c00fa030 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.146+0000 7f7434242740 1 --2- 172.21.15.38:0/2108770717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c00d8350 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.146+0000 7f7434242740 1 --2- 172.21.15.38:0/2108770717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 0x55b7c00d3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.146+0000 7f7434242740 1 --2- 172.21.15.38:0/2108770717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 0x55b7c00d2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.146+0000 7f7434242740 1 -- 172.21.15.38:0/2108770717 >> 172.21.15.38:0/2108770717 conn(0x55b7bfd254e0 msgr2=0x55b7c00cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:33.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.146+0000 7f740aff5640 0 client.16452 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:33.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.149+0000 7f7434242740 1 -- 172.21.15.38:0/2108770717 shutdown_connections 2024-05-21T15:38:33.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.151+0000 7f7434242740 1 -- 172.21.15.38:0/2108770717 wait complete. 2024-05-21T15:38:33.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.155+0000 7f7434242740 20 client.16452 trim_cache size 0 max 16384 2024-05-21T15:38:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.193+0000 7f7434242740 1 Processor -- start 2024-05-21T15:38:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.194+0000 7f7434242740 1 -- start start 2024-05-21T15:38:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.194+0000 7f7434242740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 0x55b7c00f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.194+0000 7f7434242740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 0x55b7c00d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.194+0000 7f7434242740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c00d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.194+0000 7f7434242740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b0ae0 con 0x55b7bfd31390 2024-05-21T15:38:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.194+0000 7f7434242740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b2710 con 0x55b7bfd106b0 2024-05-21T15:38:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.194+0000 7f7434242740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b11b0 con 0x55b7bfd084d0 2024-05-21T15:38:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.194+0000 7f7426ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 0x55b7c00d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.194+0000 7f7426ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 0x55b7c00d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:49528/0 (socket says 172.21.15.38:49528) 2024-05-21T15:38:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.194+0000 7f7426ffd640 1 -- 172.21.15.38:0/1069839878 learned_addr learned my addr 172.21.15.38:0/1069839878 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:33.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.197+0000 7f74267fc640 1 --2- 172.21.15.38:0/1069839878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 0x55b7c00f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.197+0000 7f74267fc640 1 --2- 172.21.15.38:0/1069839878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c00d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.197+0000 7f7426ffd640 1 -- 172.21.15.38:0/1069839878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 msgr2=0x55b7c00f9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:33.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.197+0000 7f7426ffd640 1 --2- 172.21.15.38:0/1069839878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 0x55b7c00f9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.197+0000 7f7426ffd640 1 -- 172.21.15.38:0/1069839878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 msgr2=0x55b7c00d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:33.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.197+0000 7f7426ffd640 1 --2- 172.21.15.38:0/1069839878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c00d1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.197+0000 7f7426ffd640 1 -- 172.21.15.38:0/1069839878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c00d3610 con 0x55b7bfd106b0 2024-05-21T15:38:33.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.197+0000 7f74267fc640 1 --2- 172.21.15.38:0/1069839878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c00d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:38:33.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.201+0000 7f7426ffd640 1 --2- 172.21.15.38:0/1069839878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 0x55b7c00d1550 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7418009900 tx=0x7f741802a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2cef3f996aefdd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:33.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.203+0000 7f740cff9640 1 -- 172.21.15.38:0/1069839878 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741803e040 con 0x55b7bfd106b0 2024-05-21T15:38:33.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.203+0000 7f740cff9640 1 -- 172.21.15.38:0/1069839878 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7418005850 con 0x55b7bfd106b0 2024-05-21T15:38:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.203+0000 7f740cff9640 1 -- 172.21.15.38:0/1069839878 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741803e040 con 0x55b7bfd106b0 2024-05-21T15:38:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.204+0000 7f7434242740 1 -- 172.21.15.38:0/1069839878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 msgr2=0x55b7c00d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.204+0000 7f7434242740 1 --2- 172.21.15.38:0/1069839878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 0x55b7c00d1550 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7418009900 tx=0x7f741802a420 comp rx=0 tx=0).stop 2024-05-21T15:38:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.204+0000 7f7434242740 1 -- 172.21.15.38:0/1069839878 shutdown_connections 2024-05-21T15:38:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.204+0000 7f7434242740 1 --2- 172.21.15.38:0/1069839878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c00d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.204+0000 7f7434242740 1 --2- 172.21.15.38:0/1069839878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfd106b0 0x55b7c00d1550 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7418009900 tx=0x7f741802a420 comp rx=0 tx=0).stop 2024-05-21T15:38:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.204+0000 7f7434242740 1 --2- 172.21.15.38:0/1069839878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 0x55b7c00f9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.204+0000 7f7434242740 1 -- 172.21.15.38:0/1069839878 >> 172.21.15.38:0/1069839878 conn(0x55b7bfd254e0 msgr2=0x55b7c00dcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:33.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.207+0000 7f7434242740 1 -- 172.21.15.38:0/1069839878 shutdown_connections 2024-05-21T15:38:33.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.209+0000 7f7434242740 1 -- 172.21.15.38:0/1069839878 wait complete. 2024-05-21T15:38:33.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.209+0000 7f7434242740 1 -- start start 2024-05-21T15:38:33.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.210+0000 7f7434242740 1 Processor -- start 2024-05-21T15:38:33.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.210+0000 7f7434242740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 0x55b7c010a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.210+0000 7f7434242740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c010a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.210+0000 7f7434242740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfcc53a0 0x55b7c010ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.210+0000 7f7434242740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b1f80 con 0x55b7bfd31390 2024-05-21T15:38:33.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.210+0000 7f7434242740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b1880 con 0x55b7bfcc53a0 2024-05-21T15:38:33.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.210+0000 7f7434242740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f74281aeac0 con 0x55b7bfd084d0 2024-05-21T15:38:33.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.210+0000 7f7434242740 10 client.0 _ll_register_callbacks cb 0x7f741c001440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:33.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.211+0000 7f74267fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfcc53a0 0x55b7c010ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.211+0000 7f74267fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfcc53a0 0x55b7c010ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:49532/0 (socket says 172.21.15.38:49532) 2024-05-21T15:38:33.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.211+0000 7f74267fc640 1 -- 172.21.15.38:0/337652771 learned_addr learned my addr 172.21.15.38:0/337652771 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:33.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.212+0000 7f7426ffd640 1 --2- 172.21.15.38:0/337652771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c010a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.212+0000 7f74267fc640 1 --2- 172.21.15.38:0/337652771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 0x55b7c010a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.214+0000 7f7426ffd640 1 -- 172.21.15.38:0/337652771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 msgr2=0x55b7c010a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:33.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.214+0000 7f7426ffd640 1 --2- 172.21.15.38:0/337652771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 0x55b7c010a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.214+0000 7f7426ffd640 1 -- 172.21.15.38:0/337652771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfcc53a0 msgr2=0x55b7c010ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:33.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.214+0000 7f7426ffd640 1 --2- 172.21.15.38:0/337652771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfcc53a0 0x55b7c010ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.215+0000 7f7426ffd640 1 -- 172.21.15.38:0/337652771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c0160110 con 0x55b7bfd31390 2024-05-21T15:38:33.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.215+0000 7f7426ffd640 1 --2- 172.21.15.38:0/337652771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c010a6f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x55b7c00c7e00 tx=0x7f74180056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8928638553f887 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:33.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.215+0000 7f740aff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:33.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.216+0000 7f740aff5640 1 -- 172.21.15.38:0/337652771 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741803e040 con 0x55b7bfd31390 2024-05-21T15:38:33.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.216+0000 7f7434242740 1 -- 172.21.15.38:0/337652771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7418005850 con 0x55b7bfd31390 2024-05-21T15:38:33.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.216+0000 7f740aff5640 1 -- 172.21.15.38:0/337652771 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7418004100 con 0x55b7bfd31390 2024-05-21T15:38:33.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.216+0000 7f740aff5640 1 -- 172.21.15.38:0/337652771 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7418004300 con 0x55b7bfd31390 2024-05-21T15:38:33.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.216+0000 7f7434242740 10 client.16455 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:33.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.216+0000 7f7434242740 1 -- 172.21.15.38:0/337652771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b7c0161310 con 0x55b7bfd31390 2024-05-21T15:38:33.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.217+0000 7f740aff5640 1 -- 172.21.15.38:0/337652771 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7418005850 con 0x55b7bfd31390 2024-05-21T15:38:33.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.218+0000 7f740aff5640 1 -- 172.21.15.38:0/337652771 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f741c001c80 con 0x55b7bfd31390 2024-05-21T15:38:33.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.218+0000 7f7434242740 10 client.16455 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:33.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.218+0000 7f7434242740 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:33.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.218+0000 7f7434242740 20 client.16455 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:33.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.218+0000 7f7434242740 1 --2- 172.21.15.38:0/337652771 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b7c0161ac0 0x55b7c0163ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.218+0000 7f7434242740 4 client.16455 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:38:33.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.218+0000 7f7434242740 1 -- 172.21.15.38:0/337652771 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7c00fa620 con 0x55b7c0161ac0 2024-05-21T15:38:33.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.221+0000 7f74267fc640 1 --2- 172.21.15.38:0/337652771 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b7c0161ac0 0x55b7c0163ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.222+0000 7f74267fc640 1 --2- 172.21.15.38:0/337652771 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b7c0161ac0 0x55b7c0163ee0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=464007a7926e3b93 server_cookie=5d557089334a4a95 in_seq=0 out_seq=0 2024-05-21T15:38:33.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.222+0000 7f740aff5640 10 client.16455 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.224+0000 7f740aff5640 1 -- 172.21.15.38:0/337652771 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b7c00fa620 con 0x55b7c0161ac0 2024-05-21T15:38:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.224+0000 7f740aff5640 10 client.16455 handle_command_reply: tid=0 2024-05-21T15:38:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.224+0000 7f7434242740 1 client.16455 shutdown 2024-05-21T15:38:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.227+0000 7f7434242740 1 -- 172.21.15.38:0/337652771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 msgr2=0x55b7c010a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.227+0000 7f7434242740 1 --2- 172.21.15.38:0/337652771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c010a6f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x55b7c00c7e00 tx=0x7f74180056b0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.227+0000 7f7434242740 1 -- 172.21.15.38:0/337652771 shutdown_connections 2024-05-21T15:38:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.227+0000 7f7434242740 1 --2- 172.21.15.38:0/337652771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7bfcc53a0 0x55b7c010ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.227+0000 7f7434242740 1 --2- 172.21.15.38:0/337652771 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55b7c0161ac0 0x55b7c0163ee0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.227+0000 7f7434242740 1 --2- 172.21.15.38:0/337652771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b7bfd31390 0x55b7c010a6f0 secure :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0x55b7c00c7e00 tx=0x7f74180056b0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.227+0000 7f7434242740 1 --2- 172.21.15.38:0/337652771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b7bfd084d0 0x55b7c010a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.227+0000 7f740aff5640 0 client.16455 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.227+0000 7f7434242740 1 -- 172.21.15.38:0/337652771 >> 172.21.15.38:0/337652771 conn(0x55b7bfd254e0 msgr2=0x55b7c00d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.229+0000 7f7434242740 1 -- 172.21.15.38:0/337652771 shutdown_connections 2024-05-21T15:38:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.229+0000 7f7434242740 1 -- 172.21.15.38:0/337652771 wait complete. 2024-05-21T15:38:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.230+0000 7f7434242740 20 client.16455 trim_cache size 0 max 16384 2024-05-21T15:38:33.229 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:38:33.229 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-05-21T15:38:33.229 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-05-21T15:38:33.229 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:38:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.231+0000 7f742d396640 1 -- 172.21.15.38:0/2229581108 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f74100a2d60 msgr2=0x7f74100a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.231+0000 7f742d396640 1 --2- 172.21.15.38:0/2229581108 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f74100a2d60 0x7f74100a5270 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7420017040 tx=0x7f74200058d0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.231+0000 7f742d396640 1 -- 172.21.15.38:0/2229581108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f742810f410 msgr2=0x7f74281a6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.231+0000 7f742d396640 1 --2- 172.21.15.38:0/2229581108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f742810f410 0x7f74281a6de0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f742000eb10 tx=0x7f742000c2d0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.232+0000 7f742d396640 1 -- 172.21.15.38:0/2229581108 shutdown_connections 2024-05-21T15:38:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.232+0000 7f742d396640 1 --2- 172.21.15.38:0/2229581108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f742810f410 0x7f74281a6de0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.232+0000 7f742d396640 1 --2- 172.21.15.38:0/2229581108 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f74100a2d60 0x7f74100a5270 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.232+0000 7f742d396640 1 --2- 172.21.15.38:0/2229581108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f742810a710 0x7f742819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.232+0000 7f742d396640 1 --2- 172.21.15.38:0/2229581108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74281094d0 0x7f742819f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.232+0000 7f742d396640 1 -- 172.21.15.38:0/2229581108 >> 172.21.15.38:0/2229581108 conn(0x7f742806f8e0 msgr2=0x7f742810b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:33.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.234+0000 7f742d396640 1 -- 172.21.15.38:0/2229581108 shutdown_connections 2024-05-21T15:38:33.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.235+0000 7f742d396640 1 -- 172.21.15.38:0/2229581108 wait complete. 2024-05-21T15:38:33.295 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:38:33.295 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:38:33.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.502+0000 7fa711c73640 1 -- 172.21.15.38:0/1364521205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c1019b0 msgr2=0x7fa70c1026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:33.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.502+0000 7fa711c73640 1 --2- 172.21.15.38:0/1364521205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c1019b0 0x7fa70c1026b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa704009be0 tx=0x7fa70402a6c0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.502+0000 7fa711c73640 1 -- 172.21.15.38:0/1364521205 shutdown_connections 2024-05-21T15:38:33.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.503+0000 7fa711c73640 1 --2- 172.21.15.38:0/1364521205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa70c1078f0 0x7fa70c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.503+0000 7fa711c73640 1 --2- 172.21.15.38:0/1364521205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa70c102bf0 0x7fa70c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.503+0000 7fa711c73640 1 --2- 172.21.15.38:0/1364521205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c1019b0 0x7fa70c1026b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.503+0000 7fa711c73640 1 -- 172.21.15.38:0/1364521205 >> 172.21.15.38:0/1364521205 conn(0x7fa70c0fcba0 msgr2=0x7fa70c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:33.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.503+0000 7fa711c73640 1 -- 172.21.15.38:0/1364521205 shutdown_connections 2024-05-21T15:38:33.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.503+0000 7fa711c73640 1 -- 172.21.15.38:0/1364521205 wait complete. 2024-05-21T15:38:33.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.504+0000 7fa711c73640 1 Processor -- start 2024-05-21T15:38:33.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.504+0000 7fa711c73640 1 -- start start 2024-05-21T15:38:33.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.505+0000 7fa711c73640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c1019b0 0x7fa70c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.505+0000 7fa711c73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa70c102bf0 0x7fa70c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.505+0000 7fa70b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c1019b0 0x7fa70c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.505+0000 7fa70b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c1019b0 0x7fa70c197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:45428/0 (socket says 172.21.15.38:45428) 2024-05-21T15:38:33.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.505+0000 7fa70b7fe640 1 -- 172.21.15.38:0/3176480546 learned_addr learned my addr 172.21.15.38:0/3176480546 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:33.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.505+0000 7fa70affd640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa70c102bf0 0x7fa70c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.506+0000 7fa711c73640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa70c1078f0 0x7fa70c19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.506+0000 7fa711c73640 1 -- 172.21.15.38:0/3176480546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c10d5e0 con 0x7fa70c102bf0 2024-05-21T15:38:33.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.506+0000 7fa711c73640 1 -- 172.21.15.38:0/3176480546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c10d460 con 0x7fa70c1078f0 2024-05-21T15:38:33.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.506+0000 7fa70bfff640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa70c1078f0 0x7fa70c19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.506+0000 7fa711c73640 1 -- 172.21.15.38:0/3176480546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c10d760 con 0x7fa70c1019b0 2024-05-21T15:38:33.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.507+0000 7fa70b7fe640 1 -- 172.21.15.38:0/3176480546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa70c1078f0 msgr2=0x7fa70c19f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:33.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.507+0000 7fa70b7fe640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa70c1078f0 0x7fa70c19f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.507+0000 7fa70b7fe640 1 -- 172.21.15.38:0/3176480546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa70c102bf0 msgr2=0x7fa70c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:33.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.507+0000 7fa70b7fe640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa70c102bf0 0x7fa70c198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.507+0000 7fa70b7fe640 1 -- 172.21.15.38:0/3176480546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70c19fb00 con 0x7fa70c1019b0 2024-05-21T15:38:33.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.507+0000 7fa70bfff640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa70c1078f0 0x7fa70c19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:38:33.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.507+0000 7fa70b7fe640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c1019b0 0x7fa70c197fc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa704034040 tx=0x7fa704030850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:33.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.509+0000 7fa708ff9640 1 -- 172.21.15.38:0/3176480546 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704036040 con 0x7fa70c1019b0 2024-05-21T15:38:33.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.509+0000 7fa708ff9640 1 -- 172.21.15.38:0/3176480546 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa704031920 con 0x7fa70c1019b0 2024-05-21T15:38:33.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.509+0000 7fa708ff9640 1 -- 172.21.15.38:0/3176480546 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704004740 con 0x7fa70c1019b0 2024-05-21T15:38:33.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.510+0000 7fa711c73640 1 -- 172.21.15.38:0/3176480546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa70c068fb0 con 0x7fa70c1019b0 2024-05-21T15:38:33.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.510+0000 7fa711c73640 1 -- 172.21.15.38:0/3176480546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa70c069450 con 0x7fa70c1019b0 2024-05-21T15:38:33.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.513+0000 7fa711c73640 1 -- 172.21.15.38:0/3176480546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa70c068d70 con 0x7fa70c1019b0 2024-05-21T15:38:33.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.515+0000 7fa708ff9640 1 -- 172.21.15.38:0/3176480546 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa704031440 con 0x7fa70c1019b0 2024-05-21T15:38:33.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.517+0000 7fa708ff9640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa6f40a2df0 0x7fa6f40a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:33.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.517+0000 7fa70affd640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa6f40a2df0 0x7fa6f40a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:33.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.517+0000 7fa708ff9640 1 -- 172.21.15.38:0/3176480546 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa7040f1c70 con 0x7fa70c1019b0 2024-05-21T15:38:33.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.518+0000 7fa70affd640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa6f40a2df0 0x7fa6f40a5300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa70c199060 tx=0x7fa6fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:33.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:33 smithi038 ceph-mon[31995]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 97 MiB/s wr, 16.74k op/s 2024-05-21T15:38:33.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.526+0000 7fa708ff9640 1 -- 172.21.15.38:0/3176480546 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa70404a060 con 0x7fa70c1019b0 2024-05-21T15:38:33.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:33 smithi045 ceph-mon[36409]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 97 MiB/s wr, 16.74k op/s 2024-05-21T15:38:33.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:33 smithi053 ceph-mon[36496]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 97 MiB/s wr, 16.74k op/s 2024-05-21T15:38:33.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.983+0000 7fa711c73640 1 -- 172.21.15.38:0/3176480546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa70c105fe0 con 0x7fa70c1019b0 2024-05-21T15:38:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.987+0000 7fa708ff9640 1 -- 172.21.15.38:0/3176480546 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11922 (secure 0 0 0) 0x7fa7040beb80 con 0x7fa70c1019b0 2024-05-21T15:38:33.986 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:38:33.986 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:36:59.077696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24551,24523,24602,24550,24515]},"id":1}]} 2024-05-21T15:38:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 31 2024-05-21T15:38:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.991+0000 7fa711c73640 1 -- 172.21.15.38:0/3176480546 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa6f40a2df0 msgr2=0x7fa6f40a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.991+0000 7fa711c73640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa6f40a2df0 0x7fa6f40a5300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa70c199060 tx=0x7fa6fc009260 comp rx=0 tx=0).stop 2024-05-21T15:38:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.991+0000 7fa711c73640 1 -- 172.21.15.38:0/3176480546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c1019b0 msgr2=0x7fa70c197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.991+0000 7fa711c73640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c1019b0 0x7fa70c197fc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa704034040 tx=0x7fa704030850 comp rx=0 tx=0).stop 2024-05-21T15:38:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.991+0000 7fa711c73640 1 -- 172.21.15.38:0/3176480546 shutdown_connections 2024-05-21T15:38:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.991+0000 7fa711c73640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa70c1078f0 0x7fa70c19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.991+0000 7fa711c73640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa6f40a2df0 0x7fa6f40a5300 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.991+0000 7fa711c73640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa70c102bf0 0x7fa70c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.991+0000 7fa711c73640 1 --2- 172.21.15.38:0/3176480546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa70c1019b0 0x7fa70c197fc0 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa704034040 tx=0x7fa704030850 comp rx=0 tx=0).stop 2024-05-21T15:38:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.991+0000 7fa711c73640 1 -- 172.21.15.38:0/3176480546 >> 172.21.15.38:0/3176480546 conn(0x7fa70c0fcba0 msgr2=0x7fa70c0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.992+0000 7fa711c73640 1 -- 172.21.15.38:0/3176480546 shutdown_connections 2024-05-21T15:38:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:33.992+0000 7fa711c73640 1 -- 172.21.15.38:0/3176480546 wait complete. 2024-05-21T15:38:34.013 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:38:34.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.230+0000 7f497a3e2640 1 -- 172.21.15.38:0/3781929360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4974102c10 msgr2=0x7f4974103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.230+0000 7f497a3e2640 1 --2- 172.21.15.38:0/3781929360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4974102c10 0x7f4974103020 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4964009ca0 tx=0x7f496402a8f0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.234+0000 7f497a3e2640 1 -- 172.21.15.38:0/3781929360 shutdown_connections 2024-05-21T15:38:34.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.234+0000 7f497a3e2640 1 --2- 172.21.15.38:0/3781929360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4974107910 0x7f4974109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.234+0000 7f497a3e2640 1 --2- 172.21.15.38:0/3781929360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4974102c10 0x7f4974103020 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.234+0000 7f497a3e2640 1 --2- 172.21.15.38:0/3781929360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49741019d0 0x7f49741026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.234+0000 7f497a3e2640 1 -- 172.21.15.38:0/3781929360 >> 172.21.15.38:0/3781929360 conn(0x7f49740fcbc0 msgr2=0x7f49740ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.242+0000 7f497a3e2640 1 -- 172.21.15.38:0/3781929360 shutdown_connections 2024-05-21T15:38:34.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.246+0000 7f497a3e2640 1 -- 172.21.15.38:0/3781929360 wait complete. 2024-05-21T15:38:34.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.247+0000 7f497a3e2640 1 Processor -- start 2024-05-21T15:38:34.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.247+0000 7f497a3e2640 1 -- start start 2024-05-21T15:38:34.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.248+0000 7f497a3e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49741019d0 0x7f4974112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.248+0000 7f497a3e2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4974102c10 0x7f49741113a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.248+0000 7f497a3e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4974107910 0x7f49741119f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.248+0000 7f497a3e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f497410b460 con 0x7f4974107910 2024-05-21T15:38:34.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.248+0000 7f497a3e2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f497410b2e0 con 0x7f4974102c10 2024-05-21T15:38:34.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.248+0000 7f497a3e2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f497410b5e0 con 0x7f49741019d0 2024-05-21T15:38:34.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.248+0000 7f49737fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4974102c10 0x7f49741113a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.248+0000 7f49737fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4974102c10 0x7f49741113a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:49566/0 (socket says 172.21.15.38:49566) 2024-05-21T15:38:34.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.248+0000 7f4973fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49741019d0 0x7f4974112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.248+0000 7f4973fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49741019d0 0x7f4974112d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:45452/0 (socket says 172.21.15.38:45452) 2024-05-21T15:38:34.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.249+0000 7f4973fff640 1 -- 172.21.15.38:0/1200946990 learned_addr learned my addr 172.21.15.38:0/1200946990 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.251+0000 7f49737fe640 1 -- 172.21.15.38:0/1200946990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49741019d0 msgr2=0x7f4974112d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.251+0000 7f49737fe640 1 --2- 172.21.15.38:0/1200946990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49741019d0 0x7f4974112d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.251+0000 7f49737fe640 1 -- 172.21.15.38:0/1200946990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4974107910 msgr2=0x7f49741119f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.252+0000 7f497897f640 1 --2- 172.21.15.38:0/1200946990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4974107910 0x7f49741119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.252+0000 7f49737fe640 1 --2- 172.21.15.38:0/1200946990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4974107910 0x7f49741119f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.252+0000 7f49737fe640 1 -- 172.21.15.38:0/1200946990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49741b0240 con 0x7f4974102c10 2024-05-21T15:38:34.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.253+0000 7f497897f640 1 --2- 172.21.15.38:0/1200946990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4974107910 0x7f49741119f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:38:34.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.253+0000 7f49737fe640 1 --2- 172.21.15.38:0/1200946990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4974102c10 0x7f49741113a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4964009bf0 tx=0x7f4964003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.256+0000 7f49717fa640 1 -- 172.21.15.38:0/1200946990 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964004040 con 0x7f4974102c10 2024-05-21T15:38:34.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.256+0000 7f49717fa640 1 -- 172.21.15.38:0/1200946990 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f496402d070 con 0x7f4974102c10 2024-05-21T15:38:34.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.256+0000 7f49717fa640 1 -- 172.21.15.38:0/1200946990 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496402fce0 con 0x7f4974102c10 2024-05-21T15:38:34.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.256+0000 7f497a3e2640 1 -- 172.21.15.38:0/1200946990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49741b0410 con 0x7f4974102c10 2024-05-21T15:38:34.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.256+0000 7f497a3e2640 1 -- 172.21.15.38:0/1200946990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49741b08a0 con 0x7f4974102c10 2024-05-21T15:38:34.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.263+0000 7f49717fa640 1 -- 172.21.15.38:0/1200946990 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f49640041e0 con 0x7f4974102c10 2024-05-21T15:38:34.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.264+0000 7f49717fa640 1 --2- 172.21.15.38:0/1200946990 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f49600a2d90 0x7f49600a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.264+0000 7f49717fa640 1 -- 172.21.15.38:0/1200946990 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f49640f0980 con 0x7f4974102c10 2024-05-21T15:38:34.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.264+0000 7f4973fff640 1 --2- 172.21.15.38:0/1200946990 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f49600a2d90 0x7f49600a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.267+0000 7f4981285740 1 Processor -- start 2024-05-21T15:38:34.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.267+0000 7f4981285740 1 -- start start 2024-05-21T15:38:34.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.268+0000 7f4981285740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a08d390 0x56517a43db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.268+0000 7f4981285740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a0644d0 0x56517a43e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.268+0000 7f4981285740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a06c6b0 0x56517a444f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.268+0000 7f4981285740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56517a391c40 con 0x56517a0644d0 2024-05-21T15:38:34.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.268+0000 7f4981285740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56517a339580 con 0x56517a06c6b0 2024-05-21T15:38:34.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.268+0000 7f4981285740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56517a38dc00 con 0x56517a08d390 2024-05-21T15:38:34.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.268+0000 7f497897f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a06c6b0 0x56517a444f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.268+0000 7f497897f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a06c6b0 0x56517a444f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:49576/0 (socket says 172.21.15.38:49576) 2024-05-21T15:38:34.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.268+0000 7f497897f640 1 -- 172.21.15.38:0/3357082921 learned_addr learned my addr 172.21.15.38:0/3357082921 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.268+0000 7f49737fe640 1 --2- 172.21.15.38:0/3357082921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a0644d0 0x56517a43e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.268+0000 7f497897f640 1 -- 172.21.15.38:0/3357082921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a08d390 msgr2=0x56517a43db10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:38:34.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.268+0000 7f497897f640 1 --2- 172.21.15.38:0/3357082921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a08d390 0x56517a43db10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.269+0000 7f497897f640 1 -- 172.21.15.38:0/3357082921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a0644d0 msgr2=0x56517a43e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.269+0000 7f497897f640 1 --2- 172.21.15.38:0/3357082921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a0644d0 0x56517a43e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.269+0000 7f497897f640 1 -- 172.21.15.38:0/3357082921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56517a0a53f0 con 0x56517a06c6b0 2024-05-21T15:38:34.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.270+0000 7f497897f640 1 --2- 172.21.15.38:0/3357082921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a06c6b0 0x56517a444f90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f496c00a940 tx=0x7f496c00ae00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33b9aee2f85476c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.272+0000 7f496a7fc640 1 -- 172.21.15.38:0/3357082921 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c015070 con 0x56517a06c6b0 2024-05-21T15:38:34.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.273+0000 7f496a7fc640 1 -- 172.21.15.38:0/3357082921 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f496c00cc00 con 0x56517a06c6b0 2024-05-21T15:38:34.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.274+0000 7f4973fff640 1 --2- 172.21.15.38:0/1200946990 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f49600a2d90 0x7f49600a52a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f495c006fd0 tx=0x7f495c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.274+0000 7f4981285740 1 -- 172.21.15.38:0/3357082921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a06c6b0 msgr2=0x56517a444f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.274+0000 7f4981285740 1 --2- 172.21.15.38:0/3357082921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a06c6b0 0x56517a444f90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f496c00a940 tx=0x7f496c00ae00 comp rx=0 tx=0).stop 2024-05-21T15:38:34.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.278+0000 7f4981285740 1 -- 172.21.15.38:0/3357082921 shutdown_connections 2024-05-21T15:38:34.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.278+0000 7f4981285740 1 --2- 172.21.15.38:0/3357082921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a06c6b0 0x56517a444f90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.278+0000 7f4981285740 1 --2- 172.21.15.38:0/3357082921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a0644d0 0x56517a43e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.278+0000 7f4981285740 1 --2- 172.21.15.38:0/3357082921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a08d390 0x56517a43db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.278+0000 7f4981285740 1 -- 172.21.15.38:0/3357082921 >> 172.21.15.38:0/3357082921 conn(0x56517a0814e0 msgr2=0x56517a428490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.279+0000 7f4981285740 1 -- 172.21.15.38:0/3357082921 shutdown_connections 2024-05-21T15:38:34.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.279+0000 7f4981285740 1 -- 172.21.15.38:0/3357082921 wait complete. 2024-05-21T15:38:34.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.279+0000 7f4981285740 1 -- start start 2024-05-21T15:38:34.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.279+0000 7f4981285740 1 Processor -- start 2024-05-21T15:38:34.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.280+0000 7f4981285740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a0644d0 0x56517a447760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.280+0000 7f4981285740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a06c6b0 0x56517a447ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.280+0000 7f4981285740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a08d390 0x56517a44ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.280+0000 7f4981285740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56517a445a30 con 0x56517a06c6b0 2024-05-21T15:38:34.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.280+0000 7f4981285740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56517a34cc40 con 0x56517a08d390 2024-05-21T15:38:34.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.280+0000 7f4981285740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56517a445bb0 con 0x56517a0644d0 2024-05-21T15:38:34.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.280+0000 7f4981285740 10 client.0 _ll_register_callbacks cb 0x56517a0a4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:34.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.281+0000 7f4973fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a0644d0 0x56517a447760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.281+0000 7f4973fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a0644d0 0x56517a447760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:45464/0 (socket says 172.21.15.38:45464) 2024-05-21T15:38:34.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.281+0000 7f4973fff640 1 -- 172.21.15.38:0/145949709 learned_addr learned my addr 172.21.15.38:0/145949709 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.281+0000 7f497897f640 1 --2- 172.21.15.38:0/145949709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a08d390 0x56517a44ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.281+0000 7f49737fe640 1 --2- 172.21.15.38:0/145949709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a06c6b0 0x56517a447ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.281+0000 7f4973fff640 1 -- 172.21.15.38:0/145949709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a08d390 msgr2=0x56517a44ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.281+0000 7f4973fff640 1 --2- 172.21.15.38:0/145949709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a08d390 0x56517a44ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.281+0000 7f4973fff640 1 -- 172.21.15.38:0/145949709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a06c6b0 msgr2=0x56517a447ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.281+0000 7f4973fff640 1 --2- 172.21.15.38:0/145949709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a06c6b0 0x56517a447ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.281+0000 7f4973fff640 1 -- 172.21.15.38:0/145949709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56517a16f7d0 con 0x56517a0644d0 2024-05-21T15:38:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.281+0000 7f49737fe640 1 --2- 172.21.15.38:0/145949709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a06c6b0 0x56517a447ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:38:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.282+0000 7f4973fff640 1 --2- 172.21.15.38:0/145949709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a0644d0 0x56517a447760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f495c00de20 tx=0x7f495c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2429d8f4de1e7c91 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.282+0000 7f49537fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:38:34.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.283+0000 7f49537fe640 1 -- 172.21.15.38:0/145949709 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c024060 con 0x56517a0644d0 2024-05-21T15:38:34.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.284+0000 7f49537fe640 1 -- 172.21.15.38:0/145949709 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f495c0125f0 con 0x56517a0644d0 2024-05-21T15:38:34.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.284+0000 7f4981285740 1 -- 172.21.15.38:0/145949709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f496c00cc00 con 0x56517a0644d0 2024-05-21T15:38:34.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.285+0000 7f49537fe640 1 -- 172.21.15.38:0/145949709 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f495c00de50 con 0x56517a0644d0 2024-05-21T15:38:34.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.285+0000 7f4981285740 10 client.26188 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:34.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.285+0000 7f4981285740 1 -- 172.21.15.38:0/145949709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56517a451f90 con 0x56517a0644d0 2024-05-21T15:38:34.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.287+0000 7f49537fe640 1 -- 172.21.15.38:0/145949709 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f496c00cc00 con 0x56517a0644d0 2024-05-21T15:38:34.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.287+0000 7f49537fe640 1 -- 172.21.15.38:0/145949709 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f495c013070 con 0x56517a0644d0 2024-05-21T15:38:34.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.287+0000 7f4981285740 10 client.26188 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:34.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.287+0000 7f4981285740 10 client.26188 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:38:34.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.287+0000 7f4981285740 20 client.26188 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:34.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.287+0000 7f4981285740 1 --2- 172.21.15.38:0/145949709 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56517a04bf60 0x56517a455610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.287+0000 7f4981285740 4 client.26188 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:38:34.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.287+0000 7f4981285740 1 -- 172.21.15.38:0/145949709 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56517a39f370 con 0x56517a04bf60 2024-05-21T15:38:34.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.290+0000 7f49737fe640 1 --2- 172.21.15.38:0/145949709 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56517a04bf60 0x56517a455610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.296+0000 7f49537fe640 10 client.26188 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:38:34.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.296+0000 7f49737fe640 1 --2- 172.21.15.38:0/145949709 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56517a04bf60 0x56517a455610 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7e1fca8239d31ad7 server_cookie=6cdf4cd23cdd629f in_seq=0 out_seq=0 2024-05-21T15:38:34.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.297+0000 7f49537fe640 1 -- 172.21.15.38:0/145949709 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56517a39f370 con 0x56517a04bf60 2024-05-21T15:38:34.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.297+0000 7f49537fe640 10 client.26188 handle_command_reply: tid=0 2024-05-21T15:38:34.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.298+0000 7f4981285740 1 client.26188 shutdown 2024-05-21T15:38:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.298+0000 7f4981285740 1 -- 172.21.15.38:0/145949709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a0644d0 msgr2=0x56517a447760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.298+0000 7f4981285740 1 --2- 172.21.15.38:0/145949709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a0644d0 0x56517a447760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f495c00de20 tx=0x7f495c004770 comp rx=0 tx=0).stop 2024-05-21T15:38:34.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.299+0000 7f4981285740 1 -- 172.21.15.38:0/145949709 shutdown_connections 2024-05-21T15:38:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.299+0000 7f4981285740 1 --2- 172.21.15.38:0/145949709 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56517a04bf60 0x56517a455610 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.299+0000 7f4981285740 1 --2- 172.21.15.38:0/145949709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a08d390 0x56517a44ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.299+0000 7f4981285740 1 --2- 172.21.15.38:0/145949709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a06c6b0 0x56517a447ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.299+0000 7f4981285740 1 --2- 172.21.15.38:0/145949709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a0644d0 0x56517a447760 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.299+0000 7f4981285740 1 -- 172.21.15.38:0/145949709 >> 172.21.15.38:0/145949709 conn(0x56517a0814e0 msgr2=0x56517a427570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.301+0000 7f49537fe640 0 client.26188 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:38:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.302+0000 7f4981285740 1 -- 172.21.15.38:0/145949709 shutdown_connections 2024-05-21T15:38:34.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.302+0000 7f4981285740 1 -- 172.21.15.38:0/145949709 wait complete. 2024-05-21T15:38:34.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.302+0000 7f4981285740 20 client.26188 trim_cache size 0 max 16384 2024-05-21T15:38:34.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.331+0000 7f4981285740 1 Processor -- start 2024-05-21T15:38:34.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.332+0000 7f4981285740 1 -- start start 2024-05-21T15:38:34.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.332+0000 7f4981285740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a0644d0 0x56517a4285e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.332+0000 7f4981285740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a06c6b0 0x56517a44eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.333+0000 7f4981285740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a08d390 0x56517a42b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.333+0000 7f4981285740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f495400e700 con 0x56517a08d390 2024-05-21T15:38:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.333+0000 7f4981285740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4954010330 con 0x56517a0644d0 2024-05-21T15:38:34.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.333+0000 7f4981285740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f495400edd0 con 0x56517a06c6b0 2024-05-21T15:38:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.333+0000 7f4973fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a0644d0 0x56517a4285e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.333+0000 7f4973fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a0644d0 0x56517a4285e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:49596/0 (socket says 172.21.15.38:49596) 2024-05-21T15:38:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.333+0000 7f4973fff640 1 -- 172.21.15.38:0/2823386590 learned_addr learned my addr 172.21.15.38:0/2823386590 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.333+0000 7f497897f640 1 --2- 172.21.15.38:0/2823386590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a08d390 0x56517a42b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.333+0000 7f49737fe640 1 --2- 172.21.15.38:0/2823386590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a06c6b0 0x56517a44eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.334+0000 7f4973fff640 1 -- 172.21.15.38:0/2823386590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a06c6b0 msgr2=0x56517a44eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.334+0000 7f4973fff640 1 --2- 172.21.15.38:0/2823386590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a06c6b0 0x56517a44eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.334+0000 7f4973fff640 1 -- 172.21.15.38:0/2823386590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a08d390 msgr2=0x56517a42b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.334+0000 7f4973fff640 1 --2- 172.21.15.38:0/2823386590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a08d390 0x56517a42b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.334+0000 7f4973fff640 1 -- 172.21.15.38:0/2823386590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56517a423d20 con 0x56517a0644d0 2024-05-21T15:38:34.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.334+0000 7f4973fff640 1 --2- 172.21.15.38:0/2823386590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a0644d0 0x56517a4285e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f495c00d530 tx=0x7f495c028c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b1ea60505d1f082 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.334+0000 7f4969ffb640 1 -- 172.21.15.38:0/2823386590 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c02e080 con 0x56517a0644d0 2024-05-21T15:38:34.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.335+0000 7f4969ffb640 1 -- 172.21.15.38:0/2823386590 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495c009100 con 0x56517a0644d0 2024-05-21T15:38:34.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.335+0000 7f4969ffb640 1 -- 172.21.15.38:0/2823386590 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c028780 con 0x56517a0644d0 2024-05-21T15:38:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.336+0000 7f4981285740 1 -- 172.21.15.38:0/2823386590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a0644d0 msgr2=0x56517a4285e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.336+0000 7f4981285740 1 --2- 172.21.15.38:0/2823386590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a0644d0 0x56517a4285e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f495c00d530 tx=0x7f495c028c60 comp rx=0 tx=0).stop 2024-05-21T15:38:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.336+0000 7f4981285740 1 -- 172.21.15.38:0/2823386590 shutdown_connections 2024-05-21T15:38:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.336+0000 7f4981285740 1 --2- 172.21.15.38:0/2823386590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a08d390 0x56517a42b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.336+0000 7f4981285740 1 --2- 172.21.15.38:0/2823386590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a06c6b0 0x56517a44eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.336+0000 7f4981285740 1 --2- 172.21.15.38:0/2823386590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a0644d0 0x56517a4285e0 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f495c00d530 tx=0x7f495c028c60 comp rx=0 tx=0).stop 2024-05-21T15:38:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.336+0000 7f4981285740 1 -- 172.21.15.38:0/2823386590 >> 172.21.15.38:0/2823386590 conn(0x56517a0814e0 msgr2=0x56517a427ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.337+0000 7f4981285740 1 -- 172.21.15.38:0/2823386590 shutdown_connections 2024-05-21T15:38:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.337+0000 7f4981285740 1 -- 172.21.15.38:0/2823386590 wait complete. 2024-05-21T15:38:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.338+0000 7f4981285740 1 -- start start 2024-05-21T15:38:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.338+0000 7f4981285740 1 Processor -- start 2024-05-21T15:38:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.339+0000 7f4981285740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a06c6b0 0x56517a44c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.339+0000 7f4981285740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a08d390 0x56517a44c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.339+0000 7f4973fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a06c6b0 0x56517a44c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.339+0000 7f4973fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a06c6b0 0x56517a44c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:49610/0 (socket says 172.21.15.38:49610) 2024-05-21T15:38:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.339+0000 7f4973fff640 1 -- 172.21.15.38:0/801424818 learned_addr learned my addr 172.21.15.38:0/801424818 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.339+0000 7f4981285740 1 --2- 172.21.15.38:0/801424818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a0213a0 0x56517a44d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.339+0000 7f49737fe640 1 --2- 172.21.15.38:0/801424818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a08d390 0x56517a44c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.340+0000 7f4981285740 1 -- 172.21.15.38:0/801424818 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f495400fba0 con 0x56517a0213a0 2024-05-21T15:38:34.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.340+0000 7f4981285740 1 -- 172.21.15.38:0/801424818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f495400f4a0 con 0x56517a06c6b0 2024-05-21T15:38:34.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.340+0000 7f4981285740 1 -- 172.21.15.38:0/801424818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f495400c6e0 con 0x56517a08d390 2024-05-21T15:38:34.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.340+0000 7f4973fff640 1 -- 172.21.15.38:0/801424818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a08d390 msgr2=0x56517a44c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.340+0000 7f4973fff640 1 --2- 172.21.15.38:0/801424818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a08d390 0x56517a44c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.340+0000 7f4973fff640 1 -- 172.21.15.38:0/801424818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a0213a0 msgr2=0x56517a44d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.340+0000 7f4973fff640 1 --2- 172.21.15.38:0/801424818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a0213a0 0x56517a44d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.340+0000 7f4973fff640 1 -- 172.21.15.38:0/801424818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56517a44dd30 con 0x56517a06c6b0 2024-05-21T15:38:34.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.341+0000 7f4973fff640 1 --2- 172.21.15.38:0/801424818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a06c6b0 0x56517a44c3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f495c008010 tx=0x7f495c0286b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=585d9d6e72a06eae server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.341+0000 7f49537fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:38:34.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.341+0000 7f49537fe640 1 -- 172.21.15.38:0/801424818 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c036020 con 0x56517a06c6b0 2024-05-21T15:38:34.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.341+0000 7f49537fe640 1 -- 172.21.15.38:0/801424818 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f495c005ba0 con 0x56517a06c6b0 2024-05-21T15:38:34.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.342+0000 7f4981285740 10 client.0 _ll_register_callbacks cb 0x7f495c009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:34.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.342+0000 7f4981285740 1 -- 172.21.15.38:0/801424818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56517a3996c0 con 0x56517a06c6b0 2024-05-21T15:38:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.344+0000 7f49537fe640 1 -- 172.21.15.38:0/801424818 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f495c024070 con 0x56517a06c6b0 2024-05-21T15:38:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.344+0000 7f4981285740 10 client.26132 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.344+0000 7f4981285740 1 -- 172.21.15.38:0/801424818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=31}) -- 0x56517a44ed50 con 0x56517a06c6b0 2024-05-21T15:38:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.344+0000 7f49537fe640 1 -- 172.21.15.38:0/801424818 <== mon.2 v2:172.21.15.45:3300/0 4 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f495c0036b0 con 0x56517a06c6b0 2024-05-21T15:38:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.344+0000 7f4981285740 10 client.26132 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.344+0000 7f4981285740 10 client.26132 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:38:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.344+0000 7f4981285740 20 client.26132 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.344+0000 7f4981285740 1 --2- 172.21.15.38:0/801424818 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56517a4bddd0 0x56517a4c0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.344+0000 7f4981285740 4 client.26132 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:38:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.344+0000 7f4981285740 1 -- 172.21.15.38:0/801424818 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56517a316820 con 0x56517a4bddd0 2024-05-21T15:38:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.345+0000 7f49737fe640 1 --2- 172.21.15.38:0/801424818 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56517a4bddd0 0x56517a4c0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.346+0000 7f49737fe640 1 --2- 172.21.15.38:0/801424818 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56517a4bddd0 0x56517a4c0280 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d86c66c687f0b75d server_cookie=bf1219ff7e8c1389 in_seq=0 out_seq=0 2024-05-21T15:38:34.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.346+0000 7f49537fe640 10 client.26132 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:38:34.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.346+0000 7f49537fe640 1 -- 172.21.15.38:0/801424818 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56517a316820 con 0x56517a4bddd0 2024-05-21T15:38:34.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.346+0000 7f49537fe640 10 client.26132 handle_command_reply: tid=0 2024-05-21T15:38:34.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.347+0000 7f4981285740 1 client.26132 shutdown 2024-05-21T15:38:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.348+0000 7f4981285740 1 -- 172.21.15.38:0/801424818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a06c6b0 msgr2=0x56517a44c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.348+0000 7f4981285740 1 --2- 172.21.15.38:0/801424818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a06c6b0 0x56517a44c3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f495c008010 tx=0x7f495c0286b0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.348+0000 7f4981285740 1 -- 172.21.15.38:0/801424818 shutdown_connections 2024-05-21T15:38:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.348+0000 7f4981285740 1 --2- 172.21.15.38:0/801424818 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56517a4bddd0 0x56517a4c0280 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.348+0000 7f4981285740 1 --2- 172.21.15.38:0/801424818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56517a0213a0 0x56517a44d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.348+0000 7f49537fe640 0 client.26132 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:38:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.348+0000 7f4981285740 1 --2- 172.21.15.38:0/801424818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56517a08d390 0x56517a44c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.348+0000 7f4981285740 1 --2- 172.21.15.38:0/801424818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56517a06c6b0 0x56517a44c3c0 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f495c008010 tx=0x7f495c0286b0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.348+0000 7f4981285740 1 -- 172.21.15.38:0/801424818 >> 172.21.15.38:0/801424818 conn(0x56517a0814e0 msgr2=0x56517a426b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.349+0000 7f4981285740 1 -- 172.21.15.38:0/801424818 shutdown_connections 2024-05-21T15:38:34.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.349+0000 7f4981285740 1 -- 172.21.15.38:0/801424818 wait complete. 2024-05-21T15:38:34.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.349+0000 7f4981285740 20 client.26132 trim_cache size 0 max 16384 2024-05-21T15:38:34.350 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:38:34.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.352+0000 7f497a3e2640 1 -- 172.21.15.38:0/1200946990 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f49600a2d90 msgr2=0x7f49600a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.352+0000 7f497a3e2640 1 --2- 172.21.15.38:0/1200946990 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f49600a2d90 0x7f49600a52a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f495c006fd0 tx=0x7f495c008040 comp rx=0 tx=0).stop 2024-05-21T15:38:34.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.353+0000 7f497a3e2640 1 -- 172.21.15.38:0/1200946990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4974102c10 msgr2=0x7f49741113a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.353+0000 7f497a3e2640 1 --2- 172.21.15.38:0/1200946990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4974102c10 0x7f49741113a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4964009bf0 tx=0x7f4964003ba0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.353+0000 7f497a3e2640 1 -- 172.21.15.38:0/1200946990 shutdown_connections 2024-05-21T15:38:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.353+0000 7f497a3e2640 1 --2- 172.21.15.38:0/1200946990 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f49600a2d90 0x7f49600a52a0 secure :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f495c006fd0 tx=0x7f495c008040 comp rx=0 tx=0).stop 2024-05-21T15:38:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.353+0000 7f497a3e2640 1 --2- 172.21.15.38:0/1200946990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4974107910 0x7f49741119f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.353+0000 7f497a3e2640 1 --2- 172.21.15.38:0/1200946990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4974102c10 0x7f49741113a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.353+0000 7f497a3e2640 1 --2- 172.21.15.38:0/1200946990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49741019d0 0x7f4974112d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.353+0000 7f497a3e2640 1 -- 172.21.15.38:0/1200946990 >> 172.21.15.38:0/1200946990 conn(0x7f49740fcbc0 msgr2=0x7f49741038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.354+0000 7f497a3e2640 1 -- 172.21.15.38:0/1200946990 shutdown_connections 2024-05-21T15:38:34.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.354+0000 7f497a3e2640 1 -- 172.21.15.38:0/1200946990 wait complete. 2024-05-21T15:38:34.375 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:38:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.441+0000 7f8d3759e640 1 -- 172.21.15.38:0/3085047595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d300a72d0 msgr2=0x7f8d300a76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.441+0000 7f8d3759e640 1 --2- 172.21.15.38:0/3085047595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d300a72d0 0x7f8d300a76e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009900 tx=0x7f8d2802a3f0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.441+0000 7f8d3759e640 1 -- 172.21.15.38:0/3085047595 shutdown_connections 2024-05-21T15:38:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.441+0000 7f8d3759e640 1 --2- 172.21.15.38:0/3085047595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d300abfd0 0x7f8d300ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.441+0000 7f8d3759e640 1 --2- 172.21.15.38:0/3085047595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d300a72d0 0x7f8d300a76e0 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009900 tx=0x7f8d2802a3f0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.441+0000 7f8d3759e640 1 --2- 172.21.15.38:0/3085047595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d300a61d0 0x7f8d300a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.441+0000 7f8d3759e640 1 -- 172.21.15.38:0/3085047595 >> 172.21.15.38:0/3085047595 conn(0x7f8d300a09b0 msgr2=0x7f8d300a2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.442+0000 7f8d3759e640 1 -- 172.21.15.38:0/3085047595 shutdown_connections 2024-05-21T15:38:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.442+0000 7f8d3759e640 1 -- 172.21.15.38:0/3085047595 wait complete. 2024-05-21T15:38:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.443+0000 7f8d3759e640 1 Processor -- start 2024-05-21T15:38:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.443+0000 7f8d3759e640 1 -- start start 2024-05-21T15:38:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.443+0000 7f8d3759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d300a61d0 0x7f8d30144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.443+0000 7f8d3759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d300abfd0 0x7f8d30144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.443+0000 7f8d3759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d30149d20 0x7f8d3014c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.443+0000 7f8d3759e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8d300b1470 con 0x7f8d30149d20 2024-05-21T15:38:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.443+0000 7f8d3759e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8d300b12f0 con 0x7f8d300a61d0 2024-05-21T15:38:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.443+0000 7f8d3759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8d300b15f0 con 0x7f8d300abfd0 2024-05-21T15:38:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.443+0000 7f8d35d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d300abfd0 0x7f8d30144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.443+0000 7f8d35d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d300abfd0 0x7f8d30144e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:45502/0 (socket says 172.21.15.38:45502) 2024-05-21T15:38:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.443+0000 7f8d35d9b640 1 -- 172.21.15.38:0/4213673880 learned_addr learned my addr 172.21.15.38:0/4213673880 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.444+0000 7f8d36d9d640 1 --2- 172.21.15.38:0/4213673880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d30149d20 0x7f8d3014c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.444+0000 7f8d3659c640 1 --2- 172.21.15.38:0/4213673880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d300a61d0 0x7f8d30144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.444+0000 7f8d35d9b640 1 -- 172.21.15.38:0/4213673880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d300a61d0 msgr2=0x7f8d30144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.444+0000 7f8d35d9b640 1 --2- 172.21.15.38:0/4213673880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d300a61d0 0x7f8d30144860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.444+0000 7f8d35d9b640 1 -- 172.21.15.38:0/4213673880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d30149d20 msgr2=0x7f8d3014c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.444+0000 7f8d35d9b640 1 --2- 172.21.15.38:0/4213673880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d30149d20 0x7f8d3014c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.444+0000 7f8d35d9b640 1 -- 172.21.15.38:0/4213673880 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d3014c7e0 con 0x7f8d300abfd0 2024-05-21T15:38:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.445+0000 7f8d35d9b640 1 --2- 172.21.15.38:0/4213673880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d300abfd0 0x7f8d30144e20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8d2802a9e0 tx=0x7f8d28032c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.445+0000 7f8d277fe640 1 -- 172.21.15.38:0/4213673880 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2803f030 con 0x7f8d300abfd0 2024-05-21T15:38:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.445+0000 7f8d277fe640 1 -- 172.21.15.38:0/4213673880 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d28032e50 con 0x7f8d300abfd0 2024-05-21T15:38:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.445+0000 7f8d277fe640 1 -- 172.21.15.38:0/4213673880 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28035070 con 0x7f8d300abfd0 2024-05-21T15:38:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.446+0000 7f8d3759e640 1 -- 172.21.15.38:0/4213673880 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d3014ca70 con 0x7f8d300abfd0 2024-05-21T15:38:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.446+0000 7f8d3759e640 1 -- 172.21.15.38:0/4213673880 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d3014cee0 con 0x7f8d300abfd0 2024-05-21T15:38:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.451+0000 7f8d43843740 1 Processor -- start 2024-05-21T15:38:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.452+0000 7f8d277fe640 1 -- 172.21.15.38:0/4213673880 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8d28042030 con 0x7f8d300abfd0 2024-05-21T15:38:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.452+0000 7f8d43843740 1 -- start start 2024-05-21T15:38:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.453+0000 7f8d277fe640 1 --2- 172.21.15.38:0/4213673880 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8d200a2da0 0x7f8d200a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.453+0000 7f8d277fe640 1 -- 172.21.15.38:0/4213673880 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8d280ef330 con 0x7f8d300abfd0 2024-05-21T15:38:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.454+0000 7f8d43843740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346dae1b0 0x55c34714e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.454+0000 7f8d43843740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346d852f0 0x55c34714e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.454+0000 7f8d43843740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c3471554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.454+0000 7f8d43843740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c3470b2340 con 0x55c346d8d4d0 2024-05-21T15:38:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.454+0000 7f8d43843740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c34705a290 con 0x55c346d852f0 2024-05-21T15:38:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.454+0000 7f8d43843740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c3470ae300 con 0x55c346dae1b0 2024-05-21T15:38:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.454+0000 7f8d3659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346dae1b0 0x55c34714e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.454+0000 7f8d3659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346dae1b0 0x55c34714e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:45512/0 (socket says 172.21.15.38:45512) 2024-05-21T15:38:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.454+0000 7f8d3659c640 1 -- 172.21.15.38:0/2252460484 learned_addr learned my addr 172.21.15.38:0/2252460484 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.455+0000 7f8d3659c640 1 -- 172.21.15.38:0/2252460484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346d852f0 msgr2=0x55c34714e5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:38:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.455+0000 7f8d3659c640 1 --2- 172.21.15.38:0/2252460484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346d852f0 0x55c34714e5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.455+0000 7f8d3659c640 1 -- 172.21.15.38:0/2252460484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 msgr2=0x55c3471554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.455+0000 7f8d3659c640 1 --2- 172.21.15.38:0/2252460484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c3471554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.455+0000 7f8d3659c640 1 -- 172.21.15.38:0/2252460484 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c34707e8e0 con 0x55c346dae1b0 2024-05-21T15:38:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.455+0000 7f8d36d9d640 1 --2- 172.21.15.38:0/4213673880 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8d200a2da0 0x7f8d200a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.455+0000 7f8d3659c640 1 --2- 172.21.15.38:0/2252460484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346dae1b0 0x55c34714e050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8d38067be0 tx=0x7f8d38069ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48de24753cdfba29 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.455+0000 7f8d24ff9640 1 -- 172.21.15.38:0/2252460484 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38075b80 con 0x55c346dae1b0 2024-05-21T15:38:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.456+0000 7f8d24ff9640 1 -- 172.21.15.38:0/2252460484 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3807a070 con 0x55c346dae1b0 2024-05-21T15:38:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.456+0000 7f8d24ff9640 1 -- 172.21.15.38:0/2252460484 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3807e680 con 0x55c346dae1b0 2024-05-21T15:38:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.457+0000 7f8d43843740 1 -- 172.21.15.38:0/2252460484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346dae1b0 msgr2=0x55c34714e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.457+0000 7f8d43843740 1 --2- 172.21.15.38:0/2252460484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346dae1b0 0x55c34714e050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8d38067be0 tx=0x7f8d38069ca0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.457+0000 7f8d36d9d640 1 --2- 172.21.15.38:0/4213673880 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8d200a2da0 0x7f8d200a52b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00a500 tx=0x7f8d2c00a490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.457+0000 7f8d43843740 1 -- 172.21.15.38:0/2252460484 shutdown_connections 2024-05-21T15:38:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.457+0000 7f8d43843740 1 --2- 172.21.15.38:0/2252460484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c3471554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.457+0000 7f8d43843740 1 --2- 172.21.15.38:0/2252460484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346d852f0 0x55c34714e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.457+0000 7f8d43843740 1 --2- 172.21.15.38:0/2252460484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346dae1b0 0x55c34714e050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.457+0000 7f8d43843740 1 -- 172.21.15.38:0/2252460484 >> 172.21.15.38:0/2252460484 conn(0x55c346da2300 msgr2=0x55c3471499e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.457+0000 7f8d43843740 1 -- 172.21.15.38:0/2252460484 shutdown_connections 2024-05-21T15:38:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.457+0000 7f8d43843740 1 -- 172.21.15.38:0/2252460484 wait complete. 2024-05-21T15:38:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.458+0000 7f8d43843740 1 -- start start 2024-05-21T15:38:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.458+0000 7f8d43843740 1 Processor -- start 2024-05-21T15:38:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.459+0000 7f8d43843740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346d852f0 0x55c347157db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.459+0000 7f8d43843740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c3471582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.459+0000 7f8d3659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346d852f0 0x55c347157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.460+0000 7f8d3659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346d852f0 0x55c347157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:45522/0 (socket says 172.21.15.38:45522) 2024-05-21T15:38:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.460+0000 7f8d35d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c3471582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.460+0000 7f8d35d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c3471582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42704/0 (socket says 172.21.15.38:42704) 2024-05-21T15:38:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.460+0000 7f8d3659c640 1 -- 172.21.15.38:0/881156892 learned_addr learned my addr 172.21.15.38:0/881156892 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.460+0000 7f8d43843740 1 --2- 172.21.15.38:0/881156892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346dae1b0 0x55c34715f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.460+0000 7f8d43843740 1 -- 172.21.15.38:0/881156892 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c347155f20 con 0x55c346d8d4d0 2024-05-21T15:38:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.460+0000 7f8d43843740 1 -- 172.21.15.38:0/881156892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c34706d790 con 0x55c346dae1b0 2024-05-21T15:38:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.460+0000 7f8d43843740 1 -- 172.21.15.38:0/881156892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c3471560a0 con 0x55c346d852f0 2024-05-21T15:38:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.460+0000 7f8d43843740 10 client.0 _ll_register_callbacks cb 0x55c3470a8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.460+0000 7f8d36d9d640 1 --2- 172.21.15.38:0/881156892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346dae1b0 0x55c34715f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.461+0000 7f8d35d9b640 1 -- 172.21.15.38:0/881156892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346d852f0 msgr2=0x55c347157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.461+0000 7f8d35d9b640 1 --2- 172.21.15.38:0/881156892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346d852f0 0x55c347157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.461+0000 7f8d35d9b640 1 -- 172.21.15.38:0/881156892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346dae1b0 msgr2=0x55c34715f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.461+0000 7f8d35d9b640 1 --2- 172.21.15.38:0/881156892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346dae1b0 0x55c34715f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.461+0000 7f8d35d9b640 1 -- 172.21.15.38:0/881156892 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c346dc6210 con 0x55c346d8d4d0 2024-05-21T15:38:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.461+0000 7f8d35d9b640 1 --2- 172.21.15.38:0/881156892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c3471582f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x55c34714f430 tx=0x7f8d2802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9826fa096d398adc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.461+0000 7f8d1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.461+0000 7f8d1dffb640 1 -- 172.21.15.38:0/881156892 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28049040 con 0x55c346d8d4d0 2024-05-21T15:38:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.461+0000 7f8d43843740 1 -- 172.21.15.38:0/881156892 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d3807a070 con 0x55c346d8d4d0 2024-05-21T15:38:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.462+0000 7f8d1dffb640 1 -- 172.21.15.38:0/881156892 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8d280b8150 con 0x55c346d8d4d0 2024-05-21T15:38:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.462+0000 7f8d1dffb640 1 -- 172.21.15.38:0/881156892 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8d280d9050 con 0x55c346d8d4d0 2024-05-21T15:38:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.462+0000 7f8d43843740 10 client.16479 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.462+0000 7f8d43843740 1 -- 172.21.15.38:0/881156892 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c346e92620 con 0x55c346d8d4d0 2024-05-21T15:38:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.462+0000 7f8d1dffb640 1 -- 172.21.15.38:0/881156892 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3807a070 con 0x55c346d8d4d0 2024-05-21T15:38:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.462+0000 7f8d1dffb640 1 -- 172.21.15.38:0/881156892 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f8d280d4ce0 con 0x55c346d8d4d0 2024-05-21T15:38:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.462+0000 7f8d43843740 10 client.16479 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.462+0000 7f8d43843740 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.462+0000 7f8d43843740 20 client.16479 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.462+0000 7f8d43843740 1 --2- 172.21.15.38:0/881156892 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c346d6cd80 0x55c347165730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.462+0000 7f8d43843740 4 client.16479 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:38:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.462+0000 7f8d43843740 1 -- 172.21.15.38:0/881156892 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3470bfa70 con 0x55c346d6cd80 2024-05-21T15:38:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.463+0000 7f8d3659c640 1 --2- 172.21.15.38:0/881156892 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c346d6cd80 0x55c347165730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.463+0000 7f8d1dffb640 10 client.16479 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.463+0000 7f8d3659c640 1 --2- 172.21.15.38:0/881156892 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c346d6cd80 0x55c347165730 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75851e7d1716c9a server_cookie=4f48a2d61653ebc8 in_seq=0 out_seq=0 2024-05-21T15:38:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.464+0000 7f8d1dffb640 1 -- 172.21.15.38:0/881156892 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c3470bfa70 con 0x55c346d6cd80 2024-05-21T15:38:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.464+0000 7f8d1dffb640 10 client.16479 handle_command_reply: tid=0 2024-05-21T15:38:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.464+0000 7f8d43843740 1 client.16479 shutdown 2024-05-21T15:38:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.464+0000 7f8d43843740 1 -- 172.21.15.38:0/881156892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 msgr2=0x55c3471582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.464+0000 7f8d43843740 1 --2- 172.21.15.38:0/881156892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c3471582f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x55c34714f430 tx=0x7f8d2802a3f0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.464+0000 7f8d43843740 1 -- 172.21.15.38:0/881156892 shutdown_connections 2024-05-21T15:38:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.464+0000 7f8d43843740 1 --2- 172.21.15.38:0/881156892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346dae1b0 0x55c34715f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.464+0000 7f8d43843740 1 --2- 172.21.15.38:0/881156892 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c346d6cd80 0x55c347165730 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.464+0000 7f8d43843740 1 --2- 172.21.15.38:0/881156892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c3471582f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.464+0000 7f8d43843740 1 --2- 172.21.15.38:0/881156892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346d852f0 0x55c347157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.464+0000 7f8d1dffb640 0 client.16479 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.464+0000 7f8d43843740 1 -- 172.21.15.38:0/881156892 >> 172.21.15.38:0/881156892 conn(0x55c346da2300 msgr2=0x55c347148ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.465+0000 7f8d43843740 1 -- 172.21.15.38:0/881156892 shutdown_connections 2024-05-21T15:38:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.465+0000 7f8d43843740 1 -- 172.21.15.38:0/881156892 wait complete. 2024-05-21T15:38:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.465+0000 7f8d43843740 20 client.16479 trim_cache size 0 max 16384 2024-05-21T15:38:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.486+0000 7f8d43843740 1 Processor -- start 2024-05-21T15:38:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.486+0000 7f8d43843740 1 -- start start 2024-05-21T15:38:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.487+0000 7f8d43843740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346d852f0 0x55c347149440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.487+0000 7f8d43843740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346d8d4d0 0x55c347149980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.487+0000 7f8d43843740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346dae1b0 0x55c34715f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.487+0000 7f8d43843740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8d30155dd0 con 0x55c346dae1b0 2024-05-21T15:38:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.487+0000 7f8d43843740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8d30157a00 con 0x55c346d8d4d0 2024-05-21T15:38:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.487+0000 7f8d3659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346d852f0 0x55c347149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.487+0000 7f8d3659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346d852f0 0x55c347149440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:45538/0 (socket says 172.21.15.38:45538) 2024-05-21T15:38:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.487+0000 7f8d3659c640 1 -- 172.21.15.38:0/3282275814 learned_addr learned my addr 172.21.15.38:0/3282275814 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.488+0000 7f8d43843740 1 -- 172.21.15.38:0/3282275814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8d301564a0 con 0x55c346d852f0 2024-05-21T15:38:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.488+0000 7f8d36d9d640 1 --2- 172.21.15.38:0/3282275814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346dae1b0 0x55c34715f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.488+0000 7f8d35d9b640 1 --2- 172.21.15.38:0/3282275814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346d8d4d0 0x55c347149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.489+0000 7f8d3659c640 1 -- 172.21.15.38:0/3282275814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346d8d4d0 msgr2=0x55c347149980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.489+0000 7f8d3659c640 1 --2- 172.21.15.38:0/3282275814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346d8d4d0 0x55c347149980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.489+0000 7f8d3659c640 1 -- 172.21.15.38:0/3282275814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346dae1b0 msgr2=0x55c34715f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.489+0000 7f8d3659c640 1 --2- 172.21.15.38:0/3282275814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346dae1b0 0x55c34715f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.489+0000 7f8d3659c640 1 -- 172.21.15.38:0/3282275814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c34714dc00 con 0x55c346d852f0 2024-05-21T15:38:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.490+0000 7f8d3659c640 1 --2- 172.21.15.38:0/3282275814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346d852f0 0x55c347149440 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8d38050f00 tx=0x7f8d38080040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffcb43d4f47b7316 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.570+0000 7fbf9ba61640 1 -- 172.21.15.38:0/2602194598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf94101980 msgr2=0x7fbf94102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.570+0000 7fbf9ba61640 1 --2- 172.21.15.38:0/2602194598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf94101980 0x7fbf94102680 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009900 tx=0x7fbf9002a410 comp rx=0 tx=0).stop 2024-05-21T15:38:34.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.570+0000 7fbf9ba61640 1 -- 172.21.15.38:0/2602194598 shutdown_connections 2024-05-21T15:38:34.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.570+0000 7fbf9ba61640 1 --2- 172.21.15.38:0/2602194598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf941078c0 0x7fbf94109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.570+0000 7fbf9ba61640 1 --2- 172.21.15.38:0/2602194598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf94102bc0 0x7fbf94102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.570+0000 7fbf9ba61640 1 --2- 172.21.15.38:0/2602194598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf94101980 0x7fbf94102680 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.570+0000 7fbf9ba61640 1 -- 172.21.15.38:0/2602194598 >> 172.21.15.38:0/2602194598 conn(0x7fbf940fcb70 msgr2=0x7fbf940ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.570+0000 7fbf9ba61640 1 -- 172.21.15.38:0/2602194598 shutdown_connections 2024-05-21T15:38:34.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.570+0000 7fbf9ba61640 1 -- 172.21.15.38:0/2602194598 wait complete. 2024-05-21T15:38:34.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.571+0000 7fbf9ba61640 1 Processor -- start 2024-05-21T15:38:34.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.571+0000 7fbf9ba61640 1 -- start start 2024-05-21T15:38:34.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.571+0000 7fbf9ba61640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf94101980 0x7fbf9419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.571+0000 7fbf9ba61640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf94102bc0 0x7fbf9419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.571+0000 7fbf9ba61640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf941078c0 0x7fbf941a2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.571+0000 7fbf9ba61640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410c530 con 0x7fbf941078c0 2024-05-21T15:38:34.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.571+0000 7fbf9ba61640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410c3b0 con 0x7fbf94101980 2024-05-21T15:38:34.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.571+0000 7fbf9ba61640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410c6b0 con 0x7fbf94102bc0 2024-05-21T15:38:34.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.572+0000 7fbf99ffe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf941078c0 0x7fbf941a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.572+0000 7fbf99ffe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf941078c0 0x7fbf941a2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42728/0 (socket says 172.21.15.38:42728) 2024-05-21T15:38:34.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.572+0000 7fbf99ffe640 1 -- 172.21.15.38:0/219758516 learned_addr learned my addr 172.21.15.38:0/219758516 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.572+0000 7fbf997fd640 1 --2- 172.21.15.38:0/219758516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf94101980 0x7fbf9419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.572+0000 7fbf98ffc640 1 --2- 172.21.15.38:0/219758516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf94102bc0 0x7fbf9419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.572+0000 7fbf98ffc640 1 -- 172.21.15.38:0/219758516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf94101980 msgr2=0x7fbf9419b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.572+0000 7fbf98ffc640 1 --2- 172.21.15.38:0/219758516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf94101980 0x7fbf9419b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.572+0000 7fbf98ffc640 1 -- 172.21.15.38:0/219758516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf941078c0 msgr2=0x7fbf941a2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.572+0000 7fbf98ffc640 1 --2- 172.21.15.38:0/219758516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf941078c0 0x7fbf941a2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.572+0000 7fbf98ffc640 1 -- 172.21.15.38:0/219758516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf941a2f20 con 0x7fbf94102bc0 2024-05-21T15:38:34.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.572+0000 7fbf98ffc640 1 --2- 172.21.15.38:0/219758516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf94102bc0 0x7fbf9419b9d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400c970 tx=0x7fbf8400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.607+0000 7f8d1ffff640 1 -- 172.21.15.38:0/3282275814 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38069240 con 0x55c346d852f0 2024-05-21T15:38:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.607+0000 7fbf8a7fc640 1 -- 172.21.15.38:0/219758516 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84007bb0 con 0x7fbf94102bc0 2024-05-21T15:38:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.607+0000 7f8d1ffff640 1 -- 172.21.15.38:0/3282275814 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3807a070 con 0x55c346d852f0 2024-05-21T15:38:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.607+0000 7f8d1ffff640 1 -- 172.21.15.38:0/3282275814 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38076440 con 0x55c346d852f0 2024-05-21T15:38:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.607+0000 7fbf9ba61640 1 -- 172.21.15.38:0/219758516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf941a3210 con 0x7fbf94102bc0 2024-05-21T15:38:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.607+0000 7fbf8a7fc640 1 -- 172.21.15.38:0/219758516 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf84007d50 con 0x7fbf94102bc0 2024-05-21T15:38:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.607+0000 7fbf8a7fc640 1 -- 172.21.15.38:0/219758516 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84005260 con 0x7fbf94102bc0 2024-05-21T15:38:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.607+0000 7fbf9ba61640 1 -- 172.21.15.38:0/219758516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf941a37b0 con 0x7fbf94102bc0 2024-05-21T15:38:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.608+0000 7f8d43843740 1 -- 172.21.15.38:0/3282275814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346d852f0 msgr2=0x55c347149440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.608+0000 7f8d43843740 1 --2- 172.21.15.38:0/3282275814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346d852f0 0x55c347149440 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8d38050f00 tx=0x7f8d38080040 comp rx=0 tx=0).stop 2024-05-21T15:38:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.608+0000 7f8d43843740 1 -- 172.21.15.38:0/3282275814 shutdown_connections 2024-05-21T15:38:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.608+0000 7f8d43843740 1 --2- 172.21.15.38:0/3282275814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346dae1b0 0x55c34715f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.608+0000 7f8d43843740 1 --2- 172.21.15.38:0/3282275814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346d8d4d0 0x55c347149980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.608+0000 7f8d43843740 1 --2- 172.21.15.38:0/3282275814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346d852f0 0x55c347149440 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.608+0000 7f8d43843740 1 -- 172.21.15.38:0/3282275814 >> 172.21.15.38:0/3282275814 conn(0x55c346da2300 msgr2=0x55c3471521e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.608+0000 7f8d43843740 1 -- 172.21.15.38:0/3282275814 shutdown_connections 2024-05-21T15:38:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.608+0000 7f8d43843740 1 -- 172.21.15.38:0/3282275814 wait complete. 2024-05-21T15:38:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.608+0000 7f8d43843740 1 -- start start 2024-05-21T15:38:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.609+0000 7f8d43843740 1 Processor -- start 2024-05-21T15:38:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.609+0000 7f8d43843740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346d852f0 0x55c34715adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.609+0000 7f8d43843740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c34715b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.609+0000 7f8d43843740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346dae1b0 0x55c34715e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.609+0000 7f8d43843740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8d30157270 con 0x55c346d8d4d0 2024-05-21T15:38:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.609+0000 7f8d43843740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8d30156b70 con 0x55c346d852f0 2024-05-21T15:38:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.609+0000 7f8d43843740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8d30153db0 con 0x55c346dae1b0 2024-05-21T15:38:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.609+0000 7f8d43843740 10 client.0 _ll_register_callbacks cb 0x7f8d28005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.609+0000 7f8d35d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c34715b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.609+0000 7f8d35d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c34715b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42730/0 (socket says 172.21.15.38:42730) 2024-05-21T15:38:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.609+0000 7f8d3659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346d852f0 0x55c34715adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.609+0000 7f8d35d9b640 1 -- 172.21.15.38:0/4038879448 learned_addr learned my addr 172.21.15.38:0/4038879448 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.609+0000 7f8d36d9d640 1 --2- 172.21.15.38:0/4038879448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346dae1b0 0x55c34715e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.610+0000 7f8d35d9b640 1 -- 172.21.15.38:0/4038879448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346dae1b0 msgr2=0x55c34715e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.610+0000 7f8d35d9b640 1 --2- 172.21.15.38:0/4038879448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346dae1b0 0x55c34715e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.610+0000 7f8d35d9b640 1 -- 172.21.15.38:0/4038879448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346d852f0 msgr2=0x55c34715adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.610+0000 7f8d35d9b640 1 --2- 172.21.15.38:0/4038879448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346d852f0 0x55c34715adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.610+0000 7f8d35d9b640 1 -- 172.21.15.38:0/4038879448 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c34715e7f0 con 0x55c346d8d4d0 2024-05-21T15:38:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.610+0000 7f8d36d9d640 1 --2- 172.21.15.38:0/4038879448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346dae1b0 0x55c34715e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:38:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.610+0000 7f8d35d9b640 1 --2- 172.21.15.38:0/4038879448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c34715b2f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8d280bb7e0 tx=0x7f8d2803b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92c9d6a3b75795c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.610+0000 7f8d1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.610+0000 7fbfa29e4740 1 Processor -- start 2024-05-21T15:38:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.610+0000 7fbfa29e4740 1 -- start start 2024-05-21T15:38:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbfa29e4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 0x558e8532cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbfa29e4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 0x558e8532d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbfa29e4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f6c6b0 0x558e85333f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbfa29e4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558e85291c40 con 0x558e84f8d390 2024-05-21T15:38:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbfa29e4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558e85239580 con 0x558e84f6c6b0 2024-05-21T15:38:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbfa29e4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558e8528dc00 con 0x558e84f644d0 2024-05-21T15:38:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbf99ffe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 0x558e8532cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbf99ffe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 0x558e8532cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42746/0 (socket says 172.21.15.38:42746) 2024-05-21T15:38:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbf99ffe640 1 -- 172.21.15.38:0/3008258761 learned_addr learned my addr 172.21.15.38:0/3008258761 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbf98ffc640 1 --2- 172.21.15.38:0/3008258761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f6c6b0 0x558e85333f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.611+0000 7f8d1dffb640 1 -- 172.21.15.38:0/4038879448 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d280c9060 con 0x55c346d8d4d0 2024-05-21T15:38:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.611+0000 7f8d1dffb640 1 -- 172.21.15.38:0/4038879448 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8d280da440 con 0x55c346d8d4d0 2024-05-21T15:38:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.611+0000 7f8d43843740 1 -- 172.21.15.38:0/4038879448 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d3807a070 con 0x55c346d8d4d0 2024-05-21T15:38:34.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbf997fd640 1 --2- 172.21.15.38:0/3008258761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 0x558e8532d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbf99ffe640 1 -- 172.21.15.38:0/3008258761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 msgr2=0x558e8532d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbf99ffe640 1 --2- 172.21.15.38:0/3008258761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 0x558e8532d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbf99ffe640 1 -- 172.21.15.38:0/3008258761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f6c6b0 msgr2=0x558e85333f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbf99ffe640 1 --2- 172.21.15.38:0/3008258761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f6c6b0 0x558e85333f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbf99ffe640 1 -- 172.21.15.38:0/3008258761 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e84fa53f0 con 0x558e84f8d390 2024-05-21T15:38:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.611+0000 7f8d1dffb640 1 -- 172.21.15.38:0/4038879448 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8d28049040 con 0x55c346d8d4d0 2024-05-21T15:38:34.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbf997fd640 1 --2- 172.21.15.38:0/3008258761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 0x558e8532d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:38:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.611+0000 7f8d43843740 10 client.16491 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.611+0000 7f8d43843740 1 -- 172.21.15.38:0/4038879448 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c34715fa80 con 0x55c346d8d4d0 2024-05-21T15:38:34.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.611+0000 7fbf99ffe640 1 --2- 172.21.15.38:0/3008258761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 0x558e8532cb00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00d4a0 tx=0x7fbf8c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d98506fff30f062 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.612+0000 7f8d1dffb640 1 -- 172.21.15.38:0/4038879448 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3807a070 con 0x55c346d8d4d0 2024-05-21T15:38:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.612+0000 7f8d1dffb640 1 -- 172.21.15.38:0/4038879448 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f8d280d4ce0 con 0x55c346d8d4d0 2024-05-21T15:38:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.612+0000 7f8d43843740 10 client.16491 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.612+0000 7f8d43843740 10 client.16491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.612+0000 7f8d43843740 20 client.16491 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.612+0000 7f8d43843740 1 --2- 172.21.15.38:0/4038879448 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c346d421c0 0x55c3471cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.612+0000 7f8d43843740 4 client.16491 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:38:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.612+0000 7f8d43843740 1 -- 172.21.15.38:0/4038879448 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3470b9dc0 con 0x55c346d421c0 2024-05-21T15:38:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.612+0000 7f8d3659c640 1 --2- 172.21.15.38:0/4038879448 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c346d421c0 0x55c3471cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.612+0000 7fbf7f7fe640 1 -- 172.21.15.38:0/3008258761 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c004b20 con 0x558e84f8d390 2024-05-21T15:38:34.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.613+0000 7fbf7f7fe640 1 -- 172.21.15.38:0/3008258761 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf8c004cc0 con 0x558e84f8d390 2024-05-21T15:38:34.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.613+0000 7fbf7f7fe640 1 -- 172.21.15.38:0/3008258761 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c018820 con 0x558e84f8d390 2024-05-21T15:38:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.613+0000 7f8d3659c640 1 --2- 172.21.15.38:0/4038879448 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c346d421c0 0x55c3471cff10 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b63d48140a36be server_cookie=264b682228162a68 in_seq=0 out_seq=0 2024-05-21T15:38:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:34.613+0000 7f8d1dffb640 10 client.16491 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:34.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.613+0000 7fbfa29e4740 1 -- 172.21.15.38:0/3008258761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 msgr2=0x558e8532cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.613+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/3008258761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 0x558e8532cb00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00d4a0 tx=0x7fbf8c009500 comp rx=0 tx=0).stop 2024-05-21T15:38:34.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.613+0000 7fbfa29e4740 1 -- 172.21.15.38:0/3008258761 shutdown_connections 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.613+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/3008258761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f6c6b0 0x558e85333f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.613+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/3008258761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 0x558e8532d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.613+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/3008258761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 0x558e8532cb00 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.613+0000 7fbfa29e4740 1 -- 172.21.15.38:0/3008258761 >> 172.21.15.38:0/3008258761 conn(0x558e84f814e0 msgr2=0x558e85328490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.614+0000 7fbfa29e4740 1 -- 172.21.15.38:0/3008258761 shutdown_connections 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.614+0000 7fbfa29e4740 1 -- 172.21.15.38:0/3008258761 wait complete. 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.614+0000 7fbfa29e4740 1 -- start start 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.614+0000 7fbfa29e4740 1 Processor -- start 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.615+0000 7fbfa29e4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f644d0 0x558e853369b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.615+0000 7fbfa29e4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f6c6b0 0x558e85336ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.615+0000 7fbfa29e4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 0x558e8533de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.615+0000 7fbfa29e4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558e853349d0 con 0x558e84f8d390 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.615+0000 7fbfa29e4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558e8524cc40 con 0x558e84f644d0 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.615+0000 7fbfa29e4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558e85334b50 con 0x558e84f6c6b0 2024-05-21T15:38:34.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.615+0000 7fbf997fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f6c6b0 0x558e85336ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.615+0000 7fbf997fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f6c6b0 0x558e85336ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:45572/0 (socket says 172.21.15.38:45572) 2024-05-21T15:38:34.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.615+0000 7fbf997fd640 1 -- 172.21.15.38:0/1132618234 learned_addr learned my addr 172.21.15.38:0/1132618234 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.615+0000 7fbf99ffe640 1 --2- 172.21.15.38:0/1132618234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f644d0 0x558e853369b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.615+0000 7fbfa29e4740 10 client.0 _ll_register_callbacks cb 0x558e84fa4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:34.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.616+0000 7fbf99ffe640 1 -- 172.21.15.38:0/1132618234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f6c6b0 msgr2=0x558e85336ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.616+0000 7fbf99ffe640 1 --2- 172.21.15.38:0/1132618234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f6c6b0 0x558e85336ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.616+0000 7fbf99ffe640 1 -- 172.21.15.38:0/1132618234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 msgr2=0x558e8533de50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:38:34.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.616+0000 7fbf99ffe640 1 --2- 172.21.15.38:0/1132618234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 0x558e8533de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.616+0000 7fbf99ffe640 1 -- 172.21.15.38:0/1132618234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e8506f7d0 con 0x558e84f644d0 2024-05-21T15:38:34.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.616+0000 7fbf99ffe640 1 --2- 172.21.15.38:0/1132618234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f644d0 0x558e853369b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00ddf0 tx=0x7fbf8c005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9c8c35ad003480 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.616+0000 7fbf7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:38:34.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.617+0000 7fbf8a7fc640 1 -- 172.21.15.38:0/219758516 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbf84005440 con 0x7fbf94102bc0 2024-05-21T15:38:34.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.617+0000 7fbf7cff9640 1 -- 172.21.15.38:0/1132618234 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c028070 con 0x558e84f644d0 2024-05-21T15:38:34.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.617+0000 7fbf7cff9640 1 -- 172.21.15.38:0/1132618234 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbf8c009dc0 con 0x558e84f644d0 2024-05-21T15:38:34.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.617+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1132618234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf8c004cc0 con 0x558e84f644d0 2024-05-21T15:38:34.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.617+0000 7fbf7cff9640 1 -- 172.21.15.38:0/1132618234 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbf8c034050 con 0x558e84f644d0 2024-05-21T15:38:34.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.617+0000 7fbfa29e4740 10 client.26156 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:34.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.617+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1132618234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558e85341100 con 0x558e84f644d0 2024-05-21T15:38:34.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.617+0000 7fbf8a7fc640 1 --2- 172.21.15.38:0/219758516 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf800a2b20 0x7fbf800a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.618+0000 7fbf7cff9640 1 -- 172.21.15.38:0/1132618234 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf8c004cc0 con 0x558e84f644d0 2024-05-21T15:38:34.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.618+0000 7fbf8a7fc640 1 -- 172.21.15.38:0/219758516 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbf840cf220 con 0x7fbf94102bc0 2024-05-21T15:38:34.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.618+0000 7fbf997fd640 1 --2- 172.21.15.38:0/219758516 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf800a2b20 0x7fbf800a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.618+0000 7fbf7cff9640 1 -- 172.21.15.38:0/1132618234 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fbf8c022780 con 0x558e84f644d0 2024-05-21T15:38:34.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.618+0000 7fbfa29e4740 10 client.26156 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:34.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.618+0000 7fbfa29e4740 10 client.26156 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:38:34.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.618+0000 7fbfa29e4740 20 client.26156 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:34.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.618+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1132618234 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x558e84f4bf60 0x558e85344cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.618+0000 7fbfa29e4740 4 client.26156 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:38:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.618+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1132618234 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e8529f370 con 0x558e84f4bf60 2024-05-21T15:38:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.619+0000 7fbf997fd640 1 --2- 172.21.15.38:0/219758516 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf800a2b20 0x7fbf800a5030 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x558e85337a20 tx=0x7fbf900315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.619+0000 7fbf98ffc640 1 --2- 172.21.15.38:0/1132618234 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x558e84f4bf60 0x558e85344cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.620+0000 7fbf98ffc640 1 --2- 172.21.15.38:0/1132618234 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x558e84f4bf60 0x558e85344cb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=823ea271b33d5a73 server_cookie=2798c0c19ccb6859 in_seq=0 out_seq=0 2024-05-21T15:38:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.620+0000 7fbf7cff9640 10 client.26156 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:38:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.622+0000 7fbf7cff9640 1 -- 172.21.15.38:0/1132618234 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558e8529f370 con 0x558e84f4bf60 2024-05-21T15:38:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.622+0000 7fbf7cff9640 10 client.26156 handle_command_reply: tid=0 2024-05-21T15:38:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.622+0000 7fbfa29e4740 1 client.26156 shutdown 2024-05-21T15:38:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.622+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1132618234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f644d0 msgr2=0x558e853369b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.622+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1132618234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f644d0 0x558e853369b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00ddf0 tx=0x7fbf8c005620 comp rx=0 tx=0).stop 2024-05-21T15:38:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.622+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1132618234 shutdown_connections 2024-05-21T15:38:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.622+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1132618234 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x558e84f4bf60 0x558e85344cb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.622+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1132618234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 0x558e8533de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.622+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1132618234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f6c6b0 0x558e85336ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.622+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1132618234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f644d0 0x558e853369b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.622+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1132618234 >> 172.21.15.38:0/1132618234 conn(0x558e84f814e0 msgr2=0x558e85327570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.622+0000 7fbf7cff9640 0 client.26156 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:38:34.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.622+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1132618234 shutdown_connections 2024-05-21T15:38:34.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.622+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1132618234 wait complete. 2024-05-21T15:38:34.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.623+0000 7fbfa29e4740 20 client.26156 trim_cache size 0 max 16384 2024-05-21T15:38:34.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.646+0000 7fbfa29e4740 1 Processor -- start 2024-05-21T15:38:34.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.646+0000 7fbfa29e4740 1 -- start start 2024-05-21T15:38:34.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.646+0000 7fbfa29e4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 0x558e85336fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.647+0000 7fbfa29e4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f6c6b0 0x558e85335660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.647+0000 7fbfa29e4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 0x558e8532fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.647+0000 7fbfa29e4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941ac590 con 0x558e84f8d390 2024-05-21T15:38:34.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.647+0000 7fbfa29e4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941ae1c0 con 0x558e84f6c6b0 2024-05-21T15:38:34.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.647+0000 7fbfa29e4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941acc60 con 0x558e84f644d0 2024-05-21T15:38:34.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.647+0000 7fbf98ffc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f6c6b0 0x558e85335660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.647+0000 7fbf98ffc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f6c6b0 0x558e85335660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:49724/0 (socket says 172.21.15.38:49724) 2024-05-21T15:38:34.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.647+0000 7fbf98ffc640 1 -- 172.21.15.38:0/1868169516 learned_addr learned my addr 172.21.15.38:0/1868169516 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.647+0000 7fbf997fd640 1 --2- 172.21.15.38:0/1868169516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 0x558e85336fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.647+0000 7fbf99ffe640 1 --2- 172.21.15.38:0/1868169516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 0x558e8532fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.647+0000 7fbf98ffc640 1 -- 172.21.15.38:0/1868169516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 msgr2=0x558e85336fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.647+0000 7fbf98ffc640 1 --2- 172.21.15.38:0/1868169516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 0x558e85336fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.648+0000 7fbf98ffc640 1 -- 172.21.15.38:0/1868169516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 msgr2=0x558e8532fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.648+0000 7fbf98ffc640 1 --2- 172.21.15.38:0/1868169516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 0x558e8532fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.648+0000 7fbf98ffc640 1 -- 172.21.15.38:0/1868169516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e85323d20 con 0x558e84f6c6b0 2024-05-21T15:38:34.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.649+0000 7fbf98ffc640 1 --2- 172.21.15.38:0/1868169516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f6c6b0 0x558e85335660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbf8409b7e0 tx=0x7fbf840ccb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66ef7073c7c34db7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.650+0000 7fbf7effd640 1 -- 172.21.15.38:0/1868169516 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf840b8030 con 0x558e84f6c6b0 2024-05-21T15:38:34.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.650+0000 7fbf7effd640 1 -- 172.21.15.38:0/1868169516 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf84099b80 con 0x558e84f6c6b0 2024-05-21T15:38:34.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.650+0000 7fbf7effd640 1 -- 172.21.15.38:0/1868169516 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf840a4230 con 0x558e84f6c6b0 2024-05-21T15:38:34.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.651+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1868169516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f6c6b0 msgr2=0x558e85335660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.651+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1868169516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f6c6b0 0x558e85335660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbf8409b7e0 tx=0x7fbf840ccb40 comp rx=0 tx=0).stop 2024-05-21T15:38:34.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.652+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1868169516 shutdown_connections 2024-05-21T15:38:34.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.652+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1868169516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f8d390 0x558e8532fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.652+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1868169516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f6c6b0 0x558e85335660 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.652+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1868169516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 0x558e85336fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.652+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1868169516 >> 172.21.15.38:0/1868169516 conn(0x558e84f814e0 msgr2=0x558e85334f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.652+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1868169516 shutdown_connections 2024-05-21T15:38:34.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.652+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1868169516 wait complete. 2024-05-21T15:38:34.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.652+0000 7fbfa29e4740 1 -- start start 2024-05-21T15:38:34.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.652+0000 7fbfa29e4740 1 Processor -- start 2024-05-21T15:38:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.653+0000 7fbfa29e4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 0x558e85339700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.653+0000 7fbfa29e4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f6c6b0 0x558e85339c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.653+0000 7fbfa29e4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f8d390 0x558e8533cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.653+0000 7fbfa29e4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941ada30 con 0x558e84f6c6b0 2024-05-21T15:38:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.653+0000 7fbfa29e4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941ad330 con 0x558e84f8d390 2024-05-21T15:38:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.653+0000 7fbfa29e4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf941aa630 con 0x558e84f644d0 2024-05-21T15:38:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.653+0000 7fbfa29e4740 10 client.0 _ll_register_callbacks cb 0x7fbf8c009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.653+0000 7fbf99ffe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f8d390 0x558e8533cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.653+0000 7fbf98ffc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f6c6b0 0x558e85339c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.653+0000 7fbf99ffe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f8d390 0x558e8533cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:49730/0 (socket says 172.21.15.38:49730) 2024-05-21T15:38:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.653+0000 7fbf99ffe640 1 -- 172.21.15.38:0/1541977988 learned_addr learned my addr 172.21.15.38:0/1541977988 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.654+0000 7fbf98ffc640 1 -- 172.21.15.38:0/1541977988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 msgr2=0x558e85339700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.654+0000 7fbf98ffc640 1 --2- 172.21.15.38:0/1541977988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 0x558e85339700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.654+0000 7fbf98ffc640 1 -- 172.21.15.38:0/1541977988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f8d390 msgr2=0x558e8533cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.654+0000 7fbf98ffc640 1 --2- 172.21.15.38:0/1541977988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f8d390 0x558e8533cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.654+0000 7fbf98ffc640 1 -- 172.21.15.38:0/1541977988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e8533d140 con 0x558e84f6c6b0 2024-05-21T15:38:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.654+0000 7fbf98ffc640 1 --2- 172.21.15.38:0/1541977988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f6c6b0 0x558e85339c40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbf840984b0 tx=0x7fbf840a0010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32223bf282932252 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.654+0000 7fbf7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.655+0000 7fbf7cff9640 1 -- 172.21.15.38:0/1541977988 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf840a8070 con 0x558e84f6c6b0 2024-05-21T15:38:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.655+0000 7fbf7cff9640 1 -- 172.21.15.38:0/1541977988 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbf840a4430 con 0x558e84f6c6b0 2024-05-21T15:38:34.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.655+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1541977988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf84099b80 con 0x558e84f6c6b0 2024-05-21T15:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.655+0000 7fbf7cff9640 1 -- 172.21.15.38:0/1541977988 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbf840b8070 con 0x558e84f6c6b0 2024-05-21T15:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.655+0000 7fbfa29e4740 10 client.16500 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.655+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1541977988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558e853aba80 con 0x558e84f6c6b0 2024-05-21T15:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.656+0000 7fbf7cff9640 1 -- 172.21.15.38:0/1541977988 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf84099b80 con 0x558e84f6c6b0 2024-05-21T15:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.656+0000 7fbf7cff9640 1 -- 172.21.15.38:0/1541977988 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fbf84097830 con 0x558e84f6c6b0 2024-05-21T15:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.656+0000 7fbfa29e4740 10 client.16500 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.656+0000 7fbfa29e4740 10 client.16500 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.656+0000 7fbfa29e4740 20 client.16500 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.656+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1541977988 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x558e84f213a0 0x558e853ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.656+0000 7fbfa29e4740 4 client.16500 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.656+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1541977988 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e852996c0 con 0x558e84f213a0 2024-05-21T15:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.657+0000 7fbf997fd640 1 --2- 172.21.15.38:0/1541977988 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x558e84f213a0 0x558e853ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.658+0000 7fbf997fd640 1 --2- 172.21.15.38:0/1541977988 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x558e84f213a0 0x558e853ae3b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fcd54818b98f77b5 server_cookie=caf2f545fdc118e6 in_seq=0 out_seq=0 2024-05-21T15:38:34.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.658+0000 7fbf7cff9640 10 client.16500 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:38:34.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.658+0000 7fbf7cff9640 1 -- 172.21.15.38:0/1541977988 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e852996c0 con 0x558e84f213a0 2024-05-21T15:38:34.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.658+0000 7fbf7cff9640 10 client.16500 handle_command_reply: tid=0 2024-05-21T15:38:34.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.658+0000 7fbfa29e4740 1 client.16500 shutdown 2024-05-21T15:38:34.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.659+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1541977988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f6c6b0 msgr2=0x558e85339c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.659+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1541977988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f6c6b0 0x558e85339c40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbf840984b0 tx=0x7fbf840a0010 comp rx=0 tx=0).stop 2024-05-21T15:38:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.659+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1541977988 shutdown_connections 2024-05-21T15:38:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.659+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1541977988 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x558e84f213a0 0x558e853ae3b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.659+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1541977988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e84f8d390 0x558e8533cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.659+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1541977988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558e84f6c6b0 0x558e85339c40 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.659+0000 7fbfa29e4740 1 --2- 172.21.15.38:0/1541977988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e84f644d0 0x558e85339700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.659+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1541977988 >> 172.21.15.38:0/1541977988 conn(0x558e84f814e0 msgr2=0x558e8532fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.659+0000 7fbf7cff9640 0 client.16500 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:38:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.659+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1541977988 shutdown_connections 2024-05-21T15:38:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.659+0000 7fbfa29e4740 1 -- 172.21.15.38:0/1541977988 wait complete. 2024-05-21T15:38:34.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.659+0000 7fbfa29e4740 20 client.16500 trim_cache size 0 max 16384 2024-05-21T15:38:34.660 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:38:34.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.660+0000 7fbf9ba61640 1 -- 172.21.15.38:0/219758516 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf800a2b20 msgr2=0x7fbf800a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.660+0000 7fbf9ba61640 1 --2- 172.21.15.38:0/219758516 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf800a2b20 0x7fbf800a5030 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x558e85337a20 tx=0x7fbf900315b0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.660+0000 7fbf9ba61640 1 -- 172.21.15.38:0/219758516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf94102bc0 msgr2=0x7fbf9419b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.660+0000 7fbf9ba61640 1 --2- 172.21.15.38:0/219758516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf94102bc0 0x7fbf9419b9d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400c970 tx=0x7fbf8400ce30 comp rx=0 tx=0).stop 2024-05-21T15:38:34.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.660+0000 7fbf9ba61640 1 -- 172.21.15.38:0/219758516 shutdown_connections 2024-05-21T15:38:34.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.660+0000 7fbf9ba61640 1 --2- 172.21.15.38:0/219758516 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbf800a2b20 0x7fbf800a5030 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.660+0000 7fbf9ba61640 1 --2- 172.21.15.38:0/219758516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf941078c0 0x7fbf941a2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.660+0000 7fbf9ba61640 1 --2- 172.21.15.38:0/219758516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf94102bc0 0x7fbf9419b9d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.660+0000 7fbf9ba61640 1 --2- 172.21.15.38:0/219758516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf94101980 0x7fbf9419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.660+0000 7fbf9ba61640 1 -- 172.21.15.38:0/219758516 >> 172.21.15.38:0/219758516 conn(0x7fbf940fcb70 msgr2=0x7fbf941038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.660+0000 7fbf9ba61640 1 -- 172.21.15.38:0/219758516 shutdown_connections 2024-05-21T15:38:34.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.660+0000 7fbf9ba61640 1 -- 172.21.15.38:0/219758516 wait complete. 2024-05-21T15:38:34.675 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:34 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3176480546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:38:34.677 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:38:34.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:34 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3176480546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:38:34.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.928+0000 7ff399e13640 1 -- 172.21.15.38:0/2739651458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff39410efe0 msgr2=0x7ff39410f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.928+0000 7ff399e13640 1 --2- 172.21.15.38:0/2739651458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff39410efe0 0x7ff39410f3f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009860 tx=0x7ff38c02a360 comp rx=0 tx=0).stop 2024-05-21T15:38:34.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:34 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3176480546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:38:34.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.931+0000 7ff399e13640 1 -- 172.21.15.38:0/2739651458 shutdown_connections 2024-05-21T15:38:34.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.931+0000 7ff399e13640 1 --2- 172.21.15.38:0/2739651458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff39410efe0 0x7ff39410f3f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.931+0000 7ff399e13640 1 --2- 172.21.15.38:0/2739651458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff39410a2c0 0x7ff39410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.931+0000 7ff399e13640 1 --2- 172.21.15.38:0/2739651458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3941093d0 0x7ff394109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.931+0000 7ff399e13640 1 -- 172.21.15.38:0/2739651458 >> 172.21.15.38:0/2739651458 conn(0x7ff39406f4a0 msgr2=0x7ff394071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.934+0000 7ff399e13640 1 -- 172.21.15.38:0/2739651458 shutdown_connections 2024-05-21T15:38:34.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.935+0000 7ff399e13640 1 -- 172.21.15.38:0/2739651458 wait complete. 2024-05-21T15:38:34.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.936+0000 7ff399e13640 1 Processor -- start 2024-05-21T15:38:34.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.936+0000 7ff399e13640 1 -- start start 2024-05-21T15:38:34.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.936+0000 7ff399e13640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3941093d0 0x7ff394115f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.936+0000 7ff399e13640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff39410a2c0 0x7ff3941164c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.936+0000 7ff399e13640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff39410efe0 0x7ff394117460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.936+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3941093d0 0x7ff394115f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.936+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3941093d0 0x7ff394115f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:48828/0 (socket says 172.21.15.38:48828) 2024-05-21T15:38:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.936+0000 7ff3937fe640 1 -- 172.21.15.38:0/2390088959 learned_addr learned my addr 172.21.15.38:0/2390088959 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.936+0000 7ff399e13640 1 -- 172.21.15.38:0/2390088959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff39411d5f0 con 0x7ff39410efe0 2024-05-21T15:38:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.936+0000 7ff399e13640 1 -- 172.21.15.38:0/2390088959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff39411d470 con 0x7ff39410a2c0 2024-05-21T15:38:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.936+0000 7ff399e13640 1 -- 172.21.15.38:0/2390088959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff39411d770 con 0x7ff3941093d0 2024-05-21T15:38:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.937+0000 7ff38affd640 1 --2- 172.21.15.38:0/2390088959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff39410a2c0 0x7ff3941164c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.937+0000 7ff3937fe640 1 -- 172.21.15.38:0/2390088959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff39410a2c0 msgr2=0x7ff3941164c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.937+0000 7ff3937fe640 1 --2- 172.21.15.38:0/2390088959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff39410a2c0 0x7ff3941164c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.937+0000 7ff3937fe640 1 -- 172.21.15.38:0/2390088959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff39410efe0 msgr2=0x7ff394117460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.937+0000 7ff3937fe640 1 --2- 172.21.15.38:0/2390088959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff39410efe0 0x7ff394117460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.937+0000 7ff3937fe640 1 -- 172.21.15.38:0/2390088959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff394117ad0 con 0x7ff3941093d0 2024-05-21T15:38:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.938+0000 7ff3937fe640 1 --2- 172.21.15.38:0/2390088959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3941093d0 0x7ff394115f40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00b450 tx=0x7ff37c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.939+0000 7ff3917fa640 1 -- 172.21.15.38:0/2390088959 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c015020 con 0x7ff3941093d0 2024-05-21T15:38:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.940+0000 7ff3917fa640 1 -- 172.21.15.38:0/2390088959 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37c011d70 con 0x7ff3941093d0 2024-05-21T15:38:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.940+0000 7ff399e13640 1 -- 172.21.15.38:0/2390088959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff394117dc0 con 0x7ff3941093d0 2024-05-21T15:38:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.940+0000 7ff399e13640 1 -- 172.21.15.38:0/2390088959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff394118330 con 0x7ff3941093d0 2024-05-21T15:38:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.941+0000 7ff3917fa640 1 -- 172.21.15.38:0/2390088959 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c00f340 con 0x7ff3941093d0 2024-05-21T15:38:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.944+0000 7ff3a0d8b740 1 Processor -- start 2024-05-21T15:38:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.946+0000 7ff3a0d8b740 1 -- start start 2024-05-21T15:38:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.947+0000 7ff3a0d8b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f28574390 0x557f28913b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.947+0000 7ff3a0d8b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2854b4d0 0x557f28914080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.947+0000 7ff3a0d8b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f285536b0 0x557f2891af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.947+0000 7ff3a0d8b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557f28878c40 con 0x557f28574390 2024-05-21T15:38:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.947+0000 7ff3a0d8b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557f28820580 con 0x557f2854b4d0 2024-05-21T15:38:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.947+0000 7ff3a0d8b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557f28874c00 con 0x557f285536b0 2024-05-21T15:38:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.947+0000 7ff3917fa640 1 -- 172.21.15.38:0/2390088959 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff37c018020 con 0x7ff3941093d0 2024-05-21T15:38:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.947+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2854b4d0 0x557f28914080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.947+0000 7ff393fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f28574390 0x557f28913b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.947+0000 7ff393fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f28574390 0x557f28913b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35536/0 (socket says 172.21.15.38:35536) 2024-05-21T15:38:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.947+0000 7ff393fff640 1 -- 172.21.15.38:0/2197539579 learned_addr learned my addr 172.21.15.38:0/2197539579 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.948+0000 7ff38affd640 1 --2- 172.21.15.38:0/2197539579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f285536b0 0x557f2891af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.948+0000 7ff3917fa640 1 --2- 172.21.15.38:0/2390088959 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3800a2d40 0x7ff3800a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.949+0000 7ff3917fa640 1 -- 172.21.15.38:0/2390088959 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff37c0cebb0 con 0x7ff3941093d0 2024-05-21T15:38:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.949+0000 7ff38affd640 1 -- 172.21.15.38:0/2197539579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2854b4d0 msgr2=0x557f28914080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.949+0000 7ff38affd640 1 --2- 172.21.15.38:0/2197539579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2854b4d0 0x557f28914080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.949+0000 7ff38affd640 1 -- 172.21.15.38:0/2197539579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f28574390 msgr2=0x557f28913b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.949+0000 7ff38affd640 1 --2- 172.21.15.38:0/2197539579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f28574390 0x557f28913b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.949+0000 7ff38affd640 1 -- 172.21.15.38:0/2197539579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f2858c3f0 con 0x557f285536b0 2024-05-21T15:38:34.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.949+0000 7ff3937fe640 1 --2- 172.21.15.38:0/2197539579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2854b4d0 0x557f28914080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:38:34.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.950+0000 7ff38affd640 1 --2- 172.21.15.38:0/2197539579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f285536b0 0x557f2891af80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff384005fd0 tx=0x7ff38400f890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56b2e12098ea146c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.951+0000 7ff393fff640 1 --2- 172.21.15.38:0/2390088959 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3800a2d40 0x7ff3800a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.954+0000 7ff389ffb640 1 -- 172.21.15.38:0/2197539579 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38401d070 con 0x557f285536b0 2024-05-21T15:38:34.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.954+0000 7ff389ffb640 1 -- 172.21.15.38:0/2197539579 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38400cb20 con 0x557f285536b0 2024-05-21T15:38:34.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.954+0000 7ff389ffb640 1 -- 172.21.15.38:0/2197539579 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384018c00 con 0x557f285536b0 2024-05-21T15:38:34.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.956+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/2197539579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f285536b0 msgr2=0x557f2891af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.956+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/2197539579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f285536b0 0x557f2891af80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff384005fd0 tx=0x7ff38400f890 comp rx=0 tx=0).stop 2024-05-21T15:38:34.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.956+0000 7ff393fff640 1 --2- 172.21.15.38:0/2390088959 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3800a2d40 0x7ff3800a5250 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0057c0 tx=0x7ff38c02c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.960+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/2197539579 shutdown_connections 2024-05-21T15:38:34.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.960+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/2197539579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f285536b0 0x557f2891af80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.960+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/2197539579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f2854b4d0 0x557f28914080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.960+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/2197539579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f28574390 0x557f28913b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.960+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/2197539579 >> 172.21.15.38:0/2197539579 conn(0x557f285684e0 msgr2=0x557f2890f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.964+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/2197539579 shutdown_connections 2024-05-21T15:38:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.964+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/2197539579 wait complete. 2024-05-21T15:38:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.964+0000 7ff3a0d8b740 1 -- start start 2024-05-21T15:38:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.965+0000 7ff3a0d8b740 1 Processor -- start 2024-05-21T15:38:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.965+0000 7ff3a0d8b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2854b4d0 0x557f2891d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.965+0000 7ff3a0d8b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f285536b0 0x557f2891df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.965+0000 7ff3a0d8b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f28574390 0x557f28924e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.965+0000 7ff3a0d8b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557f2891b9d0 con 0x557f2854b4d0 2024-05-21T15:38:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.965+0000 7ff3a0d8b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557f28833c40 con 0x557f28574390 2024-05-21T15:38:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.965+0000 7ff3a0d8b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557f2891bb50 con 0x557f285536b0 2024-05-21T15:38:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.965+0000 7ff3a0d8b740 10 client.0 _ll_register_callbacks cb 0x557f2858bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.966+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2854b4d0 0x557f2891d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.966+0000 7ff393fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f28574390 0x557f28924e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.966+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2854b4d0 0x557f2891d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35552/0 (socket says 172.21.15.38:35552) 2024-05-21T15:38:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.966+0000 7ff3937fe640 1 -- 172.21.15.38:0/724551149 learned_addr learned my addr 172.21.15.38:0/724551149 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.966+0000 7ff38affd640 1 --2- 172.21.15.38:0/724551149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f285536b0 0x557f2891df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.968+0000 7ff393fff640 1 -- 172.21.15.38:0/724551149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f285536b0 msgr2=0x557f2891df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.968+0000 7ff393fff640 1 --2- 172.21.15.38:0/724551149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f285536b0 0x557f2891df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.968+0000 7ff393fff640 1 -- 172.21.15.38:0/724551149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2854b4d0 msgr2=0x557f2891d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:34.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.968+0000 7ff393fff640 1 --2- 172.21.15.38:0/724551149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2854b4d0 0x557f2891d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.968+0000 7ff393fff640 1 -- 172.21.15.38:0/724551149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f286567d0 con 0x557f28574390 2024-05-21T15:38:34.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.969+0000 7ff393fff640 1 --2- 172.21.15.38:0/724551149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f28574390 0x557f28924e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff38c030650 tx=0x7ff38c030520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a3d2d51e3e27975 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:34.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.969+0000 7ff372ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:38:34.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.970+0000 7ff372ffd640 1 -- 172.21.15.38:0/724551149 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c065050 con 0x557f28574390 2024-05-21T15:38:34.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.970+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/724551149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff38400cb20 con 0x557f28574390 2024-05-21T15:38:34.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.970+0000 7ff372ffd640 1 -- 172.21.15.38:0/724551149 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff38c003ac0 con 0x557f28574390 2024-05-21T15:38:34.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.970+0000 7ff372ffd640 1 -- 172.21.15.38:0/724551149 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff38c003d00 con 0x557f28574390 2024-05-21T15:38:34.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.970+0000 7ff3a0d8b740 10 client.26171 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:34.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.970+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/724551149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557f28928100 con 0x557f28574390 2024-05-21T15:38:34.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.971+0000 7ff372ffd640 1 -- 172.21.15.38:0/724551149 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38400cb20 con 0x557f28574390 2024-05-21T15:38:34.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.971+0000 7ff372ffd640 1 -- 172.21.15.38:0/724551149 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7ff38c05e070 con 0x557f28574390 2024-05-21T15:38:34.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.971+0000 7ff3a0d8b740 10 client.26171 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:34.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.971+0000 7ff3a0d8b740 10 client.26171 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:38:34.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.971+0000 7ff3a0d8b740 20 client.26171 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:34.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.971+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/724551149 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x557f28532f60 0x557f2892bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:34.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.971+0000 7ff3a0d8b740 4 client.26171 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:38:34.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.971+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/724551149 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f28886370 con 0x557f28532f60 2024-05-21T15:38:34.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.972+0000 7ff3937fe640 1 --2- 172.21.15.38:0/724551149 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x557f28532f60 0x557f2892bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:34.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.974+0000 7ff3937fe640 1 --2- 172.21.15.38:0/724551149 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x557f28532f60 0x557f2892bce0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=919ea1df44b28a05 server_cookie=94d5b629620048b5 in_seq=0 out_seq=0 2024-05-21T15:38:34.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.974+0000 7ff372ffd640 10 client.26171 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:38:34.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.976+0000 7ff372ffd640 1 -- 172.21.15.38:0/724551149 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f28886370 con 0x557f28532f60 2024-05-21T15:38:34.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.976+0000 7ff372ffd640 10 client.26171 handle_command_reply: tid=0 2024-05-21T15:38:34.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.976+0000 7ff3a0d8b740 1 client.26171 shutdown 2024-05-21T15:38:34.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.978+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/724551149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f28574390 msgr2=0x557f28924e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:34.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.978+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/724551149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f28574390 0x557f28924e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff38c030650 tx=0x7ff38c030520 comp rx=0 tx=0).stop 2024-05-21T15:38:34.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.979+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/724551149 shutdown_connections 2024-05-21T15:38:34.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.979+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/724551149 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x557f28532f60 0x557f2892bce0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.979+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/724551149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f28574390 0x557f28924e60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.979+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/724551149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f285536b0 0x557f2891df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.979+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/724551149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2854b4d0 0x557f2891d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:34.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.979+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/724551149 >> 172.21.15.38:0/724551149 conn(0x557f285684e0 msgr2=0x557f2890e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:34.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.979+0000 7ff372ffd640 0 client.26171 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:38:34.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.987+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/724551149 shutdown_connections 2024-05-21T15:38:34.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.987+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/724551149 wait complete. 2024-05-21T15:38:34.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:34.988+0000 7ff3a0d8b740 20 client.26171 trim_cache size 0 max 16384 2024-05-21T15:38:35.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.019+0000 7ff3a0d8b740 1 Processor -- start 2024-05-21T15:38:35.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.019+0000 7ff3a0d8b740 1 -- start start 2024-05-21T15:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.020+0000 7ff3a0d8b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f2854b4d0 0x557f2892be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.020+0000 7ff3a0d8b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f285536b0 0x557f28924d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.020+0000 7ff3a0d8b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f28574390 0x557f28917de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.020+0000 7ff3a0d8b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941b0c10 con 0x557f285536b0 2024-05-21T15:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.020+0000 7ff3a0d8b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941b2840 con 0x557f28574390 2024-05-21T15:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.020+0000 7ff3a0d8b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941b12e0 con 0x557f2854b4d0 2024-05-21T15:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.020+0000 7ff393fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f28574390 0x557f28917de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.020+0000 7ff393fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f28574390 0x557f28917de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45998/0 (socket says 172.21.15.38:45998) 2024-05-21T15:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.020+0000 7ff393fff640 1 -- 172.21.15.38:0/3455086273 learned_addr learned my addr 172.21.15.38:0/3455086273 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.021+0000 7ff38affd640 1 --2- 172.21.15.38:0/3455086273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f285536b0 0x557f28924d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.021+0000 7ff393fff640 1 -- 172.21.15.38:0/3455086273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f2854b4d0 msgr2=0x557f2892be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.021+0000 7ff393fff640 1 --2- 172.21.15.38:0/3455086273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f2854b4d0 0x557f2892be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.021+0000 7ff393fff640 1 -- 172.21.15.38:0/3455086273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f285536b0 msgr2=0x557f28924d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.021+0000 7ff393fff640 1 --2- 172.21.15.38:0/3455086273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f285536b0 0x557f28924d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.021+0000 7ff393fff640 1 -- 172.21.15.38:0/3455086273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f2890ad20 con 0x557f28574390 2024-05-21T15:38:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.021+0000 7ff38affd640 1 --2- 172.21.15.38:0/3455086273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f285536b0 0x557f28924d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:38:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.022+0000 7ff393fff640 1 --2- 172.21.15.38:0/3455086273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f28574390 0x557f28917de0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff38c005eb0 tx=0x7ff38c009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=242cde356aae02b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:35.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.024+0000 7ff3897fa640 1 -- 172.21.15.38:0/3455086273 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c082070 con 0x557f28574390 2024-05-21T15:38:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.024+0000 7ff3897fa640 1 -- 172.21.15.38:0/3455086273 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38c063030 con 0x557f28574390 2024-05-21T15:38:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.025+0000 7ff3897fa640 1 -- 172.21.15.38:0/3455086273 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c065070 con 0x557f28574390 2024-05-21T15:38:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.025+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/3455086273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f28574390 msgr2=0x557f28917de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.025+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/3455086273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f28574390 0x557f28917de0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff38c005eb0 tx=0x7ff38c009550 comp rx=0 tx=0).stop 2024-05-21T15:38:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.025+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/3455086273 shutdown_connections 2024-05-21T15:38:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.025+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/3455086273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f28574390 0x557f28917de0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.025+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/3455086273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f285536b0 0x557f28924d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.025+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/3455086273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f2854b4d0 0x557f2892be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.025+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/3455086273 >> 172.21.15.38:0/3455086273 conn(0x557f285684e0 msgr2=0x557f28916e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.026+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/3455086273 shutdown_connections 2024-05-21T15:38:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.026+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/3455086273 wait complete. 2024-05-21T15:38:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.026+0000 7ff3a0d8b740 1 -- start start 2024-05-21T15:38:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.026+0000 7ff3a0d8b740 1 Processor -- start 2024-05-21T15:38:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.027+0000 7ff3a0d8b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2854b4d0 0x557f289246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.027+0000 7ff3a0d8b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f285536b0 0x557f28924c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.027+0000 7ff3a0d8b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f28574390 0x557f28921320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.027+0000 7ff3a0d8b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941b20b0 con 0x557f2854b4d0 2024-05-21T15:38:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.027+0000 7ff3a0d8b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941b19b0 con 0x557f285536b0 2024-05-21T15:38:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.027+0000 7ff3a0d8b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941aebf0 con 0x557f28574390 2024-05-21T15:38:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.027+0000 7ff3a0d8b740 10 client.0 _ll_register_callbacks cb 0x7ff384009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:35.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.027+0000 7ff38affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f285536b0 0x557f28924c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.027+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2854b4d0 0x557f289246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.027+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2854b4d0 0x557f289246c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35568/0 (socket says 172.21.15.38:35568) 2024-05-21T15:38:35.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.027+0000 7ff3937fe640 1 -- 172.21.15.38:0/4173263005 learned_addr learned my addr 172.21.15.38:0/4173263005 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:35.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.027+0000 7ff393fff640 1 --2- 172.21.15.38:0/4173263005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f28574390 0x557f28921320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.028+0000 7ff393fff640 1 -- 172.21.15.38:0/4173263005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f285536b0 msgr2=0x557f28924c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.028+0000 7ff393fff640 1 --2- 172.21.15.38:0/4173263005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f285536b0 0x557f28924c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.028+0000 7ff393fff640 1 -- 172.21.15.38:0/4173263005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2854b4d0 msgr2=0x557f289246c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.028+0000 7ff393fff640 1 --2- 172.21.15.38:0/4173263005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2854b4d0 0x557f289246c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.028+0000 7ff393fff640 1 -- 172.21.15.38:0/4173263005 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f28921a30 con 0x557f28574390 2024-05-21T15:38:35.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.028+0000 7ff393fff640 1 --2- 172.21.15.38:0/4173263005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f28574390 0x557f28921320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff38c02c010 tx=0x7ff38c02a860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d7756b615e0e152 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:35.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.028+0000 7ff372ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:38:35.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.030+0000 7ff372ffd640 1 -- 172.21.15.38:0/4173263005 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c065070 con 0x557f28574390 2024-05-21T15:38:35.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.030+0000 7ff372ffd640 1 -- 172.21.15.38:0/4173263005 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff38c06ab10 con 0x557f28574390 2024-05-21T15:38:35.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.031+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/4173263005 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff38c063030 con 0x557f28574390 2024-05-21T15:38:35.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.034+0000 7ff372ffd640 1 -- 172.21.15.38:0/4173263005 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff38c082070 con 0x557f28574390 2024-05-21T15:38:35.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.034+0000 7ff3a0d8b740 10 client.26239 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:35.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.034+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/4173263005 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557f28992c10 con 0x557f28574390 2024-05-21T15:38:35.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.036+0000 7ff372ffd640 1 -- 172.21.15.38:0/4173263005 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38c063030 con 0x557f28574390 2024-05-21T15:38:35.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.038+0000 7ff372ffd640 1 -- 172.21.15.38:0/4173263005 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7ff38c05e070 con 0x557f28574390 2024-05-21T15:38:35.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.038+0000 7ff3a0d8b740 10 client.26239 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:35.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.038+0000 7ff3a0d8b740 10 client.26239 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:38:35.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.038+0000 7ff3a0d8b740 20 client.26239 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:35.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.038+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/4173263005 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x557f285083a0 0x557f289954c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.038+0000 7ff3a0d8b740 4 client.26239 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:38:35.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.038+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/4173263005 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f288806c0 con 0x557f285083a0 2024-05-21T15:38:35.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.040+0000 7ff3937fe640 1 --2- 172.21.15.38:0/4173263005 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x557f285083a0 0x557f289954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.043+0000 7ff3937fe640 1 --2- 172.21.15.38:0/4173263005 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x557f285083a0 0x557f289954c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8fe471057e16b57 server_cookie=434026f14479e03e in_seq=0 out_seq=0 2024-05-21T15:38:35.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.043+0000 7ff372ffd640 10 client.26239 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:38:35.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.043+0000 7ff372ffd640 1 -- 172.21.15.38:0/4173263005 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f288806c0 con 0x557f285083a0 2024-05-21T15:38:35.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.044+0000 7ff372ffd640 10 client.26239 handle_command_reply: tid=0 2024-05-21T15:38:35.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.044+0000 7ff3a0d8b740 1 client.26239 shutdown 2024-05-21T15:38:35.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.048+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/4173263005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f28574390 msgr2=0x557f28921320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.048+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/4173263005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f28574390 0x557f28921320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff38c02c010 tx=0x7ff38c02a860 comp rx=0 tx=0).stop 2024-05-21T15:38:35.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.048+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/4173263005 shutdown_connections 2024-05-21T15:38:35.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.048+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/4173263005 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x557f285083a0 0x557f289954c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.048+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/4173263005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f28574390 0x557f28921320 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.048+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/4173263005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f285536b0 0x557f28924c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.048+0000 7ff3a0d8b740 1 --2- 172.21.15.38:0/4173263005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2854b4d0 0x557f289246c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.048+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/4173263005 >> 172.21.15.38:0/4173263005 conn(0x557f285684e0 msgr2=0x557f28917de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.048+0000 7ff372ffd640 0 client.26239 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:38:35.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.052+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/4173263005 shutdown_connections 2024-05-21T15:38:35.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.053+0000 7ff3a0d8b740 1 -- 172.21.15.38:0/4173263005 wait complete. 2024-05-21T15:38:35.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.054+0000 7ff3a0d8b740 20 client.26239 trim_cache size 0 max 16384 2024-05-21T15:38:35.057 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:38:35.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.062+0000 7ff399e13640 1 -- 172.21.15.38:0/2390088959 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3800a2d40 msgr2=0x7ff3800a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.062+0000 7ff399e13640 1 --2- 172.21.15.38:0/2390088959 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3800a2d40 0x7ff3800a5250 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0057c0 tx=0x7ff38c02c040 comp rx=0 tx=0).stop 2024-05-21T15:38:35.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.062+0000 7ff399e13640 1 -- 172.21.15.38:0/2390088959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3941093d0 msgr2=0x7ff394115f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.062+0000 7ff399e13640 1 --2- 172.21.15.38:0/2390088959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3941093d0 0x7ff394115f40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00b450 tx=0x7ff37c00b910 comp rx=0 tx=0).stop 2024-05-21T15:38:35.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.062+0000 7ff399e13640 1 -- 172.21.15.38:0/2390088959 shutdown_connections 2024-05-21T15:38:35.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.062+0000 7ff399e13640 1 --2- 172.21.15.38:0/2390088959 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3800a2d40 0x7ff3800a5250 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.062+0000 7ff399e13640 1 --2- 172.21.15.38:0/2390088959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff39410efe0 0x7ff394117460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.062+0000 7ff399e13640 1 --2- 172.21.15.38:0/2390088959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff39410a2c0 0x7ff3941164c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.062+0000 7ff399e13640 1 --2- 172.21.15.38:0/2390088959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3941093d0 0x7ff394115f40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.062+0000 7ff399e13640 1 -- 172.21.15.38:0/2390088959 >> 172.21.15.38:0/2390088959 conn(0x7ff39406f4a0 msgr2=0x7ff39410afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.065+0000 7ff399e13640 1 -- 172.21.15.38:0/2390088959 shutdown_connections 2024-05-21T15:38:35.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.066+0000 7ff399e13640 1 -- 172.21.15.38:0/2390088959 wait complete. 2024-05-21T15:38:35.089 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:38:35.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.314+0000 7f850e5e3640 1 -- 172.21.15.38:0/1493382166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850810efe0 msgr2=0x7f850810f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.314+0000 7f850e5e3640 1 --2- 172.21.15.38:0/1493382166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850810efe0 0x7f850810f3f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8500009ca0 tx=0x7f850002a8f0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.316+0000 7f850e5e3640 1 -- 172.21.15.38:0/1493382166 shutdown_connections 2024-05-21T15:38:35.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.316+0000 7f850e5e3640 1 --2- 172.21.15.38:0/1493382166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850810efe0 0x7f850810f3f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.316+0000 7f850e5e3640 1 --2- 172.21.15.38:0/1493382166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f850810a2c0 0x7f850810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.316+0000 7f850e5e3640 1 --2- 172.21.15.38:0/1493382166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85081093d0 0x7f8508109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.316+0000 7f850e5e3640 1 -- 172.21.15.38:0/1493382166 >> 172.21.15.38:0/1493382166 conn(0x7f850806f4a0 msgr2=0x7f8508071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.321+0000 7f850e5e3640 1 -- 172.21.15.38:0/1493382166 shutdown_connections 2024-05-21T15:38:35.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.323+0000 7f850e5e3640 1 -- 172.21.15.38:0/1493382166 wait complete. 2024-05-21T15:38:35.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.324+0000 7f850e5e3640 1 Processor -- start 2024-05-21T15:38:35.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.325+0000 7f850e5e3640 1 -- start start 2024-05-21T15:38:35.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.325+0000 7f850e5e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85081093d0 0x7f8508116040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.325+0000 7f850e5e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850810a2c0 0x7f8508116580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.325+0000 7f850e5e3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850810efe0 0x7f85081174e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.325+0000 7f850e5e3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f850811d690 con 0x7f850810a2c0 2024-05-21T15:38:35.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.325+0000 7f850e5e3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f850811d510 con 0x7f85081093d0 2024-05-21T15:38:35.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.325+0000 7f850e5e3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f850811d810 con 0x7f850810efe0 2024-05-21T15:38:35.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.326+0000 7f850cb80640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850810efe0 0x7f85081174e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.326+0000 7f850cb80640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850810efe0 0x7f85081174e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:48880/0 (socket says 172.21.15.38:48880) 2024-05-21T15:38:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.326+0000 7f850cb80640 1 -- 172.21.15.38:0/2029675434 learned_addr learned my addr 172.21.15.38:0/2029675434 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:35.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.327+0000 7f850cb80640 1 -- 172.21.15.38:0/2029675434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85081093d0 msgr2=0x7f8508116040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.327+0000 7f850cb80640 1 --2- 172.21.15.38:0/2029675434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85081093d0 0x7f8508116040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.327+0000 7f850cb80640 1 -- 172.21.15.38:0/2029675434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850810a2c0 msgr2=0x7f8508116580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:38:35.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.328+0000 7f850cb80640 1 --2- 172.21.15.38:0/2029675434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850810a2c0 0x7f8508116580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.328+0000 7f850cb80640 1 -- 172.21.15.38:0/2029675434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8508117b50 con 0x7f850810efe0 2024-05-21T15:38:35.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.330+0000 7f850cb80640 1 --2- 172.21.15.38:0/2029675434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850810efe0 0x7f85081174e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8500033040 tx=0x7f8500003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.330+0000 7f85057fa640 1 -- 172.21.15.38:0/2029675434 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500003e40 con 0x7f850810efe0 2024-05-21T15:38:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.331+0000 7f850e5e3640 1 -- 172.21.15.38:0/2029675434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8508117de0 con 0x7f850810efe0 2024-05-21T15:38:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.331+0000 7f850e5e3640 1 -- 172.21.15.38:0/2029675434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8508118400 con 0x7f850810efe0 2024-05-21T15:38:35.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.331+0000 7f85057fa640 1 -- 172.21.15.38:0/2029675434 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f850002d070 con 0x7f850810efe0 2024-05-21T15:38:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.331+0000 7f85057fa640 1 -- 172.21.15.38:0/2029675434 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850002f9f0 con 0x7f850810efe0 2024-05-21T15:38:35.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.335+0000 7f8515564740 1 Processor -- start 2024-05-21T15:38:35.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.338+0000 7f85057fa640 1 -- 172.21.15.38:0/2029675434 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f850002fcb0 con 0x7f850810efe0 2024-05-21T15:38:35.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.338+0000 7f8515564740 1 -- start start 2024-05-21T15:38:35.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.340+0000 7f8515564740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 0x55b16acfdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.340+0000 7f8515564740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a9354d0 0x55b16acfe080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.340+0000 7f8515564740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a93d6b0 0x55b16ad04f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.340+0000 7f8515564740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b16ac62c40 con 0x55b16a93d6b0 2024-05-21T15:38:35.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.340+0000 7f8515564740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b16ac0a580 con 0x55b16a95e390 2024-05-21T15:38:35.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.340+0000 7f8515564740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b16ac5ec00 con 0x55b16a9354d0 2024-05-21T15:38:35.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.341+0000 7f8507fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a9354d0 0x55b16acfe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.342+0000 7f85057fa640 1 --2- 172.21.15.38:0/2029675434 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84f00a2ce0 0x7f84f00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.342+0000 7f85077fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 0x55b16acfdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.342+0000 7f85077fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 0x55b16acfdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46032/0 (socket says 172.21.15.38:46032) 2024-05-21T15:38:35.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.342+0000 7f85077fe640 1 -- 172.21.15.38:0/1642644262 learned_addr learned my addr 172.21.15.38:0/1642644262 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:35.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.343+0000 7f85057fa640 1 -- 172.21.15.38:0/2029675434 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f85000f0b00 con 0x7f850810efe0 2024-05-21T15:38:35.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.343+0000 7f850cb80640 1 --2- 172.21.15.38:0/2029675434 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84f00a2ce0 0x7f84f00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.344+0000 7f85077fe640 1 --2- 172.21.15.38:0/1642644262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a93d6b0 0x55b16ad04f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.347+0000 7f850cb80640 1 --2- 172.21.15.38:0/2029675434 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84f00a2ce0 0x7f84f00a51f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f85000094e0 tx=0x7f85000095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:35.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.348+0000 7f8507fff640 1 -- 172.21.15.38:0/1642644262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 msgr2=0x55b16acfdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.348+0000 7f8507fff640 1 --2- 172.21.15.38:0/1642644262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 0x55b16acfdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.348+0000 7f8507fff640 1 -- 172.21.15.38:0/1642644262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a93d6b0 msgr2=0x55b16ad04f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.348+0000 7f8507fff640 1 --2- 172.21.15.38:0/1642644262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a93d6b0 0x55b16ad04f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.348+0000 7f8507fff640 1 -- 172.21.15.38:0/1642644262 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b16a9763f0 con 0x55b16a9354d0 2024-05-21T15:38:35.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.349+0000 7f8507fff640 1 --2- 172.21.15.38:0/1642644262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a9354d0 0x55b16acfe080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009720 tx=0x7f84f8009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=287052502a9767e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.349+0000 7f85077fe640 1 --2- 172.21.15.38:0/1642644262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a93d6b0 0x55b16ad04f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:38:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.352+0000 7f84ee7fc640 1 -- 172.21.15.38:0/1642644262 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8015020 con 0x55b16a9354d0 2024-05-21T15:38:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.352+0000 7f84ee7fc640 1 -- 172.21.15.38:0/1642644262 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f8009e30 con 0x55b16a9354d0 2024-05-21T15:38:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.352+0000 7f84ee7fc640 1 -- 172.21.15.38:0/1642644262 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f80103a0 con 0x55b16a9354d0 2024-05-21T15:38:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.355+0000 7f8515564740 1 -- 172.21.15.38:0/1642644262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a9354d0 msgr2=0x55b16acfe080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.355+0000 7f8515564740 1 --2- 172.21.15.38:0/1642644262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a9354d0 0x55b16acfe080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009720 tx=0x7f84f8009be0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.356+0000 7f8515564740 1 -- 172.21.15.38:0/1642644262 shutdown_connections 2024-05-21T15:38:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.356+0000 7f8515564740 1 --2- 172.21.15.38:0/1642644262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a93d6b0 0x55b16ad04f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.356+0000 7f8515564740 1 --2- 172.21.15.38:0/1642644262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a9354d0 0x55b16acfe080 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009720 tx=0x7f84f8009be0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.356+0000 7f8515564740 1 --2- 172.21.15.38:0/1642644262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 0x55b16acfdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.356+0000 7f8515564740 1 -- 172.21.15.38:0/1642644262 >> 172.21.15.38:0/1642644262 conn(0x55b16a9524e0 msgr2=0x55b16acf9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.359+0000 7f8515564740 1 -- 172.21.15.38:0/1642644262 shutdown_connections 2024-05-21T15:38:35.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.360+0000 7f8515564740 1 -- 172.21.15.38:0/1642644262 wait complete. 2024-05-21T15:38:35.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.361+0000 7f8515564740 1 -- start start 2024-05-21T15:38:35.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.361+0000 7f8515564740 1 Processor -- start 2024-05-21T15:38:35.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.361+0000 7f8515564740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a93d6b0 0x55b16ad07b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.361+0000 7f8515564740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a95e390 0x55b16ad08050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.361+0000 7f8515564740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a91cf60 0x55b16ad0efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.361+0000 7f8515564740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b16ad059d0 con 0x55b16a95e390 2024-05-21T15:38:35.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.361+0000 7f8515564740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b16ac1dc40 con 0x55b16a91cf60 2024-05-21T15:38:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.362+0000 7f8515564740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b16ad05b50 con 0x55b16a93d6b0 2024-05-21T15:38:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.362+0000 7f8515564740 10 client.0 _ll_register_callbacks cb 0x55b16a975be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.362+0000 7f85077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a93d6b0 0x55b16ad07b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.362+0000 7f85077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a93d6b0 0x55b16ad07b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:48902/0 (socket says 172.21.15.38:48902) 2024-05-21T15:38:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.362+0000 7f85077fe640 1 -- 172.21.15.38:0/2757968916 learned_addr learned my addr 172.21.15.38:0/2757968916 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.362+0000 7f85077fe640 1 --2- 172.21.15.38:0/2757968916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a91cf60 0x55b16ad0efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.362+0000 7f8507fff640 1 --2- 172.21.15.38:0/2757968916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a95e390 0x55b16ad08050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.364+0000 7f8507fff640 1 -- 172.21.15.38:0/2757968916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a93d6b0 msgr2=0x55b16ad07b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.364+0000 7f8507fff640 1 --2- 172.21.15.38:0/2757968916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a93d6b0 0x55b16ad07b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.364+0000 7f8507fff640 1 -- 172.21.15.38:0/2757968916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a91cf60 msgr2=0x55b16ad0efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.364+0000 7f8507fff640 1 --2- 172.21.15.38:0/2757968916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a91cf60 0x55b16ad0efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.364+0000 7f8507fff640 1 -- 172.21.15.38:0/2757968916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b16aa407d0 con 0x55b16a95e390 2024-05-21T15:38:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.365+0000 7f8507fff640 1 --2- 172.21.15.38:0/2757968916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a95e390 0x55b16ad08050 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x55b16acfe930 tx=0x7f84f8011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09a05a59d6cb8cd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.366+0000 7f84e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.366+0000 7f84e77fe640 1 -- 172.21.15.38:0/2757968916 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8020070 con 0x55b16a95e390 2024-05-21T15:38:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.366+0000 7f84e77fe640 1 -- 172.21.15.38:0/2757968916 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f84f800bce0 con 0x55b16a95e390 2024-05-21T15:38:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.367+0000 7f8515564740 1 -- 172.21.15.38:0/2757968916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84f8009e30 con 0x55b16a95e390 2024-05-21T15:38:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.368+0000 7f84e77fe640 1 -- 172.21.15.38:0/2757968916 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f84f8015040 con 0x55b16a95e390 2024-05-21T15:38:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.371+0000 7f8515564740 10 client.16521 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.371+0000 7f8515564740 1 -- 172.21.15.38:0/2757968916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b16ad12270 con 0x55b16a95e390 2024-05-21T15:38:35.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.374+0000 7f84e77fe640 1 -- 172.21.15.38:0/2757968916 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f8009e30 con 0x55b16a95e390 2024-05-21T15:38:35.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.374+0000 7f84e77fe640 1 -- 172.21.15.38:0/2757968916 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f84f801a420 con 0x55b16a95e390 2024-05-21T15:38:35.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.374+0000 7f8515564740 10 client.16521 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:35.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.374+0000 7f8515564740 10 client.16521 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:38:35.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.374+0000 7f8515564740 20 client.16521 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:35.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.374+0000 7f8515564740 1 --2- 172.21.15.38:0/2757968916 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b16a8f23a0 0x55b16ad156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.374+0000 7f8515564740 4 client.16521 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:38:35.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.374+0000 7f8515564740 1 -- 172.21.15.38:0/2757968916 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b16ac70370 con 0x55b16a8f23a0 2024-05-21T15:38:35.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.375+0000 7f85077fe640 1 --2- 172.21.15.38:0/2757968916 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b16a8f23a0 0x55b16ad156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.378+0000 7f85077fe640 1 --2- 172.21.15.38:0/2757968916 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b16a8f23a0 0x55b16ad156c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e63fadff24758706 server_cookie=ede55f7d242e574a in_seq=0 out_seq=0 2024-05-21T15:38:35.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.379+0000 7f84e77fe640 10 client.16521 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:38:35.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.380+0000 7f84e77fe640 1 -- 172.21.15.38:0/2757968916 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b16ac70370 con 0x55b16a8f23a0 2024-05-21T15:38:35.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.380+0000 7f84e77fe640 10 client.16521 handle_command_reply: tid=0 2024-05-21T15:38:35.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.380+0000 7f8515564740 1 client.16521 shutdown 2024-05-21T15:38:35.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.384+0000 7f8515564740 1 -- 172.21.15.38:0/2757968916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a95e390 msgr2=0x55b16ad08050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.384+0000 7f8515564740 1 --2- 172.21.15.38:0/2757968916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a95e390 0x55b16ad08050 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x55b16acfe930 tx=0x7f84f8011650 comp rx=0 tx=0).stop 2024-05-21T15:38:35.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.384+0000 7f8515564740 1 -- 172.21.15.38:0/2757968916 shutdown_connections 2024-05-21T15:38:35.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.384+0000 7f8515564740 1 --2- 172.21.15.38:0/2757968916 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b16a8f23a0 0x55b16ad156c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.384+0000 7f8515564740 1 --2- 172.21.15.38:0/2757968916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a91cf60 0x55b16ad0efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.384+0000 7f8515564740 1 --2- 172.21.15.38:0/2757968916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a95e390 0x55b16ad08050 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.384+0000 7f8515564740 1 --2- 172.21.15.38:0/2757968916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a93d6b0 0x55b16ad07b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.384+0000 7f8515564740 1 -- 172.21.15.38:0/2757968916 >> 172.21.15.38:0/2757968916 conn(0x55b16a9524e0 msgr2=0x55b16acf8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.384+0000 7f84e77fe640 0 client.16521 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:38:35.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.389+0000 7f8515564740 1 -- 172.21.15.38:0/2757968916 shutdown_connections 2024-05-21T15:38:35.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.390+0000 7f8515564740 1 -- 172.21.15.38:0/2757968916 wait complete. 2024-05-21T15:38:35.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.393+0000 7f8515564740 20 client.16521 trim_cache size 0 max 16384 2024-05-21T15:38:35.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.425+0000 7f8515564740 1 Processor -- start 2024-05-21T15:38:35.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.425+0000 7f8515564740 1 -- start start 2024-05-21T15:38:35.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.425+0000 7f8515564740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a91cf60 0x55b16ad074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.425+0000 7f8515564740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a93d6b0 0x55b16ad0ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.425+0000 7f8515564740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 0x55b16acf8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.425+0000 7f8515564740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f84f400e640 con 0x55b16a93d6b0 2024-05-21T15:38:35.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.425+0000 7f8515564740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84f4010270 con 0x55b16a95e390 2024-05-21T15:38:35.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.425+0000 7f8515564740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f84f400ed10 con 0x55b16a91cf60 2024-05-21T15:38:35.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.426+0000 7f85077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a91cf60 0x55b16ad074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.426+0000 7f85077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a91cf60 0x55b16ad074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:48912/0 (socket says 172.21.15.38:48912) 2024-05-21T15:38:35.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.426+0000 7f85077fe640 1 -- 172.21.15.38:0/2253465339 learned_addr learned my addr 172.21.15.38:0/2253465339 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:35.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.428+0000 7f8507fff640 1 --2- 172.21.15.38:0/2253465339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a93d6b0 0x55b16ad0ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.428+0000 7f85077fe640 1 -- 172.21.15.38:0/2253465339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 msgr2=0x55b16acf8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.428+0000 7f85077fe640 1 --2- 172.21.15.38:0/2253465339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 0x55b16acf8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.428+0000 7f85077fe640 1 -- 172.21.15.38:0/2253465339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a93d6b0 msgr2=0x55b16ad0ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.428+0000 7f85077fe640 1 --2- 172.21.15.38:0/2253465339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a93d6b0 0x55b16ad0ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.428+0000 7f85077fe640 1 -- 172.21.15.38:0/2253465339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b16ad02910 con 0x55b16a91cf60 2024-05-21T15:38:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.429+0000 7f8507fff640 1 --2- 172.21.15.38:0/2253465339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a93d6b0 0x55b16ad0ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:38:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.429+0000 7f85077fe640 1 --2- 172.21.15.38:0/2253465339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a91cf60 0x55b16ad074b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55b16ad15e30 tx=0x7f84fc0068d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0750b16c6857060 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.430+0000 7f84edffb640 1 -- 172.21.15.38:0/2253465339 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc006540 con 0x55b16a91cf60 2024-05-21T15:38:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.432+0000 7f84edffb640 1 -- 172.21.15.38:0/2253465339 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84fc027070 con 0x55b16a91cf60 2024-05-21T15:38:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.432+0000 7f84edffb640 1 -- 172.21.15.38:0/2253465339 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc01a5b0 con 0x55b16a91cf60 2024-05-21T15:38:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.433+0000 7f8515564740 1 -- 172.21.15.38:0/2253465339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a91cf60 msgr2=0x55b16ad074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.433+0000 7f8515564740 1 --2- 172.21.15.38:0/2253465339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a91cf60 0x55b16ad074b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55b16ad15e30 tx=0x7f84fc0068d0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.434+0000 7f8515564740 1 -- 172.21.15.38:0/2253465339 shutdown_connections 2024-05-21T15:38:35.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.434+0000 7f8515564740 1 --2- 172.21.15.38:0/2253465339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 0x55b16acf8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.434+0000 7f8515564740 1 --2- 172.21.15.38:0/2253465339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a93d6b0 0x55b16ad0ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.434+0000 7f8515564740 1 --2- 172.21.15.38:0/2253465339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a91cf60 0x55b16ad074b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.434+0000 7f8515564740 1 -- 172.21.15.38:0/2253465339 >> 172.21.15.38:0/2253465339 conn(0x55b16a9524e0 msgr2=0x55b16ad06da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.434+0000 7f8515564740 1 -- 172.21.15.38:0/2253465339 shutdown_connections 2024-05-21T15:38:35.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.436+0000 7f8515564740 1 -- 172.21.15.38:0/2253465339 wait complete. 2024-05-21T15:38:35.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.436+0000 7f8515564740 1 -- start start 2024-05-21T15:38:35.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.437+0000 7f8515564740 1 Processor -- start 2024-05-21T15:38:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.437+0000 7f8515564740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a91cf60 0x55b16ad0c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.437+0000 7f8515564740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a93d6b0 0x55b16ad0c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.437+0000 7f8515564740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 0x55b16ad0d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.437+0000 7f8515564740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f84f400fae0 con 0x55b16a91cf60 2024-05-21T15:38:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.437+0000 7f8515564740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84f400f3e0 con 0x55b16a95e390 2024-05-21T15:38:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.437+0000 7f8515564740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f84f400c620 con 0x55b16a93d6b0 2024-05-21T15:38:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.437+0000 7f8515564740 10 client.0 _ll_register_callbacks cb 0x7f84fc00b120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.438+0000 7f85077fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a91cf60 0x55b16ad0c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.438+0000 7f85077fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a91cf60 0x55b16ad0c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35612/0 (socket says 172.21.15.38:35612) 2024-05-21T15:38:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.438+0000 7f85077fe640 1 -- 172.21.15.38:0/3012206140 learned_addr learned my addr 172.21.15.38:0/3012206140 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.438+0000 7f85077fe640 1 --2- 172.21.15.38:0/3012206140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 0x55b16ad0d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.441+0000 7f8507fff640 1 --2- 172.21.15.38:0/3012206140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a93d6b0 0x55b16ad0c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.442+0000 7f85077fe640 1 -- 172.21.15.38:0/3012206140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a93d6b0 msgr2=0x55b16ad0c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.442+0000 7f85077fe640 1 --2- 172.21.15.38:0/3012206140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a93d6b0 0x55b16ad0c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.442+0000 7f85077fe640 1 -- 172.21.15.38:0/3012206140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a91cf60 msgr2=0x55b16ad0c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.442+0000 7f85077fe640 1 --2- 172.21.15.38:0/3012206140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a91cf60 0x55b16ad0c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.442+0000 7f85077fe640 1 -- 172.21.15.38:0/3012206140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b16ad0de30 con 0x55b16a95e390 2024-05-21T15:38:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.444+0000 7f85077fe640 1 --2- 172.21.15.38:0/3012206140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 0x55b16ad0d870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0049c0 tx=0x7f84fc006040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=167a2e1404e5e67d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.445+0000 7f84e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:38:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.445+0000 7f84e77fe640 1 -- 172.21.15.38:0/3012206140 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc01a5b0 con 0x55b16a95e390 2024-05-21T15:38:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.445+0000 7f8515564740 1 -- 172.21.15.38:0/3012206140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84fc027070 con 0x55b16a95e390 2024-05-21T15:38:35.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.445+0000 7f84e77fe640 1 -- 172.21.15.38:0/3012206140 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f84fc01a750 con 0x55b16a95e390 2024-05-21T15:38:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.445+0000 7f84e77fe640 1 -- 172.21.15.38:0/3012206140 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f84fc01a950 con 0x55b16a95e390 2024-05-21T15:38:35.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.448+0000 7f8515564740 10 client.26186 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:35.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.448+0000 7f8515564740 1 -- 172.21.15.38:0/3012206140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b16ad0f0b0 con 0x55b16a95e390 2024-05-21T15:38:35.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.449+0000 7f84e77fe640 1 -- 172.21.15.38:0/3012206140 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84fc027070 con 0x55b16a95e390 2024-05-21T15:38:35.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.450+0000 7f84e77fe640 1 -- 172.21.15.38:0/3012206140 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f84fc034050 con 0x55b16a95e390 2024-05-21T15:38:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.452+0000 7f8515564740 10 client.26186 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.452+0000 7f8515564740 10 client.26186 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:38:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.452+0000 7f8515564740 20 client.26186 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.452+0000 7f8515564740 1 --2- 172.21.15.38:0/3012206140 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b16ad7df80 0x55b16ad803a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.452+0000 7f8515564740 4 client.26186 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:38:35.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.452+0000 7f8515564740 1 -- 172.21.15.38:0/3012206140 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b16ac6a6c0 con 0x55b16ad7df80 2024-05-21T15:38:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.453+0000 7f8d1dffb640 1 -- 172.21.15.38:0/4038879448 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c3470b9dc0 con 0x55c346d421c0 2024-05-21T15:38:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.453+0000 7f8d1dffb640 10 client.16491 handle_command_reply: tid=0 2024-05-21T15:38:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.454+0000 7f8d43843740 1 client.16491 shutdown 2024-05-21T15:38:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.455+0000 7f8d43843740 1 -- 172.21.15.38:0/4038879448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 msgr2=0x55c34715b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.455+0000 7f8d43843740 1 --2- 172.21.15.38:0/4038879448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c34715b2f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8d280bb7e0 tx=0x7f8d2803b460 comp rx=0 tx=0).stop 2024-05-21T15:38:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.455+0000 7f8d43843740 1 -- 172.21.15.38:0/4038879448 shutdown_connections 2024-05-21T15:38:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.455+0000 7f8d43843740 1 --2- 172.21.15.38:0/4038879448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c346dae1b0 0x55c34715e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.455+0000 7f8d43843740 1 --2- 172.21.15.38:0/4038879448 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c346d421c0 0x55c3471cff10 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.455+0000 7f8d43843740 1 --2- 172.21.15.38:0/4038879448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c346d8d4d0 0x55c34715b2f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.455+0000 7f8d43843740 1 --2- 172.21.15.38:0/4038879448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c346d852f0 0x55c34715adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.455+0000 7f8d43843740 1 -- 172.21.15.38:0/4038879448 >> 172.21.15.38:0/4038879448 conn(0x55c346da2300 msgr2=0x55c347149400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.455+0000 7f8d1dffb640 0 client.16491 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.455+0000 7f8d43843740 1 -- 172.21.15.38:0/4038879448 shutdown_connections 2024-05-21T15:38:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.455+0000 7f8d43843740 1 -- 172.21.15.38:0/4038879448 wait complete. 2024-05-21T15:38:35.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.454+0000 7f8507fff640 1 --2- 172.21.15.38:0/3012206140 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b16ad7df80 0x55b16ad803a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.456+0000 7f8d43843740 20 client.16491 trim_cache size 0 max 16384 2024-05-21T15:38:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:38:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:38:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:38:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:38:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.457+0000 7f8d3759e640 1 -- 172.21.15.38:0/4213673880 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8d200a2da0 msgr2=0x7f8d200a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.457+0000 7f8d3759e640 1 --2- 172.21.15.38:0/4213673880 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8d200a2da0 0x7f8d200a52b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00a500 tx=0x7f8d2c00a490 comp rx=0 tx=0).stop 2024-05-21T15:38:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.457+0000 7f8d3759e640 1 -- 172.21.15.38:0/4213673880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d300abfd0 msgr2=0x7f8d30144e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.457+0000 7f8d3759e640 1 --2- 172.21.15.38:0/4213673880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d300abfd0 0x7f8d30144e20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8d2802a9e0 tx=0x7f8d28032c60 comp rx=0 tx=0).stop 2024-05-21T15:38:35.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.458+0000 7f84e77fe640 10 client.26186 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:38:35.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.458+0000 7f8507fff640 1 --2- 172.21.15.38:0/3012206140 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b16ad7df80 0x55b16ad803a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28ddb5ab440db1a5 server_cookie=2da48a6e6467418f in_seq=0 out_seq=0 2024-05-21T15:38:35.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.461+0000 7f84e77fe640 1 -- 172.21.15.38:0/3012206140 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b16ac6a6c0 con 0x55b16ad7df80 2024-05-21T15:38:35.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.461+0000 7f84e77fe640 10 client.26186 handle_command_reply: tid=0 2024-05-21T15:38:35.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.461+0000 7f8515564740 1 client.26186 shutdown 2024-05-21T15:38:35.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.462+0000 7f8515564740 1 -- 172.21.15.38:0/3012206140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 msgr2=0x55b16ad0d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.462+0000 7f8515564740 1 --2- 172.21.15.38:0/3012206140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 0x55b16ad0d870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0049c0 tx=0x7f84fc006040 comp rx=0 tx=0).stop 2024-05-21T15:38:35.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.462+0000 7f8515564740 1 -- 172.21.15.38:0/3012206140 shutdown_connections 2024-05-21T15:38:35.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.462+0000 7f8515564740 1 --2- 172.21.15.38:0/3012206140 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b16ad7df80 0x55b16ad803a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.462+0000 7f8515564740 1 --2- 172.21.15.38:0/3012206140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b16a95e390 0x55b16ad0d870 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.462+0000 7f8515564740 1 --2- 172.21.15.38:0/3012206140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b16a93d6b0 0x55b16ad0c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.462+0000 7f8515564740 1 --2- 172.21.15.38:0/3012206140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b16a91cf60 0x55b16ad0c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.462+0000 7f8515564740 1 -- 172.21.15.38:0/3012206140 >> 172.21.15.38:0/3012206140 conn(0x55b16a9524e0 msgr2=0x55b16acf8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.462+0000 7f8d3759e640 1 -- 172.21.15.38:0/4213673880 shutdown_connections 2024-05-21T15:38:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.462+0000 7f8d3759e640 1 --2- 172.21.15.38:0/4213673880 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8d200a2da0 0x7f8d200a52b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.463+0000 7f8d3759e640 1 --2- 172.21.15.38:0/4213673880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d30149d20 0x7f8d3014c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.463+0000 7f8d3759e640 1 --2- 172.21.15.38:0/4213673880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d300abfd0 0x7f8d30144e20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.463+0000 7f8d3759e640 1 --2- 172.21.15.38:0/4213673880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d300a61d0 0x7f8d30144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.463+0000 7f8d3759e640 1 -- 172.21.15.38:0/4213673880 >> 172.21.15.38:0/4213673880 conn(0x7f8d300a09b0 msgr2=0x7f8d300a7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.463+0000 7f84e77fe640 0 client.26186 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:38:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.463+0000 7f8d3759e640 1 -- 172.21.15.38:0/4213673880 shutdown_connections 2024-05-21T15:38:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:35.463+0000 7f8d3759e640 1 -- 172.21.15.38:0/4213673880 wait complete. 2024-05-21T15:38:35.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.464+0000 7f8515564740 1 -- 172.21.15.38:0/3012206140 shutdown_connections 2024-05-21T15:38:35.470 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:38:35.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.466+0000 7f8515564740 1 -- 172.21.15.38:0/3012206140 wait complete. 2024-05-21T15:38:35.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.467+0000 7f8515564740 20 client.26186 trim_cache size 0 max 16384 2024-05-21T15:38:35.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.471+0000 7f850e5e3640 1 -- 172.21.15.38:0/2029675434 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84f00a2ce0 msgr2=0x7f84f00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.471+0000 7f850e5e3640 1 --2- 172.21.15.38:0/2029675434 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84f00a2ce0 0x7f84f00a51f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f85000094e0 tx=0x7f85000095c0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.471+0000 7f850e5e3640 1 -- 172.21.15.38:0/2029675434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850810efe0 msgr2=0x7f85081174e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.471+0000 7f850e5e3640 1 --2- 172.21.15.38:0/2029675434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850810efe0 0x7f85081174e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8500033040 tx=0x7f8500003ba0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.471+0000 7f850e5e3640 1 -- 172.21.15.38:0/2029675434 shutdown_connections 2024-05-21T15:38:35.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.471+0000 7f850e5e3640 1 --2- 172.21.15.38:0/2029675434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850810efe0 0x7f85081174e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.471+0000 7f850e5e3640 1 --2- 172.21.15.38:0/2029675434 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84f00a2ce0 0x7f84f00a51f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.471+0000 7f850e5e3640 1 --2- 172.21.15.38:0/2029675434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850810a2c0 0x7f8508116580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.471+0000 7f850e5e3640 1 --2- 172.21.15.38:0/2029675434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85081093d0 0x7f8508116040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.471+0000 7f850e5e3640 1 -- 172.21.15.38:0/2029675434 >> 172.21.15.38:0/2029675434 conn(0x7f850806f4a0 msgr2=0x7f850810afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.472+0000 7f850e5e3640 1 -- 172.21.15.38:0/2029675434 shutdown_connections 2024-05-21T15:38:35.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.472+0000 7f850e5e3640 1 -- 172.21.15.38:0/2029675434 wait complete. 2024-05-21T15:38:35.496 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:38:35.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.715+0000 7f7d438bc640 1 -- 172.21.15.38:0/2084610149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3c101220 msgr2=0x7f7d3c10ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.715+0000 7f7d438bc640 1 --2- 172.21.15.38:0/2084610149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3c101220 0x7f7d3c10ac80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009ca0 tx=0x7f7d3402a8f0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.715+0000 7f7d438bc640 1 -- 172.21.15.38:0/2084610149 shutdown_connections 2024-05-21T15:38:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.715+0000 7f7d438bc640 1 --2- 172.21.15.38:0/2084610149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3c101220 0x7f7d3c10ac80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.715+0000 7f7d438bc640 1 --2- 172.21.15.38:0/2084610149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d3c100720 0x7f7d3c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.715+0000 7f7d438bc640 1 --2- 172.21.15.38:0/2084610149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c104070 0x7f7d3c104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.715+0000 7f7d438bc640 1 -- 172.21.15.38:0/2084610149 >> 172.21.15.38:0/2084610149 conn(0x7f7d3c0fc320 msgr2=0x7f7d3c0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.718+0000 7f7d438bc640 1 -- 172.21.15.38:0/2084610149 shutdown_connections 2024-05-21T15:38:35.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.720+0000 7f7d438bc640 1 -- 172.21.15.38:0/2084610149 wait complete. 2024-05-21T15:38:35.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.721+0000 7f7d438bc640 1 Processor -- start 2024-05-21T15:38:35.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.721+0000 7f7d438bc640 1 -- start start 2024-05-21T15:38:35.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.721+0000 7f7d438bc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d3c100720 0x7f7d3c06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.721+0000 7f7d438bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c101220 0x7f7d3c06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.721+0000 7f7d438bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3c104070 0x7f7d3c072b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.721+0000 7f7d438bc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c06bae0 con 0x7f7d3c101220 2024-05-21T15:38:35.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.721+0000 7f7d438bc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c06b960 con 0x7f7d3c100720 2024-05-21T15:38:35.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.721+0000 7f7d438bc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c06bc60 con 0x7f7d3c104070 2024-05-21T15:38:35.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.722+0000 7f7d41e59640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3c104070 0x7f7d3c072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.722+0000 7f7d41e59640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3c104070 0x7f7d3c072b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:48942/0 (socket says 172.21.15.38:48942) 2024-05-21T15:38:35.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.722+0000 7f7d41e59640 1 -- 172.21.15.38:0/2359152481 learned_addr learned my addr 172.21.15.38:0/2359152481 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:35.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.722+0000 7f7d40e57640 1 --2- 172.21.15.38:0/2359152481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c101220 0x7f7d3c06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.722+0000 7f7d41658640 1 --2- 172.21.15.38:0/2359152481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d3c100720 0x7f7d3c06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.723+0000 7f7d41658640 1 -- 172.21.15.38:0/2359152481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3c104070 msgr2=0x7f7d3c072b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.723+0000 7f7d41658640 1 --2- 172.21.15.38:0/2359152481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3c104070 0x7f7d3c072b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.723+0000 7f7d41658640 1 -- 172.21.15.38:0/2359152481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c101220 msgr2=0x7f7d3c06fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.723+0000 7f7d41658640 1 --2- 172.21.15.38:0/2359152481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c101220 0x7f7d3c06fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.723+0000 7f7d41658640 1 -- 172.21.15.38:0/2359152481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c0731d0 con 0x7f7d3c100720 2024-05-21T15:38:35.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.725+0000 7f7d41658640 1 --2- 172.21.15.38:0/2359152481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d3c100720 0x7f7d3c06f6b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c06a4c0 tx=0x7f7d2c00db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:35.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.726+0000 7f7d327fc640 1 -- 172.21.15.38:0/2359152481 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c017070 con 0x7f7d3c100720 2024-05-21T15:38:35.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.726+0000 7f7d327fc640 1 -- 172.21.15.38:0/2359152481 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2c009d00 con 0x7f7d3c100720 2024-05-21T15:38:35.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.726+0000 7f7d327fc640 1 -- 172.21.15.38:0/2359152481 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c003cd0 con 0x7f7d3c100720 2024-05-21T15:38:35.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.727+0000 7f7d438bc640 1 -- 172.21.15.38:0/2359152481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c0734c0 con 0x7f7d3c100720 2024-05-21T15:38:35.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.727+0000 7f7d438bc640 1 -- 172.21.15.38:0/2359152481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c073a40 con 0x7f7d3c100720 2024-05-21T15:38:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.731+0000 7f7d4a836740 1 Processor -- start 2024-05-21T15:38:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.731+0000 7f7d4a836740 1 -- start start 2024-05-21T15:38:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.731+0000 7f7d4a836740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331bf390 0x564e3355eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.731+0000 7f7d4a836740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e331964d0 0x564e3355f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.731+0000 7f7d4a836740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e3319e6b0 0x564e33565f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.731+0000 7f7d4a836740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564e334c3c40 con 0x564e331964d0 2024-05-21T15:38:35.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.731+0000 7f7d4a836740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564e3346b580 con 0x564e3319e6b0 2024-05-21T15:38:35.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.731+0000 7f7d4a836740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564e334bfc00 con 0x564e331bf390 2024-05-21T15:38:35.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.733+0000 7f7d40e57640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e3319e6b0 0x564e33565f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.733+0000 7f7d40e57640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e3319e6b0 0x564e33565f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46092/0 (socket says 172.21.15.38:46092) 2024-05-21T15:38:35.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.733+0000 7f7d40e57640 1 -- 172.21.15.38:0/2900486882 learned_addr learned my addr 172.21.15.38:0/2900486882 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:35.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.733+0000 7f7d41e59640 1 --2- 172.21.15.38:0/2900486882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331bf390 0x564e3355eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.733+0000 7f7d40e57640 1 -- 172.21.15.38:0/2900486882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331bf390 msgr2=0x564e3355eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.733+0000 7f7d40e57640 1 --2- 172.21.15.38:0/2900486882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331bf390 0x564e3355eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.733+0000 7f7d40e57640 1 -- 172.21.15.38:0/2900486882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e331964d0 msgr2=0x564e3355f080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:38:35.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.733+0000 7f7d40e57640 1 --2- 172.21.15.38:0/2900486882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e331964d0 0x564e3355f080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.733+0000 7f7d40e57640 1 -- 172.21.15.38:0/2900486882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e331d73f0 con 0x564e3319e6b0 2024-05-21T15:38:35.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.734+0000 7f7d40e57640 1 --2- 172.21.15.38:0/2900486882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e3319e6b0 0x564e33565f80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009a10 tx=0x7f7d38009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c37348827d711632 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.734+0000 7f7d277fe640 1 -- 172.21.15.38:0/2900486882 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38017070 con 0x564e3319e6b0 2024-05-21T15:38:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.734+0000 7f7d277fe640 1 -- 172.21.15.38:0/2900486882 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d38013470 con 0x564e3319e6b0 2024-05-21T15:38:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.734+0000 7f7d277fe640 1 -- 172.21.15.38:0/2900486882 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38012610 con 0x564e3319e6b0 2024-05-21T15:38:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.735+0000 7f7d327fc640 1 -- 172.21.15.38:0/2359152481 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7d2c004220 con 0x7f7d3c100720 2024-05-21T15:38:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.736+0000 7f7d327fc640 1 --2- 172.21.15.38:0/2359152481 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7d280a2da0 0x7f7d280a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.736+0000 7f7d327fc640 1 -- 172.21.15.38:0/2359152481 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7d2c0ceb90 con 0x7f7d3c100720 2024-05-21T15:38:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.737+0000 7f7d4a836740 1 -- 172.21.15.38:0/2900486882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e3319e6b0 msgr2=0x564e33565f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.737+0000 7f7d4a836740 1 --2- 172.21.15.38:0/2900486882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e3319e6b0 0x564e33565f80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009a10 tx=0x7f7d38009ed0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.737+0000 7f7d4a836740 1 -- 172.21.15.38:0/2900486882 shutdown_connections 2024-05-21T15:38:35.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.737+0000 7f7d41e59640 1 --2- 172.21.15.38:0/2359152481 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7d280a2da0 0x7f7d280a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.737+0000 7f7d4a836740 1 --2- 172.21.15.38:0/2900486882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e3319e6b0 0x564e33565f80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.737+0000 7f7d4a836740 1 --2- 172.21.15.38:0/2900486882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e331964d0 0x564e3355f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.738+0000 7f7d4a836740 1 --2- 172.21.15.38:0/2900486882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331bf390 0x564e3355eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.738+0000 7f7d4a836740 1 -- 172.21.15.38:0/2900486882 >> 172.21.15.38:0/2900486882 conn(0x564e331b34e0 msgr2=0x564e3355a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.739+0000 7f7d4a836740 1 -- 172.21.15.38:0/2900486882 shutdown_connections 2024-05-21T15:38:35.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.739+0000 7f7d4a836740 1 -- 172.21.15.38:0/2900486882 wait complete. 2024-05-21T15:38:35.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.739+0000 7f7d41e59640 1 --2- 172.21.15.38:0/2359152481 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7d280a2da0 0x7f7d280a52b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x564e3355fcd0 tx=0x7f7d34031ba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:35.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.740+0000 7f7d4a836740 1 -- start start 2024-05-21T15:38:35.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.740+0000 7f7d4a836740 1 Processor -- start 2024-05-21T15:38:35.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.742+0000 7f7d4a836740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e331964d0 0x564e335689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.742+0000 7f7d4a836740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e3319e6b0 0x564e33568f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.742+0000 7f7d4a836740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331bf390 0x564e3356fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.742+0000 7f7d4a836740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564e335669d0 con 0x564e331964d0 2024-05-21T15:38:35.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.742+0000 7f7d4a836740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564e3347ec40 con 0x564e331bf390 2024-05-21T15:38:35.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.742+0000 7f7d4a836740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564e33566b50 con 0x564e3319e6b0 2024-05-21T15:38:35.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.742+0000 7f7d4a836740 10 client.0 _ll_register_callbacks cb 0x564e331d6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:35.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.742+0000 7f7d41658640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e331964d0 0x564e335689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.743+0000 7f7d41658640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e331964d0 0x564e335689c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35640/0 (socket says 172.21.15.38:35640) 2024-05-21T15:38:35.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.743+0000 7f7d41658640 1 -- 172.21.15.38:0/4178367692 learned_addr learned my addr 172.21.15.38:0/4178367692 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:35.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.743+0000 7f7d41658640 1 -- 172.21.15.38:0/4178367692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e3319e6b0 msgr2=0x564e33568f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.743+0000 7f7d41658640 1 --2- 172.21.15.38:0/4178367692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e3319e6b0 0x564e33568f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.743+0000 7f7d41658640 1 -- 172.21.15.38:0/4178367692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331bf390 msgr2=0x564e3356fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.743+0000 7f7d41e59640 1 --2- 172.21.15.38:0/4178367692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331bf390 0x564e3356fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.743+0000 7f7d41658640 1 --2- 172.21.15.38:0/4178367692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331bf390 0x564e3356fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.743+0000 7f7d41658640 1 -- 172.21.15.38:0/4178367692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e332a17d0 con 0x564e331964d0 2024-05-21T15:38:35.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.743+0000 7f7d4a836740 1 -- 172.21.15.38:0/4178367692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d38013470 con 0x564e331964d0 2024-05-21T15:38:35.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.743+0000 7f7d41e59640 1 --2- 172.21.15.38:0/4178367692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331bf390 0x564e3356fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:38:35.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.744+0000 7f7d41658640 1 --2- 172.21.15.38:0/4178367692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e331964d0 0x564e335689c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c097dc0 tx=0x7f7d2c09b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5f0488c6cbc82f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:35.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.744+0000 7f7d24ff9640 10 client.16536 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:35.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.744+0000 7f7d24ff9640 1 -- 172.21.15.38:0/4178367692 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c020070 con 0x564e331964d0 2024-05-21T15:38:35.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.744+0000 7f7d24ff9640 1 -- 172.21.15.38:0/4178367692 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7d2c097a60 con 0x564e331964d0 2024-05-21T15:38:35.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.744+0000 7f7d24ff9640 1 -- 172.21.15.38:0/4178367692 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7d2c0a3fb0 con 0x564e331964d0 2024-05-21T15:38:35.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.745+0000 7f7d4a836740 10 client.16536 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:35.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.745+0000 7f7d4a836740 1 -- 172.21.15.38:0/4178367692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({fsmap=31}) -- 0x564e335730d0 con 0x564e331964d0 2024-05-21T15:38:35.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.745+0000 7f7d24ff9640 1 -- 172.21.15.38:0/4178367692 <== mon.0 v2:172.21.15.38:3300/0 4 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f7d2c0a3b20 con 0x564e331964d0 2024-05-21T15:38:35.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.745+0000 7f7d4a836740 10 client.16536 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:35.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.745+0000 7f7d4a836740 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:35.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.746+0000 7f7d4a836740 20 client.16536 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:35.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.746+0000 7f7d4a836740 1 --2- 172.21.15.38:0/4178367692 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564e3317df60 0x564e335767b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.746+0000 7f7d4a836740 4 client.16536 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:38:35.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.746+0000 7f7d4a836740 1 -- 172.21.15.38:0/4178367692 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e334d1370 con 0x564e3317df60 2024-05-21T15:38:35.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.746+0000 7f7d40e57640 1 --2- 172.21.15.38:0/4178367692 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564e3317df60 0x564e335767b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.747+0000 7f7d40e57640 1 --2- 172.21.15.38:0/4178367692 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564e3317df60 0x564e335767b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dfef1ff2189195f server_cookie=8dd74b462bc2f145 in_seq=0 out_seq=0 2024-05-21T15:38:35.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.747+0000 7f7d24ff9640 10 client.16536 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:35.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.748+0000 7f7d24ff9640 1 -- 172.21.15.38:0/4178367692 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564e334d1370 con 0x564e3317df60 2024-05-21T15:38:35.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.748+0000 7f7d24ff9640 10 client.16536 handle_command_reply: tid=0 2024-05-21T15:38:35.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.748+0000 7f7d4a836740 1 client.16536 shutdown 2024-05-21T15:38:35.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.751+0000 7f7d4a836740 1 -- 172.21.15.38:0/4178367692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e331964d0 msgr2=0x564e335689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.751+0000 7f7d4a836740 1 --2- 172.21.15.38:0/4178367692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e331964d0 0x564e335689c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c097dc0 tx=0x7f7d2c09b060 comp rx=0 tx=0).stop 2024-05-21T15:38:35.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.753+0000 7f7d4a836740 1 -- 172.21.15.38:0/4178367692 shutdown_connections 2024-05-21T15:38:35.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.753+0000 7f7d4a836740 1 --2- 172.21.15.38:0/4178367692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331bf390 0x564e3356fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.753+0000 7f7d4a836740 1 --2- 172.21.15.38:0/4178367692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e3319e6b0 0x564e33568f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.753+0000 7f7d4a836740 1 --2- 172.21.15.38:0/4178367692 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564e3317df60 0x564e335767b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.753+0000 7f7d4a836740 1 --2- 172.21.15.38:0/4178367692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e331964d0 0x564e335689c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.753+0000 7f7d4a836740 1 -- 172.21.15.38:0/4178367692 >> 172.21.15.38:0/4178367692 conn(0x564e331b34e0 msgr2=0x564e33559570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.754+0000 7f7d24ff9640 0 client.16536 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:35.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.754+0000 7f7d4a836740 1 -- 172.21.15.38:0/4178367692 shutdown_connections 2024-05-21T15:38:35.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.759+0000 7f7d4a836740 1 -- 172.21.15.38:0/4178367692 wait complete. 2024-05-21T15:38:35.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.764+0000 7f7d4a836740 20 client.16536 trim_cache size 0 max 16384 2024-05-21T15:38:35.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:35 smithi038 ceph-mon[31995]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 100 MiB/s wr, 17.39k op/s 2024-05-21T15:38:35.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.795+0000 7f7d4a836740 1 Processor -- start 2024-05-21T15:38:35.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.795+0000 7f7d4a836740 1 -- start start 2024-05-21T15:38:35.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.795+0000 7f7d4a836740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331964d0 0x564e3355a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.795+0000 7f7d4a836740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e3319e6b0 0x564e3356fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.795+0000 7f7d4a836740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331bf390 0x564e3355ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.796+0000 7f7d4a836740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2000e5e0 con 0x564e3319e6b0 2024-05-21T15:38:35.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.796+0000 7f7d4a836740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20010210 con 0x564e331bf390 2024-05-21T15:38:35.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.796+0000 7f7d4a836740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2000ecb0 con 0x564e331964d0 2024-05-21T15:38:35.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.796+0000 7f7d41e59640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331bf390 0x564e3355ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.796+0000 7f7d41e59640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331bf390 0x564e3355ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46108/0 (socket says 172.21.15.38:46108) 2024-05-21T15:38:35.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.796+0000 7f7d41e59640 1 -- 172.21.15.38:0/301841869 learned_addr learned my addr 172.21.15.38:0/301841869 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:35.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.797+0000 7f7d41658640 1 --2- 172.21.15.38:0/301841869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331964d0 0x564e3355a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.798+0000 7f7d41e59640 1 -- 172.21.15.38:0/301841869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331964d0 msgr2=0x564e3355a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.798+0000 7f7d41e59640 1 --2- 172.21.15.38:0/301841869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331964d0 0x564e3355a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.798+0000 7f7d41e59640 1 -- 172.21.15.38:0/301841869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e3319e6b0 msgr2=0x564e3356fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.798+0000 7f7d41e59640 1 --2- 172.21.15.38:0/301841869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e3319e6b0 0x564e3356fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.798+0000 7f7d41e59640 1 -- 172.21.15.38:0/301841869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e33555d20 con 0x564e331bf390 2024-05-21T15:38:35.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.799+0000 7f7d41e59640 1 --2- 172.21.15.38:0/301841869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331bf390 0x564e3355ef10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d340319a0 tx=0x7f7d34041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30045c0b1c5cf0bb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:35.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.799+0000 7f7d26ffd640 1 -- 172.21.15.38:0/301841869 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d34004100 con 0x564e331bf390 2024-05-21T15:38:35.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.799+0000 7f7d26ffd640 1 -- 172.21.15.38:0/301841869 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d34036050 con 0x564e331bf390 2024-05-21T15:38:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.801+0000 7f7d4a836740 1 -- 172.21.15.38:0/301841869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331bf390 msgr2=0x564e3355ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.801+0000 7f7d4a836740 1 --2- 172.21.15.38:0/301841869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331bf390 0x564e3355ef10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d340319a0 tx=0x7f7d34041000 comp rx=0 tx=0).stop 2024-05-21T15:38:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.802+0000 7f7d4a836740 1 -- 172.21.15.38:0/301841869 shutdown_connections 2024-05-21T15:38:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.802+0000 7f7d4a836740 1 --2- 172.21.15.38:0/301841869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331bf390 0x564e3355ef10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.802+0000 7f7d4a836740 1 --2- 172.21.15.38:0/301841869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e3319e6b0 0x564e3356fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.802+0000 7f7d4a836740 1 --2- 172.21.15.38:0/301841869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331964d0 0x564e3355a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.802+0000 7f7d4a836740 1 -- 172.21.15.38:0/301841869 >> 172.21.15.38:0/301841869 conn(0x564e331b34e0 msgr2=0x564e3355e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.807+0000 7f7d4a836740 1 -- 172.21.15.38:0/301841869 shutdown_connections 2024-05-21T15:38:35.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.808+0000 7f7d4a836740 1 -- 172.21.15.38:0/301841869 wait complete. 2024-05-21T15:38:35.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.809+0000 7f7d4a836740 1 -- start start 2024-05-21T15:38:35.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.809+0000 7f7d4a836740 1 Processor -- start 2024-05-21T15:38:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.810+0000 7f7d4a836740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331964d0 0x564e3356ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.810+0000 7f7d4a836740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e3319e6b0 0x564e3356d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.810+0000 7f7d4a836740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331bf390 0x564e335dcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.810+0000 7f7d4a836740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2000fa80 con 0x564e3319e6b0 2024-05-21T15:38:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.810+0000 7f7d4a836740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2000f380 con 0x564e331964d0 2024-05-21T15:38:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.810+0000 7f7d4a836740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2000c5c0 con 0x564e331bf390 2024-05-21T15:38:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.810+0000 7f7d4a836740 10 client.0 _ll_register_callbacks cb 0x7f7d2c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:35.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.810+0000 7f7d40e57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e3319e6b0 0x564e3356d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.810+0000 7f7d41e59640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331bf390 0x564e335dcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.810+0000 7f7d40e57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e3319e6b0 0x564e3356d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35658/0 (socket says 172.21.15.38:35658) 2024-05-21T15:38:35.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.810+0000 7f7d40e57640 1 -- 172.21.15.38:0/309375262 learned_addr learned my addr 172.21.15.38:0/309375262 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:35.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.811+0000 7f7d40e57640 1 -- 172.21.15.38:0/309375262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331bf390 msgr2=0x564e335dcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:35.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.811+0000 7f7d40e57640 1 --2- 172.21.15.38:0/309375262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331bf390 0x564e335dcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.811+0000 7f7d40e57640 1 -- 172.21.15.38:0/309375262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331964d0 msgr2=0x564e3356ce20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:38:35.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.811+0000 7f7d40e57640 1 --2- 172.21.15.38:0/309375262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331964d0 0x564e3356ce20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.811+0000 7f7d40e57640 1 -- 172.21.15.38:0/309375262 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e335dd090 con 0x564e3319e6b0 2024-05-21T15:38:35.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.812+0000 7f7d40e57640 1 --2- 172.21.15.38:0/309375262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e3319e6b0 0x564e3356d360 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x564e3355b2c0 tx=0x7f7d3800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e025b332188616 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:35.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.812+0000 7f7d24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:35.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.812+0000 7f7d24ff9640 1 -- 172.21.15.38:0/309375262 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38017040 con 0x564e3319e6b0 2024-05-21T15:38:35.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.812+0000 7f7d24ff9640 1 -- 172.21.15.38:0/309375262 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7d38004040 con 0x564e3319e6b0 2024-05-21T15:38:35.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.813+0000 7f7d4a836740 1 -- 172.21.15.38:0/309375262 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d34036050 con 0x564e3319e6b0 2024-05-21T15:38:35.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.813+0000 7f7d24ff9640 1 -- 172.21.15.38:0/309375262 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7d38004240 con 0x564e3319e6b0 2024-05-21T15:38:35.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.813+0000 7f7d4a836740 10 client.16539 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:35.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.813+0000 7f7d4a836740 1 -- 172.21.15.38:0/309375262 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564e335de3b0 con 0x564e3319e6b0 2024-05-21T15:38:35.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.815+0000 7f7d24ff9640 1 -- 172.21.15.38:0/309375262 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d34036050 con 0x564e3319e6b0 2024-05-21T15:38:35.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.815+0000 7f7d24ff9640 1 -- 172.21.15.38:0/309375262 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f7d38024430 con 0x564e3319e6b0 2024-05-21T15:38:35.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.817+0000 7f7d4a836740 10 client.16539 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:35.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.817+0000 7f7d4a836740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:35.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.817+0000 7f7d4a836740 20 client.16539 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.817+0000 7f7d4a836740 1 --2- 172.21.15.38:0/309375262 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564e331533a0 0x564e335e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.817+0000 7f7d4a836740 4 client.16539 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:38:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.817+0000 7f7d4a836740 1 -- 172.21.15.38:0/309375262 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e334cb6c0 con 0x564e331533a0 2024-05-21T15:38:35.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.818+0000 7f7d41658640 1 --2- 172.21.15.38:0/309375262 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564e331533a0 0x564e335e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:35.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.821+0000 7f7d41658640 1 --2- 172.21.15.38:0/309375262 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564e331533a0 0x564e335e0b70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b53f05bb24292028 server_cookie=eecbb515c4901fcb in_seq=0 out_seq=0 2024-05-21T15:38:35.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.821+0000 7f7d24ff9640 10 client.16539 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:35.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.821+0000 7f7d24ff9640 1 -- 172.21.15.38:0/309375262 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e334cb6c0 con 0x564e331533a0 2024-05-21T15:38:35.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.821+0000 7f7d24ff9640 10 client.16539 handle_command_reply: tid=0 2024-05-21T15:38:35.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.821+0000 7f7d4a836740 1 client.16539 shutdown 2024-05-21T15:38:35.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.824+0000 7f7d4a836740 1 -- 172.21.15.38:0/309375262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e3319e6b0 msgr2=0x564e3356d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.824+0000 7f7d4a836740 1 --2- 172.21.15.38:0/309375262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e3319e6b0 0x564e3356d360 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x564e3355b2c0 tx=0x7f7d3800ec40 comp rx=0 tx=0).stop 2024-05-21T15:38:35.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.825+0000 7f7d4a836740 1 -- 172.21.15.38:0/309375262 shutdown_connections 2024-05-21T15:38:35.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.825+0000 7f7d4a836740 1 --2- 172.21.15.38:0/309375262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e331bf390 0x564e335dcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.825+0000 7f7d4a836740 1 --2- 172.21.15.38:0/309375262 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564e331533a0 0x564e335e0b70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.825+0000 7f7d4a836740 1 --2- 172.21.15.38:0/309375262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564e3319e6b0 0x564e3356d360 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.825+0000 7f7d4a836740 1 --2- 172.21.15.38:0/309375262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e331964d0 0x564e3356ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.825+0000 7f7d4a836740 1 -- 172.21.15.38:0/309375262 >> 172.21.15.38:0/309375262 conn(0x564e331b34e0 msgr2=0x564e33558b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.826+0000 7f7d24ff9640 0 client.16539 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:35.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.826+0000 7f7d4a836740 1 -- 172.21.15.38:0/309375262 shutdown_connections 2024-05-21T15:38:35.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.826+0000 7f7d4a836740 1 -- 172.21.15.38:0/309375262 wait complete. 2024-05-21T15:38:35.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.826+0000 7f7d4a836740 20 client.16539 trim_cache size 0 max 16384 2024-05-21T15:38:35.826 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:38:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.830+0000 7f7d438bc640 1 -- 172.21.15.38:0/2359152481 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7d280a2da0 msgr2=0x7f7d280a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.830+0000 7f7d438bc640 1 --2- 172.21.15.38:0/2359152481 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7d280a2da0 0x7f7d280a52b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x564e3355fcd0 tx=0x7f7d34031ba0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.830+0000 7f7d438bc640 1 -- 172.21.15.38:0/2359152481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d3c100720 msgr2=0x7f7d3c06f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.830+0000 7f7d438bc640 1 --2- 172.21.15.38:0/2359152481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d3c100720 0x7f7d3c06f6b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c06a4c0 tx=0x7f7d2c00db30 comp rx=0 tx=0).stop 2024-05-21T15:38:35.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.832+0000 7f7d438bc640 1 -- 172.21.15.38:0/2359152481 shutdown_connections 2024-05-21T15:38:35.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.832+0000 7f7d438bc640 1 --2- 172.21.15.38:0/2359152481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d3c104070 0x7f7d3c072b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.832+0000 7f7d438bc640 1 --2- 172.21.15.38:0/2359152481 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7d280a2da0 0x7f7d280a52b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.832+0000 7f7d438bc640 1 --2- 172.21.15.38:0/2359152481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d3c101220 0x7f7d3c06fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.832+0000 7f7d438bc640 1 --2- 172.21.15.38:0/2359152481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d3c100720 0x7f7d3c06f6b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:35.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.832+0000 7f7d438bc640 1 -- 172.21.15.38:0/2359152481 >> 172.21.15.38:0/2359152481 conn(0x7f7d3c0fc320 msgr2=0x7f7d3c104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:35.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.834+0000 7f7d438bc640 1 -- 172.21.15.38:0/2359152481 shutdown_connections 2024-05-21T15:38:35.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:35.834+0000 7f7d438bc640 1 -- 172.21.15.38:0/2359152481 wait complete. 2024-05-21T15:38:35.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:35 smithi053 ceph-mon[36496]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 100 MiB/s wr, 17.39k op/s 2024-05-21T15:38:36.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:35 smithi045 ceph-mon[36409]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 100 MiB/s wr, 17.39k op/s 2024-05-21T15:38:36.856 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:38:36.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:38:36.856 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:38:37.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.035+0000 7f77c8944640 1 -- 172.21.15.38:0/2110248916 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc030e30 con 0x7f77c41078c0 2024-05-21T15:38:37.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.036+0000 7f77cbbaa640 1 -- 172.21.15.38:0/2110248916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77c41078c0 msgr2=0x7f77c4109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.036+0000 7f77cbbaa640 1 --2- 172.21.15.38:0/2110248916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77c41078c0 0x7f77c4109ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009be0 tx=0x7f77bc02a6d0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.036+0000 7f77cbbaa640 1 -- 172.21.15.38:0/2110248916 shutdown_connections 2024-05-21T15:38:37.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.036+0000 7f77cbbaa640 1 --2- 172.21.15.38:0/2110248916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77c41078c0 0x7f77c4109ce0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.036+0000 7f77cbbaa640 1 --2- 172.21.15.38:0/2110248916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77c4102bc0 0x7f77c4102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.036+0000 7f77cbbaa640 1 --2- 172.21.15.38:0/2110248916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c4101980 0x7f77c4102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.036+0000 7f77cbbaa640 1 -- 172.21.15.38:0/2110248916 >> 172.21.15.38:0/2110248916 conn(0x7f77c40fcb70 msgr2=0x7f77c40ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:37.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.036+0000 7f77cbbaa640 1 -- 172.21.15.38:0/2110248916 shutdown_connections 2024-05-21T15:38:37.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.036+0000 7f77cbbaa640 1 -- 172.21.15.38:0/2110248916 wait complete. 2024-05-21T15:38:37.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.037+0000 7f77cbbaa640 1 Processor -- start 2024-05-21T15:38:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.038+0000 7f77cbbaa640 1 -- start start 2024-05-21T15:38:37.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.038+0000 7f77cbbaa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77c4101980 0x7f77c406e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.038+0000 7f77cbbaa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c4102bc0 0x7f77c406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.038+0000 7f77cbbaa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77c41078c0 0x7f77c4071e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.038+0000 7f77c9946640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77c4101980 0x7f77c406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.038+0000 7f77c9946640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77c4101980 0x7f77c406e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:48982/0 (socket says 172.21.15.38:48982) 2024-05-21T15:38:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.039+0000 7f77c9145640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c4102bc0 0x7f77c406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.039+0000 7f77cbbaa640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f77c410c470 con 0x7f77c41078c0 2024-05-21T15:38:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.039+0000 7f77cbbaa640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77c410c2f0 con 0x7f77c4102bc0 2024-05-21T15:38:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.039+0000 7f77c9946640 1 -- 172.21.15.38:0/3762004332 learned_addr learned my addr 172.21.15.38:0/3762004332 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.039+0000 7f77cbbaa640 1 -- 172.21.15.38:0/3762004332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f77c410c5f0 con 0x7f77c4101980 2024-05-21T15:38:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.039+0000 7f77ca147640 1 --2- 172.21.15.38:0/3762004332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77c41078c0 0x7f77c4071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.039+0000 7f77ca147640 1 -- 172.21.15.38:0/3762004332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77c4101980 msgr2=0x7f77c406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.039+0000 7f77ca147640 1 --2- 172.21.15.38:0/3762004332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77c4101980 0x7f77c406e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.039+0000 7f77ca147640 1 -- 172.21.15.38:0/3762004332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c4102bc0 msgr2=0x7f77c406eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.039+0000 7f77ca147640 1 --2- 172.21.15.38:0/3762004332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c4102bc0 0x7f77c406eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.039+0000 7f77ca147640 1 -- 172.21.15.38:0/3762004332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c4072490 con 0x7f77c41078c0 2024-05-21T15:38:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.039+0000 7f77c9946640 1 --2- 172.21.15.38:0/3762004332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77c4101980 0x7f77c406e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:38:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.040+0000 7f77ca147640 1 --2- 172.21.15.38:0/3762004332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77c41078c0 0x7f77c4071e20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f77bc034040 tx=0x7f77bc0305b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:37.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.040+0000 7f77baffd640 1 -- 172.21.15.38:0/3762004332 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc041020 con 0x7f77c41078c0 2024-05-21T15:38:37.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.040+0000 7f77baffd640 1 -- 172.21.15.38:0/3762004332 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77bc037040 con 0x7f77c41078c0 2024-05-21T15:38:37.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.040+0000 7f77cbbaa640 1 -- 172.21.15.38:0/3762004332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c4072720 con 0x7f77c41078c0 2024-05-21T15:38:37.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.040+0000 7f77baffd640 1 -- 172.21.15.38:0/3762004332 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc036070 con 0x7f77c41078c0 2024-05-21T15:38:37.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.040+0000 7f77cbbaa640 1 -- 172.21.15.38:0/3762004332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c4072c10 con 0x7f77c41078c0 2024-05-21T15:38:37.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.044+0000 7f77d2a50740 1 Processor -- start 2024-05-21T15:38:37.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.044+0000 7f77baffd640 1 -- 172.21.15.38:0/3762004332 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f77bc044020 con 0x7f77c41078c0 2024-05-21T15:38:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.045+0000 7f77d2a50740 1 -- start start 2024-05-21T15:38:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.045+0000 7f77d2a50740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88085e410 0x55e880bfe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.045+0000 7f77d2a50740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880bfea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.045+0000 7f77d2a50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 0x55e880c05920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.045+0000 7f77d2a50740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e880b62990 con 0x55e88085e410 2024-05-21T15:38:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.045+0000 7f77d2a50740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e880b0a5b0 con 0x55e880835550 2024-05-21T15:38:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.045+0000 7f77d2a50740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e880b5e950 con 0x55e88083d730 2024-05-21T15:38:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.045+0000 7f77c9145640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88085e410 0x55e880bfe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.045+0000 7f77c9946640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880bfea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.045+0000 7f77c9145640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88085e410 0x55e880bfe4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35694/0 (socket says 172.21.15.38:35694) 2024-05-21T15:38:37.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.045+0000 7f77c9946640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880bfea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46128/0 (socket says 172.21.15.38:46128) 2024-05-21T15:38:37.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.045+0000 7f77c9145640 1 -- 172.21.15.38:0/1162401026 learned_addr learned my addr 172.21.15.38:0/1162401026 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:37.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.045+0000 7f77c9145640 1 --2- 172.21.15.38:0/1162401026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 0x55e880c05920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.046+0000 7f77baffd640 1 --2- 172.21.15.38:0/3762004332 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f77b00a2b90 0x7f77b00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.046+0000 7f77c9946640 1 -- 172.21.15.38:0/1162401026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 msgr2=0x55e880c05920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.046+0000 7f77c9946640 1 --2- 172.21.15.38:0/1162401026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 0x55e880c05920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.046+0000 7f77c9946640 1 -- 172.21.15.38:0/1162401026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88085e410 msgr2=0x55e880bfe4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.046+0000 7f77c9946640 1 --2- 172.21.15.38:0/1162401026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88085e410 0x55e880bfe4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.046+0000 7f77c9946640 1 -- 172.21.15.38:0/1162401026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88093e7e0 con 0x55e880835550 2024-05-21T15:38:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.046+0000 7f77c9145640 1 --2- 172.21.15.38:0/1162401026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 0x55e880c05920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:38:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.046+0000 7f77baffd640 1 -- 172.21.15.38:0/3762004332 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f77bc0efcc0 con 0x7f77c41078c0 2024-05-21T15:38:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.046+0000 7f77ca147640 1 --2- 172.21.15.38:0/3762004332 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f77b00a2b90 0x7f77b00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.046+0000 7f77c9946640 1 --2- 172.21.15.38:0/1162401026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880bfea20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f77c000bfd0 tx=0x7f77c00108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60e39928792b6930 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.047+0000 7f77affff640 1 -- 172.21.15.38:0/1162401026 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0005220 con 0x55e880835550 2024-05-21T15:38:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.047+0000 7f77ca147640 1 --2- 172.21.15.38:0/3762004332 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f77b00a2b90 0x7f77b00a50a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f77bc006010 tx=0x7f77bc0b8040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.047+0000 7f77affff640 1 -- 172.21.15.38:0/1162401026 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c0015070 con 0x55e880835550 2024-05-21T15:38:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.047+0000 7f77affff640 1 -- 172.21.15.38:0/1162401026 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0011a40 con 0x55e880835550 2024-05-21T15:38:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.048+0000 7f77d2a50740 1 -- 172.21.15.38:0/1162401026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 msgr2=0x55e880bfea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.048+0000 7f77d2a50740 1 --2- 172.21.15.38:0/1162401026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880bfea20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f77c000bfd0 tx=0x7f77c00108e0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.048+0000 7f77d2a50740 1 -- 172.21.15.38:0/1162401026 shutdown_connections 2024-05-21T15:38:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.048+0000 7f77d2a50740 1 --2- 172.21.15.38:0/1162401026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 0x55e880c05920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.048+0000 7f77d2a50740 1 --2- 172.21.15.38:0/1162401026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880bfea20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.048+0000 7f77d2a50740 1 --2- 172.21.15.38:0/1162401026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88085e410 0x55e880bfe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.048+0000 7f77d2a50740 1 -- 172.21.15.38:0/1162401026 >> 172.21.15.38:0/1162401026 conn(0x55e880852560 msgr2=0x55e880bf9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:37.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.050+0000 7f77d2a50740 1 -- 172.21.15.38:0/1162401026 shutdown_connections 2024-05-21T15:38:37.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.050+0000 7f77d2a50740 1 -- 172.21.15.38:0/1162401026 wait complete. 2024-05-21T15:38:37.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.051+0000 7f77d2a50740 1 -- start start 2024-05-21T15:38:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.051+0000 7f77d2a50740 1 Processor -- start 2024-05-21T15:38:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.052+0000 7f77d2a50740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e880835550 0x55e880c08410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.052+0000 7f77d2a50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 0x55e880c08950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.052+0000 7f77d2a50740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e88085e410 0x55e880c0f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.052+0000 7f77d2a50740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e880c063c0 con 0x55e880835550 2024-05-21T15:38:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.052+0000 7f77d2a50740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e880b1db40 con 0x55e88085e410 2024-05-21T15:38:37.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.052+0000 7f77d2a50740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e880c06540 con 0x55e88083d730 2024-05-21T15:38:37.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.052+0000 7f77d2a50740 10 client.0 _ll_register_callbacks cb 0x55e880b120f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:37.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.052+0000 7f77c9145640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e88085e410 0x55e880c0f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.052+0000 7f77c9145640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e88085e410 0x55e880c0f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46140/0 (socket says 172.21.15.38:46140) 2024-05-21T15:38:37.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.052+0000 7f77c9145640 1 -- 172.21.15.38:0/508814602 learned_addr learned my addr 172.21.15.38:0/508814602 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:37.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.052+0000 7f77c9145640 1 --2- 172.21.15.38:0/508814602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e880835550 0x55e880c08410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.053+0000 7f77c9145640 1 -- 172.21.15.38:0/508814602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 msgr2=0x55e880c08950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.053+0000 7f77c9145640 1 --2- 172.21.15.38:0/508814602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 0x55e880c08950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.053+0000 7f77c9145640 1 -- 172.21.15.38:0/508814602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e880835550 msgr2=0x55e880c08410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.053+0000 7f77c9145640 1 --2- 172.21.15.38:0/508814602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e880835550 0x55e880c08410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.053+0000 7f77c9145640 1 -- 172.21.15.38:0/508814602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88099ef70 con 0x55e88085e410 2024-05-21T15:38:37.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.053+0000 7f77c9145640 1 --2- 172.21.15.38:0/508814602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e88085e410 0x55e880c0f8b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f77b4013e90 tx=0x7f77b4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b33695f3bc85f98d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.053+0000 7f77ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:38:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.053+0000 7f77ad7fa640 1 -- 172.21.15.38:0/508814602 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4005220 con 0x55e88085e410 2024-05-21T15:38:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.054+0000 7f77d2a50740 1 -- 172.21.15.38:0/508814602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c0015070 con 0x55e88085e410 2024-05-21T15:38:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.054+0000 7f77ad7fa640 1 -- 172.21.15.38:0/508814602 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f77b40053c0 con 0x55e88085e410 2024-05-21T15:38:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.054+0000 7f77ad7fa640 1 -- 172.21.15.38:0/508814602 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f77b40055c0 con 0x55e88085e410 2024-05-21T15:38:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.054+0000 7f77d2a50740 10 client.26207 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.054+0000 7f77d2a50740 1 -- 172.21.15.38:0/508814602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e880836170 con 0x55e88085e410 2024-05-21T15:38:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.054+0000 7f77ad7fa640 1 -- 172.21.15.38:0/508814602 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c0015070 con 0x55e88085e410 2024-05-21T15:38:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.055+0000 7f77ad7fa640 1 -- 172.21.15.38:0/508814602 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f77b400f900 con 0x55e88085e410 2024-05-21T15:38:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.055+0000 7f77d2a50740 10 client.26207 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.055+0000 7f77d2a50740 10 client.26207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.055+0000 7f77d2a50740 20 client.26207 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.055+0000 7f77d2a50740 1 --2- 172.21.15.38:0/508814602 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e88081d1f0 0x55e880c162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.055+0000 7f77d2a50740 4 client.26207 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:38:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.055+0000 7f77d2a50740 1 -- 172.21.15.38:0/508814602 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e880b700c0 con 0x55e88081d1f0 2024-05-21T15:38:37.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.055+0000 7f77c9946640 1 --2- 172.21.15.38:0/508814602 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e88081d1f0 0x55e880c162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.056+0000 7f77ad7fa640 10 client.26207 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:37.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.056+0000 7f77c9946640 1 --2- 172.21.15.38:0/508814602 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e88081d1f0 0x55e880c162f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c9e21c8511e52 server_cookie=d317196556afb0db in_seq=0 out_seq=0 2024-05-21T15:38:37.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.056+0000 7f77ad7fa640 1 -- 172.21.15.38:0/508814602 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e880b700c0 con 0x55e88081d1f0 2024-05-21T15:38:37.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.056+0000 7f77ad7fa640 10 client.26207 handle_command_reply: tid=0 2024-05-21T15:38:37.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.056+0000 7f77d2a50740 1 client.26207 shutdown 2024-05-21T15:38:37.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.057+0000 7f77d2a50740 1 -- 172.21.15.38:0/508814602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e88085e410 msgr2=0x55e880c0f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.057+0000 7f77d2a50740 1 --2- 172.21.15.38:0/508814602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e88085e410 0x55e880c0f8b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f77b4013e90 tx=0x7f77b4009260 comp rx=0 tx=0).stop 2024-05-21T15:38:37.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.057+0000 7f77d2a50740 1 -- 172.21.15.38:0/508814602 shutdown_connections 2024-05-21T15:38:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.057+0000 7f77d2a50740 1 --2- 172.21.15.38:0/508814602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e88085e410 0x55e880c0f8b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.057+0000 7f77d2a50740 1 --2- 172.21.15.38:0/508814602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 0x55e880c08950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.057+0000 7f77d2a50740 1 --2- 172.21.15.38:0/508814602 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e88081d1f0 0x55e880c162f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.057+0000 7f77d2a50740 1 --2- 172.21.15.38:0/508814602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e880835550 0x55e880c08410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.057+0000 7f77d2a50740 1 -- 172.21.15.38:0/508814602 >> 172.21.15.38:0/508814602 conn(0x55e880852560 msgr2=0x55e880bf8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.057+0000 7f77ad7fa640 0 client.26207 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.057+0000 7f77d2a50740 1 -- 172.21.15.38:0/508814602 shutdown_connections 2024-05-21T15:38:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.057+0000 7f77d2a50740 1 -- 172.21.15.38:0/508814602 wait complete. 2024-05-21T15:38:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.057+0000 7f77d2a50740 20 client.26207 trim_cache size 0 max 16384 2024-05-21T15:38:37.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.077+0000 7f77d2a50740 1 Processor -- start 2024-05-21T15:38:37.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.077+0000 7f77d2a50740 1 -- start start 2024-05-21T15:38:37.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.077+0000 7f77d2a50740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880bf9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.077+0000 7f77d2a50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 0x55e880bf9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.077+0000 7f77d2a50740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88085e410 0x55e880c0f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.077+0000 7f77d2a50740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41c1ab0 con 0x55e88085e410 2024-05-21T15:38:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.077+0000 7f77d2a50740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41c36e0 con 0x55e880835550 2024-05-21T15:38:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.077+0000 7f77d2a50740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41c2180 con 0x55e88083d730 2024-05-21T15:38:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.077+0000 7f77c9145640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88085e410 0x55e880c0f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.077+0000 7f77c9145640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880bf9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.077+0000 7f77c9946640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 0x55e880bf9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.077+0000 7f77c9145640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880bf9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46152/0 (socket says 172.21.15.38:46152) 2024-05-21T15:38:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.077+0000 7f77c9145640 1 -- 172.21.15.38:0/3277365591 learned_addr learned my addr 172.21.15.38:0/3277365591 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.077+0000 7f77c9145640 1 -- 172.21.15.38:0/3277365591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 msgr2=0x55e880bf9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.078+0000 7f77c9145640 1 --2- 172.21.15.38:0/3277365591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 0x55e880bf9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.078+0000 7f77c9145640 1 -- 172.21.15.38:0/3277365591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88085e410 msgr2=0x55e880c0f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.078+0000 7f77c9145640 1 --2- 172.21.15.38:0/3277365591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88085e410 0x55e880c0f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.078+0000 7f77c9145640 1 -- 172.21.15.38:0/3277365591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e880c07490 con 0x55e880835550 2024-05-21T15:38:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.078+0000 7f77c9145640 1 --2- 172.21.15.38:0/3277365591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880bf9750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f77b4010010 tx=0x7f77b4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=317ed3afb4fcb54c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.078+0000 7f77af7fe640 1 -- 172.21.15.38:0/3277365591 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b400f690 con 0x55e880835550 2024-05-21T15:38:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.078+0000 7f77af7fe640 1 -- 172.21.15.38:0/3277365591 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b400f830 con 0x55e880835550 2024-05-21T15:38:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.078+0000 7f77af7fe640 1 -- 172.21.15.38:0/3277365591 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b401f460 con 0x55e880835550 2024-05-21T15:38:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.079+0000 7f77d2a50740 1 -- 172.21.15.38:0/3277365591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 msgr2=0x55e880bf9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.079+0000 7f77d2a50740 1 --2- 172.21.15.38:0/3277365591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880bf9750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f77b4010010 tx=0x7f77b4000bc0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.079+0000 7f77d2a50740 1 -- 172.21.15.38:0/3277365591 shutdown_connections 2024-05-21T15:38:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.079+0000 7f77d2a50740 1 --2- 172.21.15.38:0/3277365591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88085e410 0x55e880c0f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.079+0000 7f77d2a50740 1 --2- 172.21.15.38:0/3277365591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88083d730 0x55e880bf9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.079+0000 7f77d2a50740 1 --2- 172.21.15.38:0/3277365591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880bf9750 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.079+0000 7f77d2a50740 1 -- 172.21.15.38:0/3277365591 >> 172.21.15.38:0/3277365591 conn(0x55e880852560 msgr2=0x55e880c027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:37.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.079+0000 7f77d2a50740 1 -- 172.21.15.38:0/3277365591 shutdown_connections 2024-05-21T15:38:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.079+0000 7f77d2a50740 1 -- 172.21.15.38:0/3277365591 wait complete. 2024-05-21T15:38:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.079+0000 7f77d2a50740 1 -- start start 2024-05-21T15:38:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77d2a50740 1 Processor -- start 2024-05-21T15:38:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77d2a50740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880c0fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77d2a50740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88083d730 0x55e880c0c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77d2a50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88085e410 0x55e880c0ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77d2a50740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41c2f50 con 0x55e88083d730 2024-05-21T15:38:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77d2a50740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41c2850 con 0x55e880835550 2024-05-21T15:38:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77d2a50740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41bfa90 con 0x55e88085e410 2024-05-21T15:38:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77d2a50740 10 client.0 _ll_register_callbacks cb 0x7f77b40130e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77c9946640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88083d730 0x55e880c0c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77c9145640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880c0fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77c9946640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88083d730 0x55e880c0c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35716/0 (socket says 172.21.15.38:35716) 2024-05-21T15:38:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77c9946640 1 -- 172.21.15.38:0/2205061869 learned_addr learned my addr 172.21.15.38:0/2205061869 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77c9145640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880c0fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46154/0 (socket says 172.21.15.38:46154) 2024-05-21T15:38:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77c9145640 1 --2- 172.21.15.38:0/2205061869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88085e410 0x55e880c0ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77c9946640 1 -- 172.21.15.38:0/2205061869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88085e410 msgr2=0x55e880c0ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77c9946640 1 --2- 172.21.15.38:0/2205061869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88085e410 0x55e880c0ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77c9946640 1 -- 172.21.15.38:0/2205061869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 msgr2=0x55e880c0fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77c9946640 1 --2- 172.21.15.38:0/2205061869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880c0fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.080+0000 7f77c9946640 1 -- 172.21.15.38:0/2205061869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e880c10380 con 0x55e88083d730 2024-05-21T15:38:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.081+0000 7f77c9946640 1 --2- 172.21.15.38:0/2205061869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88083d730 0x55e880c0c920 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f77c000b6c0 tx=0x7f77c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c1647a0c55144 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.081+0000 7f77ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.081+0000 7f77ad7fa640 1 -- 172.21.15.38:0/2205061869 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c002c070 con 0x55e88083d730 2024-05-21T15:38:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.081+0000 7f77ad7fa640 1 -- 172.21.15.38:0/2205061869 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f77c0005450 con 0x55e88083d730 2024-05-21T15:38:37.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.081+0000 7f77d2a50740 1 -- 172.21.15.38:0/2205061869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77b400f830 con 0x55e88083d730 2024-05-21T15:38:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.081+0000 7f77ad7fa640 1 -- 172.21.15.38:0/2205061869 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f77c000bb60 con 0x55e88083d730 2024-05-21T15:38:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.081+0000 7f77d2a50740 10 client.16557 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.081+0000 7f77d2a50740 1 -- 172.21.15.38:0/2205061869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e880c7d510 con 0x55e88083d730 2024-05-21T15:38:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.082+0000 7f77ad7fa640 1 -- 172.21.15.38:0/2205061869 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c0015030 con 0x55e88083d730 2024-05-21T15:38:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.082+0000 7f77ad7fa640 1 -- 172.21.15.38:0/2205061869 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f77c002f030 con 0x55e88083d730 2024-05-21T15:38:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.082+0000 7f77d2a50740 10 client.16557 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.082+0000 7f77d2a50740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.082+0000 7f77d2a50740 20 client.16557 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.082+0000 7f77d2a50740 1 --2- 172.21.15.38:0/2205061869 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8807f2630 0x55e880c7fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.082+0000 7f77d2a50740 4 client.16557 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:38:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.082+0000 7f77d2a50740 1 -- 172.21.15.38:0/2205061869 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f77b400f830 con 0x55e8807f2630 2024-05-21T15:38:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.082+0000 7f77c9145640 1 --2- 172.21.15.38:0/2205061869 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8807f2630 0x55e880c7fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.083+0000 7f77c9145640 1 --2- 172.21.15.38:0/2205061869 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8807f2630 0x55e880c7fd50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a378df401b54e43 server_cookie=ccbecc88d1d49d1c in_seq=0 out_seq=0 2024-05-21T15:38:37.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.083+0000 7f77ad7fa640 10 client.16557 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.084+0000 7f77ad7fa640 1 -- 172.21.15.38:0/2205061869 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f77b400f830 con 0x55e8807f2630 2024-05-21T15:38:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.084+0000 7f77ad7fa640 10 client.16557 handle_command_reply: tid=0 2024-05-21T15:38:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.084+0000 7f77d2a50740 1 client.16557 shutdown 2024-05-21T15:38:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.085+0000 7f77d2a50740 1 -- 172.21.15.38:0/2205061869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88083d730 msgr2=0x55e880c0c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.085+0000 7f77d2a50740 1 --2- 172.21.15.38:0/2205061869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88083d730 0x55e880c0c920 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f77c000b6c0 tx=0x7f77c0000bc0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.085+0000 7f77d2a50740 1 -- 172.21.15.38:0/2205061869 shutdown_connections 2024-05-21T15:38:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.085+0000 7f77d2a50740 1 --2- 172.21.15.38:0/2205061869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e88085e410 0x55e880c0ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.085+0000 7f77d2a50740 1 --2- 172.21.15.38:0/2205061869 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8807f2630 0x55e880c7fd50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.085+0000 7f77d2a50740 1 --2- 172.21.15.38:0/2205061869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e88083d730 0x55e880c0c920 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.085+0000 7f77d2a50740 1 --2- 172.21.15.38:0/2205061869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e880835550 0x55e880c0fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.085+0000 7f77d2a50740 1 -- 172.21.15.38:0/2205061869 >> 172.21.15.38:0/2205061869 conn(0x55e880852560 msgr2=0x55e880bf9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:37.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.085+0000 7f77ad7fa640 0 client.16557 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:37.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.085+0000 7f77d2a50740 1 -- 172.21.15.38:0/2205061869 shutdown_connections 2024-05-21T15:38:37.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.085+0000 7f77d2a50740 1 -- 172.21.15.38:0/2205061869 wait complete. 2024-05-21T15:38:37.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.085+0000 7f77d2a50740 20 client.16557 trim_cache size 0 max 16384 2024-05-21T15:38:37.090 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:38:37.090 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-05-21T15:38:37.090 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "ec33a0fc-f2ee-475d-a600-6e3b842c78da", 2024-05-21T15:38:37.091 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-05-21T15:38:37.091 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:38:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.086+0000 7f77cbbaa640 1 -- 172.21.15.38:0/3762004332 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f77b00a2b90 msgr2=0x7f77b00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.086+0000 7f77cbbaa640 1 --2- 172.21.15.38:0/3762004332 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f77b00a2b90 0x7f77b00a50a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f77bc006010 tx=0x7f77bc0b8040 comp rx=0 tx=0).stop 2024-05-21T15:38:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.086+0000 7f77cbbaa640 1 -- 172.21.15.38:0/3762004332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77c41078c0 msgr2=0x7f77c4071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.086+0000 7f77cbbaa640 1 --2- 172.21.15.38:0/3762004332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77c41078c0 0x7f77c4071e20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f77bc034040 tx=0x7f77bc0305b0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.086+0000 7f77cbbaa640 1 -- 172.21.15.38:0/3762004332 shutdown_connections 2024-05-21T15:38:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.086+0000 7f77cbbaa640 1 --2- 172.21.15.38:0/3762004332 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f77b00a2b90 0x7f77b00a50a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.086+0000 7f77cbbaa640 1 --2- 172.21.15.38:0/3762004332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77c41078c0 0x7f77c4071e20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.086+0000 7f77cbbaa640 1 --2- 172.21.15.38:0/3762004332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c4102bc0 0x7f77c406eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.086+0000 7f77cbbaa640 1 --2- 172.21.15.38:0/3762004332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77c4101980 0x7f77c406e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.086+0000 7f77cbbaa640 1 -- 172.21.15.38:0/3762004332 >> 172.21.15.38:0/3762004332 conn(0x7f77c40fcb70 msgr2=0x7f77c40fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.086+0000 7f77cbbaa640 1 -- 172.21.15.38:0/3762004332 shutdown_connections 2024-05-21T15:38:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.086+0000 7f77cbbaa640 1 -- 172.21.15.38:0/3762004332 wait complete. 2024-05-21T15:38:37.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:38:37.100 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:38:37.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.236+0000 7f9e6c2c3640 1 -- 172.21.15.38:0/37978819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e64101990 msgr2=0x7f9e64102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.236+0000 7f9e6c2c3640 1 --2- 172.21.15.38:0/37978819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e64101990 0x7f9e64102690 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009900 tx=0x7f9e6002a410 comp rx=0 tx=0).stop 2024-05-21T15:38:37.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.236+0000 7f9e6c2c3640 1 -- 172.21.15.38:0/37978819 shutdown_connections 2024-05-21T15:38:37.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.236+0000 7f9e6c2c3640 1 --2- 172.21.15.38:0/37978819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e641078d0 0x7f9e64109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.236+0000 7f9e6c2c3640 1 --2- 172.21.15.38:0/37978819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e64102bd0 0x7f9e64102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.236+0000 7f9e6c2c3640 1 --2- 172.21.15.38:0/37978819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e64101990 0x7f9e64102690 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.236+0000 7f9e6c2c3640 1 -- 172.21.15.38:0/37978819 >> 172.21.15.38:0/37978819 conn(0x7f9e640fcb80 msgr2=0x7f9e640ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:37.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.237+0000 7f9e6c2c3640 1 -- 172.21.15.38:0/37978819 shutdown_connections 2024-05-21T15:38:37.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.237+0000 7f9e6c2c3640 1 -- 172.21.15.38:0/37978819 wait complete. 2024-05-21T15:38:37.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.237+0000 7f9e6c2c3640 1 Processor -- start 2024-05-21T15:38:37.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.237+0000 7f9e6c2c3640 1 -- start start 2024-05-21T15:38:37.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6c2c3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e64101990 0x7f9e64198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6c2c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e64102bd0 0x7f9e641986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6c2c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e641078d0 0x7f9e6419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6c2c3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6410d5c0 con 0x7f9e64102bd0 2024-05-21T15:38:37.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6c2c3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6410d440 con 0x7f9e641078d0 2024-05-21T15:38:37.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6c2c3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6410d740 con 0x7f9e64101990 2024-05-21T15:38:37.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6985e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e64102bd0 0x7f9e641986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6985e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e64102bd0 0x7f9e641986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35740/0 (socket says 172.21.15.38:35740) 2024-05-21T15:38:37.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6985e640 1 -- 172.21.15.38:0/2075830614 learned_addr learned my addr 172.21.15.38:0/2075830614 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:37.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6a05f640 1 --2- 172.21.15.38:0/2075830614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e64101990 0x7f9e64198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6a860640 1 --2- 172.21.15.38:0/2075830614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e641078d0 0x7f9e6419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6985e640 1 -- 172.21.15.38:0/2075830614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e64101990 msgr2=0x7f9e64198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6985e640 1 --2- 172.21.15.38:0/2075830614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e64101990 0x7f9e64198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6985e640 1 -- 172.21.15.38:0/2075830614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e641078d0 msgr2=0x7f9e6419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6985e640 1 --2- 172.21.15.38:0/2075830614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e641078d0 0x7f9e6419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.238+0000 7f9e6985e640 1 -- 172.21.15.38:0/2075830614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e64073e00 con 0x7f9e64102bd0 2024-05-21T15:38:37.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.239+0000 7f9e6985e640 1 --2- 172.21.15.38:0/2075830614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e64102bd0 0x7f9e641986c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400ea60 tx=0x7f9e5400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:37.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.239+0000 7f9e5b7fe640 1 -- 172.21.15.38:0/2075830614 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5400cde0 con 0x7f9e64102bd0 2024-05-21T15:38:37.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.239+0000 7f9e6c2c3640 1 -- 172.21.15.38:0/2075830614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e640740f0 con 0x7f9e64102bd0 2024-05-21T15:38:37.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.239+0000 7f9e6c2c3640 1 -- 172.21.15.38:0/2075830614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e64074690 con 0x7f9e64102bd0 2024-05-21T15:38:37.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.239+0000 7f9e5b7fe640 1 -- 172.21.15.38:0/2075830614 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e540035f0 con 0x7f9e64102bd0 2024-05-21T15:38:37.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.239+0000 7f9e5b7fe640 1 -- 172.21.15.38:0/2075830614 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54010670 con 0x7f9e64102bd0 2024-05-21T15:38:37.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.242+0000 7f9e73166740 1 Processor -- start 2024-05-21T15:38:37.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.242+0000 7f9e73166740 1 -- start start 2024-05-21T15:38:37.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.242+0000 7f9e5b7fe640 1 -- 172.21.15.38:0/2075830614 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9e54010810 con 0x7f9e64102bd0 2024-05-21T15:38:37.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.242+0000 7f9e73166740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01ca390 0x55a6d0569bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.242+0000 7f9e73166740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d056a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.242+0000 7f9e73166740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01a96b0 0x55a6d0571030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.242+0000 7f9e73166740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a6d04ced00 con 0x55a6d01a14d0 2024-05-21T15:38:37.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.242+0000 7f9e73166740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a6d04764c0 con 0x55a6d01a96b0 2024-05-21T15:38:37.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.242+0000 7f9e73166740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a6d04cacc0 con 0x55a6d01ca390 2024-05-21T15:38:37.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.242+0000 7f9e6a05f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d056a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.242+0000 7f9e6a05f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d056a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35752/0 (socket says 172.21.15.38:35752) 2024-05-21T15:38:37.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.242+0000 7f9e6a05f640 1 -- 172.21.15.38:0/2582272150 learned_addr learned my addr 172.21.15.38:0/2582272150 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:37.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.242+0000 7f9e6985e640 1 --2- 172.21.15.38:0/2582272150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01a96b0 0x55a6d0571030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.243+0000 7f9e6a860640 1 --2- 172.21.15.38:0/2582272150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01ca390 0x55a6d0569bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.243+0000 7f9e6a05f640 1 -- 172.21.15.38:0/2582272150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01ca390 msgr2=0x55a6d0569bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.243+0000 7f9e6a05f640 1 --2- 172.21.15.38:0/2582272150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01ca390 0x55a6d0569bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.243+0000 7f9e6a05f640 1 -- 172.21.15.38:0/2582272150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01a96b0 msgr2=0x55a6d0571030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.243+0000 7f9e6a05f640 1 --2- 172.21.15.38:0/2582272150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01a96b0 0x55a6d0571030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.243+0000 7f9e6a05f640 1 -- 172.21.15.38:0/2582272150 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6d02ac7d0 con 0x55a6d01a14d0 2024-05-21T15:38:37.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.243+0000 7f9e6985e640 1 --2- 172.21.15.38:0/2582272150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01a96b0 0x55a6d0571030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:38:37.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.243+0000 7f9e6a05f640 1 --2- 172.21.15.38:0/2582272150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d056a130 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9e64101260 tx=0x7f9e60031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d09db711025f9c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:37.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.243+0000 7f9e5b7fe640 1 --2- 172.21.15.38:0/2075830614 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9e500a2c10 0x7f9e500a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.243+0000 7f9e58bfe640 1 -- 172.21.15.38:0/2582272150 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e60035030 con 0x55a6d01a14d0 2024-05-21T15:38:37.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.243+0000 7f9e58bfe640 1 -- 172.21.15.38:0/2582272150 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e60040070 con 0x55a6d01a14d0 2024-05-21T15:38:37.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.243+0000 7f9e58bfe640 1 -- 172.21.15.38:0/2582272150 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e600038e0 con 0x55a6d01a14d0 2024-05-21T15:38:37.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.244+0000 7f9e6a860640 1 --2- 172.21.15.38:0/2075830614 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9e500a2c10 0x7f9e500a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.244+0000 7f9e5b7fe640 1 -- 172.21.15.38:0/2075830614 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9e540cf270 con 0x7f9e64102bd0 2024-05-21T15:38:37.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.245+0000 7f9e73166740 1 -- 172.21.15.38:0/2582272150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 msgr2=0x55a6d056a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.245+0000 7f9e73166740 1 --2- 172.21.15.38:0/2582272150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d056a130 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9e64101260 tx=0x7f9e60031650 comp rx=0 tx=0).stop 2024-05-21T15:38:37.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.245+0000 7f9e6a860640 1 --2- 172.21.15.38:0/2075830614 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9e500a2c10 0x7f9e500a5120 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55a6d056ad80 tx=0x7f9e5c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:37.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.245+0000 7f9e73166740 1 -- 172.21.15.38:0/2582272150 shutdown_connections 2024-05-21T15:38:37.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.245+0000 7f9e73166740 1 --2- 172.21.15.38:0/2582272150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01a96b0 0x55a6d0571030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.245+0000 7f9e73166740 1 --2- 172.21.15.38:0/2582272150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d056a130 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.245+0000 7f9e73166740 1 --2- 172.21.15.38:0/2582272150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01ca390 0x55a6d0569bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.245+0000 7f9e73166740 1 -- 172.21.15.38:0/2582272150 >> 172.21.15.38:0/2582272150 conn(0x55a6d01be4e0 msgr2=0x55a6d0565540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:37.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.245+0000 7f9e73166740 1 -- 172.21.15.38:0/2582272150 shutdown_connections 2024-05-21T15:38:37.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.245+0000 7f9e73166740 1 -- 172.21.15.38:0/2582272150 wait complete. 2024-05-21T15:38:37.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.245+0000 7f9e73166740 1 -- start start 2024-05-21T15:38:37.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.245+0000 7f9e73166740 1 Processor -- start 2024-05-21T15:38:37.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e73166740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d0573ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e73166740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 0x55a6d0574010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e73166740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01ca390 0x55a6d057af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e73166740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a6d0571a20 con 0x55a6d01a14d0 2024-05-21T15:38:37.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e73166740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a6d0489b90 con 0x55a6d01ca390 2024-05-21T15:38:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e73166740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a6d0571ba0 con 0x55a6d01a96b0 2024-05-21T15:38:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e6a05f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d0573ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e6a05f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d0573ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35754/0 (socket says 172.21.15.38:35754) 2024-05-21T15:38:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e6a05f640 1 -- 172.21.15.38:0/1151798941 learned_addr learned my addr 172.21.15.38:0/1151798941 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e73166740 10 client.0 _ll_register_callbacks cb 0x55a6d04c5da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e6a860640 1 --2- 172.21.15.38:0/1151798941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01ca390 0x55a6d057af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e6985e640 1 --2- 172.21.15.38:0/1151798941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 0x55a6d0574010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e6a05f640 1 -- 172.21.15.38:0/1151798941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 msgr2=0x55a6d0574010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e6a05f640 1 --2- 172.21.15.38:0/1151798941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 0x55a6d0574010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e6a05f640 1 -- 172.21.15.38:0/1151798941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01ca390 msgr2=0x55a6d057af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e6a05f640 1 --2- 172.21.15.38:0/1151798941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01ca390 0x55a6d057af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e6a05f640 1 -- 172.21.15.38:0/1151798941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6d01e23f0 con 0x55a6d01a14d0 2024-05-21T15:38:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e6985e640 1 --2- 172.21.15.38:0/1151798941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 0x55a6d0574010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:38:37.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e6a05f640 1 --2- 172.21.15.38:0/1151798941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d0573ad0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009900 tx=0x7f9e60005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7328cc0faa38a959 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:37.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.246+0000 7f9e4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:37.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.247+0000 7f9e4dffb640 1 -- 172.21.15.38:0/1151798941 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e60056060 con 0x55a6d01a14d0 2024-05-21T15:38:37.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.247+0000 7f9e4dffb640 1 -- 172.21.15.38:0/1151798941 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9e600319a0 con 0x55a6d01a14d0 2024-05-21T15:38:37.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.247+0000 7f9e73166740 1 -- 172.21.15.38:0/1151798941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e60040070 con 0x55a6d01a14d0 2024-05-21T15:38:37.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.247+0000 7f9e4dffb640 1 -- 172.21.15.38:0/1151798941 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e60035070 con 0x55a6d01a14d0 2024-05-21T15:38:37.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.247+0000 7f9e73166740 10 client.16569 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:37.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.247+0000 7f9e73166740 1 -- 172.21.15.38:0/1151798941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a6d01a20f0 con 0x55a6d01a14d0 2024-05-21T15:38:37.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.248+0000 7f9e4dffb640 1 -- 172.21.15.38:0/1151798941 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e60040070 con 0x55a6d01a14d0 2024-05-21T15:38:37.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.248+0000 7f9e4dffb640 1 -- 172.21.15.38:0/1151798941 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f9e6005c430 con 0x55a6d01a14d0 2024-05-21T15:38:37.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.248+0000 7f9e73166740 10 client.16569 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:37.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.248+0000 7f9e73166740 10 client.16569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:37.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.248+0000 7f9e73166740 20 client.16569 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:37.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.248+0000 7f9e73166740 1 --2- 172.21.15.38:0/1151798941 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a6d0188f60 0x55a6d0581400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.248+0000 7f9e73166740 4 client.16569 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.248+0000 7f9e73166740 1 -- 172.21.15.38:0/1151798941 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6d04dc430 con 0x55a6d0188f60 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.248+0000 7f9e6985e640 1 --2- 172.21.15.38:0/1151798941 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a6d0188f60 0x55a6d0581400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.248+0000 7f9e4dffb640 10 client.16569 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.249+0000 7f9e6985e640 1 --2- 172.21.15.38:0/1151798941 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a6d0188f60 0x55a6d0581400 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b18a7fff319f40a server_cookie=e96045543951c7cd in_seq=0 out_seq=0 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.249+0000 7f9e4dffb640 1 -- 172.21.15.38:0/1151798941 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a6d04dc430 con 0x55a6d0188f60 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.249+0000 7f9e4dffb640 10 client.16569 handle_command_reply: tid=0 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.249+0000 7f9e73166740 1 client.16569 shutdown 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.249+0000 7f9e73166740 1 -- 172.21.15.38:0/1151798941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 msgr2=0x55a6d0573ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.249+0000 7f9e73166740 1 --2- 172.21.15.38:0/1151798941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d0573ad0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009900 tx=0x7f9e60005970 comp rx=0 tx=0).stop 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.249+0000 7f9e73166740 1 -- 172.21.15.38:0/1151798941 shutdown_connections 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.249+0000 7f9e73166740 1 --2- 172.21.15.38:0/1151798941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01ca390 0x55a6d057af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.249+0000 7f9e73166740 1 --2- 172.21.15.38:0/1151798941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 0x55a6d0574010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.249+0000 7f9e73166740 1 --2- 172.21.15.38:0/1151798941 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a6d0188f60 0x55a6d0581400 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.250+0000 7f9e4dffb640 0 client.16569 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:37.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.250+0000 7f9e73166740 1 --2- 172.21.15.38:0/1151798941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d0573ad0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.250+0000 7f9e73166740 1 -- 172.21.15.38:0/1151798941 >> 172.21.15.38:0/1151798941 conn(0x55a6d01be4e0 msgr2=0x55a6d05645c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:37.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.250+0000 7f9e73166740 1 -- 172.21.15.38:0/1151798941 shutdown_connections 2024-05-21T15:38:37.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.250+0000 7f9e73166740 1 -- 172.21.15.38:0/1151798941 wait complete. 2024-05-21T15:38:37.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.250+0000 7f9e73166740 20 client.16569 trim_cache size 0 max 16384 2024-05-21T15:38:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e73166740 1 Processor -- start 2024-05-21T15:38:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e73166740 1 -- start start 2024-05-21T15:38:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e73166740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d0188f60 0x55a6d056e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e73166740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01a14d0 0x55a6d057af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e73166740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 0x55a6d0572800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e73166740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4800e5b0 con 0x55a6d0188f60 2024-05-21T15:38:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e73166740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e480101e0 con 0x55a6d01a14d0 2024-05-21T15:38:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e73166740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4800ec80 con 0x55a6d01a96b0 2024-05-21T15:38:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e6a05f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d0188f60 0x55a6d056e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e6a05f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d0188f60 0x55a6d056e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35764/0 (socket says 172.21.15.38:35764) 2024-05-21T15:38:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e6a05f640 1 -- 172.21.15.38:0/2408320082 learned_addr learned my addr 172.21.15.38:0/2408320082 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e6a860640 1 --2- 172.21.15.38:0/2408320082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 0x55a6d0572800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e6985e640 1 --2- 172.21.15.38:0/2408320082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01a14d0 0x55a6d057af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e6a05f640 1 -- 172.21.15.38:0/2408320082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 msgr2=0x55a6d0572800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e6a05f640 1 --2- 172.21.15.38:0/2408320082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 0x55a6d0572800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e6a05f640 1 -- 172.21.15.38:0/2408320082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01a14d0 msgr2=0x55a6d057af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e6a05f640 1 --2- 172.21.15.38:0/2408320082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01a14d0 0x55a6d057af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e6a05f640 1 -- 172.21.15.38:0/2408320082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6d05696c0 con 0x55a6d0188f60 2024-05-21T15:38:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.270+0000 7f9e6985e640 1 --2- 172.21.15.38:0/2408320082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01a14d0 0x55a6d057af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:38:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.271+0000 7f9e6a05f640 1 --2- 172.21.15.38:0/2408320082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d0188f60 0x55a6d056e560 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9e60005940 tx=0x7f9e60000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba12740d3aefd27 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.271+0000 7f9e4ffff640 1 -- 172.21.15.38:0/2408320082 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e60056070 con 0x55a6d0188f60 2024-05-21T15:38:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.271+0000 7f9e4ffff640 1 -- 172.21.15.38:0/2408320082 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e60040040 con 0x55a6d0188f60 2024-05-21T15:38:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.271+0000 7f9e4ffff640 1 -- 172.21.15.38:0/2408320082 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6003c440 con 0x55a6d0188f60 2024-05-21T15:38:37.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.272+0000 7f9e73166740 1 -- 172.21.15.38:0/2408320082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d0188f60 msgr2=0x55a6d056e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.272+0000 7f9e73166740 1 --2- 172.21.15.38:0/2408320082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d0188f60 0x55a6d056e560 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9e60005940 tx=0x7f9e60000bc0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.272+0000 7f9e73166740 1 -- 172.21.15.38:0/2408320082 shutdown_connections 2024-05-21T15:38:37.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.272+0000 7f9e73166740 1 --2- 172.21.15.38:0/2408320082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 0x55a6d0572800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.272+0000 7f9e73166740 1 --2- 172.21.15.38:0/2408320082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d01a14d0 0x55a6d057af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.272+0000 7f9e73166740 1 --2- 172.21.15.38:0/2408320082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d0188f60 0x55a6d056e560 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.272+0000 7f9e73166740 1 -- 172.21.15.38:0/2408320082 >> 172.21.15.38:0/2408320082 conn(0x55a6d01be4e0 msgr2=0x55a6d05720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:37.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.272+0000 7f9e73166740 1 -- 172.21.15.38:0/2408320082 shutdown_connections 2024-05-21T15:38:37.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.272+0000 7f9e73166740 1 -- 172.21.15.38:0/2408320082 wait complete. 2024-05-21T15:38:37.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.272+0000 7f9e73166740 1 -- start start 2024-05-21T15:38:37.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.272+0000 7f9e73166740 1 Processor -- start 2024-05-21T15:38:37.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e73166740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d0188f60 0x55a6d0579980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e73166740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d0579ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e73166740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 0x55a6d057a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e73166740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4800fa50 con 0x55a6d01a14d0 2024-05-21T15:38:37.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e73166740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4800f350 con 0x55a6d0188f60 2024-05-21T15:38:37.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e73166740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4800c590 con 0x55a6d01a96b0 2024-05-21T15:38:37.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e73166740 10 client.0 _ll_register_callbacks cb 0x7f9e5c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:37.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e6985e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d0579ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e6985e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d0579ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35774/0 (socket says 172.21.15.38:35774) 2024-05-21T15:38:37.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e6985e640 1 -- 172.21.15.38:0/2618086627 learned_addr learned my addr 172.21.15.38:0/2618086627 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:37.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e6a05f640 1 --2- 172.21.15.38:0/2618086627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d0188f60 0x55a6d0579980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e6a860640 1 --2- 172.21.15.38:0/2618086627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 0x55a6d057a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e6985e640 1 -- 172.21.15.38:0/2618086627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 msgr2=0x55a6d057a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e6985e640 1 --2- 172.21.15.38:0/2618086627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 0x55a6d057a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e6985e640 1 -- 172.21.15.38:0/2618086627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d0188f60 msgr2=0x55a6d0579980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:37.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e6985e640 1 --2- 172.21.15.38:0/2618086627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d0188f60 0x55a6d0579980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e6985e640 1 -- 172.21.15.38:0/2618086627 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6d05e7870 con 0x55a6d01a14d0 2024-05-21T15:38:37.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e6985e640 1 --2- 172.21.15.38:0/2618086627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d0579ec0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9e54098450 tx=0x7f9e5409b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b400561134269b63 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:37.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.273+0000 7f9e4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:37.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.274+0000 7f9e4dffb640 1 -- 172.21.15.38:0/2618086627 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e540a46f0 con 0x55a6d01a14d0 2024-05-21T15:38:37.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.274+0000 7f9e4dffb640 1 -- 172.21.15.38:0/2618086627 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9e54099d80 con 0x55a6d01a14d0 2024-05-21T15:38:37.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.274+0000 7f9e73166740 1 -- 172.21.15.38:0/2618086627 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e60040040 con 0x55a6d01a14d0 2024-05-21T15:38:37.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.274+0000 7f9e4dffb640 1 -- 172.21.15.38:0/2618086627 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e540980f0 con 0x55a6d01a14d0 2024-05-21T15:38:37.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.274+0000 7f9e73166740 10 client.16575 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:37.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.274+0000 7f9e73166740 1 -- 172.21.15.38:0/2618086627 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a6d05e8ae0 con 0x55a6d01a14d0 2024-05-21T15:38:37.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.274+0000 7f9e4dffb640 1 -- 172.21.15.38:0/2618086627 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e60040040 con 0x55a6d01a14d0 2024-05-21T15:38:37.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.274+0000 7f9e4dffb640 1 -- 172.21.15.38:0/2618086627 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f9e54004190 con 0x55a6d01a14d0 2024-05-21T15:38:37.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.274+0000 7f9e73166740 10 client.16575 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:37.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.274+0000 7f9e73166740 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:37.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.274+0000 7f9e73166740 20 client.16575 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:37.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.274+0000 7f9e73166740 1 --2- 172.21.15.38:0/2618086627 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a6d01ca390 0x55a6d05eb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:37.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.274+0000 7f9e73166740 4 client.16575 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:38:37.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.274+0000 7f9e73166740 1 -- 172.21.15.38:0/2618086627 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a6d04d6780 con 0x55a6d01ca390 2024-05-21T15:38:37.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e6a05f640 1 --2- 172.21.15.38:0/2618086627 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a6d01ca390 0x55a6d05eb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:37.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e6a05f640 1 --2- 172.21.15.38:0/2618086627 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a6d01ca390 0x55a6d05eb5e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=919272dd851eed64 server_cookie=d48481a33ca78e3e in_seq=0 out_seq=0 2024-05-21T15:38:37.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e4dffb640 10 client.16575 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:37.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e4dffb640 1 -- 172.21.15.38:0/2618086627 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55a6d04d6780 con 0x55a6d01ca390 2024-05-21T15:38:37.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e4dffb640 10 client.16575 handle_command_reply: tid=0 2024-05-21T15:38:37.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e73166740 1 client.16575 shutdown 2024-05-21T15:38:37.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e73166740 1 -- 172.21.15.38:0/2618086627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 msgr2=0x55a6d0579ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e73166740 1 --2- 172.21.15.38:0/2618086627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d0579ec0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9e54098450 tx=0x7f9e5409b4f0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e73166740 1 -- 172.21.15.38:0/2618086627 shutdown_connections 2024-05-21T15:38:37.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e73166740 1 --2- 172.21.15.38:0/2618086627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6d01a96b0 0x55a6d057a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e73166740 1 --2- 172.21.15.38:0/2618086627 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a6d01ca390 0x55a6d05eb5e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e73166740 1 --2- 172.21.15.38:0/2618086627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a6d01a14d0 0x55a6d0579ec0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e4dffb640 0 client.16575 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:37.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e73166740 1 --2- 172.21.15.38:0/2618086627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6d0188f60 0x55a6d0579980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e73166740 1 -- 172.21.15.38:0/2618086627 >> 172.21.15.38:0/2618086627 conn(0x55a6d01be4e0 msgr2=0x55a6d0572880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:37.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e73166740 1 -- 172.21.15.38:0/2618086627 shutdown_connections 2024-05-21T15:38:37.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.275+0000 7f9e73166740 1 -- 172.21.15.38:0/2618086627 wait complete. 2024-05-21T15:38:37.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.276+0000 7f9e73166740 20 client.16575 trim_cache size 0 max 16384 2024-05-21T15:38:37.281 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:38:37.281 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "scrub active (88 inodes in the stack)", 2024-05-21T15:38:37.281 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-05-21T15:38:37.281 INFO:teuthology.orchestra.run.smithi038.stdout: "ec33a0fc-f2ee-475d-a600-6e3b842c78da": { 2024-05-21T15:38:37.281 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-05-21T15:38:37.281 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "ec33a0fc-f2ee-475d-a600-6e3b842c78da", 2024-05-21T15:38:37.281 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-05-21T15:38:37.281 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:38:37.282 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:38:37.282 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:38:37.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.276+0000 7f9e6c2c3640 1 -- 172.21.15.38:0/2075830614 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9e500a2c10 msgr2=0x7f9e500a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.276+0000 7f9e6c2c3640 1 --2- 172.21.15.38:0/2075830614 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9e500a2c10 0x7f9e500a5120 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55a6d056ad80 tx=0x7f9e5c009500 comp rx=0 tx=0).stop 2024-05-21T15:38:37.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.276+0000 7f9e6c2c3640 1 -- 172.21.15.38:0/2075830614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e64102bd0 msgr2=0x7f9e641986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:37.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.276+0000 7f9e6c2c3640 1 --2- 172.21.15.38:0/2075830614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e64102bd0 0x7f9e641986c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400ea60 tx=0x7f9e5400ef20 comp rx=0 tx=0).stop 2024-05-21T15:38:37.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.276+0000 7f9e6c2c3640 1 -- 172.21.15.38:0/2075830614 shutdown_connections 2024-05-21T15:38:37.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.276+0000 7f9e6c2c3640 1 --2- 172.21.15.38:0/2075830614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e641078d0 0x7f9e6419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.276+0000 7f9e6c2c3640 1 --2- 172.21.15.38:0/2075830614 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9e500a2c10 0x7f9e500a5120 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.276+0000 7f9e6c2c3640 1 --2- 172.21.15.38:0/2075830614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e64102bd0 0x7f9e641986c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.276+0000 7f9e6c2c3640 1 --2- 172.21.15.38:0/2075830614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e64101990 0x7f9e64198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:37.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.276+0000 7f9e6c2c3640 1 -- 172.21.15.38:0/2075830614 >> 172.21.15.38:0/2075830614 conn(0x7f9e640fcb80 msgr2=0x7f9e641038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:37.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.276+0000 7f9e6c2c3640 1 -- 172.21.15.38:0/2075830614 shutdown_connections 2024-05-21T15:38:37.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:38:37.276+0000 7f9e6c2c3640 1 -- 172.21.15.38:0/2075830614 wait complete. 2024-05-21T15:38:37.289 INFO:tasks.cephfs.filesystem:scrub status for tag:ec33a0fc-f2ee-475d-a600-6e3b842c78da - {'path': '/', 'tag': 'ec33a0fc-f2ee-475d-a600-6e3b842c78da', 'options': 'recursive,force'} 2024-05-21T15:38:37.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:37 smithi053 ceph-mon[36496]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 93 MiB/s wr, 16.28k op/s 2024-05-21T15:38:37.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:37 smithi053 ceph-mon[36496]: scrub queued for path: / 2024-05-21T15:38:37.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:37 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:37.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:37 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:38:37.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:37 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:38:37.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:37 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:38:37.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:37 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:38:37.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:37 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:38:37.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:37 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:37.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:37 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:37.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:37 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:38.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:37 smithi045 ceph-mon[36409]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 93 MiB/s wr, 16.28k op/s 2024-05-21T15:38:38.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:37 smithi045 ceph-mon[36409]: scrub queued for path: / 2024-05-21T15:38:38.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:37 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:38.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:37 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:38:38.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:37 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:38:38.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:37 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:38:38.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:37 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:38:38.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:37 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:38:38.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:37 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:38.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:37 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:38.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:37 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:38.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:37 smithi038 ceph-mon[31995]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 93 MiB/s wr, 16.28k op/s 2024-05-21T15:38:38.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:37 smithi038 ceph-mon[31995]: scrub queued for path: / 2024-05-21T15:38:38.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:37 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:38.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:37 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:38:38.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:37 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:38:38.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:37 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:38:38.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:37 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:38:38.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:37 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:38:38.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:37 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:38.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:37 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:38.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:37 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:39.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:39 smithi053 ceph-mon[36496]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 93 MiB/s wr, 16.46k op/s 2024-05-21T15:38:40.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:39 smithi045 ceph-mon[36409]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 93 MiB/s wr, 16.46k op/s 2024-05-21T15:38:40.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:39 smithi038 ceph-mon[31995]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 93 MiB/s wr, 16.46k op/s 2024-05-21T15:38:41.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:41 smithi053 ceph-mon[36496]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 79 MiB/s wr, 14.27k op/s 2024-05-21T15:38:41.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:41 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:42.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:41 smithi045 ceph-mon[36409]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 79 MiB/s wr, 14.27k op/s 2024-05-21T15:38:42.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:41 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:42.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:41 smithi038 ceph-mon[31995]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 79 MiB/s wr, 14.27k op/s 2024-05-21T15:38:42.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:41 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:43.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:43 smithi053 ceph-mon[36496]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 87 MiB/s wr, 15.93k op/s 2024-05-21T15:38:44.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:43 smithi045 ceph-mon[36409]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 87 MiB/s wr, 15.93k op/s 2024-05-21T15:38:44.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:43 smithi038 ceph-mon[31995]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 87 MiB/s wr, 15.93k op/s 2024-05-21T15:38:45.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:45 smithi053 ceph-mon[36496]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 81 MiB/s wr, 15.13k op/s 2024-05-21T15:38:46.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:45 smithi045 ceph-mon[36409]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 81 MiB/s wr, 15.13k op/s 2024-05-21T15:38:46.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:45 smithi038 ceph-mon[31995]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 81 MiB/s wr, 15.13k op/s 2024-05-21T15:38:47.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:47 smithi053 ceph-mon[36496]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 MiB/s wr, 14.98k op/s 2024-05-21T15:38:48.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:47 smithi045 ceph-mon[36409]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 MiB/s wr, 14.98k op/s 2024-05-21T15:38:48.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:47 smithi038 ceph-mon[31995]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 MiB/s wr, 14.98k op/s 2024-05-21T15:38:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.651+0000 7ff50ce14640 1 -- 172.21.15.38:0/2128470898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff508107070 msgr2=0x7ff508109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.651+0000 7ff50ce14640 1 --2- 172.21.15.38:0/2128470898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff508107070 0x7ff508109490 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800aa10 tx=0x7ff4f802b760 comp rx=0 tx=0).stop 2024-05-21T15:38:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.653+0000 7ff50ce14640 1 -- 172.21.15.38:0/2128470898 shutdown_connections 2024-05-21T15:38:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.653+0000 7ff50ce14640 1 --2- 172.21.15.38:0/2128470898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff508107070 0x7ff508109490 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.653+0000 7ff50ce14640 1 --2- 172.21.15.38:0/2128470898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff508102370 0x7ff508102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.653+0000 7ff50ce14640 1 --2- 172.21.15.38:0/2128470898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508101130 0x7ff508101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.653+0000 7ff50ce14640 1 -- 172.21.15.38:0/2128470898 >> 172.21.15.38:0/2128470898 conn(0x7ff5080fc300 msgr2=0x7ff5080fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.656+0000 7ff50ce14640 1 -- 172.21.15.38:0/2128470898 shutdown_connections 2024-05-21T15:38:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.656+0000 7ff50ce14640 1 -- 172.21.15.38:0/2128470898 wait complete. 2024-05-21T15:38:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.657+0000 7ff50ce14640 1 Processor -- start 2024-05-21T15:38:48.656 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.658+0000 7ff50ce14640 1 -- start start 2024-05-21T15:38:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.658+0000 7ff50ce14640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508101130 0x7ff50819f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.658+0000 7ff50ce14640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff508102370 0x7ff50819fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.658+0000 7ff50ce14640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff508107070 0x7ff5081a6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.658+0000 7ff50ce14640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff50810c660 con 0x7ff508101130 2024-05-21T15:38:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.658+0000 7ff50ce14640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff50810c4e0 con 0x7ff508107070 2024-05-21T15:38:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.658+0000 7ff50ce14640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff50810c7e0 con 0x7ff508102370 2024-05-21T15:38:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.658+0000 7ff505d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff508102370 0x7ff50819fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.658+0000 7ff505d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff508102370 0x7ff50819fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:56728/0 (socket says 172.21.15.38:56728) 2024-05-21T15:38:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.658+0000 7ff50659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508101130 0x7ff50819f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.658+0000 7ff50659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508101130 0x7ff50819f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52890/0 (socket says 172.21.15.38:52890) 2024-05-21T15:38:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.658+0000 7ff50659c640 1 -- 172.21.15.38:0/1810699192 learned_addr learned my addr 172.21.15.38:0/1810699192 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.660+0000 7ff505d9b640 1 -- 172.21.15.38:0/1810699192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff508107070 msgr2=0x7ff5081a6e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T15:38:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.660+0000 7ff506d9d640 1 --2- 172.21.15.38:0/1810699192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff508107070 0x7ff5081a6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.660+0000 7ff505d9b640 1 --2- 172.21.15.38:0/1810699192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff508107070 0x7ff5081a6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.660+0000 7ff505d9b640 1 -- 172.21.15.38:0/1810699192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508101130 msgr2=0x7ff50819f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.660+0000 7ff505d9b640 1 --2- 172.21.15.38:0/1810699192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508101130 0x7ff50819f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.660+0000 7ff505d9b640 1 -- 172.21.15.38:0/1810699192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff508196280 con 0x7ff508102370 2024-05-21T15:38:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.661+0000 7ff505d9b640 1 --2- 172.21.15.38:0/1810699192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff508102370 0x7ff50819fec0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009fd0 tx=0x7ff4fc00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.661+0000 7ff4f77fe640 1 -- 172.21.15.38:0/1810699192 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc015020 con 0x7ff508102370 2024-05-21T15:38:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.662+0000 7ff50ce14640 1 -- 172.21.15.38:0/1810699192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff508196570 con 0x7ff508102370 2024-05-21T15:38:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.662+0000 7ff50ce14640 1 -- 172.21.15.38:0/1810699192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff508196b90 con 0x7ff508102370 2024-05-21T15:38:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.662+0000 7ff4f77fe640 1 -- 172.21.15.38:0/1810699192 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4fc011d10 con 0x7ff508102370 2024-05-21T15:38:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.662+0000 7ff4f77fe640 1 -- 172.21.15.38:0/1810699192 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00f2e0 con 0x7ff508102370 2024-05-21T15:38:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.667+0000 7ff4f77fe640 1 -- 172.21.15.38:0/1810699192 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff4fc018020 con 0x7ff508102370 2024-05-21T15:38:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.669+0000 7ff513d9a740 1 Processor -- start 2024-05-21T15:38:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.669+0000 7ff513d9a740 1 -- start start 2024-05-21T15:38:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.669+0000 7ff513d9a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d77e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.669+0000 7ff513d9a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049af2f0 0x556e04d783e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.669+0000 7ff513d9a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049b74d0 0x556e04d7f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.669+0000 7ff513d9a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556e04cdc340 con 0x556e049b74d0 2024-05-21T15:38:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.669+0000 7ff513d9a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556e04c84290 con 0x556e049af2f0 2024-05-21T15:38:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.669+0000 7ff513d9a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556e04cd8300 con 0x556e049d81b0 2024-05-21T15:38:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.670+0000 7ff4f77fe640 1 --2- 172.21.15.38:0/1810699192 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff4f00a2d30 0x7ff4f00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.670+0000 7ff506d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d77e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.670+0000 7ff506d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d77e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:56732/0 (socket says 172.21.15.38:56732) 2024-05-21T15:38:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.670+0000 7ff506d9d640 1 -- 172.21.15.38:0/3037244903 learned_addr learned my addr 172.21.15.38:0/3037244903 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.670+0000 7ff506d9d640 1 --2- 172.21.15.38:0/1810699192 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff4f00a2d30 0x7ff4f00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.671+0000 7ff506d9d640 1 -- 172.21.15.38:0/3037244903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049af2f0 msgr2=0x556e04d783e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.671+0000 7ff506d9d640 1 --2- 172.21.15.38:0/3037244903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049af2f0 0x556e04d783e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.671+0000 7ff506d9d640 1 -- 172.21.15.38:0/3037244903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049b74d0 msgr2=0x556e04d7f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.671+0000 7ff505d9b640 1 --2- 172.21.15.38:0/3037244903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049b74d0 0x556e04d7f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.673+0000 7ff506d9d640 1 --2- 172.21.15.38:0/3037244903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049b74d0 0x556e04d7f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.673+0000 7ff506d9d640 1 -- 172.21.15.38:0/3037244903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e04ca88e0 con 0x556e049d81b0 2024-05-21T15:38:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.673+0000 7ff505d9b640 1 --2- 172.21.15.38:0/3037244903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049b74d0 0x556e04d7f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:38:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.673+0000 7ff506d9d640 1 --2- 172.21.15.38:0/3037244903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d77e60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800ab30 tx=0x7ff4f8004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7299cecfd5810b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:48.672 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.674+0000 7ff4f4ff9640 1 -- 172.21.15.38:0/3037244903 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f804d070 con 0x556e049d81b0 2024-05-21T15:38:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.674+0000 7ff4f77fe640 1 -- 172.21.15.38:0/1810699192 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff4fc0ce8c0 con 0x7ff508102370 2024-05-21T15:38:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.674+0000 7ff506d9d640 1 --2- 172.21.15.38:0/1810699192 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff4f00a2d30 0x7ff4f00a5240 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8030720 tx=0x7ff4f803f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.674+0000 7ff4f4ff9640 1 -- 172.21.15.38:0/3037244903 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f8005d90 con 0x556e049d81b0 2024-05-21T15:38:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.676+0000 7ff4f4ff9640 1 -- 172.21.15.38:0/3037244903 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8036070 con 0x556e049d81b0 2024-05-21T15:38:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.676+0000 7ff513d9a740 1 -- 172.21.15.38:0/3037244903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 msgr2=0x556e04d77e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.676+0000 7ff513d9a740 1 --2- 172.21.15.38:0/3037244903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d77e60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800ab30 tx=0x7ff4f8004610 comp rx=0 tx=0).stop 2024-05-21T15:38:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.677+0000 7ff513d9a740 1 -- 172.21.15.38:0/3037244903 shutdown_connections 2024-05-21T15:38:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.677+0000 7ff513d9a740 1 --2- 172.21.15.38:0/3037244903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049b74d0 0x556e04d7f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.677+0000 7ff513d9a740 1 --2- 172.21.15.38:0/3037244903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049af2f0 0x556e04d783e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.677+0000 7ff513d9a740 1 --2- 172.21.15.38:0/3037244903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d77e60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.677+0000 7ff513d9a740 1 -- 172.21.15.38:0/3037244903 >> 172.21.15.38:0/3037244903 conn(0x556e049cc300 msgr2=0x556e04d737f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.677+0000 7ff513d9a740 1 -- 172.21.15.38:0/3037244903 shutdown_connections 2024-05-21T15:38:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.677+0000 7ff513d9a740 1 -- 172.21.15.38:0/3037244903 wait complete. 2024-05-21T15:38:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.677+0000 7ff513d9a740 1 -- start start 2024-05-21T15:38:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.678+0000 7ff513d9a740 1 Processor -- start 2024-05-21T15:38:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.678+0000 7ff513d9a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049af2f0 0x556e04d81bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.678+0000 7ff513d9a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049b74d0 0x556e04d82130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.678+0000 7ff513d9a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d89090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.678+0000 7ff513d9a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556e04d7fd70 con 0x556e049b74d0 2024-05-21T15:38:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.678+0000 7ff513d9a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556e04c97790 con 0x556e049af2f0 2024-05-21T15:38:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.678+0000 7ff513d9a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556e04d7fef0 con 0x556e049d81b0 2024-05-21T15:38:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.678+0000 7ff513d9a740 10 client.0 _ll_register_callbacks cb 0x556e04cd2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.678+0000 7ff50659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049af2f0 0x556e04d81bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.678+0000 7ff50659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049af2f0 0x556e04d81bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:38190/0 (socket says 172.21.15.38:38190) 2024-05-21T15:38:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.678+0000 7ff50659c640 1 -- 172.21.15.38:0/391824735 learned_addr learned my addr 172.21.15.38:0/391824735 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.679+0000 7ff505d9b640 1 --2- 172.21.15.38:0/391824735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049b74d0 0x556e04d82130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.679+0000 7ff506d9d640 1 --2- 172.21.15.38:0/391824735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d89090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.679+0000 7ff505d9b640 1 -- 172.21.15.38:0/391824735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 msgr2=0x556e04d89090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.680+0000 7ff505d9b640 1 --2- 172.21.15.38:0/391824735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d89090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.680+0000 7ff505d9b640 1 -- 172.21.15.38:0/391824735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049af2f0 msgr2=0x556e04d81bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.680+0000 7ff505d9b640 1 --2- 172.21.15.38:0/391824735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049af2f0 0x556e04d81bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.680+0000 7ff505d9b640 1 -- 172.21.15.38:0/391824735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e049f0210 con 0x556e049b74d0 2024-05-21T15:38:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.680+0000 7ff50659c640 1 --2- 172.21.15.38:0/391824735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049af2f0 0x556e04d81bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:38:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.680+0000 7ff505d9b640 1 --2- 172.21.15.38:0/391824735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049b74d0 0x556e04d82130 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc097a40 tx=0x7ff4fc09ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625b37781bc21dd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.680+0000 7ff4edffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.683+0000 7ff4edffb640 1 -- 172.21.15.38:0/391824735 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc09f420 con 0x556e049b74d0 2024-05-21T15:38:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.683+0000 7ff4edffb640 1 -- 172.21.15.38:0/391824735 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff4fc09f0d0 con 0x556e049b74d0 2024-05-21T15:38:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.684+0000 7ff513d9a740 1 -- 172.21.15.38:0/391824735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4f8005d90 con 0x556e049b74d0 2024-05-21T15:38:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.684+0000 7ff4edffb640 1 -- 172.21.15.38:0/391824735 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff4fc0a3ce0 con 0x556e049b74d0 2024-05-21T15:38:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.684+0000 7ff513d9a740 10 client.16581 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.684+0000 7ff513d9a740 1 -- 172.21.15.38:0/391824735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556e04abc620 con 0x556e049b74d0 2024-05-21T15:38:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.685+0000 7ff4edffb640 1 -- 172.21.15.38:0/391824735 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f8005d90 con 0x556e049b74d0 2024-05-21T15:38:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.685+0000 7ff4edffb640 1 -- 172.21.15.38:0/391824735 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7ff4fc09a4c0 con 0x556e049b74d0 2024-05-21T15:38:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.685+0000 7ff513d9a740 10 client.16581 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.685+0000 7ff513d9a740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.685+0000 7ff513d9a740 20 client.16581 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.685+0000 7ff513d9a740 1 --2- 172.21.15.38:0/391824735 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556e04996d80 0x556e04d8f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.685+0000 7ff513d9a740 4 client.16581 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:38:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.685+0000 7ff513d9a740 1 -- 172.21.15.38:0/391824735 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e04ce9a70 con 0x556e04996d80 2024-05-21T15:38:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.686+0000 7ff50659c640 1 --2- 172.21.15.38:0/391824735 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556e04996d80 0x556e04d8f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.686+0000 7ff4edffb640 10 client.16581 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.687+0000 7ff50659c640 1 --2- 172.21.15.38:0/391824735 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556e04996d80 0x556e04d8f560 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc139068a090eb96 server_cookie=ab76f37a6efccdfb in_seq=0 out_seq=0 2024-05-21T15:38:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.688+0000 7ff4edffb640 1 -- 172.21.15.38:0/391824735 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556e04ce9a70 con 0x556e04996d80 2024-05-21T15:38:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.688+0000 7ff4edffb640 10 client.16581 handle_command_reply: tid=0 2024-05-21T15:38:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.688+0000 7ff513d9a740 1 client.16581 shutdown 2024-05-21T15:38:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.689+0000 7ff513d9a740 1 -- 172.21.15.38:0/391824735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049b74d0 msgr2=0x556e04d82130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.689+0000 7ff513d9a740 1 --2- 172.21.15.38:0/391824735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049b74d0 0x556e04d82130 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc097a40 tx=0x7ff4fc09ac70 comp rx=0 tx=0).stop 2024-05-21T15:38:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.689+0000 7ff513d9a740 1 -- 172.21.15.38:0/391824735 shutdown_connections 2024-05-21T15:38:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.689+0000 7ff513d9a740 1 --2- 172.21.15.38:0/391824735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d89090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.689+0000 7ff513d9a740 1 --2- 172.21.15.38:0/391824735 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556e04996d80 0x556e04d8f560 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.689+0000 7ff513d9a740 1 --2- 172.21.15.38:0/391824735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049b74d0 0x556e04d82130 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.689+0000 7ff513d9a740 1 --2- 172.21.15.38:0/391824735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049af2f0 0x556e04d81bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.689+0000 7ff513d9a740 1 -- 172.21.15.38:0/391824735 >> 172.21.15.38:0/391824735 conn(0x556e049cc300 msgr2=0x556e04d72810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:48.688 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.689+0000 7ff4edffb640 0 client.16581 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.689+0000 7ff513d9a740 1 -- 172.21.15.38:0/391824735 shutdown_connections 2024-05-21T15:38:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.689+0000 7ff513d9a740 1 -- 172.21.15.38:0/391824735 wait complete. 2024-05-21T15:38:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.689+0000 7ff513d9a740 20 client.16581 trim_cache size 0 max 16384 2024-05-21T15:38:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.720+0000 7ff513d9a740 1 Processor -- start 2024-05-21T15:38:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.720+0000 7ff513d9a740 1 -- start start 2024-05-21T15:38:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.721+0000 7ff513d9a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049af2f0 0x556e04d73160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.721+0000 7ff513d9a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049b74d0 0x556e04d736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.721+0000 7ff513d9a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d76690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.721+0000 7ff513d9a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e800e5a0 con 0x556e049af2f0 2024-05-21T15:38:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.721+0000 7ff513d9a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e80101d0 con 0x556e049b74d0 2024-05-21T15:38:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.721+0000 7ff513d9a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e800ec70 con 0x556e049d81b0 2024-05-21T15:38:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.721+0000 7ff505d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049b74d0 0x556e04d736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.721+0000 7ff506d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d76690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.722+0000 7ff506d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d76690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:56746/0 (socket says 172.21.15.38:56746) 2024-05-21T15:38:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.722+0000 7ff506d9d640 1 -- 172.21.15.38:0/763713702 learned_addr learned my addr 172.21.15.38:0/763713702 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.722+0000 7ff50659c640 1 --2- 172.21.15.38:0/763713702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049af2f0 0x556e04d73160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.723+0000 7ff506d9d640 1 -- 172.21.15.38:0/763713702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049b74d0 msgr2=0x556e04d736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.723+0000 7ff506d9d640 1 --2- 172.21.15.38:0/763713702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049b74d0 0x556e04d736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.723+0000 7ff506d9d640 1 -- 172.21.15.38:0/763713702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049af2f0 msgr2=0x556e04d73160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.723+0000 7ff506d9d640 1 --2- 172.21.15.38:0/763713702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049af2f0 0x556e04d73160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.723+0000 7ff506d9d640 1 -- 172.21.15.38:0/763713702 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e04d77a10 con 0x556e049d81b0 2024-05-21T15:38:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.723+0000 7ff506d9d640 1 --2- 172.21.15.38:0/763713702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d76690 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800ab30 tx=0x7ff4f800a370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d734c9663855185e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.724+0000 7ff4effff640 1 -- 172.21.15.38:0/763713702 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f804d070 con 0x556e049d81b0 2024-05-21T15:38:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.724+0000 7ff4effff640 1 -- 172.21.15.38:0/763713702 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f8003ea0 con 0x556e049d81b0 2024-05-21T15:38:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.724+0000 7ff4effff640 1 -- 172.21.15.38:0/763713702 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8036040 con 0x556e049d81b0 2024-05-21T15:38:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.725+0000 7ff513d9a740 1 -- 172.21.15.38:0/763713702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 msgr2=0x556e04d76690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.725+0000 7ff513d9a740 1 --2- 172.21.15.38:0/763713702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d76690 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800ab30 tx=0x7ff4f800a370 comp rx=0 tx=0).stop 2024-05-21T15:38:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.726+0000 7ff513d9a740 1 -- 172.21.15.38:0/763713702 shutdown_connections 2024-05-21T15:38:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.726+0000 7ff513d9a740 1 --2- 172.21.15.38:0/763713702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049d81b0 0x556e04d76690 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.726+0000 7ff513d9a740 1 --2- 172.21.15.38:0/763713702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049b74d0 0x556e04d736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.726+0000 7ff513d9a740 1 --2- 172.21.15.38:0/763713702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049af2f0 0x556e04d73160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.726+0000 7ff513d9a740 1 -- 172.21.15.38:0/763713702 >> 172.21.15.38:0/763713702 conn(0x556e049cc300 msgr2=0x556e04d7bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.726+0000 7ff513d9a740 1 -- 172.21.15.38:0/763713702 shutdown_connections 2024-05-21T15:38:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.727+0000 7ff513d9a740 1 -- 172.21.15.38:0/763713702 wait complete. 2024-05-21T15:38:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.727+0000 7ff513d9a740 1 -- start start 2024-05-21T15:38:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.728+0000 7ff513d9a740 1 Processor -- start 2024-05-21T15:38:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.728+0000 7ff513d9a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049af2f0 0x556e04d83330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.728+0000 7ff513d9a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049b74d0 0x556e04d83870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.728+0000 7ff513d9a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049d81b0 0x556e04df58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.728+0000 7ff513d9a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e800fa40 con 0x556e049d81b0 2024-05-21T15:38:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.728+0000 7ff513d9a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e800f340 con 0x556e049b74d0 2024-05-21T15:38:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.728+0000 7ff513d9a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e800c580 con 0x556e049af2f0 2024-05-21T15:38:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.728+0000 7ff513d9a740 10 client.0 _ll_register_callbacks cb 0x7ff4fc011ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:38:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.729+0000 7ff50659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049af2f0 0x556e04d83330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.729+0000 7ff506d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049d81b0 0x556e04df58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.729+0000 7ff506d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049d81b0 0x556e04df58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52926/0 (socket says 172.21.15.38:52926) 2024-05-21T15:38:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.729+0000 7ff50659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049af2f0 0x556e04d83330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:56760/0 (socket says 172.21.15.38:56760) 2024-05-21T15:38:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.729+0000 7ff506d9d640 1 -- 172.21.15.38:0/2284973490 learned_addr learned my addr 172.21.15.38:0/2284973490 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:38:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.729+0000 7ff505d9b640 1 --2- 172.21.15.38:0/2284973490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049b74d0 0x556e04d83870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.729+0000 7ff506d9d640 1 -- 172.21.15.38:0/2284973490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049af2f0 msgr2=0x556e04d83330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.729+0000 7ff506d9d640 1 --2- 172.21.15.38:0/2284973490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049af2f0 0x556e04d83330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.729+0000 7ff506d9d640 1 -- 172.21.15.38:0/2284973490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049b74d0 msgr2=0x556e04d83870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:38:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.729+0000 7ff506d9d640 1 --2- 172.21.15.38:0/2284973490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049b74d0 0x556e04d83870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.729+0000 7ff506d9d640 1 -- 172.21.15.38:0/2284973490 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e04d887f0 con 0x556e049d81b0 2024-05-21T15:38:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.730+0000 7ff506d9d640 1 --2- 172.21.15.38:0/2284973490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049d81b0 0x556e04df58c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8004110 tx=0x7ff4f800a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e66f957bb5de557 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:38:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.731+0000 7ff4edffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:38:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.731+0000 7ff4edffb640 1 -- 172.21.15.38:0/2284973490 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8036040 con 0x556e049d81b0 2024-05-21T15:38:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.731+0000 7ff513d9a740 1 -- 172.21.15.38:0/2284973490 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556e04ce3dc0 con 0x556e049d81b0 2024-05-21T15:38:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.731+0000 7ff4edffb640 1 -- 172.21.15.38:0/2284973490 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff4f80039c0 con 0x556e049d81b0 2024-05-21T15:38:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.732+0000 7ff4edffb640 1 -- 172.21.15.38:0/2284973490 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff4f804d040 con 0x556e049d81b0 2024-05-21T15:38:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.732+0000 7ff513d9a740 10 client.16587 fetch_fsmap learned FSMap version 31 2024-05-21T15:38:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.732+0000 7ff513d9a740 1 -- 172.21.15.38:0/2284973490 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556e04df6c60 con 0x556e049d81b0 2024-05-21T15:38:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.733+0000 7ff4edffb640 1 -- 172.21.15.38:0/2284973490 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556e04ce3dc0 con 0x556e049d81b0 2024-05-21T15:38:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.734+0000 7ff4edffb640 1 -- 172.21.15.38:0/2284973490 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7ff4f803b040 con 0x556e049d81b0 2024-05-21T15:38:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.735+0000 7ff513d9a740 10 client.16587 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:38:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.735+0000 7ff513d9a740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:38:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.735+0000 7ff513d9a740 20 client.16587 populate_metadata read hostname 'smithi038' 2024-05-21T15:38:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.735+0000 7ff513d9a740 1 --2- 172.21.15.38:0/2284973490 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556e0496c1c0 0x556e04df9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:38:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.735+0000 7ff513d9a740 4 client.16587 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:38:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.735+0000 7ff513d9a740 1 -- 172.21.15.38:0/2284973490 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e04c618e0 con 0x556e0496c1c0 2024-05-21T15:38:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.735+0000 7ff50659c640 1 --2- 172.21.15.38:0/2284973490 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556e0496c1c0 0x556e04df9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:38:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.736+0000 7ff50659c640 1 --2- 172.21.15.38:0/2284973490 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556e0496c1c0 0x556e04df9740 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=128c423b226faabb server_cookie=c46a266e985d0167 in_seq=0 out_seq=0 2024-05-21T15:38:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:48.736+0000 7ff4edffb640 10 client.16587 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:48.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:48 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:49.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:48 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:49.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:48 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:38:49.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:49 smithi053 ceph-mon[36496]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 74 MiB/s wr, 14.24k op/s 2024-05-21T15:38:50.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:49 smithi045 ceph-mon[36409]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 74 MiB/s wr, 14.24k op/s 2024-05-21T15:38:50.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:49 smithi038 ceph-mon[31995]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 74 MiB/s wr, 14.24k op/s 2024-05-21T15:38:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.648+0000 7ff4edffb640 1 -- 172.21.15.38:0/2284973490 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e04c618e0 con 0x556e0496c1c0 2024-05-21T15:38:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.648+0000 7ff4edffb640 10 client.16587 handle_command_reply: tid=0 2024-05-21T15:38:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.648+0000 7ff513d9a740 1 client.16587 shutdown 2024-05-21T15:38:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.649+0000 7ff513d9a740 1 -- 172.21.15.38:0/2284973490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049d81b0 msgr2=0x556e04df58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.649+0000 7ff513d9a740 1 --2- 172.21.15.38:0/2284973490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049d81b0 0x556e04df58c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8004110 tx=0x7ff4f800a690 comp rx=0 tx=0).stop 2024-05-21T15:38:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.649+0000 7ff513d9a740 1 -- 172.21.15.38:0/2284973490 shutdown_connections 2024-05-21T15:38:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.649+0000 7ff513d9a740 1 --2- 172.21.15.38:0/2284973490 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556e0496c1c0 0x556e04df9740 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.649+0000 7ff513d9a740 1 --2- 172.21.15.38:0/2284973490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556e049d81b0 0x556e04df58c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.649+0000 7ff513d9a740 1 --2- 172.21.15.38:0/2284973490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556e049b74d0 0x556e04d83870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.649+0000 7ff513d9a740 1 --2- 172.21.15.38:0/2284973490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e049af2f0 0x556e04d83330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.649+0000 7ff513d9a740 1 -- 172.21.15.38:0/2284973490 >> 172.21.15.38:0/2284973490 conn(0x556e049cc300 msgr2=0x556e04d73160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.649+0000 7ff4edffb640 0 client.16587 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:38:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.649+0000 7ff513d9a740 1 -- 172.21.15.38:0/2284973490 shutdown_connections 2024-05-21T15:38:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.649+0000 7ff513d9a740 1 -- 172.21.15.38:0/2284973490 wait complete. 2024-05-21T15:38:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.649+0000 7ff513d9a740 20 client.16587 trim_cache size 0 max 16384 2024-05-21T15:38:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:38:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:38:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:38:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:38:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.650+0000 7ff50ce14640 1 -- 172.21.15.38:0/1810699192 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff4f00a2d30 msgr2=0x7ff4f00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.650+0000 7ff50ce14640 1 --2- 172.21.15.38:0/1810699192 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff4f00a2d30 0x7ff4f00a5240 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8030720 tx=0x7ff4f803f000 comp rx=0 tx=0).stop 2024-05-21T15:38:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.650+0000 7ff50ce14640 1 -- 172.21.15.38:0/1810699192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff508102370 msgr2=0x7ff50819fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:38:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.650+0000 7ff50ce14640 1 --2- 172.21.15.38:0/1810699192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff508102370 0x7ff50819fec0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009fd0 tx=0x7ff4fc00b870 comp rx=0 tx=0).stop 2024-05-21T15:38:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.650+0000 7ff50ce14640 1 -- 172.21.15.38:0/1810699192 shutdown_connections 2024-05-21T15:38:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.650+0000 7ff50ce14640 1 --2- 172.21.15.38:0/1810699192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff508107070 0x7ff5081a6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.650+0000 7ff50ce14640 1 --2- 172.21.15.38:0/1810699192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff508102370 0x7ff50819fec0 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009fd0 tx=0x7ff4fc00b870 comp rx=0 tx=0).stop 2024-05-21T15:38:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.650+0000 7ff50ce14640 1 --2- 172.21.15.38:0/1810699192 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff4f00a2d30 0x7ff4f00a5240 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.650+0000 7ff50ce14640 1 --2- 172.21.15.38:0/1810699192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff508101130 0x7ff50819f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:38:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.650+0000 7ff50ce14640 1 -- 172.21.15.38:0/1810699192 >> 172.21.15.38:0/1810699192 conn(0x7ff5080fc300 msgr2=0x7ff5080fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:38:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.650+0000 7ff50ce14640 1 -- 172.21.15.38:0/1810699192 shutdown_connections 2024-05-21T15:38:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:38:50.650+0000 7ff50ce14640 1 -- 172.21.15.38:0/1810699192 wait complete. 2024-05-21T15:38:51.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:51 smithi053 ceph-mon[36496]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 68 MiB/s wr, 13.20k op/s 2024-05-21T15:38:52.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:51 smithi045 ceph-mon[36409]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 68 MiB/s wr, 13.20k op/s 2024-05-21T15:38:52.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:51 smithi038 ceph-mon[31995]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 68 MiB/s wr, 13.20k op/s 2024-05-21T15:38:53.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:53 smithi053 ceph-mon[36496]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 77 MiB/s wr, 15.07k op/s 2024-05-21T15:38:54.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:53 smithi045 ceph-mon[36409]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 77 MiB/s wr, 15.07k op/s 2024-05-21T15:38:54.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:53 smithi038 ceph-mon[31995]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 77 MiB/s wr, 15.07k op/s 2024-05-21T15:38:54.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:54 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:38:54.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:54 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:38:54.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:54 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:38:54.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:54 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:38:54.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:54 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:38:55.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:54 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:38:55.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:54 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:38:55.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:54 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:38:55.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:54 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:38:55.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:54 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:38:55.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:54 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:38:55.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:54 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:38:55.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:54 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:38:55.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:54 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:38:55.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:54 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:38:55.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:55 smithi053 ceph-mon[36496]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 69 MiB/s wr, 13.57k op/s 2024-05-21T15:38:56.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:55 smithi045 ceph-mon[36409]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 69 MiB/s wr, 13.57k op/s 2024-05-21T15:38:56.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:55 smithi038 ceph-mon[31995]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 69 MiB/s wr, 13.57k op/s 2024-05-21T15:38:57.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:57 smithi053 ceph-mon[36496]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 68 MiB/s wr, 13.56k op/s 2024-05-21T15:38:58.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:57 smithi045 ceph-mon[36409]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 68 MiB/s wr, 13.56k op/s 2024-05-21T15:38:58.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:57 smithi038 ceph-mon[31995]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 68 MiB/s wr, 13.56k op/s 2024-05-21T15:38:59.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:38:59 smithi053 ceph-mon[36496]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 MiB/s wr, 11.92k op/s 2024-05-21T15:39:00.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:38:59 smithi045 ceph-mon[36409]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 MiB/s wr, 11.92k op/s 2024-05-21T15:39:00.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:38:59 smithi038 ceph-mon[31995]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 MiB/s wr, 11.92k op/s 2024-05-21T15:39:01.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:01 smithi053 ceph-mon[36496]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 MiB/s wr, 11.65k op/s 2024-05-21T15:39:02.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:01 smithi045 ceph-mon[36409]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 MiB/s wr, 11.65k op/s 2024-05-21T15:39:02.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:01 smithi038 ceph-mon[31995]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 MiB/s wr, 11.65k op/s 2024-05-21T15:39:03.816 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.817+0000 7fb269d79640 1 -- 172.21.15.38:0/1194548840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26410b000 msgr2=0x7fb26410b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.817+0000 7fb269d79640 1 --2- 172.21.15.38:0/1194548840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26410b000 0x7fb26410b410 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009860 tx=0x7fb25c02a380 comp rx=0 tx=0).stop 2024-05-21T15:39:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.817+0000 7fb269d79640 1 -- 172.21.15.38:0/1194548840 shutdown_connections 2024-05-21T15:39:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.817+0000 7fb269d79640 1 --2- 172.21.15.38:0/1194548840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26410b000 0x7fb26410b410 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.817+0000 7fb269d79640 1 --2- 172.21.15.38:0/1194548840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26410a300 0x7fb26410a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.817+0000 7fb269d79640 1 --2- 172.21.15.38:0/1194548840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2641093d0 0x7fb264109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.817+0000 7fb269d79640 1 -- 172.21.15.38:0/1194548840 >> 172.21.15.38:0/1194548840 conn(0x7fb26406f4c0 msgr2=0x7fb264071970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.818+0000 7fb269d79640 1 -- 172.21.15.38:0/1194548840 shutdown_connections 2024-05-21T15:39:03.817 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.818+0000 7fb269d79640 1 -- 172.21.15.38:0/1194548840 wait complete. 2024-05-21T15:39:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.819+0000 7fb269d79640 1 Processor -- start 2024-05-21T15:39:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.819+0000 7fb269d79640 1 -- start start 2024-05-21T15:39:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.819+0000 7fb269d79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2641093d0 0x7fb26411abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.819+0000 7fb269d79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26410a300 0x7fb26411b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.819+0000 7fb269d79640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26410b000 0x7fb26411e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.820 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.819+0000 7fb269d79640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb26410e490 con 0x7fb2641093d0 2024-05-21T15:39:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.819+0000 7fb269d79640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb26410e310 con 0x7fb26410a300 2024-05-21T15:39:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.819+0000 7fb269d79640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb26410e610 con 0x7fb26410b000 2024-05-21T15:39:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.819+0000 7fb2637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2641093d0 0x7fb26411abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.820+0000 7fb2637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2641093d0 0x7fb26411abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50656/0 (socket says 172.21.15.38:50656) 2024-05-21T15:39:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.820+0000 7fb2637fe640 1 -- 172.21.15.38:0/2846393574 learned_addr learned my addr 172.21.15.38:0/2846393574 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.820+0000 7fb263fff640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26410b000 0x7fb26411e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.820+0000 7fb262ffd640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26410a300 0x7fb26411b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.820+0000 7fb2637fe640 1 -- 172.21.15.38:0/2846393574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26410b000 msgr2=0x7fb26411e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.820+0000 7fb2637fe640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26410b000 0x7fb26411e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.820+0000 7fb2637fe640 1 -- 172.21.15.38:0/2846393574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26410a300 msgr2=0x7fb26411b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.820+0000 7fb2637fe640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26410a300 0x7fb26411b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.820+0000 7fb2637fe640 1 -- 172.21.15.38:0/2846393574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26411e6d0 con 0x7fb2641093d0 2024-05-21T15:39:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.820+0000 7fb263fff640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26410b000 0x7fb26411e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.820+0000 7fb2637fe640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2641093d0 0x7fb26411abb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb25400c470 tx=0x7fb25400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.821+0000 7fb260ff9640 1 -- 172.21.15.38:0/2846393574 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb254003970 con 0x7fb2641093d0 2024-05-21T15:39:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.821+0000 7fb260ff9640 1 -- 172.21.15.38:0/2846393574 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb25400b6f0 con 0x7fb2641093d0 2024-05-21T15:39:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.822+0000 7fb260ff9640 1 -- 172.21.15.38:0/2846393574 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb254004480 con 0x7fb2641093d0 2024-05-21T15:39:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.822+0000 7fb269d79640 1 -- 172.21.15.38:0/2846393574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26411e9c0 con 0x7fb2641093d0 2024-05-21T15:39:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.822+0000 7fb269d79640 1 -- 172.21.15.38:0/2846393574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb26411ef40 con 0x7fb2641093d0 2024-05-21T15:39:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.828+0000 7fb260ff9640 1 -- 172.21.15.38:0/2846393574 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb25401d020 con 0x7fb2641093d0 2024-05-21T15:39:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.829+0000 7fb260ff9640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb2500a2e20 0x7fb2500a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.830+0000 7fb260ff9640 1 -- 172.21.15.38:0/2846393574 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fb2540cee50 con 0x7fb2641093d0 2024-05-21T15:39:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.830+0000 7fb262ffd640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb2500a2e20 0x7fb2500a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.830+0000 7fb262ffd640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb2500a2e20 0x7fb2500a5330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb26411bc50 tx=0x7fb24c0090e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.831+0000 7fb270c1a740 1 Processor -- start 2024-05-21T15:39:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.832+0000 7fb270c1a740 1 -- start start 2024-05-21T15:39:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.832+0000 7fb270c1a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b541b0 0x560292ef3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.832+0000 7fb270c1a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 0x560292ef43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.832+0000 7fb270c1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b334d0 0x560292efb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.832+0000 7fb270c1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560292e58340 con 0x560292b334d0 2024-05-21T15:39:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.832+0000 7fb270c1a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560292e00290 con 0x560292b541b0 2024-05-21T15:39:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.832+0000 7fb270c1a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560292e54300 con 0x560292b2b2f0 2024-05-21T15:39:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.832+0000 7fb262ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 0x560292ef43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.833+0000 7fb263fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b334d0 0x560292efb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.833+0000 7fb263fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b334d0 0x560292efb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50672/0 (socket says 172.21.15.38:50672) 2024-05-21T15:39:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.833+0000 7fb262ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 0x560292ef43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41944/0 (socket says 172.21.15.38:41944) 2024-05-21T15:39:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.833+0000 7fb263fff640 1 -- 172.21.15.38:0/2384052064 learned_addr learned my addr 172.21.15.38:0/2384052064 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.833+0000 7fb2637fe640 1 --2- 172.21.15.38:0/2384052064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b541b0 0x560292ef3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.833+0000 7fb263fff640 1 -- 172.21.15.38:0/2384052064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 msgr2=0x560292ef43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.833+0000 7fb263fff640 1 --2- 172.21.15.38:0/2384052064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 0x560292ef43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.833+0000 7fb263fff640 1 -- 172.21.15.38:0/2384052064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b541b0 msgr2=0x560292ef3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.833+0000 7fb263fff640 1 --2- 172.21.15.38:0/2384052064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b541b0 0x560292ef3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.833+0000 7fb263fff640 1 -- 172.21.15.38:0/2384052064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560292e248e0 con 0x560292b334d0 2024-05-21T15:39:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.833+0000 7fb263fff640 1 --2- 172.21.15.38:0/2384052064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b334d0 0x560292efb2e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009860 tx=0x7fb25c0306d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4ae93ba103f0c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.834+0000 7fb259ffb640 1 -- 172.21.15.38:0/2384052064 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c041020 con 0x560292b334d0 2024-05-21T15:39:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.834+0000 7fb259ffb640 1 -- 172.21.15.38:0/2384052064 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb25c030d80 con 0x560292b334d0 2024-05-21T15:39:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.834+0000 7fb259ffb640 1 -- 172.21.15.38:0/2384052064 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c037040 con 0x560292b334d0 2024-05-21T15:39:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.835+0000 7fb270c1a740 1 -- 172.21.15.38:0/2384052064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b334d0 msgr2=0x560292efb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.835+0000 7fb270c1a740 1 --2- 172.21.15.38:0/2384052064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b334d0 0x560292efb2e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009860 tx=0x7fb25c0306d0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.835+0000 7fb270c1a740 1 -- 172.21.15.38:0/2384052064 shutdown_connections 2024-05-21T15:39:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.835+0000 7fb270c1a740 1 --2- 172.21.15.38:0/2384052064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b334d0 0x560292efb2e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.835+0000 7fb270c1a740 1 --2- 172.21.15.38:0/2384052064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 0x560292ef43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.835+0000 7fb270c1a740 1 --2- 172.21.15.38:0/2384052064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b541b0 0x560292ef3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.835+0000 7fb270c1a740 1 -- 172.21.15.38:0/2384052064 >> 172.21.15.38:0/2384052064 conn(0x560292b48300 msgr2=0x560292eef7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.835+0000 7fb270c1a740 1 -- 172.21.15.38:0/2384052064 shutdown_connections 2024-05-21T15:39:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.835+0000 7fb270c1a740 1 -- 172.21.15.38:0/2384052064 wait complete. 2024-05-21T15:39:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.836+0000 7fb270c1a740 1 -- start start 2024-05-21T15:39:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.836+0000 7fb270c1a740 1 Processor -- start 2024-05-21T15:39:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.836+0000 7fb270c1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b2b2f0 0x560292efdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.836+0000 7fb270c1a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b334d0 0x560292efe080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.836+0000 7fb270c1a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b541b0 0x560292f04fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.836+0000 7fb270c1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560292efbcd0 con 0x560292b2b2f0 2024-05-21T15:39:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.836+0000 7fb270c1a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560292e13790 con 0x560292b541b0 2024-05-21T15:39:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.836+0000 7fb270c1a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560292efbe50 con 0x560292b334d0 2024-05-21T15:39:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.836+0000 7fb270c1a740 10 client.0 _ll_register_callbacks cb 0x560292e4ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.836+0000 7fb2637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b2b2f0 0x560292efdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.836+0000 7fb2637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b2b2f0 0x560292efdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50678/0 (socket says 172.21.15.38:50678) 2024-05-21T15:39:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.836+0000 7fb2637fe640 1 -- 172.21.15.38:0/1972518661 learned_addr learned my addr 172.21.15.38:0/1972518661 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.836+0000 7fb262ffd640 1 --2- 172.21.15.38:0/1972518661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b334d0 0x560292efe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.837+0000 7fb263fff640 1 --2- 172.21.15.38:0/1972518661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b541b0 0x560292f04fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.837+0000 7fb2637fe640 1 -- 172.21.15.38:0/1972518661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b334d0 msgr2=0x560292efe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.837+0000 7fb2637fe640 1 --2- 172.21.15.38:0/1972518661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b334d0 0x560292efe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.837+0000 7fb2637fe640 1 -- 172.21.15.38:0/1972518661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b541b0 msgr2=0x560292f04fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.837+0000 7fb2637fe640 1 --2- 172.21.15.38:0/1972518661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b541b0 0x560292f04fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.837+0000 7fb2637fe640 1 -- 172.21.15.38:0/1972518661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560292b6c210 con 0x560292b2b2f0 2024-05-21T15:39:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.837+0000 7fb263fff640 1 --2- 172.21.15.38:0/1972518661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b541b0 0x560292f04fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:39:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.837+0000 7fb2637fe640 1 --2- 172.21.15.38:0/1972518661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b2b2f0 0x560292efdb40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb2540cc8a0 tx=0x7fb25409bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140a98723712fecf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.837+0000 7fb242ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.838+0000 7fb242ffd640 1 -- 172.21.15.38:0/1972518661 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2540b4020 con 0x560292b2b2f0 2024-05-21T15:39:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.838+0000 7fb242ffd640 1 -- 172.21.15.38:0/1972518661 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fb254004620 con 0x560292b2b2f0 2024-05-21T15:39:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.838+0000 7fb270c1a740 1 -- 172.21.15.38:0/1972518661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb25c030d80 con 0x560292b2b2f0 2024-05-21T15:39:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.838+0000 7fb242ffd640 1 -- 172.21.15.38:0/1972518661 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb254097040 con 0x560292b2b2f0 2024-05-21T15:39:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.839+0000 7fb270c1a740 10 client.16599 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.839+0000 7fb270c1a740 1 -- 172.21.15.38:0/1972518661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560292c38620 con 0x560292b2b2f0 2024-05-21T15:39:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.839+0000 7fb242ffd640 1 -- 172.21.15.38:0/1972518661 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb25c030d80 con 0x560292b2b2f0 2024-05-21T15:39:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.840+0000 7fb242ffd640 1 -- 172.21.15.38:0/1972518661 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fb25400b040 con 0x560292b2b2f0 2024-05-21T15:39:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.841+0000 7fb270c1a740 10 client.16599 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.841+0000 7fb270c1a740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.841+0000 7fb270c1a740 20 client.16599 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.841+0000 7fb270c1a740 1 --2- 172.21.15.38:0/1972518661 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560292b12d80 0x560292f0b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.841+0000 7fb270c1a740 4 client.16599 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.841+0000 7fb270c1a740 1 -- 172.21.15.38:0/1972518661 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560292e65a70 con 0x560292b12d80 2024-05-21T15:39:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.842+0000 7fb262ffd640 1 --2- 172.21.15.38:0/1972518661 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560292b12d80 0x560292f0b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.844+0000 7fb242ffd640 10 client.16599 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.844+0000 7fb262ffd640 1 --2- 172.21.15.38:0/1972518661 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560292b12d80 0x560292f0b660 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef0eef37bebe53e4 server_cookie=f0ca27b68cc5a761 in_seq=0 out_seq=0 2024-05-21T15:39:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.846+0000 7fb242ffd640 1 -- 172.21.15.38:0/1972518661 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560292e65a70 con 0x560292b12d80 2024-05-21T15:39:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.846+0000 7fb242ffd640 10 client.16599 handle_command_reply: tid=0 2024-05-21T15:39:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.846+0000 7fb270c1a740 1 client.16599 shutdown 2024-05-21T15:39:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.847+0000 7fb270c1a740 1 -- 172.21.15.38:0/1972518661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b2b2f0 msgr2=0x560292efdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.847+0000 7fb270c1a740 1 --2- 172.21.15.38:0/1972518661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b2b2f0 0x560292efdb40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb2540cc8a0 tx=0x7fb25409bdc0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.847+0000 7fb270c1a740 1 -- 172.21.15.38:0/1972518661 shutdown_connections 2024-05-21T15:39:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.847+0000 7fb270c1a740 1 --2- 172.21.15.38:0/1972518661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b541b0 0x560292f04fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.847+0000 7fb270c1a740 1 --2- 172.21.15.38:0/1972518661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b334d0 0x560292efe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.847+0000 7fb270c1a740 1 --2- 172.21.15.38:0/1972518661 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560292b12d80 0x560292f0b660 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.847+0000 7fb270c1a740 1 --2- 172.21.15.38:0/1972518661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b2b2f0 0x560292efdb40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.847+0000 7fb242ffd640 0 client.16599 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.848+0000 7fb270c1a740 1 -- 172.21.15.38:0/1972518661 >> 172.21.15.38:0/1972518661 conn(0x560292b48300 msgr2=0x560292eee810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.848+0000 7fb270c1a740 1 -- 172.21.15.38:0/1972518661 shutdown_connections 2024-05-21T15:39:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.848+0000 7fb270c1a740 1 -- 172.21.15.38:0/1972518661 wait complete. 2024-05-21T15:39:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.848+0000 7fb270c1a740 20 client.16599 trim_cache size 0 max 16384 2024-05-21T15:39:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.870+0000 7fb270c1a740 1 Processor -- start 2024-05-21T15:39:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.870+0000 7fb270c1a740 1 -- start start 2024-05-21T15:39:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.870+0000 7fb270c1a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b12d80 0x560292eef160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.870+0000 7fb270c1a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 0x560292eef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.870+0000 7fb270c1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b334d0 0x560292efc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.870+0000 7fb270c1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb2641c1a40 con 0x560292b334d0 2024-05-21T15:39:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.870+0000 7fb270c1a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb2641c3670 con 0x560292b12d80 2024-05-21T15:39:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.870+0000 7fb270c1a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb2641c2110 con 0x560292b2b2f0 2024-05-21T15:39:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.870+0000 7fb263fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b334d0 0x560292efc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.871+0000 7fb262ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 0x560292eef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.871+0000 7fb2637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b12d80 0x560292eef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.871+0000 7fb263fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b334d0 0x560292efc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50682/0 (socket says 172.21.15.38:50682) 2024-05-21T15:39:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.871+0000 7fb263fff640 1 -- 172.21.15.38:0/2295578369 learned_addr learned my addr 172.21.15.38:0/2295578369 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.871+0000 7fb263fff640 1 -- 172.21.15.38:0/2295578369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 msgr2=0x560292eef6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.871+0000 7fb263fff640 1 --2- 172.21.15.38:0/2295578369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 0x560292eef6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.871+0000 7fb263fff640 1 -- 172.21.15.38:0/2295578369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b12d80 msgr2=0x560292eef160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.871+0000 7fb263fff640 1 --2- 172.21.15.38:0/2295578369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b12d80 0x560292eef160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.871+0000 7fb263fff640 1 -- 172.21.15.38:0/2295578369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560292dfb0d0 con 0x560292b334d0 2024-05-21T15:39:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.871+0000 7fb2637fe640 1 --2- 172.21.15.38:0/2295578369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b12d80 0x560292eef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.871+0000 7fb263fff640 1 --2- 172.21.15.38:0/2295578369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b334d0 0x560292efc1f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009860 tx=0x7fb25c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a34d2b9bb1e9f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.871+0000 7fb2597fa640 1 -- 172.21.15.38:0/2295578369 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c04a020 con 0x560292b334d0 2024-05-21T15:39:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.871+0000 7fb2597fa640 1 -- 172.21.15.38:0/2295578369 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb25c030d80 con 0x560292b334d0 2024-05-21T15:39:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.871+0000 7fb2597fa640 1 -- 172.21.15.38:0/2295578369 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c037070 con 0x560292b334d0 2024-05-21T15:39:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.872+0000 7fb270c1a740 1 -- 172.21.15.38:0/2295578369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b334d0 msgr2=0x560292efc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.872+0000 7fb270c1a740 1 --2- 172.21.15.38:0/2295578369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b334d0 0x560292efc1f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009860 tx=0x7fb25c000bc0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.872+0000 7fb270c1a740 1 -- 172.21.15.38:0/2295578369 shutdown_connections 2024-05-21T15:39:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.872+0000 7fb270c1a740 1 --2- 172.21.15.38:0/2295578369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b334d0 0x560292efc1f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.872+0000 7fb270c1a740 1 --2- 172.21.15.38:0/2295578369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 0x560292eef6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.872+0000 7fb270c1a740 1 --2- 172.21.15.38:0/2295578369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b12d80 0x560292eef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.872+0000 7fb270c1a740 1 -- 172.21.15.38:0/2295578369 >> 172.21.15.38:0/2295578369 conn(0x560292b48300 msgr2=0x560292ef8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.872+0000 7fb270c1a740 1 -- 172.21.15.38:0/2295578369 shutdown_connections 2024-05-21T15:39:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.872+0000 7fb270c1a740 1 -- 172.21.15.38:0/2295578369 wait complete. 2024-05-21T15:39:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.873+0000 7fb270c1a740 1 -- start start 2024-05-21T15:39:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.873+0000 7fb270c1a740 1 Processor -- start 2024-05-21T15:39:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.873+0000 7fb270c1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b12d80 0x560292f01520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.873+0000 7fb270c1a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 0x560292f01a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.873+0000 7fb270c1a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b334d0 0x560292f71980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.873+0000 7fb270c1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb2641c2ee0 con 0x560292b12d80 2024-05-21T15:39:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.873+0000 7fb270c1a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb2641c27e0 con 0x560292b334d0 2024-05-21T15:39:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.873+0000 7fb270c1a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb2641bfa20 con 0x560292b2b2f0 2024-05-21T15:39:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.873+0000 7fb270c1a740 10 client.0 _ll_register_callbacks cb 0x7fb25409b4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.873+0000 7fb2637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b12d80 0x560292f01520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.873+0000 7fb262ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 0x560292f01a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.873+0000 7fb2637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b12d80 0x560292f01520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50686/0 (socket says 172.21.15.38:50686) 2024-05-21T15:39:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.873+0000 7fb2637fe640 1 -- 172.21.15.38:0/3481034993 learned_addr learned my addr 172.21.15.38:0/3481034993 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.873+0000 7fb263fff640 1 --2- 172.21.15.38:0/3481034993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b334d0 0x560292f71980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.874+0000 7fb262ffd640 1 -- 172.21.15.38:0/3481034993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b334d0 msgr2=0x560292f71980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.874+0000 7fb262ffd640 1 --2- 172.21.15.38:0/3481034993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b334d0 0x560292f71980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.874+0000 7fb262ffd640 1 -- 172.21.15.38:0/3481034993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b12d80 msgr2=0x560292f01520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.874+0000 7fb262ffd640 1 --2- 172.21.15.38:0/3481034993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b12d80 0x560292f01520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.874+0000 7fb262ffd640 1 -- 172.21.15.38:0/3481034993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560292f02050 con 0x560292b2b2f0 2024-05-21T15:39:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.874+0000 7fb2637fe640 1 --2- 172.21.15.38:0/3481034993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b12d80 0x560292f01520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.874+0000 7fb262ffd640 1 --2- 172.21.15.38:0/3481034993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 0x560292f01a60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x560292f0bd80 tx=0x7fb24c011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46200751a9022096 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.874+0000 7fb242ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:39:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.874+0000 7fb242ffd640 1 -- 172.21.15.38:0/3481034993 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c01b040 con 0x560292b2b2f0 2024-05-21T15:39:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.874+0000 7fb242ffd640 1 -- 172.21.15.38:0/3481034993 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fb24c029260 con 0x560292b2b2f0 2024-05-21T15:39:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.874+0000 7fb270c1a740 1 -- 172.21.15.38:0/3481034993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb25c030d80 con 0x560292b2b2f0 2024-05-21T15:39:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.874+0000 7fb242ffd640 1 -- 172.21.15.38:0/3481034993 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb24c029460 con 0x560292b2b2f0 2024-05-21T15:39:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.874+0000 7fb270c1a740 10 client.26335 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.874+0000 7fb270c1a740 1 -- 172.21.15.38:0/3481034993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560292f72fe0 con 0x560292b2b2f0 2024-05-21T15:39:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.875+0000 7fb242ffd640 1 -- 172.21.15.38:0/3481034993 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb25c030d80 con 0x560292b2b2f0 2024-05-21T15:39:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.875+0000 7fb242ffd640 1 -- 172.21.15.38:0/3481034993 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fb24c029ce0 con 0x560292b2b2f0 2024-05-21T15:39:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.875+0000 7fb270c1a740 10 client.26335 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.875+0000 7fb270c1a740 10 client.26335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.875+0000 7fb270c1a740 20 client.26335 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.875+0000 7fb270c1a740 1 --2- 172.21.15.38:0/3481034993 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560292b541b0 0x560292f75a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.875+0000 7fb270c1a740 4 client.26335 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:39:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.875+0000 7fb270c1a740 1 -- 172.21.15.38:0/3481034993 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560292e5fdc0 con 0x560292b541b0 2024-05-21T15:39:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.875+0000 7fb2637fe640 1 --2- 172.21.15.38:0/3481034993 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560292b541b0 0x560292f75a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.876+0000 7fb242ffd640 10 client.26335 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:03.876+0000 7fb2637fe640 1 --2- 172.21.15.38:0/3481034993 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560292b541b0 0x560292f75a90 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=558935076fcd8901 server_cookie=4b632254853a7dda in_seq=0 out_seq=0 2024-05-21T15:39:04.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:03 smithi045 ceph-mon[36409]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 MiB/s wr, 13.04k op/s 2024-05-21T15:39:04.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:03 smithi038 ceph-mon[31995]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 MiB/s wr, 13.04k op/s 2024-05-21T15:39:04.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:03 smithi053 ceph-mon[36496]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 MiB/s wr, 13.04k op/s 2024-05-21T15:39:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.666+0000 7fb242ffd640 1 -- 172.21.15.38:0/3481034993 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560292e5fdc0 con 0x560292b541b0 2024-05-21T15:39:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.666+0000 7fb242ffd640 10 client.26335 handle_command_reply: tid=0 2024-05-21T15:39:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.666+0000 7fb270c1a740 1 client.26335 shutdown 2024-05-21T15:39:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.666+0000 7fb270c1a740 1 -- 172.21.15.38:0/3481034993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 msgr2=0x560292f01a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.667+0000 7fb270c1a740 1 --2- 172.21.15.38:0/3481034993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 0x560292f01a60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x560292f0bd80 tx=0x7fb24c011000 comp rx=0 tx=0).stop 2024-05-21T15:39:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.667+0000 7fb270c1a740 1 -- 172.21.15.38:0/3481034993 shutdown_connections 2024-05-21T15:39:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.667+0000 7fb270c1a740 1 --2- 172.21.15.38:0/3481034993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560292b334d0 0x560292f71980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.667+0000 7fb270c1a740 1 --2- 172.21.15.38:0/3481034993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560292b2b2f0 0x560292f01a60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.667+0000 7fb270c1a740 1 --2- 172.21.15.38:0/3481034993 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560292b541b0 0x560292f75a90 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.667+0000 7fb242ffd640 0 client.26335 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.667+0000 7fb270c1a740 1 --2- 172.21.15.38:0/3481034993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560292b12d80 0x560292f01520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.667+0000 7fb270c1a740 1 -- 172.21.15.38:0/3481034993 >> 172.21.15.38:0/3481034993 conn(0x560292b48300 msgr2=0x560292eef160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.667+0000 7fb270c1a740 1 -- 172.21.15.38:0/3481034993 shutdown_connections 2024-05-21T15:39:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.668+0000 7fb270c1a740 1 -- 172.21.15.38:0/3481034993 wait complete. 2024-05-21T15:39:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.668+0000 7fb270c1a740 20 client.26335 trim_cache size 0 max 16384 2024-05-21T15:39:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:39:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:39:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:39:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:39:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.669+0000 7fb269d79640 1 -- 172.21.15.38:0/2846393574 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb2500a2e20 msgr2=0x7fb2500a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.669+0000 7fb269d79640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb2500a2e20 0x7fb2500a5330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb26411bc50 tx=0x7fb24c0090e0 comp rx=0 tx=0).stop 2024-05-21T15:39:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.669+0000 7fb269d79640 1 -- 172.21.15.38:0/2846393574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2641093d0 msgr2=0x7fb26411abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.669+0000 7fb269d79640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2641093d0 0x7fb26411abb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb25400c470 tx=0x7fb25400c930 comp rx=0 tx=0).stop 2024-05-21T15:39:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.669+0000 7fb269d79640 1 -- 172.21.15.38:0/2846393574 shutdown_connections 2024-05-21T15:39:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.669+0000 7fb269d79640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb26410b000 0x7fb26411e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.669+0000 7fb269d79640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26410a300 0x7fb26411b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.669+0000 7fb269d79640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fb2500a2e20 0x7fb2500a5330 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.669+0000 7fb269d79640 1 --2- 172.21.15.38:0/2846393574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2641093d0 0x7fb26411abb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.670+0000 7fb269d79640 1 -- 172.21.15.38:0/2846393574 >> 172.21.15.38:0/2846393574 conn(0x7fb26406f4c0 msgr2=0x7fb2640718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.670+0000 7fb269d79640 1 -- 172.21.15.38:0/2846393574 shutdown_connections 2024-05-21T15:39:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:04.670+0000 7fb269d79640 1 -- 172.21.15.38:0/2846393574 wait complete. 2024-05-21T15:39:06.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:05 smithi053 ceph-mon[36496]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 56 MiB/s wr, 11.51k op/s 2024-05-21T15:39:06.006 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:05 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:39:06.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:05 smithi045 ceph-mon[36409]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 56 MiB/s wr, 11.51k op/s 2024-05-21T15:39:06.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:05 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:39:06.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:05 smithi038 ceph-mon[31995]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 56 MiB/s wr, 11.51k op/s 2024-05-21T15:39:06.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:05 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:39:07.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:39:07.290 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:39:07.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.421+0000 7fdf8d0fe640 1 -- 172.21.15.38:0/1151553892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88101980 msgr2=0x7fdf88102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.421+0000 7fdf8d0fe640 1 --2- 172.21.15.38:0/1151553892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88101980 0x7fdf88102680 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009900 tx=0x7fdf7802a410 comp rx=0 tx=0).stop 2024-05-21T15:39:07.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.421+0000 7fdf8d0fe640 1 -- 172.21.15.38:0/1151553892 shutdown_connections 2024-05-21T15:39:07.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.421+0000 7fdf8d0fe640 1 --2- 172.21.15.38:0/1151553892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf881078c0 0x7fdf88109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.421+0000 7fdf8d0fe640 1 --2- 172.21.15.38:0/1151553892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf88102bc0 0x7fdf88102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.421+0000 7fdf8d0fe640 1 --2- 172.21.15.38:0/1151553892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88101980 0x7fdf88102680 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.421+0000 7fdf8d0fe640 1 -- 172.21.15.38:0/1151553892 >> 172.21.15.38:0/1151553892 conn(0x7fdf880fcb70 msgr2=0x7fdf880ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:07.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.421+0000 7fdf8d0fe640 1 -- 172.21.15.38:0/1151553892 shutdown_connections 2024-05-21T15:39:07.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.421+0000 7fdf8d0fe640 1 -- 172.21.15.38:0/1151553892 wait complete. 2024-05-21T15:39:07.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.422+0000 7fdf8d0fe640 1 Processor -- start 2024-05-21T15:39:07.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.422+0000 7fdf8d0fe640 1 -- start start 2024-05-21T15:39:07.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.422+0000 7fdf8d0fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf88101980 0x7fdf88069a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.422+0000 7fdf8d0fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88102bc0 0x7fdf8819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.422+0000 7fdf8d0fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf881078c0 0x7fdf881a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.422+0000 7fdf8d0fe640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8810d620 con 0x7fdf88102bc0 2024-05-21T15:39:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.422+0000 7fdf8d0fe640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8810d4a0 con 0x7fdf88101980 2024-05-21T15:39:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.422+0000 7fdf8d0fe640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8810d7a0 con 0x7fdf881078c0 2024-05-21T15:39:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.423+0000 7fdf8659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88102bc0 0x7fdf8819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.423+0000 7fdf8759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf881078c0 0x7fdf881a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.423+0000 7fdf8659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88102bc0 0x7fdf8819e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40970/0 (socket says 172.21.15.38:40970) 2024-05-21T15:39:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.423+0000 7fdf8659c640 1 -- 172.21.15.38:0/23901470 learned_addr learned my addr 172.21.15.38:0/23901470 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.423+0000 7fdf86d9d640 1 --2- 172.21.15.38:0/23901470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf88101980 0x7fdf88069a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.423+0000 7fdf8759e640 1 -- 172.21.15.38:0/23901470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf88101980 msgr2=0x7fdf88069a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.423+0000 7fdf8759e640 1 --2- 172.21.15.38:0/23901470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf88101980 0x7fdf88069a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.423+0000 7fdf8759e640 1 -- 172.21.15.38:0/23901470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88102bc0 msgr2=0x7fdf8819e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.423+0000 7fdf8759e640 1 --2- 172.21.15.38:0/23901470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88102bc0 0x7fdf8819e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.423+0000 7fdf8759e640 1 -- 172.21.15.38:0/23901470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf881a0db0 con 0x7fdf881078c0 2024-05-21T15:39:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.423+0000 7fdf8659c640 1 --2- 172.21.15.38:0/23901470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88102bc0 0x7fdf8819e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.423+0000 7fdf8759e640 1 --2- 172.21.15.38:0/23901470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf881078c0 0x7fdf881a0590 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000e940 tx=0x7fdf8000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.424+0000 7fdf6ffff640 1 -- 172.21.15.38:0/23901470 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80004d20 con 0x7fdf881078c0 2024-05-21T15:39:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.424+0000 7fdf8d0fe640 1 -- 172.21.15.38:0/23901470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf881a10a0 con 0x7fdf881078c0 2024-05-21T15:39:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.424+0000 7fdf8d0fe640 1 -- 172.21.15.38:0/23901470 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf881a1640 con 0x7fdf881078c0 2024-05-21T15:39:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.424+0000 7fdf6ffff640 1 -- 172.21.15.38:0/23901470 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf80005020 con 0x7fdf881078c0 2024-05-21T15:39:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.424+0000 7fdf6ffff640 1 -- 172.21.15.38:0/23901470 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf800116d0 con 0x7fdf881078c0 2024-05-21T15:39:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.426+0000 7fdf6ffff640 1 -- 172.21.15.38:0/23901470 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdf80011930 con 0x7fdf881078c0 2024-05-21T15:39:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.427+0000 7fdf93fa2740 1 Processor -- start 2024-05-21T15:39:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.427+0000 7fdf93fa2740 1 -- start start 2024-05-21T15:39:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.427+0000 7fdf93fa2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310eb6390 0x556311259fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.427+0000 7fdf93fa2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310e8d4d0 0x55631125a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.427+0000 7fdf6ffff640 1 --2- 172.21.15.38:0/23901470 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdf700a2b90 0x7fdf700a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.427+0000 7fdf93fa2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e956b0 0x556311261440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.427+0000 7fdf93fa2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5563111bad00 con 0x556310eb6390 2024-05-21T15:39:07.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.427+0000 7fdf93fa2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5563111624c0 con 0x556310e8d4d0 2024-05-21T15:39:07.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.427+0000 7fdf93fa2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5563111b6cc0 con 0x556310e956b0 2024-05-21T15:39:07.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.427+0000 7fdf8659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310eb6390 0x556311259fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.428+0000 7fdf8659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310eb6390 0x556311259fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40972/0 (socket says 172.21.15.38:40972) 2024-05-21T15:39:07.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.428+0000 7fdf8659c640 1 -- 172.21.15.38:0/936004109 learned_addr learned my addr 172.21.15.38:0/936004109 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:07.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.428+0000 7fdf8759e640 1 --2- 172.21.15.38:0/936004109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e956b0 0x556311261440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.428+0000 7fdf8659c640 1 --2- 172.21.15.38:0/936004109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310e8d4d0 0x55631125a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.428+0000 7fdf6ffff640 1 -- 172.21.15.38:0/23901470 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdf800cf450 con 0x7fdf881078c0 2024-05-21T15:39:07.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.428+0000 7fdf86d9d640 1 --2- 172.21.15.38:0/23901470 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdf700a2b90 0x7fdf700a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.428+0000 7fdf8659c640 1 -- 172.21.15.38:0/936004109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e956b0 msgr2=0x556311261440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.428+0000 7fdf8659c640 1 --2- 172.21.15.38:0/936004109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e956b0 0x556311261440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.428+0000 7fdf8659c640 1 -- 172.21.15.38:0/936004109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310e8d4d0 msgr2=0x55631125a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.428+0000 7fdf8659c640 1 --2- 172.21.15.38:0/936004109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310e8d4d0 0x55631125a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.428+0000 7fdf8659c640 1 -- 172.21.15.38:0/936004109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556310f987d0 con 0x556310eb6390 2024-05-21T15:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.428+0000 7fdf8659c640 1 --2- 172.21.15.38:0/936004109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310eb6390 0x556311259fc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c013ad0 tx=0x7fdf7c009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40deccb7432a1b09 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.428+0000 7fdf86d9d640 1 --2- 172.21.15.38:0/23901470 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdf700a2b90 0x7fdf700a50a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdf7802aa00 tx=0x7fdf780315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.429+0000 7fdf6d7fa640 1 -- 172.21.15.38:0/936004109 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c00fa40 con 0x556310eb6390 2024-05-21T15:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.429+0000 7fdf6d7fa640 1 -- 172.21.15.38:0/936004109 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf7c005500 con 0x556310eb6390 2024-05-21T15:39:07.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.429+0000 7fdf6d7fa640 1 -- 172.21.15.38:0/936004109 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c004b90 con 0x556310eb6390 2024-05-21T15:39:07.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.429+0000 7fdf93fa2740 1 -- 172.21.15.38:0/936004109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310eb6390 msgr2=0x556311259fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.429+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/936004109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310eb6390 0x556311259fc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c013ad0 tx=0x7fdf7c009c10 comp rx=0 tx=0).stop 2024-05-21T15:39:07.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.430+0000 7fdf93fa2740 1 -- 172.21.15.38:0/936004109 shutdown_connections 2024-05-21T15:39:07.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.430+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/936004109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e956b0 0x556311261440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.430+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/936004109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310e8d4d0 0x55631125a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.430+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/936004109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310eb6390 0x556311259fc0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.430+0000 7fdf93fa2740 1 -- 172.21.15.38:0/936004109 >> 172.21.15.38:0/936004109 conn(0x556310eaa4e0 msgr2=0x556311251540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:07.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.430+0000 7fdf93fa2740 1 -- 172.21.15.38:0/936004109 shutdown_connections 2024-05-21T15:39:07.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.430+0000 7fdf93fa2740 1 -- 172.21.15.38:0/936004109 wait complete. 2024-05-21T15:39:07.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.430+0000 7fdf93fa2740 1 -- start start 2024-05-21T15:39:07.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.430+0000 7fdf93fa2740 1 Processor -- start 2024-05-21T15:39:07.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.430+0000 7fdf93fa2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e8d4d0 0x556311263ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.430+0000 7fdf93fa2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e956b0 0x556311264530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.430+0000 7fdf93fa2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310eb6390 0x55631126b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf93fa2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556311261e90 con 0x556310e8d4d0 2024-05-21T15:39:07.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf93fa2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556311175b90 con 0x556310eb6390 2024-05-21T15:39:07.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf93fa2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556311262010 con 0x556310e956b0 2024-05-21T15:39:07.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf93fa2740 10 client.0 _ll_register_callbacks cb 0x5563111b1da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:07.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf86d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e8d4d0 0x556311263ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf86d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e8d4d0 0x556311263ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40974/0 (socket says 172.21.15.38:40974) 2024-05-21T15:39:07.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf86d9d640 1 -- 172.21.15.38:0/2292616493 learned_addr learned my addr 172.21.15.38:0/2292616493 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:07.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf8659c640 1 --2- 172.21.15.38:0/2292616493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e956b0 0x556311264530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf8759e640 1 --2- 172.21.15.38:0/2292616493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310eb6390 0x55631126b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf86d9d640 1 -- 172.21.15.38:0/2292616493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e956b0 msgr2=0x556311264530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:07.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf86d9d640 1 --2- 172.21.15.38:0/2292616493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e956b0 0x556311264530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf86d9d640 1 -- 172.21.15.38:0/2292616493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310eb6390 msgr2=0x55631126b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:07.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf86d9d640 1 --2- 172.21.15.38:0/2292616493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310eb6390 0x55631126b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf86d9d640 1 -- 172.21.15.38:0/2292616493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556310ece3f0 con 0x556310e8d4d0 2024-05-21T15:39:07.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf8659c640 1 --2- 172.21.15.38:0/2292616493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e956b0 0x556311264530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:07.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf86d9d640 1 --2- 172.21.15.38:0/2292616493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e8d4d0 0x556311263ff0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009900 tx=0x7fdf78009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db36d87a6ec0a7f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:07.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:07.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf6aff5640 1 -- 172.21.15.38:0/2292616493 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7804c020 con 0x556310e8d4d0 2024-05-21T15:39:07.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf93fa2740 1 -- 172.21.15.38:0/2292616493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf7c005500 con 0x556310e8d4d0 2024-05-21T15:39:07.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.431+0000 7fdf6aff5640 1 -- 172.21.15.38:0/2292616493 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdf780053d0 con 0x556310e8d4d0 2024-05-21T15:39:07.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.432+0000 7fdf6aff5640 1 -- 172.21.15.38:0/2292616493 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdf78057050 con 0x556310e8d4d0 2024-05-21T15:39:07.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.432+0000 7fdf93fa2740 10 client.16617 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:07.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.432+0000 7fdf93fa2740 1 -- 172.21.15.38:0/2292616493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556310e8e0f0 con 0x556310e8d4d0 2024-05-21T15:39:07.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.432+0000 7fdf6aff5640 1 -- 172.21.15.38:0/2292616493 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf7c005500 con 0x556310e8d4d0 2024-05-21T15:39:07.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.432+0000 7fdf6aff5640 1 -- 172.21.15.38:0/2292616493 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fdf780329c0 con 0x556310e8d4d0 2024-05-21T15:39:07.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.432+0000 7fdf93fa2740 10 client.16617 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:07.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.432+0000 7fdf93fa2740 10 client.16617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:07.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.432+0000 7fdf93fa2740 20 client.16617 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:07.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.432+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/2292616493 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556310e74f60 0x556311271da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.432+0000 7fdf93fa2740 4 client.16617 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:07.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.432+0000 7fdf93fa2740 1 -- 172.21.15.38:0/2292616493 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563111c8430 con 0x556310e74f60 2024-05-21T15:39:07.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.432+0000 7fdf8659c640 1 --2- 172.21.15.38:0/2292616493 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556310e74f60 0x556311271da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.433+0000 7fdf6aff5640 10 client.16617 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:07.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.433+0000 7fdf8659c640 1 --2- 172.21.15.38:0/2292616493 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556310e74f60 0x556311271da0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feefb09b58ac7716 server_cookie=76e64098319e77ca in_seq=0 out_seq=0 2024-05-21T15:39:07.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.433+0000 7fdf6aff5640 1 -- 172.21.15.38:0/2292616493 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5563111c8430 con 0x556310e74f60 2024-05-21T15:39:07.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.433+0000 7fdf6aff5640 10 client.16617 handle_command_reply: tid=0 2024-05-21T15:39:07.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.434+0000 7fdf93fa2740 1 client.16617 shutdown 2024-05-21T15:39:07.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.434+0000 7fdf93fa2740 1 -- 172.21.15.38:0/2292616493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e8d4d0 msgr2=0x556311263ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.434+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/2292616493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e8d4d0 0x556311263ff0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009900 tx=0x7fdf78009260 comp rx=0 tx=0).stop 2024-05-21T15:39:07.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.434+0000 7fdf93fa2740 1 -- 172.21.15.38:0/2292616493 shutdown_connections 2024-05-21T15:39:07.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.434+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/2292616493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310eb6390 0x55631126b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.434+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/2292616493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e956b0 0x556311264530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.434+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/2292616493 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556310e74f60 0x556311271da0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.434+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/2292616493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e8d4d0 0x556311263ff0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.434+0000 7fdf6aff5640 0 client.16617 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:07.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.434+0000 7fdf93fa2740 1 -- 172.21.15.38:0/2292616493 >> 172.21.15.38:0/2292616493 conn(0x556310eaa4e0 msgr2=0x5563112505c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:07.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.434+0000 7fdf93fa2740 1 -- 172.21.15.38:0/2292616493 shutdown_connections 2024-05-21T15:39:07.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.434+0000 7fdf93fa2740 1 -- 172.21.15.38:0/2292616493 wait complete. 2024-05-21T15:39:07.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.434+0000 7fdf93fa2740 20 client.16617 trim_cache size 0 max 16384 2024-05-21T15:39:07.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.452+0000 7fdf93fa2740 1 Processor -- start 2024-05-21T15:39:07.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.452+0000 7fdf93fa2740 1 -- start start 2024-05-21T15:39:07.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf93fa2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e8d4d0 0x556311262f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf93fa2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310e956b0 0x55631126b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf93fa2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310eb6390 0x556311276590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf93fa2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881a8240 con 0x556310e8d4d0 2024-05-21T15:39:07.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf93fa2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881a9e70 con 0x556310e956b0 2024-05-21T15:39:07.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf93fa2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881a8910 con 0x556310eb6390 2024-05-21T15:39:07.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf86d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e8d4d0 0x556311262f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf86d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e8d4d0 0x556311262f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40984/0 (socket says 172.21.15.38:40984) 2024-05-21T15:39:07.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf86d9d640 1 -- 172.21.15.38:0/1485661693 learned_addr learned my addr 172.21.15.38:0/1485661693 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:07.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf8759e640 1 --2- 172.21.15.38:0/1485661693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310eb6390 0x556311276590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf8659c640 1 --2- 172.21.15.38:0/1485661693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310e956b0 0x55631126b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf86d9d640 1 -- 172.21.15.38:0/1485661693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310eb6390 msgr2=0x556311276590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:07.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf86d9d640 1 --2- 172.21.15.38:0/1485661693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310eb6390 0x556311276590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf86d9d640 1 -- 172.21.15.38:0/1485661693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310e956b0 msgr2=0x55631126b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:07.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf86d9d640 1 --2- 172.21.15.38:0/1485661693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310e956b0 0x55631126b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf86d9d640 1 -- 172.21.15.38:0/1485661693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556311242990 con 0x556310e8d4d0 2024-05-21T15:39:07.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf86d9d640 1 --2- 172.21.15.38:0/1485661693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e8d4d0 0x556311262f10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009230 tx=0x7fdf78003850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923a806934a13ae3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:07.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.453+0000 7fdf6cff9640 1 -- 172.21.15.38:0/1485661693 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7805f020 con 0x556310e8d4d0 2024-05-21T15:39:07.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.454+0000 7fdf6cff9640 1 -- 172.21.15.38:0/1485661693 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf78032ba0 con 0x556310e8d4d0 2024-05-21T15:39:07.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.454+0000 7fdf6cff9640 1 -- 172.21.15.38:0/1485661693 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7804c070 con 0x556310e8d4d0 2024-05-21T15:39:07.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.454+0000 7fdf93fa2740 1 -- 172.21.15.38:0/1485661693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e8d4d0 msgr2=0x556311262f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.454+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/1485661693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e8d4d0 0x556311262f10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009230 tx=0x7fdf78003850 comp rx=0 tx=0).stop 2024-05-21T15:39:07.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.454+0000 7fdf93fa2740 1 -- 172.21.15.38:0/1485661693 shutdown_connections 2024-05-21T15:39:07.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.454+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/1485661693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310eb6390 0x556311276590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.454+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/1485661693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310e956b0 0x55631126b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.454+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/1485661693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e8d4d0 0x556311262f10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.454+0000 7fdf93fa2740 1 -- 172.21.15.38:0/1485661693 >> 172.21.15.38:0/1485661693 conn(0x556310eaa4e0 msgr2=0x556311264230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:07.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf93fa2740 1 -- 172.21.15.38:0/1485661693 shutdown_connections 2024-05-21T15:39:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf93fa2740 1 -- 172.21.15.38:0/1485661693 wait complete. 2024-05-21T15:39:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf93fa2740 1 -- start start 2024-05-21T15:39:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf93fa2740 1 Processor -- start 2024-05-21T15:39:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf93fa2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e8d4d0 0x55631126a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf93fa2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e956b0 0x55631126ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf93fa2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310eb6390 0x5563112d8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf93fa2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881a96e0 con 0x556310e956b0 2024-05-21T15:39:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf93fa2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881a8fe0 con 0x556310eb6390 2024-05-21T15:39:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf93fa2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881a6220 con 0x556310e8d4d0 2024-05-21T15:39:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf93fa2740 10 client.0 _ll_register_callbacks cb 0x7fdf78040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf8659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e956b0 0x55631126ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf8659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e956b0 0x55631126ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40988/0 (socket says 172.21.15.38:40988) 2024-05-21T15:39:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf8659c640 1 -- 172.21.15.38:0/3556298032 learned_addr learned my addr 172.21.15.38:0/3556298032 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf86d9d640 1 --2- 172.21.15.38:0/3556298032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e8d4d0 0x55631126a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.455+0000 7fdf8759e640 1 --2- 172.21.15.38:0/3556298032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310eb6390 0x5563112d8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.456+0000 7fdf8659c640 1 -- 172.21.15.38:0/3556298032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e8d4d0 msgr2=0x55631126a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.456+0000 7fdf8659c640 1 --2- 172.21.15.38:0/3556298032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e8d4d0 0x55631126a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.456+0000 7fdf8659c640 1 -- 172.21.15.38:0/3556298032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310eb6390 msgr2=0x5563112d8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.456+0000 7fdf8659c640 1 --2- 172.21.15.38:0/3556298032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310eb6390 0x5563112d8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.456+0000 7fdf8659c640 1 -- 172.21.15.38:0/3556298032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556311264940 con 0x556310e956b0 2024-05-21T15:39:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.456+0000 7fdf8659c640 1 --2- 172.21.15.38:0/3556298032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e956b0 0x55631126ad20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x55631125b190 tx=0x7fdf7c009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6292cf619eb98bd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.456+0000 7fdf6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.456+0000 7fdf6aff5640 1 -- 172.21.15.38:0/3556298032 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c02c030 con 0x556310e956b0 2024-05-21T15:39:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.456+0000 7fdf6aff5640 1 -- 172.21.15.38:0/3556298032 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdf7c00fb40 con 0x556310e956b0 2024-05-21T15:39:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.456+0000 7fdf93fa2740 1 -- 172.21.15.38:0/3556298032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf78032ba0 con 0x556310e956b0 2024-05-21T15:39:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.456+0000 7fdf6aff5640 1 -- 172.21.15.38:0/3556298032 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdf7c00fd40 con 0x556310e956b0 2024-05-21T15:39:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.456+0000 7fdf93fa2740 10 client.16623 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.456+0000 7fdf93fa2740 1 -- 172.21.15.38:0/3556298032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5563112d9a00 con 0x556310e956b0 2024-05-21T15:39:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.456+0000 7fdf6aff5640 1 -- 172.21.15.38:0/3556298032 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf78032ba0 con 0x556310e956b0 2024-05-21T15:39:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.457+0000 7fdf6aff5640 1 -- 172.21.15.38:0/3556298032 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fdf7c0035f0 con 0x556310e956b0 2024-05-21T15:39:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.457+0000 7fdf93fa2740 10 client.16623 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.457+0000 7fdf93fa2740 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.457+0000 7fdf93fa2740 20 client.16623 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.457+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/3556298032 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556310e4a3a0 0x5563112dc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.457+0000 7fdf93fa2740 4 client.16623 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:39:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.457+0000 7fdf93fa2740 1 -- 172.21.15.38:0/3556298032 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5563111c2780 con 0x556310e4a3a0 2024-05-21T15:39:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.457+0000 7fdf86d9d640 1 --2- 172.21.15.38:0/3556298032 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556310e4a3a0 0x5563112dc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.457+0000 7fdf6aff5640 10 client.16623 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.457+0000 7fdf86d9d640 1 --2- 172.21.15.38:0/3556298032 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556310e4a3a0 0x5563112dc310 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa05a8992aed9ed server_cookie=a539359208f5e8df in_seq=0 out_seq=0 2024-05-21T15:39:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf6aff5640 1 -- 172.21.15.38:0/3556298032 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5563111c2780 con 0x556310e4a3a0 2024-05-21T15:39:07.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf6aff5640 10 client.16623 handle_command_reply: tid=0 2024-05-21T15:39:07.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf93fa2740 1 client.16623 shutdown 2024-05-21T15:39:07.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf93fa2740 1 -- 172.21.15.38:0/3556298032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e956b0 msgr2=0x55631126ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/3556298032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e956b0 0x55631126ad20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x55631125b190 tx=0x7fdf7c009fa0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf93fa2740 1 -- 172.21.15.38:0/3556298032 shutdown_connections 2024-05-21T15:39:07.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/3556298032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556310eb6390 0x5563112d8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/3556298032 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556310e4a3a0 0x5563112dc310 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/3556298032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556310e956b0 0x55631126ad20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf93fa2740 1 --2- 172.21.15.38:0/3556298032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556310e8d4d0 0x55631126a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf6aff5640 0 client.16623 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:07.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf93fa2740 1 -- 172.21.15.38:0/3556298032 >> 172.21.15.38:0/3556298032 conn(0x556310eaa4e0 msgr2=0x556311264230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:07.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf93fa2740 1 -- 172.21.15.38:0/3556298032 shutdown_connections 2024-05-21T15:39:07.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf93fa2740 1 -- 172.21.15.38:0/3556298032 wait complete. 2024-05-21T15:39:07.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf93fa2740 20 client.16623 trim_cache size 0 max 16384 2024-05-21T15:39:07.468 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:39:07.468 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-05-21T15:39:07.468 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-05-21T15:39:07.468 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:39:07.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf8d0fe640 1 -- 172.21.15.38:0/23901470 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdf700a2b90 msgr2=0x7fdf700a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf8d0fe640 1 --2- 172.21.15.38:0/23901470 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdf700a2b90 0x7fdf700a50a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdf7802aa00 tx=0x7fdf780315b0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf8d0fe640 1 -- 172.21.15.38:0/23901470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf881078c0 msgr2=0x7fdf881a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf8d0fe640 1 --2- 172.21.15.38:0/23901470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf881078c0 0x7fdf881a0590 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000e940 tx=0x7fdf8000ee00 comp rx=0 tx=0).stop 2024-05-21T15:39:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf8d0fe640 1 -- 172.21.15.38:0/23901470 shutdown_connections 2024-05-21T15:39:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf8d0fe640 1 --2- 172.21.15.38:0/23901470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf881078c0 0x7fdf881a0590 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf8d0fe640 1 --2- 172.21.15.38:0/23901470 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdf700a2b90 0x7fdf700a50a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf8d0fe640 1 --2- 172.21.15.38:0/23901470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf88102bc0 0x7fdf8819e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.458+0000 7fdf8d0fe640 1 --2- 172.21.15.38:0/23901470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf88101980 0x7fdf88069a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.459+0000 7fdf8d0fe640 1 -- 172.21.15.38:0/23901470 >> 172.21.15.38:0/23901470 conn(0x7fdf880fcb70 msgr2=0x7fdf881038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.459+0000 7fdf8d0fe640 1 -- 172.21.15.38:0/23901470 shutdown_connections 2024-05-21T15:39:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.459+0000 7fdf8d0fe640 1 -- 172.21.15.38:0/23901470 wait complete. 2024-05-21T15:39:07.471 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:39:07.471 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:39:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.596+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/2072102011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd8102bf0 msgr2=0x7f3cd8103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.596+0000 7f3cdf8c3640 1 --2- 172.21.15.38:0/2072102011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd8102bf0 0x7f3cd8103000 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009900 tx=0x7f3ccc02a410 comp rx=0 tx=0).stop 2024-05-21T15:39:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.596+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/2072102011 shutdown_connections 2024-05-21T15:39:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.596+0000 7f3cdf8c3640 1 --2- 172.21.15.38:0/2072102011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cd81078f0 0x7f3cd8109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.596+0000 7f3cdf8c3640 1 --2- 172.21.15.38:0/2072102011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd8102bf0 0x7f3cd8103000 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.596+0000 7f3cdf8c3640 1 --2- 172.21.15.38:0/2072102011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd81019b0 0x7f3cd81026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.596+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/2072102011 >> 172.21.15.38:0/2072102011 conn(0x7f3cd80fcba0 msgr2=0x7f3cd80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.596+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/2072102011 shutdown_connections 2024-05-21T15:39:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.596+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/2072102011 wait complete. 2024-05-21T15:39:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.596+0000 7f3cdf8c3640 1 Processor -- start 2024-05-21T15:39:07.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.596+0000 7f3cdf8c3640 1 -- start start 2024-05-21T15:39:07.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdf8c3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cd81019b0 0x7f3cd80747f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdf8c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd8102bf0 0x7f3cd819dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdf8c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd81078f0 0x7f3cd81a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdf8c3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd810d680 con 0x7f3cd8102bf0 2024-05-21T15:39:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdf8c3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd810d500 con 0x7f3cd81078f0 2024-05-21T15:39:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdf8c3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd810d800 con 0x7f3cd81019b0 2024-05-21T15:39:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdd65f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cd81019b0 0x7f3cd80747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdde60640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd81078f0 0x7f3cd81a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdd65f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cd81019b0 0x7f3cd80747f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:60776/0 (socket says 172.21.15.38:60776) 2024-05-21T15:39:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdde60640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd81078f0 0x7f3cd81a04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:39758/0 (socket says 172.21.15.38:39758) 2024-05-21T15:39:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdd65f640 1 -- 172.21.15.38:0/710758557 learned_addr learned my addr 172.21.15.38:0/710758557 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdce5e640 1 --2- 172.21.15.38:0/710758557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd8102bf0 0x7f3cd819dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdd65f640 1 -- 172.21.15.38:0/710758557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd81078f0 msgr2=0x7f3cd81a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdd65f640 1 --2- 172.21.15.38:0/710758557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd81078f0 0x7f3cd81a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdd65f640 1 -- 172.21.15.38:0/710758557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd8102bf0 msgr2=0x7f3cd819dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdd65f640 1 --2- 172.21.15.38:0/710758557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd8102bf0 0x7f3cd819dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.597+0000 7f3cdd65f640 1 -- 172.21.15.38:0/710758557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd81a0d10 con 0x7f3cd81019b0 2024-05-21T15:39:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.598+0000 7f3cdde60640 1 --2- 172.21.15.38:0/710758557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd81078f0 0x7f3cd81a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.598+0000 7f3cdd65f640 1 --2- 172.21.15.38:0/710758557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cd81019b0 0x7f3cd80747f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800eb90 tx=0x7f3cc800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.598+0000 7f3cd67fc640 1 -- 172.21.15.38:0/710758557 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8003b70 con 0x7f3cd81019b0 2024-05-21T15:39:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.598+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/710758557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd81a0fa0 con 0x7f3cd81019b0 2024-05-21T15:39:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.598+0000 7f3cd67fc640 1 -- 172.21.15.38:0/710758557 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cc800ee90 con 0x7f3cd81019b0 2024-05-21T15:39:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.598+0000 7f3cd67fc640 1 -- 172.21.15.38:0/710758557 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8019800 con 0x7f3cd81019b0 2024-05-21T15:39:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.598+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/710758557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd81a1490 con 0x7f3cd81019b0 2024-05-21T15:39:07.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.599+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/710758557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd8105ff0 con 0x7f3cd81019b0 2024-05-21T15:39:07.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.600+0000 7f3cd67fc640 1 -- 172.21.15.38:0/710758557 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3cc8004b60 con 0x7f3cd81019b0 2024-05-21T15:39:07.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.601+0000 7f3cd67fc640 1 --2- 172.21.15.38:0/710758557 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3cc00a2c30 0x7f3cc00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.601+0000 7f3cdce5e640 1 --2- 172.21.15.38:0/710758557 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3cc00a2c30 0x7f3cc00a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.601+0000 7f3cd67fc640 1 -- 172.21.15.38:0/710758557 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f3cc80cf9f0 con 0x7f3cd81019b0 2024-05-21T15:39:07.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.602+0000 7f3cdce5e640 1 --2- 172.21.15.38:0/710758557 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3cc00a2c30 0x7f3cc00a5140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc005ab0 tx=0x7f3ccc0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:07.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.605+0000 7f3cd67fc640 1 -- 172.21.15.38:0/710758557 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3cc809c930 con 0x7f3cd81019b0 2024-05-21T15:39:07.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.851+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/710758557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3cd802d2a0 con 0x7f3cd81019b0 2024-05-21T15:39:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.852+0000 7f3cd67fc640 1 -- 172.21.15.38:0/710758557 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11922 (secure 0 0 0) 0x7f3cc8003d10 con 0x7f3cd81019b0 2024-05-21T15:39:07.851 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:39:07.851 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:36:59.077696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24551,24523,24602,24550,24515]},"id":1}]} 2024-05-21T15:39:07.852 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 31 2024-05-21T15:39:07.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.855+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/710758557 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3cc00a2c30 msgr2=0x7f3cc00a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.855+0000 7f3cdf8c3640 1 --2- 172.21.15.38:0/710758557 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3cc00a2c30 0x7f3cc00a5140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc005ab0 tx=0x7f3ccc0315b0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.855+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/710758557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cd81019b0 msgr2=0x7f3cd80747f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.855+0000 7f3cdf8c3640 1 --2- 172.21.15.38:0/710758557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cd81019b0 0x7f3cd80747f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800eb90 tx=0x7f3cc800c6a0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.855+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/710758557 shutdown_connections 2024-05-21T15:39:07.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.855+0000 7f3cdf8c3640 1 --2- 172.21.15.38:0/710758557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd81078f0 0x7f3cd81a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.855+0000 7f3cdf8c3640 1 --2- 172.21.15.38:0/710758557 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3cc00a2c30 0x7f3cc00a5140 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.855+0000 7f3cdf8c3640 1 --2- 172.21.15.38:0/710758557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd8102bf0 0x7f3cd819dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.855+0000 7f3cdf8c3640 1 --2- 172.21.15.38:0/710758557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cd81019b0 0x7f3cd80747f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.855+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/710758557 >> 172.21.15.38:0/710758557 conn(0x7f3cd80fcba0 msgr2=0x7f3cd80fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:07.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.855+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/710758557 shutdown_connections 2024-05-21T15:39:07.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.855+0000 7f3cdf8c3640 1 -- 172.21.15.38:0/710758557 wait complete. 2024-05-21T15:39:07.866 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:39:07.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.996+0000 7fe11f2f7640 1 -- 172.21.15.38:0/716679432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe118101980 msgr2=0x7fe118102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.996+0000 7fe11f2f7640 1 --2- 172.21.15.38:0/716679432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe118101980 0x7fe118102680 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe114009900 tx=0x7fe11402a410 comp rx=0 tx=0).stop 2024-05-21T15:39:07.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.996+0000 7fe11f2f7640 1 -- 172.21.15.38:0/716679432 shutdown_connections 2024-05-21T15:39:07.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.996+0000 7fe11f2f7640 1 --2- 172.21.15.38:0/716679432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1181078c0 0x7fe118109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.996+0000 7fe11f2f7640 1 --2- 172.21.15.38:0/716679432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe118102bc0 0x7fe118102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.996+0000 7fe11f2f7640 1 --2- 172.21.15.38:0/716679432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe118101980 0x7fe118102680 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.996+0000 7fe11f2f7640 1 -- 172.21.15.38:0/716679432 >> 172.21.15.38:0/716679432 conn(0x7fe1180fcb70 msgr2=0x7fe1180ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:07.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.996+0000 7fe11f2f7640 1 -- 172.21.15.38:0/716679432 shutdown_connections 2024-05-21T15:39:07.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.996+0000 7fe11f2f7640 1 -- 172.21.15.38:0/716679432 wait complete. 2024-05-21T15:39:07.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.997+0000 7fe11f2f7640 1 Processor -- start 2024-05-21T15:39:07.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.997+0000 7fe11f2f7640 1 -- start start 2024-05-21T15:39:07.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.997+0000 7fe11f2f7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe118101980 0x7fe118198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.997+0000 7fe11f2f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe118102bc0 0x7fe1181986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.997+0000 7fe11f2f7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1181078c0 0x7fe11819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:07.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.997+0000 7fe11f2f7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe11810d5d0 con 0x7fe118102bc0 2024-05-21T15:39:07.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.997+0000 7fe11f2f7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe11810d450 con 0x7fe1181078c0 2024-05-21T15:39:07.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.997+0000 7fe11f2f7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe11810d750 con 0x7fe118101980 2024-05-21T15:39:07.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.997+0000 7fe11c892640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe118102bc0 0x7fe1181986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.997+0000 7fe11d093640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe118101980 0x7fe118198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.997+0000 7fe11c892640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe118102bc0 0x7fe1181986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41040/0 (socket says 172.21.15.38:41040) 2024-05-21T15:39:07.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.997+0000 7fe11d093640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe118101980 0x7fe118198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:60804/0 (socket says 172.21.15.38:60804) 2024-05-21T15:39:07.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.997+0000 7fe11c892640 1 -- 172.21.15.38:0/3898230135 learned_addr learned my addr 172.21.15.38:0/3898230135 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:07.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.997+0000 7fe11d894640 1 --2- 172.21.15.38:0/3898230135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1181078c0 0x7fe11819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:07.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.998+0000 7fe11c892640 1 -- 172.21.15.38:0/3898230135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe118101980 msgr2=0x7fe118198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.998+0000 7fe11c892640 1 --2- 172.21.15.38:0/3898230135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe118101980 0x7fe118198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.998+0000 7fe11c892640 1 -- 172.21.15.38:0/3898230135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1181078c0 msgr2=0x7fe11819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:07.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.998+0000 7fe11c892640 1 --2- 172.21.15.38:0/3898230135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1181078c0 0x7fe11819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:07.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.998+0000 7fe11c892640 1 -- 172.21.15.38:0/3898230135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11819fcb0 con 0x7fe118102bc0 2024-05-21T15:39:07.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.998+0000 7fe11d093640 1 --2- 172.21.15.38:0/3898230135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe118101980 0x7fe118198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:07.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.998+0000 7fe11c892640 1 --2- 172.21.15.38:0/3898230135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe118102bc0 0x7fe1181986b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe108009970 tx=0x7fe108009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:07.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.998+0000 7fe10e7fc640 1 -- 172.21.15.38:0/3898230135 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10800d020 con 0x7fe118102bc0 2024-05-21T15:39:07.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.998+0000 7fe10e7fc640 1 -- 172.21.15.38:0/3898230135 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe108007d40 con 0x7fe118102bc0 2024-05-21T15:39:07.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.998+0000 7fe11f2f7640 1 -- 172.21.15.38:0/3898230135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe118069010 con 0x7fe118102bc0 2024-05-21T15:39:07.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.998+0000 7fe10e7fc640 1 -- 172.21.15.38:0/3898230135 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10800ccc0 con 0x7fe118102bc0 2024-05-21T15:39:07.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:07.998+0000 7fe11f2f7640 1 -- 172.21.15.38:0/3898230135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe118069560 con 0x7fe118102bc0 2024-05-21T15:39:08.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.002+0000 7fe10e7fc640 1 -- 172.21.15.38:0/3898230135 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe10800c460 con 0x7fe118102bc0 2024-05-21T15:39:08.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.002+0000 7fe12619d740 1 Processor -- start 2024-05-21T15:39:08.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.002+0000 7fe12619d740 1 -- start start 2024-05-21T15:39:08.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.002+0000 7fe12619d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fde2390 0x562090181b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.002+0000 7fe12619d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdb94d0 0x562090182080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.002+0000 7fe12619d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fdc16b0 0x562090188f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.002+0000 7fe12619d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5620900e6c40 con 0x56208fdb94d0 2024-05-21T15:39:08.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.002+0000 7fe12619d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56209008e580 con 0x56208fde2390 2024-05-21T15:39:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.002+0000 7fe12619d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5620900e2c00 con 0x56208fdc16b0 2024-05-21T15:39:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.003+0000 7fe11d093640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdb94d0 0x562090182080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.003+0000 7fe11d093640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdb94d0 0x562090182080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41054/0 (socket says 172.21.15.38:41054) 2024-05-21T15:39:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.003+0000 7fe11d093640 1 -- 172.21.15.38:0/3760500428 learned_addr learned my addr 172.21.15.38:0/3760500428 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.003+0000 7fe11c892640 1 --2- 172.21.15.38:0/3760500428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fdc16b0 0x562090188f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.003+0000 7fe11d894640 1 --2- 172.21.15.38:0/3760500428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fde2390 0x562090181b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.003+0000 7fe11d093640 1 -- 172.21.15.38:0/3760500428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fdc16b0 msgr2=0x562090188f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.003+0000 7fe11d093640 1 --2- 172.21.15.38:0/3760500428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fdc16b0 0x562090188f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.003+0000 7fe11d093640 1 -- 172.21.15.38:0/3760500428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fde2390 msgr2=0x562090181b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.003+0000 7fe11d093640 1 --2- 172.21.15.38:0/3760500428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fde2390 0x562090181b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.003+0000 7fe11d093640 1 -- 172.21.15.38:0/3760500428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56208fdfa3f0 con 0x56208fdb94d0 2024-05-21T15:39:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.003+0000 7fe10e7fc640 1 --2- 172.21.15.38:0/3898230135 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1040a2b90 0x7fe1040a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.003+0000 7fe11d093640 1 --2- 172.21.15.38:0/3760500428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdb94d0 0x562090182080 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe114009230 tx=0x7fe1140316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be14b36789e5828 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.003+0000 7fe11d894640 1 --2- 172.21.15.38:0/3898230135 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1040a2b90 0x7fe1040a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.003+0000 7fe10e7fc640 1 -- 172.21.15.38:0/3898230135 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe1080d0470 con 0x7fe118102bc0 2024-05-21T15:39:08.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.004+0000 7fe1037fe640 1 -- 172.21.15.38:0/3760500428 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114040070 con 0x56208fdb94d0 2024-05-21T15:39:08.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.004+0000 7fe1037fe640 1 -- 172.21.15.38:0/3760500428 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe114005040 con 0x56208fdb94d0 2024-05-21T15:39:08.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.004+0000 7fe1037fe640 1 -- 172.21.15.38:0/3760500428 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114035040 con 0x56208fdb94d0 2024-05-21T15:39:08.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.005+0000 7fe12619d740 1 -- 172.21.15.38:0/3760500428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdb94d0 msgr2=0x562090182080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.005+0000 7fe11d894640 1 --2- 172.21.15.38:0/3898230135 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1040a2b90 0x7fe1040a50a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x562090182cd0 tx=0x7fe110009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.005+0000 7fe12619d740 1 --2- 172.21.15.38:0/3760500428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdb94d0 0x562090182080 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe114009230 tx=0x7fe1140316b0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.005+0000 7fe12619d740 1 -- 172.21.15.38:0/3760500428 shutdown_connections 2024-05-21T15:39:08.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.005+0000 7fe12619d740 1 --2- 172.21.15.38:0/3760500428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fdc16b0 0x562090188f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.005+0000 7fe12619d740 1 --2- 172.21.15.38:0/3760500428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdb94d0 0x562090182080 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.005+0000 7fe12619d740 1 --2- 172.21.15.38:0/3760500428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fde2390 0x562090181b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.005+0000 7fe12619d740 1 -- 172.21.15.38:0/3760500428 >> 172.21.15.38:0/3760500428 conn(0x56208fdd64e0 msgr2=0x56209017d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.005+0000 7fe12619d740 1 -- 172.21.15.38:0/3760500428 shutdown_connections 2024-05-21T15:39:08.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.005+0000 7fe12619d740 1 -- 172.21.15.38:0/3760500428 wait complete. 2024-05-21T15:39:08.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.005+0000 7fe12619d740 1 -- start start 2024-05-21T15:39:08.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.005+0000 7fe12619d740 1 Processor -- start 2024-05-21T15:39:08.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe12619d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdb94d0 0x56209018b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe12619d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fdc16b0 0x56209018be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe12619d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fde2390 0x562090192d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe12619d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562090189970 con 0x56208fdb94d0 2024-05-21T15:39:08.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe12619d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5620900a1c40 con 0x56208fde2390 2024-05-21T15:39:08.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe12619d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562090189af0 con 0x56208fdc16b0 2024-05-21T15:39:08.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe12619d740 10 client.0 _ll_register_callbacks cb 0x56208fdf9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:08.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe11d093640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdb94d0 0x56209018b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe11c892640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fdc16b0 0x56209018be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe11d093640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdb94d0 0x56209018b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41068/0 (socket says 172.21.15.38:41068) 2024-05-21T15:39:08.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe11c892640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fdc16b0 0x56209018be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:60822/0 (socket says 172.21.15.38:60822) 2024-05-21T15:39:08.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe11d093640 1 -- 172.21.15.38:0/2056102061 learned_addr learned my addr 172.21.15.38:0/2056102061 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe11d894640 1 --2- 172.21.15.38:0/2056102061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fde2390 0x562090192d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe11c892640 1 -- 172.21.15.38:0/2056102061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fde2390 msgr2=0x562090192d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe11c892640 1 --2- 172.21.15.38:0/2056102061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fde2390 0x562090192d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe11c892640 1 -- 172.21.15.38:0/2056102061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdb94d0 msgr2=0x56209018b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe11c892640 1 --2- 172.21.15.38:0/2056102061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdb94d0 0x56209018b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.006+0000 7fe11c892640 1 -- 172.21.15.38:0/2056102061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56208fec47d0 con 0x56208fdc16b0 2024-05-21T15:39:08.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.007+0000 7fe11c892640 1 --2- 172.21.15.38:0/2056102061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fdc16b0 0x56209018be30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe1080cdf70 tx=0x7fe108009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87c00a6782d84f35 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.007+0000 7fe100ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:39:08.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.007+0000 7fe100ff9640 1 -- 172.21.15.38:0/2056102061 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10809c690 con 0x56208fdc16b0 2024-05-21T15:39:08.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.007+0000 7fe100ff9640 1 -- 172.21.15.38:0/2056102061 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe10809c180 con 0x56208fdc16b0 2024-05-21T15:39:08.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.007+0000 7fe12619d740 1 -- 172.21.15.38:0/2056102061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe114005040 con 0x56208fdc16b0 2024-05-21T15:39:08.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.007+0000 7fe100ff9640 1 -- 172.21.15.38:0/2056102061 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe10800ce60 con 0x56208fdc16b0 2024-05-21T15:39:08.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.007+0000 7fe12619d740 10 client.26371 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:08.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.007+0000 7fe12619d740 1 -- 172.21.15.38:0/2056102061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562090196110 con 0x56208fdc16b0 2024-05-21T15:39:08.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.007+0000 7fe100ff9640 1 -- 172.21.15.38:0/2056102061 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe114005040 con 0x56208fdc16b0 2024-05-21T15:39:08.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.008+0000 7fe100ff9640 1 -- 172.21.15.38:0/2056102061 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fe108098a80 con 0x56208fdc16b0 2024-05-21T15:39:08.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.008+0000 7fe12619d740 10 client.26371 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:08.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.008+0000 7fe12619d740 10 client.26371 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:39:08.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.008+0000 7fe12619d740 20 client.26371 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:08.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.008+0000 7fe12619d740 1 --2- 172.21.15.38:0/2056102061 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56208fda0f60 0x5620901997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.008+0000 7fe12619d740 4 client.26371 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:08.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.008+0000 7fe12619d740 1 -- 172.21.15.38:0/2056102061 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620900f4370 con 0x56208fda0f60 2024-05-21T15:39:08.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.008+0000 7fe11d093640 1 --2- 172.21.15.38:0/2056102061 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56208fda0f60 0x5620901997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.008+0000 7fe100ff9640 10 client.26371 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:39:08.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.008+0000 7fe11d093640 1 --2- 172.21.15.38:0/2056102061 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56208fda0f60 0x5620901997a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=79065decbde931e5 server_cookie=18b2a8600fce26bc in_seq=0 out_seq=0 2024-05-21T15:39:08.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.009+0000 7fe100ff9640 1 -- 172.21.15.38:0/2056102061 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5620900f4370 con 0x56208fda0f60 2024-05-21T15:39:08.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.009+0000 7fe100ff9640 10 client.26371 handle_command_reply: tid=0 2024-05-21T15:39:08.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.009+0000 7fe12619d740 1 client.26371 shutdown 2024-05-21T15:39:08.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.010+0000 7fe12619d740 1 -- 172.21.15.38:0/2056102061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fdc16b0 msgr2=0x56209018be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.010+0000 7fe12619d740 1 --2- 172.21.15.38:0/2056102061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fdc16b0 0x56209018be30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe1080cdf70 tx=0x7fe108009f60 comp rx=0 tx=0).stop 2024-05-21T15:39:08.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.010+0000 7fe12619d740 1 -- 172.21.15.38:0/2056102061 shutdown_connections 2024-05-21T15:39:08.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.010+0000 7fe12619d740 1 --2- 172.21.15.38:0/2056102061 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56208fda0f60 0x5620901997a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.010+0000 7fe12619d740 1 --2- 172.21.15.38:0/2056102061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fde2390 0x562090192d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.010+0000 7fe100ff9640 0 client.26371 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:39:08.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.010+0000 7fe12619d740 1 --2- 172.21.15.38:0/2056102061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fdc16b0 0x56209018be30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.010+0000 7fe12619d740 1 --2- 172.21.15.38:0/2056102061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdb94d0 0x56209018b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.010+0000 7fe12619d740 1 -- 172.21.15.38:0/2056102061 >> 172.21.15.38:0/2056102061 conn(0x56208fdd64e0 msgr2=0x56209017c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.010+0000 7fe12619d740 1 -- 172.21.15.38:0/2056102061 shutdown_connections 2024-05-21T15:39:08.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.010+0000 7fe12619d740 1 -- 172.21.15.38:0/2056102061 wait complete. 2024-05-21T15:39:08.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.010+0000 7fe12619d740 20 client.26371 trim_cache size 0 max 16384 2024-05-21T15:39:08.013 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:07 smithi045 ceph-mon[36409]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 58 MiB/s wr, 12.06k op/s 2024-05-21T15:39:08.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.024+0000 7fe12619d740 1 Processor -- start 2024-05-21T15:39:08.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.024+0000 7fe12619d740 1 -- start start 2024-05-21T15:39:08.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.024+0000 7fe12619d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fdb94d0 0x562090180a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.024+0000 7fe12619d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdc16b0 0x562090192cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe12619d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fde2390 0x562090189ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:07 smithi038 ceph-mon[31995]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 58 MiB/s wr, 12.06k op/s 2024-05-21T15:39:08.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe12619d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe1181a81f0 con 0x56208fdc16b0 2024-05-21T15:39:08.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe12619d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe1181a9e20 con 0x56208fdb94d0 2024-05-21T15:39:08.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe12619d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe1181a88c0 con 0x56208fde2390 2024-05-21T15:39:08.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe11d894640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fde2390 0x562090189ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe11d093640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fdb94d0 0x562090180a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe11c892640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdc16b0 0x562090192cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe11d894640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fde2390 0x562090189ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:60826/0 (socket says 172.21.15.38:60826) 2024-05-21T15:39:08.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe11d093640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fdb94d0 0x562090180a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:39804/0 (socket says 172.21.15.38:39804) 2024-05-21T15:39:08.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe11d894640 1 -- 172.21.15.38:0/1528414893 learned_addr learned my addr 172.21.15.38:0/1528414893 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe11c892640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdc16b0 0x562090192cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41072/0 (socket says 172.21.15.38:41072) 2024-05-21T15:39:08.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe11c892640 1 -- 172.21.15.38:0/1528414893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fde2390 msgr2=0x562090189ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe11c892640 1 --2- 172.21.15.38:0/1528414893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fde2390 0x562090189ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe11c892640 1 -- 172.21.15.38:0/1528414893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fdb94d0 msgr2=0x562090180a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe11c892640 1 --2- 172.21.15.38:0/1528414893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fdb94d0 0x562090180a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe11c892640 1 -- 172.21.15.38:0/1528414893 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562090189160 con 0x56208fdc16b0 2024-05-21T15:39:08.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.025+0000 7fe11c892640 1 --2- 172.21.15.38:0/1528414893 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdc16b0 0x562090192cb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe108007590 tx=0x7fe10809b240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72288dffc09eff9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.026+0000 7fe102ffd640 1 -- 172.21.15.38:0/1528414893 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1080a58f0 con 0x56208fdc16b0 2024-05-21T15:39:08.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.026+0000 7fe102ffd640 1 -- 172.21.15.38:0/1528414893 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10809cd30 con 0x56208fdc16b0 2024-05-21T15:39:08.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.026+0000 7fe102ffd640 1 -- 172.21.15.38:0/1528414893 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10809bf70 con 0x56208fdc16b0 2024-05-21T15:39:08.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.027+0000 7fe12619d740 1 -- 172.21.15.38:0/1528414893 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdc16b0 msgr2=0x562090192cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.027+0000 7fe12619d740 1 --2- 172.21.15.38:0/1528414893 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdc16b0 0x562090192cb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe108007590 tx=0x7fe10809b240 comp rx=0 tx=0).stop 2024-05-21T15:39:08.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.027+0000 7fe12619d740 1 -- 172.21.15.38:0/1528414893 shutdown_connections 2024-05-21T15:39:08.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.027+0000 7fe12619d740 1 --2- 172.21.15.38:0/1528414893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fde2390 0x562090189ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.027+0000 7fe12619d740 1 --2- 172.21.15.38:0/1528414893 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdc16b0 0x562090192cb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.027+0000 7fe12619d740 1 --2- 172.21.15.38:0/1528414893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fdb94d0 0x562090180a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.027+0000 7fe12619d740 1 -- 172.21.15.38:0/1528414893 >> 172.21.15.38:0/1528414893 conn(0x56208fdd64e0 msgr2=0x562090180330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.027+0000 7fe12619d740 1 -- 172.21.15.38:0/1528414893 shutdown_connections 2024-05-21T15:39:08.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.027+0000 7fe12619d740 1 -- 172.21.15.38:0/1528414893 wait complete. 2024-05-21T15:39:08.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.027+0000 7fe12619d740 1 -- start start 2024-05-21T15:39:08.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.027+0000 7fe12619d740 1 Processor -- start 2024-05-21T15:39:08.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe12619d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fdb94d0 0x562090190010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe12619d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdc16b0 0x562090190550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe12619d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fde2390 0x562090191480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe12619d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe1181a9690 con 0x56208fdc16b0 2024-05-21T15:39:08.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe12619d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe1181a8f90 con 0x56208fdb94d0 2024-05-21T15:39:08.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe12619d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe1181a61d0 con 0x56208fde2390 2024-05-21T15:39:08.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe12619d740 10 client.0 _ll_register_callbacks cb 0x7fe10800d3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:08.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe11c892640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdc16b0 0x562090190550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe11d093640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fdb94d0 0x562090190010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe11d894640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fde2390 0x562090191480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe11d894640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fde2390 0x562090191480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:60830/0 (socket says 172.21.15.38:60830) 2024-05-21T15:39:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe11d093640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fdb94d0 0x562090190010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:39810/0 (socket says 172.21.15.38:39810) 2024-05-21T15:39:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe11d894640 1 -- 172.21.15.38:0/3660353445 learned_addr learned my addr 172.21.15.38:0/3660353445 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe11c892640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdc16b0 0x562090190550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41084/0 (socket says 172.21.15.38:41084) 2024-05-21T15:39:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe11d093640 1 -- 172.21.15.38:0/3660353445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fde2390 msgr2=0x562090191480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe11d093640 1 --2- 172.21.15.38:0/3660353445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fde2390 0x562090191480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe11d093640 1 -- 172.21.15.38:0/3660353445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdc16b0 msgr2=0x562090190550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe11d093640 1 --2- 172.21.15.38:0/3660353445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdc16b0 0x562090190550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe11d093640 1 -- 172.21.15.38:0/3660353445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562090191a40 con 0x56208fdb94d0 2024-05-21T15:39:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.028+0000 7fe11c892640 1 --2- 172.21.15.38:0/3660353445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdc16b0 0x562090190550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.029+0000 7fe11d093640 1 --2- 172.21.15.38:0/3660353445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fdb94d0 0x562090190010 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe1140095a0 tx=0x7fe114031a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8976440dc104ecf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.029+0000 7fe100ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:39:08.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.029+0000 7fe100ff9640 1 -- 172.21.15.38:0/3660353445 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114035040 con 0x56208fdb94d0 2024-05-21T15:39:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.029+0000 7fe12619d740 1 -- 172.21.15.38:0/3660353445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe10809cd30 con 0x56208fdb94d0 2024-05-21T15:39:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.029+0000 7fe100ff9640 1 -- 172.21.15.38:0/3660353445 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe114004040 con 0x56208fdb94d0 2024-05-21T15:39:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.029+0000 7fe100ff9640 1 -- 172.21.15.38:0/3660353445 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe114040040 con 0x56208fdb94d0 2024-05-21T15:39:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.029+0000 7fe12619d740 10 client.26294 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.029+0000 7fe12619d740 1 -- 172.21.15.38:0/3660353445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562090192d30 con 0x56208fdb94d0 2024-05-21T15:39:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.030+0000 7fe100ff9640 1 -- 172.21.15.38:0/3660353445 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10809cd30 con 0x56208fdb94d0 2024-05-21T15:39:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.030+0000 7fe100ff9640 1 -- 172.21.15.38:0/3660353445 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fe114004780 con 0x56208fdb94d0 2024-05-21T15:39:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.030+0000 7fe12619d740 10 client.26294 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.030+0000 7fe12619d740 10 client.26294 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:39:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.030+0000 7fe12619d740 20 client.26294 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.030+0000 7fe12619d740 1 --2- 172.21.15.38:0/3660353445 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56208fd763a0 0x562090203b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.030+0000 7fe12619d740 4 client.26294 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:39:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.030+0000 7fe12619d740 1 -- 172.21.15.38:0/3660353445 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620900ee6c0 con 0x56208fd763a0 2024-05-21T15:39:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.030+0000 7fe11c892640 1 --2- 172.21.15.38:0/3660353445 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56208fd763a0 0x562090203b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe11c892640 1 --2- 172.21.15.38:0/3660353445 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56208fd763a0 0x562090203b70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=36482f0320ecdad8 server_cookie=18367edae2ab16cb in_seq=0 out_seq=0 2024-05-21T15:39:08.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe100ff9640 10 client.26294 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:39:08.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe100ff9640 1 -- 172.21.15.38:0/3660353445 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620900ee6c0 con 0x56208fd763a0 2024-05-21T15:39:08.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe100ff9640 10 client.26294 handle_command_reply: tid=0 2024-05-21T15:39:08.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe12619d740 1 client.26294 shutdown 2024-05-21T15:39:08.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe12619d740 1 -- 172.21.15.38:0/3660353445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fdb94d0 msgr2=0x562090190010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe12619d740 1 --2- 172.21.15.38:0/3660353445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fdb94d0 0x562090190010 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe1140095a0 tx=0x7fe114031a70 comp rx=0 tx=0).stop 2024-05-21T15:39:08.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe12619d740 1 -- 172.21.15.38:0/3660353445 shutdown_connections 2024-05-21T15:39:08.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe12619d740 1 --2- 172.21.15.38:0/3660353445 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56208fd763a0 0x562090203b70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe12619d740 1 --2- 172.21.15.38:0/3660353445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56208fde2390 0x562090191480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe12619d740 1 --2- 172.21.15.38:0/3660353445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56208fdc16b0 0x562090190550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe12619d740 1 --2- 172.21.15.38:0/3660353445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56208fdb94d0 0x562090190010 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe100ff9640 0 client.26294 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:39:08.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe12619d740 1 -- 172.21.15.38:0/3660353445 >> 172.21.15.38:0/3660353445 conn(0x56208fdd64e0 msgr2=0x562090180d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe12619d740 1 -- 172.21.15.38:0/3660353445 shutdown_connections 2024-05-21T15:39:08.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe12619d740 1 -- 172.21.15.38:0/3660353445 wait complete. 2024-05-21T15:39:08.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.031+0000 7fe12619d740 20 client.26294 trim_cache size 0 max 16384 2024-05-21T15:39:08.035 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:39:08.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.032+0000 7fe11f2f7640 1 -- 172.21.15.38:0/3898230135 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1040a2b90 msgr2=0x7fe1040a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.032+0000 7fe11f2f7640 1 --2- 172.21.15.38:0/3898230135 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1040a2b90 0x7fe1040a50a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x562090182cd0 tx=0x7fe110009500 comp rx=0 tx=0).stop 2024-05-21T15:39:08.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.032+0000 7fe11f2f7640 1 -- 172.21.15.38:0/3898230135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe118102bc0 msgr2=0x7fe1181986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.032+0000 7fe11f2f7640 1 --2- 172.21.15.38:0/3898230135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe118102bc0 0x7fe1181986b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe108009970 tx=0x7fe108009e30 comp rx=0 tx=0).stop 2024-05-21T15:39:08.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.032+0000 7fe11f2f7640 1 -- 172.21.15.38:0/3898230135 shutdown_connections 2024-05-21T15:39:08.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.032+0000 7fe11f2f7640 1 --2- 172.21.15.38:0/3898230135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1181078c0 0x7fe11819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.032+0000 7fe11f2f7640 1 --2- 172.21.15.38:0/3898230135 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe1040a2b90 0x7fe1040a50a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.032+0000 7fe11f2f7640 1 --2- 172.21.15.38:0/3898230135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe118102bc0 0x7fe1181986b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.032+0000 7fe11f2f7640 1 --2- 172.21.15.38:0/3898230135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe118101980 0x7fe118198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.032+0000 7fe11f2f7640 1 -- 172.21.15.38:0/3898230135 >> 172.21.15.38:0/3898230135 conn(0x7fe1180fcb70 msgr2=0x7fe1181038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.032+0000 7fe11f2f7640 1 -- 172.21.15.38:0/3898230135 shutdown_connections 2024-05-21T15:39:08.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.032+0000 7fe11f2f7640 1 -- 172.21.15.38:0/3898230135 wait complete. 2024-05-21T15:39:08.045 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:39:08.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.168+0000 7faa4dd2f640 1 -- 172.21.15.38:0/3547040458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa481078d0 msgr2=0x7faa48109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.168+0000 7faa4dd2f640 1 --2- 172.21.15.38:0/3547040458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa481078d0 0x7faa48109cf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7faa40009ca0 tx=0x7faa4002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.168+0000 7faa4dd2f640 1 -- 172.21.15.38:0/3547040458 shutdown_connections 2024-05-21T15:39:08.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.168+0000 7faa4dd2f640 1 --2- 172.21.15.38:0/3547040458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa481078d0 0x7faa48109cf0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.168+0000 7faa4dd2f640 1 --2- 172.21.15.38:0/3547040458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa48102bd0 0x7faa48102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.168+0000 7faa4dd2f640 1 --2- 172.21.15.38:0/3547040458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa48101990 0x7faa48102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.168+0000 7faa4dd2f640 1 -- 172.21.15.38:0/3547040458 >> 172.21.15.38:0/3547040458 conn(0x7faa480fcb80 msgr2=0x7faa480ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.168+0000 7faa4dd2f640 1 -- 172.21.15.38:0/3547040458 shutdown_connections 2024-05-21T15:39:08.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.168+0000 7faa4dd2f640 1 -- 172.21.15.38:0/3547040458 wait complete. 2024-05-21T15:39:08.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.169+0000 7faa4dd2f640 1 Processor -- start 2024-05-21T15:39:08.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.169+0000 7faa4dd2f640 1 -- start start 2024-05-21T15:39:08.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.169+0000 7faa4dd2f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa48101990 0x7faa48199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.169+0000 7faa4dd2f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa48102bd0 0x7faa481997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.169+0000 7faa4dd2f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa481078d0 0x7faa481a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.169+0000 7faa4dd2f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810e710 con 0x7faa48102bd0 2024-05-21T15:39:08.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.169+0000 7faa4dd2f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810e590 con 0x7faa481078d0 2024-05-21T15:39:08.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.169+0000 7faa4dd2f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810e890 con 0x7faa48101990 2024-05-21T15:39:08.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.169+0000 7faa46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa48102bd0 0x7faa481997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.169+0000 7faa46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa48102bd0 0x7faa481997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41098/0 (socket says 172.21.15.38:41098) 2024-05-21T15:39:08.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.169+0000 7faa46ffd640 1 -- 172.21.15.38:0/1277103332 learned_addr learned my addr 172.21.15.38:0/1277103332 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.169+0000 7faa477fe640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa48101990 0x7faa48199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.169+0000 7faa47fff640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa481078d0 0x7faa481a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.169+0000 7faa46ffd640 1 -- 172.21.15.38:0/1277103332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa48101990 msgr2=0x7faa48199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.170+0000 7faa46ffd640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa48101990 0x7faa48199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.170+0000 7faa46ffd640 1 -- 172.21.15.38:0/1277103332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa481078d0 msgr2=0x7faa481a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.170+0000 7faa46ffd640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa481078d0 0x7faa481a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.170+0000 7faa46ffd640 1 -- 172.21.15.38:0/1277103332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa481a0dd0 con 0x7faa48102bd0 2024-05-21T15:39:08.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.170+0000 7faa47fff640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa481078d0 0x7faa481a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:39:08.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.170+0000 7faa46ffd640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa48102bd0 0x7faa481997d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00e9f0 tx=0x7faa3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.170+0000 7faa44ff9640 1 -- 172.21.15.38:0/1277103332 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c00cd80 con 0x7faa48102bd0 2024-05-21T15:39:08.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.170+0000 7faa4dd2f640 1 -- 172.21.15.38:0/1277103332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa481a10c0 con 0x7faa48102bd0 2024-05-21T15:39:08.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.170+0000 7faa44ff9640 1 -- 172.21.15.38:0/1277103332 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa3c005500 con 0x7faa48102bd0 2024-05-21T15:39:08.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.170+0000 7faa44ff9640 1 -- 172.21.15.38:0/1277103332 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c010690 con 0x7faa48102bd0 2024-05-21T15:39:08.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.170+0000 7faa4dd2f640 1 -- 172.21.15.38:0/1277103332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa481a15c0 con 0x7faa48102bd0 2024-05-21T15:39:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.173+0000 7faa44ff9640 1 -- 172.21.15.38:0/1277103332 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faa3c0046c0 con 0x7faa48102bd0 2024-05-21T15:39:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.173+0000 7faa54bd3740 1 Processor -- start 2024-05-21T15:39:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.173+0000 7faa54bd3740 1 -- start start 2024-05-21T15:39:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.173+0000 7faa54bd3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da48779390 0x55da48b18b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.173+0000 7faa54bd3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487504d0 0x55da48b19080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.173+0000 7faa44ff9640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faa300a2b90 0x7faa300a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.173+0000 7faa54bd3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da487586b0 0x55da48b1ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.173+0000 7faa54bd3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55da48a7dc40 con 0x55da487504d0 2024-05-21T15:39:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.173+0000 7faa54bd3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55da48a25580 con 0x55da487586b0 2024-05-21T15:39:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.173+0000 7faa54bd3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55da48a79c00 con 0x55da48779390 2024-05-21T15:39:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487504d0 0x55da48b19080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa477fe640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faa300a2b90 0x7faa300a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487504d0 0x55da48b19080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41104/0 (socket says 172.21.15.38:41104) 2024-05-21T15:39:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa46ffd640 1 -- 172.21.15.38:0/1426379204 learned_addr learned my addr 172.21.15.38:0/1426379204 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa44ff9640 1 -- 172.21.15.38:0/1277103332 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7faa3c0cf4e0 con 0x7faa48102bd0 2024-05-21T15:39:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa47fff640 1 --2- 172.21.15.38:0/1426379204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da48779390 0x55da48b18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa47fff640 1 --2- 172.21.15.38:0/1426379204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da487586b0 0x55da48b1ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa46ffd640 1 -- 172.21.15.38:0/1426379204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da48779390 msgr2=0x55da48b18b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa46ffd640 1 --2- 172.21.15.38:0/1426379204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da48779390 0x55da48b18b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa46ffd640 1 -- 172.21.15.38:0/1426379204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da487586b0 msgr2=0x55da48b1ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa46ffd640 1 --2- 172.21.15.38:0/1426379204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da487586b0 0x55da48b1ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa46ffd640 1 -- 172.21.15.38:0/1426379204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da487913f0 con 0x55da487504d0 2024-05-21T15:39:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa47fff640 1 --2- 172.21.15.38:0/1426379204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da487586b0 0x55da48b1ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:39:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa477fe640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faa300a2b90 0x7faa300a50a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faa48102590 tx=0x7faa38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa46ffd640 1 --2- 172.21.15.38:0/1426379204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487504d0 0x55da48b19080 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0986c0 tx=0x7faa3c01bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3559f5bc74f3bcd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa2dffb640 1 -- 172.21.15.38:0/1426379204 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c069070 con 0x55da487504d0 2024-05-21T15:39:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa2dffb640 1 -- 172.21.15.38:0/1426379204 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa3c0610d0 con 0x55da487504d0 2024-05-21T15:39:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.174+0000 7faa2dffb640 1 -- 172.21.15.38:0/1426379204 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0a4960 con 0x55da487504d0 2024-05-21T15:39:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.175+0000 7faa54bd3740 1 -- 172.21.15.38:0/1426379204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487504d0 msgr2=0x55da48b19080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.175+0000 7faa54bd3740 1 --2- 172.21.15.38:0/1426379204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487504d0 0x55da48b19080 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0986c0 tx=0x7faa3c01bb30 comp rx=0 tx=0).stop 2024-05-21T15:39:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.175+0000 7faa54bd3740 1 -- 172.21.15.38:0/1426379204 shutdown_connections 2024-05-21T15:39:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.175+0000 7faa54bd3740 1 --2- 172.21.15.38:0/1426379204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da487586b0 0x55da48b1ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.175+0000 7faa54bd3740 1 --2- 172.21.15.38:0/1426379204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487504d0 0x55da48b19080 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.175+0000 7faa54bd3740 1 --2- 172.21.15.38:0/1426379204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da48779390 0x55da48b18b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.175+0000 7faa54bd3740 1 -- 172.21.15.38:0/1426379204 >> 172.21.15.38:0/1426379204 conn(0x55da4876d4e0 msgr2=0x55da48b14490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.175+0000 7faa54bd3740 1 -- 172.21.15.38:0/1426379204 shutdown_connections 2024-05-21T15:39:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.175+0000 7faa54bd3740 1 -- 172.21.15.38:0/1426379204 wait complete. 2024-05-21T15:39:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.176+0000 7faa54bd3740 1 -- start start 2024-05-21T15:39:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.176+0000 7faa54bd3740 1 Processor -- start 2024-05-21T15:39:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.176+0000 7faa54bd3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da487504d0 0x55da48b22a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.176+0000 7faa54bd3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487586b0 0x55da48b22f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.176+0000 7faa54bd3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da48779390 0x55da48b29ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.176+0000 7faa54bd3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55da48b209d0 con 0x55da487586b0 2024-05-21T15:39:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.176+0000 7faa54bd3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55da48a38c40 con 0x55da48779390 2024-05-21T15:39:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.176+0000 7faa54bd3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55da48b20b50 con 0x55da487504d0 2024-05-21T15:39:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.176+0000 7faa54bd3740 10 client.0 _ll_register_callbacks cb 0x55da48790be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.176+0000 7faa46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487586b0 0x55da48b22f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.176+0000 7faa46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487586b0 0x55da48b22f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41114/0 (socket says 172.21.15.38:41114) 2024-05-21T15:39:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.176+0000 7faa46ffd640 1 -- 172.21.15.38:0/1501212373 learned_addr learned my addr 172.21.15.38:0/1501212373 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.176+0000 7faa47fff640 1 --2- 172.21.15.38:0/1501212373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da48779390 0x55da48b29ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.176+0000 7faa477fe640 1 --2- 172.21.15.38:0/1501212373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da487504d0 0x55da48b22a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.177+0000 7faa46ffd640 1 -- 172.21.15.38:0/1501212373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da487504d0 msgr2=0x55da48b22a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.177+0000 7faa46ffd640 1 --2- 172.21.15.38:0/1501212373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da487504d0 0x55da48b22a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.177+0000 7faa46ffd640 1 -- 172.21.15.38:0/1501212373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da48779390 msgr2=0x55da48b29ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.177+0000 7faa46ffd640 1 --2- 172.21.15.38:0/1501212373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da48779390 0x55da48b29ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.177+0000 7faa46ffd640 1 -- 172.21.15.38:0/1501212373 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da4885b7d0 con 0x55da487586b0 2024-05-21T15:39:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.177+0000 7faa46ffd640 1 --2- 172.21.15.38:0/1501212373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487586b0 0x55da48b22f70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7faa3c09b9f0 tx=0x7faa3c09b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a4e11c88de79f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.177+0000 7faa26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.177+0000 7faa26ffd640 1 -- 172.21.15.38:0/1501212373 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c069030 con 0x55da487586b0 2024-05-21T15:39:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.177+0000 7faa26ffd640 1 -- 172.21.15.38:0/1501212373 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7faa3c099d90 con 0x55da487586b0 2024-05-21T15:39:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.177+0000 7faa54bd3740 1 -- 172.21.15.38:0/1501212373 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa3c0610d0 con 0x55da487586b0 2024-05-21T15:39:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.177+0000 7faa26ffd640 1 -- 172.21.15.38:0/1501212373 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faa3c09c3f0 con 0x55da487586b0 2024-05-21T15:39:08.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.177+0000 7faa54bd3740 10 client.16659 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.177+0000 7faa54bd3740 1 -- 172.21.15.38:0/1501212373 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55da48b2d190 con 0x55da487586b0 2024-05-21T15:39:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.178+0000 7faa26ffd640 1 -- 172.21.15.38:0/1501212373 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa3c0610d0 con 0x55da487586b0 2024-05-21T15:39:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.178+0000 7faa26ffd640 1 -- 172.21.15.38:0/1501212373 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7faa3c094070 con 0x55da487586b0 2024-05-21T15:39:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.178+0000 7faa54bd3740 10 client.16659 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.178+0000 7faa54bd3740 10 client.16659 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:39:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.178+0000 7faa54bd3740 20 client.16659 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.178+0000 7faa54bd3740 1 --2- 172.21.15.38:0/1501212373 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55da48737f60 0x55da48b304f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.178+0000 7faa54bd3740 4 client.16659 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.178+0000 7faa54bd3740 1 -- 172.21.15.38:0/1501212373 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da48a8b370 con 0x55da48737f60 2024-05-21T15:39:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.178+0000 7faa477fe640 1 --2- 172.21.15.38:0/1501212373 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55da48737f60 0x55da48b304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.179+0000 7faa26ffd640 10 client.16659 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:39:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.179+0000 7faa477fe640 1 --2- 172.21.15.38:0/1501212373 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55da48737f60 0x55da48b304f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=44ba24571e6bea6 server_cookie=d6b6c1ef1614c8d1 in_seq=0 out_seq=0 2024-05-21T15:39:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa26ffd640 1 -- 172.21.15.38:0/1501212373 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55da48a8b370 con 0x55da48737f60 2024-05-21T15:39:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa26ffd640 10 client.16659 handle_command_reply: tid=0 2024-05-21T15:39:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa54bd3740 1 client.16659 shutdown 2024-05-21T15:39:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa54bd3740 1 -- 172.21.15.38:0/1501212373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487586b0 msgr2=0x55da48b22f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa54bd3740 1 --2- 172.21.15.38:0/1501212373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487586b0 0x55da48b22f70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7faa3c09b9f0 tx=0x7faa3c09b6c0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa54bd3740 1 -- 172.21.15.38:0/1501212373 shutdown_connections 2024-05-21T15:39:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa54bd3740 1 --2- 172.21.15.38:0/1501212373 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55da48737f60 0x55da48b304f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa54bd3740 1 --2- 172.21.15.38:0/1501212373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da48779390 0x55da48b29ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa54bd3740 1 --2- 172.21.15.38:0/1501212373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487586b0 0x55da48b22f70 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa26ffd640 0 client.16659 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:39:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa54bd3740 1 --2- 172.21.15.38:0/1501212373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da487504d0 0x55da48b22a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa54bd3740 1 -- 172.21.15.38:0/1501212373 >> 172.21.15.38:0/1501212373 conn(0x55da4876d4e0 msgr2=0x55da48b13570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa54bd3740 1 -- 172.21.15.38:0/1501212373 shutdown_connections 2024-05-21T15:39:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa54bd3740 1 -- 172.21.15.38:0/1501212373 wait complete. 2024-05-21T15:39:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.180+0000 7faa54bd3740 20 client.16659 trim_cache size 0 max 16384 2024-05-21T15:39:08.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.194+0000 7faa54bd3740 1 Processor -- start 2024-05-21T15:39:08.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.194+0000 7faa54bd3740 1 -- start start 2024-05-21T15:39:08.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.194+0000 7faa54bd3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487504d0 0x55da48b13ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.194+0000 7faa54bd3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da487586b0 0x55da48b29df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.194+0000 7faa54bd3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da48779390 0x55da48b14410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.194+0000 7faa54bd3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faa481aa420 con 0x55da487504d0 2024-05-21T15:39:08.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.194+0000 7faa54bd3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faa481ac050 con 0x55da48779390 2024-05-21T15:39:08.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.194+0000 7faa54bd3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faa481aaaf0 con 0x55da487586b0 2024-05-21T15:39:08.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.195+0000 7faa477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487504d0 0x55da48b13ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.195+0000 7faa477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487504d0 0x55da48b13ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41116/0 (socket says 172.21.15.38:41116) 2024-05-21T15:39:08.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.195+0000 7faa477fe640 1 -- 172.21.15.38:0/3209625076 learned_addr learned my addr 172.21.15.38:0/3209625076 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.195+0000 7faa47fff640 1 --2- 172.21.15.38:0/3209625076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da48779390 0x55da48b14410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.195+0000 7faa46ffd640 1 --2- 172.21.15.38:0/3209625076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da487586b0 0x55da48b29df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.195+0000 7faa477fe640 1 -- 172.21.15.38:0/3209625076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da487586b0 msgr2=0x55da48b29df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.195+0000 7faa477fe640 1 --2- 172.21.15.38:0/3209625076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da487586b0 0x55da48b29df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.195+0000 7faa477fe640 1 -- 172.21.15.38:0/3209625076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da48779390 msgr2=0x55da48b14410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.195+0000 7faa477fe640 1 --2- 172.21.15.38:0/3209625076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da48779390 0x55da48b14410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.195+0000 7faa477fe640 1 -- 172.21.15.38:0/3209625076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da48b0fd20 con 0x55da487504d0 2024-05-21T15:39:08.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.195+0000 7faa477fe640 1 --2- 172.21.15.38:0/3209625076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487504d0 0x55da48b13ed0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x55da48b152c0 tx=0x7faa38014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b47f81c36c6b890 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.195+0000 7faa2d7fa640 1 -- 172.21.15.38:0/3209625076 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3801fe50 con 0x55da487504d0 2024-05-21T15:39:08.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.195+0000 7faa2d7fa640 1 -- 172.21.15.38:0/3209625076 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa3801f0d0 con 0x55da487504d0 2024-05-21T15:39:08.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.195+0000 7faa2d7fa640 1 -- 172.21.15.38:0/3209625076 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3801da40 con 0x55da487504d0 2024-05-21T15:39:08.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.196+0000 7faa54bd3740 1 -- 172.21.15.38:0/3209625076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487504d0 msgr2=0x55da48b13ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.196+0000 7faa54bd3740 1 --2- 172.21.15.38:0/3209625076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487504d0 0x55da48b13ed0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x55da48b152c0 tx=0x7faa38014000 comp rx=0 tx=0).stop 2024-05-21T15:39:08.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.196+0000 7faa54bd3740 1 -- 172.21.15.38:0/3209625076 shutdown_connections 2024-05-21T15:39:08.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.196+0000 7faa54bd3740 1 --2- 172.21.15.38:0/3209625076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da48779390 0x55da48b14410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.196+0000 7faa54bd3740 1 --2- 172.21.15.38:0/3209625076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da487586b0 0x55da48b29df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.196+0000 7faa54bd3740 1 --2- 172.21.15.38:0/3209625076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da487504d0 0x55da48b13ed0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.196+0000 7faa54bd3740 1 -- 172.21.15.38:0/3209625076 >> 172.21.15.38:0/3209625076 conn(0x55da4876d4e0 msgr2=0x55da48b21070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.196+0000 7faa54bd3740 1 -- 172.21.15.38:0/3209625076 shutdown_connections 2024-05-21T15:39:08.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.196+0000 7faa54bd3740 1 -- 172.21.15.38:0/3209625076 wait complete. 2024-05-21T15:39:08.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.197+0000 7faa54bd3740 1 -- start start 2024-05-21T15:39:08.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.197+0000 7faa54bd3740 1 Processor -- start 2024-05-21T15:39:08.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.197+0000 7faa54bd3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da487504d0 0x55da48b27520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.197+0000 7faa54bd3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da487586b0 0x55da48b27a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.197+0000 7faa54bd3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da48779390 0x55da48b28990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.197+0000 7faa54bd3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faa481ab8c0 con 0x55da48779390 2024-05-21T15:39:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.197+0000 7faa54bd3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faa481ab1c0 con 0x55da487504d0 2024-05-21T15:39:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.197+0000 7faa54bd3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faa481a8400 con 0x55da487586b0 2024-05-21T15:39:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.197+0000 7faa54bd3740 10 client.0 _ll_register_callbacks cb 0x7faa3c0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.197+0000 7faa47fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da48779390 0x55da48b28990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.197+0000 7faa47fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da48779390 0x55da48b28990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41126/0 (socket says 172.21.15.38:41126) 2024-05-21T15:39:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.197+0000 7faa47fff640 1 -- 172.21.15.38:0/354393577 learned_addr learned my addr 172.21.15.38:0/354393577 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.197+0000 7faa477fe640 1 --2- 172.21.15.38:0/354393577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da487504d0 0x55da48b27520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.198+0000 7faa47fff640 1 -- 172.21.15.38:0/354393577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da487586b0 msgr2=0x55da48b27a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.198+0000 7faa47fff640 1 --2- 172.21.15.38:0/354393577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da487586b0 0x55da48b27a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.198+0000 7faa47fff640 1 -- 172.21.15.38:0/354393577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da487504d0 msgr2=0x55da48b27520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.198+0000 7faa47fff640 1 --2- 172.21.15.38:0/354393577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da487504d0 0x55da48b27520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.198+0000 7faa47fff640 1 -- 172.21.15.38:0/354393577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da48b1bef0 con 0x55da48779390 2024-05-21T15:39:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.198+0000 7faa47fff640 1 --2- 172.21.15.38:0/354393577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da48779390 0x55da48b28990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7faa4002ade0 tx=0x7faa400319e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc339f7d71a617f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.198+0000 7faa26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.198+0000 7faa26ffd640 1 -- 172.21.15.38:0/354393577 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40003970 con 0x55da48779390 2024-05-21T15:39:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.198+0000 7faa26ffd640 1 -- 172.21.15.38:0/354393577 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7faa40003b10 con 0x55da48779390 2024-05-21T15:39:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.198+0000 7faa54bd3740 1 -- 172.21.15.38:0/354393577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa3801f0d0 con 0x55da48779390 2024-05-21T15:39:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.198+0000 7faa26ffd640 1 -- 172.21.15.38:0/354393577 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faa40003d10 con 0x55da48779390 2024-05-21T15:39:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.198+0000 7faa54bd3740 10 client.16665 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.198+0000 7faa54bd3740 1 -- 172.21.15.38:0/354393577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55da48b2a070 con 0x55da48779390 2024-05-21T15:39:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.198+0000 7faa26ffd640 1 -- 172.21.15.38:0/354393577 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa3801f0d0 con 0x55da48779390 2024-05-21T15:39:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.199+0000 7faa26ffd640 1 -- 172.21.15.38:0/354393577 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7faa4002f980 con 0x55da48779390 2024-05-21T15:39:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.199+0000 7faa54bd3740 10 client.16665 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.199+0000 7faa54bd3740 10 client.16665 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:39:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.199+0000 7faa54bd3740 20 client.16665 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.199+0000 7faa54bd3740 1 --2- 172.21.15.38:0/354393577 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55da4870d3a0 0x55da48b9ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.199+0000 7faa54bd3740 4 client.16665 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:39:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.199+0000 7faa54bd3740 1 -- 172.21.15.38:0/354393577 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da48a856c0 con 0x55da4870d3a0 2024-05-21T15:39:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.199+0000 7faa477fe640 1 --2- 172.21.15.38:0/354393577 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55da4870d3a0 0x55da48b9ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.199+0000 7faa477fe640 1 --2- 172.21.15.38:0/354393577 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55da4870d3a0 0x55da48b9ae50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=772b8553b7d048c4 server_cookie=12e3848662957768 in_seq=0 out_seq=0 2024-05-21T15:39:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.199+0000 7faa26ffd640 10 client.16665 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:39:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa26ffd640 1 -- 172.21.15.38:0/354393577 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da48a856c0 con 0x55da4870d3a0 2024-05-21T15:39:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa26ffd640 10 client.16665 handle_command_reply: tid=0 2024-05-21T15:39:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa54bd3740 1 client.16665 shutdown 2024-05-21T15:39:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa54bd3740 1 -- 172.21.15.38:0/354393577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da48779390 msgr2=0x55da48b28990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa54bd3740 1 --2- 172.21.15.38:0/354393577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da48779390 0x55da48b28990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7faa4002ade0 tx=0x7faa400319e0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa54bd3740 1 -- 172.21.15.38:0/354393577 shutdown_connections 2024-05-21T15:39:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa54bd3740 1 --2- 172.21.15.38:0/354393577 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55da4870d3a0 0x55da48b9ae50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa54bd3740 1 --2- 172.21.15.38:0/354393577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55da48779390 0x55da48b28990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa54bd3740 1 --2- 172.21.15.38:0/354393577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da487586b0 0x55da48b27a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa26ffd640 0 client.16665 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:39:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa54bd3740 1 --2- 172.21.15.38:0/354393577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55da487504d0 0x55da48b27520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa54bd3740 1 -- 172.21.15.38:0/354393577 >> 172.21.15.38:0/354393577 conn(0x55da4876d4e0 msgr2=0x55da48b20ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa54bd3740 1 -- 172.21.15.38:0/354393577 shutdown_connections 2024-05-21T15:39:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa54bd3740 1 -- 172.21.15.38:0/354393577 wait complete. 2024-05-21T15:39:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.200+0000 7faa54bd3740 20 client.16665 trim_cache size 0 max 16384 2024-05-21T15:39:08.204 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:39:08.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.201+0000 7faa4dd2f640 1 -- 172.21.15.38:0/1277103332 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faa300a2b90 msgr2=0x7faa300a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.201+0000 7faa4dd2f640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faa300a2b90 0x7faa300a50a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faa48102590 tx=0x7faa38005f90 comp rx=0 tx=0).stop 2024-05-21T15:39:08.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.201+0000 7faa4dd2f640 1 -- 172.21.15.38:0/1277103332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa48102bd0 msgr2=0x7faa481997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.201+0000 7faa4dd2f640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa48102bd0 0x7faa481997d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00e9f0 tx=0x7faa3c00edf0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.201+0000 7faa4dd2f640 1 -- 172.21.15.38:0/1277103332 shutdown_connections 2024-05-21T15:39:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.201+0000 7faa4dd2f640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa481078d0 0x7faa481a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.201+0000 7faa4dd2f640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faa300a2b90 0x7faa300a50a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.201+0000 7faa4dd2f640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa48102bd0 0x7faa481997d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.201+0000 7faa4dd2f640 1 --2- 172.21.15.38:0/1277103332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa48101990 0x7faa48199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.201+0000 7faa4dd2f640 1 -- 172.21.15.38:0/1277103332 >> 172.21.15.38:0/1277103332 conn(0x7faa480fcb80 msgr2=0x7faa480fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.201+0000 7faa4dd2f640 1 -- 172.21.15.38:0/1277103332 shutdown_connections 2024-05-21T15:39:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.201+0000 7faa4dd2f640 1 -- 172.21.15.38:0/1277103332 wait complete. 2024-05-21T15:39:08.206 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:07 smithi053 ceph-mon[36496]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 58 MiB/s wr, 12.06k op/s 2024-05-21T15:39:08.211 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:39:08.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.331+0000 7f9e9b26c640 1 -- 172.21.15.38:0/4236043019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e94108830 msgr2=0x7f9e9410ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.331+0000 7f9e9b26c640 1 --2- 172.21.15.38:0/4236043019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e94108830 0x7f9e9410ac50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009ca0 tx=0x7f9e8c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.331+0000 7f9e9b26c640 1 -- 172.21.15.38:0/4236043019 shutdown_connections 2024-05-21T15:39:08.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.331+0000 7f9e9b26c640 1 --2- 172.21.15.38:0/4236043019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e94108830 0x7f9e9410ac50 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.331+0000 7f9e9b26c640 1 --2- 172.21.15.38:0/4236043019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e94105ed0 0x7f9e941082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.331+0000 7f9e9b26c640 1 --2- 172.21.15.38:0/4236043019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e94103570 0x7f9e94105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.331+0000 7f9e9b26c640 1 -- 172.21.15.38:0/4236043019 >> 172.21.15.38:0/4236043019 conn(0x7f9e940fcb70 msgr2=0x7f9e940ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.331+0000 7f9e9b26c640 1 -- 172.21.15.38:0/4236043019 shutdown_connections 2024-05-21T15:39:08.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.331+0000 7f9e9b26c640 1 -- 172.21.15.38:0/4236043019 wait complete. 2024-05-21T15:39:08.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.332+0000 7f9e9b26c640 1 Processor -- start 2024-05-21T15:39:08.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.332+0000 7f9e9b26c640 1 -- start start 2024-05-21T15:39:08.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.332+0000 7f9e9b26c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e94103570 0x7f9e941029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.332+0000 7f9e9b26c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e94105ed0 0x7f9e94101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.332+0000 7f9e9b26c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e94108830 0x7f9e941015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.332+0000 7f9e9b26c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9410e700 con 0x7f9e94103570 2024-05-21T15:39:08.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.332+0000 7f9e9b26c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9410e580 con 0x7f9e94105ed0 2024-05-21T15:39:08.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.332+0000 7f9e9b26c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9410e880 con 0x7f9e94108830 2024-05-21T15:39:08.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.332+0000 7f9e99008640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e94103570 0x7f9e941029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.332+0000 7f9e98807640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e94105ed0 0x7f9e94101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.332+0000 7f9e99008640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e94103570 0x7f9e941029c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41142/0 (socket says 172.21.15.38:41142) 2024-05-21T15:39:08.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.332+0000 7f9e99008640 1 -- 172.21.15.38:0/340749136 learned_addr learned my addr 172.21.15.38:0/340749136 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.332+0000 7f9e99809640 1 --2- 172.21.15.38:0/340749136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e94108830 0x7f9e941015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.333+0000 7f9e98807640 1 -- 172.21.15.38:0/340749136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e94108830 msgr2=0x7f9e941015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.333+0000 7f9e98807640 1 --2- 172.21.15.38:0/340749136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e94108830 0x7f9e941015a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.333+0000 7f9e98807640 1 -- 172.21.15.38:0/340749136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e94103570 msgr2=0x7f9e941029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.333+0000 7f9e98807640 1 --2- 172.21.15.38:0/340749136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e94103570 0x7f9e941029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.333+0000 7f9e98807640 1 -- 172.21.15.38:0/340749136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e94101cd0 con 0x7f9e94105ed0 2024-05-21T15:39:08.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.333+0000 7f9e99008640 1 --2- 172.21.15.38:0/340749136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e94103570 0x7f9e941029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:08.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.333+0000 7f9e98807640 1 --2- 172.21.15.38:0/340749136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e94105ed0 0x7f9e94101060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400d720 tx=0x7f9e8400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.333+0000 7f9e8a7fc640 1 -- 172.21.15.38:0/340749136 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84017070 con 0x7f9e94105ed0 2024-05-21T15:39:08.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.333+0000 7f9e9b26c640 1 -- 172.21.15.38:0/340749136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e941a3300 con 0x7f9e94105ed0 2024-05-21T15:39:08.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.333+0000 7f9e8a7fc640 1 -- 172.21.15.38:0/340749136 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e8400de30 con 0x7f9e94105ed0 2024-05-21T15:39:08.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.333+0000 7f9e8a7fc640 1 -- 172.21.15.38:0/340749136 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e840047a0 con 0x7f9e94105ed0 2024-05-21T15:39:08.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.333+0000 7f9e9b26c640 1 -- 172.21.15.38:0/340749136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e941a3770 con 0x7f9e94105ed0 2024-05-21T15:39:08.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.336+0000 7f9ea21ed740 1 Processor -- start 2024-05-21T15:39:08.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.336+0000 7f9ea21ed740 1 -- start start 2024-05-21T15:39:08.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.336+0000 7f9ea21ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b81390 0x55faa1f20b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.336+0000 7f9ea21ed740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f21080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.336+0000 7f9ea21ed740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b606b0 0x55faa1f27f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.336+0000 7f9ea21ed740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55faa1e85c40 con 0x55faa1b606b0 2024-05-21T15:39:08.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.336+0000 7f9ea21ed740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55faa1e2d580 con 0x55faa1b584d0 2024-05-21T15:39:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.336+0000 7f9ea21ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55faa1e81c00 con 0x55faa1b81390 2024-05-21T15:39:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.337+0000 7f9e8a7fc640 1 -- 172.21.15.38:0/340749136 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9e840099d0 con 0x7f9e94105ed0 2024-05-21T15:39:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.337+0000 7f9e99008640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f21080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.337+0000 7f9e99008640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f21080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:39856/0 (socket says 172.21.15.38:39856) 2024-05-21T15:39:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.337+0000 7f9e99008640 1 -- 172.21.15.38:0/1517021644 learned_addr learned my addr 172.21.15.38:0/1517021644 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.337+0000 7f9e99809640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b81390 0x55faa1f20b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.337+0000 7f9e98807640 1 --2- 172.21.15.38:0/1517021644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b606b0 0x55faa1f27f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.337+0000 7f9e99008640 1 -- 172.21.15.38:0/1517021644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b81390 msgr2=0x55faa1f20b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.337+0000 7f9e99008640 1 --2- 172.21.15.38:0/1517021644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b81390 0x55faa1f20b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.337+0000 7f9e99008640 1 -- 172.21.15.38:0/1517021644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b606b0 msgr2=0x55faa1f27f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.337+0000 7f9e99008640 1 --2- 172.21.15.38:0/1517021644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b606b0 0x55faa1f27f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.337+0000 7f9e99008640 1 -- 172.21.15.38:0/1517021644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa1b993f0 con 0x55faa1b584d0 2024-05-21T15:39:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.337+0000 7f9e99809640 1 --2- 172.21.15.38:0/1517021644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b81390 0x55faa1f20b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:39:08.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.337+0000 7f9e8a7fc640 1 --2- 172.21.15.38:0/340749136 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9e800a2b90 0x7f9e800a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.337+0000 7f9e99809640 1 --2- 172.21.15.38:0/340749136 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9e800a2b90 0x7f9e800a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.338+0000 7f9e8a7fc640 1 -- 172.21.15.38:0/340749136 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9e840cec70 con 0x7f9e94105ed0 2024-05-21T15:39:08.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.338+0000 7f9e99008640 1 --2- 172.21.15.38:0/1517021644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f21080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9e900115f0 tx=0x7f9e90011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2ccd585207c1fc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.338+0000 7f9e7f7fe640 1 -- 172.21.15.38:0/1517021644 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9001e070 con 0x55faa1b584d0 2024-05-21T15:39:08.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.338+0000 7f9e7f7fe640 1 -- 172.21.15.38:0/1517021644 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9000f660 con 0x55faa1b584d0 2024-05-21T15:39:08.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.338+0000 7f9e7f7fe640 1 -- 172.21.15.38:0/1517021644 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90003960 con 0x55faa1b584d0 2024-05-21T15:39:08.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.339+0000 7f9ea21ed740 1 -- 172.21.15.38:0/1517021644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 msgr2=0x55faa1f21080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.339+0000 7f9e99809640 1 --2- 172.21.15.38:0/340749136 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9e800a2b90 0x7f9e800a50a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55faa1f21cd0 tx=0x7f9e8c03d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.339+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/1517021644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f21080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9e900115f0 tx=0x7f9e90011ab0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.339+0000 7f9ea21ed740 1 -- 172.21.15.38:0/1517021644 shutdown_connections 2024-05-21T15:39:08.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.339+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/1517021644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b606b0 0x55faa1f27f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.339+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/1517021644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f21080 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.339+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/1517021644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b81390 0x55faa1f20b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.339+0000 7f9ea21ed740 1 -- 172.21.15.38:0/1517021644 >> 172.21.15.38:0/1517021644 conn(0x55faa1b754e0 msgr2=0x55faa1f1c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.339+0000 7f9ea21ed740 1 -- 172.21.15.38:0/1517021644 shutdown_connections 2024-05-21T15:39:08.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.339+0000 7f9ea21ed740 1 -- 172.21.15.38:0/1517021644 wait complete. 2024-05-21T15:39:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.339+0000 7f9ea21ed740 1 -- start start 2024-05-21T15:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.339+0000 7f9ea21ed740 1 Processor -- start 2024-05-21T15:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9ea21ed740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f2a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9ea21ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b606b0 0x55faa1f2aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9ea21ed740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b81390 0x55faa1f31e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9ea21ed740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55faa1f289d0 con 0x55faa1b81390 2024-05-21T15:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9ea21ed740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55faa1e40c40 con 0x55faa1b584d0 2024-05-21T15:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9ea21ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55faa1f28b50 con 0x55faa1b606b0 2024-05-21T15:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9ea21ed740 10 client.0 _ll_register_callbacks cb 0x55faa1b98be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9e99809640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b81390 0x55faa1f31e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9e99008640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f2a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9e99008640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f2a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:39870/0 (socket says 172.21.15.38:39870) 2024-05-21T15:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9e99809640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b81390 0x55faa1f31e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41160/0 (socket says 172.21.15.38:41160) 2024-05-21T15:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9e99008640 1 -- 172.21.15.38:0/1246073854 learned_addr learned my addr 172.21.15.38:0/1246073854 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9e98807640 1 --2- 172.21.15.38:0/1246073854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b606b0 0x55faa1f2aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9e99008640 1 -- 172.21.15.38:0/1246073854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b606b0 msgr2=0x55faa1f2aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9e99008640 1 --2- 172.21.15.38:0/1246073854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b606b0 0x55faa1f2aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9e99008640 1 -- 172.21.15.38:0/1246073854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b81390 msgr2=0x55faa1f31e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9e99008640 1 --2- 172.21.15.38:0/1246073854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b81390 0x55faa1f31e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9e99008640 1 -- 172.21.15.38:0/1246073854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa1c637d0 con 0x55faa1b584d0 2024-05-21T15:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9e98807640 1 --2- 172.21.15.38:0/1246073854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b606b0 0x55faa1f2aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.340+0000 7f9e99809640 1 --2- 172.21.15.38:0/1246073854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b81390 0x55faa1f31e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.341+0000 7f9e99008640 1 --2- 172.21.15.38:0/1246073854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f2a9b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9e900115c0 tx=0x7f9e900052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21c19514d13f3b59 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.341+0000 7f9e7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.341+0000 7f9e7cff9640 1 -- 172.21.15.38:0/1246073854 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90026020 con 0x55faa1b584d0 2024-05-21T15:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.341+0000 7f9ea21ed740 1 -- 172.21.15.38:0/1246073854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e9000f660 con 0x55faa1b584d0 2024-05-21T15:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.341+0000 7f9e7cff9640 1 -- 172.21.15.38:0/1246073854 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9e90005510 con 0x55faa1b584d0 2024-05-21T15:39:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.341+0000 7f9e7cff9640 1 -- 172.21.15.38:0/1246073854 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e9001e040 con 0x55faa1b584d0 2024-05-21T15:39:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.341+0000 7f9ea21ed740 10 client.26324 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.341+0000 7f9ea21ed740 1 -- 172.21.15.38:0/1246073854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55faa1f35110 con 0x55faa1b584d0 2024-05-21T15:39:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.342+0000 7f9e7cff9640 1 -- 172.21.15.38:0/1246073854 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9000f660 con 0x55faa1b584d0 2024-05-21T15:39:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.342+0000 7f9e7cff9640 1 -- 172.21.15.38:0/1246073854 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f9e90033020 con 0x55faa1b584d0 2024-05-21T15:39:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.342+0000 7f9ea21ed740 10 client.26324 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.342+0000 7f9ea21ed740 10 client.26324 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:39:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.342+0000 7f9ea21ed740 20 client.26324 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.342+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/1246073854 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55faa1b3ff60 0x55faa1f386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.342+0000 7f9ea21ed740 4 client.26324 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.342+0000 7f9ea21ed740 1 -- 172.21.15.38:0/1246073854 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faa1e93370 con 0x55faa1b3ff60 2024-05-21T15:39:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.342+0000 7f9e98807640 1 --2- 172.21.15.38:0/1246073854 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55faa1b3ff60 0x55faa1f386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.343+0000 7f9e7cff9640 10 client.26324 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:39:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.343+0000 7f9e98807640 1 --2- 172.21.15.38:0/1246073854 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55faa1b3ff60 0x55faa1f386d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c86fd30f296fabb server_cookie=8611e5b4e9ba5a63 in_seq=0 out_seq=0 2024-05-21T15:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9e7cff9640 1 -- 172.21.15.38:0/1246073854 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55faa1e93370 con 0x55faa1b3ff60 2024-05-21T15:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9e7cff9640 10 client.26324 handle_command_reply: tid=0 2024-05-21T15:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9ea21ed740 1 client.26324 shutdown 2024-05-21T15:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9ea21ed740 1 -- 172.21.15.38:0/1246073854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 msgr2=0x55faa1f2a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/1246073854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f2a9b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9e900115c0 tx=0x7f9e900052f0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9ea21ed740 1 -- 172.21.15.38:0/1246073854 shutdown_connections 2024-05-21T15:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/1246073854 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55faa1b3ff60 0x55faa1f386d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/1246073854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b81390 0x55faa1f31e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/1246073854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b606b0 0x55faa1f2aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/1246073854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f2a9b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9e7cff9640 0 client.26324 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:39:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9ea21ed740 1 -- 172.21.15.38:0/1246073854 >> 172.21.15.38:0/1246073854 conn(0x55faa1b754e0 msgr2=0x55faa1f1b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9ea21ed740 1 -- 172.21.15.38:0/1246073854 shutdown_connections 2024-05-21T15:39:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9ea21ed740 1 -- 172.21.15.38:0/1246073854 wait complete. 2024-05-21T15:39:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.345+0000 7f9ea21ed740 20 client.26324 trim_cache size 0 max 16384 2024-05-21T15:39:08.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.360+0000 7f9ea21ed740 1 Processor -- start 2024-05-21T15:39:08.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9ea21ed740 1 -- start start 2024-05-21T15:39:08.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9ea21ed740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f2ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9ea21ed740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b606b0 0x55faa1f31d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9ea21ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b81390 0x55faa1f3cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9ea21ed740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941ac670 con 0x55faa1b606b0 2024-05-21T15:39:08.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9ea21ed740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941ae2a0 con 0x55faa1b584d0 2024-05-21T15:39:08.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9ea21ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941acd40 con 0x55faa1b81390 2024-05-21T15:39:08.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9e98807640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b606b0 0x55faa1f31d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9e99008640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f2ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9e98807640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b606b0 0x55faa1f31d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41166/0 (socket says 172.21.15.38:41166) 2024-05-21T15:39:08.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9e98807640 1 -- 172.21.15.38:0/2951552159 learned_addr learned my addr 172.21.15.38:0/2951552159 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9e98807640 1 -- 172.21.15.38:0/2951552159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b81390 msgr2=0x55faa1f3cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9e98807640 1 --2- 172.21.15.38:0/2951552159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b81390 0x55faa1f3cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9e98807640 1 -- 172.21.15.38:0/2951552159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 msgr2=0x55faa1f2ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9e98807640 1 --2- 172.21.15.38:0/2951552159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f2ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.361+0000 7f9e98807640 1 -- 172.21.15.38:0/2951552159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa1f2b5b0 con 0x55faa1b606b0 2024-05-21T15:39:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.362+0000 7f9e99008640 1 --2- 172.21.15.38:0/2951552159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f2ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.362+0000 7f9e98807640 1 --2- 172.21.15.38:0/2951552159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b606b0 0x55faa1f31d70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9e8409b410 tx=0x7f9e84005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82946cadb2868423 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.362+0000 7f9e7effd640 1 -- 172.21.15.38:0/2951552159 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8409afc0 con 0x55faa1b606b0 2024-05-21T15:39:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.362+0000 7f9e7effd640 1 -- 172.21.15.38:0/2951552159 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e8409bb80 con 0x55faa1b606b0 2024-05-21T15:39:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.362+0000 7f9e7effd640 1 -- 172.21.15.38:0/2951552159 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8409ade0 con 0x55faa1b606b0 2024-05-21T15:39:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.363+0000 7f9ea21ed740 1 -- 172.21.15.38:0/2951552159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b606b0 msgr2=0x55faa1f31d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.363+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/2951552159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b606b0 0x55faa1f31d70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9e8409b410 tx=0x7f9e84005240 comp rx=0 tx=0).stop 2024-05-21T15:39:08.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.363+0000 7f9ea21ed740 1 -- 172.21.15.38:0/2951552159 shutdown_connections 2024-05-21T15:39:08.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.363+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/2951552159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b81390 0x55faa1f3cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.363+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/2951552159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b606b0 0x55faa1f31d70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.363+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/2951552159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f2ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.363+0000 7f9ea21ed740 1 -- 172.21.15.38:0/2951552159 >> 172.21.15.38:0/2951552159 conn(0x55faa1b754e0 msgr2=0x55faa1f2a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.363+0000 7f9ea21ed740 1 -- 172.21.15.38:0/2951552159 shutdown_connections 2024-05-21T15:39:08.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.363+0000 7f9ea21ed740 1 -- 172.21.15.38:0/2951552159 wait complete. 2024-05-21T15:39:08.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.363+0000 7f9ea21ed740 1 -- start start 2024-05-21T15:39:08.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.363+0000 7f9ea21ed740 1 Processor -- start 2024-05-21T15:39:08.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.364+0000 7f9ea21ed740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f31c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.364+0000 7f9ea21ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b606b0 0x55faa1f2e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.364+0000 7f9ea21ed740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b81390 0x55faa1f2ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.364+0000 7f9ea21ed740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941adb10 con 0x55faa1b81390 2024-05-21T15:39:08.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.364+0000 7f9ea21ed740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941ad410 con 0x55faa1b584d0 2024-05-21T15:39:08.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.364+0000 7f9ea21ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941aa650 con 0x55faa1b606b0 2024-05-21T15:39:08.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.364+0000 7f9ea21ed740 10 client.0 _ll_register_callbacks cb 0x7f9e9001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:08.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.364+0000 7f9e99809640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b81390 0x55faa1f2ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.364+0000 7f9e99809640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b81390 0x55faa1f2ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41180/0 (socket says 172.21.15.38:41180) 2024-05-21T15:39:08.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.364+0000 7f9e99809640 1 -- 172.21.15.38:0/2119471413 learned_addr learned my addr 172.21.15.38:0/2119471413 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.364+0000 7f9e98807640 1 --2- 172.21.15.38:0/2119471413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b606b0 0x55faa1f2e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.364+0000 7f9e99008640 1 --2- 172.21.15.38:0/2119471413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f31c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.365+0000 7f9e99809640 1 -- 172.21.15.38:0/2119471413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b606b0 msgr2=0x55faa1f2e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.365+0000 7f9e99809640 1 --2- 172.21.15.38:0/2119471413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b606b0 0x55faa1f2e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.365+0000 7f9e99809640 1 -- 172.21.15.38:0/2119471413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 msgr2=0x55faa1f31c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.365+0000 7f9e99809640 1 --2- 172.21.15.38:0/2119471413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f31c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.365+0000 7f9e99809640 1 -- 172.21.15.38:0/2119471413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa1f9ea70 con 0x55faa1b81390 2024-05-21T15:39:08.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.365+0000 7f9e99809640 1 --2- 172.21.15.38:0/2119471413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b81390 0x55faa1f2ee40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9e9406a090 tx=0x7f9e8c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8301b0b5bc17df2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.365+0000 7f9e7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:08.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.365+0000 7f9e7cff9640 1 -- 172.21.15.38:0/2119471413 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c02f610 con 0x55faa1b81390 2024-05-21T15:39:08.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.365+0000 7f9e7cff9640 1 -- 172.21.15.38:0/2119471413 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9e8c02f7b0 con 0x55faa1b81390 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.365+0000 7f9ea21ed740 1 -- 172.21.15.38:0/2119471413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e8409bb80 con 0x55faa1b81390 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.365+0000 7f9e7cff9640 1 -- 172.21.15.38:0/2119471413 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e8c02f9b0 con 0x55faa1b81390 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.365+0000 7f9ea21ed740 10 client.16683 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.365+0000 7f9ea21ed740 1 -- 172.21.15.38:0/2119471413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55faa1f9fd00 con 0x55faa1b81390 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.366+0000 7f9e7cff9640 1 -- 172.21.15.38:0/2119471413 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e8409bb80 con 0x55faa1b81390 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.366+0000 7f9e7cff9640 1 -- 172.21.15.38:0/2119471413 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f9e8c030bc0 con 0x55faa1b81390 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.366+0000 7f9ea21ed740 10 client.16683 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.366+0000 7f9ea21ed740 10 client.16683 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.366+0000 7f9ea21ed740 20 client.16683 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.366+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/2119471413 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55faa1b153a0 0x55faa1fa2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.366+0000 7f9ea21ed740 4 client.16683 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.366+0000 7f9ea21ed740 1 -- 172.21.15.38:0/2119471413 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55faa1e8d6c0 con 0x55faa1b153a0 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.367+0000 7f9e99008640 1 --2- 172.21.15.38:0/2119471413 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55faa1b153a0 0x55faa1fa2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.367+0000 7f9e7cff9640 10 client.16683 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:39:08.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.367+0000 7f9e99008640 1 --2- 172.21.15.38:0/2119471413 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55faa1b153a0 0x55faa1fa2740 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75c059567752d788 server_cookie=f7b4fe49b8752f9d in_seq=0 out_seq=0 2024-05-21T15:39:08.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9e7cff9640 1 -- 172.21.15.38:0/2119471413 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55faa1e8d6c0 con 0x55faa1b153a0 2024-05-21T15:39:08.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9e7cff9640 10 client.16683 handle_command_reply: tid=0 2024-05-21T15:39:08.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9ea21ed740 1 client.16683 shutdown 2024-05-21T15:39:08.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9ea21ed740 1 -- 172.21.15.38:0/2119471413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b81390 msgr2=0x55faa1f2ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/2119471413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b81390 0x55faa1f2ee40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9e9406a090 tx=0x7f9e8c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9ea21ed740 1 -- 172.21.15.38:0/2119471413 shutdown_connections 2024-05-21T15:39:08.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/2119471413 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55faa1b153a0 0x55faa1fa2740 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/2119471413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55faa1b81390 0x55faa1f2ee40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/2119471413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55faa1b606b0 0x55faa1f2e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9ea21ed740 1 --2- 172.21.15.38:0/2119471413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55faa1b584d0 0x55faa1f31c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9e7cff9640 0 client.16683 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:39:08.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9ea21ed740 1 -- 172.21.15.38:0/2119471413 >> 172.21.15.38:0/2119471413 conn(0x55faa1b754e0 msgr2=0x55faa1f1f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9ea21ed740 1 -- 172.21.15.38:0/2119471413 shutdown_connections 2024-05-21T15:39:08.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9ea21ed740 1 -- 172.21.15.38:0/2119471413 wait complete. 2024-05-21T15:39:08.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.368+0000 7f9ea21ed740 20 client.16683 trim_cache size 0 max 16384 2024-05-21T15:39:08.371 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:39:08.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.369+0000 7f9e9b26c640 1 -- 172.21.15.38:0/340749136 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9e800a2b90 msgr2=0x7f9e800a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.369+0000 7f9e9b26c640 1 --2- 172.21.15.38:0/340749136 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9e800a2b90 0x7f9e800a50a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55faa1f21cd0 tx=0x7f9e8c03d040 comp rx=0 tx=0).stop 2024-05-21T15:39:08.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.369+0000 7f9e9b26c640 1 -- 172.21.15.38:0/340749136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e94105ed0 msgr2=0x7f9e94101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.369+0000 7f9e9b26c640 1 --2- 172.21.15.38:0/340749136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e94105ed0 0x7f9e94101060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400d720 tx=0x7f9e8400dbe0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.369+0000 7f9e9b26c640 1 -- 172.21.15.38:0/340749136 shutdown_connections 2024-05-21T15:39:08.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.369+0000 7f9e9b26c640 1 --2- 172.21.15.38:0/340749136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e94108830 0x7f9e941015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.369+0000 7f9e9b26c640 1 --2- 172.21.15.38:0/340749136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e94105ed0 0x7f9e94101060 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.369+0000 7f9e9b26c640 1 --2- 172.21.15.38:0/340749136 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9e800a2b90 0x7f9e800a50a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.369+0000 7f9e9b26c640 1 --2- 172.21.15.38:0/340749136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e94103570 0x7f9e941029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.369+0000 7f9e9b26c640 1 -- 172.21.15.38:0/340749136 >> 172.21.15.38:0/340749136 conn(0x7f9e940fcb70 msgr2=0x7f9e94104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.369+0000 7f9e9b26c640 1 -- 172.21.15.38:0/340749136 shutdown_connections 2024-05-21T15:39:08.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.369+0000 7f9e9b26c640 1 -- 172.21.15.38:0/340749136 wait complete. 2024-05-21T15:39:08.383 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:39:08.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.523+0000 7f45636fe640 1 -- 172.21.15.38:0/2814668771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c103570 msgr2=0x7f455c105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.523+0000 7f45636fe640 1 --2- 172.21.15.38:0/2814668771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c103570 0x7f455c105990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4558009900 tx=0x7f455802a410 comp rx=0 tx=0).stop 2024-05-21T15:39:08.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.523+0000 7f45636fe640 1 -- 172.21.15.38:0/2814668771 shutdown_connections 2024-05-21T15:39:08.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.523+0000 7f45636fe640 1 --2- 172.21.15.38:0/2814668771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f455c108830 0x7f455c10ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.523+0000 7f45636fe640 1 --2- 172.21.15.38:0/2814668771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f455c105ed0 0x7f455c1082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.523+0000 7f45636fe640 1 --2- 172.21.15.38:0/2814668771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c103570 0x7f455c105990 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.523+0000 7f45636fe640 1 -- 172.21.15.38:0/2814668771 >> 172.21.15.38:0/2814668771 conn(0x7f455c0fcb70 msgr2=0x7f455c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.523+0000 7f45636fe640 1 -- 172.21.15.38:0/2814668771 shutdown_connections 2024-05-21T15:39:08.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.523+0000 7f45636fe640 1 -- 172.21.15.38:0/2814668771 wait complete. 2024-05-21T15:39:08.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.524+0000 7f45636fe640 1 Processor -- start 2024-05-21T15:39:08.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.524+0000 7f45636fe640 1 -- start start 2024-05-21T15:39:08.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.524+0000 7f45636fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f455c103570 0x7f455c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.524+0000 7f45636fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f455c105ed0 0x7f455c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.524+0000 7f45636fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c108830 0x7f455c1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.524+0000 7f45636fe640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f455c10e720 con 0x7f455c108830 2024-05-21T15:39:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.524+0000 7f45636fe640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f455c10e5a0 con 0x7f455c105ed0 2024-05-21T15:39:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.524+0000 7f45636fe640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f455c10e8a0 con 0x7f455c103570 2024-05-21T15:39:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.524+0000 7f4561c9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c108830 0x7f455c1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.524+0000 7f4561c9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c108830 0x7f455c1a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41194/0 (socket says 172.21.15.38:41194) 2024-05-21T15:39:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.524+0000 7f4561c9b640 1 -- 172.21.15.38:0/1627092205 learned_addr learned my addr 172.21.15.38:0/1627092205 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.524+0000 7f4560c99640 1 --2- 172.21.15.38:0/1627092205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f455c105ed0 0x7f455c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.524+0000 7f456149a640 1 --2- 172.21.15.38:0/1627092205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f455c103570 0x7f455c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.525+0000 7f4561c9b640 1 -- 172.21.15.38:0/1627092205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f455c103570 msgr2=0x7f455c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.525+0000 7f4561c9b640 1 --2- 172.21.15.38:0/1627092205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f455c103570 0x7f455c1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.525+0000 7f4561c9b640 1 -- 172.21.15.38:0/1627092205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f455c105ed0 msgr2=0x7f455c1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.525+0000 7f4561c9b640 1 --2- 172.21.15.38:0/1627092205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f455c105ed0 0x7f455c1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.525+0000 7f4561c9b640 1 -- 172.21.15.38:0/1627092205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c1a0de0 con 0x7f455c108830 2024-05-21T15:39:08.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.525+0000 7f4561c9b640 1 --2- 172.21.15.38:0/1627092205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c108830 0x7f455c1a0770 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4554009c90 tx=0x7f455400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.525+0000 7f45527fc640 1 -- 172.21.15.38:0/1627092205 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554010630 con 0x7f455c108830 2024-05-21T15:39:08.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.525+0000 7f45527fc640 1 -- 172.21.15.38:0/1627092205 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4554010cb0 con 0x7f455c108830 2024-05-21T15:39:08.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.525+0000 7f45636fe640 1 -- 172.21.15.38:0/1627092205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c1a10d0 con 0x7f455c108830 2024-05-21T15:39:08.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.525+0000 7f45527fc640 1 -- 172.21.15.38:0/1627092205 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554017d20 con 0x7f455c108830 2024-05-21T15:39:08.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.525+0000 7f45636fe640 1 -- 172.21.15.38:0/1627092205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c1a1670 con 0x7f455c108830 2024-05-21T15:39:08.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.528+0000 7f45527fc640 1 -- 172.21.15.38:0/1627092205 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f45540107d0 con 0x7f455c108830 2024-05-21T15:39:08.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.528+0000 7f456a5a6740 1 Processor -- start 2024-05-21T15:39:08.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.528+0000 7f456a5a6740 1 -- start start 2024-05-21T15:39:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.528+0000 7f456a5a6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a964390 0x55b21ad03b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f456a5a6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a93b4d0 0x55b21ad04080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f456a5a6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a9436b0 0x55b21ad0af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f456a5a6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b21ac68c40 con 0x55b21a9436b0 2024-05-21T15:39:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f456a5a6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b21ac10580 con 0x55b21a93b4d0 2024-05-21T15:39:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f456a5a6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b21ac64c00 con 0x55b21a964390 2024-05-21T15:39:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f4560c99640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a9436b0 0x55b21ad0af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f456149a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a93b4d0 0x55b21ad04080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f456149a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a93b4d0 0x55b21ad04080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:39906/0 (socket says 172.21.15.38:39906) 2024-05-21T15:39:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f4560c99640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a9436b0 0x55b21ad0af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41198/0 (socket says 172.21.15.38:41198) 2024-05-21T15:39:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f456149a640 1 -- 172.21.15.38:0/2933891906 learned_addr learned my addr 172.21.15.38:0/2933891906 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f4560c99640 1 --2- 172.21.15.38:0/2933891906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a964390 0x55b21ad03b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f45527fc640 1 --2- 172.21.15.38:0/1627092205 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f45480a2c10 0x7f45480a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f4561c9b640 1 --2- 172.21.15.38:0/1627092205 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f45480a2c10 0x7f45480a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f456149a640 1 -- 172.21.15.38:0/2933891906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a964390 msgr2=0x55b21ad03b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f456149a640 1 --2- 172.21.15.38:0/2933891906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a964390 0x55b21ad03b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f456149a640 1 -- 172.21.15.38:0/2933891906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a9436b0 msgr2=0x55b21ad0af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f456149a640 1 --2- 172.21.15.38:0/2933891906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a9436b0 0x55b21ad0af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f456149a640 1 -- 172.21.15.38:0/2933891906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b21a97c3f0 con 0x55b21a93b4d0 2024-05-21T15:39:08.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f4560c99640 1 --2- 172.21.15.38:0/2933891906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a9436b0 0x55b21ad0af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:08.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.529+0000 7f45527fc640 1 -- 172.21.15.38:0/1627092205 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f45540cf8a0 con 0x7f455c108830 2024-05-21T15:39:08.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.530+0000 7f456149a640 1 --2- 172.21.15.38:0/2933891906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a93b4d0 0x55b21ad04080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f455c0699d0 tx=0x7f4558009660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2720f8992ec1e68 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.530+0000 7f4561c9b640 1 --2- 172.21.15.38:0/1627092205 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f45480a2c10 0x7f45480a5120 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f455409bdb0 tx=0x7f4554066c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.530+0000 7f45477fe640 1 -- 172.21.15.38:0/2933891906 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558035030 con 0x55b21a93b4d0 2024-05-21T15:39:08.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.530+0000 7f45477fe640 1 -- 172.21.15.38:0/2933891906 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45580043c0 con 0x55b21a93b4d0 2024-05-21T15:39:08.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.530+0000 7f45477fe640 1 -- 172.21.15.38:0/2933891906 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558004e50 con 0x55b21a93b4d0 2024-05-21T15:39:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.531+0000 7f456a5a6740 1 -- 172.21.15.38:0/2933891906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a93b4d0 msgr2=0x55b21ad04080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.531+0000 7f456a5a6740 1 --2- 172.21.15.38:0/2933891906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a93b4d0 0x55b21ad04080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f455c0699d0 tx=0x7f4558009660 comp rx=0 tx=0).stop 2024-05-21T15:39:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.531+0000 7f456a5a6740 1 -- 172.21.15.38:0/2933891906 shutdown_connections 2024-05-21T15:39:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.531+0000 7f456a5a6740 1 --2- 172.21.15.38:0/2933891906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a9436b0 0x55b21ad0af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.531+0000 7f456a5a6740 1 --2- 172.21.15.38:0/2933891906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a93b4d0 0x55b21ad04080 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.531+0000 7f456a5a6740 1 --2- 172.21.15.38:0/2933891906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a964390 0x55b21ad03b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.531+0000 7f456a5a6740 1 -- 172.21.15.38:0/2933891906 >> 172.21.15.38:0/2933891906 conn(0x55b21a9584e0 msgr2=0x55b21acff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.531+0000 7f456a5a6740 1 -- 172.21.15.38:0/2933891906 shutdown_connections 2024-05-21T15:39:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.531+0000 7f456a5a6740 1 -- 172.21.15.38:0/2933891906 wait complete. 2024-05-21T15:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.532+0000 7f456a5a6740 1 -- start start 2024-05-21T15:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.532+0000 7f456a5a6740 1 Processor -- start 2024-05-21T15:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.532+0000 7f456a5a6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a93b4d0 0x55b21ad0d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.532+0000 7f456a5a6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a9436b0 0x55b21ad0de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.532+0000 7f456a5a6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a964390 0x55b21ad14dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.532+0000 7f456a5a6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b21ad0b970 con 0x55b21a9436b0 2024-05-21T15:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.532+0000 7f456a5a6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b21ac23c40 con 0x55b21a964390 2024-05-21T15:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.532+0000 7f456a5a6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b21ad0baf0 con 0x55b21a93b4d0 2024-05-21T15:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.532+0000 7f456a5a6740 10 client.0 _ll_register_callbacks cb 0x55b21a97bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f456149a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a9436b0 0x55b21ad0de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f456149a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a9436b0 0x55b21ad0de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41204/0 (socket says 172.21.15.38:41204) 2024-05-21T15:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f456149a640 1 -- 172.21.15.38:0/3975564050 learned_addr learned my addr 172.21.15.38:0/3975564050 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f4560c99640 1 --2- 172.21.15.38:0/3975564050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a964390 0x55b21ad14dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f4560c99640 1 --2- 172.21.15.38:0/3975564050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a93b4d0 0x55b21ad0d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f456149a640 1 -- 172.21.15.38:0/3975564050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a93b4d0 msgr2=0x55b21ad0d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f456149a640 1 --2- 172.21.15.38:0/3975564050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a93b4d0 0x55b21ad0d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f456149a640 1 -- 172.21.15.38:0/3975564050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a964390 msgr2=0x55b21ad14dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f456149a640 1 --2- 172.21.15.38:0/3975564050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a964390 0x55b21ad14dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f456149a640 1 -- 172.21.15.38:0/3975564050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b21aa467d0 con 0x55b21a9436b0 2024-05-21T15:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f4560c99640 1 --2- 172.21.15.38:0/3975564050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a964390 0x55b21ad14dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f456149a640 1 --2- 172.21.15.38:0/3975564050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a9436b0 0x55b21ad0de60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f455802a930 tx=0x7f45580318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01cbcb6bce345cd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f4544ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f4544ff9640 1 -- 172.21.15.38:0/3975564050 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558035030 con 0x55b21a9436b0 2024-05-21T15:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f4544ff9640 1 -- 172.21.15.38:0/3975564050 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f45580041c0 con 0x55b21a9436b0 2024-05-21T15:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.533+0000 7f456a5a6740 1 -- 172.21.15.38:0/3975564050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45580043c0 con 0x55b21a9436b0 2024-05-21T15:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.534+0000 7f4544ff9640 1 -- 172.21.15.38:0/3975564050 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f45580035f0 con 0x55b21a9436b0 2024-05-21T15:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.534+0000 7f456a5a6740 10 client.16695 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:08.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.534+0000 7f456a5a6740 1 -- 172.21.15.38:0/3975564050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b21ad18080 con 0x55b21a9436b0 2024-05-21T15:39:08.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.534+0000 7f4544ff9640 1 -- 172.21.15.38:0/3975564050 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45580043c0 con 0x55b21a9436b0 2024-05-21T15:39:08.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.534+0000 7f4544ff9640 1 -- 172.21.15.38:0/3975564050 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f4558031ce0 con 0x55b21a9436b0 2024-05-21T15:39:08.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.534+0000 7f456a5a6740 10 client.16695 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:08.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.534+0000 7f456a5a6740 10 client.16695 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:39:08.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.534+0000 7f456a5a6740 20 client.16695 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:08.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.534+0000 7f456a5a6740 1 --2- 172.21.15.38:0/3975564050 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b21a922f60 0x55b21ad1b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.534+0000 7f456a5a6740 4 client.16695 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:08.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.534+0000 7f456a5a6740 1 -- 172.21.15.38:0/3975564050 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b21ac76370 con 0x55b21a922f60 2024-05-21T15:39:08.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.534+0000 7f4560c99640 1 --2- 172.21.15.38:0/3975564050 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b21a922f60 0x55b21ad1b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.535+0000 7f4560c99640 1 --2- 172.21.15.38:0/3975564050 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b21a922f60 0x55b21ad1b3e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50ed684d61012883 server_cookie=c22ca51079627fef in_seq=0 out_seq=0 2024-05-21T15:39:08.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.535+0000 7f4544ff9640 10 client.16695 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:39:08.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f4544ff9640 1 -- 172.21.15.38:0/3975564050 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b21ac76370 con 0x55b21a922f60 2024-05-21T15:39:08.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f4544ff9640 10 client.16695 handle_command_reply: tid=0 2024-05-21T15:39:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f456a5a6740 1 client.16695 shutdown 2024-05-21T15:39:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f456a5a6740 1 -- 172.21.15.38:0/3975564050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a9436b0 msgr2=0x55b21ad0de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f456a5a6740 1 --2- 172.21.15.38:0/3975564050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a9436b0 0x55b21ad0de60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f455802a930 tx=0x7f45580318e0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f456a5a6740 1 -- 172.21.15.38:0/3975564050 shutdown_connections 2024-05-21T15:39:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f456a5a6740 1 --2- 172.21.15.38:0/3975564050 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b21a922f60 0x55b21ad1b3e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f456a5a6740 1 --2- 172.21.15.38:0/3975564050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a964390 0x55b21ad14dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f456a5a6740 1 --2- 172.21.15.38:0/3975564050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a9436b0 0x55b21ad0de60 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f4544ff9640 0 client.16695 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:39:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f456a5a6740 1 --2- 172.21.15.38:0/3975564050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a93b4d0 0x55b21ad0d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f456a5a6740 1 -- 172.21.15.38:0/3975564050 >> 172.21.15.38:0/3975564050 conn(0x55b21a9584e0 msgr2=0x55b21acfe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f456a5a6740 1 -- 172.21.15.38:0/3975564050 shutdown_connections 2024-05-21T15:39:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f456a5a6740 1 -- 172.21.15.38:0/3975564050 wait complete. 2024-05-21T15:39:08.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.536+0000 7f456a5a6740 20 client.16695 trim_cache size 0 max 16384 2024-05-21T15:39:08.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.551+0000 7f456a5a6740 1 Processor -- start 2024-05-21T15:39:08.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.551+0000 7f456a5a6740 1 -- start start 2024-05-21T15:39:08.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.551+0000 7f456a5a6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a93b4d0 0x55b21ad02a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.551+0000 7f456a5a6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a9436b0 0x55b21ad14ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.551+0000 7f456a5a6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a964390 0x55b21ad20ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.551+0000 7f456a5a6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1aa430 con 0x55b21a964390 2024-05-21T15:39:08.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.551+0000 7f456a5a6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ac060 con 0x55b21a9436b0 2024-05-21T15:39:08.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.552+0000 7f456a5a6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1aab00 con 0x55b21a93b4d0 2024-05-21T15:39:08.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.552+0000 7f4560c99640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a964390 0x55b21ad20ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.552+0000 7f456149a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a9436b0 0x55b21ad14ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.552+0000 7f4560c99640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a964390 0x55b21ad20ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41210/0 (socket says 172.21.15.38:41210) 2024-05-21T15:39:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.552+0000 7f4560c99640 1 -- 172.21.15.38:0/4047619911 learned_addr learned my addr 172.21.15.38:0/4047619911 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.552+0000 7f456149a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a9436b0 0x55b21ad14ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:39926/0 (socket says 172.21.15.38:39926) 2024-05-21T15:39:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.552+0000 7f4560c99640 1 -- 172.21.15.38:0/4047619911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a93b4d0 msgr2=0x55b21ad02a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.552+0000 7f4560c99640 1 --2- 172.21.15.38:0/4047619911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a93b4d0 0x55b21ad02a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.552+0000 7f4560c99640 1 -- 172.21.15.38:0/4047619911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a9436b0 msgr2=0x55b21ad14ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.552+0000 7f4560c99640 1 --2- 172.21.15.38:0/4047619911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a9436b0 0x55b21ad14ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.552+0000 7f4560c99640 1 -- 172.21.15.38:0/4047619911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b21ad000c0 con 0x55b21a964390 2024-05-21T15:39:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.552+0000 7f4560c99640 1 --2- 172.21.15.38:0/4047619911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a964390 0x55b21ad20ad0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x55b21ad04cd0 tx=0x7f454c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e2755e5d86ab9d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.552+0000 7f4546ffd640 1 -- 172.21.15.38:0/4047619911 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c004050 con 0x55b21a964390 2024-05-21T15:39:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.553+0000 7f4546ffd640 1 -- 172.21.15.38:0/4047619911 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c021d60 con 0x55b21a964390 2024-05-21T15:39:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.553+0000 7f4546ffd640 1 -- 172.21.15.38:0/4047619911 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c004da0 con 0x55b21a964390 2024-05-21T15:39:08.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.554+0000 7f456a5a6740 1 -- 172.21.15.38:0/4047619911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a964390 msgr2=0x55b21ad20ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.554+0000 7f456a5a6740 1 --2- 172.21.15.38:0/4047619911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a964390 0x55b21ad20ad0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x55b21ad04cd0 tx=0x7f454c005f90 comp rx=0 tx=0).stop 2024-05-21T15:39:08.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.554+0000 7f456a5a6740 1 -- 172.21.15.38:0/4047619911 shutdown_connections 2024-05-21T15:39:08.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.554+0000 7f456a5a6740 1 --2- 172.21.15.38:0/4047619911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a964390 0x55b21ad20ad0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.554+0000 7f456a5a6740 1 --2- 172.21.15.38:0/4047619911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a9436b0 0x55b21ad14ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.554+0000 7f456a5a6740 1 --2- 172.21.15.38:0/4047619911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a93b4d0 0x55b21ad02a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.554+0000 7f456a5a6740 1 -- 172.21.15.38:0/4047619911 >> 172.21.15.38:0/4047619911 conn(0x55b21a9584e0 msgr2=0x55b21ad02330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.554+0000 7f456a5a6740 1 -- 172.21.15.38:0/4047619911 shutdown_connections 2024-05-21T15:39:08.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.554+0000 7f456a5a6740 1 -- 172.21.15.38:0/4047619911 wait complete. 2024-05-21T15:39:08.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.554+0000 7f456a5a6740 1 -- start start 2024-05-21T15:39:08.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.554+0000 7f456a5a6740 1 Processor -- start 2024-05-21T15:39:08.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.555+0000 7f456a5a6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a93b4d0 0x55b21ad11f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.555+0000 7f456a5a6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a9436b0 0x55b21ad12490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.555+0000 7f456a5a6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a964390 0x55b21ad133c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.555+0000 7f456a5a6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ab8d0 con 0x55b21a964390 2024-05-21T15:39:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.555+0000 7f456a5a6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ab1d0 con 0x55b21a9436b0 2024-05-21T15:39:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.555+0000 7f456a5a6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1a8410 con 0x55b21a93b4d0 2024-05-21T15:39:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.555+0000 7f456a5a6740 10 client.0 _ll_register_callbacks cb 0x7f454c009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.555+0000 7f4560c99640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a964390 0x55b21ad133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.555+0000 7f4560c99640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a964390 0x55b21ad133c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41220/0 (socket says 172.21.15.38:41220) 2024-05-21T15:39:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.555+0000 7f456149a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a9436b0 0x55b21ad12490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.555+0000 7f4560c99640 1 -- 172.21.15.38:0/3177542526 learned_addr learned my addr 172.21.15.38:0/3177542526 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f4560c99640 1 -- 172.21.15.38:0/3177542526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a93b4d0 msgr2=0x55b21ad11f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f4560c99640 1 --2- 172.21.15.38:0/3177542526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a93b4d0 0x55b21ad11f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f4560c99640 1 -- 172.21.15.38:0/3177542526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a9436b0 msgr2=0x55b21ad12490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f4560c99640 1 --2- 172.21.15.38:0/3177542526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a9436b0 0x55b21ad12490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f4560c99640 1 -- 172.21.15.38:0/3177542526 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b21ad15220 con 0x55b21a964390 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f456149a640 1 --2- 172.21.15.38:0/3177542526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a9436b0 0x55b21ad12490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f4560c99640 1 --2- 172.21.15.38:0/3177542526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a964390 0x55b21ad133c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f454c0097a0 tx=0x7f454c011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d13a4d415240ec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f4544ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f4544ff9640 1 -- 172.21.15.38:0/3177542526 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c004b20 con 0x55b21a964390 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f4544ff9640 1 -- 172.21.15.38:0/3177542526 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f454c004cc0 con 0x55b21a964390 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f456a5a6740 1 -- 172.21.15.38:0/3177542526 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454c021d60 con 0x55b21a964390 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f4544ff9640 1 -- 172.21.15.38:0/3177542526 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f454c021880 con 0x55b21a964390 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f456a5a6740 10 client.16701 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f456a5a6740 1 -- 172.21.15.38:0/3177542526 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b21ad14a30 con 0x55b21a964390 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.556+0000 7f4544ff9640 1 -- 172.21.15.38:0/3177542526 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c021d60 con 0x55b21a964390 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.557+0000 7f4544ff9640 1 -- 172.21.15.38:0/3177542526 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f454c02b030 con 0x55b21a964390 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.557+0000 7f456a5a6740 10 client.16701 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.557+0000 7f456a5a6740 10 client.16701 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.557+0000 7f456a5a6740 20 client.16701 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.557+0000 7f456a5a6740 1 --2- 172.21.15.38:0/3177542526 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b21a8f83a0 0x55b21ad85810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.557+0000 7f456a5a6740 4 client.16701 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:39:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.557+0000 7f456a5a6740 1 -- 172.21.15.38:0/3177542526 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b21ac706c0 con 0x55b21a8f83a0 2024-05-21T15:39:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.557+0000 7f456149a640 1 --2- 172.21.15.38:0/3177542526 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b21a8f83a0 0x55b21ad85810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.557+0000 7f4544ff9640 10 client.16701 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:39:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.557+0000 7f456149a640 1 --2- 172.21.15.38:0/3177542526 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b21a8f83a0 0x55b21ad85810 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7bd6286662d0e39 server_cookie=bbd8ff0e5740f391 in_seq=0 out_seq=0 2024-05-21T15:39:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f4544ff9640 1 -- 172.21.15.38:0/3177542526 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b21ac706c0 con 0x55b21a8f83a0 2024-05-21T15:39:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f4544ff9640 10 client.16701 handle_command_reply: tid=0 2024-05-21T15:39:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f456a5a6740 1 client.16701 shutdown 2024-05-21T15:39:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f456a5a6740 1 -- 172.21.15.38:0/3177542526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a964390 msgr2=0x55b21ad133c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f456a5a6740 1 --2- 172.21.15.38:0/3177542526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a964390 0x55b21ad133c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f454c0097a0 tx=0x7f454c011e30 comp rx=0 tx=0).stop 2024-05-21T15:39:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f456a5a6740 1 -- 172.21.15.38:0/3177542526 shutdown_connections 2024-05-21T15:39:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f456a5a6740 1 --2- 172.21.15.38:0/3177542526 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b21a8f83a0 0x55b21ad85810 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f456a5a6740 1 --2- 172.21.15.38:0/3177542526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b21a964390 0x55b21ad133c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f456a5a6740 1 --2- 172.21.15.38:0/3177542526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b21a9436b0 0x55b21ad12490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f456a5a6740 1 --2- 172.21.15.38:0/3177542526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b21a93b4d0 0x55b21ad11f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f456a5a6740 1 -- 172.21.15.38:0/3177542526 >> 172.21.15.38:0/3177542526 conn(0x55b21a9584e0 msgr2=0x55b21ad02d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f4544ff9640 0 client.16701 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:39:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f456a5a6740 1 -- 172.21.15.38:0/3177542526 shutdown_connections 2024-05-21T15:39:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f456a5a6740 1 -- 172.21.15.38:0/3177542526 wait complete. 2024-05-21T15:39:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.558+0000 7f456a5a6740 20 client.16701 trim_cache size 0 max 16384 2024-05-21T15:39:08.561 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:39:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.559+0000 7f45636fe640 1 -- 172.21.15.38:0/1627092205 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f45480a2c10 msgr2=0x7f45480a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.559+0000 7f45636fe640 1 --2- 172.21.15.38:0/1627092205 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f45480a2c10 0x7f45480a5120 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f455409bdb0 tx=0x7f4554066c80 comp rx=0 tx=0).stop 2024-05-21T15:39:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.559+0000 7f45636fe640 1 -- 172.21.15.38:0/1627092205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c108830 msgr2=0x7f455c1a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.559+0000 7f45636fe640 1 --2- 172.21.15.38:0/1627092205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c108830 0x7f455c1a0770 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4554009c90 tx=0x7f455400c6a0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.559+0000 7f45636fe640 1 -- 172.21.15.38:0/1627092205 shutdown_connections 2024-05-21T15:39:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.559+0000 7f45636fe640 1 --2- 172.21.15.38:0/1627092205 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f45480a2c10 0x7f45480a5120 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.559+0000 7f45636fe640 1 --2- 172.21.15.38:0/1627092205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c108830 0x7f455c1a0770 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.559+0000 7f45636fe640 1 --2- 172.21.15.38:0/1627092205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f455c105ed0 0x7f455c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.559+0000 7f45636fe640 1 --2- 172.21.15.38:0/1627092205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f455c103570 0x7f455c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.559+0000 7f45636fe640 1 -- 172.21.15.38:0/1627092205 >> 172.21.15.38:0/1627092205 conn(0x7f455c0fcb70 msgr2=0x7f455c1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.559+0000 7f45636fe640 1 -- 172.21.15.38:0/1627092205 shutdown_connections 2024-05-21T15:39:08.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.559+0000 7f45636fe640 1 -- 172.21.15.38:0/1627092205 wait complete. 2024-05-21T15:39:08.573 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:39:08.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.706+0000 7f8b08ef0640 1 -- 172.21.15.38:0/2887333959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b04102ae0 msgr2=0x7f8b041037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.706+0000 7f8b08ef0640 1 --2- 172.21.15.38:0/2887333959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b04102ae0 0x7f8b041037e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009900 tx=0x7f8af402a410 comp rx=0 tx=0).stop 2024-05-21T15:39:08.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.706+0000 7f8b08ef0640 1 -- 172.21.15.38:0/2887333959 shutdown_connections 2024-05-21T15:39:08.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.706+0000 7f8b08ef0640 1 --2- 172.21.15.38:0/2887333959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04108a20 0x7f8b0410ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.706+0000 7f8b08ef0640 1 --2- 172.21.15.38:0/2887333959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b04103d20 0x7f8b04104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.706+0000 7f8b08ef0640 1 --2- 172.21.15.38:0/2887333959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b04102ae0 0x7f8b041037e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.706+0000 7f8b08ef0640 1 -- 172.21.15.38:0/2887333959 >> 172.21.15.38:0/2887333959 conn(0x7f8b040fdcf0 msgr2=0x7f8b04100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.706+0000 7f8b08ef0640 1 -- 172.21.15.38:0/2887333959 shutdown_connections 2024-05-21T15:39:08.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.706+0000 7f8b08ef0640 1 -- 172.21.15.38:0/2887333959 wait complete. 2024-05-21T15:39:08.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.707+0000 7f8b08ef0640 1 Processor -- start 2024-05-21T15:39:08.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.707+0000 7f8b08ef0640 1 -- start start 2024-05-21T15:39:08.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.707+0000 7f8b08ef0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b04102ae0 0x7f8b041992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.707+0000 7f8b08ef0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b04103d20 0x7f8b041997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.707+0000 7f8b08ef0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04108a20 0x7f8b041a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.707+0000 7f8b08ef0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0410e730 con 0x7f8b04102ae0 2024-05-21T15:39:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.707+0000 7f8b08ef0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0410e5b0 con 0x7f8b04108a20 2024-05-21T15:39:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.707+0000 7f8b08ef0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0410e8b0 con 0x7f8b04103d20 2024-05-21T15:39:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.707+0000 7f8b0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b04102ae0 0x7f8b041992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.707+0000 7f8b0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b04102ae0 0x7f8b041992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41242/0 (socket says 172.21.15.38:41242) 2024-05-21T15:39:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.707+0000 7f8b0259c640 1 -- 172.21.15.38:0/183864559 learned_addr learned my addr 172.21.15.38:0/183864559 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.707+0000 7f8b02d9d640 1 --2- 172.21.15.38:0/183864559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04108a20 0x7f8b041a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.708+0000 7f8af9d9b640 1 --2- 172.21.15.38:0/183864559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b04103d20 0x7f8b041997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.708+0000 7f8b0259c640 1 -- 172.21.15.38:0/183864559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b04103d20 msgr2=0x7f8b041997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.708+0000 7f8b0259c640 1 --2- 172.21.15.38:0/183864559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b04103d20 0x7f8b041997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.708+0000 7f8b0259c640 1 -- 172.21.15.38:0/183864559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04108a20 msgr2=0x7f8b041a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.708+0000 7f8b0259c640 1 --2- 172.21.15.38:0/183864559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04108a20 0x7f8b041a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.708+0000 7f8b0259c640 1 -- 172.21.15.38:0/183864559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b041a0de0 con 0x7f8b04102ae0 2024-05-21T15:39:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.708+0000 7f8b0259c640 1 --2- 172.21.15.38:0/183864559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b04102ae0 0x7f8b041992a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8af4006010 tx=0x7f8af40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.708+0000 7f8afbfff640 1 -- 172.21.15.38:0/183864559 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af403f030 con 0x7f8b04102ae0 2024-05-21T15:39:08.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.708+0000 7f8b08ef0640 1 -- 172.21.15.38:0/183864559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b041a1070 con 0x7f8b04102ae0 2024-05-21T15:39:08.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.708+0000 7f8afbfff640 1 -- 172.21.15.38:0/183864559 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8af4003ce0 con 0x7f8b04102ae0 2024-05-21T15:39:08.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.708+0000 7f8afbfff640 1 -- 172.21.15.38:0/183864559 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4035070 con 0x7f8b04102ae0 2024-05-21T15:39:08.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.708+0000 7f8b08ef0640 1 -- 172.21.15.38:0/183864559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b041a1560 con 0x7f8b04102ae0 2024-05-21T15:39:08.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.711+0000 7f8afbfff640 1 -- 172.21.15.38:0/183864559 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8af4042030 con 0x7f8b04102ae0 2024-05-21T15:39:08.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.711+0000 7f8b0fd96740 1 Processor -- start 2024-05-21T15:39:08.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.711+0000 7f8b0fd96740 1 -- start start 2024-05-21T15:39:08.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0fd96740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a594e390 0x5603a5cedb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0fd96740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cee080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0fd96740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a592d6b0 0x5603a5cf4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0fd96740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5603a5c52c40 con 0x5603a59254d0 2024-05-21T15:39:08.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0fd96740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5603a5bfa580 con 0x5603a592d6b0 2024-05-21T15:39:08.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0fd96740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5603a5c4ec00 con 0x5603a594e390 2024-05-21T15:39:08.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cee080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41258/0 (socket says 172.21.15.38:41258) 2024-05-21T15:39:08.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0259c640 1 -- 172.21.15.38:0/3807340192 learned_addr learned my addr 172.21.15.38:0/3807340192 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8af9d9b640 1 --2- 172.21.15.38:0/3807340192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a592d6b0 0x5603a5cf4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b02d9d640 1 --2- 172.21.15.38:0/3807340192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a594e390 0x5603a5cedb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8afbfff640 1 --2- 172.21.15.38:0/183864559 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8af00a2b90 0x7f8af00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b02d9d640 1 --2- 172.21.15.38:0/183864559 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8af00a2b90 0x7f8af00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0259c640 1 -- 172.21.15.38:0/3807340192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a594e390 msgr2=0x5603a5cedb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0259c640 1 --2- 172.21.15.38:0/3807340192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a594e390 0x5603a5cedb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0259c640 1 -- 172.21.15.38:0/3807340192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a592d6b0 msgr2=0x5603a5cf4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0259c640 1 --2- 172.21.15.38:0/3807340192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a592d6b0 0x5603a5cf4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0259c640 1 -- 172.21.15.38:0/3807340192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a59663f0 con 0x5603a59254d0 2024-05-21T15:39:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8afbfff640 1 -- 172.21.15.38:0/183864559 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8af40ef5b0 con 0x7f8b04102ae0 2024-05-21T15:39:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.712+0000 7f8b0259c640 1 --2- 172.21.15.38:0/3807340192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cee080 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8af40b8840 tx=0x7f8af4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed3a68c176a2684 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.713+0000 7f8b02d9d640 1 --2- 172.21.15.38:0/183864559 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8af00a2b90 0x7f8af00a50a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009fd0 tx=0x7f8afc004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.713+0000 7f8af8d99640 1 -- 172.21.15.38:0/3807340192 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4049070 con 0x5603a59254d0 2024-05-21T15:39:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.713+0000 7f8af8d99640 1 -- 172.21.15.38:0/3807340192 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8af40bbe70 con 0x5603a59254d0 2024-05-21T15:39:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.713+0000 7f8af8d99640 1 -- 172.21.15.38:0/3807340192 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af40c0170 con 0x5603a59254d0 2024-05-21T15:39:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 -- 172.21.15.38:0/3807340192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 msgr2=0x5603a5cee080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/3807340192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cee080 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8af40b8840 tx=0x7f8af4031400 comp rx=0 tx=0).stop 2024-05-21T15:39:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 -- 172.21.15.38:0/3807340192 shutdown_connections 2024-05-21T15:39:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/3807340192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a592d6b0 0x5603a5cf4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/3807340192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cee080 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/3807340192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a594e390 0x5603a5cedb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 -- 172.21.15.38:0/3807340192 >> 172.21.15.38:0/3807340192 conn(0x5603a59424e0 msgr2=0x5603a5ce9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 -- 172.21.15.38:0/3807340192 shutdown_connections 2024-05-21T15:39:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 -- 172.21.15.38:0/3807340192 wait complete. 2024-05-21T15:39:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 -- start start 2024-05-21T15:39:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 Processor -- start 2024-05-21T15:39:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cf7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a592d6b0 0x5603a5cf7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a594e390 0x5603a5cfead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5603a5cf5790 con 0x5603a59254d0 2024-05-21T15:39:08.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5603a5c0dc40 con 0x5603a594e390 2024-05-21T15:39:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.714+0000 7f8b0fd96740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5603a5cf5910 con 0x5603a592d6b0 2024-05-21T15:39:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8b0fd96740 10 client.0 _ll_register_callbacks cb 0x5603a5965be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8b0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cf7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8b0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cf7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41274/0 (socket says 172.21.15.38:41274) 2024-05-21T15:39:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8b0259c640 1 -- 172.21.15.38:0/462661833 learned_addr learned my addr 172.21.15.38:0/462661833 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8b02d9d640 1 --2- 172.21.15.38:0/462661833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a594e390 0x5603a5cfead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8af9d9b640 1 --2- 172.21.15.38:0/462661833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a592d6b0 0x5603a5cf7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8b0259c640 1 -- 172.21.15.38:0/462661833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a592d6b0 msgr2=0x5603a5cf7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8b0259c640 1 --2- 172.21.15.38:0/462661833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a592d6b0 0x5603a5cf7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8b0259c640 1 -- 172.21.15.38:0/462661833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a594e390 msgr2=0x5603a5cfead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8b0259c640 1 --2- 172.21.15.38:0/462661833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a594e390 0x5603a5cfead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8b0259c640 1 -- 172.21.15.38:0/462661833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603a5a307d0 con 0x5603a59254d0 2024-05-21T15:39:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8b02d9d640 1 --2- 172.21.15.38:0/462661833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a594e390 0x5603a5cfead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:08.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8b0259c640 1 --2- 172.21.15.38:0/462661833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cf7630 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8af40c4500 tx=0x7f8af4034260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d3459273ae6cc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8ae9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.715+0000 7f8ae9ffb640 1 -- 172.21.15.38:0/462661833 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4049040 con 0x5603a59254d0 2024-05-21T15:39:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.716+0000 7f8ae9ffb640 1 -- 172.21.15.38:0/462661833 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8af40b7c40 con 0x5603a59254d0 2024-05-21T15:39:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.716+0000 7f8b0fd96740 1 -- 172.21.15.38:0/462661833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8af40bbe70 con 0x5603a59254d0 2024-05-21T15:39:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.716+0000 7f8ae9ffb640 1 -- 172.21.15.38:0/462661833 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8af40bbc40 con 0x5603a59254d0 2024-05-21T15:39:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.716+0000 7f8b0fd96740 10 client.16713 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.716+0000 7f8b0fd96740 1 -- 172.21.15.38:0/462661833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5603a5cf5790 con 0x5603a59254d0 2024-05-21T15:39:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.716+0000 7f8ae9ffb640 1 -- 172.21.15.38:0/462661833 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8af40bbe70 con 0x5603a59254d0 2024-05-21T15:39:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.716+0000 7f8ae9ffb640 1 -- 172.21.15.38:0/462661833 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f8af403b460 con 0x5603a59254d0 2024-05-21T15:39:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.716+0000 7f8b0fd96740 10 client.16713 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.716+0000 7f8b0fd96740 10 client.16713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.716+0000 7f8b0fd96740 20 client.16713 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:08.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.716+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/462661833 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5603a590cf60 0x5603a5d052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.716+0000 7f8b0fd96740 4 client.16713 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:08.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.716+0000 7f8b0fd96740 1 -- 172.21.15.38:0/462661833 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603a5c60370 con 0x5603a590cf60 2024-05-21T15:39:08.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.717+0000 7f8af9d9b640 1 --2- 172.21.15.38:0/462661833 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5603a590cf60 0x5603a5d052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.717+0000 7f8ae9ffb640 10 client.16713 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:08.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.717+0000 7f8af9d9b640 1 --2- 172.21.15.38:0/462661833 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5603a590cf60 0x5603a5d052f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d59f90ee6a157e server_cookie=891333ec7321601 in_seq=0 out_seq=0 2024-05-21T15:39:08.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.718+0000 7f8ae9ffb640 1 -- 172.21.15.38:0/462661833 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5603a5c60370 con 0x5603a590cf60 2024-05-21T15:39:08.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.718+0000 7f8ae9ffb640 10 client.16713 handle_command_reply: tid=0 2024-05-21T15:39:08.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.718+0000 7f8b0fd96740 1 client.16713 shutdown 2024-05-21T15:39:08.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.718+0000 7f8b0fd96740 1 -- 172.21.15.38:0/462661833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 msgr2=0x5603a5cf7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.718+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/462661833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cf7630 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8af40c4500 tx=0x7f8af4034260 comp rx=0 tx=0).stop 2024-05-21T15:39:08.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.719+0000 7f8b0fd96740 1 -- 172.21.15.38:0/462661833 shutdown_connections 2024-05-21T15:39:08.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.719+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/462661833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a594e390 0x5603a5cfead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.719+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/462661833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a592d6b0 0x5603a5cf7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.719+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/462661833 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5603a590cf60 0x5603a5d052f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.719+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/462661833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cf7630 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.719+0000 7f8ae9ffb640 0 client.16713 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:08.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.719+0000 7f8b0fd96740 1 -- 172.21.15.38:0/462661833 >> 172.21.15.38:0/462661833 conn(0x5603a59424e0 msgr2=0x5603a5ce8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.719+0000 7f8b0fd96740 1 -- 172.21.15.38:0/462661833 shutdown_connections 2024-05-21T15:39:08.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.719+0000 7f8b0fd96740 1 -- 172.21.15.38:0/462661833 wait complete. 2024-05-21T15:39:08.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.719+0000 7f8b0fd96740 20 client.16713 trim_cache size 0 max 16384 2024-05-21T15:39:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.736+0000 7f8b0fd96740 1 Processor -- start 2024-05-21T15:39:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.736+0000 7f8b0fd96740 1 -- start start 2024-05-21T15:39:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8b0fd96740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a590cf60 0x5603a5cf2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8b0fd96740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cf7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8b0fd96740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a592d6b0 0x5603a5cec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8b0fd96740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8b041aa450 con 0x5603a59254d0 2024-05-21T15:39:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8b0fd96740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b041ac080 con 0x5603a590cf60 2024-05-21T15:39:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8b0fd96740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8b041aab20 con 0x5603a592d6b0 2024-05-21T15:39:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8af9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cf7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8af9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cf7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41286/0 (socket says 172.21.15.38:41286) 2024-05-21T15:39:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8b02d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a592d6b0 0x5603a5cec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8af9d9b640 1 -- 172.21.15.38:0/1841848900 learned_addr learned my addr 172.21.15.38:0/1841848900 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8b0259c640 1 --2- 172.21.15.38:0/1841848900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a590cf60 0x5603a5cf2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8af9d9b640 1 -- 172.21.15.38:0/1841848900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a592d6b0 msgr2=0x5603a5cec330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8af9d9b640 1 --2- 172.21.15.38:0/1841848900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a592d6b0 0x5603a5cec330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8af9d9b640 1 -- 172.21.15.38:0/1841848900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a590cf60 msgr2=0x5603a5cf2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8af9d9b640 1 --2- 172.21.15.38:0/1841848900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a590cf60 0x5603a5cf2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8af9d9b640 1 -- 172.21.15.38:0/1841848900 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a5ce9e10 con 0x5603a59254d0 2024-05-21T15:39:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.737+0000 7f8b0259c640 1 --2- 172.21.15.38:0/1841848900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a590cf60 0x5603a5cf2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.738+0000 7f8af9d9b640 1 --2- 172.21.15.38:0/1841848900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cf7910 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x5603a5cea2c0 tx=0x7f8aec00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce586088df121f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.738+0000 7f8aebfff640 1 -- 172.21.15.38:0/1841848900 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec005020 con 0x5603a59254d0 2024-05-21T15:39:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.738+0000 7f8aebfff640 1 -- 172.21.15.38:0/1841848900 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aec023030 con 0x5603a59254d0 2024-05-21T15:39:08.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.738+0000 7f8aebfff640 1 -- 172.21.15.38:0/1841848900 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec01f350 con 0x5603a59254d0 2024-05-21T15:39:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.739+0000 7f8b0fd96740 1 -- 172.21.15.38:0/1841848900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 msgr2=0x5603a5cf7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.739+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/1841848900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cf7910 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x5603a5cea2c0 tx=0x7f8aec00b390 comp rx=0 tx=0).stop 2024-05-21T15:39:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.739+0000 7f8b0fd96740 1 -- 172.21.15.38:0/1841848900 shutdown_connections 2024-05-21T15:39:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.739+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/1841848900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a592d6b0 0x5603a5cec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.739+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/1841848900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cf7910 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.739+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/1841848900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a590cf60 0x5603a5cf2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.739+0000 7f8b0fd96740 1 -- 172.21.15.38:0/1841848900 >> 172.21.15.38:0/1841848900 conn(0x5603a59424e0 msgr2=0x5603a5cf7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.739+0000 7f8b0fd96740 1 -- 172.21.15.38:0/1841848900 shutdown_connections 2024-05-21T15:39:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.739+0000 7f8b0fd96740 1 -- 172.21.15.38:0/1841848900 wait complete. 2024-05-21T15:39:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.739+0000 7f8b0fd96740 1 -- start start 2024-05-21T15:39:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8b0fd96740 1 Processor -- start 2024-05-21T15:39:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8b0fd96740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a590cf60 0x5603a5cfad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8b0fd96740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cfb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8b0fd96740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a592d6b0 0x5603a5d6b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8b0fd96740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8b041ab8f0 con 0x5603a59254d0 2024-05-21T15:39:08.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8b0fd96740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b041ab1f0 con 0x5603a592d6b0 2024-05-21T15:39:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8b0fd96740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8b041a8430 con 0x5603a590cf60 2024-05-21T15:39:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8b0fd96740 10 client.0 _ll_register_callbacks cb 0x7f8aec004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8af9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cfb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8af9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cfb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41298/0 (socket says 172.21.15.38:41298) 2024-05-21T15:39:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8af9d9b640 1 -- 172.21.15.38:0/397258079 learned_addr learned my addr 172.21.15.38:0/397258079 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8b02d9d640 1 --2- 172.21.15.38:0/397258079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a592d6b0 0x5603a5d6b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8b0259c640 1 --2- 172.21.15.38:0/397258079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a590cf60 0x5603a5cfad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8af9d9b640 1 -- 172.21.15.38:0/397258079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a590cf60 msgr2=0x5603a5cfad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8af9d9b640 1 --2- 172.21.15.38:0/397258079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a590cf60 0x5603a5cfad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8af9d9b640 1 -- 172.21.15.38:0/397258079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a592d6b0 msgr2=0x5603a5d6b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8af9d9b640 1 --2- 172.21.15.38:0/397258079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a592d6b0 0x5603a5d6b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8af9d9b640 1 -- 172.21.15.38:0/397258079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603a5cfe220 con 0x5603a59254d0 2024-05-21T15:39:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.740+0000 7f8b02d9d640 1 --2- 172.21.15.38:0/397258079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a592d6b0 0x5603a5d6b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:08.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.741+0000 7f8af9d9b640 1 --2- 172.21.15.38:0/397258079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cfb260 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8aec005d50 tx=0x7f8aec00b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9784995626cdfd78 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.741+0000 7f8ae9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.741+0000 7f8ae9ffb640 1 -- 172.21.15.38:0/397258079 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec02c070 con 0x5603a59254d0 2024-05-21T15:39:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.741+0000 7f8ae9ffb640 1 -- 172.21.15.38:0/397258079 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8aec004730 con 0x5603a59254d0 2024-05-21T15:39:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.741+0000 7f8b0fd96740 1 -- 172.21.15.38:0/397258079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aec023030 con 0x5603a59254d0 2024-05-21T15:39:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.741+0000 7f8ae9ffb640 1 -- 172.21.15.38:0/397258079 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8aec038050 con 0x5603a59254d0 2024-05-21T15:39:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.741+0000 7f8b0fd96740 10 client.16719 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.741+0000 7f8b0fd96740 1 -- 172.21.15.38:0/397258079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5603a5d6ca30 con 0x5603a59254d0 2024-05-21T15:39:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.741+0000 7f8ae9ffb640 1 -- 172.21.15.38:0/397258079 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aec023030 con 0x5603a59254d0 2024-05-21T15:39:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.742+0000 7f8ae9ffb640 1 -- 172.21.15.38:0/397258079 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f8aec028580 con 0x5603a59254d0 2024-05-21T15:39:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.742+0000 7f8b0fd96740 10 client.16719 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.742+0000 7f8b0fd96740 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.742+0000 7f8b0fd96740 20 client.16719 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.742+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/397258079 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5603a594e390 0x5603a5d6f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:08.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.742+0000 7f8b0fd96740 4 client.16719 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:39:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.742+0000 7f8b0fd96740 1 -- 172.21.15.38:0/397258079 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603a5cf18f0 con 0x5603a594e390 2024-05-21T15:39:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.742+0000 7f8b0259c640 1 --2- 172.21.15.38:0/397258079 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5603a594e390 0x5603a5d6f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8ae9ffb640 10 client.16719 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8b0259c640 1 --2- 172.21.15.38:0/397258079 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5603a594e390 0x5603a5d6f4c0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c81964f2d619761 server_cookie=ff925cc2c5ca71a8 in_seq=0 out_seq=0 2024-05-21T15:39:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8ae9ffb640 1 -- 172.21.15.38:0/397258079 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603a5cf18f0 con 0x5603a594e390 2024-05-21T15:39:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8ae9ffb640 10 client.16719 handle_command_reply: tid=0 2024-05-21T15:39:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8b0fd96740 1 client.16719 shutdown 2024-05-21T15:39:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8b0fd96740 1 -- 172.21.15.38:0/397258079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 msgr2=0x5603a5cfb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/397258079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cfb260 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8aec005d50 tx=0x7f8aec00b9a0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8b0fd96740 1 -- 172.21.15.38:0/397258079 shutdown_connections 2024-05-21T15:39:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/397258079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603a592d6b0 0x5603a5d6b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/397258079 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5603a594e390 0x5603a5d6f4c0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/397258079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5603a59254d0 0x5603a5cfb260 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8ae9ffb640 0 client.16719 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8b0fd96740 1 --2- 172.21.15.38:0/397258079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5603a590cf60 0x5603a5cfad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8b0fd96740 1 -- 172.21.15.38:0/397258079 >> 172.21.15.38:0/397258079 conn(0x5603a59424e0 msgr2=0x5603a5ce93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8b0fd96740 1 -- 172.21.15.38:0/397258079 shutdown_connections 2024-05-21T15:39:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8b0fd96740 1 -- 172.21.15.38:0/397258079 wait complete. 2024-05-21T15:39:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.743+0000 7f8b0fd96740 20 client.16719 trim_cache size 0 max 16384 2024-05-21T15:39:08.747 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:39:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.744+0000 7f8b08ef0640 1 -- 172.21.15.38:0/183864559 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8af00a2b90 msgr2=0x7f8af00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.744+0000 7f8b08ef0640 1 --2- 172.21.15.38:0/183864559 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8af00a2b90 0x7f8af00a50a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009fd0 tx=0x7f8afc004ab0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.744+0000 7f8b08ef0640 1 -- 172.21.15.38:0/183864559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b04102ae0 msgr2=0x7f8b041992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.744+0000 7f8b08ef0640 1 --2- 172.21.15.38:0/183864559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b04102ae0 0x7f8b041992a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8af4006010 tx=0x7f8af40043b0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.744+0000 7f8b08ef0640 1 -- 172.21.15.38:0/183864559 shutdown_connections 2024-05-21T15:39:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.744+0000 7f8b08ef0640 1 --2- 172.21.15.38:0/183864559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04108a20 0x7f8b041a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.744+0000 7f8b08ef0640 1 --2- 172.21.15.38:0/183864559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b04103d20 0x7f8b041997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.744+0000 7f8b08ef0640 1 --2- 172.21.15.38:0/183864559 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8af00a2b90 0x7f8af00a50a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.744+0000 7f8b08ef0640 1 --2- 172.21.15.38:0/183864559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b04102ae0 0x7f8b041992a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.744+0000 7f8b08ef0640 1 -- 172.21.15.38:0/183864559 >> 172.21.15.38:0/183864559 conn(0x7f8b040fdcf0 msgr2=0x7f8b04104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.744+0000 7f8b08ef0640 1 -- 172.21.15.38:0/183864559 shutdown_connections 2024-05-21T15:39:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:08.744+0000 7f8b08ef0640 1 -- 172.21.15.38:0/183864559 wait complete. 2024-05-21T15:39:09.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:08 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/710758557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:39:09.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:08 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/710758557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:39:09.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:08 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/710758557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:39:09.758 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:39:09.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:39:09.758 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:39:09.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.974+0000 7fe7662b8640 1 -- 172.21.15.38:0/878178683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7601078c0 msgr2=0x7fe760109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:09.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.974+0000 7fe7662b8640 1 --2- 172.21.15.38:0/878178683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7601078c0 0x7fe760109ce0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe750009900 tx=0x7fe75002a400 comp rx=0 tx=0).stop 2024-05-21T15:39:09.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.981+0000 7fe7662b8640 1 -- 172.21.15.38:0/878178683 shutdown_connections 2024-05-21T15:39:09.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.981+0000 7fe7662b8640 1 --2- 172.21.15.38:0/878178683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7601078c0 0x7fe760109ce0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:09.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.981+0000 7fe7662b8640 1 --2- 172.21.15.38:0/878178683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe760102bc0 0x7fe760102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:09.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.981+0000 7fe7662b8640 1 --2- 172.21.15.38:0/878178683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe760101980 0x7fe760102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:09.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.981+0000 7fe7662b8640 1 -- 172.21.15.38:0/878178683 >> 172.21.15.38:0/878178683 conn(0x7fe7600fcb70 msgr2=0x7fe7600ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.987+0000 7fe7662b8640 1 -- 172.21.15.38:0/878178683 shutdown_connections 2024-05-21T15:39:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.987+0000 7fe7662b8640 1 -- 172.21.15.38:0/878178683 wait complete. 2024-05-21T15:39:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.988+0000 7fe7662b8640 1 Processor -- start 2024-05-21T15:39:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.989+0000 7fe7662b8640 1 -- start start 2024-05-21T15:39:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.989+0000 7fe7662b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe760101980 0x7fe760112330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.989+0000 7fe7662b8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe760102bc0 0x7fe760112870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.989+0000 7fe7662b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7601078c0 0x7fe760119800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.989+0000 7fe7662b8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe76010c540 con 0x7fe760101980 2024-05-21T15:39:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.989+0000 7fe7662b8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe76010c3c0 con 0x7fe760102bc0 2024-05-21T15:39:09.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.989+0000 7fe7662b8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe76010c6c0 con 0x7fe7601078c0 2024-05-21T15:39:09.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.990+0000 7fe75ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe760101980 0x7fe760112330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:09.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.990+0000 7fe75ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe760101980 0x7fe760112330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41328/0 (socket says 172.21.15.38:41328) 2024-05-21T15:39:09.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.990+0000 7fe75ffff640 1 -- 172.21.15.38:0/2580876567 learned_addr learned my addr 172.21.15.38:0/2580876567 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:09.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.991+0000 7fe75ffff640 1 -- 172.21.15.38:0/2580876567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7601078c0 msgr2=0x7fe760119800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:09.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.991+0000 7fe75ffff640 1 --2- 172.21.15.38:0/2580876567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7601078c0 0x7fe760119800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:09.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.991+0000 7fe75ffff640 1 -- 172.21.15.38:0/2580876567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe760102bc0 msgr2=0x7fe760112870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:39:09.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.991+0000 7fe75ffff640 1 --2- 172.21.15.38:0/2580876567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe760102bc0 0x7fe760112870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:09.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.991+0000 7fe75ffff640 1 -- 172.21.15.38:0/2580876567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe760119e70 con 0x7fe760101980 2024-05-21T15:39:09.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.991+0000 7fe75ffff640 1 --2- 172.21.15.38:0/2580876567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe760101980 0x7fe760112330 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe75400d740 tx=0x7fe75400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:09.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.991+0000 7fe75d7fa640 1 -- 172.21.15.38:0/2580876567 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754017070 con 0x7fe760101980 2024-05-21T15:39:09.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.991+0000 7fe7662b8640 1 -- 172.21.15.38:0/2580876567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76019efb0 con 0x7fe760101980 2024-05-21T15:39:09.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.992+0000 7fe7662b8640 1 -- 172.21.15.38:0/2580876567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe76019f580 con 0x7fe760101980 2024-05-21T15:39:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.993+0000 7fe75d7fa640 1 -- 172.21.15.38:0/2580876567 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe75400de50 con 0x7fe760101980 2024-05-21T15:39:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.993+0000 7fe75d7fa640 1 -- 172.21.15.38:0/2580876567 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754003d60 con 0x7fe760101980 2024-05-21T15:39:09.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.997+0000 7fe75d7fa640 1 -- 172.21.15.38:0/2580876567 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe754004020 con 0x7fe760101980 2024-05-21T15:39:09.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.998+0000 7fe75d7fa640 1 --2- 172.21.15.38:0/2580876567 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7480a2d90 0x7fe7480a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:09.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.999+0000 7fe76d235740 1 Processor -- start 2024-05-21T15:39:09.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.999+0000 7fe75d7fa640 1 -- 172.21.15.38:0/2580876567 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe7540cea90 con 0x7fe760101980 2024-05-21T15:39:09.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.999+0000 7fe76d235740 1 -- start start 2024-05-21T15:39:09.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.999+0000 7fe76d235740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a70410 0x55feb8e104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:09.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.999+0000 7fe76d235740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 0x55feb8e10a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:09.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:09.999+0000 7fe76d235740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a4f730 0x55feb8e17920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:09.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.000+0000 7fe76d235740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55feb8d74990 con 0x55feb8a47550 2024-05-21T15:39:09.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.000+0000 7fe76d235740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55feb8d1c5b0 con 0x55feb8a70410 2024-05-21T15:39:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.000+0000 7fe764855640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a4f730 0x55feb8e17920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.000+0000 7fe764855640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a4f730 0x55feb8e17920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:32808/0 (socket says 172.21.15.38:32808) 2024-05-21T15:39:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.000+0000 7fe764855640 1 -- 172.21.15.38:0/347792601 learned_addr learned my addr 172.21.15.38:0/347792601 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.000+0000 7fe76d235740 1 -- 172.21.15.38:0/347792601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55feb8d70950 con 0x55feb8a4f730 2024-05-21T15:39:10.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.004+0000 7fe75ffff640 1 --2- 172.21.15.38:0/347792601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a70410 0x55feb8e104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.004+0000 7fe764855640 1 -- 172.21.15.38:0/347792601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a70410 msgr2=0x55feb8e104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:10.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.004+0000 7fe764855640 1 --2- 172.21.15.38:0/347792601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a70410 0x55feb8e104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.004+0000 7fe764855640 1 -- 172.21.15.38:0/347792601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 msgr2=0x55feb8e10a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:10.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.004+0000 7fe764855640 1 --2- 172.21.15.38:0/347792601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 0x55feb8e10a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.004+0000 7fe764855640 1 -- 172.21.15.38:0/347792601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feb8b507e0 con 0x55feb8a4f730 2024-05-21T15:39:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.005+0000 7fe75f7fe640 1 --2- 172.21.15.38:0/2580876567 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7480a2d90 0x7fe7480a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.005+0000 7fe764855640 1 --2- 172.21.15.38:0/347792601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a4f730 0x55feb8e17920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe750009850 tx=0x7fe750005f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9d2b6758a49aa43 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:10.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:09 smithi045 ceph-mon[36409]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 58 MiB/s wr, 12.54k op/s 2024-05-21T15:39:10.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.011+0000 7fe7467fc640 1 -- 172.21.15.38:0/347792601 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75003e070 con 0x55feb8a4f730 2024-05-21T15:39:10.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.011+0000 7fe7467fc640 1 -- 172.21.15.38:0/347792601 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe750005530 con 0x55feb8a4f730 2024-05-21T15:39:10.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.011+0000 7fe7467fc640 1 -- 172.21.15.38:0/347792601 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750035040 con 0x55feb8a4f730 2024-05-21T15:39:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.012+0000 7fe76d235740 1 -- 172.21.15.38:0/347792601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a4f730 msgr2=0x55feb8e17920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.012+0000 7fe76d235740 1 --2- 172.21.15.38:0/347792601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a4f730 0x55feb8e17920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe750009850 tx=0x7fe750005f20 comp rx=0 tx=0).stop 2024-05-21T15:39:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.012+0000 7fe76d235740 1 -- 172.21.15.38:0/347792601 shutdown_connections 2024-05-21T15:39:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.012+0000 7fe76d235740 1 --2- 172.21.15.38:0/347792601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a4f730 0x55feb8e17920 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.012+0000 7fe76d235740 1 --2- 172.21.15.38:0/347792601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 0x55feb8e10a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.012+0000 7fe76d235740 1 --2- 172.21.15.38:0/347792601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a70410 0x55feb8e104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.012+0000 7fe76d235740 1 -- 172.21.15.38:0/347792601 >> 172.21.15.38:0/347792601 conn(0x55feb8a64560 msgr2=0x55feb8e0be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.016+0000 7fe75f7fe640 1 --2- 172.21.15.38:0/2580876567 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7480a2d90 0x7fe7480a52a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55feb8e11880 tx=0x7fe75800b250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.016+0000 7fe76d235740 1 -- 172.21.15.38:0/347792601 shutdown_connections 2024-05-21T15:39:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.016+0000 7fe76d235740 1 -- 172.21.15.38:0/347792601 wait complete. 2024-05-21T15:39:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.016+0000 7fe76d235740 1 -- start start 2024-05-21T15:39:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.017+0000 7fe76d235740 1 Processor -- start 2024-05-21T15:39:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.017+0000 7fe76d235740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 0x55feb8e1a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.017+0000 7fe76d235740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a4f730 0x55feb8e1a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.017+0000 7fe76d235740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a70410 0x55feb8e21810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.017+0000 7fe76d235740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55feb8e18310 con 0x55feb8a47550 2024-05-21T15:39:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.017+0000 7fe76d235740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55feb8d2fb40 con 0x55feb8a4f730 2024-05-21T15:39:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.017+0000 7fe76d235740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55feb8e18490 con 0x55feb8a70410 2024-05-21T15:39:10.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.017+0000 7fe76d235740 10 client.0 _ll_register_callbacks cb 0x55feb8d240f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.018+0000 7fe75f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a4f730 0x55feb8e1a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.018+0000 7fe75f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a4f730 0x55feb8e1a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:40006/0 (socket says 172.21.15.38:40006) 2024-05-21T15:39:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.018+0000 7fe75f7fe640 1 -- 172.21.15.38:0/27662510 learned_addr learned my addr 172.21.15.38:0/27662510 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.018+0000 7fe764855640 1 --2- 172.21.15.38:0/27662510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a70410 0x55feb8e21810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.018+0000 7fe75ffff640 1 --2- 172.21.15.38:0/27662510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 0x55feb8e1a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.019+0000 7fe75ffff640 1 -- 172.21.15.38:0/27662510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a70410 msgr2=0x55feb8e21810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.019+0000 7fe75ffff640 1 --2- 172.21.15.38:0/27662510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a70410 0x55feb8e21810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.019+0000 7fe75ffff640 1 -- 172.21.15.38:0/27662510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a4f730 msgr2=0x55feb8e1a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.019+0000 7fe75ffff640 1 --2- 172.21.15.38:0/27662510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a4f730 0x55feb8e1a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.019+0000 7fe75ffff640 1 -- 172.21.15.38:0/27662510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feb8bb0f70 con 0x55feb8a47550 2024-05-21T15:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.020+0000 7fe75ffff640 1 --2- 172.21.15.38:0/27662510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 0x55feb8e1a370 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe754009fd0 tx=0x7fe75409baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b487911b0e73892c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.020+0000 7fe73f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.020+0000 7fe73f7fe640 1 -- 172.21.15.38:0/27662510 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754020050 con 0x55feb8a47550 2024-05-21T15:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.020+0000 7fe76d235740 1 -- 172.21.15.38:0/27662510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe750005530 con 0x55feb8a47550 2024-05-21T15:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.021+0000 7fe73f7fe640 1 -- 172.21.15.38:0/27662510 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe7540978b0 con 0x55feb8a47550 2024-05-21T15:39:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.021+0000 7fe73f7fe640 1 -- 172.21.15.38:0/27662510 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe7540b4200 con 0x55feb8a47550 2024-05-21T15:39:10.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.022+0000 7fe76d235740 10 client.16728 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:10.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.022+0000 7fe76d235740 1 -- 172.21.15.38:0/27662510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55feb8a48170 con 0x55feb8a47550 2024-05-21T15:39:10.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.022+0000 7fe73f7fe640 1 -- 172.21.15.38:0/27662510 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe750005530 con 0x55feb8a47550 2024-05-21T15:39:10.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:09 smithi038 ceph-mon[31995]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 58 MiB/s wr, 12.54k op/s 2024-05-21T15:39:10.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.027+0000 7fe73f7fe640 1 -- 172.21.15.38:0/27662510 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fe7540a3a20 con 0x55feb8a47550 2024-05-21T15:39:10.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.027+0000 7fe76d235740 10 client.16728 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:10.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.027+0000 7fe76d235740 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:10.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.027+0000 7fe76d235740 20 client.16728 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:10.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.027+0000 7fe76d235740 1 --2- 172.21.15.38:0/27662510 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55feb8a2f1f0 0x55feb8e27d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.027+0000 7fe76d235740 4 client.16728 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:10.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.027+0000 7fe76d235740 1 -- 172.21.15.38:0/27662510 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55feb8d820c0 con 0x55feb8a2f1f0 2024-05-21T15:39:10.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.027+0000 7fe75f7fe640 1 --2- 172.21.15.38:0/27662510 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55feb8a2f1f0 0x55feb8e27d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.030+0000 7fe75f7fe640 1 --2- 172.21.15.38:0/27662510 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55feb8a2f1f0 0x55feb8e27d20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=809f5ba7b9f98465 server_cookie=352ff7096b47ff1a in_seq=0 out_seq=0 2024-05-21T15:39:10.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.031+0000 7fe73f7fe640 10 client.16728 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:10.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.032+0000 7fe73f7fe640 1 -- 172.21.15.38:0/27662510 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55feb8d820c0 con 0x55feb8a2f1f0 2024-05-21T15:39:10.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.032+0000 7fe73f7fe640 10 client.16728 handle_command_reply: tid=0 2024-05-21T15:39:10.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.032+0000 7fe76d235740 1 client.16728 shutdown 2024-05-21T15:39:10.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.033+0000 7fe76d235740 1 -- 172.21.15.38:0/27662510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 msgr2=0x55feb8e1a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:10.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.033+0000 7fe76d235740 1 --2- 172.21.15.38:0/27662510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 0x55feb8e1a370 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe754009fd0 tx=0x7fe75409baa0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.033+0000 7fe76d235740 1 -- 172.21.15.38:0/27662510 shutdown_connections 2024-05-21T15:39:10.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.033+0000 7fe76d235740 1 --2- 172.21.15.38:0/27662510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a70410 0x55feb8e21810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.033+0000 7fe76d235740 1 --2- 172.21.15.38:0/27662510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a4f730 0x55feb8e1a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.033+0000 7fe76d235740 1 --2- 172.21.15.38:0/27662510 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55feb8a2f1f0 0x55feb8e27d20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.033+0000 7fe76d235740 1 --2- 172.21.15.38:0/27662510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 0x55feb8e1a370 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.033+0000 7fe76d235740 1 -- 172.21.15.38:0/27662510 >> 172.21.15.38:0/27662510 conn(0x55feb8a64560 msgr2=0x55feb8e0ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:10.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.033+0000 7fe73f7fe640 0 client.16728 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:10.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.033+0000 7fe76d235740 1 -- 172.21.15.38:0/27662510 shutdown_connections 2024-05-21T15:39:10.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.033+0000 7fe76d235740 1 -- 172.21.15.38:0/27662510 wait complete. 2024-05-21T15:39:10.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.033+0000 7fe76d235740 20 client.16728 trim_cache size 0 max 16384 2024-05-21T15:39:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.070+0000 7fe76d235740 1 Processor -- start 2024-05-21T15:39:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.070+0000 7fe76d235740 1 -- start start 2024-05-21T15:39:10.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.070+0000 7fe76d235740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a47550 0x55feb8e0be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.070+0000 7fe76d235740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a4f730 0x55feb8e19f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.070+0000 7fe76d235740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a70410 0x55feb8e1a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.070+0000 7fe76d235740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c00e680 con 0x55feb8a70410 2024-05-21T15:39:10.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.070+0000 7fe76d235740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c0102b0 con 0x55feb8a4f730 2024-05-21T15:39:10.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.070+0000 7fe76d235740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c00ed50 con 0x55feb8a47550 2024-05-21T15:39:10.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.070+0000 7fe75ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a47550 0x55feb8e0be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.070+0000 7fe75ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a47550 0x55feb8e0be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:32830/0 (socket says 172.21.15.38:32830) 2024-05-21T15:39:10.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.070+0000 7fe75ffff640 1 -- 172.21.15.38:0/2154455926 learned_addr learned my addr 172.21.15.38:0/2154455926 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:10.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.070+0000 7fe764855640 1 --2- 172.21.15.38:0/2154455926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a70410 0x55feb8e1a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.071+0000 7fe75ffff640 1 -- 172.21.15.38:0/2154455926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a4f730 msgr2=0x55feb8e19f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:39:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.071+0000 7fe75ffff640 1 --2- 172.21.15.38:0/2154455926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a4f730 0x55feb8e19f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.071+0000 7fe75ffff640 1 -- 172.21.15.38:0/2154455926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a70410 msgr2=0x55feb8e1a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.071+0000 7fe75ffff640 1 --2- 172.21.15.38:0/2154455926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a70410 0x55feb8e1a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.071+0000 7fe75ffff640 1 -- 172.21.15.38:0/2154455926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feb8e06e90 con 0x55feb8a47550 2024-05-21T15:39:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.072+0000 7fe75ffff640 1 --2- 172.21.15.38:0/2154455926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a47550 0x55feb8e0be60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe7540a3980 tx=0x7fe75409b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97e531e030c8b3bf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.073+0000 7fe745ffb640 1 -- 172.21.15.38:0/2154455926 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754020070 con 0x55feb8a47550 2024-05-21T15:39:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.073+0000 7fe745ffb640 1 -- 172.21.15.38:0/2154455926 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe754009dc0 con 0x55feb8a47550 2024-05-21T15:39:10.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.074+0000 7fe745ffb640 1 -- 172.21.15.38:0/2154455926 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75409a690 con 0x55feb8a47550 2024-05-21T15:39:10.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.075+0000 7fe76d235740 1 -- 172.21.15.38:0/2154455926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a47550 msgr2=0x55feb8e0be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:10.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.075+0000 7fe76d235740 1 --2- 172.21.15.38:0/2154455926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a47550 0x55feb8e0be60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe7540a3980 tx=0x7fe75409b940 comp rx=0 tx=0).stop 2024-05-21T15:39:10.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.077+0000 7fe76d235740 1 -- 172.21.15.38:0/2154455926 shutdown_connections 2024-05-21T15:39:10.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.077+0000 7fe76d235740 1 --2- 172.21.15.38:0/2154455926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a70410 0x55feb8e1a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.077+0000 7fe76d235740 1 --2- 172.21.15.38:0/2154455926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a4f730 0x55feb8e19f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.077+0000 7fe76d235740 1 --2- 172.21.15.38:0/2154455926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a47550 0x55feb8e0be60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.077+0000 7fe76d235740 1 -- 172.21.15.38:0/2154455926 >> 172.21.15.38:0/2154455926 conn(0x55feb8a64560 msgr2=0x55feb8e0b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:10.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.079+0000 7fe76d235740 1 -- 172.21.15.38:0/2154455926 shutdown_connections 2024-05-21T15:39:10.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.080+0000 7fe76d235740 1 -- 172.21.15.38:0/2154455926 wait complete. 2024-05-21T15:39:10.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.080+0000 7fe76d235740 1 -- start start 2024-05-21T15:39:10.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.081+0000 7fe76d235740 1 Processor -- start 2024-05-21T15:39:10.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.081+0000 7fe76d235740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 0x55feb8e1dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.081+0000 7fe76d235740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a4f730 0x55feb8e1e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.081+0000 7fe76d235740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a70410 0x55feb8e8e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.081+0000 7fe76d235740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c00fb20 con 0x55feb8a47550 2024-05-21T15:39:10.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.081+0000 7fe76d235740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c00f420 con 0x55feb8a4f730 2024-05-21T15:39:10.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.081+0000 7fe76d235740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c00c660 con 0x55feb8a70410 2024-05-21T15:39:10.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.082+0000 7fe76d235740 10 client.0 _ll_register_callbacks cb 0x7fe75409ac90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:10.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.082+0000 7fe75ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 0x55feb8e1dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.082+0000 7fe75ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 0x55feb8e1dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41372/0 (socket says 172.21.15.38:41372) 2024-05-21T15:39:10.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.082+0000 7fe75ffff640 1 -- 172.21.15.38:0/1259706889 learned_addr learned my addr 172.21.15.38:0/1259706889 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:10.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.083+0000 7fe75ffff640 1 -- 172.21.15.38:0/1259706889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a70410 msgr2=0x55feb8e8e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:10.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.083+0000 7fe764855640 1 --2- 172.21.15.38:0/1259706889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a70410 0x55feb8e8e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.083+0000 7fe75ffff640 1 --2- 172.21.15.38:0/1259706889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a70410 0x55feb8e8e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.083+0000 7fe75ffff640 1 -- 172.21.15.38:0/1259706889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a4f730 msgr2=0x55feb8e1e160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:39:10.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.083+0000 7fe75ffff640 1 --2- 172.21.15.38:0/1259706889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a4f730 0x55feb8e1e160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.083+0000 7fe75ffff640 1 -- 172.21.15.38:0/1259706889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feb8e21120 con 0x55feb8a47550 2024-05-21T15:39:10.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.084+0000 7fe75ffff640 1 --2- 172.21.15.38:0/1259706889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 0x55feb8e1dc20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe75409af40 tx=0x7fe754009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43a4d9fce95c440 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:10.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.084+0000 7fe73f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:10.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.084+0000 7fe73f7fe640 1 -- 172.21.15.38:0/1259706889 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754020070 con 0x55feb8a47550 2024-05-21T15:39:10.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.084+0000 7fe76d235740 1 -- 172.21.15.38:0/1259706889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe754009dc0 con 0x55feb8a47550 2024-05-21T15:39:10.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.084+0000 7fe73f7fe640 1 -- 172.21.15.38:0/1259706889 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe75409a8a0 con 0x55feb8a47550 2024-05-21T15:39:10.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.085+0000 7fe73f7fe640 1 -- 172.21.15.38:0/1259706889 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe7540035f0 con 0x55feb8a47550 2024-05-21T15:39:10.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.088+0000 7fe76d235740 10 client.16734 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:10.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.088+0000 7fe76d235740 1 -- 172.21.15.38:0/1259706889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55feb8ea05e0 con 0x55feb8a47550 2024-05-21T15:39:10.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.089+0000 7fe73f7fe640 1 -- 172.21.15.38:0/1259706889 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe754009dc0 con 0x55feb8a47550 2024-05-21T15:39:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.089+0000 7fe73f7fe640 1 -- 172.21.15.38:0/1259706889 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fe7540a9070 con 0x55feb8a47550 2024-05-21T15:39:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.089+0000 7fe76d235740 10 client.16734 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.089+0000 7fe76d235740 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.089+0000 7fe76d235740 20 client.16734 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.089+0000 7fe76d235740 1 --2- 172.21.15.38:0/1259706889 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55feb8a04630 0x55feb8ea3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.089+0000 7fe76d235740 4 client.16734 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:39:10.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.089+0000 7fe76d235740 1 -- 172.21.15.38:0/1259706889 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55feb8d7c410 con 0x55feb8a04630 2024-05-21T15:39:10.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.092+0000 7fe75f7fe640 1 --2- 172.21.15.38:0/1259706889 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55feb8a04630 0x55feb8ea3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.093+0000 7fe75f7fe640 1 --2- 172.21.15.38:0/1259706889 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55feb8a04630 0x55feb8ea3000 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6118ae363249a973 server_cookie=805541903933f2ce in_seq=0 out_seq=0 2024-05-21T15:39:10.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.093+0000 7fe73f7fe640 10 client.16734 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.096+0000 7fe73f7fe640 1 -- 172.21.15.38:0/1259706889 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55feb8d7c410 con 0x55feb8a04630 2024-05-21T15:39:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.096+0000 7fe73f7fe640 10 client.16734 handle_command_reply: tid=0 2024-05-21T15:39:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.097+0000 7fe76d235740 1 client.16734 shutdown 2024-05-21T15:39:10.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.097+0000 7fe76d235740 1 -- 172.21.15.38:0/1259706889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 msgr2=0x55feb8e1dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:10.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.097+0000 7fe76d235740 1 --2- 172.21.15.38:0/1259706889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 0x55feb8e1dc20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe75409af40 tx=0x7fe754009cb0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.100+0000 7fe76d235740 1 -- 172.21.15.38:0/1259706889 shutdown_connections 2024-05-21T15:39:10.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.100+0000 7fe76d235740 1 --2- 172.21.15.38:0/1259706889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55feb8a70410 0x55feb8e8e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.100+0000 7fe76d235740 1 --2- 172.21.15.38:0/1259706889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55feb8a4f730 0x55feb8e1e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.100+0000 7fe76d235740 1 --2- 172.21.15.38:0/1259706889 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55feb8a04630 0x55feb8ea3000 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.100+0000 7fe76d235740 1 --2- 172.21.15.38:0/1259706889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55feb8a47550 0x55feb8e1dc20 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.100+0000 7fe76d235740 1 -- 172.21.15.38:0/1259706889 >> 172.21.15.38:0/1259706889 conn(0x55feb8a64560 msgr2=0x55feb8e189b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:10.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.101+0000 7fe73f7fe640 0 client.16734 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:10.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.101+0000 7fe76d235740 1 -- 172.21.15.38:0/1259706889 shutdown_connections 2024-05-21T15:39:10.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.101+0000 7fe76d235740 1 -- 172.21.15.38:0/1259706889 wait complete. 2024-05-21T15:39:10.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.102+0000 7fe76d235740 20 client.16734 trim_cache size 0 max 16384 2024-05-21T15:39:10.102 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:39:10.102 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-05-21T15:39:10.103 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "bce6877c-ded0-48cd-90be-bd40d9b63b34", 2024-05-21T15:39:10.103 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-05-21T15:39:10.103 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:39:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.105+0000 7fe7662b8640 1 -- 172.21.15.38:0/2580876567 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7480a2d90 msgr2=0x7fe7480a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.105+0000 7fe7662b8640 1 --2- 172.21.15.38:0/2580876567 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7480a2d90 0x7fe7480a52a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55feb8e11880 tx=0x7fe75800b250 comp rx=0 tx=0).stop 2024-05-21T15:39:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.105+0000 7fe7662b8640 1 -- 172.21.15.38:0/2580876567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe760101980 msgr2=0x7fe760112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.105+0000 7fe7662b8640 1 --2- 172.21.15.38:0/2580876567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe760101980 0x7fe760112330 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe75400d740 tx=0x7fe75400dc00 comp rx=0 tx=0).stop 2024-05-21T15:39:10.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.105+0000 7fe7662b8640 1 -- 172.21.15.38:0/2580876567 shutdown_connections 2024-05-21T15:39:10.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.105+0000 7fe7662b8640 1 --2- 172.21.15.38:0/2580876567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7601078c0 0x7fe760119800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.105+0000 7fe7662b8640 1 --2- 172.21.15.38:0/2580876567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe760102bc0 0x7fe760112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.105+0000 7fe7662b8640 1 --2- 172.21.15.38:0/2580876567 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7480a2d90 0x7fe7480a52a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.106+0000 7fe7662b8640 1 --2- 172.21.15.38:0/2580876567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe760101980 0x7fe760112330 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.106+0000 7fe7662b8640 1 -- 172.21.15.38:0/2580876567 >> 172.21.15.38:0/2580876567 conn(0x7fe7600fcb70 msgr2=0x7fe7601038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:10.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.106+0000 7fe7662b8640 1 -- 172.21.15.38:0/2580876567 shutdown_connections 2024-05-21T15:39:10.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.106+0000 7fe7662b8640 1 -- 172.21.15.38:0/2580876567 wait complete. 2024-05-21T15:39:10.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:39:10.127 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:39:10.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:09 smithi053 ceph-mon[36496]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 58 MiB/s wr, 12.54k op/s 2024-05-21T15:39:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.328+0000 7f9b5a029640 1 -- 172.21.15.38:0/3471259909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b4c096b20 msgr2=0x7f9b4c096f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.328+0000 7f9b5a029640 1 --2- 172.21.15.38:0/3471259909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b4c096b20 0x7f9b4c096f30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000d910 tx=0x7f9b5002e5c0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.329+0000 7f9b5a029640 1 -- 172.21.15.38:0/3471259909 shutdown_connections 2024-05-21T15:39:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.329+0000 7f9b5a029640 1 --2- 172.21.15.38:0/3471259909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b4c09b820 0x7f9b4c09dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.329+0000 7f9b5a029640 1 --2- 172.21.15.38:0/3471259909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b4c096b20 0x7f9b4c096f30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.329+0000 7f9b5a029640 1 --2- 172.21.15.38:0/3471259909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b4c0958e0 0x7f9b4c0965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.329+0000 7f9b5a029640 1 -- 172.21.15.38:0/3471259909 >> 172.21.15.38:0/3471259909 conn(0x7f9b4c090ab0 msgr2=0x7f9b4c092f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:10.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.331+0000 7f9b5a029640 1 -- 172.21.15.38:0/3471259909 shutdown_connections 2024-05-21T15:39:10.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.332+0000 7f9b5a029640 1 -- 172.21.15.38:0/3471259909 wait complete. 2024-05-21T15:39:10.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.333+0000 7f9b5a029640 1 Processor -- start 2024-05-21T15:39:10.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.334+0000 7f9b5a029640 1 -- start start 2024-05-21T15:39:10.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.334+0000 7f9b5a029640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b4c0958e0 0x7f9b4c1449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.334+0000 7f9b5a029640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b4c096b20 0x7f9b4c144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.334+0000 7f9b5a029640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b4c09b820 0x7f9b4c14bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.334+0000 7f9b5a029640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c0a0540 con 0x7f9b4c096b20 2024-05-21T15:39:10.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.334+0000 7f9b5a029640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c0a03c0 con 0x7f9b4c09b820 2024-05-21T15:39:10.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.334+0000 7f9b5a029640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c0a06c0 con 0x7f9b4c0958e0 2024-05-21T15:39:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.335+0000 7f9b59828640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b4c09b820 0x7f9b4c14bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.335+0000 7f9b59828640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b4c09b820 0x7f9b4c14bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:40028/0 (socket says 172.21.15.38:40028) 2024-05-21T15:39:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.335+0000 7f9b59828640 1 -- 172.21.15.38:0/1397986477 learned_addr learned my addr 172.21.15.38:0/1397986477 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.335+0000 7f9b59828640 1 -- 172.21.15.38:0/1397986477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b4c0958e0 msgr2=0x7f9b4c1449f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T15:39:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.335+0000 7f9b59828640 1 --2- 172.21.15.38:0/1397986477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b4c0958e0 0x7f9b4c1449f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.335+0000 7f9b59828640 1 -- 172.21.15.38:0/1397986477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b4c096b20 msgr2=0x7f9b4c144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.335+0000 7f9b59828640 1 --2- 172.21.15.38:0/1397986477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b4c096b20 0x7f9b4c144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.335+0000 7f9b59828640 1 -- 172.21.15.38:0/1397986477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c14c530 con 0x7f9b4c09b820 2024-05-21T15:39:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.335+0000 7f9b59828640 1 --2- 172.21.15.38:0/1397986477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b4c09b820 0x7f9b4c14bec0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9b540501a0 tx=0x7f9b54070570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.337+0000 7f9b4a7fc640 1 -- 172.21.15.38:0/1397986477 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54070e30 con 0x7f9b4c09b820 2024-05-21T15:39:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.338+0000 7f9b5a029640 1 -- 172.21.15.38:0/1397986477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4c14c820 con 0x7f9b4c09b820 2024-05-21T15:39:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.338+0000 7f9b5a029640 1 -- 172.21.15.38:0/1397986477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b4c14cd20 con 0x7f9b4c09b820 2024-05-21T15:39:10.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.342+0000 7f9b6220d740 1 Processor -- start 2024-05-21T15:39:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.343+0000 7f9b6220d740 1 -- start start 2024-05-21T15:39:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.344+0000 7f9b6220d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9fbe390 0x55faba35ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.344+0000 7f9b6220d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f954d0 0x55faba35e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.344+0000 7f9b6220d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f9d6b0 0x55faba365230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.344+0000 7f9b6220d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55faba2c2d00 con 0x55fab9f9d6b0 2024-05-21T15:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.344+0000 7f9b6220d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55faba26a4c0 con 0x55fab9fbe390 2024-05-21T15:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.344+0000 7f9b6220d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55faba2becc0 con 0x55fab9f954d0 2024-05-21T15:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.345+0000 7f9b59027640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f954d0 0x55faba35e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.345+0000 7f9b59027640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f954d0 0x55faba35e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:32860/0 (socket says 172.21.15.38:32860) 2024-05-21T15:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.345+0000 7f9b59027640 1 -- 172.21.15.38:0/2301116810 learned_addr learned my addr 172.21.15.38:0/2301116810 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.345+0000 7f9b4a7fc640 1 -- 172.21.15.38:0/1397986477 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b54075a70 con 0x7f9b4c09b820 2024-05-21T15:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.346+0000 7f9b58826640 1 --2- 172.21.15.38:0/2301116810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f9d6b0 0x55faba365230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.346+0000 7f9b4a7fc640 1 -- 172.21.15.38:0/1397986477 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54076b60 con 0x7f9b4c09b820 2024-05-21T15:39:10.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.347+0000 7f9b58826640 1 --2- 172.21.15.38:0/2301116810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9fbe390 0x55faba35ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.347+0000 7f9b59027640 1 -- 172.21.15.38:0/2301116810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9fbe390 msgr2=0x55faba35ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:10.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.348+0000 7f9b59027640 1 --2- 172.21.15.38:0/2301116810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9fbe390 0x55faba35ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.348+0000 7f9b59027640 1 -- 172.21.15.38:0/2301116810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f9d6b0 msgr2=0x55faba365230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:10.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.348+0000 7f9b59027640 1 --2- 172.21.15.38:0/2301116810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f9d6b0 0x55faba365230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.348+0000 7f9b59027640 1 -- 172.21.15.38:0/2301116810 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faba0a07d0 con 0x55fab9f954d0 2024-05-21T15:39:10.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.348+0000 7f9b58826640 1 --2- 172.21.15.38:0/2301116810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9fbe390 0x55faba35ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:39:10.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.349+0000 7f9b4a7fc640 1 -- 172.21.15.38:0/1397986477 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9b5407d030 con 0x7f9b4c09b820 2024-05-21T15:39:10.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.350+0000 7f9b59027640 1 --2- 172.21.15.38:0/2301116810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f954d0 0x55faba35e330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9b440096f0 tx=0x7f9b44009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0940eeace98d12 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:10.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.351+0000 7f9b4a7fc640 1 --2- 172.21.15.38:0/1397986477 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9b400a2c90 0x7f9b400a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.351+0000 7f9b3f7fe640 1 -- 172.21.15.38:0/2301116810 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44015020 con 0x55fab9f954d0 2024-05-21T15:39:10.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.351+0000 7f9b3f7fe640 1 -- 172.21.15.38:0/2301116810 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b4400bdb0 con 0x55fab9f954d0 2024-05-21T15:39:10.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.351+0000 7f9b3f7fe640 1 -- 172.21.15.38:0/2301116810 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b440103a0 con 0x55fab9f954d0 2024-05-21T15:39:10.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.351+0000 7f9b4a7fc640 1 -- 172.21.15.38:0/1397986477 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9b54131bf0 con 0x7f9b4c09b820 2024-05-21T15:39:10.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.352+0000 7f9b58826640 1 --2- 172.21.15.38:0/1397986477 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9b400a2c90 0x7f9b400a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.353+0000 7f9b6220d740 1 -- 172.21.15.38:0/2301116810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f954d0 msgr2=0x55faba35e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:10.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.353+0000 7f9b6220d740 1 --2- 172.21.15.38:0/2301116810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f954d0 0x55faba35e330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9b440096f0 tx=0x7f9b44009bb0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.353+0000 7f9b58826640 1 --2- 172.21.15.38:0/1397986477 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9b400a2c90 0x7f9b400a51a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9b50004e60 tx=0x7f9b50004df0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:10.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.354+0000 7f9b6220d740 1 -- 172.21.15.38:0/2301116810 shutdown_connections 2024-05-21T15:39:10.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.354+0000 7f9b6220d740 1 --2- 172.21.15.38:0/2301116810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f9d6b0 0x55faba365230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.354+0000 7f9b6220d740 1 --2- 172.21.15.38:0/2301116810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f954d0 0x55faba35e330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.354+0000 7f9b6220d740 1 --2- 172.21.15.38:0/2301116810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9fbe390 0x55faba35ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.354+0000 7f9b6220d740 1 -- 172.21.15.38:0/2301116810 >> 172.21.15.38:0/2301116810 conn(0x55fab9fb24e0 msgr2=0x55faba359740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:10.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.354+0000 7f9b6220d740 1 -- 172.21.15.38:0/2301116810 shutdown_connections 2024-05-21T15:39:10.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.354+0000 7f9b6220d740 1 -- 172.21.15.38:0/2301116810 wait complete. 2024-05-21T15:39:10.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.355+0000 7f9b6220d740 1 -- start start 2024-05-21T15:39:10.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.355+0000 7f9b6220d740 1 Processor -- start 2024-05-21T15:39:10.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.356+0000 7f9b6220d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f954d0 0x55faba367d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.356+0000 7f9b6220d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f9d6b0 0x55faba3682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.356+0000 7f9b6220d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9fbe390 0x55faba36f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.356+0000 7f9b6220d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55faba365c80 con 0x55fab9f9d6b0 2024-05-21T15:39:10.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.357+0000 7f9b6220d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55faba27db90 con 0x55fab9f954d0 2024-05-21T15:39:10.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.357+0000 7f9b6220d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55faba365e00 con 0x55fab9fbe390 2024-05-21T15:39:10.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.357+0000 7f9b59027640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f954d0 0x55faba367d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.357+0000 7f9b6220d740 10 client.0 _ll_register_callbacks cb 0x55faba2b9da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:10.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.357+0000 7f9b59027640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f954d0 0x55faba367d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:40056/0 (socket says 172.21.15.38:40056) 2024-05-21T15:39:10.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.357+0000 7f9b59027640 1 -- 172.21.15.38:0/3400786210 learned_addr learned my addr 172.21.15.38:0/3400786210 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:10.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.357+0000 7f9b59828640 1 --2- 172.21.15.38:0/3400786210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9fbe390 0x55faba36f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.357+0000 7f9b58826640 1 --2- 172.21.15.38:0/3400786210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f9d6b0 0x55faba3682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.358+0000 7f9b59828640 1 -- 172.21.15.38:0/3400786210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f954d0 msgr2=0x55faba367d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:10.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.358+0000 7f9b59828640 1 --2- 172.21.15.38:0/3400786210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f954d0 0x55faba367d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.358+0000 7f9b59828640 1 -- 172.21.15.38:0/3400786210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f9d6b0 msgr2=0x55faba3682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:10.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.358+0000 7f9b59828640 1 --2- 172.21.15.38:0/3400786210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f9d6b0 0x55faba3682a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.358+0000 7f9b59828640 1 -- 172.21.15.38:0/3400786210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fab9fd63f0 con 0x55fab9fbe390 2024-05-21T15:39:10.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.359+0000 7f9b59828640 1 --2- 172.21.15.38:0/3400786210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9fbe390 0x55faba36f200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b54050f00 tx=0x7f9b540febf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e167e725536ddc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:10.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.359+0000 7f9b3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:39:10.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.360+0000 7f9b3cff9640 1 -- 172.21.15.38:0/3400786210 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54117060 con 0x55fab9fbe390 2024-05-21T15:39:10.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.360+0000 7f9b3cff9640 1 -- 172.21.15.38:0/3400786210 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9b54117200 con 0x55fab9fbe390 2024-05-21T15:39:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.361+0000 7f9b6220d740 1 -- 172.21.15.38:0/3400786210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b4400bdb0 con 0x55fab9fbe390 2024-05-21T15:39:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.363+0000 7f9b3cff9640 1 -- 172.21.15.38:0/3400786210 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9b5411c050 con 0x55fab9fbe390 2024-05-21T15:39:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.364+0000 7f9b6220d740 10 client.26458 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.364+0000 7f9b6220d740 1 -- 172.21.15.38:0/3400786210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fab9f960f0 con 0x55fab9fbe390 2024-05-21T15:39:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.365+0000 7f9b3cff9640 1 -- 172.21.15.38:0/3400786210 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b54107010 con 0x55fab9fbe390 2024-05-21T15:39:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.365+0000 7f9b3cff9640 1 -- 172.21.15.38:0/3400786210 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f9b540fd8a0 con 0x55fab9fbe390 2024-05-21T15:39:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.365+0000 7f9b6220d740 10 client.26458 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.365+0000 7f9b6220d740 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.365+0000 7f9b6220d740 20 client.26458 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.365+0000 7f9b6220d740 1 --2- 172.21.15.38:0/3400786210 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fab9f7cf60 0x55faba375a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.365+0000 7f9b6220d740 4 client.26458 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.365+0000 7f9b6220d740 1 -- 172.21.15.38:0/3400786210 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9b4400bdb0 con 0x55fab9f7cf60 2024-05-21T15:39:10.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.367+0000 7f9b59027640 1 --2- 172.21.15.38:0/3400786210 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fab9f7cf60 0x55faba375a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.370+0000 7f9b59027640 1 --2- 172.21.15.38:0/3400786210 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fab9f7cf60 0x55faba375a10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1ab6d68148da217 server_cookie=5cc660da54e9346c in_seq=0 out_seq=0 2024-05-21T15:39:10.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.370+0000 7f9b3cff9640 10 client.26458 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:10.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.371+0000 7f9b3cff9640 1 -- 172.21.15.38:0/3400786210 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f9b4400bdb0 con 0x55fab9f7cf60 2024-05-21T15:39:10.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.371+0000 7f9b3cff9640 10 client.26458 handle_command_reply: tid=0 2024-05-21T15:39:10.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.371+0000 7f9b6220d740 1 client.26458 shutdown 2024-05-21T15:39:10.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.373+0000 7f9b6220d740 1 -- 172.21.15.38:0/3400786210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9fbe390 msgr2=0x55faba36f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:10.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.373+0000 7f9b6220d740 1 --2- 172.21.15.38:0/3400786210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9fbe390 0x55faba36f200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b54050f00 tx=0x7f9b540febf0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.377+0000 7f9b6220d740 1 -- 172.21.15.38:0/3400786210 shutdown_connections 2024-05-21T15:39:10.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.377+0000 7f9b6220d740 1 --2- 172.21.15.38:0/3400786210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9fbe390 0x55faba36f200 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.377+0000 7f9b6220d740 1 --2- 172.21.15.38:0/3400786210 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fab9f7cf60 0x55faba375a10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.377+0000 7f9b3cff9640 0 client.26458 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:10.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.377+0000 7f9b6220d740 1 --2- 172.21.15.38:0/3400786210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f9d6b0 0x55faba3682a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.377+0000 7f9b6220d740 1 --2- 172.21.15.38:0/3400786210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f954d0 0x55faba367d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.377+0000 7f9b6220d740 1 -- 172.21.15.38:0/3400786210 >> 172.21.15.38:0/3400786210 conn(0x55fab9fb24e0 msgr2=0x55faba358670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:10.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.378+0000 7f9b6220d740 1 -- 172.21.15.38:0/3400786210 shutdown_connections 2024-05-21T15:39:10.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.378+0000 7f9b6220d740 1 -- 172.21.15.38:0/3400786210 wait complete. 2024-05-21T15:39:10.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.378+0000 7f9b6220d740 20 client.26458 trim_cache size 0 max 16384 2024-05-21T15:39:10.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.415+0000 7f9b6220d740 1 Processor -- start 2024-05-21T15:39:10.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.415+0000 7f9b6220d740 1 -- start start 2024-05-21T15:39:10.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.415+0000 7f9b6220d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f7cf60 0x55faba37a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.416+0000 7f9b6220d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f954d0 0x55faba36f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.416+0000 7f9b6220d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f9d6b0 0x55faba37a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.416+0000 7f9b6220d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3800e5e0 con 0x55fab9f954d0 2024-05-21T15:39:10.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.416+0000 7f9b6220d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9b38010210 con 0x55fab9f9d6b0 2024-05-21T15:39:10.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.416+0000 7f9b59828640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f9d6b0 0x55faba37a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.416+0000 7f9b59828640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f9d6b0 0x55faba37a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:40062/0 (socket says 172.21.15.38:40062) 2024-05-21T15:39:10.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.416+0000 7f9b59828640 1 -- 172.21.15.38:0/1313867833 learned_addr learned my addr 172.21.15.38:0/1313867833 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:10.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.416+0000 7f9b6220d740 1 -- 172.21.15.38:0/1313867833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3800ecb0 con 0x55fab9f7cf60 2024-05-21T15:39:10.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.416+0000 7f9b59027640 1 --2- 172.21.15.38:0/1313867833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f7cf60 0x55faba37a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.417+0000 7f9b58826640 1 --2- 172.21.15.38:0/1313867833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f954d0 0x55faba36f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.419+0000 7f9b59828640 1 -- 172.21.15.38:0/1313867833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f7cf60 msgr2=0x55faba37a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:10.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.419+0000 7f9b59828640 1 --2- 172.21.15.38:0/1313867833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f7cf60 0x55faba37a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.419+0000 7f9b59828640 1 -- 172.21.15.38:0/1313867833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f954d0 msgr2=0x55faba36f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:10.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.419+0000 7f9b59828640 1 --2- 172.21.15.38:0/1313867833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f954d0 0x55faba36f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.419+0000 7f9b59828640 1 -- 172.21.15.38:0/1313867833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faba36f660 con 0x55fab9f9d6b0 2024-05-21T15:39:10.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.419+0000 7f9b59828640 1 --2- 172.21.15.38:0/1313867833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f9d6b0 0x55faba37a700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9b54064af0 tx=0x7f9b5407bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c28aedfb540f45a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.421+0000 7f9b3effd640 1 -- 172.21.15.38:0/1313867833 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54117040 con 0x55fab9f9d6b0 2024-05-21T15:39:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.421+0000 7f9b3effd640 1 -- 172.21.15.38:0/1313867833 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b54107010 con 0x55fab9f9d6b0 2024-05-21T15:39:10.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.421+0000 7f9b3effd640 1 -- 172.21.15.38:0/1313867833 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b540fab40 con 0x55fab9f9d6b0 2024-05-21T15:39:10.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.423+0000 7f9b6220d740 1 -- 172.21.15.38:0/1313867833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f9d6b0 msgr2=0x55faba37a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:10.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.423+0000 7f9b6220d740 1 --2- 172.21.15.38:0/1313867833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f9d6b0 0x55faba37a700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9b54064af0 tx=0x7f9b5407bf60 comp rx=0 tx=0).stop 2024-05-21T15:39:10.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.423+0000 7f9b6220d740 1 -- 172.21.15.38:0/1313867833 shutdown_connections 2024-05-21T15:39:10.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.423+0000 7f9b6220d740 1 --2- 172.21.15.38:0/1313867833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f9d6b0 0x55faba37a700 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.423+0000 7f9b6220d740 1 --2- 172.21.15.38:0/1313867833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f954d0 0x55faba36f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.423+0000 7f9b6220d740 1 --2- 172.21.15.38:0/1313867833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f7cf60 0x55faba37a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.423+0000 7f9b6220d740 1 -- 172.21.15.38:0/1313867833 >> 172.21.15.38:0/1313867833 conn(0x55fab9fb24e0 msgr2=0x55faba366380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:10.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.427+0000 7f9b6220d740 1 -- 172.21.15.38:0/1313867833 shutdown_connections 2024-05-21T15:39:10.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.429+0000 7f9b6220d740 1 -- 172.21.15.38:0/1313867833 wait complete. 2024-05-21T15:39:10.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.429+0000 7f9b6220d740 1 -- start start 2024-05-21T15:39:10.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.429+0000 7f9b6220d740 1 Processor -- start 2024-05-21T15:39:10.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.430+0000 7f9b6220d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f7cf60 0x55faba36e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.430+0000 7f9b6220d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f954d0 0x55faba36eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.430+0000 7f9b6220d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f9d6b0 0x55faba3dc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.430+0000 7f9b6220d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3800fa80 con 0x55fab9f954d0 2024-05-21T15:39:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.430+0000 7f9b6220d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3800f380 con 0x55fab9f7cf60 2024-05-21T15:39:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.430+0000 7f9b6220d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3800c5c0 con 0x55fab9f9d6b0 2024-05-21T15:39:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.430+0000 7f9b6220d740 10 client.0 _ll_register_callbacks cb 0x7f9b540fe200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:10.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.432+0000 7f9b59027640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f7cf60 0x55faba36e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.432+0000 7f9b59027640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f7cf60 0x55faba36e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:40070/0 (socket says 172.21.15.38:40070) 2024-05-21T15:39:10.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.432+0000 7f9b59027640 1 -- 172.21.15.38:0/18981921 learned_addr learned my addr 172.21.15.38:0/18981921 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:10.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.432+0000 7f9b58826640 1 --2- 172.21.15.38:0/18981921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f954d0 0x55faba36eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.432+0000 7f9b59828640 1 --2- 172.21.15.38:0/18981921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f9d6b0 0x55faba3dc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.434+0000 7f9b59027640 1 -- 172.21.15.38:0/18981921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f9d6b0 msgr2=0x55faba3dc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:10.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.434+0000 7f9b59027640 1 --2- 172.21.15.38:0/18981921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f9d6b0 0x55faba3dc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.434+0000 7f9b59027640 1 -- 172.21.15.38:0/18981921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f954d0 msgr2=0x55faba36eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:10.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.434+0000 7f9b59027640 1 --2- 172.21.15.38:0/18981921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f954d0 0x55faba36eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.434+0000 7f9b59027640 1 -- 172.21.15.38:0/18981921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faba3ed6d0 con 0x55fab9f7cf60 2024-05-21T15:39:10.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.434+0000 7f9b59027640 1 --2- 172.21.15.38:0/18981921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f7cf60 0x55faba36e580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55faba376140 tx=0x7f9b44009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48982d94db517b7c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:10.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.434+0000 7f9b3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:39:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.435+0000 7f9b3cff9640 1 -- 172.21.15.38:0/18981921 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44015040 con 0x55fab9f7cf60 2024-05-21T15:39:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.435+0000 7f9b3cff9640 1 -- 172.21.15.38:0/18981921 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9b4400bda0 con 0x55fab9f7cf60 2024-05-21T15:39:10.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.437+0000 7f9b6220d740 1 -- 172.21.15.38:0/18981921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b54107010 con 0x55fab9f7cf60 2024-05-21T15:39:10.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.438+0000 7f9b3cff9640 1 -- 172.21.15.38:0/18981921 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9b44033050 con 0x55fab9f7cf60 2024-05-21T15:39:10.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.438+0000 7f9b6220d740 10 client.26384 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:10.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.438+0000 7f9b6220d740 1 -- 172.21.15.38:0/18981921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55faba3ee9c0 con 0x55fab9f7cf60 2024-05-21T15:39:10.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.443+0000 7f9b3cff9640 1 -- 172.21.15.38:0/18981921 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b54107010 con 0x55fab9f7cf60 2024-05-21T15:39:10.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.443+0000 7f9b3cff9640 1 -- 172.21.15.38:0/18981921 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f9b44024940 con 0x55fab9f7cf60 2024-05-21T15:39:10.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.444+0000 7f9b6220d740 10 client.26384 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:10.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.444+0000 7f9b6220d740 10 client.26384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:10.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.444+0000 7f9b6220d740 20 client.26384 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:10.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.445+0000 7f9b6220d740 1 --2- 172.21.15.38:0/18981921 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fab9fbe390 0x55faba3f1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:10.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.445+0000 7f9b6220d740 4 client.26384 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:39:10.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.445+0000 7f9b6220d740 1 -- 172.21.15.38:0/18981921 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55faba2d0430 con 0x55fab9fbe390 2024-05-21T15:39:10.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.447+0000 7f9b58826640 1 --2- 172.21.15.38:0/18981921 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fab9fbe390 0x55faba3f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:10.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.450+0000 7f9b58826640 1 --2- 172.21.15.38:0/18981921 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fab9fbe390 0x55faba3f1340 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe6812a4cba3a9ca server_cookie=60f5aa71bd3d0502 in_seq=0 out_seq=0 2024-05-21T15:39:10.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.453+0000 7f9b3cff9640 10 client.26384 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:10.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.454+0000 7f9b3cff9640 1 -- 172.21.15.38:0/18981921 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55faba2d0430 con 0x55fab9fbe390 2024-05-21T15:39:10.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.454+0000 7f9b3cff9640 10 client.26384 handle_command_reply: tid=0 2024-05-21T15:39:10.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.456+0000 7f9b6220d740 1 client.26384 shutdown 2024-05-21T15:39:10.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.458+0000 7f9b6220d740 1 -- 172.21.15.38:0/18981921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f7cf60 msgr2=0x55faba36e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:10.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.458+0000 7f9b6220d740 1 --2- 172.21.15.38:0/18981921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f7cf60 0x55faba36e580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55faba376140 tx=0x7f9b44009460 comp rx=0 tx=0).stop 2024-05-21T15:39:10.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.459+0000 7f9b6220d740 1 -- 172.21.15.38:0/18981921 shutdown_connections 2024-05-21T15:39:10.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.459+0000 7f9b6220d740 1 --2- 172.21.15.38:0/18981921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab9f9d6b0 0x55faba3dc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.459+0000 7f9b6220d740 1 --2- 172.21.15.38:0/18981921 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fab9fbe390 0x55faba3f1340 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.459+0000 7f9b6220d740 1 --2- 172.21.15.38:0/18981921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fab9f954d0 0x55faba36eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.459+0000 7f9b6220d740 1 --2- 172.21.15.38:0/18981921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fab9f7cf60 0x55faba36e580 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.459+0000 7f9b6220d740 1 -- 172.21.15.38:0/18981921 >> 172.21.15.38:0/18981921 conn(0x55fab9fb24e0 msgr2=0x55faba35c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:10.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.459+0000 7f9b3cff9640 0 client.26384 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:10.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.461+0000 7f9b6220d740 1 -- 172.21.15.38:0/18981921 shutdown_connections 2024-05-21T15:39:10.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.463+0000 7f9b6220d740 1 -- 172.21.15.38:0/18981921 wait complete. 2024-05-21T15:39:10.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.464+0000 7f9b6220d740 20 client.26384 trim_cache size 0 max 16384 2024-05-21T15:39:10.462 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:39:10.462 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "scrub active (4 inodes in the stack)", 2024-05-21T15:39:10.462 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-05-21T15:39:10.463 INFO:teuthology.orchestra.run.smithi038.stdout: "bce6877c-ded0-48cd-90be-bd40d9b63b34": { 2024-05-21T15:39:10.463 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-05-21T15:39:10.463 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "bce6877c-ded0-48cd-90be-bd40d9b63b34", 2024-05-21T15:39:10.463 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-05-21T15:39:10.463 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:39:10.463 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:39:10.463 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:39:10.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.466+0000 7f9b5a029640 1 -- 172.21.15.38:0/1397986477 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9b400a2c90 msgr2=0x7f9b400a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:10.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.466+0000 7f9b5a029640 1 --2- 172.21.15.38:0/1397986477 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9b400a2c90 0x7f9b400a51a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9b50004e60 tx=0x7f9b50004df0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.466+0000 7f9b5a029640 1 -- 172.21.15.38:0/1397986477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b4c09b820 msgr2=0x7f9b4c14bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:10.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.466+0000 7f9b5a029640 1 --2- 172.21.15.38:0/1397986477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b4c09b820 0x7f9b4c14bec0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9b540501a0 tx=0x7f9b54070570 comp rx=0 tx=0).stop 2024-05-21T15:39:10.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.466+0000 7f9b5a029640 1 -- 172.21.15.38:0/1397986477 shutdown_connections 2024-05-21T15:39:10.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.466+0000 7f9b5a029640 1 --2- 172.21.15.38:0/1397986477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b4c09b820 0x7f9b4c14bec0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.466+0000 7f9b5a029640 1 --2- 172.21.15.38:0/1397986477 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9b400a2c90 0x7f9b400a51a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.466+0000 7f9b5a029640 1 --2- 172.21.15.38:0/1397986477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b4c096b20 0x7f9b4c144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.466+0000 7f9b5a029640 1 --2- 172.21.15.38:0/1397986477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b4c0958e0 0x7f9b4c1449f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:10.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.466+0000 7f9b5a029640 1 -- 172.21.15.38:0/1397986477 >> 172.21.15.38:0/1397986477 conn(0x7f9b4c090ab0 msgr2=0x7f9b4c097800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:10.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.467+0000 7f9b5a029640 1 -- 172.21.15.38:0/1397986477 shutdown_connections 2024-05-21T15:39:10.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:10.467+0000 7f9b5a029640 1 -- 172.21.15.38:0/1397986477 wait complete. 2024-05-21T15:39:10.483 INFO:tasks.cephfs.filesystem:scrub status for tag:bce6877c-ded0-48cd-90be-bd40d9b63b34 - {'path': '/', 'tag': 'bce6877c-ded0-48cd-90be-bd40d9b63b34', 'options': 'recursive,force'} 2024-05-21T15:39:10.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:10 smithi038 ceph-mon[31995]: scrub queued for path: / 2024-05-21T15:39:10.792 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:10 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:10.792 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:10 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:39:10.792 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:10 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:39:10.792 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:10 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:39:10.792 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:10 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:39:10.792 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:10 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:39:11.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:10 smithi045 ceph-mon[36409]: scrub queued for path: / 2024-05-21T15:39:11.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:10 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:11.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:10 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:39:11.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:10 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:39:11.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:10 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:39:11.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:10 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:39:11.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:10 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:39:11.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:10 smithi053 ceph-mon[36496]: scrub queued for path: / 2024-05-21T15:39:11.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:10 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:11.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:10 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:39:11.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:10 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:39:11.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:10 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:39:11.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:10 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:39:11.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:10 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:39:12.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:11 smithi045 ceph-mon[36409]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 59 MiB/s wr, 12.69k op/s 2024-05-21T15:39:12.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:11 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:12.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:11 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:12.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:11 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:12.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:11 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:12.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:11 smithi038 ceph-mon[31995]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 59 MiB/s wr, 12.69k op/s 2024-05-21T15:39:12.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:11 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:12.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:11 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:12.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:11 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:12.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:11 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:12.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:11 smithi053 ceph-mon[36496]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 59 MiB/s wr, 12.69k op/s 2024-05-21T15:39:12.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:11 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:12.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:11 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:12.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:11 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:12.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:11 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:14.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:13 smithi045 ceph-mon[36409]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 60 MiB/s wr, 13.28k op/s 2024-05-21T15:39:14.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:13 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:14.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:13 smithi038 ceph-mon[31995]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 60 MiB/s wr, 13.28k op/s 2024-05-21T15:39:14.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:13 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:14.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:13 smithi053 ceph-mon[36496]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 60 MiB/s wr, 13.28k op/s 2024-05-21T15:39:14.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:13 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:16.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:15 smithi045 ceph-mon[36409]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 56 MiB/s wr, 12.57k op/s 2024-05-21T15:39:16.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:15 smithi038 ceph-mon[31995]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 56 MiB/s wr, 12.57k op/s 2024-05-21T15:39:16.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:15 smithi053 ceph-mon[36496]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 56 MiB/s wr, 12.57k op/s 2024-05-21T15:39:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.868+0000 7f1da4faa640 1 -- 172.21.15.38:0/2420967340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1da0077b50 msgr2=0x7f1da0077f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.868+0000 7f1da4faa640 1 --2- 172.21.15.38:0/2420967340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1da0077b50 0x7f1da0077f60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009900 tx=0x7f1d9402a420 comp rx=0 tx=0).stop 2024-05-21T15:39:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.871+0000 7f1da4faa640 1 -- 172.21.15.38:0/2420967340 shutdown_connections 2024-05-21T15:39:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.871+0000 7f1da4faa640 1 --2- 172.21.15.38:0/2420967340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0078850 0x7f1da0078c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.871+0000 7f1da4faa640 1 --2- 172.21.15.38:0/2420967340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1da0077b50 0x7f1da0077f60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.871+0000 7f1da4faa640 1 --2- 172.21.15.38:0/2420967340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da00768c0 0x7f1da0077610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.871+0000 7f1da4faa640 1 -- 172.21.15.38:0/2420967340 >> 172.21.15.38:0/2420967340 conn(0x7f1da00ffee0 msgr2=0x7f1da0102370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.874+0000 7f1da4faa640 1 -- 172.21.15.38:0/2420967340 shutdown_connections 2024-05-21T15:39:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.875+0000 7f1da4faa640 1 -- 172.21.15.38:0/2420967340 wait complete. 2024-05-21T15:39:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.876+0000 7f1da4faa640 1 Processor -- start 2024-05-21T15:39:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.877+0000 7f1da4faa640 1 -- start start 2024-05-21T15:39:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.877+0000 7f1da4faa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da00768c0 0x7f1da0106f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.877+0000 7f1da4faa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0077b50 0x7f1da0107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.877+0000 7f1da4faa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1da0078850 0x7f1da010e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.877+0000 7f1da4faa640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0114de0 con 0x7f1da0078850 2024-05-21T15:39:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.877+0000 7f1da4faa640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0114c60 con 0x7f1da00768c0 2024-05-21T15:39:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.877+0000 7f1da4faa640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0114f60 con 0x7f1da0077b50 2024-05-21T15:39:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.877+0000 7f1d9e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da00768c0 0x7f1da0106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.877+0000 7f1d9e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da00768c0 0x7f1da0106f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:54906/0 (socket says 172.21.15.38:54906) 2024-05-21T15:39:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.877+0000 7f1d9e59c640 1 -- 172.21.15.38:0/828666793 learned_addr learned my addr 172.21.15.38:0/828666793 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.878+0000 7f1d9ed9d640 1 --2- 172.21.15.38:0/828666793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1da0078850 0x7f1da010e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.878+0000 7f1d9e59c640 1 -- 172.21.15.38:0/828666793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0077b50 msgr2=0x7f1da0107450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T15:39:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.878+0000 7f1d9e59c640 1 --2- 172.21.15.38:0/828666793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0077b50 0x7f1da0107450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.878+0000 7f1d9e59c640 1 -- 172.21.15.38:0/828666793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1da0078850 msgr2=0x7f1da010e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.878+0000 7f1d9e59c640 1 --2- 172.21.15.38:0/828666793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1da0078850 0x7f1da010e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.878+0000 7f1d9e59c640 1 -- 172.21.15.38:0/828666793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da010ea50 con 0x7f1da00768c0 2024-05-21T15:39:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.878+0000 7f1d9ed9d640 1 --2- 172.21.15.38:0/828666793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1da0078850 0x7f1da010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:39:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.878+0000 7f1d9e59c640 1 --2- 172.21.15.38:0/828666793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da00768c0 0x7f1da0106f10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800b9e0 tx=0x7f1d9800bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.880+0000 7f1d8f7fe640 1 -- 172.21.15.38:0/828666793 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98019070 con 0x7f1da00768c0 2024-05-21T15:39:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.881+0000 7f1d8f7fe640 1 -- 172.21.15.38:0/828666793 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9800cba0 con 0x7f1da00768c0 2024-05-21T15:39:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.881+0000 7f1d8f7fe640 1 -- 172.21.15.38:0/828666793 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98012720 con 0x7f1da00768c0 2024-05-21T15:39:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.881+0000 7f1da4faa640 1 -- 172.21.15.38:0/828666793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da010ed40 con 0x7f1da00768c0 2024-05-21T15:39:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.881+0000 7f1da4faa640 1 -- 172.21.15.38:0/828666793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da010f2c0 con 0x7f1da00768c0 2024-05-21T15:39:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.887+0000 7f1d8f7fe640 1 -- 172.21.15.38:0/828666793 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1d9801c070 con 0x7f1da00768c0 2024-05-21T15:39:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.890+0000 7f1d8f7fe640 1 --2- 172.21.15.38:0/828666793 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1d880a2e70 0x7f1d880a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.891+0000 7f1d8f7fe640 1 -- 172.21.15.38:0/828666793 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f1d980cf6e0 con 0x7f1da00768c0 2024-05-21T15:39:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.891+0000 7f1dabe51740 1 Processor -- start 2024-05-21T15:39:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.891+0000 7f1dabe51740 1 -- start start 2024-05-21T15:39:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.892+0000 7f1d9dd9b640 1 --2- 172.21.15.38:0/828666793 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1d880a2e70 0x7f1d880a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.892+0000 7f1dabe51740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b1001b0 0x561d9b49fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.892+0000 7f1dabe51740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0d72f0 0x561d9b4a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.892+0000 7f1dabe51740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0df4d0 0x561d9b4a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.892+0000 7f1dabe51740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561d9b404340 con 0x561d9b0df4d0 2024-05-21T15:39:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.892+0000 7f1dabe51740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561d9b3ac290 con 0x561d9b0d72f0 2024-05-21T15:39:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.892+0000 7f1dabe51740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561d9b400300 con 0x561d9b1001b0 2024-05-21T15:39:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.892+0000 7f1d9e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b1001b0 0x561d9b49fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.892+0000 7f1d9e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b1001b0 0x561d9b49fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:60420/0 (socket says 172.21.15.38:60420) 2024-05-21T15:39:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.892+0000 7f1d9e59c640 1 -- 172.21.15.38:0/4027258968 learned_addr learned my addr 172.21.15.38:0/4027258968 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.892+0000 7f1d9dd9b640 1 --2- 172.21.15.38:0/4027258968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0d72f0 0x561d9b4a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.893+0000 7f1d9ed9d640 1 --2- 172.21.15.38:0/4027258968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0df4d0 0x561d9b4a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.893+0000 7f1d9e59c640 1 -- 172.21.15.38:0/4027258968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0d72f0 msgr2=0x561d9b4a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.893+0000 7f1d9e59c640 1 --2- 172.21.15.38:0/4027258968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0d72f0 0x561d9b4a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.893+0000 7f1d9e59c640 1 -- 172.21.15.38:0/4027258968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0df4d0 msgr2=0x561d9b4a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.893+0000 7f1d9e59c640 1 --2- 172.21.15.38:0/4027258968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0df4d0 0x561d9b4a72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.893+0000 7f1d9e59c640 1 -- 172.21.15.38:0/4027258968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d9b3d08e0 con 0x561d9b1001b0 2024-05-21T15:39:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.893+0000 7f1d9e59c640 1 --2- 172.21.15.38:0/4027258968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b1001b0 0x561d9b49fe60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800bfd0 tx=0x7f1d9809b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e16479ef91748710 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.894+0000 7f1d9dd9b640 1 --2- 172.21.15.38:0/828666793 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1d880a2e70 0x7f1d880a5380 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009230 tx=0x7f1d94000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.896+0000 7f1d8cff9640 1 -- 172.21.15.38:0/4027258968 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d980a0240 con 0x561d9b1001b0 2024-05-21T15:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.896+0000 7f1d8cff9640 1 -- 172.21.15.38:0/4027258968 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d980a4b00 con 0x561d9b1001b0 2024-05-21T15:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.896+0000 7f1d8cff9640 1 -- 172.21.15.38:0/4027258968 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d980a4670 con 0x561d9b1001b0 2024-05-21T15:39:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.898+0000 7f1dabe51740 1 -- 172.21.15.38:0/4027258968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b1001b0 msgr2=0x561d9b49fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.898+0000 7f1dabe51740 1 --2- 172.21.15.38:0/4027258968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b1001b0 0x561d9b49fe60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800bfd0 tx=0x7f1d9809b8b0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.899+0000 7f1dabe51740 1 -- 172.21.15.38:0/4027258968 shutdown_connections 2024-05-21T15:39:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.899+0000 7f1dabe51740 1 --2- 172.21.15.38:0/4027258968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0df4d0 0x561d9b4a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.899+0000 7f1dabe51740 1 --2- 172.21.15.38:0/4027258968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0d72f0 0x561d9b4a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.899+0000 7f1dabe51740 1 --2- 172.21.15.38:0/4027258968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b1001b0 0x561d9b49fe60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.899+0000 7f1dabe51740 1 -- 172.21.15.38:0/4027258968 >> 172.21.15.38:0/4027258968 conn(0x561d9b0f4300 msgr2=0x561d9b49b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.902+0000 7f1dabe51740 1 -- 172.21.15.38:0/4027258968 shutdown_connections 2024-05-21T15:39:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.902+0000 7f1dabe51740 1 -- 172.21.15.38:0/4027258968 wait complete. 2024-05-21T15:39:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.903+0000 7f1dabe51740 1 -- start start 2024-05-21T15:39:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.903+0000 7f1dabe51740 1 Processor -- start 2024-05-21T15:39:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.903+0000 7f1dabe51740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0d72f0 0x561d9b4a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.904+0000 7f1dabe51740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0df4d0 0x561d9b4aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.904+0000 7f1dabe51740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b1001b0 0x561d9b4b1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.904+0000 7f1dabe51740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561d9b4a7d30 con 0x561d9b0d72f0 2024-05-21T15:39:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.904+0000 7f1dabe51740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561d9b3bf790 con 0x561d9b0df4d0 2024-05-21T15:39:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.904+0000 7f1dabe51740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561d9b4a7eb0 con 0x561d9b1001b0 2024-05-21T15:39:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.904+0000 7f1dabe51740 10 client.0 _ll_register_callbacks cb 0x561d9b3faa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.904+0000 7f1d9e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0d72f0 0x561d9b4a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.904+0000 7f1d9e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0d72f0 0x561d9b4a9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48566/0 (socket says 172.21.15.38:48566) 2024-05-21T15:39:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.904+0000 7f1d9e59c640 1 -- 172.21.15.38:0/120378748 learned_addr learned my addr 172.21.15.38:0/120378748 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.905+0000 7f1d9dd9b640 1 --2- 172.21.15.38:0/120378748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0df4d0 0x561d9b4aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.905+0000 7f1d9e59c640 1 -- 172.21.15.38:0/120378748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b1001b0 msgr2=0x561d9b4b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.905+0000 7f1d9e59c640 1 --2- 172.21.15.38:0/120378748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b1001b0 0x561d9b4b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.905+0000 7f1d9e59c640 1 -- 172.21.15.38:0/120378748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0df4d0 msgr2=0x561d9b4aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.905+0000 7f1d9e59c640 1 --2- 172.21.15.38:0/120378748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0df4d0 0x561d9b4aa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.905+0000 7f1d9e59c640 1 -- 172.21.15.38:0/120378748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d9b118210 con 0x561d9b0d72f0 2024-05-21T15:39:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.905+0000 7f1d9e59c640 1 --2- 172.21.15.38:0/120378748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0d72f0 0x561d9b4a9bf0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1d98098860 tx=0x7f1d980a0240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff71f60327d8bfac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.905+0000 7f1d85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.905+0000 7f1d9dd9b640 1 --2- 172.21.15.38:0/120378748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0df4d0 0x561d9b4aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:39:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.906+0000 7f1d85ffb640 1 -- 172.21.15.38:0/120378748 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d980a4870 con 0x561d9b0d72f0 2024-05-21T15:39:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.906+0000 7f1dabe51740 1 -- 172.21.15.38:0/120378748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d980a4b00 con 0x561d9b0d72f0 2024-05-21T15:39:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.906+0000 7f1d85ffb640 1 -- 172.21.15.38:0/120378748 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f1d9809b2e0 con 0x561d9b0d72f0 2024-05-21T15:39:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.906+0000 7f1d85ffb640 1 -- 172.21.15.38:0/120378748 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1d9809b4e0 con 0x561d9b0d72f0 2024-05-21T15:39:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.908+0000 7f1dabe51740 10 client.16755 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.908+0000 7f1dabe51740 1 -- 172.21.15.38:0/120378748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561d9b1e4620 con 0x561d9b0d72f0 2024-05-21T15:39:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.910+0000 7f1d85ffb640 1 -- 172.21.15.38:0/120378748 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d980a4b00 con 0x561d9b0d72f0 2024-05-21T15:39:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.911+0000 7f1d85ffb640 1 -- 172.21.15.38:0/120378748 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f1d9809aed0 con 0x561d9b0d72f0 2024-05-21T15:39:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.911+0000 7f1dabe51740 10 client.16755 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.911+0000 7f1dabe51740 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.911+0000 7f1dabe51740 20 client.16755 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.911+0000 7f1dabe51740 1 --2- 172.21.15.38:0/120378748 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561d9b0bed80 0x561d9b4b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.911+0000 7f1dabe51740 4 client.16755 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.911+0000 7f1dabe51740 1 -- 172.21.15.38:0/120378748 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d9b411a70 con 0x561d9b0bed80 2024-05-21T15:39:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.913+0000 7f1d9dd9b640 1 --2- 172.21.15.38:0/120378748 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561d9b0bed80 0x561d9b4b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.915+0000 7f1d9dd9b640 1 --2- 172.21.15.38:0/120378748 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561d9b0bed80 0x561d9b4b7530 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9a55e6d2921cdb6 server_cookie=bd75a67d7837284e in_seq=0 out_seq=0 2024-05-21T15:39:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.915+0000 7f1d85ffb640 10 client.16755 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.918+0000 7f1d85ffb640 1 -- 172.21.15.38:0/120378748 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561d9b411a70 con 0x561d9b0bed80 2024-05-21T15:39:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.918+0000 7f1d85ffb640 10 client.16755 handle_command_reply: tid=0 2024-05-21T15:39:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.919+0000 7f1dabe51740 1 client.16755 shutdown 2024-05-21T15:39:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.919+0000 7f1dabe51740 1 -- 172.21.15.38:0/120378748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0d72f0 msgr2=0x561d9b4a9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.919+0000 7f1dabe51740 1 --2- 172.21.15.38:0/120378748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0d72f0 0x561d9b4a9bf0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1d98098860 tx=0x7f1d980a0240 comp rx=0 tx=0).stop 2024-05-21T15:39:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.921+0000 7f1dabe51740 1 -- 172.21.15.38:0/120378748 shutdown_connections 2024-05-21T15:39:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.921+0000 7f1dabe51740 1 --2- 172.21.15.38:0/120378748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b1001b0 0x561d9b4b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.921+0000 7f1dabe51740 1 --2- 172.21.15.38:0/120378748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0df4d0 0x561d9b4aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.921+0000 7f1dabe51740 1 --2- 172.21.15.38:0/120378748 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561d9b0bed80 0x561d9b4b7530 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.921+0000 7f1dabe51740 1 --2- 172.21.15.38:0/120378748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0d72f0 0x561d9b4a9bf0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.921+0000 7f1dabe51740 1 -- 172.21.15.38:0/120378748 >> 172.21.15.38:0/120378748 conn(0x561d9b0f4300 msgr2=0x561d9b49a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.922+0000 7f1d85ffb640 0 client.16755 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.929+0000 7f1dabe51740 1 -- 172.21.15.38:0/120378748 shutdown_connections 2024-05-21T15:39:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.932+0000 7f1dabe51740 1 -- 172.21.15.38:0/120378748 wait complete. 2024-05-21T15:39:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.933+0000 7f1dabe51740 20 client.16755 trim_cache size 0 max 16384 2024-05-21T15:39:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.972+0000 7f1dabe51740 1 Processor -- start 2024-05-21T15:39:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.972+0000 7f1dabe51740 1 -- start start 2024-05-21T15:39:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.973+0000 7f1dabe51740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b0d72f0 0x561d9b49eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.973+0000 7f1dabe51740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0df4d0 0x561d9b4b1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.973+0000 7f1dabe51740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b1001b0 0x561d9b4b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.973+0000 7f1dabe51740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8000e6d0 con 0x561d9b1001b0 2024-05-21T15:39:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.973+0000 7f1dabe51740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1d80010300 con 0x561d9b0df4d0 2024-05-21T15:39:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.973+0000 7f1d9ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b1001b0 0x561d9b4b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.973+0000 7f1d9ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b1001b0 0x561d9b4b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48576/0 (socket says 172.21.15.38:48576) 2024-05-21T15:39:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.973+0000 7f1d9ed9d640 1 -- 172.21.15.38:0/4058632651 learned_addr learned my addr 172.21.15.38:0/4058632651 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.973+0000 7f1dabe51740 1 -- 172.21.15.38:0/4058632651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8000eda0 con 0x561d9b0d72f0 2024-05-21T15:39:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.976+0000 7f1d9dd9b640 1 --2- 172.21.15.38:0/4058632651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0df4d0 0x561d9b4b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.977+0000 7f1d9e59c640 1 --2- 172.21.15.38:0/4058632651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b0d72f0 0x561d9b49eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.978+0000 7f1d9ed9d640 1 -- 172.21.15.38:0/4058632651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b0d72f0 msgr2=0x561d9b49eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.978+0000 7f1d9ed9d640 1 --2- 172.21.15.38:0/4058632651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b0d72f0 0x561d9b49eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.978+0000 7f1d9ed9d640 1 -- 172.21.15.38:0/4058632651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0df4d0 msgr2=0x561d9b4b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.978+0000 7f1d9ed9d640 1 --2- 172.21.15.38:0/4058632651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0df4d0 0x561d9b4b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.978+0000 7f1d9ed9d640 1 -- 172.21.15.38:0/4058632651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d9b4a0af0 con 0x561d9b1001b0 2024-05-21T15:39:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.978+0000 7f1d9ed9d640 1 --2- 172.21.15.38:0/4058632651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b1001b0 0x561d9b4b1560 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000bba0 tx=0x7f1d9000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f493b419c7df2a87 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.979+0000 7f1d87fff640 1 -- 172.21.15.38:0/4058632651 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9001b680 con 0x561d9b1001b0 2024-05-21T15:39:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.979+0000 7f1d87fff640 1 -- 172.21.15.38:0/4058632651 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9001bd00 con 0x561d9b1001b0 2024-05-21T15:39:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.979+0000 7f1d87fff640 1 -- 172.21.15.38:0/4058632651 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90010cc0 con 0x561d9b1001b0 2024-05-21T15:39:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.981+0000 7f1dabe51740 1 -- 172.21.15.38:0/4058632651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b1001b0 msgr2=0x561d9b4b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.981+0000 7f1dabe51740 1 --2- 172.21.15.38:0/4058632651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b1001b0 0x561d9b4b1560 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000bba0 tx=0x7f1d9000f790 comp rx=0 tx=0).stop 2024-05-21T15:39:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.983+0000 7f1dabe51740 1 -- 172.21.15.38:0/4058632651 shutdown_connections 2024-05-21T15:39:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.983+0000 7f1dabe51740 1 --2- 172.21.15.38:0/4058632651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b1001b0 0x561d9b4b1560 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.983+0000 7f1dabe51740 1 --2- 172.21.15.38:0/4058632651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b0df4d0 0x561d9b4b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.983+0000 7f1dabe51740 1 --2- 172.21.15.38:0/4058632651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b0d72f0 0x561d9b49eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.983+0000 7f1dabe51740 1 -- 172.21.15.38:0/4058632651 >> 172.21.15.38:0/4058632651 conn(0x561d9b0f4300 msgr2=0x561d9b49e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.987+0000 7f1dabe51740 1 -- 172.21.15.38:0/4058632651 shutdown_connections 2024-05-21T15:39:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.988+0000 7f1dabe51740 1 -- 172.21.15.38:0/4058632651 wait complete. 2024-05-21T15:39:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.988+0000 7f1dabe51740 1 -- start start 2024-05-21T15:39:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.989+0000 7f1dabe51740 1 Processor -- start 2024-05-21T15:39:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.989+0000 7f1dabe51740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0d72f0 0x561d9b4ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.989+0000 7f1dabe51740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b0df4d0 0x561d9b4abe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.989+0000 7f1dabe51740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b1001b0 0x561d9b4b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.989+0000 7f1dabe51740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8000fb70 con 0x561d9b0d72f0 2024-05-21T15:39:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.989+0000 7f1dabe51740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8000f470 con 0x561d9b1001b0 2024-05-21T15:39:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.989+0000 7f1dabe51740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8000c6b0 con 0x561d9b0df4d0 2024-05-21T15:39:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.989+0000 7f1dabe51740 10 client.0 _ll_register_callbacks cb 0x7f1d9800ce00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.990+0000 7f1d9ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b1001b0 0x561d9b4b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.990+0000 7f1d9ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b1001b0 0x561d9b4b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:54938/0 (socket says 172.21.15.38:54938) 2024-05-21T15:39:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.990+0000 7f1d9ed9d640 1 -- 172.21.15.38:0/4089639153 learned_addr learned my addr 172.21.15.38:0/4089639153 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.992+0000 7f1d9dd9b640 1 --2- 172.21.15.38:0/4089639153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b0df4d0 0x561d9b4abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.992+0000 7f1d9ed9d640 1 -- 172.21.15.38:0/4089639153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b0df4d0 msgr2=0x561d9b4abe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.992+0000 7f1d9ed9d640 1 --2- 172.21.15.38:0/4089639153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b0df4d0 0x561d9b4abe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.992+0000 7f1d9ed9d640 1 -- 172.21.15.38:0/4089639153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0d72f0 msgr2=0x561d9b4ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.992+0000 7f1d9ed9d640 1 --2- 172.21.15.38:0/4089639153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0d72f0 0x561d9b4ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.992+0000 7f1d9ed9d640 1 -- 172.21.15.38:0/4089639153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d9b4b1330 con 0x561d9b1001b0 2024-05-21T15:39:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.992+0000 7f1d9dd9b640 1 --2- 172.21.15.38:0/4089639153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b0df4d0 0x561d9b4abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:39:17.992 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.994+0000 7f1d9ed9d640 1 --2- 172.21.15.38:0/4089639153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b1001b0 0x561d9b4b0d70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000b910 tx=0x7f1d90010f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb11f3f3c3092d9c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:17.992 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.994+0000 7f1d85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:39:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.994+0000 7f1d85ffb640 1 -- 172.21.15.38:0/4089639153 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9001b890 con 0x561d9b1001b0 2024-05-21T15:39:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.995+0000 7f1d85ffb640 1 -- 172.21.15.38:0/4089639153 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f1d9001ba30 con 0x561d9b1001b0 2024-05-21T15:39:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.996+0000 7f1dabe51740 1 -- 172.21.15.38:0/4089639153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d9001bd00 con 0x561d9b1001b0 2024-05-21T15:39:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.998+0000 7f1d85ffb640 1 -- 172.21.15.38:0/4089639153 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1d90010430 con 0x561d9b1001b0 2024-05-21T15:39:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.999+0000 7f1dabe51740 10 client.26396 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:17.999+0000 7f1dabe51740 1 -- 172.21.15.38:0/4089639153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561d9b51e0c0 con 0x561d9b1001b0 2024-05-21T15:39:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:18.003+0000 7f1d85ffb640 1 -- 172.21.15.38:0/4089639153 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9001bd00 con 0x561d9b1001b0 2024-05-21T15:39:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:18.004+0000 7f1d85ffb640 1 -- 172.21.15.38:0/4089639153 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f1d90019380 con 0x561d9b1001b0 2024-05-21T15:39:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:18.005+0000 7f1dabe51740 10 client.26396 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:18.005+0000 7f1dabe51740 10 client.26396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:18.005+0000 7f1dabe51740 20 client.26396 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:18.005+0000 7f1dabe51740 1 --2- 172.21.15.38:0/4089639153 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561d9b0941c0 0x561d9b520a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:18.005+0000 7f1dabe51740 4 client.26396 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:39:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:18.005+0000 7f1dabe51740 1 -- 172.21.15.38:0/4089639153 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d9b40bdc0 con 0x561d9b0941c0 2024-05-21T15:39:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:18.006+0000 7f1d9e59c640 1 --2- 172.21.15.38:0/4089639153 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561d9b0941c0 0x561d9b520a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:18.009+0000 7f1d9e59c640 1 --2- 172.21.15.38:0/4089639153 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561d9b0941c0 0x561d9b520a70 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da5aa0027e16f8b0 server_cookie=8a53c0ae57d15760 in_seq=0 out_seq=0 2024-05-21T15:39:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:18.009+0000 7f1d85ffb640 10 client.26396 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:18.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:17 smithi045 ceph-mon[36409]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 MiB/s wr, 11.67k op/s 2024-05-21T15:39:18.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:17 smithi038 ceph-mon[31995]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 MiB/s wr, 11.67k op/s 2024-05-21T15:39:18.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:17 smithi053 ceph-mon[36496]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 MiB/s wr, 11.67k op/s 2024-05-21T15:39:20.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:19 smithi038 ceph-mon[31995]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 55 MiB/s wr, 12.57k op/s 2024-05-21T15:39:20.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:19 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:39:20.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:19 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:39:20.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:19 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:39:20.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:19 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:39:20.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:19 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:39:20.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:19 smithi053 ceph-mon[36496]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 55 MiB/s wr, 12.57k op/s 2024-05-21T15:39:20.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:19 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:39:20.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:19 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:39:20.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:19 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:39:20.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:19 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:39:20.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:19 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:39:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:19 smithi045 ceph-mon[36409]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 55 MiB/s wr, 12.57k op/s 2024-05-21T15:39:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:19 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:39:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:19 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:39:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:19 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:39:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:19 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:39:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:19 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:39:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.871+0000 7f1d85ffb640 1 -- 172.21.15.38:0/4089639153 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d9b40bdc0 con 0x561d9b0941c0 2024-05-21T15:39:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.871+0000 7f1d85ffb640 10 client.26396 handle_command_reply: tid=0 2024-05-21T15:39:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.872+0000 7f1dabe51740 1 client.26396 shutdown 2024-05-21T15:39:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.872+0000 7f1dabe51740 1 -- 172.21.15.38:0/4089639153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b1001b0 msgr2=0x561d9b4b0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.872+0000 7f1dabe51740 1 --2- 172.21.15.38:0/4089639153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b1001b0 0x561d9b4b0d70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000b910 tx=0x7f1d90010f90 comp rx=0 tx=0).stop 2024-05-21T15:39:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.872+0000 7f1dabe51740 1 -- 172.21.15.38:0/4089639153 shutdown_connections 2024-05-21T15:39:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.872+0000 7f1dabe51740 1 --2- 172.21.15.38:0/4089639153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d9b1001b0 0x561d9b4b0d70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.872+0000 7f1dabe51740 1 --2- 172.21.15.38:0/4089639153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561d9b0df4d0 0x561d9b4abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.872+0000 7f1dabe51740 1 --2- 172.21.15.38:0/4089639153 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561d9b0941c0 0x561d9b520a70 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.872+0000 7f1dabe51740 1 --2- 172.21.15.38:0/4089639153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561d9b0d72f0 0x561d9b4ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.872+0000 7f1dabe51740 1 -- 172.21.15.38:0/4089639153 >> 172.21.15.38:0/4089639153 conn(0x561d9b0f4300 msgr2=0x561d9b4a8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.872+0000 7f1d85ffb640 0 client.26396 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.872+0000 7f1dabe51740 1 -- 172.21.15.38:0/4089639153 shutdown_connections 2024-05-21T15:39:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.872+0000 7f1dabe51740 1 -- 172.21.15.38:0/4089639153 wait complete. 2024-05-21T15:39:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.873+0000 7f1dabe51740 20 client.26396 trim_cache size 0 max 16384 2024-05-21T15:39:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:39:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:39:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:39:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:39:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.873+0000 7f1da4faa640 1 -- 172.21.15.38:0/828666793 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1d880a2e70 msgr2=0x7f1d880a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.873+0000 7f1da4faa640 1 --2- 172.21.15.38:0/828666793 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1d880a2e70 0x7f1d880a5380 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009230 tx=0x7f1d94000bc0 comp rx=0 tx=0).stop 2024-05-21T15:39:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.873+0000 7f1da4faa640 1 -- 172.21.15.38:0/828666793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da00768c0 msgr2=0x7f1da0106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.873+0000 7f1da4faa640 1 --2- 172.21.15.38:0/828666793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da00768c0 0x7f1da0106f10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800b9e0 tx=0x7f1d9800bea0 comp rx=0 tx=0).stop 2024-05-21T15:39:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.874+0000 7f1da4faa640 1 -- 172.21.15.38:0/828666793 shutdown_connections 2024-05-21T15:39:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.874+0000 7f1da4faa640 1 --2- 172.21.15.38:0/828666793 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1d880a2e70 0x7f1d880a5380 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.874+0000 7f1da4faa640 1 --2- 172.21.15.38:0/828666793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1da0078850 0x7f1da010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.874+0000 7f1da4faa640 1 --2- 172.21.15.38:0/828666793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0077b50 0x7f1da0107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.874+0000 7f1da4faa640 1 --2- 172.21.15.38:0/828666793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1da00768c0 0x7f1da0106f10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.874+0000 7f1da4faa640 1 -- 172.21.15.38:0/828666793 >> 172.21.15.38:0/828666793 conn(0x7f1da00ffee0 msgr2=0x7f1da006d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.874+0000 7f1da4faa640 1 -- 172.21.15.38:0/828666793 shutdown_connections 2024-05-21T15:39:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:21.874+0000 7f1da4faa640 1 -- 172.21.15.38:0/828666793 wait complete. 2024-05-21T15:39:22.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:21 smithi038 ceph-mon[31995]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 54 MiB/s wr, 12.34k op/s 2024-05-21T15:39:22.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:21 smithi053 ceph-mon[36496]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 54 MiB/s wr, 12.34k op/s 2024-05-21T15:39:22.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:21 smithi045 ceph-mon[36409]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 54 MiB/s wr, 12.34k op/s 2024-05-21T15:39:24.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:23 smithi038 ceph-mon[31995]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 57 MiB/s wr, 12.98k op/s 2024-05-21T15:39:24.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:23 smithi053 ceph-mon[36496]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 57 MiB/s wr, 12.98k op/s 2024-05-21T15:39:24.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:23 smithi045 ceph-mon[36409]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 57 MiB/s wr, 12.98k op/s 2024-05-21T15:39:26.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:25 smithi053 ceph-mon[36496]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 MiB/s wr, 14.44k op/s 2024-05-21T15:39:26.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:25 smithi045 ceph-mon[36409]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 MiB/s wr, 14.44k op/s 2024-05-21T15:39:26.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:25 smithi038 ceph-mon[31995]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 MiB/s wr, 14.44k op/s 2024-05-21T15:39:28.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:27 smithi053 ceph-mon[36496]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 MiB/s wr, 13.93k op/s 2024-05-21T15:39:28.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:27 smithi045 ceph-mon[36409]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 MiB/s wr, 13.93k op/s 2024-05-21T15:39:28.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:27 smithi038 ceph-mon[31995]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 MiB/s wr, 13.93k op/s 2024-05-21T15:39:29.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:28 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:39:29.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:28 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:39:29.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:28 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:39:29.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:28 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:39:29.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:28 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:39:29.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:29 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:39:30.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:29 smithi053 ceph-mon[36496]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 63 MiB/s wr, 14.30k op/s 2024-05-21T15:39:30.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:29 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:39:30.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:29 smithi045 ceph-mon[36409]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 63 MiB/s wr, 14.30k op/s 2024-05-21T15:39:30.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:29 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:39:30.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:29 smithi038 ceph-mon[31995]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 63 MiB/s wr, 14.30k op/s 2024-05-21T15:39:30.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:29 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:39:32.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:31 smithi053 ceph-mon[36496]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 54 MiB/s wr, 12.23k op/s 2024-05-21T15:39:32.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:31 smithi045 ceph-mon[36409]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 54 MiB/s wr, 12.23k op/s 2024-05-21T15:39:32.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:31 smithi038 ceph-mon[31995]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 54 MiB/s wr, 12.23k op/s 2024-05-21T15:39:34.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:33 smithi053 ceph-mon[36496]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 MiB/s wr, 10.41k op/s 2024-05-21T15:39:34.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:33 smithi045 ceph-mon[36409]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 MiB/s wr, 10.41k op/s 2024-05-21T15:39:34.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:33 smithi038 ceph-mon[31995]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 MiB/s wr, 10.41k op/s 2024-05-21T15:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.059+0000 7f937214b640 1 -- 172.21.15.38:0/70884653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f936c100d30 msgr2=0x7f936c101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.059+0000 7f937214b640 1 --2- 172.21.15.38:0/70884653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f936c100d30 0x7f936c101180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f935c009900 tx=0x7f935c02a3f0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.059+0000 7f937214b640 1 -- 172.21.15.38:0/70884653 shutdown_connections 2024-05-21T15:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.059+0000 7f937214b640 1 --2- 172.21.15.38:0/70884653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f936c1068e0 0x7f936c106cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.059+0000 7f937214b640 1 --2- 172.21.15.38:0/70884653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f936c101770 0x7f936c101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.059+0000 7f937214b640 1 --2- 172.21.15.38:0/70884653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f936c100d30 0x7f936c101180 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.059+0000 7f937214b640 1 -- 172.21.15.38:0/70884653 >> 172.21.15.38:0/70884653 conn(0x7f936c0fc300 msgr2=0x7f936c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.059+0000 7f937214b640 1 -- 172.21.15.38:0/70884653 shutdown_connections 2024-05-21T15:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.059+0000 7f937214b640 1 -- 172.21.15.38:0/70884653 wait complete. 2024-05-21T15:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.060+0000 7f937214b640 1 Processor -- start 2024-05-21T15:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.060+0000 7f937214b640 1 -- start start 2024-05-21T15:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.061+0000 7f937214b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f936c101770 0x7f936c19f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.061+0000 7f937214b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f936c1068e0 0x7f936c19ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.061+0000 7f937214b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f936c1a4eb0 0x7f936c1a72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.061+0000 7f937214b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f936c114e10 con 0x7f936c1068e0 2024-05-21T15:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.061+0000 7f937214b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f936c114c90 con 0x7f936c1a4eb0 2024-05-21T15:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.061+0000 7f937214b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f936c114f90 con 0x7f936c101770 2024-05-21T15:39:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.061+0000 7f936bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f936c1a4eb0 0x7f936c1a72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.061+0000 7f936affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f936c1068e0 0x7f936c19ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.061+0000 7f936affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f936c1068e0 0x7f936c19ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33980/0 (socket says 172.21.15.38:33980) 2024-05-21T15:39:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.061+0000 7f936bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f936c1a4eb0 0x7f936c1a72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50020/0 (socket says 172.21.15.38:50020) 2024-05-21T15:39:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.061+0000 7f936affd640 1 -- 172.21.15.38:0/3248487577 learned_addr learned my addr 172.21.15.38:0/3248487577 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.061+0000 7f936b7fe640 1 --2- 172.21.15.38:0/3248487577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f936c101770 0x7f936c19f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.062+0000 7f936bfff640 1 -- 172.21.15.38:0/3248487577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f936c101770 msgr2=0x7f936c19f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.062+0000 7f936bfff640 1 --2- 172.21.15.38:0/3248487577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f936c101770 0x7f936c19f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.062+0000 7f936bfff640 1 -- 172.21.15.38:0/3248487577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f936c1068e0 msgr2=0x7f936c19ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.062+0000 7f936bfff640 1 --2- 172.21.15.38:0/3248487577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f936c1068e0 0x7f936c19ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.062+0000 7f936bfff640 1 -- 172.21.15.38:0/3248487577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936c1a7a00 con 0x7f936c1a4eb0 2024-05-21T15:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.062+0000 7f936bfff640 1 --2- 172.21.15.38:0/3248487577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f936c1a4eb0 0x7f936c1a72d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f936400be20 tx=0x7f936400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.062+0000 7f9368ff9640 1 -- 172.21.15.38:0/3248487577 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364019070 con 0x7f936c1a4eb0 2024-05-21T15:39:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.063+0000 7f937214b640 1 -- 172.21.15.38:0/3248487577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936c1a7cf0 con 0x7f936c1a4eb0 2024-05-21T15:39:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.063+0000 7f937214b640 1 -- 172.21.15.38:0/3248487577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936c1a8230 con 0x7f936c1a4eb0 2024-05-21T15:39:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.063+0000 7f9368ff9640 1 -- 172.21.15.38:0/3248487577 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9364014920 con 0x7f936c1a4eb0 2024-05-21T15:39:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.063+0000 7f9368ff9640 1 -- 172.21.15.38:0/3248487577 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364012880 con 0x7f936c1a4eb0 2024-05-21T15:39:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.069+0000 7f9368ff9640 1 -- 172.21.15.38:0/3248487577 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f936401c070 con 0x7f936c1a4eb0 2024-05-21T15:39:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.070+0000 7f9378ff1740 1 Processor -- start 2024-05-21T15:39:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.070+0000 7f9378ff1740 1 -- start start 2024-05-21T15:39:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.070+0000 7f9378ff1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac71e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.070+0000 7f9378ff1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 0x560c3ac723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.070+0000 7f9378ff1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 0x560c3ac792e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.070+0000 7f9378ff1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560c3abd6340 con 0x560c3a8a92f0 2024-05-21T15:39:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.070+0000 7f9378ff1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560c3ab7e290 con 0x560c3a8b14d0 2024-05-21T15:39:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.070+0000 7f9378ff1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560c3abd2300 con 0x560c3a8d21b0 2024-05-21T15:39:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.071+0000 7f936affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac71e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.071+0000 7f936b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 0x560c3ac723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.071+0000 7f936b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 0x560c3ac723e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33996/0 (socket says 172.21.15.38:33996) 2024-05-21T15:39:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.071+0000 7f936b7fe640 1 -- 172.21.15.38:0/1487307506 learned_addr learned my addr 172.21.15.38:0/1487307506 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.071+0000 7f936affd640 1 --2- 172.21.15.38:0/1487307506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 0x560c3ac792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.072+0000 7f9368ff9640 1 --2- 172.21.15.38:0/3248487577 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f93540a2cb0 0x7f93540a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.072+0000 7f9368ff9640 1 -- 172.21.15.38:0/3248487577 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f93640cf890 con 0x7f936c1a4eb0 2024-05-21T15:39:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.072+0000 7f936affd640 1 -- 172.21.15.38:0/1487307506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 msgr2=0x560c3ac792e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.072+0000 7f936affd640 1 --2- 172.21.15.38:0/1487307506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 0x560c3ac792e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.072+0000 7f936affd640 1 -- 172.21.15.38:0/1487307506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 msgr2=0x560c3ac723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.072+0000 7f936affd640 1 --2- 172.21.15.38:0/1487307506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 0x560c3ac723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.072+0000 7f936affd640 1 -- 172.21.15.38:0/1487307506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c3aba28e0 con 0x560c3a8d21b0 2024-05-21T15:39:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.074+0000 7f936bfff640 1 --2- 172.21.15.38:0/3248487577 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f93540a2cb0 0x7f93540a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.074+0000 7f936affd640 1 --2- 172.21.15.38:0/1487307506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac71e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9360009ce0 tx=0x7f93600094c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69bcd52631a9c3b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.074+0000 7f936bfff640 1 --2- 172.21.15.38:0/3248487577 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f93540a2cb0 0x7f93540a51c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f93640a4780 tx=0x7f93640093d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.075+0000 7f9351ffb640 1 -- 172.21.15.38:0/1487307506 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360015620 con 0x560c3a8d21b0 2024-05-21T15:39:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.075+0000 7f9351ffb640 1 -- 172.21.15.38:0/1487307506 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936001a070 con 0x560c3a8d21b0 2024-05-21T15:39:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.076+0000 7f9351ffb640 1 -- 172.21.15.38:0/1487307506 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600148c0 con 0x560c3a8d21b0 2024-05-21T15:39:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.076+0000 7f9378ff1740 1 -- 172.21.15.38:0/1487307506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 msgr2=0x560c3ac71e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.077+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1487307506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac71e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9360009ce0 tx=0x7f93600094c0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.079+0000 7f9378ff1740 1 -- 172.21.15.38:0/1487307506 shutdown_connections 2024-05-21T15:39:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.080+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1487307506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 0x560c3ac792e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.080+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1487307506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 0x560c3ac723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.080+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1487307506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac71e60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.080+0000 7f9378ff1740 1 -- 172.21.15.38:0/1487307506 >> 172.21.15.38:0/1487307506 conn(0x560c3a8c6300 msgr2=0x560c3ac6d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.082+0000 7f9378ff1740 1 -- 172.21.15.38:0/1487307506 shutdown_connections 2024-05-21T15:39:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.082+0000 7f9378ff1740 1 -- 172.21.15.38:0/1487307506 wait complete. 2024-05-21T15:39:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.083+0000 7f9378ff1740 1 -- start start 2024-05-21T15:39:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.083+0000 7f9378ff1740 1 Processor -- start 2024-05-21T15:39:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.083+0000 7f9378ff1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 0x560c3ac8ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.084+0000 7f9378ff1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 0x560c3ac8d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.084+0000 7f9378ff1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac94150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.084+0000 7f9378ff1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560c3ac8ad90 con 0x560c3a8a92f0 2024-05-21T15:39:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.084+0000 7f9378ff1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560c3ab91790 con 0x560c3a8b14d0 2024-05-21T15:39:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.084+0000 7f9378ff1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560c3ac8af10 con 0x560c3a8d21b0 2024-05-21T15:39:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.084+0000 7f9378ff1740 10 client.0 _ll_register_callbacks cb 0x560c3abcca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.086+0000 7f936affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac94150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.086+0000 7f936affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac94150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:36444/0 (socket says 172.21.15.38:36444) 2024-05-21T15:39:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.086+0000 7f936affd640 1 -- 172.21.15.38:0/1494104890 learned_addr learned my addr 172.21.15.38:0/1494104890 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.086+0000 7f936affd640 1 --2- 172.21.15.38:0/1494104890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 0x560c3ac8ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.086+0000 7f936b7fe640 1 --2- 172.21.15.38:0/1494104890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 0x560c3ac8d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.087+0000 7f936affd640 1 -- 172.21.15.38:0/1494104890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 msgr2=0x560c3ac8d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.087+0000 7f936affd640 1 --2- 172.21.15.38:0/1494104890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 0x560c3ac8d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.087+0000 7f936affd640 1 -- 172.21.15.38:0/1494104890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 msgr2=0x560c3ac8ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.087+0000 7f936affd640 1 --2- 172.21.15.38:0/1494104890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 0x560c3ac8ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.087+0000 7f936affd640 1 -- 172.21.15.38:0/1494104890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c3a8ea210 con 0x560c3a8d21b0 2024-05-21T15:39:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.090+0000 7f936affd640 1 --2- 172.21.15.38:0/1494104890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac94150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9360009a00 tx=0x7f9360014380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cdc7dfa2c34ffc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.090+0000 7f934affd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:39:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.094+0000 7f934affd640 1 -- 172.21.15.38:0/1494104890 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360014c40 con 0x560c3a8d21b0 2024-05-21T15:39:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.094+0000 7f9378ff1740 1 -- 172.21.15.38:0/1494104890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936001a070 con 0x560c3a8d21b0 2024-05-21T15:39:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.094+0000 7f934affd640 1 -- 172.21.15.38:0/1494104890 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9360014de0 con 0x560c3a8d21b0 2024-05-21T15:39:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.095+0000 7f934affd640 1 -- 172.21.15.38:0/1494104890 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9360030050 con 0x560c3a8d21b0 2024-05-21T15:39:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.097+0000 7f9378ff1740 10 client.26479 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.097+0000 7f9378ff1740 1 -- 172.21.15.38:0/1494104890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560c3a9b6620 con 0x560c3a8d21b0 2024-05-21T15:39:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.101+0000 7f934affd640 1 -- 172.21.15.38:0/1494104890 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936001a070 con 0x560c3a8d21b0 2024-05-21T15:39:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.101+0000 7f934affd640 1 -- 172.21.15.38:0/1494104890 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f936002b690 con 0x560c3a8d21b0 2024-05-21T15:39:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.101+0000 7f9378ff1740 10 client.26479 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.101+0000 7f9378ff1740 10 client.26479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.101+0000 7f9378ff1740 20 client.26479 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.102+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1494104890 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c3a890d80 0x560c3ac9aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.102+0000 7f9378ff1740 4 client.26479 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.102+0000 7f9378ff1740 1 -- 172.21.15.38:0/1494104890 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c3abe3a70 con 0x560c3a890d80 2024-05-21T15:39:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.103+0000 7f936b7fe640 1 --2- 172.21.15.38:0/1494104890 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c3a890d80 0x560c3ac9aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.105+0000 7f936b7fe640 1 --2- 172.21.15.38:0/1494104890 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c3a890d80 0x560c3ac9aad0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f40490a0df5eea8 server_cookie=d10b511cb7ca6ea7 in_seq=0 out_seq=0 2024-05-21T15:39:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.108+0000 7f934affd640 10 client.26479 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.108+0000 7f934affd640 1 -- 172.21.15.38:0/1494104890 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560c3abe3a70 con 0x560c3a890d80 2024-05-21T15:39:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.108+0000 7f934affd640 10 client.26479 handle_command_reply: tid=0 2024-05-21T15:39:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.108+0000 7f9378ff1740 1 client.26479 shutdown 2024-05-21T15:39:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.113+0000 7f9378ff1740 1 -- 172.21.15.38:0/1494104890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 msgr2=0x560c3ac94150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.113+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1494104890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac94150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9360009a00 tx=0x7f9360014380 comp rx=0 tx=0).stop 2024-05-21T15:39:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.114+0000 7f9378ff1740 1 -- 172.21.15.38:0/1494104890 shutdown_connections 2024-05-21T15:39:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.114+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1494104890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac94150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.114+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1494104890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 0x560c3ac8d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.114+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1494104890 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c3a890d80 0x560c3ac9aad0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.114+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1494104890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 0x560c3ac8ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.114+0000 7f9378ff1740 1 -- 172.21.15.38:0/1494104890 >> 172.21.15.38:0/1494104890 conn(0x560c3a8c6300 msgr2=0x560c3ac6c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.114+0000 7f934affd640 0 client.26479 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.115+0000 7f9378ff1740 1 -- 172.21.15.38:0/1494104890 shutdown_connections 2024-05-21T15:39:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.115+0000 7f9378ff1740 1 -- 172.21.15.38:0/1494104890 wait complete. 2024-05-21T15:39:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.115+0000 7f9378ff1740 20 client.26479 trim_cache size 0 max 16384 2024-05-21T15:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.151+0000 7f9378ff1740 1 Processor -- start 2024-05-21T15:39:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.151+0000 7f9378ff1740 1 -- start start 2024-05-21T15:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.151+0000 7f9378ff1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 0x560c3ac6d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.151+0000 7f9378ff1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 0x560c3ac6d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.151+0000 7f9378ff1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac94070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.151+0000 7f9378ff1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f935800e6d0 con 0x560c3a8a92f0 2024-05-21T15:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.151+0000 7f9378ff1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9358010300 con 0x560c3a8b14d0 2024-05-21T15:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.151+0000 7f9378ff1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f935800eda0 con 0x560c3a8d21b0 2024-05-21T15:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.151+0000 7f936affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 0x560c3ac6d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.152+0000 7f936affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 0x560c3ac6d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34014/0 (socket says 172.21.15.38:34014) 2024-05-21T15:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.152+0000 7f936affd640 1 -- 172.21.15.38:0/2722993691 learned_addr learned my addr 172.21.15.38:0/2722993691 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.152+0000 7f936b7fe640 1 --2- 172.21.15.38:0/2722993691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 0x560c3ac6d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.153+0000 7f936affd640 1 --2- 172.21.15.38:0/2722993691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac94070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.153+0000 7f936b7fe640 1 -- 172.21.15.38:0/2722993691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 msgr2=0x560c3ac94070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.153+0000 7f936b7fe640 1 --2- 172.21.15.38:0/2722993691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac94070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.153+0000 7f936b7fe640 1 -- 172.21.15.38:0/2722993691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 msgr2=0x560c3ac6d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.153+0000 7f936b7fe640 1 --2- 172.21.15.38:0/2722993691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 0x560c3ac6d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.153+0000 7f936b7fe640 1 -- 172.21.15.38:0/2722993691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c3ac68b90 con 0x560c3a8b14d0 2024-05-21T15:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.156+0000 7f936b7fe640 1 --2- 172.21.15.38:0/2722993691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 0x560c3ac6d6a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f935c02a9e0 tx=0x7f935c0315b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd0706527c477e8e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.156+0000 7f93517fa640 1 -- 172.21.15.38:0/2722993691 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c048020 con 0x560c3a8b14d0 2024-05-21T15:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.156+0000 7f93517fa640 1 -- 172.21.15.38:0/2722993691 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f935c0047a0 con 0x560c3a8b14d0 2024-05-21T15:39:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.156+0000 7f93517fa640 1 -- 172.21.15.38:0/2722993691 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c035040 con 0x560c3a8b14d0 2024-05-21T15:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.158+0000 7f9378ff1740 1 -- 172.21.15.38:0/2722993691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 msgr2=0x560c3ac6d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.158+0000 7f9378ff1740 1 --2- 172.21.15.38:0/2722993691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 0x560c3ac6d6a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f935c02a9e0 tx=0x7f935c0315b0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.158+0000 7f9378ff1740 1 -- 172.21.15.38:0/2722993691 shutdown_connections 2024-05-21T15:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.159+0000 7f9378ff1740 1 --2- 172.21.15.38:0/2722993691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8d21b0 0x560c3ac94070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.159+0000 7f9378ff1740 1 --2- 172.21.15.38:0/2722993691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8b14d0 0x560c3ac6d6a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.159+0000 7f9378ff1740 1 --2- 172.21.15.38:0/2722993691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8a92f0 0x560c3ac6d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.159+0000 7f9378ff1740 1 -- 172.21.15.38:0/2722993691 >> 172.21.15.38:0/2722993691 conn(0x560c3a8c6300 msgr2=0x560c3ac75fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.159+0000 7f9378ff1740 1 -- 172.21.15.38:0/2722993691 shutdown_connections 2024-05-21T15:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.159+0000 7f9378ff1740 1 -- 172.21.15.38:0/2722993691 wait complete. 2024-05-21T15:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.160+0000 7f9378ff1740 1 -- start start 2024-05-21T15:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.160+0000 7f9378ff1740 1 Processor -- start 2024-05-21T15:39:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.160+0000 7f9378ff1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8a92f0 0x560c3ac8f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.160+0000 7f9378ff1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8b14d0 0x560c3ac8fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f9378ff1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8d21b0 0x560c3ad00e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f9378ff1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f935800fb70 con 0x560c3a8b14d0 2024-05-21T15:39:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f9378ff1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f935800f470 con 0x560c3a8d21b0 2024-05-21T15:39:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f9378ff1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f935800c6b0 con 0x560c3a8a92f0 2024-05-21T15:39:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f936b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8b14d0 0x560c3ac8fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f9378ff1740 10 client.0 _ll_register_callbacks cb 0x7f9360009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f936b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8b14d0 0x560c3ac8fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34028/0 (socket says 172.21.15.38:34028) 2024-05-21T15:39:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f936b7fe640 1 -- 172.21.15.38:0/1480616578 learned_addr learned my addr 172.21.15.38:0/1480616578 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f936affd640 1 --2- 172.21.15.38:0/1480616578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8a92f0 0x560c3ac8f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f936affd640 1 --2- 172.21.15.38:0/1480616578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8d21b0 0x560c3ad00e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f936b7fe640 1 -- 172.21.15.38:0/1480616578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8a92f0 msgr2=0x560c3ac8f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f936b7fe640 1 --2- 172.21.15.38:0/1480616578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8a92f0 0x560c3ac8f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f936b7fe640 1 -- 172.21.15.38:0/1480616578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8d21b0 msgr2=0x560c3ad00e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f936b7fe640 1 --2- 172.21.15.38:0/1480616578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8d21b0 0x560c3ad00e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.161+0000 7f936b7fe640 1 -- 172.21.15.38:0/1480616578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c3ac94bb0 con 0x560c3a8b14d0 2024-05-21T15:39:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.162+0000 7f936b7fe640 1 --2- 172.21.15.38:0/1480616578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8b14d0 0x560c3ac8fc50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f935c0057c0 tx=0x7f935c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d155d6d66fc344b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.162+0000 7f934affd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.162+0000 7f934affd640 1 -- 172.21.15.38:0/1480616578 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c048070 con 0x560c3a8b14d0 2024-05-21T15:39:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.162+0000 7f934affd640 1 -- 172.21.15.38:0/1480616578 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f935c005200 con 0x560c3a8b14d0 2024-05-21T15:39:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.163+0000 7f9378ff1740 1 -- 172.21.15.38:0/1480616578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f935c0047a0 con 0x560c3a8b14d0 2024-05-21T15:39:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.163+0000 7f934affd640 1 -- 172.21.15.38:0/1480616578 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f935c035050 con 0x560c3a8b14d0 2024-05-21T15:39:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.163+0000 7f9378ff1740 10 client.16773 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.163+0000 7f9378ff1740 1 -- 172.21.15.38:0/1480616578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560c3ad022a0 con 0x560c3a8b14d0 2024-05-21T15:39:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.164+0000 7f934affd640 1 -- 172.21.15.38:0/1480616578 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f935c051070 con 0x560c3a8b14d0 2024-05-21T15:39:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.165+0000 7f934affd640 1 -- 172.21.15.38:0/1480616578 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f935c031830 con 0x560c3a8b14d0 2024-05-21T15:39:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.166+0000 7f9378ff1740 10 client.16773 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.166+0000 7f9378ff1740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.166+0000 7f9378ff1740 20 client.16773 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.166+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1480616578 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c3a8661c0 0x560c3ad04d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.166+0000 7f9378ff1740 4 client.16773 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:39:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.166+0000 7f9378ff1740 1 -- 172.21.15.38:0/1480616578 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f935c0047a0 con 0x560c3a8661c0 2024-05-21T15:39:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.166+0000 7f936affd640 1 --2- 172.21.15.38:0/1480616578 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c3a8661c0 0x560c3ad04d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.168+0000 7f936affd640 1 --2- 172.21.15.38:0/1480616578 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c3a8661c0 0x560c3ad04d50 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5cba6da4343d8ec server_cookie=ba013cb1e66d6d70 in_seq=0 out_seq=0 2024-05-21T15:39:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.168+0000 7f934affd640 10 client.16773 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.741+0000 7f934affd640 1 -- 172.21.15.38:0/1480616578 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f935c0047a0 con 0x560c3a8661c0 2024-05-21T15:39:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.741+0000 7f934affd640 10 client.16773 handle_command_reply: tid=0 2024-05-21T15:39:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.741+0000 7f9378ff1740 1 client.16773 shutdown 2024-05-21T15:39:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.742+0000 7f9378ff1740 1 -- 172.21.15.38:0/1480616578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8b14d0 msgr2=0x560c3ac8fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.742+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1480616578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8b14d0 0x560c3ac8fc50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f935c0057c0 tx=0x7f935c000bc0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.742+0000 7f9378ff1740 1 -- 172.21.15.38:0/1480616578 shutdown_connections 2024-05-21T15:39:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.742+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1480616578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c3a8d21b0 0x560c3ad00e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.742+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1480616578 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c3a8661c0 0x560c3ad04d50 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.742+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1480616578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c3a8b14d0 0x560c3ac8fc50 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.742+0000 7f934affd640 0 client.16773 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.742+0000 7f9378ff1740 1 --2- 172.21.15.38:0/1480616578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c3a8a92f0 0x560c3ac8f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.743+0000 7f9378ff1740 1 -- 172.21.15.38:0/1480616578 >> 172.21.15.38:0/1480616578 conn(0x560c3a8c6300 msgr2=0x560c3ac6d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.743+0000 7f9378ff1740 1 -- 172.21.15.38:0/1480616578 shutdown_connections 2024-05-21T15:39:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.743+0000 7f9378ff1740 1 -- 172.21.15.38:0/1480616578 wait complete. 2024-05-21T15:39:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.743+0000 7f9378ff1740 20 client.16773 trim_cache size 0 max 16384 2024-05-21T15:39:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:39:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:39:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:39:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:39:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.744+0000 7f937214b640 1 -- 172.21.15.38:0/3248487577 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f93540a2cb0 msgr2=0x7f93540a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.744+0000 7f937214b640 1 --2- 172.21.15.38:0/3248487577 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f93540a2cb0 0x7f93540a51c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f93640a4780 tx=0x7f93640093d0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.744+0000 7f937214b640 1 -- 172.21.15.38:0/3248487577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f936c1a4eb0 msgr2=0x7f936c1a72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.744+0000 7f937214b640 1 --2- 172.21.15.38:0/3248487577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f936c1a4eb0 0x7f936c1a72d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f936400be20 tx=0x7f936400bf10 comp rx=0 tx=0).stop 2024-05-21T15:39:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.744+0000 7f937214b640 1 -- 172.21.15.38:0/3248487577 shutdown_connections 2024-05-21T15:39:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.744+0000 7f937214b640 1 --2- 172.21.15.38:0/3248487577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f936c1a4eb0 0x7f936c1a72d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.744+0000 7f937214b640 1 --2- 172.21.15.38:0/3248487577 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f93540a2cb0 0x7f93540a51c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.744+0000 7f937214b640 1 --2- 172.21.15.38:0/3248487577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f936c1068e0 0x7f936c19ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.744+0000 7f937214b640 1 --2- 172.21.15.38:0/3248487577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f936c101770 0x7f936c19f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.744+0000 7f937214b640 1 -- 172.21.15.38:0/3248487577 >> 172.21.15.38:0/3248487577 conn(0x7f936c0fc300 msgr2=0x7f936c1028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.744+0000 7f937214b640 1 -- 172.21.15.38:0/3248487577 shutdown_connections 2024-05-21T15:39:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:35.744+0000 7f937214b640 1 -- 172.21.15.38:0/3248487577 wait complete. 2024-05-21T15:39:36.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:35 smithi053 ceph-mon[36496]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 51 MiB/s wr, 11.33k op/s 2024-05-21T15:39:36.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:35 smithi045 ceph-mon[36409]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 51 MiB/s wr, 11.33k op/s 2024-05-21T15:39:36.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:35 smithi038 ceph-mon[31995]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 51 MiB/s wr, 11.33k op/s 2024-05-21T15:39:38.061 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:37 smithi045 ceph-mon[36409]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 MiB/s wr, 9.77k op/s 2024-05-21T15:39:38.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:37 smithi053 ceph-mon[36496]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 MiB/s wr, 9.77k op/s 2024-05-21T15:39:38.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:37 smithi038 ceph-mon[31995]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 MiB/s wr, 9.77k op/s 2024-05-21T15:39:40.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:39 smithi053 ceph-mon[36496]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 8.75k op/s 2024-05-21T15:39:40.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:39 smithi045 ceph-mon[36409]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 8.75k op/s 2024-05-21T15:39:40.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:39 smithi038 ceph-mon[31995]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 8.75k op/s 2024-05-21T15:39:40.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:39:40.485 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:39:40.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.713+0000 7fedd088d640 1 -- 172.21.15.38:0/1752979632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedcc107900 msgr2=0x7fedcc109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:40.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.713+0000 7fedd088d640 1 --2- 172.21.15.38:0/1752979632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedcc107900 0x7fedcc109d20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00d910 tx=0x7fedbc02c390 comp rx=0 tx=0).stop 2024-05-21T15:39:40.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.714+0000 7fedd088d640 1 -- 172.21.15.38:0/1752979632 shutdown_connections 2024-05-21T15:39:40.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.714+0000 7fedd088d640 1 --2- 172.21.15.38:0/1752979632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedcc107900 0x7fedcc109d20 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.714+0000 7fedd088d640 1 --2- 172.21.15.38:0/1752979632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedcc102c00 0x7fedcc103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.714+0000 7fedd088d640 1 --2- 172.21.15.38:0/1752979632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedcc1019c0 0x7fedcc1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.714+0000 7fedd088d640 1 -- 172.21.15.38:0/1752979632 >> 172.21.15.38:0/1752979632 conn(0x7fedcc0fcbb0 msgr2=0x7fedcc0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:40.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.715+0000 7fedd088d640 1 -- 172.21.15.38:0/1752979632 shutdown_connections 2024-05-21T15:39:40.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.715+0000 7fedd088d640 1 -- 172.21.15.38:0/1752979632 wait complete. 2024-05-21T15:39:40.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.716+0000 7fedd088d640 1 Processor -- start 2024-05-21T15:39:40.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.716+0000 7fedd088d640 1 -- start start 2024-05-21T15:39:40.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.716+0000 7fedd088d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedcc1019c0 0x7fedcc073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.716+0000 7fedd088d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedcc102c00 0x7fedcc071ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.716+0000 7fedd088d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedcc107900 0x7fedcc0720e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.716+0000 7fedd088d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc06bf20 con 0x7fedcc102c00 2024-05-21T15:39:40.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.716+0000 7fedd088d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc06bda0 con 0x7fedcc107900 2024-05-21T15:39:40.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.716+0000 7fedd088d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc06c0a0 con 0x7fedcc1019c0 2024-05-21T15:39:40.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.716+0000 7fedcad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedcc107900 0x7fedcc0720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.717+0000 7fedcad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedcc107900 0x7fedcc0720e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50096/0 (socket says 172.21.15.38:50096) 2024-05-21T15:39:40.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.717+0000 7fedcad9d640 1 -- 172.21.15.38:0/129313461 learned_addr learned my addr 172.21.15.38:0/129313461 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:40.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.717+0000 7fedca59c640 1 --2- 172.21.15.38:0/129313461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedcc1019c0 0x7fedcc073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.717+0000 7fedcad9d640 1 -- 172.21.15.38:0/129313461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedcc1019c0 msgr2=0x7fedcc073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:40.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.717+0000 7fedcad9d640 1 --2- 172.21.15.38:0/129313461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedcc1019c0 0x7fedcc073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.717+0000 7fedcad9d640 1 -- 172.21.15.38:0/129313461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedcc102c00 msgr2=0x7fedcc071ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:40.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.717+0000 7fedcad9d640 1 --2- 172.21.15.38:0/129313461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedcc102c00 0x7fedcc071ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.717+0000 7fedcad9d640 1 -- 172.21.15.38:0/129313461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedcc072750 con 0x7fedcc107900 2024-05-21T15:39:40.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.717+0000 7fedcad9d640 1 --2- 172.21.15.38:0/129313461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedcc107900 0x7fedcc0720e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00d540 tx=0x7fedbc034790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:40.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.718+0000 7fedc37fe640 1 -- 172.21.15.38:0/129313461 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc034c00 con 0x7fedcc107900 2024-05-21T15:39:40.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.718+0000 7fedd088d640 1 -- 172.21.15.38:0/129313461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedcc0729e0 con 0x7fedcc107900 2024-05-21T15:39:40.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.718+0000 7fedd088d640 1 -- 172.21.15.38:0/129313461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedcc1a7c70 con 0x7fedcc107900 2024-05-21T15:39:40.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.719+0000 7fedc37fe640 1 -- 172.21.15.38:0/129313461 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedbc041070 con 0x7fedcc107900 2024-05-21T15:39:40.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.719+0000 7fedc37fe640 1 -- 172.21.15.38:0/129313461 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc033620 con 0x7fedcc107900 2024-05-21T15:39:40.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.723+0000 7fedd780c740 1 Processor -- start 2024-05-21T15:39:40.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.725+0000 7fedd780c740 1 -- start start 2024-05-21T15:39:40.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.726+0000 7fedd780c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 0x556301055bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.726+0000 7fedd780c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x556301056130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.726+0000 7fedd780c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c956b0 0x55630105d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.726+0000 7fedd780c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556300fbad00 con 0x556300c956b0 2024-05-21T15:39:40.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.726+0000 7fedd780c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556300f624c0 con 0x556300c8d4d0 2024-05-21T15:39:40.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.726+0000 7fedd780c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556300fb6cc0 con 0x556300cb6390 2024-05-21T15:39:40.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.729+0000 7fedca59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x556301056130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.729+0000 7fedc9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c956b0 0x55630105d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.729+0000 7fedc9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c956b0 0x55630105d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34040/0 (socket says 172.21.15.38:34040) 2024-05-21T15:39:40.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.729+0000 7fedc9d9b640 1 -- 172.21.15.38:0/1223935888 learned_addr learned my addr 172.21.15.38:0/1223935888 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:40.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.729+0000 7fedc37fe640 1 -- 172.21.15.38:0/129313461 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fedbc00c050 con 0x7fedcc107900 2024-05-21T15:39:40.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.729+0000 7fedc9d9b640 1 --2- 172.21.15.38:0/1223935888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 0x556301055bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.730+0000 7fedc37fe640 1 --2- 172.21.15.38:0/129313461 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fedb00a2d10 0x7fedb00a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.730+0000 7fedc9d9b640 1 -- 172.21.15.38:0/1223935888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 msgr2=0x556301056130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:40.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.730+0000 7fedc37fe640 1 -- 172.21.15.38:0/129313461 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fedbc0eee30 con 0x7fedcc107900 2024-05-21T15:39:40.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.730+0000 7fedc9d9b640 1 --2- 172.21.15.38:0/1223935888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x556301056130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.730+0000 7fedc9d9b640 1 -- 172.21.15.38:0/1223935888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c956b0 msgr2=0x55630105d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:40.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.730+0000 7fedc9d9b640 1 --2- 172.21.15.38:0/1223935888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c956b0 0x55630105d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.730+0000 7fedc9d9b640 1 -- 172.21.15.38:0/1223935888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556300d987d0 con 0x556300cb6390 2024-05-21T15:39:40.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.733+0000 7fedcad9d640 1 --2- 172.21.15.38:0/129313461 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fedb00a2d10 0x7fedb00a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.734+0000 7fedc9d9b640 1 --2- 172.21.15.38:0/1223935888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 0x556301055bb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fedc4012a70 tx=0x7fedc4005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d574145d19fce552 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:40.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.734+0000 7fedcad9d640 1 --2- 172.21.15.38:0/129313461 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fedb00a2d10 0x7fedb00a5220 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0bb5d0 tx=0x7fedbc02c390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:40.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.737+0000 7fedc0ff9640 1 -- 172.21.15.38:0/1223935888 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4004d20 con 0x556300cb6390 2024-05-21T15:39:40.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.738+0000 7fedc0ff9640 1 -- 172.21.15.38:0/1223935888 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc400fd00 con 0x556300cb6390 2024-05-21T15:39:40.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.738+0000 7fedc0ff9640 1 -- 172.21.15.38:0/1223935888 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc40058b0 con 0x556300cb6390 2024-05-21T15:39:40.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.739+0000 7fedd780c740 1 -- 172.21.15.38:0/1223935888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 msgr2=0x556301055bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:40.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.739+0000 7fedd780c740 1 --2- 172.21.15.38:0/1223935888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 0x556301055bb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fedc4012a70 tx=0x7fedc4005bd0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.742+0000 7fedd780c740 1 -- 172.21.15.38:0/1223935888 shutdown_connections 2024-05-21T15:39:40.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.742+0000 7fedd780c740 1 --2- 172.21.15.38:0/1223935888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c956b0 0x55630105d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.742+0000 7fedd780c740 1 --2- 172.21.15.38:0/1223935888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x556301056130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.742+0000 7fedd780c740 1 --2- 172.21.15.38:0/1223935888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 0x556301055bb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.742+0000 7fedd780c740 1 -- 172.21.15.38:0/1223935888 >> 172.21.15.38:0/1223935888 conn(0x556300caa4e0 msgr2=0x556301051540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:40.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.745+0000 7fedd780c740 1 -- 172.21.15.38:0/1223935888 shutdown_connections 2024-05-21T15:39:40.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.745+0000 7fedd780c740 1 -- 172.21.15.38:0/1223935888 wait complete. 2024-05-21T15:39:40.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.746+0000 7fedd780c740 1 -- start start 2024-05-21T15:39:40.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.746+0000 7fedd780c740 1 Processor -- start 2024-05-21T15:39:40.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.749+0000 7fedd780c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x55630105fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.749+0000 7fedd780c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300c956b0 0x556301060100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.749+0000 7fedd780c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300cb6390 0x556301067060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.749+0000 7fedd780c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55630105da80 con 0x556300cb6390 2024-05-21T15:39:40.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.749+0000 7fedd780c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556300f75b90 con 0x556300c8d4d0 2024-05-21T15:39:40.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.749+0000 7fedd780c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55630105dc00 con 0x556300c956b0 2024-05-21T15:39:40.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.749+0000 7fedd780c740 10 client.0 _ll_register_callbacks cb 0x556300fb1da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:40.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.750+0000 7fedc9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300cb6390 0x556301067060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.750+0000 7fedc9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300cb6390 0x556301067060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34046/0 (socket says 172.21.15.38:34046) 2024-05-21T15:39:40.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.750+0000 7fedc9d9b640 1 -- 172.21.15.38:0/3622043764 learned_addr learned my addr 172.21.15.38:0/3622043764 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:40.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.750+0000 7fedc9d9b640 1 --2- 172.21.15.38:0/3622043764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x55630105fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.751+0000 7fedc9d9b640 1 -- 172.21.15.38:0/3622043764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300c956b0 msgr2=0x556301060100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:39:40.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.751+0000 7fedc9d9b640 1 --2- 172.21.15.38:0/3622043764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300c956b0 0x556301060100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.751+0000 7fedc9d9b640 1 -- 172.21.15.38:0/3622043764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300cb6390 msgr2=0x556301067060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:40.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.751+0000 7fedc9d9b640 1 --2- 172.21.15.38:0/3622043764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300cb6390 0x556301067060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.751+0000 7fedc9d9b640 1 -- 172.21.15.38:0/3622043764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556300cce3f0 con 0x556300c8d4d0 2024-05-21T15:39:40.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.752+0000 7fedc9d9b640 1 --2- 172.21.15.38:0/3622043764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x55630105fbc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fedc4005c40 tx=0x7fedc4009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1e35fb885b87c9d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:40.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.752+0000 7fedadffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:39:40.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.753+0000 7fedadffb640 1 -- 172.21.15.38:0/3622043764 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4004b20 con 0x556300c8d4d0 2024-05-21T15:39:40.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.754+0000 7fedadffb640 1 -- 172.21.15.38:0/3622043764 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fedc4004cc0 con 0x556300c8d4d0 2024-05-21T15:39:40.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.754+0000 7fedd780c740 1 -- 172.21.15.38:0/3622043764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc400fd00 con 0x556300c8d4d0 2024-05-21T15:39:40.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.755+0000 7fedadffb640 1 -- 172.21.15.38:0/3622043764 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fedc400f820 con 0x556300c8d4d0 2024-05-21T15:39:40.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.756+0000 7fedd780c740 10 client.26420 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:40.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.756+0000 7fedd780c740 1 -- 172.21.15.38:0/3622043764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556300c8e0f0 con 0x556300c8d4d0 2024-05-21T15:39:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.762+0000 7fedadffb640 1 -- 172.21.15.38:0/3622043764 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc400fd00 con 0x556300c8d4d0 2024-05-21T15:39:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.762+0000 7fedadffb640 1 -- 172.21.15.38:0/3622043764 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fedc4036660 con 0x556300c8d4d0 2024-05-21T15:39:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.762+0000 7fedd780c740 10 client.26420 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.762+0000 7fedd780c740 10 client.26420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.762+0000 7fedd780c740 20 client.26420 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.763+0000 7fedd780c740 1 --2- 172.21.15.38:0/3622043764 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556300c74f60 0x55630106d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.763+0000 7fedd780c740 4 client.26420 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.763+0000 7fedd780c740 1 -- 172.21.15.38:0/3622043764 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556300fc8430 con 0x556300c74f60 2024-05-21T15:39:40.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.766+0000 7fedca59c640 1 --2- 172.21.15.38:0/3622043764 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556300c74f60 0x55630106d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.772+0000 7fedca59c640 1 --2- 172.21.15.38:0/3622043764 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556300c74f60 0x55630106d800 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbda055afdc67ee5 server_cookie=b27c94bbd821f69e in_seq=0 out_seq=0 2024-05-21T15:39:40.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.772+0000 7fedadffb640 10 client.26420 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:40.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.774+0000 7fedadffb640 1 -- 172.21.15.38:0/3622043764 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556300fc8430 con 0x556300c74f60 2024-05-21T15:39:40.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.774+0000 7fedadffb640 10 client.26420 handle_command_reply: tid=0 2024-05-21T15:39:40.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.774+0000 7fedd780c740 1 client.26420 shutdown 2024-05-21T15:39:40.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.779+0000 7fedd780c740 1 -- 172.21.15.38:0/3622043764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 msgr2=0x55630105fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:40.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.779+0000 7fedd780c740 1 --2- 172.21.15.38:0/3622043764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x55630105fbc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fedc4005c40 tx=0x7fedc4009400 comp rx=0 tx=0).stop 2024-05-21T15:39:40.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.780+0000 7fedd780c740 1 -- 172.21.15.38:0/3622043764 shutdown_connections 2024-05-21T15:39:40.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.780+0000 7fedd780c740 1 --2- 172.21.15.38:0/3622043764 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556300c74f60 0x55630106d800 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.780+0000 7fedd780c740 1 --2- 172.21.15.38:0/3622043764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300cb6390 0x556301067060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.780+0000 7fedd780c740 1 --2- 172.21.15.38:0/3622043764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300c956b0 0x556301060100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.780+0000 7fedd780c740 1 --2- 172.21.15.38:0/3622043764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x55630105fbc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.780+0000 7fedd780c740 1 -- 172.21.15.38:0/3622043764 >> 172.21.15.38:0/3622043764 conn(0x556300caa4e0 msgr2=0x5563010505c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:40.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.780+0000 7fedadffb640 0 client.26420 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:40.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.787+0000 7fedd780c740 1 -- 172.21.15.38:0/3622043764 shutdown_connections 2024-05-21T15:39:40.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.790+0000 7fedd780c740 1 -- 172.21.15.38:0/3622043764 wait complete. 2024-05-21T15:39:40.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.791+0000 7fedd780c740 20 client.26420 trim_cache size 0 max 16384 2024-05-21T15:39:40.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.833+0000 7fedd780c740 1 Processor -- start 2024-05-21T15:39:40.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.833+0000 7fedd780c740 1 -- start start 2024-05-21T15:39:40.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.833+0000 7fedd780c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c8d4d0 0x55630105a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.833+0000 7fedd780c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c956b0 0x556301066ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.833+0000 7fedd780c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 0x55630105fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.833+0000 7fedd780c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fedb400e5d0 con 0x556300c8d4d0 2024-05-21T15:39:40.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.833+0000 7fedd780c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fedb4010200 con 0x556300c956b0 2024-05-21T15:39:40.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.833+0000 7fedd780c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fedb400eca0 con 0x556300cb6390 2024-05-21T15:39:40.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.834+0000 7fedca59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c956b0 0x556301066ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.834+0000 7fedca59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c956b0 0x556301066ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50108/0 (socket says 172.21.15.38:50108) 2024-05-21T15:39:40.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.834+0000 7fedca59c640 1 -- 172.21.15.38:0/1601330609 learned_addr learned my addr 172.21.15.38:0/1601330609 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:40.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.835+0000 7fedc9d9b640 1 --2- 172.21.15.38:0/1601330609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c8d4d0 0x55630105a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.835+0000 7fedc9d9b640 1 --2- 172.21.15.38:0/1601330609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 0x55630105fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.835+0000 7fedca59c640 1 -- 172.21.15.38:0/1601330609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 msgr2=0x55630105fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:40.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.835+0000 7fedca59c640 1 --2- 172.21.15.38:0/1601330609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 0x55630105fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.835+0000 7fedca59c640 1 -- 172.21.15.38:0/1601330609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c8d4d0 msgr2=0x55630105a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:40.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.835+0000 7fedca59c640 1 --2- 172.21.15.38:0/1601330609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c8d4d0 0x55630105a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.835+0000 7fedca59c640 1 -- 172.21.15.38:0/1601330609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556301042990 con 0x556300c956b0 2024-05-21T15:39:40.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.835+0000 7fedca59c640 1 --2- 172.21.15.38:0/1601330609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c956b0 0x556301066ff0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fedcc1025c0 tx=0x7fedb8008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ca8ecfc7a37b3f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:40.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.835+0000 7fedaffff640 1 -- 172.21.15.38:0/1601330609 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb8014590 con 0x556300c956b0 2024-05-21T15:39:40.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.836+0000 7fedaffff640 1 -- 172.21.15.38:0/1601330609 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedb8023070 con 0x556300c956b0 2024-05-21T15:39:40.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.837+0000 7fedd780c740 1 -- 172.21.15.38:0/1601330609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c956b0 msgr2=0x556301066ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:40.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.837+0000 7fedd780c740 1 --2- 172.21.15.38:0/1601330609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c956b0 0x556301066ff0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fedcc1025c0 tx=0x7fedb8008040 comp rx=0 tx=0).stop 2024-05-21T15:39:40.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.840+0000 7fedd780c740 1 -- 172.21.15.38:0/1601330609 shutdown_connections 2024-05-21T15:39:40.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.840+0000 7fedd780c740 1 --2- 172.21.15.38:0/1601330609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 0x55630105fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.840+0000 7fedd780c740 1 --2- 172.21.15.38:0/1601330609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c956b0 0x556301066ff0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.840+0000 7fedd780c740 1 --2- 172.21.15.38:0/1601330609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c8d4d0 0x55630105a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.840+0000 7fedd780c740 1 -- 172.21.15.38:0/1601330609 >> 172.21.15.38:0/1601330609 conn(0x556300caa4e0 msgr2=0x55630105f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:40.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.843+0000 7fedd780c740 1 -- 172.21.15.38:0/1601330609 shutdown_connections 2024-05-21T15:39:40.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.845+0000 7fedd780c740 1 -- 172.21.15.38:0/1601330609 wait complete. 2024-05-21T15:39:40.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.845+0000 7fedd780c740 1 -- start start 2024-05-21T15:39:40.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.845+0000 7fedd780c740 1 Processor -- start 2024-05-21T15:39:40.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.847+0000 7fedd780c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x556301066000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.847+0000 7fedd780c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c956b0 0x556301066540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.848+0000 7fedd780c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 0x5563010d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.848+0000 7fedd780c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fedb400fa70 con 0x556300c956b0 2024-05-21T15:39:40.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.848+0000 7fedd780c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fedb400f370 con 0x556300c8d4d0 2024-05-21T15:39:40.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.848+0000 7fedd780c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fedb400c5b0 con 0x556300cb6390 2024-05-21T15:39:40.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.848+0000 7fedd780c740 10 client.0 _ll_register_callbacks cb 0x7fedc4012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:40.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.849+0000 7fedc9d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x556301066000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.849+0000 7fedc9d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x556301066000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50112/0 (socket says 172.21.15.38:50112) 2024-05-21T15:39:40.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.849+0000 7fedc9d9b640 1 -- 172.21.15.38:0/1167958592 learned_addr learned my addr 172.21.15.38:0/1167958592 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:40.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.849+0000 7fedc9d9b640 1 --2- 172.21.15.38:0/1167958592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 0x5563010d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.849+0000 7fedc9d9b640 1 -- 172.21.15.38:0/1167958592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 msgr2=0x5563010d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:40.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.849+0000 7fedc9d9b640 1 --2- 172.21.15.38:0/1167958592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 0x5563010d3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.849+0000 7fedc9d9b640 1 -- 172.21.15.38:0/1167958592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c956b0 msgr2=0x556301066540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:40.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.849+0000 7fedc9d9b640 1 --2- 172.21.15.38:0/1167958592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c956b0 0x556301066540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.849+0000 7fedc9d9b640 1 -- 172.21.15.38:0/1167958592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563010d40e0 con 0x556300c8d4d0 2024-05-21T15:39:40.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.850+0000 7fedc9d9b640 1 --2- 172.21.15.38:0/1167958592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x556301066000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fedc4006010 tx=0x7fedc401f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b020747184fa7f29 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:40.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.852+0000 7fedadffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:39:40.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.852+0000 7fedadffb640 1 -- 172.21.15.38:0/1167958592 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4004d30 con 0x556300c8d4d0 2024-05-21T15:39:40.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.852+0000 7fedadffb640 1 -- 172.21.15.38:0/1167958592 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fedc4026040 con 0x556300c8d4d0 2024-05-21T15:39:40.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.852+0000 7fedd780c740 1 -- 172.21.15.38:0/1167958592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedb8023070 con 0x556300c8d4d0 2024-05-21T15:39:40.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.853+0000 7fedadffb640 1 -- 172.21.15.38:0/1167958592 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fedc4036470 con 0x556300c8d4d0 2024-05-21T15:39:40.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.854+0000 7fedd780c740 10 client.26426 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:40.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.854+0000 7fedd780c740 1 -- 172.21.15.38:0/1167958592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5563010d5320 con 0x556300c8d4d0 2024-05-21T15:39:40.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.854+0000 7fedadffb640 1 -- 172.21.15.38:0/1167958592 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedb8023070 con 0x556300c8d4d0 2024-05-21T15:39:40.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.856+0000 7fedadffb640 1 -- 172.21.15.38:0/1167958592 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fedc40052c0 con 0x556300c8d4d0 2024-05-21T15:39:40.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.856+0000 7fedd780c740 10 client.26426 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:40.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.856+0000 7fedd780c740 10 client.26426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:40.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.856+0000 7fedd780c740 20 client.26426 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:40.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.856+0000 7fedd780c740 1 --2- 172.21.15.38:0/1167958592 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556300c4a3a0 0x5563010d7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:40.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.856+0000 7fedd780c740 4 client.26426 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:39:40.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.856+0000 7fedd780c740 1 -- 172.21.15.38:0/1167958592 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556300fc2780 con 0x556300c4a3a0 2024-05-21T15:39:40.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.857+0000 7fedca59c640 1 --2- 172.21.15.38:0/1167958592 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556300c4a3a0 0x5563010d7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:40.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.865+0000 7fedca59c640 1 --2- 172.21.15.38:0/1167958592 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556300c4a3a0 0x5563010d7c30 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a5cee7facece82 server_cookie=dba8a56ccb6a0a50 in_seq=0 out_seq=0 2024-05-21T15:39:40.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.865+0000 7fedadffb640 10 client.26426 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:40.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.866+0000 7fedadffb640 1 -- 172.21.15.38:0/1167958592 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556300fc2780 con 0x556300c4a3a0 2024-05-21T15:39:40.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.866+0000 7fedadffb640 10 client.26426 handle_command_reply: tid=0 2024-05-21T15:39:40.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.867+0000 7fedd780c740 1 client.26426 shutdown 2024-05-21T15:39:40.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.873+0000 7fedd780c740 1 -- 172.21.15.38:0/1167958592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 msgr2=0x556301066000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:40.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.873+0000 7fedd780c740 1 --2- 172.21.15.38:0/1167958592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x556301066000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fedc4006010 tx=0x7fedc401f040 comp rx=0 tx=0).stop 2024-05-21T15:39:40.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.874+0000 7fedd780c740 1 -- 172.21.15.38:0/1167958592 shutdown_connections 2024-05-21T15:39:40.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.874+0000 7fedd780c740 1 --2- 172.21.15.38:0/1167958592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556300cb6390 0x5563010d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.874+0000 7fedd780c740 1 --2- 172.21.15.38:0/1167958592 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x556300c4a3a0 0x5563010d7c30 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.874+0000 7fedd780c740 1 --2- 172.21.15.38:0/1167958592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556300c956b0 0x556301066540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.874+0000 7fedd780c740 1 --2- 172.21.15.38:0/1167958592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556300c8d4d0 0x556301066000 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.874+0000 7fedd780c740 1 -- 172.21.15.38:0/1167958592 >> 172.21.15.38:0/1167958592 conn(0x556300caa4e0 msgr2=0x556301054340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.874+0000 7fedadffb640 0 client.26426 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.874+0000 7fedd780c740 1 -- 172.21.15.38:0/1167958592 shutdown_connections 2024-05-21T15:39:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.874+0000 7fedd780c740 1 -- 172.21.15.38:0/1167958592 wait complete. 2024-05-21T15:39:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.875+0000 7fedd780c740 20 client.26426 trim_cache size 0 max 16384 2024-05-21T15:39:40.877 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:39:40.877 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-05-21T15:39:40.877 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-05-21T15:39:40.877 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:39:40.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.881+0000 7fedd088d640 1 -- 172.21.15.38:0/129313461 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fedb00a2d10 msgr2=0x7fedb00a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:40.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.881+0000 7fedd088d640 1 --2- 172.21.15.38:0/129313461 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fedb00a2d10 0x7fedb00a5220 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0bb5d0 tx=0x7fedbc02c390 comp rx=0 tx=0).stop 2024-05-21T15:39:40.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.881+0000 7fedd088d640 1 -- 172.21.15.38:0/129313461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedcc107900 msgr2=0x7fedcc0720e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:40.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.881+0000 7fedd088d640 1 --2- 172.21.15.38:0/129313461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedcc107900 0x7fedcc0720e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00d540 tx=0x7fedbc034790 comp rx=0 tx=0).stop 2024-05-21T15:39:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.881+0000 7fedd088d640 1 -- 172.21.15.38:0/129313461 shutdown_connections 2024-05-21T15:39:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.881+0000 7fedd088d640 1 --2- 172.21.15.38:0/129313461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedcc107900 0x7fedcc0720e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.881+0000 7fedd088d640 1 --2- 172.21.15.38:0/129313461 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fedb00a2d10 0x7fedb00a5220 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.881+0000 7fedd088d640 1 --2- 172.21.15.38:0/129313461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedcc102c00 0x7fedcc071ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.881+0000 7fedd088d640 1 --2- 172.21.15.38:0/129313461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedcc1019c0 0x7fedcc073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.881+0000 7fedd088d640 1 -- 172.21.15.38:0/129313461 >> 172.21.15.38:0/129313461 conn(0x7fedcc0fcbb0 msgr2=0x7fedcc1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.881+0000 7fedd088d640 1 -- 172.21.15.38:0/129313461 shutdown_connections 2024-05-21T15:39:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:40.881+0000 7fedd088d640 1 -- 172.21.15.38:0/129313461 wait complete. 2024-05-21T15:39:40.901 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:39:40.902 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:39:41.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.059+0000 7f434859d640 1 -- 172.21.15.38:0/53391334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43401089f0 msgr2=0x7f434010ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:41.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.059+0000 7f434859d640 1 --2- 172.21.15.38:0/53391334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43401089f0 0x7f434010ae10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4338009ca0 tx=0x7f433802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.059+0000 7f434859d640 1 -- 172.21.15.38:0/53391334 shutdown_connections 2024-05-21T15:39:41.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.059+0000 7f434859d640 1 --2- 172.21.15.38:0/53391334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43401089f0 0x7f434010ae10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.059+0000 7f434859d640 1 --2- 172.21.15.38:0/53391334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4340103cf0 0x7f4340104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.059+0000 7f434859d640 1 --2- 172.21.15.38:0/53391334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4340102ab0 0x7f43401037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.060+0000 7f434859d640 1 -- 172.21.15.38:0/53391334 >> 172.21.15.38:0/53391334 conn(0x7f43400fdca0 msgr2=0x7f4340100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:41.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.060+0000 7f434859d640 1 -- 172.21.15.38:0/53391334 shutdown_connections 2024-05-21T15:39:41.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.060+0000 7f434859d640 1 -- 172.21.15.38:0/53391334 wait complete. 2024-05-21T15:39:41.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.060+0000 7f434859d640 1 Processor -- start 2024-05-21T15:39:41.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.060+0000 7f434859d640 1 -- start start 2024-05-21T15:39:41.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.060+0000 7f434859d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4340102ab0 0x7f4340075800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.060+0000 7f434859d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4340103cf0 0x7f4340073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.060+0000 7f434859d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43401089f0 0x7f43400743e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.060+0000 7f434859d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f434010e710 con 0x7f43401089f0 2024-05-21T15:39:41.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.060+0000 7f434859d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f434010e590 con 0x7f4340102ab0 2024-05-21T15:39:41.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.060+0000 7f434859d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f434010e890 con 0x7f4340103cf0 2024-05-21T15:39:41.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f4346b3a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43401089f0 0x7f43400743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f4346b3a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43401089f0 0x7f43400743e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34080/0 (socket says 172.21.15.38:34080) 2024-05-21T15:39:41.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f4346b3a640 1 -- 172.21.15.38:0/3378935410 learned_addr learned my addr 172.21.15.38:0/3378935410 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:41.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f4346339640 1 --2- 172.21.15.38:0/3378935410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4340102ab0 0x7f4340075800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f4346b3a640 1 -- 172.21.15.38:0/3378935410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4340103cf0 msgr2=0x7f4340073ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:41.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f4346b3a640 1 --2- 172.21.15.38:0/3378935410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4340103cf0 0x7f4340073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f4346b3a640 1 -- 172.21.15.38:0/3378935410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4340102ab0 msgr2=0x7f4340075800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:41.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f4346b3a640 1 --2- 172.21.15.38:0/3378935410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4340102ab0 0x7f4340075800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f4346b3a640 1 -- 172.21.15.38:0/3378935410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4340074b10 con 0x7f43401089f0 2024-05-21T15:39:41.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f4346339640 1 --2- 172.21.15.38:0/3378935410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4340102ab0 0x7f4340075800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:39:41.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f4346b3a640 1 --2- 172.21.15.38:0/3378935410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43401089f0 0x7f43400743e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4338005e30 tx=0x7f4338031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:41.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f43377fe640 1 -- 172.21.15.38:0/3378935410 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338003b70 con 0x7f43401089f0 2024-05-21T15:39:41.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f43377fe640 1 -- 172.21.15.38:0/3378935410 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4338003d10 con 0x7f43401089f0 2024-05-21T15:39:41.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f434859d640 1 -- 172.21.15.38:0/3378935410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43401a3200 con 0x7f43401089f0 2024-05-21T15:39:41.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f43377fe640 1 -- 172.21.15.38:0/3378935410 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433802fb70 con 0x7f43401089f0 2024-05-21T15:39:41.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.061+0000 7f434859d640 1 -- 172.21.15.38:0/3378935410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43401a36c0 con 0x7f43401089f0 2024-05-21T15:39:41.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.062+0000 7f434859d640 1 -- 172.21.15.38:0/3378935410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43401070f0 con 0x7f43401089f0 2024-05-21T15:39:41.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.064+0000 7f43377fe640 1 -- 172.21.15.38:0/3378935410 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f433802fd10 con 0x7f43401089f0 2024-05-21T15:39:41.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.064+0000 7f43377fe640 1 --2- 172.21.15.38:0/3378935410 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f432c0a2cb0 0x7f432c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.065+0000 7f43377fe640 1 -- 172.21.15.38:0/3378935410 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f43380f02a0 con 0x7f43401089f0 2024-05-21T15:39:41.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.065+0000 7f4346339640 1 --2- 172.21.15.38:0/3378935410 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f432c0a2cb0 0x7f432c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.065+0000 7f4346339640 1 --2- 172.21.15.38:0/3378935410 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f432c0a2cb0 0x7f432c0a51c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f43401036b0 tx=0x7f433c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:41.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.069+0000 7f43377fe640 1 -- 172.21.15.38:0/3378935410 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f43380bd1b0 con 0x7f43401089f0 2024-05-21T15:39:41.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.492+0000 7f434859d640 1 -- 172.21.15.38:0/3378935410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f434002d2a0 con 0x7f43401089f0 2024-05-21T15:39:41.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.494+0000 7f43377fe640 1 -- 172.21.15.38:0/3378935410 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11922 (secure 0 0 0) 0x7f43380bcab0 con 0x7f43401089f0 2024-05-21T15:39:41.534 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:39:41.534 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:36:59.077696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24551,24523,24602,24550,24515]},"id":1}]} 2024-05-21T15:39:41.536 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 31 2024-05-21T15:39:41.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.498+0000 7f434859d640 1 -- 172.21.15.38:0/3378935410 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f432c0a2cb0 msgr2=0x7f432c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:41.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.498+0000 7f434859d640 1 --2- 172.21.15.38:0/3378935410 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f432c0a2cb0 0x7f432c0a51c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f43401036b0 tx=0x7f433c005f90 comp rx=0 tx=0).stop 2024-05-21T15:39:41.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.498+0000 7f434859d640 1 -- 172.21.15.38:0/3378935410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43401089f0 msgr2=0x7f43400743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:41.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.498+0000 7f434859d640 1 --2- 172.21.15.38:0/3378935410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43401089f0 0x7f43400743e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4338005e30 tx=0x7f4338031ee0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.501+0000 7f434859d640 1 -- 172.21.15.38:0/3378935410 shutdown_connections 2024-05-21T15:39:41.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.502+0000 7f434859d640 1 --2- 172.21.15.38:0/3378935410 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f432c0a2cb0 0x7f432c0a51c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.502+0000 7f434859d640 1 --2- 172.21.15.38:0/3378935410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43401089f0 0x7f43400743e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.502+0000 7f434859d640 1 --2- 172.21.15.38:0/3378935410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4340103cf0 0x7f4340073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.502+0000 7f434859d640 1 --2- 172.21.15.38:0/3378935410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4340102ab0 0x7f4340075800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.502+0000 7f434859d640 1 -- 172.21.15.38:0/3378935410 >> 172.21.15.38:0/3378935410 conn(0x7f43400fdca0 msgr2=0x7f4340100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:41.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.503+0000 7f434859d640 1 -- 172.21.15.38:0/3378935410 shutdown_connections 2024-05-21T15:39:41.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.503+0000 7f434859d640 1 -- 172.21.15.38:0/3378935410 wait complete. 2024-05-21T15:39:41.537 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:39:41.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.752+0000 7fd3334fe640 1 -- 172.21.15.38:0/1056534036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd32c102c10 msgr2=0x7fd32c103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:41.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.752+0000 7fd3334fe640 1 --2- 172.21.15.38:0/1056534036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd32c102c10 0x7fd32c103020 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009900 tx=0x7fd31c02a3f0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.753+0000 7fd3334fe640 1 -- 172.21.15.38:0/1056534036 shutdown_connections 2024-05-21T15:39:41.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.753+0000 7fd3334fe640 1 --2- 172.21.15.38:0/1056534036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd32c06a370 0x7fd32c06a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.753+0000 7fd3334fe640 1 --2- 172.21.15.38:0/1056534036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd32c102c10 0x7fd32c103020 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.753+0000 7fd3334fe640 1 --2- 172.21.15.38:0/1056534036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c1019d0 0x7fd32c1026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.753+0000 7fd3334fe640 1 -- 172.21.15.38:0/1056534036 >> 172.21.15.38:0/1056534036 conn(0x7fd32c0fcbc0 msgr2=0x7fd32c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:41.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.756+0000 7fd3334fe640 1 -- 172.21.15.38:0/1056534036 shutdown_connections 2024-05-21T15:39:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.756+0000 7fd3334fe640 1 -- 172.21.15.38:0/1056534036 wait complete. 2024-05-21T15:39:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.757+0000 7fd3334fe640 1 Processor -- start 2024-05-21T15:39:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.758+0000 7fd3334fe640 1 -- start start 2024-05-21T15:39:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.758+0000 7fd3334fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd32c06a370 0x7fd32c070c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.758+0000 7fd3334fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c1019d0 0x7fd32c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.758+0000 7fd3334fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd32c102c10 0x7fd32c072170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.758+0000 7fd3334fe640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c1109d0 con 0x7fd32c102c10 2024-05-21T15:39:41.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.758+0000 7fd3334fe640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c110850 con 0x7fd32c1019d0 2024-05-21T15:39:41.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.758+0000 7fd3334fe640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c110b50 con 0x7fd32c06a370 2024-05-21T15:39:41.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.759+0000 7fd331a9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd32c102c10 0x7fd32c072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.759+0000 7fd330a99640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c1019d0 0x7fd32c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.759+0000 7fd330a99640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c1019d0 0x7fd32c0711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50148/0 (socket says 172.21.15.38:50148) 2024-05-21T15:39:41.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.759+0000 7fd330a99640 1 -- 172.21.15.38:0/2513818998 learned_addr learned my addr 172.21.15.38:0/2513818998 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:41.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.759+0000 7fd33129a640 1 --2- 172.21.15.38:0/2513818998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd32c06a370 0x7fd32c070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.761+0000 7fd331a9b640 1 -- 172.21.15.38:0/2513818998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd32c06a370 msgr2=0x7fd32c070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:41.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.761+0000 7fd331a9b640 1 --2- 172.21.15.38:0/2513818998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd32c06a370 0x7fd32c070c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.761+0000 7fd331a9b640 1 -- 172.21.15.38:0/2513818998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c1019d0 msgr2=0x7fd32c0711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:41.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.761+0000 7fd331a9b640 1 --2- 172.21.15.38:0/2513818998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c1019d0 0x7fd32c0711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.762+0000 7fd331a9b640 1 -- 172.21.15.38:0/2513818998 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c0727e0 con 0x7fd32c102c10 2024-05-21T15:39:41.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.762+0000 7fd331a9b640 1 --2- 172.21.15.38:0/2513818998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd32c102c10 0x7fd32c072170 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd32400e940 tx=0x7fd32400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:41.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.763+0000 7fd3227fc640 1 -- 172.21.15.38:0/2513818998 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd324004d20 con 0x7fd32c102c10 2024-05-21T15:39:41.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.763+0000 7fd3334fe640 1 -- 172.21.15.38:0/2513818998 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c072ad0 con 0x7fd32c102c10 2024-05-21T15:39:41.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.763+0000 7fd3334fe640 1 -- 172.21.15.38:0/2513818998 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c073020 con 0x7fd32c102c10 2024-05-21T15:39:41.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.763+0000 7fd3227fc640 1 -- 172.21.15.38:0/2513818998 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd324011470 con 0x7fd32c102c10 2024-05-21T15:39:41.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.763+0000 7fd3227fc640 1 -- 172.21.15.38:0/2513818998 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32401f060 con 0x7fd32c102c10 2024-05-21T15:39:41.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.768+0000 7fd3227fc640 1 -- 172.21.15.38:0/2513818998 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd32400cb20 con 0x7fd32c102c10 2024-05-21T15:39:41.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.769+0000 7fd3227fc640 1 --2- 172.21.15.38:0/2513818998 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3180a2cf0 0x7fd3180a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.769+0000 7fd3227fc640 1 -- 172.21.15.38:0/2513818998 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fd3240cf080 con 0x7fd32c102c10 2024-05-21T15:39:41.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.769+0000 7fd33a3a4740 1 Processor -- start 2024-05-21T15:39:41.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.769+0000 7fd33a3a4740 1 -- start start 2024-05-21T15:39:41.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.770+0000 7fd33a3a4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bdb390 0x558925f7ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.770+0000 7fd33a3a4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bb24d0 0x558925f7b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.770+0000 7fd33a3a4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f81f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.770+0000 7fd33a3a4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558925edfc40 con 0x558925bba6b0 2024-05-21T15:39:41.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.770+0000 7fd33a3a4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558925e87580 con 0x558925bb24d0 2024-05-21T15:39:41.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.770+0000 7fd33a3a4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558925edbc00 con 0x558925bdb390 2024-05-21T15:39:41.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.770+0000 7fd330a99640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bb24d0 0x558925f7b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.770+0000 7fd331a9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f81f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.770+0000 7fd331a9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f81f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34092/0 (socket says 172.21.15.38:34092) 2024-05-21T15:39:41.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.770+0000 7fd331a9b640 1 -- 172.21.15.38:0/3473517655 learned_addr learned my addr 172.21.15.38:0/3473517655 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:41.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.771+0000 7fd331a9b640 1 -- 172.21.15.38:0/3473517655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bdb390 msgr2=0x558925f7ab00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:39:41.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.771+0000 7fd331a9b640 1 --2- 172.21.15.38:0/3473517655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bdb390 0x558925f7ab00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.771+0000 7fd331a9b640 1 -- 172.21.15.38:0/3473517655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bb24d0 msgr2=0x558925f7b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:41.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.771+0000 7fd331a9b640 1 --2- 172.21.15.38:0/3473517655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bb24d0 0x558925f7b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.771+0000 7fd331a9b640 1 -- 172.21.15.38:0/3473517655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558925bf33f0 con 0x558925bba6b0 2024-05-21T15:39:41.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.771+0000 7fd331a9b640 1 --2- 172.21.15.38:0/3473517655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f81f80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd3240a3f70 tx=0x7fd32409fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4d965c805f9078 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:41.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.771+0000 7fd3177fe640 1 -- 172.21.15.38:0/3473517655 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3240ccbe0 con 0x558925bba6b0 2024-05-21T15:39:41.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.772+0000 7fd3177fe640 1 -- 172.21.15.38:0/3473517655 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd324097f50 con 0x558925bba6b0 2024-05-21T15:39:41.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.772+0000 7fd3177fe640 1 -- 172.21.15.38:0/3473517655 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32409a7a0 con 0x558925bba6b0 2024-05-21T15:39:41.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.772+0000 7fd33129a640 1 --2- 172.21.15.38:0/2513818998 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3180a2cf0 0x7fd3180a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.773+0000 7fd33a3a4740 1 -- 172.21.15.38:0/3473517655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 msgr2=0x558925f81f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:41.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.773+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/3473517655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f81f80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd3240a3f70 tx=0x7fd32409fca0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.773+0000 7fd33a3a4740 1 -- 172.21.15.38:0/3473517655 shutdown_connections 2024-05-21T15:39:41.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.773+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/3473517655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f81f80 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.773+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/3473517655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bb24d0 0x558925f7b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.773+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/3473517655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bdb390 0x558925f7ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.773+0000 7fd33a3a4740 1 -- 172.21.15.38:0/3473517655 >> 172.21.15.38:0/3473517655 conn(0x558925bcf4e0 msgr2=0x558925f76490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:41.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.774+0000 7fd33129a640 1 --2- 172.21.15.38:0/2513818998 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3180a2cf0 0x7fd3180a5200 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x558925f7bcd0 tx=0x7fd328014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:41.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.774+0000 7fd33a3a4740 1 -- 172.21.15.38:0/3473517655 shutdown_connections 2024-05-21T15:39:41.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.774+0000 7fd33a3a4740 1 -- 172.21.15.38:0/3473517655 wait complete. 2024-05-21T15:39:41.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.774+0000 7fd33a3a4740 1 -- start start 2024-05-21T15:39:41.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.774+0000 7fd33a3a4740 1 Processor -- start 2024-05-21T15:39:41.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.775+0000 7fd33a3a4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bb24d0 0x558925f84720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.775+0000 7fd33a3a4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bba6b0 0x558925f84c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.775+0000 7fd33129a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bb24d0 0x558925f84720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.776+0000 7fd33129a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bb24d0 0x558925f84720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34094/0 (socket says 172.21.15.38:34094) 2024-05-21T15:39:41.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.776+0000 7fd33129a640 1 -- 172.21.15.38:0/1169847572 learned_addr learned my addr 172.21.15.38:0/1169847572 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:41.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.776+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/1169847572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bdb390 0x558925f8bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.776+0000 7fd33a3a4740 1 -- 172.21.15.38:0/1169847572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558925f82970 con 0x558925bb24d0 2024-05-21T15:39:41.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.776+0000 7fd33a3a4740 1 -- 172.21.15.38:0/1169847572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558925e9ac40 con 0x558925bdb390 2024-05-21T15:39:41.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.776+0000 7fd33a3a4740 1 -- 172.21.15.38:0/1169847572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558925f82af0 con 0x558925bba6b0 2024-05-21T15:39:41.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.776+0000 7fd330a99640 1 --2- 172.21.15.38:0/1169847572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bba6b0 0x558925f84c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.776+0000 7fd33a3a4740 10 client.0 _ll_register_callbacks cb 0x558925bf2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:41.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.777+0000 7fd331a9b640 1 --2- 172.21.15.38:0/1169847572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bdb390 0x558925f8bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.778+0000 7fd330a99640 1 -- 172.21.15.38:0/1169847572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bdb390 msgr2=0x558925f8bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:41.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.778+0000 7fd330a99640 1 --2- 172.21.15.38:0/1169847572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bdb390 0x558925f8bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.778+0000 7fd330a99640 1 -- 172.21.15.38:0/1169847572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bb24d0 msgr2=0x558925f84720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:41.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.778+0000 7fd330a99640 1 --2- 172.21.15.38:0/1169847572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bb24d0 0x558925f84720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.778+0000 7fd330a99640 1 -- 172.21.15.38:0/1169847572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558925cbd7d0 con 0x558925bba6b0 2024-05-21T15:39:41.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.778+0000 7fd330a99640 1 --2- 172.21.15.38:0/1169847572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bba6b0 0x558925f84c60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd31c000c00 tx=0x7fd31c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c80b2502072c1643 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:41.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.779+0000 7fd314ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:39:41.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.779+0000 7fd314ff9640 1 -- 172.21.15.38:0/1169847572 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c035040 con 0x558925bba6b0 2024-05-21T15:39:41.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.779+0000 7fd314ff9640 1 -- 172.21.15.38:0/1169847572 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fd31c0316b0 con 0x558925bba6b0 2024-05-21T15:39:41.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.779+0000 7fd33a3a4740 1 -- 172.21.15.38:0/1169847572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd324097f50 con 0x558925bba6b0 2024-05-21T15:39:41.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.780+0000 7fd314ff9640 1 -- 172.21.15.38:0/1169847572 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd31c005610 con 0x558925bba6b0 2024-05-21T15:39:41.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.780+0000 7fd33a3a4740 10 client.26509 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:41.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.780+0000 7fd33a3a4740 1 -- 172.21.15.38:0/1169847572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558925f8eef0 con 0x558925bba6b0 2024-05-21T15:39:41.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.780+0000 7fd314ff9640 1 -- 172.21.15.38:0/1169847572 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd324097f50 con 0x558925bba6b0 2024-05-21T15:39:41.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.780+0000 7fd314ff9640 1 -- 172.21.15.38:0/1169847572 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fd31c0035f0 con 0x558925bba6b0 2024-05-21T15:39:41.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.781+0000 7fd33a3a4740 10 client.26509 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:41.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.781+0000 7fd33a3a4740 10 client.26509 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:39:41.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.781+0000 7fd33a3a4740 20 client.26509 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.781+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/1169847572 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x558925b99f60 0x558925f92580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.781+0000 7fd33a3a4740 4 client.26509 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.781+0000 7fd33a3a4740 1 -- 172.21.15.38:0/1169847572 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558925eed370 con 0x558925b99f60 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.781+0000 7fd33129a640 1 --2- 172.21.15.38:0/1169847572 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x558925b99f60 0x558925f92580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.782+0000 7fd314ff9640 10 client.26509 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.782+0000 7fd33129a640 1 --2- 172.21.15.38:0/1169847572 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x558925b99f60 0x558925f92580 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e54e4ae4d8f850e5 server_cookie=f4a11fc8a9da9685 in_seq=0 out_seq=0 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.783+0000 7fd314ff9640 1 -- 172.21.15.38:0/1169847572 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558925eed370 con 0x558925b99f60 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.783+0000 7fd314ff9640 10 client.26509 handle_command_reply: tid=0 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.783+0000 7fd33a3a4740 1 client.26509 shutdown 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.784+0000 7fd33a3a4740 1 -- 172.21.15.38:0/1169847572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bba6b0 msgr2=0x558925f84c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.784+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/1169847572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bba6b0 0x558925f84c60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd31c000c00 tx=0x7fd31c004610 comp rx=0 tx=0).stop 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.785+0000 7fd33a3a4740 1 -- 172.21.15.38:0/1169847572 shutdown_connections 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.785+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/1169847572 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x558925b99f60 0x558925f92580 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.785+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/1169847572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bdb390 0x558925f8bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.785+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/1169847572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bba6b0 0x558925f84c60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.785+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/1169847572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bb24d0 0x558925f84720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.785+0000 7fd33a3a4740 1 -- 172.21.15.38:0/1169847572 >> 172.21.15.38:0/1169847572 conn(0x558925bcf4e0 msgr2=0x558925f75570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:41.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.785+0000 7fd314ff9640 0 client.26509 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:39:41.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.785+0000 7fd33a3a4740 1 -- 172.21.15.38:0/1169847572 shutdown_connections 2024-05-21T15:39:41.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.785+0000 7fd33a3a4740 1 -- 172.21.15.38:0/1169847572 wait complete. 2024-05-21T15:39:41.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.785+0000 7fd33a3a4740 20 client.26509 trim_cache size 0 max 16384 2024-05-21T15:39:41.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.814+0000 7fd33a3a4740 1 Processor -- start 2024-05-21T15:39:41.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.814+0000 7fd33a3a4740 1 -- start start 2024-05-21T15:39:41.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.814+0000 7fd33a3a4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bcf4e0 0x558925f76540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.815+0000 7fd33a3a4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bb24d0 0x558925f8bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.815+0000 7fd33129a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bcf4e0 0x558925f76540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.815+0000 7fd33129a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bcf4e0 0x558925f76540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:36576/0 (socket says 172.21.15.38:36576) 2024-05-21T15:39:41.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.815+0000 7fd33129a640 1 -- 172.21.15.38:0/2204403563 learned_addr learned my addr 172.21.15.38:0/2204403563 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:41.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.815+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/2204403563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f75250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.815+0000 7fd330a99640 1 --2- 172.21.15.38:0/2204403563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bb24d0 0x558925f8bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.816+0000 7fd33a3a4740 1 -- 172.21.15.38:0/2204403563 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd31000e5b0 con 0x558925bba6b0 2024-05-21T15:39:41.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.816+0000 7fd331a9b640 1 --2- 172.21.15.38:0/2204403563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f75250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.816+0000 7fd33a3a4740 1 -- 172.21.15.38:0/2204403563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd3100101e0 con 0x558925bb24d0 2024-05-21T15:39:41.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.816+0000 7fd33a3a4740 1 -- 172.21.15.38:0/2204403563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd31000ec80 con 0x558925bcf4e0 2024-05-21T15:39:41.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.817+0000 7fd33129a640 1 -- 172.21.15.38:0/2204403563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bb24d0 msgr2=0x558925f8bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:41.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.817+0000 7fd33129a640 1 --2- 172.21.15.38:0/2204403563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bb24d0 0x558925f8bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.817+0000 7fd33129a640 1 -- 172.21.15.38:0/2204403563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 msgr2=0x558925f75250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:41.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.817+0000 7fd33129a640 1 --2- 172.21.15.38:0/2204403563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f75250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.817+0000 7fd33129a640 1 -- 172.21.15.38:0/2204403563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558925f8c020 con 0x558925bcf4e0 2024-05-21T15:39:41.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.817+0000 7fd331a9b640 1 --2- 172.21.15.38:0/2204403563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f75250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:41.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.817+0000 7fd33129a640 1 --2- 172.21.15.38:0/2204403563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bcf4e0 0x558925f76540 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x558925f92d20 tx=0x7fd32801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf4d7875710bc3e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:41.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.817+0000 7fd316ffd640 1 -- 172.21.15.38:0/2204403563 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd328024040 con 0x558925bcf4e0 2024-05-21T15:39:41.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.817+0000 7fd316ffd640 1 -- 172.21.15.38:0/2204403563 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd328003ad0 con 0x558925bcf4e0 2024-05-21T15:39:41.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.817+0000 7fd316ffd640 1 -- 172.21.15.38:0/2204403563 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd328030a10 con 0x558925bcf4e0 2024-05-21T15:39:41.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.819+0000 7fd33a3a4740 1 -- 172.21.15.38:0/2204403563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bcf4e0 msgr2=0x558925f76540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:41.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.819+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/2204403563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bcf4e0 0x558925f76540 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x558925f92d20 tx=0x7fd32801a000 comp rx=0 tx=0).stop 2024-05-21T15:39:41.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.821+0000 7fd33a3a4740 1 -- 172.21.15.38:0/2204403563 shutdown_connections 2024-05-21T15:39:41.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.821+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/2204403563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f75250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.821+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/2204403563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bb24d0 0x558925f8bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.821+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/2204403563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bcf4e0 0x558925f76540 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.821+0000 7fd33a3a4740 1 -- 172.21.15.38:0/2204403563 >> 172.21.15.38:0/2204403563 conn(0x558925f7f340 msgr2=0x558925f74b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:41.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.821+0000 7fd33a3a4740 1 -- 172.21.15.38:0/2204403563 shutdown_connections 2024-05-21T15:39:41.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.822+0000 7fd33a3a4740 1 -- 172.21.15.38:0/2204403563 wait complete. 2024-05-21T15:39:41.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.822+0000 7fd33a3a4740 1 -- start start 2024-05-21T15:39:41.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.823+0000 7fd33a3a4740 1 Processor -- start 2024-05-21T15:39:41.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.823+0000 7fd33a3a4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bb24d0 0x558925f898e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.823+0000 7fd33a3a4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f89e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.823+0000 7fd33a3a4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bcf4e0 0x558925f8ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.824+0000 7fd33a3a4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd31000fa50 con 0x558925bba6b0 2024-05-21T15:39:41.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.824+0000 7fd33129a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bb24d0 0x558925f898e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.824+0000 7fd33129a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bb24d0 0x558925f898e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:36578/0 (socket says 172.21.15.38:36578) 2024-05-21T15:39:41.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.824+0000 7fd33129a640 1 -- 172.21.15.38:0/384951424 learned_addr learned my addr 172.21.15.38:0/384951424 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:41.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.824+0000 7fd331a9b640 1 --2- 172.21.15.38:0/384951424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bcf4e0 0x558925f8ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.824+0000 7fd33a3a4740 1 -- 172.21.15.38:0/384951424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd31000f350 con 0x558925bcf4e0 2024-05-21T15:39:41.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.824+0000 7fd330a99640 1 --2- 172.21.15.38:0/384951424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f89e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.824+0000 7fd33a3a4740 1 -- 172.21.15.38:0/384951424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd31000c590 con 0x558925bb24d0 2024-05-21T15:39:41.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.824+0000 7fd33a3a4740 10 client.0 _ll_register_callbacks cb 0x7fd31c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:41.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.825+0000 7fd33129a640 1 -- 172.21.15.38:0/384951424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bcf4e0 msgr2=0x558925f8ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:41.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.825+0000 7fd33129a640 1 --2- 172.21.15.38:0/384951424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bcf4e0 0x558925f8ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.825+0000 7fd33129a640 1 -- 172.21.15.38:0/384951424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 msgr2=0x558925f89e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:41.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.825+0000 7fd33129a640 1 --2- 172.21.15.38:0/384951424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f89e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.825+0000 7fd33129a640 1 -- 172.21.15.38:0/384951424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558925f8b2c0 con 0x558925bb24d0 2024-05-21T15:39:41.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.825+0000 7fd33129a640 1 --2- 172.21.15.38:0/384951424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bb24d0 0x558925f898e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd32801e6c0 tx=0x7fd328003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33d9c6e1edb9dbd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:41.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.825+0000 7fd314ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:39:41.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.826+0000 7fd314ff9640 1 -- 172.21.15.38:0/384951424 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3280309a0 con 0x558925bb24d0 2024-05-21T15:39:41.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.826+0000 7fd314ff9640 1 -- 172.21.15.38:0/384951424 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fd32801ee00 con 0x558925bb24d0 2024-05-21T15:39:41.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.826+0000 7fd33a3a4740 1 -- 172.21.15.38:0/384951424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd328003ad0 con 0x558925bb24d0 2024-05-21T15:39:41.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.954+0000 7fd314ff9640 1 -- 172.21.15.38:0/384951424 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd3280035f0 con 0x558925bb24d0 2024-05-21T15:39:41.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.954+0000 7fd33a3a4740 10 client.26515 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:41.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.954+0000 7fd33a3a4740 1 -- 172.21.15.38:0/384951424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558925ffaee0 con 0x558925bb24d0 2024-05-21T15:39:41.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.954+0000 7fd314ff9640 1 -- 172.21.15.38:0/384951424 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd328003ad0 con 0x558925bb24d0 2024-05-21T15:39:41.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.955+0000 7fd314ff9640 1 -- 172.21.15.38:0/384951424 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fd328038030 con 0x558925bb24d0 2024-05-21T15:39:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.955+0000 7fd33a3a4740 10 client.26515 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.955+0000 7fd33a3a4740 10 client.26515 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:39:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.955+0000 7fd33a3a4740 20 client.26515 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.955+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/384951424 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x558925bdb390 0x558925ffd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.955+0000 7fd33a3a4740 4 client.26515 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:39:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.955+0000 7fd33a3a4740 1 -- 172.21.15.38:0/384951424 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558925ee76c0 con 0x558925bdb390 2024-05-21T15:39:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.955+0000 7fd330a99640 1 --2- 172.21.15.38:0/384951424 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x558925bdb390 0x558925ffd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd314ff9640 10 client.26515 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:39:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd330a99640 1 --2- 172.21.15.38:0/384951424 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x558925bdb390 0x558925ffd7c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ae3f85b65fcbca2e server_cookie=930c3e27c55ad1f1 in_seq=0 out_seq=0 2024-05-21T15:39:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd314ff9640 1 -- 172.21.15.38:0/384951424 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558925ee76c0 con 0x558925bdb390 2024-05-21T15:39:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd314ff9640 10 client.26515 handle_command_reply: tid=0 2024-05-21T15:39:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd33a3a4740 1 client.26515 shutdown 2024-05-21T15:39:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd33a3a4740 1 -- 172.21.15.38:0/384951424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bb24d0 msgr2=0x558925f898e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/384951424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bb24d0 0x558925f898e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd32801e6c0 tx=0x7fd328003c80 comp rx=0 tx=0).stop 2024-05-21T15:39:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd33a3a4740 1 -- 172.21.15.38:0/384951424 shutdown_connections 2024-05-21T15:39:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/384951424 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x558925bdb390 0x558925ffd7c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/384951424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558925bcf4e0 0x558925f8ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/384951424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558925bba6b0 0x558925f89e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd314ff9640 0 client.26515 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:39:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd33a3a4740 1 --2- 172.21.15.38:0/384951424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558925bb24d0 0x558925f898e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd33a3a4740 1 -- 172.21.15.38:0/384951424 >> 172.21.15.38:0/384951424 conn(0x558925f7f340 msgr2=0x558925f7ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:41.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd33a3a4740 1 -- 172.21.15.38:0/384951424 shutdown_connections 2024-05-21T15:39:41.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.956+0000 7fd33a3a4740 1 -- 172.21.15.38:0/384951424 wait complete. 2024-05-21T15:39:41.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.957+0000 7fd33a3a4740 20 client.26515 trim_cache size 0 max 16384 2024-05-21T15:39:41.957 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:39:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.957+0000 7fd3334fe640 1 -- 172.21.15.38:0/2513818998 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3180a2cf0 msgr2=0x7fd3180a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.957+0000 7fd3334fe640 1 --2- 172.21.15.38:0/2513818998 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3180a2cf0 0x7fd3180a5200 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x558925f7bcd0 tx=0x7fd328014040 comp rx=0 tx=0).stop 2024-05-21T15:39:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.957+0000 7fd3334fe640 1 -- 172.21.15.38:0/2513818998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd32c102c10 msgr2=0x7fd32c072170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.957+0000 7fd3334fe640 1 --2- 172.21.15.38:0/2513818998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd32c102c10 0x7fd32c072170 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd32400e940 tx=0x7fd32400ee00 comp rx=0 tx=0).stop 2024-05-21T15:39:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.957+0000 7fd3334fe640 1 -- 172.21.15.38:0/2513818998 shutdown_connections 2024-05-21T15:39:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.957+0000 7fd3334fe640 1 --2- 172.21.15.38:0/2513818998 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd3180a2cf0 0x7fd3180a5200 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.957+0000 7fd3334fe640 1 --2- 172.21.15.38:0/2513818998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd32c102c10 0x7fd32c072170 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.957+0000 7fd3334fe640 1 --2- 172.21.15.38:0/2513818998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c1019d0 0x7fd32c0711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.957+0000 7fd3334fe640 1 --2- 172.21.15.38:0/2513818998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd32c06a370 0x7fd32c070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:41.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.957+0000 7fd3334fe640 1 -- 172.21.15.38:0/2513818998 >> 172.21.15.38:0/2513818998 conn(0x7fd32c0fcbc0 msgr2=0x7fd32c107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:41.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.957+0000 7fd3334fe640 1 -- 172.21.15.38:0/2513818998 shutdown_connections 2024-05-21T15:39:41.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:41.957+0000 7fd3334fe640 1 -- 172.21.15.38:0/2513818998 wait complete. 2024-05-21T15:39:41.972 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:39:42.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.189+0000 7f059f9ce640 1 -- 172.21.15.38:0/2184090238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059810a710 msgr2=0x7f059810ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.189+0000 7f059f9ce640 1 --2- 172.21.15.38:0/2184090238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059810a710 0x7f059810ab20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0588009900 tx=0x7f058802a440 comp rx=0 tx=0).stop 2024-05-21T15:39:42.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.190+0000 7f059f9ce640 1 -- 172.21.15.38:0/2184090238 shutdown_connections 2024-05-21T15:39:42.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.191+0000 7f059f9ce640 1 --2- 172.21.15.38:0/2184090238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f059810f410 0x7f0598111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.191+0000 7f059f9ce640 1 --2- 172.21.15.38:0/2184090238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059810a710 0x7f059810ab20 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.191+0000 7f059f9ce640 1 --2- 172.21.15.38:0/2184090238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05981094d0 0x7f059810a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.191+0000 7f059f9ce640 1 -- 172.21.15.38:0/2184090238 >> 172.21.15.38:0/2184090238 conn(0x7f059806f4a0 msgr2=0x7f0598071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:42.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.191+0000 7f059f9ce640 1 -- 172.21.15.38:0/2184090238 shutdown_connections 2024-05-21T15:39:42.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.191+0000 7f059f9ce640 1 -- 172.21.15.38:0/2184090238 wait complete. 2024-05-21T15:39:42.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.193+0000 7f059f9ce640 1 Processor -- start 2024-05-21T15:39:42.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.193+0000 7f059f9ce640 1 -- start start 2024-05-21T15:39:42.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.194+0000 7f059f9ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05981094d0 0x7f05981a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.194+0000 7f059f9ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f059810a710 0x7f05981a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.194+0000 7f059f9ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059810f410 0x7f05981a7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.194+0000 7f059f9ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0598115650 con 0x7f059810f410 2024-05-21T15:39:42.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.194+0000 7f059f9ce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f05981154d0 con 0x7f05981094d0 2024-05-21T15:39:42.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.194+0000 7f059f9ce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f05981157d0 con 0x7f059810a710 2024-05-21T15:39:42.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.195+0000 7f059df6b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059810f410 0x7f05981a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.195+0000 7f059df6b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059810f410 0x7f05981a7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34116/0 (socket says 172.21.15.38:34116) 2024-05-21T15:39:42.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.195+0000 7f059df6b640 1 -- 172.21.15.38:0/3121120257 learned_addr learned my addr 172.21.15.38:0/3121120257 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:42.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.195+0000 7f059cf69640 1 --2- 172.21.15.38:0/3121120257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f059810a710 0x7f05981a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.195+0000 7f059d76a640 1 --2- 172.21.15.38:0/3121120257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05981094d0 0x7f05981a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.196+0000 7f059df6b640 1 -- 172.21.15.38:0/3121120257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f059810a710 msgr2=0x7f05981a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.196+0000 7f059df6b640 1 --2- 172.21.15.38:0/3121120257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f059810a710 0x7f05981a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.196+0000 7f059df6b640 1 -- 172.21.15.38:0/3121120257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05981094d0 msgr2=0x7f05981a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.196+0000 7f059df6b640 1 --2- 172.21.15.38:0/3121120257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05981094d0 0x7f05981a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.196+0000 7f059df6b640 1 -- 172.21.15.38:0/3121120257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05981a7cb0 con 0x7f059810f410 2024-05-21T15:39:42.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.196+0000 7f059df6b640 1 --2- 172.21.15.38:0/3121120257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059810f410 0x7f05981a7640 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f059000b970 tx=0x7f059000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.197+0000 7f058e7fc640 1 -- 172.21.15.38:0/3121120257 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059001b070 con 0x7f059810f410 2024-05-21T15:39:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.197+0000 7f058e7fc640 1 -- 172.21.15.38:0/3121120257 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f059000ed50 con 0x7f059810f410 2024-05-21T15:39:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.197+0000 7f058e7fc640 1 -- 172.21.15.38:0/3121120257 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05900078c0 con 0x7f059810f410 2024-05-21T15:39:42.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.198+0000 7f059f9ce640 1 -- 172.21.15.38:0/3121120257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0598111dd0 con 0x7f059810f410 2024-05-21T15:39:42.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.198+0000 7f059f9ce640 1 -- 172.21.15.38:0/3121120257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981122a0 con 0x7f059810f410 2024-05-21T15:39:42.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:41 smithi053 ceph-mon[36496]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 51 MiB/s wr, 11.18k op/s 2024-05-21T15:39:42.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:41 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3378935410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:39:42.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.208+0000 7f058e7fc640 1 -- 172.21.15.38:0/3121120257 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f059001e070 con 0x7f059810f410 2024-05-21T15:39:42.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.209+0000 7f058e7fc640 1 --2- 172.21.15.38:0/3121120257 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05840a2d90 0x7f05840a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.210+0000 7f058e7fc640 1 -- 172.21.15.38:0/3121120257 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f05900d17f0 con 0x7f059810f410 2024-05-21T15:39:42.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.211+0000 7f05a6877740 1 Processor -- start 2024-05-21T15:39:42.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.212+0000 7f059d76a640 1 --2- 172.21.15.38:0/3121120257 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05840a2d90 0x7f05840a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.213+0000 7f05a6877740 1 -- start start 2024-05-21T15:39:42.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.214+0000 7f05a6877740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81c2390 0x5634e8561b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.214+0000 7f05a6877740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 0x5634e8562080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.214+0000 7f05a6877740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81a16b0 0x5634e8568f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.214+0000 7f05a6877740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5634e84c6c40 con 0x5634e81a16b0 2024-05-21T15:39:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.214+0000 7f05a6877740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5634e846e580 con 0x5634e81994d0 2024-05-21T15:39:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.214+0000 7f05a6877740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5634e84c2c00 con 0x5634e81c2390 2024-05-21T15:39:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.214+0000 7f059df6b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81a16b0 0x5634e8568f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.214+0000 7f059df6b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81a16b0 0x5634e8568f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34126/0 (socket says 172.21.15.38:34126) 2024-05-21T15:39:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.214+0000 7f059df6b640 1 -- 172.21.15.38:0/3759782973 learned_addr learned my addr 172.21.15.38:0/3759782973 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.215+0000 7f059df6b640 1 -- 172.21.15.38:0/3759782973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81c2390 msgr2=0x5634e8561b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:39:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.215+0000 7f059df6b640 1 --2- 172.21.15.38:0/3759782973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81c2390 0x5634e8561b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.215+0000 7f059df6b640 1 -- 172.21.15.38:0/3759782973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 msgr2=0x5634e8562080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:39:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.215+0000 7f059df6b640 1 --2- 172.21.15.38:0/3759782973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 0x5634e8562080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.215+0000 7f059df6b640 1 -- 172.21.15.38:0/3759782973 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e81da3f0 con 0x5634e81a16b0 2024-05-21T15:39:42.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.215+0000 7f059df6b640 1 --2- 172.21.15.38:0/3759782973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81a16b0 0x5634e8568f80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f059000b940 tx=0x7f0590016df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5024e05a097a4e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:42.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.217+0000 7f05837fe640 1 -- 172.21.15.38:0/3759782973 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05900be070 con 0x5634e81a16b0 2024-05-21T15:39:42.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.217+0000 7f05837fe640 1 -- 172.21.15.38:0/3759782973 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05900a2350 con 0x5634e81a16b0 2024-05-21T15:39:42.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.217+0000 7f05837fe640 1 -- 172.21.15.38:0/3759782973 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05900a6780 con 0x5634e81a16b0 2024-05-21T15:39:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.219+0000 7f05a6877740 1 -- 172.21.15.38:0/3759782973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81a16b0 msgr2=0x5634e8568f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.219+0000 7f05a6877740 1 --2- 172.21.15.38:0/3759782973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81a16b0 0x5634e8568f80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f059000b940 tx=0x7f0590016df0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.222+0000 7f05a6877740 1 -- 172.21.15.38:0/3759782973 shutdown_connections 2024-05-21T15:39:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.222+0000 7f05a6877740 1 --2- 172.21.15.38:0/3759782973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81a16b0 0x5634e8568f80 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.222+0000 7f05a6877740 1 --2- 172.21.15.38:0/3759782973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 0x5634e8562080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.222+0000 7f05a6877740 1 --2- 172.21.15.38:0/3759782973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81c2390 0x5634e8561b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.222+0000 7f05a6877740 1 -- 172.21.15.38:0/3759782973 >> 172.21.15.38:0/3759782973 conn(0x5634e81b64e0 msgr2=0x5634e855d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.222+0000 7f059d76a640 1 --2- 172.21.15.38:0/3121120257 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05840a2d90 0x7f05840a52a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f059810a0d0 tx=0x7f0594005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.222+0000 7f05a6877740 1 -- 172.21.15.38:0/3759782973 shutdown_connections 2024-05-21T15:39:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.222+0000 7f05a6877740 1 -- 172.21.15.38:0/3759782973 wait complete. 2024-05-21T15:39:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.222+0000 7f05a6877740 1 -- start start 2024-05-21T15:39:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.222+0000 7f05a6877740 1 Processor -- start 2024-05-21T15:39:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.223+0000 7f05a6877740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 0x5634e856a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.223+0000 7f05a6877740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 0x5634e856ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.223+0000 7f05a6877740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81c2390 0x5634e8571d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.223+0000 7f05a6877740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5634e8569910 con 0x5634e81c2390 2024-05-21T15:39:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.223+0000 7f059cf69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 0x5634e856ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.224+0000 7f059cf69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 0x5634e856ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:36608/0 (socket says 172.21.15.38:36608) 2024-05-21T15:39:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.224+0000 7f059cf69640 1 -- 172.21.15.38:0/1875635424 learned_addr learned my addr 172.21.15.38:0/1875635424 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.223+0000 7f059d76a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 0x5634e856a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.224+0000 7f05a6877740 1 -- 172.21.15.38:0/1875635424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5634e8481c40 con 0x5634e81994d0 2024-05-21T15:39:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.224+0000 7f05a6877740 1 -- 172.21.15.38:0/1875635424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5634e8569a90 con 0x5634e81a16b0 2024-05-21T15:39:42.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.224+0000 7f05a6877740 10 client.0 _ll_register_callbacks cb 0x5634e81d9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:42.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.224+0000 7f059d76a640 1 -- 172.21.15.38:0/1875635424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 msgr2=0x5634e856ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:42.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.224+0000 7f059d76a640 1 --2- 172.21.15.38:0/1875635424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 0x5634e856ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.224+0000 7f059d76a640 1 -- 172.21.15.38:0/1875635424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81c2390 msgr2=0x5634e8571d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:42.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.224+0000 7f059d76a640 1 --2- 172.21.15.38:0/1875635424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81c2390 0x5634e8571d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.225+0000 7f059d76a640 1 -- 172.21.15.38:0/1875635424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634e82a47d0 con 0x5634e81994d0 2024-05-21T15:39:42.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.226+0000 7f059d76a640 1 --2- 172.21.15.38:0/1875635424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 0x5634e856a8a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5634e8562cd0 tx=0x7f0594004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a6ee42a82d16934 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:42.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.228+0000 7f0580ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:39:42.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.228+0000 7f0580ff9640 1 -- 172.21.15.38:0/1875635424 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059400fb20 con 0x5634e81994d0 2024-05-21T15:39:42.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.228+0000 7f05a6877740 1 -- 172.21.15.38:0/1875635424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05900a2350 con 0x5634e81994d0 2024-05-21T15:39:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.230+0000 7f0580ff9640 1 -- 172.21.15.38:0/1875635424 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f059400fcc0 con 0x5634e81994d0 2024-05-21T15:39:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.230+0000 7f0580ff9640 1 -- 172.21.15.38:0/1875635424 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f059401f420 con 0x5634e81994d0 2024-05-21T15:39:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.230+0000 7f05a6877740 10 client.26453 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.230+0000 7f05a6877740 1 -- 172.21.15.38:0/1875635424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5634e85761f0 con 0x5634e81994d0 2024-05-21T15:39:42.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.233+0000 7f0580ff9640 1 -- 172.21.15.38:0/1875635424 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05900a2350 con 0x5634e81994d0 2024-05-21T15:39:42.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.234+0000 7f0580ff9640 1 -- 172.21.15.38:0/1875635424 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f05940276a0 con 0x5634e81994d0 2024-05-21T15:39:42.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.234+0000 7f05a6877740 10 client.26453 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:42.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.234+0000 7f05a6877740 10 client.26453 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:39:42.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.234+0000 7f05a6877740 20 client.26453 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:42.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.234+0000 7f05a6877740 1 --2- 172.21.15.38:0/1875635424 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5634e8180f60 0x5634e8579d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.234+0000 7f05a6877740 4 client.26453 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:42.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.234+0000 7f05a6877740 1 -- 172.21.15.38:0/1875635424 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634e84d4370 con 0x5634e8180f60 2024-05-21T15:39:42.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.236+0000 7f059cf69640 1 --2- 172.21.15.38:0/1875635424 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5634e8180f60 0x5634e8579d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.239+0000 7f059cf69640 1 --2- 172.21.15.38:0/1875635424 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5634e8180f60 0x5634e8579d30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1b37161b8e3cd404 server_cookie=4aacc89b009aa100 in_seq=0 out_seq=0 2024-05-21T15:39:42.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.239+0000 7f0580ff9640 10 client.26453 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:39:42.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.243+0000 7f0580ff9640 1 -- 172.21.15.38:0/1875635424 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5634e84d4370 con 0x5634e8180f60 2024-05-21T15:39:42.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.243+0000 7f0580ff9640 10 client.26453 handle_command_reply: tid=0 2024-05-21T15:39:42.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.243+0000 7f05a6877740 1 client.26453 shutdown 2024-05-21T15:39:42.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.245+0000 7f05a6877740 1 -- 172.21.15.38:0/1875635424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 msgr2=0x5634e856a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.245+0000 7f05a6877740 1 --2- 172.21.15.38:0/1875635424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 0x5634e856a8a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5634e8562cd0 tx=0x7f0594004730 comp rx=0 tx=0).stop 2024-05-21T15:39:42.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.247+0000 7f05a6877740 1 -- 172.21.15.38:0/1875635424 shutdown_connections 2024-05-21T15:39:42.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.247+0000 7f05a6877740 1 --2- 172.21.15.38:0/1875635424 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5634e8180f60 0x5634e8579d30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.247+0000 7f05a6877740 1 --2- 172.21.15.38:0/1875635424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81c2390 0x5634e8571d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.247+0000 7f05a6877740 1 --2- 172.21.15.38:0/1875635424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 0x5634e856ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.247+0000 7f05a6877740 1 --2- 172.21.15.38:0/1875635424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 0x5634e856a8a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.247+0000 7f05a6877740 1 -- 172.21.15.38:0/1875635424 >> 172.21.15.38:0/1875635424 conn(0x5634e81b64e0 msgr2=0x5634e855c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:42.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.247+0000 7f0580ff9640 0 client.26453 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:39:42.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.248+0000 7f05a6877740 1 -- 172.21.15.38:0/1875635424 shutdown_connections 2024-05-21T15:39:42.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.248+0000 7f05a6877740 1 -- 172.21.15.38:0/1875635424 wait complete. 2024-05-21T15:39:42.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.248+0000 7f05a6877740 20 client.26453 trim_cache size 0 max 16384 2024-05-21T15:39:42.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:41 smithi045 ceph-mon[36409]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 51 MiB/s wr, 11.18k op/s 2024-05-21T15:39:42.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:41 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3378935410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:39:42.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:41 smithi038 ceph-mon[31995]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 51 MiB/s wr, 11.18k op/s 2024-05-21T15:39:42.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:41 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3378935410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:39:42.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.282+0000 7f05a6877740 1 Processor -- start 2024-05-21T15:39:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.282+0000 7f05a6877740 1 -- start start 2024-05-21T15:39:42.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.283+0000 7f05a6877740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 0x5634e8579e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.283+0000 7f05a6877740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 0x5634e8571cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.283+0000 7f05a6877740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81c2390 0x5634e8564e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.283+0000 7f059d76a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 0x5634e8579e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.283+0000 7f059d76a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 0x5634e8579e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50244/0 (socket says 172.21.15.38:50244) 2024-05-21T15:39:42.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.283+0000 7f059d76a640 1 -- 172.21.15.38:0/2873989888 learned_addr learned my addr 172.21.15.38:0/2873989888 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:42.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.283+0000 7f059cf69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 0x5634e8571cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.283+0000 7f05a6877740 1 -- 172.21.15.38:0/2873989888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05981b0b20 con 0x5634e81c2390 2024-05-21T15:39:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.283+0000 7f05a6877740 1 -- 172.21.15.38:0/2873989888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f05981b2750 con 0x5634e81994d0 2024-05-21T15:39:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.283+0000 7f05a6877740 1 -- 172.21.15.38:0/2873989888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f05981b11f0 con 0x5634e81a16b0 2024-05-21T15:39:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.286+0000 7f059d76a640 1 -- 172.21.15.38:0/2873989888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 msgr2=0x5634e8571cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.286+0000 7f059d76a640 1 --2- 172.21.15.38:0/2873989888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 0x5634e8571cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.286+0000 7f059d76a640 1 -- 172.21.15.38:0/2873989888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81c2390 msgr2=0x5634e8564e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.286+0000 7f059d76a640 1 --2- 172.21.15.38:0/2873989888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81c2390 0x5634e8564e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.286+0000 7f059d76a640 1 -- 172.21.15.38:0/2873989888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e8558d20 con 0x5634e81994d0 2024-05-21T15:39:42.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.290+0000 7f059d76a640 1 --2- 172.21.15.38:0/2873989888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 0x5634e8579e70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0594012fa0 tx=0x7f0594027660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f14c8a286ad672db server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:42.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.293+0000 7f0582ffd640 1 -- 172.21.15.38:0/2873989888 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594027850 con 0x5634e81994d0 2024-05-21T15:39:42.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.294+0000 7f0582ffd640 1 -- 172.21.15.38:0/2873989888 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f059400fd80 con 0x5634e81994d0 2024-05-21T15:39:42.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.295+0000 7f05a6877740 1 -- 172.21.15.38:0/2873989888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 msgr2=0x5634e8579e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.295+0000 7f05a6877740 1 --2- 172.21.15.38:0/2873989888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 0x5634e8579e70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0594012fa0 tx=0x7f0594027660 comp rx=0 tx=0).stop 2024-05-21T15:39:42.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.299+0000 7f05a6877740 1 -- 172.21.15.38:0/2873989888 shutdown_connections 2024-05-21T15:39:42.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.299+0000 7f05a6877740 1 --2- 172.21.15.38:0/2873989888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81c2390 0x5634e8564e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.299+0000 7f05a6877740 1 --2- 172.21.15.38:0/2873989888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 0x5634e8571cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.299+0000 7f05a6877740 1 --2- 172.21.15.38:0/2873989888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81994d0 0x5634e8579e70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.299+0000 7f05a6877740 1 -- 172.21.15.38:0/2873989888 >> 172.21.15.38:0/2873989888 conn(0x5634e81b64e0 msgr2=0x5634e8572700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:42.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.306+0000 7f05a6877740 1 -- 172.21.15.38:0/2873989888 shutdown_connections 2024-05-21T15:39:42.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.307+0000 7f05a6877740 1 -- 172.21.15.38:0/2873989888 wait complete. 2024-05-21T15:39:42.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.308+0000 7f05a6877740 1 -- start start 2024-05-21T15:39:42.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.308+0000 7f05a6877740 1 Processor -- start 2024-05-21T15:39:42.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.309+0000 7f05a6877740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81994d0 0x5634e8571600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.309+0000 7f05a6877740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 0x5634e8571b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.309+0000 7f059d76a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81994d0 0x5634e8571600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.309+0000 7f059d76a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81994d0 0x5634e8571600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34152/0 (socket says 172.21.15.38:34152) 2024-05-21T15:39:42.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.309+0000 7f059d76a640 1 -- 172.21.15.38:0/4237903563 learned_addr learned my addr 172.21.15.38:0/4237903563 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:42.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.309+0000 7f05a6877740 1 --2- 172.21.15.38:0/4237903563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81c2390 0x5634e856fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.310+0000 7f05a6877740 1 -- 172.21.15.38:0/4237903563 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05981b1fc0 con 0x5634e81994d0 2024-05-21T15:39:42.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.310+0000 7f059cf69640 1 --2- 172.21.15.38:0/4237903563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 0x5634e8571b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.310+0000 7f05a6877740 1 -- 172.21.15.38:0/4237903563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f05981b18c0 con 0x5634e81c2390 2024-05-21T15:39:42.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.310+0000 7f05a6877740 1 -- 172.21.15.38:0/4237903563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f05981aeb00 con 0x5634e81a16b0 2024-05-21T15:39:42.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.310+0000 7f059df6b640 1 --2- 172.21.15.38:0/4237903563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81c2390 0x5634e856fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.310+0000 7f05a6877740 10 client.0 _ll_register_callbacks cb 0x7f0594009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:42.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.310+0000 7f059cf69640 1 -- 172.21.15.38:0/4237903563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81c2390 msgr2=0x5634e856fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:42.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.310+0000 7f059cf69640 1 --2- 172.21.15.38:0/4237903563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81c2390 0x5634e856fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.310+0000 7f059cf69640 1 -- 172.21.15.38:0/4237903563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81994d0 msgr2=0x5634e8571600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:42.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.310+0000 7f059cf69640 1 --2- 172.21.15.38:0/4237903563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81994d0 0x5634e8571600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.310+0000 7f059cf69640 1 -- 172.21.15.38:0/4237903563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634e85703d0 con 0x5634e81a16b0 2024-05-21T15:39:42.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.311+0000 7f059cf69640 1 --2- 172.21.15.38:0/4237903563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 0x5634e8571b40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5634e856b910 tx=0x7f0588003640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c67a9fc91f39672e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:42.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.312+0000 7f0580ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:39:42.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.312+0000 7f0580ff9640 1 -- 172.21.15.38:0/4237903563 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588005000 con 0x5634e81a16b0 2024-05-21T15:39:42.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.312+0000 7f0580ff9640 1 -- 172.21.15.38:0/4237903563 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f05880051a0 con 0x5634e81a16b0 2024-05-21T15:39:42.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.312+0000 7f05a6877740 1 -- 172.21.15.38:0/4237903563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f059400fd80 con 0x5634e81a16b0 2024-05-21T15:39:42.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.314+0000 7f0580ff9640 1 -- 172.21.15.38:0/4237903563 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0588057050 con 0x5634e81a16b0 2024-05-21T15:39:42.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.314+0000 7f05a6877740 10 client.26530 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:42.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.314+0000 7f05a6877740 1 -- 172.21.15.38:0/4237903563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5634e8603040 con 0x5634e81a16b0 2024-05-21T15:39:42.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.315+0000 7f0580ff9640 1 -- 172.21.15.38:0/4237903563 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f058804b070 con 0x5634e81a16b0 2024-05-21T15:39:42.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.315+0000 7f0580ff9640 1 -- 172.21.15.38:0/4237903563 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f058803b550 con 0x5634e81a16b0 2024-05-21T15:39:42.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.317+0000 7f05a6877740 10 client.26530 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:42.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.317+0000 7f05a6877740 10 client.26530 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:39:42.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.317+0000 7f05a6877740 20 client.26530 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:42.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.317+0000 7f05a6877740 1 --2- 172.21.15.38:0/4237903563 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5634e81563a0 0x5634e8605a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.317+0000 7f05a6877740 4 client.26530 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:39:42.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.317+0000 7f05a6877740 1 -- 172.21.15.38:0/4237903563 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f059400fd80 con 0x5634e81563a0 2024-05-21T15:39:42.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.318+0000 7f059d76a640 1 --2- 172.21.15.38:0/4237903563 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5634e81563a0 0x5634e8605a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.321+0000 7f059d76a640 1 --2- 172.21.15.38:0/4237903563 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5634e81563a0 0x5634e8605a10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eb60ca2911712e6d server_cookie=b0fbe67c0fc4d693 in_seq=0 out_seq=0 2024-05-21T15:39:42.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.325+0000 7f0580ff9640 10 client.26530 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:39:42.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.325+0000 7f0580ff9640 1 -- 172.21.15.38:0/4237903563 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f059400fd80 con 0x5634e81563a0 2024-05-21T15:39:42.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.325+0000 7f0580ff9640 10 client.26530 handle_command_reply: tid=0 2024-05-21T15:39:42.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.325+0000 7f05a6877740 1 client.26530 shutdown 2024-05-21T15:39:42.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.328+0000 7f05a6877740 1 -- 172.21.15.38:0/4237903563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 msgr2=0x5634e8571b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.328+0000 7f05a6877740 1 --2- 172.21.15.38:0/4237903563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 0x5634e8571b40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5634e856b910 tx=0x7f0588003640 comp rx=0 tx=0).stop 2024-05-21T15:39:42.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.332+0000 7f05a6877740 1 -- 172.21.15.38:0/4237903563 shutdown_connections 2024-05-21T15:39:42.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.332+0000 7f05a6877740 1 --2- 172.21.15.38:0/4237903563 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5634e81563a0 0x5634e8605a10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.332+0000 7f05a6877740 1 --2- 172.21.15.38:0/4237903563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634e81c2390 0x5634e856fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.332+0000 7f05a6877740 1 --2- 172.21.15.38:0/4237903563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5634e81a16b0 0x5634e8571b40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.332+0000 7f05a6877740 1 --2- 172.21.15.38:0/4237903563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5634e81994d0 0x5634e8571600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.332+0000 7f05a6877740 1 -- 172.21.15.38:0/4237903563 >> 172.21.15.38:0/4237903563 conn(0x5634e81b64e0 msgr2=0x5634e855ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:42.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.332+0000 7f0580ff9640 0 client.26530 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:39:42.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.338+0000 7f05a6877740 1 -- 172.21.15.38:0/4237903563 shutdown_connections 2024-05-21T15:39:42.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.340+0000 7f05a6877740 1 -- 172.21.15.38:0/4237903563 wait complete. 2024-05-21T15:39:42.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.341+0000 7f05a6877740 20 client.26530 trim_cache size 0 max 16384 2024-05-21T15:39:42.342 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:39:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.347+0000 7f059f9ce640 1 -- 172.21.15.38:0/3121120257 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05840a2d90 msgr2=0x7f05840a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.347+0000 7f059f9ce640 1 --2- 172.21.15.38:0/3121120257 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05840a2d90 0x7f05840a52a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f059810a0d0 tx=0x7f0594005f90 comp rx=0 tx=0).stop 2024-05-21T15:39:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.347+0000 7f059f9ce640 1 -- 172.21.15.38:0/3121120257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059810f410 msgr2=0x7f05981a7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.347+0000 7f059f9ce640 1 --2- 172.21.15.38:0/3121120257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059810f410 0x7f05981a7640 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f059000b970 tx=0x7f059000be30 comp rx=0 tx=0).stop 2024-05-21T15:39:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.347+0000 7f059f9ce640 1 -- 172.21.15.38:0/3121120257 shutdown_connections 2024-05-21T15:39:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.347+0000 7f059f9ce640 1 --2- 172.21.15.38:0/3121120257 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05840a2d90 0x7f05840a52a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.347+0000 7f059f9ce640 1 --2- 172.21.15.38:0/3121120257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f059810f410 0x7f05981a7640 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.347+0000 7f059f9ce640 1 --2- 172.21.15.38:0/3121120257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f059810a710 0x7f05981a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.347+0000 7f059f9ce640 1 --2- 172.21.15.38:0/3121120257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05981094d0 0x7f05981a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.347+0000 7f059f9ce640 1 -- 172.21.15.38:0/3121120257 >> 172.21.15.38:0/3121120257 conn(0x7f059806f4a0 msgr2=0x7f059810b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:42.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.349+0000 7f059f9ce640 1 -- 172.21.15.38:0/3121120257 shutdown_connections 2024-05-21T15:39:42.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.349+0000 7f059f9ce640 1 -- 172.21.15.38:0/3121120257 wait complete. 2024-05-21T15:39:42.370 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:39:42.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.587+0000 7f85774e7640 1 -- 172.21.15.38:0/294041829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85700701b0 msgr2=0x7f85700705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.587+0000 7f85774e7640 1 --2- 172.21.15.38:0/294041829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85700701b0 0x7f85700705c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f856801acb0 tx=0x7f856803b920 comp rx=0 tx=0).stop 2024-05-21T15:39:42.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.590+0000 7f85774e7640 1 -- 172.21.15.38:0/294041829 shutdown_connections 2024-05-21T15:39:42.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.590+0000 7f85774e7640 1 --2- 172.21.15.38:0/294041829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85700701b0 0x7f85700705c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.590+0000 7f85774e7640 1 --2- 172.21.15.38:0/294041829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857006f4b0 0x7f857006f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.590+0000 7f85774e7640 1 --2- 172.21.15.38:0/294041829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f857006e220 0x7f857006ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.590+0000 7f85774e7640 1 -- 172.21.15.38:0/294041829 >> 172.21.15.38:0/294041829 conn(0x7f857006d430 msgr2=0x7f857010ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:42.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.596+0000 7f85774e7640 1 -- 172.21.15.38:0/294041829 shutdown_connections 2024-05-21T15:39:42.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.599+0000 7f85774e7640 1 -- 172.21.15.38:0/294041829 wait complete. 2024-05-21T15:39:42.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.600+0000 7f85774e7640 1 Processor -- start 2024-05-21T15:39:42.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.603+0000 7f85774e7640 1 -- start start 2024-05-21T15:39:42.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.603+0000 7f85774e7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857006e220 0x7f857019f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.603+0000 7f85774e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f857006f4b0 0x7f857019feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.603+0000 7f85774e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85700701b0 0x7f85701a6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.603+0000 7f85774e7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f857010eda0 con 0x7f85700701b0 2024-05-21T15:39:42.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.603+0000 7f85774e7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f857010ec20 con 0x7f857006e220 2024-05-21T15:39:42.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.603+0000 7f85774e7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f857010ef20 con 0x7f857006f4b0 2024-05-21T15:39:42.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.604+0000 7f8575a84640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85700701b0 0x7f85701a6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.604+0000 7f8575a84640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85700701b0 0x7f85701a6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34174/0 (socket says 172.21.15.38:34174) 2024-05-21T15:39:42.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.604+0000 7f8575a84640 1 -- 172.21.15.38:0/300462957 learned_addr learned my addr 172.21.15.38:0/300462957 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:42.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.606+0000 7f8575a84640 1 -- 172.21.15.38:0/300462957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f857006f4b0 msgr2=0x7f857019feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.606+0000 7f8575a84640 1 --2- 172.21.15.38:0/300462957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f857006f4b0 0x7f857019feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.606+0000 7f8575a84640 1 -- 172.21.15.38:0/300462957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857006e220 msgr2=0x7f857019f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.606+0000 7f8575a84640 1 --2- 172.21.15.38:0/300462957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857006e220 0x7f857019f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.606+0000 7f8575a84640 1 -- 172.21.15.38:0/300462957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85701a74b0 con 0x7f85700701b0 2024-05-21T15:39:42.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.609+0000 7f8575a84640 1 --2- 172.21.15.38:0/300462957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85700701b0 0x7f85701a6e40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8568040c70 tx=0x7f8568040ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:42.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.611+0000 7f85667fc640 1 -- 172.21.15.38:0/300462957 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568045070 con 0x7f85700701b0 2024-05-21T15:39:42.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.611+0000 7f85774e7640 1 -- 172.21.15.38:0/300462957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85701a7740 con 0x7f85700701b0 2024-05-21T15:39:42.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.611+0000 7f85774e7640 1 -- 172.21.15.38:0/300462957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85701a7cb0 con 0x7f85700701b0 2024-05-21T15:39:42.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.611+0000 7f85667fc640 1 -- 172.21.15.38:0/300462957 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8568040ec0 con 0x7f85700701b0 2024-05-21T15:39:42.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.611+0000 7f85667fc640 1 -- 172.21.15.38:0/300462957 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856804a430 con 0x7f85700701b0 2024-05-21T15:39:42.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.617+0000 7f857e382740 1 Processor -- start 2024-05-21T15:39:42.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.617+0000 7f857e382740 1 -- start start 2024-05-21T15:39:42.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.617+0000 7f857e382740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe7396390 0x55abe7735b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.617+0000 7f857e382740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe736d4d0 0x55abe7736080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.617+0000 7f857e382740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 0x55abe773cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.617+0000 7f857e382740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55abe769ac40 con 0x55abe7396390 2024-05-21T15:39:42.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.617+0000 7f857e382740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55abe7642580 con 0x55abe73756b0 2024-05-21T15:39:42.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.617+0000 7f857e382740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55abe7696c00 con 0x55abe736d4d0 2024-05-21T15:39:42.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.620+0000 7f8575283640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe736d4d0 0x55abe7736080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.620+0000 7f8575283640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe736d4d0 0x55abe7736080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:36640/0 (socket says 172.21.15.38:36640) 2024-05-21T15:39:42.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.620+0000 7f8575283640 1 -- 172.21.15.38:0/109993723 learned_addr learned my addr 172.21.15.38:0/109993723 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:42.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.620+0000 7f8574a82640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 0x55abe773cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.620+0000 7f8574a82640 1 --2- 172.21.15.38:0/109993723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe7396390 0x55abe7735b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.623+0000 7f8575283640 1 -- 172.21.15.38:0/109993723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 msgr2=0x55abe773cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:42.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.623+0000 7f85667fc640 1 -- 172.21.15.38:0/300462957 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8568048020 con 0x7f85700701b0 2024-05-21T15:39:42.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.624+0000 7f8575283640 1 --2- 172.21.15.38:0/109993723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 0x55abe773cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.624+0000 7f8575283640 1 -- 172.21.15.38:0/109993723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe7396390 msgr2=0x55abe7735b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:42.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.624+0000 7f8575283640 1 --2- 172.21.15.38:0/109993723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe7396390 0x55abe7735b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.624+0000 7f8575283640 1 -- 172.21.15.38:0/109993723 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abe73ae3f0 con 0x55abe736d4d0 2024-05-21T15:39:42.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.624+0000 7f8574a82640 1 --2- 172.21.15.38:0/109993723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe7396390 0x55abe7735b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:42.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.625+0000 7f85667fc640 1 --2- 172.21.15.38:0/300462957 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f855c0a2ce0 0x7f855c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.625+0000 7f8575283640 1 --2- 172.21.15.38:0/109993723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe736d4d0 0x55abe7736080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f856c0097b0 tx=0x7f856c009c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2195e285687699e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:42.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.625+0000 7f85667fc640 1 -- 172.21.15.38:0/300462957 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f85681010e0 con 0x7f85700701b0 2024-05-21T15:39:42.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.626+0000 7f855b7fe640 1 -- 172.21.15.38:0/109993723 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c015020 con 0x55abe736d4d0 2024-05-21T15:39:42.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.626+0000 7f855b7fe640 1 -- 172.21.15.38:0/109993723 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f856c011420 con 0x55abe736d4d0 2024-05-21T15:39:42.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.627+0000 7f857e382740 1 -- 172.21.15.38:0/109993723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe736d4d0 msgr2=0x55abe7736080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.627+0000 7f857e382740 1 --2- 172.21.15.38:0/109993723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe736d4d0 0x55abe7736080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f856c0097b0 tx=0x7f856c009c70 comp rx=0 tx=0).stop 2024-05-21T15:39:42.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.628+0000 7f8574a82640 1 --2- 172.21.15.38:0/300462957 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f855c0a2ce0 0x7f855c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.632+0000 7f8574a82640 1 --2- 172.21.15.38:0/300462957 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f855c0a2ce0 0x7f855c0a51f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55abe7736cd0 tx=0x7f8560006940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:42.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.633+0000 7f857e382740 1 -- 172.21.15.38:0/109993723 shutdown_connections 2024-05-21T15:39:42.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.633+0000 7f857e382740 1 --2- 172.21.15.38:0/109993723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 0x55abe773cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.633+0000 7f857e382740 1 --2- 172.21.15.38:0/109993723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe736d4d0 0x55abe7736080 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.633+0000 7f857e382740 1 --2- 172.21.15.38:0/109993723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe7396390 0x55abe7735b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.633+0000 7f857e382740 1 -- 172.21.15.38:0/109993723 >> 172.21.15.38:0/109993723 conn(0x55abe738a4e0 msgr2=0x55abe7731490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:42.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.637+0000 7f857e382740 1 -- 172.21.15.38:0/109993723 shutdown_connections 2024-05-21T15:39:42.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.639+0000 7f857e382740 1 -- 172.21.15.38:0/109993723 wait complete. 2024-05-21T15:39:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.639+0000 7f857e382740 1 -- start start 2024-05-21T15:39:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.639+0000 7f857e382740 1 Processor -- start 2024-05-21T15:39:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.640+0000 7f857e382740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe736d4d0 0x55abe773f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.640+0000 7f857e382740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 0x55abe773fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.640+0000 7f857e382740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe7396390 0x55abe7746dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.640+0000 7f857e382740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55abe773da20 con 0x55abe736d4d0 2024-05-21T15:39:42.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.640+0000 7f857e382740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55abe7655c40 con 0x55abe73756b0 2024-05-21T15:39:42.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.640+0000 7f857e382740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55abe773dba0 con 0x55abe7396390 2024-05-21T15:39:42.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.640+0000 7f857e382740 10 client.0 _ll_register_callbacks cb 0x55abe73adbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:42.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.641+0000 7f8575a84640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe7396390 0x55abe7746dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.641+0000 7f8575a84640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe7396390 0x55abe7746dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:36644/0 (socket says 172.21.15.38:36644) 2024-05-21T15:39:42.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.641+0000 7f8575a84640 1 -- 172.21.15.38:0/4141146755 learned_addr learned my addr 172.21.15.38:0/4141146755 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:42.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.643+0000 7f8575a84640 1 -- 172.21.15.38:0/4141146755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 msgr2=0x55abe773fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:39:42.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.643+0000 7f8575a84640 1 --2- 172.21.15.38:0/4141146755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 0x55abe773fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.643+0000 7f8575a84640 1 -- 172.21.15.38:0/4141146755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe736d4d0 msgr2=0x55abe773f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:42.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.643+0000 7f8575a84640 1 --2- 172.21.15.38:0/4141146755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe736d4d0 0x55abe773f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.643+0000 7f8575a84640 1 -- 172.21.15.38:0/4141146755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abe74787d0 con 0x55abe7396390 2024-05-21T15:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.648+0000 7f8575a84640 1 --2- 172.21.15.38:0/4141146755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe7396390 0x55abe7746dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f85680c9ca0 tx=0x7f856801a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=922e3fb4ebfa6acf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:42.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.648+0000 7f8558ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:39:42.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.650+0000 7f8558ff9640 1 -- 172.21.15.38:0/4141146755 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85680e5020 con 0x55abe7396390 2024-05-21T15:39:42.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.650+0000 7f857e382740 1 -- 172.21.15.38:0/4141146755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f856c011420 con 0x55abe7396390 2024-05-21T15:39:42.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.650+0000 7f8558ff9640 1 -- 172.21.15.38:0/4141146755 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f85680c9f00 con 0x55abe7396390 2024-05-21T15:39:42.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.650+0000 7f8558ff9640 1 -- 172.21.15.38:0/4141146755 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f85680d6500 con 0x55abe7396390 2024-05-21T15:39:42.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.650+0000 7f857e382740 10 client.26536 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:42.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.650+0000 7f857e382740 1 -- 172.21.15.38:0/4141146755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55abe773db20 con 0x55abe7396390 2024-05-21T15:39:42.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.652+0000 7f8558ff9640 1 -- 172.21.15.38:0/4141146755 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f856c011420 con 0x55abe7396390 2024-05-21T15:39:42.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.652+0000 7f8558ff9640 1 -- 172.21.15.38:0/4141146755 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f85680cebc0 con 0x55abe7396390 2024-05-21T15:39:42.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.653+0000 7f857e382740 10 client.26536 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:42.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.653+0000 7f857e382740 10 client.26536 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:39:42.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.653+0000 7f857e382740 20 client.26536 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:42.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.653+0000 7f857e382740 1 --2- 172.21.15.38:0/4141146755 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55abe7354f60 0x55abe774db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.653+0000 7f857e382740 4 client.26536 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:42.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.653+0000 7f857e382740 1 -- 172.21.15.38:0/4141146755 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abe76a8370 con 0x55abe7354f60 2024-05-21T15:39:42.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.653+0000 7f8575283640 1 --2- 172.21.15.38:0/4141146755 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55abe7354f60 0x55abe774db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.655+0000 7f8558ff9640 10 client.26536 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:39:42.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.656+0000 7f8575283640 1 --2- 172.21.15.38:0/4141146755 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55abe7354f60 0x55abe774db40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=255ecaaa4f47d6 server_cookie=a948ad5e1fbbf201 in_seq=0 out_seq=0 2024-05-21T15:39:42.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.659+0000 7f8558ff9640 1 -- 172.21.15.38:0/4141146755 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55abe76a8370 con 0x55abe7354f60 2024-05-21T15:39:42.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.659+0000 7f8558ff9640 10 client.26536 handle_command_reply: tid=0 2024-05-21T15:39:42.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.659+0000 7f857e382740 1 client.26536 shutdown 2024-05-21T15:39:42.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.660+0000 7f857e382740 1 -- 172.21.15.38:0/4141146755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe7396390 msgr2=0x55abe7746dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.660+0000 7f857e382740 1 --2- 172.21.15.38:0/4141146755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe7396390 0x55abe7746dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f85680c9ca0 tx=0x7f856801a5d0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.666+0000 7f857e382740 1 -- 172.21.15.38:0/4141146755 shutdown_connections 2024-05-21T15:39:42.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.666+0000 7f857e382740 1 --2- 172.21.15.38:0/4141146755 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55abe7354f60 0x55abe774db40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.666+0000 7f857e382740 1 --2- 172.21.15.38:0/4141146755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe7396390 0x55abe7746dd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.666+0000 7f857e382740 1 --2- 172.21.15.38:0/4141146755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 0x55abe773fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.666+0000 7f857e382740 1 --2- 172.21.15.38:0/4141146755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe736d4d0 0x55abe773f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.666+0000 7f857e382740 1 -- 172.21.15.38:0/4141146755 >> 172.21.15.38:0/4141146755 conn(0x55abe738a4e0 msgr2=0x55abe7730570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:42.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.667+0000 7f8558ff9640 0 client.26536 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:39:42.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.667+0000 7f857e382740 1 -- 172.21.15.38:0/4141146755 shutdown_connections 2024-05-21T15:39:42.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.669+0000 7f857e382740 1 -- 172.21.15.38:0/4141146755 wait complete. 2024-05-21T15:39:42.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.671+0000 7f857e382740 20 client.26536 trim_cache size 0 max 16384 2024-05-21T15:39:42.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.701+0000 7f857e382740 1 Processor -- start 2024-05-21T15:39:42.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.703+0000 7f857e382740 1 -- start start 2024-05-21T15:39:42.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.704+0000 7f857e382740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe736d4d0 0x55abe7738f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.704+0000 7f857e382740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 0x55abe7746cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.704+0000 7f857e382740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe7396390 0x55abe773e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.704+0000 7f857e382740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f855400e620 con 0x55abe736d4d0 2024-05-21T15:39:42.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.704+0000 7f857e382740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8554010250 con 0x55abe73756b0 2024-05-21T15:39:42.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.704+0000 7f857e382740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f855400ecf0 con 0x55abe7396390 2024-05-21T15:39:42.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.706+0000 7f8574a82640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 0x55abe7746cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.706+0000 7f8574a82640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 0x55abe7746cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50294/0 (socket says 172.21.15.38:50294) 2024-05-21T15:39:42.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.706+0000 7f8574a82640 1 -- 172.21.15.38:0/51214806 learned_addr learned my addr 172.21.15.38:0/51214806 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:42.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.706+0000 7f8575a84640 1 --2- 172.21.15.38:0/51214806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe7396390 0x55abe773e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.706+0000 7f8575283640 1 --2- 172.21.15.38:0/51214806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe736d4d0 0x55abe7738f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.707+0000 7f8575283640 1 -- 172.21.15.38:0/51214806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe7396390 msgr2=0x55abe773e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:42.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.707+0000 7f8575283640 1 --2- 172.21.15.38:0/51214806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe7396390 0x55abe773e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.707+0000 7f8575283640 1 -- 172.21.15.38:0/51214806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 msgr2=0x55abe7746cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:42.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.707+0000 7f8575283640 1 --2- 172.21.15.38:0/51214806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 0x55abe7746cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.707+0000 7f8575283640 1 -- 172.21.15.38:0/51214806 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abe7747230 con 0x55abe736d4d0 2024-05-21T15:39:42.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.707+0000 7f8575283640 1 --2- 172.21.15.38:0/51214806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe736d4d0 0x55abe7738f20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x55abe77322c0 tx=0x7f856c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb51b4ff9564169 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:42.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.708+0000 7f855affd640 1 -- 172.21.15.38:0/51214806 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c02a070 con 0x55abe736d4d0 2024-05-21T15:39:42.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.708+0000 7f855affd640 1 -- 172.21.15.38:0/51214806 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f856c00bdb0 con 0x55abe736d4d0 2024-05-21T15:39:42.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.709+0000 7f857e382740 1 -- 172.21.15.38:0/51214806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe736d4d0 msgr2=0x55abe7738f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.709+0000 7f857e382740 1 --2- 172.21.15.38:0/51214806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe736d4d0 0x55abe7738f20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x55abe77322c0 tx=0x7f856c008040 comp rx=0 tx=0).stop 2024-05-21T15:39:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.713+0000 7f857e382740 1 -- 172.21.15.38:0/51214806 shutdown_connections 2024-05-21T15:39:42.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.713+0000 7f857e382740 1 --2- 172.21.15.38:0/51214806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe7396390 0x55abe773e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.713+0000 7f857e382740 1 --2- 172.21.15.38:0/51214806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe73756b0 0x55abe7746cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.713+0000 7f857e382740 1 --2- 172.21.15.38:0/51214806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe736d4d0 0x55abe7738f20 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.713+0000 7f857e382740 1 -- 172.21.15.38:0/51214806 >> 172.21.15.38:0/51214806 conn(0x55abe738a4e0 msgr2=0x55abe7739c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:42.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.717+0000 7f857e382740 1 -- 172.21.15.38:0/51214806 shutdown_connections 2024-05-21T15:39:42.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.717+0000 7f857e382740 1 -- 172.21.15.38:0/51214806 wait complete. 2024-05-21T15:39:42.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.718+0000 7f857e382740 1 -- start start 2024-05-21T15:39:42.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.718+0000 7f857e382740 1 Processor -- start 2024-05-21T15:39:42.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.719+0000 7f857e382740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe736d4d0 0x55abe7743d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.719+0000 7f857e382740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe73756b0 0x55abe7744270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.719+0000 7f857e382740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe7396390 0x55abe77b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.719+0000 7f8575283640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe736d4d0 0x55abe7743d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.719+0000 7f8575283640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe736d4d0 0x55abe7743d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50302/0 (socket says 172.21.15.38:50302) 2024-05-21T15:39:42.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.720+0000 7f857e382740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f855400fac0 con 0x55abe7396390 2024-05-21T15:39:42.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.720+0000 7f857e382740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f855400f3c0 con 0x55abe736d4d0 2024-05-21T15:39:42.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.720+0000 7f857e382740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f855400c600 con 0x55abe73756b0 2024-05-21T15:39:42.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.720+0000 7f8574a82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe73756b0 0x55abe7744270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.720+0000 7f8574a82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe73756b0 0x55abe7744270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:36660/0 (socket says 172.21.15.38:36660) 2024-05-21T15:39:42.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.720+0000 7f8574a82640 1 -- 172.21.15.38:0/404453252 learned_addr learned my addr 172.21.15.38:0/404453252 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:42.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.720+0000 7f857e382740 10 client.0 _ll_register_callbacks cb 0x7f85680c9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:42.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.720+0000 7f8574a82640 1 -- 172.21.15.38:0/404453252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe736d4d0 msgr2=0x55abe7743d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:42.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.720+0000 7f8574a82640 1 --2- 172.21.15.38:0/404453252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe736d4d0 0x55abe7743d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.720+0000 7f8574a82640 1 -- 172.21.15.38:0/404453252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe7396390 msgr2=0x55abe77b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:42.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.720+0000 7f8574a82640 1 --2- 172.21.15.38:0/404453252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe7396390 0x55abe77b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.720+0000 7f8574a82640 1 -- 172.21.15.38:0/404453252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abe77b4420 con 0x55abe73756b0 2024-05-21T15:39:42.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.721+0000 7f8574a82640 1 --2- 172.21.15.38:0/404453252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe73756b0 0x55abe7744270 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f856000dbd0 tx=0x7f85600068d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f182c13db8b5b54 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:42.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.721+0000 7f8558ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:39:42.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.723+0000 7f8558ff9640 1 -- 172.21.15.38:0/404453252 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856002b060 con 0x55abe73756b0 2024-05-21T15:39:42.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.723+0000 7f8558ff9640 1 -- 172.21.15.38:0/404453252 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8560016030 con 0x55abe73756b0 2024-05-21T15:39:42.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.724+0000 7f857e382740 1 -- 172.21.15.38:0/404453252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f856c00bdb0 con 0x55abe73756b0 2024-05-21T15:39:42.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.728+0000 7f8558ff9640 1 -- 172.21.15.38:0/404453252 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f856000d020 con 0x55abe73756b0 2024-05-21T15:39:42.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.728+0000 7f857e382740 10 client.26542 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:42.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.729+0000 7f857e382740 1 -- 172.21.15.38:0/404453252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55abe77b56d0 con 0x55abe73756b0 2024-05-21T15:39:42.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.740+0000 7f8558ff9640 1 -- 172.21.15.38:0/404453252 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f856000d1c0 con 0x55abe73756b0 2024-05-21T15:39:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.740+0000 7f8558ff9640 1 -- 172.21.15.38:0/404453252 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f856000c870 con 0x55abe73756b0 2024-05-21T15:39:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.740+0000 7f857e382740 10 client.26542 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.740+0000 7f857e382740 10 client.26542 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:39:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.740+0000 7f857e382740 20 client.26542 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.740+0000 7f857e382740 1 --2- 172.21.15.38:0/404453252 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55abe732a3a0 0x55abe77b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.740+0000 7f857e382740 4 client.26542 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:39:42.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.740+0000 7f857e382740 1 -- 172.21.15.38:0/404453252 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f856c00bdb0 con 0x55abe732a3a0 2024-05-21T15:39:42.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.743+0000 7f8575283640 1 --2- 172.21.15.38:0/404453252 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55abe732a3a0 0x55abe77b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:42.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.747+0000 7f8575283640 1 --2- 172.21.15.38:0/404453252 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55abe732a3a0 0x55abe77b80d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2f976caa3e95eee server_cookie=3ccb76633065e4ec in_seq=0 out_seq=0 2024-05-21T15:39:42.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.748+0000 7f8558ff9640 10 client.26542 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:39:42.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.749+0000 7f8558ff9640 1 -- 172.21.15.38:0/404453252 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f856c00bdb0 con 0x55abe732a3a0 2024-05-21T15:39:42.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.749+0000 7f8558ff9640 10 client.26542 handle_command_reply: tid=0 2024-05-21T15:39:42.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.749+0000 7f857e382740 1 client.26542 shutdown 2024-05-21T15:39:42.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.753+0000 7f857e382740 1 -- 172.21.15.38:0/404453252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe73756b0 msgr2=0x55abe7744270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.753+0000 7f857e382740 1 --2- 172.21.15.38:0/404453252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe73756b0 0x55abe7744270 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f856000dbd0 tx=0x7f85600068d0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.756+0000 7f857e382740 1 -- 172.21.15.38:0/404453252 shutdown_connections 2024-05-21T15:39:42.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.756+0000 7f857e382740 1 --2- 172.21.15.38:0/404453252 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55abe732a3a0 0x55abe77b80d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.756+0000 7f8558ff9640 0 client.26542 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:39:42.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.756+0000 7f857e382740 1 --2- 172.21.15.38:0/404453252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55abe7396390 0x55abe77b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.756+0000 7f857e382740 1 --2- 172.21.15.38:0/404453252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55abe73756b0 0x55abe7744270 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.756+0000 7f857e382740 1 --2- 172.21.15.38:0/404453252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abe736d4d0 0x55abe7743d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.756+0000 7f857e382740 1 -- 172.21.15.38:0/404453252 >> 172.21.15.38:0/404453252 conn(0x55abe738a4e0 msgr2=0x55abe773e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:42.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.763+0000 7f857e382740 1 -- 172.21.15.38:0/404453252 shutdown_connections 2024-05-21T15:39:42.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.763+0000 7f857e382740 1 -- 172.21.15.38:0/404453252 wait complete. 2024-05-21T15:39:42.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.764+0000 7f857e382740 20 client.26542 trim_cache size 0 max 16384 2024-05-21T15:39:42.763 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:39:42.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.769+0000 7f85774e7640 1 -- 172.21.15.38:0/300462957 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f855c0a2ce0 msgr2=0x7f855c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.769+0000 7f85774e7640 1 --2- 172.21.15.38:0/300462957 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f855c0a2ce0 0x7f855c0a51f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55abe7736cd0 tx=0x7f8560006940 comp rx=0 tx=0).stop 2024-05-21T15:39:42.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.769+0000 7f85774e7640 1 -- 172.21.15.38:0/300462957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85700701b0 msgr2=0x7f85701a6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:42.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.769+0000 7f85774e7640 1 --2- 172.21.15.38:0/300462957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85700701b0 0x7f85701a6e40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8568040c70 tx=0x7f8568040ca0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.769+0000 7f85774e7640 1 -- 172.21.15.38:0/300462957 shutdown_connections 2024-05-21T15:39:42.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.769+0000 7f85774e7640 1 --2- 172.21.15.38:0/300462957 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f855c0a2ce0 0x7f855c0a51f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.769+0000 7f85774e7640 1 --2- 172.21.15.38:0/300462957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85700701b0 0x7f85701a6e40 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.769+0000 7f85774e7640 1 --2- 172.21.15.38:0/300462957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f857006f4b0 0x7f857019feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.769+0000 7f85774e7640 1 --2- 172.21.15.38:0/300462957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f857006e220 0x7f857019f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:42.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.769+0000 7f85774e7640 1 -- 172.21.15.38:0/300462957 >> 172.21.15.38:0/300462957 conn(0x7f857006d430 msgr2=0x7f8570196490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:42.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.770+0000 7f85774e7640 1 -- 172.21.15.38:0/300462957 shutdown_connections 2024-05-21T15:39:42.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:42.770+0000 7f85774e7640 1 -- 172.21.15.38:0/300462957 wait complete. 2024-05-21T15:39:42.790 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:39:43.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.009+0000 7f4c84938640 1 -- 172.21.15.38:0/2986596186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c80103cc0 msgr2=0x7f4c801060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.009+0000 7f4c84938640 1 --2- 172.21.15.38:0/2986596186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c80103cc0 0x7f4c801060e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009900 tx=0x7f4c7002a420 comp rx=0 tx=0).stop 2024-05-21T15:39:43.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.014+0000 7f4c84938640 1 -- 172.21.15.38:0/2986596186 shutdown_connections 2024-05-21T15:39:43.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.014+0000 7f4c84938640 1 --2- 172.21.15.38:0/2986596186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c80106620 0x7f4c80106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.014+0000 7f4c84938640 1 --2- 172.21.15.38:0/2986596186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c80103cc0 0x7f4c801060e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.014+0000 7f4c84938640 1 --2- 172.21.15.38:0/2986596186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c80101360 0x7f4c80103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.014+0000 7f4c84938640 1 -- 172.21.15.38:0/2986596186 >> 172.21.15.38:0/2986596186 conn(0x7f4c800fcb70 msgr2=0x7f4c800ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:43.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.020+0000 7f4c84938640 1 -- 172.21.15.38:0/2986596186 shutdown_connections 2024-05-21T15:39:43.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.022+0000 7f4c84938640 1 -- 172.21.15.38:0/2986596186 wait complete. 2024-05-21T15:39:43.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.023+0000 7f4c84938640 1 Processor -- start 2024-05-21T15:39:43.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.024+0000 7f4c84938640 1 -- start start 2024-05-21T15:39:43.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.024+0000 7f4c84938640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c80101360 0x7f4c800709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.024+0000 7f4c84938640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c80103cc0 0x7f4c80070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.024+0000 7f4c84938640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c80106620 0x7f4c80071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.024+0000 7f4c84938640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4c80114da0 con 0x7f4c80103cc0 2024-05-21T15:39:43.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.024+0000 7f4c84938640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c80114c20 con 0x7f4c80101360 2024-05-21T15:39:43.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.024+0000 7f4c84938640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c80114f20 con 0x7f4c80106620 2024-05-21T15:39:43.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.025+0000 7f4c7ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c80106620 0x7f4c80071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.026+0000 7f4c7e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c80101360 0x7f4c800709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.026+0000 7f4c7e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c80101360 0x7f4c800709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50320/0 (socket says 172.21.15.38:50320) 2024-05-21T15:39:43.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.026+0000 7f4c7e59c640 1 -- 172.21.15.38:0/2361695429 learned_addr learned my addr 172.21.15.38:0/2361695429 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:43.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.029+0000 7f4c7e59c640 1 -- 172.21.15.38:0/2361695429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c80106620 msgr2=0x7f4c80071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.029+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/2361695429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c80106620 0x7f4c80071e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.029+0000 7f4c7e59c640 1 -- 172.21.15.38:0/2361695429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c80103cc0 msgr2=0x7f4c80070f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:39:43.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.029+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/2361695429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c80103cc0 0x7f4c80070f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.029+0000 7f4c7e59c640 1 -- 172.21.15.38:0/2361695429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c800724f0 con 0x7f4c80101360 2024-05-21T15:39:43.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.029+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/2361695429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c80101360 0x7f4c800709e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400b470 tx=0x7f4c7400b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:43.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.031+0000 7f4c6f7fe640 1 -- 172.21.15.38:0/2361695429 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74015020 con 0x7f4c80101360 2024-05-21T15:39:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.031+0000 7f4c84938640 1 -- 172.21.15.38:0/2361695429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c800727e0 con 0x7f4c80101360 2024-05-21T15:39:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.031+0000 7f4c84938640 1 -- 172.21.15.38:0/2361695429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c80072db0 con 0x7f4c80101360 2024-05-21T15:39:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.032+0000 7f4c6f7fe640 1 -- 172.21.15.38:0/2361695429 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c74011d70 con 0x7f4c80101360 2024-05-21T15:39:43.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.032+0000 7f4c6f7fe640 1 -- 172.21.15.38:0/2361695429 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400f340 con 0x7f4c80101360 2024-05-21T15:39:43.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.038+0000 7f4c6f7fe640 1 -- 172.21.15.38:0/2361695429 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4c74018020 con 0x7f4c80101360 2024-05-21T15:39:43.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.041+0000 7f4c6f7fe640 1 --2- 172.21.15.38:0/2361695429 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4c680a2d90 0x7f4c680a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.041+0000 7f4c8b7e8740 1 Processor -- start 2024-05-21T15:39:43.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.041+0000 7f4c6f7fe640 1 -- 172.21.15.38:0/2361695429 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f4c740cfb10 con 0x7f4c80101360 2024-05-21T15:39:43.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.042+0000 7f4c8b7e8740 1 -- start start 2024-05-21T15:39:43.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.043+0000 7f4c7dd9b640 1 --2- 172.21.15.38:0/2361695429 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4c680a2d90 0x7f4c680a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.044+0000 7f4c8b7e8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 0x55b6f1ae8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.044+0000 7f4c8b7e8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17204d0 0x55b6f1ae9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.044+0000 7f4c8b7e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17286b0 0x55b6f1aeff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.044+0000 7f4c8b7e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b6f1a4dc40 con 0x55b6f17286b0 2024-05-21T15:39:43.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.044+0000 7f4c8b7e8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b6f19f5580 con 0x55b6f17204d0 2024-05-21T15:39:43.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.044+0000 7f4c8b7e8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b6f1a49c00 con 0x55b6f1749390 2024-05-21T15:39:43.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.045+0000 7f4c7ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17286b0 0x55b6f1aeff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.046+0000 7f4c7ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17286b0 0x55b6f1aeff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34230/0 (socket says 172.21.15.38:34230) 2024-05-21T15:39:43.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.046+0000 7f4c7ed9d640 1 -- 172.21.15.38:0/2356449320 learned_addr learned my addr 172.21.15.38:0/2356449320 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:43.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.047+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/2356449320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 0x55b6f1ae8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.048+0000 7f4c7e59c640 1 -- 172.21.15.38:0/2356449320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17204d0 msgr2=0x55b6f1ae9080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:39:43.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.048+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/2356449320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17204d0 0x55b6f1ae9080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.048+0000 7f4c7e59c640 1 -- 172.21.15.38:0/2356449320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17286b0 msgr2=0x55b6f1aeff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:43.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.048+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/2356449320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17286b0 0x55b6f1aeff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.048+0000 7f4c7e59c640 1 -- 172.21.15.38:0/2356449320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f17613f0 con 0x55b6f1749390 2024-05-21T15:39:43.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.048+0000 7f4c7dd9b640 1 --2- 172.21.15.38:0/2361695429 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4c680a2d90 0x7f4c680a52a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009230 tx=0x7f4c70005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:43.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.049+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/2356449320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 0x55b6f1ae8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c7409c250 tx=0x7f4c7409c1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d50392a567167f54 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:43.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.055+0000 7f4c6cff9640 1 -- 172.21.15.38:0/2356449320 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c740b5070 con 0x55b6f1749390 2024-05-21T15:39:43.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.055+0000 7f4c6cff9640 1 -- 172.21.15.38:0/2356449320 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c740a4f30 con 0x55b6f1749390 2024-05-21T15:39:43.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.055+0000 7f4c6cff9640 1 -- 172.21.15.38:0/2356449320 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c740a0170 con 0x55b6f1749390 2024-05-21T15:39:43.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.056+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2356449320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 msgr2=0x55b6f1ae8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.056+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2356449320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 0x55b6f1ae8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c7409c250 tx=0x7f4c7409c1a0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.057+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2356449320 shutdown_connections 2024-05-21T15:39:43.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.057+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2356449320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17286b0 0x55b6f1aeff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.057+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2356449320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17204d0 0x55b6f1ae9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.057+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2356449320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 0x55b6f1ae8b00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.057+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2356449320 >> 172.21.15.38:0/2356449320 conn(0x55b6f173d4e0 msgr2=0x55b6f1ae4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:43.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.063+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2356449320 shutdown_connections 2024-05-21T15:39:43.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.063+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2356449320 wait complete. 2024-05-21T15:39:43.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.063+0000 7f4c8b7e8740 1 -- start start 2024-05-21T15:39:43.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.063+0000 7f4c8b7e8740 1 Processor -- start 2024-05-21T15:39:43.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.064+0000 7f4c8b7e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 0x55b6f1af2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.064+0000 7f4c8b7e8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17286b0 0x55b6f1af2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.064+0000 7f4c8b7e8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 0x55b6f1af9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.064+0000 7f4c8b7e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b6f1af0970 con 0x55b6f17204d0 2024-05-21T15:39:43.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.064+0000 7f4c8b7e8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b6f1a08c40 con 0x55b6f17286b0 2024-05-21T15:39:43.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.064+0000 7f4c8b7e8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b6f1af0af0 con 0x55b6f1749390 2024-05-21T15:39:43.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.064+0000 7f4c8b7e8740 10 client.0 _ll_register_callbacks cb 0x55b6f1760be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:43.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.064+0000 7f4c7dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17286b0 0x55b6f1af2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.065+0000 7f4c7dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17286b0 0x55b6f1af2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50326/0 (socket says 172.21.15.38:50326) 2024-05-21T15:39:43.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.065+0000 7f4c7dd9b640 1 -- 172.21.15.38:0/2498932906 learned_addr learned my addr 172.21.15.38:0/2498932906 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:43.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.065+0000 7f4c7ed9d640 1 --2- 172.21.15.38:0/2498932906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 0x55b6f1af9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.065+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/2498932906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 0x55b6f1af2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.065+0000 7f4c7dd9b640 1 -- 172.21.15.38:0/2498932906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 msgr2=0x55b6f1af9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:43.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.065+0000 7f4c7dd9b640 1 --2- 172.21.15.38:0/2498932906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 0x55b6f1af9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.065+0000 7f4c7dd9b640 1 -- 172.21.15.38:0/2498932906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 msgr2=0x55b6f1af2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:43.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.065+0000 7f4c7dd9b640 1 --2- 172.21.15.38:0/2498932906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 0x55b6f1af2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.065+0000 7f4c7dd9b640 1 -- 172.21.15.38:0/2498932906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6f182b7d0 con 0x55b6f17286b0 2024-05-21T15:39:43.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.065+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/2498932906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 0x55b6f1af2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:39:43.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.066+0000 7f4c7dd9b640 1 --2- 172.21.15.38:0/2498932906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17286b0 0x55b6f1af2f60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009900 tx=0x7f4c70031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7336822e3260a82 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:43.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.070+0000 7f4c65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:39:43.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.070+0000 7f4c65ffb640 1 -- 172.21.15.38:0/2498932906 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7004b020 con 0x55b6f17286b0 2024-05-21T15:39:43.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.070+0000 7f4c65ffb640 1 -- 172.21.15.38:0/2498932906 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f4c70031d80 con 0x55b6f17286b0 2024-05-21T15:39:43.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.070+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2498932906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c740a4f30 con 0x55b6f17286b0 2024-05-21T15:39:43.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.072+0000 7f4c65ffb640 1 -- 172.21.15.38:0/2498932906 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4c70035040 con 0x55b6f17286b0 2024-05-21T15:39:43.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.075+0000 7f4c8b7e8740 10 client.26477 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:43.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.075+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2498932906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b6f1afd220 con 0x55b6f17286b0 2024-05-21T15:39:43.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.077+0000 7f4c65ffb640 1 -- 172.21.15.38:0/2498932906 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c740a4f30 con 0x55b6f17286b0 2024-05-21T15:39:43.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.078+0000 7f4c65ffb640 1 -- 172.21.15.38:0/2498932906 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f4c700478f0 con 0x55b6f17286b0 2024-05-21T15:39:43.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.078+0000 7f4c8b7e8740 10 client.26477 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:43.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.078+0000 7f4c8b7e8740 10 client.26477 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:39:43.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.078+0000 7f4c8b7e8740 20 client.26477 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:43.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.078+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2498932906 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b6f1707f60 0x55b6f1b00930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.078+0000 7f4c8b7e8740 4 client.26477 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:43.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.078+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2498932906 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6f1a5b370 con 0x55b6f1707f60 2024-05-21T15:39:43.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.079+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/2498932906 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b6f1707f60 0x55b6f1b00930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.084+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/2498932906 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b6f1707f60 0x55b6f1b00930 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47c77ebfd587f214 server_cookie=a3267ec35c0121e8 in_seq=0 out_seq=0 2024-05-21T15:39:43.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.085+0000 7f4c65ffb640 10 client.26477 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:39:43.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.088+0000 7f4c65ffb640 1 -- 172.21.15.38:0/2498932906 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f4c740abc70 con 0x55b6f1707f60 2024-05-21T15:39:43.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.088+0000 7f4c65ffb640 10 client.26477 handle_command_reply: tid=0 2024-05-21T15:39:43.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.089+0000 7f4c8b7e8740 1 client.26477 shutdown 2024-05-21T15:39:43.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.095+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2498932906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17286b0 msgr2=0x55b6f1af2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.095+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2498932906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17286b0 0x55b6f1af2f60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009900 tx=0x7f4c70031400 comp rx=0 tx=0).stop 2024-05-21T15:39:43.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.096+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2498932906 shutdown_connections 2024-05-21T15:39:43.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.096+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2498932906 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b6f1707f60 0x55b6f1b00930 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.097+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2498932906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 0x55b6f1af9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.097+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2498932906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17286b0 0x55b6f1af2f60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.097+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2498932906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 0x55b6f1af2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.097+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2498932906 >> 172.21.15.38:0/2498932906 conn(0x55b6f173d4e0 msgr2=0x55b6f1ae3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:43.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.098+0000 7f4c65ffb640 0 client.26477 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:39:43.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.100+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2498932906 shutdown_connections 2024-05-21T15:39:43.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.102+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2498932906 wait complete. 2024-05-21T15:39:43.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.102+0000 7f4c8b7e8740 20 client.26477 trim_cache size 0 max 16384 2024-05-21T15:39:43.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.135+0000 7f4c8b7e8740 1 Processor -- start 2024-05-21T15:39:43.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.135+0000 7f4c8b7e8740 1 -- start start 2024-05-21T15:39:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.135+0000 7f4c8b7e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 0x55b6f1ae3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.135+0000 7f4c8b7e8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17286b0 0x55b6f1af9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.135+0000 7f4c8b7e8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 0x55b6f1ae4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.135+0000 7f4c8b7e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6000e460 con 0x55b6f17204d0 2024-05-21T15:39:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.135+0000 7f4c8b7e8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60010090 con 0x55b6f17286b0 2024-05-21T15:39:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.135+0000 7f4c8b7e8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6000eb30 con 0x55b6f1749390 2024-05-21T15:39:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.135+0000 7f4c7ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 0x55b6f1ae4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.135+0000 7f4c7ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 0x55b6f1ae4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:36702/0 (socket says 172.21.15.38:36702) 2024-05-21T15:39:43.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.135+0000 7f4c7ed9d640 1 -- 172.21.15.38:0/628489922 learned_addr learned my addr 172.21.15.38:0/628489922 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:43.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.136+0000 7f4c7dd9b640 1 --2- 172.21.15.38:0/628489922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17286b0 0x55b6f1af9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.137+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/628489922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 0x55b6f1ae3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.137+0000 7f4c7e59c640 1 -- 172.21.15.38:0/628489922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 msgr2=0x55b6f1ae4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:43.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.137+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/628489922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 0x55b6f1ae4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.137+0000 7f4c7e59c640 1 -- 172.21.15.38:0/628489922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17286b0 msgr2=0x55b6f1af9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:43.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.137+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/628489922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17286b0 0x55b6f1af9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.137+0000 7f4c7e59c640 1 -- 172.21.15.38:0/628489922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f1ae98f0 con 0x55b6f17204d0 2024-05-21T15:39:43.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.138+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/628489922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 0x55b6f1ae3ef0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4c7409a810 tx=0x7f4c740bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80c49eb5d63587e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:43.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.140+0000 7f4c67fff640 1 -- 172.21.15.38:0/628489922 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c740b5040 con 0x55b6f17204d0 2024-05-21T15:39:43.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.140+0000 7f4c67fff640 1 -- 172.21.15.38:0/628489922 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c740a6070 con 0x55b6f17204d0 2024-05-21T15:39:43.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.140+0000 7f4c67fff640 1 -- 172.21.15.38:0/628489922 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c740bc0b0 con 0x55b6f17204d0 2024-05-21T15:39:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.143+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/628489922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 msgr2=0x55b6f1ae3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.143+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/628489922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 0x55b6f1ae3ef0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4c7409a810 tx=0x7f4c740bc040 comp rx=0 tx=0).stop 2024-05-21T15:39:43.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.146+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/628489922 shutdown_connections 2024-05-21T15:39:43.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.146+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/628489922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f1749390 0x55b6f1ae4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.146+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/628489922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f17286b0 0x55b6f1af9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.146+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/628489922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 0x55b6f1ae3ef0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.146+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/628489922 >> 172.21.15.38:0/628489922 conn(0x55b6f173d4e0 msgr2=0x55b6f1aece70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:43.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.154+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/628489922 shutdown_connections 2024-05-21T15:39:43.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.156+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/628489922 wait complete. 2024-05-21T15:39:43.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.156+0000 7f4c8b7e8740 1 -- start start 2024-05-21T15:39:43.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.156+0000 7f4c8b7e8740 1 Processor -- start 2024-05-21T15:39:43.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.159+0000 7f4c8b7e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 0x55b6f1af6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.159+0000 7f4c8b7e8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f17286b0 0x55b6f1af7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.159+0000 7f4c8b7e8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f1749390 0x55b6f1b66cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.159+0000 7f4c8b7e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6000f900 con 0x55b6f17204d0 2024-05-21T15:39:43.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.159+0000 7f4c8b7e8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6000f200 con 0x55b6f1749390 2024-05-21T15:39:43.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.159+0000 7f4c8b7e8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6000c4f0 con 0x55b6f17286b0 2024-05-21T15:39:43.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.160+0000 7f4c8b7e8740 10 client.0 _ll_register_callbacks cb 0x7f4c780093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:43.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.162+0000 7f4c7ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f1749390 0x55b6f1b66cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.162+0000 7f4c7ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f1749390 0x55b6f1b66cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50354/0 (socket says 172.21.15.38:50354) 2024-05-21T15:39:43.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.162+0000 7f4c7ed9d640 1 -- 172.21.15.38:0/2691948027 learned_addr learned my addr 172.21.15.38:0/2691948027 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:43.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.162+0000 7f4c7dd9b640 1 --2- 172.21.15.38:0/2691948027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f17286b0 0x55b6f1af7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.164+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/2691948027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 0x55b6f1af6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.165+0000 7f4c7dd9b640 1 -- 172.21.15.38:0/2691948027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f1749390 msgr2=0x55b6f1b66cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:43.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.165+0000 7f4c7dd9b640 1 --2- 172.21.15.38:0/2691948027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f1749390 0x55b6f1b66cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.165+0000 7f4c7dd9b640 1 -- 172.21.15.38:0/2691948027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 msgr2=0x55b6f1af6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:43.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.165+0000 7f4c7dd9b640 1 --2- 172.21.15.38:0/2691948027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 0x55b6f1af6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.166+0000 7f4c7dd9b640 1 -- 172.21.15.38:0/2691948027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6f1afa010 con 0x55b6f17286b0 2024-05-21T15:39:43.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.166+0000 7f4c7dd9b640 1 --2- 172.21.15.38:0/2691948027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f17286b0 0x55b6f1af7050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c70047f00 tx=0x7f4c70047670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad3ccad22f5ed597 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:43.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.169+0000 7f4c65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:39:43.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.169+0000 7f4c65ffb640 1 -- 172.21.15.38:0/2691948027 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7005d070 con 0x55b6f17286b0 2024-05-21T15:39:43.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.169+0000 7f4c65ffb640 1 -- 172.21.15.38:0/2691948027 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f4c700035f0 con 0x55b6f17286b0 2024-05-21T15:39:43.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.169+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2691948027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c740a6070 con 0x55b6f17286b0 2024-05-21T15:39:43.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.171+0000 7f4c65ffb640 1 -- 172.21.15.38:0/2691948027 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4c7004b070 con 0x55b6f17286b0 2024-05-21T15:39:43.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.171+0000 7f4c8b7e8740 10 client.26554 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:43.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.171+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2691948027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b6f1b681f0 con 0x55b6f17286b0 2024-05-21T15:39:43.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.173+0000 7f4c65ffb640 1 -- 172.21.15.38:0/2691948027 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c740a6070 con 0x55b6f17286b0 2024-05-21T15:39:43.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.175+0000 7f4c65ffb640 1 -- 172.21.15.38:0/2691948027 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f4c70059820 con 0x55b6f17286b0 2024-05-21T15:39:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.179+0000 7f4c8b7e8740 10 client.26554 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.179+0000 7f4c8b7e8740 10 client.26554 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:39:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.179+0000 7f4c8b7e8740 20 client.26554 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.179+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2691948027 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b6f16dd3a0 0x55b6f1b6aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.179+0000 7f4c8b7e8740 4 client.26554 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:39:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.179+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2691948027 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6f1a556c0 con 0x55b6f16dd3a0 2024-05-21T15:39:43.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.180+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/2691948027 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b6f16dd3a0 0x55b6f1b6aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.188+0000 7f4c7e59c640 1 --2- 172.21.15.38:0/2691948027 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b6f16dd3a0 0x55b6f1b6aa30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6cea9b870968c1fd server_cookie=6457271b9a360f9e in_seq=0 out_seq=0 2024-05-21T15:39:43.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.189+0000 7f4c65ffb640 10 client.26554 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:39:43.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.192+0000 7f4c65ffb640 1 -- 172.21.15.38:0/2691948027 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6f1a556c0 con 0x55b6f16dd3a0 2024-05-21T15:39:43.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.192+0000 7f4c65ffb640 10 client.26554 handle_command_reply: tid=0 2024-05-21T15:39:43.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.194+0000 7f4c8b7e8740 1 client.26554 shutdown 2024-05-21T15:39:43.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.197+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2691948027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f17286b0 msgr2=0x55b6f1af7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.197+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2691948027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f17286b0 0x55b6f1af7050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c70047f00 tx=0x7f4c70047670 comp rx=0 tx=0).stop 2024-05-21T15:39:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.198+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2691948027 shutdown_connections 2024-05-21T15:39:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.198+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2691948027 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55b6f16dd3a0 0x55b6f1b6aa30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.198+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2691948027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6f1749390 0x55b6f1b66cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.198+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2691948027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b6f17286b0 0x55b6f1af7050 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c70047f00 tx=0x7f4c70047670 comp rx=0 tx=0).stop 2024-05-21T15:39:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.198+0000 7f4c8b7e8740 1 --2- 172.21.15.38:0/2691948027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b6f17204d0 0x55b6f1af6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.198+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2691948027 >> 172.21.15.38:0/2691948027 conn(0x55b6f173d4e0 msgr2=0x55b6f1ae3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:43.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.198+0000 7f4c65ffb640 0 client.26554 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:39:43.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.201+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2691948027 shutdown_connections 2024-05-21T15:39:43.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.204+0000 7f4c8b7e8740 1 -- 172.21.15.38:0/2691948027 wait complete. 2024-05-21T15:39:43.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.207+0000 7f4c8b7e8740 20 client.26554 trim_cache size 0 max 16384 2024-05-21T15:39:43.207 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:39:43.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.210+0000 7f4c84938640 1 -- 172.21.15.38:0/2361695429 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4c680a2d90 msgr2=0x7f4c680a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.210+0000 7f4c84938640 1 --2- 172.21.15.38:0/2361695429 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4c680a2d90 0x7f4c680a52a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009230 tx=0x7f4c70005fd0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.210+0000 7f4c84938640 1 -- 172.21.15.38:0/2361695429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c80101360 msgr2=0x7f4c800709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.210+0000 7f4c84938640 1 --2- 172.21.15.38:0/2361695429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c80101360 0x7f4c800709e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400b470 tx=0x7f4c7400b930 comp rx=0 tx=0).stop 2024-05-21T15:39:43.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.210+0000 7f4c84938640 1 -- 172.21.15.38:0/2361695429 shutdown_connections 2024-05-21T15:39:43.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.210+0000 7f4c84938640 1 --2- 172.21.15.38:0/2361695429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c80106620 0x7f4c80071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.210+0000 7f4c84938640 1 --2- 172.21.15.38:0/2361695429 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f4c680a2d90 0x7f4c680a52a0 secure :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009230 tx=0x7f4c70005fd0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.210+0000 7f4c84938640 1 --2- 172.21.15.38:0/2361695429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c80103cc0 0x7f4c80070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.210+0000 7f4c84938640 1 --2- 172.21.15.38:0/2361695429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c80101360 0x7f4c800709e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.210+0000 7f4c84938640 1 -- 172.21.15.38:0/2361695429 >> 172.21.15.38:0/2361695429 conn(0x7f4c800fcb70 msgr2=0x7f4c80102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:43.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.211+0000 7f4c84938640 1 -- 172.21.15.38:0/2361695429 shutdown_connections 2024-05-21T15:39:43.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.211+0000 7f4c84938640 1 -- 172.21.15.38:0/2361695429 wait complete. 2024-05-21T15:39:43.229 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:39:43.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.449+0000 7fe9472c3640 1 -- 172.21.15.38:0/1028419470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9401093d0 msgr2=0x7fe940109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.449+0000 7fe9472c3640 1 --2- 172.21.15.38:0/1028419470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9401093d0 0x7fe940109d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe930009900 tx=0x7fe93002a420 comp rx=0 tx=0).stop 2024-05-21T15:39:43.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.452+0000 7fe9472c3640 1 -- 172.21.15.38:0/1028419470 shutdown_connections 2024-05-21T15:39:43.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.452+0000 7fe9472c3640 1 --2- 172.21.15.38:0/1028419470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94010efe0 0x7fe94010f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.452+0000 7fe9472c3640 1 --2- 172.21.15.38:0/1028419470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe94010a2c0 0x7fe94010a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.452+0000 7fe9472c3640 1 --2- 172.21.15.38:0/1028419470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9401093d0 0x7fe940109d80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.452+0000 7fe9472c3640 1 -- 172.21.15.38:0/1028419470 >> 172.21.15.38:0/1028419470 conn(0x7fe94006f4a0 msgr2=0x7fe940071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:43.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.459+0000 7fe9472c3640 1 -- 172.21.15.38:0/1028419470 shutdown_connections 2024-05-21T15:39:43.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.460+0000 7fe9472c3640 1 -- 172.21.15.38:0/1028419470 wait complete. 2024-05-21T15:39:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.462+0000 7fe9472c3640 1 Processor -- start 2024-05-21T15:39:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.462+0000 7fe9472c3640 1 -- start start 2024-05-21T15:39:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.462+0000 7fe9472c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9401093d0 0x7fe9401a8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.462+0000 7fe9472c3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94010a2c0 0x7fe9401a8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.462+0000 7fe9472c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94010efe0 0x7fe9401af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.462+0000 7fe9472c3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe94011d5e0 con 0x7fe9401093d0 2024-05-21T15:39:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.462+0000 7fe9472c3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe94011d460 con 0x7fe94010efe0 2024-05-21T15:39:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.462+0000 7fe9472c3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe94011d760 con 0x7fe94010a2c0 2024-05-21T15:39:43.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.465+0000 7fe945860640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94010efe0 0x7fe9401af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.465+0000 7fe94505f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9401093d0 0x7fe9401a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.465+0000 7fe94505f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9401093d0 0x7fe9401a8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34248/0 (socket says 172.21.15.38:34248) 2024-05-21T15:39:43.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.465+0000 7fe94505f640 1 -- 172.21.15.38:0/864406845 learned_addr learned my addr 172.21.15.38:0/864406845 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:43.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.467+0000 7fe94485e640 1 --2- 172.21.15.38:0/864406845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94010a2c0 0x7fe9401a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.468+0000 7fe945860640 1 -- 172.21.15.38:0/864406845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94010a2c0 msgr2=0x7fe9401a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.468+0000 7fe945860640 1 --2- 172.21.15.38:0/864406845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94010a2c0 0x7fe9401a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.468+0000 7fe945860640 1 -- 172.21.15.38:0/864406845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9401093d0 msgr2=0x7fe9401a8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.468+0000 7fe945860640 1 --2- 172.21.15.38:0/864406845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9401093d0 0x7fe9401a8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.468+0000 7fe945860640 1 -- 172.21.15.38:0/864406845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9401afc60 con 0x7fe94010efe0 2024-05-21T15:39:43.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.468+0000 7fe945860640 1 --2- 172.21.15.38:0/864406845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94010efe0 0x7fe9401af5f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe938009fd0 tx=0x7fe93800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:43.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.469+0000 7fe9367fc640 1 -- 172.21.15.38:0/864406845 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93800ce20 con 0x7fe94010efe0 2024-05-21T15:39:43.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.470+0000 7fe9367fc640 1 -- 172.21.15.38:0/864406845 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe938005020 con 0x7fe94010efe0 2024-05-21T15:39:43.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.470+0000 7fe9472c3640 1 -- 172.21.15.38:0/864406845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9401aff50 con 0x7fe94010efe0 2024-05-21T15:39:43.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.471+0000 7fe9367fc640 1 -- 172.21.15.38:0/864406845 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938010720 con 0x7fe94010efe0 2024-05-21T15:39:43.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.474+0000 7fe9472c3640 1 -- 172.21.15.38:0/864406845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9401b03b0 con 0x7fe94010efe0 2024-05-21T15:39:43.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.477+0000 7fe94e165740 1 Processor -- start 2024-05-21T15:39:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.479+0000 7fe94e165740 1 -- start start 2024-05-21T15:39:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.479+0000 7fe9367fc640 1 -- 172.21.15.38:0/864406845 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe938004a00 con 0x7fe94010efe0 2024-05-21T15:39:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.479+0000 7fe94e165740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cf7390 0x560bb5096b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.479+0000 7fe94e165740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb5097080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.479+0000 7fe94e165740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4cd66b0 0x560bb509df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.479+0000 7fe94e165740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560bb4ffbc40 con 0x560bb4cce4d0 2024-05-21T15:39:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.482+0000 7fe94e165740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560bb4fa3580 con 0x560bb4cf7390 2024-05-21T15:39:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.481+0000 7fe94505f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb5097080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.482+0000 7fe94505f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb5097080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34254/0 (socket says 172.21.15.38:34254) 2024-05-21T15:39:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.482+0000 7fe94505f640 1 -- 172.21.15.38:0/2291741952 learned_addr learned my addr 172.21.15.38:0/2291741952 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.482+0000 7fe94e165740 1 -- 172.21.15.38:0/2291741952 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560bb4ff7c00 con 0x560bb4cd66b0 2024-05-21T15:39:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.481+0000 7fe94485e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cf7390 0x560bb5096b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.482+0000 7fe94485e640 1 --2- 172.21.15.38:0/2291741952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4cd66b0 0x560bb509df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.483+0000 7fe94485e640 1 -- 172.21.15.38:0/2291741952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4cd66b0 msgr2=0x560bb509df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.483+0000 7fe94485e640 1 --2- 172.21.15.38:0/2291741952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4cd66b0 0x560bb509df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.483+0000 7fe94485e640 1 -- 172.21.15.38:0/2291741952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 msgr2=0x560bb5097080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.483+0000 7fe94485e640 1 --2- 172.21.15.38:0/2291741952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb5097080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.483+0000 7fe94485e640 1 -- 172.21.15.38:0/2291741952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bb4d0f3f0 con 0x560bb4cf7390 2024-05-21T15:39:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.484+0000 7fe94485e640 1 --2- 172.21.15.38:0/2291741952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cf7390 0x560bb5096b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe93c005fd0 tx=0x7fe93c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=433df34023635c12 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.484+0000 7fe9367fc640 1 --2- 172.21.15.38:0/864406845 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe92c0a2b30 0x7fe92c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.484+0000 7fe9367fc640 1 -- 172.21.15.38:0/864406845 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe9380d3060 con 0x7fe94010efe0 2024-05-21T15:39:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.485+0000 7fe94505f640 1 --2- 172.21.15.38:0/864406845 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe92c0a2b30 0x7fe92c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.486+0000 7fe92b7fe640 1 -- 172.21.15.38:0/2291741952 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c01b070 con 0x560bb4cf7390 2024-05-21T15:39:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.486+0000 7fe92b7fe640 1 -- 172.21.15.38:0/2291741952 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe93c010b60 con 0x560bb4cf7390 2024-05-21T15:39:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.488+0000 7fe94e165740 1 -- 172.21.15.38:0/2291741952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cf7390 msgr2=0x560bb5096b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.488+0000 7fe94e165740 1 --2- 172.21.15.38:0/2291741952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cf7390 0x560bb5096b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe93c005fd0 tx=0x7fe93c009310 comp rx=0 tx=0).stop 2024-05-21T15:39:43.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.490+0000 7fe94505f640 1 --2- 172.21.15.38:0/864406845 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe92c0a2b30 0x7fe92c0a5040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x560bb5097ee0 tx=0x7fe930005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:43.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.494+0000 7fe94e165740 1 -- 172.21.15.38:0/2291741952 shutdown_connections 2024-05-21T15:39:43.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.494+0000 7fe94e165740 1 --2- 172.21.15.38:0/2291741952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4cd66b0 0x560bb509df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.494+0000 7fe94e165740 1 --2- 172.21.15.38:0/2291741952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb5097080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.494+0000 7fe94e165740 1 --2- 172.21.15.38:0/2291741952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cf7390 0x560bb5096b00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.495+0000 7fe94e165740 1 -- 172.21.15.38:0/2291741952 >> 172.21.15.38:0/2291741952 conn(0x560bb4ceb4e0 msgr2=0x560bb5092490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.498+0000 7fe94e165740 1 -- 172.21.15.38:0/2291741952 shutdown_connections 2024-05-21T15:39:43.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.504+0000 7fe94e165740 1 -- 172.21.15.38:0/2291741952 wait complete. 2024-05-21T15:39:43.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.504+0000 7fe94e165740 1 -- start start 2024-05-21T15:39:43.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.505+0000 7fe94e165740 1 Processor -- start 2024-05-21T15:39:43.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.506+0000 7fe94e165740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb50b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.506+0000 7fe94e165740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4cd66b0 0x560bb50b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.506+0000 7fe94e165740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cf7390 0x560bb50b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.506+0000 7fe94e165740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560bb50afa30 con 0x560bb4cce4d0 2024-05-21T15:39:43.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.506+0000 7fe94e165740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560bb4fb6c40 con 0x560bb4cf7390 2024-05-21T15:39:43.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.506+0000 7fe94e165740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560bb50afbb0 con 0x560bb4cd66b0 2024-05-21T15:39:43.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.506+0000 7fe94e165740 10 client.0 _ll_register_callbacks cb 0x560bb4d0ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:43.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.507+0000 7fe945860640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cf7390 0x560bb50b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.507+0000 7fe94505f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb50b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.507+0000 7fe94505f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb50b1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34270/0 (socket says 172.21.15.38:34270) 2024-05-21T15:39:43.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.507+0000 7fe94505f640 1 -- 172.21.15.38:0/1063439146 learned_addr learned my addr 172.21.15.38:0/1063439146 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:43.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.507+0000 7fe94505f640 1 -- 172.21.15.38:0/1063439146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4cd66b0 msgr2=0x560bb50b1fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:39:43.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.507+0000 7fe94505f640 1 --2- 172.21.15.38:0/1063439146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4cd66b0 0x560bb50b1fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.507+0000 7fe94505f640 1 -- 172.21.15.38:0/1063439146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cf7390 msgr2=0x560bb50b8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:43.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.507+0000 7fe94505f640 1 --2- 172.21.15.38:0/1063439146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cf7390 0x560bb50b8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.507+0000 7fe94505f640 1 -- 172.21.15.38:0/1063439146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bb4dd97d0 con 0x560bb4cce4d0 2024-05-21T15:39:43.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.508+0000 7fe94505f640 1 --2- 172.21.15.38:0/1063439146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb50b1a90 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe9300308c0 tx=0x7fe93002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b241b4b5269f101 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:43.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.508+0000 7fe928ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:43.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.508+0000 7fe928ff9640 1 -- 172.21.15.38:0/1063439146 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930004100 con 0x560bb4cce4d0 2024-05-21T15:39:43.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.509+0000 7fe94e165740 1 -- 172.21.15.38:0/1063439146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe93c010b60 con 0x560bb4cce4d0 2024-05-21T15:39:43.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.509+0000 7fe928ff9640 1 -- 172.21.15.38:0/1063439146 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe9300042a0 con 0x560bb4cce4d0 2024-05-21T15:39:43.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.511+0000 7fe928ff9640 1 -- 172.21.15.38:0/1063439146 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe9300044a0 con 0x560bb4cce4d0 2024-05-21T15:39:43.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.513+0000 7fe94e165740 10 client.16860 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:43.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.513+0000 7fe94e165740 1 -- 172.21.15.38:0/1063439146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560bb50bc2e0 con 0x560bb4cce4d0 2024-05-21T15:39:43.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.515+0000 7fe928ff9640 1 -- 172.21.15.38:0/1063439146 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe93c010b60 con 0x560bb4cce4d0 2024-05-21T15:39:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.515+0000 7fe928ff9640 1 -- 172.21.15.38:0/1063439146 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fe93005a7a0 con 0x560bb4cce4d0 2024-05-21T15:39:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.515+0000 7fe94e165740 10 client.16860 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.515+0000 7fe94e165740 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.515+0000 7fe94e165740 20 client.16860 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.515+0000 7fe94e165740 1 --2- 172.21.15.38:0/1063439146 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560bb4cb5f60 0x560bb50bff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.515+0000 7fe94e165740 4 client.16860 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.515+0000 7fe94e165740 1 -- 172.21.15.38:0/1063439146 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bb5009370 con 0x560bb4cb5f60 2024-05-21T15:39:43.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.517+0000 7fe94485e640 1 --2- 172.21.15.38:0/1063439146 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560bb4cb5f60 0x560bb50bff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.522+0000 7fe94485e640 1 --2- 172.21.15.38:0/1063439146 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560bb4cb5f60 0x560bb50bff50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75f07ebf28626314 server_cookie=2b94857319391c3e in_seq=0 out_seq=0 2024-05-21T15:39:43.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.522+0000 7fe928ff9640 10 client.16860 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:43.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.524+0000 7fe928ff9640 1 -- 172.21.15.38:0/1063439146 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560bb5009370 con 0x560bb4cb5f60 2024-05-21T15:39:43.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.525+0000 7fe928ff9640 10 client.16860 handle_command_reply: tid=0 2024-05-21T15:39:43.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.528+0000 7fe94e165740 1 client.16860 shutdown 2024-05-21T15:39:43.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.535+0000 7fe94e165740 1 -- 172.21.15.38:0/1063439146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 msgr2=0x560bb50b1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.535+0000 7fe94e165740 1 --2- 172.21.15.38:0/1063439146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb50b1a90 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe9300308c0 tx=0x7fe93002a420 comp rx=0 tx=0).stop 2024-05-21T15:39:43.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.536+0000 7fe94e165740 1 -- 172.21.15.38:0/1063439146 shutdown_connections 2024-05-21T15:39:43.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.536+0000 7fe94e165740 1 --2- 172.21.15.38:0/1063439146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cf7390 0x560bb50b8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.536+0000 7fe94e165740 1 --2- 172.21.15.38:0/1063439146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4cd66b0 0x560bb50b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.536+0000 7fe94e165740 1 --2- 172.21.15.38:0/1063439146 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560bb4cb5f60 0x560bb50bff50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.536+0000 7fe94e165740 1 --2- 172.21.15.38:0/1063439146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb50b1a90 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.536+0000 7fe94e165740 1 -- 172.21.15.38:0/1063439146 >> 172.21.15.38:0/1063439146 conn(0x560bb4ceb4e0 msgr2=0x560bb5091570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:43.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.537+0000 7fe928ff9640 0 client.16860 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:43.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.540+0000 7fe94e165740 1 -- 172.21.15.38:0/1063439146 shutdown_connections 2024-05-21T15:39:43.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.542+0000 7fe94e165740 1 -- 172.21.15.38:0/1063439146 wait complete. 2024-05-21T15:39:43.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.542+0000 7fe94e165740 20 client.16860 trim_cache size 0 max 16384 2024-05-21T15:39:43.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.574+0000 7fe94e165740 1 Processor -- start 2024-05-21T15:39:43.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.574+0000 7fe94e165740 1 -- start start 2024-05-21T15:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.574+0000 7fe94e165740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4cce4d0 0x560bb50b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.575+0000 7fe94e165740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cd66b0 0x560bb50b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.575+0000 7fe94e165740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cf7390 0x560bb50b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.575+0000 7fe94e165740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe9400740f0 con 0x560bb4cf7390 2024-05-21T15:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.575+0000 7fe94e165740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe940075d20 con 0x560bb4cd66b0 2024-05-21T15:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.575+0000 7fe94e165740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe9400747c0 con 0x560bb4cce4d0 2024-05-21T15:39:43.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.575+0000 7fe945860640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cf7390 0x560bb50b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.575+0000 7fe945860640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cf7390 0x560bb50b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34282/0 (socket says 172.21.15.38:34282) 2024-05-21T15:39:43.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.575+0000 7fe945860640 1 -- 172.21.15.38:0/658290288 learned_addr learned my addr 172.21.15.38:0/658290288 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:43.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.576+0000 7fe945860640 1 -- 172.21.15.38:0/658290288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4cce4d0 msgr2=0x560bb50b0740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:39:43.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.576+0000 7fe945860640 1 --2- 172.21.15.38:0/658290288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4cce4d0 0x560bb50b0740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.576+0000 7fe945860640 1 -- 172.21.15.38:0/658290288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cd66b0 msgr2=0x560bb50b8ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:39:43.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.576+0000 7fe945860640 1 --2- 172.21.15.38:0/658290288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cd66b0 0x560bb50b8ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.576+0000 7fe945860640 1 -- 172.21.15.38:0/658290288 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bb509b7e0 con 0x560bb4cf7390 2024-05-21T15:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.577+0000 7fe945860640 1 --2- 172.21.15.38:0/658290288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cf7390 0x560bb50b1660 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe938010910 tx=0x7fe93809b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490831c6cdd84863 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.577+0000 7fe92affd640 1 -- 172.21.15.38:0/658290288 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93809fa00 con 0x560bb4cf7390 2024-05-21T15:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.578+0000 7fe92affd640 1 -- 172.21.15.38:0/658290288 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9380d2020 con 0x560bb4cf7390 2024-05-21T15:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.579+0000 7fe94e165740 1 -- 172.21.15.38:0/658290288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cf7390 msgr2=0x560bb50b1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.579+0000 7fe94e165740 1 --2- 172.21.15.38:0/658290288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cf7390 0x560bb50b1660 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe938010910 tx=0x7fe93809b280 comp rx=0 tx=0).stop 2024-05-21T15:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.579+0000 7fe94e165740 1 -- 172.21.15.38:0/658290288 shutdown_connections 2024-05-21T15:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.579+0000 7fe94e165740 1 --2- 172.21.15.38:0/658290288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cf7390 0x560bb50b1660 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.579+0000 7fe94e165740 1 --2- 172.21.15.38:0/658290288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cd66b0 0x560bb50b8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.579+0000 7fe94e165740 1 --2- 172.21.15.38:0/658290288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4cce4d0 0x560bb50b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.579+0000 7fe94e165740 1 -- 172.21.15.38:0/658290288 >> 172.21.15.38:0/658290288 conn(0x560bb4ceb4e0 msgr2=0x560bb50b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:43.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.585+0000 7fe94e165740 1 -- 172.21.15.38:0/658290288 shutdown_connections 2024-05-21T15:39:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.586+0000 7fe94e165740 1 -- 172.21.15.38:0/658290288 wait complete. 2024-05-21T15:39:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.587+0000 7fe94e165740 1 -- start start 2024-05-21T15:39:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.587+0000 7fe94e165740 1 Processor -- start 2024-05-21T15:39:43.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.592+0000 7fe94e165740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb50b6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.592+0000 7fe94e165740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cd66b0 0x560bb50b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.592+0000 7fe94e165740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4c8b3a0 0x560bb50b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.592+0000 7fe94e165740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe940075590 con 0x560bb4cce4d0 2024-05-21T15:39:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.592+0000 7fe94e165740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe940074e90 con 0x560bb4cd66b0 2024-05-21T15:39:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.592+0000 7fe94e165740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe940117630 con 0x560bb4c8b3a0 2024-05-21T15:39:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.592+0000 7fe94e165740 10 client.0 _ll_register_callbacks cb 0x7fe930009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.593+0000 7fe94505f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb50b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.593+0000 7fe94505f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb50b6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34294/0 (socket says 172.21.15.38:34294) 2024-05-21T15:39:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.593+0000 7fe94505f640 1 -- 172.21.15.38:0/2924128243 learned_addr learned my addr 172.21.15.38:0/2924128243 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.594+0000 7fe94485e640 1 --2- 172.21.15.38:0/2924128243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cd66b0 0x560bb50b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.594+0000 7fe94505f640 1 -- 172.21.15.38:0/2924128243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4c8b3a0 msgr2=0x560bb50b7960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:39:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.594+0000 7fe94505f640 1 --2- 172.21.15.38:0/2924128243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4c8b3a0 0x560bb50b7960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.594+0000 7fe94505f640 1 -- 172.21.15.38:0/2924128243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cd66b0 msgr2=0x560bb50b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.594+0000 7fe94505f640 1 --2- 172.21.15.38:0/2924128243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cd66b0 0x560bb50b6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.594+0000 7fe94505f640 1 -- 172.21.15.38:0/2924128243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bb50b7f20 con 0x560bb4cce4d0 2024-05-21T15:39:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.594+0000 7fe94505f640 1 --2- 172.21.15.38:0/2924128243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb50b6540 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe930009850 tx=0x7fe9300307e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d48d089b9c552c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.594+0000 7fe928ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.596+0000 7fe928ff9640 1 -- 172.21.15.38:0/2924128243 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93005a650 con 0x560bb4cce4d0 2024-05-21T15:39:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.596+0000 7fe928ff9640 1 -- 172.21.15.38:0/2924128243 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe93005a7f0 con 0x560bb4cce4d0 2024-05-21T15:39:43.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.598+0000 7fe94e165740 1 -- 172.21.15.38:0/2924128243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560bb50036c0 con 0x560bb4cce4d0 2024-05-21T15:39:43.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.600+0000 7fe928ff9640 1 -- 172.21.15.38:0/2924128243 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe9300035f0 con 0x560bb4cce4d0 2024-05-21T15:39:43.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.601+0000 7fe94e165740 10 client.16866 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:43.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.601+0000 7fe94e165740 1 -- 172.21.15.38:0/2924128243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560bb50b92a0 con 0x560bb4cce4d0 2024-05-21T15:39:43.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.602+0000 7fe928ff9640 1 -- 172.21.15.38:0/2924128243 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560bb50036c0 con 0x560bb4cce4d0 2024-05-21T15:39:43.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.603+0000 7fe928ff9640 1 -- 172.21.15.38:0/2924128243 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fe930046a20 con 0x560bb4cce4d0 2024-05-21T15:39:43.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.604+0000 7fe94e165740 10 client.16866 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:43.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.604+0000 7fe94e165740 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:43.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.604+0000 7fe94e165740 20 client.16866 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:43.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.604+0000 7fe94e165740 1 --2- 172.21.15.38:0/2924128243 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560bb5128b20 0x560bb512af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:43.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.604+0000 7fe94e165740 4 client.16866 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:39:43.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.604+0000 7fe94e165740 1 -- 172.21.15.38:0/2924128243 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560bb4f80820 con 0x560bb5128b20 2024-05-21T15:39:43.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.611+0000 7fe94485e640 1 --2- 172.21.15.38:0/2924128243 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560bb5128b20 0x560bb512af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:43.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.623+0000 7fe94485e640 1 --2- 172.21.15.38:0/2924128243 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560bb5128b20 0x560bb512af40 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3d166964209c2b5 server_cookie=168fc9b219c98e7 in_seq=0 out_seq=0 2024-05-21T15:39:43.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.623+0000 7fe928ff9640 10 client.16866 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:43.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.623+0000 7fe928ff9640 1 -- 172.21.15.38:0/2924128243 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560bb4f80820 con 0x560bb5128b20 2024-05-21T15:39:43.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.623+0000 7fe928ff9640 10 client.16866 handle_command_reply: tid=0 2024-05-21T15:39:43.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.624+0000 7fe94e165740 1 client.16866 shutdown 2024-05-21T15:39:43.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.625+0000 7fe94e165740 1 -- 172.21.15.38:0/2924128243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 msgr2=0x560bb50b6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.625+0000 7fe94e165740 1 --2- 172.21.15.38:0/2924128243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb50b6540 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe930009850 tx=0x7fe9300307e0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.628+0000 7fe94e165740 1 -- 172.21.15.38:0/2924128243 shutdown_connections 2024-05-21T15:39:43.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.628+0000 7fe94e165740 1 --2- 172.21.15.38:0/2924128243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560bb4c8b3a0 0x560bb50b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.628+0000 7fe94e165740 1 --2- 172.21.15.38:0/2924128243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560bb4cd66b0 0x560bb50b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.628+0000 7fe94e165740 1 --2- 172.21.15.38:0/2924128243 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560bb5128b20 0x560bb512af40 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.628+0000 7fe94e165740 1 --2- 172.21.15.38:0/2924128243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560bb4cce4d0 0x560bb50b6540 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.628+0000 7fe928ff9640 0 client.16866 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:43.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.628+0000 7fe94e165740 1 -- 172.21.15.38:0/2924128243 >> 172.21.15.38:0/2924128243 conn(0x560bb4ceb4e0 msgr2=0x560bb50b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:43.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.629+0000 7fe94e165740 1 -- 172.21.15.38:0/2924128243 shutdown_connections 2024-05-21T15:39:43.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.629+0000 7fe94e165740 1 -- 172.21.15.38:0/2924128243 wait complete. 2024-05-21T15:39:43.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.629+0000 7fe94e165740 20 client.16866 trim_cache size 0 max 16384 2024-05-21T15:39:43.630 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:39:43.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.632+0000 7fe9472c3640 1 -- 172.21.15.38:0/864406845 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe92c0a2b30 msgr2=0x7fe92c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.632+0000 7fe9472c3640 1 --2- 172.21.15.38:0/864406845 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe92c0a2b30 0x7fe92c0a5040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x560bb5097ee0 tx=0x7fe930005ef0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.632+0000 7fe9472c3640 1 -- 172.21.15.38:0/864406845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94010efe0 msgr2=0x7fe9401af5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:43.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.632+0000 7fe9472c3640 1 --2- 172.21.15.38:0/864406845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94010efe0 0x7fe9401af5f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe938009fd0 tx=0x7fe93800ef10 comp rx=0 tx=0).stop 2024-05-21T15:39:43.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.632+0000 7fe9472c3640 1 -- 172.21.15.38:0/864406845 shutdown_connections 2024-05-21T15:39:43.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.632+0000 7fe9472c3640 1 --2- 172.21.15.38:0/864406845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94010efe0 0x7fe9401af5f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.632+0000 7fe9472c3640 1 --2- 172.21.15.38:0/864406845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe94010a2c0 0x7fe9401a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.632+0000 7fe9472c3640 1 --2- 172.21.15.38:0/864406845 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe92c0a2b30 0x7fe92c0a5040 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.632+0000 7fe9472c3640 1 --2- 172.21.15.38:0/864406845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9401093d0 0x7fe9401a8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:43.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.632+0000 7fe9472c3640 1 -- 172.21.15.38:0/864406845 >> 172.21.15.38:0/864406845 conn(0x7fe94006f4a0 msgr2=0x7fe94010afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:43.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.634+0000 7fe9472c3640 1 -- 172.21.15.38:0/864406845 shutdown_connections 2024-05-21T15:39:43.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:43.634+0000 7fe9472c3640 1 -- 172.21.15.38:0/864406845 wait complete. 2024-05-21T15:39:44.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:43 smithi038 ceph-mon[31995]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 MiB/s wr, 11.86k op/s 2024-05-21T15:39:44.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:43 smithi053 ceph-mon[36496]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 MiB/s wr, 11.86k op/s 2024-05-21T15:39:44.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:43 smithi045 ceph-mon[36409]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 MiB/s wr, 11.86k op/s 2024-05-21T15:39:44.655 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:39:44.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:39:44.655 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:39:44.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.779+0000 7f7e6ce1b640 1 -- 172.21.15.38:0/2205992360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e68102eb0 msgr2=0x7f7e68101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.779+0000 7f7e6ce1b640 1 --2- 172.21.15.38:0/2205992360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e68102eb0 0x7f7e68101790 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009900 tx=0x7f7e5802a410 comp rx=0 tx=0).stop 2024-05-21T15:39:44.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.779+0000 7f7e6ce1b640 1 -- 172.21.15.38:0/2205992360 shutdown_connections 2024-05-21T15:39:44.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.779+0000 7f7e6ce1b640 1 --2- 172.21.15.38:0/2205992360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e681079e0 0x7f7e68109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.779+0000 7f7e6ce1b640 1 --2- 172.21.15.38:0/2205992360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e68103390 0x7f7e68101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.779+0000 7f7e6ce1b640 1 --2- 172.21.15.38:0/2205992360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e68102eb0 0x7f7e68101790 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.779+0000 7f7e6ce1b640 1 -- 172.21.15.38:0/2205992360 >> 172.21.15.38:0/2205992360 conn(0x7f7e680fc330 msgr2=0x7f7e680fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:44.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.779+0000 7f7e6ce1b640 1 -- 172.21.15.38:0/2205992360 shutdown_connections 2024-05-21T15:39:44.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.779+0000 7f7e6ce1b640 1 -- 172.21.15.38:0/2205992360 wait complete. 2024-05-21T15:39:44.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.780+0000 7f7e6ce1b640 1 Processor -- start 2024-05-21T15:39:44.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.780+0000 7f7e6ce1b640 1 -- start start 2024-05-21T15:39:44.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.780+0000 7f7e6ce1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e68102eb0 0x7f7e68198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.780+0000 7f7e6ce1b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e681079e0 0x7f7e681993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.780+0000 7f7e6ce1b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e6819e320 0x7f7e681a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.780+0000 7f7e6ce1b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6810e740 con 0x7f7e6819e320 2024-05-21T15:39:44.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.780+0000 7f7e6ce1b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6810e5c0 con 0x7f7e68102eb0 2024-05-21T15:39:44.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.780+0000 7f7e6ce1b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6810e8c0 con 0x7f7e681079e0 2024-05-21T15:39:44.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.780+0000 7f7e66d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e6819e320 0x7f7e681a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.780+0000 7f7e66d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e6819e320 0x7f7e681a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56376/0 (socket says 172.21.15.38:56376) 2024-05-21T15:39:44.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.780+0000 7f7e66d9d640 1 -- 172.21.15.38:0/1430268826 learned_addr learned my addr 172.21.15.38:0/1430268826 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:44.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.781+0000 7f7e6659c640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e68102eb0 0x7f7e68198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.781+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e681079e0 0x7f7e681993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.781+0000 7f7e66d9d640 1 -- 172.21.15.38:0/1430268826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e681079e0 msgr2=0x7f7e681993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.781+0000 7f7e66d9d640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e681079e0 0x7f7e681993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.781+0000 7f7e66d9d640 1 -- 172.21.15.38:0/1430268826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e68102eb0 msgr2=0x7f7e68198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.781+0000 7f7e66d9d640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e68102eb0 0x7f7e68198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.781+0000 7f7e66d9d640 1 -- 172.21.15.38:0/1430268826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e681a0d00 con 0x7f7e6819e320 2024-05-21T15:39:44.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.781+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e681079e0 0x7f7e681993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:39:44.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.781+0000 7f7e66d9d640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e6819e320 0x7f7e681a0740 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000eaa0 tx=0x7f7e6000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:44.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.781+0000 7f7e577fe640 1 -- 172.21.15.38:0/1430268826 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6000cdb0 con 0x7f7e6819e320 2024-05-21T15:39:44.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.781+0000 7f7e577fe640 1 -- 172.21.15.38:0/1430268826 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e60005020 con 0x7f7e6819e320 2024-05-21T15:39:44.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.781+0000 7f7e6ce1b640 1 -- 172.21.15.38:0/1430268826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e681a0ff0 con 0x7f7e6819e320 2024-05-21T15:39:44.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.781+0000 7f7e577fe640 1 -- 172.21.15.38:0/1430268826 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e600116b0 con 0x7f7e6819e320 2024-05-21T15:39:44.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.781+0000 7f7e6ce1b640 1 -- 172.21.15.38:0/1430268826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e681a1400 con 0x7f7e6819e320 2024-05-21T15:39:44.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.784+0000 7f7e577fe640 1 -- 172.21.15.38:0/1430268826 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7e60005560 con 0x7f7e6819e320 2024-05-21T15:39:44.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.784+0000 7f7e73da6740 1 Processor -- start 2024-05-21T15:39:44.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.784+0000 7f7e73da6740 1 -- start start 2024-05-21T15:39:44.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.784+0000 7f7e73da6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d93410 0x562e721334a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e73da6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 0x562e72133a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e73da6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d72730 0x562e7213a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e73da6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562e72097990 con 0x562e71d93410 2024-05-21T15:39:44.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e73da6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562e7203f5b0 con 0x562e71d6a550 2024-05-21T15:39:44.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e73da6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562e72093950 con 0x562e71d72730 2024-05-21T15:39:44.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e577fe640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e500a2b90 0x7f7e500a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e65d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d93410 0x562e721334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e65d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d93410 0x562e721334a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56380/0 (socket says 172.21.15.38:56380) 2024-05-21T15:39:44.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e65d9b640 1 -- 172.21.15.38:0/3037629475 learned_addr learned my addr 172.21.15.38:0/3037629475 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:44.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e66d9d640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e500a2b90 0x7f7e500a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e6659c640 1 --2- 172.21.15.38:0/3037629475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 0x562e72133a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/3037629475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d72730 0x562e7213a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e577fe640 1 -- 172.21.15.38:0/1430268826 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7e600cf310 con 0x7f7e6819e320 2024-05-21T15:39:44.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e65d9b640 1 -- 172.21.15.38:0/3037629475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d72730 msgr2=0x562e7213a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/3037629475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d72730 0x562e7213a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e65d9b640 1 -- 172.21.15.38:0/3037629475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 msgr2=0x562e72133a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/3037629475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 0x562e72133a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e65d9b640 1 -- 172.21.15.38:0/3037629475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e71e737e0 con 0x562e71d93410 2024-05-21T15:39:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e6659c640 1 --2- 172.21.15.38:0/3037629475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 0x562e72133a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:39:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/3037629475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d93410 0x562e721334a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c008fc0 tx=0x7f7e5c008640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739b06e1bf592dc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e66d9d640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e500a2b90 0x7f7e500a50a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7e600cd130 tx=0x7f7e6006b9d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e54ff9640 1 -- 172.21.15.38:0/3037629475 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c018030 con 0x562e71d93410 2024-05-21T15:39:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e54ff9640 1 -- 172.21.15.38:0/3037629475 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e5c005500 con 0x562e71d93410 2024-05-21T15:39:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.785+0000 7f7e54ff9640 1 -- 172.21.15.38:0/3037629475 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0108a0 con 0x562e71d93410 2024-05-21T15:39:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.786+0000 7f7e73da6740 1 -- 172.21.15.38:0/3037629475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d93410 msgr2=0x562e721334a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.786+0000 7f7e73da6740 1 --2- 172.21.15.38:0/3037629475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d93410 0x562e721334a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c008fc0 tx=0x7f7e5c008640 comp rx=0 tx=0).stop 2024-05-21T15:39:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.786+0000 7f7e73da6740 1 -- 172.21.15.38:0/3037629475 shutdown_connections 2024-05-21T15:39:44.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.786+0000 7f7e73da6740 1 --2- 172.21.15.38:0/3037629475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d72730 0x562e7213a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.786+0000 7f7e73da6740 1 --2- 172.21.15.38:0/3037629475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 0x562e72133a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.786+0000 7f7e73da6740 1 --2- 172.21.15.38:0/3037629475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d93410 0x562e721334a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.786+0000 7f7e73da6740 1 -- 172.21.15.38:0/3037629475 >> 172.21.15.38:0/3037629475 conn(0x562e71d87560 msgr2=0x562e7212ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.786+0000 7f7e73da6740 1 -- 172.21.15.38:0/3037629475 shutdown_connections 2024-05-21T15:39:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.786+0000 7f7e73da6740 1 -- 172.21.15.38:0/3037629475 wait complete. 2024-05-21T15:39:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.787+0000 7f7e73da6740 1 -- start start 2024-05-21T15:39:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.787+0000 7f7e73da6740 1 Processor -- start 2024-05-21T15:39:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.787+0000 7f7e73da6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 0x562e7213d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.787+0000 7f7e73da6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d72730 0x562e7213d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.787+0000 7f7e73da6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d93410 0x562e72144820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.787+0000 7f7e73da6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562e7213b370 con 0x562e71d93410 2024-05-21T15:39:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.787+0000 7f7e73da6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562e72052b40 con 0x562e71d6a550 2024-05-21T15:39:44.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.787+0000 7f7e73da6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562e7213b4f0 con 0x562e71d72730 2024-05-21T15:39:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.787+0000 7f7e73da6740 10 client.0 _ll_register_callbacks cb 0x562e720470f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.787+0000 7f7e65d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d93410 0x562e72144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.787+0000 7f7e65d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d93410 0x562e72144820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56392/0 (socket says 172.21.15.38:56392) 2024-05-21T15:39:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.787+0000 7f7e65d9b640 1 -- 172.21.15.38:0/1407233131 learned_addr learned my addr 172.21.15.38:0/1407233131 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.787+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/1407233131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 0x562e7213d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e6659c640 1 --2- 172.21.15.38:0/1407233131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d72730 0x562e7213d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e65d9b640 1 -- 172.21.15.38:0/1407233131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d72730 msgr2=0x562e7213d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/1407233131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d72730 0x562e7213d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e65d9b640 1 -- 172.21.15.38:0/1407233131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 msgr2=0x562e7213d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/1407233131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 0x562e7213d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e65d9b640 1 -- 172.21.15.38:0/1407233131 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e71ed3f70 con 0x562e71d93410 2024-05-21T15:39:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e6659c640 1 --2- 172.21.15.38:0/1407233131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d72730 0x562e7213d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:39:44.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/1407233131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d93410 0x562e72144820 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c008930 tx=0x7f7e5c008880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d849731f1b27924e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:44.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:44.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e4dffb640 1 -- 172.21.15.38:0/1407233131 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c018050 con 0x562e71d93410 2024-05-21T15:39:44.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e4dffb640 1 -- 172.21.15.38:0/1407233131 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7e5c010640 con 0x562e71d93410 2024-05-21T15:39:44.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e73da6740 1 -- 172.21.15.38:0/1407233131 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e5c005500 con 0x562e71d93410 2024-05-21T15:39:44.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e4dffb640 1 -- 172.21.15.38:0/1407233131 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7e5c033050 con 0x562e71d93410 2024-05-21T15:39:44.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e73da6740 10 client.16878 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:44.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e73da6740 1 -- 172.21.15.38:0/1407233131 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562e71d6b170 con 0x562e71d93410 2024-05-21T15:39:44.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.788+0000 7f7e4dffb640 1 -- 172.21.15.38:0/1407233131 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e5c005500 con 0x562e71d93410 2024-05-21T15:39:44.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.789+0000 7f7e4dffb640 1 -- 172.21.15.38:0/1407233131 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f7e5c02b020 con 0x562e71d93410 2024-05-21T15:39:44.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.789+0000 7f7e73da6740 10 client.16878 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:44.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.789+0000 7f7e73da6740 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:44.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.789+0000 7f7e73da6740 20 client.16878 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:44.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.789+0000 7f7e73da6740 1 --2- 172.21.15.38:0/1407233131 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562e71d521f0 0x562e7214ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.789+0000 7f7e73da6740 4 client.16878 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:44.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.789+0000 7f7e73da6740 1 -- 172.21.15.38:0/1407233131 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e720a50c0 con 0x562e71d521f0 2024-05-21T15:39:44.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.789+0000 7f7e6659c640 1 --2- 172.21.15.38:0/1407233131 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562e71d521f0 0x562e7214ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.789+0000 7f7e4dffb640 10 client.16878 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:44.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.789+0000 7f7e6659c640 1 --2- 172.21.15.38:0/1407233131 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562e71d521f0 0x562e7214ab10 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bb15506dfc6dfbc server_cookie=c0cddfb194fdd411 in_seq=0 out_seq=0 2024-05-21T15:39:44.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.790+0000 7f7e4dffb640 1 -- 172.21.15.38:0/1407233131 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e720a50c0 con 0x562e71d521f0 2024-05-21T15:39:44.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.790+0000 7f7e4dffb640 10 client.16878 handle_command_reply: tid=0 2024-05-21T15:39:44.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.790+0000 7f7e73da6740 1 client.16878 shutdown 2024-05-21T15:39:44.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.790+0000 7f7e73da6740 1 -- 172.21.15.38:0/1407233131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d93410 msgr2=0x562e72144820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.790+0000 7f7e73da6740 1 --2- 172.21.15.38:0/1407233131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d93410 0x562e72144820 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c008930 tx=0x7f7e5c008880 comp rx=0 tx=0).stop 2024-05-21T15:39:44.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.790+0000 7f7e73da6740 1 -- 172.21.15.38:0/1407233131 shutdown_connections 2024-05-21T15:39:44.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.790+0000 7f7e73da6740 1 --2- 172.21.15.38:0/1407233131 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562e71d521f0 0x562e7214ab10 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.791+0000 7f7e73da6740 1 --2- 172.21.15.38:0/1407233131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d93410 0x562e72144820 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.791+0000 7f7e4dffb640 0 client.16878 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:44.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.791+0000 7f7e73da6740 1 --2- 172.21.15.38:0/1407233131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d72730 0x562e7213d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.791+0000 7f7e73da6740 1 --2- 172.21.15.38:0/1407233131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 0x562e7213d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.791+0000 7f7e73da6740 1 -- 172.21.15.38:0/1407233131 >> 172.21.15.38:0/1407233131 conn(0x562e71d87560 msgr2=0x562e7212de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:44.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.791+0000 7f7e73da6740 1 -- 172.21.15.38:0/1407233131 shutdown_connections 2024-05-21T15:39:44.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.791+0000 7f7e73da6740 1 -- 172.21.15.38:0/1407233131 wait complete. 2024-05-21T15:39:44.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.791+0000 7f7e73da6740 20 client.16878 trim_cache size 0 max 16384 2024-05-21T15:39:44.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.809+0000 7f7e73da6740 1 Processor -- start 2024-05-21T15:39:44.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.809+0000 7f7e73da6740 1 -- start start 2024-05-21T15:39:44.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.809+0000 7f7e73da6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d6a550 0x562e7213bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.809+0000 7f7e73da6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d72730 0x562e72144740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.809+0000 7f7e73da6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d93410 0x562e7212d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.809+0000 7f7e73da6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681aa400 con 0x562e71d6a550 2024-05-21T15:39:44.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.809+0000 7f7e73da6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681ac030 con 0x562e71d72730 2024-05-21T15:39:44.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.809+0000 7f7e73da6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681aaad0 con 0x562e71d93410 2024-05-21T15:39:44.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.809+0000 7f7e65d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d6a550 0x562e7213bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.809+0000 7f7e65d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d6a550 0x562e7213bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56394/0 (socket says 172.21.15.38:56394) 2024-05-21T15:39:44.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.809+0000 7f7e65d9b640 1 -- 172.21.15.38:0/3810292888 learned_addr learned my addr 172.21.15.38:0/3810292888 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:44.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.809+0000 7f7e6659c640 1 --2- 172.21.15.38:0/3810292888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d72730 0x562e72144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.809+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/3810292888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d93410 0x562e7212d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.810+0000 7f7e65d9b640 1 -- 172.21.15.38:0/3810292888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d93410 msgr2=0x562e7212d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.810+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/3810292888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d93410 0x562e7212d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.810+0000 7f7e65d9b640 1 -- 172.21.15.38:0/3810292888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d72730 msgr2=0x562e72144740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.810+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/3810292888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d72730 0x562e72144740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.810+0000 7f7e65d9b640 1 -- 172.21.15.38:0/3810292888 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e7212f790 con 0x562e71d6a550 2024-05-21T15:39:44.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.810+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/3810292888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d6a550 0x562e7213bfd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c005b90 tx=0x7f7e5c005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a2426bdce6de4d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:44.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.810+0000 7f7e4ffff640 1 -- 172.21.15.38:0/3810292888 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c03c070 con 0x562e71d6a550 2024-05-21T15:39:44.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.810+0000 7f7e4ffff640 1 -- 172.21.15.38:0/3810292888 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e5c022070 con 0x562e71d6a550 2024-05-21T15:39:44.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.810+0000 7f7e4ffff640 1 -- 172.21.15.38:0/3810292888 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c018070 con 0x562e71d6a550 2024-05-21T15:39:44.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.811+0000 7f7e73da6740 1 -- 172.21.15.38:0/3810292888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d6a550 msgr2=0x562e7213bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.811+0000 7f7e73da6740 1 --2- 172.21.15.38:0/3810292888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d6a550 0x562e7213bfd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c005b90 tx=0x7f7e5c005da0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.811+0000 7f7e73da6740 1 -- 172.21.15.38:0/3810292888 shutdown_connections 2024-05-21T15:39:44.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.811+0000 7f7e73da6740 1 --2- 172.21.15.38:0/3810292888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d93410 0x562e7212d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.811+0000 7f7e73da6740 1 --2- 172.21.15.38:0/3810292888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d72730 0x562e72144740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.811+0000 7f7e73da6740 1 --2- 172.21.15.38:0/3810292888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d6a550 0x562e7213bfd0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.811+0000 7f7e73da6740 1 -- 172.21.15.38:0/3810292888 >> 172.21.15.38:0/3810292888 conn(0x562e71d87560 msgr2=0x562e7213b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:44.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.811+0000 7f7e73da6740 1 -- 172.21.15.38:0/3810292888 shutdown_connections 2024-05-21T15:39:44.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.811+0000 7f7e73da6740 1 -- 172.21.15.38:0/3810292888 wait complete. 2024-05-21T15:39:44.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.811+0000 7f7e73da6740 1 -- start start 2024-05-21T15:39:44.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.811+0000 7f7e73da6740 1 Processor -- start 2024-05-21T15:39:44.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.811+0000 7f7e73da6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 0x562e721419d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.811+0000 7f7e73da6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d72730 0x562e72141f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e73da6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d93410 0x562e72142e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e73da6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681ab8a0 con 0x562e71d72730 2024-05-21T15:39:44.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e73da6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681ab1a0 con 0x562e71d6a550 2024-05-21T15:39:44.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e73da6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681a83e0 con 0x562e71d93410 2024-05-21T15:39:44.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e73da6740 10 client.0 _ll_register_callbacks cb 0x7f7e5c0123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:44.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e6659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d72730 0x562e72141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e6659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d72730 0x562e72141f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56396/0 (socket says 172.21.15.38:56396) 2024-05-21T15:39:44.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e6659c640 1 -- 172.21.15.38:0/2732162640 learned_addr learned my addr 172.21.15.38:0/2732162640 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:44.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/2732162640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 0x562e721419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/2732162640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d93410 0x562e72142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e6659c640 1 -- 172.21.15.38:0/2732162640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d93410 msgr2=0x562e72142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e6659c640 1 --2- 172.21.15.38:0/2732162640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d93410 0x562e72142e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e6659c640 1 -- 172.21.15.38:0/2732162640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 msgr2=0x562e721419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e6659c640 1 --2- 172.21.15.38:0/2732162640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 0x562e721419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e6659c640 1 -- 172.21.15.38:0/2732162640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e72143400 con 0x562e71d72730 2024-05-21T15:39:44.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e6659c640 1 --2- 172.21.15.38:0/2732162640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d72730 0x562e72141f10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009230 tx=0x7f7e5802abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5660708c51abced server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:44.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:44.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e4dffb640 1 -- 172.21.15.38:0/2732162640 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58035040 con 0x562e71d72730 2024-05-21T15:39:44.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e4dffb640 1 -- 172.21.15.38:0/2732162640 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7e58005040 con 0x562e71d72730 2024-05-21T15:39:44.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e73da6740 1 -- 172.21.15.38:0/2732162640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e5c022070 con 0x562e71d72730 2024-05-21T15:39:44.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.812+0000 7f7e4dffb640 1 -- 172.21.15.38:0/2732162640 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7e58053050 con 0x562e71d72730 2024-05-21T15:39:44.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.813+0000 7f7e73da6740 10 client.16884 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:44.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.813+0000 7f7e73da6740 1 -- 172.21.15.38:0/2732162640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562e72144730 con 0x562e71d72730 2024-05-21T15:39:44.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.813+0000 7f7e4dffb640 1 -- 172.21.15.38:0/2732162640 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e5c022070 con 0x562e71d72730 2024-05-21T15:39:44.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.813+0000 7f7e4dffb640 1 -- 172.21.15.38:0/2732162640 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f7e580443b0 con 0x562e71d72730 2024-05-21T15:39:44.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.813+0000 7f7e73da6740 10 client.16884 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:44.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.813+0000 7f7e73da6740 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:44.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.813+0000 7f7e73da6740 20 client.16884 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:44.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.813+0000 7f7e73da6740 1 --2- 172.21.15.38:0/2732162640 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562e71d27630 0x562e721b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.813+0000 7f7e73da6740 4 client.16884 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:39:44.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.813+0000 7f7e73da6740 1 -- 172.21.15.38:0/2732162640 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562e7209f410 con 0x562e71d27630 2024-05-21T15:39:44.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.813+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/2732162640 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562e71d27630 0x562e721b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.814+0000 7f7e65d9b640 1 --2- 172.21.15.38:0/2732162640 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562e71d27630 0x562e721b53e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e581765216fbe741 server_cookie=edf5d226939c38da in_seq=0 out_seq=0 2024-05-21T15:39:44.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.814+0000 7f7e4dffb640 10 client.16884 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:44.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e4dffb640 1 -- 172.21.15.38:0/2732162640 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562e7209f410 con 0x562e71d27630 2024-05-21T15:39:44.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e4dffb640 10 client.16884 handle_command_reply: tid=0 2024-05-21T15:39:44.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e73da6740 1 client.16884 shutdown 2024-05-21T15:39:44.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e73da6740 1 -- 172.21.15.38:0/2732162640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d72730 msgr2=0x562e72141f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e73da6740 1 --2- 172.21.15.38:0/2732162640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d72730 0x562e72141f10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009230 tx=0x7f7e5802abc0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e73da6740 1 -- 172.21.15.38:0/2732162640 shutdown_connections 2024-05-21T15:39:44.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e73da6740 1 --2- 172.21.15.38:0/2732162640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71d93410 0x562e72142e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e73da6740 1 --2- 172.21.15.38:0/2732162640 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562e71d27630 0x562e721b53e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e73da6740 1 --2- 172.21.15.38:0/2732162640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e71d72730 0x562e72141f10 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e73da6740 1 --2- 172.21.15.38:0/2732162640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562e71d6a550 0x562e721419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e4dffb640 0 client.16884 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:44.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e73da6740 1 -- 172.21.15.38:0/2732162640 >> 172.21.15.38:0/2732162640 conn(0x562e71d87560 msgr2=0x562e7212e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:44.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e73da6740 1 -- 172.21.15.38:0/2732162640 shutdown_connections 2024-05-21T15:39:44.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e73da6740 1 -- 172.21.15.38:0/2732162640 wait complete. 2024-05-21T15:39:44.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.815+0000 7f7e73da6740 20 client.16884 trim_cache size 0 max 16384 2024-05-21T15:39:44.819 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:39:44.820 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-05-21T15:39:44.820 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "ed10b802-c372-4ddd-94f1-e1989666efbb", 2024-05-21T15:39:44.820 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-05-21T15:39:44.820 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:39:44.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.816+0000 7f7e6ce1b640 1 -- 172.21.15.38:0/1430268826 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e500a2b90 msgr2=0x7f7e500a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.816+0000 7f7e6ce1b640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e500a2b90 0x7f7e500a50a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7e600cd130 tx=0x7f7e6006b9d0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.816+0000 7f7e6ce1b640 1 -- 172.21.15.38:0/1430268826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e6819e320 msgr2=0x7f7e681a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.816+0000 7f7e6ce1b640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e6819e320 0x7f7e681a0740 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000eaa0 tx=0x7f7e6000ef60 comp rx=0 tx=0).stop 2024-05-21T15:39:44.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.816+0000 7f7e6ce1b640 1 -- 172.21.15.38:0/1430268826 shutdown_connections 2024-05-21T15:39:44.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.816+0000 7f7e6ce1b640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e500a2b90 0x7f7e500a50a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.816+0000 7f7e6ce1b640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e6819e320 0x7f7e681a0740 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.816+0000 7f7e6ce1b640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e681079e0 0x7f7e681993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.816+0000 7f7e6ce1b640 1 --2- 172.21.15.38:0/1430268826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e68102eb0 0x7f7e68198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.816+0000 7f7e6ce1b640 1 -- 172.21.15.38:0/1430268826 >> 172.21.15.38:0/1430268826 conn(0x7f7e680fc330 msgr2=0x7f7e680fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:44.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.816+0000 7f7e6ce1b640 1 -- 172.21.15.38:0/1430268826 shutdown_connections 2024-05-21T15:39:44.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.816+0000 7f7e6ce1b640 1 -- 172.21.15.38:0/1430268826 wait complete. 2024-05-21T15:39:44.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:39:44.826 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:39:44.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.954+0000 7f939b6bc640 1 -- 172.21.15.38:0/1268679744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9394102ab0 msgr2=0x7f93941037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.954+0000 7f939b6bc640 1 --2- 172.21.15.38:0/1268679744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9394102ab0 0x7f93941037b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9390009900 tx=0x7f939002a410 comp rx=0 tx=0).stop 2024-05-21T15:39:44.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.954+0000 7f939b6bc640 1 -- 172.21.15.38:0/1268679744 shutdown_connections 2024-05-21T15:39:44.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.954+0000 7f939b6bc640 1 --2- 172.21.15.38:0/1268679744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93941089f0 0x7f939410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.954+0000 7f939b6bc640 1 --2- 172.21.15.38:0/1268679744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9394103cf0 0x7f9394104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.954+0000 7f939b6bc640 1 --2- 172.21.15.38:0/1268679744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9394102ab0 0x7f93941037b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.954+0000 7f939b6bc640 1 -- 172.21.15.38:0/1268679744 >> 172.21.15.38:0/1268679744 conn(0x7f93940fdcc0 msgr2=0x7f9394100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:44.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.954+0000 7f939b6bc640 1 -- 172.21.15.38:0/1268679744 shutdown_connections 2024-05-21T15:39:44.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.954+0000 7f939b6bc640 1 -- 172.21.15.38:0/1268679744 wait complete. 2024-05-21T15:39:44.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f939b6bc640 1 Processor -- start 2024-05-21T15:39:44.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f939b6bc640 1 -- start start 2024-05-21T15:39:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f939b6bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9394102ab0 0x7f9394199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f939b6bc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9394103cf0 0x7f93941997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f939b6bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93941089f0 0x7f93941a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f939b6bc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f939410e700 con 0x7f93941089f0 2024-05-21T15:39:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f939b6bc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f939410e580 con 0x7f9394103cf0 2024-05-21T15:39:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f939b6bc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f939410e880 con 0x7f9394102ab0 2024-05-21T15:39:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f9399458640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9394102ab0 0x7f9394199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f9399c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93941089f0 0x7f93941a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f9399458640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9394102ab0 0x7f9394199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:41598/0 (socket says 172.21.15.38:41598) 2024-05-21T15:39:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f9399c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93941089f0 0x7f93941a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56418/0 (socket says 172.21.15.38:56418) 2024-05-21T15:39:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f9399458640 1 -- 172.21.15.38:0/319884368 learned_addr learned my addr 172.21.15.38:0/319884368 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f9398c57640 1 --2- 172.21.15.38:0/319884368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9394103cf0 0x7f93941997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f9399458640 1 -- 172.21.15.38:0/319884368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9394103cf0 msgr2=0x7f93941997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f9399458640 1 --2- 172.21.15.38:0/319884368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9394103cf0 0x7f93941997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f9399458640 1 -- 172.21.15.38:0/319884368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93941089f0 msgr2=0x7f93941a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f9399458640 1 --2- 172.21.15.38:0/319884368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93941089f0 0x7f93941a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.955+0000 7f9399458640 1 -- 172.21.15.38:0/319884368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93941a0db0 con 0x7f9394102ab0 2024-05-21T15:39:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.956+0000 7f9399458640 1 --2- 172.21.15.38:0/319884368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9394102ab0 0x7f9394199270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9390005ab0 tx=0x7f9390004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.956+0000 7f938a7fc640 1 -- 172.21.15.38:0/319884368 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939003f030 con 0x7f9394102ab0 2024-05-21T15:39:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.956+0000 7f938a7fc640 1 -- 172.21.15.38:0/319884368 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9390003ad0 con 0x7f9394102ab0 2024-05-21T15:39:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.956+0000 7f939b6bc640 1 -- 172.21.15.38:0/319884368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93941a1040 con 0x7f9394102ab0 2024-05-21T15:39:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.956+0000 7f938a7fc640 1 -- 172.21.15.38:0/319884368 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390035070 con 0x7f9394102ab0 2024-05-21T15:39:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.956+0000 7f939b6bc640 1 -- 172.21.15.38:0/319884368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93941a1580 con 0x7f9394102ab0 2024-05-21T15:39:44.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.959+0000 7f938a7fc640 1 -- 172.21.15.38:0/319884368 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9390042030 con 0x7f9394102ab0 2024-05-21T15:39:44.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.959+0000 7f93a21fd740 1 Processor -- start 2024-05-21T15:39:44.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.959+0000 7f93a21fd740 1 -- start start 2024-05-21T15:39:44.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.959+0000 7f93a21fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe969390 0x555ebed08bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.959+0000 7f93a21fd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9404d0 0x555ebed09130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.959+0000 7f93a21fd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9486b0 0x555ebed10030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.959+0000 7f93a21fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555ebec6dd00 con 0x555ebe969390 2024-05-21T15:39:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.959+0000 7f93a21fd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555ebec154c0 con 0x555ebe9404d0 2024-05-21T15:39:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.959+0000 7f93a21fd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555ebec69cc0 con 0x555ebe9486b0 2024-05-21T15:39:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.959+0000 7f9399c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe969390 0x555ebed08bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f9399c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe969390 0x555ebed08bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56430/0 (socket says 172.21.15.38:56430) 2024-05-21T15:39:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f9399458640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9404d0 0x555ebed09130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f9399c59640 1 -- 172.21.15.38:0/3074072282 learned_addr learned my addr 172.21.15.38:0/3074072282 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f9398c57640 1 --2- 172.21.15.38:0/3074072282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9486b0 0x555ebed10030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f9399c59640 1 -- 172.21.15.38:0/3074072282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9486b0 msgr2=0x555ebed10030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f9399c59640 1 --2- 172.21.15.38:0/3074072282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9486b0 0x555ebed10030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f9399c59640 1 -- 172.21.15.38:0/3074072282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9404d0 msgr2=0x555ebed09130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f9399c59640 1 --2- 172.21.15.38:0/3074072282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9404d0 0x555ebed09130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f9399c59640 1 -- 172.21.15.38:0/3074072282 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ebea4b7d0 con 0x555ebe969390 2024-05-21T15:39:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f938a7fc640 1 --2- 172.21.15.38:0/319884368 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f93800a2c10 0x7f93800a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f9399458640 1 --2- 172.21.15.38:0/319884368 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f93800a2c10 0x7f93800a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f9399c59640 1 --2- 172.21.15.38:0/3074072282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe969390 0x555ebed08bb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f938c009fd0 tx=0x7f938c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560bfef582f0b253 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f938a7fc640 1 -- 172.21.15.38:0/319884368 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f93900ef620 con 0x7f9394102ab0 2024-05-21T15:39:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f937f7fe640 1 -- 172.21.15.38:0/3074072282 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c004d20 con 0x555ebe969390 2024-05-21T15:39:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f937f7fe640 1 -- 172.21.15.38:0/3074072282 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f938c004050 con 0x555ebe969390 2024-05-21T15:39:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.960+0000 7f937f7fe640 1 -- 172.21.15.38:0/3074072282 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c005280 con 0x555ebe969390 2024-05-21T15:39:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.961+0000 7f93a21fd740 1 -- 172.21.15.38:0/3074072282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe969390 msgr2=0x555ebed08bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.961+0000 7f9399458640 1 --2- 172.21.15.38:0/319884368 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f93800a2c10 0x7f93800a5120 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f93900c4510 tx=0x7f93900f50e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.961+0000 7f93a21fd740 1 --2- 172.21.15.38:0/3074072282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe969390 0x555ebed08bb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f938c009fd0 tx=0x7f938c0118d0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.961+0000 7f93a21fd740 1 -- 172.21.15.38:0/3074072282 shutdown_connections 2024-05-21T15:39:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.961+0000 7f93a21fd740 1 --2- 172.21.15.38:0/3074072282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9486b0 0x555ebed10030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.961+0000 7f93a21fd740 1 --2- 172.21.15.38:0/3074072282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9404d0 0x555ebed09130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.961+0000 7f93a21fd740 1 --2- 172.21.15.38:0/3074072282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe969390 0x555ebed08bb0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.961+0000 7f93a21fd740 1 -- 172.21.15.38:0/3074072282 >> 172.21.15.38:0/3074072282 conn(0x555ebe95d4e0 msgr2=0x555ebed04540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:44.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.961+0000 7f93a21fd740 1 -- 172.21.15.38:0/3074072282 shutdown_connections 2024-05-21T15:39:44.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.961+0000 7f93a21fd740 1 -- 172.21.15.38:0/3074072282 wait complete. 2024-05-21T15:39:44.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.962+0000 7f93a21fd740 1 -- start start 2024-05-21T15:39:44.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.962+0000 7f93a21fd740 1 Processor -- start 2024-05-21T15:39:44.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.962+0000 7f93a21fd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9404d0 0x555ebed12ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.962+0000 7f93a21fd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9486b0 0x555ebed12ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.962+0000 7f93a21fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe969390 0x555ebed19f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.962+0000 7f93a21fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555ebed10a80 con 0x555ebe969390 2024-05-21T15:39:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.962+0000 7f93a21fd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555ebec28b90 con 0x555ebe9486b0 2024-05-21T15:39:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.962+0000 7f93a21fd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555ebed10c00 con 0x555ebe9404d0 2024-05-21T15:39:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.962+0000 7f93a21fd740 10 client.0 _ll_register_callbacks cb 0x555ebec64da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.962+0000 7f9399c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe969390 0x555ebed19f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.962+0000 7f9398c57640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9486b0 0x555ebed12ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f9398c57640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9486b0 0x555ebed12ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:37654/0 (socket says 172.21.15.38:37654) 2024-05-21T15:39:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f9399c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe969390 0x555ebed19f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56446/0 (socket says 172.21.15.38:56446) 2024-05-21T15:39:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f9398c57640 1 -- 172.21.15.38:0/2151232921 learned_addr learned my addr 172.21.15.38:0/2151232921 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f9399c59640 1 --2- 172.21.15.38:0/2151232921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9404d0 0x555ebed12ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f9398c57640 1 -- 172.21.15.38:0/2151232921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9404d0 msgr2=0x555ebed12ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f9398c57640 1 --2- 172.21.15.38:0/2151232921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9404d0 0x555ebed12ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f9398c57640 1 -- 172.21.15.38:0/2151232921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe969390 msgr2=0x555ebed19f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f9398c57640 1 --2- 172.21.15.38:0/2151232921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe969390 0x555ebed19f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f9398c57640 1 -- 172.21.15.38:0/2151232921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ebe9813f0 con 0x555ebe9486b0 2024-05-21T15:39:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f9399c59640 1 --2- 172.21.15.38:0/2151232921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe969390 0x555ebed19f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f9398c57640 1 --2- 172.21.15.38:0/2151232921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9486b0 0x555ebed12ff0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9384012700 tx=0x7f9384012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b1b7642e374d5d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f937cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:39:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f937cff9640 1 -- 172.21.15.38:0/2151232921 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384004030 con 0x555ebe9486b0 2024-05-21T15:39:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f93a21fd740 1 -- 172.21.15.38:0/2151232921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f938c004050 con 0x555ebe9486b0 2024-05-21T15:39:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f937cff9640 1 -- 172.21.15.38:0/2151232921 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f93840041d0 con 0x555ebe9486b0 2024-05-21T15:39:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f937cff9640 1 -- 172.21.15.38:0/2151232921 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f93840043d0 con 0x555ebe9486b0 2024-05-21T15:39:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f93a21fd740 10 client.26519 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.963+0000 7f93a21fd740 1 -- 172.21.15.38:0/2151232921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555ebe9410f0 con 0x555ebe9486b0 2024-05-21T15:39:44.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.964+0000 7f937cff9640 1 -- 172.21.15.38:0/2151232921 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f938c004050 con 0x555ebe9486b0 2024-05-21T15:39:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.964+0000 7f937cff9640 1 -- 172.21.15.38:0/2151232921 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f9384003ae0 con 0x555ebe9486b0 2024-05-21T15:39:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.964+0000 7f93a21fd740 10 client.26519 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.964+0000 7f93a21fd740 10 client.26519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.964+0000 7f93a21fd740 20 client.26519 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.964+0000 7f93a21fd740 1 --2- 172.21.15.38:0/2151232921 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555ebe927f60 0x555ebed20830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.964+0000 7f93a21fd740 4 client.26519 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.964+0000 7f93a21fd740 1 -- 172.21.15.38:0/2151232921 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ebec7b430 con 0x555ebe927f60 2024-05-21T15:39:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.964+0000 7f9399c59640 1 --2- 172.21.15.38:0/2151232921 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555ebe927f60 0x555ebed20830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.965+0000 7f937cff9640 10 client.26519 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.965+0000 7f9399c59640 1 --2- 172.21.15.38:0/2151232921 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555ebe927f60 0x555ebed20830 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ead3489bff59813 server_cookie=fd68bb5dd82d682 in_seq=0 out_seq=0 2024-05-21T15:39:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.965+0000 7f937cff9640 1 -- 172.21.15.38:0/2151232921 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ebec7b430 con 0x555ebe927f60 2024-05-21T15:39:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.965+0000 7f937cff9640 10 client.26519 handle_command_reply: tid=0 2024-05-21T15:39:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.965+0000 7f93a21fd740 1 client.26519 shutdown 2024-05-21T15:39:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.965+0000 7f93a21fd740 1 -- 172.21.15.38:0/2151232921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9486b0 msgr2=0x555ebed12ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.965+0000 7f93a21fd740 1 --2- 172.21.15.38:0/2151232921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9486b0 0x555ebed12ff0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9384012700 tx=0x7f9384012bc0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.965+0000 7f93a21fd740 1 -- 172.21.15.38:0/2151232921 shutdown_connections 2024-05-21T15:39:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.965+0000 7f93a21fd740 1 --2- 172.21.15.38:0/2151232921 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555ebe927f60 0x555ebed20830 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.965+0000 7f93a21fd740 1 --2- 172.21.15.38:0/2151232921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe969390 0x555ebed19f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.965+0000 7f93a21fd740 1 --2- 172.21.15.38:0/2151232921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9486b0 0x555ebed12ff0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.965+0000 7f93a21fd740 1 --2- 172.21.15.38:0/2151232921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9404d0 0x555ebed12ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.965+0000 7f937cff9640 0 client.26519 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.965+0000 7f93a21fd740 1 -- 172.21.15.38:0/2151232921 >> 172.21.15.38:0/2151232921 conn(0x555ebe95d4e0 msgr2=0x555ebed035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.966+0000 7f93a21fd740 1 -- 172.21.15.38:0/2151232921 shutdown_connections 2024-05-21T15:39:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.966+0000 7f93a21fd740 1 -- 172.21.15.38:0/2151232921 wait complete. 2024-05-21T15:39:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.966+0000 7f93a21fd740 20 client.26519 trim_cache size 0 max 16384 2024-05-21T15:39:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.984+0000 7f93a21fd740 1 Processor -- start 2024-05-21T15:39:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.984+0000 7f93a21fd740 1 -- start start 2024-05-21T15:39:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.984+0000 7f93a21fd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9404d0 0x555ebed110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.984+0000 7f93a21fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe9486b0 0x555ebed11630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.984+0000 7f93a21fd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe969390 0x555ebed19e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.984+0000 7f93a21fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f93941aa3b0 con 0x555ebe9486b0 2024-05-21T15:39:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.984+0000 7f93a21fd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f93941abfe0 con 0x555ebe9404d0 2024-05-21T15:39:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.984+0000 7f93a21fd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f93941aaa80 con 0x555ebe969390 2024-05-21T15:39:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.985+0000 7f9398c57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe9486b0 0x555ebed11630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.985+0000 7f9398c57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe9486b0 0x555ebed11630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56460/0 (socket says 172.21.15.38:56460) 2024-05-21T15:39:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.985+0000 7f9398c57640 1 -- 172.21.15.38:0/165590153 learned_addr learned my addr 172.21.15.38:0/165590153 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.985+0000 7f9399c59640 1 --2- 172.21.15.38:0/165590153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9404d0 0x555ebed110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.985+0000 7f9399c59640 1 --2- 172.21.15.38:0/165590153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe969390 0x555ebed19e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.985+0000 7f9398c57640 1 -- 172.21.15.38:0/165590153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe969390 msgr2=0x555ebed19e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.985+0000 7f9398c57640 1 --2- 172.21.15.38:0/165590153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe969390 0x555ebed19e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.985+0000 7f9398c57640 1 -- 172.21.15.38:0/165590153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9404d0 msgr2=0x555ebed110f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.985+0000 7f9398c57640 1 --2- 172.21.15.38:0/165590153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9404d0 0x555ebed110f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.985+0000 7f9398c57640 1 -- 172.21.15.38:0/165590153 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ebed1a3b0 con 0x555ebe9486b0 2024-05-21T15:39:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.985+0000 7f9398c57640 1 --2- 172.21.15.38:0/165590153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe9486b0 0x555ebed11630 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9384003eb0 tx=0x7f9384005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dae55378a010cc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.985+0000 7f937effd640 1 -- 172.21.15.38:0/165590153 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938401c580 con 0x555ebe9486b0 2024-05-21T15:39:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.985+0000 7f937effd640 1 -- 172.21.15.38:0/165590153 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f938401ce30 con 0x555ebe9486b0 2024-05-21T15:39:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.985+0000 7f937effd640 1 -- 172.21.15.38:0/165590153 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384003a90 con 0x555ebe9486b0 2024-05-21T15:39:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.986+0000 7f93a21fd740 1 -- 172.21.15.38:0/165590153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe9486b0 msgr2=0x555ebed11630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.986+0000 7f93a21fd740 1 --2- 172.21.15.38:0/165590153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe9486b0 0x555ebed11630 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9384003eb0 tx=0x7f9384005810 comp rx=0 tx=0).stop 2024-05-21T15:39:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.986+0000 7f93a21fd740 1 -- 172.21.15.38:0/165590153 shutdown_connections 2024-05-21T15:39:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.986+0000 7f93a21fd740 1 --2- 172.21.15.38:0/165590153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe969390 0x555ebed19e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.986+0000 7f93a21fd740 1 --2- 172.21.15.38:0/165590153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe9486b0 0x555ebed11630 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.986+0000 7f93a21fd740 1 --2- 172.21.15.38:0/165590153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe9404d0 0x555ebed110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.986+0000 7f93a21fd740 1 -- 172.21.15.38:0/165590153 >> 172.21.15.38:0/165590153 conn(0x555ebe95d4e0 msgr2=0x555ebed0cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.986+0000 7f93a21fd740 1 -- 172.21.15.38:0/165590153 shutdown_connections 2024-05-21T15:39:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.986+0000 7f93a21fd740 1 -- 172.21.15.38:0/165590153 wait complete. 2024-05-21T15:39:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.987+0000 7f93a21fd740 1 -- start start 2024-05-21T15:39:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.987+0000 7f93a21fd740 1 Processor -- start 2024-05-21T15:39:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.987+0000 7f93a21fd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe9404d0 0x555ebed185b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.987+0000 7f93a21fd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9486b0 0x555ebed18af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.987+0000 7f93a21fd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe969390 0x555ebed19030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.987+0000 7f93a21fd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f93941ab850 con 0x555ebe9404d0 2024-05-21T15:39:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.987+0000 7f93a21fd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f93941ab150 con 0x555ebe969390 2024-05-21T15:39:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.987+0000 7f93a21fd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f93941a8390 con 0x555ebe9486b0 2024-05-21T15:39:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.987+0000 7f93a21fd740 10 client.0 _ll_register_callbacks cb 0x7f9384009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.987+0000 7f9399c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe9404d0 0x555ebed185b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.987+0000 7f9399c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe9404d0 0x555ebed185b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56468/0 (socket says 172.21.15.38:56468) 2024-05-21T15:39:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.987+0000 7f9399c59640 1 -- 172.21.15.38:0/1236542593 learned_addr learned my addr 172.21.15.38:0/1236542593 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.987+0000 7f9398c57640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9486b0 0x555ebed18af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f9399c59640 1 --2- 172.21.15.38:0/1236542593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe969390 0x555ebed19030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f9399c59640 1 -- 172.21.15.38:0/1236542593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9486b0 msgr2=0x555ebed18af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f9399c59640 1 --2- 172.21.15.38:0/1236542593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9486b0 0x555ebed18af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f9399c59640 1 -- 172.21.15.38:0/1236542593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe969390 msgr2=0x555ebed19030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f9399c59640 1 --2- 172.21.15.38:0/1236542593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe969390 0x555ebed19030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f9399c59640 1 -- 172.21.15.38:0/1236542593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ebed86bd0 con 0x555ebe9404d0 2024-05-21T15:39:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f9399c59640 1 --2- 172.21.15.38:0/1236542593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe9404d0 0x555ebed185b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f938c005cc0 tx=0x7f938c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ada338e8bac62c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f937cff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f937cff9640 1 -- 172.21.15.38:0/1236542593 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c004050 con 0x555ebe9404d0 2024-05-21T15:39:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f937cff9640 1 -- 172.21.15.38:0/1236542593 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f938c0041f0 con 0x555ebe9404d0 2024-05-21T15:39:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f93a21fd740 1 -- 172.21.15.38:0/1236542593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f938401ce30 con 0x555ebe9404d0 2024-05-21T15:39:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f937cff9640 1 -- 172.21.15.38:0/1236542593 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f938c0043f0 con 0x555ebe9404d0 2024-05-21T15:39:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f93a21fd740 10 client.16902 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f93a21fd740 1 -- 172.21.15.38:0/1236542593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555ebed87dc0 con 0x555ebe9404d0 2024-05-21T15:39:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.988+0000 7f937cff9640 1 -- 172.21.15.38:0/1236542593 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f938401ce30 con 0x555ebe9404d0 2024-05-21T15:39:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.989+0000 7f937cff9640 1 -- 172.21.15.38:0/1236542593 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f938c00f4e0 con 0x555ebe9404d0 2024-05-21T15:39:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.989+0000 7f93a21fd740 10 client.16902 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.989+0000 7f93a21fd740 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.989+0000 7f93a21fd740 20 client.16902 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.989+0000 7f93a21fd740 1 --2- 172.21.15.38:0/1236542593 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555ebe8fd3a0 0x555ebed8a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.989+0000 7f93a21fd740 4 client.16902 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:39:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.989+0000 7f93a21fd740 1 -- 172.21.15.38:0/1236542593 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ebec75780 con 0x555ebe8fd3a0 2024-05-21T15:39:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.989+0000 7f9398c57640 1 --2- 172.21.15.38:0/1236542593 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555ebe8fd3a0 0x555ebed8a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.989+0000 7f937cff9640 10 client.16902 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.989+0000 7f9398c57640 1 --2- 172.21.15.38:0/1236542593 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555ebe8fd3a0 0x555ebed8a6f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9c8ec44c508463 server_cookie=69310938ffe6899f in_seq=0 out_seq=0 2024-05-21T15:39:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f937cff9640 1 -- 172.21.15.38:0/1236542593 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x555ebec75780 con 0x555ebe8fd3a0 2024-05-21T15:39:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f937cff9640 10 client.16902 handle_command_reply: tid=0 2024-05-21T15:39:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f93a21fd740 1 client.16902 shutdown 2024-05-21T15:39:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f93a21fd740 1 -- 172.21.15.38:0/1236542593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe9404d0 msgr2=0x555ebed185b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f93a21fd740 1 --2- 172.21.15.38:0/1236542593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe9404d0 0x555ebed185b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f938c005cc0 tx=0x7f938c0118d0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f93a21fd740 1 -- 172.21.15.38:0/1236542593 shutdown_connections 2024-05-21T15:39:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f93a21fd740 1 --2- 172.21.15.38:0/1236542593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ebe969390 0x555ebed19030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f93a21fd740 1 --2- 172.21.15.38:0/1236542593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555ebe9486b0 0x555ebed18af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f93a21fd740 1 --2- 172.21.15.38:0/1236542593 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x555ebe8fd3a0 0x555ebed8a6f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f93a21fd740 1 --2- 172.21.15.38:0/1236542593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555ebe9404d0 0x555ebed185b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f937cff9640 0 client.16902 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:44.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f93a21fd740 1 -- 172.21.15.38:0/1236542593 >> 172.21.15.38:0/1236542593 conn(0x555ebe95d4e0 msgr2=0x555ebed11080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:44.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f93a21fd740 1 -- 172.21.15.38:0/1236542593 shutdown_connections 2024-05-21T15:39:44.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f93a21fd740 1 -- 172.21.15.38:0/1236542593 wait complete. 2024-05-21T15:39:44.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.990+0000 7f93a21fd740 20 client.16902 trim_cache size 0 max 16384 2024-05-21T15:39:44.996 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:39:44.996 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "scrub active (0 inodes in the stack)", 2024-05-21T15:39:44.996 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-05-21T15:39:44.996 INFO:teuthology.orchestra.run.smithi038.stdout: "ed10b802-c372-4ddd-94f1-e1989666efbb": { 2024-05-21T15:39:44.997 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-05-21T15:39:44.997 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "ed10b802-c372-4ddd-94f1-e1989666efbb", 2024-05-21T15:39:44.997 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-05-21T15:39:44.997 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:39:44.997 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:39:44.997 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:39:44.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.991+0000 7f939b6bc640 1 -- 172.21.15.38:0/319884368 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f93800a2c10 msgr2=0x7f93800a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.991+0000 7f939b6bc640 1 --2- 172.21.15.38:0/319884368 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f93800a2c10 0x7f93800a5120 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f93900c4510 tx=0x7f93900f50e0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.991+0000 7f939b6bc640 1 -- 172.21.15.38:0/319884368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9394102ab0 msgr2=0x7f9394199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:44.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.991+0000 7f939b6bc640 1 --2- 172.21.15.38:0/319884368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9394102ab0 0x7f9394199270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9390005ab0 tx=0x7f9390004270 comp rx=0 tx=0).stop 2024-05-21T15:39:44.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.991+0000 7f939b6bc640 1 -- 172.21.15.38:0/319884368 shutdown_connections 2024-05-21T15:39:44.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.991+0000 7f939b6bc640 1 --2- 172.21.15.38:0/319884368 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f93800a2c10 0x7f93800a5120 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.991+0000 7f939b6bc640 1 --2- 172.21.15.38:0/319884368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93941089f0 0x7f93941a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.991+0000 7f939b6bc640 1 --2- 172.21.15.38:0/319884368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9394103cf0 0x7f93941997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.991+0000 7f939b6bc640 1 --2- 172.21.15.38:0/319884368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9394102ab0 0x7f9394199270 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:44.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.991+0000 7f939b6bc640 1 -- 172.21.15.38:0/319884368 >> 172.21.15.38:0/319884368 conn(0x7f93940fdcc0 msgr2=0x7f93941049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:44.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.991+0000 7f939b6bc640 1 -- 172.21.15.38:0/319884368 shutdown_connections 2024-05-21T15:39:44.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:39:44.991+0000 7f939b6bc640 1 -- 172.21.15.38:0/319884368 wait complete. 2024-05-21T15:39:45.000 INFO:tasks.cephfs.filesystem:scrub status for tag:ed10b802-c372-4ddd-94f1-e1989666efbb - {'path': '/', 'tag': 'ed10b802-c372-4ddd-94f1-e1989666efbb', 'options': 'recursive,force'} 2024-05-21T15:39:46.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:45 smithi053 ceph-mon[36496]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 72 MiB/s wr, 15.80k op/s 2024-05-21T15:39:46.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:45 smithi053 ceph-mon[36496]: scrub queued for path: / 2024-05-21T15:39:46.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:45 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:46.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:45 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:39:46.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:45 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:39:46.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:45 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:39:46.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:45 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:39:46.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:45 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:39:46.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:45 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:46.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:45 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:46.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:45 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:46.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:45 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:46.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:45 smithi045 ceph-mon[36409]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 72 MiB/s wr, 15.80k op/s 2024-05-21T15:39:46.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:45 smithi045 ceph-mon[36409]: scrub queued for path: / 2024-05-21T15:39:46.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:45 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:46.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:45 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:39:46.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:45 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:39:46.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:45 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:39:46.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:45 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:39:46.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:45 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:39:46.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:45 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:46.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:45 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:46.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:45 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:46.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:45 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:46.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:45 smithi038 ceph-mon[31995]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 72 MiB/s wr, 15.80k op/s 2024-05-21T15:39:46.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:45 smithi038 ceph-mon[31995]: scrub queued for path: / 2024-05-21T15:39:46.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:45 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:46.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:45 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:39:46.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:45 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:39:46.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:45 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:39:46.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:45 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:39:46.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:45 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:39:46.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:45 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:46.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:45 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:46.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:45 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:46.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:45 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:47.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:46 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:47.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:46 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:47.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:46 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:39:48.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:47 smithi053 ceph-mon[36496]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 60 MiB/s wr, 13.21k op/s 2024-05-21T15:39:48.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:47 smithi045 ceph-mon[36409]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 60 MiB/s wr, 13.21k op/s 2024-05-21T15:39:48.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:47 smithi038 ceph-mon[31995]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 60 MiB/s wr, 13.21k op/s 2024-05-21T15:39:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.861+0000 7f2c06faa640 1 -- 172.21.15.38:0/484068864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0010aac0 msgr2=0x7f2c0010cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.861+0000 7f2c06faa640 1 --2- 172.21.15.38:0/484068864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0010aac0 0x7f2c0010cee0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009ca0 tx=0x7f2bf802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.861+0000 7f2c06faa640 1 -- 172.21.15.38:0/484068864 shutdown_connections 2024-05-21T15:39:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.861+0000 7f2c06faa640 1 --2- 172.21.15.38:0/484068864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0010aac0 0x7f2c0010cee0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.861+0000 7f2c06faa640 1 --2- 172.21.15.38:0/484068864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c00074830 0x7f2c0010a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.861+0000 7f2c06faa640 1 --2- 172.21.15.38:0/484068864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c00073ee0 0x7f2c000742f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.861+0000 7f2c06faa640 1 -- 172.21.15.38:0/484068864 >> 172.21.15.38:0/484068864 conn(0x7f2c000fdce0 msgr2=0x7f2c00100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.861+0000 7f2c06faa640 1 -- 172.21.15.38:0/484068864 shutdown_connections 2024-05-21T15:39:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.861+0000 7f2c06faa640 1 -- 172.21.15.38:0/484068864 wait complete. 2024-05-21T15:39:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.861+0000 7f2c06faa640 1 Processor -- start 2024-05-21T15:39:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.861+0000 7f2c06faa640 1 -- start start 2024-05-21T15:39:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2c06faa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c00073ee0 0x7f2c0019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2c06faa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c00074830 0x7f2c0019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2c06faa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0010aac0 0x7f2c001a2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2c06faa640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00110940 con 0x7f2c0010aac0 2024-05-21T15:39:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2c06faa640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001107c0 con 0x7f2c00073ee0 2024-05-21T15:39:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2c06faa640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00110ac0 con 0x7f2c00074830 2024-05-21T15:39:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2c05547640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0010aac0 0x7f2c001a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2bfffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c00074830 0x7f2c0019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2c05547640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0010aac0 0x7f2c001a2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56488/0 (socket says 172.21.15.38:56488) 2024-05-21T15:39:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2c05547640 1 -- 172.21.15.38:0/3083986400 learned_addr learned my addr 172.21.15.38:0/3083986400 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2c04d46640 1 --2- 172.21.15.38:0/3083986400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c00073ee0 0x7f2c0019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2bfffff640 1 -- 172.21.15.38:0/3083986400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c00073ee0 msgr2=0x7f2c0019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2bfffff640 1 --2- 172.21.15.38:0/3083986400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c00073ee0 0x7f2c0019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2bfffff640 1 -- 172.21.15.38:0/3083986400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0010aac0 msgr2=0x7f2c001a2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2bfffff640 1 --2- 172.21.15.38:0/3083986400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0010aac0 0x7f2c001a2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2bfffff640 1 -- 172.21.15.38:0/3083986400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c001a2ea0 con 0x7f2c00074830 2024-05-21T15:39:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.862+0000 7f2c05547640 1 --2- 172.21.15.38:0/3083986400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0010aac0 0x7f2c001a2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.863+0000 7f2bfffff640 1 --2- 172.21.15.38:0/3083986400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c00074830 0x7f2c0019b8a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00e9f0 tx=0x7f2bec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.863+0000 7f2bfdffb640 1 -- 172.21.15.38:0/3083986400 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec00ccc0 con 0x7f2c00074830 2024-05-21T15:39:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.863+0000 7f2c06faa640 1 -- 172.21.15.38:0/3083986400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c001a3190 con 0x7f2c00074830 2024-05-21T15:39:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.863+0000 7f2bfdffb640 1 -- 172.21.15.38:0/3083986400 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bec010910 con 0x7f2c00074830 2024-05-21T15:39:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.863+0000 7f2bfdffb640 1 -- 172.21.15.38:0/3083986400 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec018960 con 0x7f2c00074830 2024-05-21T15:39:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.863+0000 7f2c06faa640 1 -- 172.21.15.38:0/3083986400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c001a36e0 con 0x7f2c00074830 2024-05-21T15:39:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.865+0000 7f2bfdffb640 1 -- 172.21.15.38:0/3083986400 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2bec010ac0 con 0x7f2c00074830 2024-05-21T15:39:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.866+0000 7f2bfdffb640 1 --2- 172.21.15.38:0/3083986400 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2bf00a2c30 0x7f2bf00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.866+0000 7f2c0de52740 1 Processor -- start 2024-05-21T15:39:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.866+0000 7f2c0de52740 1 -- start start 2024-05-21T15:39:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.866+0000 7f2bfdffb640 1 -- 172.21.15.38:0/3083986400 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2bec0cf480 con 0x7f2c00074830 2024-05-21T15:39:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.866+0000 7f2c04d46640 1 --2- 172.21.15.38:0/3083986400 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2bf00a2c30 0x7f2bf00a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.866+0000 7f2c0de52740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351c021b0 0x55c351fa1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.866+0000 7f2c0de52740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351bd92f0 0x55c351fa23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.866+0000 7f2c0de52740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351be14d0 0x55c351fa92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.866+0000 7f2c0de52740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c351f06340 con 0x55c351be14d0 2024-05-21T15:39:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.866+0000 7f2c0de52740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c351eae290 con 0x55c351c021b0 2024-05-21T15:39:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.866+0000 7f2c0de52740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c351f02300 con 0x55c351bd92f0 2024-05-21T15:39:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.867+0000 7f2c05547640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351be14d0 0x55c351fa92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.867+0000 7f2c05547640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351be14d0 0x55c351fa92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56494/0 (socket says 172.21.15.38:56494) 2024-05-21T15:39:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.867+0000 7f2c05547640 1 -- 172.21.15.38:0/989081832 learned_addr learned my addr 172.21.15.38:0/989081832 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.867+0000 7f2c04d46640 1 --2- 172.21.15.38:0/989081832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351c021b0 0x55c351fa1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.867+0000 7f2bfffff640 1 --2- 172.21.15.38:0/989081832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351bd92f0 0x55c351fa23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.867+0000 7f2c04d46640 1 --2- 172.21.15.38:0/3083986400 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2bf00a2c30 0x7f2bf00a5140 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2c00075710 tx=0x7f2bf4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.867+0000 7f2c05547640 1 -- 172.21.15.38:0/989081832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351bd92f0 msgr2=0x55c351fa23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.867+0000 7f2c05547640 1 --2- 172.21.15.38:0/989081832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351bd92f0 0x55c351fa23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.867+0000 7f2c05547640 1 -- 172.21.15.38:0/989081832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351c021b0 msgr2=0x55c351fa1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.867+0000 7f2c05547640 1 --2- 172.21.15.38:0/989081832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351c021b0 0x55c351fa1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.867+0000 7f2c05547640 1 -- 172.21.15.38:0/989081832 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c351ed28e0 con 0x55c351be14d0 2024-05-21T15:39:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.867+0000 7f2bfffff640 1 --2- 172.21.15.38:0/989081832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351bd92f0 0x55c351fa23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.867+0000 7f2c05547640 1 --2- 172.21.15.38:0/989081832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351be14d0 0x55c351fa92e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8004af0 tx=0x7f2bf802f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8b4bb47403a9be server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.867+0000 7f2beaffd640 1 -- 172.21.15.38:0/989081832 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8049070 con 0x55c351be14d0 2024-05-21T15:39:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.868+0000 7f2beaffd640 1 -- 172.21.15.38:0/989081832 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf8036040 con 0x55c351be14d0 2024-05-21T15:39:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.868+0000 7f2beaffd640 1 -- 172.21.15.38:0/989081832 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80305c0 con 0x55c351be14d0 2024-05-21T15:39:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.868+0000 7f2c0de52740 1 -- 172.21.15.38:0/989081832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351be14d0 msgr2=0x55c351fa92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.868+0000 7f2c0de52740 1 --2- 172.21.15.38:0/989081832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351be14d0 0x55c351fa92e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8004af0 tx=0x7f2bf802f310 comp rx=0 tx=0).stop 2024-05-21T15:39:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.868+0000 7f2c0de52740 1 -- 172.21.15.38:0/989081832 shutdown_connections 2024-05-21T15:39:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.868+0000 7f2c0de52740 1 --2- 172.21.15.38:0/989081832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351be14d0 0x55c351fa92e0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.868+0000 7f2c0de52740 1 --2- 172.21.15.38:0/989081832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351bd92f0 0x55c351fa23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.868+0000 7f2c0de52740 1 --2- 172.21.15.38:0/989081832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351c021b0 0x55c351fa1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.868+0000 7f2c0de52740 1 -- 172.21.15.38:0/989081832 >> 172.21.15.38:0/989081832 conn(0x55c351bf6300 msgr2=0x55c351f9d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.868+0000 7f2c0de52740 1 -- 172.21.15.38:0/989081832 shutdown_connections 2024-05-21T15:39:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.868+0000 7f2c0de52740 1 -- 172.21.15.38:0/989081832 wait complete. 2024-05-21T15:39:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.869+0000 7f2c0de52740 1 -- start start 2024-05-21T15:39:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.869+0000 7f2c0de52740 1 Processor -- start 2024-05-21T15:39:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.869+0000 7f2c0de52740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351bd92f0 0x55c351fab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.869+0000 7f2c0de52740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351be14d0 0x55c351fabef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.869+0000 7f2c0de52740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351c021b0 0x55c351fb2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.869+0000 7f2c0de52740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c351fa9b50 con 0x55c351c021b0 2024-05-21T15:39:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.869+0000 7f2c0de52740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c351ec1790 con 0x55c351be14d0 2024-05-21T15:39:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.869+0000 7f2c0de52740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c351fa9cd0 con 0x55c351bd92f0 2024-05-21T15:39:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.869+0000 7f2c0de52740 10 client.0 _ll_register_callbacks cb 0x55c351efca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.869+0000 7f2c05547640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351c021b0 0x55c351fb2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.869+0000 7f2c05547640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351c021b0 0x55c351fb2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56500/0 (socket says 172.21.15.38:56500) 2024-05-21T15:39:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.869+0000 7f2bfffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351be14d0 0x55c351fabef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.869+0000 7f2c05547640 1 -- 172.21.15.38:0/3414517048 learned_addr learned my addr 172.21.15.38:0/3414517048 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.869+0000 7f2c04d46640 1 --2- 172.21.15.38:0/3414517048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351bd92f0 0x55c351fab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2c05547640 1 -- 172.21.15.38:0/3414517048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351bd92f0 msgr2=0x55c351fab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2c05547640 1 --2- 172.21.15.38:0/3414517048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351bd92f0 0x55c351fab9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2c05547640 1 -- 172.21.15.38:0/3414517048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351be14d0 msgr2=0x55c351fabef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2c05547640 1 --2- 172.21.15.38:0/3414517048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351be14d0 0x55c351fabef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2c05547640 1 -- 172.21.15.38:0/3414517048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c351c1a210 con 0x55c351c021b0 2024-05-21T15:39:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2c04d46640 1 --2- 172.21.15.38:0/3414517048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351bd92f0 0x55c351fab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:39:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2c05547640 1 --2- 172.21.15.38:0/3414517048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351c021b0 0x55c351fb2e50 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2bf802f380 tx=0x7f2bf8009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83682e644bdce73d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2bdffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2bdffff640 1 -- 172.21.15.38:0/3414517048 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8049070 con 0x55c351c021b0 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2bdffff640 1 -- 172.21.15.38:0/3414517048 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2bf802f820 con 0x55c351c021b0 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2c0de52740 1 -- 172.21.15.38:0/3414517048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bf8036040 con 0x55c351c021b0 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2bdffff640 1 -- 172.21.15.38:0/3414517048 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2bf8031e60 con 0x55c351c021b0 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2c0de52740 10 client.16914 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2c0de52740 1 -- 172.21.15.38:0/3414517048 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c351ce6620 con 0x55c351c021b0 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.870+0000 7f2bdffff640 1 -- 172.21.15.38:0/3414517048 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf8036040 con 0x55c351c021b0 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.871+0000 7f2bdffff640 1 -- 172.21.15.38:0/3414517048 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f2bf8044590 con 0x55c351c021b0 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.871+0000 7f2c0de52740 10 client.16914 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.871+0000 7f2c0de52740 10 client.16914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.871+0000 7f2c0de52740 20 client.16914 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.871+0000 7f2c0de52740 1 --2- 172.21.15.38:0/3414517048 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c351bc0d80 0x55c351fb90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.871+0000 7f2c0de52740 4 client.16914 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:39:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.871+0000 7f2c0de52740 1 -- 172.21.15.38:0/3414517048 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c351f13a70 con 0x55c351bc0d80 2024-05-21T15:39:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.871+0000 7f2c04d46640 1 --2- 172.21.15.38:0/3414517048 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c351bc0d80 0x55c351fb90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.871+0000 7f2bdffff640 10 client.16914 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.871+0000 7f2c04d46640 1 --2- 172.21.15.38:0/3414517048 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c351bc0d80 0x55c351fb90e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d189cfa4971a9a79 server_cookie=95c9067f2a71848b in_seq=0 out_seq=0 2024-05-21T15:39:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.872+0000 7f2bdffff640 1 -- 172.21.15.38:0/3414517048 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c351f13a70 con 0x55c351bc0d80 2024-05-21T15:39:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.872+0000 7f2bdffff640 10 client.16914 handle_command_reply: tid=0 2024-05-21T15:39:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.873+0000 7f2c0de52740 1 client.16914 shutdown 2024-05-21T15:39:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.873+0000 7f2c0de52740 1 -- 172.21.15.38:0/3414517048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351c021b0 msgr2=0x55c351fb2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.873+0000 7f2c0de52740 1 --2- 172.21.15.38:0/3414517048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351c021b0 0x55c351fb2e50 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2bf802f380 tx=0x7f2bf8009360 comp rx=0 tx=0).stop 2024-05-21T15:39:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.873+0000 7f2c0de52740 1 -- 172.21.15.38:0/3414517048 shutdown_connections 2024-05-21T15:39:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.873+0000 7f2c0de52740 1 --2- 172.21.15.38:0/3414517048 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c351bc0d80 0x55c351fb90e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.873+0000 7f2bdffff640 0 client.16914 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.873+0000 7f2c0de52740 1 --2- 172.21.15.38:0/3414517048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351c021b0 0x55c351fb2e50 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.873+0000 7f2c0de52740 1 --2- 172.21.15.38:0/3414517048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351be14d0 0x55c351fabef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.873+0000 7f2c0de52740 1 --2- 172.21.15.38:0/3414517048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351bd92f0 0x55c351fab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.873+0000 7f2c0de52740 1 -- 172.21.15.38:0/3414517048 >> 172.21.15.38:0/3414517048 conn(0x55c351bf6300 msgr2=0x55c351f9c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.873+0000 7f2c0de52740 1 -- 172.21.15.38:0/3414517048 shutdown_connections 2024-05-21T15:39:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.873+0000 7f2c0de52740 1 -- 172.21.15.38:0/3414517048 wait complete. 2024-05-21T15:39:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.873+0000 7f2c0de52740 20 client.16914 trim_cache size 0 max 16384 2024-05-21T15:39:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.890+0000 7f2c0de52740 1 Processor -- start 2024-05-21T15:39:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.890+0000 7f2c0de52740 1 -- start start 2024-05-21T15:39:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2c0de52740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351bc0d80 0x55c351f9d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2c0de52740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351bd92f0 0x55c351fab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2c0de52740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351be14d0 0x55c351fabef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2c0de52740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001ac470 con 0x55c351bd92f0 2024-05-21T15:39:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2c0de52740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001ae0a0 con 0x55c351bc0d80 2024-05-21T15:39:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2c0de52740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001acb40 con 0x55c351be14d0 2024-05-21T15:39:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2bfffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351bd92f0 0x55c351fab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2bfffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351bd92f0 0x55c351fab9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56514/0 (socket says 172.21.15.38:56514) 2024-05-21T15:39:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2bfffff640 1 -- 172.21.15.38:0/1992129902 learned_addr learned my addr 172.21.15.38:0/1992129902 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2c04d46640 1 --2- 172.21.15.38:0/1992129902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351bc0d80 0x55c351f9d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2c05547640 1 --2- 172.21.15.38:0/1992129902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351be14d0 0x55c351fabef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2bfffff640 1 -- 172.21.15.38:0/1992129902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351be14d0 msgr2=0x55c351fabef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2bfffff640 1 --2- 172.21.15.38:0/1992129902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351be14d0 0x55c351fabef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2bfffff640 1 -- 172.21.15.38:0/1992129902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351bc0d80 msgr2=0x55c351f9d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2bfffff640 1 --2- 172.21.15.38:0/1992129902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351bc0d80 0x55c351f9d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2bfffff640 1 -- 172.21.15.38:0/1992129902 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c351fa2010 con 0x55c351bd92f0 2024-05-21T15:39:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2c05547640 1 --2- 172.21.15.38:0/1992129902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351be14d0 0x55c351fabef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:39:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2bfffff640 1 --2- 172.21.15.38:0/1992129902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351bd92f0 0x55c351fab9b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2bec098140 tx=0x7f2bec00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab005a121c33fbcc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.891+0000 7f2bea7fc640 1 -- 172.21.15.38:0/1992129902 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec0b9070 con 0x55c351bd92f0 2024-05-21T15:39:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.892+0000 7f2bea7fc640 1 -- 172.21.15.38:0/1992129902 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bec0ccfe0 con 0x55c351bd92f0 2024-05-21T15:39:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.892+0000 7f2bea7fc640 1 -- 172.21.15.38:0/1992129902 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec018d60 con 0x55c351bd92f0 2024-05-21T15:39:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.893+0000 7f2c0de52740 1 -- 172.21.15.38:0/1992129902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351bd92f0 msgr2=0x55c351fab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.893+0000 7f2c0de52740 1 --2- 172.21.15.38:0/1992129902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351bd92f0 0x55c351fab9b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2bec098140 tx=0x7f2bec00cf60 comp rx=0 tx=0).stop 2024-05-21T15:39:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.893+0000 7f2c0de52740 1 -- 172.21.15.38:0/1992129902 shutdown_connections 2024-05-21T15:39:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.893+0000 7f2c0de52740 1 --2- 172.21.15.38:0/1992129902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351be14d0 0x55c351fabef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.893+0000 7f2c0de52740 1 --2- 172.21.15.38:0/1992129902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351bd92f0 0x55c351fab9b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.893+0000 7f2c0de52740 1 --2- 172.21.15.38:0/1992129902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351bc0d80 0x55c351f9d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.893+0000 7f2c0de52740 1 -- 172.21.15.38:0/1992129902 >> 172.21.15.38:0/1992129902 conn(0x55c351bf6300 msgr2=0x55c351f9d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.893+0000 7f2c0de52740 1 -- 172.21.15.38:0/1992129902 shutdown_connections 2024-05-21T15:39:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.893+0000 7f2c0de52740 1 -- 172.21.15.38:0/1992129902 wait complete. 2024-05-21T15:39:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.893+0000 7f2c0de52740 1 -- start start 2024-05-21T15:39:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.893+0000 7f2c0de52740 1 Processor -- start 2024-05-21T15:39:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c0de52740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351bc0d80 0x55c351fafae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c0de52740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351bd92f0 0x55c351fb0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c0de52740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351be14d0 0x55c351fb0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c0de52740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001ad910 con 0x55c351bc0d80 2024-05-21T15:39:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c0de52740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001ad210 con 0x55c351bd92f0 2024-05-21T15:39:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c0de52740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001aa450 con 0x55c351be14d0 2024-05-21T15:39:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c0de52740 10 client.0 _ll_register_callbacks cb 0x7f2bf8047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:39:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c04d46640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351bc0d80 0x55c351fafae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c04d46640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351bc0d80 0x55c351fafae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56524/0 (socket says 172.21.15.38:56524) 2024-05-21T15:39:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c04d46640 1 -- 172.21.15.38:0/1882474419 learned_addr learned my addr 172.21.15.38:0/1882474419 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:39:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c05547640 1 --2- 172.21.15.38:0/1882474419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351be14d0 0x55c351fb0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2bfffff640 1 --2- 172.21.15.38:0/1882474419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351bd92f0 0x55c351fb0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c04d46640 1 -- 172.21.15.38:0/1882474419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351be14d0 msgr2=0x55c351fb0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c04d46640 1 --2- 172.21.15.38:0/1882474419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351be14d0 0x55c351fb0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c04d46640 1 -- 172.21.15.38:0/1882474419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351bd92f0 msgr2=0x55c351fb0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:39:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c04d46640 1 --2- 172.21.15.38:0/1882474419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351bd92f0 0x55c351fb0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c04d46640 1 -- 172.21.15.38:0/1882474419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c351fb1510 con 0x55c351bc0d80 2024-05-21T15:39:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c04d46640 1 --2- 172.21.15.38:0/1882474419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351bc0d80 0x55c351fafae0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x55c351fb9840 tx=0x7f2bf4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ca8618392d3cc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:39:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2bdffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:39:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2bdffff640 1 -- 172.21.15.38:0/1882474419 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4005300 con 0x55c351bc0d80 2024-05-21T15:39:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2bdffff640 1 -- 172.21.15.38:0/1882474419 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2bf40054a0 con 0x55c351bc0d80 2024-05-21T15:39:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.894+0000 7f2c0de52740 1 -- 172.21.15.38:0/1882474419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bec0ccfe0 con 0x55c351bc0d80 2024-05-21T15:39:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.895+0000 7f2bdffff640 1 -- 172.21.15.38:0/1882474419 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2bf40056e0 con 0x55c351bc0d80 2024-05-21T15:39:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.895+0000 7f2c0de52740 10 client.16920 fetch_fsmap learned FSMap version 31 2024-05-21T15:39:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.895+0000 7f2c0de52740 1 -- 172.21.15.38:0/1882474419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c351fb27e0 con 0x55c351bc0d80 2024-05-21T15:39:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.895+0000 7f2bdffff640 1 -- 172.21.15.38:0/1882474419 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bec0ccfe0 con 0x55c351bc0d80 2024-05-21T15:39:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.895+0000 7f2bdffff640 1 -- 172.21.15.38:0/1882474419 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f2bf4032020 con 0x55c351bc0d80 2024-05-21T15:39:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.895+0000 7f2c0de52740 10 client.16920 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:39:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.895+0000 7f2c0de52740 10 client.16920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:39:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.895+0000 7f2c0de52740 20 client.16920 populate_metadata read hostname 'smithi038' 2024-05-21T15:39:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.895+0000 7f2c0de52740 1 --2- 172.21.15.38:0/1882474419 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c351c021b0 0x55c352023c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:39:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.895+0000 7f2c0de52740 4 client.16920 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:39:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.895+0000 7f2c0de52740 1 -- 172.21.15.38:0/1882474419 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c351f0ddc0 con 0x55c351c021b0 2024-05-21T15:39:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.895+0000 7f2bfffff640 1 --2- 172.21.15.38:0/1882474419 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c351c021b0 0x55c352023c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:39:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.896+0000 7f2bdffff640 10 client.16920 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:48.896+0000 7f2bfffff640 1 --2- 172.21.15.38:0/1882474419 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c351c021b0 0x55c352023c90 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef2593055b1ea81 server_cookie=993d9916d15b7e in_seq=0 out_seq=0 2024-05-21T15:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.146+0000 7f2bdffff640 1 -- 172.21.15.38:0/1882474419 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c351f0ddc0 con 0x55c351c021b0 2024-05-21T15:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.147+0000 7f2bdffff640 10 client.16920 handle_command_reply: tid=0 2024-05-21T15:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.147+0000 7f2c0de52740 1 client.16920 shutdown 2024-05-21T15:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.147+0000 7f2c0de52740 1 -- 172.21.15.38:0/1882474419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351bc0d80 msgr2=0x55c351fafae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.147+0000 7f2c0de52740 1 --2- 172.21.15.38:0/1882474419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351bc0d80 0x55c351fafae0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x55c351fb9840 tx=0x7f2bf4004730 comp rx=0 tx=0).stop 2024-05-21T15:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.147+0000 7f2c0de52740 1 -- 172.21.15.38:0/1882474419 shutdown_connections 2024-05-21T15:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.147+0000 7f2c0de52740 1 --2- 172.21.15.38:0/1882474419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c351be14d0 0x55c351fb0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.147+0000 7f2c0de52740 1 --2- 172.21.15.38:0/1882474419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c351bd92f0 0x55c351fb0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.147+0000 7f2c0de52740 1 --2- 172.21.15.38:0/1882474419 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c351c021b0 0x55c352023c90 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.147+0000 7f2c0de52740 1 --2- 172.21.15.38:0/1882474419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c351bc0d80 0x55c351fafae0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.147+0000 7f2bdffff640 0 client.16920 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.147+0000 7f2c0de52740 1 -- 172.21.15.38:0/1882474419 >> 172.21.15.38:0/1882474419 conn(0x55c351bf6300 msgr2=0x55c351fab990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.147+0000 7f2c0de52740 1 -- 172.21.15.38:0/1882474419 shutdown_connections 2024-05-21T15:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.147+0000 7f2c0de52740 1 -- 172.21.15.38:0/1882474419 wait complete. 2024-05-21T15:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.147+0000 7f2c0de52740 20 client.16920 trim_cache size 0 max 16384 2024-05-21T15:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.148+0000 7f2c06faa640 1 -- 172.21.15.38:0/3083986400 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2bf00a2c30 msgr2=0x7f2bf00a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.148+0000 7f2c06faa640 1 --2- 172.21.15.38:0/3083986400 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2bf00a2c30 0x7f2bf00a5140 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2c00075710 tx=0x7f2bf4005f90 comp rx=0 tx=0).stop 2024-05-21T15:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.148+0000 7f2c06faa640 1 -- 172.21.15.38:0/3083986400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c00074830 msgr2=0x7f2c0019b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.148+0000 7f2c06faa640 1 --2- 172.21.15.38:0/3083986400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c00074830 0x7f2c0019b8a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00e9f0 tx=0x7f2bec00eeb0 comp rx=0 tx=0).stop 2024-05-21T15:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.148+0000 7f2c06faa640 1 -- 172.21.15.38:0/3083986400 shutdown_connections 2024-05-21T15:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.148+0000 7f2c06faa640 1 --2- 172.21.15.38:0/3083986400 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2bf00a2c30 0x7f2bf00a5140 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.148+0000 7f2c06faa640 1 --2- 172.21.15.38:0/3083986400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c0010aac0 0x7f2c001a2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.148+0000 7f2c06faa640 1 --2- 172.21.15.38:0/3083986400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c00074830 0x7f2c0019b8a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.148+0000 7f2c06faa640 1 --2- 172.21.15.38:0/3083986400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c00073ee0 0x7f2c0019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.148+0000 7f2c06faa640 1 -- 172.21.15.38:0/3083986400 >> 172.21.15.38:0/3083986400 conn(0x7f2c000fdce0 msgr2=0x7f2c00106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.148+0000 7f2c06faa640 1 -- 172.21.15.38:0/3083986400 shutdown_connections 2024-05-21T15:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:39:49.148+0000 7f2c06faa640 1 -- 172.21.15.38:0/3083986400 wait complete. 2024-05-21T15:39:50.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:49 smithi053 ceph-mon[36496]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 55 MiB/s wr, 12.19k op/s 2024-05-21T15:39:50.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:49 smithi045 ceph-mon[36409]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 55 MiB/s wr, 12.19k op/s 2024-05-21T15:39:50.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:49 smithi038 ceph-mon[31995]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 55 MiB/s wr, 12.19k op/s 2024-05-21T15:39:52.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:51 smithi053 ceph-mon[36496]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 MiB/s wr, 14.23k op/s 2024-05-21T15:39:52.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:51 smithi045 ceph-mon[36409]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 MiB/s wr, 14.23k op/s 2024-05-21T15:39:52.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:51 smithi038 ceph-mon[31995]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 MiB/s wr, 14.23k op/s 2024-05-21T15:39:54.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:53 smithi053 ceph-mon[36496]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 50 MiB/s wr, 11.05k op/s 2024-05-21T15:39:54.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:53 smithi045 ceph-mon[36409]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 50 MiB/s wr, 11.05k op/s 2024-05-21T15:39:54.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:53 smithi038 ceph-mon[31995]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 50 MiB/s wr, 11.05k op/s 2024-05-21T15:39:55.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:54 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:39:55.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:54 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:39:55.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:54 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:39:55.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:54 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:39:55.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:54 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:39:55.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:54 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:39:55.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:54 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:39:55.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:54 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:39:55.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:54 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:39:55.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:54 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:39:55.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:54 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:39:55.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:54 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:39:55.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:54 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:39:55.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:54 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:39:55.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:54 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:39:56.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:55 smithi053 ceph-mon[36496]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 52 MiB/s wr, 11.66k op/s 2024-05-21T15:39:56.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:55 smithi045 ceph-mon[36409]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 52 MiB/s wr, 11.66k op/s 2024-05-21T15:39:56.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:55 smithi038 ceph-mon[31995]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 52 MiB/s wr, 11.66k op/s 2024-05-21T15:39:58.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:57 smithi053 ceph-mon[36496]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 33 MiB/s wr, 7.57k op/s 2024-05-21T15:39:58.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:57 smithi045 ceph-mon[36409]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 33 MiB/s wr, 7.57k op/s 2024-05-21T15:39:58.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:57 smithi038 ceph-mon[31995]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 33 MiB/s wr, 7.57k op/s 2024-05-21T15:40:00.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:39:59 smithi053 ceph-mon[36496]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 36 MiB/s wr, 8.15k op/s 2024-05-21T15:40:00.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:39:59 smithi045 ceph-mon[36409]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 36 MiB/s wr, 8.15k op/s 2024-05-21T15:40:00.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:39:59 smithi038 ceph-mon[31995]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 36 MiB/s wr, 8.15k op/s 2024-05-21T15:40:01.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:00 smithi053 ceph-mon[36496]: overall HEALTH_OK 2024-05-21T15:40:01.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:00 smithi045 ceph-mon[36409]: overall HEALTH_OK 2024-05-21T15:40:01.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:00 smithi038 ceph-mon[31995]: overall HEALTH_OK 2024-05-21T15:40:02.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:01 smithi053 ceph-mon[36496]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 MiB/s wr, 9.25k op/s 2024-05-21T15:40:02.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:01 smithi045 ceph-mon[36409]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 MiB/s wr, 9.25k op/s 2024-05-21T15:40:02.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:01 smithi038 ceph-mon[31995]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 MiB/s wr, 9.25k op/s 2024-05-21T15:40:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.277+0000 7f029fbea640 1 -- 172.21.15.38:0/102828420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0298101310 msgr2=0x7f02981080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.277+0000 7f029fbea640 1 --2- 172.21.15.38:0/102828420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0298101310 0x7f02981080a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0288009860 tx=0x7f028802a340 comp rx=0 tx=0).stop 2024-05-21T15:40:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.277+0000 7f029fbea640 1 -- 172.21.15.38:0/102828420 shutdown_connections 2024-05-21T15:40:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.277+0000 7f029fbea640 1 --2- 172.21.15.38:0/102828420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0298108730 0x7f029810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.277+0000 7f029fbea640 1 --2- 172.21.15.38:0/102828420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0298101310 0x7f02981080a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.277+0000 7f029fbea640 1 --2- 172.21.15.38:0/102828420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0298100e10 0x7f0298105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.277+0000 7f029fbea640 1 -- 172.21.15.38:0/102828420 >> 172.21.15.38:0/102828420 conn(0x7f02980fc300 msgr2=0x7f02980fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.277+0000 7f029fbea640 1 -- 172.21.15.38:0/102828420 shutdown_connections 2024-05-21T15:40:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.277+0000 7f029fbea640 1 -- 172.21.15.38:0/102828420 wait complete. 2024-05-21T15:40:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.278+0000 7f029fbea640 1 Processor -- start 2024-05-21T15:40:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.278+0000 7f029fbea640 1 -- start start 2024-05-21T15:40:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.278+0000 7f029fbea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0298100e10 0x7f02981991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.278+0000 7f029fbea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0298101310 0x7f02981996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.278+0000 7f029fbea640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0298108730 0x7f02981a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.278+0000 7f029fbea640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f029810e6f0 con 0x7f0298100e10 2024-05-21T15:40:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.278+0000 7f029fbea640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f029810e570 con 0x7f0298101310 2024-05-21T15:40:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.278+0000 7f029fbea640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f029810e870 con 0x7f0298108730 2024-05-21T15:40:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.279+0000 7f029e187640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0298108730 0x7f02981a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.279+0000 7f029e187640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0298108730 0x7f02981a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:40878/0 (socket says 172.21.15.38:40878) 2024-05-21T15:40:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.279+0000 7f029e187640 1 -- 172.21.15.38:0/4109629912 learned_addr learned my addr 172.21.15.38:0/4109629912 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.279+0000 7f029e187640 1 -- 172.21.15.38:0/4109629912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0298101310 msgr2=0x7f02981996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.279+0000 7f029d986640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0298100e10 0x7f02981991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.279+0000 7f029d185640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0298101310 0x7f02981996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.279+0000 7f029e187640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0298101310 0x7f02981996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.279+0000 7f029e187640 1 -- 172.21.15.38:0/4109629912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0298100e10 msgr2=0x7f02981991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.279+0000 7f029e187640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0298100e10 0x7f02981991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.279+0000 7f029e187640 1 -- 172.21.15.38:0/4109629912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0298101990 con 0x7f0298108730 2024-05-21T15:40:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.279+0000 7f029d986640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0298100e10 0x7f02981991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:40:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.280+0000 7f029e187640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0298108730 0x7f02981a0670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0290009fd0 tx=0x7f029000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.280+0000 7f028effd640 1 -- 172.21.15.38:0/4109629912 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0290004b20 con 0x7f0298108730 2024-05-21T15:40:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.280+0000 7f028effd640 1 -- 172.21.15.38:0/4109629912 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0290003e50 con 0x7f0298108730 2024-05-21T15:40:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.280+0000 7f028effd640 1 -- 172.21.15.38:0/4109629912 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029000b630 con 0x7f0298108730 2024-05-21T15:40:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.280+0000 7f029fbea640 1 -- 172.21.15.38:0/4109629912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0298101c80 con 0x7f0298108730 2024-05-21T15:40:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.280+0000 7f029fbea640 1 -- 172.21.15.38:0/4109629912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0298102180 con 0x7f0298108730 2024-05-21T15:40:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.284+0000 7f028effd640 1 -- 172.21.15.38:0/4109629912 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f029001d020 con 0x7f0298108730 2024-05-21T15:40:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.284+0000 7f02a6b6d740 1 Processor -- start 2024-05-21T15:40:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.284+0000 7f02a6b6d740 1 -- start start 2024-05-21T15:40:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.285+0000 7f02a6b6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010b11b0 0x560e01450e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.285+0000 7f02a6b6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e014513e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.285+0000 7f029d185640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010b11b0 0x560e01450e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.285+0000 7f029d185640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010b11b0 0x560e01450e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50066/0 (socket says 172.21.15.38:50066) 2024-05-21T15:40:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.285+0000 7f029d185640 1 -- 172.21.15.38:0/516631146 learned_addr learned my addr 172.21.15.38:0/516631146 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.285+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/516631146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010904d0 0x560e014582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.286+0000 7f02a6b6d740 1 -- 172.21.15.38:0/516631146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560e013b5340 con 0x560e010b11b0 2024-05-21T15:40:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.286+0000 7f02a6b6d740 1 -- 172.21.15.38:0/516631146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560e0135d290 con 0x560e010882f0 2024-05-21T15:40:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.286+0000 7f02a6b6d740 1 -- 172.21.15.38:0/516631146 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560e013b1300 con 0x560e010904d0 2024-05-21T15:40:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.286+0000 7f029d986640 1 --2- 172.21.15.38:0/516631146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e014513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.286+0000 7f028effd640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f02840a2cb0 0x7f02840a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.286+0000 7f029d185640 1 --2- 172.21.15.38:0/516631146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010904d0 0x560e014582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.286+0000 7f029e187640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f02840a2cb0 0x7f02840a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.286+0000 7f029d986640 1 -- 172.21.15.38:0/516631146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010904d0 msgr2=0x560e014582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.286+0000 7f029d986640 1 --2- 172.21.15.38:0/516631146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010904d0 0x560e014582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.286+0000 7f029d986640 1 -- 172.21.15.38:0/516631146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010b11b0 msgr2=0x560e01450e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.286+0000 7f029d986640 1 --2- 172.21.15.38:0/516631146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010b11b0 0x560e01450e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.286+0000 7f029d986640 1 -- 172.21.15.38:0/516631146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e013818e0 con 0x560e010882f0 2024-05-21T15:40:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.286+0000 7f028effd640 1 -- 172.21.15.38:0/4109629912 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f02900cf130 con 0x7f0298108730 2024-05-21T15:40:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.286+0000 7f029d185640 1 --2- 172.21.15.38:0/516631146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010904d0 0x560e014582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:40:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.287+0000 7f029e187640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f02840a2cb0 0x7f02840a51c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f02900982b0 tx=0x7f029001c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.287+0000 7f029d986640 1 --2- 172.21.15.38:0/516631146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e014513e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0294009b40 tx=0x7f0294010370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77be7d979756975d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.288+0000 7f0283fff640 1 -- 172.21.15.38:0/516631146 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294010da0 con 0x560e010882f0 2024-05-21T15:40:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.288+0000 7f0283fff640 1 -- 172.21.15.38:0/516631146 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0294005500 con 0x560e010882f0 2024-05-21T15:40:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.288+0000 7f0283fff640 1 -- 172.21.15.38:0/516631146 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02940126b0 con 0x560e010882f0 2024-05-21T15:40:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.289+0000 7f02a6b6d740 1 -- 172.21.15.38:0/516631146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 msgr2=0x560e014513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.289+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/516631146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e014513e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0294009b40 tx=0x7f0294010370 comp rx=0 tx=0).stop 2024-05-21T15:40:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.289+0000 7f02a6b6d740 1 -- 172.21.15.38:0/516631146 shutdown_connections 2024-05-21T15:40:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.289+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/516631146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010904d0 0x560e014582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.289+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/516631146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e014513e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.289+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/516631146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010b11b0 0x560e01450e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.289+0000 7f02a6b6d740 1 -- 172.21.15.38:0/516631146 >> 172.21.15.38:0/516631146 conn(0x560e010a5300 msgr2=0x560e0144c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.289+0000 7f02a6b6d740 1 -- 172.21.15.38:0/516631146 shutdown_connections 2024-05-21T15:40:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.289+0000 7f02a6b6d740 1 -- 172.21.15.38:0/516631146 wait complete. 2024-05-21T15:40:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.290+0000 7f02a6b6d740 1 -- start start 2024-05-21T15:40:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.290+0000 7f02a6b6d740 1 Processor -- start 2024-05-21T15:40:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.290+0000 7f02a6b6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010882f0 0x560e0145ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.290+0000 7f02a6b6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010904d0 0x560e0145b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.290+0000 7f02a6b6d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010b11b0 0x560e01462110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.290+0000 7f02a6b6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560e01458d80 con 0x560e010882f0 2024-05-21T15:40:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.290+0000 7f02a6b6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560e01370790 con 0x560e010904d0 2024-05-21T15:40:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.290+0000 7f02a6b6d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560e01458f00 con 0x560e010b11b0 2024-05-21T15:40:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.290+0000 7f02a6b6d740 10 client.0 _ll_register_callbacks cb 0x560e013aba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.290+0000 7f029d185640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010882f0 0x560e0145ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.290+0000 7f029d185640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010882f0 0x560e0145ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50070/0 (socket says 172.21.15.38:50070) 2024-05-21T15:40:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.290+0000 7f029d185640 1 -- 172.21.15.38:0/38887951 learned_addr learned my addr 172.21.15.38:0/38887951 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.290+0000 7f029d986640 1 --2- 172.21.15.38:0/38887951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010904d0 0x560e0145b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.291+0000 7f029d185640 1 --2- 172.21.15.38:0/38887951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010b11b0 0x560e01462110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.291+0000 7f029d986640 1 -- 172.21.15.38:0/38887951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010b11b0 msgr2=0x560e01462110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.291+0000 7f029d986640 1 --2- 172.21.15.38:0/38887951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010b11b0 0x560e01462110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.291+0000 7f029d986640 1 -- 172.21.15.38:0/38887951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010882f0 msgr2=0x560e0145ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.291+0000 7f029d986640 1 --2- 172.21.15.38:0/38887951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010882f0 0x560e0145ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.291+0000 7f029d986640 1 -- 172.21.15.38:0/38887951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e010c9210 con 0x560e010904d0 2024-05-21T15:40:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.291+0000 7f029d185640 1 --2- 172.21.15.38:0/38887951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010882f0 0x560e0145ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.291+0000 7f029d986640 1 --2- 172.21.15.38:0/38887951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010904d0 0x560e0145b1b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0294000c00 tx=0x7f0294010e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79b595bf1a5a84d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.291+0000 7f02817fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:40:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.291+0000 7f02817fa640 1 -- 172.21.15.38:0/38887951 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294005100 con 0x560e010904d0 2024-05-21T15:40:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.291+0000 7f02a6b6d740 1 -- 172.21.15.38:0/38887951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0294005500 con 0x560e010904d0 2024-05-21T15:40:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.291+0000 7f02817fa640 1 -- 172.21.15.38:0/38887951 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f02940052a0 con 0x560e010904d0 2024-05-21T15:40:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.292+0000 7f02817fa640 1 -- 172.21.15.38:0/38887951 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0294004020 con 0x560e010904d0 2024-05-21T15:40:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.292+0000 7f02a6b6d740 10 client.26549 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.292+0000 7f02a6b6d740 1 -- 172.21.15.38:0/38887951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560e01195620 con 0x560e010904d0 2024-05-21T15:40:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.292+0000 7f02817fa640 1 -- 172.21.15.38:0/38887951 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0294005500 con 0x560e010904d0 2024-05-21T15:40:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.292+0000 7f02817fa640 1 -- 172.21.15.38:0/38887951 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f029400f040 con 0x560e010904d0 2024-05-21T15:40:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.292+0000 7f02a6b6d740 10 client.26549 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.292+0000 7f02a6b6d740 10 client.26549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.292+0000 7f02a6b6d740 20 client.26549 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.292+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/38887951 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e0106fd80 0x560e01467ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.292+0000 7f02a6b6d740 4 client.26549 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.292+0000 7f02a6b6d740 1 -- 172.21.15.38:0/38887951 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e013c2a70 con 0x560e0106fd80 2024-05-21T15:40:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.292+0000 7f029d185640 1 --2- 172.21.15.38:0/38887951 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e0106fd80 0x560e01467ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.293+0000 7f02817fa640 10 client.26549 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.293+0000 7f029d185640 1 --2- 172.21.15.38:0/38887951 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e0106fd80 0x560e01467ff0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=465d3573894cdca9 server_cookie=9076a330f7cf9ee4 in_seq=0 out_seq=0 2024-05-21T15:40:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.294+0000 7f02817fa640 1 -- 172.21.15.38:0/38887951 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e013c2a70 con 0x560e0106fd80 2024-05-21T15:40:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.294+0000 7f02817fa640 10 client.26549 handle_command_reply: tid=0 2024-05-21T15:40:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.294+0000 7f02a6b6d740 1 client.26549 shutdown 2024-05-21T15:40:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.294+0000 7f02a6b6d740 1 -- 172.21.15.38:0/38887951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010904d0 msgr2=0x560e0145b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.294+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/38887951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010904d0 0x560e0145b1b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0294000c00 tx=0x7f0294010e40 comp rx=0 tx=0).stop 2024-05-21T15:40:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.294+0000 7f02a6b6d740 1 -- 172.21.15.38:0/38887951 shutdown_connections 2024-05-21T15:40:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.294+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/38887951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010b11b0 0x560e01462110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.294+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/38887951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010904d0 0x560e0145b1b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.294+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/38887951 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e0106fd80 0x560e01467ff0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.294+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/38887951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010882f0 0x560e0145ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.294+0000 7f02817fa640 0 client.26549 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.294+0000 7f02a6b6d740 1 -- 172.21.15.38:0/38887951 >> 172.21.15.38:0/38887951 conn(0x560e010a5300 msgr2=0x560e0144f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.295+0000 7f02a6b6d740 1 -- 172.21.15.38:0/38887951 shutdown_connections 2024-05-21T15:40:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.295+0000 7f02a6b6d740 1 -- 172.21.15.38:0/38887951 wait complete. 2024-05-21T15:40:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.295+0000 7f02a6b6d740 20 client.26549 trim_cache size 0 max 16384 2024-05-21T15:40:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.316+0000 7f02a6b6d740 1 Processor -- start 2024-05-21T15:40:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.316+0000 7f02a6b6d740 1 -- start start 2024-05-21T15:40:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.316+0000 7f02a6b6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e014592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.316+0000 7f02a6b6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010904d0 0x560e01459810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.316+0000 7f02a6b6d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010b11b0 0x560e01455050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.316+0000 7f029d986640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010904d0 0x560e01459810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.316+0000 7f029d986640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010904d0 0x560e01459810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50074/0 (socket says 172.21.15.38:50074) 2024-05-21T15:40:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.316+0000 7f029d185640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e014592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.317+0000 7f029d185640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e014592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:35504/0 (socket says 172.21.15.38:35504) 2024-05-21T15:40:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.317+0000 7f029d986640 1 -- 172.21.15.38:0/1979959477 learned_addr learned my addr 172.21.15.38:0/1979959477 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.317+0000 7f02a6b6d740 1 -- 172.21.15.38:0/1979959477 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02981b0a80 con 0x560e010904d0 2024-05-21T15:40:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.317+0000 7f02a6b6d740 1 -- 172.21.15.38:0/1979959477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f02981b26b0 con 0x560e010882f0 2024-05-21T15:40:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.317+0000 7f02a6b6d740 1 -- 172.21.15.38:0/1979959477 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f02981b1150 con 0x560e010b11b0 2024-05-21T15:40:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.317+0000 7f029d185640 1 --2- 172.21.15.38:0/1979959477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010b11b0 0x560e01455050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.317+0000 7f029d185640 1 -- 172.21.15.38:0/1979959477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010b11b0 msgr2=0x560e01455050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.317+0000 7f029d185640 1 --2- 172.21.15.38:0/1979959477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010b11b0 0x560e01455050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.317+0000 7f029d185640 1 -- 172.21.15.38:0/1979959477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010904d0 msgr2=0x560e01459810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.317+0000 7f029d185640 1 --2- 172.21.15.38:0/1979959477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010904d0 0x560e01459810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.317+0000 7f029d185640 1 -- 172.21.15.38:0/1979959477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e01455590 con 0x560e010882f0 2024-05-21T15:40:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.317+0000 7f029d986640 1 --2- 172.21.15.38:0/1979959477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010904d0 0x560e01459810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.317+0000 7f029d185640 1 --2- 172.21.15.38:0/1979959477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e014592d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0288035040 tx=0x7f0288009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=923c343b9423ed3e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.318+0000 7f02837fe640 1 -- 172.21.15.38:0/1979959477 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028804a070 con 0x560e010882f0 2024-05-21T15:40:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.318+0000 7f02837fe640 1 -- 172.21.15.38:0/1979959477 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0288038040 con 0x560e010882f0 2024-05-21T15:40:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.318+0000 7f02837fe640 1 -- 172.21.15.38:0/1979959477 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288037040 con 0x560e010882f0 2024-05-21T15:40:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.319+0000 7f02a6b6d740 1 -- 172.21.15.38:0/1979959477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 msgr2=0x560e014592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.319+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/1979959477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e014592d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0288035040 tx=0x7f0288009530 comp rx=0 tx=0).stop 2024-05-21T15:40:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.319+0000 7f02a6b6d740 1 -- 172.21.15.38:0/1979959477 shutdown_connections 2024-05-21T15:40:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.319+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/1979959477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010b11b0 0x560e01455050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.319+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/1979959477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010904d0 0x560e01459810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.319+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/1979959477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e014592d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.319+0000 7f02a6b6d740 1 -- 172.21.15.38:0/1979959477 >> 172.21.15.38:0/1979959477 conn(0x560e010a5300 msgr2=0x560e01454160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.319+0000 7f02a6b6d740 1 -- 172.21.15.38:0/1979959477 shutdown_connections 2024-05-21T15:40:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.319+0000 7f02a6b6d740 1 -- 172.21.15.38:0/1979959477 wait complete. 2024-05-21T15:40:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.320+0000 7f02a6b6d740 1 -- start start 2024-05-21T15:40:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.320+0000 7f02a6b6d740 1 Processor -- start 2024-05-21T15:40:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.320+0000 7f02a6b6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e0145e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.320+0000 7f02a6b6d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010904d0 0x560e0145e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f029d185640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e0145e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f029d185640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e0145e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:35510/0 (socket says 172.21.15.38:35510) 2024-05-21T15:40:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f02a6b6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010b11b0 0x560e014ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f029d185640 1 -- 172.21.15.38:0/2190353310 learned_addr learned my addr 172.21.15.38:0/2190353310 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f029d986640 1 --2- 172.21.15.38:0/2190353310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010904d0 0x560e0145e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f02a6b6d740 1 -- 172.21.15.38:0/2190353310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02981b1f20 con 0x560e010b11b0 2024-05-21T15:40:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f02a6b6d740 1 -- 172.21.15.38:0/2190353310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f02981b1820 con 0x560e010882f0 2024-05-21T15:40:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f02a6b6d740 1 -- 172.21.15.38:0/2190353310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f02981aea60 con 0x560e010904d0 2024-05-21T15:40:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f02a6b6d740 10 client.0 _ll_register_callbacks cb 0x7f0288035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f029d185640 1 --2- 172.21.15.38:0/2190353310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010b11b0 0x560e014ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f029d185640 1 -- 172.21.15.38:0/2190353310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010904d0 msgr2=0x560e0145e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f029d185640 1 --2- 172.21.15.38:0/2190353310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010904d0 0x560e0145e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f029d185640 1 -- 172.21.15.38:0/2190353310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010b11b0 msgr2=0x560e014ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f029d185640 1 --2- 172.21.15.38:0/2190353310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010b11b0 0x560e014ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.321+0000 7f029d185640 1 -- 172.21.15.38:0/2190353310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e01461880 con 0x560e010882f0 2024-05-21T15:40:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.322+0000 7f029d185640 1 --2- 172.21.15.38:0/2190353310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e0145e380 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0288009780 tx=0x7f02880057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f332bb9742c5ad7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.322+0000 7f02817fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:40:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.322+0000 7f02817fa640 1 -- 172.21.15.38:0/2190353310 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288053030 con 0x560e010882f0 2024-05-21T15:40:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.322+0000 7f02a6b6d740 1 -- 172.21.15.38:0/2190353310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0288038040 con 0x560e010882f0 2024-05-21T15:40:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.322+0000 7f02817fa640 1 -- 172.21.15.38:0/2190353310 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f02880035f0 con 0x560e010882f0 2024-05-21T15:40:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.322+0000 7f02817fa640 1 -- 172.21.15.38:0/2190353310 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f028805e050 con 0x560e010882f0 2024-05-21T15:40:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.323+0000 7f02a6b6d740 10 client.26555 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.323+0000 7f02a6b6d740 1 -- 172.21.15.38:0/2190353310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560e014cf620 con 0x560e010882f0 2024-05-21T15:40:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.323+0000 7f02817fa640 1 -- 172.21.15.38:0/2190353310 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0288038040 con 0x560e010882f0 2024-05-21T15:40:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.323+0000 7f02817fa640 1 -- 172.21.15.38:0/2190353310 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f0288032cb0 con 0x560e010882f0 2024-05-21T15:40:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.323+0000 7f02a6b6d740 10 client.26555 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.323+0000 7f02a6b6d740 10 client.26555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.323+0000 7f02a6b6d740 20 client.26555 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.323+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/2190353310 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e010451c0 0x560e014d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.323+0000 7f02a6b6d740 4 client.26555 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:40:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.323+0000 7f02a6b6d740 1 -- 172.21.15.38:0/2190353310 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e013bcdc0 con 0x560e010451c0 2024-05-21T15:40:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.324+0000 7f029d986640 1 --2- 172.21.15.38:0/2190353310 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e010451c0 0x560e014d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.324+0000 7f029d986640 1 --2- 172.21.15.38:0/2190353310 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e010451c0 0x560e014d1f90 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58f231802558727b server_cookie=f1e76492d55e15c3 in_seq=0 out_seq=0 2024-05-21T15:40:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.324+0000 7f02817fa640 10 client.26555 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.344+0000 7f02817fa640 1 -- 172.21.15.38:0/2190353310 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e013bcdc0 con 0x560e010451c0 2024-05-21T15:40:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.344+0000 7f02817fa640 10 client.26555 handle_command_reply: tid=0 2024-05-21T15:40:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.344+0000 7f02a6b6d740 1 client.26555 shutdown 2024-05-21T15:40:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.345+0000 7f02a6b6d740 1 -- 172.21.15.38:0/2190353310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 msgr2=0x560e0145e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.345+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/2190353310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e0145e380 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0288009780 tx=0x7f02880057f0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.345+0000 7f02a6b6d740 1 -- 172.21.15.38:0/2190353310 shutdown_connections 2024-05-21T15:40:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.345+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/2190353310 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560e010451c0 0x560e014d1f90 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.345+0000 7f02817fa640 0 client.26555 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.345+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/2190353310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560e010b11b0 0x560e014ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.345+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/2190353310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560e010904d0 0x560e0145e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.345+0000 7f02a6b6d740 1 --2- 172.21.15.38:0/2190353310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e010882f0 0x560e0145e380 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.345+0000 7f02a6b6d740 1 -- 172.21.15.38:0/2190353310 >> 172.21.15.38:0/2190353310 conn(0x560e010a5300 msgr2=0x560e014592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.345+0000 7f02a6b6d740 1 -- 172.21.15.38:0/2190353310 shutdown_connections 2024-05-21T15:40:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.345+0000 7f02a6b6d740 1 -- 172.21.15.38:0/2190353310 wait complete. 2024-05-21T15:40:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.345+0000 7f02a6b6d740 20 client.26555 trim_cache size 0 max 16384 2024-05-21T15:40:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:40:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:40:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:40:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:40:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.347+0000 7f029fbea640 1 -- 172.21.15.38:0/4109629912 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f02840a2cb0 msgr2=0x7f02840a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.348+0000 7f029fbea640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f02840a2cb0 0x7f02840a51c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f02900982b0 tx=0x7f029001c000 comp rx=0 tx=0).stop 2024-05-21T15:40:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.348+0000 7f029fbea640 1 -- 172.21.15.38:0/4109629912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0298108730 msgr2=0x7f02981a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.348+0000 7f029fbea640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0298108730 0x7f02981a0670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0290009fd0 tx=0x7f029000c800 comp rx=0 tx=0).stop 2024-05-21T15:40:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.348+0000 7f029fbea640 1 -- 172.21.15.38:0/4109629912 shutdown_connections 2024-05-21T15:40:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.348+0000 7f029fbea640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0298108730 0x7f02981a0670 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.348+0000 7f029fbea640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0298101310 0x7f02981996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.348+0000 7f029fbea640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f02840a2cb0 0x7f02840a51c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.348+0000 7f029fbea640 1 --2- 172.21.15.38:0/4109629912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0298100e10 0x7f02981991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.348+0000 7f029fbea640 1 -- 172.21.15.38:0/4109629912 >> 172.21.15.38:0/4109629912 conn(0x7f02980fc300 msgr2=0x7f02981045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.348+0000 7f029fbea640 1 -- 172.21.15.38:0/4109629912 shutdown_connections 2024-05-21T15:40:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:02.348+0000 7f029fbea640 1 -- 172.21.15.38:0/4109629912 wait complete. 2024-05-21T15:40:04.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:03 smithi053 ceph-mon[36496]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 MiB/s wr, 6.77k op/s 2024-05-21T15:40:04.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:03 smithi045 ceph-mon[36409]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 MiB/s wr, 6.77k op/s 2024-05-21T15:40:04.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:03 smithi038 ceph-mon[31995]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 MiB/s wr, 6.77k op/s 2024-05-21T15:40:06.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:05 smithi053 ceph-mon[36496]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 6.94k op/s 2024-05-21T15:40:06.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:05 smithi045 ceph-mon[36409]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 6.94k op/s 2024-05-21T15:40:06.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:05 smithi038 ceph-mon[31995]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 6.94k op/s 2024-05-21T15:40:08.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:07 smithi053 ceph-mon[36496]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 4.75k op/s 2024-05-21T15:40:08.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:07 smithi045 ceph-mon[36409]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 4.75k op/s 2024-05-21T15:40:08.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:07 smithi038 ceph-mon[31995]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 4.75k op/s 2024-05-21T15:40:10.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:09 smithi053 ceph-mon[36496]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 4.51k op/s 2024-05-21T15:40:10.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:09 smithi045 ceph-mon[36409]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 4.51k op/s 2024-05-21T15:40:10.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:09 smithi038 ceph-mon[31995]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 4.51k op/s 2024-05-21T15:40:12.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:11 smithi053 ceph-mon[36496]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 5.61k op/s 2024-05-21T15:40:12.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:11 smithi045 ceph-mon[36409]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 5.61k op/s 2024-05-21T15:40:12.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:11 smithi038 ceph-mon[31995]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 5.61k op/s 2024-05-21T15:40:14.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:13 smithi053 ceph-mon[36496]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 3.69k op/s 2024-05-21T15:40:14.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:13 smithi045 ceph-mon[36409]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 3.69k op/s 2024-05-21T15:40:14.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:13 smithi038 ceph-mon[31995]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 3.69k op/s 2024-05-21T15:40:15.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:40:15.002 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:40:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.160+0000 7fa07fd4c640 1 -- 172.21.15.38:0/49024248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa078105ed0 msgr2=0x7fa0781082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.160+0000 7fa07fd4c640 1 --2- 172.21.15.38:0/49024248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa078105ed0 0x7fa0781082f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa068009900 tx=0x7fa06802a440 comp rx=0 tx=0).stop 2024-05-21T15:40:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.161+0000 7fa07fd4c640 1 -- 172.21.15.38:0/49024248 shutdown_connections 2024-05-21T15:40:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.161+0000 7fa07fd4c640 1 --2- 172.21.15.38:0/49024248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa078108830 0x7fa07810ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.161+0000 7fa07fd4c640 1 --2- 172.21.15.38:0/49024248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa078105ed0 0x7fa0781082f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.161+0000 7fa07fd4c640 1 --2- 172.21.15.38:0/49024248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa078103570 0x7fa078105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.161+0000 7fa07fd4c640 1 -- 172.21.15.38:0/49024248 >> 172.21.15.38:0/49024248 conn(0x7fa0780fcb70 msgr2=0x7fa0780ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:15.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.161+0000 7fa07fd4c640 1 -- 172.21.15.38:0/49024248 shutdown_connections 2024-05-21T15:40:15.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.161+0000 7fa07fd4c640 1 -- 172.21.15.38:0/49024248 wait complete. 2024-05-21T15:40:15.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.162+0000 7fa07fd4c640 1 Processor -- start 2024-05-21T15:40:15.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.162+0000 7fa07fd4c640 1 -- start start 2024-05-21T15:40:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.162+0000 7fa07fd4c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa078103570 0x7fa0781123f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.162+0000 7fa07fd4c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa078105ed0 0x7fa078112930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.162+0000 7fa07fd4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa078108830 0x7fa0781178b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.162+0000 7fa07dae8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa078103570 0x7fa0781123f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.163+0000 7fa07dae8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa078103570 0x7fa0781123f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59028/0 (socket says 172.21.15.38:59028) 2024-05-21T15:40:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.163+0000 7fa07dae8640 1 -- 172.21.15.38:0/1991538979 learned_addr learned my addr 172.21.15.38:0/1991538979 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.163+0000 7fa07d2e7640 1 --2- 172.21.15.38:0/1991538979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa078105ed0 0x7fa078112930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.163+0000 7fa07fd4c640 1 -- 172.21.15.38:0/1991538979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa07810c590 con 0x7fa078103570 2024-05-21T15:40:15.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.163+0000 7fa07fd4c640 1 -- 172.21.15.38:0/1991538979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa07810c410 con 0x7fa078105ed0 2024-05-21T15:40:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.163+0000 7fa07fd4c640 1 -- 172.21.15.38:0/1991538979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa07810c710 con 0x7fa078108830 2024-05-21T15:40:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.163+0000 7fa07e2e9640 1 --2- 172.21.15.38:0/1991538979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa078108830 0x7fa0781178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.163+0000 7fa07dae8640 1 -- 172.21.15.38:0/1991538979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa078108830 msgr2=0x7fa0781178b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.163+0000 7fa07dae8640 1 --2- 172.21.15.38:0/1991538979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa078108830 0x7fa0781178b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.163+0000 7fa07dae8640 1 -- 172.21.15.38:0/1991538979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa078105ed0 msgr2=0x7fa078112930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.163+0000 7fa07dae8640 1 --2- 172.21.15.38:0/1991538979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa078105ed0 0x7fa078112930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.163+0000 7fa07dae8640 1 -- 172.21.15.38:0/1991538979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa078117f20 con 0x7fa078103570 2024-05-21T15:40:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.164+0000 7fa07dae8640 1 --2- 172.21.15.38:0/1991538979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa078103570 0x7fa0781123f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa07401c460 tx=0x7fa07401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.164+0000 7fa06effd640 1 -- 172.21.15.38:0/1991538979 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074025070 con 0x7fa078103570 2024-05-21T15:40:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.164+0000 7fa07fd4c640 1 -- 172.21.15.38:0/1991538979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa07806b570 con 0x7fa078103570 2024-05-21T15:40:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.164+0000 7fa06effd640 1 -- 172.21.15.38:0/1991538979 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa074021ce0 con 0x7fa078103570 2024-05-21T15:40:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.164+0000 7fa06effd640 1 -- 172.21.15.38:0/1991538979 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074028e50 con 0x7fa078103570 2024-05-21T15:40:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.165+0000 7fa07fd4c640 1 -- 172.21.15.38:0/1991538979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa07806bb10 con 0x7fa078103570 2024-05-21T15:40:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.169+0000 7fa086bf9740 1 Processor -- start 2024-05-21T15:40:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.169+0000 7fa06effd640 1 -- 172.21.15.38:0/1991538979 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa074028070 con 0x7fa078103570 2024-05-21T15:40:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.169+0000 7fa086bf9740 1 -- start start 2024-05-21T15:40:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.170+0000 7fa086bf9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc9163e390 0x55fc919ddbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.170+0000 7fa086bf9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc916154d0 0x55fc919de130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.170+0000 7fa086bf9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9161d6b0 0x55fc919e5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.170+0000 7fa07e2e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc9163e390 0x55fc919ddbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.170+0000 7fa07e2e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc9163e390 0x55fc919ddbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:38798/0 (socket says 172.21.15.38:38798) 2024-05-21T15:40:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.170+0000 7fa086bf9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fc91942d00 con 0x55fc9161d6b0 2024-05-21T15:40:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.170+0000 7fa086bf9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fc918ea4c0 con 0x55fc916154d0 2024-05-21T15:40:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.170+0000 7fa07e2e9640 1 -- 172.21.15.38:0/3215586296 learned_addr learned my addr 172.21.15.38:0/3215586296 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.170+0000 7fa086bf9740 1 -- 172.21.15.38:0/3215586296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55fc9193ecc0 con 0x55fc9163e390 2024-05-21T15:40:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.170+0000 7fa07d2e7640 1 --2- 172.21.15.38:0/3215586296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9161d6b0 0x55fc919e5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.170+0000 7fa07dae8640 1 --2- 172.21.15.38:0/3215586296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc916154d0 0x55fc919de130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.171+0000 7fa06effd640 1 --2- 172.21.15.38:0/1991538979 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0640a2e00 0x7fa0640a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.171+0000 7fa07d2e7640 1 -- 172.21.15.38:0/3215586296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc9163e390 msgr2=0x55fc919ddbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.171+0000 7fa07d2e7640 1 --2- 172.21.15.38:0/3215586296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc9163e390 0x55fc919ddbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.171+0000 7fa07d2e7640 1 -- 172.21.15.38:0/3215586296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc916154d0 msgr2=0x55fc919de130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.171+0000 7fa07d2e7640 1 --2- 172.21.15.38:0/3215586296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc916154d0 0x55fc919de130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.171+0000 7fa07d2e7640 1 -- 172.21.15.38:0/3215586296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc917207d0 con 0x55fc9161d6b0 2024-05-21T15:40:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.171+0000 7fa07e2e9640 1 --2- 172.21.15.38:0/3215586296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc9163e390 0x55fc919ddbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.171+0000 7fa06effd640 1 -- 172.21.15.38:0/1991538979 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa0740df3e0 con 0x7fa078103570 2024-05-21T15:40:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.171+0000 7fa07d2e7640 1 --2- 172.21.15.38:0/3215586296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9161d6b0 0x55fc919e5030 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa06802aa30 tx=0x7fa068004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db280802d5cafd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.171+0000 7fa07e2e9640 1 --2- 172.21.15.38:0/1991538979 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0640a2e00 0x7fa0640a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.171+0000 7fa063fff640 1 -- 172.21.15.38:0/3215586296 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068035040 con 0x55fc9161d6b0 2024-05-21T15:40:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.172+0000 7fa063fff640 1 -- 172.21.15.38:0/3215586296 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa068009540 con 0x55fc9161d6b0 2024-05-21T15:40:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.172+0000 7fa063fff640 1 -- 172.21.15.38:0/3215586296 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0680318a0 con 0x55fc9161d6b0 2024-05-21T15:40:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.173+0000 7fa07e2e9640 1 --2- 172.21.15.38:0/1991538979 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0640a2e00 0x7fa0640a5310 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55fc919ded80 tx=0x7fa070004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.173+0000 7fa086bf9740 1 -- 172.21.15.38:0/3215586296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9161d6b0 msgr2=0x55fc919e5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.173+0000 7fa086bf9740 1 --2- 172.21.15.38:0/3215586296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9161d6b0 0x55fc919e5030 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa06802aa30 tx=0x7fa068004310 comp rx=0 tx=0).stop 2024-05-21T15:40:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.173+0000 7fa086bf9740 1 -- 172.21.15.38:0/3215586296 shutdown_connections 2024-05-21T15:40:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.173+0000 7fa086bf9740 1 --2- 172.21.15.38:0/3215586296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9161d6b0 0x55fc919e5030 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.174+0000 7fa086bf9740 1 --2- 172.21.15.38:0/3215586296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc916154d0 0x55fc919de130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.174+0000 7fa086bf9740 1 --2- 172.21.15.38:0/3215586296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc9163e390 0x55fc919ddbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.174+0000 7fa086bf9740 1 -- 172.21.15.38:0/3215586296 >> 172.21.15.38:0/3215586296 conn(0x55fc916324e0 msgr2=0x55fc919d9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.174+0000 7fa086bf9740 1 -- 172.21.15.38:0/3215586296 shutdown_connections 2024-05-21T15:40:15.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.174+0000 7fa086bf9740 1 -- 172.21.15.38:0/3215586296 wait complete. 2024-05-21T15:40:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.175+0000 7fa086bf9740 1 -- start start 2024-05-21T15:40:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.175+0000 7fa086bf9740 1 Processor -- start 2024-05-21T15:40:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.176+0000 7fa086bf9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc916154d0 0x55fc919e7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.176+0000 7fa086bf9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9161d6b0 0x55fc919e7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.176+0000 7fa086bf9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc9163e390 0x55fc919eed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.176+0000 7fa07dae8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc916154d0 0x55fc919e7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.176+0000 7fa07dae8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc916154d0 0x55fc919e7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46328/0 (socket says 172.21.15.38:46328) 2024-05-21T15:40:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.176+0000 7fa07dae8640 1 -- 172.21.15.38:0/3112307042 learned_addr learned my addr 172.21.15.38:0/3112307042 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:15.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.176+0000 7fa086bf9740 1 -- 172.21.15.38:0/3112307042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fc919e5a20 con 0x55fc9161d6b0 2024-05-21T15:40:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.176+0000 7fa07e2e9640 1 --2- 172.21.15.38:0/3112307042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc9163e390 0x55fc919eed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.176+0000 7fa086bf9740 1 -- 172.21.15.38:0/3112307042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fc918fdb90 con 0x55fc916154d0 2024-05-21T15:40:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.177+0000 7fa086bf9740 1 -- 172.21.15.38:0/3112307042 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55fc919e5ba0 con 0x55fc9163e390 2024-05-21T15:40:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.177+0000 7fa07d2e7640 1 --2- 172.21.15.38:0/3112307042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9161d6b0 0x55fc919e7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.177+0000 7fa086bf9740 10 client.0 _ll_register_callbacks cb 0x55fc91939da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.177+0000 7fa07dae8640 1 -- 172.21.15.38:0/3112307042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc9163e390 msgr2=0x55fc919eed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.177+0000 7fa07dae8640 1 --2- 172.21.15.38:0/3112307042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc9163e390 0x55fc919eed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.177+0000 7fa07dae8640 1 -- 172.21.15.38:0/3112307042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9161d6b0 msgr2=0x55fc919e7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.177+0000 7fa07dae8640 1 --2- 172.21.15.38:0/3112307042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9161d6b0 0x55fc919e7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.177+0000 7fa07dae8640 1 -- 172.21.15.38:0/3112307042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc916563f0 con 0x55fc916154d0 2024-05-21T15:40:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.177+0000 7fa086bf9740 1 -- 172.21.15.38:0/3112307042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa068009540 con 0x55fc916154d0 2024-05-21T15:40:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.178+0000 7fa07dae8640 1 --2- 172.21.15.38:0/3112307042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc916154d0 0x55fc919e7880 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa0740b42d0 tx=0x7fa0740ab780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11c842a39dfd41a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:15.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.178+0000 7fa0617fa640 10 client.26567 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:40:15.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.178+0000 7fa0617fa640 1 -- 172.21.15.38:0/3112307042 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07402f050 con 0x55fc916154d0 2024-05-21T15:40:15.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.178+0000 7fa0617fa640 1 -- 172.21.15.38:0/3112307042 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa0740dcf40 con 0x55fc916154d0 2024-05-21T15:40:15.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.178+0000 7fa0617fa640 1 -- 172.21.15.38:0/3112307042 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa0740abc40 con 0x55fc916154d0 2024-05-21T15:40:15.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.178+0000 7fa086bf9740 10 client.26567 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:15.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.178+0000 7fa086bf9740 1 -- 172.21.15.38:0/3112307042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55fc916160f0 con 0x55fc916154d0 2024-05-21T15:40:15.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.179+0000 7fa0617fa640 1 -- 172.21.15.38:0/3112307042 <== mon.2 v2:172.21.15.45:3300/0 4 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fa0740de070 con 0x55fc916154d0 2024-05-21T15:40:15.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.179+0000 7fa086bf9740 10 client.26567 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:15.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.179+0000 7fa086bf9740 10 client.26567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:15.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.179+0000 7fa086bf9740 20 client.26567 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:15.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.179+0000 7fa086bf9740 1 --2- 172.21.15.38:0/3112307042 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fc915fcf60 0x55fc919f5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.179+0000 7fa086bf9740 4 client.26567 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:15.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.179+0000 7fa086bf9740 1 -- 172.21.15.38:0/3112307042 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc91950430 con 0x55fc915fcf60 2024-05-21T15:40:15.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.179+0000 7fa07d2e7640 1 --2- 172.21.15.38:0/3112307042 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fc915fcf60 0x55fc919f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.180+0000 7fa07d2e7640 1 --2- 172.21.15.38:0/3112307042 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fc915fcf60 0x55fc919f5500 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca23f6765586fff1 server_cookie=d486e62c76cab8da in_seq=0 out_seq=0 2024-05-21T15:40:15.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.180+0000 7fa0617fa640 10 client.26567 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:15.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.181+0000 7fa0617fa640 1 -- 172.21.15.38:0/3112307042 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fc91950430 con 0x55fc915fcf60 2024-05-21T15:40:15.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.181+0000 7fa0617fa640 10 client.26567 handle_command_reply: tid=0 2024-05-21T15:40:15.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.181+0000 7fa086bf9740 1 client.26567 shutdown 2024-05-21T15:40:15.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.181+0000 7fa086bf9740 1 -- 172.21.15.38:0/3112307042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc916154d0 msgr2=0x55fc919e7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.181+0000 7fa086bf9740 1 --2- 172.21.15.38:0/3112307042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc916154d0 0x55fc919e7880 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa0740b42d0 tx=0x7fa0740ab780 comp rx=0 tx=0).stop 2024-05-21T15:40:15.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.182+0000 7fa086bf9740 1 -- 172.21.15.38:0/3112307042 shutdown_connections 2024-05-21T15:40:15.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.182+0000 7fa086bf9740 1 --2- 172.21.15.38:0/3112307042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc9163e390 0x55fc919eed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.182+0000 7fa086bf9740 1 --2- 172.21.15.38:0/3112307042 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fc915fcf60 0x55fc919f5500 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.182+0000 7fa086bf9740 1 --2- 172.21.15.38:0/3112307042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc9161d6b0 0x55fc919e7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.182+0000 7fa0617fa640 0 client.26567 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:15.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.182+0000 7fa086bf9740 1 --2- 172.21.15.38:0/3112307042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc916154d0 0x55fc919e7880 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.182+0000 7fa086bf9740 1 -- 172.21.15.38:0/3112307042 >> 172.21.15.38:0/3112307042 conn(0x55fc916324e0 msgr2=0x55fc919d85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:15.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.183+0000 7fa086bf9740 1 -- 172.21.15.38:0/3112307042 shutdown_connections 2024-05-21T15:40:15.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.183+0000 7fa086bf9740 1 -- 172.21.15.38:0/3112307042 wait complete. 2024-05-21T15:40:15.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.184+0000 7fa086bf9740 20 client.26567 trim_cache size 0 max 16384 2024-05-21T15:40:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.213+0000 7fa086bf9740 1 Processor -- start 2024-05-21T15:40:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.213+0000 7fa086bf9740 1 -- start start 2024-05-21T15:40:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa086bf9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc915fcf60 0x55fc919e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa086bf9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc916154d0 0x55fc919eecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa086bf9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc9161d6b0 0x55fc919d7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa086bf9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa0781b0ab0 con 0x55fc915fcf60 2024-05-21T15:40:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa086bf9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0781b26e0 con 0x55fc9161d6b0 2024-05-21T15:40:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa086bf9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa0781b1180 con 0x55fc916154d0 2024-05-21T15:40:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa07d2e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc916154d0 0x55fc919eecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa07dae8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc915fcf60 0x55fc919e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa07dae8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc915fcf60 0x55fc919e7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59058/0 (socket says 172.21.15.38:59058) 2024-05-21T15:40:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa07dae8640 1 -- 172.21.15.38:0/1469367571 learned_addr learned my addr 172.21.15.38:0/1469367571 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa07d2e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc916154d0 0x55fc919eecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:38816/0 (socket says 172.21.15.38:38816) 2024-05-21T15:40:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa07e2e9640 1 --2- 172.21.15.38:0/1469367571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc9161d6b0 0x55fc919d7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa07dae8640 1 -- 172.21.15.38:0/1469367571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc916154d0 msgr2=0x55fc919eecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa07dae8640 1 --2- 172.21.15.38:0/1469367571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc916154d0 0x55fc919eecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa07dae8640 1 -- 172.21.15.38:0/1469367571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc9161d6b0 msgr2=0x55fc919d7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa07dae8640 1 --2- 172.21.15.38:0/1469367571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc9161d6b0 0x55fc919d7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.214+0000 7fa07dae8640 1 -- 172.21.15.38:0/1469367571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc919dd6c0 con 0x55fc915fcf60 2024-05-21T15:40:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.215+0000 7fa07d2e7640 1 --2- 172.21.15.38:0/1469367571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc916154d0 0x55fc919eecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.215+0000 7fa07e2e9640 1 --2- 172.21.15.38:0/1469367571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc9161d6b0 0x55fc919d7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.215+0000 7fa07dae8640 1 --2- 172.21.15.38:0/1469367571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc915fcf60 0x55fc919e7b60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa0740a85c0 tx=0x7fa0740e7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f07a7ab430cc68 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.215+0000 7fa0637fe640 1 -- 172.21.15.38:0/1469367571 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07402f050 con 0x55fc915fcf60 2024-05-21T15:40:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.215+0000 7fa0637fe640 1 -- 172.21.15.38:0/1469367571 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa068009540 con 0x55fc915fcf60 2024-05-21T15:40:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.215+0000 7fa0637fe640 1 -- 172.21.15.38:0/1469367571 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0740a7c80 con 0x55fc915fcf60 2024-05-21T15:40:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.216+0000 7fa086bf9740 1 -- 172.21.15.38:0/1469367571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc915fcf60 msgr2=0x55fc919e7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.216+0000 7fa086bf9740 1 --2- 172.21.15.38:0/1469367571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc915fcf60 0x55fc919e7b60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa0740a85c0 tx=0x7fa0740e7000 comp rx=0 tx=0).stop 2024-05-21T15:40:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.216+0000 7fa086bf9740 1 -- 172.21.15.38:0/1469367571 shutdown_connections 2024-05-21T15:40:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.216+0000 7fa086bf9740 1 --2- 172.21.15.38:0/1469367571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc9161d6b0 0x55fc919d7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.216+0000 7fa086bf9740 1 --2- 172.21.15.38:0/1469367571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc916154d0 0x55fc919eecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.216+0000 7fa086bf9740 1 --2- 172.21.15.38:0/1469367571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc915fcf60 0x55fc919e7b60 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.216+0000 7fa086bf9740 1 -- 172.21.15.38:0/1469367571 >> 172.21.15.38:0/1469367571 conn(0x55fc916324e0 msgr2=0x55fc919e7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.216+0000 7fa086bf9740 1 -- 172.21.15.38:0/1469367571 shutdown_connections 2024-05-21T15:40:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.217+0000 7fa086bf9740 1 -- 172.21.15.38:0/1469367571 wait complete. 2024-05-21T15:40:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.217+0000 7fa086bf9740 1 -- start start 2024-05-21T15:40:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.217+0000 7fa086bf9740 1 Processor -- start 2024-05-21T15:40:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.218+0000 7fa086bf9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc915fcf60 0x55fc919eba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.218+0000 7fa086bf9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc916154d0 0x55fc919ebf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.218+0000 7fa086bf9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc9161d6b0 0x55fc91a5b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.218+0000 7fa086bf9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa0781b1f50 con 0x55fc915fcf60 2024-05-21T15:40:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.218+0000 7fa086bf9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0781b1850 con 0x55fc9161d6b0 2024-05-21T15:40:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.218+0000 7fa07dae8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc915fcf60 0x55fc919eba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.218+0000 7fa07e2e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc9161d6b0 0x55fc91a5b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.218+0000 7fa07dae8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc915fcf60 0x55fc919eba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59072/0 (socket says 172.21.15.38:59072) 2024-05-21T15:40:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.218+0000 7fa07e2e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc9161d6b0 0x55fc91a5b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46346/0 (socket says 172.21.15.38:46346) 2024-05-21T15:40:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.218+0000 7fa07dae8640 1 -- 172.21.15.38:0/2855968249 learned_addr learned my addr 172.21.15.38:0/2855968249 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.218+0000 7fa086bf9740 1 -- 172.21.15.38:0/2855968249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa0780727f0 con 0x55fc916154d0 2024-05-21T15:40:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.218+0000 7fa086bf9740 10 client.0 _ll_register_callbacks cb 0x7fa0740e2b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.219+0000 7fa07dae8640 1 -- 172.21.15.38:0/2855968249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc916154d0 msgr2=0x55fc919ebf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.219+0000 7fa07dae8640 1 --2- 172.21.15.38:0/2855968249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc916154d0 0x55fc919ebf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.219+0000 7fa07dae8640 1 -- 172.21.15.38:0/2855968249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc9161d6b0 msgr2=0x55fc91a5b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.219+0000 7fa07dae8640 1 --2- 172.21.15.38:0/2855968249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc9161d6b0 0x55fc91a5b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.219+0000 7fa07dae8640 1 -- 172.21.15.38:0/2855968249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc919eeee0 con 0x55fc915fcf60 2024-05-21T15:40:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.219+0000 7fa07dae8640 1 --2- 172.21.15.38:0/2855968249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc915fcf60 0x55fc919eba30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa0740a8610 tx=0x7fa0740a7f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc9152db5f92927 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.219+0000 7fa0617fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.219+0000 7fa0617fa640 1 -- 172.21.15.38:0/2855968249 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07402f030 con 0x55fc915fcf60 2024-05-21T15:40:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.219+0000 7fa0617fa640 1 -- 172.21.15.38:0/2855968249 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa0740ca510 con 0x55fc915fcf60 2024-05-21T15:40:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.220+0000 7fa086bf9740 1 -- 172.21.15.38:0/2855968249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa068009540 con 0x55fc915fcf60 2024-05-21T15:40:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.220+0000 7fa0617fa640 1 -- 172.21.15.38:0/2855968249 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa0740aff40 con 0x55fc915fcf60 2024-05-21T15:40:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.221+0000 7fa086bf9740 10 client.16950 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.221+0000 7fa086bf9740 1 -- 172.21.15.38:0/2855968249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fc91a5cc90 con 0x55fc915fcf60 2024-05-21T15:40:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.221+0000 7fa0617fa640 1 -- 172.21.15.38:0/2855968249 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0740aab00 con 0x55fc915fcf60 2024-05-21T15:40:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.221+0000 7fa0617fa640 1 -- 172.21.15.38:0/2855968249 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fa0740de030 con 0x55fc915fcf60 2024-05-21T15:40:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.221+0000 7fa086bf9740 10 client.16950 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.221+0000 7fa086bf9740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.221+0000 7fa086bf9740 20 client.16950 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.221+0000 7fa086bf9740 1 --2- 172.21.15.38:0/2855968249 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fc9163e390 0x55fc91a5f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.222+0000 7fa086bf9740 4 client.16950 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:40:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.222+0000 7fa086bf9740 1 -- 172.21.15.38:0/2855968249 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa068009540 con 0x55fc9163e390 2024-05-21T15:40:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.222+0000 7fa07d2e7640 1 --2- 172.21.15.38:0/2855968249 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fc9163e390 0x55fc91a5f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.223+0000 7fa07d2e7640 1 --2- 172.21.15.38:0/2855968249 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fc9163e390 0x55fc91a5f4d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b321ff2ead6b53e0 server_cookie=6fb0675f8de88ec in_seq=0 out_seq=0 2024-05-21T15:40:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.223+0000 7fa0617fa640 10 client.16950 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.223+0000 7fa0617fa640 1 -- 172.21.15.38:0/2855968249 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fa068009540 con 0x55fc9163e390 2024-05-21T15:40:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.223+0000 7fa0617fa640 10 client.16950 handle_command_reply: tid=0 2024-05-21T15:40:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.223+0000 7fa086bf9740 1 client.16950 shutdown 2024-05-21T15:40:15.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.225+0000 7fa086bf9740 1 -- 172.21.15.38:0/2855968249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc915fcf60 msgr2=0x55fc919eba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.225+0000 7fa086bf9740 1 --2- 172.21.15.38:0/2855968249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc915fcf60 0x55fc919eba30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa0740a8610 tx=0x7fa0740a7f70 comp rx=0 tx=0).stop 2024-05-21T15:40:15.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.225+0000 7fa086bf9740 1 -- 172.21.15.38:0/2855968249 shutdown_connections 2024-05-21T15:40:15.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.225+0000 7fa086bf9740 1 --2- 172.21.15.38:0/2855968249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc9161d6b0 0x55fc91a5b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.225+0000 7fa086bf9740 1 --2- 172.21.15.38:0/2855968249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc916154d0 0x55fc919ebf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.225+0000 7fa086bf9740 1 --2- 172.21.15.38:0/2855968249 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fc9163e390 0x55fc91a5f4d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.225+0000 7fa086bf9740 1 --2- 172.21.15.38:0/2855968249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fc915fcf60 0x55fc919eba30 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.225+0000 7fa0617fa640 0 client.16950 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:15.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.225+0000 7fa086bf9740 1 -- 172.21.15.38:0/2855968249 >> 172.21.15.38:0/2855968249 conn(0x55fc916324e0 msgr2=0x55fc919dc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:15.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.225+0000 7fa086bf9740 1 -- 172.21.15.38:0/2855968249 shutdown_connections 2024-05-21T15:40:15.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.225+0000 7fa086bf9740 1 -- 172.21.15.38:0/2855968249 wait complete. 2024-05-21T15:40:15.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.225+0000 7fa086bf9740 20 client.16950 trim_cache size 0 max 16384 2024-05-21T15:40:15.225 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:40:15.225 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-05-21T15:40:15.225 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-05-21T15:40:15.225 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:40:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.226+0000 7fa07fd4c640 1 -- 172.21.15.38:0/1991538979 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0640a2e00 msgr2=0x7fa0640a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.226+0000 7fa07fd4c640 1 --2- 172.21.15.38:0/1991538979 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0640a2e00 0x7fa0640a5310 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55fc919ded80 tx=0x7fa070004a40 comp rx=0 tx=0).stop 2024-05-21T15:40:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.227+0000 7fa07fd4c640 1 -- 172.21.15.38:0/1991538979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa078103570 msgr2=0x7fa0781123f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.227+0000 7fa07fd4c640 1 --2- 172.21.15.38:0/1991538979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa078103570 0x7fa0781123f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa07401c460 tx=0x7fa07401c920 comp rx=0 tx=0).stop 2024-05-21T15:40:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.227+0000 7fa07fd4c640 1 -- 172.21.15.38:0/1991538979 shutdown_connections 2024-05-21T15:40:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.227+0000 7fa07fd4c640 1 --2- 172.21.15.38:0/1991538979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa078108830 0x7fa0781178b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.227+0000 7fa07fd4c640 1 --2- 172.21.15.38:0/1991538979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa078105ed0 0x7fa078112930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.227+0000 7fa07fd4c640 1 --2- 172.21.15.38:0/1991538979 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa0640a2e00 0x7fa0640a5310 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.227+0000 7fa07fd4c640 1 --2- 172.21.15.38:0/1991538979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa078103570 0x7fa0781123f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.227+0000 7fa07fd4c640 1 -- 172.21.15.38:0/1991538979 >> 172.21.15.38:0/1991538979 conn(0x7fa0780fcb70 msgr2=0x7fa0781069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:15.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.227+0000 7fa07fd4c640 1 -- 172.21.15.38:0/1991538979 shutdown_connections 2024-05-21T15:40:15.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.227+0000 7fa07fd4c640 1 -- 172.21.15.38:0/1991538979 wait complete. 2024-05-21T15:40:15.244 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:40:15.244 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:40:15.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.444+0000 7fe39e626640 1 -- 172.21.15.38:0/2091690723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3981068e0 msgr2=0x7fe398106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.445+0000 7fe39e626640 1 --2- 172.21.15.38:0/2091690723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3981068e0 0x7fe398106cf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe390009900 tx=0x7fe39002a3f0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.445+0000 7fe39e626640 1 -- 172.21.15.38:0/2091690723 shutdown_connections 2024-05-21T15:40:15.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.445+0000 7fe39e626640 1 --2- 172.21.15.38:0/2091690723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3981068e0 0x7fe398106cf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.445+0000 7fe39e626640 1 --2- 172.21.15.38:0/2091690723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398101770 0x7fe398101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.445+0000 7fe39e626640 1 --2- 172.21.15.38:0/2091690723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe398100d30 0x7fe398101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.445+0000 7fe39e626640 1 -- 172.21.15.38:0/2091690723 >> 172.21.15.38:0/2091690723 conn(0x7fe3980fc340 msgr2=0x7fe3980fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:15.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.446+0000 7fe39e626640 1 -- 172.21.15.38:0/2091690723 shutdown_connections 2024-05-21T15:40:15.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.446+0000 7fe39e626640 1 -- 172.21.15.38:0/2091690723 wait complete. 2024-05-21T15:40:15.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.447+0000 7fe39e626640 1 Processor -- start 2024-05-21T15:40:15.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.448+0000 7fe39e626640 1 -- start start 2024-05-21T15:40:15.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.448+0000 7fe39e626640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe398100d30 0x7fe39819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.448+0000 7fe39e626640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398101770 0x7fe39819fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.449+0000 7fe397fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe398100d30 0x7fe39819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.449+0000 7fe397fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe398100d30 0x7fe39819f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:38852/0 (socket says 172.21.15.38:38852) 2024-05-21T15:40:15.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.449+0000 7fe397fff640 1 -- 172.21.15.38:0/753116441 learned_addr learned my addr 172.21.15.38:0/753116441 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:15.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.449+0000 7fe3977fe640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398101770 0x7fe39819fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.449+0000 7fe39e626640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3981068e0 0x7fe3981a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.449+0000 7fe39e626640 1 -- 172.21.15.38:0/753116441 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe398114d00 con 0x7fe398101770 2024-05-21T15:40:15.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.449+0000 7fe39e626640 1 -- 172.21.15.38:0/753116441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe398114b80 con 0x7fe3981068e0 2024-05-21T15:40:15.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.449+0000 7fe39cbc3640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3981068e0 0x7fe3981a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.450+0000 7fe39e626640 1 -- 172.21.15.38:0/753116441 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe398114e80 con 0x7fe398100d30 2024-05-21T15:40:15.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.450+0000 7fe39cbc3640 1 -- 172.21.15.38:0/753116441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe398100d30 msgr2=0x7fe39819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.450+0000 7fe39cbc3640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe398100d30 0x7fe39819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.450+0000 7fe39cbc3640 1 -- 172.21.15.38:0/753116441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398101770 msgr2=0x7fe39819fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.450+0000 7fe39cbc3640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398101770 0x7fe39819fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.450+0000 7fe39cbc3640 1 -- 172.21.15.38:0/753116441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3981a70c0 con 0x7fe3981068e0 2024-05-21T15:40:15.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.451+0000 7fe39cbc3640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3981068e0 0x7fe3981a6a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe39002a9e0 tx=0x7fe390003f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.451+0000 7fe3957fa640 1 -- 172.21.15.38:0/753116441 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39003f070 con 0x7fe3981068e0 2024-05-21T15:40:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.451+0000 7fe3957fa640 1 -- 172.21.15.38:0/753116441 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe39003b460 con 0x7fe3981068e0 2024-05-21T15:40:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.451+0000 7fe3957fa640 1 -- 172.21.15.38:0/753116441 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390035030 con 0x7fe3981068e0 2024-05-21T15:40:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.451+0000 7fe39e626640 1 -- 172.21.15.38:0/753116441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3981a7350 con 0x7fe3981068e0 2024-05-21T15:40:15.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.452+0000 7fe39e626640 1 -- 172.21.15.38:0/753116441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3981a7730 con 0x7fe3981068e0 2024-05-21T15:40:15.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.454+0000 7fe39e626640 1 -- 172.21.15.38:0/753116441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe384005290 con 0x7fe3981068e0 2024-05-21T15:40:15.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.456+0000 7fe3957fa640 1 -- 172.21.15.38:0/753116441 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe390042070 con 0x7fe3981068e0 2024-05-21T15:40:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.457+0000 7fe3957fa640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe3800a2d80 0x7fe3800a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.457+0000 7fe397fff640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe3800a2d80 0x7fe3800a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.457+0000 7fe3957fa640 1 -- 172.21.15.38:0/753116441 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe3900ef8b0 con 0x7fe3981068e0 2024-05-21T15:40:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.458+0000 7fe397fff640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe3800a2d80 0x7fe3800a5290 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe388009c90 tx=0x7fe388005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:15.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.464+0000 7fe3957fa640 1 -- 172.21.15.38:0/753116441 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe3900bc1a0 con 0x7fe3981068e0 2024-05-21T15:40:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.541+0000 7f5045a79640 1 -- 172.21.15.38:0/3342296981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5040102370 msgr2=0x7f5040102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.541+0000 7f5045a79640 1 --2- 172.21.15.38:0/3342296981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5040102370 0x7f5040102810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5038009ca0 tx=0x7f503802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.541+0000 7f5045a79640 1 -- 172.21.15.38:0/3342296981 shutdown_connections 2024-05-21T15:40:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.541+0000 7f5045a79640 1 --2- 172.21.15.38:0/3342296981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5040102370 0x7f5040102810 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.541+0000 7f5045a79640 1 --2- 172.21.15.38:0/3342296981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50401012e0 0x7f5040101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.541+0000 7f5045a79640 1 --2- 172.21.15.38:0/3342296981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5040069070 0x7f5040100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.541+0000 7f5045a79640 1 -- 172.21.15.38:0/3342296981 >> 172.21.15.38:0/3342296981 conn(0x7f50400fc300 msgr2=0x7f50400fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.541+0000 7f5045a79640 1 -- 172.21.15.38:0/3342296981 shutdown_connections 2024-05-21T15:40:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.541+0000 7f5045a79640 1 -- 172.21.15.38:0/3342296981 wait complete. 2024-05-21T15:40:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.542+0000 7f5045a79640 1 Processor -- start 2024-05-21T15:40:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.543+0000 7f5045a79640 1 -- start start 2024-05-21T15:40:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.543+0000 7f5045a79640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5040069070 0x7f5040199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.543+0000 7f5045a79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50401012e0 0x7f5040199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.543+0000 7f5045a79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5040102370 0x7f50401a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.543+0000 7f5045a79640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f504010d610 con 0x7f5040102370 2024-05-21T15:40:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.543+0000 7f5045a79640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f504010d490 con 0x7f50401012e0 2024-05-21T15:40:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.543+0000 7f5045a79640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f504010d790 con 0x7f5040069070 2024-05-21T15:40:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.543+0000 7f503effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50401012e0 0x7f5040199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.543+0000 7f503f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5040069070 0x7f5040199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.543+0000 7f503ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5040102370 0x7f50401a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.543+0000 7f503effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50401012e0 0x7f5040199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46370/0 (socket says 172.21.15.38:46370) 2024-05-21T15:40:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.543+0000 7f503effd640 1 -- 172.21.15.38:0/1686196839 learned_addr learned my addr 172.21.15.38:0/1686196839 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.543+0000 7f503ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5040102370 0x7f50401a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59096/0 (socket says 172.21.15.38:59096) 2024-05-21T15:40:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.544+0000 7f503effd640 1 -- 172.21.15.38:0/1686196839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5040069070 msgr2=0x7f5040199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.544+0000 7f503effd640 1 --2- 172.21.15.38:0/1686196839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5040069070 0x7f5040199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.544+0000 7f503effd640 1 -- 172.21.15.38:0/1686196839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5040102370 msgr2=0x7f50401a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.544+0000 7f503effd640 1 --2- 172.21.15.38:0/1686196839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5040102370 0x7f50401a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.544+0000 7f503effd640 1 -- 172.21.15.38:0/1686196839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50401a0cd0 con 0x7f50401012e0 2024-05-21T15:40:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.544+0000 7f503effd640 1 --2- 172.21.15.38:0/1686196839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50401012e0 0x7f5040199740 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5034009c20 tx=0x7f503400b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.544+0000 7f503cff9640 1 -- 172.21.15.38:0/1686196839 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503401e070 con 0x7f50401012e0 2024-05-21T15:40:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.545+0000 7f5045a79640 1 -- 172.21.15.38:0/1686196839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50401041c0 con 0x7f50401012e0 2024-05-21T15:40:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.545+0000 7f5045a79640 1 -- 172.21.15.38:0/1686196839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5040104760 con 0x7f50401012e0 2024-05-21T15:40:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.545+0000 7f503cff9640 1 -- 172.21.15.38:0/1686196839 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5034019930 con 0x7f50401012e0 2024-05-21T15:40:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.545+0000 7f503cff9640 1 -- 172.21.15.38:0/1686196839 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50340188e0 con 0x7f50401012e0 2024-05-21T15:40:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.548+0000 7f504c9f3740 1 Processor -- start 2024-05-21T15:40:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.548+0000 7f504c9f3740 1 -- start start 2024-05-21T15:40:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.548+0000 7f504c9f3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56365507a1b0 0x563655419e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.548+0000 7f504c9f3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636550512f0 0x56365541a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.548+0000 7f504c9f3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x5636554212e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.548+0000 7f504c9f3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56365537e340 con 0x5636550512f0 2024-05-21T15:40:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.548+0000 7f504c9f3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563655326290 con 0x5636550594d0 2024-05-21T15:40:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.548+0000 7f504c9f3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56365537a300 con 0x56365507a1b0 2024-05-21T15:40:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.548+0000 7f503f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636550512f0 0x56365541a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.548+0000 7f503f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636550512f0 0x56365541a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59112/0 (socket says 172.21.15.38:59112) 2024-05-21T15:40:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.549+0000 7f503f7fe640 1 -- 172.21.15.38:0/3199976736 learned_addr learned my addr 172.21.15.38:0/3199976736 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.549+0000 7f503ffff640 1 --2- 172.21.15.38:0/3199976736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56365507a1b0 0x563655419e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.549+0000 7f503cff9640 1 -- 172.21.15.38:0/1686196839 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5034021070 con 0x7f50401012e0 2024-05-21T15:40:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.549+0000 7f503effd640 1 --2- 172.21.15.38:0/3199976736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x5636554212e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.549+0000 7f503f7fe640 1 -- 172.21.15.38:0/3199976736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56365507a1b0 msgr2=0x563655419e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.549+0000 7f503f7fe640 1 --2- 172.21.15.38:0/3199976736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56365507a1b0 0x563655419e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.549+0000 7f503f7fe640 1 -- 172.21.15.38:0/3199976736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 msgr2=0x5636554212e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.549+0000 7f503f7fe640 1 --2- 172.21.15.38:0/3199976736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x5636554212e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.549+0000 7f503f7fe640 1 -- 172.21.15.38:0/3199976736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365534a8e0 con 0x5636550512f0 2024-05-21T15:40:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.549+0000 7f503effd640 1 --2- 172.21.15.38:0/3199976736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x5636554212e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:40:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.549+0000 7f503f7fe640 1 --2- 172.21.15.38:0/3199976736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636550512f0 0x56365541a3e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5030009c50 tx=0x7f5030011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e05057a7f4a7e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.550+0000 7f5025ffb640 1 -- 172.21.15.38:0/3199976736 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030004030 con 0x5636550512f0 2024-05-21T15:40:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.550+0000 7f5025ffb640 1 -- 172.21.15.38:0/3199976736 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50300041d0 con 0x5636550512f0 2024-05-21T15:40:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.550+0000 7f5025ffb640 1 -- 172.21.15.38:0/3199976736 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030019800 con 0x5636550512f0 2024-05-21T15:40:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.551+0000 7f503cff9640 1 --2- 172.21.15.38:0/1686196839 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f50280a2d80 0x7f50280a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.551+0000 7f503cff9640 1 -- 172.21.15.38:0/1686196839 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f50340d29e0 con 0x7f50401012e0 2024-05-21T15:40:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.552+0000 7f503ffff640 1 --2- 172.21.15.38:0/1686196839 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f50280a2d80 0x7f50280a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.552+0000 7f504c9f3740 1 -- 172.21.15.38:0/3199976736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636550512f0 msgr2=0x56365541a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.552+0000 7f504c9f3740 1 --2- 172.21.15.38:0/3199976736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636550512f0 0x56365541a3e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5030009c50 tx=0x7f5030011810 comp rx=0 tx=0).stop 2024-05-21T15:40:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.552+0000 7f504c9f3740 1 -- 172.21.15.38:0/3199976736 shutdown_connections 2024-05-21T15:40:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.552+0000 7f504c9f3740 1 --2- 172.21.15.38:0/3199976736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x5636554212e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.553+0000 7f504c9f3740 1 --2- 172.21.15.38:0/3199976736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5636550512f0 0x56365541a3e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.553+0000 7f504c9f3740 1 --2- 172.21.15.38:0/3199976736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56365507a1b0 0x563655419e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.553+0000 7f504c9f3740 1 -- 172.21.15.38:0/3199976736 >> 172.21.15.38:0/3199976736 conn(0x56365506e300 msgr2=0x5636554157f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.553+0000 7f503ffff640 1 --2- 172.21.15.38:0/1686196839 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f50280a2d80 0x7f50280a5290 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x56365541b030 tx=0x7f5038009460 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.553+0000 7f504c9f3740 1 -- 172.21.15.38:0/3199976736 shutdown_connections 2024-05-21T15:40:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.553+0000 7f504c9f3740 1 -- 172.21.15.38:0/3199976736 wait complete. 2024-05-21T15:40:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.554+0000 7f504c9f3740 1 -- start start 2024-05-21T15:40:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.554+0000 7f504c9f3740 1 Processor -- start 2024-05-21T15:40:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.555+0000 7f504c9f3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550512f0 0x563655423c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.555+0000 7f504c9f3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x5636554241c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.555+0000 7f504c9f3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 0x56365542b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.556+0000 7f503effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x5636554241c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.556+0000 7f503effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x5636554241c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46382/0 (socket says 172.21.15.38:46382) 2024-05-21T15:40:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.556+0000 7f503effd640 1 -- 172.21.15.38:0/4075182118 learned_addr learned my addr 172.21.15.38:0/4075182118 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.556+0000 7f504c9f3740 1 -- 172.21.15.38:0/4075182118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563655421d30 con 0x56365507a1b0 2024-05-21T15:40:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.556+0000 7f504c9f3740 1 -- 172.21.15.38:0/4075182118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563655339790 con 0x5636550594d0 2024-05-21T15:40:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.556+0000 7f504c9f3740 1 -- 172.21.15.38:0/4075182118 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563655421eb0 con 0x5636550512f0 2024-05-21T15:40:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.556+0000 7f504c9f3740 10 client.0 _ll_register_callbacks cb 0x563655374a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.556+0000 7f503ffff640 1 --2- 172.21.15.38:0/4075182118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 0x56365542b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.556+0000 7f503effd640 1 -- 172.21.15.38:0/4075182118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550512f0 msgr2=0x563655423c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.556+0000 7f503effd640 1 --2- 172.21.15.38:0/4075182118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550512f0 0x563655423c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.556+0000 7f503effd640 1 -- 172.21.15.38:0/4075182118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 msgr2=0x56365542b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.556+0000 7f503effd640 1 --2- 172.21.15.38:0/4075182118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 0x56365542b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.556+0000 7f503effd640 1 -- 172.21.15.38:0/4075182118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563655092210 con 0x5636550594d0 2024-05-21T15:40:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.557+0000 7f503effd640 1 --2- 172.21.15.38:0/4075182118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x5636554241c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f503400bb90 tx=0x7f503409b600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=333028c8f0cb5ce8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.557+0000 7f501effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:40:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.558+0000 7f501effd640 1 -- 172.21.15.38:0/4075182118 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50340a3540 con 0x5636550594d0 2024-05-21T15:40:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.558+0000 7f501effd640 1 -- 172.21.15.38:0/4075182118 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f50340a3150 con 0x5636550594d0 2024-05-21T15:40:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.558+0000 7f504c9f3740 1 -- 172.21.15.38:0/4075182118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50300041d0 con 0x5636550594d0 2024-05-21T15:40:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.558+0000 7f501effd640 1 -- 172.21.15.38:0/4075182118 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f50340a7e00 con 0x5636550594d0 2024-05-21T15:40:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.559+0000 7f504c9f3740 10 client.26588 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.559+0000 7f504c9f3740 1 -- 172.21.15.38:0/4075182118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56365515e620 con 0x5636550594d0 2024-05-21T15:40:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.559+0000 7f501effd640 1 -- 172.21.15.38:0/4075182118 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50300041d0 con 0x5636550594d0 2024-05-21T15:40:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.559+0000 7f501effd640 1 -- 172.21.15.38:0/4075182118 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f5034019ae0 con 0x5636550594d0 2024-05-21T15:40:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.559+0000 7f504c9f3740 10 client.26588 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.559+0000 7f504c9f3740 10 client.26588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.559+0000 7f504c9f3740 20 client.26588 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.559+0000 7f504c9f3740 1 --2- 172.21.15.38:0/4075182118 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x563655038d80 0x563655431cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.559+0000 7f504c9f3740 4 client.26588 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.559+0000 7f504c9f3740 1 -- 172.21.15.38:0/4075182118 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56365538ba70 con 0x563655038d80 2024-05-21T15:40:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.560+0000 7f503f7fe640 1 --2- 172.21.15.38:0/4075182118 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x563655038d80 0x563655431cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.561+0000 7f503f7fe640 1 --2- 172.21.15.38:0/4075182118 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x563655038d80 0x563655431cc0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=597ec8693f354d65 server_cookie=271be19a9281f193 in_seq=0 out_seq=0 2024-05-21T15:40:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.561+0000 7f501effd640 10 client.26588 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.562+0000 7f501effd640 1 -- 172.21.15.38:0/4075182118 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56365538ba70 con 0x563655038d80 2024-05-21T15:40:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.562+0000 7f501effd640 10 client.26588 handle_command_reply: tid=0 2024-05-21T15:40:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.562+0000 7f504c9f3740 1 client.26588 shutdown 2024-05-21T15:40:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.562+0000 7f504c9f3740 1 -- 172.21.15.38:0/4075182118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 msgr2=0x5636554241c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.562+0000 7f504c9f3740 1 --2- 172.21.15.38:0/4075182118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x5636554241c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f503400bb90 tx=0x7f503409b600 comp rx=0 tx=0).stop 2024-05-21T15:40:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.563+0000 7f504c9f3740 1 -- 172.21.15.38:0/4075182118 shutdown_connections 2024-05-21T15:40:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.563+0000 7f504c9f3740 1 --2- 172.21.15.38:0/4075182118 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x563655038d80 0x563655431cc0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.563+0000 7f504c9f3740 1 --2- 172.21.15.38:0/4075182118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 0x56365542b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.563+0000 7f504c9f3740 1 --2- 172.21.15.38:0/4075182118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x5636554241c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.563+0000 7f501effd640 0 client.26588 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.563+0000 7f504c9f3740 1 --2- 172.21.15.38:0/4075182118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550512f0 0x563655423c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.563+0000 7f504c9f3740 1 -- 172.21.15.38:0/4075182118 >> 172.21.15.38:0/4075182118 conn(0x56365506e300 msgr2=0x5636554187a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.563+0000 7f504c9f3740 1 -- 172.21.15.38:0/4075182118 shutdown_connections 2024-05-21T15:40:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.563+0000 7f504c9f3740 1 -- 172.21.15.38:0/4075182118 wait complete. 2024-05-21T15:40:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.563+0000 7f504c9f3740 20 client.26588 trim_cache size 0 max 16384 2024-05-21T15:40:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.589+0000 7f504c9f3740 1 Processor -- start 2024-05-21T15:40:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.589+0000 7f504c9f3740 1 -- start start 2024-05-21T15:40:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.590+0000 7f504c9f3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550512f0 0x563655422950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.590+0000 7f504c9f3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550594d0 0x563655415920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.590+0000 7f504c9f3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 0x56365542afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.590+0000 7f503f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550512f0 0x563655422950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.590+0000 7f503f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550512f0 0x563655422950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46398/0 (socket says 172.21.15.38:46398) 2024-05-21T15:40:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.590+0000 7f503effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550594d0 0x563655415920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.590+0000 7f503effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550594d0 0x563655415920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:38882/0 (socket says 172.21.15.38:38882) 2024-05-21T15:40:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.591+0000 7f503f7fe640 1 -- 172.21.15.38:0/17205532 learned_addr learned my addr 172.21.15.38:0/17205532 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.591+0000 7f504c9f3740 1 -- 172.21.15.38:0/17205532 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f502c00e5e0 con 0x56365507a1b0 2024-05-21T15:40:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.591+0000 7f504c9f3740 1 -- 172.21.15.38:0/17205532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f502c010210 con 0x5636550512f0 2024-05-21T15:40:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.591+0000 7f504c9f3740 1 -- 172.21.15.38:0/17205532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f502c00ecb0 con 0x5636550594d0 2024-05-21T15:40:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.591+0000 7f503ffff640 1 --2- 172.21.15.38:0/17205532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 0x56365542afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.591+0000 7f503effd640 1 -- 172.21.15.38:0/17205532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550512f0 msgr2=0x563655422950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.591+0000 7f503effd640 1 --2- 172.21.15.38:0/17205532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550512f0 0x563655422950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.591+0000 7f503effd640 1 -- 172.21.15.38:0/17205532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 msgr2=0x56365542afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.591+0000 7f503effd640 1 --2- 172.21.15.38:0/17205532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 0x56365542afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.591+0000 7f503effd640 1 -- 172.21.15.38:0/17205532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365541ab50 con 0x5636550594d0 2024-05-21T15:40:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.591+0000 7f503effd640 1 --2- 172.21.15.38:0/17205532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550594d0 0x563655415920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f503409bc10 tx=0x7f50340058b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6a985f48c90a553 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.592+0000 7f50257fa640 1 -- 172.21.15.38:0/17205532 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50340a3540 con 0x5636550594d0 2024-05-21T15:40:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.592+0000 7f50257fa640 1 -- 172.21.15.38:0/17205532 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50340bf0c0 con 0x5636550594d0 2024-05-21T15:40:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.592+0000 7f50257fa640 1 -- 172.21.15.38:0/17205532 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503409e100 con 0x5636550594d0 2024-05-21T15:40:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.593+0000 7f504c9f3740 1 -- 172.21.15.38:0/17205532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550594d0 msgr2=0x563655415920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.593+0000 7f504c9f3740 1 --2- 172.21.15.38:0/17205532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550594d0 0x563655415920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f503409bc10 tx=0x7f50340058b0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.593+0000 7f504c9f3740 1 -- 172.21.15.38:0/17205532 shutdown_connections 2024-05-21T15:40:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.594+0000 7f504c9f3740 1 --2- 172.21.15.38:0/17205532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 0x56365542afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.594+0000 7f504c9f3740 1 --2- 172.21.15.38:0/17205532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550594d0 0x563655415920 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.594+0000 7f504c9f3740 1 --2- 172.21.15.38:0/17205532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550512f0 0x563655422950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.594+0000 7f504c9f3740 1 -- 172.21.15.38:0/17205532 >> 172.21.15.38:0/17205532 conn(0x56365506e300 msgr2=0x563655415210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.594+0000 7f504c9f3740 1 -- 172.21.15.38:0/17205532 shutdown_connections 2024-05-21T15:40:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.594+0000 7f504c9f3740 1 -- 172.21.15.38:0/17205532 wait complete. 2024-05-21T15:40:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.595+0000 7f504c9f3740 1 -- start start 2024-05-21T15:40:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.595+0000 7f504c9f3740 1 Processor -- start 2024-05-21T15:40:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.596+0000 7f504c9f3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550512f0 0x563655428730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.596+0000 7f504c9f3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x563655428c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.596+0000 7f503f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550512f0 0x563655428730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.596+0000 7f504c9f3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 0x563655498020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.596+0000 7f503f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550512f0 0x563655428730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:38890/0 (socket says 172.21.15.38:38890) 2024-05-21T15:40:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.596+0000 7f504c9f3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f502c00fa80 con 0x56365507a1b0 2024-05-21T15:40:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.596+0000 7f504c9f3740 1 -- 172.21.15.38:0/84151967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f502c00f380 con 0x5636550594d0 2024-05-21T15:40:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.596+0000 7f503f7fe640 1 -- 172.21.15.38:0/84151967 learned_addr learned my addr 172.21.15.38:0/84151967 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.596+0000 7f504c9f3740 1 -- 172.21.15.38:0/84151967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f502c00c5c0 con 0x5636550512f0 2024-05-21T15:40:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.596+0000 7f504c9f3740 10 client.0 _ll_register_callbacks cb 0x7f50300096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.596+0000 7f503effd640 1 --2- 172.21.15.38:0/84151967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x563655428c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.596+0000 7f503ffff640 1 --2- 172.21.15.38:0/84151967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 0x563655498020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.596+0000 7f503f7fe640 1 -- 172.21.15.38:0/84151967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 msgr2=0x563655428c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.596+0000 7f503f7fe640 1 --2- 172.21.15.38:0/84151967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x563655428c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.597+0000 7f503f7fe640 1 -- 172.21.15.38:0/84151967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 msgr2=0x563655498020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.597+0000 7f503f7fe640 1 --2- 172.21.15.38:0/84151967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 0x563655498020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.597+0000 7f503f7fe640 1 -- 172.21.15.38:0/84151967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563655498560 con 0x5636550512f0 2024-05-21T15:40:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.597+0000 7f503ffff640 1 --2- 172.21.15.38:0/84151967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 0x563655498020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:40:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.597+0000 7f503f7fe640 1 --2- 172.21.15.38:0/84151967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550512f0 0x563655428730 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x56365541b240 tx=0x7f5030009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8acfe70283516827 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.597+0000 7f501effd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:40:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.597+0000 7f501effd640 1 -- 172.21.15.38:0/84151967 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503001f040 con 0x5636550512f0 2024-05-21T15:40:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.597+0000 7f501effd640 1 -- 172.21.15.38:0/84151967 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f50300053f0 con 0x5636550512f0 2024-05-21T15:40:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.597+0000 7f504c9f3740 1 -- 172.21.15.38:0/84151967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50340bf0c0 con 0x5636550512f0 2024-05-21T15:40:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.598+0000 7f501effd640 1 -- 172.21.15.38:0/84151967 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5030041050 con 0x5636550512f0 2024-05-21T15:40:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.598+0000 7f504c9f3740 10 client.26659 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.598+0000 7f504c9f3740 1 -- 172.21.15.38:0/84151967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563655499680 con 0x5636550512f0 2024-05-21T15:40:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.599+0000 7f501effd640 1 -- 172.21.15.38:0/84151967 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50340bf0c0 con 0x5636550512f0 2024-05-21T15:40:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.599+0000 7f501effd640 1 -- 172.21.15.38:0/84151967 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f5030018070 con 0x5636550512f0 2024-05-21T15:40:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.599+0000 7f504c9f3740 10 client.26659 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.599+0000 7f504c9f3740 10 client.26659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.599+0000 7f504c9f3740 20 client.26659 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.599+0000 7f504c9f3740 1 --2- 172.21.15.38:0/84151967 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56365500e1c0 0x56365549bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.599+0000 7f504c9f3740 4 client.26659 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:40:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.599+0000 7f504c9f3740 1 -- 172.21.15.38:0/84151967 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563655385dc0 con 0x56365500e1c0 2024-05-21T15:40:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.600+0000 7f503effd640 1 --2- 172.21.15.38:0/84151967 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56365500e1c0 0x56365549bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.600+0000 7f501effd640 10 client.26659 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.600+0000 7f503effd640 1 --2- 172.21.15.38:0/84151967 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56365500e1c0 0x56365549bfb0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbcda0d502c04a66 server_cookie=1c083a0ed016d3be in_seq=0 out_seq=0 2024-05-21T15:40:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.619+0000 7f501effd640 1 -- 172.21.15.38:0/84151967 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563655385dc0 con 0x56365500e1c0 2024-05-21T15:40:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.619+0000 7f501effd640 10 client.26659 handle_command_reply: tid=0 2024-05-21T15:40:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.619+0000 7f504c9f3740 1 client.26659 shutdown 2024-05-21T15:40:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.620+0000 7f504c9f3740 1 -- 172.21.15.38:0/84151967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550512f0 msgr2=0x563655428730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.620+0000 7f504c9f3740 1 --2- 172.21.15.38:0/84151967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550512f0 0x563655428730 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x56365541b240 tx=0x7f5030009e20 comp rx=0 tx=0).stop 2024-05-21T15:40:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.620+0000 7f504c9f3740 1 -- 172.21.15.38:0/84151967 shutdown_connections 2024-05-21T15:40:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.620+0000 7f504c9f3740 1 --2- 172.21.15.38:0/84151967 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56365500e1c0 0x56365549bfb0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.620+0000 7f504c9f3740 1 --2- 172.21.15.38:0/84151967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56365507a1b0 0x563655498020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.620+0000 7f504c9f3740 1 --2- 172.21.15.38:0/84151967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636550594d0 0x563655428c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.620+0000 7f501effd640 0 client.26659 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.620+0000 7f504c9f3740 1 --2- 172.21.15.38:0/84151967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636550512f0 0x563655428730 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.620+0000 7f504c9f3740 1 -- 172.21.15.38:0/84151967 >> 172.21.15.38:0/84151967 conn(0x56365506e300 msgr2=0x563655436470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.620+0000 7f504c9f3740 1 -- 172.21.15.38:0/84151967 shutdown_connections 2024-05-21T15:40:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.620+0000 7f504c9f3740 1 -- 172.21.15.38:0/84151967 wait complete. 2024-05-21T15:40:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.620+0000 7f504c9f3740 20 client.26659 trim_cache size 0 max 16384 2024-05-21T15:40:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:40:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:40:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:40:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:40:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.621+0000 7f5045a79640 1 -- 172.21.15.38:0/1686196839 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f50280a2d80 msgr2=0x7f50280a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.621+0000 7f5045a79640 1 --2- 172.21.15.38:0/1686196839 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f50280a2d80 0x7f50280a5290 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x56365541b030 tx=0x7f5038009460 comp rx=0 tx=0).stop 2024-05-21T15:40:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.621+0000 7f5045a79640 1 -- 172.21.15.38:0/1686196839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50401012e0 msgr2=0x7f5040199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.621+0000 7f5045a79640 1 --2- 172.21.15.38:0/1686196839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50401012e0 0x7f5040199740 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5034009c20 tx=0x7f503400b690 comp rx=0 tx=0).stop 2024-05-21T15:40:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.622+0000 7f5045a79640 1 -- 172.21.15.38:0/1686196839 shutdown_connections 2024-05-21T15:40:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.622+0000 7f5045a79640 1 --2- 172.21.15.38:0/1686196839 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f50280a2d80 0x7f50280a5290 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.622+0000 7f5045a79640 1 --2- 172.21.15.38:0/1686196839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5040102370 0x7f50401a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.622+0000 7f5045a79640 1 --2- 172.21.15.38:0/1686196839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50401012e0 0x7f5040199740 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.622+0000 7f5045a79640 1 --2- 172.21.15.38:0/1686196839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5040069070 0x7f5040199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.622+0000 7f5045a79640 1 -- 172.21.15.38:0/1686196839 >> 172.21.15.38:0/1686196839 conn(0x7f50400fc300 msgr2=0x7f50400fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.622+0000 7f5045a79640 1 -- 172.21.15.38:0/1686196839 shutdown_connections 2024-05-21T15:40:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:15.622+0000 7f5045a79640 1 -- 172.21.15.38:0/1686196839 wait complete. 2024-05-21T15:40:15.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.834+0000 7fe39e626640 1 -- 172.21.15.38:0/753116441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe384005580 con 0x7fe3981068e0 2024-05-21T15:40:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.835+0000 7fe3957fa640 1 -- 172.21.15.38:0/753116441 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11922 (secure 0 0 0) 0x7fe3900c04a0 con 0x7fe3981068e0 2024-05-21T15:40:15.833 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:40:15.834 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:36:59.077696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24551,24523,24602,24550,24515]},"id":1}]} 2024-05-21T15:40:15.834 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 31 2024-05-21T15:40:15.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.838+0000 7fe39e626640 1 -- 172.21.15.38:0/753116441 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe3800a2d80 msgr2=0x7fe3800a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.838+0000 7fe39e626640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe3800a2d80 0x7fe3800a5290 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe388009c90 tx=0x7fe388005f90 comp rx=0 tx=0).stop 2024-05-21T15:40:15.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.838+0000 7fe39e626640 1 -- 172.21.15.38:0/753116441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3981068e0 msgr2=0x7fe3981a6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:15.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.838+0000 7fe39e626640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3981068e0 0x7fe3981a6a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe39002a9e0 tx=0x7fe390003f00 comp rx=0 tx=0).stop 2024-05-21T15:40:15.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.838+0000 7fe39e626640 1 -- 172.21.15.38:0/753116441 shutdown_connections 2024-05-21T15:40:15.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.838+0000 7fe39e626640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3981068e0 0x7fe3981a6a50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.838+0000 7fe39e626640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe3800a2d80 0x7fe3800a5290 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.838+0000 7fe39e626640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe398101770 0x7fe39819fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.838+0000 7fe39e626640 1 --2- 172.21.15.38:0/753116441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe398100d30 0x7fe39819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:15.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.838+0000 7fe39e626640 1 -- 172.21.15.38:0/753116441 >> 172.21.15.38:0/753116441 conn(0x7fe3980fc340 msgr2=0x7fe3980fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:15.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.838+0000 7fe39e626640 1 -- 172.21.15.38:0/753116441 shutdown_connections 2024-05-21T15:40:15.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:15.838+0000 7fe39e626640 1 -- 172.21.15.38:0/753116441 wait complete. 2024-05-21T15:40:15.852 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:40:16.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.008+0000 7f7e2455e640 1 -- 172.21.15.38:0/1585179163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e1c104cc0 msgr2=0x7f7e1c1059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.008+0000 7f7e2455e640 1 --2- 172.21.15.38:0/1585179163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e1c104cc0 0x7f7e1c1059c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009900 tx=0x7f7e1802a410 comp rx=0 tx=0).stop 2024-05-21T15:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.008+0000 7f7e2455e640 1 -- 172.21.15.38:0/1585179163 shutdown_connections 2024-05-21T15:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.008+0000 7f7e2455e640 1 --2- 172.21.15.38:0/1585179163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e1c10ac00 0x7f7e1c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.008+0000 7f7e2455e640 1 --2- 172.21.15.38:0/1585179163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e1c105f00 0x7f7e1c106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.008+0000 7f7e2455e640 1 --2- 172.21.15.38:0/1585179163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e1c104cc0 0x7f7e1c1059c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.008+0000 7f7e2455e640 1 -- 172.21.15.38:0/1585179163 >> 172.21.15.38:0/1585179163 conn(0x7f7e1c0ffeb0 msgr2=0x7f7e1c102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.008+0000 7f7e2455e640 1 -- 172.21.15.38:0/1585179163 shutdown_connections 2024-05-21T15:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.008+0000 7f7e2455e640 1 -- 172.21.15.38:0/1585179163 wait complete. 2024-05-21T15:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.009+0000 7f7e2455e640 1 Processor -- start 2024-05-21T15:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.009+0000 7f7e2455e640 1 -- start start 2024-05-21T15:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.009+0000 7f7e2455e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e1c104cc0 0x7f7e1c19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.009+0000 7f7e2455e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e1c105f00 0x7f7e1c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.009+0000 7f7e2455e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e1c10ac00 0x7f7e1c1a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.009+0000 7f7e2455e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c110960 con 0x7f7e1c10ac00 2024-05-21T15:40:16.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.009+0000 7f7e2455e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c1107e0 con 0x7f7e1c104cc0 2024-05-21T15:40:16.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.009+0000 7f7e2455e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c110ae0 con 0x7f7e1c105f00 2024-05-21T15:40:16.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.009+0000 7f7e222fa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e1c104cc0 0x7f7e1c19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.009+0000 7f7e22afb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e1c10ac00 0x7f7e1c1a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.009+0000 7f7e222fa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e1c104cc0 0x7f7e1c19b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46430/0 (socket says 172.21.15.38:46430) 2024-05-21T15:40:16.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.009+0000 7f7e222fa640 1 -- 172.21.15.38:0/1849817215 learned_addr learned my addr 172.21.15.38:0/1849817215 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.009+0000 7f7e21af9640 1 --2- 172.21.15.38:0/1849817215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e1c105f00 0x7f7e1c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.010+0000 7f7e222fa640 1 -- 172.21.15.38:0/1849817215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e1c105f00 msgr2=0x7f7e1c19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.010+0000 7f7e222fa640 1 --2- 172.21.15.38:0/1849817215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e1c105f00 0x7f7e1c19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.010+0000 7f7e222fa640 1 -- 172.21.15.38:0/1849817215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e1c10ac00 msgr2=0x7f7e1c1a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.010+0000 7f7e222fa640 1 --2- 172.21.15.38:0/1849817215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e1c10ac00 0x7f7e1c1a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.010+0000 7f7e222fa640 1 -- 172.21.15.38:0/1849817215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1c1a2fc0 con 0x7f7e1c104cc0 2024-05-21T15:40:16.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.010+0000 7f7e22afb640 1 --2- 172.21.15.38:0/1849817215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e1c10ac00 0x7f7e1c1a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:16.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.010+0000 7f7e222fa640 1 --2- 172.21.15.38:0/1849817215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e1c104cc0 0x7f7e1c19b480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7e18031440 tx=0x7f7e180045c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.010+0000 7f7e137fe640 1 -- 172.21.15.38:0/1849817215 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18035070 con 0x7f7e1c104cc0 2024-05-21T15:40:16.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.010+0000 7f7e2455e640 1 -- 172.21.15.38:0/1849817215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1c075f90 con 0x7f7e1c104cc0 2024-05-21T15:40:16.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.010+0000 7f7e137fe640 1 -- 172.21.15.38:0/1849817215 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e18041b60 con 0x7f7e1c104cc0 2024-05-21T15:40:16.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.010+0000 7f7e2455e640 1 -- 172.21.15.38:0/1849817215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e1c0764d0 con 0x7f7e1c104cc0 2024-05-21T15:40:16.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.010+0000 7f7e137fe640 1 -- 172.21.15.38:0/1849817215 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e180496c0 con 0x7f7e1c104cc0 2024-05-21T15:40:16.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.013+0000 7f7e137fe640 1 -- 172.21.15.38:0/1849817215 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7e18041d10 con 0x7f7e1c104cc0 2024-05-21T15:40:16.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.014+0000 7f7e2b40a740 1 Processor -- start 2024-05-21T15:40:16.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.014+0000 7f7e2b40a740 1 -- start start 2024-05-21T15:40:16.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.014+0000 7f7e137fe640 1 --2- 172.21.15.38:0/1849817215 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e080a2d90 0x7f7e080a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e2b40a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e55907390 0x555e55ca6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e2b40a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558de4d0 0x555e55ca7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e2b40a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558e66b0 0x555e55cadf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e2b40a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555e55c0bc40 con 0x555e55907390 2024-05-21T15:40:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e2b40a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555e55bb3580 con 0x555e558de4d0 2024-05-21T15:40:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e2b40a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555e55c07c00 con 0x555e558e66b0 2024-05-21T15:40:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e22afb640 1 --2- 172.21.15.38:0/1849817215 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e080a2d90 0x7f7e080a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e21af9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558de4d0 0x555e55ca7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e21af9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558de4d0 0x555e55ca7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46434/0 (socket says 172.21.15.38:46434) 2024-05-21T15:40:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e21af9640 1 -- 172.21.15.38:0/3228994932 learned_addr learned my addr 172.21.15.38:0/3228994932 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e222fa640 1 --2- 172.21.15.38:0/3228994932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e55907390 0x555e55ca6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e22afb640 1 --2- 172.21.15.38:0/3228994932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558e66b0 0x555e55cadf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e137fe640 1 -- 172.21.15.38:0/1849817215 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7e18049460 con 0x7f7e1c104cc0 2024-05-21T15:40:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e21af9640 1 -- 172.21.15.38:0/3228994932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558e66b0 msgr2=0x555e55cadf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e21af9640 1 --2- 172.21.15.38:0/3228994932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558e66b0 0x555e55cadf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e21af9640 1 -- 172.21.15.38:0/3228994932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e55907390 msgr2=0x555e55ca6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e21af9640 1 --2- 172.21.15.38:0/3228994932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e55907390 0x555e55ca6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.015+0000 7f7e21af9640 1 -- 172.21.15.38:0/3228994932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5591f3f0 con 0x555e558de4d0 2024-05-21T15:40:16.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.016+0000 7f7e22afb640 1 --2- 172.21.15.38:0/1849817215 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e080a2d90 0x7f7e080a52a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7e14005fd0 tx=0x7f7e14011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.016+0000 7f7e21af9640 1 --2- 172.21.15.38:0/3228994932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558de4d0 0x555e55ca7080 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009ca0 tx=0x7f7e0c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2238c4edf12f1b8d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.016+0000 7f7e10bfe640 1 -- 172.21.15.38:0/3228994932 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c005220 con 0x555e558de4d0 2024-05-21T15:40:16.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.016+0000 7f7e10bfe640 1 -- 172.21.15.38:0/3228994932 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e0c004c80 con 0x555e558de4d0 2024-05-21T15:40:16.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.016+0000 7f7e10bfe640 1 -- 172.21.15.38:0/3228994932 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c0187d0 con 0x555e558de4d0 2024-05-21T15:40:16.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.017+0000 7f7e2b40a740 1 -- 172.21.15.38:0/3228994932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558de4d0 msgr2=0x555e55ca7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.017+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/3228994932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558de4d0 0x555e55ca7080 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009ca0 tx=0x7f7e0c011810 comp rx=0 tx=0).stop 2024-05-21T15:40:16.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.017+0000 7f7e2b40a740 1 -- 172.21.15.38:0/3228994932 shutdown_connections 2024-05-21T15:40:16.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.017+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/3228994932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558e66b0 0x555e55cadf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.017+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/3228994932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558de4d0 0x555e55ca7080 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.017+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/3228994932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e55907390 0x555e55ca6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.017+0000 7f7e2b40a740 1 -- 172.21.15.38:0/3228994932 >> 172.21.15.38:0/3228994932 conn(0x555e558fb4e0 msgr2=0x555e55ca2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.017+0000 7f7e2b40a740 1 -- 172.21.15.38:0/3228994932 shutdown_connections 2024-05-21T15:40:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.017+0000 7f7e2b40a740 1 -- 172.21.15.38:0/3228994932 wait complete. 2024-05-21T15:40:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.017+0000 7f7e2b40a740 1 -- start start 2024-05-21T15:40:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.017+0000 7f7e2b40a740 1 Processor -- start 2024-05-21T15:40:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e2b40a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558de4d0 0x555e55cb08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e2b40a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558e66b0 0x555e55cb0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e2b40a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e55907390 0x555e55cb7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e2b40a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555e55cae9d0 con 0x555e55907390 2024-05-21T15:40:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e2b40a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555e55bc6c40 con 0x555e558e66b0 2024-05-21T15:40:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e2b40a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555e55caeb50 con 0x555e558de4d0 2024-05-21T15:40:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e2b40a740 10 client.0 _ll_register_callbacks cb 0x555e5591ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e222fa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558de4d0 0x555e55cb08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e222fa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558de4d0 0x555e55cb08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:38916/0 (socket says 172.21.15.38:38916) 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e21af9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558e66b0 0x555e55cb0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e222fa640 1 -- 172.21.15.38:0/1488578632 learned_addr learned my addr 172.21.15.38:0/1488578632 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e22afb640 1 --2- 172.21.15.38:0/1488578632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e55907390 0x555e55cb7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e22afb640 1 -- 172.21.15.38:0/1488578632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558de4d0 msgr2=0x555e55cb08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e22afb640 1 --2- 172.21.15.38:0/1488578632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558de4d0 0x555e55cb08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e22afb640 1 -- 172.21.15.38:0/1488578632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558e66b0 msgr2=0x555e55cb0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e22afb640 1 --2- 172.21.15.38:0/1488578632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558e66b0 0x555e55cb0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.018+0000 7f7e22afb640 1 -- 172.21.15.38:0/1488578632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e559e97d0 con 0x555e55907390 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.019+0000 7f7e22afb640 1 --2- 172.21.15.38:0/1488578632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e55907390 0x555e55cb7d90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400fc20 tx=0x7f7e14009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54853e549d925ec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.019+0000 7f7e05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.019+0000 7f7e05ffb640 1 -- 172.21.15.38:0/1488578632 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14004600 con 0x555e55907390 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.019+0000 7f7e05ffb640 1 -- 172.21.15.38:0/1488578632 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7e140047a0 con 0x555e55907390 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.019+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1488578632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e0c004c80 con 0x555e55907390 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.019+0000 7f7e05ffb640 1 -- 172.21.15.38:0/1488578632 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7e14028430 con 0x555e55907390 2024-05-21T15:40:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.019+0000 7f7e2b40a740 10 client.16983 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.019+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1488578632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555e55cbb0c0 con 0x555e55907390 2024-05-21T15:40:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.019+0000 7f7e05ffb640 1 -- 172.21.15.38:0/1488578632 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e0c004c80 con 0x555e55907390 2024-05-21T15:40:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.021+0000 7f7e05ffb640 1 -- 172.21.15.38:0/1488578632 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f7e1401e050 con 0x555e55907390 2024-05-21T15:40:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.021+0000 7f7e2b40a740 10 client.16983 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.021+0000 7f7e2b40a740 10 client.16983 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:40:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.021+0000 7f7e2b40a740 20 client.16983 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.021+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/1488578632 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x555e558c5f60 0x555e55cbe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.021+0000 7f7e2b40a740 4 client.16983 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.021+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1488578632 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e55c19370 con 0x555e558c5f60 2024-05-21T15:40:16.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.021+0000 7f7e222fa640 1 --2- 172.21.15.38:0/1488578632 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x555e558c5f60 0x555e55cbe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.021+0000 7f7e05ffb640 10 client.16983 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:40:16.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.021+0000 7f7e222fa640 1 --2- 172.21.15.38:0/1488578632 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x555e558c5f60 0x555e55cbe810 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=306ea0529c054453 server_cookie=2f53c91e0e233084 in_seq=0 out_seq=0 2024-05-21T15:40:16.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.023+0000 7f7e05ffb640 1 -- 172.21.15.38:0/1488578632 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555e55c19370 con 0x555e558c5f60 2024-05-21T15:40:16.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.023+0000 7f7e05ffb640 10 client.16983 handle_command_reply: tid=0 2024-05-21T15:40:16.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.023+0000 7f7e2b40a740 1 client.16983 shutdown 2024-05-21T15:40:16.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.023+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1488578632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e55907390 msgr2=0x555e55cb7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.023+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/1488578632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e55907390 0x555e55cb7d90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400fc20 tx=0x7f7e14009500 comp rx=0 tx=0).stop 2024-05-21T15:40:16.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.023+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1488578632 shutdown_connections 2024-05-21T15:40:16.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.023+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/1488578632 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x555e558c5f60 0x555e55cbe810 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.023+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/1488578632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e55907390 0x555e55cb7d90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.023+0000 7f7e05ffb640 0 client.16983 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:40:16.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.023+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/1488578632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558e66b0 0x555e55cb0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.023+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/1488578632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558de4d0 0x555e55cb08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.023+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1488578632 >> 172.21.15.38:0/1488578632 conn(0x555e558fb4e0 msgr2=0x555e55ca1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.024+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1488578632 shutdown_connections 2024-05-21T15:40:16.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.024+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1488578632 wait complete. 2024-05-21T15:40:16.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.024+0000 7f7e2b40a740 20 client.16983 trim_cache size 0 max 16384 2024-05-21T15:40:16.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.045+0000 7f7e2b40a740 1 Processor -- start 2024-05-21T15:40:16.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.045+0000 7f7e2b40a740 1 -- start start 2024-05-21T15:40:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e2b40a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558c5f60 0x555e55caeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e2b40a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558de4d0 0x555e55caf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e2b40a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e558e66b0 0x555e55ca1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e2b40a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c1a7fe0 con 0x555e558e66b0 2024-05-21T15:40:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e2b40a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c1a9c10 con 0x555e558c5f60 2024-05-21T15:40:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e2b40a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c1a86b0 con 0x555e558de4d0 2024-05-21T15:40:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e21af9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558de4d0 0x555e55caf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e222fa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558c5f60 0x555e55caeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e21af9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558de4d0 0x555e55caf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:38928/0 (socket says 172.21.15.38:38928) 2024-05-21T15:40:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e21af9640 1 -- 172.21.15.38:0/525752660 learned_addr learned my addr 172.21.15.38:0/525752660 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e222fa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558c5f60 0x555e55caeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46462/0 (socket says 172.21.15.38:46462) 2024-05-21T15:40:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e22afb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e558e66b0 0x555e55ca1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e22afb640 1 -- 172.21.15.38:0/525752660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558de4d0 msgr2=0x555e55caf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e22afb640 1 --2- 172.21.15.38:0/525752660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558de4d0 0x555e55caf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e22afb640 1 -- 172.21.15.38:0/525752660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558c5f60 msgr2=0x555e55caeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e22afb640 1 --2- 172.21.15.38:0/525752660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558c5f60 0x555e55caeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e22afb640 1 -- 172.21.15.38:0/525752660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e55caf970 con 0x555e558e66b0 2024-05-21T15:40:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.046+0000 7f7e222fa640 1 --2- 172.21.15.38:0/525752660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558c5f60 0x555e55caeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.047+0000 7f7e22afb640 1 --2- 172.21.15.38:0/525752660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e558e66b0 0x555e55ca1ca0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7e14032930 tx=0x7f7e140049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9ca4422b58a686 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.047+0000 7f7e07fff640 1 -- 172.21.15.38:0/525752660 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14032450 con 0x555e558e66b0 2024-05-21T15:40:16.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.047+0000 7f7e07fff640 1 -- 172.21.15.38:0/525752660 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e14037020 con 0x555e558e66b0 2024-05-21T15:40:16.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.047+0000 7f7e07fff640 1 -- 172.21.15.38:0/525752660 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14011040 con 0x555e558e66b0 2024-05-21T15:40:16.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.048+0000 7f7e2b40a740 1 -- 172.21.15.38:0/525752660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e558e66b0 msgr2=0x555e55ca1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.048+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/525752660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e558e66b0 0x555e55ca1ca0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7e14032930 tx=0x7f7e140049c0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.048+0000 7f7e2b40a740 1 -- 172.21.15.38:0/525752660 shutdown_connections 2024-05-21T15:40:16.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.048+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/525752660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e558e66b0 0x555e55ca1ca0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.048+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/525752660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558de4d0 0x555e55caf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.048+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/525752660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558c5f60 0x555e55caeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.048+0000 7f7e2b40a740 1 -- 172.21.15.38:0/525752660 >> 172.21.15.38:0/525752660 conn(0x555e558fb4e0 msgr2=0x555e55caad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.048+0000 7f7e2b40a740 1 -- 172.21.15.38:0/525752660 shutdown_connections 2024-05-21T15:40:16.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.048+0000 7f7e2b40a740 1 -- 172.21.15.38:0/525752660 wait complete. 2024-05-21T15:40:16.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.049+0000 7f7e2b40a740 1 -- start start 2024-05-21T15:40:16.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.049+0000 7f7e2b40a740 1 Processor -- start 2024-05-21T15:40:16.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.049+0000 7f7e2b40a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e558c5f60 0x555e55cb42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.049+0000 7f7e2b40a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558de4d0 0x555e55cb4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.049+0000 7f7e2b40a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558e66b0 0x555e55d24b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.049+0000 7f7e2b40a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c1a9480 con 0x555e558c5f60 2024-05-21T15:40:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.049+0000 7f7e2b40a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c1a8d80 con 0x555e558de4d0 2024-05-21T15:40:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.049+0000 7f7e2b40a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c1a6070 con 0x555e558e66b0 2024-05-21T15:40:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.049+0000 7f7e2b40a740 10 client.0 _ll_register_callbacks cb 0x7f7e140119f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.049+0000 7f7e222fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e558c5f60 0x555e55cb42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.049+0000 7f7e22afb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558e66b0 0x555e55d24b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.049+0000 7f7e21af9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558de4d0 0x555e55cb4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.049+0000 7f7e222fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e558c5f60 0x555e55cb42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59206/0 (socket says 172.21.15.38:59206) 2024-05-21T15:40:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.049+0000 7f7e222fa640 1 -- 172.21.15.38:0/1656908547 learned_addr learned my addr 172.21.15.38:0/1656908547 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.050+0000 7f7e222fa640 1 -- 172.21.15.38:0/1656908547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558e66b0 msgr2=0x555e55d24b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.050+0000 7f7e222fa640 1 --2- 172.21.15.38:0/1656908547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558e66b0 0x555e55d24b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.050+0000 7f7e222fa640 1 -- 172.21.15.38:0/1656908547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558de4d0 msgr2=0x555e55cb4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.050+0000 7f7e222fa640 1 --2- 172.21.15.38:0/1656908547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558de4d0 0x555e55cb4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.050+0000 7f7e222fa640 1 -- 172.21.15.38:0/1656908547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e55cb4e20 con 0x555e558c5f60 2024-05-21T15:40:16.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.050+0000 7f7e22afb640 1 --2- 172.21.15.38:0/1656908547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558e66b0 0x555e55d24b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:40:16.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.050+0000 7f7e222fa640 1 --2- 172.21.15.38:0/1656908547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e558c5f60 0x555e55cb42d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7e180f5b60 tx=0x7f7e180a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198c21f897e45730 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.050+0000 7f7e05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:16.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.050+0000 7f7e05ffb640 1 -- 172.21.15.38:0/1656908547 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e180c8900 con 0x555e558c5f60 2024-05-21T15:40:16.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.050+0000 7f7e05ffb640 1 -- 172.21.15.38:0/1656908547 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f7e180c2850 con 0x555e558c5f60 2024-05-21T15:40:16.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.050+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1656908547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e14037020 con 0x555e558c5f60 2024-05-21T15:40:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.051+0000 7f7e05ffb640 1 -- 172.21.15.38:0/1656908547 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7e180cd1c0 con 0x555e558c5f60 2024-05-21T15:40:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.051+0000 7f7e2b40a740 10 client.16989 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.051+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1656908547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555e55d261d0 con 0x555e558c5f60 2024-05-21T15:40:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.051+0000 7f7e05ffb640 1 -- 172.21.15.38:0/1656908547 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e14037020 con 0x555e558c5f60 2024-05-21T15:40:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.052+0000 7f7e05ffb640 1 -- 172.21.15.38:0/1656908547 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f7e180b3b80 con 0x555e558c5f60 2024-05-21T15:40:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.052+0000 7f7e2b40a740 10 client.16989 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.052+0000 7f7e2b40a740 10 client.16989 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:40:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.052+0000 7f7e2b40a740 20 client.16989 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.052+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/1656908547 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x555e55907390 0x555e55d28b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.052+0000 7f7e2b40a740 4 client.16989 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:40:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.052+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1656908547 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e55c136c0 con 0x555e55907390 2024-05-21T15:40:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.052+0000 7f7e21af9640 1 --2- 172.21.15.38:0/1656908547 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x555e55907390 0x555e55d28b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.053+0000 7f7e21af9640 1 --2- 172.21.15.38:0/1656908547 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x555e55907390 0x555e55d28b90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dd4dc5d4c4baa210 server_cookie=b2120fcbb750772b in_seq=0 out_seq=0 2024-05-21T15:40:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.053+0000 7f7e05ffb640 10 client.16989 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:40:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.053+0000 7f7e05ffb640 1 -- 172.21.15.38:0/1656908547 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e55c136c0 con 0x555e55907390 2024-05-21T15:40:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.053+0000 7f7e05ffb640 10 client.16989 handle_command_reply: tid=0 2024-05-21T15:40:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.053+0000 7f7e2b40a740 1 client.16989 shutdown 2024-05-21T15:40:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.053+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1656908547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e558c5f60 msgr2=0x555e55cb42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.053+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/1656908547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e558c5f60 0x555e55cb42d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7e180f5b60 tx=0x7f7e180a9040 comp rx=0 tx=0).stop 2024-05-21T15:40:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.053+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1656908547 shutdown_connections 2024-05-21T15:40:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.053+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/1656908547 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x555e55907390 0x555e55d28b90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.053+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/1656908547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555e558e66b0 0x555e55d24b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.053+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/1656908547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555e558de4d0 0x555e55cb4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.054+0000 7f7e2b40a740 1 --2- 172.21.15.38:0/1656908547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555e558c5f60 0x555e55cb42d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.054+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1656908547 >> 172.21.15.38:0/1656908547 conn(0x555e558fb4e0 msgr2=0x555e55cb08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.054+0000 7f7e05ffb640 0 client.16989 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:40:16.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.054+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1656908547 shutdown_connections 2024-05-21T15:40:16.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.054+0000 7f7e2b40a740 1 -- 172.21.15.38:0/1656908547 wait complete. 2024-05-21T15:40:16.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.054+0000 7f7e2b40a740 20 client.16989 trim_cache size 0 max 16384 2024-05-21T15:40:16.055 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:40:16.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.055+0000 7f7e2455e640 1 -- 172.21.15.38:0/1849817215 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e080a2d90 msgr2=0x7f7e080a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.055+0000 7f7e2455e640 1 --2- 172.21.15.38:0/1849817215 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e080a2d90 0x7f7e080a52a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7e14005fd0 tx=0x7f7e14011720 comp rx=0 tx=0).stop 2024-05-21T15:40:16.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.055+0000 7f7e2455e640 1 -- 172.21.15.38:0/1849817215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e1c104cc0 msgr2=0x7f7e1c19b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.055+0000 7f7e2455e640 1 --2- 172.21.15.38:0/1849817215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e1c104cc0 0x7f7e1c19b480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7e18031440 tx=0x7f7e180045c0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.055+0000 7f7e2455e640 1 -- 172.21.15.38:0/1849817215 shutdown_connections 2024-05-21T15:40:16.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.055+0000 7f7e2455e640 1 --2- 172.21.15.38:0/1849817215 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f7e080a2d90 0x7f7e080a52a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.055+0000 7f7e2455e640 1 --2- 172.21.15.38:0/1849817215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e1c10ac00 0x7f7e1c1a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.055+0000 7f7e2455e640 1 --2- 172.21.15.38:0/1849817215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e1c105f00 0x7f7e1c19b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.055+0000 7f7e2455e640 1 --2- 172.21.15.38:0/1849817215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e1c104cc0 0x7f7e1c19b480 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.055+0000 7f7e2455e640 1 -- 172.21.15.38:0/1849817215 >> 172.21.15.38:0/1849817215 conn(0x7f7e1c0ffeb0 msgr2=0x7f7e1c106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.055+0000 7f7e2455e640 1 -- 172.21.15.38:0/1849817215 shutdown_connections 2024-05-21T15:40:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.055+0000 7f7e2455e640 1 -- 172.21.15.38:0/1849817215 wait complete. 2024-05-21T15:40:16.069 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:40:16.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:15 smithi053 ceph-mon[36496]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 4.85k op/s 2024-05-21T15:40:16.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:15 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/753116441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:40:16.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.230+0000 7fed85c73640 1 -- 172.21.15.38:0/2997217816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed80102ab0 msgr2=0x7fed80102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.230+0000 7fed85c73640 1 --2- 172.21.15.38:0/2997217816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed80102ab0 0x7fed80102f80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fed74009900 tx=0x7fed7402a410 comp rx=0 tx=0).stop 2024-05-21T15:40:16.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.230+0000 7fed85c73640 1 -- 172.21.15.38:0/2997217816 shutdown_connections 2024-05-21T15:40:16.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.230+0000 7fed85c73640 1 --2- 172.21.15.38:0/2997217816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed80103610 0x7fed80109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.230+0000 7fed85c73640 1 --2- 172.21.15.38:0/2997217816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed80102ab0 0x7fed80102f80 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.230+0000 7fed85c73640 1 --2- 172.21.15.38:0/2997217816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80101f50 0x7fed80102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.230+0000 7fed85c73640 1 -- 172.21.15.38:0/2997217816 >> 172.21.15.38:0/2997217816 conn(0x7fed800fc2f0 msgr2=0x7fed800fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.230+0000 7fed85c73640 1 -- 172.21.15.38:0/2997217816 shutdown_connections 2024-05-21T15:40:16.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.230+0000 7fed85c73640 1 -- 172.21.15.38:0/2997217816 wait complete. 2024-05-21T15:40:16.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.231+0000 7fed85c73640 1 Processor -- start 2024-05-21T15:40:16.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.231+0000 7fed85c73640 1 -- start start 2024-05-21T15:40:16.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed85c73640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80101f50 0x7fed80196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed85c73640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed80102ab0 0x7fed801973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed85c73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed8019c340 0x7fed8019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed85c73640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed8010c720 con 0x7fed8019c340 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed85c73640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fed8010c5a0 con 0x7fed80101f50 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed85c73640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed8010c8a0 con 0x7fed80102ab0 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed7ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed8019c340 0x7fed8019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed7f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80101f50 0x7fed80196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed7effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed80102ab0 0x7fed801973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed7ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed8019c340 0x7fed8019e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59220/0 (socket says 172.21.15.38:59220) 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed7f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80101f50 0x7fed80196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46486/0 (socket says 172.21.15.38:46486) 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed7ffff640 1 -- 172.21.15.38:0/2176994117 learned_addr learned my addr 172.21.15.38:0/2176994117 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed7ffff640 1 -- 172.21.15.38:0/2176994117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed80102ab0 msgr2=0x7fed801973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed7ffff640 1 --2- 172.21.15.38:0/2176994117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed80102ab0 0x7fed801973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed7ffff640 1 -- 172.21.15.38:0/2176994117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80101f50 msgr2=0x7fed80196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed7ffff640 1 --2- 172.21.15.38:0/2176994117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80101f50 0x7fed80196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.232+0000 7fed7ffff640 1 -- 172.21.15.38:0/2176994117 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8019edd0 con 0x7fed8019c340 2024-05-21T15:40:16.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.233+0000 7fed7ffff640 1 --2- 172.21.15.38:0/2176994117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed8019c340 0x7fed8019e760 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fed7800eb10 tx=0x7fed7800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.233+0000 7fed7cff9640 1 -- 172.21.15.38:0/2176994117 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78004b20 con 0x7fed8019c340 2024-05-21T15:40:16.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.233+0000 7fed7cff9640 1 -- 172.21.15.38:0/2176994117 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed78004cc0 con 0x7fed8019c340 2024-05-21T15:40:16.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.233+0000 7fed7cff9640 1 -- 172.21.15.38:0/2176994117 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78010760 con 0x7fed8019c340 2024-05-21T15:40:16.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.233+0000 7fed85c73640 1 -- 172.21.15.38:0/2176994117 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8019f0c0 con 0x7fed8019c340 2024-05-21T15:40:16.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.233+0000 7fed85c73640 1 -- 172.21.15.38:0/2176994117 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8019f5c0 con 0x7fed8019c340 2024-05-21T15:40:16.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.237+0000 7fed8cbfc740 1 Processor -- start 2024-05-21T15:40:16.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.237+0000 7fed7cff9640 1 -- 172.21.15.38:0/2176994117 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fed78005020 con 0x7fed8019c340 2024-05-21T15:40:16.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.237+0000 7fed8cbfc740 1 -- start start 2024-05-21T15:40:16.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed8cbfc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641001d390 0x5564103c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed8cbfc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed8cbfc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fffc6b0 0x5564103cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed8cbfc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556410321c40 con 0x55640fff44d0 2024-05-21T15:40:16.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed8cbfc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5564102c9580 con 0x55641001d390 2024-05-21T15:40:16.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed8cbfc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55641031dc00 con 0x55640fffc6b0 2024-05-21T15:40:16.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59234/0 (socket says 172.21.15.38:59234) 2024-05-21T15:40:16.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed7f7fe640 1 -- 172.21.15.38:0/2627396284 learned_addr learned my addr 172.21.15.38:0/2627396284 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed7effd640 1 --2- 172.21.15.38:0/2627396284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fffc6b0 0x5564103cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed7effd640 1 --2- 172.21.15.38:0/2627396284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641001d390 0x5564103c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed7effd640 1 -- 172.21.15.38:0/2627396284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641001d390 msgr2=0x5564103c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed7effd640 1 --2- 172.21.15.38:0/2627396284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641001d390 0x5564103c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed7effd640 1 -- 172.21.15.38:0/2627396284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 msgr2=0x5564103c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed7effd640 1 --2- 172.21.15.38:0/2627396284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.238+0000 7fed7effd640 1 -- 172.21.15.38:0/2627396284 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564100353f0 con 0x55640fffc6b0 2024-05-21T15:40:16.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.239+0000 7fed7cff9640 1 --2- 172.21.15.38:0/2176994117 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fed680a2b90 0x7fed680a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.239+0000 7fed7f7fe640 1 --2- 172.21.15.38:0/2176994117 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fed680a2b90 0x7fed680a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.239+0000 7fed7cff9640 1 -- 172.21.15.38:0/2176994117 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fed780cf730 con 0x7fed8019c340 2024-05-21T15:40:16.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.239+0000 7fed7effd640 1 --2- 172.21.15.38:0/2627396284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fffc6b0 0x5564103cc6b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fed74005ab0 tx=0x7fed74000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b373551ae3e63011 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.239+0000 7fed7f7fe640 1 --2- 172.21.15.38:0/2176994117 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fed680a2b90 0x7fed680a50a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5564103bbe30 tx=0x7fed70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.239+0000 7fed65ffb640 1 -- 172.21.15.38:0/2627396284 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74035070 con 0x55640fffc6b0 2024-05-21T15:40:16.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.240+0000 7fed65ffb640 1 -- 172.21.15.38:0/2627396284 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed740035f0 con 0x55640fffc6b0 2024-05-21T15:40:16.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.240+0000 7fed65ffb640 1 -- 172.21.15.38:0/2627396284 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7403d670 con 0x55640fffc6b0 2024-05-21T15:40:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.241+0000 7fed8cbfc740 1 -- 172.21.15.38:0/2627396284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fffc6b0 msgr2=0x5564103cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.241+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/2627396284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fffc6b0 0x5564103cc6b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fed74005ab0 tx=0x7fed74000bc0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.241+0000 7fed8cbfc740 1 -- 172.21.15.38:0/2627396284 shutdown_connections 2024-05-21T15:40:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.241+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/2627396284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fffc6b0 0x5564103cc6b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.241+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/2627396284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.241+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/2627396284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641001d390 0x5564103c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.241+0000 7fed8cbfc740 1 -- 172.21.15.38:0/2627396284 >> 172.21.15.38:0/2627396284 conn(0x5564100114e0 msgr2=0x5564103b8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.241+0000 7fed8cbfc740 1 -- 172.21.15.38:0/2627396284 shutdown_connections 2024-05-21T15:40:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.241+0000 7fed8cbfc740 1 -- 172.21.15.38:0/2627396284 wait complete. 2024-05-21T15:40:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.242+0000 7fed8cbfc740 1 -- start start 2024-05-21T15:40:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.242+0000 7fed8cbfc740 1 Processor -- start 2024-05-21T15:40:16.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.242+0000 7fed8cbfc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fff44d0 0x5564103cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.242+0000 7fed8cbfc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fffc6b0 0x5564103cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.242+0000 7fed8cbfc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641001d390 0x5564103d6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed7effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fffc6b0 0x5564103cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed7effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fffc6b0 0x5564103cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59242/0 (socket says 172.21.15.38:59242) 2024-05-21T15:40:16.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed7f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fff44d0 0x5564103cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed7effd640 1 -- 172.21.15.38:0/273543288 learned_addr learned my addr 172.21.15.38:0/273543288 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed8cbfc740 1 -- 172.21.15.38:0/273543288 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5564103cd170 con 0x55640fffc6b0 2024-05-21T15:40:16.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed8cbfc740 1 -- 172.21.15.38:0/273543288 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5564102dcc40 con 0x55641001d390 2024-05-21T15:40:16.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed8cbfc740 1 -- 172.21.15.38:0/273543288 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5564103cd2f0 con 0x55640fff44d0 2024-05-21T15:40:16.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed8cbfc740 10 client.0 _ll_register_callbacks cb 0x556410034be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:16.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed7ffff640 1 --2- 172.21.15.38:0/273543288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641001d390 0x5564103d6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed7f7fe640 1 -- 172.21.15.38:0/273543288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641001d390 msgr2=0x5564103d6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed7f7fe640 1 --2- 172.21.15.38:0/273543288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641001d390 0x5564103d6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed7f7fe640 1 -- 172.21.15.38:0/273543288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fffc6b0 msgr2=0x5564103cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed7f7fe640 1 --2- 172.21.15.38:0/273543288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fffc6b0 0x5564103cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed7f7fe640 1 -- 172.21.15.38:0/273543288 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564100ff7d0 con 0x55640fff44d0 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed7effd640 1 --2- 172.21.15.38:0/273543288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fffc6b0 0x5564103cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed7f7fe640 1 --2- 172.21.15.38:0/273543288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fff44d0 0x5564103cf1c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed70013010 tx=0x7fed70004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=390ee5b81ddc8ea9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.243+0000 7fed5effd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.244+0000 7fed5effd640 1 -- 172.21.15.38:0/273543288 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70009ad0 con 0x55640fff44d0 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.244+0000 7fed5effd640 1 -- 172.21.15.38:0/273543288 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fed7001f630 con 0x55640fff44d0 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.244+0000 7fed8cbfc740 1 -- 172.21.15.38:0/273543288 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed740035f0 con 0x55640fff44d0 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.244+0000 7fed5effd640 1 -- 172.21.15.38:0/273543288 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fed7001f830 con 0x55640fff44d0 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.244+0000 7fed8cbfc740 10 client.26689 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.244+0000 7fed8cbfc740 1 -- 172.21.15.38:0/273543288 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5564103d99b0 con 0x55640fff44d0 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.245+0000 7fed5effd640 1 -- 172.21.15.38:0/273543288 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed740035f0 con 0x55640fff44d0 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.245+0000 7fed5effd640 1 -- 172.21.15.38:0/273543288 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fed70027070 con 0x55640fff44d0 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.245+0000 7fed8cbfc740 10 client.26689 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.245+0000 7fed8cbfc740 10 client.26689 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:40:16.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.245+0000 7fed8cbfc740 20 client.26689 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:16.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.245+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/273543288 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55640ffdbf60 0x5564103dd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.245+0000 7fed8cbfc740 4 client.26689 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:16.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.245+0000 7fed8cbfc740 1 -- 172.21.15.38:0/273543288 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55641032f370 con 0x55640ffdbf60 2024-05-21T15:40:16.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.245+0000 7fed7effd640 1 --2- 172.21.15.38:0/273543288 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55640ffdbf60 0x5564103dd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.246+0000 7fed7effd640 1 --2- 172.21.15.38:0/273543288 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55640ffdbf60 0x5564103dd0d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=62fa23491c5e3bdf server_cookie=ae7b1d996fbedb09 in_seq=0 out_seq=0 2024-05-21T15:40:16.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.246+0000 7fed5effd640 10 client.26689 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:40:16.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.246+0000 7fed5effd640 1 -- 172.21.15.38:0/273543288 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55641032f370 con 0x55640ffdbf60 2024-05-21T15:40:16.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.246+0000 7fed5effd640 10 client.26689 handle_command_reply: tid=0 2024-05-21T15:40:16.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.247+0000 7fed8cbfc740 1 client.26689 shutdown 2024-05-21T15:40:16.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.247+0000 7fed8cbfc740 1 -- 172.21.15.38:0/273543288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fff44d0 msgr2=0x5564103cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.247+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/273543288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fff44d0 0x5564103cf1c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed70013010 tx=0x7fed70004730 comp rx=0 tx=0).stop 2024-05-21T15:40:16.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.247+0000 7fed8cbfc740 1 -- 172.21.15.38:0/273543288 shutdown_connections 2024-05-21T15:40:16.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.247+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/273543288 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55640ffdbf60 0x5564103dd0d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.247+0000 7fed5effd640 0 client.26689 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:40:16.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.247+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/273543288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55641001d390 0x5564103d6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.247+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/273543288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fffc6b0 0x5564103cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.247+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/273543288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fff44d0 0x5564103cf1c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.247+0000 7fed8cbfc740 1 -- 172.21.15.38:0/273543288 >> 172.21.15.38:0/273543288 conn(0x5564100114e0 msgr2=0x5564103b7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.247+0000 7fed8cbfc740 1 -- 172.21.15.38:0/273543288 shutdown_connections 2024-05-21T15:40:16.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.247+0000 7fed8cbfc740 1 -- 172.21.15.38:0/273543288 wait complete. 2024-05-21T15:40:16.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.247+0000 7fed8cbfc740 20 client.26689 trim_cache size 0 max 16384 2024-05-21T15:40:16.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:15 smithi045 ceph-mon[36409]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 4.85k op/s 2024-05-21T15:40:16.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:15 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/753116441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:40:16.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.268+0000 7fed8cbfc740 1 Processor -- start 2024-05-21T15:40:16.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.269+0000 7fed8cbfc740 1 -- start start 2024-05-21T15:40:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.269+0000 7fed8cbfc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55640ffdbf60 0x5564103e2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.269+0000 7fed8cbfc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103d65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.269+0000 7fed8cbfc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fffc6b0 0x5564103e2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.269+0000 7fed8cbfc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed801a82c0 con 0x55640fff44d0 2024-05-21T15:40:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.269+0000 7fed8cbfc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fed801a9ef0 con 0x55640ffdbf60 2024-05-21T15:40:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.269+0000 7fed8cbfc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed801a8990 con 0x55640fffc6b0 2024-05-21T15:40:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.269+0000 7fed7effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103d65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.269+0000 7fed7effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103d65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59246/0 (socket says 172.21.15.38:59246) 2024-05-21T15:40:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.269+0000 7fed7effd640 1 -- 172.21.15.38:0/3093792899 learned_addr learned my addr 172.21.15.38:0/3093792899 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.269+0000 7fed7f7fe640 1 --2- 172.21.15.38:0/3093792899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55640ffdbf60 0x5564103e2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.269+0000 7fed7ffff640 1 --2- 172.21.15.38:0/3093792899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fffc6b0 0x5564103e2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.270+0000 7fed7effd640 1 -- 172.21.15.38:0/3093792899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fffc6b0 msgr2=0x5564103e2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.270+0000 7fed7effd640 1 --2- 172.21.15.38:0/3093792899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fffc6b0 0x5564103e2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.270+0000 7fed7effd640 1 -- 172.21.15.38:0/3093792899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55640ffdbf60 msgr2=0x5564103e2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.270+0000 7fed7effd640 1 --2- 172.21.15.38:0/3093792899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55640ffdbf60 0x5564103e2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.270+0000 7fed7effd640 1 -- 172.21.15.38:0/3093792899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564103b3d20 con 0x55640fff44d0 2024-05-21T15:40:16.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.270+0000 7fed7effd640 1 --2- 172.21.15.38:0/3093792899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103d65f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fed74005ab0 tx=0x7fed74005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40eae298f5dad2c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.270+0000 7fed657fa640 1 -- 172.21.15.38:0/3093792899 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74035050 con 0x55640fff44d0 2024-05-21T15:40:16.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.270+0000 7fed657fa640 1 -- 172.21.15.38:0/3093792899 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed74005500 con 0x55640fff44d0 2024-05-21T15:40:16.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.270+0000 7fed657fa640 1 -- 172.21.15.38:0/3093792899 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed740038e0 con 0x55640fff44d0 2024-05-21T15:40:16.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.271+0000 7fed8cbfc740 1 -- 172.21.15.38:0/3093792899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 msgr2=0x5564103d65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.271+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/3093792899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103d65f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fed74005ab0 tx=0x7fed74005dd0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.272+0000 7fed8cbfc740 1 -- 172.21.15.38:0/3093792899 shutdown_connections 2024-05-21T15:40:16.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.272+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/3093792899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640fffc6b0 0x5564103e2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.272+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/3093792899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103d65f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.272+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/3093792899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55640ffdbf60 0x5564103e2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.272+0000 7fed8cbfc740 1 -- 172.21.15.38:0/3093792899 >> 172.21.15.38:0/3093792899 conn(0x5564100114e0 msgr2=0x5564103c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.272+0000 7fed8cbfc740 1 -- 172.21.15.38:0/3093792899 shutdown_connections 2024-05-21T15:40:16.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.272+0000 7fed8cbfc740 1 -- 172.21.15.38:0/3093792899 wait complete. 2024-05-21T15:40:16.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.273+0000 7fed8cbfc740 1 -- start start 2024-05-21T15:40:16.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.273+0000 7fed8cbfc740 1 Processor -- start 2024-05-21T15:40:16.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.274+0000 7fed8cbfc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640ffdbf60 0x5564103d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.274+0000 7fed8cbfc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103d5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.274+0000 7fed8cbfc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55640fffc6b0 0x5564103d59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.274+0000 7fed8cbfc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed801a9760 con 0x55640fff44d0 2024-05-21T15:40:16.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.274+0000 7fed8cbfc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fed801a9060 con 0x55640fffc6b0 2024-05-21T15:40:16.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.274+0000 7fed8cbfc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed801a62a0 con 0x55640ffdbf60 2024-05-21T15:40:16.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.274+0000 7fed8cbfc740 10 client.0 _ll_register_callbacks cb 0x7fed70009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:16.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.274+0000 7fed7effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103d5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.274+0000 7fed7f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640ffdbf60 0x5564103d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.274+0000 7fed7effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103d5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59252/0 (socket says 172.21.15.38:59252) 2024-05-21T15:40:16.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.274+0000 7fed7effd640 1 -- 172.21.15.38:0/4153464682 learned_addr learned my addr 172.21.15.38:0/4153464682 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.274+0000 7fed7ffff640 1 --2- 172.21.15.38:0/4153464682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55640fffc6b0 0x5564103d59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:15 smithi038 ceph-mon[31995]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 4.85k op/s 2024-05-21T15:40:16.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:15 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/753116441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:40:16.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.275+0000 7fed7effd640 1 -- 172.21.15.38:0/4153464682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640ffdbf60 msgr2=0x5564103d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.275+0000 7fed7effd640 1 --2- 172.21.15.38:0/4153464682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640ffdbf60 0x5564103d4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.275+0000 7fed7effd640 1 -- 172.21.15.38:0/4153464682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55640fffc6b0 msgr2=0x5564103d59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.275+0000 7fed7effd640 1 --2- 172.21.15.38:0/4153464682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55640fffc6b0 0x5564103d59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.275+0000 7fed7effd640 1 -- 172.21.15.38:0/4153464682 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556410443430 con 0x55640fff44d0 2024-05-21T15:40:16.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.275+0000 7fed7effd640 1 --2- 172.21.15.38:0/4153464682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103d5460 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fed74056040 tx=0x7fed74031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04e7537bcac9e4a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.275+0000 7fed5effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:16.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.275+0000 7fed5effd640 1 -- 172.21.15.38:0/4153464682 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed740543e0 con 0x55640fff44d0 2024-05-21T15:40:16.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.275+0000 7fed5effd640 1 -- 172.21.15.38:0/4153464682 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fed74054580 con 0x55640fff44d0 2024-05-21T15:40:16.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.276+0000 7fed8cbfc740 1 -- 172.21.15.38:0/4153464682 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed74005500 con 0x55640fff44d0 2024-05-21T15:40:16.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.276+0000 7fed5effd640 1 -- 172.21.15.38:0/4153464682 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fed74054780 con 0x55640fff44d0 2024-05-21T15:40:16.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.276+0000 7fed8cbfc740 10 client.17007 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:16.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.276+0000 7fed8cbfc740 1 -- 172.21.15.38:0/4153464682 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556410444640 con 0x55640fff44d0 2024-05-21T15:40:16.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.277+0000 7fed5effd640 1 -- 172.21.15.38:0/4153464682 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed74054920 con 0x55640fff44d0 2024-05-21T15:40:16.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.278+0000 7fed5effd640 1 -- 172.21.15.38:0/4153464682 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fed7403d450 con 0x55640fff44d0 2024-05-21T15:40:16.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.278+0000 7fed8cbfc740 10 client.17007 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:16.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.278+0000 7fed8cbfc740 10 client.17007 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:40:16.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.278+0000 7fed8cbfc740 20 client.17007 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:16.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.278+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/4153464682 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55641001d390 0x556410446be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.278+0000 7fed8cbfc740 4 client.17007 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:40:16.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.278+0000 7fed8cbfc740 1 -- 172.21.15.38:0/4153464682 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fed74005500 con 0x55641001d390 2024-05-21T15:40:16.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.279+0000 7fed7f7fe640 1 --2- 172.21.15.38:0/4153464682 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55641001d390 0x556410446be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.279+0000 7fed7f7fe640 1 --2- 172.21.15.38:0/4153464682 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55641001d390 0x556410446be0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b319ffc21366f8fb server_cookie=5daf225733341a7d in_seq=0 out_seq=0 2024-05-21T15:40:16.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.279+0000 7fed5effd640 10 client.17007 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:40:16.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.279+0000 7fed5effd640 1 -- 172.21.15.38:0/4153464682 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fed74005500 con 0x55641001d390 2024-05-21T15:40:16.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.279+0000 7fed5effd640 10 client.17007 handle_command_reply: tid=0 2024-05-21T15:40:16.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.279+0000 7fed8cbfc740 1 client.17007 shutdown 2024-05-21T15:40:16.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.280+0000 7fed8cbfc740 1 -- 172.21.15.38:0/4153464682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 msgr2=0x5564103d5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.280+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/4153464682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103d5460 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fed74056040 tx=0x7fed74031400 comp rx=0 tx=0).stop 2024-05-21T15:40:16.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.280+0000 7fed8cbfc740 1 -- 172.21.15.38:0/4153464682 shutdown_connections 2024-05-21T15:40:16.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.280+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/4153464682 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x55641001d390 0x556410446be0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.280+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/4153464682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55640fffc6b0 0x5564103d59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.280+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/4153464682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55640fff44d0 0x5564103d5460 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.280+0000 7fed5effd640 0 client.17007 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:40:16.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.280+0000 7fed8cbfc740 1 --2- 172.21.15.38:0/4153464682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55640ffdbf60 0x5564103d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.280+0000 7fed8cbfc740 1 -- 172.21.15.38:0/4153464682 >> 172.21.15.38:0/4153464682 conn(0x5564100114e0 msgr2=0x5564103cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.280+0000 7fed8cbfc740 1 -- 172.21.15.38:0/4153464682 shutdown_connections 2024-05-21T15:40:16.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.280+0000 7fed8cbfc740 1 -- 172.21.15.38:0/4153464682 wait complete. 2024-05-21T15:40:16.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.281+0000 7fed8cbfc740 20 client.17007 trim_cache size 0 max 16384 2024-05-21T15:40:16.280 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:40:16.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.281+0000 7fed85c73640 1 -- 172.21.15.38:0/2176994117 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fed680a2b90 msgr2=0x7fed680a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.281+0000 7fed85c73640 1 --2- 172.21.15.38:0/2176994117 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fed680a2b90 0x7fed680a50a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5564103bbe30 tx=0x7fed70005f90 comp rx=0 tx=0).stop 2024-05-21T15:40:16.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.281+0000 7fed85c73640 1 -- 172.21.15.38:0/2176994117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed8019c340 msgr2=0x7fed8019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.281+0000 7fed85c73640 1 --2- 172.21.15.38:0/2176994117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed8019c340 0x7fed8019e760 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fed7800eb10 tx=0x7fed7800c2d0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.281+0000 7fed85c73640 1 -- 172.21.15.38:0/2176994117 shutdown_connections 2024-05-21T15:40:16.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.281+0000 7fed85c73640 1 --2- 172.21.15.38:0/2176994117 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fed680a2b90 0x7fed680a50a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.281+0000 7fed85c73640 1 --2- 172.21.15.38:0/2176994117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed8019c340 0x7fed8019e760 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.281+0000 7fed85c73640 1 --2- 172.21.15.38:0/2176994117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed80102ab0 0x7fed801973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.281+0000 7fed85c73640 1 --2- 172.21.15.38:0/2176994117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80101f50 0x7fed80196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.281+0000 7fed85c73640 1 -- 172.21.15.38:0/2176994117 >> 172.21.15.38:0/2176994117 conn(0x7fed800fc2f0 msgr2=0x7fed800fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.281+0000 7fed85c73640 1 -- 172.21.15.38:0/2176994117 shutdown_connections 2024-05-21T15:40:16.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.281+0000 7fed85c73640 1 -- 172.21.15.38:0/2176994117 wait complete. 2024-05-21T15:40:16.295 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:40:16.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.440+0000 7fe59788d640 1 -- 172.21.15.38:0/257586053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe590101990 msgr2=0x7fe590102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.440+0000 7fe59788d640 1 --2- 172.21.15.38:0/257586053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe590101990 0x7fe590102690 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009900 tx=0x7fe58c02a410 comp rx=0 tx=0).stop 2024-05-21T15:40:16.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.440+0000 7fe59788d640 1 -- 172.21.15.38:0/257586053 shutdown_connections 2024-05-21T15:40:16.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.440+0000 7fe59788d640 1 --2- 172.21.15.38:0/257586053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5901078d0 0x7fe590109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.440+0000 7fe59788d640 1 --2- 172.21.15.38:0/257586053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe590102bd0 0x7fe590102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.440+0000 7fe59788d640 1 --2- 172.21.15.38:0/257586053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe590101990 0x7fe590102690 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.440+0000 7fe59788d640 1 -- 172.21.15.38:0/257586053 >> 172.21.15.38:0/257586053 conn(0x7fe5900fcb80 msgr2=0x7fe5900ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.440+0000 7fe59788d640 1 -- 172.21.15.38:0/257586053 shutdown_connections 2024-05-21T15:40:16.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.440+0000 7fe59788d640 1 -- 172.21.15.38:0/257586053 wait complete. 2024-05-21T15:40:16.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.441+0000 7fe59788d640 1 Processor -- start 2024-05-21T15:40:16.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.442+0000 7fe59788d640 1 -- start start 2024-05-21T15:40:16.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.442+0000 7fe59788d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe590101990 0x7fe5901992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.442+0000 7fe59788d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe590102bd0 0x7fe5901997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.442+0000 7fe59788d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5901078d0 0x7fe5901a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.442+0000 7fe59788d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe59010e700 con 0x7fe590101990 2024-05-21T15:40:16.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.442+0000 7fe59788d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe59010e580 con 0x7fe590102bd0 2024-05-21T15:40:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.442+0000 7fe59788d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe59010e880 con 0x7fe5901078d0 2024-05-21T15:40:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.442+0000 7fe595629640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe590101990 0x7fe5901992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.442+0000 7fe594e28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe590102bd0 0x7fe5901997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.442+0000 7fe595e2a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5901078d0 0x7fe5901a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.442+0000 7fe595629640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe590101990 0x7fe5901992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59268/0 (socket says 172.21.15.38:59268) 2024-05-21T15:40:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.442+0000 7fe594e28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe590102bd0 0x7fe5901997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46530/0 (socket says 172.21.15.38:46530) 2024-05-21T15:40:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.442+0000 7fe595e2a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5901078d0 0x7fe5901a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39028/0 (socket says 172.21.15.38:39028) 2024-05-21T15:40:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.442+0000 7fe595629640 1 -- 172.21.15.38:0/3751960421 learned_addr learned my addr 172.21.15.38:0/3751960421 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.443+0000 7fe595629640 1 -- 172.21.15.38:0/3751960421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5901078d0 msgr2=0x7fe5901a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.443+0000 7fe595629640 1 --2- 172.21.15.38:0/3751960421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5901078d0 0x7fe5901a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.443+0000 7fe595629640 1 -- 172.21.15.38:0/3751960421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe590102bd0 msgr2=0x7fe5901997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.443+0000 7fe595629640 1 --2- 172.21.15.38:0/3751960421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe590102bd0 0x7fe5901997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.443+0000 7fe595629640 1 -- 172.21.15.38:0/3751960421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5901a0de0 con 0x7fe590101990 2024-05-21T15:40:16.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.443+0000 7fe594e28640 1 --2- 172.21.15.38:0/3751960421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe590102bd0 0x7fe5901997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:16.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.443+0000 7fe595629640 1 --2- 172.21.15.38:0/3751960421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe590101990 0x7fe5901992a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe58c005ab0 tx=0x7fe58c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.443+0000 7fe5867fc640 1 -- 172.21.15.38:0/3751960421 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c03f030 con 0x7fe590101990 2024-05-21T15:40:16.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.443+0000 7fe5867fc640 1 -- 172.21.15.38:0/3751960421 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe58c003ad0 con 0x7fe590101990 2024-05-21T15:40:16.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.443+0000 7fe5867fc640 1 -- 172.21.15.38:0/3751960421 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c035070 con 0x7fe590101990 2024-05-21T15:40:16.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.444+0000 7fe59788d640 1 -- 172.21.15.38:0/3751960421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5901a1070 con 0x7fe590101990 2024-05-21T15:40:16.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.444+0000 7fe59788d640 1 -- 172.21.15.38:0/3751960421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5901a1510 con 0x7fe590101990 2024-05-21T15:40:16.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.447+0000 7fe59e80c740 1 Processor -- start 2024-05-21T15:40:16.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.447+0000 7fe59e80c740 1 -- start start 2024-05-21T15:40:16.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe59e80c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c3e390 0x55c389fe1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe59e80c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c154d0 0x55c389fe2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe59e80c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c1d6b0 0x55c389fe9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe59e80c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c389f42c40 con 0x55c389c1d6b0 2024-05-21T15:40:16.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe59e80c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c389eea580 con 0x55c389c3e390 2024-05-21T15:40:16.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe59e80c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c389f3ec00 con 0x55c389c154d0 2024-05-21T15:40:16.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe595e2a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c3e390 0x55c389fe1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe595e2a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c3e390 0x55c389fe1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46546/0 (socket says 172.21.15.38:46546) 2024-05-21T15:40:16.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe595e2a640 1 -- 172.21.15.38:0/2639595951 learned_addr learned my addr 172.21.15.38:0/2639595951 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe5867fc640 1 -- 172.21.15.38:0/3751960421 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe58c042030 con 0x7fe590101990 2024-05-21T15:40:16.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe594e28640 1 --2- 172.21.15.38:0/2639595951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c1d6b0 0x55c389fe9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe595e2a640 1 -- 172.21.15.38:0/2639595951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c154d0 msgr2=0x55c389fe2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe595e2a640 1 --2- 172.21.15.38:0/2639595951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c154d0 0x55c389fe2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe595e2a640 1 -- 172.21.15.38:0/2639595951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c1d6b0 msgr2=0x55c389fe9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe595e2a640 1 --2- 172.21.15.38:0/2639595951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c1d6b0 0x55c389fe9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe595e2a640 1 -- 172.21.15.38:0/2639595951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c389c563f0 con 0x55c389c3e390 2024-05-21T15:40:16.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.448+0000 7fe594e28640 1 --2- 172.21.15.38:0/2639595951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c1d6b0 0x55c389fe9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:40:16.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.449+0000 7fe595e2a640 1 --2- 172.21.15.38:0/2639595951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c3e390 0x55c389fe1f10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe588009fd0 tx=0x7fe5880118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cffc60c64485692 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.449+0000 7fe5867fc640 1 --2- 172.21.15.38:0/3751960421 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe57c0a2b90 0x7fe57c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.449+0000 7fe595629640 1 --2- 172.21.15.38:0/3751960421 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe57c0a2b90 0x7fe57c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.450+0000 7fe5867fc640 1 -- 172.21.15.38:0/3751960421 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe58c0ef4f0 con 0x7fe590101990 2024-05-21T15:40:16.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.450+0000 7fe57b7fe640 1 -- 172.21.15.38:0/2639595951 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588004d20 con 0x55c389c3e390 2024-05-21T15:40:16.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.450+0000 7fe57b7fe640 1 -- 172.21.15.38:0/2639595951 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe588004050 con 0x55c389c3e390 2024-05-21T15:40:16.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.450+0000 7fe57b7fe640 1 -- 172.21.15.38:0/2639595951 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588005260 con 0x55c389c3e390 2024-05-21T15:40:16.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.451+0000 7fe59e80c740 1 -- 172.21.15.38:0/2639595951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c3e390 msgr2=0x55c389fe1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.451+0000 7fe59e80c740 1 --2- 172.21.15.38:0/2639595951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c3e390 0x55c389fe1f10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe588009fd0 tx=0x7fe5880118d0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.451+0000 7fe595629640 1 --2- 172.21.15.38:0/3751960421 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe57c0a2b90 0x7fe57c0a50a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0bbce0 tx=0x7fe58c0f1040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.452+0000 7fe59e80c740 1 -- 172.21.15.38:0/2639595951 shutdown_connections 2024-05-21T15:40:16.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.452+0000 7fe59e80c740 1 --2- 172.21.15.38:0/2639595951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c1d6b0 0x55c389fe9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.452+0000 7fe59e80c740 1 --2- 172.21.15.38:0/2639595951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c154d0 0x55c389fe2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.452+0000 7fe59e80c740 1 --2- 172.21.15.38:0/2639595951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c3e390 0x55c389fe1f10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.452+0000 7fe59e80c740 1 -- 172.21.15.38:0/2639595951 >> 172.21.15.38:0/2639595951 conn(0x55c389c324e0 msgr2=0x55c389fd9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.452+0000 7fe59e80c740 1 -- 172.21.15.38:0/2639595951 shutdown_connections 2024-05-21T15:40:16.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.452+0000 7fe59e80c740 1 -- 172.21.15.38:0/2639595951 wait complete. 2024-05-21T15:40:16.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.452+0000 7fe59e80c740 1 -- start start 2024-05-21T15:40:16.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.452+0000 7fe59e80c740 1 Processor -- start 2024-05-21T15:40:16.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.453+0000 7fe59e80c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c154d0 0x55c389febe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.453+0000 7fe59e80c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c1d6b0 0x55c389fec370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.453+0000 7fe59e80c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c3e390 0x55c389ff32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.453+0000 7fe59e80c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c389fe9de0 con 0x55c389c154d0 2024-05-21T15:40:16.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.453+0000 7fe59e80c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c389efdc40 con 0x55c389c1d6b0 2024-05-21T15:40:16.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.453+0000 7fe59e80c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c389fe9f60 con 0x55c389c3e390 2024-05-21T15:40:16.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.453+0000 7fe59e80c740 10 client.0 _ll_register_callbacks cb 0x55c389c55be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:16.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.454+0000 7fe594e28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c1d6b0 0x55c389fec370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.454+0000 7fe594e28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c1d6b0 0x55c389fec370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46562/0 (socket says 172.21.15.38:46562) 2024-05-21T15:40:16.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.454+0000 7fe594e28640 1 -- 172.21.15.38:0/1518505087 learned_addr learned my addr 172.21.15.38:0/1518505087 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.454+0000 7fe595e2a640 1 --2- 172.21.15.38:0/1518505087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c3e390 0x55c389ff32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.454+0000 7fe595e2a640 1 --2- 172.21.15.38:0/1518505087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c154d0 0x55c389febe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.454+0000 7fe594e28640 1 -- 172.21.15.38:0/1518505087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c3e390 msgr2=0x55c389ff32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.454+0000 7fe594e28640 1 --2- 172.21.15.38:0/1518505087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c3e390 0x55c389ff32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.454+0000 7fe594e28640 1 -- 172.21.15.38:0/1518505087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c154d0 msgr2=0x55c389febe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.454+0000 7fe594e28640 1 --2- 172.21.15.38:0/1518505087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c154d0 0x55c389febe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.454+0000 7fe594e28640 1 -- 172.21.15.38:0/1518505087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c389d207d0 con 0x55c389c1d6b0 2024-05-21T15:40:16.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.454+0000 7fe594e28640 1 --2- 172.21.15.38:0/1518505087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c1d6b0 0x55c389fec370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe580013d30 tx=0x7fe5800117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ff26ff6cb623202 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.454+0000 7fe578ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:40:16.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.454+0000 7fe578ff9640 1 -- 172.21.15.38:0/1518505087 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580004050 con 0x55c389c1d6b0 2024-05-21T15:40:16.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.455+0000 7fe578ff9640 1 -- 172.21.15.38:0/1518505087 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe5800041f0 con 0x55c389c1d6b0 2024-05-21T15:40:16.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.455+0000 7fe59e80c740 1 -- 172.21.15.38:0/1518505087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe588004050 con 0x55c389c1d6b0 2024-05-21T15:40:16.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.455+0000 7fe578ff9640 1 -- 172.21.15.38:0/1518505087 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe5800043f0 con 0x55c389c1d6b0 2024-05-21T15:40:16.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.455+0000 7fe59e80c740 10 client.26642 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:16.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.455+0000 7fe59e80c740 1 -- 172.21.15.38:0/1518505087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c389ff65d0 con 0x55c389c1d6b0 2024-05-21T15:40:16.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.455+0000 7fe578ff9640 1 -- 172.21.15.38:0/1518505087 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe588004050 con 0x55c389c1d6b0 2024-05-21T15:40:16.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.455+0000 7fe578ff9640 1 -- 172.21.15.38:0/1518505087 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fe58001a070 con 0x55c389c1d6b0 2024-05-21T15:40:16.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.455+0000 7fe59e80c740 10 client.26642 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:16.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.455+0000 7fe59e80c740 10 client.26642 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:40:16.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.455+0000 7fe59e80c740 20 client.26642 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:16.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.456+0000 7fe59e80c740 1 --2- 172.21.15.38:0/1518505087 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55c389bfcf60 0x55c389ffa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.456+0000 7fe59e80c740 4 client.26642 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:16.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.456+0000 7fe59e80c740 1 -- 172.21.15.38:0/1518505087 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c389f50370 con 0x55c389bfcf60 2024-05-21T15:40:16.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.456+0000 7fe595e2a640 1 --2- 172.21.15.38:0/1518505087 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55c389bfcf60 0x55c389ffa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.456+0000 7fe578ff9640 10 client.26642 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:40:16.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.457+0000 7fe595e2a640 1 --2- 172.21.15.38:0/1518505087 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55c389bfcf60 0x55c389ffa100 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4160776e43011974 server_cookie=bbca36f24294c76a in_seq=0 out_seq=0 2024-05-21T15:40:16.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.458+0000 7fe578ff9640 1 -- 172.21.15.38:0/1518505087 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c389f50370 con 0x55c389bfcf60 2024-05-21T15:40:16.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.458+0000 7fe578ff9640 10 client.26642 handle_command_reply: tid=0 2024-05-21T15:40:16.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.458+0000 7fe59e80c740 1 client.26642 shutdown 2024-05-21T15:40:16.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.459+0000 7fe59e80c740 1 -- 172.21.15.38:0/1518505087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c1d6b0 msgr2=0x55c389fec370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.459+0000 7fe59e80c740 1 --2- 172.21.15.38:0/1518505087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c1d6b0 0x55c389fec370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe580013d30 tx=0x7fe5800117b0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.459+0000 7fe59e80c740 1 -- 172.21.15.38:0/1518505087 shutdown_connections 2024-05-21T15:40:16.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.459+0000 7fe59e80c740 1 --2- 172.21.15.38:0/1518505087 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55c389bfcf60 0x55c389ffa100 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.459+0000 7fe59e80c740 1 --2- 172.21.15.38:0/1518505087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c3e390 0x55c389ff32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.459+0000 7fe59e80c740 1 --2- 172.21.15.38:0/1518505087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c1d6b0 0x55c389fec370 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.459+0000 7fe59e80c740 1 --2- 172.21.15.38:0/1518505087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c154d0 0x55c389febe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.459+0000 7fe59e80c740 1 -- 172.21.15.38:0/1518505087 >> 172.21.15.38:0/1518505087 conn(0x55c389c324e0 msgr2=0x55c389fd8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.459+0000 7fe578ff9640 0 client.26642 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:40:16.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.459+0000 7fe59e80c740 1 -- 172.21.15.38:0/1518505087 shutdown_connections 2024-05-21T15:40:16.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.459+0000 7fe59e80c740 1 -- 172.21.15.38:0/1518505087 wait complete. 2024-05-21T15:40:16.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.459+0000 7fe59e80c740 20 client.26642 trim_cache size 0 max 16384 2024-05-21T15:40:16.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.477+0000 7fe59e80c740 1 Processor -- start 2024-05-21T15:40:16.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.477+0000 7fe59e80c740 1 -- start start 2024-05-21T15:40:16.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe59e80c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c154d0 0x55c389feb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe59e80c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c1d6b0 0x55c389ff31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe59e80c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c3e390 0x55c389fd8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe59e80c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901a8290 con 0x55c389c3e390 2024-05-21T15:40:16.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe59e80c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901a9ec0 con 0x55c389c154d0 2024-05-21T15:40:16.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe59e80c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901a8960 con 0x55c389c1d6b0 2024-05-21T15:40:16.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe595e2a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c3e390 0x55c389fd8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe595e2a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c154d0 0x55c389feb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe595e2a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c154d0 0x55c389feb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46576/0 (socket says 172.21.15.38:46576) 2024-05-21T15:40:16.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe595e2a640 1 -- 172.21.15.38:0/3293122143 learned_addr learned my addr 172.21.15.38:0/3293122143 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe594e28640 1 --2- 172.21.15.38:0/3293122143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c1d6b0 0x55c389ff31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe595e2a640 1 -- 172.21.15.38:0/3293122143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c1d6b0 msgr2=0x55c389ff31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe595e2a640 1 --2- 172.21.15.38:0/3293122143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c1d6b0 0x55c389ff31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe595e2a640 1 -- 172.21.15.38:0/3293122143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c3e390 msgr2=0x55c389fd8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe595e2a640 1 --2- 172.21.15.38:0/3293122143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c3e390 0x55c389fd8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.478+0000 7fe595e2a640 1 -- 172.21.15.38:0/3293122143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c389bfd190 con 0x55c389c154d0 2024-05-21T15:40:16.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.479+0000 7fe595e2a640 1 --2- 172.21.15.38:0/3293122143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c154d0 0x55c389feb7d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe588009f70 tx=0x7fe588005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bde5d7d081e2e3bf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.479+0000 7fe57affd640 1 -- 172.21.15.38:0/3293122143 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58802e050 con 0x55c389c154d0 2024-05-21T15:40:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.479+0000 7fe57affd640 1 -- 172.21.15.38:0/3293122143 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe588004830 con 0x55c389c154d0 2024-05-21T15:40:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.479+0000 7fe57affd640 1 -- 172.21.15.38:0/3293122143 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58800f4e0 con 0x55c389c154d0 2024-05-21T15:40:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.480+0000 7fe59e80c740 1 -- 172.21.15.38:0/3293122143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c154d0 msgr2=0x55c389feb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.480+0000 7fe59e80c740 1 --2- 172.21.15.38:0/3293122143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c154d0 0x55c389feb7d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe588009f70 tx=0x7fe588005fd0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.480+0000 7fe59e80c740 1 -- 172.21.15.38:0/3293122143 shutdown_connections 2024-05-21T15:40:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.480+0000 7fe59e80c740 1 --2- 172.21.15.38:0/3293122143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c3e390 0x55c389fd8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.480+0000 7fe59e80c740 1 --2- 172.21.15.38:0/3293122143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c1d6b0 0x55c389ff31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.480+0000 7fe59e80c740 1 --2- 172.21.15.38:0/3293122143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c154d0 0x55c389feb7d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.480+0000 7fe59e80c740 1 -- 172.21.15.38:0/3293122143 >> 172.21.15.38:0/3293122143 conn(0x55c389c324e0 msgr2=0x55c389feb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.480+0000 7fe59e80c740 1 -- 172.21.15.38:0/3293122143 shutdown_connections 2024-05-21T15:40:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.480+0000 7fe59e80c740 1 -- 172.21.15.38:0/3293122143 wait complete. 2024-05-21T15:40:16.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.480+0000 7fe59e80c740 1 -- start start 2024-05-21T15:40:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.481+0000 7fe59e80c740 1 Processor -- start 2024-05-21T15:40:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.481+0000 7fe59e80c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c154d0 0x55c389ff0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.481+0000 7fe59e80c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c1d6b0 0x55c389ff0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.481+0000 7fe59e80c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c3e390 0x55c38a0604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.481+0000 7fe59e80c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901a9730 con 0x55c389c3e390 2024-05-21T15:40:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.481+0000 7fe59e80c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901a9030 con 0x55c389c154d0 2024-05-21T15:40:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.481+0000 7fe59e80c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901a6270 con 0x55c389c1d6b0 2024-05-21T15:40:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.481+0000 7fe59e80c740 10 client.0 _ll_register_callbacks cb 0x7fe580009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.481+0000 7fe595e2a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c3e390 0x55c38a0604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.481+0000 7fe594e28640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c1d6b0 0x55c389ff0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.481+0000 7fe595e2a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c3e390 0x55c38a0604a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59292/0 (socket says 172.21.15.38:59292) 2024-05-21T15:40:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.481+0000 7fe595e2a640 1 -- 172.21.15.38:0/3339408916 learned_addr learned my addr 172.21.15.38:0/3339408916 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.481+0000 7fe595e2a640 1 --2- 172.21.15.38:0/3339408916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c154d0 0x55c389ff0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.482+0000 7fe594e28640 1 -- 172.21.15.38:0/3339408916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c154d0 msgr2=0x55c389ff0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.482+0000 7fe594e28640 1 --2- 172.21.15.38:0/3339408916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c154d0 0x55c389ff0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.482+0000 7fe594e28640 1 -- 172.21.15.38:0/3339408916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c3e390 msgr2=0x55c38a0604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.482+0000 7fe594e28640 1 --2- 172.21.15.38:0/3339408916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c3e390 0x55c38a0604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.482+0000 7fe594e28640 1 -- 172.21.15.38:0/3339408916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c389ff0b70 con 0x55c389c1d6b0 2024-05-21T15:40:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.482+0000 7fe595e2a640 1 --2- 172.21.15.38:0/3339408916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c3e390 0x55c38a0604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.482+0000 7fe594e28640 1 --2- 172.21.15.38:0/3339408916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c1d6b0 0x55c389ff0560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe580025a70 tx=0x7fe580011cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9cd43ebbf1b3897 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.482+0000 7fe578ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:40:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.482+0000 7fe578ff9640 1 -- 172.21.15.38:0/3339408916 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58001b850 con 0x55c389c1d6b0 2024-05-21T15:40:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.482+0000 7fe578ff9640 1 -- 172.21.15.38:0/3339408916 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe58001b9f0 con 0x55c389c1d6b0 2024-05-21T15:40:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.482+0000 7fe59e80c740 1 -- 172.21.15.38:0/3339408916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe588004830 con 0x55c389c1d6b0 2024-05-21T15:40:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.483+0000 7fe578ff9640 1 -- 172.21.15.38:0/3339408916 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe58001bbf0 con 0x55c389c1d6b0 2024-05-21T15:40:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.483+0000 7fe59e80c740 10 client.26710 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.483+0000 7fe59e80c740 1 -- 172.21.15.38:0/3339408916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c38a061aa0 con 0x55c389c1d6b0 2024-05-21T15:40:16.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.483+0000 7fe578ff9640 1 -- 172.21.15.38:0/3339408916 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe58001bd90 con 0x55c389c1d6b0 2024-05-21T15:40:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.483+0000 7fe578ff9640 1 -- 172.21.15.38:0/3339408916 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fe58001a070 con 0x55c389c1d6b0 2024-05-21T15:40:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.483+0000 7fe59e80c740 10 client.26710 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.483+0000 7fe59e80c740 10 client.26710 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:40:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.483+0000 7fe59e80c740 20 client.26710 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.483+0000 7fe59e80c740 1 --2- 172.21.15.38:0/3339408916 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55c389bd23a0 0x55c38a064470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.483+0000 7fe59e80c740 4 client.26710 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:40:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.483+0000 7fe59e80c740 1 -- 172.21.15.38:0/3339408916 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe588004830 con 0x55c389bd23a0 2024-05-21T15:40:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.483+0000 7fe595e2a640 1 --2- 172.21.15.38:0/3339408916 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55c389bd23a0 0x55c38a064470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.485+0000 7fe595e2a640 1 --2- 172.21.15.38:0/3339408916 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55c389bd23a0 0x55c38a064470 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa34d4a7dc1b4359 server_cookie=1370d6dc5c9146d5 in_seq=0 out_seq=0 2024-05-21T15:40:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.485+0000 7fe578ff9640 10 client.26710 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:40:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.485+0000 7fe578ff9640 1 -- 172.21.15.38:0/3339408916 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe588004830 con 0x55c389bd23a0 2024-05-21T15:40:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.485+0000 7fe578ff9640 10 client.26710 handle_command_reply: tid=0 2024-05-21T15:40:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.485+0000 7fe59e80c740 1 client.26710 shutdown 2024-05-21T15:40:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.485+0000 7fe59e80c740 1 -- 172.21.15.38:0/3339408916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c1d6b0 msgr2=0x55c389ff0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.485+0000 7fe59e80c740 1 --2- 172.21.15.38:0/3339408916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c1d6b0 0x55c389ff0560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe580025a70 tx=0x7fe580011cb0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.485+0000 7fe59e80c740 1 -- 172.21.15.38:0/3339408916 shutdown_connections 2024-05-21T15:40:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.485+0000 7fe59e80c740 1 --2- 172.21.15.38:0/3339408916 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55c389bd23a0 0x55c38a064470 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.486+0000 7fe59e80c740 1 --2- 172.21.15.38:0/3339408916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c389c3e390 0x55c38a0604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.486+0000 7fe578ff9640 0 client.26710 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:40:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.486+0000 7fe59e80c740 1 --2- 172.21.15.38:0/3339408916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c389c1d6b0 0x55c389ff0560 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.486+0000 7fe59e80c740 1 --2- 172.21.15.38:0/3339408916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c389c154d0 0x55c389ff0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.486+0000 7fe59e80c740 1 -- 172.21.15.38:0/3339408916 >> 172.21.15.38:0/3339408916 conn(0x55c389c324e0 msgr2=0x55c389fd8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.486+0000 7fe59e80c740 1 -- 172.21.15.38:0/3339408916 shutdown_connections 2024-05-21T15:40:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.486+0000 7fe59e80c740 1 -- 172.21.15.38:0/3339408916 wait complete. 2024-05-21T15:40:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.486+0000 7fe59e80c740 20 client.26710 trim_cache size 0 max 16384 2024-05-21T15:40:16.488 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:40:16.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.487+0000 7fe59788d640 1 -- 172.21.15.38:0/3751960421 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe57c0a2b90 msgr2=0x7fe57c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.487+0000 7fe59788d640 1 --2- 172.21.15.38:0/3751960421 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe57c0a2b90 0x7fe57c0a50a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0bbce0 tx=0x7fe58c0f1040 comp rx=0 tx=0).stop 2024-05-21T15:40:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.487+0000 7fe59788d640 1 -- 172.21.15.38:0/3751960421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe590101990 msgr2=0x7fe5901992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.487+0000 7fe59788d640 1 --2- 172.21.15.38:0/3751960421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe590101990 0x7fe5901992a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe58c005ab0 tx=0x7fe58c004270 comp rx=0 tx=0).stop 2024-05-21T15:40:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.487+0000 7fe59788d640 1 -- 172.21.15.38:0/3751960421 shutdown_connections 2024-05-21T15:40:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.487+0000 7fe59788d640 1 --2- 172.21.15.38:0/3751960421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5901078d0 0x7fe5901a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.487+0000 7fe59788d640 1 --2- 172.21.15.38:0/3751960421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe590102bd0 0x7fe5901997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.487+0000 7fe59788d640 1 --2- 172.21.15.38:0/3751960421 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe57c0a2b90 0x7fe57c0a50a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.487+0000 7fe59788d640 1 --2- 172.21.15.38:0/3751960421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe590101990 0x7fe5901992a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.487+0000 7fe59788d640 1 -- 172.21.15.38:0/3751960421 >> 172.21.15.38:0/3751960421 conn(0x7fe5900fcb80 msgr2=0x7fe5901038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.487+0000 7fe59788d640 1 -- 172.21.15.38:0/3751960421 shutdown_connections 2024-05-21T15:40:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.487+0000 7fe59788d640 1 -- 172.21.15.38:0/3751960421 wait complete. 2024-05-21T15:40:16.500 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:40:16.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.623+0000 7f844f945640 1 -- 172.21.15.38:0/4086458062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84481078c0 msgr2=0x7f8448109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.623+0000 7f844f945640 1 --2- 172.21.15.38:0/4086458062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84481078c0 0x7f8448109ce0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8440009ca0 tx=0x7f844002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.623+0000 7f844f945640 1 -- 172.21.15.38:0/4086458062 shutdown_connections 2024-05-21T15:40:16.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.623+0000 7f844f945640 1 --2- 172.21.15.38:0/4086458062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84481078c0 0x7f8448109ce0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.623+0000 7f844f945640 1 --2- 172.21.15.38:0/4086458062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8448102bc0 0x7f8448102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.623+0000 7f844f945640 1 --2- 172.21.15.38:0/4086458062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8448101980 0x7f8448102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.623+0000 7f844f945640 1 -- 172.21.15.38:0/4086458062 >> 172.21.15.38:0/4086458062 conn(0x7f84480fcb70 msgr2=0x7f84480ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.623+0000 7f844f945640 1 -- 172.21.15.38:0/4086458062 shutdown_connections 2024-05-21T15:40:16.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.623+0000 7f844f945640 1 -- 172.21.15.38:0/4086458062 wait complete. 2024-05-21T15:40:16.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.624+0000 7f844f945640 1 Processor -- start 2024-05-21T15:40:16.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.624+0000 7f844f945640 1 -- start start 2024-05-21T15:40:16.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.624+0000 7f844f945640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8448101980 0x7f84480699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.624+0000 7f844f945640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448102bc0 0x7f844819e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.624+0000 7f844f945640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84481078c0 0x7f84481a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.624+0000 7f844f945640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f844810d600 con 0x7f8448102bc0 2024-05-21T15:40:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.624+0000 7f844f945640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f844810d480 con 0x7f8448101980 2024-05-21T15:40:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.624+0000 7f844f945640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f844810d780 con 0x7f84481078c0 2024-05-21T15:40:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.624+0000 7f844cee0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448102bc0 0x7f844819e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.624+0000 7f844cee0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448102bc0 0x7f844819e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59308/0 (socket says 172.21.15.38:59308) 2024-05-21T15:40:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.624+0000 7f844cee0640 1 -- 172.21.15.38:0/1088209867 learned_addr learned my addr 172.21.15.38:0/1088209867 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.624+0000 7f844d6e1640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8448101980 0x7f84480699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.624+0000 7f844dee2640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84481078c0 0x7f84481a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.625+0000 7f844cee0640 1 -- 172.21.15.38:0/1088209867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84481078c0 msgr2=0x7f84481a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.625+0000 7f844cee0640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84481078c0 0x7f84481a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.625+0000 7f844cee0640 1 -- 172.21.15.38:0/1088209867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8448101980 msgr2=0x7f84480699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.625+0000 7f844cee0640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8448101980 0x7f84480699b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.625+0000 7f844cee0640 1 -- 172.21.15.38:0/1088209867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84481a0d50 con 0x7f8448102bc0 2024-05-21T15:40:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.625+0000 7f844dee2640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84481078c0 0x7f84481a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:40:16.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.625+0000 7f844cee0640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448102bc0 0x7f844819e030 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f843800d950 tx=0x7f843800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.625+0000 7f843e7fc640 1 -- 172.21.15.38:0/1088209867 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438017070 con 0x7f8448102bc0 2024-05-21T15:40:16.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.625+0000 7f844f945640 1 -- 172.21.15.38:0/1088209867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84481a0fe0 con 0x7f8448102bc0 2024-05-21T15:40:16.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.625+0000 7f843e7fc640 1 -- 172.21.15.38:0/1088209867 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8438009c30 con 0x7f8448102bc0 2024-05-21T15:40:16.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.625+0000 7f843e7fc640 1 -- 172.21.15.38:0/1088209867 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843800f6b0 con 0x7f8448102bc0 2024-05-21T15:40:16.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.625+0000 7f844f945640 1 -- 172.21.15.38:0/1088209867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84481a1520 con 0x7f8448102bc0 2024-05-21T15:40:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f843e7fc640 1 -- 172.21.15.38:0/1088209867 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f84380109d0 con 0x7f8448102bc0 2024-05-21T15:40:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f84567f5740 1 Processor -- start 2024-05-21T15:40:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f84567f5740 1 -- start start 2024-05-21T15:40:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f84567f5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca585390 0x55e3ca928f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f84567f5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca55c4d0 0x55e3ca929490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f84567f5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 0x55e3ca930390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f84567f5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e3ca889c40 con 0x55e3ca585390 2024-05-21T15:40:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f84567f5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e3ca831580 con 0x55e3ca55c4d0 2024-05-21T15:40:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f84567f5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e3ca885c00 con 0x55e3ca5646b0 2024-05-21T15:40:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f844dee2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca585390 0x55e3ca928f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f844dee2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca585390 0x55e3ca928f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59318/0 (socket says 172.21.15.38:59318) 2024-05-21T15:40:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f844dee2640 1 -- 172.21.15.38:0/431097554 learned_addr learned my addr 172.21.15.38:0/431097554 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f844cee0640 1 --2- 172.21.15.38:0/431097554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 0x55e3ca930390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f844d6e1640 1 --2- 172.21.15.38:0/431097554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca55c4d0 0x55e3ca929490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f844dee2640 1 -- 172.21.15.38:0/431097554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 msgr2=0x55e3ca930390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f844dee2640 1 --2- 172.21.15.38:0/431097554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 0x55e3ca930390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f844dee2640 1 -- 172.21.15.38:0/431097554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca55c4d0 msgr2=0x55e3ca929490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f844dee2640 1 --2- 172.21.15.38:0/431097554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca55c4d0 0x55e3ca929490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f844dee2640 1 -- 172.21.15.38:0/431097554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3ca59d3f0 con 0x55e3ca585390 2024-05-21T15:40:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f844d6e1640 1 --2- 172.21.15.38:0/431097554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca55c4d0 0x55e3ca929490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:40:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.628+0000 7f843e7fc640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84340a2b90 0x7f84340a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.629+0000 7f844d6e1640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84340a2b90 0x7f84340a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.629+0000 7f843e7fc640 1 -- 172.21.15.38:0/1088209867 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f84380ced00 con 0x7f8448102bc0 2024-05-21T15:40:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.629+0000 7f844dee2640 1 --2- 172.21.15.38:0/431097554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca585390 0x55e3ca928f10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8440035000 tx=0x7f844002f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a019d3eb825190 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.629+0000 7f84337fe640 1 -- 172.21.15.38:0/431097554 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440031cb0 con 0x55e3ca585390 2024-05-21T15:40:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.629+0000 7f84337fe640 1 -- 172.21.15.38:0/431097554 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8440036050 con 0x55e3ca585390 2024-05-21T15:40:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.629+0000 7f84337fe640 1 -- 172.21.15.38:0/431097554 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84400042f0 con 0x55e3ca585390 2024-05-21T15:40:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.630+0000 7f84567f5740 1 -- 172.21.15.38:0/431097554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca585390 msgr2=0x55e3ca928f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.630+0000 7f84567f5740 1 --2- 172.21.15.38:0/431097554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca585390 0x55e3ca928f10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8440035000 tx=0x7f844002f390 comp rx=0 tx=0).stop 2024-05-21T15:40:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.630+0000 7f844d6e1640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84340a2b90 0x7f84340a50a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55e3ca92a2f0 tx=0x7f8444005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.630+0000 7f84567f5740 1 -- 172.21.15.38:0/431097554 shutdown_connections 2024-05-21T15:40:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.630+0000 7f84567f5740 1 --2- 172.21.15.38:0/431097554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 0x55e3ca930390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.630+0000 7f84567f5740 1 --2- 172.21.15.38:0/431097554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca55c4d0 0x55e3ca929490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.630+0000 7f84567f5740 1 --2- 172.21.15.38:0/431097554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca585390 0x55e3ca928f10 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.630+0000 7f84567f5740 1 -- 172.21.15.38:0/431097554 >> 172.21.15.38:0/431097554 conn(0x55e3ca5794e0 msgr2=0x55e3ca920490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.630+0000 7f84567f5740 1 -- 172.21.15.38:0/431097554 shutdown_connections 2024-05-21T15:40:16.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.630+0000 7f84567f5740 1 -- 172.21.15.38:0/431097554 wait complete. 2024-05-21T15:40:16.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.630+0000 7f84567f5740 1 -- start start 2024-05-21T15:40:16.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.630+0000 7f84567f5740 1 Processor -- start 2024-05-21T15:40:16.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f84567f5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca55c4d0 0x55e3ca932c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f84567f5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 0x55e3ca933140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f84567f5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca585390 0x55e3ca93a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f84567f5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e3ca930c00 con 0x55e3ca55c4d0 2024-05-21T15:40:16.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f84567f5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e3ca844c40 con 0x55e3ca585390 2024-05-21T15:40:16.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f84567f5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e3ca930d80 con 0x55e3ca5646b0 2024-05-21T15:40:16.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f84567f5740 10 client.0 _ll_register_callbacks cb 0x55e3ca59cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f844d6e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca55c4d0 0x55e3ca932c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f844d6e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca55c4d0 0x55e3ca932c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59330/0 (socket says 172.21.15.38:59330) 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f844d6e1640 1 -- 172.21.15.38:0/1509938744 learned_addr learned my addr 172.21.15.38:0/1509938744 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f844cee0640 1 --2- 172.21.15.38:0/1509938744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 0x55e3ca933140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f844dee2640 1 --2- 172.21.15.38:0/1509938744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca585390 0x55e3ca93a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f844d6e1640 1 -- 172.21.15.38:0/1509938744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 msgr2=0x55e3ca933140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f844d6e1640 1 --2- 172.21.15.38:0/1509938744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 0x55e3ca933140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f844d6e1640 1 -- 172.21.15.38:0/1509938744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca585390 msgr2=0x55e3ca93a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f844d6e1640 1 --2- 172.21.15.38:0/1509938744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca585390 0x55e3ca93a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f844d6e1640 1 -- 172.21.15.38:0/1509938744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3ca6677d0 con 0x55e3ca55c4d0 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f844dee2640 1 --2- 172.21.15.38:0/1509938744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca585390 0x55e3ca93a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f844d6e1640 1 --2- 172.21.15.38:0/1509938744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca55c4d0 0x55e3ca932c00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8444013010 tx=0x7f8444004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d58135b8d5e0c59 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.631+0000 7f8430ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f8430ff9640 1 -- 172.21.15.38:0/1509938744 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444029440 con 0x55e3ca55c4d0 2024-05-21T15:40:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f8430ff9640 1 -- 172.21.15.38:0/1509938744 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f844401f430 con 0x55e3ca55c4d0 2024-05-21T15:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f84567f5740 1 -- 172.21.15.38:0/1509938744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8440036050 con 0x55e3ca55c4d0 2024-05-21T15:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f8430ff9640 1 -- 172.21.15.38:0/1509938744 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f844401f630 con 0x55e3ca55c4d0 2024-05-21T15:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f84567f5740 10 client.17034 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f84567f5740 1 -- 172.21.15.38:0/1509938744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e3ca93d430 con 0x55e3ca55c4d0 2024-05-21T15:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f8430ff9640 1 -- 172.21.15.38:0/1509938744 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8440036050 con 0x55e3ca55c4d0 2024-05-21T15:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f8430ff9640 1 -- 172.21.15.38:0/1509938744 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f8444027070 con 0x55e3ca55c4d0 2024-05-21T15:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f84567f5740 10 client.17034 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f84567f5740 10 client.17034 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f84567f5740 20 client.17034 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f84567f5740 1 --2- 172.21.15.38:0/1509938744 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55e3ca543f60 0x55e3ca940b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f84567f5740 4 client.17034 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f84567f5740 1 -- 172.21.15.38:0/1509938744 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3ca897370 con 0x55e3ca543f60 2024-05-21T15:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.632+0000 7f844cee0640 1 --2- 172.21.15.38:0/1509938744 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55e3ca543f60 0x55e3ca940b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.633+0000 7f8430ff9640 10 client.17034 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:40:16.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.633+0000 7f844cee0640 1 --2- 172.21.15.38:0/1509938744 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55e3ca543f60 0x55e3ca940b10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b77649bd0fefd102 server_cookie=5e6ba9db630bd366 in_seq=0 out_seq=0 2024-05-21T15:40:16.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f8430ff9640 1 -- 172.21.15.38:0/1509938744 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e3ca897370 con 0x55e3ca543f60 2024-05-21T15:40:16.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f8430ff9640 10 client.17034 handle_command_reply: tid=0 2024-05-21T15:40:16.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f84567f5740 1 client.17034 shutdown 2024-05-21T15:40:16.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f84567f5740 1 -- 172.21.15.38:0/1509938744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca55c4d0 msgr2=0x55e3ca932c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f84567f5740 1 --2- 172.21.15.38:0/1509938744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca55c4d0 0x55e3ca932c00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8444013010 tx=0x7f8444004730 comp rx=0 tx=0).stop 2024-05-21T15:40:16.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f84567f5740 1 -- 172.21.15.38:0/1509938744 shutdown_connections 2024-05-21T15:40:16.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f84567f5740 1 --2- 172.21.15.38:0/1509938744 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55e3ca543f60 0x55e3ca940b10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f84567f5740 1 --2- 172.21.15.38:0/1509938744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca585390 0x55e3ca93a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f8430ff9640 0 client.17034 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:40:16.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f84567f5740 1 --2- 172.21.15.38:0/1509938744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 0x55e3ca933140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f84567f5740 1 --2- 172.21.15.38:0/1509938744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca55c4d0 0x55e3ca932c00 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f84567f5740 1 -- 172.21.15.38:0/1509938744 >> 172.21.15.38:0/1509938744 conn(0x55e3ca5794e0 msgr2=0x55e3ca91f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f84567f5740 1 -- 172.21.15.38:0/1509938744 shutdown_connections 2024-05-21T15:40:16.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f84567f5740 1 -- 172.21.15.38:0/1509938744 wait complete. 2024-05-21T15:40:16.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.634+0000 7f84567f5740 20 client.17034 trim_cache size 0 max 16384 2024-05-21T15:40:16.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f84567f5740 1 Processor -- start 2024-05-21T15:40:16.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f84567f5740 1 -- start start 2024-05-21T15:40:16.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f84567f5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca543f60 0x55e3ca91f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f84567f5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca55c4d0 0x55e3ca93a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f84567f5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca5646b0 0x55e3ca931140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f84567f5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f84481a8260 con 0x55e3ca543f60 2024-05-21T15:40:16.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f84567f5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84481a9e90 con 0x55e3ca5646b0 2024-05-21T15:40:16.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f84567f5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f84481a8930 con 0x55e3ca55c4d0 2024-05-21T15:40:16.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f844d6e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca543f60 0x55e3ca91f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f844d6e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca543f60 0x55e3ca91f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59338/0 (socket says 172.21.15.38:59338) 2024-05-21T15:40:16.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f844d6e1640 1 -- 172.21.15.38:0/4006445440 learned_addr learned my addr 172.21.15.38:0/4006445440 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f844dee2640 1 --2- 172.21.15.38:0/4006445440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca5646b0 0x55e3ca931140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f844cee0640 1 --2- 172.21.15.38:0/4006445440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca55c4d0 0x55e3ca93a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f844d6e1640 1 -- 172.21.15.38:0/4006445440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca55c4d0 msgr2=0x55e3ca93a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f844d6e1640 1 --2- 172.21.15.38:0/4006445440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca55c4d0 0x55e3ca93a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f844d6e1640 1 -- 172.21.15.38:0/4006445440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca5646b0 msgr2=0x55e3ca931140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f844d6e1640 1 --2- 172.21.15.38:0/4006445440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca5646b0 0x55e3ca931140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f844d6e1640 1 -- 172.21.15.38:0/4006445440 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3ca92a110 con 0x55e3ca543f60 2024-05-21T15:40:16.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f844dee2640 1 --2- 172.21.15.38:0/4006445440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca5646b0 0x55e3ca931140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:40:16.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.648+0000 7f844cee0640 1 --2- 172.21.15.38:0/4006445440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca55c4d0 0x55e3ca93a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:40:16.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.649+0000 7f844d6e1640 1 --2- 172.21.15.38:0/4006445440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca543f60 0x55e3ca91f250 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f84440059d0 tx=0x7f8444003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e3bbc2bcec2932 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.649+0000 7f8432ffd640 1 -- 172.21.15.38:0/4006445440 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444038030 con 0x55e3ca543f60 2024-05-21T15:40:16.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.649+0000 7f8432ffd640 1 -- 172.21.15.38:0/4006445440 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8444029920 con 0x55e3ca543f60 2024-05-21T15:40:16.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.649+0000 7f8432ffd640 1 -- 172.21.15.38:0/4006445440 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84440336d0 con 0x55e3ca543f60 2024-05-21T15:40:16.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.649+0000 7f84567f5740 1 -- 172.21.15.38:0/4006445440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca543f60 msgr2=0x55e3ca91f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.649+0000 7f84567f5740 1 --2- 172.21.15.38:0/4006445440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca543f60 0x55e3ca91f250 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f84440059d0 tx=0x7f8444003ed0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.649+0000 7f84567f5740 1 -- 172.21.15.38:0/4006445440 shutdown_connections 2024-05-21T15:40:16.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.649+0000 7f84567f5740 1 --2- 172.21.15.38:0/4006445440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca5646b0 0x55e3ca931140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.649+0000 7f84567f5740 1 --2- 172.21.15.38:0/4006445440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca55c4d0 0x55e3ca93a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.649+0000 7f84567f5740 1 --2- 172.21.15.38:0/4006445440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca543f60 0x55e3ca91f250 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.649+0000 7f84567f5740 1 -- 172.21.15.38:0/4006445440 >> 172.21.15.38:0/4006445440 conn(0x55e3ca5794e0 msgr2=0x55e3ca91eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.649+0000 7f84567f5740 1 -- 172.21.15.38:0/4006445440 shutdown_connections 2024-05-21T15:40:16.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.650+0000 7f84567f5740 1 -- 172.21.15.38:0/4006445440 wait complete. 2024-05-21T15:40:16.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.650+0000 7f84567f5740 1 -- start start 2024-05-21T15:40:16.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.650+0000 7f84567f5740 1 Processor -- start 2024-05-21T15:40:16.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.650+0000 7f84567f5740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca543f60 0x55e3ca93a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.650+0000 7f84567f5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca55c4d0 0x55e3ca936cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.650+0000 7f84567f5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 0x55e3ca9372b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.650+0000 7f84567f5740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f84481a9700 con 0x55e3ca543f60 2024-05-21T15:40:16.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.650+0000 7f84567f5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84481a9000 con 0x55e3ca55c4d0 2024-05-21T15:40:16.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.650+0000 7f84567f5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f84481a6240 con 0x55e3ca5646b0 2024-05-21T15:40:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.650+0000 7f84567f5740 10 client.0 _ll_register_callbacks cb 0x7f8444009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.650+0000 7f844d6e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca543f60 0x55e3ca93a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.650+0000 7f844d6e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca543f60 0x55e3ca93a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59344/0 (socket says 172.21.15.38:59344) 2024-05-21T15:40:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.650+0000 7f844d6e1640 1 -- 172.21.15.38:0/2703218935 learned_addr learned my addr 172.21.15.38:0/2703218935 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.650+0000 7f844dee2640 1 --2- 172.21.15.38:0/2703218935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 0x55e3ca9372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f844cee0640 1 --2- 172.21.15.38:0/2703218935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca55c4d0 0x55e3ca936cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f844d6e1640 1 -- 172.21.15.38:0/2703218935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 msgr2=0x55e3ca9372b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f844d6e1640 1 --2- 172.21.15.38:0/2703218935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 0x55e3ca9372b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f844d6e1640 1 -- 172.21.15.38:0/2703218935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca55c4d0 msgr2=0x55e3ca936cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f844d6e1640 1 --2- 172.21.15.38:0/2703218935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca55c4d0 0x55e3ca936cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f844d6e1640 1 -- 172.21.15.38:0/2703218935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3ca9a6ee0 con 0x55e3ca543f60 2024-05-21T15:40:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f844d6e1640 1 --2- 172.21.15.38:0/2703218935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca543f60 0x55e3ca93a0e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8444029c20 tx=0x7f844401f530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd400d4a7b7b85e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f8430ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f8430ff9640 1 -- 172.21.15.38:0/2703218935 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444042060 con 0x55e3ca543f60 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f8430ff9640 1 -- 172.21.15.38:0/2703218935 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f844401f7d0 con 0x55e3ca543f60 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f84567f5740 1 -- 172.21.15.38:0/2703218935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8444029920 con 0x55e3ca543f60 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f8430ff9640 1 -- 172.21.15.38:0/2703218935 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8444038040 con 0x55e3ca543f60 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f84567f5740 10 client.17040 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f84567f5740 1 -- 172.21.15.38:0/2703218935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e3ca9a8070 con 0x55e3ca543f60 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.651+0000 7f8430ff9640 1 -- 172.21.15.38:0/2703218935 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8444029920 con 0x55e3ca543f60 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.652+0000 7f8430ff9640 1 -- 172.21.15.38:0/2703218935 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f8444027040 con 0x55e3ca543f60 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.652+0000 7f84567f5740 10 client.17040 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.652+0000 7f84567f5740 10 client.17040 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.652+0000 7f84567f5740 20 client.17040 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.652+0000 7f84567f5740 1 --2- 172.21.15.38:0/2703218935 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55e3ca585390 0x55e3ca9aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.652+0000 7f84567f5740 4 client.17040 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.652+0000 7f84567f5740 1 -- 172.21.15.38:0/2703218935 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3ca8916c0 con 0x55e3ca585390 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.652+0000 7f844cee0640 1 --2- 172.21.15.38:0/2703218935 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55e3ca585390 0x55e3ca9aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.652+0000 7f8430ff9640 10 client.17040 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.652+0000 7f844cee0640 1 --2- 172.21.15.38:0/2703218935 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55e3ca585390 0x55e3ca9aa9e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc25b7f7300ea8d2 server_cookie=e4128b8eeac9fec5 in_seq=0 out_seq=0 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.652+0000 7f8430ff9640 1 -- 172.21.15.38:0/2703218935 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3ca8916c0 con 0x55e3ca585390 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.652+0000 7f8430ff9640 10 client.17040 handle_command_reply: tid=0 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.652+0000 7f84567f5740 1 client.17040 shutdown 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f84567f5740 1 -- 172.21.15.38:0/2703218935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca543f60 msgr2=0x55e3ca93a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f84567f5740 1 --2- 172.21.15.38:0/2703218935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca543f60 0x55e3ca93a0e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8444029c20 tx=0x7f844401f530 comp rx=0 tx=0).stop 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f84567f5740 1 -- 172.21.15.38:0/2703218935 shutdown_connections 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f84567f5740 1 --2- 172.21.15.38:0/2703218935 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55e3ca585390 0x55e3ca9aa9e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f84567f5740 1 --2- 172.21.15.38:0/2703218935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e3ca5646b0 0x55e3ca9372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f84567f5740 1 --2- 172.21.15.38:0/2703218935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3ca55c4d0 0x55e3ca936cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f84567f5740 1 --2- 172.21.15.38:0/2703218935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3ca543f60 0x55e3ca93a0e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f8430ff9640 0 client.17040 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f84567f5740 1 -- 172.21.15.38:0/2703218935 >> 172.21.15.38:0/2703218935 conn(0x55e3ca5794e0 msgr2=0x55e3ca91f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f84567f5740 1 -- 172.21.15.38:0/2703218935 shutdown_connections 2024-05-21T15:40:16.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f84567f5740 1 -- 172.21.15.38:0/2703218935 wait complete. 2024-05-21T15:40:16.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f84567f5740 20 client.17040 trim_cache size 0 max 16384 2024-05-21T15:40:16.658 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:40:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f844f945640 1 -- 172.21.15.38:0/1088209867 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84340a2b90 msgr2=0x7f84340a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f844f945640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84340a2b90 0x7f84340a50a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55e3ca92a2f0 tx=0x7f8444005f90 comp rx=0 tx=0).stop 2024-05-21T15:40:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f844f945640 1 -- 172.21.15.38:0/1088209867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448102bc0 msgr2=0x7f844819e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f844f945640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448102bc0 0x7f844819e030 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f843800d950 tx=0x7f843800de10 comp rx=0 tx=0).stop 2024-05-21T15:40:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f844f945640 1 -- 172.21.15.38:0/1088209867 shutdown_connections 2024-05-21T15:40:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f844f945640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84481078c0 0x7f84481a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f844f945640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84340a2b90 0x7f84340a50a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f844f945640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8448102bc0 0x7f844819e030 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f844f945640 1 --2- 172.21.15.38:0/1088209867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8448101980 0x7f84480699b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.653+0000 7f844f945640 1 -- 172.21.15.38:0/1088209867 >> 172.21.15.38:0/1088209867 conn(0x7f84480fcb70 msgr2=0x7f84481038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.654+0000 7f844f945640 1 -- 172.21.15.38:0/1088209867 shutdown_connections 2024-05-21T15:40:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.654+0000 7f844f945640 1 -- 172.21.15.38:0/1088209867 wait complete. 2024-05-21T15:40:16.663 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:40:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.794+0000 7fa158b02640 1 -- 172.21.15.38:0/2151761161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa154101100 msgr2=0x7fa154101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.794+0000 7fa158b02640 1 --2- 172.21.15.38:0/2151761161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa154101100 0x7fa154101e00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa144009860 tx=0x7fa14402a340 comp rx=0 tx=0).stop 2024-05-21T15:40:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.794+0000 7fa158b02640 1 -- 172.21.15.38:0/2151761161 shutdown_connections 2024-05-21T15:40:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.794+0000 7fa158b02640 1 --2- 172.21.15.38:0/2151761161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa154107040 0x7fa154109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.794+0000 7fa158b02640 1 --2- 172.21.15.38:0/2151761161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa154102340 0x7fa154102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.794+0000 7fa158b02640 1 --2- 172.21.15.38:0/2151761161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa154101100 0x7fa154101e00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.794+0000 7fa158b02640 1 -- 172.21.15.38:0/2151761161 >> 172.21.15.38:0/2151761161 conn(0x7fa1540fc2f0 msgr2=0x7fa1540fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.794+0000 7fa158b02640 1 -- 172.21.15.38:0/2151761161 shutdown_connections 2024-05-21T15:40:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.795+0000 7fa158b02640 1 -- 172.21.15.38:0/2151761161 wait complete. 2024-05-21T15:40:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.796+0000 7fa158b02640 1 Processor -- start 2024-05-21T15:40:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.796+0000 7fa158b02640 1 -- start start 2024-05-21T15:40:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.796+0000 7fa158b02640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa154101100 0x7fa154198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.796+0000 7fa158b02640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa154102340 0x7fa1541986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.796+0000 7fa158b02640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa154107040 0x7fa15419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.796+0000 7fa158b02640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa15410d5b0 con 0x7fa154101100 2024-05-21T15:40:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.796+0000 7fa158b02640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa15410d430 con 0x7fa154102340 2024-05-21T15:40:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.796+0000 7fa158b02640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa15410d730 con 0x7fa154107040 2024-05-21T15:40:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.796+0000 7fa15259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa154101100 0x7fa154198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.796+0000 7fa15259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa154101100 0x7fa154198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59358/0 (socket says 172.21.15.38:59358) 2024-05-21T15:40:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.796+0000 7fa15259c640 1 -- 172.21.15.38:0/2313586783 learned_addr learned my addr 172.21.15.38:0/2313586783 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.796+0000 7fa151d9b640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa154102340 0x7fa1541986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.796+0000 7fa152d9d640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa154107040 0x7fa15419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.797+0000 7fa151d9b640 1 -- 172.21.15.38:0/2313586783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa154107040 msgr2=0x7fa15419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.797+0000 7fa151d9b640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa154107040 0x7fa15419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.797+0000 7fa151d9b640 1 -- 172.21.15.38:0/2313586783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa154101100 msgr2=0x7fa154198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.797+0000 7fa151d9b640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa154101100 0x7fa154198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.797+0000 7fa151d9b640 1 -- 172.21.15.38:0/2313586783 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15419fca0 con 0x7fa154102340 2024-05-21T15:40:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.797+0000 7fa15259c640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa154101100 0x7fa154198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.797+0000 7fa151d9b640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa154102340 0x7fa1541986a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa148005fd0 tx=0x7fa14800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.797+0000 7fa1437fe640 1 -- 172.21.15.38:0/2313586783 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14800cd00 con 0x7fa154102340 2024-05-21T15:40:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.797+0000 7fa158b02640 1 -- 172.21.15.38:0/2313586783 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15419ff90 con 0x7fa154102340 2024-05-21T15:40:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.797+0000 7fa1437fe640 1 -- 172.21.15.38:0/2313586783 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1480035f0 con 0x7fa154102340 2024-05-21T15:40:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.797+0000 7fa1437fe640 1 -- 172.21.15.38:0/2313586783 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148010690 con 0x7fa154102340 2024-05-21T15:40:16.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.797+0000 7fa158b02640 1 -- 172.21.15.38:0/2313586783 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1541a0400 con 0x7fa154102340 2024-05-21T15:40:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.800+0000 7fa15f9a9740 1 Processor -- start 2024-05-21T15:40:16.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.800+0000 7fa15f9a9740 1 -- start start 2024-05-21T15:40:16.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa15f9a9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d540443390 0x55d5407e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa15f9a9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d54041a4d0 0x55d5407ebb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa15f9a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5404226b0 0x55d5407ee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa15f9a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d540747c40 con 0x55d5404226b0 2024-05-21T15:40:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa15f9a9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d5406ef580 con 0x55d54041a4d0 2024-05-21T15:40:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa15f9a9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d540743c00 con 0x55d540443390 2024-05-21T15:40:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa1437fe640 1 -- 172.21.15.38:0/2313586783 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa148010830 con 0x7fa154102340 2024-05-21T15:40:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa152d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d540443390 0x55d5407e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa15259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d54041a4d0 0x55d5407ebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa152d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d540443390 0x55d5407e95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39136/0 (socket says 172.21.15.38:39136) 2024-05-21T15:40:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa152d9d640 1 -- 172.21.15.38:0/3920336032 learned_addr learned my addr 172.21.15.38:0/3920336032 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa151d9b640 1 --2- 172.21.15.38:0/3920336032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5404226b0 0x55d5407ee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa152d9d640 1 -- 172.21.15.38:0/3920336032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d54041a4d0 msgr2=0x55d5407ebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa152d9d640 1 --2- 172.21.15.38:0/3920336032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d54041a4d0 0x55d5407ebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa152d9d640 1 -- 172.21.15.38:0/3920336032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5404226b0 msgr2=0x55d5407ee2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa152d9d640 1 --2- 172.21.15.38:0/3920336032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5404226b0 0x55d5407ee2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa152d9d640 1 -- 172.21.15.38:0/3920336032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d54045b3f0 con 0x55d540443390 2024-05-21T15:40:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa15259c640 1 --2- 172.21.15.38:0/3920336032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d54041a4d0 0x55d5407ebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.801+0000 7fa151d9b640 1 --2- 172.21.15.38:0/3920336032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5404226b0 0x55d5407ee2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:40:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.802+0000 7fa152d9d640 1 --2- 172.21.15.38:0/3920336032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d540443390 0x55d5407e95c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009fd0 tx=0x7fa14c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b844aab4676da3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.802+0000 7fa140ff9640 1 -- 172.21.15.38:0/3920336032 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c004d20 con 0x55d540443390 2024-05-21T15:40:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.802+0000 7fa140ff9640 1 -- 172.21.15.38:0/3920336032 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa14c004050 con 0x55d540443390 2024-05-21T15:40:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.802+0000 7fa140ff9640 1 -- 172.21.15.38:0/3920336032 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c005240 con 0x55d540443390 2024-05-21T15:40:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.802+0000 7fa1437fe640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa13c0a2b90 0x7fa13c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.802+0000 7fa15259c640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa13c0a2b90 0x7fa13c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.802+0000 7fa1437fe640 1 -- 172.21.15.38:0/2313586783 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa1480cf2a0 con 0x7fa154102340 2024-05-21T15:40:16.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.803+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3920336032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d540443390 msgr2=0x55d5407e95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.803+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3920336032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d540443390 0x55d5407e95c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009fd0 tx=0x7fa14c0118d0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.803+0000 7fa15259c640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa13c0a2b90 0x7fa13c0a50a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa14402a930 tx=0x7fa144000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.803+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3920336032 shutdown_connections 2024-05-21T15:40:16.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.803+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3920336032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5404226b0 0x55d5407ee2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.803+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3920336032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d54041a4d0 0x55d5407ebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.803+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3920336032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d540443390 0x55d5407e95c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.803+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3920336032 >> 172.21.15.38:0/3920336032 conn(0x55d5404374e0 msgr2=0x55d5407ddee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.803+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3920336032 shutdown_connections 2024-05-21T15:40:16.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.803+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3920336032 wait complete. 2024-05-21T15:40:16.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.804+0000 7fa15f9a9740 1 -- start start 2024-05-21T15:40:16.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.804+0000 7fa15f9a9740 1 Processor -- start 2024-05-21T15:40:16.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa15f9a9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d54041a4d0 0x55d5407f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa15f9a9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5404226b0 0x55d5407f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa15f9a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d540443390 0x55d5407f8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa15f9a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d5407eedf0 con 0x55d540443390 2024-05-21T15:40:16.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa15f9a9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d540702c40 con 0x55d54041a4d0 2024-05-21T15:40:16.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa15f9a9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d5407eef70 con 0x55d5404226b0 2024-05-21T15:40:16.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa151d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5404226b0 0x55d5407f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa15259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d54041a4d0 0x55d5407f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa151d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5404226b0 0x55d5407f13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39138/0 (socket says 172.21.15.38:39138) 2024-05-21T15:40:16.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa151d9b640 1 -- 172.21.15.38:0/3328034454 learned_addr learned my addr 172.21.15.38:0/3328034454 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa15259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d54041a4d0 0x55d5407f0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46690/0 (socket says 172.21.15.38:46690) 2024-05-21T15:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa15f9a9740 10 client.0 _ll_register_callbacks cb 0x55d54045abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa152d9d640 1 --2- 172.21.15.38:0/3328034454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d540443390 0x55d5407f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa15259c640 1 -- 172.21.15.38:0/3328034454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5404226b0 msgr2=0x55d5407f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa15259c640 1 --2- 172.21.15.38:0/3328034454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5404226b0 0x55d5407f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa15259c640 1 -- 172.21.15.38:0/3328034454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d540443390 msgr2=0x55d5407f8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa15259c640 1 --2- 172.21.15.38:0/3328034454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d540443390 0x55d5407f8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa15259c640 1 -- 172.21.15.38:0/3328034454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5405257d0 con 0x55d54041a4d0 2024-05-21T15:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa151d9b640 1 --2- 172.21.15.38:0/3328034454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5404226b0 0x55d5407f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.805+0000 7fa152d9d640 1 --2- 172.21.15.38:0/3328034454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d540443390 0x55d5407f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.806+0000 7fa15259c640 1 --2- 172.21.15.38:0/3328034454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d54041a4d0 0x55d5407f0e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa1440305b0 tx=0x7fa14402a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5da85f163b5566ac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.806+0000 7fa139ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.806+0000 7fa139ffb640 1 -- 172.21.15.38:0/3328034454 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144037040 con 0x55d54041a4d0 2024-05-21T15:40:16.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.806+0000 7fa139ffb640 1 -- 172.21.15.38:0/3328034454 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa144005050 con 0x55d54041a4d0 2024-05-21T15:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.806+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3328034454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa14c004050 con 0x55d54041a4d0 2024-05-21T15:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.806+0000 7fa139ffb640 1 -- 172.21.15.38:0/3328034454 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa1440309f0 con 0x55d54041a4d0 2024-05-21T15:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.806+0000 7fa15f9a9740 10 client.26678 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.806+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3328034454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d5407fb610 con 0x55d54041a4d0 2024-05-21T15:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.806+0000 7fa139ffb640 1 -- 172.21.15.38:0/3328034454 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa14c004050 con 0x55d54041a4d0 2024-05-21T15:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.807+0000 7fa139ffb640 1 -- 172.21.15.38:0/3328034454 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fa144050030 con 0x55d54041a4d0 2024-05-21T15:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.807+0000 7fa15f9a9740 10 client.26678 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.807+0000 7fa15f9a9740 10 client.26678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.807+0000 7fa15f9a9740 20 client.26678 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.807+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3328034454 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d540401f60 0x55d5407fede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.807+0000 7fa15f9a9740 4 client.26678 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.807+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3328034454 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d540755370 con 0x55d540401f60 2024-05-21T15:40:16.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.807+0000 7fa151d9b640 1 --2- 172.21.15.38:0/3328034454 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d540401f60 0x55d5407fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.808+0000 7fa139ffb640 10 client.26678 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.808+0000 7fa151d9b640 1 --2- 172.21.15.38:0/3328034454 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d540401f60 0x55d5407fede0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6718ed428e9f4eb8 server_cookie=303e864b70cc291c in_seq=0 out_seq=0 2024-05-21T15:40:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa139ffb640 1 -- 172.21.15.38:0/3328034454 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d540755370 con 0x55d540401f60 2024-05-21T15:40:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa139ffb640 10 client.26678 handle_command_reply: tid=0 2024-05-21T15:40:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa15f9a9740 1 client.26678 shutdown 2024-05-21T15:40:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3328034454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d54041a4d0 msgr2=0x55d5407f0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3328034454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d54041a4d0 0x55d5407f0e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa1440305b0 tx=0x7fa14402a340 comp rx=0 tx=0).stop 2024-05-21T15:40:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3328034454 shutdown_connections 2024-05-21T15:40:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3328034454 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d540401f60 0x55d5407fede0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3328034454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d540443390 0x55d5407f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3328034454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5404226b0 0x55d5407f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa139ffb640 0 client.26678 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3328034454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d54041a4d0 0x55d5407f0e60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3328034454 >> 172.21.15.38:0/3328034454 conn(0x55d5404374e0 msgr2=0x55d5407dd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3328034454 shutdown_connections 2024-05-21T15:40:16.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3328034454 wait complete. 2024-05-21T15:40:16.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.809+0000 7fa15f9a9740 20 client.26678 trim_cache size 0 max 16384 2024-05-21T15:40:16.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.828+0000 7fa15f9a9740 1 Processor -- start 2024-05-21T15:40:16.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.828+0000 7fa15f9a9740 1 -- start start 2024-05-21T15:40:16.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.828+0000 7fa15f9a9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d540401f60 0x55d5407efb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.828+0000 7fa15f9a9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d54041a4d0 0x55d5407dcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.828+0000 7fa15f9a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5404226b0 0x55d5407dd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.828+0000 7fa15f9a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa1541a8210 con 0x55d5404226b0 2024-05-21T15:40:16.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.828+0000 7fa15f9a9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1541a9e40 con 0x55d540401f60 2024-05-21T15:40:16.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.828+0000 7fa15f9a9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa1541a88e0 con 0x55d54041a4d0 2024-05-21T15:40:16.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.828+0000 7fa152d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5404226b0 0x55d5407dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.829+0000 7fa152d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5404226b0 0x55d5407dd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59372/0 (socket says 172.21.15.38:59372) 2024-05-21T15:40:16.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.829+0000 7fa15259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d540401f60 0x55d5407efb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.829+0000 7fa152d9d640 1 -- 172.21.15.38:0/1768619484 learned_addr learned my addr 172.21.15.38:0/1768619484 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.829+0000 7fa151d9b640 1 --2- 172.21.15.38:0/1768619484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d54041a4d0 0x55d5407dcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.829+0000 7fa15259c640 1 -- 172.21.15.38:0/1768619484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d54041a4d0 msgr2=0x55d5407dcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.829+0000 7fa15259c640 1 --2- 172.21.15.38:0/1768619484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d54041a4d0 0x55d5407dcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.829+0000 7fa15259c640 1 -- 172.21.15.38:0/1768619484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5404226b0 msgr2=0x55d5407dd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.829+0000 7fa15259c640 1 --2- 172.21.15.38:0/1768619484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5404226b0 0x55d5407dd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.829+0000 7fa15259c640 1 -- 172.21.15.38:0/1768619484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5407dec30 con 0x55d540401f60 2024-05-21T15:40:16.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.829+0000 7fa15259c640 1 --2- 172.21.15.38:0/1768619484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d540401f60 0x55d5407efb90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa14402a860 tx=0x7fa144004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=499a00e894fc8ecd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.830+0000 7fa13bfff640 1 -- 172.21.15.38:0/1768619484 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144061070 con 0x55d540401f60 2024-05-21T15:40:16.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.830+0000 7fa13bfff640 1 -- 172.21.15.38:0/1768619484 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa144053500 con 0x55d540401f60 2024-05-21T15:40:16.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.830+0000 7fa13bfff640 1 -- 172.21.15.38:0/1768619484 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144037040 con 0x55d540401f60 2024-05-21T15:40:16.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.831+0000 7fa15f9a9740 1 -- 172.21.15.38:0/1768619484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d540401f60 msgr2=0x55d5407efb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.831+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/1768619484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d540401f60 0x55d5407efb90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa14402a860 tx=0x7fa144004610 comp rx=0 tx=0).stop 2024-05-21T15:40:16.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.831+0000 7fa15f9a9740 1 -- 172.21.15.38:0/1768619484 shutdown_connections 2024-05-21T15:40:16.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.831+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/1768619484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5404226b0 0x55d5407dd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.831+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/1768619484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d54041a4d0 0x55d5407dcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.831+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/1768619484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d540401f60 0x55d5407efb90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.831+0000 7fa15f9a9740 1 -- 172.21.15.38:0/1768619484 >> 172.21.15.38:0/1768619484 conn(0x55d5404374e0 msgr2=0x55d5407ef480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.831+0000 7fa15f9a9740 1 -- 172.21.15.38:0/1768619484 shutdown_connections 2024-05-21T15:40:16.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.831+0000 7fa15f9a9740 1 -- 172.21.15.38:0/1768619484 wait complete. 2024-05-21T15:40:16.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.832+0000 7fa15f9a9740 1 -- start start 2024-05-21T15:40:16.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.832+0000 7fa15f9a9740 1 Processor -- start 2024-05-21T15:40:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.832+0000 7fa15f9a9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d540401f60 0x55d5407f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.832+0000 7fa15f9a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d54041a4d0 0x55d5407f5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.832+0000 7fa15f9a9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5404226b0 0x55d5407f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.832+0000 7fa15f9a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa1541a96b0 con 0x55d54041a4d0 2024-05-21T15:40:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.832+0000 7fa15f9a9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1541a8fb0 con 0x55d5404226b0 2024-05-21T15:40:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.832+0000 7fa15f9a9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa1541a61f0 con 0x55d540401f60 2024-05-21T15:40:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.833+0000 7fa15f9a9740 10 client.0 _ll_register_callbacks cb 0x7fa14c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.833+0000 7fa152d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5404226b0 0x55d5407f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.833+0000 7fa15259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d540401f60 0x55d5407f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.833+0000 7fa152d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5404226b0 0x55d5407f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46722/0 (socket says 172.21.15.38:46722) 2024-05-21T15:40:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.833+0000 7fa15259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d540401f60 0x55d5407f4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39166/0 (socket says 172.21.15.38:39166) 2024-05-21T15:40:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.833+0000 7fa152d9d640 1 -- 172.21.15.38:0/3204432572 learned_addr learned my addr 172.21.15.38:0/3204432572 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.833+0000 7fa151d9b640 1 --2- 172.21.15.38:0/3204432572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d54041a4d0 0x55d5407f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.833+0000 7fa152d9d640 1 -- 172.21.15.38:0/3204432572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d540401f60 msgr2=0x55d5407f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.833+0000 7fa152d9d640 1 --2- 172.21.15.38:0/3204432572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d540401f60 0x55d5407f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.833+0000 7fa152d9d640 1 -- 172.21.15.38:0/3204432572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d54041a4d0 msgr2=0x55d5407f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.833+0000 7fa152d9d640 1 --2- 172.21.15.38:0/3204432572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d54041a4d0 0x55d5407f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.833+0000 7fa152d9d640 1 -- 172.21.15.38:0/3204432572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5407e4030 con 0x55d5404226b0 2024-05-21T15:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.833+0000 7fa15259c640 1 --2- 172.21.15.38:0/3204432572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d540401f60 0x55d5407f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.833+0000 7fa151d9b640 1 --2- 172.21.15.38:0/3204432572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d54041a4d0 0x55d5407f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.834+0000 7fa152d9d640 1 --2- 172.21.15.38:0/3204432572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5404226b0 0x55d5407f5f80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55d5407dee10 tx=0x7fa14c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f0e25cad76f87d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.834+0000 7fa139ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.834+0000 7fa139ffb640 1 -- 172.21.15.38:0/3204432572 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c0055e0 con 0x55d5404226b0 2024-05-21T15:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.834+0000 7fa139ffb640 1 -- 172.21.15.38:0/3204432572 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa14c005780 con 0x55d5404226b0 2024-05-21T15:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.834+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3204432572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa144053500 con 0x55d5404226b0 2024-05-21T15:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.834+0000 7fa139ffb640 1 -- 172.21.15.38:0/3204432572 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa14c005980 con 0x55d5404226b0 2024-05-21T15:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.834+0000 7fa15f9a9740 10 client.26684 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.834+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3204432572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d5407f7440 con 0x55d5404226b0 2024-05-21T15:40:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.834+0000 7fa139ffb640 1 -- 172.21.15.38:0/3204432572 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa144053500 con 0x55d5404226b0 2024-05-21T15:40:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.835+0000 7fa139ffb640 1 -- 172.21.15.38:0/3204432572 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fa14c02d800 con 0x55d5404226b0 2024-05-21T15:40:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.835+0000 7fa15f9a9740 10 client.26684 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.835+0000 7fa15f9a9740 10 client.26684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.835+0000 7fa15f9a9740 20 client.26684 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.835+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3204432572 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d540443390 0x55d54086d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.835+0000 7fa15f9a9740 4 client.26684 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:40:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.835+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3204432572 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d54074f6c0 con 0x55d540443390 2024-05-21T15:40:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.835+0000 7fa15259c640 1 --2- 172.21.15.38:0/3204432572 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d540443390 0x55d54086d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.835+0000 7fa139ffb640 10 client.26684 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:16.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.835+0000 7fa15259c640 1 --2- 172.21.15.38:0/3204432572 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d540443390 0x55d54086d990 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60b5096998276070 server_cookie=fc5edc147e0df06e in_seq=0 out_seq=0 2024-05-21T15:40:16.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa139ffb640 1 -- 172.21.15.38:0/3204432572 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d54074f6c0 con 0x55d540443390 2024-05-21T15:40:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa139ffb640 10 client.26684 handle_command_reply: tid=0 2024-05-21T15:40:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa15f9a9740 1 client.26684 shutdown 2024-05-21T15:40:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3204432572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5404226b0 msgr2=0x55d5407f5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3204432572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5404226b0 0x55d5407f5f80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55d5407dee10 tx=0x7fa14c005270 comp rx=0 tx=0).stop 2024-05-21T15:40:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3204432572 shutdown_connections 2024-05-21T15:40:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3204432572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5404226b0 0x55d5407f5f80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3204432572 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d540443390 0x55d54086d990 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3204432572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d54041a4d0 0x55d5407f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa15f9a9740 1 --2- 172.21.15.38:0/3204432572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d540401f60 0x55d5407f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa139ffb640 0 client.26684 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3204432572 >> 172.21.15.38:0/3204432572 conn(0x55d5404374e0 msgr2=0x55d5407f1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3204432572 shutdown_connections 2024-05-21T15:40:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa15f9a9740 1 -- 172.21.15.38:0/3204432572 wait complete. 2024-05-21T15:40:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.836+0000 7fa15f9a9740 20 client.26684 trim_cache size 0 max 16384 2024-05-21T15:40:16.839 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:40:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.837+0000 7fa158b02640 1 -- 172.21.15.38:0/2313586783 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa13c0a2b90 msgr2=0x7fa13c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.837+0000 7fa158b02640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa13c0a2b90 0x7fa13c0a50a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa14402a930 tx=0x7fa144000bc0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.837+0000 7fa158b02640 1 -- 172.21.15.38:0/2313586783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa154102340 msgr2=0x7fa1541986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.837+0000 7fa158b02640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa154102340 0x7fa1541986a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa148005fd0 tx=0x7fa14800ee50 comp rx=0 tx=0).stop 2024-05-21T15:40:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.837+0000 7fa158b02640 1 -- 172.21.15.38:0/2313586783 shutdown_connections 2024-05-21T15:40:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.837+0000 7fa158b02640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa154107040 0x7fa15419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.837+0000 7fa158b02640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa154102340 0x7fa1541986a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.837+0000 7fa158b02640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa13c0a2b90 0x7fa13c0a50a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.837+0000 7fa158b02640 1 --2- 172.21.15.38:0/2313586783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa154101100 0x7fa154198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.837+0000 7fa158b02640 1 -- 172.21.15.38:0/2313586783 >> 172.21.15.38:0/2313586783 conn(0x7fa1540fc2f0 msgr2=0x7fa154103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.837+0000 7fa158b02640 1 -- 172.21.15.38:0/2313586783 shutdown_connections 2024-05-21T15:40:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:16.837+0000 7fa158b02640 1 -- 172.21.15.38:0/2313586783 wait complete. 2024-05-21T15:40:17.853 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:40:17.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:40:17.853 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:40:18.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.056+0000 7fcd53cca640 1 -- 172.21.15.38:0/1007256618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4c101740 msgr2=0x7fcd4c101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.056+0000 7fcd53cca640 1 --2- 172.21.15.38:0/1007256618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4c101740 0x7fcd4c101c10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009900 tx=0x7fcd4402a440 comp rx=0 tx=0).stop 2024-05-21T15:40:18.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.059+0000 7fcd53cca640 1 -- 172.21.15.38:0/1007256618 shutdown_connections 2024-05-21T15:40:18.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.059+0000 7fcd53cca640 1 --2- 172.21.15.38:0/1007256618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd4c1068b0 0x7fcd4c106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.059+0000 7fcd53cca640 1 --2- 172.21.15.38:0/1007256618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4c101740 0x7fcd4c101c10 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.059+0000 7fcd53cca640 1 --2- 172.21.15.38:0/1007256618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd4c100d00 0x7fcd4c101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.059+0000 7fcd53cca640 1 -- 172.21.15.38:0/1007256618 >> 172.21.15.38:0/1007256618 conn(0x7fcd4c0fc2f0 msgr2=0x7fcd4c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:18.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.060+0000 7fcd53cca640 1 -- 172.21.15.38:0/1007256618 shutdown_connections 2024-05-21T15:40:18.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.061+0000 7fcd53cca640 1 -- 172.21.15.38:0/1007256618 wait complete. 2024-05-21T15:40:18.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.062+0000 7fcd53cca640 1 Processor -- start 2024-05-21T15:40:18.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.062+0000 7fcd53cca640 1 -- start start 2024-05-21T15:40:18.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.062+0000 7fcd53cca640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd4c100d00 0x7fcd4c19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.062+0000 7fcd53cca640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4c101740 0x7fcd4c19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.062+0000 7fcd53cca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd4c1068b0 0x7fcd4c1a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.062+0000 7fcd53cca640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c114e30 con 0x7fcd4c101740 2024-05-21T15:40:18.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.062+0000 7fcd53cca640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c114cb0 con 0x7fcd4c100d00 2024-05-21T15:40:18.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.062+0000 7fcd53cca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c114fb0 con 0x7fcd4c1068b0 2024-05-21T15:40:18.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.063+0000 7fcd52267640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd4c1068b0 0x7fcd4c1a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.064+0000 7fcd52267640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd4c1068b0 0x7fcd4c1a6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39170/0 (socket says 172.21.15.38:39170) 2024-05-21T15:40:18.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.064+0000 7fcd52267640 1 -- 172.21.15.38:0/1762904308 learned_addr learned my addr 172.21.15.38:0/1762904308 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:18.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.065+0000 7fcd51265640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4c101740 0x7fcd4c19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.065+0000 7fcd52267640 1 -- 172.21.15.38:0/1762904308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd4c100d00 msgr2=0x7fcd4c19f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.065+0000 7fcd51a66640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd4c100d00 0x7fcd4c19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.065+0000 7fcd52267640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd4c100d00 0x7fcd4c19f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.065+0000 7fcd52267640 1 -- 172.21.15.38:0/1762904308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4c101740 msgr2=0x7fcd4c19fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.065+0000 7fcd52267640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4c101740 0x7fcd4c19fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.065+0000 7fcd52267640 1 -- 172.21.15.38:0/1762904308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4c1a73f0 con 0x7fcd4c1068b0 2024-05-21T15:40:18.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.065+0000 7fcd51265640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4c101740 0x7fcd4c19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:40:18.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.065+0000 7fcd51a66640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd4c100d00 0x7fcd4c19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:40:18.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.066+0000 7fcd52267640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd4c1068b0 0x7fcd4c1a6d80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00eb10 tx=0x7fcd3c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:18.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.067+0000 7fcd4affd640 1 -- 172.21.15.38:0/1762904308 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c004b20 con 0x7fcd4c1068b0 2024-05-21T15:40:18.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.067+0000 7fcd4affd640 1 -- 172.21.15.38:0/1762904308 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd3c004cc0 con 0x7fcd4c1068b0 2024-05-21T15:40:18.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.067+0000 7fcd4affd640 1 -- 172.21.15.38:0/1762904308 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c010760 con 0x7fcd4c1068b0 2024-05-21T15:40:18.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.068+0000 7fcd53cca640 1 -- 172.21.15.38:0/1762904308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4c1a76e0 con 0x7fcd4c1068b0 2024-05-21T15:40:18.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.068+0000 7fcd53cca640 1 -- 172.21.15.38:0/1762904308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd4c1a7c60 con 0x7fcd4c1068b0 2024-05-21T15:40:18.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.074+0000 7fcd5ab6d740 1 Processor -- start 2024-05-21T15:40:18.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.079+0000 7fcd4affd640 1 -- 172.21.15.38:0/1762904308 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcd3c010900 con 0x7fcd4c1068b0 2024-05-21T15:40:18.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.080+0000 7fcd5ab6d740 1 -- start start 2024-05-21T15:40:18.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.080+0000 7fcd5ab6d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12268410 0x55cc126084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.080+0000 7fcd5ab6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc1223f550 0x55cc12608a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.080+0000 7fcd5ab6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc12247730 0x55cc1260f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.080+0000 7fcd5ab6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cc1256c990 con 0x55cc12247730 2024-05-21T15:40:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.080+0000 7fcd5ab6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cc125145b0 con 0x55cc1223f550 2024-05-21T15:40:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.080+0000 7fcd5ab6d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cc12568950 con 0x55cc12268410 2024-05-21T15:40:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.081+0000 7fcd4affd640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcd340a2d90 0x7fcd340a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.081+0000 7fcd52267640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcd340a2d90 0x7fcd340a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.082+0000 7fcd51265640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12268410 0x55cc126084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.082+0000 7fcd51265640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12268410 0x55cc126084a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39172/0 (socket says 172.21.15.38:39172) 2024-05-21T15:40:18.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.082+0000 7fcd51265640 1 -- 172.21.15.38:0/3832688450 learned_addr learned my addr 172.21.15.38:0/3832688450 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:18.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.082+0000 7fcd4affd640 1 -- 172.21.15.38:0/1762904308 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fcd3c0cf6e0 con 0x7fcd4c1068b0 2024-05-21T15:40:18.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.082+0000 7fcd51a66640 1 --2- 172.21.15.38:0/3832688450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc1223f550 0x55cc12608a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.085+0000 7fcd51265640 1 --2- 172.21.15.38:0/3832688450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc12247730 0x55cc1260f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.086+0000 7fcd51265640 1 -- 172.21.15.38:0/3832688450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc1223f550 msgr2=0x55cc12608a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.086+0000 7fcd51265640 1 --2- 172.21.15.38:0/3832688450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc1223f550 0x55cc12608a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.086+0000 7fcd51265640 1 -- 172.21.15.38:0/3832688450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc12247730 msgr2=0x55cc1260f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.086+0000 7fcd51265640 1 --2- 172.21.15.38:0/3832688450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc12247730 0x55cc1260f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.086+0000 7fcd51265640 1 -- 172.21.15.38:0/3832688450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc123487e0 con 0x55cc12268410 2024-05-21T15:40:18.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.086+0000 7fcd51265640 1 --2- 172.21.15.38:0/3832688450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12268410 0x55cc126084a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcd44031610 tx=0x7fcd44031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=325aa6d27192a74c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:18.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.087+0000 7fcd33fff640 1 -- 172.21.15.38:0/3832688450 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44045070 con 0x55cc12268410 2024-05-21T15:40:18.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.087+0000 7fcd33fff640 1 -- 172.21.15.38:0/3832688450 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd44031b90 con 0x55cc12268410 2024-05-21T15:40:18.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.087+0000 7fcd33fff640 1 -- 172.21.15.38:0/3832688450 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44035040 con 0x55cc12268410 2024-05-21T15:40:18.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.089+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/3832688450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12268410 msgr2=0x55cc126084a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.089+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/3832688450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12268410 0x55cc126084a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcd44031610 tx=0x7fcd44031400 comp rx=0 tx=0).stop 2024-05-21T15:40:18.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.093+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/3832688450 shutdown_connections 2024-05-21T15:40:18.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.093+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/3832688450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc12247730 0x55cc1260f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.093+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/3832688450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc1223f550 0x55cc12608a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.093+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/3832688450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12268410 0x55cc126084a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.093+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/3832688450 >> 172.21.15.38:0/3832688450 conn(0x55cc1225c560 msgr2=0x55cc12603e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:18.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.093+0000 7fcd52267640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcd340a2d90 0x7fcd340a52a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c09bb90 tx=0x7fcd3c097a80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:18.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.096+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/3832688450 shutdown_connections 2024-05-21T15:40:18.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.097+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/3832688450 wait complete. 2024-05-21T15:40:18.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.098+0000 7fcd5ab6d740 1 -- start start 2024-05-21T15:40:18.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.098+0000 7fcd5ab6d740 1 Processor -- start 2024-05-21T15:40:18.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.100+0000 7fcd5ab6d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1223f550 0x55cc12612420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.100+0000 7fcd5ab6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc12247730 0x55cc12612960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.100+0000 7fcd5ab6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc12268410 0x55cc126198c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.100+0000 7fcd5ab6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cc12610310 con 0x55cc12247730 2024-05-21T15:40:18.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.100+0000 7fcd5ab6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cc12527b40 con 0x55cc12268410 2024-05-21T15:40:18.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.100+0000 7fcd5ab6d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cc12610490 con 0x55cc1223f550 2024-05-21T15:40:18.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.100+0000 7fcd5ab6d740 10 client.0 _ll_register_callbacks cb 0x55cc1251c0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:18.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.104+0000 7fcd51a66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc12247730 0x55cc12612960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.104+0000 7fcd51265640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1223f550 0x55cc12612420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.104+0000 7fcd51265640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1223f550 0x55cc12612420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39184/0 (socket says 172.21.15.38:39184) 2024-05-21T15:40:18.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.104+0000 7fcd51265640 1 -- 172.21.15.38:0/2342024020 learned_addr learned my addr 172.21.15.38:0/2342024020 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:18.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.105+0000 7fcd51265640 1 --2- 172.21.15.38:0/2342024020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc12268410 0x55cc126198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.108+0000 7fcd51265640 1 -- 172.21.15.38:0/2342024020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc12268410 msgr2=0x55cc126198c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.108+0000 7fcd51265640 1 --2- 172.21.15.38:0/2342024020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc12268410 0x55cc126198c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.108+0000 7fcd51265640 1 -- 172.21.15.38:0/2342024020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc12247730 msgr2=0x55cc12612960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.108+0000 7fcd51265640 1 --2- 172.21.15.38:0/2342024020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc12247730 0x55cc12612960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.108+0000 7fcd51265640 1 -- 172.21.15.38:0/2342024020 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc123a8f70 con 0x55cc1223f550 2024-05-21T15:40:18.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.110+0000 7fcd51265640 1 --2- 172.21.15.38:0/2342024020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1223f550 0x55cc12612420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcd44004650 tx=0x7fcd44009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c51909e6c964e9c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.111+0000 7fcd317fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:40:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.111+0000 7fcd317fa640 1 -- 172.21.15.38:0/2342024020 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44035070 con 0x55cc1223f550 2024-05-21T15:40:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.111+0000 7fcd317fa640 1 -- 172.21.15.38:0/2342024020 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fcd44005050 con 0x55cc1223f550 2024-05-21T15:40:18.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.112+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2342024020 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd44031b90 con 0x55cc1223f550 2024-05-21T15:40:18.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.115+0000 7fcd317fa640 1 -- 172.21.15.38:0/2342024020 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcd44045040 con 0x55cc1223f550 2024-05-21T15:40:18.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.116+0000 7fcd5ab6d740 10 client.26752 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:18.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.116+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2342024020 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cc12240170 con 0x55cc1223f550 2024-05-21T15:40:18.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.118+0000 7fcd317fa640 1 -- 172.21.15.38:0/2342024020 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd44031b90 con 0x55cc1223f550 2024-05-21T15:40:18.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.118+0000 7fcd317fa640 1 -- 172.21.15.38:0/2342024020 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fcd4404b070 con 0x55cc1223f550 2024-05-21T15:40:18.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.119+0000 7fcd5ab6d740 10 client.26752 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:18.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.119+0000 7fcd5ab6d740 10 client.26752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:18.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.119+0000 7fcd5ab6d740 20 client.26752 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:18.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.119+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/2342024020 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55cc122271f0 0x55cc12630990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.119+0000 7fcd5ab6d740 4 client.26752 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:18.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.119+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2342024020 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc1257a0c0 con 0x55cc122271f0 2024-05-21T15:40:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.121+0000 7fcd51a66640 1 --2- 172.21.15.38:0/2342024020 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55cc122271f0 0x55cc12630990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.130+0000 7fcd51a66640 1 --2- 172.21.15.38:0/2342024020 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55cc122271f0 0x55cc12630990 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f9e322bf7696394 server_cookie=7d91538529c13e23 in_seq=0 out_seq=0 2024-05-21T15:40:18.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.130+0000 7fcd317fa640 10 client.26752 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:18.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.132+0000 7fcd317fa640 1 -- 172.21.15.38:0/2342024020 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cc1257a0c0 con 0x55cc122271f0 2024-05-21T15:40:18.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.132+0000 7fcd317fa640 10 client.26752 handle_command_reply: tid=0 2024-05-21T15:40:18.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.132+0000 7fcd5ab6d740 1 client.26752 shutdown 2024-05-21T15:40:18.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.133+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2342024020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1223f550 msgr2=0x55cc12612420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.133+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/2342024020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1223f550 0x55cc12612420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcd44004650 tx=0x7fcd44009640 comp rx=0 tx=0).stop 2024-05-21T15:40:18.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.134+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2342024020 shutdown_connections 2024-05-21T15:40:18.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.134+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/2342024020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc12268410 0x55cc126198c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.134+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/2342024020 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55cc122271f0 0x55cc12630990 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.134+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/2342024020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc12247730 0x55cc12612960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.134+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/2342024020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc1223f550 0x55cc12612420 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.134+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2342024020 >> 172.21.15.38:0/2342024020 conn(0x55cc1225c560 msgr2=0x55cc12602e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:18.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.134+0000 7fcd317fa640 0 client.26752 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:18.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.135+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2342024020 shutdown_connections 2024-05-21T15:40:18.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.137+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2342024020 wait complete. 2024-05-21T15:40:18.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.139+0000 7fcd5ab6d740 20 client.26752 trim_cache size 0 max 16384 2024-05-21T15:40:18.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.180+0000 7fcd5ab6d740 1 Processor -- start 2024-05-21T15:40:18.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.180+0000 7fcd5ab6d740 1 -- start start 2024-05-21T15:40:18.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.181+0000 7fcd5ab6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc1223f550 0x55cc126116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.181+0000 7fcd5ab6d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12247730 0x55cc12611bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.181+0000 7fcd5ab6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc12268410 0x55cc12603790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.181+0000 7fcd5ab6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3800e600 con 0x55cc12268410 2024-05-21T15:40:18.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.181+0000 7fcd5ab6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd38010230 con 0x55cc1223f550 2024-05-21T15:40:18.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.181+0000 7fcd5ab6d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3800ecd0 con 0x55cc12247730 2024-05-21T15:40:18.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.181+0000 7fcd51a66640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12247730 0x55cc12611bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.181+0000 7fcd51a66640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12247730 0x55cc12611bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39190/0 (socket says 172.21.15.38:39190) 2024-05-21T15:40:18.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.181+0000 7fcd51a66640 1 -- 172.21.15.38:0/3670900607 learned_addr learned my addr 172.21.15.38:0/3670900607 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:18.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.181+0000 7fcd51265640 1 --2- 172.21.15.38:0/3670900607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc1223f550 0x55cc126116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.182+0000 7fcd51a66640 1 -- 172.21.15.38:0/3670900607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc1223f550 msgr2=0x55cc126116b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.182+0000 7fcd51a66640 1 --2- 172.21.15.38:0/3670900607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc1223f550 0x55cc126116b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.182+0000 7fcd51a66640 1 -- 172.21.15.38:0/3670900607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc12268410 msgr2=0x55cc12603790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.182+0000 7fcd51a66640 1 --2- 172.21.15.38:0/3670900607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc12268410 0x55cc12603790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.182+0000 7fcd51a66640 1 -- 172.21.15.38:0/3670900607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc125fee90 con 0x55cc12247730 2024-05-21T15:40:18.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.183+0000 7fcd51a66640 1 --2- 172.21.15.38:0/3670900607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12247730 0x55cc12611bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55cc12604bc0 tx=0x7fcd40008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28e2f3f97473dfdc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:18.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.188+0000 7fcd337fe640 1 -- 172.21.15.38:0/3670900607 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd400158d0 con 0x55cc12247730 2024-05-21T15:40:18.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.188+0000 7fcd337fe640 1 -- 172.21.15.38:0/3670900607 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd4001b250 con 0x55cc12247730 2024-05-21T15:40:18.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.189+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/3670900607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12247730 msgr2=0x55cc12611bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.189+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/3670900607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12247730 0x55cc12611bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55cc12604bc0 tx=0x7fcd40008040 comp rx=0 tx=0).stop 2024-05-21T15:40:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.190+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/3670900607 shutdown_connections 2024-05-21T15:40:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.190+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/3670900607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc12268410 0x55cc12603790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.190+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/3670900607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12247730 0x55cc12611bf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.190+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/3670900607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc1223f550 0x55cc126116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.190+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/3670900607 >> 172.21.15.38:0/3670900607 conn(0x55cc1225c560 msgr2=0x55cc12610a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:18.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.194+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/3670900607 shutdown_connections 2024-05-21T15:40:18.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.194+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/3670900607 wait complete. 2024-05-21T15:40:18.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.194+0000 7fcd5ab6d740 1 -- start start 2024-05-21T15:40:18.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.194+0000 7fcd5ab6d740 1 Processor -- start 2024-05-21T15:40:18.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.195+0000 7fcd5ab6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc1223f550 0x55cc126163f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.195+0000 7fcd5ab6d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12247730 0x55cc12616930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.195+0000 7fcd5ab6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc12268410 0x55cc12617860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.195+0000 7fcd5ab6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3800faa0 con 0x55cc1223f550 2024-05-21T15:40:18.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.195+0000 7fcd5ab6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3800f3a0 con 0x55cc12268410 2024-05-21T15:40:18.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.195+0000 7fcd5ab6d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3800c5e0 con 0x55cc12247730 2024-05-21T15:40:18.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.195+0000 7fcd5ab6d740 10 client.0 _ll_register_callbacks cb 0x7fcd44043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:18.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.196+0000 7fcd51265640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc1223f550 0x55cc126163f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.196+0000 7fcd51265640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc1223f550 0x55cc126163f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59442/0 (socket says 172.21.15.38:59442) 2024-05-21T15:40:18.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.196+0000 7fcd51265640 1 -- 172.21.15.38:0/2882191127 learned_addr learned my addr 172.21.15.38:0/2882191127 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:18.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.196+0000 7fcd51265640 1 --2- 172.21.15.38:0/2882191127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc12268410 0x55cc12617860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.196+0000 7fcd51a66640 1 --2- 172.21.15.38:0/2882191127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12247730 0x55cc12616930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.196+0000 7fcd51a66640 1 -- 172.21.15.38:0/2882191127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc12268410 msgr2=0x55cc12617860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.196+0000 7fcd51a66640 1 --2- 172.21.15.38:0/2882191127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc12268410 0x55cc12617860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.196+0000 7fcd51a66640 1 -- 172.21.15.38:0/2882191127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc1223f550 msgr2=0x55cc126163f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.196+0000 7fcd51a66640 1 --2- 172.21.15.38:0/2882191127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc1223f550 0x55cc126163f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.196+0000 7fcd51a66640 1 -- 172.21.15.38:0/2882191127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc12617e20 con 0x55cc12247730 2024-05-21T15:40:18.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.197+0000 7fcd51a66640 1 --2- 172.21.15.38:0/2882191127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12247730 0x55cc12616930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcd400047b0 tx=0x7fcd4001b0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5be4c2959fb46638 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:18.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.198+0000 7fcd317fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:40:18.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.198+0000 7fcd317fa640 1 -- 172.21.15.38:0/2882191127 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4001b810 con 0x55cc12247730 2024-05-21T15:40:18.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.198+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2882191127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd4001b250 con 0x55cc12247730 2024-05-21T15:40:18.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.198+0000 7fcd317fa640 1 -- 172.21.15.38:0/2882191127 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fcd400158d0 con 0x55cc12247730 2024-05-21T15:40:18.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.199+0000 7fcd317fa640 1 -- 172.21.15.38:0/2882191127 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcd40015af0 con 0x55cc12247730 2024-05-21T15:40:18.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.199+0000 7fcd5ab6d740 10 client.26758 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:18.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.200+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2882191127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cc12619110 con 0x55cc12247730 2024-05-21T15:40:18.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.201+0000 7fcd317fa640 1 -- 172.21.15.38:0/2882191127 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd4001b250 con 0x55cc12247730 2024-05-21T15:40:18.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.201+0000 7fcd317fa640 1 -- 172.21.15.38:0/2882191127 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fcd4001bce0 con 0x55cc12247730 2024-05-21T15:40:18.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.201+0000 7fcd5ab6d740 10 client.26758 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:18.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.201+0000 7fcd5ab6d740 10 client.26758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:18.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.201+0000 7fcd5ab6d740 20 client.26758 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:18.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.201+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/2882191127 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55cc121fc630 0x55cc126ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.201+0000 7fcd5ab6d740 4 client.26758 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:40:18.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.201+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2882191127 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cc12574410 con 0x55cc121fc630 2024-05-21T15:40:18.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.201+0000 7fcd51265640 1 --2- 172.21.15.38:0/2882191127 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55cc121fc630 0x55cc126ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.202+0000 7fcd317fa640 10 client.26758 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:18.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.202+0000 7fcd51265640 1 --2- 172.21.15.38:0/2882191127 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55cc121fc630 0x55cc126ac350 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14f6039bf2f6363f server_cookie=29685c42df61a224 in_seq=0 out_seq=0 2024-05-21T15:40:18.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:17 smithi053 ceph-mon[36496]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.14k op/s 2024-05-21T15:40:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.206+0000 7fcd317fa640 1 -- 172.21.15.38:0/2882191127 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cc12574410 con 0x55cc121fc630 2024-05-21T15:40:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.206+0000 7fcd317fa640 10 client.26758 handle_command_reply: tid=0 2024-05-21T15:40:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.206+0000 7fcd5ab6d740 1 client.26758 shutdown 2024-05-21T15:40:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.210+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2882191127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12247730 msgr2=0x55cc12616930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.210+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/2882191127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12247730 0x55cc12616930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcd400047b0 tx=0x7fcd4001b0e0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.210+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2882191127 shutdown_connections 2024-05-21T15:40:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.210+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/2882191127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc12268410 0x55cc12617860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.210+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/2882191127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc12247730 0x55cc12616930 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.210+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/2882191127 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55cc121fc630 0x55cc126ac350 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.210+0000 7fcd5ab6d740 1 --2- 172.21.15.38:0/2882191127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cc1223f550 0x55cc126163f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.210+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2882191127 >> 172.21.15.38:0/2882191127 conn(0x55cc1225c560 msgr2=0x55cc126116b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.210+0000 7fcd317fa640 0 client.26758 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.210+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2882191127 shutdown_connections 2024-05-21T15:40:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.210+0000 7fcd5ab6d740 1 -- 172.21.15.38:0/2882191127 wait complete. 2024-05-21T15:40:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.210+0000 7fcd5ab6d740 20 client.26758 trim_cache size 0 max 16384 2024-05-21T15:40:18.210 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:40:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-05-21T15:40:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "da2fb5b5-af83-44d2-bc65-2c1d09f57ed3", 2024-05-21T15:40:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-05-21T15:40:18.210 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:40:18.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.212+0000 7fcd53cca640 1 -- 172.21.15.38:0/1762904308 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcd340a2d90 msgr2=0x7fcd340a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.212+0000 7fcd53cca640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcd340a2d90 0x7fcd340a52a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c09bb90 tx=0x7fcd3c097a80 comp rx=0 tx=0).stop 2024-05-21T15:40:18.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.212+0000 7fcd53cca640 1 -- 172.21.15.38:0/1762904308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd4c1068b0 msgr2=0x7fcd4c1a6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.212+0000 7fcd53cca640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd4c1068b0 0x7fcd4c1a6d80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00eb10 tx=0x7fcd3c00c2d0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.215+0000 7fcd53cca640 1 -- 172.21.15.38:0/1762904308 shutdown_connections 2024-05-21T15:40:18.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.215+0000 7fcd53cca640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd4c1068b0 0x7fcd4c1a6d80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.215+0000 7fcd53cca640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcd340a2d90 0x7fcd340a52a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.215+0000 7fcd53cca640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4c101740 0x7fcd4c19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.215+0000 7fcd53cca640 1 --2- 172.21.15.38:0/1762904308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd4c100d00 0x7fcd4c19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.215+0000 7fcd53cca640 1 -- 172.21.15.38:0/1762904308 >> 172.21.15.38:0/1762904308 conn(0x7fcd4c0fc2f0 msgr2=0x7fcd4c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:18.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.217+0000 7fcd53cca640 1 -- 172.21.15.38:0/1762904308 shutdown_connections 2024-05-21T15:40:18.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.217+0000 7fcd53cca640 1 -- 172.21.15.38:0/1762904308 wait complete. 2024-05-21T15:40:18.234 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:17 smithi038 ceph-mon[31995]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.14k op/s 2024-05-21T15:40:18.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:40:18.236 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:40:18.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:17 smithi045 ceph-mon[36409]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.14k op/s 2024-05-21T15:40:18.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.449+0000 7f53f054b640 1 -- 172.21.15.38:0/4145079071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e81019a0 msgr2=0x7f53e81026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.449+0000 7f53f054b640 1 --2- 172.21.15.38:0/4145079071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e81019a0 0x7f53e81026a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009ca0 tx=0x7f53e002a8d0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.450+0000 7f53f054b640 1 -- 172.21.15.38:0/4145079071 shutdown_connections 2024-05-21T15:40:18.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.450+0000 7f53f054b640 1 --2- 172.21.15.38:0/4145079071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e81078e0 0x7f53e8109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.450+0000 7f53f054b640 1 --2- 172.21.15.38:0/4145079071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53e8102be0 0x7f53e8102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.450+0000 7f53f054b640 1 --2- 172.21.15.38:0/4145079071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e81019a0 0x7f53e81026a0 secure :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009ca0 tx=0x7f53e002a8d0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.450+0000 7f53f054b640 1 -- 172.21.15.38:0/4145079071 >> 172.21.15.38:0/4145079071 conn(0x7f53e80fcb90 msgr2=0x7f53e80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:18.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.451+0000 7f53f054b640 1 -- 172.21.15.38:0/4145079071 shutdown_connections 2024-05-21T15:40:18.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.451+0000 7f53f054b640 1 -- 172.21.15.38:0/4145079071 wait complete. 2024-05-21T15:40:18.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.452+0000 7f53f054b640 1 Processor -- start 2024-05-21T15:40:18.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.452+0000 7f53f054b640 1 -- start start 2024-05-21T15:40:18.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.458+0000 7f53f054b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53e8102be0 0x7f53e819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.458+0000 7f53f054b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e81078e0 0x7f53e81a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.458+0000 7f53f054b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e81a4ee0 0x7f53e81a7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.459+0000 7f53f054b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f53e810d6c0 con 0x7f53e81a4ee0 2024-05-21T15:40:18.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.459+0000 7f53f054b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f53e810d540 con 0x7f53e81078e0 2024-05-21T15:40:18.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.459+0000 7f53f054b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f53e810d840 con 0x7f53e8102be0 2024-05-21T15:40:18.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.460+0000 7f53eeae8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e81a4ee0 0x7f53e81a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.460+0000 7f53eeae8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e81a4ee0 0x7f53e81a7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59464/0 (socket says 172.21.15.38:59464) 2024-05-21T15:40:18.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.460+0000 7f53eeae8640 1 -- 172.21.15.38:0/4127466650 learned_addr learned my addr 172.21.15.38:0/4127466650 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:18.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.461+0000 7f53ee2e7640 1 --2- 172.21.15.38:0/4127466650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53e8102be0 0x7f53e819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.461+0000 7f53edae6640 1 --2- 172.21.15.38:0/4127466650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e81078e0 0x7f53e81a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.461+0000 7f53eeae8640 1 -- 172.21.15.38:0/4127466650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53e8102be0 msgr2=0x7f53e819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.461+0000 7f53eeae8640 1 --2- 172.21.15.38:0/4127466650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53e8102be0 0x7f53e819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.461+0000 7f53eeae8640 1 -- 172.21.15.38:0/4127466650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e81078e0 msgr2=0x7f53e81a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.462+0000 7f53eeae8640 1 --2- 172.21.15.38:0/4127466650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e81078e0 0x7f53e81a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.462+0000 7f53eeae8640 1 -- 172.21.15.38:0/4127466650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e81a7a50 con 0x7f53e81a4ee0 2024-05-21T15:40:18.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.462+0000 7f53eeae8640 1 --2- 172.21.15.38:0/4127466650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e81a4ee0 0x7f53e81a7300 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f53e400d950 tx=0x7f53e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:18.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.463+0000 7f53df7fe640 1 -- 172.21.15.38:0/4127466650 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4017030 con 0x7f53e81a4ee0 2024-05-21T15:40:18.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.463+0000 7f53df7fe640 1 -- 172.21.15.38:0/4127466650 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e4004020 con 0x7f53e81a4ee0 2024-05-21T15:40:18.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.463+0000 7f53df7fe640 1 -- 172.21.15.38:0/4127466650 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4003c50 con 0x7f53e81a4ee0 2024-05-21T15:40:18.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.464+0000 7f53f054b640 1 -- 172.21.15.38:0/4127466650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e81a7d40 con 0x7f53e81a4ee0 2024-05-21T15:40:18.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.464+0000 7f53f054b640 1 -- 172.21.15.38:0/4127466650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e81a81b0 con 0x7f53e81a4ee0 2024-05-21T15:40:18.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.469+0000 7f53f73f1740 1 Processor -- start 2024-05-21T15:40:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.470+0000 7f53f73f1740 1 -- start start 2024-05-21T15:40:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.470+0000 7f53f73f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2f02390 0x55a2a32a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.470+0000 7f53f73f1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2ed94d0 0x55a2a32a2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.470+0000 7f53f73f1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2ee16b0 0x55a2a32a9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.470+0000 7f53f73f1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a2a3206d00 con 0x55a2a2ed94d0 2024-05-21T15:40:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.470+0000 7f53f73f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a2a31ae4c0 con 0x55a2a2f02390 2024-05-21T15:40:18.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.470+0000 7f53f73f1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a2a3202cc0 con 0x55a2a2ee16b0 2024-05-21T15:40:18.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.474+0000 7f53ee2e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2ed94d0 0x55a2a32a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.474+0000 7f53ee2e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2ed94d0 0x55a2a32a2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59478/0 (socket says 172.21.15.38:59478) 2024-05-21T15:40:18.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.474+0000 7f53ee2e7640 1 -- 172.21.15.38:0/1484065487 learned_addr learned my addr 172.21.15.38:0/1484065487 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:18.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.475+0000 7f53edae6640 1 --2- 172.21.15.38:0/1484065487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2f02390 0x55a2a32a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.476+0000 7f53ee2e7640 1 -- 172.21.15.38:0/1484065487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2ee16b0 msgr2=0x55a2a32a9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.476+0000 7f53ee2e7640 1 --2- 172.21.15.38:0/1484065487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2ee16b0 0x55a2a32a9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.476+0000 7f53ee2e7640 1 -- 172.21.15.38:0/1484065487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2f02390 msgr2=0x55a2a32a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.476+0000 7f53ee2e7640 1 --2- 172.21.15.38:0/1484065487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2f02390 0x55a2a32a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.476+0000 7f53ee2e7640 1 -- 172.21.15.38:0/1484065487 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2a2fe47d0 con 0x55a2a2ed94d0 2024-05-21T15:40:18.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.476+0000 7f53edae6640 1 --2- 172.21.15.38:0/1484065487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2f02390 0x55a2a32a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:40:18.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.480+0000 7f53ee2e7640 1 --2- 172.21.15.38:0/1484065487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2ed94d0 0x55a2a32a2130 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f53e002aec0 tx=0x7f53e00099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ba0fb833d55257 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:18.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.480+0000 7f53dcbfe640 1 -- 172.21.15.38:0/1484065487 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0031460 con 0x55a2a2ed94d0 2024-05-21T15:40:18.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.480+0000 7f53dcbfe640 1 -- 172.21.15.38:0/1484065487 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e0036040 con 0x55a2a2ed94d0 2024-05-21T15:40:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.481+0000 7f53f73f1740 1 -- 172.21.15.38:0/1484065487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2ed94d0 msgr2=0x55a2a32a2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.481+0000 7f53f73f1740 1 --2- 172.21.15.38:0/1484065487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2ed94d0 0x55a2a32a2130 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f53e002aec0 tx=0x7f53e00099a0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.482+0000 7f53df7fe640 1 -- 172.21.15.38:0/4127466650 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f53e401a070 con 0x7f53e81a4ee0 2024-05-21T15:40:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.483+0000 7f53f73f1740 1 -- 172.21.15.38:0/1484065487 shutdown_connections 2024-05-21T15:40:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.483+0000 7f53f73f1740 1 --2- 172.21.15.38:0/1484065487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2ee16b0 0x55a2a32a9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.483+0000 7f53f73f1740 1 --2- 172.21.15.38:0/1484065487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2ed94d0 0x55a2a32a2130 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.483+0000 7f53f73f1740 1 --2- 172.21.15.38:0/1484065487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2f02390 0x55a2a32a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.483+0000 7f53f73f1740 1 -- 172.21.15.38:0/1484065487 >> 172.21.15.38:0/1484065487 conn(0x55a2a2ef64e0 msgr2=0x55a2a329d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.484+0000 7f53f73f1740 1 -- 172.21.15.38:0/1484065487 shutdown_connections 2024-05-21T15:40:18.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.484+0000 7f53df7fe640 1 --2- 172.21.15.38:0/4127466650 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f53d40a2d10 0x7f53d40a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.484+0000 7f53f73f1740 1 -- 172.21.15.38:0/1484065487 wait complete. 2024-05-21T15:40:18.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.484+0000 7f53df7fe640 1 -- 172.21.15.38:0/4127466650 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f53e40cdf90 con 0x7f53e81a4ee0 2024-05-21T15:40:18.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.484+0000 7f53ee2e7640 1 --2- 172.21.15.38:0/4127466650 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f53d40a2d10 0x7f53d40a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.485+0000 7f53f73f1740 1 -- start start 2024-05-21T15:40:18.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.485+0000 7f53f73f1740 1 Processor -- start 2024-05-21T15:40:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.485+0000 7f53f73f1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2ed94d0 0x55a2a32bcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.485+0000 7f53f73f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2ee16b0 0x55a2a32bcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.485+0000 7f53f73f1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2f02390 0x55a2a32c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.485+0000 7f53f73f1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a2a32baa30 con 0x55a2a2ed94d0 2024-05-21T15:40:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.485+0000 7f53f73f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a2a31c1b90 con 0x55a2a2ee16b0 2024-05-21T15:40:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.485+0000 7f53f73f1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a2a32babb0 con 0x55a2a2f02390 2024-05-21T15:40:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.485+0000 7f53f73f1740 10 client.0 _ll_register_callbacks cb 0x55a2a31fdda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.486+0000 7f53edae6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2ee16b0 0x55a2a32bcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.486+0000 7f53edae6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2ee16b0 0x55a2a32bcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46814/0 (socket says 172.21.15.38:46814) 2024-05-21T15:40:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.486+0000 7f53edae6640 1 -- 172.21.15.38:0/1409236695 learned_addr learned my addr 172.21.15.38:0/1409236695 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:18.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.487+0000 7f53ee2e7640 1 --2- 172.21.15.38:0/1409236695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2ed94d0 0x55a2a32bcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.487+0000 7f53eeae8640 1 --2- 172.21.15.38:0/1409236695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2f02390 0x55a2a32c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.487+0000 7f53ee2e7640 1 --2- 172.21.15.38:0/4127466650 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f53d40a2d10 0x7f53d40a5220 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f53e0000f80 tx=0x7f53e002a8d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:18.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.488+0000 7f53ee2e7640 1 -- 172.21.15.38:0/1409236695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2f02390 msgr2=0x55a2a32c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.488+0000 7f53ee2e7640 1 --2- 172.21.15.38:0/1409236695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2f02390 0x55a2a32c3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.488+0000 7f53ee2e7640 1 -- 172.21.15.38:0/1409236695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2ee16b0 msgr2=0x55a2a32bcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.488+0000 7f53ee2e7640 1 --2- 172.21.15.38:0/1409236695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2ee16b0 0x55a2a32bcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.488+0000 7f53ee2e7640 1 -- 172.21.15.38:0/1409236695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2a2f1a3f0 con 0x55a2a2ed94d0 2024-05-21T15:40:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.488+0000 7f53ee2e7640 1 --2- 172.21.15.38:0/1409236695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2ed94d0 0x55a2a32bcaa0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55a2a329e2d0 tx=0x7f53e00099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ad57ad5ff9df1d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.488+0000 7f53d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.489+0000 7f53d1ffb640 1 -- 172.21.15.38:0/1409236695 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0030de0 con 0x55a2a2ed94d0 2024-05-21T15:40:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.489+0000 7f53f73f1740 1 -- 172.21.15.38:0/1409236695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e0036040 con 0x55a2a2ed94d0 2024-05-21T15:40:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.489+0000 7f53d1ffb640 1 -- 172.21.15.38:0/1409236695 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f53e0004000 con 0x55a2a2ed94d0 2024-05-21T15:40:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.489+0000 7f53d1ffb640 1 -- 172.21.15.38:0/1409236695 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f53e0004200 con 0x55a2a2ed94d0 2024-05-21T15:40:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.490+0000 7f53f73f1740 10 client.17079 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.490+0000 7f53f73f1740 1 -- 172.21.15.38:0/1409236695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a2a2eda0f0 con 0x55a2a2ed94d0 2024-05-21T15:40:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.492+0000 7f53d1ffb640 1 -- 172.21.15.38:0/1409236695 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e0036040 con 0x55a2a2ed94d0 2024-05-21T15:40:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.493+0000 7f53d1ffb640 1 -- 172.21.15.38:0/1409236695 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f53e004e070 con 0x55a2a2ed94d0 2024-05-21T15:40:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.493+0000 7f53f73f1740 10 client.17079 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:18.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.493+0000 7f53f73f1740 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:18.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.493+0000 7f53f73f1740 20 client.17079 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:18.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.493+0000 7f53f73f1740 1 --2- 172.21.15.38:0/1409236695 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2a2ec0f60 0x55a2a32ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.493+0000 7f53f73f1740 4 client.17079 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:18.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.493+0000 7f53f73f1740 1 -- 172.21.15.38:0/1409236695 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2a3214430 con 0x55a2a2ec0f60 2024-05-21T15:40:18.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.493+0000 7f53edae6640 1 --2- 172.21.15.38:0/1409236695 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2a2ec0f60 0x55a2a32ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.500+0000 7f53edae6640 1 --2- 172.21.15.38:0/1409236695 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2a2ec0f60 0x55a2a32ca980 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de7c842c07c99e2 server_cookie=b5995f51fa2051d in_seq=0 out_seq=0 2024-05-21T15:40:18.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.501+0000 7f53d1ffb640 10 client.17079 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:18.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.502+0000 7f53d1ffb640 1 -- 172.21.15.38:0/1409236695 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a2a3214430 con 0x55a2a2ec0f60 2024-05-21T15:40:18.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.502+0000 7f53d1ffb640 10 client.17079 handle_command_reply: tid=0 2024-05-21T15:40:18.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.503+0000 7f53f73f1740 1 client.17079 shutdown 2024-05-21T15:40:18.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.506+0000 7f53f73f1740 1 -- 172.21.15.38:0/1409236695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2ed94d0 msgr2=0x55a2a32bcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.506+0000 7f53f73f1740 1 --2- 172.21.15.38:0/1409236695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2ed94d0 0x55a2a32bcaa0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55a2a329e2d0 tx=0x7f53e00099a0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.507+0000 7f53f73f1740 1 -- 172.21.15.38:0/1409236695 shutdown_connections 2024-05-21T15:40:18.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.507+0000 7f53f73f1740 1 --2- 172.21.15.38:0/1409236695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2f02390 0x55a2a32c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.507+0000 7f53f73f1740 1 --2- 172.21.15.38:0/1409236695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2ee16b0 0x55a2a32bcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.507+0000 7f53f73f1740 1 --2- 172.21.15.38:0/1409236695 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2a2ec0f60 0x55a2a32ca980 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.507+0000 7f53f73f1740 1 --2- 172.21.15.38:0/1409236695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2ed94d0 0x55a2a32bcaa0 secure :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55a2a329e2d0 tx=0x7f53e00099a0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.507+0000 7f53f73f1740 1 -- 172.21.15.38:0/1409236695 >> 172.21.15.38:0/1409236695 conn(0x55a2a2ef64e0 msgr2=0x55a2a329c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:18.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.507+0000 7f53d1ffb640 0 client.17079 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:18.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.510+0000 7f53f73f1740 1 -- 172.21.15.38:0/1409236695 shutdown_connections 2024-05-21T15:40:18.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.512+0000 7f53f73f1740 1 -- 172.21.15.38:0/1409236695 wait complete. 2024-05-21T15:40:18.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.514+0000 7f53f73f1740 20 client.17079 trim_cache size 0 max 16384 2024-05-21T15:40:18.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.559+0000 7f53f73f1740 1 Processor -- start 2024-05-21T15:40:18.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.560+0000 7f53f73f1740 1 -- start start 2024-05-21T15:40:18.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.560+0000 7f53f73f1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2ee16b0 0x55a2a32bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.560+0000 7f53f73f1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2f02390 0x55a2a32c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.560+0000 7f53f73f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2e963a0 0x55a2a32a5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.560+0000 7f53f73f1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc01f700 con 0x55a2a2f02390 2024-05-21T15:40:18.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.560+0000 7f53f73f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc021330 con 0x55a2a2e963a0 2024-05-21T15:40:18.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.560+0000 7f53f73f1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc01fdd0 con 0x55a2a2ee16b0 2024-05-21T15:40:18.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.561+0000 7f53eeae8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2e963a0 0x55a2a32a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.561+0000 7f53eeae8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2e963a0 0x55a2a32a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46824/0 (socket says 172.21.15.38:46824) 2024-05-21T15:40:18.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.561+0000 7f53eeae8640 1 -- 172.21.15.38:0/2958280298 learned_addr learned my addr 172.21.15.38:0/2958280298 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:18.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.563+0000 7f53eeae8640 1 -- 172.21.15.38:0/2958280298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2ee16b0 msgr2=0x55a2a32bb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.563+0000 7f53eeae8640 1 --2- 172.21.15.38:0/2958280298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2ee16b0 0x55a2a32bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.563+0000 7f53eeae8640 1 -- 172.21.15.38:0/2958280298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2f02390 msgr2=0x55a2a32c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.563+0000 7f53eeae8640 1 --2- 172.21.15.38:0/2958280298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2f02390 0x55a2a32c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.563+0000 7f53eeae8640 1 -- 172.21.15.38:0/2958280298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2a32bada0 con 0x55a2a2e963a0 2024-05-21T15:40:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.565+0000 7f53eeae8640 1 --2- 172.21.15.38:0/2958280298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2e963a0 0x55a2a32a5480 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009610 tx=0x7f53e409a450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd1d143f2be11ef server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.568+0000 7f53d3fff640 1 -- 172.21.15.38:0/2958280298 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e409a850 con 0x55a2a2e963a0 2024-05-21T15:40:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.568+0000 7f53d3fff640 1 -- 172.21.15.38:0/2958280298 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e40b3070 con 0x55a2a2e963a0 2024-05-21T15:40:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.568+0000 7f53d3fff640 1 -- 172.21.15.38:0/2958280298 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4096ec0 con 0x55a2a2e963a0 2024-05-21T15:40:18.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.570+0000 7f53f73f1740 1 -- 172.21.15.38:0/2958280298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2e963a0 msgr2=0x55a2a32a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.570+0000 7f53f73f1740 1 --2- 172.21.15.38:0/2958280298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2e963a0 0x55a2a32a5480 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009610 tx=0x7f53e409a450 comp rx=0 tx=0).stop 2024-05-21T15:40:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.573+0000 7f53f73f1740 1 -- 172.21.15.38:0/2958280298 shutdown_connections 2024-05-21T15:40:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.573+0000 7f53f73f1740 1 --2- 172.21.15.38:0/2958280298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2e963a0 0x55a2a32a5480 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.573+0000 7f53f73f1740 1 --2- 172.21.15.38:0/2958280298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2f02390 0x55a2a32c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.573+0000 7f53f73f1740 1 --2- 172.21.15.38:0/2958280298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2ee16b0 0x55a2a32bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.573+0000 7f53f73f1740 1 -- 172.21.15.38:0/2958280298 >> 172.21.15.38:0/2958280298 conn(0x55a2a2ef64e0 msgr2=0x55a2a32a4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:18.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.577+0000 7f53f73f1740 1 -- 172.21.15.38:0/2958280298 shutdown_connections 2024-05-21T15:40:18.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.577+0000 7f53f73f1740 1 -- 172.21.15.38:0/2958280298 wait complete. 2024-05-21T15:40:18.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.577+0000 7f53f73f1740 1 -- start start 2024-05-21T15:40:18.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.577+0000 7f53f73f1740 1 Processor -- start 2024-05-21T15:40:18.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.578+0000 7f53f73f1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2e963a0 0x55a2a32c0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.578+0000 7f53f73f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2ee16b0 0x55a2a32c1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.578+0000 7f53f73f1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2f02390 0x55a2a3330ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.578+0000 7f53f73f1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc020ba0 con 0x55a2a2f02390 2024-05-21T15:40:18.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.578+0000 7f53f73f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc0204a0 con 0x55a2a2ee16b0 2024-05-21T15:40:18.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.578+0000 7f53f73f1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc01d6e0 con 0x55a2a2e963a0 2024-05-21T15:40:18.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.578+0000 7f53f73f1740 10 client.0 _ll_register_callbacks cb 0x7f53e0009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:18.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.578+0000 7f53edae6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2ee16b0 0x55a2a32c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.578+0000 7f53edae6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2ee16b0 0x55a2a32c1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46828/0 (socket says 172.21.15.38:46828) 2024-05-21T15:40:18.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.578+0000 7f53edae6640 1 -- 172.21.15.38:0/3535284325 learned_addr learned my addr 172.21.15.38:0/3535284325 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:18.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.578+0000 7f53ee2e7640 1 --2- 172.21.15.38:0/3535284325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2e963a0 0x55a2a32c0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.578+0000 7f53eeae8640 1 --2- 172.21.15.38:0/3535284325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2f02390 0x55a2a3330ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.579+0000 7f53edae6640 1 -- 172.21.15.38:0/3535284325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2e963a0 msgr2=0x55a2a32c0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.579+0000 7f53edae6640 1 --2- 172.21.15.38:0/3535284325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2e963a0 0x55a2a32c0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.579+0000 7f53edae6640 1 -- 172.21.15.38:0/3535284325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2f02390 msgr2=0x55a2a3330ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:18.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.579+0000 7f53edae6640 1 --2- 172.21.15.38:0/3535284325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2f02390 0x55a2a3330ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.579+0000 7f53edae6640 1 -- 172.21.15.38:0/3535284325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2a32c3fc0 con 0x55a2a2ee16b0 2024-05-21T15:40:18.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.582+0000 7f53edae6640 1 --2- 172.21.15.38:0/3535284325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2ee16b0 0x55a2a32c1070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f53d8006a20 tx=0x7f53d800c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7daa8a62981fbb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:18.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.583+0000 7f53d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:40:18.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.583+0000 7f53d1ffb640 1 -- 172.21.15.38:0/3535284325 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8015d20 con 0x55a2a2ee16b0 2024-05-21T15:40:18.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.583+0000 7f53d1ffb640 1 -- 172.21.15.38:0/3535284325 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f53d800f040 con 0x55a2a2ee16b0 2024-05-21T15:40:18.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.584+0000 7f53f73f1740 1 -- 172.21.15.38:0/3535284325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e40b3070 con 0x55a2a2ee16b0 2024-05-21T15:40:18.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.589+0000 7f53d1ffb640 1 -- 172.21.15.38:0/3535284325 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f53d801b250 con 0x55a2a2ee16b0 2024-05-21T15:40:18.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.591+0000 7f53f73f1740 10 client.26711 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:18.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.591+0000 7f53f73f1740 1 -- 172.21.15.38:0/3535284325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a2a3332160 con 0x55a2a2ee16b0 2024-05-21T15:40:18.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.592+0000 7f53d1ffb640 1 -- 172.21.15.38:0/3535284325 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e40b3070 con 0x55a2a2ee16b0 2024-05-21T15:40:18.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.592+0000 7f53d1ffb640 1 -- 172.21.15.38:0/3535284325 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f53d801bce0 con 0x55a2a2ee16b0 2024-05-21T15:40:18.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.593+0000 7f53f73f1740 10 client.26711 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:18.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.593+0000 7f53f73f1740 10 client.26711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:18.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.593+0000 7f53f73f1740 20 client.26711 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:18.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.593+0000 7f53f73f1740 1 --2- 172.21.15.38:0/3535284325 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2a3332ba0 0x55a2a3334fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:18.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.593+0000 7f53f73f1740 4 client.26711 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:40:18.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.593+0000 7f53f73f1740 1 -- 172.21.15.38:0/3535284325 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a2a320e780 con 0x55a2a3332ba0 2024-05-21T15:40:18.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.600+0000 7f53ee2e7640 1 --2- 172.21.15.38:0/3535284325 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2a3332ba0 0x55a2a3334fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.610+0000 7f53d1ffb640 10 client.26711 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:18.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.610+0000 7f53ee2e7640 1 --2- 172.21.15.38:0/3535284325 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2a3332ba0 0x55a2a3334fc0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642ad660e0170862 server_cookie=dac308f309f86020 in_seq=0 out_seq=0 2024-05-21T15:40:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.613+0000 7f53d1ffb640 1 -- 172.21.15.38:0/3535284325 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55a2a320e780 con 0x55a2a3332ba0 2024-05-21T15:40:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.614+0000 7f53d1ffb640 10 client.26711 handle_command_reply: tid=0 2024-05-21T15:40:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.614+0000 7f53f73f1740 1 client.26711 shutdown 2024-05-21T15:40:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.617+0000 7f53f73f1740 1 -- 172.21.15.38:0/3535284325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2ee16b0 msgr2=0x55a2a32c1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.617+0000 7f53f73f1740 1 --2- 172.21.15.38:0/3535284325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2ee16b0 0x55a2a32c1070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f53d8006a20 tx=0x7f53d800c980 comp rx=0 tx=0).stop 2024-05-21T15:40:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.617+0000 7f53f73f1740 1 -- 172.21.15.38:0/3535284325 shutdown_connections 2024-05-21T15:40:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.617+0000 7f53f73f1740 1 --2- 172.21.15.38:0/3535284325 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2a3332ba0 0x55a2a3334fc0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.617+0000 7f53f73f1740 1 --2- 172.21.15.38:0/3535284325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2a2f02390 0x55a2a3330ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.617+0000 7f53f73f1740 1 --2- 172.21.15.38:0/3535284325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2a2ee16b0 0x55a2a32c1070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.617+0000 7f53f73f1740 1 --2- 172.21.15.38:0/3535284325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2a2e963a0 0x55a2a32c0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.617+0000 7f53f73f1740 1 -- 172.21.15.38:0/3535284325 >> 172.21.15.38:0/3535284325 conn(0x55a2a2ef64e0 msgr2=0x55a2a32a0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.617+0000 7f53d1ffb640 0 client.26711 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.620+0000 7f53f73f1740 1 -- 172.21.15.38:0/3535284325 shutdown_connections 2024-05-21T15:40:18.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.620+0000 7f53f73f1740 1 -- 172.21.15.38:0/3535284325 wait complete. 2024-05-21T15:40:18.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.623+0000 7f53f73f1740 20 client.26711 trim_cache size 0 max 16384 2024-05-21T15:40:18.625 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:40:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "scrub active (1 inodes in the stack)", 2024-05-21T15:40:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-05-21T15:40:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "da2fb5b5-af83-44d2-bc65-2c1d09f57ed3": { 2024-05-21T15:40:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-05-21T15:40:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "da2fb5b5-af83-44d2-bc65-2c1d09f57ed3", 2024-05-21T15:40:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-05-21T15:40:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:40:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:40:18.625 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:40:18.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.629+0000 7f53f054b640 1 -- 172.21.15.38:0/4127466650 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f53d40a2d10 msgr2=0x7f53d40a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.629+0000 7f53f054b640 1 --2- 172.21.15.38:0/4127466650 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f53d40a2d10 0x7f53d40a5220 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f53e0000f80 tx=0x7f53e002a8d0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.629+0000 7f53f054b640 1 -- 172.21.15.38:0/4127466650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e81a4ee0 msgr2=0x7f53e81a7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:18.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.629+0000 7f53f054b640 1 --2- 172.21.15.38:0/4127466650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e81a4ee0 0x7f53e81a7300 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f53e400d950 tx=0x7f53e400de10 comp rx=0 tx=0).stop 2024-05-21T15:40:18.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.629+0000 7f53f054b640 1 -- 172.21.15.38:0/4127466650 shutdown_connections 2024-05-21T15:40:18.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.629+0000 7f53f054b640 1 --2- 172.21.15.38:0/4127466650 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f53d40a2d10 0x7f53d40a5220 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.629+0000 7f53f054b640 1 --2- 172.21.15.38:0/4127466650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53e81a4ee0 0x7f53e81a7300 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.629+0000 7f53f054b640 1 --2- 172.21.15.38:0/4127466650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e81078e0 0x7f53e81a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.629+0000 7f53f054b640 1 --2- 172.21.15.38:0/4127466650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53e8102be0 0x7f53e819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:18.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.629+0000 7f53f054b640 1 -- 172.21.15.38:0/4127466650 >> 172.21.15.38:0/4127466650 conn(0x7f53e80fcb90 msgr2=0x7f53e80fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:18.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.632+0000 7f53f054b640 1 -- 172.21.15.38:0/4127466650 shutdown_connections 2024-05-21T15:40:18.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:18.632+0000 7f53f054b640 1 -- 172.21.15.38:0/4127466650 wait complete. 2024-05-21T15:40:18.652 INFO:tasks.cephfs.filesystem:scrub status for tag:da2fb5b5-af83-44d2-bc65-2c1d09f57ed3 - {'path': '/', 'tag': 'da2fb5b5-af83-44d2-bc65-2c1d09f57ed3', 'options': 'recursive,force'} 2024-05-21T15:40:20.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:19 smithi053 ceph-mon[36496]: scrub queued for path: / 2024-05-21T15:40:20.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:19 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:20.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:19 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:40:20.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:19 smithi053 ceph-mon[36496]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.09k op/s 2024-05-21T15:40:20.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:19 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:40:20.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:19 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:40:20.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:19 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:40:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:19 smithi045 ceph-mon[36409]: scrub queued for path: / 2024-05-21T15:40:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:19 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:19 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:40:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:19 smithi045 ceph-mon[36409]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.09k op/s 2024-05-21T15:40:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:19 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:40:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:19 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:40:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:19 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:40:20.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:19 smithi038 ceph-mon[31995]: scrub queued for path: / 2024-05-21T15:40:20.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:19 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:20.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:19 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:40:20.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:19 smithi038 ceph-mon[31995]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.09k op/s 2024-05-21T15:40:20.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:19 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:40:20.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:19 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:40:20.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:19 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:40:21.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:20 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:40:21.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:20 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:40:21.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:20 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:40:22.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:21 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:22.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:21 smithi053 ceph-mon[36496]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 MiB/s wr, 5.61k op/s 2024-05-21T15:40:22.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:21 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:22.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:21 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:22.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:21 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:22.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:21 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:22.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:21 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:22.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:21 smithi045 ceph-mon[36409]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 MiB/s wr, 5.61k op/s 2024-05-21T15:40:22.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:21 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:22.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:21 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:22.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:21 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:22.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:21 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:22.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:21 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:22.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:21 smithi038 ceph-mon[31995]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 MiB/s wr, 5.61k op/s 2024-05-21T15:40:22.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:21 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:22.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:21 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:22.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:21 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:22.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:21 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:24.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:23 smithi053 ceph-mon[36496]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 3.92k op/s 2024-05-21T15:40:24.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:23 smithi045 ceph-mon[36409]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 3.92k op/s 2024-05-21T15:40:24.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:23 smithi038 ceph-mon[31995]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 3.92k op/s 2024-05-21T15:40:26.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:25 smithi053 ceph-mon[36496]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 4.01k op/s 2024-05-21T15:40:26.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:25 smithi045 ceph-mon[36409]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 4.01k op/s 2024-05-21T15:40:26.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:25 smithi038 ceph-mon[31995]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 4.01k op/s 2024-05-21T15:40:28.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:28 smithi053 ceph-mon[36496]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 25 MiB/s wr, 5.57k op/s 2024-05-21T15:40:28.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:28 smithi045 ceph-mon[36409]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 25 MiB/s wr, 5.57k op/s 2024-05-21T15:40:28.522 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:28 smithi038 ceph-mon[31995]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 25 MiB/s wr, 5.57k op/s 2024-05-21T15:40:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.740+0000 7f19b5ad3640 1 -- 172.21.15.38:0/2556241875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b0101250 msgr2=0x7f19b010acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.740+0000 7f19b5ad3640 1 --2- 172.21.15.38:0/2556241875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b0101250 0x7f19b010acb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009ca0 tx=0x7f19a802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.741+0000 7f19b5ad3640 1 -- 172.21.15.38:0/2556241875 shutdown_connections 2024-05-21T15:40:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.741+0000 7f19b5ad3640 1 --2- 172.21.15.38:0/2556241875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b0101250 0x7f19b010acb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.741+0000 7f19b5ad3640 1 --2- 172.21.15.38:0/2556241875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b0100750 0x7f19b0100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.741+0000 7f19b5ad3640 1 --2- 172.21.15.38:0/2556241875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19b01040a0 0x7f19b01044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.741+0000 7f19b5ad3640 1 -- 172.21.15.38:0/2556241875 >> 172.21.15.38:0/2556241875 conn(0x7f19b00fc390 msgr2=0x7f19b00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.741+0000 7f19b5ad3640 1 -- 172.21.15.38:0/2556241875 shutdown_connections 2024-05-21T15:40:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.741+0000 7f19b5ad3640 1 -- 172.21.15.38:0/2556241875 wait complete. 2024-05-21T15:40:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.741+0000 7f19b5ad3640 1 Processor -- start 2024-05-21T15:40:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.741+0000 7f19b5ad3640 1 -- start start 2024-05-21T15:40:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19b5ad3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b0100750 0x7f19b01991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19b5ad3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b0101250 0x7f19b01996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19b5ad3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19b01040a0 0x7f19b01a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19b5ad3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f19b010e780 con 0x7f19b0100750 2024-05-21T15:40:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19b5ad3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19b010e600 con 0x7f19b0101250 2024-05-21T15:40:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19b5ad3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f19b010e900 con 0x7f19b01040a0 2024-05-21T15:40:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b0100750 0x7f19b01991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b0100750 0x7f19b01991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50820/0 (socket says 172.21.15.38:50820) 2024-05-21T15:40:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19af7fe640 1 -- 172.21.15.38:0/2124583625 learned_addr learned my addr 172.21.15.38:0/2124583625 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19aeffd640 1 --2- 172.21.15.38:0/2124583625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b0101250 0x7f19b01996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19affff640 1 --2- 172.21.15.38:0/2124583625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19b01040a0 0x7f19b01a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19af7fe640 1 -- 172.21.15.38:0/2124583625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19b01040a0 msgr2=0x7f19b01a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19af7fe640 1 --2- 172.21.15.38:0/2124583625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19b01040a0 0x7f19b01a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19af7fe640 1 -- 172.21.15.38:0/2124583625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b0101250 msgr2=0x7f19b01996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19af7fe640 1 --2- 172.21.15.38:0/2124583625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b0101250 0x7f19b01996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19af7fe640 1 -- 172.21.15.38:0/2124583625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b01a0cf0 con 0x7f19b0100750 2024-05-21T15:40:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.742+0000 7f19af7fe640 1 --2- 172.21.15.38:0/2124583625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b0100750 0x7f19b01991b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f19a400e9f0 tx=0x7f19a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.743+0000 7f19acff9640 1 -- 172.21.15.38:0/2124583625 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a400ccc0 con 0x7f19b0100750 2024-05-21T15:40:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.743+0000 7f19acff9640 1 -- 172.21.15.38:0/2124583625 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19a40035f0 con 0x7f19b0100750 2024-05-21T15:40:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.743+0000 7f19acff9640 1 -- 172.21.15.38:0/2124583625 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a400ce60 con 0x7f19b0100750 2024-05-21T15:40:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.743+0000 7f19b5ad3640 1 -- 172.21.15.38:0/2124583625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b01a0fe0 con 0x7f19b0100750 2024-05-21T15:40:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.743+0000 7f19b5ad3640 1 -- 172.21.15.38:0/2124583625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b01a13f0 con 0x7f19b0100750 2024-05-21T15:40:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.746+0000 7f19acff9640 1 -- 172.21.15.38:0/2124583625 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f19a40037a0 con 0x7f19b0100750 2024-05-21T15:40:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.746+0000 7f19bc970740 1 Processor -- start 2024-05-21T15:40:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.747+0000 7f19bc970740 1 -- start start 2024-05-21T15:40:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.747+0000 7f19bc970740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8becb41b0 0x55e8bf053e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.747+0000 7f19bc970740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf0543e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.747+0000 7f19bc970740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8bec934d0 0x55e8bf05b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.747+0000 7f19bc970740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e8befb8340 con 0x55e8bec8b2f0 2024-05-21T15:40:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.747+0000 7f19bc970740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e8bef60290 con 0x55e8bec934d0 2024-05-21T15:40:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.747+0000 7f19bc970740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e8befb4300 con 0x55e8becb41b0 2024-05-21T15:40:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.747+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf0543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.747+0000 7f19affff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8becb41b0 0x55e8bf053e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.747+0000 7f19affff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8becb41b0 0x55e8bf053e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:36510/0 (socket says 172.21.15.38:36510) 2024-05-21T15:40:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.747+0000 7f19affff640 1 -- 172.21.15.38:0/3499393454 learned_addr learned my addr 172.21.15.38:0/3499393454 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.747+0000 7f19aeffd640 1 --2- 172.21.15.38:0/3499393454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8bec934d0 0x55e8bf05b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.748+0000 7f19acff9640 1 --2- 172.21.15.38:0/2124583625 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f19980a2c30 0x7f19980a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.748+0000 7f19aeffd640 1 -- 172.21.15.38:0/3499393454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8becb41b0 msgr2=0x55e8bf053e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.748+0000 7f19aeffd640 1 --2- 172.21.15.38:0/3499393454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8becb41b0 0x55e8bf053e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.748+0000 7f19aeffd640 1 -- 172.21.15.38:0/3499393454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 msgr2=0x55e8bf0543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.748+0000 7f19aeffd640 1 --2- 172.21.15.38:0/3499393454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf0543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.748+0000 7f19aeffd640 1 -- 172.21.15.38:0/3499393454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8bef848e0 con 0x55e8bec934d0 2024-05-21T15:40:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.748+0000 7f19affff640 1 --2- 172.21.15.38:0/3499393454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8becb41b0 0x55e8bf053e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.748+0000 7f19affff640 1 --2- 172.21.15.38:0/2124583625 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f19980a2c30 0x7f19980a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.748+0000 7f19acff9640 1 -- 172.21.15.38:0/2124583625 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f19a40cf550 con 0x7f19b0100750 2024-05-21T15:40:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.748+0000 7f19aeffd640 1 --2- 172.21.15.38:0/3499393454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8bec934d0 0x55e8bf05b2e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009bf0 tx=0x7f19a0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=253bcf61de69802c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.748+0000 7f1995ffb640 1 -- 172.21.15.38:0/3499393454 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0004050 con 0x55e8bec934d0 2024-05-21T15:40:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.748+0000 7f1995ffb640 1 -- 172.21.15.38:0/3499393454 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19a0005500 con 0x55e8bec934d0 2024-05-21T15:40:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.748+0000 7f1995ffb640 1 -- 172.21.15.38:0/3499393454 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a00035f0 con 0x55e8bec934d0 2024-05-21T15:40:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.748+0000 7f19affff640 1 --2- 172.21.15.38:0/2124583625 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f19980a2c30 0x7f19980a5140 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55e8bf055030 tx=0x7f19a80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.749+0000 7f19bc970740 1 -- 172.21.15.38:0/3499393454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8bec934d0 msgr2=0x55e8bf05b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.749+0000 7f19bc970740 1 --2- 172.21.15.38:0/3499393454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8bec934d0 0x55e8bf05b2e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009bf0 tx=0x7f19a0011810 comp rx=0 tx=0).stop 2024-05-21T15:40:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.749+0000 7f19bc970740 1 -- 172.21.15.38:0/3499393454 shutdown_connections 2024-05-21T15:40:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.749+0000 7f19bc970740 1 --2- 172.21.15.38:0/3499393454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8bec934d0 0x55e8bf05b2e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.749+0000 7f19bc970740 1 --2- 172.21.15.38:0/3499393454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf0543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.749+0000 7f19bc970740 1 --2- 172.21.15.38:0/3499393454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8becb41b0 0x55e8bf053e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.749+0000 7f19bc970740 1 -- 172.21.15.38:0/3499393454 >> 172.21.15.38:0/3499393454 conn(0x55e8beca8300 msgr2=0x55e8bf04f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.749+0000 7f19bc970740 1 -- 172.21.15.38:0/3499393454 shutdown_connections 2024-05-21T15:40:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.749+0000 7f19bc970740 1 -- 172.21.15.38:0/3499393454 wait complete. 2024-05-21T15:40:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.750+0000 7f19bc970740 1 -- start start 2024-05-21T15:40:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.750+0000 7f19bc970740 1 Processor -- start 2024-05-21T15:40:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.750+0000 7f19bc970740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf061fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.750+0000 7f19bc970740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec934d0 0x55e8bf0624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19bc970740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8becb41b0 0x55e8bf069440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19bc970740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e8bf060140 con 0x55e8bec8b2f0 2024-05-21T15:40:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19bc970740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e8bef73790 con 0x55e8becb41b0 2024-05-21T15:40:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf061fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19bc970740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e8bf0602c0 con 0x55e8bec934d0 2024-05-21T15:40:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf061fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50836/0 (socket says 172.21.15.38:50836) 2024-05-21T15:40:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8becb41b0 0x55e8bf069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19af7fe640 1 -- 172.21.15.38:0/2212003128 learned_addr learned my addr 172.21.15.38:0/2212003128 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19aeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec934d0 0x55e8bf0624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19bc970740 10 client.0 _ll_register_callbacks cb 0x55e8befaea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19af7fe640 1 -- 172.21.15.38:0/2212003128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec934d0 msgr2=0x55e8bf0624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19af7fe640 1 --2- 172.21.15.38:0/2212003128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec934d0 0x55e8bf0624e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19af7fe640 1 -- 172.21.15.38:0/2212003128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8becb41b0 msgr2=0x55e8bf069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19af7fe640 1 --2- 172.21.15.38:0/2212003128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8becb41b0 0x55e8bf069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19af7fe640 1 -- 172.21.15.38:0/2212003128 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8beccc210 con 0x55e8bec8b2f0 2024-05-21T15:40:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19aeffd640 1 --2- 172.21.15.38:0/2212003128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec934d0 0x55e8bf0624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19affff640 1 --2- 172.21.15.38:0/2212003128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8becb41b0 0x55e8bf069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f19af7fe640 1 --2- 172.21.15.38:0/2212003128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf061fa0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f19a40981c0 tx=0x7f19a40978e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de66f2b4f812e76 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.751+0000 7f198effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.752+0000 7f198effd640 1 -- 172.21.15.38:0/2212003128 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a409b6c0 con 0x55e8bec8b2f0 2024-05-21T15:40:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.752+0000 7f198effd640 1 -- 172.21.15.38:0/2212003128 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f19a40cf0d0 con 0x55e8bec8b2f0 2024-05-21T15:40:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.752+0000 7f19bc970740 1 -- 172.21.15.38:0/2212003128 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19a0005500 con 0x55e8bec8b2f0 2024-05-21T15:40:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.752+0000 7f198effd640 1 -- 172.21.15.38:0/2212003128 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f19a409c460 con 0x55e8bec8b2f0 2024-05-21T15:40:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.752+0000 7f19bc970740 10 client.17094 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.752+0000 7f19bc970740 1 -- 172.21.15.38:0/2212003128 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e8bed98620 con 0x55e8bec8b2f0 2024-05-21T15:40:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.752+0000 7f198effd640 1 -- 172.21.15.38:0/2212003128 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19a0005500 con 0x55e8bec8b2f0 2024-05-21T15:40:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.752+0000 7f198effd640 1 -- 172.21.15.38:0/2212003128 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f19a4086050 con 0x55e8bec8b2f0 2024-05-21T15:40:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.753+0000 7f19bc970740 10 client.17094 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.753+0000 7f19bc970740 10 client.17094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.753+0000 7f19bc970740 20 client.17094 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.753+0000 7f19bc970740 1 --2- 172.21.15.38:0/2212003128 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8bec72d80 0x55e8bf06f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.753+0000 7f19bc970740 4 client.17094 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.753+0000 7f19bc970740 1 -- 172.21.15.38:0/2212003128 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8befc5a70 con 0x55e8bec72d80 2024-05-21T15:40:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.753+0000 7f19aeffd640 1 --2- 172.21.15.38:0/2212003128 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8bec72d80 0x55e8bf06f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.753+0000 7f198effd640 10 client.17094 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.753+0000 7f19aeffd640 1 --2- 172.21.15.38:0/2212003128 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8bec72d80 0x55e8bf06f9f0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fc9f9c02b312c69 server_cookie=aabc56afbad1e87 in_seq=0 out_seq=0 2024-05-21T15:40:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.754+0000 7f198effd640 1 -- 172.21.15.38:0/2212003128 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e8befc5a70 con 0x55e8bec72d80 2024-05-21T15:40:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.754+0000 7f198effd640 10 client.17094 handle_command_reply: tid=0 2024-05-21T15:40:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.754+0000 7f19bc970740 1 client.17094 shutdown 2024-05-21T15:40:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.755+0000 7f19bc970740 1 -- 172.21.15.38:0/2212003128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 msgr2=0x55e8bf061fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.755+0000 7f19bc970740 1 --2- 172.21.15.38:0/2212003128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf061fa0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f19a40981c0 tx=0x7f19a40978e0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.755+0000 7f19bc970740 1 -- 172.21.15.38:0/2212003128 shutdown_connections 2024-05-21T15:40:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.755+0000 7f19bc970740 1 --2- 172.21.15.38:0/2212003128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8becb41b0 0x55e8bf069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.755+0000 7f19bc970740 1 --2- 172.21.15.38:0/2212003128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec934d0 0x55e8bf0624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.755+0000 7f19bc970740 1 --2- 172.21.15.38:0/2212003128 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8bec72d80 0x55e8bf06f9f0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.755+0000 7f19bc970740 1 --2- 172.21.15.38:0/2212003128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf061fa0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.755+0000 7f198effd640 0 client.17094 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.755+0000 7f19bc970740 1 -- 172.21.15.38:0/2212003128 >> 172.21.15.38:0/2212003128 conn(0x55e8beca8300 msgr2=0x55e8bf04e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.755+0000 7f19bc970740 1 -- 172.21.15.38:0/2212003128 shutdown_connections 2024-05-21T15:40:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.755+0000 7f19bc970740 1 -- 172.21.15.38:0/2212003128 wait complete. 2024-05-21T15:40:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.755+0000 7f19bc970740 20 client.17094 trim_cache size 0 max 16384 2024-05-21T15:40:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19bc970740 1 Processor -- start 2024-05-21T15:40:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19bc970740 1 -- start start 2024-05-21T15:40:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19bc970740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec8b2f0 0x55e8bf0576e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19bc970740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8bec934d0 0x55e8bf04f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19bc970740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8becb41b0 0x55e8bf04dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19bc970740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01a81f0 con 0x55e8becb41b0 2024-05-21T15:40:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19bc970740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01a9e20 con 0x55e8bec934d0 2024-05-21T15:40:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19bc970740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01a88c0 con 0x55e8bec8b2f0 2024-05-21T15:40:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19affff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8becb41b0 0x55e8bf04dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19aeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8bec934d0 0x55e8bf04f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec8b2f0 0x55e8bf0576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19affff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8becb41b0 0x55e8bf04dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50838/0 (socket says 172.21.15.38:50838) 2024-05-21T15:40:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec8b2f0 0x55e8bf0576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:36524/0 (socket says 172.21.15.38:36524) 2024-05-21T15:40:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19aeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8bec934d0 0x55e8bf04f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:54248/0 (socket says 172.21.15.38:54248) 2024-05-21T15:40:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.775+0000 7f19affff640 1 -- 172.21.15.38:0/1613955654 learned_addr learned my addr 172.21.15.38:0/1613955654 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.776+0000 7f19af7fe640 1 -- 172.21.15.38:0/1613955654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8bec934d0 msgr2=0x55e8bf04f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.776+0000 7f19af7fe640 1 --2- 172.21.15.38:0/1613955654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8bec934d0 0x55e8bf04f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.776+0000 7f19af7fe640 1 -- 172.21.15.38:0/1613955654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8becb41b0 msgr2=0x55e8bf04dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.776+0000 7f19af7fe640 1 --2- 172.21.15.38:0/1613955654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8becb41b0 0x55e8bf04dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.776+0000 7f19af7fe640 1 -- 172.21.15.38:0/1613955654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8bf063010 con 0x55e8bec8b2f0 2024-05-21T15:40:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.776+0000 7f19affff640 1 --2- 172.21.15.38:0/1613955654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8becb41b0 0x55e8bf04dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.776+0000 7f19af7fe640 1 --2- 172.21.15.38:0/1613955654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec8b2f0 0x55e8bf0576e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f19a409b340 tx=0x7f19a409a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5592e39ce727619 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.776+0000 7f19957fa640 1 -- 172.21.15.38:0/1613955654 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a409b6c0 con 0x55e8bec8b2f0 2024-05-21T15:40:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.776+0000 7f19957fa640 1 -- 172.21.15.38:0/1613955654 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19a40a0110 con 0x55e8bec8b2f0 2024-05-21T15:40:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.776+0000 7f19957fa640 1 -- 172.21.15.38:0/1613955654 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a409bb30 con 0x55e8bec8b2f0 2024-05-21T15:40:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.777+0000 7f19bc970740 1 -- 172.21.15.38:0/1613955654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec8b2f0 msgr2=0x55e8bf0576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.777+0000 7f19bc970740 1 --2- 172.21.15.38:0/1613955654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec8b2f0 0x55e8bf0576e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f19a409b340 tx=0x7f19a409a270 comp rx=0 tx=0).stop 2024-05-21T15:40:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.777+0000 7f19bc970740 1 -- 172.21.15.38:0/1613955654 shutdown_connections 2024-05-21T15:40:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.777+0000 7f19bc970740 1 --2- 172.21.15.38:0/1613955654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8becb41b0 0x55e8bf04dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.777+0000 7f19bc970740 1 --2- 172.21.15.38:0/1613955654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8bec934d0 0x55e8bf04f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.777+0000 7f19bc970740 1 --2- 172.21.15.38:0/1613955654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec8b2f0 0x55e8bf0576e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.777+0000 7f19bc970740 1 -- 172.21.15.38:0/1613955654 >> 172.21.15.38:0/1613955654 conn(0x55e8beca8300 msgr2=0x55e8bf04f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.778+0000 7f19bc970740 1 -- 172.21.15.38:0/1613955654 shutdown_connections 2024-05-21T15:40:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.778+0000 7f19bc970740 1 -- 172.21.15.38:0/1613955654 wait complete. 2024-05-21T15:40:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.778+0000 7f19bc970740 1 -- start start 2024-05-21T15:40:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.778+0000 7f19bc970740 1 Processor -- start 2024-05-21T15:40:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.778+0000 7f19bc970740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf065a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.778+0000 7f19bc970740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec934d0 0x55e8bf065f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.778+0000 7f19bc970740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8becb41b0 0x55e8bf0d5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19bc970740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01a9690 con 0x55e8bec8b2f0 2024-05-21T15:40:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19bc970740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01a8f90 con 0x55e8becb41b0 2024-05-21T15:40:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19bc970740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01a61d0 con 0x55e8bec934d0 2024-05-21T15:40:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19bc970740 10 client.0 _ll_register_callbacks cb 0x7f19a40d2d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19aeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec934d0 0x55e8bf065f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19aeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec934d0 0x55e8bf065f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:36526/0 (socket says 172.21.15.38:36526) 2024-05-21T15:40:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf065a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19aeffd640 1 -- 172.21.15.38:0/3338399929 learned_addr learned my addr 172.21.15.38:0/3338399929 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19affff640 1 --2- 172.21.15.38:0/3338399929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8becb41b0 0x55e8bf0d5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19aeffd640 1 -- 172.21.15.38:0/3338399929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8becb41b0 msgr2=0x55e8bf0d5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19aeffd640 1 --2- 172.21.15.38:0/3338399929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8becb41b0 0x55e8bf0d5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19aeffd640 1 -- 172.21.15.38:0/3338399929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 msgr2=0x55e8bf065a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19aeffd640 1 --2- 172.21.15.38:0/3338399929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf065a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19aeffd640 1 -- 172.21.15.38:0/3338399929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8bf0d62d0 con 0x55e8bec934d0 2024-05-21T15:40:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19af7fe640 1 --2- 172.21.15.38:0/3338399929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf065a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f19aeffd640 1 --2- 172.21.15.38:0/3338399929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec934d0 0x55e8bf065f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19a00092e0 tx=0x7f19a0005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a84dd50efa88fd2a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.779+0000 7f198effd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:40:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.780+0000 7f198effd640 1 -- 172.21.15.38:0/3338399929 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0032070 con 0x55e8bec934d0 2024-05-21T15:40:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.780+0000 7f198effd640 1 -- 172.21.15.38:0/3338399929 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f19a002e3a0 con 0x55e8bec934d0 2024-05-21T15:40:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.780+0000 7f19bc970740 1 -- 172.21.15.38:0/3338399929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19a40a0110 con 0x55e8bec934d0 2024-05-21T15:40:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.780+0000 7f198effd640 1 -- 172.21.15.38:0/3338399929 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f19a003e050 con 0x55e8bec934d0 2024-05-21T15:40:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.780+0000 7f19bc970740 10 client.26785 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.780+0000 7f19bc970740 1 -- 172.21.15.38:0/3338399929 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e8bf0d7490 con 0x55e8bec934d0 2024-05-21T15:40:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.780+0000 7f198effd640 1 -- 172.21.15.38:0/3338399929 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19a40a0110 con 0x55e8bec934d0 2024-05-21T15:40:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.781+0000 7f198effd640 1 -- 172.21.15.38:0/3338399929 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f19a0019430 con 0x55e8bec934d0 2024-05-21T15:40:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.781+0000 7f19bc970740 10 client.26785 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.781+0000 7f19bc970740 10 client.26785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.781+0000 7f19bc970740 20 client.26785 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.781+0000 7f19bc970740 1 --2- 172.21.15.38:0/3338399929 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8bec481c0 0x55e8bf0d9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.781+0000 7f19bc970740 4 client.26785 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:40:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.781+0000 7f19bc970740 1 -- 172.21.15.38:0/3338399929 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8befbfdc0 con 0x55e8bec481c0 2024-05-21T15:40:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.781+0000 7f19af7fe640 1 --2- 172.21.15.38:0/3338399929 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8bec481c0 0x55e8bf0d9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.781+0000 7f198effd640 10 client.26785 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.781+0000 7f19af7fe640 1 --2- 172.21.15.38:0/3338399929 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8bec481c0 0x55e8bf0d9d60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=873eb6292f59f91f server_cookie=129cff0537e1dcd6 in_seq=0 out_seq=0 2024-05-21T15:40:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f198effd640 1 -- 172.21.15.38:0/3338399929 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8befbfdc0 con 0x55e8bec481c0 2024-05-21T15:40:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f198effd640 10 client.26785 handle_command_reply: tid=0 2024-05-21T15:40:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f19bc970740 1 client.26785 shutdown 2024-05-21T15:40:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f19bc970740 1 -- 172.21.15.38:0/3338399929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec934d0 msgr2=0x55e8bf065f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f19bc970740 1 --2- 172.21.15.38:0/3338399929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec934d0 0x55e8bf065f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19a00092e0 tx=0x7f19a0005ce0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f19bc970740 1 -- 172.21.15.38:0/3338399929 shutdown_connections 2024-05-21T15:40:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f19bc970740 1 --2- 172.21.15.38:0/3338399929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8becb41b0 0x55e8bf0d5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f19bc970740 1 --2- 172.21.15.38:0/3338399929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8bec934d0 0x55e8bf065f80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f19bc970740 1 --2- 172.21.15.38:0/3338399929 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8bec481c0 0x55e8bf0d9d60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f19bc970740 1 --2- 172.21.15.38:0/3338399929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8bec8b2f0 0x55e8bf065a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f198effd640 0 client.26785 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f19bc970740 1 -- 172.21.15.38:0/3338399929 >> 172.21.15.38:0/3338399929 conn(0x55e8beca8300 msgr2=0x55e8bf052690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f19bc970740 1 -- 172.21.15.38:0/3338399929 shutdown_connections 2024-05-21T15:40:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f19bc970740 1 -- 172.21.15.38:0/3338399929 wait complete. 2024-05-21T15:40:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.802+0000 7f19bc970740 20 client.26785 trim_cache size 0 max 16384 2024-05-21T15:40:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:40:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:40:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:40:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:40:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.803+0000 7f19b5ad3640 1 -- 172.21.15.38:0/2124583625 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f19980a2c30 msgr2=0x7f19980a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.803+0000 7f19b5ad3640 1 --2- 172.21.15.38:0/2124583625 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f19980a2c30 0x7f19980a5140 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55e8bf055030 tx=0x7f19a80095c0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.803+0000 7f19b5ad3640 1 -- 172.21.15.38:0/2124583625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b0100750 msgr2=0x7f19b01991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.803+0000 7f19b5ad3640 1 --2- 172.21.15.38:0/2124583625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b0100750 0x7f19b01991b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f19a400e9f0 tx=0x7f19a400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.803+0000 7f19b5ad3640 1 -- 172.21.15.38:0/2124583625 shutdown_connections 2024-05-21T15:40:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.803+0000 7f19b5ad3640 1 --2- 172.21.15.38:0/2124583625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19b01040a0 0x7f19b01a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.803+0000 7f19b5ad3640 1 --2- 172.21.15.38:0/2124583625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19b0101250 0x7f19b01996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.803+0000 7f19b5ad3640 1 --2- 172.21.15.38:0/2124583625 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f19980a2c30 0x7f19980a5140 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.803+0000 7f19b5ad3640 1 --2- 172.21.15.38:0/2124583625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b0100750 0x7f19b01991b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.803+0000 7f19b5ad3640 1 -- 172.21.15.38:0/2124583625 >> 172.21.15.38:0/2124583625 conn(0x7f19b00fc390 msgr2=0x7f19b0106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.803+0000 7f19b5ad3640 1 -- 172.21.15.38:0/2124583625 shutdown_connections 2024-05-21T15:40:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:28.803+0000 7f19b5ad3640 1 -- 172.21.15.38:0/2124583625 wait complete. 2024-05-21T15:40:29.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:29 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:40:29.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:29 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:40:29.522 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:29 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:40:30.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:30 smithi053 ceph-mon[36496]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 26 MiB/s wr, 5.92k op/s 2024-05-21T15:40:30.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:30 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:40:30.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:30 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:40:30.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:30 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:40:30.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:30 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:40:30.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:30 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:40:30.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:30 smithi045 ceph-mon[36409]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 26 MiB/s wr, 5.92k op/s 2024-05-21T15:40:30.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:30 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:40:30.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:30 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:40:30.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:30 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:40:30.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:30 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:40:30.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:30 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:40:30.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:30 smithi038 ceph-mon[31995]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 26 MiB/s wr, 5.92k op/s 2024-05-21T15:40:30.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:30 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:40:30.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:30 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:40:30.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:30 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:40:30.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:30 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:40:30.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:30 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:40:31.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:40:31.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:40:31.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:31 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:40:31.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:40:31.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:40:31.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:31 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:40:31.522 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:40:31.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:40:31.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:31 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:40:32.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:32 smithi053 ceph-mon[36496]: pgmap v534: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 32 MiB/s wr, 7.06k op/s 2024-05-21T15:40:32.508 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:32 smithi045 ceph-mon[36409]: pgmap v534: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 32 MiB/s wr, 7.06k op/s 2024-05-21T15:40:32.522 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:32 smithi038 ceph-mon[31995]: pgmap v534: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 32 MiB/s wr, 7.06k op/s 2024-05-21T15:40:33.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:33 smithi053 ceph-mon[36496]: pgmap v535: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 5.62k op/s 2024-05-21T15:40:33.508 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:33 smithi045 ceph-mon[36409]: pgmap v535: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 5.62k op/s 2024-05-21T15:40:33.522 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:33 smithi038 ceph-mon[31995]: pgmap v535: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 5.62k op/s 2024-05-21T15:40:35.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:35 smithi053 ceph-mon[36496]: pgmap v536: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 24 MiB/s wr, 5.35k op/s 2024-05-21T15:40:35.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:35 smithi045 ceph-mon[36409]: pgmap v536: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 24 MiB/s wr, 5.35k op/s 2024-05-21T15:40:35.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:35 smithi038 ceph-mon[31995]: pgmap v536: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 24 MiB/s wr, 5.35k op/s 2024-05-21T15:40:37.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:37 smithi053 ceph-mon[36496]: pgmap v537: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 27 MiB/s wr, 6.21k op/s 2024-05-21T15:40:37.758 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:37 smithi045 ceph-mon[36409]: pgmap v537: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 27 MiB/s wr, 6.21k op/s 2024-05-21T15:40:37.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:37 smithi038 ceph-mon[31995]: pgmap v537: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 27 MiB/s wr, 6.21k op/s 2024-05-21T15:40:39.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:39 smithi053 ceph-mon[36496]: pgmap v538: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.06k op/s 2024-05-21T15:40:39.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:39 smithi045 ceph-mon[36409]: pgmap v538: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.06k op/s 2024-05-21T15:40:39.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:39 smithi038 ceph-mon[31995]: pgmap v538: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.06k op/s 2024-05-21T15:40:41.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:41 smithi053 ceph-mon[36496]: pgmap v539: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 4.78k op/s 2024-05-21T15:40:41.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:41 smithi045 ceph-mon[36409]: pgmap v539: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 4.78k op/s 2024-05-21T15:40:41.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:41 smithi038 ceph-mon[31995]: pgmap v539: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 4.78k op/s 2024-05-21T15:40:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.926+0000 7f198e17e640 1 -- 172.21.15.38:0/3575833664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1988105f20 msgr2=0x7f1988108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.926+0000 7f198e17e640 1 --2- 172.21.15.38:0/3575833664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1988105f20 0x7f1988108340 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f197c009900 tx=0x7f197c02a410 comp rx=0 tx=0).stop 2024-05-21T15:40:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.926+0000 7f198e17e640 1 -- 172.21.15.38:0/3575833664 shutdown_connections 2024-05-21T15:40:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.926+0000 7f198e17e640 1 --2- 172.21.15.38:0/3575833664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1988108880 0x7f198810aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.926+0000 7f198e17e640 1 --2- 172.21.15.38:0/3575833664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1988105f20 0x7f1988108340 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.926+0000 7f198e17e640 1 --2- 172.21.15.38:0/3575833664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19881035c0 0x7f19881059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.926+0000 7f198e17e640 1 -- 172.21.15.38:0/3575833664 >> 172.21.15.38:0/3575833664 conn(0x7f19880fcbc0 msgr2=0x7f19880ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.927+0000 7f198e17e640 1 -- 172.21.15.38:0/3575833664 shutdown_connections 2024-05-21T15:40:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.927+0000 7f198e17e640 1 -- 172.21.15.38:0/3575833664 wait complete. 2024-05-21T15:40:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.927+0000 7f198e17e640 1 Processor -- start 2024-05-21T15:40:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.928+0000 7f198e17e640 1 -- start start 2024-05-21T15:40:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.928+0000 7f198e17e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881035c0 0x7f1988199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.928+0000 7f198e17e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1988105f20 0x7f1988199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.928+0000 7f198e17e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1988108880 0x7f19881a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.928+0000 7f198e17e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f198810e730 con 0x7f19881035c0 2024-05-21T15:40:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.928+0000 7f198e17e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f198810e5b0 con 0x7f1988108880 2024-05-21T15:40:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.928+0000 7f198e17e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f198810e8b0 con 0x7f1988105f20 2024-05-21T15:40:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.928+0000 7f19877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881035c0 0x7f1988199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.928+0000 7f19877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881035c0 0x7f1988199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60702/0 (socket says 172.21.15.38:60702) 2024-05-21T15:40:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.928+0000 7f19877fe640 1 -- 172.21.15.38:0/3774682710 learned_addr learned my addr 172.21.15.38:0/3774682710 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.928+0000 7f1986ffd640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1988105f20 0x7f1988199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.928+0000 7f1987fff640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1988108880 0x7f19881a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.929+0000 7f19877fe640 1 -- 172.21.15.38:0/3774682710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1988105f20 msgr2=0x7f1988199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.929+0000 7f19877fe640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1988105f20 0x7f1988199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.929+0000 7f19877fe640 1 -- 172.21.15.38:0/3774682710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1988108880 msgr2=0x7f19881a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.929+0000 7f19877fe640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1988108880 0x7f19881a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.929+0000 7f19877fe640 1 -- 172.21.15.38:0/3774682710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19881a0d60 con 0x7f19881035c0 2024-05-21T15:40:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.929+0000 7f1987fff640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1988108880 0x7f19881a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.929+0000 7f19877fe640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881035c0 0x7f1988199220 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f197800e9c0 tx=0x7f197800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.929+0000 7f1984ff9640 1 -- 172.21.15.38:0/3774682710 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197800cd80 con 0x7f19881035c0 2024-05-21T15:40:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.929+0000 7f1984ff9640 1 -- 172.21.15.38:0/3774682710 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1978005500 con 0x7f19881035c0 2024-05-21T15:40:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.929+0000 7f198e17e640 1 -- 172.21.15.38:0/3774682710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19881a0ff0 con 0x7f19881035c0 2024-05-21T15:40:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.929+0000 7f1984ff9640 1 -- 172.21.15.38:0/3774682710 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978010690 con 0x7f19881035c0 2024-05-21T15:40:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.929+0000 7f198e17e640 1 -- 172.21.15.38:0/3774682710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19881a14b0 con 0x7f19881035c0 2024-05-21T15:40:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.932+0000 7f199502b740 1 Processor -- start 2024-05-21T15:40:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.932+0000 7f1984ff9640 1 -- 172.21.15.38:0/3774682710 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1978010830 con 0x7f19881035c0 2024-05-21T15:40:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.932+0000 7f199502b740 1 -- start start 2024-05-21T15:40:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.932+0000 7f199502b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c865111b0 0x562c868b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.932+0000 7f199502b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f199502b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c864f04d0 0x562c868b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f19877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f1987fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c865111b0 0x562c868b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f1987fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c865111b0 0x562c868b0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60714/0 (socket says 172.21.15.38:60714) 2024-05-21T15:40:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f19877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868b13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45978/0 (socket says 172.21.15.38:45978) 2024-05-21T15:40:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f199502b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562c86815340 con 0x562c865111b0 2024-05-21T15:40:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f199502b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562c867bd290 con 0x562c864e82f0 2024-05-21T15:40:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f199502b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562c86811300 con 0x562c864f04d0 2024-05-21T15:40:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f1987fff640 1 -- 172.21.15.38:0/2883088154 learned_addr learned my addr 172.21.15.38:0/2883088154 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f1986ffd640 1 --2- 172.21.15.38:0/2883088154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c864f04d0 0x562c868b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f1984ff9640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f19700a2c30 0x7f19700a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f1986ffd640 1 -- 172.21.15.38:0/2883088154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 msgr2=0x562c868b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f1986ffd640 1 --2- 172.21.15.38:0/2883088154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f1986ffd640 1 -- 172.21.15.38:0/2883088154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c865111b0 msgr2=0x562c868b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f1986ffd640 1 --2- 172.21.15.38:0/2883088154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c865111b0 0x562c868b0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f1986ffd640 1 -- 172.21.15.38:0/2883088154 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c867e18e0 con 0x562c864f04d0 2024-05-21T15:40:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f19877fe640 1 --2- 172.21.15.38:0/2883088154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.933+0000 7f1984ff9640 1 -- 172.21.15.38:0/3774682710 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f19780cf430 con 0x7f19881035c0 2024-05-21T15:40:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.934+0000 7f1987fff640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f19700a2c30 0x7f19700a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.934+0000 7f1986ffd640 1 --2- 172.21.15.38:0/2883088154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c864f04d0 0x562c868b82e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f197c005ab0 tx=0x7f197c009600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e579de9ab22725ae server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.934+0000 7f196dffb640 1 -- 172.21.15.38:0/2883088154 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c040070 con 0x562c864f04d0 2024-05-21T15:40:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.934+0000 7f196dffb640 1 -- 172.21.15.38:0/2883088154 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f197c004030 con 0x562c864f04d0 2024-05-21T15:40:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.934+0000 7f196dffb640 1 -- 172.21.15.38:0/2883088154 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c035040 con 0x562c864f04d0 2024-05-21T15:40:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.935+0000 7f199502b740 1 -- 172.21.15.38:0/2883088154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c864f04d0 msgr2=0x562c868b82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.935+0000 7f1987fff640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f19700a2c30 0x7f19700a5140 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1980005c40 tx=0x7f1980014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.935+0000 7f199502b740 1 --2- 172.21.15.38:0/2883088154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c864f04d0 0x562c868b82e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f197c005ab0 tx=0x7f197c009600 comp rx=0 tx=0).stop 2024-05-21T15:40:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.935+0000 7f199502b740 1 -- 172.21.15.38:0/2883088154 shutdown_connections 2024-05-21T15:40:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.935+0000 7f199502b740 1 --2- 172.21.15.38:0/2883088154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c864f04d0 0x562c868b82e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.935+0000 7f199502b740 1 --2- 172.21.15.38:0/2883088154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.935+0000 7f199502b740 1 --2- 172.21.15.38:0/2883088154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c865111b0 0x562c868b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.935+0000 7f199502b740 1 -- 172.21.15.38:0/2883088154 >> 172.21.15.38:0/2883088154 conn(0x562c86505300 msgr2=0x562c868ac7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.935+0000 7f199502b740 1 -- 172.21.15.38:0/2883088154 shutdown_connections 2024-05-21T15:40:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.935+0000 7f199502b740 1 -- 172.21.15.38:0/2883088154 wait complete. 2024-05-21T15:40:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.935+0000 7f199502b740 1 -- start start 2024-05-21T15:40:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.936+0000 7f199502b740 1 Processor -- start 2024-05-21T15:40:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.936+0000 7f199502b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c864e82f0 0x562c868bab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.936+0000 7f199502b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864f04d0 0x562c868bb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.936+0000 7f19877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c864e82f0 0x562c868bab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.936+0000 7f19877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c864e82f0 0x562c868bab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60720/0 (socket says 172.21.15.38:60720) 2024-05-21T15:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.936+0000 7f19877fe640 1 -- 172.21.15.38:0/2594579079 learned_addr learned my addr 172.21.15.38:0/2594579079 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.936+0000 7f199502b740 1 --2- 172.21.15.38:0/2594579079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c865111b0 0x562c868c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.936+0000 7f199502b740 1 -- 172.21.15.38:0/2594579079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562c868b8cd0 con 0x562c864e82f0 2024-05-21T15:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.936+0000 7f199502b740 1 -- 172.21.15.38:0/2594579079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562c867d0790 con 0x562c864f04d0 2024-05-21T15:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.936+0000 7f199502b740 1 -- 172.21.15.38:0/2594579079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562c868b8e50 con 0x562c865111b0 2024-05-21T15:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.936+0000 7f199502b740 10 client.0 _ll_register_callbacks cb 0x562c8680ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.937+0000 7f1986ffd640 1 --2- 172.21.15.38:0/2594579079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864f04d0 0x562c868bb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.937+0000 7f1987fff640 1 --2- 172.21.15.38:0/2594579079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c865111b0 0x562c868c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.937+0000 7f19877fe640 1 -- 172.21.15.38:0/2594579079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c865111b0 msgr2=0x562c868c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.937+0000 7f19877fe640 1 --2- 172.21.15.38:0/2594579079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c865111b0 0x562c868c1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.937+0000 7f19877fe640 1 -- 172.21.15.38:0/2594579079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864f04d0 msgr2=0x562c868bb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.937+0000 7f19877fe640 1 --2- 172.21.15.38:0/2594579079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864f04d0 0x562c868bb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.937+0000 7f19877fe640 1 -- 172.21.15.38:0/2594579079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c86529210 con 0x562c864e82f0 2024-05-21T15:40:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.937+0000 7f19877fe640 1 --2- 172.21.15.38:0/2594579079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c864e82f0 0x562c868bab30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f197809bbd0 tx=0x7f197800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9450f5af4d2236e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.937+0000 7f1966ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.937+0000 7f1966ffd640 1 -- 172.21.15.38:0/2594579079 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197809b780 con 0x562c864e82f0 2024-05-21T15:40:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.937+0000 7f1966ffd640 1 -- 172.21.15.38:0/2594579079 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f19780046c0 con 0x562c864e82f0 2024-05-21T15:40:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.937+0000 7f199502b740 1 -- 172.21.15.38:0/2594579079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f197c004030 con 0x562c864e82f0 2024-05-21T15:40:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.938+0000 7f1966ffd640 1 -- 172.21.15.38:0/2594579079 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f197809b5a0 con 0x562c864e82f0 2024-05-21T15:40:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.938+0000 7f199502b740 10 client.17112 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.938+0000 7f199502b740 1 -- 172.21.15.38:0/2594579079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c865f5620 con 0x562c864e82f0 2024-05-21T15:40:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.938+0000 7f1966ffd640 1 -- 172.21.15.38:0/2594579079 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f197c004030 con 0x562c864e82f0 2024-05-21T15:40:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.938+0000 7f1966ffd640 1 -- 172.21.15.38:0/2594579079 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f19780a4420 con 0x562c864e82f0 2024-05-21T15:40:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.938+0000 7f199502b740 10 client.17112 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.938+0000 7f199502b740 10 client.17112 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.938+0000 7f199502b740 20 client.17112 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.938+0000 7f199502b740 1 --2- 172.21.15.38:0/2594579079 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c864cfd80 0x562c868c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.938+0000 7f199502b740 4 client.17112 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.938+0000 7f199502b740 1 -- 172.21.15.38:0/2594579079 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c86822a70 con 0x562c864cfd80 2024-05-21T15:40:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.938+0000 7f1986ffd640 1 --2- 172.21.15.38:0/2594579079 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c864cfd80 0x562c868c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.939+0000 7f1966ffd640 10 client.17112 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.939+0000 7f1986ffd640 1 --2- 172.21.15.38:0/2594579079 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c864cfd80 0x562c868c85d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a702415670f608 server_cookie=ddc6a8eac3e088e8 in_seq=0 out_seq=0 2024-05-21T15:40:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.940+0000 7f1966ffd640 1 -- 172.21.15.38:0/2594579079 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562c86822a70 con 0x562c864cfd80 2024-05-21T15:40:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.940+0000 7f1966ffd640 10 client.17112 handle_command_reply: tid=0 2024-05-21T15:40:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.940+0000 7f199502b740 1 client.17112 shutdown 2024-05-21T15:40:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.940+0000 7f199502b740 1 -- 172.21.15.38:0/2594579079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c864e82f0 msgr2=0x562c868bab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.940+0000 7f199502b740 1 --2- 172.21.15.38:0/2594579079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c864e82f0 0x562c868bab30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f197809bbd0 tx=0x7f197800cf20 comp rx=0 tx=0).stop 2024-05-21T15:40:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.940+0000 7f199502b740 1 -- 172.21.15.38:0/2594579079 shutdown_connections 2024-05-21T15:40:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.940+0000 7f199502b740 1 --2- 172.21.15.38:0/2594579079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c865111b0 0x562c868c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.940+0000 7f199502b740 1 --2- 172.21.15.38:0/2594579079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864f04d0 0x562c868bb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.940+0000 7f199502b740 1 --2- 172.21.15.38:0/2594579079 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c864cfd80 0x562c868c85d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.940+0000 7f199502b740 1 --2- 172.21.15.38:0/2594579079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c864e82f0 0x562c868bab30 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.940+0000 7f199502b740 1 -- 172.21.15.38:0/2594579079 >> 172.21.15.38:0/2594579079 conn(0x562c86505300 msgr2=0x562c868ab810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.940+0000 7f1966ffd640 0 client.17112 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.940+0000 7f199502b740 1 -- 172.21.15.38:0/2594579079 shutdown_connections 2024-05-21T15:40:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.941+0000 7f199502b740 1 -- 172.21.15.38:0/2594579079 wait complete. 2024-05-21T15:40:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.941+0000 7f199502b740 20 client.17112 trim_cache size 0 max 16384 2024-05-21T15:40:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.963+0000 7f199502b740 1 Processor -- start 2024-05-21T15:40:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.963+0000 7f199502b740 1 -- start start 2024-05-21T15:40:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.963+0000 7f199502b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868ac160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.963+0000 7f199502b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c864f04d0 0x562c868ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.963+0000 7f199502b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c865111b0 0x562c868b9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.963+0000 7f199502b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f19881aa390 con 0x562c865111b0 2024-05-21T15:40:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.963+0000 7f199502b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19881abfc0 con 0x562c864e82f0 2024-05-21T15:40:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.963+0000 7f199502b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f19881aaa60 con 0x562c864f04d0 2024-05-21T15:40:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.963+0000 7f1987fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c865111b0 0x562c868b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.963+0000 7f1986ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c864f04d0 0x562c868ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.963+0000 7f1987fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c865111b0 0x562c868b9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60726/0 (socket says 172.21.15.38:60726) 2024-05-21T15:40:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.963+0000 7f19877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.963+0000 7f1987fff640 1 -- 172.21.15.38:0/4087616995 learned_addr learned my addr 172.21.15.38:0/4087616995 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.964+0000 7f1987fff640 1 -- 172.21.15.38:0/4087616995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c864f04d0 msgr2=0x562c868ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.964+0000 7f1987fff640 1 --2- 172.21.15.38:0/4087616995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c864f04d0 0x562c868ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.964+0000 7f1987fff640 1 -- 172.21.15.38:0/4087616995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 msgr2=0x562c868ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.964+0000 7f1987fff640 1 --2- 172.21.15.38:0/4087616995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.964+0000 7f1987fff640 1 -- 172.21.15.38:0/4087616995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c864cffb0 con 0x562c865111b0 2024-05-21T15:40:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.964+0000 7f1986ffd640 1 --2- 172.21.15.38:0/4087616995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c864f04d0 0x562c868ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.964+0000 7f1987fff640 1 --2- 172.21.15.38:0/4087616995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c865111b0 0x562c868b9220 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1980012600 tx=0x7f1980017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876a443805a2b6a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.964+0000 7f196d7fa640 1 -- 172.21.15.38:0/4087616995 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198001c480 con 0x562c865111b0 2024-05-21T15:40:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.964+0000 7f196d7fa640 1 -- 172.21.15.38:0/4087616995 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f198001cb00 con 0x562c865111b0 2024-05-21T15:40:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.964+0000 7f196d7fa640 1 -- 172.21.15.38:0/4087616995 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980010c70 con 0x562c865111b0 2024-05-21T15:40:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.965+0000 7f199502b740 1 -- 172.21.15.38:0/4087616995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c865111b0 msgr2=0x562c868b9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.965+0000 7f199502b740 1 --2- 172.21.15.38:0/4087616995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c865111b0 0x562c868b9220 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1980012600 tx=0x7f1980017000 comp rx=0 tx=0).stop 2024-05-21T15:40:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.965+0000 7f199502b740 1 -- 172.21.15.38:0/4087616995 shutdown_connections 2024-05-21T15:40:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.965+0000 7f199502b740 1 --2- 172.21.15.38:0/4087616995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c865111b0 0x562c868b9220 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.965+0000 7f199502b740 1 --2- 172.21.15.38:0/4087616995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c864f04d0 0x562c868ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.965+0000 7f199502b740 1 --2- 172.21.15.38:0/4087616995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.965+0000 7f199502b740 1 -- 172.21.15.38:0/4087616995 >> 172.21.15.38:0/4087616995 conn(0x562c86505300 msgr2=0x562c868b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.965+0000 7f199502b740 1 -- 172.21.15.38:0/4087616995 shutdown_connections 2024-05-21T15:40:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.965+0000 7f199502b740 1 -- 172.21.15.38:0/4087616995 wait complete. 2024-05-21T15:40:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.966+0000 7f199502b740 1 -- start start 2024-05-21T15:40:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.966+0000 7f199502b740 1 Processor -- start 2024-05-21T15:40:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.966+0000 7f199502b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868be4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.966+0000 7f199502b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c864f04d0 0x562c868be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.966+0000 7f199502b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c865111b0 0x562c8692e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f199502b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f19881ab830 con 0x562c864f04d0 2024-05-21T15:40:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f199502b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19881ab130 con 0x562c864e82f0 2024-05-21T15:40:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f199502b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f19881a8370 con 0x562c865111b0 2024-05-21T15:40:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f19877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868be4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f1987fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c865111b0 0x562c8692e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f1986ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c864f04d0 0x562c868be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f19877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868be4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:46008/0 (socket says 172.21.15.38:46008) 2024-05-21T15:40:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f199502b740 10 client.0 _ll_register_callbacks cb 0x7f1978009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f19877fe640 1 -- 172.21.15.38:0/2684467440 learned_addr learned my addr 172.21.15.38:0/2684467440 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f19877fe640 1 -- 172.21.15.38:0/2684467440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c865111b0 msgr2=0x562c8692e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f19877fe640 1 --2- 172.21.15.38:0/2684467440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c865111b0 0x562c8692e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f19877fe640 1 -- 172.21.15.38:0/2684467440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c864f04d0 msgr2=0x562c868be9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f19877fe640 1 --2- 172.21.15.38:0/2684467440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c864f04d0 0x562c868be9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f19877fe640 1 -- 172.21.15.38:0/2684467440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c868befe0 con 0x562c864e82f0 2024-05-21T15:40:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f1987fff640 1 --2- 172.21.15.38:0/2684467440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c865111b0 0x562c8692e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f1986ffd640 1 --2- 172.21.15.38:0/2684467440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c864f04d0 0x562c868be9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f19877fe640 1 --2- 172.21.15.38:0/2684467440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868be4b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1978009360 tx=0x7f197809b2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6df558605526a66e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.967+0000 7f1966ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:40:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f1966ffd640 1 -- 172.21.15.38:0/2684467440 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197807e020 con 0x562c864e82f0 2024-05-21T15:40:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f1966ffd640 1 -- 172.21.15.38:0/2684467440 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f197809b330 con 0x562c864e82f0 2024-05-21T15:40:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f199502b740 1 -- 172.21.15.38:0/2684467440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f198001cb00 con 0x562c864e82f0 2024-05-21T15:40:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f1966ffd640 1 -- 172.21.15.38:0/2684467440 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f197809b780 con 0x562c864e82f0 2024-05-21T15:40:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f199502b740 10 client.26747 fetch_fsmap learned FSMap version 31 2024-05-21T15:40:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f199502b740 1 -- 172.21.15.38:0/2684467440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c8692ff60 con 0x562c864e82f0 2024-05-21T15:40:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f1966ffd640 1 -- 172.21.15.38:0/2684467440 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f198001cb00 con 0x562c864e82f0 2024-05-21T15:40:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f1966ffd640 1 -- 172.21.15.38:0/2684467440 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f19780b4040 con 0x562c864e82f0 2024-05-21T15:40:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f199502b740 10 client.26747 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:40:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f199502b740 10 client.26747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f199502b740 20 client.26747 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f199502b740 1 --2- 172.21.15.38:0/2684467440 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c864a51c0 0x562c86932890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f199502b740 4 client.26747 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:40:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f199502b740 1 -- 172.21.15.38:0/2684467440 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c8681cdc0 con 0x562c864a51c0 2024-05-21T15:40:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.968+0000 7f1986ffd640 1 --2- 172.21.15.38:0/2684467440 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c864a51c0 0x562c86932890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.969+0000 7f1966ffd640 10 client.26747 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.969+0000 7f1986ffd640 1 --2- 172.21.15.38:0/2684467440 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c864a51c0 0x562c86932890 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52cc24f665ee2e48 server_cookie=a209afa0cbae994b in_seq=0 out_seq=0 2024-05-21T15:40:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f1966ffd640 1 -- 172.21.15.38:0/2684467440 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c8681cdc0 con 0x562c864a51c0 2024-05-21T15:40:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f1966ffd640 10 client.26747 handle_command_reply: tid=0 2024-05-21T15:40:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f199502b740 1 client.26747 shutdown 2024-05-21T15:40:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f199502b740 1 -- 172.21.15.38:0/2684467440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 msgr2=0x562c868be4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f199502b740 1 --2- 172.21.15.38:0/2684467440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868be4b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1978009360 tx=0x7f197809b2c0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f199502b740 1 -- 172.21.15.38:0/2684467440 shutdown_connections 2024-05-21T15:40:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f199502b740 1 --2- 172.21.15.38:0/2684467440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c865111b0 0x562c8692e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f199502b740 1 --2- 172.21.15.38:0/2684467440 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c864a51c0 0x562c86932890 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f199502b740 1 --2- 172.21.15.38:0/2684467440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c864f04d0 0x562c868be9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f199502b740 1 --2- 172.21.15.38:0/2684467440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c864e82f0 0x562c868be4b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f1966ffd640 0 client.26747 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f199502b740 1 -- 172.21.15.38:0/2684467440 >> 172.21.15.38:0/2684467440 conn(0x562c86505300 msgr2=0x562c868ac160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f199502b740 1 -- 172.21.15.38:0/2684467440 shutdown_connections 2024-05-21T15:40:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f199502b740 1 -- 172.21.15.38:0/2684467440 wait complete. 2024-05-21T15:40:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.985+0000 7f199502b740 20 client.26747 trim_cache size 0 max 16384 2024-05-21T15:40:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:40:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:40:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:40:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:40:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.986+0000 7f198e17e640 1 -- 172.21.15.38:0/3774682710 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f19700a2c30 msgr2=0x7f19700a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.986+0000 7f198e17e640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f19700a2c30 0x7f19700a5140 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1980005c40 tx=0x7f1980014040 comp rx=0 tx=0).stop 2024-05-21T15:40:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.986+0000 7f198e17e640 1 -- 172.21.15.38:0/3774682710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881035c0 msgr2=0x7f1988199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.986+0000 7f198e17e640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881035c0 0x7f1988199220 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f197800e9c0 tx=0x7f197800ee80 comp rx=0 tx=0).stop 2024-05-21T15:40:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.986+0000 7f198e17e640 1 -- 172.21.15.38:0/3774682710 shutdown_connections 2024-05-21T15:40:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.986+0000 7f198e17e640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1988108880 0x7f19881a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.986+0000 7f198e17e640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1988105f20 0x7f1988199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.986+0000 7f198e17e640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f19700a2c30 0x7f19700a5140 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.986+0000 7f198e17e640 1 --2- 172.21.15.38:0/3774682710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881035c0 0x7f1988199220 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.986+0000 7f198e17e640 1 -- 172.21.15.38:0/3774682710 >> 172.21.15.38:0/3774682710 conn(0x7f19880fcbc0 msgr2=0x7f19881044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.986+0000 7f198e17e640 1 -- 172.21.15.38:0/3774682710 shutdown_connections 2024-05-21T15:40:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:41.986+0000 7f198e17e640 1 -- 172.21.15.38:0/3774682710 wait complete. 2024-05-21T15:40:43.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:43 smithi053 ceph-mon[36496]: pgmap v540: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 4.43k op/s 2024-05-21T15:40:43.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:43 smithi045 ceph-mon[36409]: pgmap v540: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 4.43k op/s 2024-05-21T15:40:43.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:43 smithi038 ceph-mon[31995]: pgmap v540: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 4.43k op/s 2024-05-21T15:40:45.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:45 smithi045 ceph-mon[36409]: pgmap v541: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 18 MiB/s wr, 4.09k op/s 2024-05-21T15:40:45.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:45 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:40:45.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:45 smithi038 ceph-mon[31995]: pgmap v541: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 18 MiB/s wr, 4.09k op/s 2024-05-21T15:40:45.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:45 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:40:45.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:45 smithi053 ceph-mon[36496]: pgmap v541: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 18 MiB/s wr, 4.09k op/s 2024-05-21T15:40:45.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:45 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:40:47.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:47 smithi045 ceph-mon[36409]: pgmap v542: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 4.89k op/s 2024-05-21T15:40:47.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:47 smithi038 ceph-mon[31995]: pgmap v542: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 4.89k op/s 2024-05-21T15:40:47.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:47 smithi053 ceph-mon[36496]: pgmap v542: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 4.89k op/s 2024-05-21T15:40:48.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:40:48.654 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:40:48.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.804+0000 7f9155d1a640 1 -- 172.21.15.38:0/792589622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9150107910 msgr2=0x7f9150109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:48.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.804+0000 7f9155d1a640 1 --2- 172.21.15.38:0/792589622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9150107910 0x7f9150109d30 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9148009ca0 tx=0x7f914802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.804+0000 7f9155d1a640 1 -- 172.21.15.38:0/792589622 shutdown_connections 2024-05-21T15:40:48.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.804+0000 7f9155d1a640 1 --2- 172.21.15.38:0/792589622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9150107910 0x7f9150109d30 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.804+0000 7f9155d1a640 1 --2- 172.21.15.38:0/792589622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9150102c10 0x7f9150103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.804+0000 7f9155d1a640 1 --2- 172.21.15.38:0/792589622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91501019d0 0x7f91501026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.804+0000 7f9155d1a640 1 -- 172.21.15.38:0/792589622 >> 172.21.15.38:0/792589622 conn(0x7f91500fcbc0 msgr2=0x7f91500ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:48.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.805+0000 7f9155d1a640 1 -- 172.21.15.38:0/792589622 shutdown_connections 2024-05-21T15:40:48.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.805+0000 7f9155d1a640 1 -- 172.21.15.38:0/792589622 wait complete. 2024-05-21T15:40:48.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.805+0000 7f9155d1a640 1 Processor -- start 2024-05-21T15:40:48.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f9155d1a640 1 -- start start 2024-05-21T15:40:48.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f9155d1a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91501019d0 0x7f9150199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f9155d1a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9150102c10 0x7f9150199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f9155d1a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9150107910 0x7f91501a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f9155d1a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f915010d6a0 con 0x7f9150107910 2024-05-21T15:40:48.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f9155d1a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f915010d520 con 0x7f91501019d0 2024-05-21T15:40:48.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f9155d1a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f915010d820 con 0x7f9150102c10 2024-05-21T15:40:48.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f914ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9150107910 0x7f91501a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f914effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9150102c10 0x7f9150199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f914f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91501019d0 0x7f9150199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f914effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9150102c10 0x7f9150199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:51576/0 (socket says 172.21.15.38:51576) 2024-05-21T15:40:48.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f914ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9150107910 0x7f91501a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33488/0 (socket says 172.21.15.38:33488) 2024-05-21T15:40:48.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f914effd640 1 -- 172.21.15.38:0/356734975 learned_addr learned my addr 172.21.15.38:0/356734975 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:48.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f914effd640 1 -- 172.21.15.38:0/356734975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91501019d0 msgr2=0x7f9150199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:48.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f914effd640 1 --2- 172.21.15.38:0/356734975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91501019d0 0x7f9150199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f914effd640 1 -- 172.21.15.38:0/356734975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9150107910 msgr2=0x7f91501a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:48.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f914effd640 1 --2- 172.21.15.38:0/356734975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9150107910 0x7f91501a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f914effd640 1 -- 172.21.15.38:0/356734975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91501a10b0 con 0x7f9150102c10 2024-05-21T15:40:48.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.806+0000 7f914ffff640 1 --2- 172.21.15.38:0/356734975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9150107910 0x7f91501a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:48.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.807+0000 7f914effd640 1 --2- 172.21.15.38:0/356734975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9150102c10 0x7f9150199ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f914400e9f0 tx=0x7f914400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:48.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.807+0000 7f914cff9640 1 -- 172.21.15.38:0/356734975 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914400cd80 con 0x7f9150102c10 2024-05-21T15:40:48.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.807+0000 7f9155d1a640 1 -- 172.21.15.38:0/356734975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91501a13a0 con 0x7f9150102c10 2024-05-21T15:40:48.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.807+0000 7f914cff9640 1 -- 172.21.15.38:0/356734975 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9144010910 con 0x7f9150102c10 2024-05-21T15:40:48.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.807+0000 7f914cff9640 1 -- 172.21.15.38:0/356734975 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91440188f0 con 0x7f9150102c10 2024-05-21T15:40:48.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.807+0000 7f9155d1a640 1 -- 172.21.15.38:0/356734975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91501a1940 con 0x7f9150102c10 2024-05-21T15:40:48.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.810+0000 7f914cff9640 1 -- 172.21.15.38:0/356734975 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9144018b90 con 0x7f9150102c10 2024-05-21T15:40:48.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.810+0000 7f915cbc4740 1 Processor -- start 2024-05-21T15:40:48.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.811+0000 7f915cbc4740 1 -- start start 2024-05-21T15:40:48.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.811+0000 7f915cbc4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605c0d390 0x55f605facbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.811+0000 7f915cbc4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605be44d0 0x55f605fad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.811+0000 7f915cbc4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bec6b0 0x55f605fb4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.811+0000 7f915cbc4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f605f11d00 con 0x55f605be44d0 2024-05-21T15:40:48.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.811+0000 7f915cbc4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f605eb94c0 con 0x55f605c0d390 2024-05-21T15:40:48.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.811+0000 7f915cbc4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f605f0dcc0 con 0x55f605bec6b0 2024-05-21T15:40:48.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.811+0000 7f914effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bec6b0 0x55f605fb4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.811+0000 7f914ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605c0d390 0x55f605facbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.811+0000 7f914ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605c0d390 0x55f605facbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:51112/0 (socket says 172.21.15.38:51112) 2024-05-21T15:40:48.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.811+0000 7f914ffff640 1 -- 172.21.15.38:0/2970820108 learned_addr learned my addr 172.21.15.38:0/2970820108 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:48.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.811+0000 7f914f7fe640 1 --2- 172.21.15.38:0/2970820108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605be44d0 0x55f605fad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.812+0000 7f914ffff640 1 -- 172.21.15.38:0/2970820108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bec6b0 msgr2=0x55f605fb4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:48.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.812+0000 7f914ffff640 1 --2- 172.21.15.38:0/2970820108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bec6b0 0x55f605fb4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.812+0000 7f914ffff640 1 -- 172.21.15.38:0/2970820108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605be44d0 msgr2=0x55f605fad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:48.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.812+0000 7f914ffff640 1 --2- 172.21.15.38:0/2970820108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605be44d0 0x55f605fad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.812+0000 7f914ffff640 1 -- 172.21.15.38:0/2970820108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f605cef7d0 con 0x55f605c0d390 2024-05-21T15:40:48.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.812+0000 7f914cff9640 1 --2- 172.21.15.38:0/356734975 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f91380a2b90 0x7f91380a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.813+0000 7f914ffff640 1 --2- 172.21.15.38:0/2970820108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605c0d390 0x55f605facbb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9148035000 tx=0x7f91480093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b99ae180682cbc38 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.813+0000 7f9135ffb640 1 -- 172.21.15.38:0/2970820108 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9148031650 con 0x55f605c0d390 2024-05-21T15:40:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.813+0000 7f9135ffb640 1 -- 172.21.15.38:0/2970820108 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9148036040 con 0x55f605c0d390 2024-05-21T15:40:48.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.813+0000 7f914cff9640 1 -- 172.21.15.38:0/356734975 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f91440cf460 con 0x7f9150102c10 2024-05-21T15:40:48.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.813+0000 7f914f7fe640 1 --2- 172.21.15.38:0/356734975 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f91380a2b90 0x7f91380a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.814+0000 7f9135ffb640 1 -- 172.21.15.38:0/2970820108 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91480047d0 con 0x55f605c0d390 2024-05-21T15:40:48.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.814+0000 7f915cbc4740 1 -- 172.21.15.38:0/2970820108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605c0d390 msgr2=0x55f605facbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:48.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.814+0000 7f915cbc4740 1 --2- 172.21.15.38:0/2970820108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605c0d390 0x55f605facbb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9148035000 tx=0x7f91480093e0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.814+0000 7f914f7fe640 1 --2- 172.21.15.38:0/356734975 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f91380a2b90 0x7f91380a50a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55f605fadf90 tx=0x7f9140005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:48.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.814+0000 7f915cbc4740 1 -- 172.21.15.38:0/2970820108 shutdown_connections 2024-05-21T15:40:48.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.814+0000 7f915cbc4740 1 --2- 172.21.15.38:0/2970820108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bec6b0 0x55f605fb4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.814+0000 7f915cbc4740 1 --2- 172.21.15.38:0/2970820108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605be44d0 0x55f605fad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.814+0000 7f915cbc4740 1 --2- 172.21.15.38:0/2970820108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605c0d390 0x55f605facbb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.814+0000 7f915cbc4740 1 -- 172.21.15.38:0/2970820108 >> 172.21.15.38:0/2970820108 conn(0x55f605c014e0 msgr2=0x55f605fa8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:48.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.814+0000 7f915cbc4740 1 -- 172.21.15.38:0/2970820108 shutdown_connections 2024-05-21T15:40:48.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.814+0000 7f915cbc4740 1 -- 172.21.15.38:0/2970820108 wait complete. 2024-05-21T15:40:48.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.815+0000 7f915cbc4740 1 -- start start 2024-05-21T15:40:48.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.815+0000 7f915cbc4740 1 Processor -- start 2024-05-21T15:40:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f915cbc4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605be44d0 0x55f605fbada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f915cbc4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bec6b0 0x55f605fbb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f915cbc4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605c0d390 0x55f605fc2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f915cbc4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f605fb8cb0 con 0x55f605bec6b0 2024-05-21T15:40:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f915cbc4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f605eccb90 con 0x55f605c0d390 2024-05-21T15:40:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f915cbc4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f605fb8e30 con 0x55f605be44d0 2024-05-21T15:40:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f914f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605be44d0 0x55f605fbada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f914f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605be44d0 0x55f605fbada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:51596/0 (socket says 172.21.15.38:51596) 2024-05-21T15:40:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f914f7fe640 1 -- 172.21.15.38:0/2272706738 learned_addr learned my addr 172.21.15.38:0/2272706738 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:48.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f914effd640 1 --2- 172.21.15.38:0/2272706738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bec6b0 0x55f605fbb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f914ffff640 1 --2- 172.21.15.38:0/2272706738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605c0d390 0x55f605fc2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f915cbc4740 10 client.0 _ll_register_callbacks cb 0x55f605f08da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f914ffff640 1 -- 172.21.15.38:0/2272706738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605be44d0 msgr2=0x55f605fbada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f914ffff640 1 --2- 172.21.15.38:0/2272706738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605be44d0 0x55f605fbada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f914ffff640 1 -- 172.21.15.38:0/2272706738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bec6b0 msgr2=0x55f605fbb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f914ffff640 1 --2- 172.21.15.38:0/2272706738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bec6b0 0x55f605fbb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f914ffff640 1 -- 172.21.15.38:0/2272706738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f605c253f0 con 0x55f605c0d390 2024-05-21T15:40:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.816+0000 7f914f7fe640 1 --2- 172.21.15.38:0/2272706738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605be44d0 0x55f605fbada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.817+0000 7f914ffff640 1 --2- 172.21.15.38:0/2272706738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605c0d390 0x55f605fc2240 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9148009ca0 tx=0x7f9148031700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5620170ed114b647 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.817+0000 7f912effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:40:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.817+0000 7f912effd640 1 -- 172.21.15.38:0/2272706738 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9148003b80 con 0x55f605c0d390 2024-05-21T15:40:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.817+0000 7f915cbc4740 1 -- 172.21.15.38:0/2272706738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9148036040 con 0x55f605c0d390 2024-05-21T15:40:48.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.817+0000 7f912effd640 1 -- 172.21.15.38:0/2272706738 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9148003d20 con 0x55f605c0d390 2024-05-21T15:40:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.817+0000 7f912effd640 1 -- 172.21.15.38:0/2272706738 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9148003f20 con 0x55f605c0d390 2024-05-21T15:40:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.817+0000 7f915cbc4740 10 client.26759 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:48.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.817+0000 7f915cbc4740 1 -- 172.21.15.38:0/2272706738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f605be50f0 con 0x55f605c0d390 2024-05-21T15:40:48.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.818+0000 7f912effd640 1 -- 172.21.15.38:0/2272706738 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9148036040 con 0x55f605c0d390 2024-05-21T15:40:48.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.818+0000 7f912effd640 1 -- 172.21.15.38:0/2272706738 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f914804f070 con 0x55f605c0d390 2024-05-21T15:40:48.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.818+0000 7f915cbc4740 10 client.26759 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:48.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.818+0000 7f915cbc4740 10 client.26759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:48.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.818+0000 7f915cbc4740 20 client.26759 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:48.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.818+0000 7f915cbc4740 1 --2- 172.21.15.38:0/2272706738 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f605bcbf60 0x55f605fc8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.818+0000 7f915cbc4740 4 client.26759 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.818+0000 7f915cbc4740 1 -- 172.21.15.38:0/2272706738 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f605f1f430 con 0x55f605bcbf60 2024-05-21T15:40:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.819+0000 7f914f7fe640 1 --2- 172.21.15.38:0/2272706738 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f605bcbf60 0x55f605fc8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.819+0000 7f914f7fe640 1 --2- 172.21.15.38:0/2272706738 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f605bcbf60 0x55f605fc8800 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=696fdf52b99f79be server_cookie=d53fb0cebbe67411 in_seq=0 out_seq=0 2024-05-21T15:40:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.819+0000 7f912effd640 10 client.26759 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.820+0000 7f912effd640 1 -- 172.21.15.38:0/2272706738 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f605f1f430 con 0x55f605bcbf60 2024-05-21T15:40:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.820+0000 7f912effd640 10 client.26759 handle_command_reply: tid=0 2024-05-21T15:40:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.820+0000 7f915cbc4740 1 client.26759 shutdown 2024-05-21T15:40:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.821+0000 7f915cbc4740 1 -- 172.21.15.38:0/2272706738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605c0d390 msgr2=0x55f605fc2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.821+0000 7f915cbc4740 1 --2- 172.21.15.38:0/2272706738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605c0d390 0x55f605fc2240 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9148009ca0 tx=0x7f9148031700 comp rx=0 tx=0).stop 2024-05-21T15:40:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.821+0000 7f915cbc4740 1 -- 172.21.15.38:0/2272706738 shutdown_connections 2024-05-21T15:40:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.821+0000 7f915cbc4740 1 --2- 172.21.15.38:0/2272706738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605c0d390 0x55f605fc2240 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.821+0000 7f915cbc4740 1 --2- 172.21.15.38:0/2272706738 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f605bcbf60 0x55f605fc8800 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.821+0000 7f912effd640 0 client.26759 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:48.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.821+0000 7f915cbc4740 1 --2- 172.21.15.38:0/2272706738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bec6b0 0x55f605fbb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.821+0000 7f915cbc4740 1 --2- 172.21.15.38:0/2272706738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605be44d0 0x55f605fbada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.821+0000 7f915cbc4740 1 -- 172.21.15.38:0/2272706738 >> 172.21.15.38:0/2272706738 conn(0x55f605c014e0 msgr2=0x55f605fa75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:48.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.821+0000 7f915cbc4740 1 -- 172.21.15.38:0/2272706738 shutdown_connections 2024-05-21T15:40:48.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.821+0000 7f915cbc4740 1 -- 172.21.15.38:0/2272706738 wait complete. 2024-05-21T15:40:48.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.821+0000 7f915cbc4740 20 client.26759 trim_cache size 0 max 16384 2024-05-21T15:40:48.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.848+0000 7f915cbc4740 1 Processor -- start 2024-05-21T15:40:48.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.848+0000 7f915cbc4740 1 -- start start 2024-05-21T15:40:48.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.848+0000 7f915cbc4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bcbf60 0x55f605fba3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.848+0000 7f915cbc4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605be44d0 0x55f605fc2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.848+0000 7f915cbc4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bec6b0 0x55f605fbb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.848+0000 7f915cbc4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f91501a8100 con 0x55f605bec6b0 2024-05-21T15:40:48.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.848+0000 7f915cbc4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91501a9cd0 con 0x55f605be44d0 2024-05-21T15:40:48.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.848+0000 7f915cbc4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f91501a87d0 con 0x55f605bcbf60 2024-05-21T15:40:48.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.848+0000 7f914ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bec6b0 0x55f605fbb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.848+0000 7f914ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bec6b0 0x55f605fbb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33508/0 (socket says 172.21.15.38:33508) 2024-05-21T15:40:48.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.848+0000 7f914effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605be44d0 0x55f605fc2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.848+0000 7f914ffff640 1 -- 172.21.15.38:0/3270428647 learned_addr learned my addr 172.21.15.38:0/3270428647 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:48.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.848+0000 7f914f7fe640 1 --2- 172.21.15.38:0/3270428647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bcbf60 0x55f605fba3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.849+0000 7f914ffff640 1 -- 172.21.15.38:0/3270428647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bcbf60 msgr2=0x55f605fba3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:48.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.849+0000 7f914ffff640 1 --2- 172.21.15.38:0/3270428647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bcbf60 0x55f605fba3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.849+0000 7f914ffff640 1 -- 172.21.15.38:0/3270428647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605be44d0 msgr2=0x55f605fc2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:48.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.849+0000 7f914ffff640 1 --2- 172.21.15.38:0/3270428647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605be44d0 0x55f605fc2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.849+0000 7f914ffff640 1 -- 172.21.15.38:0/3270428647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f605fadd30 con 0x55f605bec6b0 2024-05-21T15:40:48.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.849+0000 7f914effd640 1 --2- 172.21.15.38:0/3270428647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605be44d0 0x55f605fc2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:48.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.849+0000 7f914ffff640 1 --2- 172.21.15.38:0/3270428647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bec6b0 0x55f605fbb410 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f914802ade0 tx=0x7f91480048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6753277ceb40135d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:48.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.849+0000 7f91357fa640 1 -- 172.21.15.38:0/3270428647 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9148004120 con 0x55f605bec6b0 2024-05-21T15:40:48.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.849+0000 7f91357fa640 1 -- 172.21.15.38:0/3270428647 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9148036070 con 0x55f605bec6b0 2024-05-21T15:40:48.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.849+0000 7f91357fa640 1 -- 172.21.15.38:0/3270428647 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914802fd50 con 0x55f605bec6b0 2024-05-21T15:40:48.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.850+0000 7f915cbc4740 1 -- 172.21.15.38:0/3270428647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bec6b0 msgr2=0x55f605fbb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:48.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.850+0000 7f915cbc4740 1 --2- 172.21.15.38:0/3270428647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bec6b0 0x55f605fbb410 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f914802ade0 tx=0x7f91480048e0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.850+0000 7f915cbc4740 1 -- 172.21.15.38:0/3270428647 shutdown_connections 2024-05-21T15:40:48.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.850+0000 7f915cbc4740 1 --2- 172.21.15.38:0/3270428647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bec6b0 0x55f605fbb410 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.850+0000 7f915cbc4740 1 --2- 172.21.15.38:0/3270428647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605be44d0 0x55f605fc2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.850+0000 7f915cbc4740 1 --2- 172.21.15.38:0/3270428647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bcbf60 0x55f605fba3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.850+0000 7f915cbc4740 1 -- 172.21.15.38:0/3270428647 >> 172.21.15.38:0/3270428647 conn(0x55f605c014e0 msgr2=0x55f605fabd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:48.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.850+0000 7f915cbc4740 1 -- 172.21.15.38:0/3270428647 shutdown_connections 2024-05-21T15:40:48.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.850+0000 7f915cbc4740 1 -- 172.21.15.38:0/3270428647 wait complete. 2024-05-21T15:40:48.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.850+0000 7f915cbc4740 1 -- start start 2024-05-21T15:40:48.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.851+0000 7f915cbc4740 1 Processor -- start 2024-05-21T15:40:48.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.851+0000 7f915cbc4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bcbf60 0x55f605fc11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.851+0000 7f915cbc4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605be44d0 0x55f605fc16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.851+0000 7f915cbc4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bec6b0 0x55f60602eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.851+0000 7f915cbc4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f91501a95a0 con 0x55f605bcbf60 2024-05-21T15:40:48.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.851+0000 7f915cbc4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91501a8ea0 con 0x55f605be44d0 2024-05-21T15:40:48.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.851+0000 7f915cbc4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f91501a60e0 con 0x55f605bec6b0 2024-05-21T15:40:48.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.851+0000 7f915cbc4740 10 client.0 _ll_register_callbacks cb 0x7f9140009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:48.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.851+0000 7f914f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bcbf60 0x55f605fc11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.851+0000 7f914f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bcbf60 0x55f605fc11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33524/0 (socket says 172.21.15.38:33524) 2024-05-21T15:40:48.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.851+0000 7f914f7fe640 1 -- 172.21.15.38:0/998660907 learned_addr learned my addr 172.21.15.38:0/998660907 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:48.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.851+0000 7f914effd640 1 --2- 172.21.15.38:0/998660907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605be44d0 0x55f605fc16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.851+0000 7f914ffff640 1 --2- 172.21.15.38:0/998660907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bec6b0 0x55f60602eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.852+0000 7f914f7fe640 1 -- 172.21.15.38:0/998660907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bec6b0 msgr2=0x55f60602eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:48.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.852+0000 7f914f7fe640 1 --2- 172.21.15.38:0/998660907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bec6b0 0x55f60602eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.852+0000 7f914f7fe640 1 -- 172.21.15.38:0/998660907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605be44d0 msgr2=0x55f605fc16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:48.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.852+0000 7f914f7fe640 1 --2- 172.21.15.38:0/998660907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605be44d0 0x55f605fc16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.852+0000 7f914f7fe640 1 -- 172.21.15.38:0/998660907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f60602f0e0 con 0x55f605bcbf60 2024-05-21T15:40:48.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.852+0000 7f914f7fe640 1 --2- 172.21.15.38:0/998660907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bcbf60 0x55f605fc11a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9140009fb0 tx=0x7f91400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f596899585ef13c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:48.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.852+0000 7f912effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:48.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.852+0000 7f912effd640 1 -- 172.21.15.38:0/998660907 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140003770 con 0x55f605bcbf60 2024-05-21T15:40:48.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.852+0000 7f912effd640 1 -- 172.21.15.38:0/998660907 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9140003910 con 0x55f605bcbf60 2024-05-21T15:40:48.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.852+0000 7f915cbc4740 1 -- 172.21.15.38:0/998660907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9148036070 con 0x55f605bcbf60 2024-05-21T15:40:48.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.852+0000 7f912effd640 1 -- 172.21.15.38:0/998660907 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9140003b10 con 0x55f605bcbf60 2024-05-21T15:40:48.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.852+0000 7f915cbc4740 10 client.17136 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:48.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.852+0000 7f915cbc4740 1 -- 172.21.15.38:0/998660907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f606030410 con 0x55f605bcbf60 2024-05-21T15:40:48.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.853+0000 7f912effd640 1 -- 172.21.15.38:0/998660907 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9148036070 con 0x55f605bcbf60 2024-05-21T15:40:48.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.853+0000 7f912effd640 1 -- 172.21.15.38:0/998660907 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f914003e400 con 0x55f605bcbf60 2024-05-21T15:40:48.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.853+0000 7f915cbc4740 10 client.17136 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:48.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.853+0000 7f915cbc4740 10 client.17136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:48.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.853+0000 7f915cbc4740 20 client.17136 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:48.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.853+0000 7f915cbc4740 1 --2- 172.21.15.38:0/998660907 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f605c0d390 0x55f606032e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:48.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.853+0000 7f915cbc4740 4 client.17136 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:40:48.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.853+0000 7f915cbc4740 1 -- 172.21.15.38:0/998660907 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f605f19780 con 0x55f605c0d390 2024-05-21T15:40:48.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.853+0000 7f914effd640 1 --2- 172.21.15.38:0/998660907 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f605c0d390 0x55f606032e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:48.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f914effd640 1 --2- 172.21.15.38:0/998660907 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f605c0d390 0x55f606032e70 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c466d35567c2e5 server_cookie=9679195576fa791b in_seq=0 out_seq=0 2024-05-21T15:40:48.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f912effd640 10 client.17136 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:48.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f912effd640 1 -- 172.21.15.38:0/998660907 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f605f19780 con 0x55f605c0d390 2024-05-21T15:40:48.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f912effd640 10 client.17136 handle_command_reply: tid=0 2024-05-21T15:40:48.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f915cbc4740 1 client.17136 shutdown 2024-05-21T15:40:48.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f915cbc4740 1 -- 172.21.15.38:0/998660907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bcbf60 msgr2=0x55f605fc11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:48.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f915cbc4740 1 --2- 172.21.15.38:0/998660907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bcbf60 0x55f605fc11a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9140009fb0 tx=0x7f91400046c0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f915cbc4740 1 -- 172.21.15.38:0/998660907 shutdown_connections 2024-05-21T15:40:48.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f915cbc4740 1 --2- 172.21.15.38:0/998660907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f605bec6b0 0x55f60602eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f915cbc4740 1 --2- 172.21.15.38:0/998660907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f605be44d0 0x55f605fc16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f915cbc4740 1 --2- 172.21.15.38:0/998660907 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55f605c0d390 0x55f606032e70 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f915cbc4740 1 --2- 172.21.15.38:0/998660907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f605bcbf60 0x55f605fc11a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f912effd640 0 client.17136 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:48.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f915cbc4740 1 -- 172.21.15.38:0/998660907 >> 172.21.15.38:0/998660907 conn(0x55f605c014e0 msgr2=0x55f605fabad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:48.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f915cbc4740 1 -- 172.21.15.38:0/998660907 shutdown_connections 2024-05-21T15:40:48.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f915cbc4740 1 -- 172.21.15.38:0/998660907 wait complete. 2024-05-21T15:40:48.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.854+0000 7f915cbc4740 20 client.17136 trim_cache size 0 max 16384 2024-05-21T15:40:48.858 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:40:48.858 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-05-21T15:40:48.858 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-05-21T15:40:48.858 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:40:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.855+0000 7f9155d1a640 1 -- 172.21.15.38:0/356734975 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f91380a2b90 msgr2=0x7f91380a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.855+0000 7f9155d1a640 1 --2- 172.21.15.38:0/356734975 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f91380a2b90 0x7f91380a50a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55f605fadf90 tx=0x7f9140005f90 comp rx=0 tx=0).stop 2024-05-21T15:40:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.855+0000 7f9155d1a640 1 -- 172.21.15.38:0/356734975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9150102c10 msgr2=0x7f9150199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.855+0000 7f9155d1a640 1 --2- 172.21.15.38:0/356734975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9150102c10 0x7f9150199ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f914400e9f0 tx=0x7f914400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.855+0000 7f9155d1a640 1 -- 172.21.15.38:0/356734975 shutdown_connections 2024-05-21T15:40:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.855+0000 7f9155d1a640 1 --2- 172.21.15.38:0/356734975 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f91380a2b90 0x7f91380a50a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.855+0000 7f9155d1a640 1 --2- 172.21.15.38:0/356734975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9150107910 0x7f91501a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.855+0000 7f9155d1a640 1 --2- 172.21.15.38:0/356734975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9150102c10 0x7f9150199ab0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.855+0000 7f9155d1a640 1 --2- 172.21.15.38:0/356734975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91501019d0 0x7f9150199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.855+0000 7f9155d1a640 1 -- 172.21.15.38:0/356734975 >> 172.21.15.38:0/356734975 conn(0x7f91500fcbc0 msgr2=0x7f9150108de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:48.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.855+0000 7f9155d1a640 1 -- 172.21.15.38:0/356734975 shutdown_connections 2024-05-21T15:40:48.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:48.855+0000 7f9155d1a640 1 -- 172.21.15.38:0/356734975 wait complete. 2024-05-21T15:40:48.866 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:40:48.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:40:49.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.017+0000 7fe9e44a6640 1 -- 172.21.15.38:0/191649578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9dc108750 msgr2=0x7fe9dc10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.017+0000 7fe9e44a6640 1 --2- 172.21.15.38:0/191649578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9dc108750 0x7fe9dc10ac60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009ca0 tx=0x7fe9d402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.017+0000 7fe9e44a6640 1 -- 172.21.15.38:0/191649578 shutdown_connections 2024-05-21T15:40:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.017+0000 7fe9e44a6640 1 --2- 172.21.15.38:0/191649578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9dc108750 0x7fe9dc10ac60 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.017+0000 7fe9e44a6640 1 --2- 172.21.15.38:0/191649578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc105be0 0x7fe9dc1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.017+0000 7fe9e44a6640 1 --2- 172.21.15.38:0/191649578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9dc100e50 0x7fe9dc101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.017+0000 7fe9e44a6640 1 -- 172.21.15.38:0/191649578 >> 172.21.15.38:0/191649578 conn(0x7fe9dc0fc340 msgr2=0x7fe9dc0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.017+0000 7fe9e44a6640 1 -- 172.21.15.38:0/191649578 shutdown_connections 2024-05-21T15:40:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.017+0000 7fe9e44a6640 1 -- 172.21.15.38:0/191649578 wait complete. 2024-05-21T15:40:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.018+0000 7fe9e44a6640 1 Processor -- start 2024-05-21T15:40:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.018+0000 7fe9e44a6640 1 -- start start 2024-05-21T15:40:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.018+0000 7fe9e44a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9dc100e50 0x7fe9dc199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e44a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc105be0 0x7fe9dc199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e44a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9dc108750 0x7fe9dc1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e44a6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc10e730 con 0x7fe9dc100e50 2024-05-21T15:40:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e44a6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc10e5b0 con 0x7fe9dc105be0 2024-05-21T15:40:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e44a6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc10e8b0 con 0x7fe9dc108750 2024-05-21T15:40:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e2242640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9dc100e50 0x7fe9dc199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e2242640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9dc100e50 0x7fe9dc199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33550/0 (socket says 172.21.15.38:33550) 2024-05-21T15:40:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e2242640 1 -- 172.21.15.38:0/1335781018 learned_addr learned my addr 172.21.15.38:0/1335781018 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e2a43640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9dc108750 0x7fe9dc1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e1a41640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc105be0 0x7fe9dc199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e2242640 1 -- 172.21.15.38:0/1335781018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9dc108750 msgr2=0x7fe9dc1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e2242640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9dc108750 0x7fe9dc1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e2242640 1 -- 172.21.15.38:0/1335781018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc105be0 msgr2=0x7fe9dc199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e2242640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc105be0 0x7fe9dc199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e2242640 1 -- 172.21.15.38:0/1335781018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9dc1a0d60 con 0x7fe9dc100e50 2024-05-21T15:40:49.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.019+0000 7fe9e1a41640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc105be0 0x7fe9dc199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:40:49.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.020+0000 7fe9e2242640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9dc100e50 0x7fe9dc199220 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800e9f0 tx=0x7fe9d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.020+0000 7fe9d37fe640 1 -- 172.21.15.38:0/1335781018 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d800ccc0 con 0x7fe9dc100e50 2024-05-21T15:40:49.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.020+0000 7fe9d37fe640 1 -- 172.21.15.38:0/1335781018 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9d80035f0 con 0x7fe9dc100e50 2024-05-21T15:40:49.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.020+0000 7fe9e44a6640 1 -- 172.21.15.38:0/1335781018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9dc1a0ff0 con 0x7fe9dc100e50 2024-05-21T15:40:49.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.020+0000 7fe9e44a6640 1 -- 172.21.15.38:0/1335781018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9dc1a1460 con 0x7fe9dc100e50 2024-05-21T15:40:49.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.021+0000 7fe9d37fe640 1 -- 172.21.15.38:0/1335781018 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d800ce60 con 0x7fe9dc100e50 2024-05-21T15:40:49.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.021+0000 7fe9e44a6640 1 -- 172.21.15.38:0/1335781018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9dc068340 con 0x7fe9dc100e50 2024-05-21T15:40:49.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.023+0000 7fe9d37fe640 1 -- 172.21.15.38:0/1335781018 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe9d80037a0 con 0x7fe9dc100e50 2024-05-21T15:40:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.024+0000 7fe9d37fe640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe9c80a2c30 0x7fe9c80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.024+0000 7fe9d37fe640 1 -- 172.21.15.38:0/1335781018 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe9d80cf630 con 0x7fe9dc100e50 2024-05-21T15:40:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.025+0000 7fe9e1a41640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe9c80a2c30 0x7fe9c80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.025+0000 7fe9e1a41640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe9c80a2c30 0x7fe9c80a5140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc19a2c0 tx=0x7fe9cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.031+0000 7fe9d37fe640 1 -- 172.21.15.38:0/1335781018 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe9d809c540 con 0x7fe9dc100e50 2024-05-21T15:40:49.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.302+0000 7fe9e44a6640 1 -- 172.21.15.38:0/1335781018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9dc0fde60 con 0x7fe9dc100e50 2024-05-21T15:40:49.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.303+0000 7fe9d37fe640 1 -- 172.21.15.38:0/1335781018 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11918 (secure 0 0 0) 0x7fe9d809be40 con 0x7fe9dc100e50 2024-05-21T15:40:49.302 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:40:49.302 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:40:44.080316+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24515,24550,24602,24551,24523]},"id":1}]} 2024-05-21T15:40:49.302 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 32 2024-05-21T15:40:49.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.307+0000 7fe9e44a6640 1 -- 172.21.15.38:0/1335781018 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe9c80a2c30 msgr2=0x7fe9c80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.307+0000 7fe9e44a6640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe9c80a2c30 0x7fe9c80a5140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc19a2c0 tx=0x7fe9cc005f90 comp rx=0 tx=0).stop 2024-05-21T15:40:49.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.307+0000 7fe9e44a6640 1 -- 172.21.15.38:0/1335781018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9dc100e50 msgr2=0x7fe9dc199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.307+0000 7fe9e44a6640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9dc100e50 0x7fe9dc199220 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800e9f0 tx=0x7fe9d800eeb0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.307+0000 7fe9e44a6640 1 -- 172.21.15.38:0/1335781018 shutdown_connections 2024-05-21T15:40:49.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.307+0000 7fe9e44a6640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9dc108750 0x7fe9dc1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.307+0000 7fe9e44a6640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc105be0 0x7fe9dc199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.307+0000 7fe9e44a6640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe9c80a2c30 0x7fe9c80a5140 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.307+0000 7fe9e44a6640 1 --2- 172.21.15.38:0/1335781018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9dc100e50 0x7fe9dc199220 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.307+0000 7fe9e44a6640 1 -- 172.21.15.38:0/1335781018 >> 172.21.15.38:0/1335781018 conn(0x7fe9dc0fc340 msgr2=0x7fe9dc106b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.308+0000 7fe9e44a6640 1 -- 172.21.15.38:0/1335781018 shutdown_connections 2024-05-21T15:40:49.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.308+0000 7fe9e44a6640 1 -- 172.21.15.38:0/1335781018 wait complete. 2024-05-21T15:40:49.322 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:40:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.473+0000 7f231f0f7640 1 -- 172.21.15.38:0/360078035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23181089d0 msgr2=0x7f231810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.473+0000 7f231f0f7640 1 --2- 172.21.15.38:0/360078035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23181089d0 0x7f231810adf0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2310009ca0 tx=0x7f231002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.473+0000 7f231f0f7640 1 -- 172.21.15.38:0/360078035 shutdown_connections 2024-05-21T15:40:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.473+0000 7f231f0f7640 1 --2- 172.21.15.38:0/360078035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23181089d0 0x7f231810adf0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.473+0000 7f231f0f7640 1 --2- 172.21.15.38:0/360078035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2318103cd0 0x7f23181040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.473+0000 7f231f0f7640 1 --2- 172.21.15.38:0/360078035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2318102a90 0x7f2318103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.473+0000 7f231f0f7640 1 -- 172.21.15.38:0/360078035 >> 172.21.15.38:0/360078035 conn(0x7f23180fdc80 msgr2=0x7f2318100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.473+0000 7f231f0f7640 1 -- 172.21.15.38:0/360078035 shutdown_connections 2024-05-21T15:40:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.473+0000 7f231f0f7640 1 -- 172.21.15.38:0/360078035 wait complete. 2024-05-21T15:40:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.474+0000 7f231f0f7640 1 Processor -- start 2024-05-21T15:40:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.474+0000 7f231f0f7640 1 -- start start 2024-05-21T15:40:49.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.474+0000 7f231f0f7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2318102a90 0x7f2318199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.474+0000 7f231f0f7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2318103cd0 0x7f2318199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.474+0000 7f231f0f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23181089d0 0x7f23181a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.474+0000 7f231f0f7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f231810e760 con 0x7f23181089d0 2024-05-21T15:40:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.474+0000 7f231f0f7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f231810e5e0 con 0x7f2318103cd0 2024-05-21T15:40:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.474+0000 7f231f0f7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f231810e8e0 con 0x7f2318102a90 2024-05-21T15:40:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.474+0000 7f231d694640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23181089d0 0x7f23181a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.474+0000 7f231d694640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23181089d0 0x7f23181a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33562/0 (socket says 172.21.15.38:33562) 2024-05-21T15:40:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.474+0000 7f231ce93640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2318102a90 0x7f2318199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.474+0000 7f231d694640 1 -- 172.21.15.38:0/72538338 learned_addr learned my addr 172.21.15.38:0/72538338 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.474+0000 7f2317fff640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2318103cd0 0x7f2318199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.475+0000 7f2317fff640 1 -- 172.21.15.38:0/72538338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2318102a90 msgr2=0x7f2318199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.475+0000 7f2317fff640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2318102a90 0x7f2318199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.475+0000 7f2317fff640 1 -- 172.21.15.38:0/72538338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23181089d0 msgr2=0x7f23181a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.475+0000 7f2317fff640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23181089d0 0x7f23181a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.475+0000 7f2317fff640 1 -- 172.21.15.38:0/72538338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23181a0d80 con 0x7f2318103cd0 2024-05-21T15:40:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.475+0000 7f231ce93640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2318102a90 0x7f2318199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.475+0000 7f231d694640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23181089d0 0x7f23181a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.475+0000 7f2317fff640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2318103cd0 0x7f2318199780 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f230400e9f0 tx=0x7f230400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.475+0000 7f2315ffb640 1 -- 172.21.15.38:0/72538338 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230400ccc0 con 0x7f2318103cd0 2024-05-21T15:40:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.475+0000 7f2315ffb640 1 -- 172.21.15.38:0/72538338 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2304010910 con 0x7f2318103cd0 2024-05-21T15:40:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.475+0000 7f231f0f7640 1 -- 172.21.15.38:0/72538338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23181a1070 con 0x7f2318103cd0 2024-05-21T15:40:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.475+0000 7f2315ffb640 1 -- 172.21.15.38:0/72538338 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304018960 con 0x7f2318103cd0 2024-05-21T15:40:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.475+0000 7f231f0f7640 1 -- 172.21.15.38:0/72538338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23181a15c0 con 0x7f2318103cd0 2024-05-21T15:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.478+0000 7f2315ffb640 1 -- 172.21.15.38:0/72538338 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2304018b00 con 0x7f2318103cd0 2024-05-21T15:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.478+0000 7f2325f9d740 1 Processor -- start 2024-05-21T15:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2325f9d740 1 -- start start 2024-05-21T15:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2315ffb640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f23080a2b90 0x7f23080a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2325f9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf7860390 0x55caf7c03f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2325f9d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c04490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2325f9d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf783f6b0 0x55caf7c0b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2325f9d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55caf7b64c40 con 0x55caf783f6b0 2024-05-21T15:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2325f9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55caf7b0c580 con 0x55caf7860390 2024-05-21T15:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2325f9d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55caf7b60c00 con 0x55caf78374d0 2024-05-21T15:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2315ffb640 1 -- 172.21.15.38:0/72538338 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f23040cf450 con 0x7f2318103cd0 2024-05-21T15:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f231d694640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f23080a2b90 0x7f23080a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f231d694640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf783f6b0 0x55caf7c0b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f231d694640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf783f6b0 0x55caf7c0b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33578/0 (socket says 172.21.15.38:33578) 2024-05-21T15:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f231d694640 1 -- 172.21.15.38:0/2047210848 learned_addr learned my addr 172.21.15.38:0/2047210848 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2317fff640 1 --2- 172.21.15.38:0/2047210848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c04490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f231ce93640 1 --2- 172.21.15.38:0/2047210848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf7860390 0x55caf7c03f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2317fff640 1 -- 172.21.15.38:0/2047210848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf7860390 msgr2=0x55caf7c03f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2317fff640 1 --2- 172.21.15.38:0/2047210848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf7860390 0x55caf7c03f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2317fff640 1 -- 172.21.15.38:0/2047210848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf783f6b0 msgr2=0x55caf7c0b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2317fff640 1 --2- 172.21.15.38:0/2047210848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf783f6b0 0x55caf7c0b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.479+0000 7f2317fff640 1 -- 172.21.15.38:0/2047210848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf78783f0 con 0x55caf78374d0 2024-05-21T15:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.480+0000 7f231d694640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f23080a2b90 0x7f23080a50a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2310031860 tx=0x7f231002f500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.480+0000 7f2317fff640 1 --2- 172.21.15.38:0/2047210848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c04490 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f230400c6a0 tx=0x7f230400cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23288f97eb38887 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.480+0000 7f2302ffd640 1 -- 172.21.15.38:0/2047210848 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23040b4e60 con 0x55caf78374d0 2024-05-21T15:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.480+0000 7f2302ffd640 1 -- 172.21.15.38:0/2047210848 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23040b1070 con 0x55caf78374d0 2024-05-21T15:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.480+0000 7f2302ffd640 1 -- 172.21.15.38:0/2047210848 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304098510 con 0x55caf78374d0 2024-05-21T15:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.481+0000 7f2325f9d740 1 -- 172.21.15.38:0/2047210848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 msgr2=0x55caf7c04490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.481+0000 7f2325f9d740 1 --2- 172.21.15.38:0/2047210848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c04490 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f230400c6a0 tx=0x7f230400cf60 comp rx=0 tx=0).stop 2024-05-21T15:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.481+0000 7f2325f9d740 1 -- 172.21.15.38:0/2047210848 shutdown_connections 2024-05-21T15:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.481+0000 7f2325f9d740 1 --2- 172.21.15.38:0/2047210848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf783f6b0 0x55caf7c0b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.481+0000 7f2325f9d740 1 --2- 172.21.15.38:0/2047210848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c04490 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.481+0000 7f2325f9d740 1 --2- 172.21.15.38:0/2047210848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf7860390 0x55caf7c03f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.481+0000 7f2325f9d740 1 -- 172.21.15.38:0/2047210848 >> 172.21.15.38:0/2047210848 conn(0x55caf78544e0 msgr2=0x55caf7bfb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.481+0000 7f2325f9d740 1 -- 172.21.15.38:0/2047210848 shutdown_connections 2024-05-21T15:40:49.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.481+0000 7f2325f9d740 1 -- 172.21.15.38:0/2047210848 wait complete. 2024-05-21T15:40:49.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.481+0000 7f2325f9d740 1 -- start start 2024-05-21T15:40:49.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.481+0000 7f2325f9d740 1 Processor -- start 2024-05-21T15:40:49.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f2325f9d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf78374d0 0x55caf7c0ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f2325f9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf783f6b0 0x55caf7c0e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f2325f9d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf7860390 0x55caf7c15220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f2325f9d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55caf7c0bd80 con 0x55caf78374d0 2024-05-21T15:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f2325f9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55caf7b1fc40 con 0x55caf783f6b0 2024-05-21T15:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f2325f9d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55caf7c0bf00 con 0x55caf7860390 2024-05-21T15:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f2325f9d740 10 client.0 _ll_register_callbacks cb 0x55caf7877be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f231ce93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf78374d0 0x55caf7c0ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f231ce93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf78374d0 0x55caf7c0ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33594/0 (socket says 172.21.15.38:33594) 2024-05-21T15:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f231d694640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf7860390 0x55caf7c15220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f231ce93640 1 -- 172.21.15.38:0/198746719 learned_addr learned my addr 172.21.15.38:0/198746719 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f2317fff640 1 --2- 172.21.15.38:0/198746719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf783f6b0 0x55caf7c0e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f231ce93640 1 -- 172.21.15.38:0/198746719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf7860390 msgr2=0x55caf7c15220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f231ce93640 1 --2- 172.21.15.38:0/198746719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf7860390 0x55caf7c15220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f231ce93640 1 -- 172.21.15.38:0/198746719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf783f6b0 msgr2=0x55caf7c0e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f231ce93640 1 --2- 172.21.15.38:0/198746719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf783f6b0 0x55caf7c0e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f231ce93640 1 -- 172.21.15.38:0/198746719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf79427d0 con 0x55caf78374d0 2024-05-21T15:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f231ce93640 1 --2- 172.21.15.38:0/198746719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf78374d0 0x55caf7c0ddf0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f230c012d20 tx=0x7f230c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24b89a0b146b490 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.482+0000 7f22f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f22f7fff640 1 -- 172.21.15.38:0/198746719 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c012110 con 0x55caf78374d0 2024-05-21T15:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f22f7fff640 1 -- 172.21.15.38:0/198746719 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f230c004050 con 0x55caf78374d0 2024-05-21T15:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f2325f9d740 1 -- 172.21.15.38:0/198746719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23040b1070 con 0x55caf78374d0 2024-05-21T15:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f22f7fff640 1 -- 172.21.15.38:0/198746719 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f230c0122b0 con 0x55caf78374d0 2024-05-21T15:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f2325f9d740 10 client.17154 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f2325f9d740 1 -- 172.21.15.38:0/198746719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55caf7c185b0 con 0x55caf78374d0 2024-05-21T15:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f22f7fff640 1 -- 172.21.15.38:0/198746719 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23040b1070 con 0x55caf78374d0 2024-05-21T15:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f22f7fff640 1 -- 172.21.15.38:0/198746719 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f230c010430 con 0x55caf78374d0 2024-05-21T15:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f2325f9d740 10 client.17154 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f2325f9d740 10 client.17154 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f2325f9d740 20 client.17154 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:49.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f2325f9d740 1 --2- 172.21.15.38:0/198746719 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55caf781ef60 0x55caf7c1bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f2325f9d740 4 client.17154 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f2325f9d740 1 -- 172.21.15.38:0/198746719 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caf7b72370 con 0x55caf781ef60 2024-05-21T15:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.483+0000 7f2317fff640 1 --2- 172.21.15.38:0/198746719 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55caf781ef60 0x55caf7c1bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.484+0000 7f22f7fff640 10 client.17154 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.484+0000 7f2317fff640 1 --2- 172.21.15.38:0/198746719 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55caf781ef60 0x55caf7c1bc90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8e7713621d591e01 server_cookie=37d0630744ea3338 in_seq=0 out_seq=0 2024-05-21T15:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f22f7fff640 1 -- 172.21.15.38:0/198746719 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55caf7b72370 con 0x55caf781ef60 2024-05-21T15:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f22f7fff640 10 client.17154 handle_command_reply: tid=0 2024-05-21T15:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f2325f9d740 1 client.17154 shutdown 2024-05-21T15:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f2325f9d740 1 -- 172.21.15.38:0/198746719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf78374d0 msgr2=0x55caf7c0ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f2325f9d740 1 --2- 172.21.15.38:0/198746719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf78374d0 0x55caf7c0ddf0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f230c012d20 tx=0x7f230c00f790 comp rx=0 tx=0).stop 2024-05-21T15:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f2325f9d740 1 -- 172.21.15.38:0/198746719 shutdown_connections 2024-05-21T15:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f2325f9d740 1 --2- 172.21.15.38:0/198746719 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55caf781ef60 0x55caf7c1bc90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f2325f9d740 1 --2- 172.21.15.38:0/198746719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf7860390 0x55caf7c15220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f2325f9d740 1 --2- 172.21.15.38:0/198746719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf783f6b0 0x55caf7c0e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f22f7fff640 0 client.17154 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:40:49.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f2325f9d740 1 --2- 172.21.15.38:0/198746719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf78374d0 0x55caf7c0ddf0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f2325f9d740 1 -- 172.21.15.38:0/198746719 >> 172.21.15.38:0/198746719 conn(0x55caf78544e0 msgr2=0x55caf7bfa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f2325f9d740 1 -- 172.21.15.38:0/198746719 shutdown_connections 2024-05-21T15:40:49.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f2325f9d740 1 -- 172.21.15.38:0/198746719 wait complete. 2024-05-21T15:40:49.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.485+0000 7f2325f9d740 20 client.17154 trim_cache size 0 max 16384 2024-05-21T15:40:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.499+0000 7f2325f9d740 1 Processor -- start 2024-05-21T15:40:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.499+0000 7f2325f9d740 1 -- start start 2024-05-21T15:40:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.499+0000 7f2325f9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf781ef60 0x55caf7bfb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.499+0000 7f2325f9d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c15140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.499+0000 7f2325f9d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf783f6b0 0x55caf7c0d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.499+0000 7f2325f9d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f23181a8240 con 0x55caf783f6b0 2024-05-21T15:40:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.499+0000 7f2325f9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f23181a9e10 con 0x55caf781ef60 2024-05-21T15:40:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.499+0000 7f2325f9d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f23181a8910 con 0x55caf78374d0 2024-05-21T15:40:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.499+0000 7f231d694640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf783f6b0 0x55caf7c0d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.499+0000 7f231d694640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf783f6b0 0x55caf7c0d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33608/0 (socket says 172.21.15.38:33608) 2024-05-21T15:40:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.499+0000 7f231d694640 1 -- 172.21.15.38:0/2387075820 learned_addr learned my addr 172.21.15.38:0/2387075820 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.499+0000 7f231ce93640 1 --2- 172.21.15.38:0/2387075820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf781ef60 0x55caf7bfb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.499+0000 7f2317fff640 1 --2- 172.21.15.38:0/2387075820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c15140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.500+0000 7f231d694640 1 -- 172.21.15.38:0/2387075820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 msgr2=0x55caf7c15140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.500+0000 7f231d694640 1 --2- 172.21.15.38:0/2387075820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c15140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.500+0000 7f231d694640 1 -- 172.21.15.38:0/2387075820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf781ef60 msgr2=0x55caf7bfb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.500+0000 7f231d694640 1 --2- 172.21.15.38:0/2387075820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf781ef60 0x55caf7bfb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.500+0000 7f231d694640 1 -- 172.21.15.38:0/2387075820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf7c0e9f0 con 0x55caf783f6b0 2024-05-21T15:40:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.500+0000 7f231ce93640 1 --2- 172.21.15.38:0/2387075820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf781ef60 0x55caf7bfb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:49.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.500+0000 7f2317fff640 1 --2- 172.21.15.38:0/2387075820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c15140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:40:49.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.500+0000 7f231d694640 1 --2- 172.21.15.38:0/2387075820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf783f6b0 0x55caf7c0d9c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2310031560 tx=0x7f231003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1400d106370211 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.500+0000 7f23027fc640 1 -- 172.21.15.38:0/2387075820 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231002fc10 con 0x55caf783f6b0 2024-05-21T15:40:49.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.500+0000 7f23027fc640 1 -- 172.21.15.38:0/2387075820 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2310041040 con 0x55caf783f6b0 2024-05-21T15:40:49.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.500+0000 7f23027fc640 1 -- 172.21.15.38:0/2387075820 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231004c450 con 0x55caf783f6b0 2024-05-21T15:40:49.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.501+0000 7f2325f9d740 1 -- 172.21.15.38:0/2387075820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf783f6b0 msgr2=0x55caf7c0d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.501+0000 7f2325f9d740 1 --2- 172.21.15.38:0/2387075820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf783f6b0 0x55caf7c0d9c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2310031560 tx=0x7f231003b000 comp rx=0 tx=0).stop 2024-05-21T15:40:49.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.501+0000 7f2325f9d740 1 -- 172.21.15.38:0/2387075820 shutdown_connections 2024-05-21T15:40:49.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.501+0000 7f2325f9d740 1 --2- 172.21.15.38:0/2387075820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf783f6b0 0x55caf7c0d9c0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.501+0000 7f2325f9d740 1 --2- 172.21.15.38:0/2387075820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c15140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.501+0000 7f2325f9d740 1 --2- 172.21.15.38:0/2387075820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf781ef60 0x55caf7bfb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.501+0000 7f2325f9d740 1 -- 172.21.15.38:0/2387075820 >> 172.21.15.38:0/2387075820 conn(0x55caf78544e0 msgr2=0x55caf7bfaed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.501+0000 7f2325f9d740 1 -- 172.21.15.38:0/2387075820 shutdown_connections 2024-05-21T15:40:49.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.501+0000 7f2325f9d740 1 -- 172.21.15.38:0/2387075820 wait complete. 2024-05-21T15:40:49.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.501+0000 7f2325f9d740 1 -- start start 2024-05-21T15:40:49.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.501+0000 7f2325f9d740 1 Processor -- start 2024-05-21T15:40:49.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f2325f9d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf781ef60 0x55caf7c122f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f2325f9d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c12830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f2325f9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf783f6b0 0x55caf7c13760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f2325f9d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f23181a96e0 con 0x55caf781ef60 2024-05-21T15:40:49.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f2325f9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f23181a8fe0 con 0x55caf783f6b0 2024-05-21T15:40:49.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f2325f9d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f23181a6220 con 0x55caf78374d0 2024-05-21T15:40:49.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f2325f9d740 10 client.0 _ll_register_callbacks cb 0x7f230c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:49.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f231ce93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf781ef60 0x55caf7c122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f231ce93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf781ef60 0x55caf7c122f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33622/0 (socket says 172.21.15.38:33622) 2024-05-21T15:40:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f231ce93640 1 -- 172.21.15.38:0/1210885504 learned_addr learned my addr 172.21.15.38:0/1210885504 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f231d694640 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf783f6b0 0x55caf7c13760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f2317fff640 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c12830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f231ce93640 1 -- 172.21.15.38:0/1210885504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 msgr2=0x55caf7c12830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f231ce93640 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c12830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f231ce93640 1 -- 172.21.15.38:0/1210885504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf783f6b0 msgr2=0x55caf7c13760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f231ce93640 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf783f6b0 0x55caf7c13760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f231ce93640 1 -- 172.21.15.38:0/1210885504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf7c13d20 con 0x55caf781ef60 2024-05-21T15:40:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f2317fff640 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c12830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f231d694640 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf783f6b0 0x55caf7c13760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f231ce93640 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf781ef60 0x55caf7c122f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f230c010880 tx=0x7f230c0106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a1aec0112b2a94 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f22f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:49.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f22f7fff640 1 -- 172.21.15.38:0/1210885504 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c000e60 con 0x55caf781ef60 2024-05-21T15:40:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f22f7fff640 1 -- 172.21.15.38:0/1210885504 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f230c0050a0 con 0x55caf781ef60 2024-05-21T15:40:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.502+0000 7f2325f9d740 1 -- 172.21.15.38:0/1210885504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2310041040 con 0x55caf781ef60 2024-05-21T15:40:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.503+0000 7f22f7fff640 1 -- 172.21.15.38:0/1210885504 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f230c02c070 con 0x55caf781ef60 2024-05-21T15:40:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.503+0000 7f2325f9d740 10 client.17160 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.503+0000 7f2325f9d740 1 -- 172.21.15.38:0/1210885504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55caf7c15010 con 0x55caf781ef60 2024-05-21T15:40:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.503+0000 7f22f7fff640 1 -- 172.21.15.38:0/1210885504 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2310041040 con 0x55caf781ef60 2024-05-21T15:40:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.503+0000 7f22f7fff640 1 -- 172.21.15.38:0/1210885504 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f230c003c80 con 0x55caf781ef60 2024-05-21T15:40:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.503+0000 7f2325f9d740 10 client.17160 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.503+0000 7f2325f9d740 10 client.17160 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:40:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.503+0000 7f2325f9d740 20 client.17160 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.503+0000 7f2325f9d740 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55caf7860390 0x55caf7c863b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.503+0000 7f2325f9d740 4 client.17160 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:40:49.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.503+0000 7f2325f9d740 1 -- 172.21.15.38:0/1210885504 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55caf7b6c6c0 con 0x55caf7860390 2024-05-21T15:40:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.503+0000 7f2317fff640 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55caf7860390 0x55caf7c863b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f22f7fff640 10 client.17160 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:40:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f2317fff640 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55caf7860390 0x55caf7c863b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d563d649c58cdf1 server_cookie=b76d89c2132ae16b in_seq=0 out_seq=0 2024-05-21T15:40:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f22f7fff640 1 -- 172.21.15.38:0/1210885504 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55caf7b6c6c0 con 0x55caf7860390 2024-05-21T15:40:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f22f7fff640 10 client.17160 handle_command_reply: tid=0 2024-05-21T15:40:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f2325f9d740 1 client.17160 shutdown 2024-05-21T15:40:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f2325f9d740 1 -- 172.21.15.38:0/1210885504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf781ef60 msgr2=0x55caf7c122f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f2325f9d740 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf781ef60 0x55caf7c122f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f230c010880 tx=0x7f230c0106e0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f2325f9d740 1 -- 172.21.15.38:0/1210885504 shutdown_connections 2024-05-21T15:40:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f2325f9d740 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55caf7860390 0x55caf7c863b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f2325f9d740 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55caf783f6b0 0x55caf7c13760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f22f7fff640 0 client.17160 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:40:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f2325f9d740 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf78374d0 0x55caf7c12830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f2325f9d740 1 --2- 172.21.15.38:0/1210885504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55caf781ef60 0x55caf7c122f0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f2325f9d740 1 -- 172.21.15.38:0/1210885504 >> 172.21.15.38:0/1210885504 conn(0x55caf78544e0 msgr2=0x55caf7c0d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f2325f9d740 1 -- 172.21.15.38:0/1210885504 shutdown_connections 2024-05-21T15:40:49.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f2325f9d740 1 -- 172.21.15.38:0/1210885504 wait complete. 2024-05-21T15:40:49.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.504+0000 7f2325f9d740 20 client.17160 trim_cache size 0 max 16384 2024-05-21T15:40:49.509 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:40:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.505+0000 7f231f0f7640 1 -- 172.21.15.38:0/72538338 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f23080a2b90 msgr2=0x7f23080a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.505+0000 7f231f0f7640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f23080a2b90 0x7f23080a50a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2310031860 tx=0x7f231002f500 comp rx=0 tx=0).stop 2024-05-21T15:40:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.505+0000 7f231f0f7640 1 -- 172.21.15.38:0/72538338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2318103cd0 msgr2=0x7f2318199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.505+0000 7f231f0f7640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2318103cd0 0x7f2318199780 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f230400e9f0 tx=0x7f230400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.505+0000 7f231f0f7640 1 -- 172.21.15.38:0/72538338 shutdown_connections 2024-05-21T15:40:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.505+0000 7f231f0f7640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f23080a2b90 0x7f23080a50a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.505+0000 7f231f0f7640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23181089d0 0x7f23181a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.505+0000 7f231f0f7640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2318103cd0 0x7f2318199780 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.505+0000 7f231f0f7640 1 --2- 172.21.15.38:0/72538338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2318102a90 0x7f2318199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.505+0000 7f231f0f7640 1 -- 172.21.15.38:0/72538338 >> 172.21.15.38:0/72538338 conn(0x7f23180fdc80 msgr2=0x7f23181049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.505+0000 7f231f0f7640 1 -- 172.21.15.38:0/72538338 shutdown_connections 2024-05-21T15:40:49.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.505+0000 7f231f0f7640 1 -- 172.21.15.38:0/72538338 wait complete. 2024-05-21T15:40:49.515 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:40:49.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.669+0000 7f11dc1ce640 1 -- 172.21.15.38:0/2596854128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4103570 msgr2=0x7f11d4105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.669+0000 7f11dc1ce640 1 --2- 172.21.15.38:0/2596854128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4103570 0x7f11d4105990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009900 tx=0x7f11d002a410 comp rx=0 tx=0).stop 2024-05-21T15:40:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.669+0000 7f11dc1ce640 1 -- 172.21.15.38:0/2596854128 shutdown_connections 2024-05-21T15:40:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.669+0000 7f11dc1ce640 1 --2- 172.21.15.38:0/2596854128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11d4108830 0x7f11d410ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.669+0000 7f11dc1ce640 1 --2- 172.21.15.38:0/2596854128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11d4105ed0 0x7f11d41082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.669+0000 7f11dc1ce640 1 --2- 172.21.15.38:0/2596854128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4103570 0x7f11d4105990 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.669+0000 7f11dc1ce640 1 -- 172.21.15.38:0/2596854128 >> 172.21.15.38:0/2596854128 conn(0x7f11d40fcb70 msgr2=0x7f11d40ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.670+0000 7f11dc1ce640 1 -- 172.21.15.38:0/2596854128 shutdown_connections 2024-05-21T15:40:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.670+0000 7f11dc1ce640 1 -- 172.21.15.38:0/2596854128 wait complete. 2024-05-21T15:40:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.670+0000 7f11dc1ce640 1 Processor -- start 2024-05-21T15:40:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.670+0000 7f11dc1ce640 1 -- start start 2024-05-21T15:40:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11dc1ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11d4103570 0x7f11d4199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11dc1ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11d4105ed0 0x7f11d41997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11dc1ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4108830 0x7f11d41a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11dc1ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410e6b0 con 0x7f11d4108830 2024-05-21T15:40:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11dc1ce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410e530 con 0x7f11d4103570 2024-05-21T15:40:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11dc1ce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410e830 con 0x7f11d4105ed0 2024-05-21T15:40:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11da76b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4108830 0x7f11d41a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11d9769640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11d4105ed0 0x7f11d41997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11da76b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4108830 0x7f11d41a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33626/0 (socket says 172.21.15.38:33626) 2024-05-21T15:40:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11da76b640 1 -- 172.21.15.38:0/1127190204 learned_addr learned my addr 172.21.15.38:0/1127190204 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11d9f6a640 1 --2- 172.21.15.38:0/1127190204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11d4103570 0x7f11d4199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11d9769640 1 -- 172.21.15.38:0/1127190204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11d4103570 msgr2=0x7f11d4199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11d9769640 1 --2- 172.21.15.38:0/1127190204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11d4103570 0x7f11d4199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11d9769640 1 -- 172.21.15.38:0/1127190204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4108830 msgr2=0x7f11d41a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11d9769640 1 --2- 172.21.15.38:0/1127190204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4108830 0x7f11d41a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11d9769640 1 -- 172.21.15.38:0/1127190204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d41a0dd0 con 0x7f11d4105ed0 2024-05-21T15:40:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.671+0000 7f11da76b640 1 --2- 172.21.15.38:0/1127190204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4108830 0x7f11d41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.672+0000 7f11d9769640 1 --2- 172.21.15.38:0/1127190204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11d4105ed0 0x7f11d41997d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f11c400e9f0 tx=0x7f11c400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.672+0000 7f11caffd640 1 -- 172.21.15.38:0/1127190204 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c400ccc0 con 0x7f11d4105ed0 2024-05-21T15:40:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.672+0000 7f11dc1ce640 1 -- 172.21.15.38:0/1127190204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d4100fa0 con 0x7f11d4105ed0 2024-05-21T15:40:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.672+0000 7f11caffd640 1 -- 172.21.15.38:0/1127190204 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c4010910 con 0x7f11d4105ed0 2024-05-21T15:40:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.672+0000 7f11caffd640 1 -- 172.21.15.38:0/1127190204 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4018960 con 0x7f11d4105ed0 2024-05-21T15:40:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.672+0000 7f11dc1ce640 1 -- 172.21.15.38:0/1127190204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d4101540 con 0x7f11d4105ed0 2024-05-21T15:40:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.675+0000 7f11caffd640 1 -- 172.21.15.38:0/1127190204 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f11c4010ac0 con 0x7f11d4105ed0 2024-05-21T15:40:49.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.675+0000 7f11caffd640 1 --2- 172.21.15.38:0/1127190204 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11c00a2b90 0x7f11c00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.675+0000 7f11e3077740 1 Processor -- start 2024-05-21T15:40:49.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.675+0000 7f11d9f6a640 1 --2- 172.21.15.38:0/1127190204 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11c00a2b90 0x7f11c00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.675+0000 7f11caffd640 1 -- 172.21.15.38:0/1127190204 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f11c40cf420 con 0x7f11d4105ed0 2024-05-21T15:40:49.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.676+0000 7f11e3077740 1 -- start start 2024-05-21T15:40:49.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.676+0000 7f11e3077740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 0x559494741b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.676+0000 7f11e3077740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x559494742080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.676+0000 7f11e3077740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943816b0 0x559494748f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.676+0000 7f11e3077740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5594946a6c40 con 0x5594943816b0 2024-05-21T15:40:49.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.676+0000 7f11e3077740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55949464e580 con 0x5594943a2390 2024-05-21T15:40:49.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.676+0000 7f11e3077740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5594946a2c00 con 0x5594943794d0 2024-05-21T15:40:49.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.676+0000 7f11d9f6a640 1 --2- 172.21.15.38:0/1127190204 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11c00a2b90 0x7f11c00a50a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009230 tx=0x7f11d0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.676+0000 7f11d9769640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x559494742080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.676+0000 7f11d9f6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 0x559494741b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.676+0000 7f11da76b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943816b0 0x559494748f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.676+0000 7f11d9769640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x559494742080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:51708/0 (socket says 172.21.15.38:51708) 2024-05-21T15:40:49.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.676+0000 7f11d9769640 1 -- 172.21.15.38:0/3798558673 learned_addr learned my addr 172.21.15.38:0/3798558673 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.677+0000 7f11d9f6a640 1 -- 172.21.15.38:0/3798558673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 msgr2=0x559494742080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.677+0000 7f11d9f6a640 1 --2- 172.21.15.38:0/3798558673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x559494742080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.677+0000 7f11d9f6a640 1 -- 172.21.15.38:0/3798558673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943816b0 msgr2=0x559494748f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.677+0000 7f11d9f6a640 1 --2- 172.21.15.38:0/3798558673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943816b0 0x559494748f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.677+0000 7f11d9f6a640 1 -- 172.21.15.38:0/3798558673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594943ba3f0 con 0x5594943a2390 2024-05-21T15:40:49.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.677+0000 7f11da76b640 1 --2- 172.21.15.38:0/3798558673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943816b0 0x559494748f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:49.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.677+0000 7f11d9769640 1 --2- 172.21.15.38:0/3798558673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x559494742080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:49.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.677+0000 7f11d9f6a640 1 --2- 172.21.15.38:0/3798558673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 0x559494741b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f11d0006010 tx=0x7f11d003d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22b06203c924c664 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.677+0000 7f11bffff640 1 -- 172.21.15.38:0/3798558673 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d004d070 con 0x5594943a2390 2024-05-21T15:40:49.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.677+0000 7f11bffff640 1 -- 172.21.15.38:0/3798558673 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d0005520 con 0x5594943a2390 2024-05-21T15:40:49.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.677+0000 7f11bffff640 1 -- 172.21.15.38:0/3798558673 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0035040 con 0x5594943a2390 2024-05-21T15:40:49.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.679+0000 7f11e3077740 1 -- 172.21.15.38:0/3798558673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 msgr2=0x559494741b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.679+0000 7f11e3077740 1 --2- 172.21.15.38:0/3798558673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 0x559494741b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f11d0006010 tx=0x7f11d003d000 comp rx=0 tx=0).stop 2024-05-21T15:40:49.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.679+0000 7f11e3077740 1 -- 172.21.15.38:0/3798558673 shutdown_connections 2024-05-21T15:40:49.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.679+0000 7f11e3077740 1 --2- 172.21.15.38:0/3798558673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943816b0 0x559494748f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.679+0000 7f11e3077740 1 --2- 172.21.15.38:0/3798558673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x559494742080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.679+0000 7f11e3077740 1 --2- 172.21.15.38:0/3798558673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 0x559494741b00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.679+0000 7f11e3077740 1 -- 172.21.15.38:0/3798558673 >> 172.21.15.38:0/3798558673 conn(0x5594943964e0 msgr2=0x55949473d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.679+0000 7f11e3077740 1 -- 172.21.15.38:0/3798558673 shutdown_connections 2024-05-21T15:40:49.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.679+0000 7f11e3077740 1 -- 172.21.15.38:0/3798558673 wait complete. 2024-05-21T15:40:49.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.680+0000 7f11e3077740 1 -- start start 2024-05-21T15:40:49.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.680+0000 7f11e3077740 1 Processor -- start 2024-05-21T15:40:49.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.681+0000 7f11e3077740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x55949474b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.681+0000 7f11e3077740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943816b0 0x55949474be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.681+0000 7f11d9f6a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x55949474b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.681+0000 7f11d9f6a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x55949474b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:51710/0 (socket says 172.21.15.38:51710) 2024-05-21T15:40:49.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.681+0000 7f11d9f6a640 1 -- 172.21.15.38:0/4222987026 learned_addr learned my addr 172.21.15.38:0/4222987026 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.681+0000 7f11d9769640 1 --2- 172.21.15.38:0/4222987026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943816b0 0x55949474be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.681+0000 7f11e3077740 1 --2- 172.21.15.38:0/4222987026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943a2390 0x559494752df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.681+0000 7f11e3077740 1 -- 172.21.15.38:0/4222987026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559494749970 con 0x5594943a2390 2024-05-21T15:40:49.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.682+0000 7f11e3077740 1 -- 172.21.15.38:0/4222987026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559494661c40 con 0x5594943816b0 2024-05-21T15:40:49.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.682+0000 7f11da76b640 1 --2- 172.21.15.38:0/4222987026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943a2390 0x559494752df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.682+0000 7f11e3077740 1 -- 172.21.15.38:0/4222987026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559494749af0 con 0x5594943794d0 2024-05-21T15:40:49.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.682+0000 7f11e3077740 10 client.0 _ll_register_callbacks cb 0x5594943b9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:49.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.682+0000 7f11d9769640 1 -- 172.21.15.38:0/4222987026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 msgr2=0x55949474b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.682+0000 7f11d9769640 1 --2- 172.21.15.38:0/4222987026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x55949474b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.682+0000 7f11d9769640 1 -- 172.21.15.38:0/4222987026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943a2390 msgr2=0x559494752df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.682+0000 7f11d9769640 1 --2- 172.21.15.38:0/4222987026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943a2390 0x559494752df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.682+0000 7f11d9769640 1 -- 172.21.15.38:0/4222987026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594944847d0 con 0x5594943816b0 2024-05-21T15:40:49.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.682+0000 7f11d9769640 1 --2- 172.21.15.38:0/4222987026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943816b0 0x55949474be90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f11c40a4370 tx=0x7f11c409b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3817a34d4f6699bd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.682+0000 7f11bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:40:49.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.683+0000 7f11bd7fa640 1 -- 172.21.15.38:0/4222987026 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c40a48a0 con 0x5594943816b0 2024-05-21T15:40:49.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.683+0000 7f11bd7fa640 1 -- 172.21.15.38:0/4222987026 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f11c409b130 con 0x5594943816b0 2024-05-21T15:40:49.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.683+0000 7f11e3077740 1 -- 172.21.15.38:0/4222987026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d0005520 con 0x5594943816b0 2024-05-21T15:40:49.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.683+0000 7f11bd7fa640 1 -- 172.21.15.38:0/4222987026 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f11c409b330 con 0x5594943816b0 2024-05-21T15:40:49.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.683+0000 7f11e3077740 10 client.26801 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:49.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.683+0000 7f11e3077740 1 -- 172.21.15.38:0/4222987026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5594947560a0 con 0x5594943816b0 2024-05-21T15:40:49.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.683+0000 7f11bd7fa640 1 -- 172.21.15.38:0/4222987026 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d0005520 con 0x5594943816b0 2024-05-21T15:40:49.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.683+0000 7f11bd7fa640 1 -- 172.21.15.38:0/4222987026 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f11c4097a30 con 0x5594943816b0 2024-05-21T15:40:49.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.683+0000 7f11e3077740 10 client.26801 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:49.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.683+0000 7f11e3077740 10 client.26801 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:40:49.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.683+0000 7f11e3077740 20 client.26801 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:49.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.683+0000 7f11e3077740 1 --2- 172.21.15.38:0/4222987026 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x559494360f60 0x559494759820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.683+0000 7f11e3077740 4 client.26801 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:49.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.683+0000 7f11e3077740 1 -- 172.21.15.38:0/4222987026 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594946b4370 con 0x559494360f60 2024-05-21T15:40:49.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.684+0000 7f11d9f6a640 1 --2- 172.21.15.38:0/4222987026 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x559494360f60 0x559494759820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.684+0000 7f11bd7fa640 10 client.26801 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:40:49.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.684+0000 7f11d9f6a640 1 --2- 172.21.15.38:0/4222987026 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x559494360f60 0x559494759820 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a7067d1199e2d413 server_cookie=4f88598ac33d5b5c in_seq=0 out_seq=0 2024-05-21T15:40:49.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.685+0000 7f11bd7fa640 1 -- 172.21.15.38:0/4222987026 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5594946b4370 con 0x559494360f60 2024-05-21T15:40:49.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.685+0000 7f11bd7fa640 10 client.26801 handle_command_reply: tid=0 2024-05-21T15:40:49.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.685+0000 7f11e3077740 1 client.26801 shutdown 2024-05-21T15:40:49.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.685+0000 7f11e3077740 1 -- 172.21.15.38:0/4222987026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943816b0 msgr2=0x55949474be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.685+0000 7f11e3077740 1 --2- 172.21.15.38:0/4222987026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943816b0 0x55949474be90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f11c40a4370 tx=0x7f11c409b730 comp rx=0 tx=0).stop 2024-05-21T15:40:49.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.686+0000 7f11e3077740 1 -- 172.21.15.38:0/4222987026 shutdown_connections 2024-05-21T15:40:49.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.686+0000 7f11e3077740 1 --2- 172.21.15.38:0/4222987026 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x559494360f60 0x559494759820 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.686+0000 7f11e3077740 1 --2- 172.21.15.38:0/4222987026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943a2390 0x559494752df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.686+0000 7f11e3077740 1 --2- 172.21.15.38:0/4222987026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943816b0 0x55949474be90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.686+0000 7f11e3077740 1 --2- 172.21.15.38:0/4222987026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x55949474b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.686+0000 7f11e3077740 1 -- 172.21.15.38:0/4222987026 >> 172.21.15.38:0/4222987026 conn(0x5594943964e0 msgr2=0x55949473c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.686+0000 7f11bd7fa640 0 client.26801 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:40:49.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.687+0000 7f11e3077740 1 -- 172.21.15.38:0/4222987026 shutdown_connections 2024-05-21T15:40:49.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.687+0000 7f11e3077740 1 -- 172.21.15.38:0/4222987026 wait complete. 2024-05-21T15:40:49.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.687+0000 7f11e3077740 20 client.26801 trim_cache size 0 max 16384 2024-05-21T15:40:49.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.704+0000 7f11e3077740 1 Processor -- start 2024-05-21T15:40:49.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.704+0000 7f11e3077740 1 -- start start 2024-05-21T15:40:49.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.705+0000 7f11e3077740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943794d0 0x559494749e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.705+0000 7f11e3077740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943816b0 0x559494752d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.705+0000 7f11e3077740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 0x55949474a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.705+0000 7f11e3077740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41a8240 con 0x5594943794d0 2024-05-21T15:40:49.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.705+0000 7f11e3077740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41a9e10 con 0x5594943a2390 2024-05-21T15:40:49.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.705+0000 7f11e3077740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41a8910 con 0x5594943816b0 2024-05-21T15:40:49.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.705+0000 7f11d9f6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943794d0 0x559494749e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.705+0000 7f11d9769640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943816b0 0x559494752d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.705+0000 7f11d9f6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943794d0 0x559494749e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33660/0 (socket says 172.21.15.38:33660) 2024-05-21T15:40:49.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.705+0000 7f11d9f6a640 1 -- 172.21.15.38:0/2971588825 learned_addr learned my addr 172.21.15.38:0/2971588825 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.705+0000 7f11da76b640 1 --2- 172.21.15.38:0/2971588825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 0x55949474a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.705+0000 7f11d9f6a640 1 -- 172.21.15.38:0/2971588825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943816b0 msgr2=0x559494752d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.705+0000 7f11d9f6a640 1 --2- 172.21.15.38:0/2971588825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943816b0 0x559494752d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.706+0000 7f11d9f6a640 1 -- 172.21.15.38:0/2971588825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 msgr2=0x55949474a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.706+0000 7f11d9f6a640 1 --2- 172.21.15.38:0/2971588825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 0x55949474a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.706+0000 7f11d9f6a640 1 -- 172.21.15.38:0/2971588825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559494738d20 con 0x5594943794d0 2024-05-21T15:40:49.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.706+0000 7f11d9f6a640 1 --2- 172.21.15.38:0/2971588825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943794d0 0x559494749e90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f11d0005ab0 tx=0x7f11d0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1749785953d603c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.706+0000 7f11bf7fe640 1 -- 172.21.15.38:0/2971588825 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0063070 con 0x5594943794d0 2024-05-21T15:40:49.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.706+0000 7f11bf7fe640 1 -- 172.21.15.38:0/2971588825 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d005e8d0 con 0x5594943794d0 2024-05-21T15:40:49.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.706+0000 7f11bf7fe640 1 -- 172.21.15.38:0/2971588825 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0040040 con 0x5594943794d0 2024-05-21T15:40:49.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.707+0000 7f11e3077740 1 -- 172.21.15.38:0/2971588825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943794d0 msgr2=0x559494749e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.707+0000 7f11e3077740 1 --2- 172.21.15.38:0/2971588825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943794d0 0x559494749e90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f11d0005ab0 tx=0x7f11d0009260 comp rx=0 tx=0).stop 2024-05-21T15:40:49.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.708+0000 7f11e3077740 1 -- 172.21.15.38:0/2971588825 shutdown_connections 2024-05-21T15:40:49.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.708+0000 7f11e3077740 1 --2- 172.21.15.38:0/2971588825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 0x55949474a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.708+0000 7f11e3077740 1 --2- 172.21.15.38:0/2971588825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943816b0 0x559494752d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.708+0000 7f11e3077740 1 --2- 172.21.15.38:0/2971588825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943794d0 0x559494749e90 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.708+0000 7f11e3077740 1 -- 172.21.15.38:0/2971588825 >> 172.21.15.38:0/2971588825 conn(0x5594943964e0 msgr2=0x559494745e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.708+0000 7f11e3077740 1 -- 172.21.15.38:0/2971588825 shutdown_connections 2024-05-21T15:40:49.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.708+0000 7f11e3077740 1 -- 172.21.15.38:0/2971588825 wait complete. 2024-05-21T15:40:49.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.709+0000 7f11e3077740 1 -- start start 2024-05-21T15:40:49.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.709+0000 7f11e3077740 1 Processor -- start 2024-05-21T15:40:49.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.709+0000 7f11e3077740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x55949474fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.709+0000 7f11e3077740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943816b0 0x5594947501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.709+0000 7f11e3077740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 0x5594947bfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11e3077740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41a96e0 con 0x5594943816b0 2024-05-21T15:40:49.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11e3077740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41a8fe0 con 0x5594943a2390 2024-05-21T15:40:49.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11e3077740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41a6220 con 0x5594943794d0 2024-05-21T15:40:49.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11e3077740 10 client.0 _ll_register_callbacks cb 0x7f11c409b910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:49.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11d9f6a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x55949474fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11d9769640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943816b0 0x5594947501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11d9f6a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x55949474fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:51720/0 (socket says 172.21.15.38:51720) 2024-05-21T15:40:49.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11d9f6a640 1 -- 172.21.15.38:0/2601043744 learned_addr learned my addr 172.21.15.38:0/2601043744 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11da76b640 1 --2- 172.21.15.38:0/2601043744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 0x5594947bfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11d9769640 1 -- 172.21.15.38:0/2601043744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 msgr2=0x55949474fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11d9769640 1 --2- 172.21.15.38:0/2601043744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x55949474fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11d9769640 1 -- 172.21.15.38:0/2601043744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 msgr2=0x5594947bfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11d9769640 1 --2- 172.21.15.38:0/2601043744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 0x5594947bfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11d9769640 1 -- 172.21.15.38:0/2601043744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594947c0090 con 0x5594943816b0 2024-05-21T15:40:49.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11da76b640 1 --2- 172.21.15.38:0/2601043744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 0x5594947bfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:49.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11d9769640 1 --2- 172.21.15.38:0/2601043744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943816b0 0x5594947501c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f11c4018fd0 tx=0x7f11c4097c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc91a57e49dfbd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.710+0000 7f11bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:49.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.711+0000 7f11bd7fa640 1 -- 172.21.15.38:0/2601043744 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c409af40 con 0x5594943816b0 2024-05-21T15:40:49.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.711+0000 7f11bd7fa640 1 -- 172.21.15.38:0/2601043744 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f11c407ab50 con 0x5594943816b0 2024-05-21T15:40:49.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.711+0000 7f11e3077740 1 -- 172.21.15.38:0/2601043744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d005e8d0 con 0x5594943816b0 2024-05-21T15:40:49.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.711+0000 7f11bd7fa640 1 -- 172.21.15.38:0/2601043744 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f11c40a48a0 con 0x5594943816b0 2024-05-21T15:40:49.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.711+0000 7f11e3077740 10 client.17178 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:49.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.711+0000 7f11e3077740 1 -- 172.21.15.38:0/2601043744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5594947c1250 con 0x5594943816b0 2024-05-21T15:40:49.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.711+0000 7f11bd7fa640 1 -- 172.21.15.38:0/2601043744 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d005e8d0 con 0x5594943816b0 2024-05-21T15:40:49.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.711+0000 7f11bd7fa640 1 -- 172.21.15.38:0/2601043744 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f11c40a13c0 con 0x5594943816b0 2024-05-21T15:40:49.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.711+0000 7f11e3077740 10 client.17178 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:49.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.711+0000 7f11e3077740 10 client.17178 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:40:49.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.711+0000 7f11e3077740 20 client.17178 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:49.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.711+0000 7f11e3077740 1 --2- 172.21.15.38:0/2601043744 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5594943363a0 0x5594947c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.711+0000 7f11e3077740 4 client.17178 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:40:49.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.711+0000 7f11e3077740 1 -- 172.21.15.38:0/2601043744 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594946ae6c0 con 0x5594943363a0 2024-05-21T15:40:49.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.712+0000 7f11d9f6a640 1 --2- 172.21.15.38:0/2601043744 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5594943363a0 0x5594947c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.712+0000 7f11bd7fa640 10 client.17178 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:40:49.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.712+0000 7f11d9f6a640 1 --2- 172.21.15.38:0/2601043744 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5594943363a0 0x5594947c3ab0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=638600df57300bca server_cookie=5d9d98623489da99 in_seq=0 out_seq=0 2024-05-21T15:40:49.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11bd7fa640 1 -- 172.21.15.38:0/2601043744 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594946ae6c0 con 0x5594943363a0 2024-05-21T15:40:49.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11bd7fa640 10 client.17178 handle_command_reply: tid=0 2024-05-21T15:40:49.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11e3077740 1 client.17178 shutdown 2024-05-21T15:40:49.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11e3077740 1 -- 172.21.15.38:0/2601043744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943816b0 msgr2=0x5594947501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11e3077740 1 --2- 172.21.15.38:0/2601043744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943816b0 0x5594947501c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f11c4018fd0 tx=0x7f11c4097c40 comp rx=0 tx=0).stop 2024-05-21T15:40:49.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11e3077740 1 -- 172.21.15.38:0/2601043744 shutdown_connections 2024-05-21T15:40:49.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11e3077740 1 --2- 172.21.15.38:0/2601043744 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5594943363a0 0x5594947c3ab0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11e3077740 1 --2- 172.21.15.38:0/2601043744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594943a2390 0x5594947bfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11e3077740 1 --2- 172.21.15.38:0/2601043744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5594943816b0 0x5594947501c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11e3077740 1 --2- 172.21.15.38:0/2601043744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5594943794d0 0x55949474fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11e3077740 1 -- 172.21.15.38:0/2601043744 >> 172.21.15.38:0/2601043744 conn(0x5594943964e0 msgr2=0x559494749e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11bd7fa640 0 client.17178 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:40:49.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11e3077740 1 -- 172.21.15.38:0/2601043744 shutdown_connections 2024-05-21T15:40:49.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11e3077740 1 -- 172.21.15.38:0/2601043744 wait complete. 2024-05-21T15:40:49.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.713+0000 7f11e3077740 20 client.17178 trim_cache size 0 max 16384 2024-05-21T15:40:49.729 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:40:49.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.714+0000 7f11dc1ce640 1 -- 172.21.15.38:0/1127190204 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11c00a2b90 msgr2=0x7f11c00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.714+0000 7f11dc1ce640 1 --2- 172.21.15.38:0/1127190204 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11c00a2b90 0x7f11c00a50a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009230 tx=0x7f11d0031400 comp rx=0 tx=0).stop 2024-05-21T15:40:49.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.714+0000 7f11dc1ce640 1 -- 172.21.15.38:0/1127190204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11d4105ed0 msgr2=0x7f11d41997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.714+0000 7f11dc1ce640 1 --2- 172.21.15.38:0/1127190204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11d4105ed0 0x7f11d41997d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f11c400e9f0 tx=0x7f11c400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.714+0000 7f11dc1ce640 1 -- 172.21.15.38:0/1127190204 shutdown_connections 2024-05-21T15:40:49.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.714+0000 7f11dc1ce640 1 --2- 172.21.15.38:0/1127190204 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11c00a2b90 0x7f11c00a50a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.714+0000 7f11dc1ce640 1 --2- 172.21.15.38:0/1127190204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11d4108830 0x7f11d41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.714+0000 7f11dc1ce640 1 --2- 172.21.15.38:0/1127190204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11d4105ed0 0x7f11d41997d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.714+0000 7f11dc1ce640 1 --2- 172.21.15.38:0/1127190204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11d4103570 0x7f11d4199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.714+0000 7f11dc1ce640 1 -- 172.21.15.38:0/1127190204 >> 172.21.15.38:0/1127190204 conn(0x7f11d40fcb70 msgr2=0x7f11d4104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.714+0000 7f11dc1ce640 1 -- 172.21.15.38:0/1127190204 shutdown_connections 2024-05-21T15:40:49.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.714+0000 7f11dc1ce640 1 -- 172.21.15.38:0/1127190204 wait complete. 2024-05-21T15:40:49.732 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:49 smithi038 ceph-mon[31995]: pgmap v543: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 MiB/s wr, 4.43k op/s 2024-05-21T15:40:49.732 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:49 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1335781018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:40:49.734 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:40:49.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:49 smithi045 ceph-mon[36409]: pgmap v543: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 MiB/s wr, 4.43k op/s 2024-05-21T15:40:49.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:49 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1335781018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:40:49.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.878+0000 7f0f8eb77640 1 -- 172.21.15.38:0/2994020902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f88101990 msgr2=0x7f0f88102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.878+0000 7f0f8eb77640 1 --2- 172.21.15.38:0/2994020902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f88101990 0x7f0f88102690 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009900 tx=0x7f0f7c02a410 comp rx=0 tx=0).stop 2024-05-21T15:40:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.878+0000 7f0f8eb77640 1 -- 172.21.15.38:0/2994020902 shutdown_connections 2024-05-21T15:40:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.878+0000 7f0f8eb77640 1 --2- 172.21.15.38:0/2994020902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f881078d0 0x7f0f88109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.878+0000 7f0f8eb77640 1 --2- 172.21.15.38:0/2994020902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f88102bd0 0x7f0f88102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.878+0000 7f0f8eb77640 1 --2- 172.21.15.38:0/2994020902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f88101990 0x7f0f88102690 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.878+0000 7f0f8eb77640 1 -- 172.21.15.38:0/2994020902 >> 172.21.15.38:0/2994020902 conn(0x7f0f880fcb80 msgr2=0x7f0f880ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.878+0000 7f0f8eb77640 1 -- 172.21.15.38:0/2994020902 shutdown_connections 2024-05-21T15:40:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.878+0000 7f0f8eb77640 1 -- 172.21.15.38:0/2994020902 wait complete. 2024-05-21T15:40:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.879+0000 7f0f8eb77640 1 Processor -- start 2024-05-21T15:40:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.879+0000 7f0f8eb77640 1 -- start start 2024-05-21T15:40:49.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.879+0000 7f0f8eb77640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f88101990 0x7f0f881981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.879+0000 7f0f8eb77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f88102bd0 0x7f0f88198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.879+0000 7f0f8eb77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f881078d0 0x7f0f8819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.879+0000 7f0f8eb77640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8810d5e0 con 0x7f0f88101990 2024-05-21T15:40:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.879+0000 7f0f8eb77640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8810d460 con 0x7f0f88102bd0 2024-05-21T15:40:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.879+0000 7f0f8eb77640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8810d760 con 0x7f0f881078d0 2024-05-21T15:40:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.879+0000 7f0f8c913640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f88101990 0x7f0f881981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.879+0000 7f0f8c913640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f88101990 0x7f0f881981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33674/0 (socket says 172.21.15.38:33674) 2024-05-21T15:40:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.879+0000 7f0f8c913640 1 -- 172.21.15.38:0/2201464408 learned_addr learned my addr 172.21.15.38:0/2201464408 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.879+0000 7f0f8d114640 1 --2- 172.21.15.38:0/2201464408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f881078d0 0x7f0f8819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.879+0000 7f0f87fff640 1 --2- 172.21.15.38:0/2201464408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f88102bd0 0x7f0f88198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.880+0000 7f0f8c913640 1 -- 172.21.15.38:0/2201464408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f881078d0 msgr2=0x7f0f8819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.880+0000 7f0f8c913640 1 --2- 172.21.15.38:0/2201464408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f881078d0 0x7f0f8819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.880+0000 7f0f8c913640 1 -- 172.21.15.38:0/2201464408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f88102bd0 msgr2=0x7f0f88198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.880+0000 7f0f8c913640 1 --2- 172.21.15.38:0/2201464408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f88102bd0 0x7f0f88198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.880+0000 7f0f8c913640 1 -- 172.21.15.38:0/2201464408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8819fcc0 con 0x7f0f88101990 2024-05-21T15:40:49.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.880+0000 7f0f8c913640 1 --2- 172.21.15.38:0/2201464408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f88101990 0x7f0f881981f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c005ab0 tx=0x7f0f7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.880+0000 7f0f85ffb640 1 -- 172.21.15.38:0/2201464408 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c03f030 con 0x7f0f88101990 2024-05-21T15:40:49.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.880+0000 7f0f85ffb640 1 -- 172.21.15.38:0/2201464408 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f7c003ad0 con 0x7f0f88101990 2024-05-21T15:40:49.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.880+0000 7f0f8eb77640 1 -- 172.21.15.38:0/2201464408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8819ff50 con 0x7f0f88101990 2024-05-21T15:40:49.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.880+0000 7f0f85ffb640 1 -- 172.21.15.38:0/2201464408 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c035070 con 0x7f0f88101990 2024-05-21T15:40:49.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.880+0000 7f0f8eb77640 1 -- 172.21.15.38:0/2201464408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f881a0350 con 0x7f0f88101990 2024-05-21T15:40:49.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f85ffb640 1 -- 172.21.15.38:0/2201464408 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0f7c042030 con 0x7f0f88101990 2024-05-21T15:40:49.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f95a26740 1 Processor -- start 2024-05-21T15:40:49.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f95a26740 1 -- start start 2024-05-21T15:40:49.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f95a26740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da75390 0x55a85de14b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f95a26740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 0x55a85de15080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f95a26740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 0x55a85de1bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f95a26740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a85dd79c40 con 0x55a85da75390 2024-05-21T15:40:49.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f95a26740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a85dd21580 con 0x55a85da4c4d0 2024-05-21T15:40:49.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f95a26740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a85dd75c00 con 0x55a85da546b0 2024-05-21T15:40:49.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f8d114640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da75390 0x55a85de14b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f8d114640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da75390 0x55a85de14b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33684/0 (socket says 172.21.15.38:33684) 2024-05-21T15:40:49.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f8d114640 1 -- 172.21.15.38:0/3823504183 learned_addr learned my addr 172.21.15.38:0/3823504183 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f87fff640 1 --2- 172.21.15.38:0/3823504183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 0x55a85de1bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f85ffb640 1 --2- 172.21.15.38:0/2201464408 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0f780a2b90 0x7f0f780a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f8c913640 1 --2- 172.21.15.38:0/3823504183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 0x55a85de15080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f8d114640 1 -- 172.21.15.38:0/3823504183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 msgr2=0x55a85de1bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f8d114640 1 --2- 172.21.15.38:0/3823504183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 0x55a85de1bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f8d114640 1 -- 172.21.15.38:0/3823504183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 msgr2=0x55a85de15080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f8d114640 1 --2- 172.21.15.38:0/3823504183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 0x55a85de15080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f8d114640 1 -- 172.21.15.38:0/3823504183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a85da8d3f0 con 0x55a85da75390 2024-05-21T15:40:49.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.883+0000 7f0f8c913640 1 --2- 172.21.15.38:0/3823504183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 0x55a85de15080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:40:49.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.884+0000 7f0f8d114640 1 --2- 172.21.15.38:0/2201464408 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0f780a2b90 0x7f0f780a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.884+0000 7f0f85ffb640 1 -- 172.21.15.38:0/2201464408 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f0f7c0ef640 con 0x7f0f88101990 2024-05-21T15:40:49.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.884+0000 7f0f8d114640 1 --2- 172.21.15.38:0/3823504183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da75390 0x55a85de14b00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0f80011590 tx=0x7f0f80011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7b86d99bba6fa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.884+0000 7f0f72ffd640 1 -- 172.21.15.38:0/3823504183 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80017050 con 0x55a85da75390 2024-05-21T15:40:49.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.884+0000 7f0f8d114640 1 --2- 172.21.15.38:0/2201464408 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0f780a2b90 0x7f0f780a50a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0f80022000 tx=0x7f0f80023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.884+0000 7f0f72ffd640 1 -- 172.21.15.38:0/3823504183 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f8000fd20 con 0x55a85da75390 2024-05-21T15:40:49.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.884+0000 7f0f72ffd640 1 -- 172.21.15.38:0/3823504183 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8001bc60 con 0x55a85da75390 2024-05-21T15:40:49.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.885+0000 7f0f95a26740 1 -- 172.21.15.38:0/3823504183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da75390 msgr2=0x55a85de14b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.885+0000 7f0f95a26740 1 --2- 172.21.15.38:0/3823504183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da75390 0x55a85de14b00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0f80011590 tx=0x7f0f80011a50 comp rx=0 tx=0).stop 2024-05-21T15:40:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.885+0000 7f0f95a26740 1 -- 172.21.15.38:0/3823504183 shutdown_connections 2024-05-21T15:40:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.885+0000 7f0f95a26740 1 --2- 172.21.15.38:0/3823504183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 0x55a85de1bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.885+0000 7f0f95a26740 1 --2- 172.21.15.38:0/3823504183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 0x55a85de15080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.885+0000 7f0f95a26740 1 --2- 172.21.15.38:0/3823504183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da75390 0x55a85de14b00 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.885+0000 7f0f95a26740 1 -- 172.21.15.38:0/3823504183 >> 172.21.15.38:0/3823504183 conn(0x55a85da694e0 msgr2=0x55a85de10490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.885+0000 7f0f95a26740 1 -- 172.21.15.38:0/3823504183 shutdown_connections 2024-05-21T15:40:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.885+0000 7f0f95a26740 1 -- 172.21.15.38:0/3823504183 wait complete. 2024-05-21T15:40:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.885+0000 7f0f95a26740 1 -- start start 2024-05-21T15:40:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.885+0000 7f0f95a26740 1 Processor -- start 2024-05-21T15:40:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f95a26740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 0x55a85de1e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f95a26740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da546b0 0x55a85de1eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f95a26740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da75390 0x55a85de25e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f95a26740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a85de1c9d0 con 0x55a85da546b0 2024-05-21T15:40:49.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f95a26740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a85dd34c40 con 0x55a85da4c4d0 2024-05-21T15:40:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f95a26740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a85de1cb50 con 0x55a85da75390 2024-05-21T15:40:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f95a26740 10 client.0 _ll_register_callbacks cb 0x55a85da8cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f87fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da546b0 0x55a85de1eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f87fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da546b0 0x55a85de1eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33690/0 (socket says 172.21.15.38:33690) 2024-05-21T15:40:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f87fff640 1 -- 172.21.15.38:0/43136798 learned_addr learned my addr 172.21.15.38:0/43136798 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f8d114640 1 --2- 172.21.15.38:0/43136798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da75390 0x55a85de25e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f8c913640 1 --2- 172.21.15.38:0/43136798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 0x55a85de1e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f87fff640 1 -- 172.21.15.38:0/43136798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da75390 msgr2=0x55a85de25e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f87fff640 1 --2- 172.21.15.38:0/43136798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da75390 0x55a85de25e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f87fff640 1 -- 172.21.15.38:0/43136798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 msgr2=0x55a85de1e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f87fff640 1 --2- 172.21.15.38:0/43136798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 0x55a85de1e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f87fff640 1 -- 172.21.15.38:0/43136798 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a85db577d0 con 0x55a85da546b0 2024-05-21T15:40:49.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f87fff640 1 --2- 172.21.15.38:0/43136798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da546b0 0x55a85de1eec0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0f74012700 tx=0x7f0f74012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea8e23808951168 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f67fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.886+0000 7f0f67fff640 1 -- 172.21.15.38:0/43136798 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74004030 con 0x55a85da546b0 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.887+0000 7f0f67fff640 1 -- 172.21.15.38:0/43136798 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f0f740041d0 con 0x55a85da546b0 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.887+0000 7f0f95a26740 1 -- 172.21.15.38:0/43136798 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f8000fd20 con 0x55a85da546b0 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.887+0000 7f0f67fff640 1 -- 172.21.15.38:0/43136798 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0f740043d0 con 0x55a85da546b0 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.887+0000 7f0f95a26740 10 client.17190 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.887+0000 7f0f95a26740 1 -- 172.21.15.38:0/43136798 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a85de2b330 con 0x55a85da546b0 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.887+0000 7f0f67fff640 1 -- 172.21.15.38:0/43136798 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f8000fd20 con 0x55a85da546b0 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.887+0000 7f0f67fff640 1 -- 172.21.15.38:0/43136798 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f0f74003ac0 con 0x55a85da546b0 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.887+0000 7f0f95a26740 10 client.17190 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.887+0000 7f0f95a26740 10 client.17190 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.887+0000 7f0f95a26740 20 client.17190 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.887+0000 7f0f95a26740 1 --2- 172.21.15.38:0/43136798 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55a85da33f60 0x55a85de2ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.887+0000 7f0f95a26740 4 client.17190 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:49.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.887+0000 7f0f95a26740 1 -- 172.21.15.38:0/43136798 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a85dd87370 con 0x55a85da33f60 2024-05-21T15:40:49.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.887+0000 7f0f8c913640 1 --2- 172.21.15.38:0/43136798 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55a85da33f60 0x55a85de2ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.888+0000 7f0f8c913640 1 --2- 172.21.15.38:0/43136798 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55a85da33f60 0x55a85de2ea10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b76a12e94c59bd53 server_cookie=51ae240c38c9e9d9 in_seq=0 out_seq=0 2024-05-21T15:40:49.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.888+0000 7f0f67fff640 10 client.17190 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:40:49.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.888+0000 7f0f67fff640 1 -- 172.21.15.38:0/43136798 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a85dd87370 con 0x55a85da33f60 2024-05-21T15:40:49.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.888+0000 7f0f67fff640 10 client.17190 handle_command_reply: tid=0 2024-05-21T15:40:49.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.888+0000 7f0f95a26740 1 client.17190 shutdown 2024-05-21T15:40:49.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.889+0000 7f0f95a26740 1 -- 172.21.15.38:0/43136798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da546b0 msgr2=0x55a85de1eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.889+0000 7f0f95a26740 1 --2- 172.21.15.38:0/43136798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da546b0 0x55a85de1eec0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0f74012700 tx=0x7f0f74012bc0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.889+0000 7f0f95a26740 1 -- 172.21.15.38:0/43136798 shutdown_connections 2024-05-21T15:40:49.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.889+0000 7f0f95a26740 1 --2- 172.21.15.38:0/43136798 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55a85da33f60 0x55a85de2ea10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.889+0000 7f0f95a26740 1 --2- 172.21.15.38:0/43136798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da75390 0x55a85de25e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.889+0000 7f0f95a26740 1 --2- 172.21.15.38:0/43136798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da546b0 0x55a85de1eec0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.889+0000 7f0f67fff640 0 client.17190 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:40:49.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.889+0000 7f0f95a26740 1 --2- 172.21.15.38:0/43136798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 0x55a85de1e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.889+0000 7f0f95a26740 1 -- 172.21.15.38:0/43136798 >> 172.21.15.38:0/43136798 conn(0x55a85da694e0 msgr2=0x55a85de0f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.889+0000 7f0f95a26740 1 -- 172.21.15.38:0/43136798 shutdown_connections 2024-05-21T15:40:49.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.889+0000 7f0f95a26740 1 -- 172.21.15.38:0/43136798 wait complete. 2024-05-21T15:40:49.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.889+0000 7f0f95a26740 20 client.17190 trim_cache size 0 max 16384 2024-05-21T15:40:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f95a26740 1 Processor -- start 2024-05-21T15:40:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f95a26740 1 -- start start 2024-05-21T15:40:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f95a26740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da33f60 0x55a85de1d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f95a26740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da4c4d0 0x55a85de33200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f95a26740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 0x55a85de33740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f95a26740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881a81a0 con 0x55a85da4c4d0 2024-05-21T15:40:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f95a26740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881a9d70 con 0x55a85da33f60 2024-05-21T15:40:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f95a26740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881a8870 con 0x55a85da546b0 2024-05-21T15:40:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f87fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da4c4d0 0x55a85de33200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f87fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da4c4d0 0x55a85de33200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33706/0 (socket says 172.21.15.38:33706) 2024-05-21T15:40:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f87fff640 1 -- 172.21.15.38:0/2852680512 learned_addr learned my addr 172.21.15.38:0/2852680512 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f8d114640 1 --2- 172.21.15.38:0/2852680512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 0x55a85de33740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f8c913640 1 --2- 172.21.15.38:0/2852680512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da33f60 0x55a85de1d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f87fff640 1 -- 172.21.15.38:0/2852680512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 msgr2=0x55a85de33740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f87fff640 1 --2- 172.21.15.38:0/2852680512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 0x55a85de33740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f87fff640 1 -- 172.21.15.38:0/2852680512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da33f60 msgr2=0x55a85de1d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f87fff640 1 --2- 172.21.15.38:0/2852680512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da33f60 0x55a85de1d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f87fff640 1 -- 172.21.15.38:0/2852680512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a85de0bd20 con 0x55a85da4c4d0 2024-05-21T15:40:49.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.903+0000 7f0f8c913640 1 --2- 172.21.15.38:0/2852680512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da33f60 0x55a85de1d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:40:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.904+0000 7f0f87fff640 1 --2- 172.21.15.38:0/2852680512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da4c4d0 0x55a85de33200 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009e00 tx=0x7f0f74005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747d91916dcb484b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.904+0000 7f0f727fc640 1 -- 172.21.15.38:0/2852680512 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f740040a0 con 0x55a85da4c4d0 2024-05-21T15:40:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.904+0000 7f0f727fc640 1 -- 172.21.15.38:0/2852680512 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f74003ad0 con 0x55a85da4c4d0 2024-05-21T15:40:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.904+0000 7f0f727fc640 1 -- 172.21.15.38:0/2852680512 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7401c580 con 0x55a85da4c4d0 2024-05-21T15:40:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.905+0000 7f0f95a26740 1 -- 172.21.15.38:0/2852680512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da4c4d0 msgr2=0x55a85de33200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.905+0000 7f0f95a26740 1 --2- 172.21.15.38:0/2852680512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da4c4d0 0x55a85de33200 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009e00 tx=0x7f0f74005710 comp rx=0 tx=0).stop 2024-05-21T15:40:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.905+0000 7f0f95a26740 1 -- 172.21.15.38:0/2852680512 shutdown_connections 2024-05-21T15:40:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.905+0000 7f0f95a26740 1 --2- 172.21.15.38:0/2852680512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 0x55a85de33740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.905+0000 7f0f95a26740 1 --2- 172.21.15.38:0/2852680512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da4c4d0 0x55a85de33200 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.905+0000 7f0f95a26740 1 --2- 172.21.15.38:0/2852680512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da33f60 0x55a85de1d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.905+0000 7f0f95a26740 1 -- 172.21.15.38:0/2852680512 >> 172.21.15.38:0/2852680512 conn(0x55a85da694e0 msgr2=0x55a85de1cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.905+0000 7f0f95a26740 1 -- 172.21.15.38:0/2852680512 shutdown_connections 2024-05-21T15:40:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.905+0000 7f0f95a26740 1 -- 172.21.15.38:0/2852680512 wait complete. 2024-05-21T15:40:49.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.905+0000 7f0f95a26740 1 -- start start 2024-05-21T15:40:49.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.905+0000 7f0f95a26740 1 Processor -- start 2024-05-21T15:40:49.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.905+0000 7f0f95a26740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da33f60 0x55a85de22720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f95a26740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 0x55a85de22c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f95a26740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 0x55a85de23b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f95a26740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881a9640 con 0x55a85da33f60 2024-05-21T15:40:49.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f95a26740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881a8f40 con 0x55a85da4c4d0 2024-05-21T15:40:49.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f95a26740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881a6180 con 0x55a85da546b0 2024-05-21T15:40:49.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f95a26740 10 client.0 _ll_register_callbacks cb 0x7f0f74009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:49.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f8c913640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da33f60 0x55a85de22720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f8c913640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da33f60 0x55a85de22720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33712/0 (socket says 172.21.15.38:33712) 2024-05-21T15:40:49.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f8c913640 1 -- 172.21.15.38:0/2887098004 learned_addr learned my addr 172.21.15.38:0/2887098004 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:49.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f87fff640 1 --2- 172.21.15.38:0/2887098004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 0x55a85de22c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f8d114640 1 --2- 172.21.15.38:0/2887098004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 0x55a85de23b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f8c913640 1 -- 172.21.15.38:0/2887098004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 msgr2=0x55a85de23b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f8c913640 1 --2- 172.21.15.38:0/2887098004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 0x55a85de23b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f8c913640 1 -- 172.21.15.38:0/2887098004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 msgr2=0x55a85de22c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:49.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f8c913640 1 --2- 172.21.15.38:0/2887098004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 0x55a85de22c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f8c913640 1 -- 172.21.15.38:0/2887098004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a85de24100 con 0x55a85da33f60 2024-05-21T15:40:49.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f8d114640 1 --2- 172.21.15.38:0/2887098004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 0x55a85de23b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:40:49.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f8c913640 1 --2- 172.21.15.38:0/2887098004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da33f60 0x55a85de22720 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0ed460 tx=0x7f0f7c08efc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e017f1c7093527 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:49.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f67fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:49.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f67fff640 1 -- 172.21.15.38:0/2887098004 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c049040 con 0x55a85da33f60 2024-05-21T15:40:49.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f67fff640 1 -- 172.21.15.38:0/2887098004 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f0f7c0035f0 con 0x55a85da33f60 2024-05-21T15:40:49.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f95a26740 1 -- 172.21.15.38:0/2887098004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f74003ad0 con 0x55a85da33f60 2024-05-21T15:40:49.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.906+0000 7f0f67fff640 1 -- 172.21.15.38:0/2887098004 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0f7c0d9070 con 0x55a85da33f60 2024-05-21T15:40:49.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.907+0000 7f0f95a26740 10 client.17196 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:49.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.907+0000 7f0f95a26740 1 -- 172.21.15.38:0/2887098004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a85de25450 con 0x55a85da33f60 2024-05-21T15:40:49.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.907+0000 7f0f67fff640 1 -- 172.21.15.38:0/2887098004 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f74003ad0 con 0x55a85da33f60 2024-05-21T15:40:49.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.907+0000 7f0f67fff640 1 -- 172.21.15.38:0/2887098004 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f0f7c03b460 con 0x55a85da33f60 2024-05-21T15:40:49.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.907+0000 7f0f95a26740 10 client.17196 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:49.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.907+0000 7f0f95a26740 10 client.17196 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:40:49.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.907+0000 7f0f95a26740 20 client.17196 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:49.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.907+0000 7f0f95a26740 1 --2- 172.21.15.38:0/2887098004 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55a85da75390 0x55a85de99290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:49.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.907+0000 7f0f95a26740 4 client.17196 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:40:49.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.907+0000 7f0f95a26740 1 -- 172.21.15.38:0/2887098004 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a85dd816c0 con 0x55a85da75390 2024-05-21T15:40:49.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.907+0000 7f0f87fff640 1 --2- 172.21.15.38:0/2887098004 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55a85da75390 0x55a85de99290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:49.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f67fff640 10 client.17196 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:40:49.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f87fff640 1 --2- 172.21.15.38:0/2887098004 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55a85da75390 0x55a85de99290 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=630b7873fff5db5f server_cookie=c08fc826bcfee4a3 in_seq=0 out_seq=0 2024-05-21T15:40:49.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f67fff640 1 -- 172.21.15.38:0/2887098004 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a85dd816c0 con 0x55a85da75390 2024-05-21T15:40:49.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f67fff640 10 client.17196 handle_command_reply: tid=0 2024-05-21T15:40:49.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f95a26740 1 client.17196 shutdown 2024-05-21T15:40:49.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f95a26740 1 -- 172.21.15.38:0/2887098004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da33f60 msgr2=0x55a85de22720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f95a26740 1 --2- 172.21.15.38:0/2887098004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da33f60 0x55a85de22720 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0ed460 tx=0x7f0f7c08efc0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f95a26740 1 -- 172.21.15.38:0/2887098004 shutdown_connections 2024-05-21T15:40:49.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f95a26740 1 --2- 172.21.15.38:0/2887098004 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x55a85da75390 0x55a85de99290 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f95a26740 1 --2- 172.21.15.38:0/2887098004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a85da546b0 0x55a85de23b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f95a26740 1 --2- 172.21.15.38:0/2887098004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a85da4c4d0 0x55a85de22c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f67fff640 0 client.17196 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:40:49.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f95a26740 1 --2- 172.21.15.38:0/2887098004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a85da33f60 0x55a85de22720 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f95a26740 1 -- 172.21.15.38:0/2887098004 >> 172.21.15.38:0/2887098004 conn(0x55a85da694e0 msgr2=0x55a85de13330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f95a26740 1 -- 172.21.15.38:0/2887098004 shutdown_connections 2024-05-21T15:40:49.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f95a26740 1 -- 172.21.15.38:0/2887098004 wait complete. 2024-05-21T15:40:49.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.908+0000 7f0f95a26740 20 client.17196 trim_cache size 0 max 16384 2024-05-21T15:40:49.914 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:40:49.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.909+0000 7f0f8eb77640 1 -- 172.21.15.38:0/2201464408 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0f780a2b90 msgr2=0x7f0f780a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.909+0000 7f0f8eb77640 1 --2- 172.21.15.38:0/2201464408 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0f780a2b90 0x7f0f780a50a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0f80022000 tx=0x7f0f80023000 comp rx=0 tx=0).stop 2024-05-21T15:40:49.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.909+0000 7f0f8eb77640 1 -- 172.21.15.38:0/2201464408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f88101990 msgr2=0x7f0f881981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:49.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.909+0000 7f0f8eb77640 1 --2- 172.21.15.38:0/2201464408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f88101990 0x7f0f881981f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c005ab0 tx=0x7f0f7c004270 comp rx=0 tx=0).stop 2024-05-21T15:40:49.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.909+0000 7f0f8eb77640 1 -- 172.21.15.38:0/2201464408 shutdown_connections 2024-05-21T15:40:49.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.909+0000 7f0f8eb77640 1 --2- 172.21.15.38:0/2201464408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f881078d0 0x7f0f8819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.909+0000 7f0f8eb77640 1 --2- 172.21.15.38:0/2201464408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f88102bd0 0x7f0f88198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.909+0000 7f0f8eb77640 1 --2- 172.21.15.38:0/2201464408 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0f780a2b90 0x7f0f780a50a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.909+0000 7f0f8eb77640 1 --2- 172.21.15.38:0/2201464408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f88101990 0x7f0f881981f0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:49.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.909+0000 7f0f8eb77640 1 -- 172.21.15.38:0/2201464408 >> 172.21.15.38:0/2201464408 conn(0x7f0f880fcb80 msgr2=0x7f0f881038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:49.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.909+0000 7f0f8eb77640 1 -- 172.21.15.38:0/2201464408 shutdown_connections 2024-05-21T15:40:49.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:49.909+0000 7f0f8eb77640 1 -- 172.21.15.38:0/2201464408 wait complete. 2024-05-21T15:40:49.918 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:40:49.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:49 smithi053 ceph-mon[36496]: pgmap v543: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 MiB/s wr, 4.43k op/s 2024-05-21T15:40:49.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:49 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1335781018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:40:50.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.047+0000 7ff1f1509640 1 -- 172.21.15.38:0/2220068993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1ec074830 msgr2=0x7ff1ec10a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.047+0000 7ff1f1509640 1 --2- 172.21.15.38:0/2220068993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1ec074830 0x7ff1ec10a520 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009900 tx=0x7ff1d402a410 comp rx=0 tx=0).stop 2024-05-21T15:40:50.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.047+0000 7ff1f1509640 1 -- 172.21.15.38:0/2220068993 shutdown_connections 2024-05-21T15:40:50.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.047+0000 7ff1f1509640 1 --2- 172.21.15.38:0/2220068993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1ec10aa60 0x7ff1ec10ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.047+0000 7ff1f1509640 1 --2- 172.21.15.38:0/2220068993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1ec074830 0x7ff1ec10a520 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.048+0000 7ff1f1509640 1 --2- 172.21.15.38:0/2220068993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1ec073ee0 0x7ff1ec0742f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.048+0000 7ff1f1509640 1 -- 172.21.15.38:0/2220068993 >> 172.21.15.38:0/2220068993 conn(0x7ff1ec0fdc80 msgr2=0x7ff1ec100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:50.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.048+0000 7ff1f1509640 1 -- 172.21.15.38:0/2220068993 shutdown_connections 2024-05-21T15:40:50.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.048+0000 7ff1f1509640 1 -- 172.21.15.38:0/2220068993 wait complete. 2024-05-21T15:40:50.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.048+0000 7ff1f1509640 1 Processor -- start 2024-05-21T15:40:50.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.048+0000 7ff1f1509640 1 -- start start 2024-05-21T15:40:50.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1f1509640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1ec073ee0 0x7ff1ec19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1f1509640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1ec074830 0x7ff1ec19b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1f1509640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1ec10aa60 0x7ff1ec1a2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1f1509640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec110900 con 0x7ff1ec074830 2024-05-21T15:40:50.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1f1509640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec110780 con 0x7ff1ec073ee0 2024-05-21T15:40:50.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1f1509640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec110a80 con 0x7ff1ec10aa60 2024-05-21T15:40:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1e3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1ec074830 0x7ff1ec19b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1eaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1ec073ee0 0x7ff1ec19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1e3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1ec074830 0x7ff1ec19b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33722/0 (socket says 172.21.15.38:33722) 2024-05-21T15:40:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1e3fff640 1 -- 172.21.15.38:0/370426884 learned_addr learned my addr 172.21.15.38:0/370426884 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1eb7fe640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1ec10aa60 0x7ff1ec1a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1e3fff640 1 -- 172.21.15.38:0/370426884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1ec10aa60 msgr2=0x7ff1ec1a2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1ec10aa60 0x7ff1ec1a2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1e3fff640 1 -- 172.21.15.38:0/370426884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1ec073ee0 msgr2=0x7ff1ec19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1ec073ee0 0x7ff1ec19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1e3fff640 1 -- 172.21.15.38:0/370426884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec1a2fb0 con 0x7ff1ec074830 2024-05-21T15:40:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1eaffd640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1ec073ee0 0x7ff1ec19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.049+0000 7ff1eb7fe640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1ec10aa60 0x7ff1ec1a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.050+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1ec074830 0x7ff1ec19b9b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4006010 tx=0x7ff1d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.050+0000 7ff1e8ff9640 1 -- 172.21.15.38:0/370426884 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d403f030 con 0x7ff1ec074830 2024-05-21T15:40:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.050+0000 7ff1e8ff9640 1 -- 172.21.15.38:0/370426884 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1d4003ce0 con 0x7ff1ec074830 2024-05-21T15:40:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.050+0000 7ff1f1509640 1 -- 172.21.15.38:0/370426884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ec1a3240 con 0x7ff1ec074830 2024-05-21T15:40:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.050+0000 7ff1e8ff9640 1 -- 172.21.15.38:0/370426884 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4035070 con 0x7ff1ec074830 2024-05-21T15:40:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.050+0000 7ff1f1509640 1 -- 172.21.15.38:0/370426884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ec1a3730 con 0x7ff1ec074830 2024-05-21T15:40:50.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.054+0000 7ff1e8ff9640 1 -- 172.21.15.38:0/370426884 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff1d4042030 con 0x7ff1ec074830 2024-05-21T15:40:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.055+0000 7ff1e8ff9640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff1d80a2c10 0x7ff1d80a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.056+0000 7ff1eaffd640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff1d80a2c10 0x7ff1d80a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.056+0000 7ff1f83b2740 1 Processor -- start 2024-05-21T15:40:50.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.056+0000 7ff1e8ff9640 1 -- 172.21.15.38:0/370426884 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff1d40ef840 con 0x7ff1ec074830 2024-05-21T15:40:50.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.056+0000 7ff1f83b2740 1 -- start start 2024-05-21T15:40:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.056+0000 7ff1f83b2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44fae6390 0x55c44fe85b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.056+0000 7ff1f83b2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fabd4d0 0x55c44fe86080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1f83b2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fac56b0 0x55c44fe8cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1e3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fabd4d0 0x55c44fe86080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1e3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fabd4d0 0x55c44fe86080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:51800/0 (socket says 172.21.15.38:51800) 2024-05-21T15:40:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1e3fff640 1 -- 172.21.15.38:0/824728936 learned_addr learned my addr 172.21.15.38:0/824728936 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1f83b2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c44fdeac40 con 0x55c44fae6390 2024-05-21T15:40:50.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1f83b2740 1 -- 172.21.15.38:0/824728936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c44fd92580 con 0x55c44fac56b0 2024-05-21T15:40:50.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1f83b2740 1 -- 172.21.15.38:0/824728936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c44fde6c00 con 0x55c44fabd4d0 2024-05-21T15:40:50.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1eb7fe640 1 --2- 172.21.15.38:0/824728936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fac56b0 0x55c44fe8cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1eaffd640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff1d80a2c10 0x7ff1d80a5120 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec075710 tx=0x7ff1dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:50.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1eaffd640 1 --2- 172.21.15.38:0/824728936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44fae6390 0x55c44fe85b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1e3fff640 1 -- 172.21.15.38:0/824728936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fac56b0 msgr2=0x55c44fe8cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/824728936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fac56b0 0x55c44fe8cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1e3fff640 1 -- 172.21.15.38:0/824728936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44fae6390 msgr2=0x55c44fe85b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/824728936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44fae6390 0x55c44fe85b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1e3fff640 1 -- 172.21.15.38:0/824728936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44fafe3f0 con 0x55c44fabd4d0 2024-05-21T15:40:50.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.057+0000 7ff1eaffd640 1 --2- 172.21.15.38:0/824728936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44fae6390 0x55c44fe85b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:50.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.058+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/824728936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fabd4d0 0x55c44fe86080 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40c4730 tx=0x7ff1d40058b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4428f567296bf149 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:50.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.058+0000 7ff1e17fa640 1 -- 172.21.15.38:0/824728936 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4049040 con 0x55c44fabd4d0 2024-05-21T15:40:50.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.058+0000 7ff1e17fa640 1 -- 172.21.15.38:0/824728936 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1d40bc0a0 con 0x55c44fabd4d0 2024-05-21T15:40:50.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.058+0000 7ff1e17fa640 1 -- 172.21.15.38:0/824728936 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4003800 con 0x55c44fabd4d0 2024-05-21T15:40:50.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.059+0000 7ff1f83b2740 1 -- 172.21.15.38:0/824728936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fabd4d0 msgr2=0x55c44fe86080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.059+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/824728936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fabd4d0 0x55c44fe86080 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40c4730 tx=0x7ff1d40058b0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.059+0000 7ff1f83b2740 1 -- 172.21.15.38:0/824728936 shutdown_connections 2024-05-21T15:40:50.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.059+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/824728936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fac56b0 0x55c44fe8cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.059+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/824728936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fabd4d0 0x55c44fe86080 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.059+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/824728936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44fae6390 0x55c44fe85b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.059+0000 7ff1f83b2740 1 -- 172.21.15.38:0/824728936 >> 172.21.15.38:0/824728936 conn(0x55c44fada4e0 msgr2=0x55c44fe81490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:50.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.059+0000 7ff1f83b2740 1 -- 172.21.15.38:0/824728936 shutdown_connections 2024-05-21T15:40:50.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.059+0000 7ff1f83b2740 1 -- 172.21.15.38:0/824728936 wait complete. 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.060+0000 7ff1f83b2740 1 -- start start 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.060+0000 7ff1f83b2740 1 Processor -- start 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.060+0000 7ff1f83b2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44fabd4d0 0x55c44fe8f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.060+0000 7ff1f83b2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fac56b0 0x55c44fe8fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.060+0000 7ff1f83b2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fae6390 0x55c44fe96b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.060+0000 7ff1f83b2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c44fe8d790 con 0x55c44fabd4d0 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.060+0000 7ff1f83b2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c44fda5c40 con 0x55c44fac56b0 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.060+0000 7ff1f83b2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c44fe8d910 con 0x55c44fae6390 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.060+0000 7ff1eaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44fabd4d0 0x55c44fe8f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.060+0000 7ff1f83b2740 10 client.0 _ll_register_callbacks cb 0x55c44fafdbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.060+0000 7ff1eaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44fabd4d0 0x55c44fe8f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33734/0 (socket says 172.21.15.38:33734) 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.060+0000 7ff1eaffd640 1 -- 172.21.15.38:0/1881657404 learned_addr learned my addr 172.21.15.38:0/1881657404 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.060+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/1881657404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fac56b0 0x55c44fe8fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.060+0000 7ff1eb7fe640 1 --2- 172.21.15.38:0/1881657404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fae6390 0x55c44fe96b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.061+0000 7ff1eaffd640 1 -- 172.21.15.38:0/1881657404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fae6390 msgr2=0x55c44fe96b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.061+0000 7ff1eaffd640 1 --2- 172.21.15.38:0/1881657404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fae6390 0x55c44fe96b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.061+0000 7ff1eaffd640 1 -- 172.21.15.38:0/1881657404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fac56b0 msgr2=0x55c44fe8fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.061+0000 7ff1eaffd640 1 --2- 172.21.15.38:0/1881657404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fac56b0 0x55c44fe8fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.061+0000 7ff1eaffd640 1 -- 172.21.15.38:0/1881657404 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44fbc87d0 con 0x55c44fabd4d0 2024-05-21T15:40:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.061+0000 7ff1eaffd640 1 --2- 172.21.15.38:0/1881657404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44fabd4d0 0x55c44fe8f6e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc003710 tx=0x7ff1dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7947354e7f08ea8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.061+0000 7ff1d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.061+0000 7ff1d27fc640 1 -- 172.21.15.38:0/1881657404 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00f890 con 0x55c44fabd4d0 2024-05-21T15:40:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.061+0000 7ff1d27fc640 1 -- 172.21.15.38:0/1881657404 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff1dc00fa30 con 0x55c44fabd4d0 2024-05-21T15:40:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.061+0000 7ff1f83b2740 1 -- 172.21.15.38:0/1881657404 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1d40bc0a0 con 0x55c44fabd4d0 2024-05-21T15:40:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.061+0000 7ff1d27fc640 1 -- 172.21.15.38:0/1881657404 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff1dc00fc30 con 0x55c44fabd4d0 2024-05-21T15:40:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.061+0000 7ff1f83b2740 10 client.17208 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.061+0000 7ff1f83b2740 1 -- 172.21.15.38:0/1881657404 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c44fe99f10 con 0x55c44fabd4d0 2024-05-21T15:40:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.062+0000 7ff1d27fc640 1 -- 172.21.15.38:0/1881657404 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1d40bc0a0 con 0x55c44fabd4d0 2024-05-21T15:40:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.062+0000 7ff1d27fc640 1 -- 172.21.15.38:0/1881657404 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7ff1dc01a070 con 0x55c44fabd4d0 2024-05-21T15:40:50.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.062+0000 7ff1f83b2740 10 client.17208 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:50.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.062+0000 7ff1f83b2740 10 client.17208 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:40:50.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.062+0000 7ff1f83b2740 20 client.17208 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:50.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.062+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/1881657404 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55c44faa4f60 0x55c44fe9d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.062+0000 7ff1f83b2740 4 client.17208 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:50.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.062+0000 7ff1f83b2740 1 -- 172.21.15.38:0/1881657404 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c44fdf8370 con 0x55c44faa4f60 2024-05-21T15:40:50.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.062+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/1881657404 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55c44faa4f60 0x55c44fe9d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.062+0000 7ff1d27fc640 10 client.17208 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:40:50.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.062+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/1881657404 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55c44faa4f60 0x55c44fe9d660 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac5dc01dcc6b5f05 server_cookie=d4081d5c71623c38 in_seq=0 out_seq=0 2024-05-21T15:40:50.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.063+0000 7ff1d27fc640 1 -- 172.21.15.38:0/1881657404 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c44fdf8370 con 0x55c44faa4f60 2024-05-21T15:40:50.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.063+0000 7ff1d27fc640 10 client.17208 handle_command_reply: tid=0 2024-05-21T15:40:50.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.063+0000 7ff1f83b2740 1 client.17208 shutdown 2024-05-21T15:40:50.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.064+0000 7ff1f83b2740 1 -- 172.21.15.38:0/1881657404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44fabd4d0 msgr2=0x55c44fe8f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.064+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/1881657404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44fabd4d0 0x55c44fe8f6e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc003710 tx=0x7ff1dc004730 comp rx=0 tx=0).stop 2024-05-21T15:40:50.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.064+0000 7ff1f83b2740 1 -- 172.21.15.38:0/1881657404 shutdown_connections 2024-05-21T15:40:50.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.064+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/1881657404 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55c44faa4f60 0x55c44fe9d660 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.064+0000 7ff1d27fc640 0 client.17208 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:40:50.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.064+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/1881657404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fae6390 0x55c44fe96b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.064+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/1881657404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fac56b0 0x55c44fe8fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.064+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/1881657404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44fabd4d0 0x55c44fe8f6e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.064+0000 7ff1f83b2740 1 -- 172.21.15.38:0/1881657404 >> 172.21.15.38:0/1881657404 conn(0x55c44fada4e0 msgr2=0x55c44fe80570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.064+0000 7ff1f83b2740 1 -- 172.21.15.38:0/1881657404 shutdown_connections 2024-05-21T15:40:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.065+0000 7ff1f83b2740 1 -- 172.21.15.38:0/1881657404 wait complete. 2024-05-21T15:40:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.065+0000 7ff1f83b2740 20 client.17208 trim_cache size 0 max 16384 2024-05-21T15:40:50.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.087+0000 7ff1f83b2740 1 Processor -- start 2024-05-21T15:40:50.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.088+0000 7ff1f83b2740 1 -- start start 2024-05-21T15:40:50.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.088+0000 7ff1f83b2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44faa4f60 0x55c44fe8f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.088+0000 7ff1f83b2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fabd4d0 0x55c44fe96b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.088+0000 7ff1f83b2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fac56b0 0x55c44fe8f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.088+0000 7ff1f83b2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1ac620 con 0x55c44faa4f60 2024-05-21T15:40:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.088+0000 7ff1f83b2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1ae1f0 con 0x55c44fabd4d0 2024-05-21T15:40:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.088+0000 7ff1f83b2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1accf0 con 0x55c44fac56b0 2024-05-21T15:40:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.088+0000 7ff1eaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44faa4f60 0x55c44fe8f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.088+0000 7ff1eaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44faa4f60 0x55c44fe8f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33738/0 (socket says 172.21.15.38:33738) 2024-05-21T15:40:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.088+0000 7ff1eaffd640 1 -- 172.21.15.38:0/1197354506 learned_addr learned my addr 172.21.15.38:0/1197354506 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.088+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/1197354506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fabd4d0 0x55c44fe96b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.088+0000 7ff1eb7fe640 1 --2- 172.21.15.38:0/1197354506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fac56b0 0x55c44fe8f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.089+0000 7ff1e3fff640 1 -- 172.21.15.38:0/1197354506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fac56b0 msgr2=0x55c44fe8f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.089+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/1197354506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fac56b0 0x55c44fe8f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.089+0000 7ff1e3fff640 1 -- 172.21.15.38:0/1197354506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44faa4f60 msgr2=0x55c44fe8f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.089+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/1197354506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44faa4f60 0x55c44fe8f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.089+0000 7ff1e3fff640 1 -- 172.21.15.38:0/1197354506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44fe902c0 con 0x55c44fabd4d0 2024-05-21T15:40:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.089+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/1197354506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fabd4d0 0x55c44fe96b10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40c4e20 tx=0x7ff1d403b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f615e32c55aabc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.089+0000 7ff1e0ff9640 1 -- 172.21.15.38:0/1197354506 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4049040 con 0x55c44fabd4d0 2024-05-21T15:40:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.090+0000 7ff1e0ff9640 1 -- 172.21.15.38:0/1197354506 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1d40bc0a0 con 0x55c44fabd4d0 2024-05-21T15:40:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.090+0000 7ff1e0ff9640 1 -- 172.21.15.38:0/1197354506 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d40bc300 con 0x55c44fabd4d0 2024-05-21T15:40:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.090+0000 7ff1f83b2740 1 -- 172.21.15.38:0/1197354506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fabd4d0 msgr2=0x55c44fe96b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.090+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/1197354506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fabd4d0 0x55c44fe96b10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40c4e20 tx=0x7ff1d403b6b0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.091+0000 7ff1f83b2740 1 -- 172.21.15.38:0/1197354506 shutdown_connections 2024-05-21T15:40:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.091+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/1197354506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fac56b0 0x55c44fe8f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.091+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/1197354506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fabd4d0 0x55c44fe96b10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.091+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/1197354506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44faa4f60 0x55c44fe8f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.091+0000 7ff1f83b2740 1 -- 172.21.15.38:0/1197354506 >> 172.21.15.38:0/1197354506 conn(0x55c44fada4e0 msgr2=0x55c44fe8dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.091+0000 7ff1f83b2740 1 -- 172.21.15.38:0/1197354506 shutdown_connections 2024-05-21T15:40:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.091+0000 7ff1f83b2740 1 -- 172.21.15.38:0/1197354506 wait complete. 2024-05-21T15:40:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.091+0000 7ff1f83b2740 1 -- start start 2024-05-21T15:40:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.091+0000 7ff1f83b2740 1 Processor -- start 2024-05-21T15:40:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.091+0000 7ff1f83b2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44faa4f60 0x55c44fe93e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.091+0000 7ff1f83b2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fabd4d0 0x55c44fe94360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1f83b2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fac56b0 0x55c44fe95290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1f83b2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1adac0 con 0x55c44faa4f60 2024-05-21T15:40:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1f83b2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1ad3c0 con 0x55c44fabd4d0 2024-05-21T15:40:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1f83b2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1aa600 con 0x55c44fac56b0 2024-05-21T15:40:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1eaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44faa4f60 0x55c44fe93e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1f83b2740 10 client.0 _ll_register_callbacks cb 0x7ff1dc0040f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1eaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44faa4f60 0x55c44fe93e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33744/0 (socket says 172.21.15.38:33744) 2024-05-21T15:40:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1eaffd640 1 -- 172.21.15.38:0/2192556923 learned_addr learned my addr 172.21.15.38:0/2192556923 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/2192556923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fabd4d0 0x55c44fe94360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1eb7fe640 1 --2- 172.21.15.38:0/2192556923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fac56b0 0x55c44fe95290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1eaffd640 1 -- 172.21.15.38:0/2192556923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fac56b0 msgr2=0x55c44fe95290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1eaffd640 1 --2- 172.21.15.38:0/2192556923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fac56b0 0x55c44fe95290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1eaffd640 1 -- 172.21.15.38:0/2192556923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fabd4d0 msgr2=0x55c44fe94360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1eaffd640 1 --2- 172.21.15.38:0/2192556923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fabd4d0 0x55c44fe94360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1eaffd640 1 -- 172.21.15.38:0/2192556923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44fe95850 con 0x55c44faa4f60 2024-05-21T15:40:50.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/2192556923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fabd4d0 0x55c44fe94360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:50.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1eaffd640 1 --2- 172.21.15.38:0/2192556923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44faa4f60 0x55c44fe93e20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0092e0 tx=0x7ff1dc004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e120537e9fbf48 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:50.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.092+0000 7ff1d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:50.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.093+0000 7ff1d27fc640 1 -- 172.21.15.38:0/2192556923 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00fe50 con 0x55c44faa4f60 2024-05-21T15:40:50.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.093+0000 7ff1d27fc640 1 -- 172.21.15.38:0/2192556923 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff1dc023040 con 0x55c44faa4f60 2024-05-21T15:40:50.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.093+0000 7ff1f83b2740 1 -- 172.21.15.38:0/2192556923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1d40bc0a0 con 0x55c44faa4f60 2024-05-21T15:40:50.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.093+0000 7ff1d27fc640 1 -- 172.21.15.38:0/2192556923 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff1dc01d740 con 0x55c44faa4f60 2024-05-21T15:40:50.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.093+0000 7ff1f83b2740 10 client.17214 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:50.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.093+0000 7ff1f83b2740 1 -- 172.21.15.38:0/2192556923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c44fe96b40 con 0x55c44faa4f60 2024-05-21T15:40:50.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.093+0000 7ff1d27fc640 1 -- 172.21.15.38:0/2192556923 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1d40bc0a0 con 0x55c44faa4f60 2024-05-21T15:40:50.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.093+0000 7ff1d27fc640 1 -- 172.21.15.38:0/2192556923 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7ff1dc01a040 con 0x55c44faa4f60 2024-05-21T15:40:50.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.094+0000 7ff1f83b2740 10 client.17214 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:50.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.094+0000 7ff1f83b2740 10 client.17214 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:40:50.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.094+0000 7ff1f83b2740 20 client.17214 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:50.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.094+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/2192556923 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55c44fae6390 0x55c44ff07fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.094+0000 7ff1f83b2740 4 client.17214 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:40:50.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.094+0000 7ff1f83b2740 1 -- 172.21.15.38:0/2192556923 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c44fe898f0 con 0x55c44fae6390 2024-05-21T15:40:50.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.094+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/2192556923 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55c44fae6390 0x55c44ff07fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.094+0000 7ff1e3fff640 1 --2- 172.21.15.38:0/2192556923 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55c44fae6390 0x55c44ff07fb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0ea483ba6b2aa12 server_cookie=efbfc8ae71251600 in_seq=0 out_seq=0 2024-05-21T15:40:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.094+0000 7ff1d27fc640 10 client.17214 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:40:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.094+0000 7ff1d27fc640 1 -- 172.21.15.38:0/2192556923 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c44fe898f0 con 0x55c44fae6390 2024-05-21T15:40:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.094+0000 7ff1d27fc640 10 client.17214 handle_command_reply: tid=0 2024-05-21T15:40:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.094+0000 7ff1f83b2740 1 client.17214 shutdown 2024-05-21T15:40:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.095+0000 7ff1f83b2740 1 -- 172.21.15.38:0/2192556923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44faa4f60 msgr2=0x55c44fe93e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.095+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/2192556923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44faa4f60 0x55c44fe93e20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0092e0 tx=0x7ff1dc004560 comp rx=0 tx=0).stop 2024-05-21T15:40:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.095+0000 7ff1f83b2740 1 -- 172.21.15.38:0/2192556923 shutdown_connections 2024-05-21T15:40:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.095+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/2192556923 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55c44fae6390 0x55c44ff07fb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.095+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/2192556923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c44fac56b0 0x55c44fe95290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.095+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/2192556923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c44fabd4d0 0x55c44fe94360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.095+0000 7ff1f83b2740 1 --2- 172.21.15.38:0/2192556923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c44faa4f60 0x55c44fe93e20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.095+0000 7ff1f83b2740 1 -- 172.21.15.38:0/2192556923 >> 172.21.15.38:0/2192556923 conn(0x55c44fada4e0 msgr2=0x55c44fe8f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.095+0000 7ff1d27fc640 0 client.17214 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:40:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.095+0000 7ff1f83b2740 1 -- 172.21.15.38:0/2192556923 shutdown_connections 2024-05-21T15:40:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.095+0000 7ff1f83b2740 1 -- 172.21.15.38:0/2192556923 wait complete. 2024-05-21T15:40:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.095+0000 7ff1f83b2740 20 client.17214 trim_cache size 0 max 16384 2024-05-21T15:40:50.097 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:40:50.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.096+0000 7ff1f1509640 1 -- 172.21.15.38:0/370426884 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff1d80a2c10 msgr2=0x7ff1d80a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.096+0000 7ff1f1509640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff1d80a2c10 0x7ff1d80a5120 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec075710 tx=0x7ff1dc005f90 comp rx=0 tx=0).stop 2024-05-21T15:40:50.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.096+0000 7ff1f1509640 1 -- 172.21.15.38:0/370426884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1ec074830 msgr2=0x7ff1ec19b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.096+0000 7ff1f1509640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1ec074830 0x7ff1ec19b9b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4006010 tx=0x7ff1d40042f0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.096+0000 7ff1f1509640 1 -- 172.21.15.38:0/370426884 shutdown_connections 2024-05-21T15:40:50.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.096+0000 7ff1f1509640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1ec10aa60 0x7ff1ec1a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.096+0000 7ff1f1509640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff1d80a2c10 0x7ff1d80a5120 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.096+0000 7ff1f1509640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1ec074830 0x7ff1ec19b9b0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.096+0000 7ff1f1509640 1 --2- 172.21.15.38:0/370426884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1ec073ee0 0x7ff1ec19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.096+0000 7ff1f1509640 1 -- 172.21.15.38:0/370426884 >> 172.21.15.38:0/370426884 conn(0x7ff1ec0fdc80 msgr2=0x7ff1ec0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:50.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.096+0000 7ff1f1509640 1 -- 172.21.15.38:0/370426884 shutdown_connections 2024-05-21T15:40:50.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.096+0000 7ff1f1509640 1 -- 172.21.15.38:0/370426884 wait complete. 2024-05-21T15:40:50.110 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:40:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.266+0000 7ff37656b640 1 -- 172.21.15.38:0/2390854121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3701078d0 msgr2=0x7ff370109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.266+0000 7ff37656b640 1 --2- 172.21.15.38:0/2390854121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3701078d0 0x7ff370109cf0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff368009ca0 tx=0x7ff36802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.266+0000 7ff37656b640 1 -- 172.21.15.38:0/2390854121 shutdown_connections 2024-05-21T15:40:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.266+0000 7ff37656b640 1 --2- 172.21.15.38:0/2390854121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3701078d0 0x7ff370109cf0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.266+0000 7ff37656b640 1 --2- 172.21.15.38:0/2390854121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff370102bd0 0x7ff370102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.266+0000 7ff37656b640 1 --2- 172.21.15.38:0/2390854121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff370101990 0x7ff370102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.266+0000 7ff37656b640 1 -- 172.21.15.38:0/2390854121 >> 172.21.15.38:0/2390854121 conn(0x7ff3700fcb60 msgr2=0x7ff3700ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.266+0000 7ff37656b640 1 -- 172.21.15.38:0/2390854121 shutdown_connections 2024-05-21T15:40:50.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.266+0000 7ff37656b640 1 -- 172.21.15.38:0/2390854121 wait complete. 2024-05-21T15:40:50.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.267+0000 7ff37656b640 1 Processor -- start 2024-05-21T15:40:50.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.267+0000 7ff37656b640 1 -- start start 2024-05-21T15:40:50.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.268+0000 7ff37656b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff370101990 0x7ff370198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.268+0000 7ff37656b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff370102bd0 0x7ff370198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.268+0000 7ff37656b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3701078d0 0x7ff37019f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.268+0000 7ff37656b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010d5f0 con 0x7ff370101990 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.268+0000 7ff37656b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010d470 con 0x7ff3701078d0 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.268+0000 7ff37656b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010d770 con 0x7ff370102bd0 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.268+0000 7ff36ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff370101990 0x7ff370198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.268+0000 7ff36ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff370101990 0x7ff370198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33756/0 (socket says 172.21.15.38:33756) 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.268+0000 7ff36ffff640 1 -- 172.21.15.38:0/2330982107 learned_addr learned my addr 172.21.15.38:0/2330982107 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.268+0000 7ff36f7fe640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff370102bd0 0x7ff370198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.268+0000 7ff374b08640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3701078d0 0x7ff37019f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.269+0000 7ff36ffff640 1 -- 172.21.15.38:0/2330982107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff370102bd0 msgr2=0x7ff370198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.269+0000 7ff36ffff640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff370102bd0 0x7ff370198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.269+0000 7ff36ffff640 1 -- 172.21.15.38:0/2330982107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3701078d0 msgr2=0x7ff37019f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.269+0000 7ff36ffff640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3701078d0 0x7ff37019f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.269+0000 7ff36ffff640 1 -- 172.21.15.38:0/2330982107 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37006a4d0 con 0x7ff370101990 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.269+0000 7ff374b08640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3701078d0 0x7ff37019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.269+0000 7ff36f7fe640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff370102bd0 0x7ff370198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:50.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.269+0000 7ff36ffff640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff370101990 0x7ff370198100 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff36000e9f0 tx=0x7ff36000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.269+0000 7ff36d7fa640 1 -- 172.21.15.38:0/2330982107 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36000ccc0 con 0x7ff370101990 2024-05-21T15:40:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.269+0000 7ff37656b640 1 -- 172.21.15.38:0/2330982107 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37006a7c0 con 0x7ff370101990 2024-05-21T15:40:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.269+0000 7ff36d7fa640 1 -- 172.21.15.38:0/2330982107 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3600035f0 con 0x7ff370101990 2024-05-21T15:40:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.269+0000 7ff36d7fa640 1 -- 172.21.15.38:0/2330982107 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36000ce60 con 0x7ff370101990 2024-05-21T15:40:50.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.269+0000 7ff37656b640 1 -- 172.21.15.38:0/2330982107 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37006ad60 con 0x7ff370101990 2024-05-21T15:40:50.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.273+0000 7ff37d40c740 1 Processor -- start 2024-05-21T15:40:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.273+0000 7ff37d40c740 1 -- start start 2024-05-21T15:40:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.273+0000 7ff37d40c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7d3390 0x55d0dcb72b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.273+0000 7ff37d40c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7aa4d0 0x55d0dcb73080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.273+0000 7ff37d40c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb79f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.273+0000 7ff37d40c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d0dcad7c40 con 0x55d0dc7d3390 2024-05-21T15:40:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.273+0000 7ff37d40c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d0dca7f580 con 0x55d0dc7b26b0 2024-05-21T15:40:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.273+0000 7ff37d40c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d0dcad3c00 con 0x55d0dc7aa4d0 2024-05-21T15:40:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.273+0000 7ff374b08640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7d3390 0x55d0dcb72b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.273+0000 7ff36f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb79f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.273+0000 7ff374b08640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7d3390 0x55d0dcb72b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33772/0 (socket says 172.21.15.38:33772) 2024-05-21T15:40:50.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.273+0000 7ff374b08640 1 -- 172.21.15.38:0/771156232 learned_addr learned my addr 172.21.15.38:0/771156232 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:50.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.274+0000 7ff36f7fe640 1 -- 172.21.15.38:0/771156232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7aa4d0 msgr2=0x55d0dcb73080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:40:50.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.274+0000 7ff36f7fe640 1 --2- 172.21.15.38:0/771156232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7aa4d0 0x55d0dcb73080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.274+0000 7ff36f7fe640 1 -- 172.21.15.38:0/771156232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7d3390 msgr2=0x55d0dcb72b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.274+0000 7ff36f7fe640 1 --2- 172.21.15.38:0/771156232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7d3390 0x55d0dcb72b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.274+0000 7ff36f7fe640 1 -- 172.21.15.38:0/771156232 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0dc7eb3f0 con 0x55d0dc7b26b0 2024-05-21T15:40:50.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.274+0000 7ff36d7fa640 1 -- 172.21.15.38:0/2330982107 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff3600037a0 con 0x7ff370101990 2024-05-21T15:40:50.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.274+0000 7ff36f7fe640 1 --2- 172.21.15.38:0/771156232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb79f80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff364009fa0 tx=0x7ff3640118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a8b72cc680f27ad server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:50.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.275+0000 7ff3567fc640 1 -- 172.21.15.38:0/771156232 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364004030 con 0x55d0dc7b26b0 2024-05-21T15:40:50.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.275+0000 7ff3567fc640 1 -- 172.21.15.38:0/771156232 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff364005500 con 0x55d0dc7b26b0 2024-05-21T15:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.275+0000 7ff36d7fa640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3580a2c10 0x7ff3580a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.275+0000 7ff36d7fa640 1 -- 172.21.15.38:0/2330982107 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff3600cf4a0 con 0x7ff370101990 2024-05-21T15:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.276+0000 7ff3567fc640 1 -- 172.21.15.38:0/771156232 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364004e10 con 0x55d0dc7b26b0 2024-05-21T15:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.276+0000 7ff37d40c740 1 -- 172.21.15.38:0/771156232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7b26b0 msgr2=0x55d0dcb79f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.276+0000 7ff37d40c740 1 --2- 172.21.15.38:0/771156232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb79f80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff364009fa0 tx=0x7ff3640118d0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.276+0000 7ff374b08640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3580a2c10 0x7ff3580a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.276+0000 7ff37d40c740 1 -- 172.21.15.38:0/771156232 shutdown_connections 2024-05-21T15:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.276+0000 7ff37d40c740 1 --2- 172.21.15.38:0/771156232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb79f80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.276+0000 7ff37d40c740 1 --2- 172.21.15.38:0/771156232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7aa4d0 0x55d0dcb73080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.276+0000 7ff37d40c740 1 --2- 172.21.15.38:0/771156232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7d3390 0x55d0dcb72b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.276+0000 7ff37d40c740 1 -- 172.21.15.38:0/771156232 >> 172.21.15.38:0/771156232 conn(0x55d0dc7c74e0 msgr2=0x55d0dcb6e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.277+0000 7ff37d40c740 1 -- 172.21.15.38:0/771156232 shutdown_connections 2024-05-21T15:40:50.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.277+0000 7ff37d40c740 1 -- 172.21.15.38:0/771156232 wait complete. 2024-05-21T15:40:50.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.277+0000 7ff374b08640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3580a2c10 0x7ff3580a5120 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff3680094e0 tx=0x7ff368031bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:50.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.278+0000 7ff37d40c740 1 -- start start 2024-05-21T15:40:50.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.278+0000 7ff37d40c740 1 Processor -- start 2024-05-21T15:40:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.279+0000 7ff37d40c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7aa4d0 0x55d0dcb7ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.279+0000 7ff37d40c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb7cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.279+0000 7ff36ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7aa4d0 0x55d0dcb7ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.279+0000 7ff36ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7aa4d0 0x55d0dcb7ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33788/0 (socket says 172.21.15.38:33788) 2024-05-21T15:40:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.279+0000 7ff37d40c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7d3390 0x55d0dcb83ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.279+0000 7ff36ffff640 1 -- 172.21.15.38:0/3504386544 learned_addr learned my addr 172.21.15.38:0/3504386544 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.279+0000 7ff37d40c740 1 -- 172.21.15.38:0/3504386544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d0dcb7aa20 con 0x55d0dc7aa4d0 2024-05-21T15:40:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.279+0000 7ff37d40c740 1 -- 172.21.15.38:0/3504386544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d0dca92c40 con 0x55d0dc7d3390 2024-05-21T15:40:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.279+0000 7ff37d40c740 1 -- 172.21.15.38:0/3504386544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d0dcb7aba0 con 0x55d0dc7b26b0 2024-05-21T15:40:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.279+0000 7ff36f7fe640 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb7cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.279+0000 7ff374b08640 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7d3390 0x55d0dcb83ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.279+0000 7ff37d40c740 10 client.0 _ll_register_callbacks cb 0x55d0dc7eabe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.280+0000 7ff36ffff640 1 -- 172.21.15.38:0/3504386544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7b26b0 msgr2=0x55d0dcb7cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.280+0000 7ff36ffff640 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb7cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.280+0000 7ff36ffff640 1 -- 172.21.15.38:0/3504386544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7d3390 msgr2=0x55d0dcb83ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.280+0000 7ff36ffff640 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7d3390 0x55d0dcb83ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.280+0000 7ff36ffff640 1 -- 172.21.15.38:0/3504386544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0dc8b57d0 con 0x55d0dc7aa4d0 2024-05-21T15:40:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.280+0000 7ff374b08640 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7d3390 0x55d0dcb83ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:40:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.280+0000 7ff36f7fe640 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb7cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:50.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.280+0000 7ff36ffff640 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7aa4d0 0x55d0dcb7ca20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff3600980b0 tx=0x7ff36009b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f779ade2cd2afb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:50.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.280+0000 7ff34f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:50.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.280+0000 7ff34f7fe640 1 -- 172.21.15.38:0/3504386544 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3600b1070 con 0x55d0dc7aa4d0 2024-05-21T15:40:50.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.280+0000 7ff37d40c740 1 -- 172.21.15.38:0/3504386544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff364005500 con 0x55d0dc7aa4d0 2024-05-21T15:40:50.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.280+0000 7ff34f7fe640 1 -- 172.21.15.38:0/3504386544 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff360003b10 con 0x55d0dc7aa4d0 2024-05-21T15:40:50.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.281+0000 7ff34f7fe640 1 -- 172.21.15.38:0/3504386544 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3600a0000 con 0x55d0dc7aa4d0 2024-05-21T15:40:50.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.281+0000 7ff37d40c740 10 client.17226 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:50.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.281+0000 7ff37d40c740 1 -- 172.21.15.38:0/3504386544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d0dcb87280 con 0x55d0dc7aa4d0 2024-05-21T15:40:50.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.281+0000 7ff34f7fe640 1 -- 172.21.15.38:0/3504386544 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff364005500 con 0x55d0dc7aa4d0 2024-05-21T15:40:50.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.281+0000 7ff34f7fe640 1 -- 172.21.15.38:0/3504386544 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7ff36009b0a0 con 0x55d0dc7aa4d0 2024-05-21T15:40:50.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.281+0000 7ff37d40c740 10 client.17226 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:50.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.281+0000 7ff37d40c740 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:50.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.281+0000 7ff37d40c740 20 client.17226 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:50.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.281+0000 7ff37d40c740 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d0dc791f60 0x55d0dcb8a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.281+0000 7ff37d40c740 4 client.17226 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:50.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.281+0000 7ff37d40c740 1 -- 172.21.15.38:0/3504386544 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0dcae5370 con 0x55d0dc791f60 2024-05-21T15:40:50.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.281+0000 7ff36f7fe640 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d0dc791f60 0x55d0dcb8a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.282+0000 7ff34f7fe640 10 client.17226 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:50.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.282+0000 7ff36f7fe640 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d0dc791f60 0x55d0dcb8a960 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13df0963e04738f1 server_cookie=da0007c207fdf42e in_seq=0 out_seq=0 2024-05-21T15:40:50.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.283+0000 7ff34f7fe640 1 -- 172.21.15.38:0/3504386544 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d0dcae5370 con 0x55d0dc791f60 2024-05-21T15:40:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.283+0000 7ff34f7fe640 10 client.17226 handle_command_reply: tid=0 2024-05-21T15:40:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.283+0000 7ff37d40c740 1 client.17226 shutdown 2024-05-21T15:40:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.283+0000 7ff37d40c740 1 -- 172.21.15.38:0/3504386544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7aa4d0 msgr2=0x55d0dcb7ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.283+0000 7ff37d40c740 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7aa4d0 0x55d0dcb7ca20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff3600980b0 tx=0x7ff36009b670 comp rx=0 tx=0).stop 2024-05-21T15:40:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.283+0000 7ff37d40c740 1 -- 172.21.15.38:0/3504386544 shutdown_connections 2024-05-21T15:40:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.283+0000 7ff37d40c740 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7d3390 0x55d0dcb83ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.283+0000 7ff37d40c740 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb7cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.283+0000 7ff37d40c740 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d0dc791f60 0x55d0dcb8a960 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.283+0000 7ff34f7fe640 0 client.17226 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:50.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.283+0000 7ff37d40c740 1 --2- 172.21.15.38:0/3504386544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7aa4d0 0x55d0dcb7ca20 secure :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff3600980b0 tx=0x7ff36009b670 comp rx=0 tx=0).stop 2024-05-21T15:40:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.283+0000 7ff37d40c740 1 -- 172.21.15.38:0/3504386544 >> 172.21.15.38:0/3504386544 conn(0x55d0dc7c74e0 msgr2=0x55d0dcb6d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.284+0000 7ff37d40c740 1 -- 172.21.15.38:0/3504386544 shutdown_connections 2024-05-21T15:40:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.284+0000 7ff37d40c740 1 -- 172.21.15.38:0/3504386544 wait complete. 2024-05-21T15:40:50.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.284+0000 7ff37d40c740 20 client.17226 trim_cache size 0 max 16384 2024-05-21T15:40:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.305+0000 7ff37d40c740 1 Processor -- start 2024-05-21T15:40:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.305+0000 7ff37d40c740 1 -- start start 2024-05-21T15:40:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.305+0000 7ff37d40c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc791f60 0x55d0dcb7b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.305+0000 7ff37d40c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb7c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff37d40c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7d3390 0x55d0dcb7cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff37d40c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701ac590 con 0x55d0dc7d3390 2024-05-21T15:40:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff37d40c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701ae160 con 0x55d0dc7b26b0 2024-05-21T15:40:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff37d40c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701acc60 con 0x55d0dc791f60 2024-05-21T15:40:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff374b08640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7d3390 0x55d0dcb7cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff36ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc791f60 0x55d0dcb7b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff36ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc791f60 0x55d0dcb7b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:51870/0 (socket says 172.21.15.38:51870) 2024-05-21T15:40:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff374b08640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7d3390 0x55d0dcb7cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33802/0 (socket says 172.21.15.38:33802) 2024-05-21T15:40:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff36ffff640 1 -- 172.21.15.38:0/3549012435 learned_addr learned my addr 172.21.15.38:0/3549012435 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff36f7fe640 1 --2- 172.21.15.38:0/3549012435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb7c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff36ffff640 1 -- 172.21.15.38:0/3549012435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7b26b0 msgr2=0x55d0dcb7c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff36ffff640 1 --2- 172.21.15.38:0/3549012435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb7c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff36ffff640 1 -- 172.21.15.38:0/3549012435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7d3390 msgr2=0x55d0dcb7cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff36ffff640 1 --2- 172.21.15.38:0/3549012435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7d3390 0x55d0dcb7cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff36ffff640 1 -- 172.21.15.38:0/3549012435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0dcb69d20 con 0x55d0dc791f60 2024-05-21T15:40:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff374b08640 1 --2- 172.21.15.38:0/3549012435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7d3390 0x55d0dcb7cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:50.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.306+0000 7ff36ffff640 1 --2- 172.21.15.38:0/3549012435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc791f60 0x55d0dcb7b730 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c6a0 tx=0x7ff36009a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9ec74ba9e17e8e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:50.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.307+0000 7ff355ffb640 1 -- 172.21.15.38:0/3549012435 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3600b1070 con 0x55d0dc791f60 2024-05-21T15:40:50.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.307+0000 7ff355ffb640 1 -- 172.21.15.38:0/3549012435 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3600b4e50 con 0x55d0dc791f60 2024-05-21T15:40:50.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.307+0000 7ff355ffb640 1 -- 172.21.15.38:0/3549012435 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3600a4430 con 0x55d0dc791f60 2024-05-21T15:40:50.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.308+0000 7ff37d40c740 1 -- 172.21.15.38:0/3549012435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc791f60 msgr2=0x55d0dcb7b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.308+0000 7ff37d40c740 1 --2- 172.21.15.38:0/3549012435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc791f60 0x55d0dcb7b730 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c6a0 tx=0x7ff36009a160 comp rx=0 tx=0).stop 2024-05-21T15:40:50.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.308+0000 7ff37d40c740 1 -- 172.21.15.38:0/3549012435 shutdown_connections 2024-05-21T15:40:50.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.308+0000 7ff37d40c740 1 --2- 172.21.15.38:0/3549012435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc7d3390 0x55d0dcb7cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.308+0000 7ff37d40c740 1 --2- 172.21.15.38:0/3549012435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb7c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.308+0000 7ff37d40c740 1 --2- 172.21.15.38:0/3549012435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc791f60 0x55d0dcb7b730 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.308+0000 7ff37d40c740 1 -- 172.21.15.38:0/3549012435 >> 172.21.15.38:0/3549012435 conn(0x55d0dc7c74e0 msgr2=0x55d0dcb7b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:50.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.308+0000 7ff37d40c740 1 -- 172.21.15.38:0/3549012435 shutdown_connections 2024-05-21T15:40:50.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.308+0000 7ff37d40c740 1 -- 172.21.15.38:0/3549012435 wait complete. 2024-05-21T15:40:50.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.308+0000 7ff37d40c740 1 -- start start 2024-05-21T15:40:50.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.309+0000 7ff37d40c740 1 Processor -- start 2024-05-21T15:40:50.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.309+0000 7ff37d40c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc791f60 0x55d0dcb83580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.309+0000 7ff37d40c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb80150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.309+0000 7ff36ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc791f60 0x55d0dcb83580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.309+0000 7ff36ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc791f60 0x55d0dcb83580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33814/0 (socket says 172.21.15.38:33814) 2024-05-21T15:40:50.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.309+0000 7ff37d40c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7d3390 0x55d0dcb807f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.309+0000 7ff37d40c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701ada30 con 0x55d0dc791f60 2024-05-21T15:40:50.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.310+0000 7ff37d40c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701ad330 con 0x55d0dc7d3390 2024-05-21T15:40:50.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.310+0000 7ff37d40c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701aa570 con 0x55d0dc7b26b0 2024-05-21T15:40:50.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.310+0000 7ff36ffff640 1 -- 172.21.15.38:0/810700518 learned_addr learned my addr 172.21.15.38:0/810700518 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:50.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.310+0000 7ff37d40c740 10 client.0 _ll_register_callbacks cb 0x7ff36009bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:50.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.310+0000 7ff36f7fe640 1 --2- 172.21.15.38:0/810700518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb80150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.310+0000 7ff374b08640 1 --2- 172.21.15.38:0/810700518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7d3390 0x55d0dcb807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.310+0000 7ff36ffff640 1 -- 172.21.15.38:0/810700518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7b26b0 msgr2=0x55d0dcb80150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.310+0000 7ff36ffff640 1 --2- 172.21.15.38:0/810700518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb80150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.310+0000 7ff36ffff640 1 -- 172.21.15.38:0/810700518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7d3390 msgr2=0x55d0dcb807f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:50.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.310+0000 7ff36ffff640 1 --2- 172.21.15.38:0/810700518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7d3390 0x55d0dcb807f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.310+0000 7ff36ffff640 1 -- 172.21.15.38:0/810700518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0dcbf0d80 con 0x55d0dc791f60 2024-05-21T15:40:50.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.310+0000 7ff36ffff640 1 --2- 172.21.15.38:0/810700518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc791f60 0x55d0dcb83580 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff36009bbe0 tx=0x7ff36009b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d140deceb140a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:50.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.310+0000 7ff34f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:50.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.311+0000 7ff34f7fe640 1 -- 172.21.15.38:0/810700518 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3600a8020 con 0x55d0dc791f60 2024-05-21T15:40:50.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.311+0000 7ff34f7fe640 1 -- 172.21.15.38:0/810700518 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff3600a4640 con 0x55d0dc791f60 2024-05-21T15:40:50.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.311+0000 7ff37d40c740 1 -- 172.21.15.38:0/810700518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3600b4e50 con 0x55d0dc791f60 2024-05-21T15:40:50.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.311+0000 7ff34f7fe640 1 -- 172.21.15.38:0/810700518 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3600b1070 con 0x55d0dc791f60 2024-05-21T15:40:50.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.311+0000 7ff37d40c740 10 client.17232 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:50.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.311+0000 7ff37d40c740 1 -- 172.21.15.38:0/810700518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d0dcbf0d80 con 0x55d0dc791f60 2024-05-21T15:40:50.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.311+0000 7ff34f7fe640 1 -- 172.21.15.38:0/810700518 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3600a48c0 con 0x55d0dc791f60 2024-05-21T15:40:50.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.312+0000 7ff34f7fe640 1 -- 172.21.15.38:0/810700518 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7ff36009ab10 con 0x55d0dc791f60 2024-05-21T15:40:50.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.312+0000 7ff37d40c740 10 client.17232 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:50.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.312+0000 7ff37d40c740 10 client.17232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:50.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.312+0000 7ff37d40c740 20 client.17232 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:50.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.312+0000 7ff37d40c740 1 --2- 172.21.15.38:0/810700518 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d0dc7673a0 0x55d0dcbf4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:50.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.312+0000 7ff37d40c740 4 client.17232 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:40:50.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.312+0000 7ff37d40c740 1 -- 172.21.15.38:0/810700518 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff3600b4e50 con 0x55d0dc7673a0 2024-05-21T15:40:50.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.312+0000 7ff36f7fe640 1 --2- 172.21.15.38:0/810700518 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d0dc7673a0 0x55d0dcbf4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:50.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.312+0000 7ff34f7fe640 10 client.17232 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:50.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.312+0000 7ff36f7fe640 1 --2- 172.21.15.38:0/810700518 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d0dc7673a0 0x55d0dcbf4580 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20fe03de05685d50 server_cookie=233a23f16f335e18 in_seq=0 out_seq=0 2024-05-21T15:40:50.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff34f7fe640 1 -- 172.21.15.38:0/810700518 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff3600b4e50 con 0x55d0dc7673a0 2024-05-21T15:40:50.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff34f7fe640 10 client.17232 handle_command_reply: tid=0 2024-05-21T15:40:50.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff37d40c740 1 client.17232 shutdown 2024-05-21T15:40:50.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff37d40c740 1 -- 172.21.15.38:0/810700518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc791f60 msgr2=0x55d0dcb83580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff37d40c740 1 --2- 172.21.15.38:0/810700518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc791f60 0x55d0dcb83580 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff36009bbe0 tx=0x7ff36009b790 comp rx=0 tx=0).stop 2024-05-21T15:40:50.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff37d40c740 1 -- 172.21.15.38:0/810700518 shutdown_connections 2024-05-21T15:40:50.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff37d40c740 1 --2- 172.21.15.38:0/810700518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0dc7d3390 0x55d0dcb807f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff37d40c740 1 --2- 172.21.15.38:0/810700518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d0dc7b26b0 0x55d0dcb80150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff37d40c740 1 --2- 172.21.15.38:0/810700518 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55d0dc7673a0 0x55d0dcbf4580 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff34f7fe640 0 client.17232 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:50.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff37d40c740 1 --2- 172.21.15.38:0/810700518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0dc791f60 0x55d0dcb83580 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff37d40c740 1 -- 172.21.15.38:0/810700518 >> 172.21.15.38:0/810700518 conn(0x55d0dc7c74e0 msgr2=0x55d0dcb71330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:50.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff37d40c740 1 -- 172.21.15.38:0/810700518 shutdown_connections 2024-05-21T15:40:50.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff37d40c740 1 -- 172.21.15.38:0/810700518 wait complete. 2024-05-21T15:40:50.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.313+0000 7ff37d40c740 20 client.17232 trim_cache size 0 max 16384 2024-05-21T15:40:50.316 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:40:50.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.314+0000 7ff37656b640 1 -- 172.21.15.38:0/2330982107 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3580a2c10 msgr2=0x7ff3580a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.314+0000 7ff37656b640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3580a2c10 0x7ff3580a5120 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff3680094e0 tx=0x7ff368031bb0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.314+0000 7ff37656b640 1 -- 172.21.15.38:0/2330982107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff370101990 msgr2=0x7ff370198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:50.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.314+0000 7ff37656b640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff370101990 0x7ff370198100 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff36000e9f0 tx=0x7ff36000eeb0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.314+0000 7ff37656b640 1 -- 172.21.15.38:0/2330982107 shutdown_connections 2024-05-21T15:40:50.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.314+0000 7ff37656b640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3701078d0 0x7ff37019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.314+0000 7ff37656b640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff370102bd0 0x7ff370198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.314+0000 7ff37656b640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff3580a2c10 0x7ff3580a5120 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.314+0000 7ff37656b640 1 --2- 172.21.15.38:0/2330982107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff370101990 0x7ff370198100 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:50.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.314+0000 7ff37656b640 1 -- 172.21.15.38:0/2330982107 >> 172.21.15.38:0/2330982107 conn(0x7ff3700fcb60 msgr2=0x7ff3701038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:50.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.314+0000 7ff37656b640 1 -- 172.21.15.38:0/2330982107 shutdown_connections 2024-05-21T15:40:50.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:50.314+0000 7ff37656b640 1 -- 172.21.15.38:0/2330982107 wait complete. 2024-05-21T15:40:51.329 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:40:51.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:40:51.329 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:40:51.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.550+0000 7fd4cbfaa640 1 -- 172.21.15.38:0/3424757114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c410a2c0 msgr2=0x7fd4c410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:51.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.550+0000 7fd4cbfaa640 1 --2- 172.21.15.38:0/3424757114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c410a2c0 0x7fd4c410a6d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009900 tx=0x7fd4bc02a440 comp rx=0 tx=0).stop 2024-05-21T15:40:51.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.552+0000 7fd4cbfaa640 1 -- 172.21.15.38:0/3424757114 shutdown_connections 2024-05-21T15:40:51.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.552+0000 7fd4cbfaa640 1 --2- 172.21.15.38:0/3424757114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4c410efe0 0x7fd4c410f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.552+0000 7fd4cbfaa640 1 --2- 172.21.15.38:0/3424757114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c410a2c0 0x7fd4c410a6d0 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009900 tx=0x7fd4bc02a440 comp rx=0 tx=0).stop 2024-05-21T15:40:51.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.552+0000 7fd4cbfaa640 1 --2- 172.21.15.38:0/3424757114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c41093d0 0x7fd4c4109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.552+0000 7fd4cbfaa640 1 -- 172.21.15.38:0/3424757114 >> 172.21.15.38:0/3424757114 conn(0x7fd4c406f4a0 msgr2=0x7fd4c4071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:51.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.557+0000 7fd4cbfaa640 1 -- 172.21.15.38:0/3424757114 shutdown_connections 2024-05-21T15:40:51.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.559+0000 7fd4cbfaa640 1 -- 172.21.15.38:0/3424757114 wait complete. 2024-05-21T15:40:51.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.560+0000 7fd4cbfaa640 1 Processor -- start 2024-05-21T15:40:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.563+0000 7fd4cbfaa640 1 -- start start 2024-05-21T15:40:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.564+0000 7fd4cbfaa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c41093d0 0x7fd4c4116020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.564+0000 7fd4cbfaa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c410efe0 0x7fd4c41165e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.564+0000 7fd4cbfaa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4c4117470 0x7fd4c4117880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.564+0000 7fd4cbfaa640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c411d640 con 0x7fd4c410efe0 2024-05-21T15:40:51.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.564+0000 7fd4cbfaa640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c411d4c0 con 0x7fd4c4117470 2024-05-21T15:40:51.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.564+0000 7fd4cbfaa640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c411d7c0 con 0x7fd4c41093d0 2024-05-21T15:40:51.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.564+0000 7fd4c9d46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c41093d0 0x7fd4c4116020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.564+0000 7fd4c9545640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c410efe0 0x7fd4c41165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.564+0000 7fd4c9545640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c410efe0 0x7fd4c41165e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33822/0 (socket says 172.21.15.38:33822) 2024-05-21T15:40:51.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.564+0000 7fd4c9545640 1 -- 172.21.15.38:0/2640593808 learned_addr learned my addr 172.21.15.38:0/2640593808 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.567+0000 7fd4c9545640 1 -- 172.21.15.38:0/2640593808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c41093d0 msgr2=0x7fd4c4116020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.567+0000 7fd4c9545640 1 --2- 172.21.15.38:0/2640593808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c41093d0 0x7fd4c4116020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.567+0000 7fd4c9545640 1 -- 172.21.15.38:0/2640593808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4c4117470 msgr2=0x7fd4c4117880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:51.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.567+0000 7fd4ca547640 1 --2- 172.21.15.38:0/2640593808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4c4117470 0x7fd4c4117880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.567+0000 7fd4c9545640 1 --2- 172.21.15.38:0/2640593808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4c4117470 0x7fd4c4117880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.567+0000 7fd4c9545640 1 -- 172.21.15.38:0/2640593808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c4117fb0 con 0x7fd4c410efe0 2024-05-21T15:40:51.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.568+0000 7fd4c9545640 1 --2- 172.21.15.38:0/2640593808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c410efe0 0x7fd4c41165e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc000c00 tx=0x7fd4bc0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:51.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.570+0000 7fd4c2ffd640 1 -- 172.21.15.38:0/2640593808 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc03f070 con 0x7fd4c410efe0 2024-05-21T15:40:51.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.570+0000 7fd4c2ffd640 1 -- 172.21.15.38:0/2640593808 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4bc031810 con 0x7fd4c410efe0 2024-05-21T15:40:51.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.570+0000 7fd4cbfaa640 1 -- 172.21.15.38:0/2640593808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c4118240 con 0x7fd4c410efe0 2024-05-21T15:40:51.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.570+0000 7fd4cbfaa640 1 -- 172.21.15.38:0/2640593808 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c41186e0 con 0x7fd4c410efe0 2024-05-21T15:40:51.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.570+0000 7fd4c2ffd640 1 -- 172.21.15.38:0/2640593808 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc02f7c0 con 0x7fd4c410efe0 2024-05-21T15:40:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.576+0000 7fd4c2ffd640 1 -- 172.21.15.38:0/2640593808 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd4bc02fa20 con 0x7fd4c410efe0 2024-05-21T15:40:51.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.577+0000 7fd4c2ffd640 1 --2- 172.21.15.38:0/2640593808 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd4b00a2da0 0x7fd4b00a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.577+0000 7fd4c2ffd640 1 -- 172.21.15.38:0/2640593808 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fd4bc0f1250 con 0x7fd4c410efe0 2024-05-21T15:40:51.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.578+0000 7fd4c9d46640 1 --2- 172.21.15.38:0/2640593808 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd4b00a2da0 0x7fd4b00a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.578+0000 7fd4c9d46640 1 --2- 172.21.15.38:0/2640593808 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd4b00a2da0 0x7fd4b00a52b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4117070 tx=0x7fd4ac009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:51.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.579+0000 7fd4d2e51740 1 Processor -- start 2024-05-21T15:40:51.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.579+0000 7fd4d2e51740 1 -- start start 2024-05-21T15:40:51.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.579+0000 7fd4d2e51740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e48410 0x561ee21e84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.579+0000 7fd4d2e51740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21e8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.579+0000 7fd4d2e51740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e27730 0x561ee21ef920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.580+0000 7fd4d2e51740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561ee214c990 con 0x561ee1e27730 2024-05-21T15:40:51.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.580+0000 7fd4d2e51740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561ee20f45b0 con 0x561ee1e1f550 2024-05-21T15:40:51.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.580+0000 7fd4d2e51740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561ee2148950 con 0x561ee1e48410 2024-05-21T15:40:51.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.580+0000 7fd4ca547640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e27730 0x561ee21ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.581+0000 7fd4ca547640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e27730 0x561ee21ef920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33836/0 (socket says 172.21.15.38:33836) 2024-05-21T15:40:51.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.581+0000 7fd4ca547640 1 -- 172.21.15.38:0/608774675 learned_addr learned my addr 172.21.15.38:0/608774675 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:51.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.581+0000 7fd4c9d46640 1 --2- 172.21.15.38:0/608774675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e48410 0x561ee21e84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.581+0000 7fd4c9545640 1 --2- 172.21.15.38:0/608774675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21e8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.582+0000 7fd4ca547640 1 -- 172.21.15.38:0/608774675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e48410 msgr2=0x561ee21e84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:51.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.582+0000 7fd4ca547640 1 --2- 172.21.15.38:0/608774675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e48410 0x561ee21e84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.582+0000 7fd4ca547640 1 -- 172.21.15.38:0/608774675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 msgr2=0x561ee21e8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:51.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.582+0000 7fd4ca547640 1 --2- 172.21.15.38:0/608774675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21e8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.582+0000 7fd4ca547640 1 -- 172.21.15.38:0/608774675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ee1f287e0 con 0x561ee1e27730 2024-05-21T15:40:51.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.583+0000 7fd4ca547640 1 --2- 172.21.15.38:0/608774675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e27730 0x561ee21ef920 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400a8a0 tx=0x7fd4b400ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a5e20b9aa20050 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:51.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.583+0000 7fd4bbfff640 1 -- 172.21.15.38:0/608774675 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4015070 con 0x561ee1e27730 2024-05-21T15:40:51.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.583+0000 7fd4bbfff640 1 -- 172.21.15.38:0/608774675 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b400cb40 con 0x561ee1e27730 2024-05-21T15:40:51.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.585+0000 7fd4d2e51740 1 -- 172.21.15.38:0/608774675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e27730 msgr2=0x561ee21ef920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:51.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.585+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/608774675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e27730 0x561ee21ef920 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400a8a0 tx=0x7fd4b400ad60 comp rx=0 tx=0).stop 2024-05-21T15:40:51.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.590+0000 7fd4d2e51740 1 -- 172.21.15.38:0/608774675 shutdown_connections 2024-05-21T15:40:51.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.590+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/608774675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e27730 0x561ee21ef920 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.590+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/608774675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21e8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.590+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/608774675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e48410 0x561ee21e84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.590+0000 7fd4d2e51740 1 -- 172.21.15.38:0/608774675 >> 172.21.15.38:0/608774675 conn(0x561ee1e3c560 msgr2=0x561ee21e3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:51.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.594+0000 7fd4d2e51740 1 -- 172.21.15.38:0/608774675 shutdown_connections 2024-05-21T15:40:51.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.595+0000 7fd4d2e51740 1 -- 172.21.15.38:0/608774675 wait complete. 2024-05-21T15:40:51.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.595+0000 7fd4d2e51740 1 -- start start 2024-05-21T15:40:51.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.595+0000 7fd4d2e51740 1 Processor -- start 2024-05-21T15:40:51.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.597+0000 7fd4d2e51740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21f2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.597+0000 7fd4d2e51740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 0x561ee21f2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.597+0000 7fd4d2e51740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 0x561ee21f9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.597+0000 7fd4d2e51740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561ee21f03c0 con 0x561ee1e48410 2024-05-21T15:40:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.597+0000 7fd4d2e51740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561ee2107b40 con 0x561ee1e1f550 2024-05-21T15:40:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.597+0000 7fd4d2e51740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561ee21f0540 con 0x561ee1e27730 2024-05-21T15:40:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.597+0000 7fd4d2e51740 10 client.0 _ll_register_callbacks cb 0x561ee20fc0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.598+0000 7fd4c9d46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.598+0000 7fd4c9545640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 0x561ee21f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.598+0000 7fd4c9d46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21f2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:51428/0 (socket says 172.21.15.38:51428) 2024-05-21T15:40:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.598+0000 7fd4c9d46640 1 -- 172.21.15.38:0/1309187066 learned_addr learned my addr 172.21.15.38:0/1309187066 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:51.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.599+0000 7fd4c9d46640 1 -- 172.21.15.38:0/1309187066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 msgr2=0x561ee21f2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:51.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.599+0000 7fd4c9d46640 1 --2- 172.21.15.38:0/1309187066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 0x561ee21f2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.599+0000 7fd4c9d46640 1 -- 172.21.15.38:0/1309187066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 msgr2=0x561ee21f9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:51.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.599+0000 7fd4c9d46640 1 --2- 172.21.15.38:0/1309187066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 0x561ee21f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.599+0000 7fd4c9d46640 1 -- 172.21.15.38:0/1309187066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ee1f88f70 con 0x561ee1e1f550 2024-05-21T15:40:51.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.600+0000 7fd4c9d46640 1 --2- 172.21.15.38:0/1309187066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21f2430 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0042e0 tx=0x7fd4ac004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a592e6320844d77b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:51.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.600+0000 7fd4b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:40:51.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.601+0000 7fd4b97fa640 1 -- 172.21.15.38:0/1309187066 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac024070 con 0x561ee1e1f550 2024-05-21T15:40:51.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.601+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1309187066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4b400cb40 con 0x561ee1e1f550 2024-05-21T15:40:51.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.601+0000 7fd4b97fa640 1 -- 172.21.15.38:0/1309187066 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fd4ac011dd0 con 0x561ee1e1f550 2024-05-21T15:40:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.601+0000 7fd4b97fa640 1 -- 172.21.15.38:0/1309187066 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd4ac030050 con 0x561ee1e1f550 2024-05-21T15:40:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.601+0000 7fd4d2e51740 10 client.26864 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.601+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1309187066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561ee1e20170 con 0x561ee1e1f550 2024-05-21T15:40:51.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.602+0000 7fd4b97fa640 1 -- 172.21.15.38:0/1309187066 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b400cb40 con 0x561ee1e1f550 2024-05-21T15:40:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.604+0000 7fd4b97fa640 1 -- 172.21.15.38:0/1309187066 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fd4ac0285f0 con 0x561ee1e1f550 2024-05-21T15:40:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.604+0000 7fd4d2e51740 10 client.26864 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.604+0000 7fd4d2e51740 10 client.26864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.604+0000 7fd4d2e51740 20 client.26864 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.604+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/1309187066 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561ee1e071f0 0x561ee22002a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.604+0000 7fd4d2e51740 4 client.26864 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.604+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1309187066 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ee215a0c0 con 0x561ee1e071f0 2024-05-21T15:40:51.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.605+0000 7fd4c9545640 1 --2- 172.21.15.38:0/1309187066 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561ee1e071f0 0x561ee22002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.606+0000 7fd4c9545640 1 --2- 172.21.15.38:0/1309187066 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561ee1e071f0 0x561ee22002a0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e18a4a3ef92fc7 server_cookie=390b37c73ac9f3fb in_seq=0 out_seq=0 2024-05-21T15:40:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.606+0000 7fd4b97fa640 10 client.26864 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.608+0000 7fd4b97fa640 1 -- 172.21.15.38:0/1309187066 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561ee215a0c0 con 0x561ee1e071f0 2024-05-21T15:40:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.608+0000 7fd4b97fa640 10 client.26864 handle_command_reply: tid=0 2024-05-21T15:40:51.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.608+0000 7fd4d2e51740 1 client.26864 shutdown 2024-05-21T15:40:51.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.611+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1309187066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 msgr2=0x561ee21f2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:51.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.611+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/1309187066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21f2430 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0042e0 tx=0x7fd4ac004ab0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.612+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1309187066 shutdown_connections 2024-05-21T15:40:51.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.612+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/1309187066 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561ee1e071f0 0x561ee22002a0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.612+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/1309187066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 0x561ee21f9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.612+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/1309187066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 0x561ee21f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.612+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/1309187066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21f2430 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.612+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1309187066 >> 172.21.15.38:0/1309187066 conn(0x561ee1e3c560 msgr2=0x561ee21e2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:51.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.613+0000 7fd4b97fa640 0 client.26864 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:51.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.616+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1309187066 shutdown_connections 2024-05-21T15:40:51.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.618+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1309187066 wait complete. 2024-05-21T15:40:51.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.619+0000 7fd4d2e51740 20 client.26864 trim_cache size 0 max 16384 2024-05-21T15:40:51.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.664+0000 7fd4d2e51740 1 Processor -- start 2024-05-21T15:40:51.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.664+0000 7fd4d2e51740 1 -- start start 2024-05-21T15:40:51.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.664+0000 7fd4d2e51740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21e2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.665+0000 7fd4d2e51740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 0x561ee21f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.665+0000 7fd4d2e51740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 0x561ee21f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.665+0000 7fd4d2e51740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a400e580 con 0x561ee1e48410 2024-05-21T15:40:51.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.665+0000 7fd4c9545640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 0x561ee21f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.665+0000 7fd4d2e51740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a4010150 con 0x561ee1e1f550 2024-05-21T15:40:51.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.665+0000 7fd4d2e51740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a400ec50 con 0x561ee1e27730 2024-05-21T15:40:51.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.665+0000 7fd4c9545640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 0x561ee21f97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:51932/0 (socket says 172.21.15.38:51932) 2024-05-21T15:40:51.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.665+0000 7fd4c9545640 1 -- 172.21.15.38:0/2884132345 learned_addr learned my addr 172.21.15.38:0/2884132345 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:51.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.665+0000 7fd4ca547640 1 --2- 172.21.15.38:0/2884132345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 0x561ee21f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.665+0000 7fd4c9d46640 1 --2- 172.21.15.38:0/2884132345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21e2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.666+0000 7fd4ca547640 1 -- 172.21.15.38:0/2884132345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 msgr2=0x561ee21f97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:51.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.666+0000 7fd4ca547640 1 --2- 172.21.15.38:0/2884132345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 0x561ee21f97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.666+0000 7fd4ca547640 1 -- 172.21.15.38:0/2884132345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 msgr2=0x561ee21e2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:51.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.666+0000 7fd4ca547640 1 --2- 172.21.15.38:0/2884132345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21e2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.666+0000 7fd4ca547640 1 -- 172.21.15.38:0/2884132345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ee21f34a0 con 0x561ee1e48410 2024-05-21T15:40:51.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.666+0000 7fd4ca547640 1 --2- 172.21.15.38:0/2884132345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 0x561ee21f9d30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400a870 tx=0x7fd4b4006be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a04c518d63dd51a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:51.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.667+0000 7fd4bb7fe640 1 -- 172.21.15.38:0/2884132345 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4020070 con 0x561ee1e48410 2024-05-21T15:40:51.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.667+0000 7fd4bb7fe640 1 -- 172.21.15.38:0/2884132345 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b4018760 con 0x561ee1e48410 2024-05-21T15:40:51.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.667+0000 7fd4bb7fe640 1 -- 172.21.15.38:0/2884132345 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4015040 con 0x561ee1e48410 2024-05-21T15:40:51.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.668+0000 7fd4d2e51740 1 -- 172.21.15.38:0/2884132345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 msgr2=0x561ee21f9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:51.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.668+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/2884132345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 0x561ee21f9d30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400a870 tx=0x7fd4b4006be0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.669+0000 7fd4d2e51740 1 -- 172.21.15.38:0/2884132345 shutdown_connections 2024-05-21T15:40:51.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.669+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/2884132345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 0x561ee21f9d30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.669+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/2884132345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 0x561ee21f97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.669+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/2884132345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21e2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.669+0000 7fd4d2e51740 1 -- 172.21.15.38:0/2884132345 >> 172.21.15.38:0/2884132345 conn(0x561ee1e3c560 msgr2=0x561ee21e23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:51.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.669+0000 7fd4d2e51740 1 -- 172.21.15.38:0/2884132345 shutdown_connections 2024-05-21T15:40:51.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.670+0000 7fd4d2e51740 1 -- 172.21.15.38:0/2884132345 wait complete. 2024-05-21T15:40:51.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.670+0000 7fd4d2e51740 1 -- start start 2024-05-21T15:40:51.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.670+0000 7fd4d2e51740 1 Processor -- start 2024-05-21T15:40:51.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.671+0000 7fd4d2e51740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.671+0000 7fd4d2e51740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 0x561ee21f7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.671+0000 7fd4d2e51740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 0x561ee2266640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.671+0000 7fd4d2e51740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a400fa20 con 0x561ee1e48410 2024-05-21T15:40:51.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.671+0000 7fd4d2e51740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a400f320 con 0x561ee1e1f550 2024-05-21T15:40:51.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.671+0000 7fd4d2e51740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a400c610 con 0x561ee1e27730 2024-05-21T15:40:51.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.671+0000 7fd4d2e51740 10 client.0 _ll_register_callbacks cb 0x7fd4ac003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:51.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.672+0000 7fd4c9d46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.672+0000 7fd4c9d46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21f6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:51452/0 (socket says 172.21.15.38:51452) 2024-05-21T15:40:51.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.672+0000 7fd4c9d46640 1 -- 172.21.15.38:0/1138221671 learned_addr learned my addr 172.21.15.38:0/1138221671 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:51.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.672+0000 7fd4ca547640 1 --2- 172.21.15.38:0/1138221671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 0x561ee2266640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.672+0000 7fd4c9d46640 1 -- 172.21.15.38:0/1138221671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 msgr2=0x561ee21f7000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:40:51.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.672+0000 7fd4c9d46640 1 --2- 172.21.15.38:0/1138221671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 0x561ee21f7000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.672+0000 7fd4c9d46640 1 -- 172.21.15.38:0/1138221671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 msgr2=0x561ee2266640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:51.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.672+0000 7fd4c9d46640 1 --2- 172.21.15.38:0/1138221671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 0x561ee2266640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.672+0000 7fd4c9d46640 1 -- 172.21.15.38:0/1138221671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ee21f9fc0 con 0x561ee1e1f550 2024-05-21T15:40:51.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.673+0000 7fd4c9d46640 1 --2- 172.21.15.38:0/1138221671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21f6ac0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00a010 tx=0x7fd4ac028df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=260c36d2bf963aac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:51.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.673+0000 7fd4b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:40:51.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.674+0000 7fd4b97fa640 1 -- 172.21.15.38:0/1138221671 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac024040 con 0x561ee1e1f550 2024-05-21T15:40:51.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.674+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1138221671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4b4018760 con 0x561ee1e1f550 2024-05-21T15:40:51.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.675+0000 7fd4b97fa640 1 -- 172.21.15.38:0/1138221671 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fd4ac011de0 con 0x561ee1e1f550 2024-05-21T15:40:51.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.675+0000 7fd4b97fa640 1 -- 172.21.15.38:0/1138221671 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd4ac0283b0 con 0x561ee1e1f550 2024-05-21T15:40:51.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.675+0000 7fd4d2e51740 10 client.26870 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:51.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.675+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1138221671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561ee2267c00 con 0x561ee1e1f550 2024-05-21T15:40:51.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.676+0000 7fd4b97fa640 1 -- 172.21.15.38:0/1138221671 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b4018760 con 0x561ee1e1f550 2024-05-21T15:40:51.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.676+0000 7fd4b97fa640 1 -- 172.21.15.38:0/1138221671 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fd4ac029ce0 con 0x561ee1e1f550 2024-05-21T15:40:51.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.676+0000 7fd4d2e51740 10 client.26870 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:51.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.676+0000 7fd4d2e51740 10 client.26870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:51.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.676+0000 7fd4d2e51740 20 client.26870 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:51.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.676+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/1138221671 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561ee1ddc630 0x561ee226a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.676+0000 7fd4d2e51740 4 client.26870 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:40:51.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.676+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1138221671 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561ee2154410 con 0x561ee1ddc630 2024-05-21T15:40:51.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.678+0000 7fd4c9545640 1 --2- 172.21.15.38:0/1138221671 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561ee1ddc630 0x561ee226a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.679+0000 7fd4b97fa640 10 client.26870 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:51.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.680+0000 7fd4c9545640 1 --2- 172.21.15.38:0/1138221671 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561ee1ddc630 0x561ee226a630 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=909b41033f27f563 server_cookie=df6ac3d5b7169385 in_seq=0 out_seq=0 2024-05-21T15:40:51.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.683+0000 7fd4b97fa640 1 -- 172.21.15.38:0/1138221671 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561ee2154410 con 0x561ee1ddc630 2024-05-21T15:40:51.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.683+0000 7fd4b97fa640 10 client.26870 handle_command_reply: tid=0 2024-05-21T15:40:51.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.685+0000 7fd4d2e51740 1 client.26870 shutdown 2024-05-21T15:40:51.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.686+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1138221671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 msgr2=0x561ee21f6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:51.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.686+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/1138221671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21f6ac0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00a010 tx=0x7fd4ac028df0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.686+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1138221671 shutdown_connections 2024-05-21T15:40:51.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.686+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/1138221671 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x561ee1ddc630 0x561ee226a630 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.686+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/1138221671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ee1e48410 0x561ee2266640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.686+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/1138221671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561ee1e27730 0x561ee21f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.686+0000 7fd4d2e51740 1 --2- 172.21.15.38:0/1138221671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ee1e1f550 0x561ee21f6ac0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.686+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1138221671 >> 172.21.15.38:0/1138221671 conn(0x561ee1e3c560 msgr2=0x561ee21e23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:51.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.687+0000 7fd4b97fa640 0 client.26870 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:51.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.687+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1138221671 shutdown_connections 2024-05-21T15:40:51.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.688+0000 7fd4d2e51740 1 -- 172.21.15.38:0/1138221671 wait complete. 2024-05-21T15:40:51.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.692+0000 7fd4d2e51740 20 client.26870 trim_cache size 0 max 16384 2024-05-21T15:40:51.692 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:40:51.692 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-05-21T15:40:51.692 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "9cc63c75-2a25-4f2d-9301-a8366dc50ef0", 2024-05-21T15:40:51.692 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-05-21T15:40:51.692 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:40:51.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.695+0000 7fd4cbfaa640 1 -- 172.21.15.38:0/2640593808 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd4b00a2da0 msgr2=0x7fd4b00a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:51.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.695+0000 7fd4cbfaa640 1 --2- 172.21.15.38:0/2640593808 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd4b00a2da0 0x7fd4b00a52b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4117070 tx=0x7fd4ac009500 comp rx=0 tx=0).stop 2024-05-21T15:40:51.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.695+0000 7fd4cbfaa640 1 -- 172.21.15.38:0/2640593808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c410efe0 msgr2=0x7fd4c41165e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:51.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.695+0000 7fd4cbfaa640 1 --2- 172.21.15.38:0/2640593808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c410efe0 0x7fd4c41165e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc000c00 tx=0x7fd4bc0056d0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.696+0000 7fd4cbfaa640 1 -- 172.21.15.38:0/2640593808 shutdown_connections 2024-05-21T15:40:51.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.696+0000 7fd4cbfaa640 1 --2- 172.21.15.38:0/2640593808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4c4117470 0x7fd4c4117880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.696+0000 7fd4cbfaa640 1 --2- 172.21.15.38:0/2640593808 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd4b00a2da0 0x7fd4b00a52b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.696+0000 7fd4cbfaa640 1 --2- 172.21.15.38:0/2640593808 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4c410efe0 0x7fd4c41165e0 secure :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc000c00 tx=0x7fd4bc0056d0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.696+0000 7fd4cbfaa640 1 --2- 172.21.15.38:0/2640593808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c41093d0 0x7fd4c4116020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.696+0000 7fd4cbfaa640 1 -- 172.21.15.38:0/2640593808 >> 172.21.15.38:0/2640593808 conn(0x7fd4c406f4a0 msgr2=0x7fd4c410afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:51.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.699+0000 7fd4cbfaa640 1 -- 172.21.15.38:0/2640593808 shutdown_connections 2024-05-21T15:40:51.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.700+0000 7fd4cbfaa640 1 -- 172.21.15.38:0/2640593808 wait complete. 2024-05-21T15:40:51.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:51 smithi038 ceph-mon[31995]: pgmap v544: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 4.20k op/s 2024-05-21T15:40:51.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:40:51.717 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:40:51.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:51 smithi045 ceph-mon[36409]: pgmap v544: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 4.20k op/s 2024-05-21T15:40:51.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.930+0000 7fa9d00e9640 1 -- 172.21.15.38:0/405695397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c810a2c0 msgr2=0x7fa9c810a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:51.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.930+0000 7fa9d00e9640 1 --2- 172.21.15.38:0/405695397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c810a2c0 0x7fa9c810a6d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009900 tx=0x7fa9b802a420 comp rx=0 tx=0).stop 2024-05-21T15:40:51.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.937+0000 7fa9d00e9640 1 -- 172.21.15.38:0/405695397 shutdown_connections 2024-05-21T15:40:51.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.937+0000 7fa9d00e9640 1 --2- 172.21.15.38:0/405695397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9c810efe0 0x7fa9c810f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.937+0000 7fa9d00e9640 1 --2- 172.21.15.38:0/405695397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c810a2c0 0x7fa9c810a6d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.937+0000 7fa9d00e9640 1 --2- 172.21.15.38:0/405695397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c81093d0 0x7fa9c8109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.937+0000 7fa9d00e9640 1 -- 172.21.15.38:0/405695397 >> 172.21.15.38:0/405695397 conn(0x7fa9c806f4a0 msgr2=0x7fa9c8071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:51.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.940+0000 7fa9d00e9640 1 -- 172.21.15.38:0/405695397 shutdown_connections 2024-05-21T15:40:51.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.940+0000 7fa9d00e9640 1 -- 172.21.15.38:0/405695397 wait complete. 2024-05-21T15:40:51.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.941+0000 7fa9d00e9640 1 Processor -- start 2024-05-21T15:40:51.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.944+0000 7fa9d00e9640 1 -- start start 2024-05-21T15:40:51.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.944+0000 7fa9d00e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c81093d0 0x7fa9c81a8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.944+0000 7fa9d00e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9c810a2c0 0x7fa9c81a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.944+0000 7fa9d00e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c810efe0 0x7fa9c81af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.944+0000 7fa9d00e9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c811d6e0 con 0x7fa9c810efe0 2024-05-21T15:40:51.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.944+0000 7fa9d00e9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c811d560 con 0x7fa9c81093d0 2024-05-21T15:40:51.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.944+0000 7fa9d00e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c811d860 con 0x7fa9c810a2c0 2024-05-21T15:40:51.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.945+0000 7fa9cd684640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9c810a2c0 0x7fa9c81a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.945+0000 7fa9cd684640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9c810a2c0 0x7fa9c81a86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:51936/0 (socket says 172.21.15.38:51936) 2024-05-21T15:40:51.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.945+0000 7fa9cd684640 1 -- 172.21.15.38:0/1208713054 learned_addr learned my addr 172.21.15.38:0/1208713054 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:51.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.946+0000 7fa9ce686640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c810efe0 0x7fa9c81af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.946+0000 7fa9cde85640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c81093d0 0x7fa9c81a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.946+0000 7fa9cd684640 1 -- 172.21.15.38:0/1208713054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c81093d0 msgr2=0x7fa9c81a8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:51.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.946+0000 7fa9cd684640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c81093d0 0x7fa9c81a8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.946+0000 7fa9cd684640 1 -- 172.21.15.38:0/1208713054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c810efe0 msgr2=0x7fa9c81af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:51.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.946+0000 7fa9cd684640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c810efe0 0x7fa9c81af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.946+0000 7fa9cd684640 1 -- 172.21.15.38:0/1208713054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c8110c40 con 0x7fa9c810a2c0 2024-05-21T15:40:51.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.946+0000 7fa9ce686640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c810efe0 0x7fa9c81af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:40:51.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.947+0000 7fa9cd684640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9c810a2c0 0x7fa9c81a86d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa9b802aa10 tx=0x7fa9b8039520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:51.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.949+0000 7fa9beffd640 1 -- 172.21.15.38:0/1208713054 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8040030 con 0x7fa9c810a2c0 2024-05-21T15:40:51.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.950+0000 7fa9d00e9640 1 -- 172.21.15.38:0/1208713054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c8110ed0 con 0x7fa9c810a2c0 2024-05-21T15:40:51.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.950+0000 7fa9d00e9640 1 -- 172.21.15.38:0/1208713054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c8111440 con 0x7fa9c810a2c0 2024-05-21T15:40:51.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.950+0000 7fa9beffd640 1 -- 172.21.15.38:0/1208713054 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b80398a0 con 0x7fa9c810a2c0 2024-05-21T15:40:51.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.950+0000 7fa9beffd640 1 -- 172.21.15.38:0/1208713054 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8035070 con 0x7fa9c810a2c0 2024-05-21T15:40:51.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:51 smithi053 ceph-mon[36496]: pgmap v544: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 4.20k op/s 2024-05-21T15:40:51.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.956+0000 7fa9beffd640 1 -- 172.21.15.38:0/1208713054 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa9b8003830 con 0x7fa9c810a2c0 2024-05-21T15:40:51.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.957+0000 7fa9d6f8c740 1 Processor -- start 2024-05-21T15:40:51.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.960+0000 7fa9d6f8c740 1 -- start start 2024-05-21T15:40:51.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.961+0000 7fa9beffd640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa9b40a2da0 0x7fa9b40a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.961+0000 7fa9beffd640 1 -- 172.21.15.38:0/1208713054 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa9b80f1360 con 0x7fa9c810a2c0 2024-05-21T15:40:51.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.961+0000 7fa9d6f8c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52f12390 0x55bf532b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.962+0000 7fa9d6f8c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52ee94d0 0x55bf532b2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.962+0000 7fa9d6f8c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 0x55bf532b9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.962+0000 7fa9d6f8c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bf53216d00 con 0x55bf52ef16b0 2024-05-21T15:40:51.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.962+0000 7fa9d6f8c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bf531be4c0 con 0x55bf52f12390 2024-05-21T15:40:51.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.962+0000 7fa9ce686640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa9b40a2da0 0x7fa9b40a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.962+0000 7fa9d6f8c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bf53212cc0 con 0x55bf52ee94d0 2024-05-21T15:40:51.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.962+0000 7fa9cd684640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52ee94d0 0x55bf532b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.962+0000 7fa9cd684640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52ee94d0 0x55bf532b2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:51942/0 (socket says 172.21.15.38:51942) 2024-05-21T15:40:51.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.962+0000 7fa9cd684640 1 -- 172.21.15.38:0/2482688483 learned_addr learned my addr 172.21.15.38:0/2482688483 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:51.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.964+0000 7fa9cd684640 1 -- 172.21.15.38:0/2482688483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52f12390 msgr2=0x55bf532b1bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:40:51.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.964+0000 7fa9cd684640 1 --2- 172.21.15.38:0/2482688483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52f12390 0x55bf532b1bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.964+0000 7fa9cd684640 1 -- 172.21.15.38:0/2482688483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 msgr2=0x55bf532b9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:51.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.964+0000 7fa9cd684640 1 --2- 172.21.15.38:0/2482688483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 0x55bf532b9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.964+0000 7fa9cd684640 1 -- 172.21.15.38:0/2482688483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf52ff47d0 con 0x55bf52ee94d0 2024-05-21T15:40:51.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.965+0000 7fa9ce686640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa9b40a2da0 0x7fa9b40a52b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4006fd0 tx=0x7fa9c400a0d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:51.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.965+0000 7fa9cd684640 1 --2- 172.21.15.38:0/2482688483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52ee94d0 0x55bf532b2130 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80ba4e0 tx=0x7fa9b80316b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7b961d4d8a8be28 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:51.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.971+0000 7fa9b3fff640 1 -- 172.21.15.38:0/2482688483 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80d5030 con 0x55bf52ee94d0 2024-05-21T15:40:51.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.971+0000 7fa9b3fff640 1 -- 172.21.15.38:0/2482688483 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b80c1ec0 con 0x55bf52ee94d0 2024-05-21T15:40:51.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.971+0000 7fa9b3fff640 1 -- 172.21.15.38:0/2482688483 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80c62f0 con 0x55bf52ee94d0 2024-05-21T15:40:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.975+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2482688483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52ee94d0 msgr2=0x55bf532b2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:51.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.975+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/2482688483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52ee94d0 0x55bf532b2130 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80ba4e0 tx=0x7fa9b80316b0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.977+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2482688483 shutdown_connections 2024-05-21T15:40:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.977+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/2482688483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 0x55bf532b9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.977+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/2482688483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52ee94d0 0x55bf532b2130 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.977+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/2482688483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52f12390 0x55bf532b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.977+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2482688483 >> 172.21.15.38:0/2482688483 conn(0x55bf52f064e0 msgr2=0x55bf532ad540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:51.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.981+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2482688483 shutdown_connections 2024-05-21T15:40:51.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.983+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2482688483 wait complete. 2024-05-21T15:40:51.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.984+0000 7fa9d6f8c740 1 -- start start 2024-05-21T15:40:51.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.984+0000 7fa9d6f8c740 1 Processor -- start 2024-05-21T15:40:51.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.986+0000 7fa9d6f8c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52ee94d0 0x55bf532bb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.986+0000 7fa9d6f8c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 0x55bf532bbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.986+0000 7fa9d6f8c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52f12390 0x55bf532c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.986+0000 7fa9cd684640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 0x55bf532bbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.986+0000 7fa9cd684640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 0x55bf532bbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33920/0 (socket says 172.21.15.38:33920) 2024-05-21T15:40:51.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.987+0000 7fa9cd684640 1 -- 172.21.15.38:0/2329358313 learned_addr learned my addr 172.21.15.38:0/2329358313 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:51.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.987+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2329358313 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bf53212cc0 con 0x55bf52ef16b0 2024-05-21T15:40:51.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.987+0000 7fa9ce686640 1 --2- 172.21.15.38:0/2329358313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52f12390 0x55bf532c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:51.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.987+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2329358313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bf531d1b90 con 0x55bf52ee94d0 2024-05-21T15:40:51.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.987+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2329358313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55bf532b9840 con 0x55bf52f12390 2024-05-21T15:40:51.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.987+0000 7fa9d6f8c740 10 client.0 _ll_register_callbacks cb 0x55bf5320dda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:51.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.989+0000 7fa9ce686640 1 -- 172.21.15.38:0/2329358313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52ee94d0 msgr2=0x55bf532bb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:51.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.989+0000 7fa9ce686640 1 --2- 172.21.15.38:0/2329358313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52ee94d0 0x55bf532bb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.989+0000 7fa9ce686640 1 -- 172.21.15.38:0/2329358313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 msgr2=0x55bf532bbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:51.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.989+0000 7fa9ce686640 1 --2- 172.21.15.38:0/2329358313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 0x55bf532bbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:51.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.989+0000 7fa9ce686640 1 -- 172.21.15.38:0/2329358313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf52f2a3f0 con 0x55bf52f12390 2024-05-21T15:40:51.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.991+0000 7fa9ce686640 1 --2- 172.21.15.38:0/2329358313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52f12390 0x55bf532c2ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4013680 tx=0x7fa9c4006d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e96a80dcf8d155 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:51.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.991+0000 7fa9b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:40:51.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.991+0000 7fa9b17fa640 1 -- 172.21.15.38:0/2329358313 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4019050 con 0x55bf52f12390 2024-05-21T15:40:51.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.991+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2329358313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b80c1ec0 con 0x55bf52f12390 2024-05-21T15:40:51.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.992+0000 7fa9b17fa640 1 -- 172.21.15.38:0/2329358313 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa9c401d870 con 0x55bf52f12390 2024-05-21T15:40:51.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.992+0000 7fa9b17fa640 1 -- 172.21.15.38:0/2329358313 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa9c401da70 con 0x55bf52f12390 2024-05-21T15:40:51.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.994+0000 7fa9d6f8c740 10 client.26938 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:51.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.994+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2329358313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bf52eea0f0 con 0x55bf52f12390 2024-05-21T15:40:51.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.995+0000 7fa9b17fa640 1 -- 172.21.15.38:0/2329358313 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c4023070 con 0x55bf52f12390 2024-05-21T15:40:51.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.995+0000 7fa9b17fa640 1 -- 172.21.15.38:0/2329358313 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fa9c4027400 con 0x55bf52f12390 2024-05-21T15:40:51.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.997+0000 7fa9d6f8c740 10 client.26938 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:51.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.997+0000 7fa9d6f8c740 10 client.26938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:51.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.997+0000 7fa9d6f8c740 20 client.26938 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:51.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.997+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/2329358313 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf52ed0f60 0x55bf532c9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:51.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.997+0000 7fa9d6f8c740 4 client.26938 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:51.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:51.997+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2329358313 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa9b80c1ec0 con 0x55bf52ed0f60 2024-05-21T15:40:51.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.002+0000 7fa9cde85640 1 --2- 172.21.15.38:0/2329358313 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf52ed0f60 0x55bf532c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:52.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.008+0000 7fa9b17fa640 10 client.26938 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:52.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.008+0000 7fa9cde85640 1 --2- 172.21.15.38:0/2329358313 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf52ed0f60 0x55bf532c9450 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8509e0bb82dd3cb5 server_cookie=2774d3ddb030424c in_seq=0 out_seq=0 2024-05-21T15:40:52.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.013+0000 7fa9b17fa640 1 -- 172.21.15.38:0/2329358313 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fa9b80c1ec0 con 0x55bf52ed0f60 2024-05-21T15:40:52.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.014+0000 7fa9b17fa640 10 client.26938 handle_command_reply: tid=0 2024-05-21T15:40:52.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.014+0000 7fa9d6f8c740 1 client.26938 shutdown 2024-05-21T15:40:52.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.016+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2329358313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52f12390 msgr2=0x55bf532c2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:52.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.017+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/2329358313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52f12390 0x55bf532c2ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4013680 tx=0x7fa9c4006d50 comp rx=0 tx=0).stop 2024-05-21T15:40:52.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.021+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2329358313 shutdown_connections 2024-05-21T15:40:52.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.021+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/2329358313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52f12390 0x55bf532c2ab0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.021+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/2329358313 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf52ed0f60 0x55bf532c9450 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.021+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/2329358313 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 0x55bf532bbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.021+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/2329358313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52ee94d0 0x55bf532bb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.021+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2329358313 >> 172.21.15.38:0/2329358313 conn(0x55bf52f064e0 msgr2=0x55bf532ac5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:52.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.021+0000 7fa9b17fa640 0 client.26938 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:52.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.022+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2329358313 shutdown_connections 2024-05-21T15:40:52.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.023+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2329358313 wait complete. 2024-05-21T15:40:52.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.025+0000 7fa9d6f8c740 20 client.26938 trim_cache size 0 max 16384 2024-05-21T15:40:52.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.068+0000 7fa9d6f8c740 1 Processor -- start 2024-05-21T15:40:52.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.068+0000 7fa9d6f8c740 1 -- start start 2024-05-21T15:40:52.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.068+0000 7fa9d6f8c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52ee94d0 0x55bf532b6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:52.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.068+0000 7fa9d6f8c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 0x55bf532b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:52.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.068+0000 7fa9d6f8c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52f12390 0x55bf532c2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:52.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.068+0000 7fa9d6f8c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac00e5d0 con 0x55bf52ef16b0 2024-05-21T15:40:52.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.068+0000 7fa9d6f8c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac0101a0 con 0x55bf52ee94d0 2024-05-21T15:40:52.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.068+0000 7fa9d6f8c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac00eca0 con 0x55bf52f12390 2024-05-21T15:40:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.069+0000 7fa9cd684640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 0x55bf532b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.069+0000 7fa9cd684640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 0x55bf532b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33936/0 (socket says 172.21.15.38:33936) 2024-05-21T15:40:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.069+0000 7fa9cd684640 1 -- 172.21.15.38:0/2722932447 learned_addr learned my addr 172.21.15.38:0/2722932447 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.069+0000 7fa9ce686640 1 --2- 172.21.15.38:0/2722932447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52f12390 0x55bf532c2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.069+0000 7fa9cde85640 1 --2- 172.21.15.38:0/2722932447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52ee94d0 0x55bf532b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.070+0000 7fa9cd684640 1 -- 172.21.15.38:0/2722932447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52f12390 msgr2=0x55bf532c2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.070+0000 7fa9cd684640 1 --2- 172.21.15.38:0/2722932447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52f12390 0x55bf532c2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.070+0000 7fa9cd684640 1 -- 172.21.15.38:0/2722932447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52ee94d0 msgr2=0x55bf532b6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:52.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.070+0000 7fa9cd684640 1 --2- 172.21.15.38:0/2722932447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52ee94d0 0x55bf532b6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.070+0000 7fa9cd684640 1 -- 172.21.15.38:0/2722932447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf532bc210 con 0x55bf52ef16b0 2024-05-21T15:40:52.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.070+0000 7fa9cd684640 1 --2- 172.21.15.38:0/2722932447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 0x55bf532b4f30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8000c00 tx=0x7fa9b80eeec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbec10bf9ed1e574 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.072+0000 7fa9b37fe640 1 -- 172.21.15.38:0/2722932447 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80d5040 con 0x55bf52ef16b0 2024-05-21T15:40:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.072+0000 7fa9b37fe640 1 -- 172.21.15.38:0/2722932447 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b80bbe10 con 0x55bf52ef16b0 2024-05-21T15:40:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.072+0000 7fa9b37fe640 1 -- 172.21.15.38:0/2722932447 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80bd160 con 0x55bf52ef16b0 2024-05-21T15:40:52.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.074+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2722932447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 msgr2=0x55bf532b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:52.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.074+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/2722932447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 0x55bf532b4f30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8000c00 tx=0x7fa9b80eeec0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.078+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2722932447 shutdown_connections 2024-05-21T15:40:52.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.078+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/2722932447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52f12390 0x55bf532c2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.078+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/2722932447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ef16b0 0x55bf532b4f30 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.078+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/2722932447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52ee94d0 0x55bf532b6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.078+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2722932447 >> 172.21.15.38:0/2722932447 conn(0x55bf52f064e0 msgr2=0x55bf532b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.078+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2722932447 shutdown_connections 2024-05-21T15:40:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.078+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/2722932447 wait complete. 2024-05-21T15:40:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.079+0000 7fa9d6f8c740 1 -- start start 2024-05-21T15:40:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.079+0000 7fa9d6f8c740 1 Processor -- start 2024-05-21T15:40:52.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.080+0000 7fa9d6f8c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ee94d0 0x55bf532bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:52.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.080+0000 7fa9d6f8c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52ef16b0 0x55bf532c0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:52.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.080+0000 7fa9d6f8c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52f12390 0x55bf5332f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:52.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.080+0000 7fa9d6f8c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac00fa70 con 0x55bf52ee94d0 2024-05-21T15:40:52.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.080+0000 7fa9d6f8c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac00f370 con 0x55bf52f12390 2024-05-21T15:40:52.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.080+0000 7fa9d6f8c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac00c5b0 con 0x55bf52ef16b0 2024-05-21T15:40:52.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.080+0000 7fa9d6f8c740 10 client.0 _ll_register_callbacks cb 0x7fa9c400a280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:52.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.081+0000 7fa9ce686640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52f12390 0x55bf5332f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:52.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.081+0000 7fa9ce686640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52f12390 0x55bf5332f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:51498/0 (socket says 172.21.15.38:51498) 2024-05-21T15:40:52.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.081+0000 7fa9ce686640 1 -- 172.21.15.38:0/3428812850 learned_addr learned my addr 172.21.15.38:0/3428812850 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:52.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.081+0000 7fa9cd684640 1 --2- 172.21.15.38:0/3428812850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52ef16b0 0x55bf532c0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:52.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.082+0000 7fa9cde85640 1 --2- 172.21.15.38:0/3428812850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ee94d0 0x55bf532bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:52.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.082+0000 7fa9cde85640 1 -- 172.21.15.38:0/3428812850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52ef16b0 msgr2=0x55bf532c0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:52.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.082+0000 7fa9cde85640 1 --2- 172.21.15.38:0/3428812850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52ef16b0 0x55bf532c0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.082+0000 7fa9cde85640 1 -- 172.21.15.38:0/3428812850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52f12390 msgr2=0x55bf5332f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:52.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.082+0000 7fa9cde85640 1 --2- 172.21.15.38:0/3428812850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52f12390 0x55bf5332f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.082+0000 7fa9cde85640 1 -- 172.21.15.38:0/3428812850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf5332fd20 con 0x55bf52ee94d0 2024-05-21T15:40:52.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.083+0000 7fa9cde85640 1 --2- 172.21.15.38:0/3428812850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ee94d0 0x55bf532bfae0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x55bf532c9b50 tx=0x7fa9c00157d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb584384323902c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:52.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.083+0000 7fa9b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:40:52.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.083+0000 7fa9b17fa640 1 -- 172.21.15.38:0/3428812850 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0017580 con 0x55bf52ee94d0 2024-05-21T15:40:52.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.083+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/3428812850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b80bbe10 con 0x55bf52ee94d0 2024-05-21T15:40:52.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.083+0000 7fa9b17fa640 1 -- 172.21.15.38:0/3428812850 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa9c0017720 con 0x55bf52ee94d0 2024-05-21T15:40:52.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.084+0000 7fa9b17fa640 1 -- 172.21.15.38:0/3428812850 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa9c0017920 con 0x55bf52ee94d0 2024-05-21T15:40:52.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.084+0000 7fa9d6f8c740 10 client.17259 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:52.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.084+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/3428812850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bf53331080 con 0x55bf52ee94d0 2024-05-21T15:40:52.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.092+0000 7fa9b17fa640 1 -- 172.21.15.38:0/3428812850 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b80bbe10 con 0x55bf52ee94d0 2024-05-21T15:40:52.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.092+0000 7fa9b17fa640 1 -- 172.21.15.38:0/3428812850 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fa9c0011040 con 0x55bf52ee94d0 2024-05-21T15:40:52.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.094+0000 7fa9d6f8c740 10 client.17259 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:52.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.094+0000 7fa9d6f8c740 10 client.17259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:52.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.094+0000 7fa9d6f8c740 20 client.17259 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:52.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.094+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/3428812850 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf52ea63a0 0x55bf533339d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:52.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.094+0000 7fa9d6f8c740 4 client.17259 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:40:52.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.094+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/3428812850 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bf53224430 con 0x55bf52ea63a0 2024-05-21T15:40:52.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.096+0000 7fa9cd684640 1 --2- 172.21.15.38:0/3428812850 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf52ea63a0 0x55bf533339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:52.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.103+0000 7fa9cd684640 1 --2- 172.21.15.38:0/3428812850 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf52ea63a0 0x55bf533339d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30789bfabaaefaed server_cookie=b23ed2cb2c0993b4 in_seq=0 out_seq=0 2024-05-21T15:40:52.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.105+0000 7fa9b17fa640 10 client.17259 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:52.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.110+0000 7fa9b17fa640 1 -- 172.21.15.38:0/3428812850 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55bf53224430 con 0x55bf52ea63a0 2024-05-21T15:40:52.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.110+0000 7fa9b17fa640 10 client.17259 handle_command_reply: tid=0 2024-05-21T15:40:52.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.111+0000 7fa9d6f8c740 1 client.17259 shutdown 2024-05-21T15:40:52.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.119+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/3428812850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ee94d0 msgr2=0x55bf532bfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:52.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.119+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/3428812850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ee94d0 0x55bf532bfae0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x55bf532c9b50 tx=0x7fa9c00157d0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.120+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/3428812850 shutdown_connections 2024-05-21T15:40:52.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.120+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/3428812850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf52f12390 0x55bf5332f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.120+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/3428812850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bf52ef16b0 0x55bf532c0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.120+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/3428812850 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55bf52ea63a0 0x55bf533339d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.120+0000 7fa9d6f8c740 1 --2- 172.21.15.38:0/3428812850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf52ee94d0 0x55bf532bfae0 secure :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0x55bf532c9b50 tx=0x7fa9c00157d0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.120+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/3428812850 >> 172.21.15.38:0/3428812850 conn(0x55bf52f064e0 msgr2=0x55bf532b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:52.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.120+0000 7fa9b17fa640 0 client.17259 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:52.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.125+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/3428812850 shutdown_connections 2024-05-21T15:40:52.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.126+0000 7fa9d6f8c740 1 -- 172.21.15.38:0/3428812850 wait complete. 2024-05-21T15:40:52.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.129+0000 7fa9d6f8c740 20 client.17259 trim_cache size 0 max 16384 2024-05-21T15:40:52.131 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:40:52.131 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "scrub active (1 inodes in the stack)", 2024-05-21T15:40:52.131 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-05-21T15:40:52.131 INFO:teuthology.orchestra.run.smithi038.stdout: "9cc63c75-2a25-4f2d-9301-a8366dc50ef0": { 2024-05-21T15:40:52.132 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-05-21T15:40:52.132 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "9cc63c75-2a25-4f2d-9301-a8366dc50ef0", 2024-05-21T15:40:52.132 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-05-21T15:40:52.132 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:40:52.132 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:40:52.132 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:40:52.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.140+0000 7fa9d00e9640 1 -- 172.21.15.38:0/1208713054 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa9b40a2da0 msgr2=0x7fa9b40a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:52.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.140+0000 7fa9d00e9640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa9b40a2da0 0x7fa9b40a52b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4006fd0 tx=0x7fa9c400a0d0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.140+0000 7fa9d00e9640 1 -- 172.21.15.38:0/1208713054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9c810a2c0 msgr2=0x7fa9c81a86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:52.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.140+0000 7fa9d00e9640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9c810a2c0 0x7fa9c81a86d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa9b802aa10 tx=0x7fa9b8039520 comp rx=0 tx=0).stop 2024-05-21T15:40:52.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.141+0000 7fa9d00e9640 1 -- 172.21.15.38:0/1208713054 shutdown_connections 2024-05-21T15:40:52.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.141+0000 7fa9d00e9640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa9b40a2da0 0x7fa9b40a52b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.141+0000 7fa9d00e9640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9c810efe0 0x7fa9c81af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.141+0000 7fa9d00e9640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9c810a2c0 0x7fa9c81a86d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.141+0000 7fa9d00e9640 1 --2- 172.21.15.38:0/1208713054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c81093d0 0x7fa9c81a8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:52.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.141+0000 7fa9d00e9640 1 -- 172.21.15.38:0/1208713054 >> 172.21.15.38:0/1208713054 conn(0x7fa9c806f4a0 msgr2=0x7fa9c8071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:52.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.142+0000 7fa9d00e9640 1 -- 172.21.15.38:0/1208713054 shutdown_connections 2024-05-21T15:40:52.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:40:52.145+0000 7fa9d00e9640 1 -- 172.21.15.38:0/1208713054 wait complete. 2024-05-21T15:40:52.163 INFO:tasks.cephfs.filesystem:scrub status for tag:9cc63c75-2a25-4f2d-9301-a8366dc50ef0 - {'path': '/', 'tag': '9cc63c75-2a25-4f2d-9301-a8366dc50ef0', 'options': 'recursive,force'} 2024-05-21T15:40:52.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:52 smithi045 ceph-mon[36409]: scrub queued for path: / 2024-05-21T15:40:52.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:52 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:52.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:52 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:40:52.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:52 smithi038 ceph-mon[31995]: scrub queued for path: / 2024-05-21T15:40:52.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:52 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:52.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:52 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:40:52.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:52 smithi053 ceph-mon[36496]: scrub queued for path: / 2024-05-21T15:40:52.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:52 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:52.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:52 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:40:53.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:53 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:40:53.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:53 smithi045 ceph-mon[36409]: pgmap v545: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 4.90k op/s 2024-05-21T15:40:53.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:53 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:40:53.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:53 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:40:53.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:53 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:40:53.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:53 smithi038 ceph-mon[31995]: pgmap v545: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 4.90k op/s 2024-05-21T15:40:53.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:53 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:40:53.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:53 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:40:53.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:53 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:40:53.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:53 smithi053 ceph-mon[36496]: pgmap v545: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 4.90k op/s 2024-05-21T15:40:53.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:53 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:40:53.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:53 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:40:54.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:54 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:40:54.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:54 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:54.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:54 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:54.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:54 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:54.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:54 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:40:54.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:54 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:54.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:54 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:54.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:54 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:54.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:54 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:40:54.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:54 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:54.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:54 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:54.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:54 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.112+0000 7f84e3aa0640 1 -- 172.21.15.38:0/2764280076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc102370 msgr2=0x7f84dc102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.112+0000 7f84e3aa0640 1 --2- 172.21.15.38:0/2764280076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc102370 0x7f84dc102780 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009900 tx=0x7f84cc02a410 comp rx=0 tx=0).stop 2024-05-21T15:40:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.112+0000 7f84e3aa0640 1 -- 172.21.15.38:0/2764280076 shutdown_connections 2024-05-21T15:40:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.112+0000 7f84e3aa0640 1 --2- 172.21.15.38:0/2764280076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84dc107070 0x7f84dc109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.112+0000 7f84e3aa0640 1 --2- 172.21.15.38:0/2764280076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc102370 0x7f84dc102780 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.112+0000 7f84e3aa0640 1 --2- 172.21.15.38:0/2764280076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc101130 0x7f84dc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.112+0000 7f84e3aa0640 1 -- 172.21.15.38:0/2764280076 >> 172.21.15.38:0/2764280076 conn(0x7f84dc0fc300 msgr2=0x7f84dc0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.112+0000 7f84e3aa0640 1 -- 172.21.15.38:0/2764280076 shutdown_connections 2024-05-21T15:40:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.112+0000 7f84e3aa0640 1 -- 172.21.15.38:0/2764280076 wait complete. 2024-05-21T15:40:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.113+0000 7f84e3aa0640 1 Processor -- start 2024-05-21T15:40:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.113+0000 7f84e3aa0640 1 -- start start 2024-05-21T15:40:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.113+0000 7f84e3aa0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc101130 0x7f84dc10a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.113+0000 7f84e3aa0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc102370 0x7f84dc19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.113+0000 7f84e3aa0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84dc107070 0x7f84dc1a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.113+0000 7f84e3aa0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc10d630 con 0x7f84dc102370 2024-05-21T15:40:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.113+0000 7f84e3aa0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc10d4b0 con 0x7f84dc101130 2024-05-21T15:40:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.113+0000 7f84e3aa0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc10d7b0 con 0x7f84dc107070 2024-05-21T15:40:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.113+0000 7f84e103b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc102370 0x7f84dc19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.113+0000 7f84e103b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc102370 0x7f84dc19df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48208/0 (socket says 172.21.15.38:48208) 2024-05-21T15:40:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.113+0000 7f84e103b640 1 -- 172.21.15.38:0/2044576868 learned_addr learned my addr 172.21.15.38:0/2044576868 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.113+0000 7f84e183c640 1 --2- 172.21.15.38:0/2044576868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc101130 0x7f84dc10a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.113+0000 7f84e203d640 1 --2- 172.21.15.38:0/2044576868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84dc107070 0x7f84dc1a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.114+0000 7f84e103b640 1 -- 172.21.15.38:0/2044576868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84dc107070 msgr2=0x7f84dc1a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.114+0000 7f84e103b640 1 --2- 172.21.15.38:0/2044576868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84dc107070 0x7f84dc1a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.114+0000 7f84e103b640 1 -- 172.21.15.38:0/2044576868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc101130 msgr2=0x7f84dc10a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.114+0000 7f84e103b640 1 --2- 172.21.15.38:0/2044576868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc101130 0x7f84dc10a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.114+0000 7f84e103b640 1 -- 172.21.15.38:0/2044576868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc1a0cd0 con 0x7f84dc102370 2024-05-21T15:40:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.114+0000 7f84e103b640 1 --2- 172.21.15.38:0/2044576868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc102370 0x7f84dc19df60 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f84cc006010 tx=0x7f84cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.114+0000 7f84d2ffd640 1 -- 172.21.15.38:0/2044576868 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc03f030 con 0x7f84dc102370 2024-05-21T15:40:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.114+0000 7f84d2ffd640 1 -- 172.21.15.38:0/2044576868 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84cc003ce0 con 0x7f84dc102370 2024-05-21T15:40:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.114+0000 7f84e3aa0640 1 -- 172.21.15.38:0/2044576868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84dc1a0f60 con 0x7f84dc102370 2024-05-21T15:40:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.114+0000 7f84d2ffd640 1 -- 172.21.15.38:0/2044576868 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc035070 con 0x7f84dc102370 2024-05-21T15:40:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.114+0000 7f84e3aa0640 1 -- 172.21.15.38:0/2044576868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84dc1a1370 con 0x7f84dc102370 2024-05-21T15:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.116+0000 7f84d2ffd640 1 -- 172.21.15.38:0/2044576868 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f84cc042030 con 0x7f84dc102370 2024-05-21T15:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.117+0000 7f84eaa21740 1 Processor -- start 2024-05-21T15:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.117+0000 7f84eaa21740 1 -- start start 2024-05-21T15:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.117+0000 7f84eaa21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 0x558041ff2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.117+0000 7f84eaa21740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c2a2f0 0x558041ff33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.117+0000 7f84eaa21740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c324d0 0x558041ffa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.117+0000 7f84eaa21740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558041f57340 con 0x558041c2a2f0 2024-05-21T15:40:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.117+0000 7f84eaa21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558041eff290 con 0x558041c531b0 2024-05-21T15:40:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.117+0000 7f84eaa21740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558041f53300 con 0x558041c324d0 2024-05-21T15:40:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.118+0000 7f84e103b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c324d0 0x558041ffa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.118+0000 7f84e203d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 0x558041ff2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.118+0000 7f84d2ffd640 1 --2- 172.21.15.38:0/2044576868 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84c80a2c30 0x7f84c80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.118+0000 7f84e203d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 0x558041ff2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:45376/0 (socket says 172.21.15.38:45376) 2024-05-21T15:40:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.118+0000 7f84e203d640 1 -- 172.21.15.38:0/510580309 learned_addr learned my addr 172.21.15.38:0/510580309 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.118+0000 7f84e183c640 1 --2- 172.21.15.38:0/510580309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c2a2f0 0x558041ff33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.118+0000 7f84e203d640 1 --2- 172.21.15.38:0/2044576868 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84c80a2c30 0x7f84c80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.118+0000 7f84e203d640 1 -- 172.21.15.38:0/510580309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c324d0 msgr2=0x558041ffa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.118+0000 7f84e203d640 1 --2- 172.21.15.38:0/510580309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c324d0 0x558041ffa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.118+0000 7f84d2ffd640 1 -- 172.21.15.38:0/2044576868 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f84cc0ef810 con 0x7f84dc102370 2024-05-21T15:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.118+0000 7f84e203d640 1 -- 172.21.15.38:0/510580309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c2a2f0 msgr2=0x558041ff33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.118+0000 7f84e203d640 1 --2- 172.21.15.38:0/510580309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c2a2f0 0x558041ff33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.118+0000 7f84e203d640 1 -- 172.21.15.38:0/510580309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558041f238e0 con 0x558041c531b0 2024-05-21T15:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.118+0000 7f84e203d640 1 --2- 172.21.15.38:0/510580309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 0x558041ff2e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f84d4005c40 tx=0x7f84d400f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=143aaf7960c23c06 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.119+0000 7f84e203d640 1 --2- 172.21.15.38:0/2044576868 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84c80a2c30 0x7f84c80a5140 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f84d4008000 tx=0x7f84d401f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.119+0000 7f84c7fff640 1 -- 172.21.15.38:0/510580309 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d400fd30 con 0x558041c531b0 2024-05-21T15:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.119+0000 7f84c7fff640 1 -- 172.21.15.38:0/510580309 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84d4022030 con 0x558041c531b0 2024-05-21T15:40:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.119+0000 7f84c7fff640 1 -- 172.21.15.38:0/510580309 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4010930 con 0x558041c531b0 2024-05-21T15:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.120+0000 7f84eaa21740 1 -- 172.21.15.38:0/510580309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 msgr2=0x558041ff2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.120+0000 7f84eaa21740 1 --2- 172.21.15.38:0/510580309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 0x558041ff2e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f84d4005c40 tx=0x7f84d400f3c0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.120+0000 7f84eaa21740 1 -- 172.21.15.38:0/510580309 shutdown_connections 2024-05-21T15:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.120+0000 7f84eaa21740 1 --2- 172.21.15.38:0/510580309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c324d0 0x558041ffa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.120+0000 7f84eaa21740 1 --2- 172.21.15.38:0/510580309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c2a2f0 0x558041ff33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.120+0000 7f84eaa21740 1 --2- 172.21.15.38:0/510580309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 0x558041ff2e60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.120+0000 7f84eaa21740 1 -- 172.21.15.38:0/510580309 >> 172.21.15.38:0/510580309 conn(0x558041c47300 msgr2=0x558041fee7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.120+0000 7f84eaa21740 1 -- 172.21.15.38:0/510580309 shutdown_connections 2024-05-21T15:40:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.120+0000 7f84eaa21740 1 -- 172.21.15.38:0/510580309 wait complete. 2024-05-21T15:40:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.120+0000 7f84eaa21740 1 -- start start 2024-05-21T15:40:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.121+0000 7f84eaa21740 1 Processor -- start 2024-05-21T15:40:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.121+0000 7f84eaa21740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558042005400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84eaa21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c324d0 0x558042005940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84eaa21740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c531b0 0x55804200c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84eaa21740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558041ffacd0 con 0x558041c531b0 2024-05-21T15:40:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84e183c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558042005400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84e183c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558042005400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:33204/0 (socket says 172.21.15.38:33204) 2024-05-21T15:40:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84e183c640 1 -- 172.21.15.38:0/236992040 learned_addr learned my addr 172.21.15.38:0/236992040 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84eaa21740 1 -- 172.21.15.38:0/236992040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558041f12790 con 0x558041c324d0 2024-05-21T15:40:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84e103b640 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c324d0 0x558042005940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84e203d640 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c531b0 0x55804200c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84eaa21740 1 -- 172.21.15.38:0/236992040 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558041ffae50 con 0x558041c2a2f0 2024-05-21T15:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84eaa21740 10 client.0 _ll_register_callbacks cb 0x558041f4da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84e103b640 1 -- 172.21.15.38:0/236992040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 msgr2=0x558042005400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84e103b640 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558042005400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84e103b640 1 -- 172.21.15.38:0/236992040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c531b0 msgr2=0x55804200c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84e103b640 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c531b0 0x55804200c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84e103b640 1 -- 172.21.15.38:0/236992040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558041c6b210 con 0x558041c324d0 2024-05-21T15:40:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84e203d640 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c531b0 0x55804200c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.122+0000 7f84e183c640 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558042005400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.123+0000 7f84e103b640 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c324d0 0x558042005940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0ed310 tx=0x7f84cc009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbb852811582ebae server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.123+0000 7f84c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.123+0000 7f84c57fa640 1 -- 172.21.15.38:0/236992040 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc049050 con 0x558041c324d0 2024-05-21T15:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.123+0000 7f84c57fa640 1 -- 172.21.15.38:0/236992040 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f84cc0ba170 con 0x558041c324d0 2024-05-21T15:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.123+0000 7f84eaa21740 1 -- 172.21.15.38:0/236992040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84d4022030 con 0x558041c324d0 2024-05-21T15:40:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.123+0000 7f84c57fa640 1 -- 172.21.15.38:0/236992040 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f84cc0bc0d0 con 0x558041c324d0 2024-05-21T15:40:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.123+0000 7f84eaa21740 10 client.26888 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.123+0000 7f84eaa21740 1 -- 172.21.15.38:0/236992040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558041d37620 con 0x558041c324d0 2024-05-21T15:40:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.124+0000 7f84c57fa640 1 -- 172.21.15.38:0/236992040 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84d4022030 con 0x558041c324d0 2024-05-21T15:40:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.125+0000 7f84c57fa640 1 -- 172.21.15.38:0/236992040 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f84cc0b8740 con 0x558041c324d0 2024-05-21T15:40:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.125+0000 7f84eaa21740 10 client.26888 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.125+0000 7f84eaa21740 10 client.26888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.125+0000 7f84eaa21740 20 client.26888 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.125+0000 7f84eaa21740 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x558041c11d80 0x558042012d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.125+0000 7f84eaa21740 4 client.26888 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.125+0000 7f84eaa21740 1 -- 172.21.15.38:0/236992040 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558041f64a70 con 0x558041c11d80 2024-05-21T15:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.125+0000 7f84e183c640 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x558041c11d80 0x558042012d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.125+0000 7f84e183c640 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x558041c11d80 0x558042012d80 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=871cede65c1bcc9e server_cookie=258baa165269162b in_seq=0 out_seq=0 2024-05-21T15:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.125+0000 7f84c57fa640 10 client.26888 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.126+0000 7f84c57fa640 1 -- 172.21.15.38:0/236992040 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558041f64a70 con 0x558041c11d80 2024-05-21T15:40:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.126+0000 7f84c57fa640 10 client.26888 handle_command_reply: tid=0 2024-05-21T15:40:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.126+0000 7f84eaa21740 1 client.26888 shutdown 2024-05-21T15:40:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.126+0000 7f84eaa21740 1 -- 172.21.15.38:0/236992040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c324d0 msgr2=0x558042005940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.126+0000 7f84eaa21740 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c324d0 0x558042005940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0ed310 tx=0x7f84cc009620 comp rx=0 tx=0).stop 2024-05-21T15:40:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.126+0000 7f84eaa21740 1 -- 172.21.15.38:0/236992040 shutdown_connections 2024-05-21T15:40:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.126+0000 7f84eaa21740 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x558041c11d80 0x558042012d80 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.126+0000 7f84eaa21740 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c531b0 0x55804200c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.126+0000 7f84eaa21740 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c324d0 0x558042005940 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.126+0000 7f84eaa21740 1 --2- 172.21.15.38:0/236992040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558042005400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.126+0000 7f84c57fa640 0 client.26888 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.126+0000 7f84eaa21740 1 -- 172.21.15.38:0/236992040 >> 172.21.15.38:0/236992040 conn(0x558041c47300 msgr2=0x558041fed810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.126+0000 7f84eaa21740 1 -- 172.21.15.38:0/236992040 shutdown_connections 2024-05-21T15:40:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.127+0000 7f84eaa21740 1 -- 172.21.15.38:0/236992040 wait complete. 2024-05-21T15:40:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.127+0000 7f84eaa21740 20 client.26888 trim_cache size 0 max 16384 2024-05-21T15:40:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.148+0000 7f84eaa21740 1 Processor -- start 2024-05-21T15:40:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.148+0000 7f84eaa21740 1 -- start start 2024-05-21T15:40:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.148+0000 7f84eaa21740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558041ff7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.148+0000 7f84eaa21740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c324d0 0x558041ff7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.148+0000 7f84eaa21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 0x558041fee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.148+0000 7f84eaa21740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1a81d0 con 0x558041c324d0 2024-05-21T15:40:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.148+0000 7f84eaa21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1a9da0 con 0x558041c531b0 2024-05-21T15:40:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.148+0000 7f84eaa21740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1a88a0 con 0x558041c2a2f0 2024-05-21T15:40:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.148+0000 7f84e103b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c324d0 0x558041ff7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.148+0000 7f84e103b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c324d0 0x558041ff7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48236/0 (socket says 172.21.15.38:48236) 2024-05-21T15:40:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.148+0000 7f84e103b640 1 -- 172.21.15.38:0/4120997541 learned_addr learned my addr 172.21.15.38:0/4120997541 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.148+0000 7f84e183c640 1 --2- 172.21.15.38:0/4120997541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558041ff7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.148+0000 7f84e203d640 1 --2- 172.21.15.38:0/4120997541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 0x558041fee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.149+0000 7f84e103b640 1 -- 172.21.15.38:0/4120997541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 msgr2=0x558041ff7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.149+0000 7f84e103b640 1 --2- 172.21.15.38:0/4120997541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558041ff7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.149+0000 7f84e103b640 1 -- 172.21.15.38:0/4120997541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 msgr2=0x558041fee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.149+0000 7f84e103b640 1 --2- 172.21.15.38:0/4120997541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 0x558041fee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.149+0000 7f84e103b640 1 -- 172.21.15.38:0/4120997541 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558041ff2a10 con 0x558041c324d0 2024-05-21T15:40:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.149+0000 7f84e103b640 1 --2- 172.21.15.38:0/4120997541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c324d0 0x558041ff7590 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0bbdd0 tx=0x7f84cc0efba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a3cfcbb4b2f103 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.149+0000 7f84c77fe640 1 -- 172.21.15.38:0/4120997541 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc049070 con 0x558041c324d0 2024-05-21T15:40:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.149+0000 7f84c77fe640 1 -- 172.21.15.38:0/4120997541 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84cc0c03d0 con 0x558041c324d0 2024-05-21T15:40:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.149+0000 7f84c77fe640 1 -- 172.21.15.38:0/4120997541 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc0ba480 con 0x558041c324d0 2024-05-21T15:40:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.150+0000 7f84eaa21740 1 -- 172.21.15.38:0/4120997541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c324d0 msgr2=0x558041ff7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.150+0000 7f84eaa21740 1 --2- 172.21.15.38:0/4120997541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c324d0 0x558041ff7590 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0bbdd0 tx=0x7f84cc0efba0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.150+0000 7f84eaa21740 1 -- 172.21.15.38:0/4120997541 shutdown_connections 2024-05-21T15:40:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.150+0000 7f84eaa21740 1 --2- 172.21.15.38:0/4120997541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 0x558041fee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.150+0000 7f84eaa21740 1 --2- 172.21.15.38:0/4120997541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c324d0 0x558041ff7590 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.150+0000 7f84eaa21740 1 --2- 172.21.15.38:0/4120997541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558041ff7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.150+0000 7f84eaa21740 1 -- 172.21.15.38:0/4120997541 >> 172.21.15.38:0/4120997541 conn(0x558041c47300 msgr2=0x558041ff2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.150+0000 7f84eaa21740 1 -- 172.21.15.38:0/4120997541 shutdown_connections 2024-05-21T15:40:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.150+0000 7f84eaa21740 1 -- 172.21.15.38:0/4120997541 wait complete. 2024-05-21T15:40:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.151+0000 7f84eaa21740 1 -- start start 2024-05-21T15:40:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.151+0000 7f84eaa21740 1 Processor -- start 2024-05-21T15:40:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.151+0000 7f84eaa21740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558042009340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.151+0000 7f84eaa21740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c324d0 0x558042009880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.151+0000 7f84eaa21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 0x55804200a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.151+0000 7f84eaa21740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1a9670 con 0x558041c324d0 2024-05-21T15:40:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.151+0000 7f84eaa21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1a8f70 con 0x558041c531b0 2024-05-21T15:40:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.151+0000 7f84eaa21740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1a61b0 con 0x558041c2a2f0 2024-05-21T15:40:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.151+0000 7f84eaa21740 10 client.0 _ll_register_callbacks cb 0x7f84cc0044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:40:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.151+0000 7f84e183c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558042009340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.151+0000 7f84e183c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558042009340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:33230/0 (socket says 172.21.15.38:33230) 2024-05-21T15:40:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.151+0000 7f84e183c640 1 -- 172.21.15.38:0/3002536090 learned_addr learned my addr 172.21.15.38:0/3002536090 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:40:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.151+0000 7f84e103b640 1 --2- 172.21.15.38:0/3002536090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c324d0 0x558042009880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.152+0000 7f84e183c640 1 -- 172.21.15.38:0/3002536090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 msgr2=0x55804200a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.152+0000 7f84e183c640 1 --2- 172.21.15.38:0/3002536090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 0x55804200a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.152+0000 7f84e183c640 1 -- 172.21.15.38:0/3002536090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c324d0 msgr2=0x558042009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:40:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.152+0000 7f84e183c640 1 --2- 172.21.15.38:0/3002536090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c324d0 0x558042009880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.152+0000 7f84e183c640 1 -- 172.21.15.38:0/3002536090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55804200ac90 con 0x558041c2a2f0 2024-05-21T15:40:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.152+0000 7f84e183c640 1 --2- 172.21.15.38:0/3002536090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558042009340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f84d8000c00 tx=0x7f84d8012ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdacc53e5ce36307 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:40:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.152+0000 7f84c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:40:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.152+0000 7f84c57fa640 1 -- 172.21.15.38:0/3002536090 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d801fb00 con 0x558041c2a2f0 2024-05-21T15:40:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.152+0000 7f84c57fa640 1 -- 172.21.15.38:0/3002536090 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f84d801fca0 con 0x558041c2a2f0 2024-05-21T15:40:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.152+0000 7f84eaa21740 1 -- 172.21.15.38:0/3002536090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84cc0c03d0 con 0x558041c2a2f0 2024-05-21T15:40:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.153+0000 7f84c57fa640 1 -- 172.21.15.38:0/3002536090 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f84d8030050 con 0x558041c2a2f0 2024-05-21T15:40:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.153+0000 7f84eaa21740 10 client.26962 fetch_fsmap learned FSMap version 32 2024-05-21T15:40:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.153+0000 7f84eaa21740 1 -- 172.21.15.38:0/3002536090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55804200bf60 con 0x558041c2a2f0 2024-05-21T15:40:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.153+0000 7f84c57fa640 1 -- 172.21.15.38:0/3002536090 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84cc0c03d0 con 0x558041c2a2f0 2024-05-21T15:40:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.153+0000 7f84c57fa640 1 -- 172.21.15.38:0/3002536090 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f84d8010550 con 0x558041c2a2f0 2024-05-21T15:40:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.153+0000 7f84eaa21740 10 client.26962 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:40:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.153+0000 7f84eaa21740 10 client.26962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:40:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.153+0000 7f84eaa21740 20 client.26962 populate_metadata read hostname 'smithi038' 2024-05-21T15:40:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.153+0000 7f84eaa21740 1 --2- 172.21.15.38:0/3002536090 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x558041be71c0 0x55804207d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:40:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.153+0000 7f84eaa21740 4 client.26962 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:40:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.153+0000 7f84eaa21740 1 -- 172.21.15.38:0/3002536090 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558041f5edc0 con 0x558041be71c0 2024-05-21T15:40:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.154+0000 7f84e103b640 1 --2- 172.21.15.38:0/3002536090 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x558041be71c0 0x55804207d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:40:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.154+0000 7f84e103b640 1 --2- 172.21.15.38:0/3002536090 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x558041be71c0 0x55804207d720 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b082f9ba0e2c47 server_cookie=bc7a416b8ad3ba18 in_seq=0 out_seq=0 2024-05-21T15:40:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.154+0000 7f84c57fa640 10 client.26962 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.169+0000 7f84c57fa640 1 -- 172.21.15.38:0/3002536090 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558041f5edc0 con 0x558041be71c0 2024-05-21T15:40:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.169+0000 7f84c57fa640 10 client.26962 handle_command_reply: tid=0 2024-05-21T15:40:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.169+0000 7f84eaa21740 1 client.26962 shutdown 2024-05-21T15:40:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.169+0000 7f84eaa21740 1 -- 172.21.15.38:0/3002536090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 msgr2=0x558042009340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.169+0000 7f84eaa21740 1 --2- 172.21.15.38:0/3002536090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558042009340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f84d8000c00 tx=0x7f84d8012ed0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.170+0000 7f84eaa21740 1 -- 172.21.15.38:0/3002536090 shutdown_connections 2024-05-21T15:40:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.170+0000 7f84eaa21740 1 --2- 172.21.15.38:0/3002536090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558041c531b0 0x55804200a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.170+0000 7f84eaa21740 1 --2- 172.21.15.38:0/3002536090 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x558041be71c0 0x55804207d720 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.170+0000 7f84eaa21740 1 --2- 172.21.15.38:0/3002536090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558041c324d0 0x558042009880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.170+0000 7f84eaa21740 1 --2- 172.21.15.38:0/3002536090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558041c2a2f0 0x558042009340 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.170+0000 7f84eaa21740 1 -- 172.21.15.38:0/3002536090 >> 172.21.15.38:0/3002536090 conn(0x558041c47300 msgr2=0x558041ff7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.170+0000 7f84c57fa640 0 client.26962 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:40:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.170+0000 7f84eaa21740 1 -- 172.21.15.38:0/3002536090 shutdown_connections 2024-05-21T15:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.170+0000 7f84eaa21740 1 -- 172.21.15.38:0/3002536090 wait complete. 2024-05-21T15:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.170+0000 7f84eaa21740 20 client.26962 trim_cache size 0 max 16384 2024-05-21T15:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:40:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:40:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.171+0000 7f84e3aa0640 1 -- 172.21.15.38:0/2044576868 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84c80a2c30 msgr2=0x7f84c80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.171+0000 7f84e3aa0640 1 --2- 172.21.15.38:0/2044576868 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84c80a2c30 0x7f84c80a5140 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f84d4008000 tx=0x7f84d401f000 comp rx=0 tx=0).stop 2024-05-21T15:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.171+0000 7f84e3aa0640 1 -- 172.21.15.38:0/2044576868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc102370 msgr2=0x7f84dc19df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.171+0000 7f84e3aa0640 1 --2- 172.21.15.38:0/2044576868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc102370 0x7f84dc19df60 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f84cc006010 tx=0x7f84cc0042f0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.171+0000 7f84e3aa0640 1 -- 172.21.15.38:0/2044576868 shutdown_connections 2024-05-21T15:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.171+0000 7f84e3aa0640 1 --2- 172.21.15.38:0/2044576868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84dc107070 0x7f84dc1a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.171+0000 7f84e3aa0640 1 --2- 172.21.15.38:0/2044576868 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f84c80a2c30 0x7f84c80a5140 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.171+0000 7f84e3aa0640 1 --2- 172.21.15.38:0/2044576868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc102370 0x7f84dc19df60 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.171+0000 7f84e3aa0640 1 --2- 172.21.15.38:0/2044576868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc101130 0x7f84dc10a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.171+0000 7f84e3aa0640 1 -- 172.21.15.38:0/2044576868 >> 172.21.15.38:0/2044576868 conn(0x7f84dc0fc300 msgr2=0x7f84dc103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.171+0000 7f84e3aa0640 1 -- 172.21.15.38:0/2044576868 shutdown_connections 2024-05-21T15:40:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:40:55.171+0000 7f84e3aa0640 1 -- 172.21.15.38:0/2044576868 wait complete. 2024-05-21T15:40:55.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:55 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:55.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:55 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:55.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:55 smithi045 ceph-mon[36409]: pgmap v546: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 5.07k op/s 2024-05-21T15:40:55.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:55 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:55.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:55 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:55.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:55 smithi038 ceph-mon[31995]: pgmap v546: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 5.07k op/s 2024-05-21T15:40:55.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:55 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:55.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:55 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:40:55.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:55 smithi053 ceph-mon[36496]: pgmap v546: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 5.07k op/s 2024-05-21T15:40:57.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:57 smithi045 ceph-mon[36409]: pgmap v547: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 MiB/s wr, 6.22k op/s 2024-05-21T15:40:57.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:57 smithi038 ceph-mon[31995]: pgmap v547: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 MiB/s wr, 6.22k op/s 2024-05-21T15:40:57.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:57 smithi053 ceph-mon[36496]: pgmap v547: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 MiB/s wr, 6.22k op/s 2024-05-21T15:40:58.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:58 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:40:58.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:58 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:40:59.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:58 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:40:59.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:59 smithi038 ceph-mon[31995]: pgmap v548: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 36 MiB/s wr, 7.75k op/s 2024-05-21T15:40:59.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:59 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:40:59.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:59 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:40:59.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:59 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:40:59.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:40:59 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:40:59.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:59 smithi053 ceph-mon[36496]: pgmap v548: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 36 MiB/s wr, 7.75k op/s 2024-05-21T15:40:59.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:59 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:40:59.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:59 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:40:59.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:59 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:40:59.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:40:59 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:41:00.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:59 smithi045 ceph-mon[36409]: pgmap v548: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 36 MiB/s wr, 7.75k op/s 2024-05-21T15:41:00.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:59 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:41:00.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:59 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:41:00.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:59 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:41:00.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:40:59 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:41:00.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:00 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:41:00.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:00 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:41:01.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:00 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:41:01.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:01 smithi038 ceph-mon[31995]: pgmap v549: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 35 MiB/s wr, 7.52k op/s 2024-05-21T15:41:01.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:01 smithi053 ceph-mon[36496]: pgmap v549: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 35 MiB/s wr, 7.52k op/s 2024-05-21T15:41:02.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:01 smithi045 ceph-mon[36409]: pgmap v549: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 35 MiB/s wr, 7.52k op/s 2024-05-21T15:41:03.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:03 smithi053 ceph-mon[36496]: pgmap v550: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 33 MiB/s wr, 7.27k op/s 2024-05-21T15:41:04.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:03 smithi045 ceph-mon[36409]: pgmap v550: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 33 MiB/s wr, 7.27k op/s 2024-05-21T15:41:04.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:03 smithi038 ceph-mon[31995]: pgmap v550: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 33 MiB/s wr, 7.27k op/s 2024-05-21T15:41:05.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:05 smithi053 ceph-mon[36496]: pgmap v551: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 36 MiB/s wr, 7.83k op/s 2024-05-21T15:41:06.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:05 smithi045 ceph-mon[36409]: pgmap v551: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 36 MiB/s wr, 7.83k op/s 2024-05-21T15:41:06.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:05 smithi038 ceph-mon[31995]: pgmap v551: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 36 MiB/s wr, 7.83k op/s 2024-05-21T15:41:07.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:07 smithi053 ceph-mon[36496]: pgmap v552: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 36 MiB/s wr, 7.84k op/s 2024-05-21T15:41:08.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:07 smithi045 ceph-mon[36409]: pgmap v552: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 36 MiB/s wr, 7.84k op/s 2024-05-21T15:41:08.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:07 smithi038 ceph-mon[31995]: pgmap v552: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 36 MiB/s wr, 7.84k op/s 2024-05-21T15:41:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.357+0000 7f9694656640 1 -- 172.21.15.38:0/316731118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f968c106670 msgr2=0x7f968c106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.357+0000 7f9694656640 1 --2- 172.21.15.38:0/316731118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f968c106670 0x7f968c106a80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9684009ca0 tx=0x7f968402a8f0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.358+0000 7f9694656640 1 -- 172.21.15.38:0/316731118 shutdown_connections 2024-05-21T15:41:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.358+0000 7f9694656640 1 --2- 172.21.15.38:0/316731118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f968c106670 0x7f968c106a80 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.358+0000 7f9694656640 1 --2- 172.21.15.38:0/316731118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c103d10 0x7f968c106130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.358+0000 7f9694656640 1 --2- 172.21.15.38:0/316731118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f968c1013b0 0x7f968c1037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.358+0000 7f9694656640 1 -- 172.21.15.38:0/316731118 >> 172.21.15.38:0/316731118 conn(0x7f968c0fcbc0 msgr2=0x7f968c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.363+0000 7f9694656640 1 -- 172.21.15.38:0/316731118 shutdown_connections 2024-05-21T15:41:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.364+0000 7f9694656640 1 -- 172.21.15.38:0/316731118 wait complete. 2024-05-21T15:41:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.365+0000 7f9694656640 1 Processor -- start 2024-05-21T15:41:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.365+0000 7f9694656640 1 -- start start 2024-05-21T15:41:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.365+0000 7f9694656640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f968c1013b0 0x7f968c111460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.365+0000 7f9694656640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c103d10 0x7f968c10fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.365+0000 7f9694656640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f968c106670 0x7f968c110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.365+0000 7f9694656640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f968c107f60 con 0x7f968c106670 2024-05-21T15:41:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.365+0000 7f9694656640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f968c107de0 con 0x7f968c1013b0 2024-05-21T15:41:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.365+0000 7f9694656640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f968c1080e0 con 0x7f968c103d10 2024-05-21T15:41:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.366+0000 7f9692bf3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f968c106670 0x7f968c110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.366+0000 7f9692bf3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f968c106670 0x7f968c110040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43410/0 (socket says 172.21.15.38:43410) 2024-05-21T15:41:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.366+0000 7f9692bf3640 1 -- 172.21.15.38:0/1905051177 learned_addr learned my addr 172.21.15.38:0/1905051177 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.367+0000 7f96923f2640 1 --2- 172.21.15.38:0/1905051177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f968c1013b0 0x7f968c111460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.372+0000 7f9691bf1640 1 --2- 172.21.15.38:0/1905051177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c103d10 0x7f968c10fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.373+0000 7f96923f2640 1 -- 172.21.15.38:0/1905051177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c103d10 msgr2=0x7f968c10fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.373+0000 7f96923f2640 1 --2- 172.21.15.38:0/1905051177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c103d10 0x7f968c10fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.373+0000 7f96923f2640 1 -- 172.21.15.38:0/1905051177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f968c106670 msgr2=0x7f968c110040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.373+0000 7f96923f2640 1 --2- 172.21.15.38:0/1905051177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f968c106670 0x7f968c110040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.373+0000 7f96923f2640 1 -- 172.21.15.38:0/1905051177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968c1105b0 con 0x7f968c1013b0 2024-05-21T15:41:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.375+0000 7f96923f2640 1 --2- 172.21.15.38:0/1905051177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f968c1013b0 0x7f968c111460 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f967c009fd0 tx=0x7f967c00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.379+0000 7f96837fe640 1 -- 172.21.15.38:0/1905051177 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c015020 con 0x7f968c1013b0 2024-05-21T15:41:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.379+0000 7f9694656640 1 -- 172.21.15.38:0/1905051177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968c1108a0 con 0x7f968c1013b0 2024-05-21T15:41:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.379+0000 7f9694656640 1 -- 172.21.15.38:0/1905051177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f968c071910 con 0x7f968c1013b0 2024-05-21T15:41:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.379+0000 7f96837fe640 1 -- 172.21.15.38:0/1905051177 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f967c011d10 con 0x7f968c1013b0 2024-05-21T15:41:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.379+0000 7f96837fe640 1 -- 172.21.15.38:0/1905051177 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c00f2e0 con 0x7f968c1013b0 2024-05-21T15:41:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.383+0000 7f969b5d3740 1 Processor -- start 2024-05-21T15:41:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.386+0000 7f96837fe640 1 -- 172.21.15.38:0/1905051177 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f967c018020 con 0x7f968c1013b0 2024-05-21T15:41:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.387+0000 7f96837fe640 1 --2- 172.21.15.38:0/1905051177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f96780a2e70 0x7f96780a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.387+0000 7f96837fe640 1 -- 172.21.15.38:0/1905051177 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f967c0ce800 con 0x7f968c1013b0 2024-05-21T15:41:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.388+0000 7f969b5d3740 1 -- start start 2024-05-21T15:41:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.388+0000 7f969b5d3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be1471b0 0x5610be4e6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.388+0000 7f969b5d3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4e73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.388+0000 7f969b5d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 0x5610be4ee2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.388+0000 7f969b5d3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5610be44b340 con 0x5610be11e2f0 2024-05-21T15:41:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.388+0000 7f969b5d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5610be3f3290 con 0x5610be1264d0 2024-05-21T15:41:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.388+0000 7f969b5d3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5610be447300 con 0x5610be1471b0 2024-05-21T15:41:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.389+0000 7f9692bf3640 1 --2- 172.21.15.38:0/1905051177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f96780a2e70 0x7f96780a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.389+0000 7f9692bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 0x5610be4ee2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.389+0000 7f9692bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 0x5610be4ee2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:59236/0 (socket says 172.21.15.38:59236) 2024-05-21T15:41:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.389+0000 7f9692bf3640 1 -- 172.21.15.38:0/2361123781 learned_addr learned my addr 172.21.15.38:0/2361123781 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.390+0000 7f9692bf3640 1 -- 172.21.15.38:0/2361123781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be1471b0 msgr2=0x5610be4e6e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:41:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.390+0000 7f9692bf3640 1 --2- 172.21.15.38:0/2361123781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be1471b0 0x5610be4e6e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.390+0000 7f9692bf3640 1 -- 172.21.15.38:0/2361123781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 msgr2=0x5610be4e73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.390+0000 7f9692bf3640 1 --2- 172.21.15.38:0/2361123781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4e73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.390+0000 7f9692bf3640 1 -- 172.21.15.38:0/2361123781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610be4178e0 con 0x5610be1264d0 2024-05-21T15:41:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.391+0000 7f9692bf3640 1 --2- 172.21.15.38:0/2361123781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 0x5610be4ee2e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f968402f4e0 tx=0x7f9684048000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfb9e2e02ad9244 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.391+0000 7f9692bf3640 1 --2- 172.21.15.38:0/1905051177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f96780a2e70 0x7f96780a5380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9684031a10 tx=0x7f9684009820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.393+0000 7f9680bfe640 1 -- 172.21.15.38:0/2361123781 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684040030 con 0x5610be1264d0 2024-05-21T15:41:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.393+0000 7f9680bfe640 1 -- 172.21.15.38:0/2361123781 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9684051070 con 0x5610be1264d0 2024-05-21T15:41:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.394+0000 7f969b5d3740 1 -- 172.21.15.38:0/2361123781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 msgr2=0x5610be4ee2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.394+0000 7f969b5d3740 1 --2- 172.21.15.38:0/2361123781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 0x5610be4ee2e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f968402f4e0 tx=0x7f9684048000 comp rx=0 tx=0).stop 2024-05-21T15:41:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.395+0000 7f969b5d3740 1 -- 172.21.15.38:0/2361123781 shutdown_connections 2024-05-21T15:41:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.395+0000 7f969b5d3740 1 --2- 172.21.15.38:0/2361123781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 0x5610be4ee2e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.395+0000 7f969b5d3740 1 --2- 172.21.15.38:0/2361123781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4e73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.395+0000 7f969b5d3740 1 --2- 172.21.15.38:0/2361123781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be1471b0 0x5610be4e6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.395+0000 7f969b5d3740 1 -- 172.21.15.38:0/2361123781 >> 172.21.15.38:0/2361123781 conn(0x5610be13b300 msgr2=0x5610be4e27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.396+0000 7f969b5d3740 1 -- 172.21.15.38:0/2361123781 shutdown_connections 2024-05-21T15:41:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.396+0000 7f969b5d3740 1 -- 172.21.15.38:0/2361123781 wait complete. 2024-05-21T15:41:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.397+0000 7f969b5d3740 1 -- start start 2024-05-21T15:41:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.397+0000 7f969b5d3740 1 Processor -- start 2024-05-21T15:41:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.397+0000 7f969b5d3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4f0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.397+0000 7f969b5d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 0x5610be4f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.397+0000 7f969b5d3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be1471b0 0x5610be4f8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.397+0000 7f969b5d3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5610be4eed20 con 0x5610be11e2f0 2024-05-21T15:41:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.397+0000 7f969b5d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5610be406790 con 0x5610be1264d0 2024-05-21T15:41:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.397+0000 7f969b5d3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5610be4eeea0 con 0x5610be1471b0 2024-05-21T15:41:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.397+0000 7f969b5d3740 10 client.0 _ll_register_callbacks cb 0x5610be441a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.398+0000 7f96923f2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4f0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.398+0000 7f96923f2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4f0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43436/0 (socket says 172.21.15.38:43436) 2024-05-21T15:41:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.398+0000 7f96923f2640 1 -- 172.21.15.38:0/4285567370 learned_addr learned my addr 172.21.15.38:0/4285567370 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.398+0000 7f9692bf3640 1 --2- 172.21.15.38:0/4285567370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be1471b0 0x5610be4f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.398+0000 7f9691bf1640 1 --2- 172.21.15.38:0/4285567370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 0x5610be4f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.398+0000 7f96923f2640 1 -- 172.21.15.38:0/4285567370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be1471b0 msgr2=0x5610be4f8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.398+0000 7f96923f2640 1 --2- 172.21.15.38:0/4285567370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be1471b0 0x5610be4f8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.398+0000 7f96923f2640 1 -- 172.21.15.38:0/4285567370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 msgr2=0x5610be4f10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.398+0000 7f96923f2640 1 --2- 172.21.15.38:0/4285567370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 0x5610be4f10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.398+0000 7f96923f2640 1 -- 172.21.15.38:0/4285567370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610be15f210 con 0x5610be11e2f0 2024-05-21T15:41:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.399+0000 7f96923f2640 1 --2- 172.21.15.38:0/4285567370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4f0bb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f967c09af40 tx=0x7f967c09a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656910c413f4d4ec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.399+0000 7f9675ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.400+0000 7f9675ffb640 1 -- 172.21.15.38:0/4285567370 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c09afb0 con 0x5610be11e2f0 2024-05-21T15:41:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.400+0000 7f9675ffb640 1 -- 172.21.15.38:0/4285567370 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f967c0cc360 con 0x5610be11e2f0 2024-05-21T15:41:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.400+0000 7f969b5d3740 1 -- 172.21.15.38:0/4285567370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9684051070 con 0x5610be11e2f0 2024-05-21T15:41:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.400+0000 7f9675ffb640 1 -- 172.21.15.38:0/4285567370 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f967c09f360 con 0x5610be11e2f0 2024-05-21T15:41:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.403+0000 7f969b5d3740 10 client.17286 fetch_fsmap learned FSMap version 33 2024-05-21T15:41:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.403+0000 7f969b5d3740 1 -- 172.21.15.38:0/4285567370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5610be22b620 con 0x5610be11e2f0 2024-05-21T15:41:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.405+0000 7f9675ffb640 1 -- 172.21.15.38:0/4285567370 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9684051070 con 0x5610be11e2f0 2024-05-21T15:41:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.405+0000 7f9675ffb640 1 -- 172.21.15.38:0/4285567370 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f967c011830 con 0x5610be11e2f0 2024-05-21T15:41:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.406+0000 7f969b5d3740 10 client.17286 fetch_fsmap finished waiting for FSMap version 33 2024-05-21T15:41:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.406+0000 7f969b5d3740 10 client.17286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.406+0000 7f969b5d3740 20 client.17286 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.406+0000 7f969b5d3740 1 --2- 172.21.15.38:0/4285567370 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5610be105d80 0x5610be4fe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.406+0000 7f969b5d3740 4 client.17286 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.406+0000 7f969b5d3740 1 -- 172.21.15.38:0/4285567370 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610be458a70 con 0x5610be105d80 2024-05-21T15:41:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.406+0000 7f9691bf1640 1 --2- 172.21.15.38:0/4285567370 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5610be105d80 0x5610be4fe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.408+0000 7f9691bf1640 1 --2- 172.21.15.38:0/4285567370 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5610be105d80 0x5610be4fe600 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b45b96c400a2a6d9 server_cookie=8cb707e0d5f2c5ec in_seq=0 out_seq=0 2024-05-21T15:41:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.408+0000 7f9675ffb640 10 client.17286 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:08.407 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.409+0000 7f9675ffb640 1 -- 172.21.15.38:0/4285567370 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5610be458a70 con 0x5610be105d80 2024-05-21T15:41:08.407 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.409+0000 7f9675ffb640 10 client.17286 handle_command_reply: tid=0 2024-05-21T15:41:08.407 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.410+0000 7f969b5d3740 1 client.17286 shutdown 2024-05-21T15:41:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.421+0000 7f969b5d3740 1 -- 172.21.15.38:0/4285567370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 msgr2=0x5610be4f0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.421+0000 7f969b5d3740 1 --2- 172.21.15.38:0/4285567370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4f0bb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f967c09af40 tx=0x7f967c09a910 comp rx=0 tx=0).stop 2024-05-21T15:41:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.423+0000 7f969b5d3740 1 -- 172.21.15.38:0/4285567370 shutdown_connections 2024-05-21T15:41:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.423+0000 7f969b5d3740 1 --2- 172.21.15.38:0/4285567370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be1471b0 0x5610be4f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.423+0000 7f969b5d3740 1 --2- 172.21.15.38:0/4285567370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 0x5610be4f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.423+0000 7f969b5d3740 1 --2- 172.21.15.38:0/4285567370 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5610be105d80 0x5610be4fe600 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.423+0000 7f969b5d3740 1 --2- 172.21.15.38:0/4285567370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4f0bb0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.423+0000 7f969b5d3740 1 -- 172.21.15.38:0/4285567370 >> 172.21.15.38:0/4285567370 conn(0x5610be13b300 msgr2=0x5610be4e1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.423+0000 7f9675ffb640 0 client.17286 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.424+0000 7f969b5d3740 1 -- 172.21.15.38:0/4285567370 shutdown_connections 2024-05-21T15:41:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.425+0000 7f969b5d3740 1 -- 172.21.15.38:0/4285567370 wait complete. 2024-05-21T15:41:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.430+0000 7f969b5d3740 20 client.17286 trim_cache size 0 max 16384 2024-05-21T15:41:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.467+0000 7f969b5d3740 1 Processor -- start 2024-05-21T15:41:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.467+0000 7f969b5d3740 1 -- start start 2024-05-21T15:41:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.467+0000 7f969b5d3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be13b300 0x5610be4eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.467+0000 7f969b5d3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4eb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.467+0000 7f969b5d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 0x5610be4e2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.467+0000 7f9691bf1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4eb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.467+0000 7f9691bf1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4eb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43442/0 (socket says 172.21.15.38:43442) 2024-05-21T15:41:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.467+0000 7f9691bf1640 1 -- 172.21.15.38:0/2952682410 learned_addr learned my addr 172.21.15.38:0/2952682410 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.467+0000 7f969b5d3740 1 -- 172.21.15.38:0/2952682410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f967000e710 con 0x5610be11e2f0 2024-05-21T15:41:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.467+0000 7f969b5d3740 1 -- 172.21.15.38:0/2952682410 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f96700102b0 con 0x5610be1264d0 2024-05-21T15:41:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.467+0000 7f969b5d3740 1 -- 172.21.15.38:0/2952682410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f967000ede0 con 0x5610be13b300 2024-05-21T15:41:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.468+0000 7f9692bf3640 1 --2- 172.21.15.38:0/2952682410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 0x5610be4e2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.468+0000 7f9691bf1640 1 -- 172.21.15.38:0/2952682410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be13b300 msgr2=0x5610be4eaf40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:41:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.468+0000 7f9691bf1640 1 --2- 172.21.15.38:0/2952682410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be13b300 0x5610be4eaf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.468+0000 7f9691bf1640 1 -- 172.21.15.38:0/2952682410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 msgr2=0x5610be4e2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.468+0000 7f9691bf1640 1 --2- 172.21.15.38:0/2952682410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 0x5610be4e2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.468+0000 7f9691bf1640 1 -- 172.21.15.38:0/2952682410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610be4f0960 con 0x5610be11e2f0 2024-05-21T15:41:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.468+0000 7f9691bf1640 1 --2- 172.21.15.38:0/2952682410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4eb480 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9688006dc0 tx=0x7f9688008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f621cdc572155a32 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.471+0000 7f9677fff640 1 -- 172.21.15.38:0/2952682410 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688016770 con 0x5610be11e2f0 2024-05-21T15:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.471+0000 7f9677fff640 1 -- 172.21.15.38:0/2952682410 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9688016df0 con 0x5610be11e2f0 2024-05-21T15:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.472+0000 7f969b5d3740 1 -- 172.21.15.38:0/2952682410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 msgr2=0x5610be4eb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.472+0000 7f969b5d3740 1 --2- 172.21.15.38:0/2952682410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4eb480 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9688006dc0 tx=0x7f9688008040 comp rx=0 tx=0).stop 2024-05-21T15:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.473+0000 7f969b5d3740 1 -- 172.21.15.38:0/2952682410 shutdown_connections 2024-05-21T15:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.473+0000 7f969b5d3740 1 --2- 172.21.15.38:0/2952682410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be1264d0 0x5610be4e2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.473+0000 7f969b5d3740 1 --2- 172.21.15.38:0/2952682410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be11e2f0 0x5610be4eb480 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.473+0000 7f969b5d3740 1 --2- 172.21.15.38:0/2952682410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be13b300 0x5610be4eaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.473+0000 7f969b5d3740 1 -- 172.21.15.38:0/2952682410 >> 172.21.15.38:0/2952682410 conn(0x5610be4dd870 msgr2=0x5610be4f0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.480+0000 7f969b5d3740 1 -- 172.21.15.38:0/2952682410 shutdown_connections 2024-05-21T15:41:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.482+0000 7f969b5d3740 1 -- 172.21.15.38:0/2952682410 wait complete. 2024-05-21T15:41:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.482+0000 7f969b5d3740 1 -- start start 2024-05-21T15:41:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.482+0000 7f969b5d3740 1 Processor -- start 2024-05-21T15:41:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.483+0000 7f969b5d3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be11e2f0 0x5610be4f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.483+0000 7f969b5d3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be1264d0 0x5610be4f5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.483+0000 7f969b5d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be13b300 0x5610be4f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.483+0000 7f969b5d3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f967000fb80 con 0x5610be1264d0 2024-05-21T15:41:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.483+0000 7f969b5d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f967000f4b0 con 0x5610be13b300 2024-05-21T15:41:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.483+0000 7f969b5d3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f967000c6f0 con 0x5610be11e2f0 2024-05-21T15:41:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.483+0000 7f969b5d3740 10 client.0 _ll_register_callbacks cb 0x7f967c00bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.483+0000 7f9692bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be13b300 0x5610be4f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.483+0000 7f9692bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be13b300 0x5610be4f5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:59274/0 (socket says 172.21.15.38:59274) 2024-05-21T15:41:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.483+0000 7f9692bf3640 1 -- 172.21.15.38:0/3035634289 learned_addr learned my addr 172.21.15.38:0/3035634289 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.483+0000 7f96923f2640 1 --2- 172.21.15.38:0/3035634289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be11e2f0 0x5610be4f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.484+0000 7f9692bf3640 1 -- 172.21.15.38:0/3035634289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be11e2f0 msgr2=0x5610be4f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.484+0000 7f9692bf3640 1 --2- 172.21.15.38:0/3035634289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be11e2f0 0x5610be4f4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.484+0000 7f9692bf3640 1 -- 172.21.15.38:0/3035634289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be1264d0 msgr2=0x5610be4f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.484+0000 7f9692bf3640 1 --2- 172.21.15.38:0/3035634289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be1264d0 0x5610be4f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.484+0000 7f9692bf3640 1 -- 172.21.15.38:0/3035634289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610be4f6460 con 0x5610be13b300 2024-05-21T15:41:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.484+0000 7f9692bf3640 1 --2- 172.21.15.38:0/3035634289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be13b300 0x5610be4f5ea0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9684050000 tx=0x7f9684009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5da286713bf1a16 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.485+0000 7f9675ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:41:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.487+0000 7f9675ffb640 1 -- 172.21.15.38:0/3035634289 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684040040 con 0x5610be13b300 2024-05-21T15:41:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.487+0000 7f9675ffb640 1 -- 172.21.15.38:0/3035634289 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f968402fe20 con 0x5610be13b300 2024-05-21T15:41:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.489+0000 7f969b5d3740 1 -- 172.21.15.38:0/3035634289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9688016df0 con 0x5610be13b300 2024-05-21T15:41:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.490+0000 7f9675ffb640 1 -- 172.21.15.38:0/3035634289 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9684004410 con 0x5610be13b300 2024-05-21T15:41:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.491+0000 7f969b5d3740 10 client.26906 fetch_fsmap learned FSMap version 33 2024-05-21T15:41:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.491+0000 7f969b5d3740 1 -- 172.21.15.38:0/3035634289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5610be4f7720 con 0x5610be13b300 2024-05-21T15:41:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.496+0000 7f9675ffb640 1 -- 172.21.15.38:0/3035634289 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9688016df0 con 0x5610be13b300 2024-05-21T15:41:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.496+0000 7f9675ffb640 1 -- 172.21.15.38:0/3035634289 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f968404b070 con 0x5610be13b300 2024-05-21T15:41:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.497+0000 7f969b5d3740 10 client.26906 fetch_fsmap finished waiting for FSMap version 33 2024-05-21T15:41:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.497+0000 7f969b5d3740 10 client.26906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.497+0000 7f969b5d3740 20 client.26906 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.497+0000 7f969b5d3740 1 --2- 172.21.15.38:0/3035634289 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5610be1471b0 0x5610be568a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.497+0000 7f969b5d3740 4 client.26906 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:41:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.497+0000 7f969b5d3740 1 -- 172.21.15.38:0/3035634289 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610be452dc0 con 0x5610be1471b0 2024-05-21T15:41:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.498+0000 7f96923f2640 1 --2- 172.21.15.38:0/3035634289 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5610be1471b0 0x5610be568a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.499+0000 7f96923f2640 1 --2- 172.21.15.38:0/3035634289 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5610be1471b0 0x5610be568a20 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7da67fe885994c6 server_cookie=bf1de8488c855155 in_seq=0 out_seq=0 2024-05-21T15:41:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:08.500+0000 7f9675ffb640 10 client.26906 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.749+0000 7f9675ffb640 1 -- 172.21.15.38:0/3035634289 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5610be452dc0 con 0x5610be1471b0 2024-05-21T15:41:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.749+0000 7f9675ffb640 10 client.26906 handle_command_reply: tid=0 2024-05-21T15:41:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.749+0000 7f969b5d3740 1 client.26906 shutdown 2024-05-21T15:41:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.749+0000 7f969b5d3740 1 -- 172.21.15.38:0/3035634289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be13b300 msgr2=0x5610be4f5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.749+0000 7f969b5d3740 1 --2- 172.21.15.38:0/3035634289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be13b300 0x5610be4f5ea0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9684050000 tx=0x7f9684009780 comp rx=0 tx=0).stop 2024-05-21T15:41:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.749+0000 7f969b5d3740 1 -- 172.21.15.38:0/3035634289 shutdown_connections 2024-05-21T15:41:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.749+0000 7f969b5d3740 1 --2- 172.21.15.38:0/3035634289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5610be13b300 0x5610be4f5ea0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.749+0000 7f969b5d3740 1 --2- 172.21.15.38:0/3035634289 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5610be1471b0 0x5610be568a20 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.749+0000 7f969b5d3740 1 --2- 172.21.15.38:0/3035634289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5610be1264d0 0x5610be4f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.749+0000 7f969b5d3740 1 --2- 172.21.15.38:0/3035634289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5610be11e2f0 0x5610be4f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.749+0000 7f969b5d3740 1 -- 172.21.15.38:0/3035634289 >> 172.21.15.38:0/3035634289 conn(0x5610be4dd870 msgr2=0x5610be4f0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.750+0000 7f9675ffb640 0 client.26906 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.750+0000 7f969b5d3740 1 -- 172.21.15.38:0/3035634289 shutdown_connections 2024-05-21T15:41:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.750+0000 7f969b5d3740 1 -- 172.21.15.38:0/3035634289 wait complete. 2024-05-21T15:41:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.750+0000 7f969b5d3740 20 client.26906 trim_cache size 0 max 16384 2024-05-21T15:41:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:41:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:41:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:41:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:41:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.751+0000 7f9694656640 1 -- 172.21.15.38:0/1905051177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f96780a2e70 msgr2=0x7f96780a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.751+0000 7f9694656640 1 --2- 172.21.15.38:0/1905051177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f96780a2e70 0x7f96780a5380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9684031a10 tx=0x7f9684009820 comp rx=0 tx=0).stop 2024-05-21T15:41:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.751+0000 7f9694656640 1 -- 172.21.15.38:0/1905051177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f968c1013b0 msgr2=0x7f968c111460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.751+0000 7f9694656640 1 --2- 172.21.15.38:0/1905051177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f968c1013b0 0x7f968c111460 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f967c009fd0 tx=0x7f967c00b870 comp rx=0 tx=0).stop 2024-05-21T15:41:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.751+0000 7f9694656640 1 -- 172.21.15.38:0/1905051177 shutdown_connections 2024-05-21T15:41:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.751+0000 7f9694656640 1 --2- 172.21.15.38:0/1905051177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f96780a2e70 0x7f96780a5380 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.751+0000 7f9694656640 1 --2- 172.21.15.38:0/1905051177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f968c106670 0x7f968c110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.751+0000 7f9694656640 1 --2- 172.21.15.38:0/1905051177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c103d10 0x7f968c10fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.751+0000 7f9694656640 1 --2- 172.21.15.38:0/1905051177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f968c1013b0 0x7f968c111460 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.751+0000 7f9694656640 1 -- 172.21.15.38:0/1905051177 >> 172.21.15.38:0/1905051177 conn(0x7f968c0fcbc0 msgr2=0x7f968c0fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.751+0000 7f9694656640 1 -- 172.21.15.38:0/1905051177 shutdown_connections 2024-05-21T15:41:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:09.751+0000 7f9694656640 1 -- 172.21.15.38:0/1905051177 wait complete. 2024-05-21T15:41:09.810 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:09 smithi045 ceph-mon[36409]: pgmap v553: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 7.08k op/s 2024-05-21T15:41:09.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:09 smithi053 ceph-mon[36496]: pgmap v553: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 7.08k op/s 2024-05-21T15:41:10.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:09 smithi038 ceph-mon[31995]: pgmap v553: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 7.08k op/s 2024-05-21T15:41:10.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:10 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:41:11.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:10 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:41:11.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:10 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:41:11.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:11 smithi053 ceph-mon[36496]: pgmap v554: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 7.29k op/s 2024-05-21T15:41:12.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:11 smithi045 ceph-mon[36409]: pgmap v554: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 7.29k op/s 2024-05-21T15:41:12.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:11 smithi038 ceph-mon[31995]: pgmap v554: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 7.29k op/s 2024-05-21T15:41:13.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:13 smithi053 ceph-mon[36496]: pgmap v555: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 7.03k op/s 2024-05-21T15:41:14.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:13 smithi045 ceph-mon[36409]: pgmap v555: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 7.03k op/s 2024-05-21T15:41:14.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:13 smithi038 ceph-mon[31995]: pgmap v555: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 7.03k op/s 2024-05-21T15:41:15.825 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:15 smithi038 ceph-mon[31995]: pgmap v556: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 33 MiB/s wr, 7.44k op/s 2024-05-21T15:41:15.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:15 smithi053 ceph-mon[36496]: pgmap v556: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 33 MiB/s wr, 7.44k op/s 2024-05-21T15:41:16.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:15 smithi045 ceph-mon[36409]: pgmap v556: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 33 MiB/s wr, 7.44k op/s 2024-05-21T15:41:17.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:17 smithi053 ceph-mon[36496]: pgmap v557: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 MiB/s wr, 7.42k op/s 2024-05-21T15:41:18.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:17 smithi045 ceph-mon[36409]: pgmap v557: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 MiB/s wr, 7.42k op/s 2024-05-21T15:41:18.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:17 smithi038 ceph-mon[31995]: pgmap v557: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 33 MiB/s wr, 7.42k op/s 2024-05-21T15:41:19.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:19 smithi053 ceph-mon[36496]: pgmap v558: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 6.63k op/s 2024-05-21T15:41:20.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:19 smithi045 ceph-mon[36409]: pgmap v558: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 6.63k op/s 2024-05-21T15:41:20.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:19 smithi038 ceph-mon[31995]: pgmap v558: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 6.63k op/s 2024-05-21T15:41:21.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:21 smithi053 ceph-mon[36496]: pgmap v559: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 MiB/s wr, 7.60k op/s 2024-05-21T15:41:22.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:21 smithi045 ceph-mon[36409]: pgmap v559: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 MiB/s wr, 7.60k op/s 2024-05-21T15:41:22.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:21 smithi038 ceph-mon[31995]: pgmap v559: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 MiB/s wr, 7.60k op/s 2024-05-21T15:41:22.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:41:22.164 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:41:22.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.315+0000 7fe74038f640 1 -- 172.21.15.38:0/1988463370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7381019d0 msgr2=0x7fe7381026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.315+0000 7fe74038f640 1 --2- 172.21.15.38:0/1988463370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7381019d0 0x7fe7381026d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe734009900 tx=0x7fe73402a410 comp rx=0 tx=0).stop 2024-05-21T15:41:22.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.315+0000 7fe74038f640 1 -- 172.21.15.38:0/1988463370 shutdown_connections 2024-05-21T15:41:22.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.315+0000 7fe74038f640 1 --2- 172.21.15.38:0/1988463370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe738107910 0x7fe738109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.315+0000 7fe74038f640 1 --2- 172.21.15.38:0/1988463370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe738102c10 0x7fe738103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.315+0000 7fe74038f640 1 --2- 172.21.15.38:0/1988463370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7381019d0 0x7fe7381026d0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.315+0000 7fe74038f640 1 -- 172.21.15.38:0/1988463370 >> 172.21.15.38:0/1988463370 conn(0x7fe7380fcbc0 msgr2=0x7fe7380ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:22.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.316+0000 7fe74038f640 1 -- 172.21.15.38:0/1988463370 shutdown_connections 2024-05-21T15:41:22.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.316+0000 7fe74038f640 1 -- 172.21.15.38:0/1988463370 wait complete. 2024-05-21T15:41:22.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.316+0000 7fe74038f640 1 Processor -- start 2024-05-21T15:41:22.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.316+0000 7fe74038f640 1 -- start start 2024-05-21T15:41:22.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe74038f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7381019d0 0x7fe73819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe74038f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe738102c10 0x7fe7381a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe74038f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe738107910 0x7fe7381a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe74038f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe73810c550 con 0x7fe7381019d0 2024-05-21T15:41:22.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe74038f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe73810c3d0 con 0x7fe738107910 2024-05-21T15:41:22.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe74038f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe73810c6d0 con 0x7fe738102c10 2024-05-21T15:41:22.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe73e12b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7381019d0 0x7fe73819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe73d92a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe738102c10 0x7fe7381a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe73d92a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe738102c10 0x7fe7381a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:59296/0 (socket says 172.21.15.38:59296) 2024-05-21T15:41:22.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe73d92a640 1 -- 172.21.15.38:0/3899714406 learned_addr learned my addr 172.21.15.38:0/3899714406 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:22.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe73e92c640 1 --2- 172.21.15.38:0/3899714406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe738107910 0x7fe7381a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe73d92a640 1 -- 172.21.15.38:0/3899714406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe738107910 msgr2=0x7fe7381a70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe73d92a640 1 --2- 172.21.15.38:0/3899714406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe738107910 0x7fe7381a70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe73d92a640 1 -- 172.21.15.38:0/3899714406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7381019d0 msgr2=0x7fe73819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe73d92a640 1 --2- 172.21.15.38:0/3899714406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7381019d0 0x7fe73819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe73d92a640 1 -- 172.21.15.38:0/3899714406 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7381a7720 con 0x7fe738102c10 2024-05-21T15:41:22.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.317+0000 7fe73e12b640 1 --2- 172.21.15.38:0/3899714406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7381019d0 0x7fe73819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:22.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.318+0000 7fe73d92a640 1 --2- 172.21.15.38:0/3899714406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe738102c10 0x7fe7381a0120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe72800e9f0 tx=0x7fe72800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:22.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.318+0000 7fe72f7fe640 1 -- 172.21.15.38:0/3899714406 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72800cd80 con 0x7fe738102c10 2024-05-21T15:41:22.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.318+0000 7fe72f7fe640 1 -- 172.21.15.38:0/3899714406 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe728010910 con 0x7fe738102c10 2024-05-21T15:41:22.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.318+0000 7fe72f7fe640 1 -- 172.21.15.38:0/3899714406 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728018990 con 0x7fe738102c10 2024-05-21T15:41:22.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.318+0000 7fe74038f640 1 -- 172.21.15.38:0/3899714406 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7381a7a10 con 0x7fe738102c10 2024-05-21T15:41:22.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.318+0000 7fe74038f640 1 -- 172.21.15.38:0/3899714406 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7381a7f10 con 0x7fe738102c10 2024-05-21T15:41:22.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.322+0000 7fe74723b740 1 Processor -- start 2024-05-21T15:41:22.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.322+0000 7fe74723b740 1 -- start start 2024-05-21T15:41:22.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.323+0000 7fe74723b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d431390 0x55fe5d7e1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.323+0000 7fe74723b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4084d0 0x55fe5d7e2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.323+0000 7fe74723b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4106b0 0x55fe5d7e9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.323+0000 7fe74723b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fe5d735d00 con 0x55fe5d4084d0 2024-05-21T15:41:22.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.323+0000 7fe74723b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fe5d6dd4c0 con 0x55fe5d431390 2024-05-21T15:41:22.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.323+0000 7fe74723b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55fe5d731cc0 con 0x55fe5d4106b0 2024-05-21T15:41:22.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.323+0000 7fe73e12b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4084d0 0x55fe5d7e2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.323+0000 7fe73e12b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4084d0 0x55fe5d7e2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40776/0 (socket says 172.21.15.38:40776) 2024-05-21T15:41:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.323+0000 7fe73e12b640 1 -- 172.21.15.38:0/1586686728 learned_addr learned my addr 172.21.15.38:0/1586686728 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.323+0000 7fe73d92a640 1 --2- 172.21.15.38:0/1586686728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4106b0 0x55fe5d7e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.323+0000 7fe72f7fe640 1 -- 172.21.15.38:0/3899714406 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe728010ac0 con 0x7fe738102c10 2024-05-21T15:41:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.324+0000 7fe73e92c640 1 --2- 172.21.15.38:0/1586686728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d431390 0x55fe5d7e1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.324+0000 7fe73e12b640 1 -- 172.21.15.38:0/1586686728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4106b0 msgr2=0x55fe5d7e9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.324+0000 7fe73e12b640 1 --2- 172.21.15.38:0/1586686728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4106b0 0x55fe5d7e9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.324+0000 7fe73e12b640 1 -- 172.21.15.38:0/1586686728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d431390 msgr2=0x55fe5d7e1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.324+0000 7fe73e12b640 1 --2- 172.21.15.38:0/1586686728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d431390 0x55fe5d7e1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.324+0000 7fe73e12b640 1 -- 172.21.15.38:0/1586686728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe5d5137d0 con 0x55fe5d4084d0 2024-05-21T15:41:22.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.324+0000 7fe73e12b640 1 --2- 172.21.15.38:0/1586686728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4084d0 0x55fe5d7e2140 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe734031440 tx=0x7fe734009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9160dbb296963af server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.325+0000 7fe72f7fe640 1 --2- 172.21.15.38:0/3899714406 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7240a2d60 0x7fe7240a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.325+0000 7fe73e92c640 1 --2- 172.21.15.38:0/3899714406 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7240a2d60 0x7fe7240a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.325+0000 7fe72f7fe640 1 -- 172.21.15.38:0/3899714406 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe7280cf2f0 con 0x7fe738102c10 2024-05-21T15:41:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.325+0000 7fe72cbfe640 1 -- 172.21.15.38:0/1586686728 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734035040 con 0x55fe5d4084d0 2024-05-21T15:41:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.326+0000 7fe72cbfe640 1 -- 172.21.15.38:0/1586686728 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe734004740 con 0x55fe5d4084d0 2024-05-21T15:41:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.326+0000 7fe72cbfe640 1 -- 172.21.15.38:0/1586686728 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734031a20 con 0x55fe5d4084d0 2024-05-21T15:41:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.327+0000 7fe74723b740 1 -- 172.21.15.38:0/1586686728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4084d0 msgr2=0x55fe5d7e2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.327+0000 7fe74723b740 1 --2- 172.21.15.38:0/1586686728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4084d0 0x55fe5d7e2140 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe734031440 tx=0x7fe734009640 comp rx=0 tx=0).stop 2024-05-21T15:41:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.327+0000 7fe73e92c640 1 --2- 172.21.15.38:0/3899714406 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7240a2d60 0x7fe7240a5270 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55fe5d7e2d90 tx=0x7fe730009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:22.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.327+0000 7fe74723b740 1 -- 172.21.15.38:0/1586686728 shutdown_connections 2024-05-21T15:41:22.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.327+0000 7fe74723b740 1 --2- 172.21.15.38:0/1586686728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4106b0 0x55fe5d7e9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.327+0000 7fe74723b740 1 --2- 172.21.15.38:0/1586686728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4084d0 0x55fe5d7e2140 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.327+0000 7fe74723b740 1 --2- 172.21.15.38:0/1586686728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d431390 0x55fe5d7e1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.327+0000 7fe74723b740 1 -- 172.21.15.38:0/1586686728 >> 172.21.15.38:0/1586686728 conn(0x55fe5d4254e0 msgr2=0x55fe5d7cc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:22.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.327+0000 7fe74723b740 1 -- 172.21.15.38:0/1586686728 shutdown_connections 2024-05-21T15:41:22.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.327+0000 7fe74723b740 1 -- 172.21.15.38:0/1586686728 wait complete. 2024-05-21T15:41:22.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.328+0000 7fe74723b740 1 -- start start 2024-05-21T15:41:22.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.328+0000 7fe74723b740 1 Processor -- start 2024-05-21T15:41:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.328+0000 7fe74723b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d4084d0 0x55fe5d7ebae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.328+0000 7fe74723b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4106b0 0x55fe5d7ec020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.328+0000 7fe74723b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d431390 0x55fe5d7f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.328+0000 7fe74723b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fe5d7e9a80 con 0x55fe5d4106b0 2024-05-21T15:41:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.328+0000 7fe74723b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fe5d6f0b90 con 0x55fe5d4084d0 2024-05-21T15:41:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.328+0000 7fe74723b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55fe5d7e9c00 con 0x55fe5d431390 2024-05-21T15:41:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.328+0000 7fe74723b740 10 client.0 _ll_register_callbacks cb 0x55fe5d72cda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.328+0000 7fe73e12b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d4084d0 0x55fe5d7ebae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.328+0000 7fe73d92a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4106b0 0x55fe5d7ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.328+0000 7fe73d92a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4106b0 0x55fe5d7ec020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40784/0 (socket says 172.21.15.38:40784) 2024-05-21T15:41:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.329+0000 7fe73d92a640 1 -- 172.21.15.38:0/1716679355 learned_addr learned my addr 172.21.15.38:0/1716679355 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.329+0000 7fe73e92c640 1 --2- 172.21.15.38:0/1716679355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d431390 0x55fe5d7f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.329+0000 7fe73d92a640 1 -- 172.21.15.38:0/1716679355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d431390 msgr2=0x55fe5d7f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.329+0000 7fe73d92a640 1 --2- 172.21.15.38:0/1716679355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d431390 0x55fe5d7f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.329+0000 7fe73d92a640 1 -- 172.21.15.38:0/1716679355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d4084d0 msgr2=0x55fe5d7ebae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.329+0000 7fe73d92a640 1 --2- 172.21.15.38:0/1716679355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d4084d0 0x55fe5d7ebae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.329+0000 7fe73d92a640 1 -- 172.21.15.38:0/1716679355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe5d4493f0 con 0x55fe5d4106b0 2024-05-21T15:41:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.329+0000 7fe73e92c640 1 --2- 172.21.15.38:0/1716679355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d431390 0x55fe5d7f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.329+0000 7fe73d92a640 1 --2- 172.21.15.38:0/1716679355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4106b0 0x55fe5d7ec020 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe7280a41e0 tx=0x7fe72809b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792afd730620e0c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.329+0000 7fe721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:22.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.329+0000 7fe721ffb640 1 -- 172.21.15.38:0/1716679355 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728018d30 con 0x55fe5d4106b0 2024-05-21T15:41:22.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.329+0000 7fe721ffb640 1 -- 172.21.15.38:0/1716679355 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe72809afa0 con 0x55fe5d4106b0 2024-05-21T15:41:22.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.330+0000 7fe74723b740 1 -- 172.21.15.38:0/1716679355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe734004740 con 0x55fe5d4106b0 2024-05-21T15:41:22.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.330+0000 7fe721ffb640 1 -- 172.21.15.38:0/1716679355 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe7280d2e20 con 0x55fe5d4106b0 2024-05-21T15:41:22.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.330+0000 7fe74723b740 10 client.17301 fetch_fsmap learned FSMap version 34 2024-05-21T15:41:22.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.330+0000 7fe74723b740 1 -- 172.21.15.38:0/1716679355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fe5d4090f0 con 0x55fe5d4106b0 2024-05-21T15:41:22.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.330+0000 7fe721ffb640 1 -- 172.21.15.38:0/1716679355 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe734004740 con 0x55fe5d4106b0 2024-05-21T15:41:22.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.330+0000 7fe721ffb640 1 -- 172.21.15.38:0/1716679355 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fe7280978a0 con 0x55fe5d4106b0 2024-05-21T15:41:22.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.331+0000 7fe74723b740 10 client.17301 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T15:41:22.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.331+0000 7fe74723b740 10 client.17301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:22.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.331+0000 7fe74723b740 20 client.17301 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:22.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.331+0000 7fe74723b740 1 --2- 172.21.15.38:0/1716679355 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fe5d3eff60 0x55fe5d7f9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.331+0000 7fe74723b740 4 client.17301 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:22.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.331+0000 7fe74723b740 1 -- 172.21.15.38:0/1716679355 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe5d743430 con 0x55fe5d3eff60 2024-05-21T15:41:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.331+0000 7fe73e12b640 1 --2- 172.21.15.38:0/1716679355 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fe5d3eff60 0x55fe5d7f9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.331+0000 7fe721ffb640 10 client.17301 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.332+0000 7fe73e12b640 1 --2- 172.21.15.38:0/1716679355 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fe5d3eff60 0x55fe5d7f9890 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c6f4440e818c5ac server_cookie=17e152553443e75d in_seq=0 out_seq=0 2024-05-21T15:41:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe721ffb640 1 -- 172.21.15.38:0/1716679355 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe5d743430 con 0x55fe5d3eff60 2024-05-21T15:41:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe721ffb640 10 client.17301 handle_command_reply: tid=0 2024-05-21T15:41:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe74723b740 1 client.17301 shutdown 2024-05-21T15:41:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe74723b740 1 -- 172.21.15.38:0/1716679355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4106b0 msgr2=0x55fe5d7ec020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe74723b740 1 --2- 172.21.15.38:0/1716679355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4106b0 0x55fe5d7ec020 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe7280a41e0 tx=0x7fe72809b920 comp rx=0 tx=0).stop 2024-05-21T15:41:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe74723b740 1 -- 172.21.15.38:0/1716679355 shutdown_connections 2024-05-21T15:41:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe74723b740 1 --2- 172.21.15.38:0/1716679355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d431390 0x55fe5d7f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe74723b740 1 --2- 172.21.15.38:0/1716679355 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fe5d3eff60 0x55fe5d7f9890 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe74723b740 1 --2- 172.21.15.38:0/1716679355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4106b0 0x55fe5d7ec020 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe74723b740 1 --2- 172.21.15.38:0/1716679355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d4084d0 0x55fe5d7ebae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe74723b740 1 -- 172.21.15.38:0/1716679355 >> 172.21.15.38:0/1716679355 conn(0x55fe5d4254e0 msgr2=0x55fe5d7cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe721ffb640 0 client.17301 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:22.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe74723b740 1 -- 172.21.15.38:0/1716679355 shutdown_connections 2024-05-21T15:41:22.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe74723b740 1 -- 172.21.15.38:0/1716679355 wait complete. 2024-05-21T15:41:22.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.333+0000 7fe74723b740 20 client.17301 trim_cache size 0 max 16384 2024-05-21T15:41:22.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.358+0000 7fe74723b740 1 Processor -- start 2024-05-21T15:41:22.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.358+0000 7fe74723b740 1 -- start start 2024-05-21T15:41:22.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.359+0000 7fe74723b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4084d0 0x55fe5d7ea730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.359+0000 7fe74723b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4106b0 0x55fe5d7f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.359+0000 7fe74723b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d431390 0x55fe5d7cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.359+0000 7fe74723b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe7381a8280 con 0x55fe5d4084d0 2024-05-21T15:41:22.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.359+0000 7fe73e12b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4084d0 0x55fe5d7ea730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.359+0000 7fe73e12b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4084d0 0x55fe5d7ea730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40792/0 (socket says 172.21.15.38:40792) 2024-05-21T15:41:22.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.359+0000 7fe73e12b640 1 -- 172.21.15.38:0/1302888836 learned_addr learned my addr 172.21.15.38:0/1302888836 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:22.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.359+0000 7fe73e92c640 1 --2- 172.21.15.38:0/1302888836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d431390 0x55fe5d7cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.359+0000 7fe74723b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe7381a9df0 con 0x55fe5d431390 2024-05-21T15:41:22.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.359+0000 7fe74723b740 1 -- 172.21.15.38:0/1302888836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe7381a8950 con 0x55fe5d4106b0 2024-05-21T15:41:22.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.359+0000 7fe73d92a640 1 --2- 172.21.15.38:0/1302888836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4106b0 0x55fe5d7f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.360+0000 7fe73e12b640 1 -- 172.21.15.38:0/1302888836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4106b0 msgr2=0x55fe5d7f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.360+0000 7fe73e12b640 1 --2- 172.21.15.38:0/1302888836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4106b0 0x55fe5d7f2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.360+0000 7fe73e12b640 1 -- 172.21.15.38:0/1302888836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d431390 msgr2=0x55fe5d7cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.360+0000 7fe73e12b640 1 --2- 172.21.15.38:0/1302888836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d431390 0x55fe5d7cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.360+0000 7fe73e12b640 1 -- 172.21.15.38:0/1302888836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe5d3f0190 con 0x55fe5d4084d0 2024-05-21T15:41:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.360+0000 7fe73e12b640 1 --2- 172.21.15.38:0/1302888836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4084d0 0x55fe5d7ea730 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe734009900 tx=0x7fe73402aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a7051851e5c600 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.360+0000 7fe723fff640 1 -- 172.21.15.38:0/1302888836 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734035040 con 0x55fe5d4084d0 2024-05-21T15:41:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.360+0000 7fe723fff640 1 -- 172.21.15.38:0/1302888836 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7340054e0 con 0x55fe5d4084d0 2024-05-21T15:41:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.360+0000 7fe723fff640 1 -- 172.21.15.38:0/1302888836 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734031cb0 con 0x55fe5d4084d0 2024-05-21T15:41:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.362+0000 7fe74723b740 1 -- 172.21.15.38:0/1302888836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4084d0 msgr2=0x55fe5d7ea730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.362+0000 7fe74723b740 1 --2- 172.21.15.38:0/1302888836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4084d0 0x55fe5d7ea730 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe734009900 tx=0x7fe73402aa90 comp rx=0 tx=0).stop 2024-05-21T15:41:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.362+0000 7fe74723b740 1 -- 172.21.15.38:0/1302888836 shutdown_connections 2024-05-21T15:41:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.362+0000 7fe74723b740 1 --2- 172.21.15.38:0/1302888836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d431390 0x55fe5d7cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.362+0000 7fe74723b740 1 --2- 172.21.15.38:0/1302888836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4106b0 0x55fe5d7f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.362+0000 7fe74723b740 1 --2- 172.21.15.38:0/1302888836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d4084d0 0x55fe5d7ea730 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.362+0000 7fe74723b740 1 -- 172.21.15.38:0/1302888836 >> 172.21.15.38:0/1302888836 conn(0x55fe5d4254e0 msgr2=0x55fe5d7ea020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.362+0000 7fe74723b740 1 -- 172.21.15.38:0/1302888836 shutdown_connections 2024-05-21T15:41:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.362+0000 7fe74723b740 1 -- 172.21.15.38:0/1302888836 wait complete. 2024-05-21T15:41:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.362+0000 7fe74723b740 1 -- start start 2024-05-21T15:41:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.362+0000 7fe74723b740 1 Processor -- start 2024-05-21T15:41:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe74723b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4084d0 0x55fe5d7efe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe74723b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d4106b0 0x55fe5d7f0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe74723b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d431390 0x55fe5d85fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe74723b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe7381a96f0 con 0x55fe5d431390 2024-05-21T15:41:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe74723b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe7381a9020 con 0x55fe5d4106b0 2024-05-21T15:41:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe74723b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe738071ea0 con 0x55fe5d4084d0 2024-05-21T15:41:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe74723b740 10 client.0 _ll_register_callbacks cb 0x7fe728009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe73e12b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4084d0 0x55fe5d7efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe73e12b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4084d0 0x55fe5d7efe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:59330/0 (socket says 172.21.15.38:59330) 2024-05-21T15:41:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe73e12b640 1 -- 172.21.15.38:0/951927619 learned_addr learned my addr 172.21.15.38:0/951927619 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe73e92c640 1 --2- 172.21.15.38:0/951927619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d431390 0x55fe5d85fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe73d92a640 1 --2- 172.21.15.38:0/951927619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d4106b0 0x55fe5d7f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe73e12b640 1 -- 172.21.15.38:0/951927619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d4106b0 msgr2=0x55fe5d7f0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe73e12b640 1 --2- 172.21.15.38:0/951927619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d4106b0 0x55fe5d7f0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe73e12b640 1 -- 172.21.15.38:0/951927619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d431390 msgr2=0x55fe5d85fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe73e12b640 1 --2- 172.21.15.38:0/951927619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d431390 0x55fe5d85fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.363+0000 7fe73e12b640 1 -- 172.21.15.38:0/951927619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe5d860130 con 0x55fe5d4084d0 2024-05-21T15:41:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.364+0000 7fe73e12b640 1 --2- 172.21.15.38:0/951927619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4084d0 0x55fe5d7efe50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe7340049a0 tx=0x7fe734005a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40fdace218cce897 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.364+0000 7fe721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:41:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.364+0000 7fe721ffb640 1 -- 172.21.15.38:0/951927619 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734062060 con 0x55fe5d4084d0 2024-05-21T15:41:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.364+0000 7fe721ffb640 1 -- 172.21.15.38:0/951927619 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe734005200 con 0x55fe5d4084d0 2024-05-21T15:41:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.364+0000 7fe74723b740 1 -- 172.21.15.38:0/951927619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7340054e0 con 0x55fe5d4084d0 2024-05-21T15:41:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.364+0000 7fe721ffb640 1 -- 172.21.15.38:0/951927619 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe734035040 con 0x55fe5d4084d0 2024-05-21T15:41:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.364+0000 7fe74723b740 10 client.26989 fetch_fsmap learned FSMap version 34 2024-05-21T15:41:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.364+0000 7fe74723b740 1 -- 172.21.15.38:0/951927619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fe5d861290 con 0x55fe5d4084d0 2024-05-21T15:41:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.365+0000 7fe721ffb640 1 -- 172.21.15.38:0/951927619 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe734004740 con 0x55fe5d4084d0 2024-05-21T15:41:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.365+0000 7fe721ffb640 1 -- 172.21.15.38:0/951927619 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fe734054500 con 0x55fe5d4084d0 2024-05-21T15:41:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.365+0000 7fe74723b740 10 client.26989 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T15:41:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.365+0000 7fe74723b740 10 client.26989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.365+0000 7fe74723b740 20 client.26989 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.365+0000 7fe74723b740 1 --2- 172.21.15.38:0/951927619 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fe5d3c53a0 0x55fe5d8637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.365+0000 7fe74723b740 4 client.26989 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:41:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.365+0000 7fe74723b740 1 -- 172.21.15.38:0/951927619 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe7340054e0 con 0x55fe5d3c53a0 2024-05-21T15:41:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.365+0000 7fe73d92a640 1 --2- 172.21.15.38:0/951927619 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fe5d3c53a0 0x55fe5d8637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.367+0000 7fe721ffb640 10 client.26989 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.367+0000 7fe73d92a640 1 --2- 172.21.15.38:0/951927619 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fe5d3c53a0 0x55fe5d8637b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c151a9e8aba846d server_cookie=8296c5db66d693cc in_seq=0 out_seq=0 2024-05-21T15:41:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.367+0000 7fe721ffb640 1 -- 172.21.15.38:0/951927619 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fe7340054e0 con 0x55fe5d3c53a0 2024-05-21T15:41:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.367+0000 7fe721ffb640 10 client.26989 handle_command_reply: tid=0 2024-05-21T15:41:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.367+0000 7fe74723b740 1 client.26989 shutdown 2024-05-21T15:41:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.367+0000 7fe74723b740 1 -- 172.21.15.38:0/951927619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4084d0 msgr2=0x55fe5d7efe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.367+0000 7fe74723b740 1 --2- 172.21.15.38:0/951927619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4084d0 0x55fe5d7efe50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe7340049a0 tx=0x7fe734005a60 comp rx=0 tx=0).stop 2024-05-21T15:41:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.367+0000 7fe74723b740 1 -- 172.21.15.38:0/951927619 shutdown_connections 2024-05-21T15:41:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.367+0000 7fe74723b740 1 --2- 172.21.15.38:0/951927619 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55fe5d3c53a0 0x55fe5d8637b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.367+0000 7fe74723b740 1 --2- 172.21.15.38:0/951927619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fe5d431390 0x55fe5d85fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.367+0000 7fe721ffb640 0 client.26989 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.367+0000 7fe74723b740 1 --2- 172.21.15.38:0/951927619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe5d4106b0 0x55fe5d7f0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.367+0000 7fe74723b740 1 --2- 172.21.15.38:0/951927619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe5d4084d0 0x55fe5d7efe50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.367+0000 7fe74723b740 1 -- 172.21.15.38:0/951927619 >> 172.21.15.38:0/951927619 conn(0x55fe5d4254e0 msgr2=0x55fe5d7cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74723b740 1 -- 172.21.15.38:0/951927619 shutdown_connections 2024-05-21T15:41:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74723b740 1 -- 172.21.15.38:0/951927619 wait complete. 2024-05-21T15:41:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74723b740 20 client.26989 trim_cache size 0 max 16384 2024-05-21T15:41:22.370 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:41:22.370 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-05-21T15:41:22.370 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-05-21T15:41:22.370 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:41:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74038f640 1 -- 172.21.15.38:0/3899714406 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7240a2d60 msgr2=0x7fe7240a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74038f640 1 --2- 172.21.15.38:0/3899714406 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7240a2d60 0x7fe7240a5270 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55fe5d7e2d90 tx=0x7fe730009500 comp rx=0 tx=0).stop 2024-05-21T15:41:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74038f640 1 -- 172.21.15.38:0/3899714406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe738102c10 msgr2=0x7fe7381a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74038f640 1 --2- 172.21.15.38:0/3899714406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe738102c10 0x7fe7381a0120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe72800e9f0 tx=0x7fe72800eeb0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74038f640 1 -- 172.21.15.38:0/3899714406 shutdown_connections 2024-05-21T15:41:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74038f640 1 --2- 172.21.15.38:0/3899714406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe738107910 0x7fe7381a70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74038f640 1 --2- 172.21.15.38:0/3899714406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe738102c10 0x7fe7381a0120 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74038f640 1 --2- 172.21.15.38:0/3899714406 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe7240a2d60 0x7fe7240a5270 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74038f640 1 --2- 172.21.15.38:0/3899714406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7381019d0 0x7fe73819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74038f640 1 -- 172.21.15.38:0/3899714406 >> 172.21.15.38:0/3899714406 conn(0x7fe7380fcbc0 msgr2=0x7fe7380fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74038f640 1 -- 172.21.15.38:0/3899714406 shutdown_connections 2024-05-21T15:41:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.368+0000 7fe74038f640 1 -- 172.21.15.38:0/3899714406 wait complete. 2024-05-21T15:41:22.379 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:41:22.379 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:41:22.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.528+0000 7f3d98e21640 1 -- 172.21.15.38:0/3146254839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94105be0 msgr2=0x7f3d941080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.528+0000 7f3d98e21640 1 --2- 172.21.15.38:0/3146254839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94105be0 0x7f3d941080c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009900 tx=0x7f3d8802a410 comp rx=0 tx=0).stop 2024-05-21T15:41:22.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.529+0000 7f3d98e21640 1 -- 172.21.15.38:0/3146254839 shutdown_connections 2024-05-21T15:41:22.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.529+0000 7f3d98e21640 1 --2- 172.21.15.38:0/3146254839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94108750 0x7f3d9410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.529+0000 7f3d98e21640 1 --2- 172.21.15.38:0/3146254839 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94105be0 0x7f3d941080c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.529+0000 7f3d98e21640 1 --2- 172.21.15.38:0/3146254839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d94100e50 0x7f3d94101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.529+0000 7f3d98e21640 1 -- 172.21.15.38:0/3146254839 >> 172.21.15.38:0/3146254839 conn(0x7f3d940fc340 msgr2=0x7f3d940fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:22.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.530+0000 7f3d98e21640 1 -- 172.21.15.38:0/3146254839 shutdown_connections 2024-05-21T15:41:22.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.530+0000 7f3d98e21640 1 -- 172.21.15.38:0/3146254839 wait complete. 2024-05-21T15:41:22.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.531+0000 7f3d98e21640 1 Processor -- start 2024-05-21T15:41:22.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.531+0000 7f3d98e21640 1 -- start start 2024-05-21T15:41:22.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d98e21640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94100e50 0x7f3d94103340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d98e21640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d94105be0 0x7f3d941019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d98e21640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94108750 0x7f3d94101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d98e21640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9410e760 con 0x7f3d94100e50 2024-05-21T15:41:22.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d98e21640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9410e5e0 con 0x7f3d94105be0 2024-05-21T15:41:22.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d98e21640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9410e8e0 con 0x7f3d94108750 2024-05-21T15:41:22.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d9259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94100e50 0x7f3d94103340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d9259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94100e50 0x7f3d94103340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40820/0 (socket says 172.21.15.38:40820) 2024-05-21T15:41:22.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d9259c640 1 -- 172.21.15.38:0/2672379013 learned_addr learned my addr 172.21.15.38:0/2672379013 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:22.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d91d9b640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d94105be0 0x7f3d941019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d92d9d640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94108750 0x7f3d94101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d9259c640 1 -- 172.21.15.38:0/2672379013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94108750 msgr2=0x7f3d94101f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d9259c640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94108750 0x7f3d94101f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d9259c640 1 -- 172.21.15.38:0/2672379013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d94105be0 msgr2=0x7f3d941019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d9259c640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d94105be0 0x7f3d941019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d9259c640 1 -- 172.21.15.38:0/2672379013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9419d090 con 0x7f3d94100e50 2024-05-21T15:41:22.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d92d9d640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94108750 0x7f3d94101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:41:22.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.532+0000 7f3d91d9b640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d94105be0 0x7f3d941019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:22.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.533+0000 7f3d9259c640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94100e50 0x7f3d94103340 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00eb90 tx=0x7f3d8c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:22.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.533+0000 7f3d837fe640 1 -- 172.21.15.38:0/2672379013 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c003b70 con 0x7f3d94100e50 2024-05-21T15:41:22.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.533+0000 7f3d98e21640 1 -- 172.21.15.38:0/2672379013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d94102580 con 0x7f3d94100e50 2024-05-21T15:41:22.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.533+0000 7f3d98e21640 1 -- 172.21.15.38:0/2672379013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d941a7c70 con 0x7f3d94100e50 2024-05-21T15:41:22.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.533+0000 7f3d837fe640 1 -- 172.21.15.38:0/2672379013 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d8c00ee90 con 0x7f3d94100e50 2024-05-21T15:41:22.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.533+0000 7f3d837fe640 1 -- 172.21.15.38:0/2672379013 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c019800 con 0x7f3d94100e50 2024-05-21T15:41:22.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.534+0000 7f3d98e21640 1 -- 172.21.15.38:0/2672379013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d940fde60 con 0x7f3d94100e50 2024-05-21T15:41:22.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.537+0000 7f3d837fe640 1 -- 172.21.15.38:0/2672379013 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3d8c019a40 con 0x7f3d94100e50 2024-05-21T15:41:22.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.537+0000 7f3d837fe640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3d7c0a2cb0 0x7f3d7c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.538+0000 7f3d837fe640 1 -- 172.21.15.38:0/2672379013 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f3d8c0cf8f0 con 0x7f3d94100e50 2024-05-21T15:41:22.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.538+0000 7f3d91d9b640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3d7c0a2cb0 0x7f3d7c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.538+0000 7f3d91d9b640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3d7c0a2cb0 0x7f3d7c0a51c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3d88005ab0 tx=0x7f3d880315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:22.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.543+0000 7f3d837fe640 1 -- 172.21.15.38:0/2672379013 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3d9419d090 con 0x7f3d94100e50 2024-05-21T15:41:22.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.860+0000 7f3d98e21640 1 -- 172.21.15.38:0/2672379013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d9402d660 con 0x7f3d94100e50 2024-05-21T15:41:22.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.861+0000 7f3d837fe640 1 -- 172.21.15.38:0/2672379013 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11915 (secure 0 0 0) 0x7f3d8c09c800 con 0x7f3d94100e50 2024-05-21T15:41:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:41:22.859 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:41:09.081844+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24550,24515,24602,24523,24551]},"id":1}]} 2024-05-21T15:41:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 34 2024-05-21T15:41:22.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.863+0000 7f3d98e21640 1 -- 172.21.15.38:0/2672379013 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3d7c0a2cb0 msgr2=0x7f3d7c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.863+0000 7f3d98e21640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3d7c0a2cb0 0x7f3d7c0a51c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3d88005ab0 tx=0x7f3d880315b0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.863+0000 7f3d98e21640 1 -- 172.21.15.38:0/2672379013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94100e50 msgr2=0x7f3d94103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.863+0000 7f3d98e21640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94100e50 0x7f3d94103340 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00eb90 tx=0x7f3d8c00c6a0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.864+0000 7f3d98e21640 1 -- 172.21.15.38:0/2672379013 shutdown_connections 2024-05-21T15:41:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.864+0000 7f3d98e21640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94108750 0x7f3d94101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.864+0000 7f3d98e21640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d94105be0 0x7f3d941019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.864+0000 7f3d98e21640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3d7c0a2cb0 0x7f3d7c0a51c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.864+0000 7f3d98e21640 1 --2- 172.21.15.38:0/2672379013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94100e50 0x7f3d94103340 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.864+0000 7f3d98e21640 1 -- 172.21.15.38:0/2672379013 >> 172.21.15.38:0/2672379013 conn(0x7f3d940fc340 msgr2=0x7f3d94106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.864+0000 7f3d98e21640 1 -- 172.21.15.38:0/2672379013 shutdown_connections 2024-05-21T15:41:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:22.864+0000 7f3d98e21640 1 -- 172.21.15.38:0/2672379013 wait complete. 2024-05-21T15:41:22.877 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:41:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.897+0000 7f8e2c656640 1 -- 172.21.15.38:0/3415313670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e241019b0 msgr2=0x7f8e241026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.897+0000 7f8e2c656640 1 --2- 172.21.15.38:0/3415313670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e241019b0 0x7f8e241026b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009900 tx=0x7f8e2002a410 comp rx=0 tx=0).stop 2024-05-21T15:41:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.897+0000 7f8e2c656640 1 -- 172.21.15.38:0/3415313670 shutdown_connections 2024-05-21T15:41:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.897+0000 7f8e2c656640 1 --2- 172.21.15.38:0/3415313670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e241078f0 0x7f8e24109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.897+0000 7f8e2c656640 1 --2- 172.21.15.38:0/3415313670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e24102bf0 0x7f8e24103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.897+0000 7f8e2c656640 1 --2- 172.21.15.38:0/3415313670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e241019b0 0x7f8e241026b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.897+0000 7f8e2c656640 1 -- 172.21.15.38:0/3415313670 >> 172.21.15.38:0/3415313670 conn(0x7f8e240fcba0 msgr2=0x7f8e240ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.897+0000 7f8e2c656640 1 -- 172.21.15.38:0/3415313670 shutdown_connections 2024-05-21T15:41:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.897+0000 7f8e2c656640 1 -- 172.21.15.38:0/3415313670 wait complete. 2024-05-21T15:41:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.898+0000 7f8e2c656640 1 Processor -- start 2024-05-21T15:41:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.899+0000 7f8e2c656640 1 -- start start 2024-05-21T15:41:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.899+0000 7f8e2c656640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e241019b0 0x7f8e24199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.899+0000 7f8e2c656640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e24102bf0 0x7f8e241995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.899+0000 7f8e2c656640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e241078f0 0x7f8e241a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.899+0000 7f8e2c656640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2410e750 con 0x7f8e24102bf0 2024-05-21T15:41:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.899+0000 7f8e2c656640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2410e5d0 con 0x7f8e241078f0 2024-05-21T15:41:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.899+0000 7f8e2c656640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2410e8d0 con 0x7f8e241019b0 2024-05-21T15:41:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.899+0000 7f8e2a3f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e241019b0 0x7f8e24199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.899+0000 7f8e2a3f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e241019b0 0x7f8e24199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:59370/0 (socket says 172.21.15.38:59370) 2024-05-21T15:41:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.899+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/3711311588 learned_addr learned my addr 172.21.15.38:0/3711311588 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.899+0000 7f8e29bf1640 1 --2- 172.21.15.38:0/3711311588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e24102bf0 0x7f8e241995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.900+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/3711311588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e241078f0 msgr2=0x7f8e241a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.900+0000 7f8e2a3f2640 1 --2- 172.21.15.38:0/3711311588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e241078f0 0x7f8e241a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.900+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/3711311588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e24102bf0 msgr2=0x7f8e241995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.900+0000 7f8e2a3f2640 1 --2- 172.21.15.38:0/3711311588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e24102bf0 0x7f8e241995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.900+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/3711311588 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e241a0ba0 con 0x7f8e241019b0 2024-05-21T15:41:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.900+0000 7f8e2a3f2640 1 --2- 172.21.15.38:0/3711311588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e241019b0 0x7f8e24199060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009850 tx=0x7f8e20031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.900+0000 7f8e1b7fe640 1 -- 172.21.15.38:0/3711311588 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20035030 con 0x7f8e241019b0 2024-05-21T15:41:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.900+0000 7f8e1b7fe640 1 -- 172.21.15.38:0/3711311588 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e20041070 con 0x7f8e241019b0 2024-05-21T15:41:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.900+0000 7f8e2c656640 1 -- 172.21.15.38:0/3711311588 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2410a800 con 0x7f8e241019b0 2024-05-21T15:41:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.900+0000 7f8e1b7fe640 1 -- 172.21.15.38:0/3711311588 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20046470 con 0x7f8e241019b0 2024-05-21T15:41:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.900+0000 7f8e2c656640 1 -- 172.21.15.38:0/3711311588 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2410ad40 con 0x7f8e241019b0 2024-05-21T15:41:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.903+0000 7f8e1b7fe640 1 -- 172.21.15.38:0/3711311588 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8e2004f3f0 con 0x7f8e241019b0 2024-05-21T15:41:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.903+0000 7f8e335d1740 1 Processor -- start 2024-05-21T15:41:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.904+0000 7f8e1b7fe640 1 --2- 172.21.15.38:0/3711311588 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8e100a2d00 0x7f8e100a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.904+0000 7f8e335d1740 1 -- start start 2024-05-21T15:41:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.904+0000 7f8e1b7fe640 1 -- 172.21.15.38:0/3711311588 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8e200f3ca0 con 0x7f8e241019b0 2024-05-21T15:41:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.904+0000 7f8e2abf3640 1 --2- 172.21.15.38:0/3711311588 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8e100a2d00 0x7f8e100a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.904+0000 7f8e335d1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8389131b0 0x55e838cb2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.904+0000 7f8e335d1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388ea2f0 0x55e838cb33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.904+0000 7f8e335d1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8388f24d0 0x55e838cba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.904+0000 7f8e335d1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e838c17340 con 0x55e8389131b0 2024-05-21T15:41:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.904+0000 7f8e335d1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e838bbf290 con 0x55e8388f24d0 2024-05-21T15:41:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.904+0000 7f8e335d1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e838c13300 con 0x55e8388ea2f0 2024-05-21T15:41:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.904+0000 7f8e2a3f2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8389131b0 0x55e838cb2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.904+0000 7f8e29bf1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388ea2f0 0x55e838cb33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.905+0000 7f8e29bf1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388ea2f0 0x55e838cb33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:59380/0 (socket says 172.21.15.38:59380) 2024-05-21T15:41:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.905+0000 7f8e29bf1640 1 -- 172.21.15.38:0/3047194103 learned_addr learned my addr 172.21.15.38:0/3047194103 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.905+0000 7f8e2a3f2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8389131b0 0x55e838cb2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40854/0 (socket says 172.21.15.38:40854) 2024-05-21T15:41:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.905+0000 7f8e2abf3640 1 --2- 172.21.15.38:0/3711311588 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8e100a2d00 0x7f8e100a5210 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c005c40 tx=0x7f8e1c005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.905+0000 7f8e2abf3640 1 --2- 172.21.15.38:0/3047194103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8388f24d0 0x55e838cba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.905+0000 7f8e29bf1640 1 -- 172.21.15.38:0/3047194103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8388f24d0 msgr2=0x55e838cba2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.905+0000 7f8e29bf1640 1 --2- 172.21.15.38:0/3047194103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8388f24d0 0x55e838cba2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.905+0000 7f8e29bf1640 1 -- 172.21.15.38:0/3047194103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8389131b0 msgr2=0x55e838cb2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.905+0000 7f8e29bf1640 1 --2- 172.21.15.38:0/3047194103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8389131b0 0x55e838cb2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.905+0000 7f8e29bf1640 1 -- 172.21.15.38:0/3047194103 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e838be38e0 con 0x55e8388ea2f0 2024-05-21T15:41:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.905+0000 7f8e29bf1640 1 --2- 172.21.15.38:0/3047194103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388ea2f0 0x55e838cb33e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009c90 tx=0x7f8e14011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96595f1b3182f637 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.905+0000 7f8e18bfe640 1 -- 172.21.15.38:0/3047194103 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14004230 con 0x55e8388ea2f0 2024-05-21T15:41:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.905+0000 7f8e18bfe640 1 -- 172.21.15.38:0/3047194103 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e140043d0 con 0x55e8388ea2f0 2024-05-21T15:41:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.905+0000 7f8e18bfe640 1 -- 172.21.15.38:0/3047194103 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e140197e0 con 0x55e8388ea2f0 2024-05-21T15:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.906+0000 7f8e335d1740 1 -- 172.21.15.38:0/3047194103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388ea2f0 msgr2=0x55e838cb33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.906+0000 7f8e335d1740 1 --2- 172.21.15.38:0/3047194103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388ea2f0 0x55e838cb33e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009c90 tx=0x7f8e14011810 comp rx=0 tx=0).stop 2024-05-21T15:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.906+0000 7f8e335d1740 1 -- 172.21.15.38:0/3047194103 shutdown_connections 2024-05-21T15:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.906+0000 7f8e335d1740 1 --2- 172.21.15.38:0/3047194103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8388f24d0 0x55e838cba2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.906+0000 7f8e335d1740 1 --2- 172.21.15.38:0/3047194103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388ea2f0 0x55e838cb33e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.906+0000 7f8e335d1740 1 --2- 172.21.15.38:0/3047194103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8389131b0 0x55e838cb2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.907+0000 7f8e335d1740 1 -- 172.21.15.38:0/3047194103 >> 172.21.15.38:0/3047194103 conn(0x55e838907300 msgr2=0x55e838cae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.907+0000 7f8e335d1740 1 -- 172.21.15.38:0/3047194103 shutdown_connections 2024-05-21T15:41:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.907+0000 7f8e335d1740 1 -- 172.21.15.38:0/3047194103 wait complete. 2024-05-21T15:41:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.907+0000 7f8e335d1740 1 -- start start 2024-05-21T15:41:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.907+0000 7f8e335d1740 1 Processor -- start 2024-05-21T15:41:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.908+0000 7f8e335d1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8388ea2f0 0x55e838cbcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.908+0000 7f8e335d1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388f24d0 0x55e838cbd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.908+0000 7f8e335d1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8389131b0 0x55e838cc3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.908+0000 7f8e2a3f2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8388ea2f0 0x55e838cbcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.908+0000 7f8e29bf1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388f24d0 0x55e838cbd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.908+0000 7f8e2a3f2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8388ea2f0 0x55e838cbcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50292/0 (socket says 172.21.15.38:50292) 2024-05-21T15:41:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.908+0000 7f8e29bf1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388f24d0 0x55e838cbd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:59382/0 (socket says 172.21.15.38:59382) 2024-05-21T15:41:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.908+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/4221306412 learned_addr learned my addr 172.21.15.38:0/4221306412 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.908+0000 7f8e2abf3640 1 --2- 172.21.15.38:0/4221306412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8389131b0 0x55e838cc3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.908+0000 7f8e335d1740 1 -- 172.21.15.38:0/4221306412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e838cbad30 con 0x55e8389131b0 2024-05-21T15:41:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.908+0000 7f8e335d1740 1 -- 172.21.15.38:0/4221306412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e838bd2790 con 0x55e8388ea2f0 2024-05-21T15:41:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.908+0000 7f8e335d1740 1 -- 172.21.15.38:0/4221306412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e838cbaeb0 con 0x55e8388f24d0 2024-05-21T15:41:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.908+0000 7f8e335d1740 10 client.0 _ll_register_callbacks cb 0x55e838c0da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.909+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/4221306412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388f24d0 msgr2=0x55e838cbd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.909+0000 7f8e2a3f2640 1 --2- 172.21.15.38:0/4221306412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388f24d0 0x55e838cbd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.909+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/4221306412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8389131b0 msgr2=0x55e838cc3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.909+0000 7f8e2a3f2640 1 --2- 172.21.15.38:0/4221306412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8389131b0 0x55e838cc3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.909+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/4221306412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e83892b210 con 0x55e8388ea2f0 2024-05-21T15:41:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.909+0000 7f8e29bf1640 1 --2- 172.21.15.38:0/4221306412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388f24d0 0x55e838cbd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:41:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.909+0000 7f8e2a3f2640 1 --2- 172.21.15.38:0/4221306412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8388ea2f0 0x55e838cbcb30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8e200096c0 tx=0x7f8e2002a910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4a90673b72d06a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.909+0000 7f8e0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:41:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.909+0000 7f8e0dffb640 1 -- 172.21.15.38:0/4221306412 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e200de060 con 0x55e8388ea2f0 2024-05-21T15:41:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.909+0000 7f8e0dffb640 1 -- 172.21.15.38:0/4221306412 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8e2003dc00 con 0x55e8388ea2f0 2024-05-21T15:41:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.909+0000 7f8e335d1740 1 -- 172.21.15.38:0/4221306412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e140043d0 con 0x55e8388ea2f0 2024-05-21T15:41:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.910+0000 7f8e0dffb640 1 -- 172.21.15.38:0/4221306412 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8e200d9070 con 0x55e8388ea2f0 2024-05-21T15:41:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.910+0000 7f8e335d1740 10 client.26939 fetch_fsmap learned FSMap version 34 2024-05-21T15:41:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.910+0000 7f8e335d1740 1 -- 172.21.15.38:0/4221306412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e8389f7620 con 0x55e8388ea2f0 2024-05-21T15:41:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.910+0000 7f8e0dffb640 1 -- 172.21.15.38:0/4221306412 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e200c4840 con 0x55e8388ea2f0 2024-05-21T15:41:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.910+0000 7f8e0dffb640 1 -- 172.21.15.38:0/4221306412 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f8e200c06f0 con 0x55e8388ea2f0 2024-05-21T15:41:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.910+0000 7f8e335d1740 10 client.26939 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T15:41:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.910+0000 7f8e335d1740 10 client.26939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.910+0000 7f8e335d1740 20 client.26939 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.910+0000 7f8e335d1740 1 --2- 172.21.15.38:0/4221306412 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8388d1d80 0x55e838cca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.910+0000 7f8e335d1740 4 client.26939 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.910+0000 7f8e335d1740 1 -- 172.21.15.38:0/4221306412 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8e140043d0 con 0x55e8388d1d80 2024-05-21T15:41:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.911+0000 7f8e29bf1640 1 --2- 172.21.15.38:0/4221306412 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8388d1d80 0x55e838cca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.911+0000 7f8e0dffb640 10 client.26939 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.911+0000 7f8e29bf1640 1 --2- 172.21.15.38:0/4221306412 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8388d1d80 0x55e838cca4f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e6e3b699445800b server_cookie=172bda8aa382a69a in_seq=0 out_seq=0 2024-05-21T15:41:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.912+0000 7f8e0dffb640 1 -- 172.21.15.38:0/4221306412 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f8e140043d0 con 0x55e8388d1d80 2024-05-21T15:41:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.912+0000 7f8e0dffb640 10 client.26939 handle_command_reply: tid=0 2024-05-21T15:41:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.912+0000 7f8e335d1740 1 client.26939 shutdown 2024-05-21T15:41:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.912+0000 7f8e335d1740 1 -- 172.21.15.38:0/4221306412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8388ea2f0 msgr2=0x55e838cbcb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.912+0000 7f8e335d1740 1 --2- 172.21.15.38:0/4221306412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8388ea2f0 0x55e838cbcb30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8e200096c0 tx=0x7f8e2002a910 comp rx=0 tx=0).stop 2024-05-21T15:41:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.913+0000 7f8e335d1740 1 -- 172.21.15.38:0/4221306412 shutdown_connections 2024-05-21T15:41:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.913+0000 7f8e335d1740 1 --2- 172.21.15.38:0/4221306412 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e8388d1d80 0x55e838cca4f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.913+0000 7f8e335d1740 1 --2- 172.21.15.38:0/4221306412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8389131b0 0x55e838cc3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.913+0000 7f8e335d1740 1 --2- 172.21.15.38:0/4221306412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388f24d0 0x55e838cbd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.913+0000 7f8e335d1740 1 --2- 172.21.15.38:0/4221306412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8388ea2f0 0x55e838cbcb30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.913+0000 7f8e335d1740 1 -- 172.21.15.38:0/4221306412 >> 172.21.15.38:0/4221306412 conn(0x55e838907300 msgr2=0x55e838cb17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.913+0000 7f8e0dffb640 0 client.26939 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.913+0000 7f8e335d1740 1 -- 172.21.15.38:0/4221306412 shutdown_connections 2024-05-21T15:41:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.913+0000 7f8e335d1740 1 -- 172.21.15.38:0/4221306412 wait complete. 2024-05-21T15:41:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.913+0000 7f8e335d1740 20 client.26939 trim_cache size 0 max 16384 2024-05-21T15:41:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.935+0000 7f8e335d1740 1 Processor -- start 2024-05-21T15:41:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.935+0000 7f8e335d1740 1 -- start start 2024-05-21T15:41:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.935+0000 7f8e335d1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388ea2f0 0x55e838cae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.935+0000 7f8e335d1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8388f24d0 0x55e838cc3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.935+0000 7f8e335d1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8389131b0 0x55e838cbc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.935+0000 7f8e335d1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241a7f70 con 0x55e8388f24d0 2024-05-21T15:41:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.935+0000 7f8e335d1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241a9ae0 con 0x55e8389131b0 2024-05-21T15:41:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.935+0000 7f8e335d1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241a8640 con 0x55e8388ea2f0 2024-05-21T15:41:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.936+0000 7f8e2a3f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388ea2f0 0x55e838cae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.936+0000 7f8e2a3f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388ea2f0 0x55e838cae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:59388/0 (socket says 172.21.15.38:59388) 2024-05-21T15:41:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.936+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/3222940010 learned_addr learned my addr 172.21.15.38:0/3222940010 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.936+0000 7f8e2abf3640 1 --2- 172.21.15.38:0/3222940010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8389131b0 0x55e838cbc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.936+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/3222940010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8389131b0 msgr2=0x55e838cbc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.936+0000 7f8e2a3f2640 1 --2- 172.21.15.38:0/3222940010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8389131b0 0x55e838cbc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.936+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/3222940010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8388f24d0 msgr2=0x55e838cc3f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:41:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.936+0000 7f8e2a3f2640 1 --2- 172.21.15.38:0/3222940010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8388f24d0 0x55e838cc3f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.936+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/3222940010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e838cb34c0 con 0x55e8388ea2f0 2024-05-21T15:41:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.936+0000 7f8e2a3f2640 1 --2- 172.21.15.38:0/3222940010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388ea2f0 0x55e838cae840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8e200bc8f0 tx=0x7f8e200be9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79fb996e023f692a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.937+0000 7f8e0ffff640 1 -- 172.21.15.38:0/3222940010 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e200c9100 con 0x55e8388ea2f0 2024-05-21T15:41:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.937+0000 7f8e0ffff640 1 -- 172.21.15.38:0/3222940010 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e200bcb00 con 0x55e8388ea2f0 2024-05-21T15:41:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.937+0000 7f8e0ffff640 1 -- 172.21.15.38:0/3222940010 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2003dc00 con 0x55e8388ea2f0 2024-05-21T15:41:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.938+0000 7f8e335d1740 1 -- 172.21.15.38:0/3222940010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388ea2f0 msgr2=0x55e838cae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.938+0000 7f8e335d1740 1 --2- 172.21.15.38:0/3222940010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388ea2f0 0x55e838cae840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8e200bc8f0 tx=0x7f8e200be9a0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.938+0000 7f8e335d1740 1 -- 172.21.15.38:0/3222940010 shutdown_connections 2024-05-21T15:41:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.938+0000 7f8e335d1740 1 --2- 172.21.15.38:0/3222940010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8389131b0 0x55e838cbc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.938+0000 7f8e335d1740 1 --2- 172.21.15.38:0/3222940010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8388f24d0 0x55e838cc3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.938+0000 7f8e335d1740 1 --2- 172.21.15.38:0/3222940010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8388ea2f0 0x55e838cae840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.938+0000 7f8e335d1740 1 -- 172.21.15.38:0/3222940010 >> 172.21.15.38:0/3222940010 conn(0x55e838907300 msgr2=0x55e838cae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.938+0000 7f8e335d1740 1 -- 172.21.15.38:0/3222940010 shutdown_connections 2024-05-21T15:41:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.938+0000 7f8e335d1740 1 -- 172.21.15.38:0/3222940010 wait complete. 2024-05-21T15:41:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.939+0000 7f8e335d1740 1 -- start start 2024-05-21T15:41:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.939+0000 7f8e335d1740 1 Processor -- start 2024-05-21T15:41:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.939+0000 7f8e335d1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8389131b0 0x55e838cc1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.939+0000 7f8e335d1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8388a71c0 0x55e838cc17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.939+0000 7f8e335d1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e838cc2700 0x55e838cc2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.939+0000 7f8e29bf1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8388a71c0 0x55e838cc17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.939+0000 7f8e2a3f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8389131b0 0x55e838cc1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.939+0000 7f8e2a3f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8389131b0 0x55e838cc1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:59396/0 (socket says 172.21.15.38:59396) 2024-05-21T15:41:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.940+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/2463363116 learned_addr learned my addr 172.21.15.38:0/2463363116 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.940+0000 7f8e335d1740 1 -- 172.21.15.38:0/2463363116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241a93e0 con 0x55e8388a71c0 2024-05-21T15:41:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.940+0000 7f8e335d1740 1 -- 172.21.15.38:0/2463363116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241a8d10 con 0x55e838cc2700 2024-05-21T15:41:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.940+0000 7f8e335d1740 1 -- 172.21.15.38:0/2463363116 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241a5f50 con 0x55e8389131b0 2024-05-21T15:41:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.940+0000 7f8e335d1740 10 client.0 _ll_register_callbacks cb 0x7f8e200bc730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.940+0000 7f8e2abf3640 1 --2- 172.21.15.38:0/2463363116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e838cc2700 0x55e838cc2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.940+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/2463363116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e838cc2700 msgr2=0x55e838cc2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.940+0000 7f8e2a3f2640 1 --2- 172.21.15.38:0/2463363116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e838cc2700 0x55e838cc2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.940+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/2463363116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8388a71c0 msgr2=0x55e838cc17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.940+0000 7f8e2a3f2640 1 --2- 172.21.15.38:0/2463363116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8388a71c0 0x55e838cc17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.940+0000 7f8e2a3f2640 1 -- 172.21.15.38:0/2463363116 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e838cc30d0 con 0x55e8389131b0 2024-05-21T15:41:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.940+0000 7f8e2a3f2640 1 --2- 172.21.15.38:0/2463363116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8389131b0 0x55e838cc1290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8e200c8bd0 tx=0x7f8e20005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e923ebc8c61ef2e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.940+0000 7f8e0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:41:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.941+0000 7f8e0dffb640 1 -- 172.21.15.38:0/2463363116 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e200cf060 con 0x55e8389131b0 2024-05-21T15:41:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.941+0000 7f8e0dffb640 1 -- 172.21.15.38:0/2463363116 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f8e200bfae0 con 0x55e8389131b0 2024-05-21T15:41:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.941+0000 7f8e335d1740 1 -- 172.21.15.38:0/2463363116 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e200bcb00 con 0x55e8389131b0 2024-05-21T15:41:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.941+0000 7f8e0dffb640 1 -- 172.21.15.38:0/2463363116 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8e200c4840 con 0x55e8389131b0 2024-05-21T15:41:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.941+0000 7f8e335d1740 10 client.27013 fetch_fsmap learned FSMap version 34 2024-05-21T15:41:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.941+0000 7f8e335d1740 1 -- 172.21.15.38:0/2463363116 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e838cc30d0 con 0x55e8389131b0 2024-05-21T15:41:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.941+0000 7f8e0dffb640 1 -- 172.21.15.38:0/2463363116 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e200c9100 con 0x55e8389131b0 2024-05-21T15:41:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.942+0000 7f8e0dffb640 1 -- 172.21.15.38:0/2463363116 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f8e200bc1e0 con 0x55e8389131b0 2024-05-21T15:41:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.942+0000 7f8e335d1740 10 client.27013 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T15:41:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.942+0000 7f8e335d1740 10 client.27013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.942+0000 7f8e335d1740 20 client.27013 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.942+0000 7f8e335d1740 1 --2- 172.21.15.38:0/2463363116 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e838d3b490 0x55e838d3d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.942+0000 7f8e335d1740 4 client.27013 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:41:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.942+0000 7f8e335d1740 1 -- 172.21.15.38:0/2463363116 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8e200bcb00 con 0x55e838d3b490 2024-05-21T15:41:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.942+0000 7f8e29bf1640 1 --2- 172.21.15.38:0/2463363116 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e838d3b490 0x55e838d3d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.943+0000 7f8e29bf1640 1 --2- 172.21.15.38:0/2463363116 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e838d3b490 0x55e838d3d8b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2adda36a4eb129e8 server_cookie=f63c9e1d32729bbe in_seq=0 out_seq=0 2024-05-21T15:41:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.943+0000 7f8e0dffb640 10 client.27013 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:22.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.965+0000 7f8e0dffb640 1 -- 172.21.15.38:0/2463363116 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8e200bcb00 con 0x55e838d3b490 2024-05-21T15:41:22.964 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.965+0000 7f8e0dffb640 10 client.27013 handle_command_reply: tid=0 2024-05-21T15:41:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.965+0000 7f8e335d1740 1 client.27013 shutdown 2024-05-21T15:41:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.966+0000 7f8e335d1740 1 -- 172.21.15.38:0/2463363116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8389131b0 msgr2=0x55e838cc1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.966+0000 7f8e335d1740 1 --2- 172.21.15.38:0/2463363116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8389131b0 0x55e838cc1290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8e200c8bd0 tx=0x7f8e20005fd0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.966+0000 7f8e335d1740 1 -- 172.21.15.38:0/2463363116 shutdown_connections 2024-05-21T15:41:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.966+0000 7f8e335d1740 1 --2- 172.21.15.38:0/2463363116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e838cc2700 0x55e838cc2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.966+0000 7f8e335d1740 1 --2- 172.21.15.38:0/2463363116 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e838d3b490 0x55e838d3d8b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.966+0000 7f8e335d1740 1 --2- 172.21.15.38:0/2463363116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e8388a71c0 0x55e838cc17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.966+0000 7f8e335d1740 1 --2- 172.21.15.38:0/2463363116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e8389131b0 0x55e838cc1290 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.966+0000 7f8e0dffb640 0 client.27013 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.966+0000 7f8e335d1740 1 -- 172.21.15.38:0/2463363116 >> 172.21.15.38:0/2463363116 conn(0x55e838907300 msgr2=0x55e838cb1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.966+0000 7f8e335d1740 1 -- 172.21.15.38:0/2463363116 shutdown_connections 2024-05-21T15:41:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.966+0000 7f8e335d1740 1 -- 172.21.15.38:0/2463363116 wait complete. 2024-05-21T15:41:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.966+0000 7f8e335d1740 20 client.27013 trim_cache size 0 max 16384 2024-05-21T15:41:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:41:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:41:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:41:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:41:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.967+0000 7f8e2c656640 1 -- 172.21.15.38:0/3711311588 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8e100a2d00 msgr2=0x7f8e100a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.967+0000 7f8e2c656640 1 --2- 172.21.15.38:0/3711311588 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8e100a2d00 0x7f8e100a5210 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c005c40 tx=0x7f8e1c005bd0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.967+0000 7f8e2c656640 1 -- 172.21.15.38:0/3711311588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e241019b0 msgr2=0x7f8e24199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.967+0000 7f8e2c656640 1 --2- 172.21.15.38:0/3711311588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e241019b0 0x7f8e24199060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009850 tx=0x7f8e20031400 comp rx=0 tx=0).stop 2024-05-21T15:41:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.967+0000 7f8e2c656640 1 -- 172.21.15.38:0/3711311588 shutdown_connections 2024-05-21T15:41:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.967+0000 7f8e2c656640 1 --2- 172.21.15.38:0/3711311588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e241078f0 0x7f8e241a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.967+0000 7f8e2c656640 1 --2- 172.21.15.38:0/3711311588 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8e100a2d00 0x7f8e100a5210 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.967+0000 7f8e2c656640 1 --2- 172.21.15.38:0/3711311588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e24102bf0 0x7f8e241995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.967+0000 7f8e2c656640 1 --2- 172.21.15.38:0/3711311588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e241019b0 0x7f8e24199060 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.967+0000 7f8e2c656640 1 -- 172.21.15.38:0/3711311588 >> 172.21.15.38:0/3711311588 conn(0x7f8e240fcba0 msgr2=0x7f8e240fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.967+0000 7f8e2c656640 1 -- 172.21.15.38:0/3711311588 shutdown_connections 2024-05-21T15:41:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:22.967+0000 7f8e2c656640 1 -- 172.21.15.38:0/3711311588 wait complete. 2024-05-21T15:41:23.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.037+0000 7f35d500d640 1 -- 172.21.15.38:0/3647462151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35d0105ed0 msgr2=0x7f35d01082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.037+0000 7f35d500d640 1 --2- 172.21.15.38:0/3647462151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35d0105ed0 0x7f35d01082f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009900 tx=0x7f35c402a410 comp rx=0 tx=0).stop 2024-05-21T15:41:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.037+0000 7f35d500d640 1 -- 172.21.15.38:0/3647462151 shutdown_connections 2024-05-21T15:41:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.037+0000 7f35d500d640 1 --2- 172.21.15.38:0/3647462151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35d0108830 0x7f35d010ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.037+0000 7f35d500d640 1 --2- 172.21.15.38:0/3647462151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35d0105ed0 0x7f35d01082f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.037+0000 7f35d500d640 1 --2- 172.21.15.38:0/3647462151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d0103570 0x7f35d0105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.037+0000 7f35d500d640 1 -- 172.21.15.38:0/3647462151 >> 172.21.15.38:0/3647462151 conn(0x7f35d00fcb70 msgr2=0x7f35d00ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.037+0000 7f35d500d640 1 -- 172.21.15.38:0/3647462151 shutdown_connections 2024-05-21T15:41:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.037+0000 7f35d500d640 1 -- 172.21.15.38:0/3647462151 wait complete. 2024-05-21T15:41:23.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.038+0000 7f35d500d640 1 Processor -- start 2024-05-21T15:41:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.039+0000 7f35d500d640 1 -- start start 2024-05-21T15:41:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.039+0000 7f35d500d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35d0103570 0x7f35d0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.039+0000 7f35d500d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35d0105ed0 0x7f35d01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.039+0000 7f35d500d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d0108830 0x7f35d01a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.039+0000 7f35d500d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f35d010e750 con 0x7f35d0103570 2024-05-21T15:41:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.039+0000 7f35d500d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35d010e5d0 con 0x7f35d0108830 2024-05-21T15:41:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.039+0000 7f35ced9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35d0103570 0x7f35d0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.039+0000 7f35d500d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f35d010e8d0 con 0x7f35d0105ed0 2024-05-21T15:41:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.039+0000 7f35ced9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35d0103570 0x7f35d0199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40888/0 (socket says 172.21.15.38:40888) 2024-05-21T15:41:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.039+0000 7f35ced9d640 1 -- 172.21.15.38:0/525039818 learned_addr learned my addr 172.21.15.38:0/525039818 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.039+0000 7f35ce59c640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35d0105ed0 0x7f35d01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.039+0000 7f35cf59e640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d0108830 0x7f35d01a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.040+0000 7f35ce59c640 1 -- 172.21.15.38:0/525039818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d0108830 msgr2=0x7f35d01a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.040+0000 7f35ce59c640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d0108830 0x7f35d01a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.040+0000 7f35ce59c640 1 -- 172.21.15.38:0/525039818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35d0103570 msgr2=0x7f35d0199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.040+0000 7f35ce59c640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35d0103570 0x7f35d0199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.040+0000 7f35ce59c640 1 -- 172.21.15.38:0/525039818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d01a0dd0 con 0x7f35d0105ed0 2024-05-21T15:41:23.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.040+0000 7f35cf59e640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d0108830 0x7f35d01a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:23.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.040+0000 7f35ce59c640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35d0105ed0 0x7f35d01997d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f35c402aa00 tx=0x7f35c4032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.041+0000 7f35b7fff640 1 -- 172.21.15.38:0/525039818 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c403f030 con 0x7f35d0105ed0 2024-05-21T15:41:23.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.041+0000 7f35b7fff640 1 -- 172.21.15.38:0/525039818 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c4003ce0 con 0x7f35d0105ed0 2024-05-21T15:41:23.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.041+0000 7f35d500d640 1 -- 172.21.15.38:0/525039818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d01a1060 con 0x7f35d0105ed0 2024-05-21T15:41:23.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.041+0000 7f35b7fff640 1 -- 172.21.15.38:0/525039818 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4035070 con 0x7f35d0105ed0 2024-05-21T15:41:23.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.041+0000 7f35d500d640 1 -- 172.21.15.38:0/525039818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d01a1570 con 0x7f35d0105ed0 2024-05-21T15:41:23.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.043+0000 7f35b7fff640 1 -- 172.21.15.38:0/525039818 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f35c4042030 con 0x7f35d0105ed0 2024-05-21T15:41:23.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.044+0000 7f35dbf8f740 1 Processor -- start 2024-05-21T15:41:23.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.044+0000 7f35dbf8f740 1 -- start start 2024-05-21T15:41:23.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.044+0000 7f35dbf8f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x56298193eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.044+0000 7f35dbf8f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 0x56298193f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.044+0000 7f35dbf8f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298157e6b0 0x562981945f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.044+0000 7f35dbf8f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5629818a3c40 con 0x5629815764d0 2024-05-21T15:41:23.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.044+0000 7f35dbf8f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56298184b580 con 0x56298157e6b0 2024-05-21T15:41:23.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.044+0000 7f35dbf8f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56298189fc00 con 0x56298159f390 2024-05-21T15:41:23.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.044+0000 7f35b7fff640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f35b80a2c10 0x7f35b80a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.045+0000 7f35cf59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x56298193eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.045+0000 7f35ce59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298157e6b0 0x562981945f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.045+0000 7f35cf59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x56298193eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:59440/0 (socket says 172.21.15.38:59440) 2024-05-21T15:41:23.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.045+0000 7f35cf59e640 1 -- 172.21.15.38:0/3656407356 learned_addr learned my addr 172.21.15.38:0/3656407356 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.045+0000 7f35cf59e640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f35b80a2c10 0x7f35b80a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.045+0000 7f35b7fff640 1 -- 172.21.15.38:0/525039818 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f35c40ef330 con 0x7f35d0105ed0 2024-05-21T15:41:23.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.045+0000 7f35ced9d640 1 --2- 172.21.15.38:0/3656407356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 0x56298193f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.045+0000 7f35cf59e640 1 -- 172.21.15.38:0/3656407356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298157e6b0 msgr2=0x562981945f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.045+0000 7f35cf59e640 1 --2- 172.21.15.38:0/3656407356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298157e6b0 0x562981945f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.045+0000 7f35cf59e640 1 -- 172.21.15.38:0/3656407356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 msgr2=0x56298193f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.045+0000 7f35cf59e640 1 --2- 172.21.15.38:0/3656407356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 0x56298193f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.045+0000 7f35cf59e640 1 -- 172.21.15.38:0/3656407356 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629815b73f0 con 0x56298159f390 2024-05-21T15:41:23.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.045+0000 7f35ced9d640 1 --2- 172.21.15.38:0/3656407356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 0x56298193f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:41:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.046+0000 7f35cf59e640 1 --2- 172.21.15.38:0/3656407356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x56298193eb00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f35c800f570 tx=0x7f35c800fa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=892436ea2c93f5db server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.046+0000 7f35cf59e640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f35b80a2c10 0x7f35b80a5120 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f35c8008000 tx=0x7f35c8020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.046+0000 7f35b57fa640 1 -- 172.21.15.38:0/3656407356 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8011630 con 0x56298159f390 2024-05-21T15:41:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.046+0000 7f35b57fa640 1 -- 172.21.15.38:0/3656407356 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c8022070 con 0x56298159f390 2024-05-21T15:41:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.046+0000 7f35b57fa640 1 -- 172.21.15.38:0/3656407356 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8026420 con 0x56298159f390 2024-05-21T15:41:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.047+0000 7f35dbf8f740 1 -- 172.21.15.38:0/3656407356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 msgr2=0x56298193eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.047+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/3656407356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x56298193eb00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f35c800f570 tx=0x7f35c800fa30 comp rx=0 tx=0).stop 2024-05-21T15:41:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.047+0000 7f35dbf8f740 1 -- 172.21.15.38:0/3656407356 shutdown_connections 2024-05-21T15:41:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.047+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/3656407356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298157e6b0 0x562981945f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.047+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/3656407356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 0x56298193f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.047+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/3656407356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x56298193eb00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.047+0000 7f35dbf8f740 1 -- 172.21.15.38:0/3656407356 >> 172.21.15.38:0/3656407356 conn(0x5629815934e0 msgr2=0x56298193a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.047+0000 7f35dbf8f740 1 -- 172.21.15.38:0/3656407356 shutdown_connections 2024-05-21T15:41:23.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.047+0000 7f35dbf8f740 1 -- 172.21.15.38:0/3656407356 wait complete. 2024-05-21T15:41:23.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.047+0000 7f35dbf8f740 1 -- start start 2024-05-21T15:41:23.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.047+0000 7f35dbf8f740 1 Processor -- start 2024-05-21T15:41:23.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35dbf8f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629815764d0 0x5629819489f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35dbf8f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56298157e6b0 0x562981948f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35dbf8f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x56298194fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35dbf8f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562981946970 con 0x56298157e6b0 2024-05-21T15:41:23.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35dbf8f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56298185ec40 con 0x5629815764d0 2024-05-21T15:41:23.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35dbf8f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562981946af0 con 0x56298159f390 2024-05-21T15:41:23.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35dbf8f740 10 client.0 _ll_register_callbacks cb 0x5629815b6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:23.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35ce59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56298157e6b0 0x562981948f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35ce59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56298157e6b0 0x562981948f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40902/0 (socket says 172.21.15.38:40902) 2024-05-21T15:41:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35ce59c640 1 -- 172.21.15.38:0/3759419958 learned_addr learned my addr 172.21.15.38:0/3759419958 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35cf59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x56298194fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35ced9d640 1 --2- 172.21.15.38:0/3759419958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629815764d0 0x5629819489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35cf59e640 1 -- 172.21.15.38:0/3759419958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629815764d0 msgr2=0x5629819489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35cf59e640 1 --2- 172.21.15.38:0/3759419958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629815764d0 0x5629819489f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35cf59e640 1 -- 172.21.15.38:0/3759419958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56298157e6b0 msgr2=0x562981948f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35cf59e640 1 --2- 172.21.15.38:0/3759419958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56298157e6b0 0x562981948f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35cf59e640 1 -- 172.21.15.38:0/3759419958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629816817d0 con 0x56298159f390 2024-05-21T15:41:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.048+0000 7f35ced9d640 1 --2- 172.21.15.38:0/3759419958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629815764d0 0x5629819489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.049+0000 7f35ce59c640 1 --2- 172.21.15.38:0/3759419958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56298157e6b0 0x562981948f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.049+0000 7f35cf59e640 1 --2- 172.21.15.38:0/3759419958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x56298194fe90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f35c800ff30 tx=0x7f35c8009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dca96ca3fb94c63d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.049+0000 7f35b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:41:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.049+0000 7f35b2ff5640 1 -- 172.21.15.38:0/3759419958 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c802a070 con 0x56298159f390 2024-05-21T15:41:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.049+0000 7f35dbf8f740 1 -- 172.21.15.38:0/3759419958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35c8022070 con 0x56298159f390 2024-05-21T15:41:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.049+0000 7f35b2ff5640 1 -- 172.21.15.38:0/3759419958 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f35c80045f0 con 0x56298159f390 2024-05-21T15:41:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.049+0000 7f35b2ff5640 1 -- 172.21.15.38:0/3759419958 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f35c8011610 con 0x56298159f390 2024-05-21T15:41:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.049+0000 7f35dbf8f740 10 client.27025 fetch_fsmap learned FSMap version 34 2024-05-21T15:41:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.049+0000 7f35dbf8f740 1 -- 172.21.15.38:0/3759419958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562981953180 con 0x56298159f390 2024-05-21T15:41:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.050+0000 7f35b2ff5640 1 -- 172.21.15.38:0/3759419958 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c8022070 con 0x56298159f390 2024-05-21T15:41:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.050+0000 7f35b2ff5640 1 -- 172.21.15.38:0/3759419958 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f35c8005350 con 0x56298159f390 2024-05-21T15:41:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.050+0000 7f35dbf8f740 10 client.27025 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T15:41:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.050+0000 7f35dbf8f740 10 client.27025 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:41:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.050+0000 7f35dbf8f740 20 client.27025 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.050+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/3759419958 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56298155df60 0x562981956660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.050+0000 7f35dbf8f740 4 client.27025 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:23.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.050+0000 7f35dbf8f740 1 -- 172.21.15.38:0/3759419958 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629818b1370 con 0x56298155df60 2024-05-21T15:41:23.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.050+0000 7f35ced9d640 1 --2- 172.21.15.38:0/3759419958 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56298155df60 0x562981956660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.051+0000 7f35b2ff5640 10 client.27025 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:41:23.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.051+0000 7f35ced9d640 1 --2- 172.21.15.38:0/3759419958 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56298155df60 0x562981956660 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7ebe55fe890822f7 server_cookie=9b8e2e73b8e134a3 in_seq=0 out_seq=0 2024-05-21T15:41:23.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35b2ff5640 1 -- 172.21.15.38:0/3759419958 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5629818b1370 con 0x56298155df60 2024-05-21T15:41:23.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35b2ff5640 10 client.27025 handle_command_reply: tid=0 2024-05-21T15:41:23.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35dbf8f740 1 client.27025 shutdown 2024-05-21T15:41:23.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35dbf8f740 1 -- 172.21.15.38:0/3759419958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 msgr2=0x56298194fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/3759419958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x56298194fe90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f35c800ff30 tx=0x7f35c8009630 comp rx=0 tx=0).stop 2024-05-21T15:41:23.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35dbf8f740 1 -- 172.21.15.38:0/3759419958 shutdown_connections 2024-05-21T15:41:23.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/3759419958 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56298155df60 0x562981956660 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/3759419958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x56298194fe90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/3759419958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56298157e6b0 0x562981948f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35b2ff5640 0 client.27025 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:41:23.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/3759419958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629815764d0 0x5629819489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35dbf8f740 1 -- 172.21.15.38:0/3759419958 >> 172.21.15.38:0/3759419958 conn(0x5629815934e0 msgr2=0x562981939570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35dbf8f740 1 -- 172.21.15.38:0/3759419958 shutdown_connections 2024-05-21T15:41:23.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35dbf8f740 1 -- 172.21.15.38:0/3759419958 wait complete. 2024-05-21T15:41:23.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.052+0000 7f35dbf8f740 20 client.27025 trim_cache size 0 max 16384 2024-05-21T15:41:23.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.074+0000 7f35dbf8f740 1 Processor -- start 2024-05-21T15:41:23.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.074+0000 7f35dbf8f740 1 -- start start 2024-05-21T15:41:23.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.074+0000 7f35dbf8f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 0x56298193a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.074+0000 7f35dbf8f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298157e6b0 0x56298194fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.074+0000 7f35dbf8f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298159f390 0x562981938b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.074+0000 7f35dbf8f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f35d01aa430 con 0x5629815764d0 2024-05-21T15:41:23.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.074+0000 7f35dbf8f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35d01abfa0 con 0x56298159f390 2024-05-21T15:41:23.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.074+0000 7f35ce59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298157e6b0 0x56298194fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.074+0000 7f35ce59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298157e6b0 0x56298194fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:59458/0 (socket says 172.21.15.38:59458) 2024-05-21T15:41:23.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.074+0000 7f35ce59c640 1 -- 172.21.15.38:0/2929623654 learned_addr learned my addr 172.21.15.38:0/2929623654 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.074+0000 7f35dbf8f740 1 -- 172.21.15.38:0/2929623654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f35d01aab00 con 0x56298157e6b0 2024-05-21T15:41:23.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.074+0000 7f35cf59e640 1 --2- 172.21.15.38:0/2929623654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298159f390 0x562981938b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.075+0000 7f35ce59c640 1 -- 172.21.15.38:0/2929623654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298159f390 msgr2=0x562981938b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.075+0000 7f35ce59c640 1 --2- 172.21.15.38:0/2929623654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298159f390 0x562981938b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.075+0000 7f35ce59c640 1 -- 172.21.15.38:0/2929623654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 msgr2=0x56298193a5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:41:23.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.075+0000 7f35ce59c640 1 --2- 172.21.15.38:0/2929623654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 0x56298193a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.075+0000 7f35ce59c640 1 -- 172.21.15.38:0/2929623654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562981935d20 con 0x56298157e6b0 2024-05-21T15:41:23.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.075+0000 7f35ce59c640 1 --2- 172.21.15.38:0/2929623654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298157e6b0 0x56298194fdb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f35c402a930 tx=0x7f35c4049770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b0c2ac00b670615 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.075+0000 7f35b4ff9640 1 -- 172.21.15.38:0/2929623654 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4049040 con 0x56298157e6b0 2024-05-21T15:41:23.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.075+0000 7f35b4ff9640 1 -- 172.21.15.38:0/2929623654 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c40d3070 con 0x56298157e6b0 2024-05-21T15:41:23.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.075+0000 7f35b4ff9640 1 -- 172.21.15.38:0/2929623654 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c40c4790 con 0x56298157e6b0 2024-05-21T15:41:23.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.077+0000 7f35dbf8f740 1 -- 172.21.15.38:0/2929623654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298157e6b0 msgr2=0x56298194fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.077+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/2929623654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298157e6b0 0x56298194fdb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f35c402a930 tx=0x7f35c4049770 comp rx=0 tx=0).stop 2024-05-21T15:41:23.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.077+0000 7f35dbf8f740 1 -- 172.21.15.38:0/2929623654 shutdown_connections 2024-05-21T15:41:23.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.077+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/2929623654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298159f390 0x562981938b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.077+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/2929623654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298157e6b0 0x56298194fdb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.077+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/2929623654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 0x56298193a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.077+0000 7f35dbf8f740 1 -- 172.21.15.38:0/2929623654 >> 172.21.15.38:0/2929623654 conn(0x5629815934e0 msgr2=0x562981939ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.077+0000 7f35dbf8f740 1 -- 172.21.15.38:0/2929623654 shutdown_connections 2024-05-21T15:41:23.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.077+0000 7f35dbf8f740 1 -- 172.21.15.38:0/2929623654 wait complete. 2024-05-21T15:41:23.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.077+0000 7f35dbf8f740 1 -- start start 2024-05-21T15:41:23.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.078+0000 7f35dbf8f740 1 Processor -- start 2024-05-21T15:41:23.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.078+0000 7f35dbf8f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 0x56298194ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.078+0000 7f35dbf8f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298157e6b0 0x56298194d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.078+0000 7f35ced9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 0x56298194ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.078+0000 7f35ced9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 0x56298194ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40922/0 (socket says 172.21.15.38:40922) 2024-05-21T15:41:23.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.078+0000 7f35ced9d640 1 -- 172.21.15.38:0/885592272 learned_addr learned my addr 172.21.15.38:0/885592272 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.078+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x5629819c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.078+0000 7f35dbf8f740 1 -- 172.21.15.38:0/885592272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f35d01ab8a0 con 0x5629815764d0 2024-05-21T15:41:23.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.078+0000 7f35dbf8f740 1 -- 172.21.15.38:0/885592272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35d01ab1d0 con 0x56298157e6b0 2024-05-21T15:41:23.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.078+0000 7f35dbf8f740 1 -- 172.21.15.38:0/885592272 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f35d01a8410 con 0x56298159f390 2024-05-21T15:41:23.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.079+0000 7f35ce59c640 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298157e6b0 0x56298194d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.079+0000 7f35dbf8f740 10 client.0 _ll_register_callbacks cb 0x7f35c4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:23.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.079+0000 7f35cf59e640 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x5629819c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.079+0000 7f35cf59e640 1 -- 172.21.15.38:0/885592272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298157e6b0 msgr2=0x56298194d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.079+0000 7f35cf59e640 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298157e6b0 0x56298194d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.079+0000 7f35cf59e640 1 -- 172.21.15.38:0/885592272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 msgr2=0x56298194ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.079+0000 7f35cf59e640 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 0x56298194ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.079+0000 7f35cf59e640 1 -- 172.21.15.38:0/885592272 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629819502f0 con 0x56298159f390 2024-05-21T15:41:23.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.079+0000 7f35ced9d640 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 0x56298194ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:23.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.079+0000 7f35cf59e640 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x5629819c51d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f35c8011fd0 tx=0x7f35c8000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=810d0f76fcb2ad51 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.079+0000 7f35b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:41:23.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.080+0000 7f35b2ff5640 1 -- 172.21.15.38:0/885592272 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8047060 con 0x56298159f390 2024-05-21T15:41:23.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.080+0000 7f35b2ff5640 1 -- 172.21.15.38:0/885592272 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f35c802f900 con 0x56298159f390 2024-05-21T15:41:23.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.080+0000 7f35dbf8f740 1 -- 172.21.15.38:0/885592272 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35c40d3070 con 0x56298159f390 2024-05-21T15:41:23.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.080+0000 7f35b2ff5640 1 -- 172.21.15.38:0/885592272 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f35c802a040 con 0x56298159f390 2024-05-21T15:41:23.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.080+0000 7f35dbf8f740 10 client.27031 fetch_fsmap learned FSMap version 34 2024-05-21T15:41:23.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.080+0000 7f35dbf8f740 1 -- 172.21.15.38:0/885592272 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5629819468b0 con 0x56298159f390 2024-05-21T15:41:23.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.080+0000 7f35b2ff5640 1 -- 172.21.15.38:0/885592272 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c40d3070 con 0x56298159f390 2024-05-21T15:41:23.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.080+0000 7f35b2ff5640 1 -- 172.21.15.38:0/885592272 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f35c803a920 con 0x56298159f390 2024-05-21T15:41:23.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.080+0000 7f35dbf8f740 10 client.27031 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T15:41:23.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.080+0000 7f35dbf8f740 10 client.27031 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:41:23.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.080+0000 7f35dbf8f740 20 client.27031 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:23.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.080+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x5629815333a0 0x5629819c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.080+0000 7f35dbf8f740 4 client.27031 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:41:23.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.080+0000 7f35dbf8f740 1 -- 172.21.15.38:0/885592272 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629818ab6c0 con 0x5629815333a0 2024-05-21T15:41:23.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.081+0000 7f35ced9d640 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x5629815333a0 0x5629819c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.081+0000 7f35b2ff5640 10 client.27031 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:41:23.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.081+0000 7f35ced9d640 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x5629815333a0 0x5629819c8f30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=38418f2fcf39b0e1 server_cookie=1fb4077a49de3b15 in_seq=0 out_seq=0 2024-05-21T15:41:23.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35b2ff5640 1 -- 172.21.15.38:0/885592272 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629818ab6c0 con 0x5629815333a0 2024-05-21T15:41:23.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35b2ff5640 10 client.27031 handle_command_reply: tid=0 2024-05-21T15:41:23.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35dbf8f740 1 client.27031 shutdown 2024-05-21T15:41:23.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35dbf8f740 1 -- 172.21.15.38:0/885592272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 msgr2=0x5629819c51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x5629819c51d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f35c8011fd0 tx=0x7f35c8000f40 comp rx=0 tx=0).stop 2024-05-21T15:41:23.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35dbf8f740 1 -- 172.21.15.38:0/885592272 shutdown_connections 2024-05-21T15:41:23.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x5629815333a0 0x5629819c8f30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56298159f390 0x5629819c51d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56298157e6b0 0x56298194d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35b2ff5640 0 client.27031 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:41:23.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35dbf8f740 1 --2- 172.21.15.38:0/885592272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5629815764d0 0x56298194ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35dbf8f740 1 -- 172.21.15.38:0/885592272 >> 172.21.15.38:0/885592272 conn(0x5629815934e0 msgr2=0x562981938b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35dbf8f740 1 -- 172.21.15.38:0/885592272 shutdown_connections 2024-05-21T15:41:23.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35dbf8f740 1 -- 172.21.15.38:0/885592272 wait complete. 2024-05-21T15:41:23.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.082+0000 7f35dbf8f740 20 client.27031 trim_cache size 0 max 16384 2024-05-21T15:41:23.084 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:41:23.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.083+0000 7f35d500d640 1 -- 172.21.15.38:0/525039818 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f35b80a2c10 msgr2=0x7f35b80a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.083+0000 7f35d500d640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f35b80a2c10 0x7f35b80a5120 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f35c8008000 tx=0x7f35c8020000 comp rx=0 tx=0).stop 2024-05-21T15:41:23.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.083+0000 7f35d500d640 1 -- 172.21.15.38:0/525039818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35d0105ed0 msgr2=0x7f35d01997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.083+0000 7f35d500d640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35d0105ed0 0x7f35d01997d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f35c402aa00 tx=0x7f35c4032ce0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.083+0000 7f35d500d640 1 -- 172.21.15.38:0/525039818 shutdown_connections 2024-05-21T15:41:23.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.083+0000 7f35d500d640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d0108830 0x7f35d01a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.083+0000 7f35d500d640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35d0105ed0 0x7f35d01997d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.083+0000 7f35d500d640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f35b80a2c10 0x7f35b80a5120 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.083+0000 7f35d500d640 1 --2- 172.21.15.38:0/525039818 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35d0103570 0x7f35d0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.083+0000 7f35d500d640 1 -- 172.21.15.38:0/525039818 >> 172.21.15.38:0/525039818 conn(0x7f35d00fcb70 msgr2=0x7f35d0104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.084+0000 7f35d500d640 1 -- 172.21.15.38:0/525039818 shutdown_connections 2024-05-21T15:41:23.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.084+0000 7f35d500d640 1 -- 172.21.15.38:0/525039818 wait complete. 2024-05-21T15:41:23.096 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:41:23.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.285+0000 7f9582d4c640 1 -- 172.21.15.38:0/2923072719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f957c101980 msgr2=0x7f957c102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.285+0000 7f9582d4c640 1 --2- 172.21.15.38:0/2923072719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f957c101980 0x7f957c102680 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9570009860 tx=0x7f957002a340 comp rx=0 tx=0).stop 2024-05-21T15:41:23.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.285+0000 7f9582d4c640 1 -- 172.21.15.38:0/2923072719 shutdown_connections 2024-05-21T15:41:23.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.285+0000 7f9582d4c640 1 --2- 172.21.15.38:0/2923072719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c1078c0 0x7f957c109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.285+0000 7f9582d4c640 1 --2- 172.21.15.38:0/2923072719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f957c102bc0 0x7f957c102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.285+0000 7f9582d4c640 1 --2- 172.21.15.38:0/2923072719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f957c101980 0x7f957c102680 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.285+0000 7f9582d4c640 1 -- 172.21.15.38:0/2923072719 >> 172.21.15.38:0/2923072719 conn(0x7f957c0fcb70 msgr2=0x7f957c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.285+0000 7f9582d4c640 1 -- 172.21.15.38:0/2923072719 shutdown_connections 2024-05-21T15:41:23.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.285+0000 7f9582d4c640 1 -- 172.21.15.38:0/2923072719 wait complete. 2024-05-21T15:41:23.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.286+0000 7f9582d4c640 1 Processor -- start 2024-05-21T15:41:23.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.286+0000 7f9582d4c640 1 -- start start 2024-05-21T15:41:23.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.286+0000 7f9582d4c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c101980 0x7f957c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.286+0000 7f9582d4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f957c102bc0 0x7f957c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.286+0000 7f9582d4c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f957c1078c0 0x7f957c1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.286+0000 7f9582d4c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f957c10e6c0 con 0x7f957c1078c0 2024-05-21T15:41:23.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.286+0000 7f9582d4c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f957c10e540 con 0x7f957c101980 2024-05-21T15:41:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.286+0000 7f9582d4c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f957c10e840 con 0x7f957c102bc0 2024-05-21T15:41:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.286+0000 7f95812e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f957c1078c0 0x7f957c1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.286+0000 7f957bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f957c102bc0 0x7f957c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.286+0000 7f95812e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f957c1078c0 0x7f957c1a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40940/0 (socket says 172.21.15.38:40940) 2024-05-21T15:41:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.286+0000 7f95812e9640 1 -- 172.21.15.38:0/800347524 learned_addr learned my addr 172.21.15.38:0/800347524 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.287+0000 7f9580ae8640 1 --2- 172.21.15.38:0/800347524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c101980 0x7f957c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.287+0000 7f957bfff640 1 -- 172.21.15.38:0/800347524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c101980 msgr2=0x7f957c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.287+0000 7f957bfff640 1 --2- 172.21.15.38:0/800347524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c101980 0x7f957c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.287+0000 7f957bfff640 1 -- 172.21.15.38:0/800347524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f957c1078c0 msgr2=0x7f957c1a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.287+0000 7f957bfff640 1 --2- 172.21.15.38:0/800347524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f957c1078c0 0x7f957c1a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.287+0000 7f957bfff640 1 -- 172.21.15.38:0/800347524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c1a0db0 con 0x7f957c102bc0 2024-05-21T15:41:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.287+0000 7f95812e9640 1 --2- 172.21.15.38:0/800347524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f957c1078c0 0x7f957c1a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.287+0000 7f957bfff640 1 --2- 172.21.15.38:0/800347524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f957c102bc0 0x7f957c1997b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9568005fd0 tx=0x7f956800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.287+0000 7f9579ffb640 1 -- 172.21.15.38:0/800347524 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956800cd00 con 0x7f957c102bc0 2024-05-21T15:41:23.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.287+0000 7f9582d4c640 1 -- 172.21.15.38:0/800347524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957c1a10a0 con 0x7f957c102bc0 2024-05-21T15:41:23.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.287+0000 7f9579ffb640 1 -- 172.21.15.38:0/800347524 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95680035f0 con 0x7f957c102bc0 2024-05-21T15:41:23.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.287+0000 7f9579ffb640 1 -- 172.21.15.38:0/800347524 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568010670 con 0x7f957c102bc0 2024-05-21T15:41:23.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.288+0000 7f9582d4c640 1 -- 172.21.15.38:0/800347524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957c1a15f0 con 0x7f957c102bc0 2024-05-21T15:41:23.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.291+0000 7f9579ffb640 1 -- 172.21.15.38:0/800347524 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9568010810 con 0x7f957c102bc0 2024-05-21T15:41:23.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.292+0000 7f9589bfb740 1 Processor -- start 2024-05-21T15:41:23.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.292+0000 7f9589bfb740 1 -- start start 2024-05-21T15:41:23.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.292+0000 7f9589bfb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564641872390 0x564641c1ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.292+0000 7f9589bfb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c1ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.292+0000 7f9589bfb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 0x564641c216b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.292+0000 7f9589bfb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564641b76c40 con 0x564641872390 2024-05-21T15:41:23.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.292+0000 7f9589bfb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564641b1e580 con 0x5646418494d0 2024-05-21T15:41:23.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.292+0000 7f9589bfb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564641b72c00 con 0x5646418516b0 2024-05-21T15:41:23.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.292+0000 7f95812e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564641872390 0x564641c1ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.292+0000 7f95812e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564641872390 0x564641c1ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40954/0 (socket says 172.21.15.38:40954) 2024-05-21T15:41:23.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.292+0000 7f95812e9640 1 -- 172.21.15.38:0/2672648072 learned_addr learned my addr 172.21.15.38:0/2672648072 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.292+0000 7f957bfff640 1 --2- 172.21.15.38:0/2672648072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 0x564641c216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.293+0000 7f95812e9640 1 -- 172.21.15.38:0/2672648072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 msgr2=0x564641c216b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.293+0000 7f95812e9640 1 --2- 172.21.15.38:0/2672648072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 0x564641c216b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.293+0000 7f95812e9640 1 -- 172.21.15.38:0/2672648072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 msgr2=0x564641c1ef50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:41:23.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.293+0000 7f95812e9640 1 --2- 172.21.15.38:0/2672648072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c1ef50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.293+0000 7f95812e9640 1 -- 172.21.15.38:0/2672648072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56464188a3f0 con 0x564641872390 2024-05-21T15:41:23.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.293+0000 7f957bfff640 1 --2- 172.21.15.38:0/2672648072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 0x564641c216b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:23.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.293+0000 7f9579ffb640 1 --2- 172.21.15.38:0/800347524 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f956c0a2c10 0x7f956c0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.293+0000 7f95812e9640 1 --2- 172.21.15.38:0/2672648072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564641872390 0x564641c1ca00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f95740118b0 tx=0x7f9574011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8d23c7701c2e8d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.293+0000 7f9579ffb640 1 -- 172.21.15.38:0/800347524 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f95680d0370 con 0x7f957c102bc0 2024-05-21T15:41:23.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.294+0000 7f9566ffd640 1 -- 172.21.15.38:0/2672648072 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957401e070 con 0x564641872390 2024-05-21T15:41:23.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.294+0000 7f9566ffd640 1 -- 172.21.15.38:0/2672648072 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9574011310 con 0x564641872390 2024-05-21T15:41:23.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.294+0000 7f9566ffd640 1 -- 172.21.15.38:0/2672648072 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574010430 con 0x564641872390 2024-05-21T15:41:23.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.294+0000 7f9580ae8640 1 --2- 172.21.15.38:0/800347524 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f956c0a2c10 0x7f956c0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.295+0000 7f9580ae8640 1 --2- 172.21.15.38:0/800347524 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f956c0a2c10 0x7f956c0a5120 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9570035040 tx=0x7f9570000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.295+0000 7f9589bfb740 1 -- 172.21.15.38:0/2672648072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564641872390 msgr2=0x564641c1ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.295+0000 7f9589bfb740 1 --2- 172.21.15.38:0/2672648072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564641872390 0x564641c1ca00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f95740118b0 tx=0x7f9574011d70 comp rx=0 tx=0).stop 2024-05-21T15:41:23.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.295+0000 7f9589bfb740 1 -- 172.21.15.38:0/2672648072 shutdown_connections 2024-05-21T15:41:23.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.295+0000 7f9589bfb740 1 --2- 172.21.15.38:0/2672648072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 0x564641c216b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.295+0000 7f9589bfb740 1 --2- 172.21.15.38:0/2672648072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c1ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.295+0000 7f9589bfb740 1 --2- 172.21.15.38:0/2672648072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564641872390 0x564641c1ca00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.295+0000 7f9589bfb740 1 -- 172.21.15.38:0/2672648072 >> 172.21.15.38:0/2672648072 conn(0x5646418664e0 msgr2=0x564641c0d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.296+0000 7f9589bfb740 1 -- 172.21.15.38:0/2672648072 shutdown_connections 2024-05-21T15:41:23.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.296+0000 7f9589bfb740 1 -- 172.21.15.38:0/2672648072 wait complete. 2024-05-21T15:41:23.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.296+0000 7f9589bfb740 1 -- start start 2024-05-21T15:41:23.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.297+0000 7f9589bfb740 1 Processor -- start 2024-05-21T15:41:23.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.297+0000 7f9589bfb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c24160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.298+0000 7f9589bfb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 0x564641c246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.298+0000 7f9589bfb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564641872390 0x564641c2b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.298+0000 7f9589bfb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564641c221d0 con 0x564641872390 2024-05-21T15:41:23.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.298+0000 7f9580ae8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c24160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.298+0000 7f9580ae8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c24160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50384/0 (socket says 172.21.15.38:50384) 2024-05-21T15:41:23.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.298+0000 7f9580ae8640 1 -- 172.21.15.38:0/2675412057 learned_addr learned my addr 172.21.15.38:0/2675412057 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.298+0000 7f9589bfb740 1 -- 172.21.15.38:0/2675412057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564641b31c40 con 0x5646418494d0 2024-05-21T15:41:23.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.298+0000 7f9589bfb740 1 -- 172.21.15.38:0/2675412057 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564641c22350 con 0x5646418516b0 2024-05-21T15:41:23.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.298+0000 7f957bfff640 1 --2- 172.21.15.38:0/2675412057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 0x564641c246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.298+0000 7f95812e9640 1 --2- 172.21.15.38:0/2675412057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564641872390 0x564641c2b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.298+0000 7f9589bfb740 10 client.0 _ll_register_callbacks cb 0x564641889be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:23.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.298+0000 7f95812e9640 1 -- 172.21.15.38:0/2675412057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 msgr2=0x564641c246a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.298+0000 7f95812e9640 1 --2- 172.21.15.38:0/2675412057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 0x564641c246a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.299+0000 7f95812e9640 1 -- 172.21.15.38:0/2675412057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 msgr2=0x564641c24160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.299+0000 7f95812e9640 1 --2- 172.21.15.38:0/2675412057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c24160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.299+0000 7f95812e9640 1 -- 172.21.15.38:0/2675412057 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646419547d0 con 0x564641872390 2024-05-21T15:41:23.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.299+0000 7f9580ae8640 1 --2- 172.21.15.38:0/2675412057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c24160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:23.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.299+0000 7f95812e9640 1 --2- 172.21.15.38:0/2675412057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564641872390 0x564641c2b600 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f957400f7b0 tx=0x7f9574019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8879c1dac1673031 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.299+0000 7f955bfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:23.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.299+0000 7f955bfff640 1 -- 172.21.15.38:0/2675412057 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574029020 con 0x564641872390 2024-05-21T15:41:23.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.299+0000 7f955bfff640 1 -- 172.21.15.38:0/2675412057 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9574004a40 con 0x564641872390 2024-05-21T15:41:23.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.299+0000 7f9589bfb740 1 -- 172.21.15.38:0/2675412057 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9574011310 con 0x564641872390 2024-05-21T15:41:23.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.300+0000 7f955bfff640 1 -- 172.21.15.38:0/2675412057 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f957401e040 con 0x564641872390 2024-05-21T15:41:23.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.300+0000 7f9589bfb740 10 client.17352 fetch_fsmap learned FSMap version 34 2024-05-21T15:41:23.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.300+0000 7f9589bfb740 1 -- 172.21.15.38:0/2675412057 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564641c2e880 con 0x564641872390 2024-05-21T15:41:23.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.300+0000 7f955bfff640 1 -- 172.21.15.38:0/2675412057 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9574011310 con 0x564641872390 2024-05-21T15:41:23.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.301+0000 7f955bfff640 1 -- 172.21.15.38:0/2675412057 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f9574005420 con 0x564641872390 2024-05-21T15:41:23.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.301+0000 7f9589bfb740 10 client.17352 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T15:41:23.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.301+0000 7f9589bfb740 10 client.17352 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:41:23.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.301+0000 7f9589bfb740 20 client.17352 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:23.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.301+0000 7f9589bfb740 1 --2- 172.21.15.38:0/2675412057 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564641830f60 0x564641c31ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.301+0000 7f9589bfb740 4 client.17352 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:23.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.301+0000 7f9589bfb740 1 -- 172.21.15.38:0/2675412057 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564641b84370 con 0x564641830f60 2024-05-21T15:41:23.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.301+0000 7f9580ae8640 1 --2- 172.21.15.38:0/2675412057 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564641830f60 0x564641c31ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.302+0000 7f955bfff640 10 client.17352 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:41:23.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.302+0000 7f9580ae8640 1 --2- 172.21.15.38:0/2675412057 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564641830f60 0x564641c31ca0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e2d5062793bdb8ba server_cookie=638dbb33cdc525bf in_seq=0 out_seq=0 2024-05-21T15:41:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f955bfff640 1 -- 172.21.15.38:0/2675412057 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564641b84370 con 0x564641830f60 2024-05-21T15:41:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f955bfff640 10 client.17352 handle_command_reply: tid=0 2024-05-21T15:41:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f9589bfb740 1 client.17352 shutdown 2024-05-21T15:41:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f9589bfb740 1 -- 172.21.15.38:0/2675412057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564641872390 msgr2=0x564641c2b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f9589bfb740 1 --2- 172.21.15.38:0/2675412057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564641872390 0x564641c2b600 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f957400f7b0 tx=0x7f9574019660 comp rx=0 tx=0).stop 2024-05-21T15:41:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f9589bfb740 1 -- 172.21.15.38:0/2675412057 shutdown_connections 2024-05-21T15:41:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f9589bfb740 1 --2- 172.21.15.38:0/2675412057 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564641830f60 0x564641c31ca0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f9589bfb740 1 --2- 172.21.15.38:0/2675412057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564641872390 0x564641c2b600 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f9589bfb740 1 --2- 172.21.15.38:0/2675412057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 0x564641c246a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f9589bfb740 1 --2- 172.21.15.38:0/2675412057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c24160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f9589bfb740 1 -- 172.21.15.38:0/2675412057 >> 172.21.15.38:0/2675412057 conn(0x5646418664e0 msgr2=0x564641c0c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f955bfff640 0 client.17352 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:41:23.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f9589bfb740 1 -- 172.21.15.38:0/2675412057 shutdown_connections 2024-05-21T15:41:23.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f9589bfb740 1 -- 172.21.15.38:0/2675412057 wait complete. 2024-05-21T15:41:23.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.303+0000 7f9589bfb740 20 client.17352 trim_cache size 0 max 16384 2024-05-21T15:41:23.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.322+0000 7f9589bfb740 1 Processor -- start 2024-05-21T15:41:23.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.322+0000 7f9589bfb740 1 -- start start 2024-05-21T15:41:23.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.322+0000 7f9589bfb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418664e0 0x564641c1a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.322+0000 7f9589bfb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646418494d0 0x564641c22ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.322+0000 7f9589bfb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 0x564641c36450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.322+0000 7f9589bfb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1a8300 con 0x5646418494d0 2024-05-21T15:41:23.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.322+0000 7f9589bfb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1a9e70 con 0x5646418664e0 2024-05-21T15:41:23.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.322+0000 7f9589bfb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1a89d0 con 0x5646418516b0 2024-05-21T15:41:23.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.322+0000 7f957bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646418494d0 0x564641c22ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.322+0000 7f9580ae8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418664e0 0x564641c1a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.322+0000 7f957bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646418494d0 0x564641c22ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40978/0 (socket says 172.21.15.38:40978) 2024-05-21T15:41:23.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.322+0000 7f957bfff640 1 -- 172.21.15.38:0/3818029186 learned_addr learned my addr 172.21.15.38:0/3818029186 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.322+0000 7f95812e9640 1 --2- 172.21.15.38:0/3818029186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 0x564641c36450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.323+0000 7f9580ae8640 1 -- 172.21.15.38:0/3818029186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 msgr2=0x564641c36450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.323+0000 7f9580ae8640 1 --2- 172.21.15.38:0/3818029186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 0x564641c36450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.323+0000 7f9580ae8640 1 -- 172.21.15.38:0/3818029186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646418494d0 msgr2=0x564641c22ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.323+0000 7f9580ae8640 1 --2- 172.21.15.38:0/3818029186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646418494d0 0x564641c22ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.323+0000 7f9580ae8640 1 -- 172.21.15.38:0/3818029186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564641c08d20 con 0x5646418664e0 2024-05-21T15:41:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.323+0000 7f9580ae8640 1 --2- 172.21.15.38:0/3818029186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418664e0 0x564641c1a9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9570030410 tx=0x7f957003c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6067e43087ce72a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.323+0000 7f95667fc640 1 -- 172.21.15.38:0/3818029186 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570037040 con 0x5646418664e0 2024-05-21T15:41:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.324+0000 7f95667fc640 1 -- 172.21.15.38:0/3818029186 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9570050070 con 0x5646418664e0 2024-05-21T15:41:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.324+0000 7f95667fc640 1 -- 172.21.15.38:0/3818029186 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570004b20 con 0x5646418664e0 2024-05-21T15:41:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.325+0000 7f9589bfb740 1 -- 172.21.15.38:0/3818029186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418664e0 msgr2=0x564641c1a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.325+0000 7f9589bfb740 1 --2- 172.21.15.38:0/3818029186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418664e0 0x564641c1a9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9570030410 tx=0x7f957003c000 comp rx=0 tx=0).stop 2024-05-21T15:41:23.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.325+0000 7f9589bfb740 1 -- 172.21.15.38:0/3818029186 shutdown_connections 2024-05-21T15:41:23.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.325+0000 7f9589bfb740 1 --2- 172.21.15.38:0/3818029186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418516b0 0x564641c36450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.325+0000 7f9589bfb740 1 --2- 172.21.15.38:0/3818029186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646418494d0 0x564641c22ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.325+0000 7f9589bfb740 1 --2- 172.21.15.38:0/3818029186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418664e0 0x564641c1a9f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.325+0000 7f9589bfb740 1 -- 172.21.15.38:0/3818029186 >> 172.21.15.38:0/3818029186 conn(0x564641c24930 msgr2=0x564641c227b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.325+0000 7f9589bfb740 1 -- 172.21.15.38:0/3818029186 shutdown_connections 2024-05-21T15:41:23.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.325+0000 7f9589bfb740 1 -- 172.21.15.38:0/3818029186 wait complete. 2024-05-21T15:41:23.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.325+0000 7f9589bfb740 1 -- start start 2024-05-21T15:41:23.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.325+0000 7f9589bfb740 1 Processor -- start 2024-05-21T15:41:23.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f9589bfb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c294d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f9589bfb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646418516b0 0x564641c29a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f9589bfb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418664e0 0x564641c29f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f9589bfb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1a9770 con 0x5646418516b0 2024-05-21T15:41:23.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f9589bfb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1a90a0 con 0x5646418494d0 2024-05-21T15:41:23.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f9589bfb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1a62e0 con 0x5646418664e0 2024-05-21T15:41:23.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f9589bfb740 10 client.0 _ll_register_callbacks cb 0x7f9574011780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:23.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f957bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646418516b0 0x564641c29a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f9580ae8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f957bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646418516b0 0x564641c29a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40984/0 (socket says 172.21.15.38:40984) 2024-05-21T15:41:23.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f9580ae8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c294d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50414/0 (socket says 172.21.15.38:50414) 2024-05-21T15:41:23.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f957bfff640 1 -- 172.21.15.38:0/1940231323 learned_addr learned my addr 172.21.15.38:0/1940231323 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f957bfff640 1 -- 172.21.15.38:0/1940231323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418664e0 msgr2=0x564641c29f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f957bfff640 1 --2- 172.21.15.38:0/1940231323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418664e0 0x564641c29f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f957bfff640 1 -- 172.21.15.38:0/1940231323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 msgr2=0x564641c294d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.326+0000 7f957bfff640 1 --2- 172.21.15.38:0/1940231323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c294d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.327+0000 7f957bfff640 1 -- 172.21.15.38:0/1940231323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564641c98000 con 0x5646418516b0 2024-05-21T15:41:23.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.327+0000 7f9580ae8640 1 --2- 172.21.15.38:0/1940231323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c294d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:23.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.327+0000 7f957bfff640 1 --2- 172.21.15.38:0/1940231323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646418516b0 0x564641c29a10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f956800c6a0 tx=0x7f9568014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b914168547579e20 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.327+0000 7f955bfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:23.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.327+0000 7f955bfff640 1 -- 172.21.15.38:0/1940231323 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568009510 con 0x5646418516b0 2024-05-21T15:41:23.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.327+0000 7f955bfff640 1 -- 172.21.15.38:0/1940231323 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f95680040c0 con 0x5646418516b0 2024-05-21T15:41:23.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.327+0000 7f9589bfb740 1 -- 172.21.15.38:0/1940231323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9570050070 con 0x5646418516b0 2024-05-21T15:41:23.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.327+0000 7f955bfff640 1 -- 172.21.15.38:0/1940231323 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f956809cb20 con 0x5646418516b0 2024-05-21T15:41:23.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.327+0000 7f9589bfb740 10 client.17358 fetch_fsmap learned FSMap version 34 2024-05-21T15:41:23.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.327+0000 7f9589bfb740 1 -- 172.21.15.38:0/1940231323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564641c992b0 con 0x5646418516b0 2024-05-21T15:41:23.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.327+0000 7f955bfff640 1 -- 172.21.15.38:0/1940231323 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9570050070 con 0x5646418516b0 2024-05-21T15:41:23.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.328+0000 7f955bfff640 1 -- 172.21.15.38:0/1940231323 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f9568098870 con 0x5646418516b0 2024-05-21T15:41:23.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.328+0000 7f9589bfb740 10 client.17358 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T15:41:23.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.328+0000 7f9589bfb740 10 client.17358 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:41:23.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.328+0000 7f9589bfb740 20 client.17358 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:23.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.328+0000 7f9589bfb740 1 --2- 172.21.15.38:0/1940231323 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564641872390 0x564641c9bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.328+0000 7f9589bfb740 4 client.17358 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:41:23.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.328+0000 7f9589bfb740 1 -- 172.21.15.38:0/1940231323 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564641b7e6c0 con 0x564641872390 2024-05-21T15:41:23.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.328+0000 7f9580ae8640 1 --2- 172.21.15.38:0/1940231323 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564641872390 0x564641c9bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.328+0000 7f955bfff640 10 client.17358 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:41:23.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.328+0000 7f9580ae8640 1 --2- 172.21.15.38:0/1940231323 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564641872390 0x564641c9bc40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8bcb9965185b6122 server_cookie=3bbf203e12c00e27 in_seq=0 out_seq=0 2024-05-21T15:41:23.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f955bfff640 1 -- 172.21.15.38:0/1940231323 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564641b7e6c0 con 0x564641872390 2024-05-21T15:41:23.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f955bfff640 10 client.17358 handle_command_reply: tid=0 2024-05-21T15:41:23.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f9589bfb740 1 client.17358 shutdown 2024-05-21T15:41:23.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f9589bfb740 1 -- 172.21.15.38:0/1940231323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646418516b0 msgr2=0x564641c29a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f9589bfb740 1 --2- 172.21.15.38:0/1940231323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646418516b0 0x564641c29a10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f956800c6a0 tx=0x7f9568014000 comp rx=0 tx=0).stop 2024-05-21T15:41:23.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f9589bfb740 1 -- 172.21.15.38:0/1940231323 shutdown_connections 2024-05-21T15:41:23.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f9589bfb740 1 --2- 172.21.15.38:0/1940231323 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x564641872390 0x564641c9bc40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f9589bfb740 1 --2- 172.21.15.38:0/1940231323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646418664e0 0x564641c29f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f955bfff640 0 client.17358 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:41:23.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f9589bfb740 1 --2- 172.21.15.38:0/1940231323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5646418516b0 0x564641c29a10 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f9589bfb740 1 --2- 172.21.15.38:0/1940231323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646418494d0 0x564641c294d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f9589bfb740 1 -- 172.21.15.38:0/1940231323 >> 172.21.15.38:0/1940231323 conn(0x564641c24930 msgr2=0x564641c199e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f9589bfb740 1 -- 172.21.15.38:0/1940231323 shutdown_connections 2024-05-21T15:41:23.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f9589bfb740 1 -- 172.21.15.38:0/1940231323 wait complete. 2024-05-21T15:41:23.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.329+0000 7f9589bfb740 20 client.17358 trim_cache size 0 max 16384 2024-05-21T15:41:23.332 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:41:23.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.330+0000 7f9582d4c640 1 -- 172.21.15.38:0/800347524 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f956c0a2c10 msgr2=0x7f956c0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.330+0000 7f9582d4c640 1 --2- 172.21.15.38:0/800347524 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f956c0a2c10 0x7f956c0a5120 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9570035040 tx=0x7f9570000bc0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.330+0000 7f9582d4c640 1 -- 172.21.15.38:0/800347524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f957c102bc0 msgr2=0x7f957c1997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.330+0000 7f9582d4c640 1 --2- 172.21.15.38:0/800347524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f957c102bc0 0x7f957c1997b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9568005fd0 tx=0x7f956800ee50 comp rx=0 tx=0).stop 2024-05-21T15:41:23.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.330+0000 7f9582d4c640 1 -- 172.21.15.38:0/800347524 shutdown_connections 2024-05-21T15:41:23.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.330+0000 7f9582d4c640 1 --2- 172.21.15.38:0/800347524 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f956c0a2c10 0x7f956c0a5120 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.330+0000 7f9582d4c640 1 --2- 172.21.15.38:0/800347524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f957c1078c0 0x7f957c1a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.330+0000 7f9582d4c640 1 --2- 172.21.15.38:0/800347524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f957c102bc0 0x7f957c1997b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.330+0000 7f9582d4c640 1 --2- 172.21.15.38:0/800347524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c101980 0x7f957c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.330+0000 7f9582d4c640 1 -- 172.21.15.38:0/800347524 >> 172.21.15.38:0/800347524 conn(0x7f957c0fcb70 msgr2=0x7f957c1038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.330+0000 7f9582d4c640 1 -- 172.21.15.38:0/800347524 shutdown_connections 2024-05-21T15:41:23.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.330+0000 7f9582d4c640 1 -- 172.21.15.38:0/800347524 wait complete. 2024-05-21T15:41:23.343 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:41:23.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.491+0000 7f1133275640 1 -- 172.21.15.38:0/3494092101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f112c102bc0 msgr2=0x7f112c102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.491+0000 7f1133275640 1 --2- 172.21.15.38:0/3494092101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f112c102bc0 0x7f112c102fd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f111c009900 tx=0x7f111c02a3f0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.491+0000 7f1133275640 1 -- 172.21.15.38:0/3494092101 shutdown_connections 2024-05-21T15:41:23.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.491+0000 7f1133275640 1 --2- 172.21.15.38:0/3494092101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f112c1078c0 0x7f112c109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.491+0000 7f1133275640 1 --2- 172.21.15.38:0/3494092101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f112c102bc0 0x7f112c102fd0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.491+0000 7f1133275640 1 --2- 172.21.15.38:0/3494092101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112c101980 0x7f112c102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.491+0000 7f1133275640 1 -- 172.21.15.38:0/3494092101 >> 172.21.15.38:0/3494092101 conn(0x7f112c0fcb70 msgr2=0x7f112c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.492+0000 7f1133275640 1 -- 172.21.15.38:0/3494092101 shutdown_connections 2024-05-21T15:41:23.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.492+0000 7f1133275640 1 -- 172.21.15.38:0/3494092101 wait complete. 2024-05-21T15:41:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.493+0000 7f1133275640 1 Processor -- start 2024-05-21T15:41:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.493+0000 7f1133275640 1 -- start start 2024-05-21T15:41:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.493+0000 7f1133275640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112c101980 0x7f112c19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.493+0000 7f1133275640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f112c102bc0 0x7f112c1a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.493+0000 7f1133275640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f112c1078c0 0x7f112c1a6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.493+0000 7f1131011640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112c101980 0x7f112c19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.494+0000 7f1131011640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112c101980 0x7f112c19fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50430/0 (socket says 172.21.15.38:50430) 2024-05-21T15:41:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.494+0000 7f1133275640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f112c10d650 con 0x7f112c102bc0 2024-05-21T15:41:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.494+0000 7f1131011640 1 -- 172.21.15.38:0/524018095 learned_addr learned my addr 172.21.15.38:0/524018095 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.494+0000 7f1133275640 1 -- 172.21.15.38:0/524018095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f112c10d4d0 con 0x7f112c101980 2024-05-21T15:41:23.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.494+0000 7f1133275640 1 -- 172.21.15.38:0/524018095 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f112c10d7d0 con 0x7f112c1078c0 2024-05-21T15:41:23.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.494+0000 7f1131812640 1 --2- 172.21.15.38:0/524018095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f112c1078c0 0x7f112c1a6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.494+0000 7f1131011640 1 -- 172.21.15.38:0/524018095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f112c1078c0 msgr2=0x7f112c1a6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.495+0000 7f1131011640 1 --2- 172.21.15.38:0/524018095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f112c1078c0 0x7f112c1a6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.495+0000 7f1131011640 1 -- 172.21.15.38:0/524018095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f112c102bc0 msgr2=0x7f112c1a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.495+0000 7f1131011640 1 --2- 172.21.15.38:0/524018095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f112c102bc0 0x7f112c1a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.495+0000 7f1131011640 1 -- 172.21.15.38:0/524018095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112c1a7380 con 0x7f112c101980 2024-05-21T15:41:23.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.495+0000 7f1131011640 1 --2- 172.21.15.38:0/524018095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112c101980 0x7f112c19fbb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1128005fd0 tx=0x7f112800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.495+0000 7f11227fc640 1 -- 172.21.15.38:0/524018095 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1128017070 con 0x7f112c101980 2024-05-21T15:41:23.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.495+0000 7f1133275640 1 -- 172.21.15.38:0/524018095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112c1a7670 con 0x7f112c101980 2024-05-21T15:41:23.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.495+0000 7f1133275640 1 -- 172.21.15.38:0/524018095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f112c1a7c10 con 0x7f112c101980 2024-05-21T15:41:23.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.496+0000 7f11227fc640 1 -- 172.21.15.38:0/524018095 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1128009c30 con 0x7f112c101980 2024-05-21T15:41:23.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.496+0000 7f11227fc640 1 -- 172.21.15.38:0/524018095 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1128003e40 con 0x7f112c101980 2024-05-21T15:41:23.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.499+0000 7f113a1f1740 1 Processor -- start 2024-05-21T15:41:23.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.499+0000 7f11227fc640 1 -- 172.21.15.38:0/524018095 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1128004020 con 0x7f112c101980 2024-05-21T15:41:23.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.499+0000 7f113a1f1740 1 -- start start 2024-05-21T15:41:23.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.499+0000 7f113a1f1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28f3390 0x559aa2c92b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.499+0000 7f113a1f1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28ca4d0 0x559aa2c93080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.499+0000 7f113a1f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28d26b0 0x559aa2c99f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.499+0000 7f113a1f1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559aa2bf7c40 con 0x559aa28f3390 2024-05-21T15:41:23.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.499+0000 7f113a1f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559aa2b9f580 con 0x559aa28d26b0 2024-05-21T15:41:23.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.499+0000 7f113a1f1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559aa2bf3c00 con 0x559aa28ca4d0 2024-05-21T15:41:23.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.500+0000 7f1131812640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28f3390 0x559aa2c92b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.500+0000 7f1131812640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28f3390 0x559aa2c92b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41012/0 (socket says 172.21.15.38:41012) 2024-05-21T15:41:23.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.500+0000 7f1131812640 1 -- 172.21.15.38:0/3714861321 learned_addr learned my addr 172.21.15.38:0/3714861321 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.500+0000 7f1131011640 1 --2- 172.21.15.38:0/3714861321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28ca4d0 0x559aa2c93080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.500+0000 7f1130810640 1 --2- 172.21.15.38:0/3714861321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28d26b0 0x559aa2c99f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.500+0000 7f1131812640 1 -- 172.21.15.38:0/3714861321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28ca4d0 msgr2=0x559aa2c93080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.500+0000 7f1131812640 1 --2- 172.21.15.38:0/3714861321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28ca4d0 0x559aa2c93080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.500+0000 7f1131812640 1 -- 172.21.15.38:0/3714861321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28d26b0 msgr2=0x559aa2c99f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.500+0000 7f1131812640 1 --2- 172.21.15.38:0/3714861321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28d26b0 0x559aa2c99f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.500+0000 7f1131812640 1 -- 172.21.15.38:0/3714861321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aa290b3f0 con 0x559aa28f3390 2024-05-21T15:41:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.500+0000 7f11227fc640 1 --2- 172.21.15.38:0/524018095 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11180a2c10 0x7f11180a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.500+0000 7f1131812640 1 --2- 172.21.15.38:0/3714861321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28f3390 0x559aa2c92b00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1124009f40 tx=0x7f1124010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3c1f79777d07bb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.501+0000 7f1131011640 1 --2- 172.21.15.38:0/524018095 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11180a2c10 0x7f11180a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.501+0000 7f11177fe640 1 -- 172.21.15.38:0/3714861321 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124003b70 con 0x559aa28f3390 2024-05-21T15:41:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.501+0000 7f11177fe640 1 -- 172.21.15.38:0/3714861321 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1124003d10 con 0x559aa28f3390 2024-05-21T15:41:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.501+0000 7f11177fe640 1 -- 172.21.15.38:0/3714861321 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124011a40 con 0x559aa28f3390 2024-05-21T15:41:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.501+0000 7f11227fc640 1 -- 172.21.15.38:0/524018095 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f11280ceda0 con 0x7f112c101980 2024-05-21T15:41:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.502+0000 7f113a1f1740 1 -- 172.21.15.38:0/3714861321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28f3390 msgr2=0x559aa2c92b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.502+0000 7f1131011640 1 --2- 172.21.15.38:0/524018095 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11180a2c10 0x7f11180a5120 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x559aa2c93ee0 tx=0x7f1128086040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.502+0000 7f113a1f1740 1 --2- 172.21.15.38:0/3714861321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28f3390 0x559aa2c92b00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1124009f40 tx=0x7f1124010860 comp rx=0 tx=0).stop 2024-05-21T15:41:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.502+0000 7f113a1f1740 1 -- 172.21.15.38:0/3714861321 shutdown_connections 2024-05-21T15:41:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.502+0000 7f113a1f1740 1 --2- 172.21.15.38:0/3714861321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28d26b0 0x559aa2c99f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.502+0000 7f113a1f1740 1 --2- 172.21.15.38:0/3714861321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28ca4d0 0x559aa2c93080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.502+0000 7f113a1f1740 1 --2- 172.21.15.38:0/3714861321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28f3390 0x559aa2c92b00 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.502+0000 7f113a1f1740 1 -- 172.21.15.38:0/3714861321 >> 172.21.15.38:0/3714861321 conn(0x559aa28e74e0 msgr2=0x559aa2c8e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.502+0000 7f113a1f1740 1 -- 172.21.15.38:0/3714861321 shutdown_connections 2024-05-21T15:41:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.502+0000 7f113a1f1740 1 -- 172.21.15.38:0/3714861321 wait complete. 2024-05-21T15:41:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.503+0000 7f113a1f1740 1 -- start start 2024-05-21T15:41:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.503+0000 7f113a1f1740 1 Processor -- start 2024-05-21T15:41:23.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.503+0000 7f113a1f1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28ca4d0 0x559aa2c9c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.503+0000 7f113a1f1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28d26b0 0x559aa2c9cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.503+0000 7f113a1f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28f3390 0x559aa2ca3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.503+0000 7f113a1f1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559aa2c9a9d0 con 0x559aa28d26b0 2024-05-21T15:41:23.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.503+0000 7f113a1f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559aa2bb2c40 con 0x559aa28f3390 2024-05-21T15:41:23.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.503+0000 7f113a1f1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559aa2c9ab50 con 0x559aa28ca4d0 2024-05-21T15:41:23.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.504+0000 7f113a1f1740 10 client.0 _ll_register_callbacks cb 0x559aa290abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:23.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.504+0000 7f1131812640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28ca4d0 0x559aa2c9c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.504+0000 7f1131812640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28ca4d0 0x559aa2c9c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:59584/0 (socket says 172.21.15.38:59584) 2024-05-21T15:41:23.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.504+0000 7f1131812640 1 -- 172.21.15.38:0/589240736 learned_addr learned my addr 172.21.15.38:0/589240736 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.504+0000 7f1131812640 1 --2- 172.21.15.38:0/589240736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28f3390 0x559aa2ca3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.504+0000 7f1131812640 1 -- 172.21.15.38:0/589240736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28f3390 msgr2=0x559aa2ca3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.504+0000 7f1131812640 1 --2- 172.21.15.38:0/589240736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28f3390 0x559aa2ca3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.504+0000 7f1131812640 1 -- 172.21.15.38:0/589240736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28d26b0 msgr2=0x559aa2c9cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.504+0000 7f1131812640 1 --2- 172.21.15.38:0/589240736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28d26b0 0x559aa2c9cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.504+0000 7f1131812640 1 -- 172.21.15.38:0/589240736 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aa29d57d0 con 0x559aa28ca4d0 2024-05-21T15:41:23.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.504+0000 7f1131812640 1 --2- 172.21.15.38:0/589240736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28ca4d0 0x559aa2c9c7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1124009d50 tx=0x7f1124013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4263f673d803a295 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.504+0000 7f1114ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:41:23.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.505+0000 7f1114ff9640 1 -- 172.21.15.38:0/589240736 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112402f060 con 0x559aa28ca4d0 2024-05-21T15:41:23.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.505+0000 7f1114ff9640 1 -- 172.21.15.38:0/589240736 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f11240119f0 con 0x559aa28ca4d0 2024-05-21T15:41:23.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.505+0000 7f113a1f1740 1 -- 172.21.15.38:0/589240736 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1124003d10 con 0x559aa28ca4d0 2024-05-21T15:41:23.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.505+0000 7f1114ff9640 1 -- 172.21.15.38:0/589240736 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1124004630 con 0x559aa28ca4d0 2024-05-21T15:41:23.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.505+0000 7f113a1f1740 10 client.27058 fetch_fsmap learned FSMap version 34 2024-05-21T15:41:23.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.505+0000 7f113a1f1740 1 -- 172.21.15.38:0/589240736 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559aa2ca6f00 con 0x559aa28ca4d0 2024-05-21T15:41:23.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.505+0000 7f1114ff9640 1 -- 172.21.15.38:0/589240736 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1124011bf0 con 0x559aa28ca4d0 2024-05-21T15:41:23.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.506+0000 7f1114ff9640 1 -- 172.21.15.38:0/589240736 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f1124033070 con 0x559aa28ca4d0 2024-05-21T15:41:23.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.506+0000 7f113a1f1740 10 client.27058 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T15:41:23.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.506+0000 7f113a1f1740 10 client.27058 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:41:23.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.506+0000 7f113a1f1740 20 client.27058 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:23.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.506+0000 7f113a1f1740 1 --2- 172.21.15.38:0/589240736 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x559aa28b1f60 0x559aa2caab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.506+0000 7f113a1f1740 4 client.27058 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:23.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.506+0000 7f113a1f1740 1 -- 172.21.15.38:0/589240736 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1124003d10 con 0x559aa28b1f60 2024-05-21T15:41:23.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.506+0000 7f1130810640 1 --2- 172.21.15.38:0/589240736 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x559aa28b1f60 0x559aa2caab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.507+0000 7f1130810640 1 --2- 172.21.15.38:0/589240736 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x559aa28b1f60 0x559aa2caab60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bfaf0df0fcbf2c79 server_cookie=cf68e986a3d9088 in_seq=0 out_seq=0 2024-05-21T15:41:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.507+0000 7f1114ff9640 10 client.27058 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:41:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.507+0000 7f1114ff9640 1 -- 172.21.15.38:0/589240736 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f1124003d10 con 0x559aa28b1f60 2024-05-21T15:41:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.507+0000 7f1114ff9640 10 client.27058 handle_command_reply: tid=0 2024-05-21T15:41:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.507+0000 7f113a1f1740 1 client.27058 shutdown 2024-05-21T15:41:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.508+0000 7f113a1f1740 1 -- 172.21.15.38:0/589240736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28ca4d0 msgr2=0x559aa2c9c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.508+0000 7f113a1f1740 1 --2- 172.21.15.38:0/589240736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28ca4d0 0x559aa2c9c7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1124009d50 tx=0x7f1124013040 comp rx=0 tx=0).stop 2024-05-21T15:41:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.508+0000 7f113a1f1740 1 -- 172.21.15.38:0/589240736 shutdown_connections 2024-05-21T15:41:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.508+0000 7f113a1f1740 1 --2- 172.21.15.38:0/589240736 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x559aa28b1f60 0x559aa2caab60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.508+0000 7f113a1f1740 1 --2- 172.21.15.38:0/589240736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28f3390 0x559aa2ca3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.508+0000 7f113a1f1740 1 --2- 172.21.15.38:0/589240736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28d26b0 0x559aa2c9cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.508+0000 7f113a1f1740 1 --2- 172.21.15.38:0/589240736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28ca4d0 0x559aa2c9c7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.508+0000 7f113a1f1740 1 -- 172.21.15.38:0/589240736 >> 172.21.15.38:0/589240736 conn(0x559aa28e74e0 msgr2=0x559aa2c91440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.508+0000 7f1114ff9640 0 client.27058 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:41:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.508+0000 7f113a1f1740 1 -- 172.21.15.38:0/589240736 shutdown_connections 2024-05-21T15:41:23.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.508+0000 7f113a1f1740 1 -- 172.21.15.38:0/589240736 wait complete. 2024-05-21T15:41:23.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.508+0000 7f113a1f1740 20 client.27058 trim_cache size 0 max 16384 2024-05-21T15:41:23.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.529+0000 7f113a1f1740 1 Processor -- start 2024-05-21T15:41:23.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.529+0000 7f113a1f1740 1 -- start start 2024-05-21T15:41:23.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.529+0000 7f113a1f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28ca4d0 0x559aa2c9bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.529+0000 7f113a1f1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28d26b0 0x559aa2ca3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.529+0000 7f113a1f1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28f3390 0x559aa2c91a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.529+0000 7f113a1f1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f111000e550 con 0x559aa28d26b0 2024-05-21T15:41:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.529+0000 7f113a1f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f11100100c0 con 0x559aa28ca4d0 2024-05-21T15:41:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.529+0000 7f113a1f1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f111000ec20 con 0x559aa28f3390 2024-05-21T15:41:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.529+0000 7f1130810640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28d26b0 0x559aa2ca3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.529+0000 7f1130810640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28d26b0 0x559aa2ca3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41036/0 (socket says 172.21.15.38:41036) 2024-05-21T15:41:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.529+0000 7f1130810640 1 -- 172.21.15.38:0/1427019798 learned_addr learned my addr 172.21.15.38:0/1427019798 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.530+0000 7f1131812640 1 --2- 172.21.15.38:0/1427019798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28ca4d0 0x559aa2c9bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.530+0000 7f1131812640 1 --2- 172.21.15.38:0/1427019798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28f3390 0x559aa2c91a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.530+0000 7f1130810640 1 -- 172.21.15.38:0/1427019798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28f3390 msgr2=0x559aa2c91a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.530+0000 7f1130810640 1 --2- 172.21.15.38:0/1427019798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28f3390 0x559aa2c91a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.530+0000 7f1130810640 1 -- 172.21.15.38:0/1427019798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28ca4d0 msgr2=0x559aa2c9bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.530+0000 7f1130810640 1 --2- 172.21.15.38:0/1427019798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28ca4d0 0x559aa2c9bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.530+0000 7f1130810640 1 -- 172.21.15.38:0/1427019798 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aa2c9a4f0 con 0x559aa28d26b0 2024-05-21T15:41:23.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.530+0000 7f1131812640 1 --2- 172.21.15.38:0/1427019798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28ca4d0 0x559aa2c9bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:41:23.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.530+0000 7f1130810640 1 --2- 172.21.15.38:0/1427019798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28d26b0 0x559aa2ca3bd0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f111c0096c0 tx=0x7f111c0314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5c1847530e0719 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.530+0000 7f1116ffd640 1 -- 172.21.15.38:0/1427019798 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c04c070 con 0x559aa28d26b0 2024-05-21T15:41:23.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.530+0000 7f1116ffd640 1 -- 172.21.15.38:0/1427019798 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111c031b90 con 0x559aa28d26b0 2024-05-21T15:41:23.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.530+0000 7f1116ffd640 1 -- 172.21.15.38:0/1427019798 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c035040 con 0x559aa28d26b0 2024-05-21T15:41:23.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.531+0000 7f113a1f1740 1 -- 172.21.15.38:0/1427019798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28d26b0 msgr2=0x559aa2ca3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.531+0000 7f113a1f1740 1 --2- 172.21.15.38:0/1427019798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28d26b0 0x559aa2ca3bd0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f111c0096c0 tx=0x7f111c0314e0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.531+0000 7f113a1f1740 1 -- 172.21.15.38:0/1427019798 shutdown_connections 2024-05-21T15:41:23.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.531+0000 7f113a1f1740 1 --2- 172.21.15.38:0/1427019798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28f3390 0x559aa2c91a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.531+0000 7f113a1f1740 1 --2- 172.21.15.38:0/1427019798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28d26b0 0x559aa2ca3bd0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.532+0000 7f113a1f1740 1 --2- 172.21.15.38:0/1427019798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28ca4d0 0x559aa2c9bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.532+0000 7f113a1f1740 1 -- 172.21.15.38:0/1427019798 >> 172.21.15.38:0/1427019798 conn(0x559aa28e74e0 msgr2=0x559aa2c91330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.532+0000 7f113a1f1740 1 -- 172.21.15.38:0/1427019798 shutdown_connections 2024-05-21T15:41:23.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.532+0000 7f113a1f1740 1 -- 172.21.15.38:0/1427019798 wait complete. 2024-05-21T15:41:23.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.532+0000 7f113a1f1740 1 -- start start 2024-05-21T15:41:23.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.532+0000 7f113a1f1740 1 Processor -- start 2024-05-21T15:41:23.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.533+0000 7f113a1f1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28ca4d0 0x559aa2ca3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.533+0000 7f113a1f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28d26b0 0x559aa2ca2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.533+0000 7f113a1f1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28f3390 0x559aa2ca28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.533+0000 7f113a1f1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f111000f9c0 con 0x559aa28ca4d0 2024-05-21T15:41:23.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.533+0000 7f113a1f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f111000f2f0 con 0x559aa28d26b0 2024-05-21T15:41:23.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.533+0000 7f113a1f1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f111000c530 con 0x559aa28f3390 2024-05-21T15:41:23.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.533+0000 7f1131812640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28ca4d0 0x559aa2ca3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.533+0000 7f1131812640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28ca4d0 0x559aa2ca3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41050/0 (socket says 172.21.15.38:41050) 2024-05-21T15:41:23.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.533+0000 7f1131812640 1 -- 172.21.15.38:0/3582017166 learned_addr learned my addr 172.21.15.38:0/3582017166 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.533+0000 7f113a1f1740 10 client.0 _ll_register_callbacks cb 0x7f11240093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:23.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.533+0000 7f1130810640 1 --2- 172.21.15.38:0/3582017166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28d26b0 0x559aa2ca2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.533+0000 7f1131812640 1 --2- 172.21.15.38:0/3582017166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28f3390 0x559aa2ca28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.534+0000 7f1131812640 1 -- 172.21.15.38:0/3582017166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28f3390 msgr2=0x559aa2ca28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.534+0000 7f1131812640 1 --2- 172.21.15.38:0/3582017166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28f3390 0x559aa2ca28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.534+0000 7f1131812640 1 -- 172.21.15.38:0/3582017166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28d26b0 msgr2=0x559aa2ca2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.534+0000 7f1131812640 1 --2- 172.21.15.38:0/3582017166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28d26b0 0x559aa2ca2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.534+0000 7f1131812640 1 -- 172.21.15.38:0/3582017166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aa2ca3040 con 0x559aa28ca4d0 2024-05-21T15:41:23.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.534+0000 7f1131812640 1 --2- 172.21.15.38:0/3582017166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28ca4d0 0x559aa2ca3bd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f112402a500 tx=0x7f112401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4306d7e1e447f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.534+0000 7f1114ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.534+0000 7f1114ff9640 1 -- 172.21.15.38:0/3582017166 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124038040 con 0x559aa28ca4d0 2024-05-21T15:41:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.534+0000 7f113a1f1740 1 -- 172.21.15.38:0/3582017166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f111c031b90 con 0x559aa28ca4d0 2024-05-21T15:41:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.534+0000 7f1114ff9640 1 -- 172.21.15.38:0/3582017166 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f1124003c80 con 0x559aa28ca4d0 2024-05-21T15:41:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.534+0000 7f1114ff9640 1 -- 172.21.15.38:0/3582017166 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f112402f040 con 0x559aa28ca4d0 2024-05-21T15:41:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.535+0000 7f113a1f1740 10 client.17370 fetch_fsmap learned FSMap version 34 2024-05-21T15:41:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.535+0000 7f113a1f1740 1 -- 172.21.15.38:0/3582017166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559aa2d11f40 con 0x559aa28ca4d0 2024-05-21T15:41:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.535+0000 7f1114ff9640 1 -- 172.21.15.38:0/3582017166 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111c031b90 con 0x559aa28ca4d0 2024-05-21T15:41:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.535+0000 7f1114ff9640 1 -- 172.21.15.38:0/3582017166 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7f112403e070 con 0x559aa28ca4d0 2024-05-21T15:41:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.535+0000 7f113a1f1740 10 client.17370 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T15:41:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.535+0000 7f113a1f1740 10 client.17370 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:41:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.535+0000 7f113a1f1740 20 client.17370 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.535+0000 7f113a1f1740 1 --2- 172.21.15.38:0/3582017166 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x559aa28873a0 0x559aa2d148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.535+0000 7f113a1f1740 4 client.17370 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:41:23.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.535+0000 7f113a1f1740 1 -- 172.21.15.38:0/3582017166 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559aa2c05370 con 0x559aa28873a0 2024-05-21T15:41:23.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.535+0000 7f1130810640 1 --2- 172.21.15.38:0/3582017166 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x559aa28873a0 0x559aa2d148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.536+0000 7f1114ff9640 10 client.17370 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:41:23.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.536+0000 7f1130810640 1 --2- 172.21.15.38:0/3582017166 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x559aa28873a0 0x559aa2d148d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bca15d75197af325 server_cookie=7df830a16aa27ca4 in_seq=0 out_seq=0 2024-05-21T15:41:23.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.536+0000 7f1114ff9640 1 -- 172.21.15.38:0/3582017166 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559aa2c05370 con 0x559aa28873a0 2024-05-21T15:41:23.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.536+0000 7f1114ff9640 10 client.17370 handle_command_reply: tid=0 2024-05-21T15:41:23.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.536+0000 7f113a1f1740 1 client.17370 shutdown 2024-05-21T15:41:23.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.536+0000 7f113a1f1740 1 -- 172.21.15.38:0/3582017166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28ca4d0 msgr2=0x559aa2ca3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.536+0000 7f113a1f1740 1 --2- 172.21.15.38:0/3582017166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28ca4d0 0x559aa2ca3bd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f112402a500 tx=0x7f112401d040 comp rx=0 tx=0).stop 2024-05-21T15:41:23.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.537+0000 7f113a1f1740 1 -- 172.21.15.38:0/3582017166 shutdown_connections 2024-05-21T15:41:23.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.537+0000 7f113a1f1740 1 --2- 172.21.15.38:0/3582017166 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x559aa28873a0 0x559aa2d148d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.537+0000 7f113a1f1740 1 --2- 172.21.15.38:0/3582017166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559aa28f3390 0x559aa2ca28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.537+0000 7f113a1f1740 1 --2- 172.21.15.38:0/3582017166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559aa28d26b0 0x559aa2ca2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.537+0000 7f113a1f1740 1 --2- 172.21.15.38:0/3582017166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559aa28ca4d0 0x559aa2ca3bd0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.537+0000 7f113a1f1740 1 -- 172.21.15.38:0/3582017166 >> 172.21.15.38:0/3582017166 conn(0x559aa28e74e0 msgr2=0x559aa2c91d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.537+0000 7f1114ff9640 0 client.17370 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:41:23.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.537+0000 7f113a1f1740 1 -- 172.21.15.38:0/3582017166 shutdown_connections 2024-05-21T15:41:23.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.537+0000 7f113a1f1740 1 -- 172.21.15.38:0/3582017166 wait complete. 2024-05-21T15:41:23.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.537+0000 7f113a1f1740 20 client.17370 trim_cache size 0 max 16384 2024-05-21T15:41:23.542 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:41:23.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.540+0000 7f1133275640 1 -- 172.21.15.38:0/524018095 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11180a2c10 msgr2=0x7f11180a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.540+0000 7f1133275640 1 --2- 172.21.15.38:0/524018095 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11180a2c10 0x7f11180a5120 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x559aa2c93ee0 tx=0x7f1128086040 comp rx=0 tx=0).stop 2024-05-21T15:41:23.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.540+0000 7f1133275640 1 -- 172.21.15.38:0/524018095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112c101980 msgr2=0x7f112c19fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.540+0000 7f1133275640 1 --2- 172.21.15.38:0/524018095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112c101980 0x7f112c19fbb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1128005fd0 tx=0x7f112800dde0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.540+0000 7f1133275640 1 -- 172.21.15.38:0/524018095 shutdown_connections 2024-05-21T15:41:23.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.540+0000 7f1133275640 1 --2- 172.21.15.38:0/524018095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f112c1078c0 0x7f112c1a6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.540+0000 7f1133275640 1 --2- 172.21.15.38:0/524018095 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f11180a2c10 0x7f11180a5120 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.540+0000 7f1133275640 1 --2- 172.21.15.38:0/524018095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f112c102bc0 0x7f112c1a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.540+0000 7f1133275640 1 --2- 172.21.15.38:0/524018095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112c101980 0x7f112c19fbb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.540+0000 7f1133275640 1 -- 172.21.15.38:0/524018095 >> 172.21.15.38:0/524018095 conn(0x7f112c0fcb70 msgr2=0x7f112c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.540+0000 7f1133275640 1 -- 172.21.15.38:0/524018095 shutdown_connections 2024-05-21T15:41:23.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.540+0000 7f1133275640 1 -- 172.21.15.38:0/524018095 wait complete. 2024-05-21T15:41:23.553 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:41:23.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.704+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2652910951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd38102bc0 msgr2=0x7fdd38102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.704+0000 7fdd3fdea640 1 --2- 172.21.15.38:0/2652910951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd38102bc0 0x7fdd38102fd0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdd28009900 tx=0x7fdd2802a410 comp rx=0 tx=0).stop 2024-05-21T15:41:23.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.705+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2652910951 shutdown_connections 2024-05-21T15:41:23.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.705+0000 7fdd3fdea640 1 --2- 172.21.15.38:0/2652910951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd381078c0 0x7fdd38109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.705+0000 7fdd3fdea640 1 --2- 172.21.15.38:0/2652910951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd38102bc0 0x7fdd38102fd0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.705+0000 7fdd3fdea640 1 --2- 172.21.15.38:0/2652910951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd38101980 0x7fdd38102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.705+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2652910951 >> 172.21.15.38:0/2652910951 conn(0x7fdd380fcb70 msgr2=0x7fdd380ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.705+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2652910951 shutdown_connections 2024-05-21T15:41:23.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.705+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2652910951 wait complete. 2024-05-21T15:41:23.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.705+0000 7fdd3fdea640 1 Processor -- start 2024-05-21T15:41:23.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.706+0000 7fdd3fdea640 1 -- start start 2024-05-21T15:41:23.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.706+0000 7fdd3fdea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd38101980 0x7fdd38197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.706+0000 7fdd3fdea640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd38102bc0 0x7fdd381975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.706+0000 7fdd3db86640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd38101980 0x7fdd38197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.706+0000 7fdd3fdea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd381078c0 0x7fdd3819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.706+0000 7fdd3db86640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd38101980 0x7fdd38197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41054/0 (socket says 172.21.15.38:41054) 2024-05-21T15:41:23.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.706+0000 7fdd3db86640 1 -- 172.21.15.38:0/2723346866 learned_addr learned my addr 172.21.15.38:0/2723346866 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.706+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2723346866 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3810c530 con 0x7fdd38101980 2024-05-21T15:41:23.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.706+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2723346866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3810c3b0 con 0x7fdd381078c0 2024-05-21T15:41:23.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.706+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2723346866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3810c6b0 con 0x7fdd38102bc0 2024-05-21T15:41:23.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.706+0000 7fdd3d385640 1 --2- 172.21.15.38:0/2723346866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd38102bc0 0x7fdd381975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.706+0000 7fdd3e387640 1 --2- 172.21.15.38:0/2723346866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd381078c0 0x7fdd3819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.706+0000 7fdd3db86640 1 -- 172.21.15.38:0/2723346866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd38102bc0 msgr2=0x7fdd381975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.706+0000 7fdd3db86640 1 --2- 172.21.15.38:0/2723346866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd38102bc0 0x7fdd381975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.707+0000 7fdd3db86640 1 -- 172.21.15.38:0/2723346866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd381078c0 msgr2=0x7fdd3819e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.707+0000 7fdd3db86640 1 --2- 172.21.15.38:0/2723346866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd381078c0 0x7fdd3819e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.707+0000 7fdd3db86640 1 -- 172.21.15.38:0/2723346866 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3819ebb0 con 0x7fdd38101980 2024-05-21T15:41:23.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.707+0000 7fdd3db86640 1 --2- 172.21.15.38:0/2723346866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd38101980 0x7fdd38197070 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400e9f0 tx=0x7fdd3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.707+0000 7fdd2effd640 1 -- 172.21.15.38:0/2723346866 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3400ccc0 con 0x7fdd38101980 2024-05-21T15:41:23.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.707+0000 7fdd2effd640 1 -- 172.21.15.38:0/2723346866 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd340035f0 con 0x7fdd38101980 2024-05-21T15:41:23.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.707+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2723346866 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3819eea0 con 0x7fdd38101980 2024-05-21T15:41:23.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.707+0000 7fdd2effd640 1 -- 172.21.15.38:0/2723346866 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3400ce60 con 0x7fdd38101980 2024-05-21T15:41:23.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.707+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2723346866 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd3819f2a0 con 0x7fdd38101980 2024-05-21T15:41:23.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.711+0000 7fdd46d69740 1 Processor -- start 2024-05-21T15:41:23.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.711+0000 7fdd2effd640 1 -- 172.21.15.38:0/2723346866 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdd34010750 con 0x7fdd38101980 2024-05-21T15:41:23.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.711+0000 7fdd46d69740 1 -- start start 2024-05-21T15:41:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd46d69740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d97390 0x55f6e7135570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd46d69740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7135af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd46d69740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d766b0 0x55f6e7132220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd46d69740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f6e709bc40 con 0x55f6e6d97390 2024-05-21T15:41:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd46d69740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f6e7043580 con 0x55f6e6d766b0 2024-05-21T15:41:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd46d69740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f6e7097c00 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd3e387640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d97390 0x55f6e7135570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd3db86640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7135af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd3e387640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d97390 0x55f6e7135570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41068/0 (socket says 172.21.15.38:41068) 2024-05-21T15:41:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd3e387640 1 -- 172.21.15.38:0/4144368384 learned_addr learned my addr 172.21.15.38:0/4144368384 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd3d385640 1 --2- 172.21.15.38:0/4144368384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d766b0 0x55f6e7132220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd3e387640 1 -- 172.21.15.38:0/4144368384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 msgr2=0x55f6e7135af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd3e387640 1 --2- 172.21.15.38:0/4144368384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7135af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd3e387640 1 -- 172.21.15.38:0/4144368384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d766b0 msgr2=0x55f6e7132220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd3e387640 1 --2- 172.21.15.38:0/4144368384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d766b0 0x55f6e7132220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.712+0000 7fdd3e387640 1 -- 172.21.15.38:0/4144368384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6e6daf3f0 con 0x55f6e6d97390 2024-05-21T15:41:23.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.713+0000 7fdd3d385640 1 --2- 172.21.15.38:0/4144368384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d766b0 0x55f6e7132220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:41:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.713+0000 7fdd3e387640 1 --2- 172.21.15.38:0/4144368384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d97390 0x55f6e7135570 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009310 tx=0x7fdd3000b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f21c18285cbd0c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.713+0000 7fdd2effd640 1 --2- 172.21.15.38:0/2723346866 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdd240a2b90 0x7fdd240a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.714+0000 7fdd23fff640 1 -- 172.21.15.38:0/4144368384 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30003b70 con 0x55f6e6d97390 2024-05-21T15:41:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.714+0000 7fdd3db86640 1 --2- 172.21.15.38:0/2723346866 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdd240a2b90 0x7fdd240a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.714+0000 7fdd23fff640 1 -- 172.21.15.38:0/4144368384 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd3001b070 con 0x55f6e6d97390 2024-05-21T15:41:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.714+0000 7fdd23fff640 1 -- 172.21.15.38:0/4144368384 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd300167a0 con 0x55f6e6d97390 2024-05-21T15:41:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.714+0000 7fdd2effd640 1 -- 172.21.15.38:0/2723346866 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdd340cf5a0 con 0x7fdd38101980 2024-05-21T15:41:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.715+0000 7fdd46d69740 1 -- 172.21.15.38:0/4144368384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d97390 msgr2=0x55f6e7135570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.715+0000 7fdd46d69740 1 --2- 172.21.15.38:0/4144368384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d97390 0x55f6e7135570 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009310 tx=0x7fdd3000b710 comp rx=0 tx=0).stop 2024-05-21T15:41:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.715+0000 7fdd3db86640 1 --2- 172.21.15.38:0/2723346866 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdd240a2b90 0x7fdd240a50a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55f6e71334f0 tx=0x7fdd340d4040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.715+0000 7fdd46d69740 1 -- 172.21.15.38:0/4144368384 shutdown_connections 2024-05-21T15:41:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.715+0000 7fdd46d69740 1 --2- 172.21.15.38:0/4144368384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d766b0 0x55f6e7132220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.715+0000 7fdd46d69740 1 --2- 172.21.15.38:0/4144368384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7135af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.715+0000 7fdd46d69740 1 --2- 172.21.15.38:0/4144368384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d97390 0x55f6e7135570 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.715+0000 7fdd46d69740 1 -- 172.21.15.38:0/4144368384 >> 172.21.15.38:0/4144368384 conn(0x55f6e6d8b4e0 msgr2=0x55f6e71368a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.715+0000 7fdd46d69740 1 -- 172.21.15.38:0/4144368384 shutdown_connections 2024-05-21T15:41:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.715+0000 7fdd46d69740 1 -- 172.21.15.38:0/4144368384 wait complete. 2024-05-21T15:41:23.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.716+0000 7fdd46d69740 1 -- start start 2024-05-21T15:41:23.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.716+0000 7fdd46d69740 1 Processor -- start 2024-05-21T15:41:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.716+0000 7fdd46d69740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7140740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.716+0000 7fdd46d69740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d766b0 0x55f6e7140c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd46d69740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d97390 0x55f6e7145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd3e387640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7140740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd3e387640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7140740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:59638/0 (socket says 172.21.15.38:59638) 2024-05-21T15:41:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd3e387640 1 -- 172.21.15.38:0/3813008935 learned_addr learned my addr 172.21.15.38:0/3813008935 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd46d69740 1 -- 172.21.15.38:0/3813008935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f6e713ab60 con 0x55f6e6d766b0 2024-05-21T15:41:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd46d69740 1 -- 172.21.15.38:0/3813008935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f6e7056c40 con 0x55f6e6d97390 2024-05-21T15:41:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd46d69740 1 -- 172.21.15.38:0/3813008935 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f6e713ace0 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd46d69740 10 client.0 _ll_register_callbacks cb 0x55f6e6daebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd3e387640 1 --2- 172.21.15.38:0/3813008935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d97390 0x55f6e7145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd3d385640 1 --2- 172.21.15.38:0/3813008935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d766b0 0x55f6e7140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd3e387640 1 -- 172.21.15.38:0/3813008935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d97390 msgr2=0x55f6e7145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd3e387640 1 --2- 172.21.15.38:0/3813008935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d97390 0x55f6e7145bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd3e387640 1 -- 172.21.15.38:0/3813008935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d766b0 msgr2=0x55f6e7140c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd3e387640 1 --2- 172.21.15.38:0/3813008935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d766b0 0x55f6e7140c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.717+0000 7fdd3e387640 1 -- 172.21.15.38:0/3813008935 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6e6e797d0 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.718+0000 7fdd3e387640 1 --2- 172.21.15.38:0/3813008935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7140740 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000b840 tx=0x7fdd30000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c03d7b51080a68b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.718+0000 7fdd217fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:41:23.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.718+0000 7fdd217fa640 1 -- 172.21.15.38:0/3813008935 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd300168f0 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.718+0000 7fdd217fa640 1 -- 172.21.15.38:0/3813008935 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdd30016a90 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.718+0000 7fdd46d69740 1 -- 172.21.15.38:0/3813008935 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd3001b070 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.718+0000 7fdd217fa640 1 -- 172.21.15.38:0/3813008935 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdd30016c90 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.718+0000 7fdd46d69740 10 client.27076 fetch_fsmap learned FSMap version 34 2024-05-21T15:41:23.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.718+0000 7fdd46d69740 1 -- 172.21.15.38:0/3813008935 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f6e715be30 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.719+0000 7fdd217fa640 1 -- 172.21.15.38:0/3813008935 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd3002b070 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.719+0000 7fdd217fa640 1 -- 172.21.15.38:0/3813008935 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fdd30023430 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.719+0000 7fdd46d69740 10 client.27076 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T15:41:23.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.719+0000 7fdd46d69740 10 client.27076 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:41:23.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.719+0000 7fdd46d69740 20 client.27076 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:23.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.719+0000 7fdd46d69740 1 --2- 172.21.15.38:0/3813008935 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55f6e6d55f60 0x55f6e715f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.719+0000 7fdd46d69740 4 client.27076 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:23.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.719+0000 7fdd46d69740 1 -- 172.21.15.38:0/3813008935 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdd3001b070 con 0x55f6e6d55f60 2024-05-21T15:41:23.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.720+0000 7fdd3d385640 1 --2- 172.21.15.38:0/3813008935 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55f6e6d55f60 0x55f6e715f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.720+0000 7fdd3d385640 1 --2- 172.21.15.38:0/3813008935 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55f6e6d55f60 0x55f6e715f2a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d51d661c25c3432f server_cookie=423def0b573b04b9 in_seq=0 out_seq=0 2024-05-21T15:41:23.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.720+0000 7fdd217fa640 10 client.27076 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:41:23.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.721+0000 7fdd217fa640 1 -- 172.21.15.38:0/3813008935 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fdd3001b070 con 0x55f6e6d55f60 2024-05-21T15:41:23.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.721+0000 7fdd217fa640 10 client.27076 handle_command_reply: tid=0 2024-05-21T15:41:23.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.721+0000 7fdd46d69740 1 client.27076 shutdown 2024-05-21T15:41:23.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.721+0000 7fdd46d69740 1 -- 172.21.15.38:0/3813008935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 msgr2=0x55f6e7140740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.721+0000 7fdd46d69740 1 --2- 172.21.15.38:0/3813008935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7140740 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000b840 tx=0x7fdd30000f40 comp rx=0 tx=0).stop 2024-05-21T15:41:23.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.722+0000 7fdd46d69740 1 -- 172.21.15.38:0/3813008935 shutdown_connections 2024-05-21T15:41:23.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.722+0000 7fdd46d69740 1 --2- 172.21.15.38:0/3813008935 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55f6e6d55f60 0x55f6e715f2a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.722+0000 7fdd46d69740 1 --2- 172.21.15.38:0/3813008935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d97390 0x55f6e7145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.722+0000 7fdd217fa640 0 client.27076 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:41:23.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.722+0000 7fdd46d69740 1 --2- 172.21.15.38:0/3813008935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d766b0 0x55f6e7140c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.722+0000 7fdd46d69740 1 --2- 172.21.15.38:0/3813008935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7140740 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.722+0000 7fdd46d69740 1 -- 172.21.15.38:0/3813008935 >> 172.21.15.38:0/3813008935 conn(0x55f6e6d8b4e0 msgr2=0x55f6e7131570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.722+0000 7fdd46d69740 1 -- 172.21.15.38:0/3813008935 shutdown_connections 2024-05-21T15:41:23.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.722+0000 7fdd46d69740 1 -- 172.21.15.38:0/3813008935 wait complete. 2024-05-21T15:41:23.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.722+0000 7fdd46d69740 20 client.27076 trim_cache size 0 max 16384 2024-05-21T15:41:23.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.741+0000 7fdd46d69740 1 Processor -- start 2024-05-21T15:41:23.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.741+0000 7fdd46d69740 1 -- start start 2024-05-21T15:41:23.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd46d69740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d55f60 0x55f6e7130b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd46d69740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7131080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd46d69740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d766b0 0x55f6e71315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd46d69740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381a82a0 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd46d69740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381a9e10 con 0x55f6e6d55f60 2024-05-21T15:41:23.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd46d69740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381a8970 con 0x55f6e6d766b0 2024-05-21T15:41:23.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd3d385640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7131080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd3d385640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7131080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41088/0 (socket says 172.21.15.38:41088) 2024-05-21T15:41:23.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd3d385640 1 -- 172.21.15.38:0/3856776788 learned_addr learned my addr 172.21.15.38:0/3856776788 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd3e387640 1 --2- 172.21.15.38:0/3856776788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d55f60 0x55f6e7130b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd3e387640 1 --2- 172.21.15.38:0/3856776788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d766b0 0x55f6e71315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd3d385640 1 -- 172.21.15.38:0/3856776788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d766b0 msgr2=0x55f6e71315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd3d385640 1 --2- 172.21.15.38:0/3856776788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d766b0 0x55f6e71315c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd3d385640 1 -- 172.21.15.38:0/3856776788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d55f60 msgr2=0x55f6e7130b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd3d385640 1 --2- 172.21.15.38:0/3856776788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d55f60 0x55f6e7130b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd3d385640 1 -- 172.21.15.38:0/3856776788 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6e713b0e0 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd3e387640 1 --2- 172.21.15.38:0/3856776788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d55f60 0x55f6e7130b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:23.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.742+0000 7fdd3d385640 1 --2- 172.21.15.38:0/3856776788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7131080 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdd28005ab0 tx=0x7fdd280315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf4959111408731 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.743+0000 7fdd237fe640 1 -- 172.21.15.38:0/3856776788 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd28035040 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.743+0000 7fdd237fe640 1 -- 172.21.15.38:0/3856776788 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd28049070 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.743+0000 7fdd237fe640 1 -- 172.21.15.38:0/3856776788 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd28004d30 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.744+0000 7fdd46d69740 1 -- 172.21.15.38:0/3856776788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d6e4d0 msgr2=0x55f6e7131080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.744+0000 7fdd46d69740 1 --2- 172.21.15.38:0/3856776788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7131080 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdd28005ab0 tx=0x7fdd280315b0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.744+0000 7fdd46d69740 1 -- 172.21.15.38:0/3856776788 shutdown_connections 2024-05-21T15:41:23.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.744+0000 7fdd46d69740 1 --2- 172.21.15.38:0/3856776788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d766b0 0x55f6e71315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.744+0000 7fdd46d69740 1 --2- 172.21.15.38:0/3856776788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d6e4d0 0x55f6e7131080 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.744+0000 7fdd46d69740 1 --2- 172.21.15.38:0/3856776788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d55f60 0x55f6e7130b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.744+0000 7fdd46d69740 1 -- 172.21.15.38:0/3856776788 >> 172.21.15.38:0/3856776788 conn(0x55f6e6d8b4e0 msgr2=0x55f6e713ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.744+0000 7fdd46d69740 1 -- 172.21.15.38:0/3856776788 shutdown_connections 2024-05-21T15:41:23.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.744+0000 7fdd46d69740 1 -- 172.21.15.38:0/3856776788 wait complete. 2024-05-21T15:41:23.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.744+0000 7fdd46d69740 1 -- start start 2024-05-21T15:41:23.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.744+0000 7fdd46d69740 1 Processor -- start 2024-05-21T15:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd46d69740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d55f60 0x55f6e716dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd46d69740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 0x55f6e716e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd46d69740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d766b0 0x55f6e716f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd46d69740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381a9710 con 0x55f6e6d766b0 2024-05-21T15:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd46d69740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381a9040 con 0x55f6e6d55f60 2024-05-21T15:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd46d69740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381a6280 con 0x55f6e6d6e4d0 2024-05-21T15:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd46d69740 10 client.0 _ll_register_callbacks cb 0x7fdd300091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd3e387640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d55f60 0x55f6e716dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd3e387640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d55f60 0x55f6e716dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:50512/0 (socket says 172.21.15.38:50512) 2024-05-21T15:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd3e387640 1 -- 172.21.15.38:0/519321908 learned_addr learned my addr 172.21.15.38:0/519321908 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd3e387640 1 --2- 172.21.15.38:0/519321908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d766b0 0x55f6e716f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd3d385640 1 --2- 172.21.15.38:0/519321908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 0x55f6e716e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd3e387640 1 -- 172.21.15.38:0/519321908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 msgr2=0x55f6e716e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd3e387640 1 --2- 172.21.15.38:0/519321908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 0x55f6e716e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd3e387640 1 -- 172.21.15.38:0/519321908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d766b0 msgr2=0x55f6e716f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.745+0000 7fdd3e387640 1 --2- 172.21.15.38:0/519321908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d766b0 0x55f6e716f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.746+0000 7fdd3e387640 1 -- 172.21.15.38:0/519321908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6e716f640 con 0x55f6e6d55f60 2024-05-21T15:41:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.746+0000 7fdd3e387640 1 --2- 172.21.15.38:0/519321908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d55f60 0x55f6e716dcd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdd30005c40 tx=0x7fdd30004640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c38f0f52bc34fe6e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.746+0000 7fdd217fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:41:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.746+0000 7fdd217fa640 1 -- 172.21.15.38:0/519321908 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30023c70 con 0x55f6e6d55f60 2024-05-21T15:41:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.746+0000 7fdd46d69740 1 -- 172.21.15.38:0/519321908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd28049070 con 0x55f6e6d55f60 2024-05-21T15:41:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.746+0000 7fdd217fa640 1 -- 172.21.15.38:0/519321908 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdd300166f0 con 0x55f6e6d55f60 2024-05-21T15:41:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.746+0000 7fdd217fa640 1 -- 172.21.15.38:0/519321908 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdd30023e10 con 0x55f6e6d55f60 2024-05-21T15:41:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.746+0000 7fdd46d69740 10 client.27011 fetch_fsmap learned FSMap version 34 2024-05-21T15:41:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.747+0000 7fdd46d69740 1 -- 172.21.15.38:0/519321908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f6e71708c0 con 0x55f6e6d55f60 2024-05-21T15:41:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.747+0000 7fdd217fa640 1 -- 172.21.15.38:0/519321908 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd28049070 con 0x55f6e6d55f60 2024-05-21T15:41:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.747+0000 7fdd217fa640 1 -- 172.21.15.38:0/519321908 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fdd300247b0 con 0x55f6e6d55f60 2024-05-21T15:41:23.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.747+0000 7fdd46d69740 10 client.27011 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T15:41:23.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.747+0000 7fdd46d69740 10 client.27011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:41:23.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.747+0000 7fdd46d69740 20 client.27011 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:23.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.747+0000 7fdd46d69740 1 --2- 172.21.15.38:0/519321908 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55f6e6d97390 0x55f6e71c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:23.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.747+0000 7fdd46d69740 4 client.27011 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:41:23.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.747+0000 7fdd46d69740 1 -- 172.21.15.38:0/519321908 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6e70a9370 con 0x55f6e6d97390 2024-05-21T15:41:23.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.748+0000 7fdd3d385640 1 --2- 172.21.15.38:0/519321908 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55f6e6d97390 0x55f6e71c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:23.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.748+0000 7fdd217fa640 10 client.27011 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:41:23.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.748+0000 7fdd3d385640 1 --2- 172.21.15.38:0/519321908 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55f6e6d97390 0x55f6e71c95a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6b9c8e36b808b5e1 server_cookie=7774df4ad60daf75 in_seq=0 out_seq=0 2024-05-21T15:41:23.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.749+0000 7fdd217fa640 1 -- 172.21.15.38:0/519321908 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6e70a9370 con 0x55f6e6d97390 2024-05-21T15:41:23.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.749+0000 7fdd217fa640 10 client.27011 handle_command_reply: tid=0 2024-05-21T15:41:23.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.749+0000 7fdd46d69740 1 client.27011 shutdown 2024-05-21T15:41:23.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.749+0000 7fdd46d69740 1 -- 172.21.15.38:0/519321908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d55f60 msgr2=0x55f6e716dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.749+0000 7fdd46d69740 1 --2- 172.21.15.38:0/519321908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d55f60 0x55f6e716dcd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdd30005c40 tx=0x7fdd30004640 comp rx=0 tx=0).stop 2024-05-21T15:41:23.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.749+0000 7fdd46d69740 1 -- 172.21.15.38:0/519321908 shutdown_connections 2024-05-21T15:41:23.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.749+0000 7fdd46d69740 1 --2- 172.21.15.38:0/519321908 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55f6e6d97390 0x55f6e71c95a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.749+0000 7fdd46d69740 1 --2- 172.21.15.38:0/519321908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f6e6d766b0 0x55f6e716f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.749+0000 7fdd46d69740 1 --2- 172.21.15.38:0/519321908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e6d6e4d0 0x55f6e716e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.749+0000 7fdd217fa640 0 client.27011 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:41:23.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.749+0000 7fdd46d69740 1 --2- 172.21.15.38:0/519321908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f6e6d55f60 0x55f6e716dcd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.749+0000 7fdd46d69740 1 -- 172.21.15.38:0/519321908 >> 172.21.15.38:0/519321908 conn(0x55f6e6d8b4e0 msgr2=0x55f6e713e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.749+0000 7fdd46d69740 1 -- 172.21.15.38:0/519321908 shutdown_connections 2024-05-21T15:41:23.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.749+0000 7fdd46d69740 1 -- 172.21.15.38:0/519321908 wait complete. 2024-05-21T15:41:23.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.750+0000 7fdd46d69740 20 client.27011 trim_cache size 0 max 16384 2024-05-21T15:41:23.752 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:41:23.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.750+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2723346866 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdd240a2b90 msgr2=0x7fdd240a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.750+0000 7fdd3fdea640 1 --2- 172.21.15.38:0/2723346866 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdd240a2b90 0x7fdd240a50a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55f6e71334f0 tx=0x7fdd340d4040 comp rx=0 tx=0).stop 2024-05-21T15:41:23.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.751+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2723346866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd38101980 msgr2=0x7fdd38197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:23.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.751+0000 7fdd3fdea640 1 --2- 172.21.15.38:0/2723346866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd38101980 0x7fdd38197070 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400e9f0 tx=0x7fdd3400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.751+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2723346866 shutdown_connections 2024-05-21T15:41:23.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.751+0000 7fdd3fdea640 1 --2- 172.21.15.38:0/2723346866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd381078c0 0x7fdd3819e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.751+0000 7fdd3fdea640 1 --2- 172.21.15.38:0/2723346866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd38102bc0 0x7fdd381975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.751+0000 7fdd3fdea640 1 --2- 172.21.15.38:0/2723346866 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdd240a2b90 0x7fdd240a50a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.751+0000 7fdd3fdea640 1 --2- 172.21.15.38:0/2723346866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd38101980 0x7fdd38197070 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:23.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.751+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2723346866 >> 172.21.15.38:0/2723346866 conn(0x7fdd380fcb70 msgr2=0x7fdd381038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:23.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.751+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2723346866 shutdown_connections 2024-05-21T15:41:23.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.751+0000 7fdd3fdea640 1 -- 172.21.15.38:0/2723346866 wait complete. 2024-05-21T15:41:23.763 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:41:23.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.908+0000 7fbdc43ea640 1 -- 172.21.15.38:0/3638238304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdbc108710 msgr2=0x7fbdbc10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:24.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.908+0000 7fbdc43ea640 1 --2- 172.21.15.38:0/3638238304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdbc108710 0x7fbdbc10ac20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009ca0 tx=0x7fbdb402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.908+0000 7fbdc43ea640 1 -- 172.21.15.38:0/3638238304 shutdown_connections 2024-05-21T15:41:24.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.908+0000 7fbdc43ea640 1 --2- 172.21.15.38:0/3638238304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdbc108710 0x7fbdbc10ac20 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.908+0000 7fbdc43ea640 1 --2- 172.21.15.38:0/3638238304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdbc105ba0 0x7fbdbc108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.908+0000 7fbdc43ea640 1 --2- 172.21.15.38:0/3638238304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdbc100dc0 0x7fbdbc1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.908+0000 7fbdc43ea640 1 -- 172.21.15.38:0/3638238304 >> 172.21.15.38:0/3638238304 conn(0x7fbdbc0fc2f0 msgr2=0x7fbdbc0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:24.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.908+0000 7fbdc43ea640 1 -- 172.21.15.38:0/3638238304 shutdown_connections 2024-05-21T15:41:24.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.908+0000 7fbdc43ea640 1 -- 172.21.15.38:0/3638238304 wait complete. 2024-05-21T15:41:24.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc43ea640 1 Processor -- start 2024-05-21T15:41:24.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc43ea640 1 -- start start 2024-05-21T15:41:24.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc43ea640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdbc100dc0 0x7fbdbc199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc43ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdbc105ba0 0x7fbdbc199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc43ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdbc108710 0x7fbdbc1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc43ea640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc10e740 con 0x7fbdbc105ba0 2024-05-21T15:41:24.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc43ea640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc10e5c0 con 0x7fbdbc108710 2024-05-21T15:41:24.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc43ea640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc10e8c0 con 0x7fbdbc100dc0 2024-05-21T15:41:24.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc1985640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdbc105ba0 0x7fbdbc199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc2186640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdbc100dc0 0x7fbdbc199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc1985640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdbc105ba0 0x7fbdbc199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41104/0 (socket says 172.21.15.38:41104) 2024-05-21T15:41:24.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc2186640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdbc100dc0 0x7fbdbc199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:59668/0 (socket says 172.21.15.38:59668) 2024-05-21T15:41:24.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc1985640 1 -- 172.21.15.38:0/1250160648 learned_addr learned my addr 172.21.15.38:0/1250160648 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:24.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc2987640 1 --2- 172.21.15.38:0/1250160648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdbc108710 0x7fbdbc1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc1985640 1 -- 172.21.15.38:0/1250160648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdbc100dc0 msgr2=0x7fbdbc199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:24.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc1985640 1 --2- 172.21.15.38:0/1250160648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdbc100dc0 0x7fbdbc199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc1985640 1 -- 172.21.15.38:0/1250160648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdbc108710 msgr2=0x7fbdbc1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:24.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc1985640 1 --2- 172.21.15.38:0/1250160648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdbc108710 0x7fbdbc1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc1985640 1 -- 172.21.15.38:0/1250160648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdbc1a0d60 con 0x7fbdbc105ba0 2024-05-21T15:41:24.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.909+0000 7fbdc2186640 1 --2- 172.21.15.38:0/1250160648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdbc100dc0 0x7fbdbc199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:24.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.910+0000 7fbdc1985640 1 --2- 172.21.15.38:0/1250160648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdbc105ba0 0x7fbdbc199760 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00e9f0 tx=0x7fbdac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:24.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.910+0000 7fbdb37fe640 1 -- 172.21.15.38:0/1250160648 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00cd80 con 0x7fbdbc105ba0 2024-05-21T15:41:24.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.910+0000 7fbdb37fe640 1 -- 172.21.15.38:0/1250160648 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdac010910 con 0x7fbdbc105ba0 2024-05-21T15:41:24.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.910+0000 7fbdb37fe640 1 -- 172.21.15.38:0/1250160648 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac018990 con 0x7fbdbc105ba0 2024-05-21T15:41:24.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.910+0000 7fbdc43ea640 1 -- 172.21.15.38:0/1250160648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdbc1a0ff0 con 0x7fbdbc105ba0 2024-05-21T15:41:24.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.910+0000 7fbdc43ea640 1 -- 172.21.15.38:0/1250160648 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdbc1a1490 con 0x7fbdbc105ba0 2024-05-21T15:41:24.074 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:23 smithi045 ceph-mon[36409]: pgmap v560: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 5.09k op/s 2024-05-21T15:41:24.074 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:23 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2672379013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:41:24.074 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:23 smithi053 ceph-mon[36496]: pgmap v560: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 5.09k op/s 2024-05-21T15:41:24.075 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:23 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2672379013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:41:24.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.913+0000 7fbdb37fe640 1 -- 172.21.15.38:0/1250160648 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbdac018b30 con 0x7fbdbc105ba0 2024-05-21T15:41:24.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.913+0000 7fbdcb369740 1 Processor -- start 2024-05-21T15:41:24.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.913+0000 7fbdcb369740 1 -- start start 2024-05-21T15:41:24.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.913+0000 7fbdcb369740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011a7390 0x564a0154af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdcb369740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 0x564a0154b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdcb369740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011866b0 0x564a01552390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdcb369740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564a014abc40 con 0x564a011a7390 2024-05-21T15:41:24.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdcb369740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564a01453580 con 0x564a011866b0 2024-05-21T15:41:24.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdcb369740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564a014a7c00 con 0x564a0117e4d0 2024-05-21T15:41:24.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdc2987640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011a7390 0x564a0154af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdb37fe640 1 --2- 172.21.15.38:0/1250160648 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbda80a2b90 0x7fbda80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdc2987640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011a7390 0x564a0154af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41116/0 (socket says 172.21.15.38:41116) 2024-05-21T15:41:24.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdc2987640 1 -- 172.21.15.38:0/2094156963 learned_addr learned my addr 172.21.15.38:0/2094156963 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:24.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdc2987640 1 --2- 172.21.15.38:0/1250160648 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbda80a2b90 0x7fbda80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdb37fe640 1 -- 172.21.15.38:0/1250160648 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbdac0cf3c0 con 0x7fbdbc105ba0 2024-05-21T15:41:24.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdc1985640 1 --2- 172.21.15.38:0/2094156963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011866b0 0x564a01552390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdc2186640 1 --2- 172.21.15.38:0/2094156963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 0x564a0154b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdc2987640 1 -- 172.21.15.38:0/2094156963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 msgr2=0x564a0154b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:24.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdc2987640 1 --2- 172.21.15.38:0/2094156963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 0x564a0154b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdc2987640 1 -- 172.21.15.38:0/2094156963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011866b0 msgr2=0x564a01552390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:24.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdc2987640 1 --2- 172.21.15.38:0/2094156963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011866b0 0x564a01552390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.914+0000 7fbdc2987640 1 -- 172.21.15.38:0/2094156963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a011bf3f0 con 0x564a011a7390 2024-05-21T15:41:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.915+0000 7fbdc2987640 1 --2- 172.21.15.38:0/2094156963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011a7390 0x564a0154af10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4035000 tx=0x7fbdb40316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a4ddfc4b4325f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.915+0000 7fbdb0bfe640 1 -- 172.21.15.38:0/2094156963 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4004040 con 0x564a011a7390 2024-05-21T15:41:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.915+0000 7fbdb0bfe640 1 -- 172.21.15.38:0/2094156963 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb4036070 con 0x564a011a7390 2024-05-21T15:41:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.915+0000 7fbdb0bfe640 1 -- 172.21.15.38:0/2094156963 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb402faa0 con 0x564a011a7390 2024-05-21T15:41:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.916+0000 7fbdc2987640 1 --2- 172.21.15.38:0/1250160648 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbda80a2b90 0x7fbda80a50a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbdb404e000 tx=0x7fbdb404c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.916+0000 7fbdcb369740 1 -- 172.21.15.38:0/2094156963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011a7390 msgr2=0x564a0154af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.916+0000 7fbdcb369740 1 --2- 172.21.15.38:0/2094156963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011a7390 0x564a0154af10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4035000 tx=0x7fbdb40316f0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.916+0000 7fbdcb369740 1 -- 172.21.15.38:0/2094156963 shutdown_connections 2024-05-21T15:41:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.916+0000 7fbdcb369740 1 --2- 172.21.15.38:0/2094156963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011866b0 0x564a01552390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.916+0000 7fbdcb369740 1 --2- 172.21.15.38:0/2094156963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 0x564a0154b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.916+0000 7fbdcb369740 1 --2- 172.21.15.38:0/2094156963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011a7390 0x564a0154af10 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.916+0000 7fbdcb369740 1 -- 172.21.15.38:0/2094156963 >> 172.21.15.38:0/2094156963 conn(0x564a0119b4e0 msgr2=0x564a01542490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.916+0000 7fbdcb369740 1 -- 172.21.15.38:0/2094156963 shutdown_connections 2024-05-21T15:41:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.916+0000 7fbdcb369740 1 -- 172.21.15.38:0/2094156963 wait complete. 2024-05-21T15:41:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.916+0000 7fbdcb369740 1 -- start start 2024-05-21T15:41:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.916+0000 7fbdcb369740 1 Processor -- start 2024-05-21T15:41:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.917+0000 7fbdcb369740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 0x564a01554c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.917+0000 7fbdcb369740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011866b0 0x564a01555150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.917+0000 7fbdcb369740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011a7390 0x564a0155c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.917+0000 7fbdcb369740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564a01552c60 con 0x564a011a7390 2024-05-21T15:41:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.917+0000 7fbdcb369740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564a01466c40 con 0x564a011866b0 2024-05-21T15:41:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.917+0000 7fbdcb369740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564a01552de0 con 0x564a0117e4d0 2024-05-21T15:41:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.917+0000 7fbdcb369740 10 client.0 _ll_register_callbacks cb 0x564a011bebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.917+0000 7fbdc2987640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011a7390 0x564a0155c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.917+0000 7fbdc2987640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011a7390 0x564a0155c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41122/0 (socket says 172.21.15.38:41122) 2024-05-21T15:41:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.917+0000 7fbdc2987640 1 -- 172.21.15.38:0/2573832294 learned_addr learned my addr 172.21.15.38:0/2573832294 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:24.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.917+0000 7fbdc1985640 1 --2- 172.21.15.38:0/2573832294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011866b0 0x564a01555150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.917+0000 7fbdc2186640 1 --2- 172.21.15.38:0/2573832294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 0x564a01554c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.917+0000 7fbdc1985640 1 -- 172.21.15.38:0/2573832294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 msgr2=0x564a01554c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.917+0000 7fbdc1985640 1 --2- 172.21.15.38:0/2573832294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 0x564a01554c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.918+0000 7fbdc1985640 1 -- 172.21.15.38:0/2573832294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011a7390 msgr2=0x564a0155c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.918+0000 7fbdc1985640 1 --2- 172.21.15.38:0/2573832294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011a7390 0x564a0155c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.918+0000 7fbdc1985640 1 -- 172.21.15.38:0/2573832294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a012897d0 con 0x564a011866b0 2024-05-21T15:41:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.918+0000 7fbdc2987640 1 --2- 172.21.15.38:0/2573832294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011a7390 0x564a0155c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.918+0000 7fbdc1985640 1 --2- 172.21.15.38:0/2573832294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011866b0 0x564a01555150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbdac09bc10 tx=0x7fbdac09bdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2d252c8b12b3ba9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.918+0000 7fbda5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:41:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.918+0000 7fbda5ffb640 1 -- 172.21.15.38:0/2573832294 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac09b2d0 con 0x564a011866b0 2024-05-21T15:41:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.918+0000 7fbda5ffb640 1 -- 172.21.15.38:0/2573832294 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbdac0863d0 con 0x564a011866b0 2024-05-21T15:41:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.918+0000 7fbdcb369740 1 -- 172.21.15.38:0/2573832294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdb4036070 con 0x564a011866b0 2024-05-21T15:41:24.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.918+0000 7fbda5ffb640 1 -- 172.21.15.38:0/2573832294 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbdac0cd040 con 0x564a011866b0 2024-05-21T15:41:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.918+0000 7fbdcb369740 10 client.27023 fetch_fsmap learned FSMap version 34 2024-05-21T15:41:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.918+0000 7fbdcb369740 1 -- 172.21.15.38:0/2573832294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564a0155f430 con 0x564a011866b0 2024-05-21T15:41:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.919+0000 7fbda5ffb640 1 -- 172.21.15.38:0/2573832294 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb4036070 con 0x564a011866b0 2024-05-21T15:41:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.919+0000 7fbda5ffb640 1 -- 172.21.15.38:0/2573832294 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fbdac09ab40 con 0x564a011866b0 2024-05-21T15:41:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.919+0000 7fbdcb369740 10 client.27023 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T15:41:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.919+0000 7fbdcb369740 10 client.27023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.919+0000 7fbdcb369740 20 client.27023 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.919+0000 7fbdcb369740 1 --2- 172.21.15.38:0/2573832294 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564a01165f60 0x564a01562b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.919+0000 7fbdcb369740 4 client.27023 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.919+0000 7fbdcb369740 1 -- 172.21.15.38:0/2573832294 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a014b9370 con 0x564a01165f60 2024-05-21T15:41:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.919+0000 7fbdc2186640 1 --2- 172.21.15.38:0/2573832294 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564a01165f60 0x564a01562b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.919+0000 7fbda5ffb640 10 client.27023 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.920+0000 7fbdc2186640 1 --2- 172.21.15.38:0/2573832294 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564a01165f60 0x564a01562b80 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc5f1bad9eabdf4e server_cookie=18b55d72f33f373d in_seq=0 out_seq=0 2024-05-21T15:41:24.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.920+0000 7fbda5ffb640 1 -- 172.21.15.38:0/2573832294 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564a014b9370 con 0x564a01165f60 2024-05-21T15:41:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.920+0000 7fbda5ffb640 10 client.27023 handle_command_reply: tid=0 2024-05-21T15:41:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.921+0000 7fbdcb369740 1 client.27023 shutdown 2024-05-21T15:41:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.921+0000 7fbdcb369740 1 -- 172.21.15.38:0/2573832294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011866b0 msgr2=0x564a01555150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.921+0000 7fbdcb369740 1 --2- 172.21.15.38:0/2573832294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011866b0 0x564a01555150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbdac09bc10 tx=0x7fbdac09bdc0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.921+0000 7fbdcb369740 1 -- 172.21.15.38:0/2573832294 shutdown_connections 2024-05-21T15:41:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.921+0000 7fbdcb369740 1 --2- 172.21.15.38:0/2573832294 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564a01165f60 0x564a01562b80 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.921+0000 7fbdcb369740 1 --2- 172.21.15.38:0/2573832294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011a7390 0x564a0155c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.921+0000 7fbdcb369740 1 --2- 172.21.15.38:0/2573832294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011866b0 0x564a01555150 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.921+0000 7fbdcb369740 1 --2- 172.21.15.38:0/2573832294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 0x564a01554c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.921+0000 7fbdcb369740 1 -- 172.21.15.38:0/2573832294 >> 172.21.15.38:0/2573832294 conn(0x564a0119b4e0 msgr2=0x564a01541570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.921+0000 7fbda5ffb640 0 client.27023 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.921+0000 7fbdcb369740 1 -- 172.21.15.38:0/2573832294 shutdown_connections 2024-05-21T15:41:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.921+0000 7fbdcb369740 1 -- 172.21.15.38:0/2573832294 wait complete. 2024-05-21T15:41:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.921+0000 7fbdcb369740 20 client.27023 trim_cache size 0 max 16384 2024-05-21T15:41:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.943+0000 7fbdcb369740 1 Processor -- start 2024-05-21T15:41:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.943+0000 7fbdcb369740 1 -- start start 2024-05-21T15:41:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.943+0000 7fbdcb369740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0117e4d0 0x564a015538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.943+0000 7fbdcb369740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011866b0 0x564a01541ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.943+0000 7fbdcb369740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a011a7390 0x564a01542410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.943+0000 7fbdcb369740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc1a8150 con 0x564a011866b0 2024-05-21T15:41:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.943+0000 7fbdcb369740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc1a9cc0 con 0x564a0117e4d0 2024-05-21T15:41:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.943+0000 7fbdcb369740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc1a8820 con 0x564a011a7390 2024-05-21T15:41:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.944+0000 7fbdc2987640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a011a7390 0x564a01542410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.944+0000 7fbdc2987640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a011a7390 0x564a01542410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:59708/0 (socket says 172.21.15.38:59708) 2024-05-21T15:41:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.944+0000 7fbdc2987640 1 -- 172.21.15.38:0/3161510273 learned_addr learned my addr 172.21.15.38:0/3161510273 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.944+0000 7fbdc1985640 1 --2- 172.21.15.38:0/3161510273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011866b0 0x564a01541ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.944+0000 7fbdc2186640 1 --2- 172.21.15.38:0/3161510273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0117e4d0 0x564a015538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.945+0000 7fbdc2987640 1 -- 172.21.15.38:0/3161510273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0117e4d0 msgr2=0x564a015538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.945+0000 7fbdc2987640 1 --2- 172.21.15.38:0/3161510273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0117e4d0 0x564a015538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.945+0000 7fbdc2987640 1 -- 172.21.15.38:0/3161510273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011866b0 msgr2=0x564a01541ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.945+0000 7fbdc2987640 1 --2- 172.21.15.38:0/3161510273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011866b0 0x564a01541ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.945+0000 7fbdc2987640 1 -- 172.21.15.38:0/3161510273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a01166190 con 0x564a011a7390 2024-05-21T15:41:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.945+0000 7fbdc2987640 1 --2- 172.21.15.38:0/3161510273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a011a7390 0x564a01542410 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbdb402f540 tx=0x7fbdb40319d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2869c41bcded6f33 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.946+0000 7fbda7fff640 1 -- 172.21.15.38:0/3161510273 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb402f680 con 0x564a011a7390 2024-05-21T15:41:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.946+0000 7fbda7fff640 1 -- 172.21.15.38:0/3161510273 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb4036040 con 0x564a011a7390 2024-05-21T15:41:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.946+0000 7fbda7fff640 1 -- 172.21.15.38:0/3161510273 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4030800 con 0x564a011a7390 2024-05-21T15:41:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.947+0000 7fbdcb369740 1 -- 172.21.15.38:0/3161510273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a011a7390 msgr2=0x564a01542410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.947+0000 7fbdcb369740 1 --2- 172.21.15.38:0/3161510273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a011a7390 0x564a01542410 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbdb402f540 tx=0x7fbdb40319d0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.947+0000 7fbdcb369740 1 -- 172.21.15.38:0/3161510273 shutdown_connections 2024-05-21T15:41:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.947+0000 7fbdcb369740 1 --2- 172.21.15.38:0/3161510273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a011a7390 0x564a01542410 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.947+0000 7fbdcb369740 1 --2- 172.21.15.38:0/3161510273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011866b0 0x564a01541ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.947+0000 7fbdcb369740 1 --2- 172.21.15.38:0/3161510273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0117e4d0 0x564a015538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.947+0000 7fbdcb369740 1 -- 172.21.15.38:0/3161510273 >> 172.21.15.38:0/3161510273 conn(0x564a0119b4e0 msgr2=0x564a015531e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.948+0000 7fbdcb369740 1 -- 172.21.15.38:0/3161510273 shutdown_connections 2024-05-21T15:41:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.948+0000 7fbdcb369740 1 -- 172.21.15.38:0/3161510273 wait complete. 2024-05-21T15:41:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.948+0000 7fbdcb369740 1 -- start start 2024-05-21T15:41:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.948+0000 7fbdcb369740 1 Processor -- start 2024-05-21T15:41:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.949+0000 7fbdcb369740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 0x564a01549c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.949+0000 7fbdcb369740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011866b0 0x564a0154a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.950+0000 7fbdcb369740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011a7390 0x564a015cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.950+0000 7fbdcb369740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc1a95c0 con 0x564a011866b0 2024-05-21T15:41:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.950+0000 7fbdcb369740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc1a8ef0 con 0x564a011a7390 2024-05-21T15:41:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.950+0000 7fbdcb369740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc1a6130 con 0x564a0117e4d0 2024-05-21T15:41:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.950+0000 7fbdcb369740 10 client.0 _ll_register_callbacks cb 0x7fbdac009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.950+0000 7fbdc1985640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011866b0 0x564a0154a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.950+0000 7fbdc1985640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011866b0 0x564a0154a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41142/0 (socket says 172.21.15.38:41142) 2024-05-21T15:41:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.950+0000 7fbdc1985640 1 -- 172.21.15.38:0/3072299992 learned_addr learned my addr 172.21.15.38:0/3072299992 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.951+0000 7fbdc2186640 1 --2- 172.21.15.38:0/3072299992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 0x564a01549c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.951+0000 7fbdc2987640 1 --2- 172.21.15.38:0/3072299992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011a7390 0x564a015cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.951+0000 7fbdc1985640 1 -- 172.21.15.38:0/3072299992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 msgr2=0x564a01549c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.951+0000 7fbdc1985640 1 --2- 172.21.15.38:0/3072299992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 0x564a01549c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.951+0000 7fbdc1985640 1 -- 172.21.15.38:0/3072299992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011a7390 msgr2=0x564a015cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.951+0000 7fbdc1985640 1 --2- 172.21.15.38:0/3072299992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011a7390 0x564a015cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.951+0000 7fbdc1985640 1 -- 172.21.15.38:0/3072299992 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a0155c430 con 0x564a011866b0 2024-05-21T15:41:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.951+0000 7fbdc2186640 1 --2- 172.21.15.38:0/3072299992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 0x564a01549c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:41:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.951+0000 7fbdc1985640 1 --2- 172.21.15.38:0/3072299992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011866b0 0x564a0154a170 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbdac0097b0 tx=0x7fbdac09b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f75e0a1aa312ae server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.951+0000 7fbda5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:24.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.952+0000 7fbda5ffb640 1 -- 172.21.15.38:0/3072299992 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac0a1110 con 0x564a011866b0 2024-05-21T15:41:24.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.952+0000 7fbdcb369740 1 -- 172.21.15.38:0/3072299992 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdb4036040 con 0x564a011866b0 2024-05-21T15:41:24.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.952+0000 7fbda5ffb640 1 -- 172.21.15.38:0/3072299992 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fbdac0863d0 con 0x564a011866b0 2024-05-21T15:41:24.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.952+0000 7fbda5ffb640 1 -- 172.21.15.38:0/3072299992 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbdac08b050 con 0x564a011866b0 2024-05-21T15:41:24.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.952+0000 7fbdcb369740 10 client.17406 fetch_fsmap learned FSMap version 34 2024-05-21T15:41:24.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.953+0000 7fbdcb369740 1 -- 172.21.15.38:0/3072299992 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564a015cebe0 con 0x564a011866b0 2024-05-21T15:41:24.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.953+0000 7fbda5ffb640 1 -- 172.21.15.38:0/3072299992 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb4036040 con 0x564a011866b0 2024-05-21T15:41:24.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.953+0000 7fbda5ffb640 1 -- 172.21.15.38:0/3072299992 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 34) ==== 8358+0+0 (secure 0 0 0) 0x7fbdac0a4a90 con 0x564a011866b0 2024-05-21T15:41:24.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.953+0000 7fbdcb369740 10 client.17406 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T15:41:24.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.953+0000 7fbdcb369740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:24.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.953+0000 7fbdcb369740 20 client.17406 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:24.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.953+0000 7fbdcb369740 1 --2- 172.21.15.38:0/3072299992 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564a0113b3a0 0x564a015d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:24.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.953+0000 7fbdcb369740 4 client.17406 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:41:24.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.953+0000 7fbdcb369740 1 -- 172.21.15.38:0/3072299992 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a014b36c0 con 0x564a0113b3a0 2024-05-21T15:41:24.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.954+0000 7fbdc2186640 1 --2- 172.21.15.38:0/3072299992 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564a0113b3a0 0x564a015d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:24.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.956+0000 7fbdc2186640 1 --2- 172.21.15.38:0/3072299992 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564a0113b3a0 0x564a015d1630 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af3d0eaf662cce2c server_cookie=502c77a3ee14da3e in_seq=0 out_seq=0 2024-05-21T15:41:24.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.956+0000 7fbda5ffb640 10 client.17406 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:24.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.956+0000 7fbda5ffb640 1 -- 172.21.15.38:0/3072299992 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a014b36c0 con 0x564a0113b3a0 2024-05-21T15:41:24.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.956+0000 7fbda5ffb640 10 client.17406 handle_command_reply: tid=0 2024-05-21T15:41:24.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.956+0000 7fbdcb369740 1 client.17406 shutdown 2024-05-21T15:41:24.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.957+0000 7fbdcb369740 1 -- 172.21.15.38:0/3072299992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011866b0 msgr2=0x564a0154a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:24.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.957+0000 7fbdcb369740 1 --2- 172.21.15.38:0/3072299992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011866b0 0x564a0154a170 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbdac0097b0 tx=0x7fbdac09b6b0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.957+0000 7fbdcb369740 1 -- 172.21.15.38:0/3072299992 shutdown_connections 2024-05-21T15:41:24.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.957+0000 7fbdcb369740 1 --2- 172.21.15.38:0/3072299992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a011a7390 0x564a015cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.957+0000 7fbdcb369740 1 --2- 172.21.15.38:0/3072299992 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x564a0113b3a0 0x564a015d1630 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.957+0000 7fbdcb369740 1 --2- 172.21.15.38:0/3072299992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a011866b0 0x564a0154a170 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.957+0000 7fbdcb369740 1 --2- 172.21.15.38:0/3072299992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a0117e4d0 0x564a01549c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.957+0000 7fbdcb369740 1 -- 172.21.15.38:0/3072299992 >> 172.21.15.38:0/3072299992 conn(0x564a0119b4e0 msgr2=0x564a01541ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:24.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.957+0000 7fbda5ffb640 0 client.17406 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:24.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.957+0000 7fbdcb369740 1 -- 172.21.15.38:0/3072299992 shutdown_connections 2024-05-21T15:41:24.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.958+0000 7fbdcb369740 1 -- 172.21.15.38:0/3072299992 wait complete. 2024-05-21T15:41:24.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.958+0000 7fbdcb369740 20 client.17406 trim_cache size 0 max 16384 2024-05-21T15:41:24.106 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:41:24.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.959+0000 7fbdc43ea640 1 -- 172.21.15.38:0/1250160648 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbda80a2b90 msgr2=0x7fbda80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:24.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.959+0000 7fbdc43ea640 1 --2- 172.21.15.38:0/1250160648 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbda80a2b90 0x7fbda80a50a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbdb404e000 tx=0x7fbdb404c000 comp rx=0 tx=0).stop 2024-05-21T15:41:24.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.959+0000 7fbdc43ea640 1 -- 172.21.15.38:0/1250160648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdbc105ba0 msgr2=0x7fbdbc199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:24.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.959+0000 7fbdc43ea640 1 --2- 172.21.15.38:0/1250160648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdbc105ba0 0x7fbdbc199760 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00e9f0 tx=0x7fbdac00eeb0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.960+0000 7fbdc43ea640 1 -- 172.21.15.38:0/1250160648 shutdown_connections 2024-05-21T15:41:24.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.960+0000 7fbdc43ea640 1 --2- 172.21.15.38:0/1250160648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdbc108710 0x7fbdbc1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.960+0000 7fbdc43ea640 1 --2- 172.21.15.38:0/1250160648 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fbda80a2b90 0x7fbda80a50a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.960+0000 7fbdc43ea640 1 --2- 172.21.15.38:0/1250160648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdbc105ba0 0x7fbdbc199760 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.960+0000 7fbdc43ea640 1 --2- 172.21.15.38:0/1250160648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdbc100dc0 0x7fbdbc199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:24.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.960+0000 7fbdc43ea640 1 -- 172.21.15.38:0/1250160648 >> 172.21.15.38:0/1250160648 conn(0x7fbdbc0fc2f0 msgr2=0x7fbdbc106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:24.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.960+0000 7fbdc43ea640 1 -- 172.21.15.38:0/1250160648 shutdown_connections 2024-05-21T15:41:24.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:23.961+0000 7fbdc43ea640 1 -- 172.21.15.38:0/1250160648 wait complete. 2024-05-21T15:41:24.109 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:23 smithi038 ceph-mon[31995]: pgmap v560: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 5.09k op/s 2024-05-21T15:41:24.109 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:23 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2672379013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:41:25.109 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:41:25.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:41:25.110 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:41:25.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.330+0000 7f956b45d640 1 -- 172.21.15.38:0/2689421982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564107900 msgr2=0x7f9564109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.330+0000 7f956b45d640 1 --2- 172.21.15.38:0/2689421982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564107900 0x7f9564109d20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f954c009ca0 tx=0x7f954c02c7b0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.332+0000 7f956b45d640 1 -- 172.21.15.38:0/2689421982 shutdown_connections 2024-05-21T15:41:25.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.332+0000 7f956b45d640 1 --2- 172.21.15.38:0/2689421982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564107900 0x7f9564109d20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.332+0000 7f956b45d640 1 --2- 172.21.15.38:0/2689421982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9564102c00 0x7f9564103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.332+0000 7f956b45d640 1 --2- 172.21.15.38:0/2689421982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95641019c0 0x7f95641026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.332+0000 7f956b45d640 1 -- 172.21.15.38:0/2689421982 >> 172.21.15.38:0/2689421982 conn(0x7f95640fcbb0 msgr2=0x7f95640ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:25.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.336+0000 7f956b45d640 1 -- 172.21.15.38:0/2689421982 shutdown_connections 2024-05-21T15:41:25.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.337+0000 7f956b45d640 1 -- 172.21.15.38:0/2689421982 wait complete. 2024-05-21T15:41:25.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.338+0000 7f956b45d640 1 Processor -- start 2024-05-21T15:41:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.339+0000 7f956b45d640 1 -- start start 2024-05-21T15:41:25.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.340+0000 7f956b45d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95641019c0 0x7f9564197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.340+0000 7f956b45d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564102c00 0x7f9564197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.340+0000 7f956b45d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9564107900 0x7f956419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.340+0000 7f956b45d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f956410c610 con 0x7f95641019c0 2024-05-21T15:41:25.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.340+0000 7f956b45d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f956410c490 con 0x7f9564107900 2024-05-21T15:41:25.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.340+0000 7f956b45d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f956410c790 con 0x7f9564102c00 2024-05-21T15:41:25.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.341+0000 7f95691f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95641019c0 0x7f9564197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.341+0000 7f95691f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95641019c0 0x7f9564197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35862/0 (socket says 172.21.15.38:35862) 2024-05-21T15:41:25.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.341+0000 7f95691f9640 1 -- 172.21.15.38:0/4162419450 learned_addr learned my addr 172.21.15.38:0/4162419450 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:25.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.342+0000 7f95691f9640 1 -- 172.21.15.38:0/4162419450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564102c00 msgr2=0x7f9564197780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T15:41:25.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.342+0000 7f95691f9640 1 --2- 172.21.15.38:0/4162419450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564102c00 0x7f9564197780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.342+0000 7f95691f9640 1 -- 172.21.15.38:0/4162419450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9564107900 msgr2=0x7f956419e710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T15:41:25.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.342+0000 7f95691f9640 1 --2- 172.21.15.38:0/4162419450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9564107900 0x7f956419e710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.342+0000 7f95691f9640 1 -- 172.21.15.38:0/4162419450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956419ed80 con 0x7f95641019c0 2024-05-21T15:41:25.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.343+0000 7f95691f9640 1 --2- 172.21.15.38:0/4162419450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95641019c0 0x7f9564197240 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f955400fed0 tx=0x7f9554009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.343+0000 7f95627fc640 1 -- 172.21.15.38:0/4162419450 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554018a50 con 0x7f95641019c0 2024-05-21T15:41:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.343+0000 7f95627fc640 1 -- 172.21.15.38:0/4162419450 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f955401d070 con 0x7f95641019c0 2024-05-21T15:41:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.343+0000 7f95627fc640 1 -- 172.21.15.38:0/4162419450 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554018a50 con 0x7f95641019c0 2024-05-21T15:41:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.345+0000 7f956b45d640 1 -- 172.21.15.38:0/4162419450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956419f070 con 0x7f95641019c0 2024-05-21T15:41:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.345+0000 7f956b45d640 1 -- 172.21.15.38:0/4162419450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956419f5c0 con 0x7f95641019c0 2024-05-21T15:41:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.350+0000 7f95627fc640 1 -- 172.21.15.38:0/4162419450 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f955400a6b0 con 0x7f95641019c0 2024-05-21T15:41:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.353+0000 7f95627fc640 1 --2- 172.21.15.38:0/4162419450 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f95500a2fd0 0x7f95500a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.353+0000 7f95723d8740 1 Processor -- start 2024-05-21T15:41:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.354+0000 7f95627fc640 1 -- 172.21.15.38:0/4162419450 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9554018850 con 0x7f95641019c0 2024-05-21T15:41:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.356+0000 7f95723d8740 1 -- start start 2024-05-21T15:41:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.356+0000 7f95689f8640 1 --2- 172.21.15.38:0/4162419450 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f95500a2fd0 0x7f95500a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.357+0000 7f95723d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2ee410 0x56530a68e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.357+0000 7f95723d8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2c5550 0x56530a68ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.357+0000 7f95723d8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2cd730 0x56530a695920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.357+0000 7f95723d8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56530a5f2990 con 0x56530a2ee410 2024-05-21T15:41:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.357+0000 7f95723d8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56530a59a5b0 con 0x56530a2c5550 2024-05-21T15:41:25.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.357+0000 7f95723d8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56530a5ee950 con 0x56530a2cd730 2024-05-21T15:41:25.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.357+0000 7f95699fa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2cd730 0x56530a695920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.357+0000 7f95699fa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2cd730 0x56530a695920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:49880/0 (socket says 172.21.15.38:49880) 2024-05-21T15:41:25.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.357+0000 7f95699fa640 1 -- 172.21.15.38:0/2917014328 learned_addr learned my addr 172.21.15.38:0/2917014328 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:25.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.357+0000 7f95691f9640 1 --2- 172.21.15.38:0/2917014328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2ee410 0x56530a68e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.357+0000 7f95699fa640 1 -- 172.21.15.38:0/2917014328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2c5550 msgr2=0x56530a68ea20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:41:25.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.357+0000 7f95699fa640 1 --2- 172.21.15.38:0/2917014328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2c5550 0x56530a68ea20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.358+0000 7f95699fa640 1 -- 172.21.15.38:0/2917014328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2ee410 msgr2=0x56530a68e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:25.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.358+0000 7f95699fa640 1 --2- 172.21.15.38:0/2917014328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2ee410 0x56530a68e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.358+0000 7f95699fa640 1 -- 172.21.15.38:0/2917014328 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56530a3ce7e0 con 0x56530a2cd730 2024-05-21T15:41:25.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.359+0000 7f95689f8640 1 --2- 172.21.15.38:0/4162419450 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f95500a2fd0 0x7f95500a54e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x56530a68f880 tx=0x7f955c006c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:25.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.359+0000 7f95691f9640 1 --2- 172.21.15.38:0/2917014328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2ee410 0x56530a68e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:41:25.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.359+0000 7f95699fa640 1 --2- 172.21.15.38:0/2917014328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2cd730 0x56530a695920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f954c02d010 tx=0x7f954c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5458594c973d52ba server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:25.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.362+0000 7f955b7fe640 1 -- 172.21.15.38:0/2917014328 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c003b70 con 0x56530a2cd730 2024-05-21T15:41:25.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.362+0000 7f955b7fe640 1 -- 172.21.15.38:0/2917014328 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f954c0339a0 con 0x56530a2cd730 2024-05-21T15:41:25.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.362+0000 7f955b7fe640 1 -- 172.21.15.38:0/2917014328 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c0047e0 con 0x56530a2cd730 2024-05-21T15:41:25.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.364+0000 7f95723d8740 1 -- 172.21.15.38:0/2917014328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2cd730 msgr2=0x56530a695920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.364+0000 7f95723d8740 1 --2- 172.21.15.38:0/2917014328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2cd730 0x56530a695920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f954c02d010 tx=0x7f954c009920 comp rx=0 tx=0).stop 2024-05-21T15:41:25.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.364+0000 7f95723d8740 1 -- 172.21.15.38:0/2917014328 shutdown_connections 2024-05-21T15:41:25.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.364+0000 7f95723d8740 1 --2- 172.21.15.38:0/2917014328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2cd730 0x56530a695920 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.364+0000 7f95723d8740 1 --2- 172.21.15.38:0/2917014328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2c5550 0x56530a68ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.364+0000 7f95723d8740 1 --2- 172.21.15.38:0/2917014328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2ee410 0x56530a68e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.364+0000 7f95723d8740 1 -- 172.21.15.38:0/2917014328 >> 172.21.15.38:0/2917014328 conn(0x56530a2e2560 msgr2=0x56530a689e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:25.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.365+0000 7f95723d8740 1 -- 172.21.15.38:0/2917014328 shutdown_connections 2024-05-21T15:41:25.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.365+0000 7f95723d8740 1 -- 172.21.15.38:0/2917014328 wait complete. 2024-05-21T15:41:25.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.365+0000 7f95723d8740 1 -- start start 2024-05-21T15:41:25.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.366+0000 7f95723d8740 1 Processor -- start 2024-05-21T15:41:25.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.366+0000 7f95723d8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2c5550 0x56530a6a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.366+0000 7f95723d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2cd730 0x56530a6a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.366+0000 7f95723d8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2ee410 0x56530a6b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.366+0000 7f95723d8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56530a6a7320 con 0x56530a2cd730 2024-05-21T15:41:25.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.366+0000 7f95723d8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56530a5adb40 con 0x56530a2ee410 2024-05-21T15:41:25.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.366+0000 7f95723d8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56530a6a74a0 con 0x56530a2c5550 2024-05-21T15:41:25.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.366+0000 7f95723d8740 10 client.0 _ll_register_callbacks cb 0x56530a5a20f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:25.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.367+0000 7f95691f9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2c5550 0x56530a6a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.367+0000 7f95689f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2cd730 0x56530a6a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.367+0000 7f95689f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2cd730 0x56530a6a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35882/0 (socket says 172.21.15.38:35882) 2024-05-21T15:41:25.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.367+0000 7f95689f8640 1 -- 172.21.15.38:0/3912475778 learned_addr learned my addr 172.21.15.38:0/3912475778 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:25.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.367+0000 7f95691f9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2c5550 0x56530a6a93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:49886/0 (socket says 172.21.15.38:49886) 2024-05-21T15:41:25.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.367+0000 7f95699fa640 1 --2- 172.21.15.38:0/3912475778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2ee410 0x56530a6b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.367+0000 7f95691f9640 1 -- 172.21.15.38:0/3912475778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2ee410 msgr2=0x56530a6b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:25.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.367+0000 7f95691f9640 1 --2- 172.21.15.38:0/3912475778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2ee410 0x56530a6b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.367+0000 7f95691f9640 1 -- 172.21.15.38:0/3912475778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2cd730 msgr2=0x56530a6a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.367+0000 7f95691f9640 1 --2- 172.21.15.38:0/3912475778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2cd730 0x56530a6a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.367+0000 7f95691f9640 1 -- 172.21.15.38:0/3912475778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56530a42ef70 con 0x56530a2c5550 2024-05-21T15:41:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.368+0000 7f95691f9640 1 --2- 172.21.15.38:0/3912475778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2c5550 0x56530a6a93a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f956419f790 tx=0x7f95540a1a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6a43a3e355cc56 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.368+0000 7f9558ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:41:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.370+0000 7f9558ff9640 1 -- 172.21.15.38:0/3912475778 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955409fc50 con 0x56530a2c5550 2024-05-21T15:41:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.370+0000 7f9558ff9640 1 -- 172.21.15.38:0/3912475778 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f955409dfc0 con 0x56530a2c5550 2024-05-21T15:41:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.370+0000 7f95723d8740 1 -- 172.21.15.38:0/3912475778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f954c0339a0 con 0x56530a2c5550 2024-05-21T15:41:25.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.371+0000 7f9558ff9640 1 -- 172.21.15.38:0/3912475778 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f955400a9a0 con 0x56530a2c5550 2024-05-21T15:41:25.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.371+0000 7f95723d8740 10 client.27109 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:25.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.371+0000 7f95723d8740 1 -- 172.21.15.38:0/3912475778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56530a2c6170 con 0x56530a2c5550 2024-05-21T15:41:25.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.373+0000 7f9558ff9640 1 -- 172.21.15.38:0/3912475778 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f954c0339a0 con 0x56530a2c5550 2024-05-21T15:41:25.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.373+0000 7f9558ff9640 1 -- 172.21.15.38:0/3912475778 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f95540bab50 con 0x56530a2c5550 2024-05-21T15:41:25.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.374+0000 7f95723d8740 10 client.27109 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:25.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.374+0000 7f95723d8740 10 client.27109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:25.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.374+0000 7f95723d8740 20 client.27109 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:25.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.374+0000 7f95723d8740 1 --2- 172.21.15.38:0/3912475778 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56530a2ad1f0 0x56530a6b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.374+0000 7f95723d8740 4 client.27109 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:25.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.374+0000 7f95723d8740 1 -- 172.21.15.38:0/3912475778 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56530a6000c0 con 0x56530a2ad1f0 2024-05-21T15:41:25.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.376+0000 7f95689f8640 1 --2- 172.21.15.38:0/3912475778 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56530a2ad1f0 0x56530a6b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.384+0000 7f95689f8640 1 --2- 172.21.15.38:0/3912475778 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56530a2ad1f0 0x56530a6b6d40 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abae563656858c8c server_cookie=69992fcfe9c0f6b0 in_seq=0 out_seq=0 2024-05-21T15:41:25.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.386+0000 7f9558ff9640 10 client.27109 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:25.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.393+0000 7f9558ff9640 1 -- 172.21.15.38:0/3912475778 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56530a6000c0 con 0x56530a2ad1f0 2024-05-21T15:41:25.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.393+0000 7f9558ff9640 10 client.27109 handle_command_reply: tid=0 2024-05-21T15:41:25.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.393+0000 7f95723d8740 1 client.27109 shutdown 2024-05-21T15:41:25.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.395+0000 7f95723d8740 1 -- 172.21.15.38:0/3912475778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2c5550 msgr2=0x56530a6a93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.395+0000 7f95723d8740 1 --2- 172.21.15.38:0/3912475778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2c5550 0x56530a6a93a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f956419f790 tx=0x7f95540a1a10 comp rx=0 tx=0).stop 2024-05-21T15:41:25.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.398+0000 7f95723d8740 1 -- 172.21.15.38:0/3912475778 shutdown_connections 2024-05-21T15:41:25.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.398+0000 7f95723d8740 1 --2- 172.21.15.38:0/3912475778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2ee410 0x56530a6b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.398+0000 7f95723d8740 1 --2- 172.21.15.38:0/3912475778 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56530a2ad1f0 0x56530a6b6d40 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.398+0000 7f95723d8740 1 --2- 172.21.15.38:0/3912475778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2cd730 0x56530a6a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.398+0000 7f95723d8740 1 --2- 172.21.15.38:0/3912475778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2c5550 0x56530a6a93a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.398+0000 7f95723d8740 1 -- 172.21.15.38:0/3912475778 >> 172.21.15.38:0/3912475778 conn(0x56530a2e2560 msgr2=0x56530a688e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:25.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.400+0000 7f9558ff9640 0 client.27109 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:25.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.404+0000 7f95723d8740 1 -- 172.21.15.38:0/3912475778 shutdown_connections 2024-05-21T15:41:25.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.406+0000 7f95723d8740 1 -- 172.21.15.38:0/3912475778 wait complete. 2024-05-21T15:41:25.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.407+0000 7f95723d8740 20 client.27109 trim_cache size 0 max 16384 2024-05-21T15:41:25.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.448+0000 7f95723d8740 1 Processor -- start 2024-05-21T15:41:25.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.449+0000 7f95723d8740 1 -- start start 2024-05-21T15:41:25.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.449+0000 7f95723d8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2e2560 0x56530a689e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.449+0000 7f95723d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2c5550 0x56530a6a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.449+0000 7f95723d8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2cd730 0x56530a6a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.450+0000 7f95723d8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f95641a8590 con 0x56530a2c5550 2024-05-21T15:41:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.450+0000 7f95723d8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f95641aa0d0 con 0x56530a2cd730 2024-05-21T15:41:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.450+0000 7f95723d8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f95641a8c60 con 0x56530a2e2560 2024-05-21T15:41:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.451+0000 7f95691f9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2e2560 0x56530a689e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.451+0000 7f95691f9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2e2560 0x56530a689e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:49888/0 (socket says 172.21.15.38:49888) 2024-05-21T15:41:25.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.451+0000 7f95691f9640 1 -- 172.21.15.38:0/3182299026 learned_addr learned my addr 172.21.15.38:0/3182299026 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:25.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.451+0000 7f95699fa640 1 --2- 172.21.15.38:0/3182299026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2cd730 0x56530a6a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.453+0000 7f95689f8640 1 --2- 172.21.15.38:0/3182299026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2c5550 0x56530a6a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.453+0000 7f95699fa640 1 -- 172.21.15.38:0/3182299026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2e2560 msgr2=0x56530a689e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:25.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.453+0000 7f95699fa640 1 --2- 172.21.15.38:0/3182299026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2e2560 0x56530a689e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.453+0000 7f95699fa640 1 -- 172.21.15.38:0/3182299026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2c5550 msgr2=0x56530a6a8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:25.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.453+0000 7f95699fa640 1 --2- 172.21.15.38:0/3182299026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2c5550 0x56530a6a8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.453+0000 7f95699fa640 1 -- 172.21.15.38:0/3182299026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56530a6a80a0 con 0x56530a2cd730 2024-05-21T15:41:25.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.453+0000 7f95699fa640 1 --2- 172.21.15.38:0/3182299026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2cd730 0x56530a6a94b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f954c009a60 tx=0x7f954c009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a161762685a48d5b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:25.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.454+0000 7f955affd640 1 -- 172.21.15.38:0/3182299026 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c003b70 con 0x56530a2cd730 2024-05-21T15:41:25.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.454+0000 7f955affd640 1 -- 172.21.15.38:0/3182299026 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f954c055070 con 0x56530a2cd730 2024-05-21T15:41:25.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.455+0000 7f95723d8740 1 -- 172.21.15.38:0/3182299026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2cd730 msgr2=0x56530a6a94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.455+0000 7f95723d8740 1 --2- 172.21.15.38:0/3182299026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2cd730 0x56530a6a94b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f954c009a60 tx=0x7f954c009540 comp rx=0 tx=0).stop 2024-05-21T15:41:25.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.455+0000 7f95723d8740 1 -- 172.21.15.38:0/3182299026 shutdown_connections 2024-05-21T15:41:25.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.455+0000 7f95723d8740 1 --2- 172.21.15.38:0/3182299026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2cd730 0x56530a6a94b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.455+0000 7f95723d8740 1 --2- 172.21.15.38:0/3182299026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2c5550 0x56530a6a8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.455+0000 7f95723d8740 1 --2- 172.21.15.38:0/3182299026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2e2560 0x56530a689e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.455+0000 7f95723d8740 1 -- 172.21.15.38:0/3182299026 >> 172.21.15.38:0/3182299026 conn(0x56530a6b7030 msgr2=0x56530a689750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:25.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.458+0000 7f95723d8740 1 -- 172.21.15.38:0/3182299026 shutdown_connections 2024-05-21T15:41:25.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.460+0000 7f95723d8740 1 -- 172.21.15.38:0/3182299026 wait complete. 2024-05-21T15:41:25.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.460+0000 7f95723d8740 1 -- start start 2024-05-21T15:41:25.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.460+0000 7f95723d8740 1 Processor -- start 2024-05-21T15:41:25.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.463+0000 7f95723d8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2c5550 0x56530a6ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.463+0000 7f95723d8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2cd730 0x56530a6ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.463+0000 7f95723d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2e2560 0x56530a6af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.463+0000 7f95723d8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f95641a9a00 con 0x56530a2e2560 2024-05-21T15:41:25.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.463+0000 7f95723d8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f95641a9330 con 0x56530a2cd730 2024-05-21T15:41:25.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.463+0000 7f95723d8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f95641a6570 con 0x56530a2c5550 2024-05-21T15:41:25.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.463+0000 7f95723d8740 10 client.0 _ll_register_callbacks cb 0x7f95540a1490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:25.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.465+0000 7f95691f9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2c5550 0x56530a6ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.465+0000 7f95691f9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2c5550 0x56530a6ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:49890/0 (socket says 172.21.15.38:49890) 2024-05-21T15:41:25.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.465+0000 7f95691f9640 1 -- 172.21.15.38:0/3424923866 learned_addr learned my addr 172.21.15.38:0/3424923866 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:25.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.465+0000 7f95699fa640 1 --2- 172.21.15.38:0/3424923866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2e2560 0x56530a6af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.465+0000 7f95689f8640 1 --2- 172.21.15.38:0/3424923866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2cd730 0x56530a6ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.466+0000 7f95691f9640 1 -- 172.21.15.38:0/3424923866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2cd730 msgr2=0x56530a6ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:25.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.466+0000 7f95691f9640 1 --2- 172.21.15.38:0/3424923866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2cd730 0x56530a6ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.466+0000 7f95691f9640 1 -- 172.21.15.38:0/3424923866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2e2560 msgr2=0x56530a6af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:25.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.468+0000 7f95691f9640 1 --2- 172.21.15.38:0/3424923866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2e2560 0x56530a6af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.468+0000 7f95691f9640 1 -- 172.21.15.38:0/3424923866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56530a6af8a0 con 0x56530a2c5550 2024-05-21T15:41:25.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.468+0000 7f95699fa640 1 --2- 172.21.15.38:0/3424923866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2e2560 0x56530a6af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.471+0000 7f95691f9640 1 --2- 172.21.15.38:0/3424923866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2c5550 0x56530a6ade70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f955400fea0 tx=0x7f955409dc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2959f51afc6e5885 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.471+0000 7f9558ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:41:25.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.472+0000 7f9558ff9640 1 -- 172.21.15.38:0/3424923866 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955400ab40 con 0x56530a2c5550 2024-05-21T15:41:25.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.472+0000 7f95723d8740 1 -- 172.21.15.38:0/3424923866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f954c055070 con 0x56530a2c5550 2024-05-21T15:41:25.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.472+0000 7f9558ff9640 1 -- 172.21.15.38:0/3424923866 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f955408c0d0 con 0x56530a2c5550 2024-05-21T15:41:25.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.472+0000 7f9558ff9640 1 -- 172.21.15.38:0/3424923866 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f95540d2b30 con 0x56530a2c5550 2024-05-21T15:41:25.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.472+0000 7f95723d8740 10 client.27115 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:25.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.472+0000 7f95723d8740 1 -- 172.21.15.38:0/3424923866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56530a71f7d0 con 0x56530a2c5550 2024-05-21T15:41:25.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.473+0000 7f9558ff9640 1 -- 172.21.15.38:0/3424923866 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f954c055070 con 0x56530a2c5550 2024-05-21T15:41:25.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.474+0000 7f9558ff9640 1 -- 172.21.15.38:0/3424923866 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f95540a0a20 con 0x56530a2c5550 2024-05-21T15:41:25.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.474+0000 7f95723d8740 10 client.27115 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:25.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.474+0000 7f95723d8740 10 client.27115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:25.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.474+0000 7f95723d8740 20 client.27115 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:25.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.474+0000 7f95723d8740 1 --2- 172.21.15.38:0/3424923866 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56530a2ee410 0x56530a721f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.474+0000 7f95723d8740 4 client.27115 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:41:25.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.474+0000 7f95723d8740 1 -- 172.21.15.38:0/3424923866 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56530a5fa410 con 0x56530a2ee410 2024-05-21T15:41:25.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.476+0000 7f95689f8640 1 --2- 172.21.15.38:0/3424923866 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56530a2ee410 0x56530a721f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.488+0000 7f95689f8640 1 --2- 172.21.15.38:0/3424923866 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56530a2ee410 0x56530a721f00 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5e271c2f3ae7320 server_cookie=5e3ee2e56c4a6e1c in_seq=0 out_seq=0 2024-05-21T15:41:25.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.489+0000 7f9558ff9640 10 client.27115 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:25.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.492+0000 7f9558ff9640 1 -- 172.21.15.38:0/3424923866 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56530a5fa410 con 0x56530a2ee410 2024-05-21T15:41:25.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.492+0000 7f9558ff9640 10 client.27115 handle_command_reply: tid=0 2024-05-21T15:41:25.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.492+0000 7f95723d8740 1 client.27115 shutdown 2024-05-21T15:41:25.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.495+0000 7f95723d8740 1 -- 172.21.15.38:0/3424923866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2c5550 msgr2=0x56530a6ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.495+0000 7f95723d8740 1 --2- 172.21.15.38:0/3424923866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2c5550 0x56530a6ade70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f955400fea0 tx=0x7f955409dc80 comp rx=0 tx=0).stop 2024-05-21T15:41:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.495+0000 7f95723d8740 1 -- 172.21.15.38:0/3424923866 shutdown_connections 2024-05-21T15:41:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.495+0000 7f95723d8740 1 --2- 172.21.15.38:0/3424923866 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56530a2ee410 0x56530a721f00 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.495+0000 7f95723d8740 1 --2- 172.21.15.38:0/3424923866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56530a2e2560 0x56530a6af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.495+0000 7f95723d8740 1 --2- 172.21.15.38:0/3424923866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56530a2cd730 0x56530a6ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.495+0000 7f95723d8740 1 --2- 172.21.15.38:0/3424923866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56530a2c5550 0x56530a6ade70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.495+0000 7f95723d8740 1 -- 172.21.15.38:0/3424923866 >> 172.21.15.38:0/3424923866 conn(0x56530a6b7030 msgr2=0x56530a6a8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:25.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.495+0000 7f9558ff9640 0 client.27115 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:25.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.497+0000 7f95723d8740 1 -- 172.21.15.38:0/3424923866 shutdown_connections 2024-05-21T15:41:25.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.497+0000 7f95723d8740 1 -- 172.21.15.38:0/3424923866 wait complete. 2024-05-21T15:41:25.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.498+0000 7f95723d8740 20 client.27115 trim_cache size 0 max 16384 2024-05-21T15:41:25.497 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:41:25.497 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-05-21T15:41:25.497 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "43e054b9-8cd5-4f98-946b-035b679d4d55", 2024-05-21T15:41:25.497 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-05-21T15:41:25.497 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:41:25.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.501+0000 7f956b45d640 1 -- 172.21.15.38:0/4162419450 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f95500a2fd0 msgr2=0x7f95500a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.501+0000 7f956b45d640 1 --2- 172.21.15.38:0/4162419450 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f95500a2fd0 0x7f95500a54e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x56530a68f880 tx=0x7f955c006c20 comp rx=0 tx=0).stop 2024-05-21T15:41:25.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.501+0000 7f956b45d640 1 -- 172.21.15.38:0/4162419450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95641019c0 msgr2=0x7f9564197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.501+0000 7f956b45d640 1 --2- 172.21.15.38:0/4162419450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95641019c0 0x7f9564197240 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f955400fed0 tx=0x7f9554009d60 comp rx=0 tx=0).stop 2024-05-21T15:41:25.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.510+0000 7f956b45d640 1 -- 172.21.15.38:0/4162419450 shutdown_connections 2024-05-21T15:41:25.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.510+0000 7f956b45d640 1 --2- 172.21.15.38:0/4162419450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9564107900 0x7f956419e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.510+0000 7f956b45d640 1 --2- 172.21.15.38:0/4162419450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9564102c00 0x7f9564197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.510+0000 7f956b45d640 1 --2- 172.21.15.38:0/4162419450 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f95500a2fd0 0x7f95500a54e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.510+0000 7f956b45d640 1 --2- 172.21.15.38:0/4162419450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95641019c0 0x7f9564197240 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.510+0000 7f956b45d640 1 -- 172.21.15.38:0/4162419450 >> 172.21.15.38:0/4162419450 conn(0x7f95640fcbb0 msgr2=0x7f95641038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:25.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.510+0000 7f956b45d640 1 -- 172.21.15.38:0/4162419450 shutdown_connections 2024-05-21T15:41:25.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.511+0000 7f956b45d640 1 -- 172.21.15.38:0/4162419450 wait complete. 2024-05-21T15:41:25.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:41:25.528 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:41:25.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.740+0000 7f393359e640 1 -- 172.21.15.38:0/1233745647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f392c09b820 msgr2=0x7f392c09dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.740+0000 7f393359e640 1 --2- 172.21.15.38:0/1233745647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f392c09b820 0x7f392c09dc40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3924009900 tx=0x7f392402a420 comp rx=0 tx=0).stop 2024-05-21T15:41:25.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.745+0000 7f393359e640 1 -- 172.21.15.38:0/1233745647 shutdown_connections 2024-05-21T15:41:25.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.745+0000 7f393359e640 1 --2- 172.21.15.38:0/1233745647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f392c09b820 0x7f392c09dc40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.745+0000 7f393359e640 1 --2- 172.21.15.38:0/1233745647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f392c096b20 0x7f392c096f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.745+0000 7f393359e640 1 --2- 172.21.15.38:0/1233745647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392c0958e0 0x7f392c0965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.745+0000 7f393359e640 1 -- 172.21.15.38:0/1233745647 >> 172.21.15.38:0/1233745647 conn(0x7f392c090ab0 msgr2=0x7f392c092f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:25.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.747+0000 7f393359e640 1 -- 172.21.15.38:0/1233745647 shutdown_connections 2024-05-21T15:41:25.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.747+0000 7f393359e640 1 -- 172.21.15.38:0/1233745647 wait complete. 2024-05-21T15:41:25.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.748+0000 7f393359e640 1 Processor -- start 2024-05-21T15:41:25.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.752+0000 7f393359e640 1 -- start start 2024-05-21T15:41:25.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.752+0000 7f393359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f392c0958e0 0x7f392c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.752+0000 7f393359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f392c096b20 0x7f392c0a6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.752+0000 7f393359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392c09b820 0x7f392c0a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.752+0000 7f393359e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f392c0a0450 con 0x7f392c09b820 2024-05-21T15:41:25.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.752+0000 7f393359e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f392c0a02d0 con 0x7f392c0958e0 2024-05-21T15:41:25.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.752+0000 7f393359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f392c0a05d0 con 0x7f392c096b20 2024-05-21T15:41:25.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.753+0000 7f3932d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392c09b820 0x7f392c0a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.753+0000 7f393259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f392c0958e0 0x7f392c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.753+0000 7f3931d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f392c096b20 0x7f392c0a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.753+0000 7f3932d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392c09b820 0x7f392c0a9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35920/0 (socket says 172.21.15.38:35920) 2024-05-21T15:41:25.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.753+0000 7f3932d9d640 1 -- 172.21.15.38:0/1281927662 learned_addr learned my addr 172.21.15.38:0/1281927662 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:25.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.755+0000 7f393259c640 1 -- 172.21.15.38:0/1281927662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f392c096b20 msgr2=0x7f392c0a6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.755+0000 7f393259c640 1 --2- 172.21.15.38:0/1281927662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f392c096b20 0x7f392c0a6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.755+0000 7f393259c640 1 -- 172.21.15.38:0/1281927662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392c09b820 msgr2=0x7f392c0a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.755+0000 7f393259c640 1 --2- 172.21.15.38:0/1281927662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392c09b820 0x7f392c0a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.755+0000 7f393259c640 1 -- 172.21.15.38:0/1281927662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392c0a9d70 con 0x7f392c0958e0 2024-05-21T15:41:25.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.756+0000 7f393259c640 1 --2- 172.21.15.38:0/1281927662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f392c0958e0 0x7f392c0a6250 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3934051020 tx=0x7f39340705f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:25.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.757+0000 7f39237fe640 1 -- 172.21.15.38:0/1281927662 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934075590 con 0x7f392c0958e0 2024-05-21T15:41:25.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.758+0000 7f39237fe640 1 -- 172.21.15.38:0/1281927662 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3934075c10 con 0x7f392c0958e0 2024-05-21T15:41:25.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.758+0000 7f39237fe640 1 -- 172.21.15.38:0/1281927662 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934076b40 con 0x7f392c0958e0 2024-05-21T15:41:25.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.758+0000 7f393359e640 1 -- 172.21.15.38:0/1281927662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392c0aa060 con 0x7f392c0958e0 2024-05-21T15:41:25.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.758+0000 7f393359e640 1 -- 172.21.15.38:0/1281927662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f392c0aa530 con 0x7f392c0958e0 2024-05-21T15:41:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.763+0000 7f393fdb4740 1 Processor -- start 2024-05-21T15:41:25.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.764+0000 7f393fdb4740 1 -- start start 2024-05-21T15:41:25.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.764+0000 7f393fdb4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106d0390 0x562c10a6fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.764+0000 7f393fdb4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a70330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.764+0000 7f393fdb4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106af6b0 0x562c10a77230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.764+0000 7f393fdb4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562c109d4d00 con 0x562c106a74d0 2024-05-21T15:41:25.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.764+0000 7f393fdb4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562c1097c4c0 con 0x562c106d0390 2024-05-21T15:41:25.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.764+0000 7f393fdb4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562c109d0cc0 con 0x562c106af6b0 2024-05-21T15:41:25.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.767+0000 7f393259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a70330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.767+0000 7f393259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a70330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35922/0 (socket says 172.21.15.38:35922) 2024-05-21T15:41:25.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.767+0000 7f393259c640 1 -- 172.21.15.38:0/1011084824 learned_addr learned my addr 172.21.15.38:0/1011084824 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:25.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.768+0000 7f3932d9d640 1 --2- 172.21.15.38:0/1011084824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106d0390 0x562c10a6fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.768+0000 7f3932d9d640 1 -- 172.21.15.38:0/1011084824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106af6b0 msgr2=0x562c10a77230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:41:25.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.768+0000 7f3932d9d640 1 --2- 172.21.15.38:0/1011084824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106af6b0 0x562c10a77230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.768+0000 7f3932d9d640 1 -- 172.21.15.38:0/1011084824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 msgr2=0x562c10a70330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:25.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.768+0000 7f3932d9d640 1 --2- 172.21.15.38:0/1011084824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a70330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.768+0000 7f3932d9d640 1 -- 172.21.15.38:0/1011084824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c107b27d0 con 0x562c106d0390 2024-05-21T15:41:25.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.769+0000 7f39237fe640 1 -- 172.21.15.38:0/1281927662 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3934075730 con 0x7f392c0958e0 2024-05-21T15:41:25.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.770+0000 7f39237fe640 1 --2- 172.21.15.38:0/1281927662 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f391c0a2da0 0x7f391c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.771+0000 7f39237fe640 1 -- 172.21.15.38:0/1281927662 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f3934131900 con 0x7f392c0958e0 2024-05-21T15:41:25.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.771+0000 7f393259c640 1 --2- 172.21.15.38:0/1281927662 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f391c0a2da0 0x7f391c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.772+0000 7f393259c640 1 --2- 172.21.15.38:0/1281927662 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f391c0a2da0 0x7f391c0a52b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3934050c70 tx=0x7f3934137530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:25.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.773+0000 7f3932d9d640 1 --2- 172.21.15.38:0/1011084824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106d0390 0x562c10a6fdb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3924006010 tx=0x7f39240318d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=369d5b2e4f25e243 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:25.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.775+0000 7f3920ff9640 1 -- 172.21.15.38:0/1011084824 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924035040 con 0x562c106d0390 2024-05-21T15:41:25.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.775+0000 7f3920ff9640 1 -- 172.21.15.38:0/1011084824 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3924005520 con 0x562c106d0390 2024-05-21T15:41:25.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.776+0000 7f393fdb4740 1 -- 172.21.15.38:0/1011084824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106d0390 msgr2=0x562c10a6fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.776+0000 7f393fdb4740 1 --2- 172.21.15.38:0/1011084824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106d0390 0x562c10a6fdb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3924006010 tx=0x7f39240318d0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.779+0000 7f393fdb4740 1 -- 172.21.15.38:0/1011084824 shutdown_connections 2024-05-21T15:41:25.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.779+0000 7f393fdb4740 1 --2- 172.21.15.38:0/1011084824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106af6b0 0x562c10a77230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.779+0000 7f393fdb4740 1 --2- 172.21.15.38:0/1011084824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a70330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.779+0000 7f393fdb4740 1 --2- 172.21.15.38:0/1011084824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106d0390 0x562c10a6fdb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.779+0000 7f393fdb4740 1 -- 172.21.15.38:0/1011084824 >> 172.21.15.38:0/1011084824 conn(0x562c106c44e0 msgr2=0x562c10a6b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:25.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.785+0000 7f393fdb4740 1 -- 172.21.15.38:0/1011084824 shutdown_connections 2024-05-21T15:41:25.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.786+0000 7f393fdb4740 1 -- 172.21.15.38:0/1011084824 wait complete. 2024-05-21T15:41:25.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.787+0000 7f393fdb4740 1 -- start start 2024-05-21T15:41:25.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.787+0000 7f393fdb4740 1 Processor -- start 2024-05-21T15:41:25.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.788+0000 7f393fdb4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a79c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.788+0000 7f393fdb4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106af6b0 0x562c10a7a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.788+0000 7f393fdb4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 0x562c10a81110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.788+0000 7f393fdb4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562c10a77c70 con 0x562c106a74d0 2024-05-21T15:41:25.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.788+0000 7f393fdb4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562c1098fb90 con 0x562c106af6b0 2024-05-21T15:41:25.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.788+0000 7f393fdb4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562c10a77df0 con 0x562c106d0390 2024-05-21T15:41:25.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.788+0000 7f393fdb4740 10 client.0 _ll_register_callbacks cb 0x562c109cbda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:25.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.789+0000 7f3932d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 0x562c10a81110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.789+0000 7f3932d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a79c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.789+0000 7f3932d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a79c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35932/0 (socket says 172.21.15.38:35932) 2024-05-21T15:41:25.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.789+0000 7f3932d9d640 1 -- 172.21.15.38:0/1441813086 learned_addr learned my addr 172.21.15.38:0/1441813086 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.790+0000 7f3932d9d640 1 -- 172.21.15.38:0/1441813086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106af6b0 msgr2=0x562c10a7a1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:41:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.790+0000 7f3932d9d640 1 --2- 172.21.15.38:0/1441813086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106af6b0 0x562c10a7a1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.790+0000 7f3932d9d640 1 -- 172.21.15.38:0/1441813086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 msgr2=0x562c10a79c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.790+0000 7f3932d9d640 1 --2- 172.21.15.38:0/1441813086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a79c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.790+0000 7f3932d9d640 1 -- 172.21.15.38:0/1441813086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c106e83f0 con 0x562c106d0390 2024-05-21T15:41:25.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.793+0000 7f3932d9d640 1 --2- 172.21.15.38:0/1441813086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 0x562c10a81110 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3924005240 tx=0x7f3924009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=971c92134d049aaf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:25.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.793+0000 7f3919ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:41:25.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.793+0000 7f3919ffb640 1 -- 172.21.15.38:0/1441813086 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924035030 con 0x562c106d0390 2024-05-21T15:41:25.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.794+0000 7f3919ffb640 1 -- 172.21.15.38:0/1441813086 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f392403d450 con 0x562c106d0390 2024-05-21T15:41:25.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.794+0000 7f393fdb4740 1 -- 172.21.15.38:0/1441813086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3924005520 con 0x562c106d0390 2024-05-21T15:41:25.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.796+0000 7f3919ffb640 1 -- 172.21.15.38:0/1441813086 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f392405b050 con 0x562c106d0390 2024-05-21T15:41:25.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.797+0000 7f393fdb4740 10 client.27124 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:25.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.797+0000 7f393fdb4740 1 -- 172.21.15.38:0/1441813086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562c106a80f0 con 0x562c106d0390 2024-05-21T15:41:25.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.801+0000 7f3919ffb640 1 -- 172.21.15.38:0/1441813086 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3924005520 con 0x562c106d0390 2024-05-21T15:41:25.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.801+0000 7f3919ffb640 1 -- 172.21.15.38:0/1441813086 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f392403a040 con 0x562c106d0390 2024-05-21T15:41:25.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.802+0000 7f393fdb4740 10 client.27124 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:25.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.802+0000 7f393fdb4740 10 client.27124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:25.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.802+0000 7f393fdb4740 20 client.27124 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:25.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.802+0000 7f393fdb4740 1 --2- 172.21.15.38:0/1441813086 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c1068ef60 0x562c10a87b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.802+0000 7f393fdb4740 4 client.27124 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:25.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.802+0000 7f393fdb4740 1 -- 172.21.15.38:0/1441813086 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c109e2430 con 0x562c1068ef60 2024-05-21T15:41:25.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.804+0000 7f3931d9b640 1 --2- 172.21.15.38:0/1441813086 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c1068ef60 0x562c10a87b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.806+0000 7f3931d9b640 1 --2- 172.21.15.38:0/1441813086 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c1068ef60 0x562c10a87b50 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ca3ef6a4ae82a52 server_cookie=ee74c78f2fe3a311 in_seq=0 out_seq=0 2024-05-21T15:41:25.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.806+0000 7f3919ffb640 10 client.27124 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:25.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.808+0000 7f3919ffb640 1 -- 172.21.15.38:0/1441813086 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562c109e2430 con 0x562c1068ef60 2024-05-21T15:41:25.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.808+0000 7f3919ffb640 10 client.27124 handle_command_reply: tid=0 2024-05-21T15:41:25.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.809+0000 7f393fdb4740 1 client.27124 shutdown 2024-05-21T15:41:25.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.812+0000 7f393fdb4740 1 -- 172.21.15.38:0/1441813086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 msgr2=0x562c10a81110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.812+0000 7f393fdb4740 1 --2- 172.21.15.38:0/1441813086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 0x562c10a81110 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3924005240 tx=0x7f3924009260 comp rx=0 tx=0).stop 2024-05-21T15:41:25.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.813+0000 7f393fdb4740 1 -- 172.21.15.38:0/1441813086 shutdown_connections 2024-05-21T15:41:25.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.813+0000 7f393fdb4740 1 --2- 172.21.15.38:0/1441813086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 0x562c10a81110 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.813+0000 7f393fdb4740 1 --2- 172.21.15.38:0/1441813086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106af6b0 0x562c10a7a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.813+0000 7f393fdb4740 1 --2- 172.21.15.38:0/1441813086 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c1068ef60 0x562c10a87b50 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.813+0000 7f393fdb4740 1 --2- 172.21.15.38:0/1441813086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a79c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.813+0000 7f393fdb4740 1 -- 172.21.15.38:0/1441813086 >> 172.21.15.38:0/1441813086 conn(0x562c106c44e0 msgr2=0x562c10a6a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:25.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.814+0000 7f3919ffb640 0 client.27124 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:25.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.815+0000 7f393fdb4740 1 -- 172.21.15.38:0/1441813086 shutdown_connections 2024-05-21T15:41:25.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.816+0000 7f393fdb4740 1 -- 172.21.15.38:0/1441813086 wait complete. 2024-05-21T15:41:25.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.817+0000 7f393fdb4740 20 client.27124 trim_cache size 0 max 16384 2024-05-21T15:41:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.856+0000 7f393fdb4740 1 Processor -- start 2024-05-21T15:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.857+0000 7f393fdb4740 1 -- start start 2024-05-21T15:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.857+0000 7f393fdb4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a74660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.858+0000 7f393fdb4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106af6b0 0x562c10a79610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.858+0000 7f393fdb4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 0x562c10a79c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.858+0000 7f393fdb4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f392c15e100 con 0x562c106a74d0 2024-05-21T15:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.858+0000 7f393fdb4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f392c15fc40 con 0x562c106af6b0 2024-05-21T15:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.858+0000 7f393fdb4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f392c15e7d0 con 0x562c106d0390 2024-05-21T15:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.860+0000 7f3932d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 0x562c10a79c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.860+0000 7f3932d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a74660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.860+0000 7f3932d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a74660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35946/0 (socket says 172.21.15.38:35946) 2024-05-21T15:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.860+0000 7f3932d9d640 1 -- 172.21.15.38:0/3653784078 learned_addr learned my addr 172.21.15.38:0/3653784078 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.862+0000 7f3932d9d640 1 -- 172.21.15.38:0/3653784078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 msgr2=0x562c10a79c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.862+0000 7f3932d9d640 1 --2- 172.21.15.38:0/3653784078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 0x562c10a79c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.862+0000 7f3932d9d640 1 -- 172.21.15.38:0/3653784078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106af6b0 msgr2=0x562c10a79610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.862+0000 7f3932d9d640 1 --2- 172.21.15.38:0/3653784078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106af6b0 0x562c10a79610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.862+0000 7f3932d9d640 1 -- 172.21.15.38:0/3653784078 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c10a7a870 con 0x562c106a74d0 2024-05-21T15:41:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.863+0000 7f3932d9d640 1 --2- 172.21.15.38:0/3653784078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a74660 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3924031810 tx=0x7f3924005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd20c31c6f7ea4d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.864+0000 7f391bfff640 1 -- 172.21.15.38:0/3653784078 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924035020 con 0x562c106a74d0 2024-05-21T15:41:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.864+0000 7f391bfff640 1 -- 172.21.15.38:0/3653784078 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3924049070 con 0x562c106a74d0 2024-05-21T15:41:25.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.864+0000 7f391bfff640 1 -- 172.21.15.38:0/3653784078 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392404f040 con 0x562c106a74d0 2024-05-21T15:41:25.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.866+0000 7f393fdb4740 1 -- 172.21.15.38:0/3653784078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 msgr2=0x562c10a74660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.866+0000 7f393fdb4740 1 --2- 172.21.15.38:0/3653784078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a74660 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3924031810 tx=0x7f3924005fd0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.868+0000 7f393fdb4740 1 -- 172.21.15.38:0/3653784078 shutdown_connections 2024-05-21T15:41:25.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.868+0000 7f393fdb4740 1 --2- 172.21.15.38:0/3653784078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 0x562c10a79c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.868+0000 7f393fdb4740 1 --2- 172.21.15.38:0/3653784078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106af6b0 0x562c10a79610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.868+0000 7f393fdb4740 1 --2- 172.21.15.38:0/3653784078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106a74d0 0x562c10a74660 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.868+0000 7f393fdb4740 1 -- 172.21.15.38:0/3653784078 >> 172.21.15.38:0/3653784078 conn(0x562c106c44e0 msgr2=0x562c10a78f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:25.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.868+0000 7f393fdb4740 1 -- 172.21.15.38:0/3653784078 shutdown_connections 2024-05-21T15:41:25.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.869+0000 7f393fdb4740 1 -- 172.21.15.38:0/3653784078 wait complete. 2024-05-21T15:41:25.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.870+0000 7f393fdb4740 1 -- start start 2024-05-21T15:41:25.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.870+0000 7f393fdb4740 1 Processor -- start 2024-05-21T15:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.871+0000 7f393fdb4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106a74d0 0x562c10a80ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.871+0000 7f393fdb4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106af6b0 0x562c10a7db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.872+0000 7f3932d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106a74d0 0x562c10a80ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.872+0000 7f3932d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106a74d0 0x562c10a80ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:57614/0 (socket says 172.21.15.38:57614) 2024-05-21T15:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.872+0000 7f3932d9d640 1 -- 172.21.15.38:0/475574109 learned_addr learned my addr 172.21.15.38:0/475574109 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.872+0000 7f393fdb4740 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 0x562c10a7e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.872+0000 7f393fdb4740 1 -- 172.21.15.38:0/475574109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f392c15f570 con 0x562c106af6b0 2024-05-21T15:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.872+0000 7f393fdb4740 1 -- 172.21.15.38:0/475574109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f392c15eea0 con 0x562c106a74d0 2024-05-21T15:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.872+0000 7f393fdb4740 1 -- 172.21.15.38:0/475574109 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f392c15c0e0 con 0x562c106d0390 2024-05-21T15:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.872+0000 7f393fdb4740 10 client.0 _ll_register_callbacks cb 0x7f3934137400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.874+0000 7f3932d9d640 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 0x562c10a7e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.874+0000 7f3932d9d640 1 -- 172.21.15.38:0/475574109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 msgr2=0x562c10a7e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:25.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.874+0000 7f3932d9d640 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 0x562c10a7e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.874+0000 7f3932d9d640 1 -- 172.21.15.38:0/475574109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106af6b0 msgr2=0x562c10a7db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:25.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.876+0000 7f3931d9b640 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106af6b0 0x562c10a7db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.876+0000 7f3932d9d640 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106af6b0 0x562c10a7db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.876+0000 7f3932d9d640 1 -- 172.21.15.38:0/475574109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c10aedef0 con 0x562c106a74d0 2024-05-21T15:41:25.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.877+0000 7f3931d9b640 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106af6b0 0x562c10a7db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:41:25.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.878+0000 7f3932d9d640 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106a74d0 0x562c10a80ff0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3924031b20 tx=0x7f39240566e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4e387b6383a64fe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:25.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.878+0000 7f3919ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:41:25.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.878+0000 7f3919ffb640 1 -- 172.21.15.38:0/475574109 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924035070 con 0x562c106a74d0 2024-05-21T15:41:25.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.878+0000 7f393fdb4740 1 -- 172.21.15.38:0/475574109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3924049070 con 0x562c106a74d0 2024-05-21T15:41:25.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.878+0000 7f3919ffb640 1 -- 172.21.15.38:0/475574109 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f3924005220 con 0x562c106a74d0 2024-05-21T15:41:25.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.879+0000 7f3919ffb640 1 -- 172.21.15.38:0/475574109 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3924005420 con 0x562c106a74d0 2024-05-21T15:41:25.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.879+0000 7f393fdb4740 10 client.27047 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:25.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.879+0000 7f393fdb4740 1 -- 172.21.15.38:0/475574109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562c10aef000 con 0x562c106a74d0 2024-05-21T15:41:25.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.883+0000 7f3919ffb640 1 -- 172.21.15.38:0/475574109 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3924049070 con 0x562c106a74d0 2024-05-21T15:41:25.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.883+0000 7f3919ffb640 1 -- 172.21.15.38:0/475574109 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f3924060440 con 0x562c106a74d0 2024-05-21T15:41:25.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.884+0000 7f393fdb4740 10 client.27047 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:25.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.884+0000 7f393fdb4740 10 client.27047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:25.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.884+0000 7f393fdb4740 20 client.27047 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:25.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.884+0000 7f393fdb4740 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c106643a0 0x562c10af16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:25.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.884+0000 7f393fdb4740 4 client.27047 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:41:25.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.884+0000 7f393fdb4740 1 -- 172.21.15.38:0/475574109 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562c109dc780 con 0x562c106643a0 2024-05-21T15:41:25.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.885+0000 7f3931d9b640 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c106643a0 0x562c10af16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:25.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.889+0000 7f3931d9b640 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c106643a0 0x562c10af16d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a479319a0f6cda6a server_cookie=e66a35807417f721 in_seq=0 out_seq=0 2024-05-21T15:41:25.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.889+0000 7f3919ffb640 10 client.27047 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:25.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.890+0000 7f3919ffb640 1 -- 172.21.15.38:0/475574109 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x562c109dc780 con 0x562c106643a0 2024-05-21T15:41:25.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.890+0000 7f3919ffb640 10 client.27047 handle_command_reply: tid=0 2024-05-21T15:41:25.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.890+0000 7f393fdb4740 1 client.27047 shutdown 2024-05-21T15:41:25.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.890+0000 7f393fdb4740 1 -- 172.21.15.38:0/475574109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106a74d0 msgr2=0x562c10a80ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.890+0000 7f393fdb4740 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106a74d0 0x562c10a80ff0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3924031b20 tx=0x7f39240566e0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.892+0000 7f393fdb4740 1 -- 172.21.15.38:0/475574109 shutdown_connections 2024-05-21T15:41:25.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.892+0000 7f393fdb4740 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c106d0390 0x562c10a7e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.892+0000 7f393fdb4740 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x562c106643a0 0x562c10af16d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.892+0000 7f393fdb4740 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c106af6b0 0x562c10a7db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.892+0000 7f393fdb4740 1 --2- 172.21.15.38:0/475574109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c106a74d0 0x562c10a80ff0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.892+0000 7f393fdb4740 1 -- 172.21.15.38:0/475574109 >> 172.21.15.38:0/475574109 conn(0x562c106c44e0 msgr2=0x562c10a6afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:25.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.892+0000 7f3919ffb640 0 client.27047 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:25.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.893+0000 7f393fdb4740 1 -- 172.21.15.38:0/475574109 shutdown_connections 2024-05-21T15:41:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.894+0000 7f393fdb4740 1 -- 172.21.15.38:0/475574109 wait complete. 2024-05-21T15:41:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.895+0000 7f393fdb4740 20 client.27047 trim_cache size 0 max 16384 2024-05-21T15:41:25.896 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:41:25.896 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "scrub active (0 inodes in the stack)", 2024-05-21T15:41:25.896 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-05-21T15:41:25.896 INFO:teuthology.orchestra.run.smithi038.stdout: "43e054b9-8cd5-4f98-946b-035b679d4d55": { 2024-05-21T15:41:25.896 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-05-21T15:41:25.896 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "43e054b9-8cd5-4f98-946b-035b679d4d55", 2024-05-21T15:41:25.896 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-05-21T15:41:25.896 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:41:25.896 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:41:25.897 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:41:25.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.901+0000 7f393359e640 1 -- 172.21.15.38:0/1281927662 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f391c0a2da0 msgr2=0x7f391c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.901+0000 7f393359e640 1 --2- 172.21.15.38:0/1281927662 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f391c0a2da0 0x7f391c0a52b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3934050c70 tx=0x7f3934137530 comp rx=0 tx=0).stop 2024-05-21T15:41:25.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.901+0000 7f393359e640 1 -- 172.21.15.38:0/1281927662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f392c0958e0 msgr2=0x7f392c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:25.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.901+0000 7f393359e640 1 --2- 172.21.15.38:0/1281927662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f392c0958e0 0x7f392c0a6250 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3934051020 tx=0x7f39340705f0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.901+0000 7f393359e640 1 -- 172.21.15.38:0/1281927662 shutdown_connections 2024-05-21T15:41:25.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.901+0000 7f393359e640 1 --2- 172.21.15.38:0/1281927662 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f391c0a2da0 0x7f391c0a52b0 secure :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3934050c70 tx=0x7f3934137530 comp rx=0 tx=0).stop 2024-05-21T15:41:25.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.901+0000 7f393359e640 1 --2- 172.21.15.38:0/1281927662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f392c09b820 0x7f392c0a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.901+0000 7f393359e640 1 --2- 172.21.15.38:0/1281927662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f392c096b20 0x7f392c0a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.901+0000 7f393359e640 1 --2- 172.21.15.38:0/1281927662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f392c0958e0 0x7f392c0a6250 secure :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3934051020 tx=0x7f39340705f0 comp rx=0 tx=0).stop 2024-05-21T15:41:25.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.901+0000 7f393359e640 1 -- 172.21.15.38:0/1281927662 >> 172.21.15.38:0/1281927662 conn(0x7f392c090ab0 msgr2=0x7f392c092ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:25.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.902+0000 7f393359e640 1 -- 172.21.15.38:0/1281927662 shutdown_connections 2024-05-21T15:41:25.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:25.902+0000 7f393359e640 1 -- 172.21.15.38:0/1281927662 wait complete. 2024-05-21T15:41:25.920 INFO:tasks.cephfs.filesystem:scrub status for tag:43e054b9-8cd5-4f98-946b-035b679d4d55 - {'path': '/', 'tag': '43e054b9-8cd5-4f98-946b-035b679d4d55', 'options': 'recursive,force'} 2024-05-21T15:41:25.920 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:25 smithi038 ceph-mon[31995]: pgmap v561: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 4.88k op/s 2024-05-21T15:41:25.920 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:25 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:41:25.920 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:25 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:41:25.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:25 smithi053 ceph-mon[36496]: pgmap v561: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 4.88k op/s 2024-05-21T15:41:25.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:25 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:41:25.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:25 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:41:26.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:25 smithi045 ceph-mon[36409]: pgmap v561: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 4.88k op/s 2024-05-21T15:41:26.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:25 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:41:26.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:25 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:41:26.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:26 smithi053 ceph-mon[36496]: scrub queued for path: / 2024-05-21T15:41:26.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:26 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:26.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:26 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:41:26.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:26 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:41:26.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:26 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:41:26.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:26 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:41:26.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:26 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:26.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:26 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:26.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:26 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:26.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:26 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:26.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:26 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:27.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:26 smithi045 ceph-mon[36409]: scrub queued for path: / 2024-05-21T15:41:27.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:26 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:27.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:26 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:41:27.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:26 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:41:27.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:26 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:41:27.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:26 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:41:27.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:26 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:27.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:26 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:27.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:26 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:27.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:26 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:27.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:26 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:27.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:26 smithi038 ceph-mon[31995]: scrub queued for path: / 2024-05-21T15:41:27.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:26 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:27.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:26 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:41:27.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:26 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:41:27.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:26 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:41:27.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:26 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:41:27.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:26 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:27.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:26 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:27.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:26 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:27.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:26 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:27.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:26 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:41:27.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:27 smithi053 ceph-mon[36496]: pgmap v562: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 5.66k op/s 2024-05-21T15:41:28.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:27 smithi045 ceph-mon[36409]: pgmap v562: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 5.66k op/s 2024-05-21T15:41:28.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:27 smithi038 ceph-mon[31995]: pgmap v562: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 5.66k op/s 2024-05-21T15:41:29.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:29 smithi053 ceph-mon[36496]: pgmap v563: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.38k op/s 2024-05-21T15:41:30.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:29 smithi045 ceph-mon[36409]: pgmap v563: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.38k op/s 2024-05-21T15:41:30.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:29 smithi038 ceph-mon[31995]: pgmap v563: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.38k op/s 2024-05-21T15:41:30.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:30 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:41:31.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:30 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:41:31.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:30 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:41:31.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:31 smithi053 ceph-mon[36496]: pgmap v564: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 3.99k op/s 2024-05-21T15:41:32.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:31 smithi045 ceph-mon[36409]: pgmap v564: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 3.99k op/s 2024-05-21T15:41:32.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:31 smithi038 ceph-mon[31995]: pgmap v564: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 3.99k op/s 2024-05-21T15:41:32.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:32.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:32.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:32.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:32.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:32.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:32.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:32.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:32.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-05-21T15:41:32.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-05-21T15:41:32.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: Unable to set osd_memory_target on smithi045 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-21T15:41:32.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: Unable to set osd_memory_target on smithi038 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-21T15:41:32.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:41:32.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:41:32.955 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:41:33.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-05-21T15:41:33.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-05-21T15:41:33.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: Unable to set osd_memory_target on smithi045 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-21T15:41:33.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: Unable to set osd_memory_target on smithi038 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-21T15:41:33.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:41:33.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:41:33.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:41:33.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:41:33.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-05-21T15:41:33.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi038 to 177.2M 2024-05-21T15:41:33.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: Unable to set osd_memory_target on smithi045 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-21T15:41:33.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: Unable to set osd_memory_target on smithi038 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-21T15:41:33.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:41:33.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:41:33.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:41:34.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:33 smithi045 ceph-mon[36409]: pgmap v565: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 22 MiB/s wr, 4.82k op/s 2024-05-21T15:41:34.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:33 smithi038 ceph-mon[31995]: pgmap v565: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 22 MiB/s wr, 4.82k op/s 2024-05-21T15:41:34.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:33 smithi053 ceph-mon[36496]: pgmap v565: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 22 MiB/s wr, 4.82k op/s 2024-05-21T15:41:35.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:34 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:41:35.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:34 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:41:35.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:34 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:41:35.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:34 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:41:35.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:34 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:41:35.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:34 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:41:35.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:34 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:41:35.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:34 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:41:35.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:34 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:41:35.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:34 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:41:35.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:34 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:41:35.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:34 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:41:35.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:34 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:41:35.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:34 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:41:35.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:34 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:41:36.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:35 smithi045 ceph-mon[36409]: pgmap v566: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 26 MiB/s wr, 5.66k op/s 2024-05-21T15:41:36.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:35 smithi038 ceph-mon[31995]: pgmap v566: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 26 MiB/s wr, 5.66k op/s 2024-05-21T15:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.098+0000 7fa524daa640 1 -- 172.21.15.38:0/1730592426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa520102ad0 msgr2=0x7fa5201037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.098+0000 7fa524daa640 1 --2- 172.21.15.38:0/1730592426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa520102ad0 0x7fa5201037d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa510009900 tx=0x7fa51002a410 comp rx=0 tx=0).stop 2024-05-21T15:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.098+0000 7fa524daa640 1 -- 172.21.15.38:0/1730592426 shutdown_connections 2024-05-21T15:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.098+0000 7fa524daa640 1 --2- 172.21.15.38:0/1730592426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa520108a10 0x7fa52010ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.098+0000 7fa524daa640 1 --2- 172.21.15.38:0/1730592426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa520103d10 0x7fa520104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.098+0000 7fa524daa640 1 --2- 172.21.15.38:0/1730592426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa520102ad0 0x7fa5201037d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.098+0000 7fa524daa640 1 -- 172.21.15.38:0/1730592426 >> 172.21.15.38:0/1730592426 conn(0x7fa5200fdce0 msgr2=0x7fa520100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.098+0000 7fa524daa640 1 -- 172.21.15.38:0/1730592426 shutdown_connections 2024-05-21T15:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.098+0000 7fa524daa640 1 -- 172.21.15.38:0/1730592426 wait complete. 2024-05-21T15:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.099+0000 7fa524daa640 1 Processor -- start 2024-05-21T15:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.099+0000 7fa524daa640 1 -- start start 2024-05-21T15:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.099+0000 7fa524daa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa520102ad0 0x7fa520075800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.099+0000 7fa524daa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa520103d10 0x7fa520073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.099+0000 7fa524daa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa520108a10 0x7fa5200743e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.099+0000 7fa524daa640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa52010e700 con 0x7fa520103d10 2024-05-21T15:41:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.099+0000 7fa524daa640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa52010e580 con 0x7fa520108a10 2024-05-21T15:41:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.100+0000 7fa524daa640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa52010e880 con 0x7fa520102ad0 2024-05-21T15:41:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.100+0000 7fa517fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa520103d10 0x7fa520073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.100+0000 7fa517fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa520103d10 0x7fa520073ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42676/0 (socket says 172.21.15.38:42676) 2024-05-21T15:41:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.100+0000 7fa517fff640 1 -- 172.21.15.38:0/3628904785 learned_addr learned my addr 172.21.15.38:0/3628904785 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.100+0000 7fa51e59c640 1 --2- 172.21.15.38:0/3628904785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa520102ad0 0x7fa520075800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.100+0000 7fa51ed9d640 1 --2- 172.21.15.38:0/3628904785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa520108a10 0x7fa5200743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.100+0000 7fa517fff640 1 -- 172.21.15.38:0/3628904785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa520102ad0 msgr2=0x7fa520075800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.100+0000 7fa517fff640 1 --2- 172.21.15.38:0/3628904785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa520102ad0 0x7fa520075800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.100+0000 7fa517fff640 1 -- 172.21.15.38:0/3628904785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa520108a10 msgr2=0x7fa5200743e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.100+0000 7fa517fff640 1 --2- 172.21.15.38:0/3628904785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa520108a10 0x7fa5200743e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.100+0000 7fa517fff640 1 -- 172.21.15.38:0/3628904785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa520074b10 con 0x7fa520103d10 2024-05-21T15:41:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.100+0000 7fa517fff640 1 --2- 172.21.15.38:0/3628904785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa520103d10 0x7fa520073ea0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa50800e9f0 tx=0x7fa50800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.100+0000 7fa5177fe640 1 -- 172.21.15.38:0/3628904785 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50800cd80 con 0x7fa520103d10 2024-05-21T15:41:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.100+0000 7fa524daa640 1 -- 172.21.15.38:0/3628904785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5201a3360 con 0x7fa520103d10 2024-05-21T15:41:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.101+0000 7fa524daa640 1 -- 172.21.15.38:0/3628904785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5201a3850 con 0x7fa520103d10 2024-05-21T15:41:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.101+0000 7fa5177fe640 1 -- 172.21.15.38:0/3628904785 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa508010910 con 0x7fa520103d10 2024-05-21T15:41:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.101+0000 7fa5177fe640 1 -- 172.21.15.38:0/3628904785 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508026060 con 0x7fa520103d10 2024-05-21T15:41:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.104+0000 7fa5177fe640 1 -- 172.21.15.38:0/3628904785 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa508018980 con 0x7fa520103d10 2024-05-21T15:41:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.105+0000 7fa52bc53740 1 Processor -- start 2024-05-21T15:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.105+0000 7fa52bc53740 1 -- start start 2024-05-21T15:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.105+0000 7fa52bc53740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e9c1b0 0x55ea3523be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.105+0000 7fa52bc53740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea3523c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.105+0000 7fa52bc53740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e7b4d0 0x55ea352432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.105+0000 7fa52bc53740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ea351a0340 con 0x55ea34e732f0 2024-05-21T15:41:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa52bc53740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ea35148290 con 0x55ea34e9c1b0 2024-05-21T15:41:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa52bc53740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ea3519c300 con 0x55ea34e7b4d0 2024-05-21T15:41:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa51e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea3523c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa51e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea3523c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42692/0 (socket says 172.21.15.38:42692) 2024-05-21T15:41:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa51e59c640 1 -- 172.21.15.38:0/2048574639 learned_addr learned my addr 172.21.15.38:0/2048574639 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa517fff640 1 --2- 172.21.15.38:0/2048574639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e7b4d0 0x55ea352432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa5177fe640 1 --2- 172.21.15.38:0/3628904785 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa50c0a2c30 0x7fa50c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa51ed9d640 1 --2- 172.21.15.38:0/2048574639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e9c1b0 0x55ea3523be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa51e59c640 1 -- 172.21.15.38:0/2048574639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e7b4d0 msgr2=0x55ea352432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa51e59c640 1 --2- 172.21.15.38:0/2048574639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e7b4d0 0x55ea352432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa51e59c640 1 -- 172.21.15.38:0/2048574639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e9c1b0 msgr2=0x55ea3523be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa51e59c640 1 --2- 172.21.15.38:0/2048574639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e9c1b0 0x55ea3523be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa51e59c640 1 -- 172.21.15.38:0/2048574639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea3516c8e0 con 0x55ea34e732f0 2024-05-21T15:41:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa517fff640 1 --2- 172.21.15.38:0/2048574639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e7b4d0 0x55ea352432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa51ed9d640 1 --2- 172.21.15.38:0/3628904785 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa50c0a2c30 0x7fa50c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa5177fe640 1 -- 172.21.15.38:0/3628904785 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa5080d0450 con 0x7fa520103d10 2024-05-21T15:41:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.106+0000 7fa51e59c640 1 --2- 172.21.15.38:0/2048574639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea3523c3e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa510009900 tx=0x7fa510005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c0a27814d5b5eb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.107+0000 7fa514ff9640 1 -- 172.21.15.38:0/2048574639 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510040070 con 0x55ea34e732f0 2024-05-21T15:41:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.107+0000 7fa514ff9640 1 -- 172.21.15.38:0/2048574639 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa510031970 con 0x55ea34e732f0 2024-05-21T15:41:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.107+0000 7fa514ff9640 1 -- 172.21.15.38:0/2048574639 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510035040 con 0x55ea34e732f0 2024-05-21T15:41:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.108+0000 7fa52bc53740 1 -- 172.21.15.38:0/2048574639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 msgr2=0x55ea3523c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.108+0000 7fa51ed9d640 1 --2- 172.21.15.38:0/3628904785 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa50c0a2c30 0x7fa50c0a5140 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55ea3523d030 tx=0x7fa518014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.108+0000 7fa52bc53740 1 --2- 172.21.15.38:0/2048574639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea3523c3e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa510009900 tx=0x7fa510005d50 comp rx=0 tx=0).stop 2024-05-21T15:41:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.108+0000 7fa52bc53740 1 -- 172.21.15.38:0/2048574639 shutdown_connections 2024-05-21T15:41:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.108+0000 7fa52bc53740 1 --2- 172.21.15.38:0/2048574639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e7b4d0 0x55ea352432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.108+0000 7fa52bc53740 1 --2- 172.21.15.38:0/2048574639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea3523c3e0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.108+0000 7fa52bc53740 1 --2- 172.21.15.38:0/2048574639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e9c1b0 0x55ea3523be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.108+0000 7fa52bc53740 1 -- 172.21.15.38:0/2048574639 >> 172.21.15.38:0/2048574639 conn(0x55ea34e90300 msgr2=0x55ea352377f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.108+0000 7fa52bc53740 1 -- 172.21.15.38:0/2048574639 shutdown_connections 2024-05-21T15:41:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.108+0000 7fa52bc53740 1 -- 172.21.15.38:0/2048574639 wait complete. 2024-05-21T15:41:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.108+0000 7fa52bc53740 1 -- start start 2024-05-21T15:41:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.108+0000 7fa52bc53740 1 Processor -- start 2024-05-21T15:41:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa52bc53740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea35245b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa52bc53740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e7b4d0 0x55ea35246040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa52bc53740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e9c1b0 0x55ea3524cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa51e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea35245b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa52bc53740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ea35243cd0 con 0x55ea34e732f0 2024-05-21T15:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa51e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea35245b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42704/0 (socket says 172.21.15.38:42704) 2024-05-21T15:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa51e59c640 1 -- 172.21.15.38:0/4027289727 learned_addr learned my addr 172.21.15.38:0/4027289727 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa52bc53740 1 -- 172.21.15.38:0/4027289727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ea3515b790 con 0x55ea34e7b4d0 2024-05-21T15:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa52bc53740 1 -- 172.21.15.38:0/4027289727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ea35243e50 con 0x55ea34e9c1b0 2024-05-21T15:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa517fff640 1 --2- 172.21.15.38:0/4027289727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e7b4d0 0x55ea35246040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa51ed9d640 1 --2- 172.21.15.38:0/4027289727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e9c1b0 0x55ea3524cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa52bc53740 10 client.0 _ll_register_callbacks cb 0x55ea35196a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa517fff640 1 -- 172.21.15.38:0/4027289727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e9c1b0 msgr2=0x55ea3524cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa517fff640 1 --2- 172.21.15.38:0/4027289727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e9c1b0 0x55ea3524cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa517fff640 1 -- 172.21.15.38:0/4027289727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 msgr2=0x55ea35245b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa517fff640 1 --2- 172.21.15.38:0/4027289727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea35245b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.109+0000 7fa517fff640 1 -- 172.21.15.38:0/4027289727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea34eb4210 con 0x55ea34e7b4d0 2024-05-21T15:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa517fff640 1 --2- 172.21.15.38:0/4027289727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e7b4d0 0x55ea35246040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa508000c00 tx=0x7fa508014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daad65aa71291395 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa505ffb640 1 -- 172.21.15.38:0/4027289727 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508018c70 con 0x55ea34e7b4d0 2024-05-21T15:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa505ffb640 1 -- 172.21.15.38:0/4027289727 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa508010550 con 0x55ea34e7b4d0 2024-05-21T15:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa52bc53740 1 -- 172.21.15.38:0/4027289727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa510031970 con 0x55ea34e7b4d0 2024-05-21T15:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa505ffb640 1 -- 172.21.15.38:0/4027289727 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa5080ba050 con 0x55ea34e7b4d0 2024-05-21T15:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa52bc53740 10 client.27059 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa52bc53740 1 -- 172.21.15.38:0/4027289727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ea34f80620 con 0x55ea34e7b4d0 2024-05-21T15:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa505ffb640 1 -- 172.21.15.38:0/4027289727 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa510031970 con 0x55ea34e7b4d0 2024-05-21T15:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa505ffb640 1 -- 172.21.15.38:0/4027289727 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fa50809c160 con 0x55ea34e7b4d0 2024-05-21T15:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa52bc53740 10 client.27059 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa52bc53740 10 client.27059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa52bc53740 20 client.27059 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa52bc53740 1 --2- 172.21.15.38:0/4027289727 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ea34e5ad80 0x55ea352534c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa52bc53740 4 client.27059 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.110+0000 7fa52bc53740 1 -- 172.21.15.38:0/4027289727 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea351ada70 con 0x55ea34e5ad80 2024-05-21T15:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.111+0000 7fa51e59c640 1 --2- 172.21.15.38:0/4027289727 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ea34e5ad80 0x55ea352534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.111+0000 7fa505ffb640 10 client.27059 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.111+0000 7fa51e59c640 1 --2- 172.21.15.38:0/4027289727 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ea34e5ad80 0x55ea352534c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e37e763b7922417 server_cookie=bf99e374a1e3708c in_seq=0 out_seq=0 2024-05-21T15:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.112+0000 7fa505ffb640 1 -- 172.21.15.38:0/4027289727 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ea351ada70 con 0x55ea34e5ad80 2024-05-21T15:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.112+0000 7fa505ffb640 10 client.27059 handle_command_reply: tid=0 2024-05-21T15:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.112+0000 7fa52bc53740 1 client.27059 shutdown 2024-05-21T15:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.113+0000 7fa52bc53740 1 -- 172.21.15.38:0/4027289727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e7b4d0 msgr2=0x55ea35246040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.113+0000 7fa52bc53740 1 --2- 172.21.15.38:0/4027289727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e7b4d0 0x55ea35246040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa508000c00 tx=0x7fa508014000 comp rx=0 tx=0).stop 2024-05-21T15:41:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.113+0000 7fa52bc53740 1 -- 172.21.15.38:0/4027289727 shutdown_connections 2024-05-21T15:41:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.113+0000 7fa52bc53740 1 --2- 172.21.15.38:0/4027289727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e9c1b0 0x55ea3524cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.113+0000 7fa52bc53740 1 --2- 172.21.15.38:0/4027289727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e7b4d0 0x55ea35246040 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.113+0000 7fa52bc53740 1 --2- 172.21.15.38:0/4027289727 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ea34e5ad80 0x55ea352534c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.113+0000 7fa52bc53740 1 --2- 172.21.15.38:0/4027289727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea35245b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.113+0000 7fa52bc53740 1 -- 172.21.15.38:0/4027289727 >> 172.21.15.38:0/4027289727 conn(0x55ea34e90300 msgr2=0x55ea3523a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.113+0000 7fa505ffb640 0 client.27059 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.113+0000 7fa52bc53740 1 -- 172.21.15.38:0/4027289727 shutdown_connections 2024-05-21T15:41:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.113+0000 7fa52bc53740 1 -- 172.21.15.38:0/4027289727 wait complete. 2024-05-21T15:41:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.113+0000 7fa52bc53740 20 client.27059 trim_cache size 0 max 16384 2024-05-21T15:41:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.131+0000 7fa52bc53740 1 Processor -- start 2024-05-21T15:41:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.131+0000 7fa52bc53740 1 -- start start 2024-05-21T15:41:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa52bc53740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea35244280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa52bc53740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e7b4d0 0x55ea352447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa52bc53740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e9c1b0 0x55ea35237130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa52bc53740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa50000e510 con 0x55ea34e732f0 2024-05-21T15:41:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa52bc53740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa500010050 con 0x55ea34e7b4d0 2024-05-21T15:41:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa52bc53740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa50000ebe0 con 0x55ea34e9c1b0 2024-05-21T15:41:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa51e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea35244280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa517fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e7b4d0 0x55ea352447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa51e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea35244280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42714/0 (socket says 172.21.15.38:42714) 2024-05-21T15:41:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa51e59c640 1 -- 172.21.15.38:0/925401961 learned_addr learned my addr 172.21.15.38:0/925401961 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa51ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e9c1b0 0x55ea35237130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa51e59c640 1 -- 172.21.15.38:0/925401961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e9c1b0 msgr2=0x55ea35237130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa51e59c640 1 --2- 172.21.15.38:0/925401961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e9c1b0 0x55ea35237130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa51e59c640 1 -- 172.21.15.38:0/925401961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e7b4d0 msgr2=0x55ea352447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa51e59c640 1 --2- 172.21.15.38:0/925401961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e7b4d0 0x55ea352447c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa51e59c640 1 -- 172.21.15.38:0/925401961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea352511c0 con 0x55ea34e732f0 2024-05-21T15:41:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa517fff640 1 --2- 172.21.15.38:0/925401961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e7b4d0 0x55ea352447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.132+0000 7fa51e59c640 1 --2- 172.21.15.38:0/925401961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea35244280 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa5100096c0 tx=0x7fa510000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3aa1381d739907 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.133+0000 7fa507fff640 1 -- 172.21.15.38:0/925401961 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510040040 con 0x55ea34e732f0 2024-05-21T15:41:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.133+0000 7fa507fff640 1 -- 172.21.15.38:0/925401961 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa510005500 con 0x55ea34e732f0 2024-05-21T15:41:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.133+0000 7fa507fff640 1 -- 172.21.15.38:0/925401961 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510035040 con 0x55ea34e732f0 2024-05-21T15:41:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.133+0000 7fa52bc53740 1 -- 172.21.15.38:0/925401961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 msgr2=0x55ea35244280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.133+0000 7fa52bc53740 1 --2- 172.21.15.38:0/925401961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea35244280 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa5100096c0 tx=0x7fa510000bc0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.134+0000 7fa52bc53740 1 -- 172.21.15.38:0/925401961 shutdown_connections 2024-05-21T15:41:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.134+0000 7fa52bc53740 1 --2- 172.21.15.38:0/925401961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e9c1b0 0x55ea35237130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.134+0000 7fa52bc53740 1 --2- 172.21.15.38:0/925401961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e7b4d0 0x55ea352447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.134+0000 7fa52bc53740 1 --2- 172.21.15.38:0/925401961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea35244280 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.134+0000 7fa52bc53740 1 -- 172.21.15.38:0/925401961 >> 172.21.15.38:0/925401961 conn(0x55ea34e90300 msgr2=0x55ea3523ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.134+0000 7fa52bc53740 1 -- 172.21.15.38:0/925401961 shutdown_connections 2024-05-21T15:41:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.134+0000 7fa52bc53740 1 -- 172.21.15.38:0/925401961 wait complete. 2024-05-21T15:41:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.134+0000 7fa52bc53740 1 -- start start 2024-05-21T15:41:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.134+0000 7fa52bc53740 1 Processor -- start 2024-05-21T15:41:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.134+0000 7fa52bc53740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea352492f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.134+0000 7fa52bc53740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e7b4d0 0x55ea35249830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.134+0000 7fa52bc53740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e9c1b0 0x55ea352b9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.134+0000 7fa52bc53740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa50000f980 con 0x55ea34e732f0 2024-05-21T15:41:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa52bc53740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa50000f2b0 con 0x55ea34e9c1b0 2024-05-21T15:41:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa52bc53740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa50000c4f0 con 0x55ea34e7b4d0 2024-05-21T15:41:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa52bc53740 10 client.0 _ll_register_callbacks cb 0x7fa50800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa51e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea352492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa517fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e7b4d0 0x55ea35249830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa51e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea352492f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42724/0 (socket says 172.21.15.38:42724) 2024-05-21T15:41:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa51e59c640 1 -- 172.21.15.38:0/693306641 learned_addr learned my addr 172.21.15.38:0/693306641 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa51ed9d640 1 --2- 172.21.15.38:0/693306641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e9c1b0 0x55ea352b9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa51e59c640 1 -- 172.21.15.38:0/693306641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e7b4d0 msgr2=0x55ea35249830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa51e59c640 1 --2- 172.21.15.38:0/693306641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e7b4d0 0x55ea35249830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa51e59c640 1 -- 172.21.15.38:0/693306641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e9c1b0 msgr2=0x55ea352b9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa51e59c640 1 --2- 172.21.15.38:0/693306641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e9c1b0 0x55ea352b9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa51e59c640 1 -- 172.21.15.38:0/693306641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea3524c7f0 con 0x55ea34e732f0 2024-05-21T15:41:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa517fff640 1 --2- 172.21.15.38:0/693306641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e7b4d0 0x55ea35249830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa51e59c640 1 --2- 172.21.15.38:0/693306641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea352492f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa510009900 tx=0x7fa51002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b965d06ab13400a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.135+0000 7fa505ffb640 1 -- 172.21.15.38:0/693306641 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51005e060 con 0x55ea34e732f0 2024-05-21T15:41:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.136+0000 7fa52bc53740 1 -- 172.21.15.38:0/693306641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa510005500 con 0x55ea34e732f0 2024-05-21T15:41:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.136+0000 7fa505ffb640 1 -- 172.21.15.38:0/693306641 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa510005230 con 0x55ea34e732f0 2024-05-21T15:41:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.136+0000 7fa505ffb640 1 -- 172.21.15.38:0/693306641 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa510040070 con 0x55ea34e732f0 2024-05-21T15:41:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.136+0000 7fa52bc53740 10 client.17451 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.136+0000 7fa52bc53740 1 -- 172.21.15.38:0/693306641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ea352bad00 con 0x55ea34e732f0 2024-05-21T15:41:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.136+0000 7fa505ffb640 1 -- 172.21.15.38:0/693306641 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa510005500 con 0x55ea34e732f0 2024-05-21T15:41:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.136+0000 7fa505ffb640 1 -- 172.21.15.38:0/693306641 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fa51004f960 con 0x55ea34e732f0 2024-05-21T15:41:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.136+0000 7fa52bc53740 10 client.17451 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.136+0000 7fa52bc53740 10 client.17451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.136+0000 7fa52bc53740 20 client.17451 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.136+0000 7fa52bc53740 1 --2- 172.21.15.38:0/693306641 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ea34e301c0 0x55ea352bd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.136+0000 7fa52bc53740 4 client.17451 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:41:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.136+0000 7fa52bc53740 1 -- 172.21.15.38:0/693306641 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea351a7dc0 con 0x55ea34e301c0 2024-05-21T15:41:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.137+0000 7fa517fff640 1 --2- 172.21.15.38:0/693306641 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ea34e301c0 0x55ea352bd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.137+0000 7fa517fff640 1 --2- 172.21.15.38:0/693306641 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ea34e301c0 0x55ea352bd2c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c68f9f92e4db03 server_cookie=5a8c78e90e7b91d4 in_seq=0 out_seq=0 2024-05-21T15:41:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.137+0000 7fa505ffb640 10 client.17451 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.154+0000 7fa505ffb640 1 -- 172.21.15.38:0/693306641 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ea351a7dc0 con 0x55ea34e301c0 2024-05-21T15:41:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.154+0000 7fa505ffb640 10 client.17451 handle_command_reply: tid=0 2024-05-21T15:41:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.154+0000 7fa52bc53740 1 client.17451 shutdown 2024-05-21T15:41:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.154+0000 7fa52bc53740 1 -- 172.21.15.38:0/693306641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 msgr2=0x55ea352492f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.154+0000 7fa52bc53740 1 --2- 172.21.15.38:0/693306641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea352492f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa510009900 tx=0x7fa51002a410 comp rx=0 tx=0).stop 2024-05-21T15:41:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.154+0000 7fa52bc53740 1 -- 172.21.15.38:0/693306641 shutdown_connections 2024-05-21T15:41:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.154+0000 7fa52bc53740 1 --2- 172.21.15.38:0/693306641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea34e9c1b0 0x55ea352b9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.154+0000 7fa52bc53740 1 --2- 172.21.15.38:0/693306641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea34e7b4d0 0x55ea35249830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.154+0000 7fa52bc53740 1 --2- 172.21.15.38:0/693306641 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ea34e301c0 0x55ea352bd2c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.154+0000 7fa52bc53740 1 --2- 172.21.15.38:0/693306641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ea34e732f0 0x55ea352492f0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.154+0000 7fa52bc53740 1 -- 172.21.15.38:0/693306641 >> 172.21.15.38:0/693306641 conn(0x55ea34e90300 msgr2=0x55ea35244280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.154+0000 7fa505ffb640 0 client.17451 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.154+0000 7fa52bc53740 1 -- 172.21.15.38:0/693306641 shutdown_connections 2024-05-21T15:41:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.155+0000 7fa52bc53740 1 -- 172.21.15.38:0/693306641 wait complete. 2024-05-21T15:41:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.155+0000 7fa52bc53740 20 client.17451 trim_cache size 0 max 16384 2024-05-21T15:41:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:41:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:41:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:41:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:41:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.155+0000 7fa524daa640 1 -- 172.21.15.38:0/3628904785 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa50c0a2c30 msgr2=0x7fa50c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.155+0000 7fa524daa640 1 --2- 172.21.15.38:0/3628904785 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa50c0a2c30 0x7fa50c0a5140 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55ea3523d030 tx=0x7fa518014040 comp rx=0 tx=0).stop 2024-05-21T15:41:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.155+0000 7fa524daa640 1 -- 172.21.15.38:0/3628904785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa520103d10 msgr2=0x7fa520073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.155+0000 7fa524daa640 1 --2- 172.21.15.38:0/3628904785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa520103d10 0x7fa520073ea0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa50800e9f0 tx=0x7fa50800eeb0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.156+0000 7fa524daa640 1 -- 172.21.15.38:0/3628904785 shutdown_connections 2024-05-21T15:41:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.156+0000 7fa524daa640 1 --2- 172.21.15.38:0/3628904785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa520108a10 0x7fa5200743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.156+0000 7fa524daa640 1 --2- 172.21.15.38:0/3628904785 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa50c0a2c30 0x7fa50c0a5140 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.156+0000 7fa524daa640 1 --2- 172.21.15.38:0/3628904785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa520103d10 0x7fa520073ea0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.156+0000 7fa524daa640 1 --2- 172.21.15.38:0/3628904785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa520102ad0 0x7fa520075800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.156+0000 7fa524daa640 1 -- 172.21.15.38:0/3628904785 >> 172.21.15.38:0/3628904785 conn(0x7fa5200fdce0 msgr2=0x7fa5201049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.156+0000 7fa524daa640 1 -- 172.21.15.38:0/3628904785 shutdown_connections 2024-05-21T15:41:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:36.156+0000 7fa524daa640 1 -- 172.21.15.38:0/3628904785 wait complete. 2024-05-21T15:41:36.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:35 smithi053 ceph-mon[36496]: pgmap v566: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 26 MiB/s wr, 5.66k op/s 2024-05-21T15:41:38.005 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:37 smithi053 ceph-mon[36496]: pgmap v567: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 MiB/s wr, 6.42k op/s 2024-05-21T15:41:38.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:37 smithi045 ceph-mon[36409]: pgmap v567: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 MiB/s wr, 6.42k op/s 2024-05-21T15:41:38.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:37 smithi038 ceph-mon[31995]: pgmap v567: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 MiB/s wr, 6.42k op/s 2024-05-21T15:41:40.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:39 smithi045 ceph-mon[36409]: pgmap v568: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 23 MiB/s wr, 5.05k op/s 2024-05-21T15:41:40.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:39 smithi038 ceph-mon[31995]: pgmap v568: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 23 MiB/s wr, 5.05k op/s 2024-05-21T15:41:40.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:39 smithi053 ceph-mon[36496]: pgmap v568: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 23 MiB/s wr, 5.05k op/s 2024-05-21T15:41:42.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:41 smithi045 ceph-mon[36409]: pgmap v569: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 23 MiB/s wr, 4.93k op/s 2024-05-21T15:41:42.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:41 smithi038 ceph-mon[31995]: pgmap v569: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 23 MiB/s wr, 4.93k op/s 2024-05-21T15:41:42.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:41 smithi053 ceph-mon[36496]: pgmap v569: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 23 MiB/s wr, 4.93k op/s 2024-05-21T15:41:44.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:43 smithi045 ceph-mon[36409]: pgmap v570: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 25 MiB/s wr, 5.53k op/s 2024-05-21T15:41:44.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:43 smithi038 ceph-mon[31995]: pgmap v570: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 25 MiB/s wr, 5.53k op/s 2024-05-21T15:41:44.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:43 smithi053 ceph-mon[36496]: pgmap v570: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 25 MiB/s wr, 5.53k op/s 2024-05-21T15:41:46.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:45 smithi045 ceph-mon[36409]: pgmap v571: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 3.98k op/s 2024-05-21T15:41:46.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:45 smithi038 ceph-mon[31995]: pgmap v571: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 3.98k op/s 2024-05-21T15:41:46.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:45 smithi053 ceph-mon[36496]: pgmap v571: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 3.98k op/s 2024-05-21T15:41:48.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:47 smithi045 ceph-mon[36409]: pgmap v572: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 3.39k op/s 2024-05-21T15:41:48.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:47 smithi038 ceph-mon[31995]: pgmap v572: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 3.39k op/s 2024-05-21T15:41:48.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:47 smithi053 ceph-mon[36496]: pgmap v572: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 3.39k op/s 2024-05-21T15:41:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.272+0000 7f8702b3e640 1 -- 172.21.15.38:0/1917866394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86fc1019d0 msgr2=0x7f86fc1026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.272+0000 7f8702b3e640 1 --2- 172.21.15.38:0/1917866394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86fc1019d0 0x7f86fc1026d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009900 tx=0x7f86f002a410 comp rx=0 tx=0).stop 2024-05-21T15:41:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.273+0000 7f8702b3e640 1 -- 172.21.15.38:0/1917866394 shutdown_connections 2024-05-21T15:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.273+0000 7f8702b3e640 1 --2- 172.21.15.38:0/1917866394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86fc107910 0x7f86fc109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.273+0000 7f8702b3e640 1 --2- 172.21.15.38:0/1917866394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86fc102c10 0x7f86fc103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.273+0000 7f8702b3e640 1 --2- 172.21.15.38:0/1917866394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86fc1019d0 0x7f86fc1026d0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.273+0000 7f8702b3e640 1 -- 172.21.15.38:0/1917866394 >> 172.21.15.38:0/1917866394 conn(0x7f86fc0fcbc0 msgr2=0x7f86fc0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.273+0000 7f8702b3e640 1 -- 172.21.15.38:0/1917866394 shutdown_connections 2024-05-21T15:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.273+0000 7f8702b3e640 1 -- 172.21.15.38:0/1917866394 wait complete. 2024-05-21T15:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.274+0000 7f8702b3e640 1 Processor -- start 2024-05-21T15:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.274+0000 7f8702b3e640 1 -- start start 2024-05-21T15:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.274+0000 7f8702b3e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86fc1019d0 0x7f86fc199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.274+0000 7f8702b3e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86fc102c10 0x7f86fc199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.274+0000 7f8702b3e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86fc107910 0x7f86fc1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.274+0000 7f8702b3e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc10d670 con 0x7f86fc107910 2024-05-21T15:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.274+0000 7f8702b3e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc10d4f0 con 0x7f86fc102c10 2024-05-21T15:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.274+0000 7f8702b3e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc10d7f0 con 0x7f86fc1019d0 2024-05-21T15:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.274+0000 7f87008da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86fc1019d0 0x7f86fc199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.274+0000 7f86fbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86fc102c10 0x7f86fc199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.274+0000 7f87008da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86fc1019d0 0x7f86fc199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46834/0 (socket says 172.21.15.38:46834) 2024-05-21T15:41:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.274+0000 7f86fbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86fc102c10 0x7f86fc199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:57528/0 (socket says 172.21.15.38:57528) 2024-05-21T15:41:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.274+0000 7f87008da640 1 -- 172.21.15.38:0/1739013413 learned_addr learned my addr 172.21.15.38:0/1739013413 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.274+0000 7f87010db640 1 --2- 172.21.15.38:0/1739013413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86fc107910 0x7f86fc1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.275+0000 7f87008da640 1 -- 172.21.15.38:0/1739013413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86fc102c10 msgr2=0x7f86fc199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.275+0000 7f87008da640 1 --2- 172.21.15.38:0/1739013413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86fc102c10 0x7f86fc199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.275+0000 7f87008da640 1 -- 172.21.15.38:0/1739013413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86fc107910 msgr2=0x7f86fc1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.275+0000 7f87008da640 1 --2- 172.21.15.38:0/1739013413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86fc107910 0x7f86fc1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.275+0000 7f87008da640 1 -- 172.21.15.38:0/1739013413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc1a0d50 con 0x7f86fc1019d0 2024-05-21T15:41:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.275+0000 7f86fbfff640 1 --2- 172.21.15.38:0/1739013413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86fc102c10 0x7f86fc199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.275+0000 7f87008da640 1 --2- 172.21.15.38:0/1739013413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86fc1019d0 0x7f86fc199210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f86f0031440 tx=0x7f86f0032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.275+0000 7f86f9ffb640 1 -- 172.21.15.38:0/1739013413 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f003f030 con 0x7f86fc1019d0 2024-05-21T15:41:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.275+0000 7f8702b3e640 1 -- 172.21.15.38:0/1739013413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86fc1a0fe0 con 0x7f86fc1019d0 2024-05-21T15:41:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.275+0000 7f86f9ffb640 1 -- 172.21.15.38:0/1739013413 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f0003ce0 con 0x7f86fc1019d0 2024-05-21T15:41:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.275+0000 7f86f9ffb640 1 -- 172.21.15.38:0/1739013413 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0035070 con 0x7f86fc1019d0 2024-05-21T15:41:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.275+0000 7f8702b3e640 1 -- 172.21.15.38:0/1739013413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86fc1a1440 con 0x7f86fc1019d0 2024-05-21T15:41:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.278+0000 7f86f9ffb640 1 -- 172.21.15.38:0/1739013413 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f86f0042030 con 0x7f86fc1019d0 2024-05-21T15:41:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.279+0000 7f86f9ffb640 1 --2- 172.21.15.38:0/1739013413 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86ec0a2cb0 0x7f86ec0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.279+0000 7f87099ef740 1 Processor -- start 2024-05-21T15:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.279+0000 7f87099ef740 1 -- start start 2024-05-21T15:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.279+0000 7f86f9ffb640 1 -- 172.21.15.38:0/1739013413 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f86f00ef320 con 0x7f86fc1019d0 2024-05-21T15:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.279+0000 7f86fbfff640 1 --2- 172.21.15.38:0/1739013413 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86ec0a2cb0 0x7f86ec0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.279+0000 7f87099ef740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b10b01b0 0x55a2b144fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.279+0000 7f87099ef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10872f0 0x55a2b14503e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.279+0000 7f87099ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b108f4d0 0x55a2b14572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.279+0000 7f87099ef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a2b13b4340 con 0x55a2b10872f0 2024-05-21T15:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.279+0000 7f87099ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a2b135c290 con 0x55a2b108f4d0 2024-05-21T15:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.279+0000 7f87099ef740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a2b13b0300 con 0x55a2b10b01b0 2024-05-21T15:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.280+0000 7f86fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10872f0 0x55a2b14503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.280+0000 7f86fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10872f0 0x55a2b14503e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46910/0 (socket says 172.21.15.38:46910) 2024-05-21T15:41:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.280+0000 7f86fbfff640 1 -- 172.21.15.38:0/3679829885 learned_addr learned my addr 172.21.15.38:0/3679829885 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.280+0000 7f87008da640 1 --2- 172.21.15.38:0/3679829885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b10b01b0 0x55a2b144fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.280+0000 7f87010db640 1 --2- 172.21.15.38:0/3679829885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b108f4d0 0x55a2b14572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.280+0000 7f86fbfff640 1 -- 172.21.15.38:0/3679829885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b10b01b0 msgr2=0x55a2b144fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.280+0000 7f86fbfff640 1 --2- 172.21.15.38:0/3679829885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b10b01b0 0x55a2b144fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.280+0000 7f86fbfff640 1 -- 172.21.15.38:0/3679829885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b108f4d0 msgr2=0x55a2b14572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.280+0000 7f86fbfff640 1 --2- 172.21.15.38:0/3679829885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b108f4d0 0x55a2b14572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.280+0000 7f86fbfff640 1 -- 172.21.15.38:0/3679829885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2b13808e0 con 0x55a2b10872f0 2024-05-21T15:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.280+0000 7f87010db640 1 --2- 172.21.15.38:0/3679829885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b108f4d0 0x55a2b14572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.280+0000 7f87008da640 1 --2- 172.21.15.38:0/3679829885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b10b01b0 0x55a2b144fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.281+0000 7f86fbfff640 1 --2- 172.21.15.38:0/1739013413 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86ec0a2cb0 0x7f86ec0a51c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009fd0 tx=0x7f86e8009f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.281+0000 7f86fbfff640 1 --2- 172.21.15.38:0/3679829885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10872f0 0x55a2b14503e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f86e800f790 tx=0x7f86e8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec64df6a737fd8e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.281+0000 7f86e6ffd640 1 -- 172.21.15.38:0/3679829885 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8013170 con 0x55a2b10872f0 2024-05-21T15:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.281+0000 7f86e6ffd640 1 -- 172.21.15.38:0/3679829885 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e8020cc0 con 0x55a2b10872f0 2024-05-21T15:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.281+0000 7f86e6ffd640 1 -- 172.21.15.38:0/3679829885 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8010690 con 0x55a2b10872f0 2024-05-21T15:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.282+0000 7f87099ef740 1 -- 172.21.15.38:0/3679829885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10872f0 msgr2=0x55a2b14503e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.282+0000 7f87099ef740 1 --2- 172.21.15.38:0/3679829885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10872f0 0x55a2b14503e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f86e800f790 tx=0x7f86e8020000 comp rx=0 tx=0).stop 2024-05-21T15:41:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.282+0000 7f87099ef740 1 -- 172.21.15.38:0/3679829885 shutdown_connections 2024-05-21T15:41:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.282+0000 7f87099ef740 1 --2- 172.21.15.38:0/3679829885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b108f4d0 0x55a2b14572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.282+0000 7f87099ef740 1 --2- 172.21.15.38:0/3679829885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10872f0 0x55a2b14503e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.282+0000 7f87099ef740 1 --2- 172.21.15.38:0/3679829885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b10b01b0 0x55a2b144fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.282+0000 7f87099ef740 1 -- 172.21.15.38:0/3679829885 >> 172.21.15.38:0/3679829885 conn(0x55a2b10a4300 msgr2=0x55a2b144b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.282+0000 7f87099ef740 1 -- 172.21.15.38:0/3679829885 shutdown_connections 2024-05-21T15:41:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.282+0000 7f87099ef740 1 -- 172.21.15.38:0/3679829885 wait complete. 2024-05-21T15:41:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.283+0000 7f87099ef740 1 -- start start 2024-05-21T15:41:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.283+0000 7f87099ef740 1 Processor -- start 2024-05-21T15:41:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.283+0000 7f87099ef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10872f0 0x55a2b146a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.283+0000 7f87099ef740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b108f4d0 0x55a2b146af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.283+0000 7f87099ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10b01b0 0x55a2b1471e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.283+0000 7f87099ef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a2b1468c80 con 0x55a2b10872f0 2024-05-21T15:41:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.283+0000 7f87099ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a2b136f790 con 0x55a2b10b01b0 2024-05-21T15:41:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.283+0000 7f87099ef740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a2b1468e00 con 0x55a2b108f4d0 2024-05-21T15:41:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.283+0000 7f87008da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10872f0 0x55a2b146a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.283+0000 7f87099ef740 10 client.0 _ll_register_callbacks cb 0x55a2b13aaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.283+0000 7f86fbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b108f4d0 0x55a2b146af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.283+0000 7f86fbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b108f4d0 0x55a2b146af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46852/0 (socket says 172.21.15.38:46852) 2024-05-21T15:41:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.283+0000 7f87008da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10872f0 0x55a2b146a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46918/0 (socket says 172.21.15.38:46918) 2024-05-21T15:41:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.283+0000 7f86fbfff640 1 -- 172.21.15.38:0/3460715218 learned_addr learned my addr 172.21.15.38:0/3460715218 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.284+0000 7f87010db640 1 --2- 172.21.15.38:0/3460715218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10b01b0 0x55a2b1471e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.284+0000 7f86fbfff640 1 -- 172.21.15.38:0/3460715218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10b01b0 msgr2=0x55a2b1471e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.284+0000 7f86fbfff640 1 --2- 172.21.15.38:0/3460715218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10b01b0 0x55a2b1471e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.284+0000 7f86fbfff640 1 -- 172.21.15.38:0/3460715218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10872f0 msgr2=0x55a2b146a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.284+0000 7f86fbfff640 1 --2- 172.21.15.38:0/3460715218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10872f0 0x55a2b146a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.284+0000 7f86fbfff640 1 -- 172.21.15.38:0/3460715218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2b10c8210 con 0x55a2b108f4d0 2024-05-21T15:41:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.284+0000 7f86fbfff640 1 --2- 172.21.15.38:0/3460715218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b108f4d0 0x55a2b146af30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f86e8005bf0 tx=0x7f86e800ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c576a9224cdca4dd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.284+0000 7f86dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:41:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.284+0000 7f86dbfff640 1 -- 172.21.15.38:0/3460715218 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8033060 con 0x55a2b108f4d0 2024-05-21T15:41:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.284+0000 7f86dbfff640 1 -- 172.21.15.38:0/3460715218 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f86e8005070 con 0x55a2b108f4d0 2024-05-21T15:41:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.284+0000 7f87099ef740 1 -- 172.21.15.38:0/3460715218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86e8020cc0 con 0x55a2b108f4d0 2024-05-21T15:41:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.285+0000 7f86dbfff640 1 -- 172.21.15.38:0/3460715218 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f86e8005270 con 0x55a2b108f4d0 2024-05-21T15:41:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.285+0000 7f87099ef740 10 client.27160 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.285+0000 7f87099ef740 1 -- 172.21.15.38:0/3460715218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a2b1194620 con 0x55a2b108f4d0 2024-05-21T15:41:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.285+0000 7f86dbfff640 1 -- 172.21.15.38:0/3460715218 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e8020cc0 con 0x55a2b108f4d0 2024-05-21T15:41:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.285+0000 7f86dbfff640 1 -- 172.21.15.38:0/3460715218 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f86e801f070 con 0x55a2b108f4d0 2024-05-21T15:41:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.285+0000 7f87099ef740 10 client.27160 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.285+0000 7f87099ef740 10 client.27160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.285+0000 7f87099ef740 20 client.27160 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.285+0000 7f87099ef740 1 --2- 172.21.15.38:0/3460715218 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2b106ed80 0x55a2b14780c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.285+0000 7f87099ef740 4 client.27160 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.285+0000 7f87099ef740 1 -- 172.21.15.38:0/3460715218 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2b13c1a70 con 0x55a2b106ed80 2024-05-21T15:41:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.286+0000 7f87008da640 1 --2- 172.21.15.38:0/3460715218 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2b106ed80 0x55a2b14780c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.286+0000 7f87008da640 1 --2- 172.21.15.38:0/3460715218 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2b106ed80 0x55a2b14780c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6575077136a4a9fa server_cookie=b6b8acf951e8753d in_seq=0 out_seq=0 2024-05-21T15:41:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.286+0000 7f86dbfff640 10 client.27160 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f86dbfff640 1 -- 172.21.15.38:0/3460715218 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a2b13c1a70 con 0x55a2b106ed80 2024-05-21T15:41:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f86dbfff640 10 client.27160 handle_command_reply: tid=0 2024-05-21T15:41:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f87099ef740 1 client.27160 shutdown 2024-05-21T15:41:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f87099ef740 1 -- 172.21.15.38:0/3460715218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b108f4d0 msgr2=0x55a2b146af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f87099ef740 1 --2- 172.21.15.38:0/3460715218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b108f4d0 0x55a2b146af30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f86e8005bf0 tx=0x7f86e800ff70 comp rx=0 tx=0).stop 2024-05-21T15:41:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f87099ef740 1 -- 172.21.15.38:0/3460715218 shutdown_connections 2024-05-21T15:41:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f87099ef740 1 --2- 172.21.15.38:0/3460715218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10b01b0 0x55a2b1471e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f87099ef740 1 --2- 172.21.15.38:0/3460715218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b108f4d0 0x55a2b146af30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f87099ef740 1 --2- 172.21.15.38:0/3460715218 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2b106ed80 0x55a2b14780c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f87099ef740 1 --2- 172.21.15.38:0/3460715218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10872f0 0x55a2b146a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f87099ef740 1 -- 172.21.15.38:0/3460715218 >> 172.21.15.38:0/3460715218 conn(0x55a2b10a4300 msgr2=0x55a2b144e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f86dbfff640 0 client.27160 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f87099ef740 1 -- 172.21.15.38:0/3460715218 shutdown_connections 2024-05-21T15:41:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f87099ef740 1 -- 172.21.15.38:0/3460715218 wait complete. 2024-05-21T15:41:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.287+0000 7f87099ef740 20 client.27160 trim_cache size 0 max 16384 2024-05-21T15:41:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.303+0000 7f87099ef740 1 Processor -- start 2024-05-21T15:41:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.303+0000 7f87099ef740 1 -- start start 2024-05-21T15:41:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f87099ef740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b10872f0 0x55a2b144eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f87099ef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b108f4d0 0x55a2b1471db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f87099ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10b01b0 0x55a2b14722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f87099ef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc072050 con 0x55a2b108f4d0 2024-05-21T15:41:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f87099ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc073b90 con 0x55a2b10b01b0 2024-05-21T15:41:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f87099ef740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc072720 con 0x55a2b10872f0 2024-05-21T15:41:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f86fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b108f4d0 0x55a2b1471db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f86fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b108f4d0 0x55a2b1471db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46932/0 (socket says 172.21.15.38:46932) 2024-05-21T15:41:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f86fbfff640 1 -- 172.21.15.38:0/1693318177 learned_addr learned my addr 172.21.15.38:0/1693318177 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f87010db640 1 --2- 172.21.15.38:0/1693318177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10b01b0 0x55a2b14722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f87008da640 1 --2- 172.21.15.38:0/1693318177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b10872f0 0x55a2b144eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f86fbfff640 1 -- 172.21.15.38:0/1693318177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b10872f0 msgr2=0x55a2b144eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f86fbfff640 1 --2- 172.21.15.38:0/1693318177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b10872f0 0x55a2b144eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f86fbfff640 1 -- 172.21.15.38:0/1693318177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10b01b0 msgr2=0x55a2b14722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f86fbfff640 1 --2- 172.21.15.38:0/1693318177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10b01b0 0x55a2b14722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f86fbfff640 1 -- 172.21.15.38:0/1693318177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2b1446bc0 con 0x55a2b108f4d0 2024-05-21T15:41:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f87010db640 1 --2- 172.21.15.38:0/1693318177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10b01b0 0x55a2b14722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.304+0000 7f87008da640 1 --2- 172.21.15.38:0/1693318177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b10872f0 0x55a2b144eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.305+0000 7f86fbfff640 1 --2- 172.21.15.38:0/1693318177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b108f4d0 0x55a2b1471db0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f86e8025a40 tx=0x7f86e8025d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bd5608e1c2d71f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.305+0000 7f86e67fc640 1 -- 172.21.15.38:0/1693318177 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8033070 con 0x55a2b108f4d0 2024-05-21T15:41:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.305+0000 7f86e67fc640 1 -- 172.21.15.38:0/1693318177 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e8025e50 con 0x55a2b108f4d0 2024-05-21T15:41:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.305+0000 7f86e67fc640 1 -- 172.21.15.38:0/1693318177 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8039690 con 0x55a2b108f4d0 2024-05-21T15:41:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.306+0000 7f87099ef740 1 -- 172.21.15.38:0/1693318177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b108f4d0 msgr2=0x55a2b1471db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.306+0000 7f87099ef740 1 --2- 172.21.15.38:0/1693318177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b108f4d0 0x55a2b1471db0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f86e8025a40 tx=0x7f86e8025d20 comp rx=0 tx=0).stop 2024-05-21T15:41:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.306+0000 7f87099ef740 1 -- 172.21.15.38:0/1693318177 shutdown_connections 2024-05-21T15:41:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.306+0000 7f87099ef740 1 --2- 172.21.15.38:0/1693318177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10b01b0 0x55a2b14722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.306+0000 7f87099ef740 1 --2- 172.21.15.38:0/1693318177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b108f4d0 0x55a2b1471db0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.306+0000 7f87099ef740 1 --2- 172.21.15.38:0/1693318177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b10872f0 0x55a2b144eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.306+0000 7f87099ef740 1 -- 172.21.15.38:0/1693318177 >> 172.21.15.38:0/1693318177 conn(0x55a2b10a4300 msgr2=0x55a2b144e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.306+0000 7f87099ef740 1 -- 172.21.15.38:0/1693318177 shutdown_connections 2024-05-21T15:41:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.307+0000 7f87099ef740 1 -- 172.21.15.38:0/1693318177 wait complete. 2024-05-21T15:41:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.307+0000 7f87099ef740 1 -- start start 2024-05-21T15:41:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.307+0000 7f87099ef740 1 Processor -- start 2024-05-21T15:41:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.307+0000 7f87099ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10872f0 0x55a2b146f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.307+0000 7f87099ef740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b108f4d0 0x55a2b146fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f87099ef740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10b01b0 0x55a2b1470d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f87099ef740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc0734c0 con 0x55a2b10b01b0 2024-05-21T15:41:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f87099ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc072df0 con 0x55a2b10872f0 2024-05-21T15:41:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f87099ef740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc070030 con 0x55a2b108f4d0 2024-05-21T15:41:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f87099ef740 10 client.0 _ll_register_callbacks cb 0x7f86e800fc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f86fbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b108f4d0 0x55a2b146fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f86fbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b108f4d0 0x55a2b146fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:46884/0 (socket says 172.21.15.38:46884) 2024-05-21T15:41:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f86fbfff640 1 -- 172.21.15.38:0/4150580576 learned_addr learned my addr 172.21.15.38:0/4150580576 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f87010db640 1 --2- 172.21.15.38:0/4150580576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10b01b0 0x55a2b1470d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f87008da640 1 --2- 172.21.15.38:0/4150580576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10872f0 0x55a2b146f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f86fbfff640 1 -- 172.21.15.38:0/4150580576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10872f0 msgr2=0x55a2b146f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f86fbfff640 1 --2- 172.21.15.38:0/4150580576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10872f0 0x55a2b146f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f86fbfff640 1 -- 172.21.15.38:0/4150580576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10b01b0 msgr2=0x55a2b1470d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f86fbfff640 1 --2- 172.21.15.38:0/4150580576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10b01b0 0x55a2b1470d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f86fbfff640 1 -- 172.21.15.38:0/4150580576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2b1471340 con 0x55a2b108f4d0 2024-05-21T15:41:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f86fbfff640 1 --2- 172.21.15.38:0/4150580576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b108f4d0 0x55a2b146fea0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f86e8000c00 tx=0x7f86e800f870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa008b1a0b5a5617 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f86dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:41:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.308+0000 7f86dbfff640 1 -- 172.21.15.38:0/4150580576 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8033030 con 0x55a2b108f4d0 2024-05-21T15:41:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.309+0000 7f87099ef740 1 -- 172.21.15.38:0/4150580576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86e8025e50 con 0x55a2b108f4d0 2024-05-21T15:41:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.309+0000 7f86dbfff640 1 -- 172.21.15.38:0/4150580576 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f86e80055d0 con 0x55a2b108f4d0 2024-05-21T15:41:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.309+0000 7f86dbfff640 1 -- 172.21.15.38:0/4150580576 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f86e8025260 con 0x55a2b108f4d0 2024-05-21T15:41:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.309+0000 7f87099ef740 10 client.27166 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.309+0000 7f87099ef740 1 -- 172.21.15.38:0/4150580576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a2b14725d0 con 0x55a2b108f4d0 2024-05-21T15:41:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.309+0000 7f86dbfff640 1 -- 172.21.15.38:0/4150580576 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e8025e50 con 0x55a2b108f4d0 2024-05-21T15:41:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.309+0000 7f86dbfff640 1 -- 172.21.15.38:0/4150580576 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f86e801f070 con 0x55a2b108f4d0 2024-05-21T15:41:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.309+0000 7f87099ef740 10 client.27166 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.309+0000 7f87099ef740 10 client.27166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.309+0000 7f87099ef740 20 client.27166 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.309+0000 7f87099ef740 1 --2- 172.21.15.38:0/4150580576 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2b10441c0 0x55a2b14e27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.309+0000 7f87099ef740 4 client.27166 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:41:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.309+0000 7f87099ef740 1 -- 172.21.15.38:0/4150580576 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2b13bbdc0 con 0x55a2b10441c0 2024-05-21T15:41:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.310+0000 7f87008da640 1 --2- 172.21.15.38:0/4150580576 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2b10441c0 0x55a2b14e27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.310+0000 7f86dbfff640 10 client.27166 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.311+0000 7f87008da640 1 --2- 172.21.15.38:0/4150580576 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2b10441c0 0x55a2b14e27c0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e03eb6786983ded7 server_cookie=fa0c295e80b32370 in_seq=0 out_seq=0 2024-05-21T15:41:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f86dbfff640 1 -- 172.21.15.38:0/4150580576 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2b13bbdc0 con 0x55a2b10441c0 2024-05-21T15:41:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f86dbfff640 10 client.27166 handle_command_reply: tid=0 2024-05-21T15:41:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f87099ef740 1 client.27166 shutdown 2024-05-21T15:41:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f87099ef740 1 -- 172.21.15.38:0/4150580576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b108f4d0 msgr2=0x55a2b146fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f87099ef740 1 --2- 172.21.15.38:0/4150580576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b108f4d0 0x55a2b146fea0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f86e8000c00 tx=0x7f86e800f870 comp rx=0 tx=0).stop 2024-05-21T15:41:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f87099ef740 1 -- 172.21.15.38:0/4150580576 shutdown_connections 2024-05-21T15:41:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f87099ef740 1 --2- 172.21.15.38:0/4150580576 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55a2b10441c0 0x55a2b14e27c0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f87099ef740 1 --2- 172.21.15.38:0/4150580576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a2b10b01b0 0x55a2b1470d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f87099ef740 1 --2- 172.21.15.38:0/4150580576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b108f4d0 0x55a2b146fea0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f86dbfff640 0 client.27166 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f87099ef740 1 --2- 172.21.15.38:0/4150580576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2b10872f0 0x55a2b146f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f87099ef740 1 -- 172.21.15.38:0/4150580576 >> 172.21.15.38:0/4150580576 conn(0x55a2b10a4300 msgr2=0x55a2b147c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f87099ef740 1 -- 172.21.15.38:0/4150580576 shutdown_connections 2024-05-21T15:41:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f87099ef740 1 -- 172.21.15.38:0/4150580576 wait complete. 2024-05-21T15:41:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.328+0000 7f87099ef740 20 client.27166 trim_cache size 0 max 16384 2024-05-21T15:41:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:41:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:41:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:41:49.331 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:41:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.329+0000 7f8702b3e640 1 -- 172.21.15.38:0/1739013413 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86ec0a2cb0 msgr2=0x7f86ec0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.329+0000 7f8702b3e640 1 --2- 172.21.15.38:0/1739013413 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86ec0a2cb0 0x7f86ec0a51c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009fd0 tx=0x7f86e8009f40 comp rx=0 tx=0).stop 2024-05-21T15:41:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.329+0000 7f8702b3e640 1 -- 172.21.15.38:0/1739013413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86fc1019d0 msgr2=0x7f86fc199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.329+0000 7f8702b3e640 1 --2- 172.21.15.38:0/1739013413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86fc1019d0 0x7f86fc199210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f86f0031440 tx=0x7f86f0032ce0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.329+0000 7f8702b3e640 1 -- 172.21.15.38:0/1739013413 shutdown_connections 2024-05-21T15:41:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.329+0000 7f8702b3e640 1 --2- 172.21.15.38:0/1739013413 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f86ec0a2cb0 0x7f86ec0a51c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.329+0000 7f8702b3e640 1 --2- 172.21.15.38:0/1739013413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86fc107910 0x7f86fc1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.329+0000 7f8702b3e640 1 --2- 172.21.15.38:0/1739013413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86fc102c10 0x7f86fc199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.329+0000 7f8702b3e640 1 --2- 172.21.15.38:0/1739013413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86fc1019d0 0x7f86fc199210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.329+0000 7f8702b3e640 1 -- 172.21.15.38:0/1739013413 >> 172.21.15.38:0/1739013413 conn(0x7f86fc0fcbc0 msgr2=0x7f86fc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.329+0000 7f8702b3e640 1 -- 172.21.15.38:0/1739013413 shutdown_connections 2024-05-21T15:41:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:41:49.330+0000 7f8702b3e640 1 -- 172.21.15.38:0/1739013413 wait complete. 2024-05-21T15:41:50.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:49 smithi045 ceph-mon[36409]: pgmap v573: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 4.54k op/s 2024-05-21T15:41:50.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:49 smithi038 ceph-mon[31995]: pgmap v573: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 4.54k op/s 2024-05-21T15:41:50.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:49 smithi053 ceph-mon[36496]: pgmap v573: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 4.54k op/s 2024-05-21T15:41:52.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:51 smithi045 ceph-mon[36409]: pgmap v574: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 5.04k op/s 2024-05-21T15:41:52.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:51 smithi038 ceph-mon[31995]: pgmap v574: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 5.04k op/s 2024-05-21T15:41:52.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:51 smithi053 ceph-mon[36496]: pgmap v574: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 5.04k op/s 2024-05-21T15:41:54.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:53 smithi045 ceph-mon[36409]: pgmap v575: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 MiB/s wr, 6.84k op/s 2024-05-21T15:41:54.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:53 smithi038 ceph-mon[31995]: pgmap v575: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 MiB/s wr, 6.84k op/s 2024-05-21T15:41:54.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:53 smithi053 ceph-mon[36496]: pgmap v575: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 MiB/s wr, 6.84k op/s 2024-05-21T15:41:55.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:41:55.921 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:41:56.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:55 smithi045 ceph-mon[36409]: pgmap v576: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 5.46k op/s 2024-05-21T15:41:56.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:55 smithi038 ceph-mon[31995]: pgmap v576: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 5.46k op/s 2024-05-21T15:41:56.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.055+0000 7fc98acde640 1 -- 172.21.15.38:0/2435887516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc984106040 msgr2=0x7fc984108460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.055+0000 7fc98acde640 1 --2- 172.21.15.38:0/2435887516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc984106040 0x7fc984108460 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc970009900 tx=0x7fc97002a410 comp rx=0 tx=0).stop 2024-05-21T15:41:56.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.055+0000 7fc98acde640 1 -- 172.21.15.38:0/2435887516 shutdown_connections 2024-05-21T15:41:56.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.055+0000 7fc98acde640 1 --2- 172.21.15.38:0/2435887516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9841089a0 0x7fc98410adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.055+0000 7fc98acde640 1 --2- 172.21.15.38:0/2435887516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc984106040 0x7fc984108460 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.055+0000 7fc98acde640 1 --2- 172.21.15.38:0/2435887516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc984100e90 0x7fc984105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.055+0000 7fc98acde640 1 -- 172.21.15.38:0/2435887516 >> 172.21.15.38:0/2435887516 conn(0x7fc9840fc2f0 msgr2=0x7fc9840fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.055+0000 7fc98acde640 1 -- 172.21.15.38:0/2435887516 shutdown_connections 2024-05-21T15:41:56.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.055+0000 7fc98acde640 1 -- 172.21.15.38:0/2435887516 wait complete. 2024-05-21T15:41:56.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.056+0000 7fc98acde640 1 Processor -- start 2024-05-21T15:41:56.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.056+0000 7fc98acde640 1 -- start start 2024-05-21T15:41:56.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.056+0000 7fc98acde640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc984100e90 0x7fc984199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.056+0000 7fc98acde640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc984106040 0x7fc984199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.056+0000 7fc98acde640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841089a0 0x7fc9841a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.056+0000 7fc98acde640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc98410e770 con 0x7fc984106040 2024-05-21T15:41:56.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.056+0000 7fc98acde640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc98410e5f0 con 0x7fc9841089a0 2024-05-21T15:41:56.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.056+0000 7fc98acde640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc98410e8f0 con 0x7fc984100e90 2024-05-21T15:41:56.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.056+0000 7fc983fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc984106040 0x7fc984199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc983fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc984106040 0x7fc984199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40458/0 (socket says 172.21.15.38:40458) 2024-05-21T15:41:56.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc983fff640 1 -- 172.21.15.38:0/1722251688 learned_addr learned my addr 172.21.15.38:0/1722251688 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc988a7a640 1 --2- 172.21.15.38:0/1722251688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc984100e90 0x7fc984199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc98927b640 1 --2- 172.21.15.38:0/1722251688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841089a0 0x7fc9841a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc983fff640 1 -- 172.21.15.38:0/1722251688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc984100e90 msgr2=0x7fc984199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc983fff640 1 --2- 172.21.15.38:0/1722251688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc984100e90 0x7fc984199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc983fff640 1 -- 172.21.15.38:0/1722251688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841089a0 msgr2=0x7fc9841a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc983fff640 1 --2- 172.21.15.38:0/1722251688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841089a0 0x7fc9841a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc983fff640 1 -- 172.21.15.38:0/1722251688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9841a0e50 con 0x7fc984106040 2024-05-21T15:41:56.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc983fff640 1 --2- 172.21.15.38:0/1722251688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc984106040 0x7fc984199850 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc970006010 tx=0x7fc970031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc981ffb640 1 -- 172.21.15.38:0/1722251688 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97003f030 con 0x7fc984106040 2024-05-21T15:41:56.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc981ffb640 1 -- 172.21.15.38:0/1722251688 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc970003ad0 con 0x7fc984106040 2024-05-21T15:41:56.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc98acde640 1 -- 172.21.15.38:0/1722251688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9841a10e0 con 0x7fc984106040 2024-05-21T15:41:56.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc981ffb640 1 -- 172.21.15.38:0/1722251688 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970035070 con 0x7fc984106040 2024-05-21T15:41:56.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.057+0000 7fc98acde640 1 -- 172.21.15.38:0/1722251688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9841a15d0 con 0x7fc984106040 2024-05-21T15:41:56.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.060+0000 7fc981ffb640 1 -- 172.21.15.38:0/1722251688 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc970042030 con 0x7fc984106040 2024-05-21T15:41:56.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.060+0000 7fc991b80740 1 Processor -- start 2024-05-21T15:41:56.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.060+0000 7fc991b80740 1 -- start start 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.060+0000 7fc991b80740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d1fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc991b80740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 0x5557b6d20130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc991b80740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 0x5557b6d27030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc991b80740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5557b6c84d00 con 0x5557b6980390 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc991b80740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5557b6c2c4c0 con 0x5557b695f6b0 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc991b80740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5557b6c80cc0 con 0x5557b69574d0 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc98927b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d1fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc98927b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d1fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40464/0 (socket says 172.21.15.38:40464) 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc98927b640 1 -- 172.21.15.38:0/48654231 learned_addr learned my addr 172.21.15.38:0/48654231 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc981ffb640 1 --2- 172.21.15.38:0/1722251688 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc9740a2b90 0x7fc9740a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc988a7a640 1 --2- 172.21.15.38:0/48654231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 0x5557b6d20130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc983fff640 1 --2- 172.21.15.38:0/48654231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 0x5557b6d27030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc98927b640 1 --2- 172.21.15.38:0/1722251688 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc9740a2b90 0x7fc9740a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc988a7a640 1 -- 172.21.15.38:0/48654231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 msgr2=0x5557b6d27030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc988a7a640 1 --2- 172.21.15.38:0/48654231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 0x5557b6d27030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc988a7a640 1 -- 172.21.15.38:0/48654231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 msgr2=0x5557b6d1fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc988a7a640 1 --2- 172.21.15.38:0/48654231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d1fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc981ffb640 1 -- 172.21.15.38:0/1722251688 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fc9700ef680 con 0x7fc984106040 2024-05-21T15:41:56.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc988a7a640 1 -- 172.21.15.38:0/48654231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557b6a627d0 con 0x5557b69574d0 2024-05-21T15:41:56.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc988a7a640 1 --2- 172.21.15.38:0/48654231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 0x5557b6d20130 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc978009bf0 tx=0x7fc978011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28577c01edbf8b42 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc98927b640 1 --2- 172.21.15.38:0/1722251688 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc9740a2b90 0x7fc9740a50a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x5557b6d20d80 tx=0x7fc97c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.061+0000 7fc96effd640 1 -- 172.21.15.38:0/48654231 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97801e070 con 0x5557b69574d0 2024-05-21T15:41:56.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.062+0000 7fc96effd640 1 -- 172.21.15.38:0/48654231 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9780035f0 con 0x5557b69574d0 2024-05-21T15:41:56.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.062+0000 7fc96effd640 1 -- 172.21.15.38:0/48654231 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978019700 con 0x5557b69574d0 2024-05-21T15:41:56.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.062+0000 7fc991b80740 1 -- 172.21.15.38:0/48654231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 msgr2=0x5557b6d20130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.062+0000 7fc991b80740 1 --2- 172.21.15.38:0/48654231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 0x5557b6d20130 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc978009bf0 tx=0x7fc978011810 comp rx=0 tx=0).stop 2024-05-21T15:41:56.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.062+0000 7fc991b80740 1 -- 172.21.15.38:0/48654231 shutdown_connections 2024-05-21T15:41:56.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.062+0000 7fc991b80740 1 --2- 172.21.15.38:0/48654231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 0x5557b6d27030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.062+0000 7fc991b80740 1 --2- 172.21.15.38:0/48654231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 0x5557b6d20130 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.062+0000 7fc991b80740 1 --2- 172.21.15.38:0/48654231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d1fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.062+0000 7fc991b80740 1 -- 172.21.15.38:0/48654231 >> 172.21.15.38:0/48654231 conn(0x5557b69744e0 msgr2=0x5557b6d1b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc991b80740 1 -- 172.21.15.38:0/48654231 shutdown_connections 2024-05-21T15:41:56.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc991b80740 1 -- 172.21.15.38:0/48654231 wait complete. 2024-05-21T15:41:56.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc991b80740 1 -- start start 2024-05-21T15:41:56.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc991b80740 1 Processor -- start 2024-05-21T15:41:56.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc991b80740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b69574d0 0x5557b6d29b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc991b80740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b695f6b0 0x5557b6d2a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc991b80740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d30f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc991b80740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5557b6d27a80 con 0x5557b6980390 2024-05-21T15:41:56.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc991b80740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5557b6c3fb90 con 0x5557b69574d0 2024-05-21T15:41:56.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc991b80740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5557b6d27c00 con 0x5557b695f6b0 2024-05-21T15:41:56.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc991b80740 10 client.0 _ll_register_callbacks cb 0x5557b6c7bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:56.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc98927b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d30f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc988a7a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b69574d0 0x5557b6d29b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc98927b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d30f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40472/0 (socket says 172.21.15.38:40472) 2024-05-21T15:41:56.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc98927b640 1 -- 172.21.15.38:0/290557235 learned_addr learned my addr 172.21.15.38:0/290557235 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.063+0000 7fc983fff640 1 --2- 172.21.15.38:0/290557235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b695f6b0 0x5557b6d2a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.064+0000 7fc988a7a640 1 -- 172.21.15.38:0/290557235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b695f6b0 msgr2=0x5557b6d2a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.064+0000 7fc988a7a640 1 --2- 172.21.15.38:0/290557235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b695f6b0 0x5557b6d2a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.064+0000 7fc988a7a640 1 -- 172.21.15.38:0/290557235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 msgr2=0x5557b6d30f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.064+0000 7fc988a7a640 1 --2- 172.21.15.38:0/290557235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d30f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.064+0000 7fc988a7a640 1 -- 172.21.15.38:0/290557235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557b69983f0 con 0x5557b69574d0 2024-05-21T15:41:56.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.064+0000 7fc988a7a640 1 --2- 172.21.15.38:0/290557235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b69574d0 0x5557b6d29b40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc978003fd0 tx=0x7fc978009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88dabc81fdb35b80 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.064+0000 7fc963fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:41:56.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.064+0000 7fc963fff640 1 -- 172.21.15.38:0/290557235 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978029020 con 0x5557b69574d0 2024-05-21T15:41:56.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.064+0000 7fc963fff640 1 -- 172.21.15.38:0/290557235 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fc9780039a0 con 0x5557b69574d0 2024-05-21T15:41:56.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.064+0000 7fc991b80740 1 -- 172.21.15.38:0/290557235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9780035f0 con 0x5557b69574d0 2024-05-21T15:41:56.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.064+0000 7fc963fff640 1 -- 172.21.15.38:0/290557235 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc978034050 con 0x5557b69574d0 2024-05-21T15:41:56.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.064+0000 7fc991b80740 10 client.27095 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:56.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.064+0000 7fc991b80740 1 -- 172.21.15.38:0/290557235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5557b69580f0 con 0x5557b69574d0 2024-05-21T15:41:56.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.065+0000 7fc963fff640 1 -- 172.21.15.38:0/290557235 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9780035f0 con 0x5557b69574d0 2024-05-21T15:41:56.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.065+0000 7fc963fff640 1 -- 172.21.15.38:0/290557235 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fc978004bc0 con 0x5557b69574d0 2024-05-21T15:41:56.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.065+0000 7fc991b80740 10 client.27095 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:56.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.065+0000 7fc991b80740 10 client.27095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:56.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.065+0000 7fc991b80740 20 client.27095 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:56.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.065+0000 7fc991b80740 1 --2- 172.21.15.38:0/290557235 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5557b693ef60 0x5557b6d37530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.065+0000 7fc991b80740 4 client.27095 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:56.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.065+0000 7fc991b80740 1 -- 172.21.15.38:0/290557235 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557b6c92430 con 0x5557b693ef60 2024-05-21T15:41:56.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.065+0000 7fc983fff640 1 --2- 172.21.15.38:0/290557235 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5557b693ef60 0x5557b6d37530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.066+0000 7fc963fff640 10 client.27095 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:56.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.066+0000 7fc983fff640 1 --2- 172.21.15.38:0/290557235 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5557b693ef60 0x5557b6d37530 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e77da8023754b8d server_cookie=b38a33d69e7c84db in_seq=0 out_seq=0 2024-05-21T15:41:56.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.066+0000 7fc963fff640 1 -- 172.21.15.38:0/290557235 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5557b6c92430 con 0x5557b693ef60 2024-05-21T15:41:56.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.066+0000 7fc963fff640 10 client.27095 handle_command_reply: tid=0 2024-05-21T15:41:56.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.066+0000 7fc991b80740 1 client.27095 shutdown 2024-05-21T15:41:56.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.067+0000 7fc991b80740 1 -- 172.21.15.38:0/290557235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b69574d0 msgr2=0x5557b6d29b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.067+0000 7fc991b80740 1 --2- 172.21.15.38:0/290557235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b69574d0 0x5557b6d29b40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc978003fd0 tx=0x7fc978009fd0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.067+0000 7fc991b80740 1 -- 172.21.15.38:0/290557235 shutdown_connections 2024-05-21T15:41:56.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.067+0000 7fc991b80740 1 --2- 172.21.15.38:0/290557235 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5557b693ef60 0x5557b6d37530 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.067+0000 7fc991b80740 1 --2- 172.21.15.38:0/290557235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d30f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.067+0000 7fc991b80740 1 --2- 172.21.15.38:0/290557235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b695f6b0 0x5557b6d2a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.067+0000 7fc963fff640 0 client.27095 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:56.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.067+0000 7fc991b80740 1 --2- 172.21.15.38:0/290557235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b69574d0 0x5557b6d29b40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.067+0000 7fc991b80740 1 -- 172.21.15.38:0/290557235 >> 172.21.15.38:0/290557235 conn(0x5557b69744e0 msgr2=0x5557b6d1a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.067+0000 7fc991b80740 1 -- 172.21.15.38:0/290557235 shutdown_connections 2024-05-21T15:41:56.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.067+0000 7fc991b80740 1 -- 172.21.15.38:0/290557235 wait complete. 2024-05-21T15:41:56.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.067+0000 7fc991b80740 20 client.27095 trim_cache size 0 max 16384 2024-05-21T15:41:56.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc991b80740 1 Processor -- start 2024-05-21T15:41:56.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc991b80740 1 -- start start 2024-05-21T15:41:56.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc991b80740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 0x5557b6d287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc991b80740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 0x5557b6d30e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc991b80740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d1b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc991b80740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841aa3a0 con 0x5557b6980390 2024-05-21T15:41:56.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc991b80740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841abee0 con 0x5557b695f6b0 2024-05-21T15:41:56.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc991b80740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841aaa70 con 0x5557b69574d0 2024-05-21T15:41:56.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc98927b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d1b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc98927b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d1b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40474/0 (socket says 172.21.15.38:40474) 2024-05-21T15:41:56.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc983fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 0x5557b6d30e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc98927b640 1 -- 172.21.15.38:0/3462215589 learned_addr learned my addr 172.21.15.38:0/3462215589 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc988a7a640 1 --2- 172.21.15.38:0/3462215589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 0x5557b6d287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc98927b640 1 -- 172.21.15.38:0/3462215589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 msgr2=0x5557b6d287f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc98927b640 1 --2- 172.21.15.38:0/3462215589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 0x5557b6d287f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc98927b640 1 -- 172.21.15.38:0/3462215589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 msgr2=0x5557b6d30e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc98927b640 1 --2- 172.21.15.38:0/3462215589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 0x5557b6d30e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.086+0000 7fc98927b640 1 -- 172.21.15.38:0/3462215589 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557b6d15830 con 0x5557b6980390 2024-05-21T15:41:56.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.087+0000 7fc98927b640 1 --2- 172.21.15.38:0/3462215589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d1b400 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc97c011ae0 tx=0x7fc97c0099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e5bbfd379dd99a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.087+0000 7fc96e7fc640 1 -- 172.21.15.38:0/3462215589 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c019070 con 0x5557b6980390 2024-05-21T15:41:56.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.087+0000 7fc96e7fc640 1 -- 172.21.15.38:0/3462215589 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97c004d20 con 0x5557b6980390 2024-05-21T15:41:56.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.087+0000 7fc96e7fc640 1 -- 172.21.15.38:0/3462215589 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c010450 con 0x5557b6980390 2024-05-21T15:41:56.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.088+0000 7fc991b80740 1 -- 172.21.15.38:0/3462215589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 msgr2=0x5557b6d1b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.088+0000 7fc991b80740 1 --2- 172.21.15.38:0/3462215589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d1b400 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc97c011ae0 tx=0x7fc97c0099c0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.088+0000 7fc991b80740 1 -- 172.21.15.38:0/3462215589 shutdown_connections 2024-05-21T15:41:56.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.088+0000 7fc991b80740 1 --2- 172.21.15.38:0/3462215589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d1b400 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.088+0000 7fc991b80740 1 --2- 172.21.15.38:0/3462215589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 0x5557b6d30e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.088+0000 7fc991b80740 1 --2- 172.21.15.38:0/3462215589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 0x5557b6d287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.088+0000 7fc991b80740 1 -- 172.21.15.38:0/3462215589 >> 172.21.15.38:0/3462215589 conn(0x5557b69744e0 msgr2=0x5557b6d1acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.088+0000 7fc991b80740 1 -- 172.21.15.38:0/3462215589 shutdown_connections 2024-05-21T15:41:56.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.088+0000 7fc991b80740 1 -- 172.21.15.38:0/3462215589 wait complete. 2024-05-21T15:41:56.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.088+0000 7fc991b80740 1 -- start start 2024-05-21T15:41:56.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.088+0000 7fc991b80740 1 Processor -- start 2024-05-21T15:41:56.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc991b80740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 0x5557b6d2e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc991b80740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 0x5557b6d2e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc991b80740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d2f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc991b80740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841ab810 con 0x5557b6980390 2024-05-21T15:41:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc991b80740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841ab140 con 0x5557b695f6b0 2024-05-21T15:41:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc991b80740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841a8380 con 0x5557b69574d0 2024-05-21T15:41:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc991b80740 10 client.0 _ll_register_callbacks cb 0x7fc97801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc98927b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d2f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc988a7a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 0x5557b6d2e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc98927b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d2f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40476/0 (socket says 172.21.15.38:40476) 2024-05-21T15:41:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc98927b640 1 -- 172.21.15.38:0/3366712918 learned_addr learned my addr 172.21.15.38:0/3366712918 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc983fff640 1 --2- 172.21.15.38:0/3366712918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 0x5557b6d2e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc98927b640 1 -- 172.21.15.38:0/3366712918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 msgr2=0x5557b6d2e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc98927b640 1 --2- 172.21.15.38:0/3366712918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 0x5557b6d2e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc98927b640 1 -- 172.21.15.38:0/3366712918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 msgr2=0x5557b6d2e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc98927b640 1 --2- 172.21.15.38:0/3366712918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 0x5557b6d2e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc98927b640 1 -- 172.21.15.38:0/3366712918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557b6d2fc90 con 0x5557b6980390 2024-05-21T15:41:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc988a7a640 1 --2- 172.21.15.38:0/3366712918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 0x5557b6d2e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc983fff640 1 --2- 172.21.15.38:0/3366712918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 0x5557b6d2e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc98927b640 1 --2- 172.21.15.38:0/3366712918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d2f6d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc97c011fd0 tx=0x7fc97c005a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e1425f8c166507 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc963fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc963fff640 1 -- 172.21.15.38:0/3366712918 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c036020 con 0x5557b6980390 2024-05-21T15:41:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc963fff640 1 -- 172.21.15.38:0/3366712918 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fc97c021a90 con 0x5557b6980390 2024-05-21T15:41:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.089+0000 7fc991b80740 1 -- 172.21.15.38:0/3366712918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc97c004d20 con 0x5557b6980390 2024-05-21T15:41:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.090+0000 7fc963fff640 1 -- 172.21.15.38:0/3366712918 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc97c019050 con 0x5557b6980390 2024-05-21T15:41:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.090+0000 7fc991b80740 10 client.17487 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.090+0000 7fc991b80740 1 -- 172.21.15.38:0/3366712918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5557b6d30f20 con 0x5557b6980390 2024-05-21T15:41:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.090+0000 7fc963fff640 1 -- 172.21.15.38:0/3366712918 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97c004d20 con 0x5557b6980390 2024-05-21T15:41:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.090+0000 7fc963fff640 1 -- 172.21.15.38:0/3366712918 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fc97c031030 con 0x5557b6980390 2024-05-21T15:41:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.090+0000 7fc991b80740 10 client.17487 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:56.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.090+0000 7fc991b80740 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:56.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.090+0000 7fc991b80740 20 client.17487 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:56.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.090+0000 7fc991b80740 1 --2- 172.21.15.38:0/3366712918 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5557b69143a0 0x5557b6da1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.090+0000 7fc991b80740 4 client.17487 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:41:56.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.090+0000 7fc991b80740 1 -- 172.21.15.38:0/3366712918 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5557b6c8c780 con 0x5557b69143a0 2024-05-21T15:41:56.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.090+0000 7fc988a7a640 1 --2- 172.21.15.38:0/3366712918 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5557b69143a0 0x5557b6da1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc988a7a640 1 --2- 172.21.15.38:0/3366712918 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5557b69143a0 0x5557b6da1e80 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628b51e1c72dbd02 server_cookie=d3d78b1d0881772b in_seq=0 out_seq=0 2024-05-21T15:41:56.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc963fff640 10 client.17487 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:56.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc963fff640 1 -- 172.21.15.38:0/3366712918 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5557b6c8c780 con 0x5557b69143a0 2024-05-21T15:41:56.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc963fff640 10 client.17487 handle_command_reply: tid=0 2024-05-21T15:41:56.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc991b80740 1 client.17487 shutdown 2024-05-21T15:41:56.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc991b80740 1 -- 172.21.15.38:0/3366712918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 msgr2=0x5557b6d2f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc991b80740 1 --2- 172.21.15.38:0/3366712918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d2f6d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc97c011fd0 tx=0x7fc97c005a30 comp rx=0 tx=0).stop 2024-05-21T15:41:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc991b80740 1 -- 172.21.15.38:0/3366712918 shutdown_connections 2024-05-21T15:41:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc991b80740 1 --2- 172.21.15.38:0/3366712918 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x5557b69143a0 0x5557b6da1e80 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc991b80740 1 --2- 172.21.15.38:0/3366712918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5557b6980390 0x5557b6d2f6d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc991b80740 1 --2- 172.21.15.38:0/3366712918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557b695f6b0 0x5557b6d2e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc991b80740 1 --2- 172.21.15.38:0/3366712918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5557b69574d0 0x5557b6d2e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc991b80740 1 -- 172.21.15.38:0/3366712918 >> 172.21.15.38:0/3366712918 conn(0x5557b69744e0 msgr2=0x5557b6d1ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc963fff640 0 client.17487 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc991b80740 1 -- 172.21.15.38:0/3366712918 shutdown_connections 2024-05-21T15:41:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc991b80740 1 -- 172.21.15.38:0/3366712918 wait complete. 2024-05-21T15:41:56.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.091+0000 7fc991b80740 20 client.17487 trim_cache size 0 max 16384 2024-05-21T15:41:56.096 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:41:56.096 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-05-21T15:41:56.096 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-05-21T15:41:56.096 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:41:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.092+0000 7fc98acde640 1 -- 172.21.15.38:0/1722251688 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc9740a2b90 msgr2=0x7fc9740a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.092+0000 7fc98acde640 1 --2- 172.21.15.38:0/1722251688 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc9740a2b90 0x7fc9740a50a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x5557b6d20d80 tx=0x7fc97c004ab0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.092+0000 7fc98acde640 1 -- 172.21.15.38:0/1722251688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc984106040 msgr2=0x7fc984199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.092+0000 7fc98acde640 1 --2- 172.21.15.38:0/1722251688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc984106040 0x7fc984199850 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc970006010 tx=0x7fc970031b60 comp rx=0 tx=0).stop 2024-05-21T15:41:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.092+0000 7fc98acde640 1 -- 172.21.15.38:0/1722251688 shutdown_connections 2024-05-21T15:41:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.092+0000 7fc98acde640 1 --2- 172.21.15.38:0/1722251688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9841089a0 0x7fc9841a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.092+0000 7fc98acde640 1 --2- 172.21.15.38:0/1722251688 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fc9740a2b90 0x7fc9740a50a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.092+0000 7fc98acde640 1 --2- 172.21.15.38:0/1722251688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc984106040 0x7fc984199850 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.092+0000 7fc98acde640 1 --2- 172.21.15.38:0/1722251688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc984100e90 0x7fc984199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.092+0000 7fc98acde640 1 -- 172.21.15.38:0/1722251688 >> 172.21.15.38:0/1722251688 conn(0x7fc9840fc2f0 msgr2=0x7fc984104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.092+0000 7fc98acde640 1 -- 172.21.15.38:0/1722251688 shutdown_connections 2024-05-21T15:41:56.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.092+0000 7fc98acde640 1 -- 172.21.15.38:0/1722251688 wait complete. 2024-05-21T15:41:56.105 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:41:56.105 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:41:56.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:55 smithi053 ceph-mon[36496]: pgmap v576: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 5.46k op/s 2024-05-21T15:41:56.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.237+0000 7f71a5131640 1 -- 172.21.15.38:0/4189995094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71a0107900 msgr2=0x7f71a0109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.237+0000 7f71a5131640 1 --2- 172.21.15.38:0/4189995094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71a0107900 0x7f71a0109d20 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7198009ca0 tx=0x7f719802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.237+0000 7f71a5131640 1 -- 172.21.15.38:0/4189995094 shutdown_connections 2024-05-21T15:41:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.237+0000 7f71a5131640 1 --2- 172.21.15.38:0/4189995094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71a0107900 0x7f71a0109d20 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.237+0000 7f71a5131640 1 --2- 172.21.15.38:0/4189995094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a0102c00 0x7f71a0103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.237+0000 7f71a5131640 1 --2- 172.21.15.38:0/4189995094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71a01019c0 0x7f71a01026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.237+0000 7f71a5131640 1 -- 172.21.15.38:0/4189995094 >> 172.21.15.38:0/4189995094 conn(0x7f71a00fcbb0 msgr2=0x7f71a00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.237+0000 7f71a5131640 1 -- 172.21.15.38:0/4189995094 shutdown_connections 2024-05-21T15:41:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.237+0000 7f71a5131640 1 -- 172.21.15.38:0/4189995094 wait complete. 2024-05-21T15:41:56.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.238+0000 7f71a5131640 1 Processor -- start 2024-05-21T15:41:56.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.238+0000 7f71a5131640 1 -- start start 2024-05-21T15:41:56.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f71a5131640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a01019c0 0x7f71a006ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f71a5131640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71a0102c00 0x7f71a019df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f71a5131640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71a0107900 0x7f71a01a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f71a5131640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f71a010d690 con 0x7f71a0107900 2024-05-21T15:41:56.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f71a5131640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f71a010d510 con 0x7f71a0102c00 2024-05-21T15:41:56.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f71a5131640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f71a010d810 con 0x7f71a01019c0 2024-05-21T15:41:56.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f719659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71a0102c00 0x7f71a019df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f719f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71a0107900 0x7f71a01a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f719659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71a0102c00 0x7f71a019df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53806/0 (socket says 172.21.15.38:53806) 2024-05-21T15:41:56.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f719659c640 1 -- 172.21.15.38:0/4241403138 learned_addr learned my addr 172.21.15.38:0/4241403138 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f719ed9d640 1 --2- 172.21.15.38:0/4241403138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a01019c0 0x7f71a006ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f719659c640 1 -- 172.21.15.38:0/4241403138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a01019c0 msgr2=0x7f71a006ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f719659c640 1 --2- 172.21.15.38:0/4241403138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a01019c0 0x7f71a006ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f719659c640 1 -- 172.21.15.38:0/4241403138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71a0107900 msgr2=0x7f71a01a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f719659c640 1 --2- 172.21.15.38:0/4241403138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71a0107900 0x7f71a01a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f719659c640 1 -- 172.21.15.38:0/4241403138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a01a0d00 con 0x7f71a0102c00 2024-05-21T15:41:56.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.239+0000 7f719659c640 1 --2- 172.21.15.38:0/4241403138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71a0102c00 0x7f71a019df30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f718800e9f0 tx=0x7f718800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.240+0000 7f719cd99640 1 -- 172.21.15.38:0/4241403138 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718800cd80 con 0x7f71a0102c00 2024-05-21T15:41:56.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.240+0000 7f719cd99640 1 -- 172.21.15.38:0/4241403138 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7188005500 con 0x7f71a0102c00 2024-05-21T15:41:56.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.240+0000 7f71a5131640 1 -- 172.21.15.38:0/4241403138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a01a0ff0 con 0x7f71a0102c00 2024-05-21T15:41:56.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.240+0000 7f719cd99640 1 -- 172.21.15.38:0/4241403138 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188010690 con 0x7f71a0102c00 2024-05-21T15:41:56.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.240+0000 7f71a5131640 1 -- 172.21.15.38:0/4241403138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a01a1460 con 0x7f71a0102c00 2024-05-21T15:41:56.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.241+0000 7f71a5131640 1 -- 172.21.15.38:0/4241403138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a0106000 con 0x7f71a0102c00 2024-05-21T15:41:56.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.242+0000 7f719cd99640 1 -- 172.21.15.38:0/4241403138 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7188010830 con 0x7f71a0102c00 2024-05-21T15:41:56.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.243+0000 7f719cd99640 1 --2- 172.21.15.38:0/4241403138 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f718c0a2c30 0x7f718c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.243+0000 7f719ed9d640 1 --2- 172.21.15.38:0/4241403138 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f718c0a2c30 0x7f718c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.243+0000 7f719cd99640 1 -- 172.21.15.38:0/4241403138 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f71880cf4a0 con 0x7f71a0102c00 2024-05-21T15:41:56.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.243+0000 7f719ed9d640 1 --2- 172.21.15.38:0/4241403138 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f718c0a2c30 0x7f718c0a5140 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f71a01025c0 tx=0x7f7190005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.249+0000 7f719cd99640 1 -- 172.21.15.38:0/4241403138 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f718809c3b0 con 0x7f71a0102c00 2024-05-21T15:41:56.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.486+0000 7f71a5131640 1 -- 172.21.15.38:0/4241403138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71a0062750 con 0x7f71a0102c00 2024-05-21T15:41:56.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.487+0000 7f719cd99640 1 -- 172.21.15.38:0/4241403138 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11914 (secure 0 0 0) 0x7f718809bcb0 con 0x7f71a0102c00 2024-05-21T15:41:56.485 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:41:56.485 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:41:24.081884+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24551,24523,24602,24550,24515]},"id":1}]} 2024-05-21T15:41:56.485 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 35 2024-05-21T15:41:56.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.489+0000 7f71a5131640 1 -- 172.21.15.38:0/4241403138 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f718c0a2c30 msgr2=0x7f718c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.489+0000 7f71a5131640 1 --2- 172.21.15.38:0/4241403138 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f718c0a2c30 0x7f718c0a5140 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f71a01025c0 tx=0x7f7190005f90 comp rx=0 tx=0).stop 2024-05-21T15:41:56.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.489+0000 7f71a5131640 1 -- 172.21.15.38:0/4241403138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71a0102c00 msgr2=0x7f71a019df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.489+0000 7f71a5131640 1 --2- 172.21.15.38:0/4241403138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71a0102c00 0x7f71a019df30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f718800e9f0 tx=0x7f718800edf0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.490+0000 7f71a5131640 1 -- 172.21.15.38:0/4241403138 shutdown_connections 2024-05-21T15:41:56.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.490+0000 7f71a5131640 1 --2- 172.21.15.38:0/4241403138 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f718c0a2c30 0x7f718c0a5140 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.490+0000 7f71a5131640 1 --2- 172.21.15.38:0/4241403138 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71a0107900 0x7f71a01a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.490+0000 7f71a5131640 1 --2- 172.21.15.38:0/4241403138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71a0102c00 0x7f71a019df30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.490+0000 7f71a5131640 1 --2- 172.21.15.38:0/4241403138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a01019c0 0x7f71a006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.490+0000 7f71a5131640 1 -- 172.21.15.38:0/4241403138 >> 172.21.15.38:0/4241403138 conn(0x7f71a00fcbb0 msgr2=0x7f71a00fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.490+0000 7f71a5131640 1 -- 172.21.15.38:0/4241403138 shutdown_connections 2024-05-21T15:41:56.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.490+0000 7f71a5131640 1 -- 172.21.15.38:0/4241403138 wait complete. 2024-05-21T15:41:56.502 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:41:56.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.633+0000 7f0560a92640 1 -- 172.21.15.38:0/2155637667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f055c103d00 msgr2=0x7f055c104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.633+0000 7f0560a92640 1 --2- 172.21.15.38:0/2155637667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f055c103d00 0x7f055c104110 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0550009900 tx=0x7f055002a410 comp rx=0 tx=0).stop 2024-05-21T15:41:56.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.633+0000 7f0560a92640 1 -- 172.21.15.38:0/2155637667 shutdown_connections 2024-05-21T15:41:56.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.633+0000 7f0560a92640 1 --2- 172.21.15.38:0/2155637667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f055c108a00 0x7f055c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.633+0000 7f0560a92640 1 --2- 172.21.15.38:0/2155637667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f055c103d00 0x7f055c104110 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.633+0000 7f0560a92640 1 --2- 172.21.15.38:0/2155637667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c102ac0 0x7f055c1037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.633+0000 7f0560a92640 1 -- 172.21.15.38:0/2155637667 >> 172.21.15.38:0/2155637667 conn(0x7f055c0fdcb0 msgr2=0x7f055c100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.633+0000 7f0560a92640 1 -- 172.21.15.38:0/2155637667 shutdown_connections 2024-05-21T15:41:56.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.633+0000 7f0560a92640 1 -- 172.21.15.38:0/2155637667 wait complete. 2024-05-21T15:41:56.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f0560a92640 1 Processor -- start 2024-05-21T15:41:56.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f0560a92640 1 -- start start 2024-05-21T15:41:56.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f0560a92640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f055c102ac0 0x7f055c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f0560a92640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c103d00 0x7f055c199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f0560a92640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f055c108a00 0x7f055c1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f0560a92640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f055c10e740 con 0x7f055c102ac0 2024-05-21T15:41:56.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f0560a92640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f055c10e5c0 con 0x7f055c103d00 2024-05-21T15:41:56.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f0560a92640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f055c10e8c0 con 0x7f055c108a00 2024-05-21T15:41:56.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f055a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f055c102ac0 0x7f055c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f055a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f055c102ac0 0x7f055c199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40508/0 (socket says 172.21.15.38:40508) 2024-05-21T15:41:56.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f055a59c640 1 -- 172.21.15.38:0/806708271 learned_addr learned my addr 172.21.15.38:0/806708271 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f0559d9b640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c103d00 0x7f055c199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f055ad9d640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f055c108a00 0x7f055c1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f055a59c640 1 -- 172.21.15.38:0/806708271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f055c108a00 msgr2=0x7f055c1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f055a59c640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f055c108a00 0x7f055c1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f055a59c640 1 -- 172.21.15.38:0/806708271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c103d00 msgr2=0x7f055c199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f055a59c640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c103d00 0x7f055c199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f055a59c640 1 -- 172.21.15.38:0/806708271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c1a0d60 con 0x7f055c102ac0 2024-05-21T15:41:56.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.634+0000 7f055ad9d640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f055c108a00 0x7f055c1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:56.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.635+0000 7f055a59c640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f055c102ac0 0x7f055c199220 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f054c00e9f0 tx=0x7f054c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.635+0000 7f054b7fe640 1 -- 172.21.15.38:0/806708271 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c00ccc0 con 0x7f055c102ac0 2024-05-21T15:41:56.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.635+0000 7f0560a92640 1 -- 172.21.15.38:0/806708271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c1a0ff0 con 0x7f055c102ac0 2024-05-21T15:41:56.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.635+0000 7f054b7fe640 1 -- 172.21.15.38:0/806708271 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f054c0035f0 con 0x7f055c102ac0 2024-05-21T15:41:56.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.635+0000 7f054b7fe640 1 -- 172.21.15.38:0/806708271 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c00ce60 con 0x7f055c102ac0 2024-05-21T15:41:56.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.635+0000 7f0560a92640 1 -- 172.21.15.38:0/806708271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c1a1500 con 0x7f055c102ac0 2024-05-21T15:41:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.638+0000 7f0567a1a740 1 Processor -- start 2024-05-21T15:41:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.638+0000 7f054b7fe640 1 -- 172.21.15.38:0/806708271 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f054c0037a0 con 0x7f055c102ac0 2024-05-21T15:41:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.638+0000 7f0567a1a740 1 -- start start 2024-05-21T15:41:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.638+0000 7f0567a1a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151cb1d390 0x56151cebcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.638+0000 7f0567a1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151caf44d0 0x56151cebd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.638+0000 7f0567a1a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cafc6b0 0x56151cec3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.638+0000 7f0567a1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56151ce21c40 con 0x56151caf44d0 2024-05-21T15:41:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.638+0000 7f0567a1a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56151cdc9580 con 0x56151cb1d390 2024-05-21T15:41:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.638+0000 7f0567a1a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56151ce1dc00 con 0x56151cafc6b0 2024-05-21T15:41:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.638+0000 7f055a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151caf44d0 0x56151cebd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.638+0000 7f055a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151caf44d0 0x56151cebd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40518/0 (socket says 172.21.15.38:40518) 2024-05-21T15:41:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.638+0000 7f055a59c640 1 -- 172.21.15.38:0/2986439908 learned_addr learned my addr 172.21.15.38:0/2986439908 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.638+0000 7f0559d9b640 1 --2- 172.21.15.38:0/2986439908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cafc6b0 0x56151cec3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.638+0000 7f055ad9d640 1 --2- 172.21.15.38:0/2986439908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151cb1d390 0x56151cebcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.639+0000 7f055a59c640 1 -- 172.21.15.38:0/2986439908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cafc6b0 msgr2=0x56151cec3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.639+0000 7f055a59c640 1 --2- 172.21.15.38:0/2986439908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cafc6b0 0x56151cec3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.639+0000 7f055a59c640 1 -- 172.21.15.38:0/2986439908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151cb1d390 msgr2=0x56151cebcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.639+0000 7f055a59c640 1 --2- 172.21.15.38:0/2986439908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151cb1d390 0x56151cebcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.639+0000 7f054b7fe640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05440a2b90 0x7f05440a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.639+0000 7f055a59c640 1 -- 172.21.15.38:0/2986439908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56151cb353f0 con 0x56151caf44d0 2024-05-21T15:41:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.639+0000 7f0559d9b640 1 --2- 172.21.15.38:0/2986439908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cafc6b0 0x56151cec3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.639+0000 7f055ad9d640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05440a2b90 0x7f05440a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.639+0000 7f055a59c640 1 --2- 172.21.15.38:0/2986439908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151caf44d0 0x56151cebd080 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f054c0abda0 tx=0x7f054c097f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85085964ba2a988d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.639+0000 7f054b7fe640 1 -- 172.21.15.38:0/806708271 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f054c0cf390 con 0x7f055c102ac0 2024-05-21T15:41:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.639+0000 7f0548ff9640 1 -- 172.21.15.38:0/2986439908 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c09b970 con 0x56151caf44d0 2024-05-21T15:41:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.639+0000 7f0548ff9640 1 -- 172.21.15.38:0/2986439908 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f054c0abbe0 con 0x56151caf44d0 2024-05-21T15:41:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.639+0000 7f0548ff9640 1 -- 172.21.15.38:0/2986439908 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c09b500 con 0x56151caf44d0 2024-05-21T15:41:56.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.640+0000 7f0567a1a740 1 -- 172.21.15.38:0/2986439908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151caf44d0 msgr2=0x56151cebd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.640+0000 7f055ad9d640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05440a2b90 0x7f05440a50a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x56151cebdcd0 tx=0x7f0554009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.640+0000 7f0567a1a740 1 --2- 172.21.15.38:0/2986439908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151caf44d0 0x56151cebd080 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f054c0abda0 tx=0x7f054c097f00 comp rx=0 tx=0).stop 2024-05-21T15:41:56.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.640+0000 7f0567a1a740 1 -- 172.21.15.38:0/2986439908 shutdown_connections 2024-05-21T15:41:56.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.640+0000 7f0567a1a740 1 --2- 172.21.15.38:0/2986439908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cafc6b0 0x56151cec3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.640+0000 7f0567a1a740 1 --2- 172.21.15.38:0/2986439908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151caf44d0 0x56151cebd080 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.640+0000 7f0567a1a740 1 --2- 172.21.15.38:0/2986439908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151cb1d390 0x56151cebcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.640+0000 7f0567a1a740 1 -- 172.21.15.38:0/2986439908 >> 172.21.15.38:0/2986439908 conn(0x56151cb114e0 msgr2=0x56151ceb8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.640+0000 7f0567a1a740 1 -- 172.21.15.38:0/2986439908 shutdown_connections 2024-05-21T15:41:56.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.640+0000 7f0567a1a740 1 -- 172.21.15.38:0/2986439908 wait complete. 2024-05-21T15:41:56.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0567a1a740 1 -- start start 2024-05-21T15:41:56.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0567a1a740 1 Processor -- start 2024-05-21T15:41:56.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0567a1a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151caf44d0 0x56151cec6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0567a1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cafc6b0 0x56151cec6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0567a1a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151cb1d390 0x56151cecdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0567a1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56151cec4910 con 0x56151cafc6b0 2024-05-21T15:41:56.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0567a1a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56151cddcc40 con 0x56151cb1d390 2024-05-21T15:41:56.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0567a1a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56151cec4a90 con 0x56151caf44d0 2024-05-21T15:41:56.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0559d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cafc6b0 0x56151cec6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0567a1a740 10 client.0 _ll_register_callbacks cb 0x56151cb34be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:56.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0559d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cafc6b0 0x56151cec6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40520/0 (socket says 172.21.15.38:40520) 2024-05-21T15:41:56.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f055a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151caf44d0 0x56151cec6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0559d9b640 1 -- 172.21.15.38:0/1360708825 learned_addr learned my addr 172.21.15.38:0/1360708825 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f055ad9d640 1 --2- 172.21.15.38:0/1360708825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151cb1d390 0x56151cecdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0559d9b640 1 -- 172.21.15.38:0/1360708825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151caf44d0 msgr2=0x56151cec6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0559d9b640 1 --2- 172.21.15.38:0/1360708825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151caf44d0 0x56151cec6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0559d9b640 1 -- 172.21.15.38:0/1360708825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151cb1d390 msgr2=0x56151cecdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0559d9b640 1 --2- 172.21.15.38:0/1360708825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151cb1d390 0x56151cecdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f0559d9b640 1 -- 172.21.15.38:0/1360708825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56151cbff7d0 con 0x56151cafc6b0 2024-05-21T15:41:56.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.641+0000 7f055a59c640 1 --2- 172.21.15.38:0/1360708825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151caf44d0 0x56151cec6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:56.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.642+0000 7f0559d9b640 1 --2- 172.21.15.38:0/1360708825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cafc6b0 0x56151cec6dd0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0550031610 tx=0x7f05500316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6239cf0979a9623f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.642+0000 7f0541ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:56.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.642+0000 7f0541ffb640 1 -- 172.21.15.38:0/1360708825 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550035040 con 0x56151cafc6b0 2024-05-21T15:41:56.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.642+0000 7f0541ffb640 1 -- 172.21.15.38:0/1360708825 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f0550004040 con 0x56151cafc6b0 2024-05-21T15:41:56.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.642+0000 7f0567a1a740 1 -- 172.21.15.38:0/1360708825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56151ce2f370 con 0x56151cafc6b0 2024-05-21T15:41:56.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.642+0000 7f0541ffb640 1 -- 172.21.15.38:0/1360708825 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0550004240 con 0x56151cafc6b0 2024-05-21T15:41:56.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.642+0000 7f0567a1a740 10 client.17505 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:56.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.642+0000 7f0567a1a740 1 -- 172.21.15.38:0/1360708825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56151ced1060 con 0x56151cafc6b0 2024-05-21T15:41:56.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.642+0000 7f0541ffb640 1 -- 172.21.15.38:0/1360708825 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56151ce2f370 con 0x56151cafc6b0 2024-05-21T15:41:56.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.643+0000 7f0541ffb640 1 -- 172.21.15.38:0/1360708825 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f0550004a60 con 0x56151cafc6b0 2024-05-21T15:41:56.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.643+0000 7f0567a1a740 10 client.17505 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:56.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.643+0000 7f0567a1a740 10 client.17505 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:41:56.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.643+0000 7f0567a1a740 20 client.17505 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:56.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.643+0000 7f0567a1a740 1 --2- 172.21.15.38:0/1360708825 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56151cadbf60 0x56151ced4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.643+0000 7f0567a1a740 4 client.17505 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:56.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.643+0000 7f0567a1a740 1 -- 172.21.15.38:0/1360708825 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56151ce296c0 con 0x56151cadbf60 2024-05-21T15:41:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.643+0000 7f055a59c640 1 --2- 172.21.15.38:0/1360708825 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56151cadbf60 0x56151ced4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.643+0000 7f0541ffb640 10 client.17505 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:41:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.643+0000 7f055a59c640 1 --2- 172.21.15.38:0/1360708825 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56151cadbf60 0x56151ced4720 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=20395c942af52322 server_cookie=65827f0e9ebed60f in_seq=0 out_seq=0 2024-05-21T15:41:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0541ffb640 1 -- 172.21.15.38:0/1360708825 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56151ce296c0 con 0x56151cadbf60 2024-05-21T15:41:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0541ffb640 10 client.17505 handle_command_reply: tid=0 2024-05-21T15:41:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0567a1a740 1 client.17505 shutdown 2024-05-21T15:41:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0567a1a740 1 -- 172.21.15.38:0/1360708825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cafc6b0 msgr2=0x56151cec6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0567a1a740 1 --2- 172.21.15.38:0/1360708825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cafc6b0 0x56151cec6dd0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0550031610 tx=0x7f05500316b0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0567a1a740 1 -- 172.21.15.38:0/1360708825 shutdown_connections 2024-05-21T15:41:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0567a1a740 1 --2- 172.21.15.38:0/1360708825 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56151cadbf60 0x56151ced4720 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0567a1a740 1 --2- 172.21.15.38:0/1360708825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151cb1d390 0x56151cecdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0567a1a740 1 --2- 172.21.15.38:0/1360708825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cafc6b0 0x56151cec6dd0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0567a1a740 1 --2- 172.21.15.38:0/1360708825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151caf44d0 0x56151cec6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0567a1a740 1 -- 172.21.15.38:0/1360708825 >> 172.21.15.38:0/1360708825 conn(0x56151cb114e0 msgr2=0x56151ceb7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0541ffb640 0 client.17505 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:41:56.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0567a1a740 1 -- 172.21.15.38:0/1360708825 shutdown_connections 2024-05-21T15:41:56.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0567a1a740 1 -- 172.21.15.38:0/1360708825 wait complete. 2024-05-21T15:41:56.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.644+0000 7f0567a1a740 20 client.17505 trim_cache size 0 max 16384 2024-05-21T15:41:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.660+0000 7f0567a1a740 1 Processor -- start 2024-05-21T15:41:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.660+0000 7f0567a1a740 1 -- start start 2024-05-21T15:41:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.660+0000 7f0567a1a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151caf44d0 0x56151cebb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.660+0000 7f0567a1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cafc6b0 0x56151cebb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.660+0000 7f0567a1a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cb1d390 0x56151cebbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.660+0000 7f0567a1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1aa400 con 0x56151cafc6b0 2024-05-21T15:41:56.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.660+0000 7f0567a1a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1abf40 con 0x56151caf44d0 2024-05-21T15:41:56.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.660+0000 7f0567a1a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1aaad0 con 0x56151cb1d390 2024-05-21T15:41:56.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.660+0000 7f0559d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cafc6b0 0x56151cebb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.660+0000 7f055ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cb1d390 0x56151cebbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.660+0000 7f0559d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cafc6b0 0x56151cebb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40532/0 (socket says 172.21.15.38:40532) 2024-05-21T15:41:56.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.660+0000 7f0559d9b640 1 -- 172.21.15.38:0/3333054280 learned_addr learned my addr 172.21.15.38:0/3333054280 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.660+0000 7f055a59c640 1 --2- 172.21.15.38:0/3333054280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151caf44d0 0x56151cebb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.661+0000 7f0559d9b640 1 -- 172.21.15.38:0/3333054280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cb1d390 msgr2=0x56151cebbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.661+0000 7f0559d9b640 1 --2- 172.21.15.38:0/3333054280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cb1d390 0x56151cebbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.661+0000 7f0559d9b640 1 -- 172.21.15.38:0/3333054280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151caf44d0 msgr2=0x56151cebb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.661+0000 7f0559d9b640 1 --2- 172.21.15.38:0/3333054280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151caf44d0 0x56151cebb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.661+0000 7f0559d9b640 1 -- 172.21.15.38:0/3333054280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56151cebc2f0 con 0x56151cafc6b0 2024-05-21T15:41:56.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.661+0000 7f055ad9d640 1 --2- 172.21.15.38:0/3333054280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cb1d390 0x56151cebbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:56.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.661+0000 7f0559d9b640 1 --2- 172.21.15.38:0/3333054280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cafc6b0 0x56151cebb870 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f055002a930 tx=0x7f0550001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0d741a524de21c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.661+0000 7f0543fff640 1 -- 172.21.15.38:0/3333054280 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550035020 con 0x56151cafc6b0 2024-05-21T15:41:56.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.661+0000 7f0543fff640 1 -- 172.21.15.38:0/3333054280 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0550004c40 con 0x56151cafc6b0 2024-05-21T15:41:56.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.661+0000 7f0543fff640 1 -- 172.21.15.38:0/3333054280 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05500055d0 con 0x56151cafc6b0 2024-05-21T15:41:56.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.662+0000 7f0567a1a740 1 -- 172.21.15.38:0/3333054280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cafc6b0 msgr2=0x56151cebb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.662+0000 7f0567a1a740 1 --2- 172.21.15.38:0/3333054280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cafc6b0 0x56151cebb870 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f055002a930 tx=0x7f0550001e70 comp rx=0 tx=0).stop 2024-05-21T15:41:56.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.662+0000 7f0567a1a740 1 -- 172.21.15.38:0/3333054280 shutdown_connections 2024-05-21T15:41:56.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.662+0000 7f0567a1a740 1 --2- 172.21.15.38:0/3333054280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cb1d390 0x56151cebbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.662+0000 7f0567a1a740 1 --2- 172.21.15.38:0/3333054280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cafc6b0 0x56151cebb870 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.662+0000 7f0567a1a740 1 --2- 172.21.15.38:0/3333054280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151caf44d0 0x56151cebb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.662+0000 7f0567a1a740 1 -- 172.21.15.38:0/3333054280 >> 172.21.15.38:0/3333054280 conn(0x56151cb114e0 msgr2=0x56151cec0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.662+0000 7f0567a1a740 1 -- 172.21.15.38:0/3333054280 shutdown_connections 2024-05-21T15:41:56.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.662+0000 7f0567a1a740 1 -- 172.21.15.38:0/3333054280 wait complete. 2024-05-21T15:41:56.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.662+0000 7f0567a1a740 1 -- start start 2024-05-21T15:41:56.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f0567a1a740 1 Processor -- start 2024-05-21T15:41:56.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f0567a1a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151caf44d0 0x56151cf36db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f0567a1a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cafc6b0 0x56151cf372f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f0567a1a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cb1d390 0x56151cf3e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f0567a1a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1ab870 con 0x56151cb1d390 2024-05-21T15:41:56.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f0567a1a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1ab1a0 con 0x56151caf44d0 2024-05-21T15:41:56.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f0567a1a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1a83e0 con 0x56151cafc6b0 2024-05-21T15:41:56.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f0567a1a740 10 client.0 _ll_register_callbacks cb 0x7f0550009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:56.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f055ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cb1d390 0x56151cf3e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f055ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cb1d390 0x56151cf3e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40534/0 (socket says 172.21.15.38:40534) 2024-05-21T15:41:56.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f055ad9d640 1 -- 172.21.15.38:0/1859975386 learned_addr learned my addr 172.21.15.38:0/1859975386 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f055a59c640 1 --2- 172.21.15.38:0/1859975386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151caf44d0 0x56151cf36db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f0559d9b640 1 --2- 172.21.15.38:0/1859975386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cafc6b0 0x56151cf372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f055ad9d640 1 -- 172.21.15.38:0/1859975386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cafc6b0 msgr2=0x56151cf372f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f055ad9d640 1 --2- 172.21.15.38:0/1859975386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cafc6b0 0x56151cf372f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f055ad9d640 1 -- 172.21.15.38:0/1859975386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151caf44d0 msgr2=0x56151cf36db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f055ad9d640 1 --2- 172.21.15.38:0/1859975386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151caf44d0 0x56151cf36db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f055ad9d640 1 -- 172.21.15.38:0/1859975386 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56151cf3e810 con 0x56151cb1d390 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.663+0000 7f055a59c640 1 --2- 172.21.15.38:0/1859975386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151caf44d0 0x56151cf36db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.664+0000 7f055ad9d640 1 --2- 172.21.15.38:0/1859975386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cb1d390 0x56151cf3e250 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f055400fef0 tx=0x7f0554004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d685df1883e6e2f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.664+0000 7f0541ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.664+0000 7f0541ffb640 1 -- 172.21.15.38:0/1859975386 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554011e10 con 0x56151cb1d390 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.664+0000 7f0541ffb640 1 -- 172.21.15.38:0/1859975386 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f0554004b20 con 0x56151cb1d390 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.664+0000 7f0567a1a740 1 -- 172.21.15.38:0/1859975386 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0550004c40 con 0x56151cb1d390 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.664+0000 7f0541ffb640 1 -- 172.21.15.38:0/1859975386 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0554004d20 con 0x56151cb1d390 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.664+0000 7f0567a1a740 10 client.17511 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:56.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.664+0000 7f0567a1a740 1 -- 172.21.15.38:0/1859975386 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56151cf3fb90 con 0x56151cb1d390 2024-05-21T15:41:56.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.664+0000 7f0541ffb640 1 -- 172.21.15.38:0/1859975386 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0550004c40 con 0x56151cb1d390 2024-05-21T15:41:56.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.664+0000 7f0541ffb640 1 -- 172.21.15.38:0/1859975386 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f0554033070 con 0x56151cb1d390 2024-05-21T15:41:56.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.664+0000 7f0567a1a740 10 client.17511 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:56.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.664+0000 7f0567a1a740 10 client.17511 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:41:56.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.664+0000 7f0567a1a740 20 client.17511 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:56.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.665+0000 7f0567a1a740 1 --2- 172.21.15.38:0/1859975386 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56151cab13a0 0x56151cf42630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.665+0000 7f0567a1a740 4 client.17511 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:41:56.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.665+0000 7f0567a1a740 1 -- 172.21.15.38:0/1859975386 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56151cda6820 con 0x56151cab13a0 2024-05-21T15:41:56.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.665+0000 7f055a59c640 1 --2- 172.21.15.38:0/1859975386 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56151cab13a0 0x56151cf42630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.665+0000 7f0541ffb640 10 client.17511 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:41:56.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.665+0000 7f055a59c640 1 --2- 172.21.15.38:0/1859975386 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56151cab13a0 0x56151cf42630 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3731162e128aaf35 server_cookie=a16f6e9aafd1baf1 in_seq=0 out_seq=0 2024-05-21T15:41:56.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.665+0000 7f0541ffb640 1 -- 172.21.15.38:0/1859975386 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56151cda6820 con 0x56151cab13a0 2024-05-21T15:41:56.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.665+0000 7f0541ffb640 10 client.17511 handle_command_reply: tid=0 2024-05-21T15:41:56.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.665+0000 7f0567a1a740 1 client.17511 shutdown 2024-05-21T15:41:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.665+0000 7f0567a1a740 1 -- 172.21.15.38:0/1859975386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cb1d390 msgr2=0x56151cf3e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.665+0000 7f0567a1a740 1 --2- 172.21.15.38:0/1859975386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cb1d390 0x56151cf3e250 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f055400fef0 tx=0x7f0554004ab0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0567a1a740 1 -- 172.21.15.38:0/1859975386 shutdown_connections 2024-05-21T15:41:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0567a1a740 1 --2- 172.21.15.38:0/1859975386 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x56151cab13a0 0x56151cf42630 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0567a1a740 1 --2- 172.21.15.38:0/1859975386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56151cb1d390 0x56151cf3e250 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0567a1a740 1 --2- 172.21.15.38:0/1859975386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56151cafc6b0 0x56151cf372f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0567a1a740 1 --2- 172.21.15.38:0/1859975386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56151caf44d0 0x56151cf36db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0541ffb640 0 client.17511 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:41:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0567a1a740 1 -- 172.21.15.38:0/1859975386 >> 172.21.15.38:0/1859975386 conn(0x56151cb114e0 msgr2=0x56151cebb330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0567a1a740 1 -- 172.21.15.38:0/1859975386 shutdown_connections 2024-05-21T15:41:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0567a1a740 1 -- 172.21.15.38:0/1859975386 wait complete. 2024-05-21T15:41:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0567a1a740 20 client.17511 trim_cache size 0 max 16384 2024-05-21T15:41:56.670 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:41:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0560a92640 1 -- 172.21.15.38:0/806708271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05440a2b90 msgr2=0x7f05440a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0560a92640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05440a2b90 0x7f05440a50a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x56151cebdcd0 tx=0x7f0554009500 comp rx=0 tx=0).stop 2024-05-21T15:41:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0560a92640 1 -- 172.21.15.38:0/806708271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f055c102ac0 msgr2=0x7f055c199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0560a92640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f055c102ac0 0x7f055c199220 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f054c00e9f0 tx=0x7f054c00eeb0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0560a92640 1 -- 172.21.15.38:0/806708271 shutdown_connections 2024-05-21T15:41:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0560a92640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f055c108a00 0x7f055c1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0560a92640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c103d00 0x7f055c199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0560a92640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f05440a2b90 0x7f05440a50a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.666+0000 7f0560a92640 1 --2- 172.21.15.38:0/806708271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f055c102ac0 0x7f055c199220 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.667+0000 7f0560a92640 1 -- 172.21.15.38:0/806708271 >> 172.21.15.38:0/806708271 conn(0x7f055c0fdcb0 msgr2=0x7f055c1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.667+0000 7f0560a92640 1 -- 172.21.15.38:0/806708271 shutdown_connections 2024-05-21T15:41:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.667+0000 7f0560a92640 1 -- 172.21.15.38:0/806708271 wait complete. 2024-05-21T15:41:56.677 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:41:56.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.811+0000 7fceca745640 1 -- 172.21.15.38:0/1248858168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcec41028b0 msgr2=0x7fcec4102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.811+0000 7fceca745640 1 --2- 172.21.15.38:0/1248858168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcec41028b0 0x7fcec4102d60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009900 tx=0x7fceb802a410 comp rx=0 tx=0).stop 2024-05-21T15:41:56.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.811+0000 7fceca745640 1 -- 172.21.15.38:0/1248858168 shutdown_connections 2024-05-21T15:41:56.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.811+0000 7fceca745640 1 --2- 172.21.15.38:0/1248858168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec41033f0 0x7fcec4109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.811+0000 7fceca745640 1 --2- 172.21.15.38:0/1248858168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcec41028b0 0x7fcec4102d60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.811+0000 7fceca745640 1 --2- 172.21.15.38:0/1248858168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec4101ed0 0x7fcec41022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.811+0000 7fceca745640 1 -- 172.21.15.38:0/1248858168 >> 172.21.15.38:0/1248858168 conn(0x7fcec40fc2f0 msgr2=0x7fcec40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.811+0000 7fceca745640 1 -- 172.21.15.38:0/1248858168 shutdown_connections 2024-05-21T15:41:56.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.812+0000 7fceca745640 1 -- 172.21.15.38:0/1248858168 wait complete. 2024-05-21T15:41:56.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.812+0000 7fceca745640 1 Processor -- start 2024-05-21T15:41:56.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.812+0000 7fceca745640 1 -- start start 2024-05-21T15:41:56.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.812+0000 7fceca745640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec4101ed0 0x7fcec4199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fceca745640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec41028b0 0x7fcec41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fceca745640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcec41033f0 0x7fcec41a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fceca745640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcec410e6f0 con 0x7fcec41028b0 2024-05-21T15:41:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fceca745640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcec410e570 con 0x7fcec4101ed0 2024-05-21T15:41:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fceca745640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcec410e870 con 0x7fcec41033f0 2024-05-21T15:41:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec41028b0 0x7fcec41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec41028b0 0x7fcec41997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40550/0 (socket says 172.21.15.38:40550) 2024-05-21T15:41:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fcec37fe640 1 -- 172.21.15.38:0/1351867431 learned_addr learned my addr 172.21.15.38:0/1351867431 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fcec8ce2640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcec41033f0 0x7fcec41a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fcec3fff640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec4101ed0 0x7fcec4199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fcec37fe640 1 -- 172.21.15.38:0/1351867431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcec41033f0 msgr2=0x7fcec41a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fcec37fe640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcec41033f0 0x7fcec41a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fcec37fe640 1 -- 172.21.15.38:0/1351867431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec4101ed0 msgr2=0x7fcec4199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fcec37fe640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec4101ed0 0x7fcec4199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fcec37fe640 1 -- 172.21.15.38:0/1351867431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec41a0db0 con 0x7fcec41028b0 2024-05-21T15:41:56.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fcec8ce2640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcec41033f0 0x7fcec41a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:56.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fcec37fe640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec41028b0 0x7fcec41997b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fceb8006010 tx=0x7fceb80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.813+0000 7fcec17fa640 1 -- 172.21.15.38:0/1351867431 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb803f030 con 0x7fcec41028b0 2024-05-21T15:41:56.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.814+0000 7fcec17fa640 1 -- 172.21.15.38:0/1351867431 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceb8003ce0 con 0x7fcec41028b0 2024-05-21T15:41:56.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.814+0000 7fceca745640 1 -- 172.21.15.38:0/1351867431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec41a1040 con 0x7fcec41028b0 2024-05-21T15:41:56.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.814+0000 7fcec17fa640 1 -- 172.21.15.38:0/1351867431 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8035070 con 0x7fcec41028b0 2024-05-21T15:41:56.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.814+0000 7fceca745640 1 -- 172.21.15.38:0/1351867431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec41a1530 con 0x7fcec41028b0 2024-05-21T15:41:56.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.816+0000 7fcec17fa640 1 -- 172.21.15.38:0/1351867431 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fceb8042030 con 0x7fcec41028b0 2024-05-21T15:41:56.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.816+0000 7fced15f2740 1 Processor -- start 2024-05-21T15:41:56.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.816+0000 7fced15f2740 1 -- start start 2024-05-21T15:41:56.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.816+0000 7fced15f2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a939c390 0x5654a973ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.816+0000 7fced15f2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a93734d0 0x5654a9740490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.816+0000 7fced15f2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a937b6b0 0x5654a9747390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.816+0000 7fced15f2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5654a96a0c40 con 0x5654a937b6b0 2024-05-21T15:41:56.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.816+0000 7fced15f2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5654a9648580 con 0x5654a939c390 2024-05-21T15:41:56.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.816+0000 7fced15f2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5654a969cc00 con 0x5654a93734d0 2024-05-21T15:41:56.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a937b6b0 0x5654a9747390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec8ce2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a939c390 0x5654a973ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a93734d0 0x5654a9740490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec8ce2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a939c390 0x5654a973ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53902/0 (socket says 172.21.15.38:53902) 2024-05-21T15:41:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a937b6b0 0x5654a9747390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40552/0 (socket says 172.21.15.38:40552) 2024-05-21T15:41:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec8ce2640 1 -- 172.21.15.38:0/3254533862 learned_addr learned my addr 172.21.15.38:0/3254533862 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec17fa640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fceac0a2b90 0x7fceac0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec3fff640 1 -- 172.21.15.38:0/3254533862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a939c390 msgr2=0x5654a973ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec3fff640 1 --2- 172.21.15.38:0/3254533862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a939c390 0x5654a973ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec8ce2640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fceac0a2b90 0x7fceac0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec3fff640 1 -- 172.21.15.38:0/3254533862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a937b6b0 msgr2=0x5654a9747390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec3fff640 1 --2- 172.21.15.38:0/3254533862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a937b6b0 0x5654a9747390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec3fff640 1 -- 172.21.15.38:0/3254533862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a93b43f0 con 0x5654a93734d0 2024-05-21T15:41:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec17fa640 1 -- 172.21.15.38:0/1351867431 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fceb80ef890 con 0x7fcec41028b0 2024-05-21T15:41:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec3fff640 1 --2- 172.21.15.38:0/3254533862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a93734d0 0x5654a9740490 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcec4068d10 tx=0x7fceb4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8649c7e592a08e21 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fceaa7fc640 1 -- 172.21.15.38:0/3254533862 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4004050 con 0x5654a93734d0 2024-05-21T15:41:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fcec8ce2640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fceac0a2b90 0x7fceac0a50a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x5654a97410e0 tx=0x7fcebc004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.817+0000 7fceaa7fc640 1 -- 172.21.15.38:0/3254533862 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceb4005500 con 0x5654a93734d0 2024-05-21T15:41:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.818+0000 7fceaa7fc640 1 -- 172.21.15.38:0/3254533862 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4004d90 con 0x5654a93734d0 2024-05-21T15:41:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.818+0000 7fced15f2740 1 -- 172.21.15.38:0/3254533862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a93734d0 msgr2=0x5654a9740490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.818+0000 7fced15f2740 1 --2- 172.21.15.38:0/3254533862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a93734d0 0x5654a9740490 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcec4068d10 tx=0x7fceb4011810 comp rx=0 tx=0).stop 2024-05-21T15:41:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.818+0000 7fced15f2740 1 -- 172.21.15.38:0/3254533862 shutdown_connections 2024-05-21T15:41:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.818+0000 7fced15f2740 1 --2- 172.21.15.38:0/3254533862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a937b6b0 0x5654a9747390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.818+0000 7fced15f2740 1 --2- 172.21.15.38:0/3254533862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a93734d0 0x5654a9740490 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.818+0000 7fced15f2740 1 --2- 172.21.15.38:0/3254533862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a939c390 0x5654a973ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.818+0000 7fced15f2740 1 -- 172.21.15.38:0/3254533862 >> 172.21.15.38:0/3254533862 conn(0x5654a93904e0 msgr2=0x5654a973b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.819+0000 7fced15f2740 1 -- 172.21.15.38:0/3254533862 shutdown_connections 2024-05-21T15:41:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.819+0000 7fced15f2740 1 -- 172.21.15.38:0/3254533862 wait complete. 2024-05-21T15:41:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.819+0000 7fced15f2740 1 -- start start 2024-05-21T15:41:56.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.819+0000 7fced15f2740 1 Processor -- start 2024-05-21T15:41:56.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.819+0000 7fced15f2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a9749b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.819+0000 7fced15f2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a937b6b0 0x5654a974a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.819+0000 7fced15f2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a939c390 0x5654a9751000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.819+0000 7fced15f2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5654a9747de0 con 0x5654a937b6b0 2024-05-21T15:41:56.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.819+0000 7fced15f2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5654a965bc40 con 0x5654a93734d0 2024-05-21T15:41:56.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.819+0000 7fced15f2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5654a9747f60 con 0x5654a939c390 2024-05-21T15:41:56.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.819+0000 7fced15f2740 10 client.0 _ll_register_callbacks cb 0x5654a93b3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:56.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.819+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a937b6b0 0x5654a974a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcec3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a9749b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcec8ce2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a939c390 0x5654a9751000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcec3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a9749b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53908/0 (socket says 172.21.15.38:53908) 2024-05-21T15:41:56.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcec3fff640 1 -- 172.21.15.38:0/2318202727 learned_addr learned my addr 172.21.15.38:0/2318202727 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a937b6b0 0x5654a974a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40560/0 (socket says 172.21.15.38:40560) 2024-05-21T15:41:56.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcec3fff640 1 -- 172.21.15.38:0/2318202727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a939c390 msgr2=0x5654a9751000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcec3fff640 1 --2- 172.21.15.38:0/2318202727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a939c390 0x5654a9751000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcec3fff640 1 -- 172.21.15.38:0/2318202727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a937b6b0 msgr2=0x5654a974a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcec3fff640 1 --2- 172.21.15.38:0/2318202727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a937b6b0 0x5654a974a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcec3fff640 1 -- 172.21.15.38:0/2318202727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a947e7d0 con 0x5654a93734d0 2024-05-21T15:41:56.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcec8ce2640 1 --2- 172.21.15.38:0/2318202727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a939c390 0x5654a9751000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:56.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcec37fe640 1 --2- 172.21.15.38:0/2318202727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a937b6b0 0x5654a974a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:56.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcec3fff640 1 --2- 172.21.15.38:0/2318202727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a9749b60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fceb4004650 tx=0x7fceb4004de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86bdecbba30cb674 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcea37fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:41:56.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcea37fe640 1 -- 172.21.15.38:0/2318202727 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4029070 con 0x5654a93734d0 2024-05-21T15:41:56.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fced15f2740 1 -- 172.21.15.38:0/2318202727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceb4005500 con 0x5654a93734d0 2024-05-21T15:41:56.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.820+0000 7fcea37fe640 1 -- 172.21.15.38:0/2318202727 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fceb40035f0 con 0x5654a93734d0 2024-05-21T15:41:56.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.821+0000 7fcea37fe640 1 -- 172.21.15.38:0/2318202727 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fceb4035050 con 0x5654a93734d0 2024-05-21T15:41:56.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.821+0000 7fced15f2740 10 client.27137 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:56.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.821+0000 7fced15f2740 1 -- 172.21.15.38:0/2318202727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5654a97542c0 con 0x5654a93734d0 2024-05-21T15:41:56.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.821+0000 7fcea37fe640 1 -- 172.21.15.38:0/2318202727 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceb4005500 con 0x5654a93734d0 2024-05-21T15:41:56.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.821+0000 7fcea37fe640 1 -- 172.21.15.38:0/2318202727 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fceb4023430 con 0x5654a93734d0 2024-05-21T15:41:56.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.822+0000 7fced15f2740 10 client.27137 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:56.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.822+0000 7fced15f2740 10 client.27137 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:41:56.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.822+0000 7fced15f2740 20 client.27137 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.822+0000 7fced15f2740 1 --2- 172.21.15.38:0/2318202727 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5654a935af60 0x5654a97577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.822+0000 7fced15f2740 4 client.27137 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.822+0000 7fced15f2740 1 -- 172.21.15.38:0/2318202727 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654a96ae370 con 0x5654a935af60 2024-05-21T15:41:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.822+0000 7fcec37fe640 1 --2- 172.21.15.38:0/2318202727 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5654a935af60 0x5654a97577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.822+0000 7fcec37fe640 1 --2- 172.21.15.38:0/2318202727 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5654a935af60 0x5654a97577a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6133ee15abd97eaa server_cookie=193693f0b77b722e in_seq=0 out_seq=0 2024-05-21T15:41:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.822+0000 7fcea37fe640 10 client.27137 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:41:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.823+0000 7fcea37fe640 1 -- 172.21.15.38:0/2318202727 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654a96ae370 con 0x5654a935af60 2024-05-21T15:41:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.823+0000 7fcea37fe640 10 client.27137 handle_command_reply: tid=0 2024-05-21T15:41:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.823+0000 7fced15f2740 1 client.27137 shutdown 2024-05-21T15:41:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.823+0000 7fced15f2740 1 -- 172.21.15.38:0/2318202727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 msgr2=0x5654a9749b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.823+0000 7fced15f2740 1 --2- 172.21.15.38:0/2318202727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a9749b60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fceb4004650 tx=0x7fceb4004de0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.823+0000 7fced15f2740 1 -- 172.21.15.38:0/2318202727 shutdown_connections 2024-05-21T15:41:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.823+0000 7fced15f2740 1 --2- 172.21.15.38:0/2318202727 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5654a935af60 0x5654a97577a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.823+0000 7fced15f2740 1 --2- 172.21.15.38:0/2318202727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a939c390 0x5654a9751000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.823+0000 7fced15f2740 1 --2- 172.21.15.38:0/2318202727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a937b6b0 0x5654a974a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.823+0000 7fcea37fe640 0 client.27137 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:41:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.823+0000 7fced15f2740 1 --2- 172.21.15.38:0/2318202727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a9749b60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.823+0000 7fced15f2740 1 -- 172.21.15.38:0/2318202727 >> 172.21.15.38:0/2318202727 conn(0x5654a93904e0 msgr2=0x5654a973e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.823+0000 7fced15f2740 1 -- 172.21.15.38:0/2318202727 shutdown_connections 2024-05-21T15:41:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.823+0000 7fced15f2740 1 -- 172.21.15.38:0/2318202727 wait complete. 2024-05-21T15:41:56.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.824+0000 7fced15f2740 20 client.27137 trim_cache size 0 max 16384 2024-05-21T15:41:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.838+0000 7fced15f2740 1 Processor -- start 2024-05-21T15:41:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.838+0000 7fced15f2740 1 -- start start 2024-05-21T15:41:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.838+0000 7fced15f2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a935af60 0x5654a9757970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.838+0000 7fced15f2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a9750f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.838+0000 7fced15f2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a937b6b0 0x5654a97401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.838+0000 7fced15f2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcec41a8200 con 0x5654a935af60 2024-05-21T15:41:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.838+0000 7fced15f2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcec41a9d40 con 0x5654a93734d0 2024-05-21T15:41:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.838+0000 7fced15f2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcec41a88d0 con 0x5654a937b6b0 2024-05-21T15:41:56.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.838+0000 7fcec3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a935af60 0x5654a9757970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.838+0000 7fcec3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a935af60 0x5654a9757970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40576/0 (socket says 172.21.15.38:40576) 2024-05-21T15:41:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.838+0000 7fcec3fff640 1 -- 172.21.15.38:0/638822140 learned_addr learned my addr 172.21.15.38:0/638822140 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.838+0000 7fcec37fe640 1 --2- 172.21.15.38:0/638822140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a9750f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.839+0000 7fcec8ce2640 1 --2- 172.21.15.38:0/638822140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a937b6b0 0x5654a97401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.839+0000 7fcec3fff640 1 -- 172.21.15.38:0/638822140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a937b6b0 msgr2=0x5654a97401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.839+0000 7fcec3fff640 1 --2- 172.21.15.38:0/638822140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a937b6b0 0x5654a97401d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.839+0000 7fcec3fff640 1 -- 172.21.15.38:0/638822140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 msgr2=0x5654a9750f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.839+0000 7fcec3fff640 1 --2- 172.21.15.38:0/638822140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a9750f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.839+0000 7fcec3fff640 1 -- 172.21.15.38:0/638822140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a9732d20 con 0x5654a935af60 2024-05-21T15:41:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.839+0000 7fcec8ce2640 1 --2- 172.21.15.38:0/638822140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a937b6b0 0x5654a97401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:41:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.839+0000 7fcec3fff640 1 --2- 172.21.15.38:0/638822140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a935af60 0x5654a9757970 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009730 tx=0x7fceb4004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df65e369b6d6f3a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.839+0000 7fcea9ffb640 1 -- 172.21.15.38:0/638822140 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4029070 con 0x5654a935af60 2024-05-21T15:41:56.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.839+0000 7fcea9ffb640 1 -- 172.21.15.38:0/638822140 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceb4004680 con 0x5654a935af60 2024-05-21T15:41:56.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.839+0000 7fcea9ffb640 1 -- 172.21.15.38:0/638822140 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb40035f0 con 0x5654a935af60 2024-05-21T15:41:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.840+0000 7fced15f2740 1 -- 172.21.15.38:0/638822140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a935af60 msgr2=0x5654a9757970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.840+0000 7fced15f2740 1 --2- 172.21.15.38:0/638822140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a935af60 0x5654a9757970 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009730 tx=0x7fceb4004a80 comp rx=0 tx=0).stop 2024-05-21T15:41:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.840+0000 7fced15f2740 1 -- 172.21.15.38:0/638822140 shutdown_connections 2024-05-21T15:41:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.840+0000 7fced15f2740 1 --2- 172.21.15.38:0/638822140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a937b6b0 0x5654a97401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.840+0000 7fced15f2740 1 --2- 172.21.15.38:0/638822140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a9750f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.840+0000 7fced15f2740 1 --2- 172.21.15.38:0/638822140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a935af60 0x5654a9757970 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.840+0000 7fced15f2740 1 -- 172.21.15.38:0/638822140 >> 172.21.15.38:0/638822140 conn(0x5654a93904e0 msgr2=0x5654a973fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.840+0000 7fced15f2740 1 -- 172.21.15.38:0/638822140 shutdown_connections 2024-05-21T15:41:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.840+0000 7fced15f2740 1 -- 172.21.15.38:0/638822140 wait complete. 2024-05-21T15:41:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.841+0000 7fced15f2740 1 -- start start 2024-05-21T15:41:56.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.841+0000 7fced15f2740 1 Processor -- start 2024-05-21T15:41:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.841+0000 7fced15f2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a935af60 0x5654a9750a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.841+0000 7fced15f2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a974d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.841+0000 7fced15f2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a937b6b0 0x5654a974dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.841+0000 7fced15f2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcec41a9670 con 0x5654a935af60 2024-05-21T15:41:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.841+0000 7fced15f2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcec41a8fa0 con 0x5654a93734d0 2024-05-21T15:41:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.841+0000 7fced15f2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcec41a61e0 con 0x5654a937b6b0 2024-05-21T15:41:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.841+0000 7fced15f2740 10 client.0 _ll_register_callbacks cb 0x7fceb80b8360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.841+0000 7fcec3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a935af60 0x5654a9750a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.841+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a974d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.841+0000 7fcec3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a935af60 0x5654a9750a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40592/0 (socket says 172.21.15.38:40592) 2024-05-21T15:41:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.841+0000 7fcec3fff640 1 -- 172.21.15.38:0/256780483 learned_addr learned my addr 172.21.15.38:0/256780483 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.841+0000 7fcec8ce2640 1 --2- 172.21.15.38:0/256780483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a937b6b0 0x5654a974dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fcec3fff640 1 -- 172.21.15.38:0/256780483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a937b6b0 msgr2=0x5654a974dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fcec3fff640 1 --2- 172.21.15.38:0/256780483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a937b6b0 0x5654a974dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fcec3fff640 1 -- 172.21.15.38:0/256780483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 msgr2=0x5654a974d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fcec3fff640 1 --2- 172.21.15.38:0/256780483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a974d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fcec3fff640 1 -- 172.21.15.38:0/256780483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a9751460 con 0x5654a935af60 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fcec8ce2640 1 --2- 172.21.15.38:0/256780483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a937b6b0 0x5654a974dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fcec37fe640 1 --2- 172.21.15.38:0/256780483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a974d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fcec3fff640 1 --2- 172.21.15.38:0/256780483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a935af60 0x5654a9750a20 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fceb40099b0 tx=0x7fceb40097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10d14f85f85fc22 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fcea37fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fcea37fe640 1 -- 172.21.15.38:0/256780483 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4029040 con 0x5654a935af60 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fcea37fe640 1 -- 172.21.15.38:0/256780483 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fceb4023ce0 con 0x5654a935af60 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fced15f2740 1 -- 172.21.15.38:0/256780483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceb4004680 con 0x5654a935af60 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fcea37fe640 1 -- 172.21.15.38:0/256780483 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fceb4004de0 con 0x5654a935af60 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fced15f2740 10 client.17529 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fced15f2740 1 -- 172.21.15.38:0/256780483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5654a97be670 con 0x5654a935af60 2024-05-21T15:41:56.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.842+0000 7fcea37fe640 1 -- 172.21.15.38:0/256780483 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceb4004680 con 0x5654a935af60 2024-05-21T15:41:56.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.843+0000 7fcea37fe640 1 -- 172.21.15.38:0/256780483 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fceb4003c60 con 0x5654a935af60 2024-05-21T15:41:56.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.843+0000 7fced15f2740 10 client.17529 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:56.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.843+0000 7fced15f2740 10 client.17529 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:41:56.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.843+0000 7fced15f2740 20 client.17529 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:56.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.843+0000 7fced15f2740 1 --2- 172.21.15.38:0/256780483 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5654a939c390 0x5654a97c0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.843+0000 7fced15f2740 4 client.17529 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:41:56.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.843+0000 7fced15f2740 1 -- 172.21.15.38:0/256780483 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654a96a86c0 con 0x5654a939c390 2024-05-21T15:41:56.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.843+0000 7fcec37fe640 1 --2- 172.21.15.38:0/256780483 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5654a939c390 0x5654a97c0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fcec37fe640 1 --2- 172.21.15.38:0/256780483 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5654a939c390 0x5654a97c0fd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c1d48b7fcae81dc server_cookie=af5946b8df01dd88 in_seq=0 out_seq=0 2024-05-21T15:41:56.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fcea37fe640 10 client.17529 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:41:56.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fcea37fe640 1 -- 172.21.15.38:0/256780483 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654a96a86c0 con 0x5654a939c390 2024-05-21T15:41:56.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fcea37fe640 10 client.17529 handle_command_reply: tid=0 2024-05-21T15:41:56.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fced15f2740 1 client.17529 shutdown 2024-05-21T15:41:56.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fced15f2740 1 -- 172.21.15.38:0/256780483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a935af60 msgr2=0x5654a9750a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fced15f2740 1 --2- 172.21.15.38:0/256780483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a935af60 0x5654a9750a20 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fceb40099b0 tx=0x7fceb40097a0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fced15f2740 1 -- 172.21.15.38:0/256780483 shutdown_connections 2024-05-21T15:41:56.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fced15f2740 1 --2- 172.21.15.38:0/256780483 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x5654a939c390 0x5654a97c0fd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fced15f2740 1 --2- 172.21.15.38:0/256780483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a937b6b0 0x5654a974dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fced15f2740 1 --2- 172.21.15.38:0/256780483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654a93734d0 0x5654a974d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fced15f2740 1 --2- 172.21.15.38:0/256780483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654a935af60 0x5654a9750a20 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fcea37fe640 0 client.17529 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:41:56.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fced15f2740 1 -- 172.21.15.38:0/256780483 >> 172.21.15.38:0/256780483 conn(0x5654a93904e0 msgr2=0x5654a973e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fced15f2740 1 -- 172.21.15.38:0/256780483 shutdown_connections 2024-05-21T15:41:56.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fced15f2740 1 -- 172.21.15.38:0/256780483 wait complete. 2024-05-21T15:41:56.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.844+0000 7fced15f2740 20 client.17529 trim_cache size 0 max 16384 2024-05-21T15:41:56.848 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:41:56.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.845+0000 7fceca745640 1 -- 172.21.15.38:0/1351867431 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fceac0a2b90 msgr2=0x7fceac0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.845+0000 7fceca745640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fceac0a2b90 0x7fceac0a50a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x5654a97410e0 tx=0x7fcebc004ab0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.845+0000 7fceca745640 1 -- 172.21.15.38:0/1351867431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec41028b0 msgr2=0x7fcec41997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.845+0000 7fceca745640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec41028b0 0x7fcec41997b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fceb8006010 tx=0x7fceb80042f0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.845+0000 7fceca745640 1 -- 172.21.15.38:0/1351867431 shutdown_connections 2024-05-21T15:41:56.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.845+0000 7fceca745640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcec41033f0 0x7fcec41a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.845+0000 7fceca745640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fceac0a2b90 0x7fceac0a50a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.845+0000 7fceca745640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcec41028b0 0x7fcec41997b0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.845+0000 7fceca745640 1 --2- 172.21.15.38:0/1351867431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcec4101ed0 0x7fcec4199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.845+0000 7fceca745640 1 -- 172.21.15.38:0/1351867431 >> 172.21.15.38:0/1351867431 conn(0x7fcec40fc2f0 msgr2=0x7fcec4103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.845+0000 7fceca745640 1 -- 172.21.15.38:0/1351867431 shutdown_connections 2024-05-21T15:41:56.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.845+0000 7fceca745640 1 -- 172.21.15.38:0/1351867431 wait complete. 2024-05-21T15:41:56.856 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:41:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.987+0000 7fafb5d02640 1 -- 172.21.15.38:0/2634140670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb0104060 msgr2=0x7fafb0104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.987+0000 7fafb5d02640 1 --2- 172.21.15.38:0/2634140670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb0104060 0x7fafb0104470 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009900 tx=0x7fafa002a410 comp rx=0 tx=0).stop 2024-05-21T15:41:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.987+0000 7fafb5d02640 1 -- 172.21.15.38:0/2634140670 shutdown_connections 2024-05-21T15:41:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.987+0000 7fafb5d02640 1 --2- 172.21.15.38:0/2634140670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafb0101210 0x7fafb010ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.987+0000 7fafb5d02640 1 --2- 172.21.15.38:0/2634140670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb0100710 0x7fafb0100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.987+0000 7fafb5d02640 1 --2- 172.21.15.38:0/2634140670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb0104060 0x7fafb0104470 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.987+0000 7fafb5d02640 1 -- 172.21.15.38:0/2634140670 >> 172.21.15.38:0/2634140670 conn(0x7fafb00fc330 msgr2=0x7fafb00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.987+0000 7fafb5d02640 1 -- 172.21.15.38:0/2634140670 shutdown_connections 2024-05-21T15:41:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.987+0000 7fafb5d02640 1 -- 172.21.15.38:0/2634140670 wait complete. 2024-05-21T15:41:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.988+0000 7fafb5d02640 1 Processor -- start 2024-05-21T15:41:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.988+0000 7fafb5d02640 1 -- start start 2024-05-21T15:41:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.988+0000 7fafb5d02640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb0100710 0x7fafb0199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.988+0000 7fafb5d02640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafb0101210 0x7fafb01997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.988+0000 7fafb5d02640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb0104060 0x7fafb01a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.988+0000 7fafb5d02640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fafb010e700 con 0x7fafb0100710 2024-05-21T15:41:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.988+0000 7fafb5d02640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fafb010e580 con 0x7fafb0104060 2024-05-21T15:41:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.988+0000 7fafb5d02640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fafb010e880 con 0x7fafb0101210 2024-05-21T15:41:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.988+0000 7fafaf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb0100710 0x7fafb0199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.988+0000 7fafaf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb0100710 0x7fafb0199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40602/0 (socket says 172.21.15.38:40602) 2024-05-21T15:41:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.988+0000 7fafaf7fe640 1 -- 172.21.15.38:0/2423990995 learned_addr learned my addr 172.21.15.38:0/2423990995 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.988+0000 7fafaeffd640 1 --2- 172.21.15.38:0/2423990995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafb0101210 0x7fafb01997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.988+0000 7fafaffff640 1 --2- 172.21.15.38:0/2423990995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb0104060 0x7fafb01a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.989+0000 7fafaf7fe640 1 -- 172.21.15.38:0/2423990995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafb0101210 msgr2=0x7fafb01997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.989+0000 7fafaf7fe640 1 --2- 172.21.15.38:0/2423990995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafb0101210 0x7fafb01997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.989+0000 7fafaf7fe640 1 -- 172.21.15.38:0/2423990995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb0104060 msgr2=0x7fafb01a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.989+0000 7fafaf7fe640 1 --2- 172.21.15.38:0/2423990995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb0104060 0x7fafb01a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.989+0000 7fafaf7fe640 1 -- 172.21.15.38:0/2423990995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb01a0db0 con 0x7fafb0100710 2024-05-21T15:41:56.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.989+0000 7fafaf7fe640 1 --2- 172.21.15.38:0/2423990995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb0100710 0x7fafb0199270 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fafa0006010 tx=0x7fafa0031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.989+0000 7fafacff9640 1 -- 172.21.15.38:0/2423990995 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa003f030 con 0x7fafb0100710 2024-05-21T15:41:56.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.989+0000 7fafb5d02640 1 -- 172.21.15.38:0/2423990995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb01a1040 con 0x7fafb0100710 2024-05-21T15:41:56.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.989+0000 7fafacff9640 1 -- 172.21.15.38:0/2423990995 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa0003ad0 con 0x7fafb0100710 2024-05-21T15:41:56.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.989+0000 7fafacff9640 1 -- 172.21.15.38:0/2423990995 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0035070 con 0x7fafb0100710 2024-05-21T15:41:56.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.989+0000 7fafb5d02640 1 -- 172.21.15.38:0/2423990995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb01a1530 con 0x7fafb0100710 2024-05-21T15:41:56.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.991+0000 7fafbcbaa740 1 Processor -- start 2024-05-21T15:41:56.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafbcbaa740 1 -- start start 2024-05-21T15:41:56.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafacff9640 1 -- 172.21.15.38:0/2423990995 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fafa0042030 con 0x7fafb0100710 2024-05-21T15:41:56.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafbcbaa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d42240390 0x563d425dfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafbcbaa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d422174d0 0x563d425e0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafbcbaa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d4221f6b0 0x563d425e6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafbcbaa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563d42544c40 con 0x563d42240390 2024-05-21T15:41:56.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafbcbaa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563d424ec580 con 0x563d422174d0 2024-05-21T15:41:56.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafbcbaa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563d42540c00 con 0x563d4221f6b0 2024-05-21T15:41:56.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafaffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d42240390 0x563d425dfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafaffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d42240390 0x563d425dfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40614/0 (socket says 172.21.15.38:40614) 2024-05-21T15:41:56.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafaf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d422174d0 0x563d425e0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafaffff640 1 -- 172.21.15.38:0/4097449114 learned_addr learned my addr 172.21.15.38:0/4097449114 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:56.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafaeffd640 1 --2- 172.21.15.38:0/4097449114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d4221f6b0 0x563d425e6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafaffff640 1 -- 172.21.15.38:0/4097449114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d4221f6b0 msgr2=0x563d425e6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafaffff640 1 --2- 172.21.15.38:0/4097449114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d4221f6b0 0x563d425e6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafaffff640 1 -- 172.21.15.38:0/4097449114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d422174d0 msgr2=0x563d425e0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:56.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafaffff640 1 --2- 172.21.15.38:0/4097449114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d422174d0 0x563d425e0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.992+0000 7fafaffff640 1 -- 172.21.15.38:0/4097449114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d422583f0 con 0x563d42240390 2024-05-21T15:41:56.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.993+0000 7fafacff9640 1 --2- 172.21.15.38:0/2423990995 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faf980a2b90 0x7faf980a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.993+0000 7fafaffff640 1 --2- 172.21.15.38:0/4097449114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d42240390 0x563d425dfb00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fafa8010370 tx=0x7fafa8010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9806cb7a6c18e865 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.993+0000 7fafaf7fe640 1 --2- 172.21.15.38:0/2423990995 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faf980a2b90 0x7faf980a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:56.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.993+0000 7faf95ffb640 1 -- 172.21.15.38:0/4097449114 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8016070 con 0x563d42240390 2024-05-21T15:41:56.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.993+0000 7faf95ffb640 1 -- 172.21.15.38:0/4097449114 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa8004d20 con 0x563d42240390 2024-05-21T15:41:56.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.993+0000 7faf95ffb640 1 -- 172.21.15.38:0/4097449114 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa80053e0 con 0x563d42240390 2024-05-21T15:41:56.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.993+0000 7fafacff9640 1 -- 172.21.15.38:0/2423990995 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fafa00ef540 con 0x7fafb0100710 2024-05-21T15:41:56.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.994+0000 7fafbcbaa740 1 -- 172.21.15.38:0/4097449114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d42240390 msgr2=0x563d425dfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:56.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.994+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/4097449114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d42240390 0x563d425dfb00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fafa8010370 tx=0x7fafa8010830 comp rx=0 tx=0).stop 2024-05-21T15:41:56.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.994+0000 7fafaf7fe640 1 --2- 172.21.15.38:0/2423990995 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faf980a2b90 0x7faf980a50a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x563d425e0ee0 tx=0x7fafa00f4040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:56.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.994+0000 7fafbcbaa740 1 -- 172.21.15.38:0/4097449114 shutdown_connections 2024-05-21T15:41:56.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.994+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/4097449114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d4221f6b0 0x563d425e6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.994+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/4097449114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d422174d0 0x563d425e0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.994+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/4097449114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d42240390 0x563d425dfb00 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:56.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.994+0000 7fafbcbaa740 1 -- 172.21.15.38:0/4097449114 >> 172.21.15.38:0/4097449114 conn(0x563d422344e0 msgr2=0x563d425db490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:56.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.994+0000 7fafbcbaa740 1 -- 172.21.15.38:0/4097449114 shutdown_connections 2024-05-21T15:41:56.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.994+0000 7fafbcbaa740 1 -- 172.21.15.38:0/4097449114 wait complete. 2024-05-21T15:41:56.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.994+0000 7fafbcbaa740 1 -- start start 2024-05-21T15:41:56.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.994+0000 7fafbcbaa740 1 Processor -- start 2024-05-21T15:41:56.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.994+0000 7fafbcbaa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d422174d0 0x563d425e9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.994+0000 7fafbcbaa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4221f6b0 0x563d425e9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafbcbaa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d42240390 0x563d425f0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:56.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafbcbaa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563d425e79d0 con 0x563d42240390 2024-05-21T15:41:57.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafbcbaa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563d424ffc40 con 0x563d4221f6b0 2024-05-21T15:41:57.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafbcbaa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563d425e7b50 con 0x563d422174d0 2024-05-21T15:41:57.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafbcbaa740 10 client.0 _ll_register_callbacks cb 0x563d42257be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:57.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafaffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d42240390 0x563d425f0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafaffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d42240390 0x563d425f0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40618/0 (socket says 172.21.15.38:40618) 2024-05-21T15:41:57.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafaffff640 1 -- 172.21.15.38:0/4104296237 learned_addr learned my addr 172.21.15.38:0/4104296237 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:57.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafaffff640 1 --2- 172.21.15.38:0/4104296237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d422174d0 0x563d425e9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafaeffd640 1 --2- 172.21.15.38:0/4104296237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4221f6b0 0x563d425e9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafaffff640 1 -- 172.21.15.38:0/4104296237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d422174d0 msgr2=0x563d425e9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafaffff640 1 --2- 172.21.15.38:0/4104296237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d422174d0 0x563d425e9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafaffff640 1 -- 172.21.15.38:0/4104296237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4221f6b0 msgr2=0x563d425e9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafaffff640 1 --2- 172.21.15.38:0/4104296237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4221f6b0 0x563d425e9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafaffff640 1 -- 172.21.15.38:0/4104296237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d423227d0 con 0x563d42240390 2024-05-21T15:41:57.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafaeffd640 1 --2- 172.21.15.38:0/4104296237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4221f6b0 0x563d425e9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:41:57.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafaffff640 1 --2- 172.21.15.38:0/4104296237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d42240390 0x563d425f0dd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fafa8010d30 tx=0x7fafa8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90374aa8fb326f53 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7faf8effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:57.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7faf8effd640 1 -- 172.21.15.38:0/4104296237 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8016070 con 0x563d42240390 2024-05-21T15:41:57.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7faf8effd640 1 -- 172.21.15.38:0/4104296237 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fafa8005820 con 0x563d42240390 2024-05-21T15:41:57.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.995+0000 7fafbcbaa740 1 -- 172.21.15.38:0/4104296237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa8004d20 con 0x563d42240390 2024-05-21T15:41:57.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.996+0000 7faf8effd640 1 -- 172.21.15.38:0/4104296237 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fafa8023040 con 0x563d42240390 2024-05-21T15:41:57.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.996+0000 7fafbcbaa740 10 client.17541 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:57.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.996+0000 7fafbcbaa740 1 -- 172.21.15.38:0/4104296237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563d425f4090 con 0x563d42240390 2024-05-21T15:41:57.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.996+0000 7faf8effd640 1 -- 172.21.15.38:0/4104296237 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa8004d20 con 0x563d42240390 2024-05-21T15:41:57.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.996+0000 7faf8effd640 1 -- 172.21.15.38:0/4104296237 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fafa800f040 con 0x563d42240390 2024-05-21T15:41:57.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.996+0000 7fafbcbaa740 10 client.17541 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:57.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.996+0000 7fafbcbaa740 10 client.17541 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:41:57.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.996+0000 7fafbcbaa740 20 client.17541 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:57.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.996+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/4104296237 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x563d421fef60 0x563d425f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.996+0000 7fafbcbaa740 4 client.17541 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:57.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.996+0000 7fafbcbaa740 1 -- 172.21.15.38:0/4104296237 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d42552370 con 0x563d421fef60 2024-05-21T15:41:57.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.996+0000 7fafaeffd640 1 --2- 172.21.15.38:0/4104296237 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x563d421fef60 0x563d425f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.997+0000 7faf8effd640 10 client.17541 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:41:57.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.997+0000 7fafaeffd640 1 --2- 172.21.15.38:0/4104296237 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x563d421fef60 0x563d425f75c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aefae688f8817f47 server_cookie=2c7a526ef38b5202 in_seq=0 out_seq=0 2024-05-21T15:41:57.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7faf8effd640 1 -- 172.21.15.38:0/4104296237 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563d42552370 con 0x563d421fef60 2024-05-21T15:41:57.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7faf8effd640 10 client.17541 handle_command_reply: tid=0 2024-05-21T15:41:57.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7fafbcbaa740 1 client.17541 shutdown 2024-05-21T15:41:57.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7fafbcbaa740 1 -- 172.21.15.38:0/4104296237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d42240390 msgr2=0x563d425f0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/4104296237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d42240390 0x563d425f0dd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fafa8010d30 tx=0x7fafa8009bb0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7fafbcbaa740 1 -- 172.21.15.38:0/4104296237 shutdown_connections 2024-05-21T15:41:57.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/4104296237 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x563d421fef60 0x563d425f75c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/4104296237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d42240390 0x563d425f0dd0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/4104296237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4221f6b0 0x563d425e9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7faf8effd640 0 client.17541 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:41:57.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/4104296237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d422174d0 0x563d425e9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7fafbcbaa740 1 -- 172.21.15.38:0/4104296237 >> 172.21.15.38:0/4104296237 conn(0x563d422344e0 msgr2=0x563d425da570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7fafbcbaa740 1 -- 172.21.15.38:0/4104296237 shutdown_connections 2024-05-21T15:41:57.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7fafbcbaa740 1 -- 172.21.15.38:0/4104296237 wait complete. 2024-05-21T15:41:57.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:56.998+0000 7fafbcbaa740 20 client.17541 trim_cache size 0 max 16384 2024-05-21T15:41:57.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:56 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4241403138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:41:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.016+0000 7fafbcbaa740 1 Processor -- start 2024-05-21T15:41:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.016+0000 7fafbcbaa740 1 -- start start 2024-05-21T15:41:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.016+0000 7fafbcbaa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d422174d0 0x563d425daca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.016+0000 7fafbcbaa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d4221f6b0 0x563d425f0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.016+0000 7fafbcbaa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d42240390 0x563d425db1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.016+0000 7fafbcbaa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fafb01aa470 con 0x563d4221f6b0 2024-05-21T15:41:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.016+0000 7fafbcbaa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fafb01abfb0 con 0x563d42240390 2024-05-21T15:41:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.016+0000 7fafbcbaa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fafb01aab40 con 0x563d422174d0 2024-05-21T15:41:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.016+0000 7fafaeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d4221f6b0 0x563d425f0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.016+0000 7fafaeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d4221f6b0 0x563d425f0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40620/0 (socket says 172.21.15.38:40620) 2024-05-21T15:41:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.016+0000 7fafaeffd640 1 -- 172.21.15.38:0/3167503143 learned_addr learned my addr 172.21.15.38:0/3167503143 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.016+0000 7fafaffff640 1 --2- 172.21.15.38:0/3167503143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d42240390 0x563d425db1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.016+0000 7fafaffff640 1 --2- 172.21.15.38:0/3167503143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d422174d0 0x563d425daca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.016+0000 7fafaeffd640 1 -- 172.21.15.38:0/3167503143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d422174d0 msgr2=0x563d425daca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.017+0000 7fafaeffd640 1 --2- 172.21.15.38:0/3167503143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d422174d0 0x563d425daca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.017+0000 7fafaeffd640 1 -- 172.21.15.38:0/3167503143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d42240390 msgr2=0x563d425db1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.017+0000 7fafaeffd640 1 --2- 172.21.15.38:0/3167503143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d42240390 0x563d425db1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.017+0000 7fafaeffd640 1 -- 172.21.15.38:0/3167503143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d425ea9a0 con 0x563d4221f6b0 2024-05-21T15:41:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.017+0000 7fafaeffd640 1 --2- 172.21.15.38:0/3167503143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d4221f6b0 0x563d425f0cf0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009d60 tx=0x7fafa4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df30618a45eff03 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.017+0000 7faf957fa640 1 -- 172.21.15.38:0/3167503143 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4004030 con 0x563d4221f6b0 2024-05-21T15:41:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.017+0000 7faf957fa640 1 -- 172.21.15.38:0/3167503143 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa4024020 con 0x563d4221f6b0 2024-05-21T15:41:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.017+0000 7faf957fa640 1 -- 172.21.15.38:0/3167503143 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4004d40 con 0x563d4221f6b0 2024-05-21T15:41:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.018+0000 7fafbcbaa740 1 -- 172.21.15.38:0/3167503143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d4221f6b0 msgr2=0x563d425f0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.018+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/3167503143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d4221f6b0 0x563d425f0cf0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009d60 tx=0x7fafa4005f90 comp rx=0 tx=0).stop 2024-05-21T15:41:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.018+0000 7fafbcbaa740 1 -- 172.21.15.38:0/3167503143 shutdown_connections 2024-05-21T15:41:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.018+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/3167503143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d42240390 0x563d425db1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.018+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/3167503143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d4221f6b0 0x563d425f0cf0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.018+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/3167503143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d422174d0 0x563d425daca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.018+0000 7fafbcbaa740 1 -- 172.21.15.38:0/3167503143 >> 172.21.15.38:0/3167503143 conn(0x563d422344e0 msgr2=0x563d425e3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.018+0000 7fafbcbaa740 1 -- 172.21.15.38:0/3167503143 shutdown_connections 2024-05-21T15:41:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.018+0000 7fafbcbaa740 1 -- 172.21.15.38:0/3167503143 wait complete. 2024-05-21T15:41:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.018+0000 7fafbcbaa740 1 -- start start 2024-05-21T15:41:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafbcbaa740 1 Processor -- start 2024-05-21T15:41:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafbcbaa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d422174d0 0x563d425edda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafbcbaa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4221f6b0 0x563d425ee2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafbcbaa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d42240390 0x563d4265d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafbcbaa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fafb01ab8e0 con 0x563d422174d0 2024-05-21T15:41:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafbcbaa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fafb01ab210 con 0x563d4221f6b0 2024-05-21T15:41:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafbcbaa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fafb01a8450 con 0x563d42240390 2024-05-21T15:41:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafbcbaa740 10 client.0 _ll_register_callbacks cb 0x7fafa00f2ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafaffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d422174d0 0x563d425edda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafaffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d422174d0 0x563d425edda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40634/0 (socket says 172.21.15.38:40634) 2024-05-21T15:41:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafaffff640 1 -- 172.21.15.38:0/1880020643 learned_addr learned my addr 172.21.15.38:0/1880020643 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafaeffd640 1 --2- 172.21.15.38:0/1880020643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4221f6b0 0x563d425ee2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafaffff640 1 --2- 172.21.15.38:0/1880020643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d42240390 0x563d4265d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafaeffd640 1 -- 172.21.15.38:0/1880020643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d42240390 msgr2=0x563d4265d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafaeffd640 1 --2- 172.21.15.38:0/1880020643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d42240390 0x563d4265d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafaeffd640 1 -- 172.21.15.38:0/1880020643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d422174d0 msgr2=0x563d425edda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafaeffd640 1 --2- 172.21.15.38:0/1880020643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d422174d0 0x563d425edda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafaeffd640 1 -- 172.21.15.38:0/1880020643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d4265df20 con 0x563d4221f6b0 2024-05-21T15:41:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.019+0000 7fafaffff640 1 --2- 172.21.15.38:0/1880020643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d422174d0 0x563d425edda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.020+0000 7fafaeffd640 1 --2- 172.21.15.38:0/1880020643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4221f6b0 0x563d425ee2e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x563d425f4060 tx=0x7fafa4004bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51b71a2cd153f26d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.020+0000 7faf8effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:41:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.020+0000 7faf8effd640 1 -- 172.21.15.38:0/1880020643 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4004040 con 0x563d4221f6b0 2024-05-21T15:41:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.020+0000 7faf8effd640 1 -- 172.21.15.38:0/1880020643 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fafa4025040 con 0x563d4221f6b0 2024-05-21T15:41:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.020+0000 7fafbcbaa740 1 -- 172.21.15.38:0/1880020643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa4024020 con 0x563d4221f6b0 2024-05-21T15:41:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.020+0000 7faf8effd640 1 -- 172.21.15.38:0/1880020643 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fafa40041e0 con 0x563d4221f6b0 2024-05-21T15:41:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.020+0000 7fafbcbaa740 10 client.27161 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.020+0000 7fafbcbaa740 1 -- 172.21.15.38:0/1880020643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563d4265f240 con 0x563d4221f6b0 2024-05-21T15:41:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.020+0000 7faf8effd640 1 -- 172.21.15.38:0/1880020643 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa4024020 con 0x563d4221f6b0 2024-05-21T15:41:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.020+0000 7faf8effd640 1 -- 172.21.15.38:0/1880020643 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fafa4003b10 con 0x563d4221f6b0 2024-05-21T15:41:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.020+0000 7fafbcbaa740 10 client.27161 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.020+0000 7fafbcbaa740 10 client.27161 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:41:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.020+0000 7fafbcbaa740 20 client.27161 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.021+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/1880020643 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x563d421d43a0 0x563d42661c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.021+0000 7fafbcbaa740 4 client.27161 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:41:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.021+0000 7fafbcbaa740 1 -- 172.21.15.38:0/1880020643 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d4254c6c0 con 0x563d421d43a0 2024-05-21T15:41:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.021+0000 7fafaffff640 1 --2- 172.21.15.38:0/1880020643 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x563d421d43a0 0x563d42661c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.021+0000 7faf8effd640 10 client.27161 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:41:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.021+0000 7fafaffff640 1 --2- 172.21.15.38:0/1880020643 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x563d421d43a0 0x563d42661c00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a15549f14798805 server_cookie=32beb1b4400f7307 in_seq=0 out_seq=0 2024-05-21T15:41:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7faf8effd640 1 -- 172.21.15.38:0/1880020643 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d4254c6c0 con 0x563d421d43a0 2024-05-21T15:41:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7faf8effd640 10 client.27161 handle_command_reply: tid=0 2024-05-21T15:41:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafbcbaa740 1 client.27161 shutdown 2024-05-21T15:41:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafbcbaa740 1 -- 172.21.15.38:0/1880020643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4221f6b0 msgr2=0x563d425ee2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/1880020643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4221f6b0 0x563d425ee2e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x563d425f4060 tx=0x7fafa4004bf0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafbcbaa740 1 -- 172.21.15.38:0/1880020643 shutdown_connections 2024-05-21T15:41:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/1880020643 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x563d421d43a0 0x563d42661c00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/1880020643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563d42240390 0x563d4265d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/1880020643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4221f6b0 0x563d425ee2e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7faf8effd640 0 client.27161 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:41:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafbcbaa740 1 --2- 172.21.15.38:0/1880020643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563d422174d0 0x563d425edda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafbcbaa740 1 -- 172.21.15.38:0/1880020643 >> 172.21.15.38:0/1880020643 conn(0x563d422344e0 msgr2=0x563d425dac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafbcbaa740 1 -- 172.21.15.38:0/1880020643 shutdown_connections 2024-05-21T15:41:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafbcbaa740 1 -- 172.21.15.38:0/1880020643 wait complete. 2024-05-21T15:41:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafbcbaa740 20 client.27161 trim_cache size 0 max 16384 2024-05-21T15:41:57.026 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:41:57.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafb5d02640 1 -- 172.21.15.38:0/2423990995 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faf980a2b90 msgr2=0x7faf980a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafb5d02640 1 --2- 172.21.15.38:0/2423990995 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faf980a2b90 0x7faf980a50a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x563d425e0ee0 tx=0x7fafa00f4040 comp rx=0 tx=0).stop 2024-05-21T15:41:57.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafb5d02640 1 -- 172.21.15.38:0/2423990995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb0100710 msgr2=0x7fafb0199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.022+0000 7fafb5d02640 1 --2- 172.21.15.38:0/2423990995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb0100710 0x7fafb0199270 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fafa0006010 tx=0x7fafa0031b60 comp rx=0 tx=0).stop 2024-05-21T15:41:57.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.023+0000 7fafb5d02640 1 -- 172.21.15.38:0/2423990995 shutdown_connections 2024-05-21T15:41:57.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.023+0000 7fafb5d02640 1 --2- 172.21.15.38:0/2423990995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb0104060 0x7fafb01a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.023+0000 7fafb5d02640 1 --2- 172.21.15.38:0/2423990995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafb0101210 0x7fafb01997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.023+0000 7fafb5d02640 1 --2- 172.21.15.38:0/2423990995 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7faf980a2b90 0x7faf980a50a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.023+0000 7fafb5d02640 1 --2- 172.21.15.38:0/2423990995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb0100710 0x7fafb0199270 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.023+0000 7fafb5d02640 1 -- 172.21.15.38:0/2423990995 >> 172.21.15.38:0/2423990995 conn(0x7fafb00fc330 msgr2=0x7fafb0104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.023+0000 7fafb5d02640 1 -- 172.21.15.38:0/2423990995 shutdown_connections 2024-05-21T15:41:57.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.023+0000 7fafb5d02640 1 -- 172.21.15.38:0/2423990995 wait complete. 2024-05-21T15:41:57.028 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:56 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4241403138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:41:57.033 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:41:57.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.162+0000 7fdae9cf0640 1 -- 172.21.15.38:0/2041084529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdae4102aa0 msgr2=0x7fdae41037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.162+0000 7fdae9cf0640 1 --2- 172.21.15.38:0/2041084529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdae4102aa0 0x7fdae41037a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009ca0 tx=0x7fdad402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.162+0000 7fdae9cf0640 1 -- 172.21.15.38:0/2041084529 shutdown_connections 2024-05-21T15:41:57.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.162+0000 7fdae9cf0640 1 --2- 172.21.15.38:0/2041084529 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae41089e0 0x7fdae410ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.162+0000 7fdae9cf0640 1 --2- 172.21.15.38:0/2041084529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae4103ce0 0x7fdae41040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.162+0000 7fdae9cf0640 1 --2- 172.21.15.38:0/2041084529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdae4102aa0 0x7fdae41037a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.162+0000 7fdae9cf0640 1 -- 172.21.15.38:0/2041084529 >> 172.21.15.38:0/2041084529 conn(0x7fdae40fdc70 msgr2=0x7fdae4100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.162+0000 7fdae9cf0640 1 -- 172.21.15.38:0/2041084529 shutdown_connections 2024-05-21T15:41:57.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.162+0000 7fdae9cf0640 1 -- 172.21.15.38:0/2041084529 wait complete. 2024-05-21T15:41:57.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.163+0000 7fdae9cf0640 1 Processor -- start 2024-05-21T15:41:57.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.163+0000 7fdae9cf0640 1 -- start start 2024-05-21T15:41:57.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.163+0000 7fdae9cf0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4102aa0 0x7fdae4199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae9cf0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae4103ce0 0x7fdae41997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae9cf0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdae41089e0 0x7fdae41a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae9cf0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdae410e6f0 con 0x7fdae4102aa0 2024-05-21T15:41:57.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae9cf0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdae410e570 con 0x7fdae4103ce0 2024-05-21T15:41:57.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae9cf0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdae410e870 con 0x7fdae41089e0 2024-05-21T15:41:57.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae4103ce0 0x7fdae41997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4102aa0 0x7fdae4199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdae41089e0 0x7fdae41a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae4103ce0 0x7fdae41997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:53982/0 (socket says 172.21.15.38:53982) 2024-05-21T15:41:57.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae2ffd640 1 -- 172.21.15.38:0/4070612550 learned_addr learned my addr 172.21.15.38:0/4070612550 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:57.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae2ffd640 1 -- 172.21.15.38:0/4070612550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdae41089e0 msgr2=0x7fdae41a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/4070612550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdae41089e0 0x7fdae41a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae2ffd640 1 -- 172.21.15.38:0/4070612550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4102aa0 msgr2=0x7fdae4199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/4070612550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4102aa0 0x7fdae4199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae2ffd640 1 -- 172.21.15.38:0/4070612550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae41a0dd0 con 0x7fdae4103ce0 2024-05-21T15:41:57.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/4070612550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae4103ce0 0x7fdae41997d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00e9f0 tx=0x7fdacc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.164+0000 7fdae0ff9640 1 -- 172.21.15.38:0/4070612550 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc00cd80 con 0x7fdae4103ce0 2024-05-21T15:41:57.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.165+0000 7fdae9cf0640 1 -- 172.21.15.38:0/4070612550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae41a10c0 con 0x7fdae4103ce0 2024-05-21T15:41:57.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.165+0000 7fdae0ff9640 1 -- 172.21.15.38:0/4070612550 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdacc005500 con 0x7fdae4103ce0 2024-05-21T15:41:57.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.165+0000 7fdae0ff9640 1 -- 172.21.15.38:0/4070612550 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc010690 con 0x7fdae4103ce0 2024-05-21T15:41:57.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.165+0000 7fdae9cf0640 1 -- 172.21.15.38:0/4070612550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae41a1610 con 0x7fdae4103ce0 2024-05-21T15:41:57.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.167+0000 7fdae0ff9640 1 -- 172.21.15.38:0/4070612550 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdacc0046c0 con 0x7fdae4103ce0 2024-05-21T15:41:57.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdaf0b97740 1 Processor -- start 2024-05-21T15:41:57.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdaf0b97740 1 -- start start 2024-05-21T15:41:57.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdae0ff9640 1 --2- 172.21.15.38:0/4070612550 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdad00a2b90 0x7fdad00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdaf0b97740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b749e390 0x55a5b7841f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdaf0b97740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b74754d0 0x55a5b7842490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdaf0b97740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b747d6b0 0x55a5b7849390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdaf0b97740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a5b77a2c40 con 0x55a5b74754d0 2024-05-21T15:41:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdaf0b97740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a5b774a580 con 0x55a5b747d6b0 2024-05-21T15:41:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdaf0b97740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a5b779ec00 con 0x55a5b749e390 2024-05-21T15:41:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdae3fff640 1 --2- 172.21.15.38:0/4070612550 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdad00a2b90 0x7fdad00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b749e390 0x55a5b7841f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdae0ff9640 1 -- 172.21.15.38:0/4070612550 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdacc0cf470 con 0x7fdae4103ce0 2024-05-21T15:41:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b749e390 0x55a5b7841f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:37680/0 (socket says 172.21.15.38:37680) 2024-05-21T15:41:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdae37fe640 1 -- 172.21.15.38:0/4032514450 learned_addr learned my addr 172.21.15.38:0/4032514450 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdae3fff640 1 --2- 172.21.15.38:0/4032514450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b747d6b0 0x55a5b7849390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.168+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/4032514450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b74754d0 0x55a5b7842490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.169+0000 7fdae37fe640 1 -- 172.21.15.38:0/4032514450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b747d6b0 msgr2=0x55a5b7849390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.169+0000 7fdae37fe640 1 --2- 172.21.15.38:0/4032514450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b747d6b0 0x55a5b7849390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.169+0000 7fdae37fe640 1 -- 172.21.15.38:0/4032514450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b74754d0 msgr2=0x55a5b7842490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.169+0000 7fdae37fe640 1 --2- 172.21.15.38:0/4032514450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b74754d0 0x55a5b7842490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.169+0000 7fdae37fe640 1 -- 172.21.15.38:0/4032514450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b74b63f0 con 0x55a5b749e390 2024-05-21T15:41:57.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.169+0000 7fdae3fff640 1 --2- 172.21.15.38:0/4070612550 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdad00a2b90 0x7fdad00a50a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009710 tx=0x7fdadc009a70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.169+0000 7fdae37fe640 1 --2- 172.21.15.38:0/4032514450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b749e390 0x55a5b7841f10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdad4031c10 tx=0x7fdad40093c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b8b3941948d6c1d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.169+0000 7fdad9ffb640 1 -- 172.21.15.38:0/4032514450 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4049070 con 0x55a5b749e390 2024-05-21T15:41:57.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.169+0000 7fdad9ffb640 1 -- 172.21.15.38:0/4032514450 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad4036040 con 0x55a5b749e390 2024-05-21T15:41:57.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.169+0000 7fdad9ffb640 1 -- 172.21.15.38:0/4032514450 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad402fcb0 con 0x55a5b749e390 2024-05-21T15:41:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.170+0000 7fdaf0b97740 1 -- 172.21.15.38:0/4032514450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b749e390 msgr2=0x55a5b7841f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.170+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/4032514450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b749e390 0x55a5b7841f10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdad4031c10 tx=0x7fdad40093c0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.170+0000 7fdaf0b97740 1 -- 172.21.15.38:0/4032514450 shutdown_connections 2024-05-21T15:41:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.170+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/4032514450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b747d6b0 0x55a5b7849390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.170+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/4032514450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b74754d0 0x55a5b7842490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.170+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/4032514450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b749e390 0x55a5b7841f10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.170+0000 7fdaf0b97740 1 -- 172.21.15.38:0/4032514450 >> 172.21.15.38:0/4032514450 conn(0x55a5b74924e0 msgr2=0x55a5b7839490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.170+0000 7fdaf0b97740 1 -- 172.21.15.38:0/4032514450 shutdown_connections 2024-05-21T15:41:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.170+0000 7fdaf0b97740 1 -- 172.21.15.38:0/4032514450 wait complete. 2024-05-21T15:41:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.170+0000 7fdaf0b97740 1 -- start start 2024-05-21T15:41:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdaf0b97740 1 Processor -- start 2024-05-21T15:41:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdaf0b97740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b74754d0 0x55a5b784bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdaf0b97740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b747d6b0 0x55a5b784c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdaf0b97740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b749e390 0x55a5b7852ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdaf0b97740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a5b7849c00 con 0x55a5b747d6b0 2024-05-21T15:41:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdaf0b97740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a5b775dc40 con 0x55a5b74754d0 2024-05-21T15:41:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdaf0b97740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a5b7849d80 con 0x55a5b749e390 2024-05-21T15:41:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdaf0b97740 10 client.0 _ll_register_callbacks cb 0x55a5b74b5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdae2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b747d6b0 0x55a5b784c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b74754d0 0x55a5b784bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdae2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b747d6b0 0x55a5b784c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40686/0 (socket says 172.21.15.38:40686) 2024-05-21T15:41:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdae2ffd640 1 -- 172.21.15.38:0/4207221020 learned_addr learned my addr 172.21.15.38:0/4207221020 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdae3fff640 1 --2- 172.21.15.38:0/4207221020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b749e390 0x55a5b7852ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdae37fe640 1 -- 172.21.15.38:0/4207221020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b749e390 msgr2=0x55a5b7852ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdae37fe640 1 --2- 172.21.15.38:0/4207221020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b749e390 0x55a5b7852ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdae37fe640 1 -- 172.21.15.38:0/4207221020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b747d6b0 msgr2=0x55a5b784c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdae37fe640 1 --2- 172.21.15.38:0/4207221020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b747d6b0 0x55a5b784c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdae37fe640 1 -- 172.21.15.38:0/4207221020 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b75807d0 con 0x55a5b74754d0 2024-05-21T15:41:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdae3fff640 1 --2- 172.21.15.38:0/4207221020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b749e390 0x55a5b7852ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.171+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/4207221020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b747d6b0 0x55a5b784c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.172+0000 7fdae37fe640 1 --2- 172.21.15.38:0/4207221020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b74754d0 0x55a5b784bb50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009ca0 tx=0x7fdad40095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=868d6a12422a34c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.172+0000 7fdac2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:41:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.172+0000 7fdac2ffd640 1 -- 172.21.15.38:0/4207221020 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4049040 con 0x55a5b74754d0 2024-05-21T15:41:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.172+0000 7fdaf0b97740 1 -- 172.21.15.38:0/4207221020 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad4036040 con 0x55a5b74754d0 2024-05-21T15:41:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.172+0000 7fdac2ffd640 1 -- 172.21.15.38:0/4207221020 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdad4031cb0 con 0x55a5b74754d0 2024-05-21T15:41:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.172+0000 7fdac2ffd640 1 -- 172.21.15.38:0/4207221020 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdad402f610 con 0x55a5b74754d0 2024-05-21T15:41:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.172+0000 7fdaf0b97740 10 client.27173 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.172+0000 7fdaf0b97740 1 -- 172.21.15.38:0/4207221020 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a5b7849c00 con 0x55a5b74754d0 2024-05-21T15:41:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.172+0000 7fdac2ffd640 1 -- 172.21.15.38:0/4207221020 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad4036040 con 0x55a5b74754d0 2024-05-21T15:41:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.173+0000 7fdac2ffd640 1 -- 172.21.15.38:0/4207221020 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fdad4030650 con 0x55a5b74754d0 2024-05-21T15:41:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.173+0000 7fdaf0b97740 10 client.27173 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.173+0000 7fdaf0b97740 10 client.27173 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:41:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.173+0000 7fdaf0b97740 20 client.27173 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.173+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/4207221020 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55a5b745cf60 0x55a5b7859640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.173+0000 7fdaf0b97740 4 client.27173 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.173+0000 7fdaf0b97740 1 -- 172.21.15.38:0/4207221020 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5b77b0370 con 0x55a5b745cf60 2024-05-21T15:41:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.173+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/4207221020 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55a5b745cf60 0x55a5b7859640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.173+0000 7fdac2ffd640 10 client.27173 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:41:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.173+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/4207221020 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55a5b745cf60 0x55a5b7859640 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df227eb726d3fcea server_cookie=12c2ec92ab67aebe in_seq=0 out_seq=0 2024-05-21T15:41:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.174+0000 7fdac2ffd640 1 -- 172.21.15.38:0/4207221020 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a5b77b0370 con 0x55a5b745cf60 2024-05-21T15:41:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.174+0000 7fdac2ffd640 10 client.27173 handle_command_reply: tid=0 2024-05-21T15:41:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.174+0000 7fdaf0b97740 1 client.27173 shutdown 2024-05-21T15:41:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.174+0000 7fdaf0b97740 1 -- 172.21.15.38:0/4207221020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b74754d0 msgr2=0x55a5b784bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.174+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/4207221020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b74754d0 0x55a5b784bb50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009ca0 tx=0x7fdad40095c0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.174+0000 7fdaf0b97740 1 -- 172.21.15.38:0/4207221020 shutdown_connections 2024-05-21T15:41:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.174+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/4207221020 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55a5b745cf60 0x55a5b7859640 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.174+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/4207221020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b749e390 0x55a5b7852ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.174+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/4207221020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b747d6b0 0x55a5b784c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.174+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/4207221020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b74754d0 0x55a5b784bb50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.174+0000 7fdac2ffd640 0 client.27173 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:41:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.174+0000 7fdaf0b97740 1 -- 172.21.15.38:0/4207221020 >> 172.21.15.38:0/4207221020 conn(0x55a5b74924e0 msgr2=0x55a5b7838570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.174+0000 7fdaf0b97740 1 -- 172.21.15.38:0/4207221020 shutdown_connections 2024-05-21T15:41:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.174+0000 7fdaf0b97740 1 -- 172.21.15.38:0/4207221020 wait complete. 2024-05-21T15:41:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.175+0000 7fdaf0b97740 20 client.27173 trim_cache size 0 max 16384 2024-05-21T15:41:57.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.189+0000 7fdaf0b97740 1 Processor -- start 2024-05-21T15:41:57.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.189+0000 7fdaf0b97740 1 -- start start 2024-05-21T15:41:57.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.189+0000 7fdaf0b97740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b745cf60 0x55a5b784a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.189+0000 7fdaf0b97740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b74754d0 0x55a5b7852f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.189+0000 7fdaf0b97740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b747d6b0 0x55a5b785ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.189+0000 7fdaf0b97740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdae41a81c0 con 0x55a5b745cf60 2024-05-21T15:41:57.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.189+0000 7fdaf0b97740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdae41a9d00 con 0x55a5b74754d0 2024-05-21T15:41:57.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.189+0000 7fdaf0b97740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdae41a8890 con 0x55a5b747d6b0 2024-05-21T15:41:57.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.189+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b745cf60 0x55a5b784a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.189+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b745cf60 0x55a5b784a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40702/0 (socket says 172.21.15.38:40702) 2024-05-21T15:41:57.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.189+0000 7fdae37fe640 1 -- 172.21.15.38:0/2782177742 learned_addr learned my addr 172.21.15.38:0/2782177742 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:57.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.189+0000 7fdae3fff640 1 --2- 172.21.15.38:0/2782177742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b747d6b0 0x55a5b785ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.190+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/2782177742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b74754d0 0x55a5b7852f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.190+0000 7fdae37fe640 1 -- 172.21.15.38:0/2782177742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b747d6b0 msgr2=0x55a5b785ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.190+0000 7fdae37fe640 1 --2- 172.21.15.38:0/2782177742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b747d6b0 0x55a5b785ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.190+0000 7fdae37fe640 1 -- 172.21.15.38:0/2782177742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b74754d0 msgr2=0x55a5b7852f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.190+0000 7fdae37fe640 1 --2- 172.21.15.38:0/2782177742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b74754d0 0x55a5b7852f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.190+0000 7fdae37fe640 1 -- 172.21.15.38:0/2782177742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b7834d20 con 0x55a5b745cf60 2024-05-21T15:41:57.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.190+0000 7fdae37fe640 1 --2- 172.21.15.38:0/2782177742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b745cf60 0x55a5b784a8b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fdad4031040 tx=0x7fdad402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729d49999e2290ae server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.190+0000 7fdad97fa640 1 -- 172.21.15.38:0/2782177742 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4049070 con 0x55a5b745cf60 2024-05-21T15:41:57.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.190+0000 7fdad97fa640 1 -- 172.21.15.38:0/2782177742 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad4036040 con 0x55a5b745cf60 2024-05-21T15:41:57.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.190+0000 7fdad97fa640 1 -- 172.21.15.38:0/2782177742 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad402fc80 con 0x55a5b745cf60 2024-05-21T15:41:57.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.191+0000 7fdaf0b97740 1 -- 172.21.15.38:0/2782177742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b745cf60 msgr2=0x55a5b784a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.191+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/2782177742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b745cf60 0x55a5b784a8b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fdad4031040 tx=0x7fdad402bfd0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.191+0000 7fdaf0b97740 1 -- 172.21.15.38:0/2782177742 shutdown_connections 2024-05-21T15:41:57.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.191+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/2782177742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b747d6b0 0x55a5b785ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.191+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/2782177742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b74754d0 0x55a5b7852f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.191+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/2782177742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b745cf60 0x55a5b784a8b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.191+0000 7fdaf0b97740 1 -- 172.21.15.38:0/2782177742 >> 172.21.15.38:0/2782177742 conn(0x55a5b74924e0 msgr2=0x55a5b784a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.191+0000 7fdaf0b97740 1 -- 172.21.15.38:0/2782177742 shutdown_connections 2024-05-21T15:41:57.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.191+0000 7fdaf0b97740 1 -- 172.21.15.38:0/2782177742 wait complete. 2024-05-21T15:41:57.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.191+0000 7fdaf0b97740 1 -- start start 2024-05-21T15:41:57.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdaf0b97740 1 Processor -- start 2024-05-21T15:41:57.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdaf0b97740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b745cf60 0x55a5b7852f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdaf0b97740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b74754d0 0x55a5b78515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdaf0b97740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b747d6b0 0x55a5b7851c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdaf0b97740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdae41a9630 con 0x55a5b745cf60 2024-05-21T15:41:57.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdaf0b97740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdae41a8f60 con 0x55a5b747d6b0 2024-05-21T15:41:57.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdaf0b97740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdae41a61a0 con 0x55a5b74754d0 2024-05-21T15:41:57.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdaf0b97740 10 client.0 _ll_register_callbacks cb 0x7fdad4047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:57.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b745cf60 0x55a5b7852f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b745cf60 0x55a5b7852f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40712/0 (socket says 172.21.15.38:40712) 2024-05-21T15:41:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdae37fe640 1 -- 172.21.15.38:0/3466313703 learned_addr learned my addr 172.21.15.38:0/3466313703 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/3466313703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b74754d0 0x55a5b78515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdae3fff640 1 --2- 172.21.15.38:0/3466313703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b747d6b0 0x55a5b7851c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdae37fe640 1 -- 172.21.15.38:0/3466313703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b74754d0 msgr2=0x55a5b78515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdae37fe640 1 --2- 172.21.15.38:0/3466313703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b74754d0 0x55a5b78515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdae37fe640 1 -- 172.21.15.38:0/3466313703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b747d6b0 msgr2=0x55a5b7851c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdae37fe640 1 --2- 172.21.15.38:0/3466313703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b747d6b0 0x55a5b7851c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdae37fe640 1 -- 172.21.15.38:0/3466313703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b7852380 con 0x55a5b745cf60 2024-05-21T15:41:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdae37fe640 1 --2- 172.21.15.38:0/3466313703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b745cf60 0x55a5b7852f80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fdad4030aa0 tx=0x7fdad40319d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de50e539a8b2654c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.192+0000 7fdac2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.193+0000 7fdac2ffd640 1 -- 172.21.15.38:0/3466313703 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4049040 con 0x55a5b745cf60 2024-05-21T15:41:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.193+0000 7fdac2ffd640 1 -- 172.21.15.38:0/3466313703 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdad4030e20 con 0x55a5b745cf60 2024-05-21T15:41:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.193+0000 7fdaf0b97740 1 -- 172.21.15.38:0/3466313703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad4036040 con 0x55a5b745cf60 2024-05-21T15:41:57.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.193+0000 7fdac2ffd640 1 -- 172.21.15.38:0/3466313703 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdad4069050 con 0x55a5b745cf60 2024-05-21T15:41:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.193+0000 7fdaf0b97740 10 client.17565 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.193+0000 7fdaf0b97740 1 -- 172.21.15.38:0/3466313703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a5b78c09c0 con 0x55a5b745cf60 2024-05-21T15:41:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.193+0000 7fdac2ffd640 1 -- 172.21.15.38:0/3466313703 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad4036040 con 0x55a5b745cf60 2024-05-21T15:41:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.193+0000 7fdac2ffd640 1 -- 172.21.15.38:0/3466313703 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fdad40040d0 con 0x55a5b745cf60 2024-05-21T15:41:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.193+0000 7fdaf0b97740 10 client.17565 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.193+0000 7fdaf0b97740 10 client.17565 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:41:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.193+0000 7fdaf0b97740 20 client.17565 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.193+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/3466313703 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55a5b749e390 0x55a5b78c3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.193+0000 7fdaf0b97740 4 client.17565 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:41:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.193+0000 7fdaf0b97740 1 -- 172.21.15.38:0/3466313703 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5b77aa6c0 con 0x55a5b749e390 2024-05-21T15:41:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/3466313703 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55a5b749e390 0x55a5b78c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdac2ffd640 10 client.17565 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:41:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/3466313703 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55a5b749e390 0x55a5b78c3390 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=600c46020b62b8f server_cookie=296f6a0bfd797945 in_seq=0 out_seq=0 2024-05-21T15:41:57.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdac2ffd640 1 -- 172.21.15.38:0/3466313703 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5b77aa6c0 con 0x55a5b749e390 2024-05-21T15:41:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdac2ffd640 10 client.17565 handle_command_reply: tid=0 2024-05-21T15:41:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdaf0b97740 1 client.17565 shutdown 2024-05-21T15:41:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdaf0b97740 1 -- 172.21.15.38:0/3466313703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b745cf60 msgr2=0x55a5b7852f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/3466313703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b745cf60 0x55a5b7852f80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fdad4030aa0 tx=0x7fdad40319d0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdaf0b97740 1 -- 172.21.15.38:0/3466313703 shutdown_connections 2024-05-21T15:41:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/3466313703 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55a5b749e390 0x55a5b78c3390 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/3466313703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a5b747d6b0 0x55a5b7851c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/3466313703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5b74754d0 0x55a5b78515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdac2ffd640 0 client.17565 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:41:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdaf0b97740 1 --2- 172.21.15.38:0/3466313703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a5b745cf60 0x55a5b7852f80 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.194+0000 7fdaf0b97740 1 -- 172.21.15.38:0/3466313703 >> 172.21.15.38:0/3466313703 conn(0x55a5b74924e0 msgr2=0x55a5b783cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdaf0b97740 1 -- 172.21.15.38:0/3466313703 shutdown_connections 2024-05-21T15:41:57.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdaf0b97740 1 -- 172.21.15.38:0/3466313703 wait complete. 2024-05-21T15:41:57.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdaf0b97740 20 client.17565 trim_cache size 0 max 16384 2024-05-21T15:41:57.199 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:41:57.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdae9cf0640 1 -- 172.21.15.38:0/4070612550 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdad00a2b90 msgr2=0x7fdad00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdae9cf0640 1 --2- 172.21.15.38:0/4070612550 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdad00a2b90 0x7fdad00a50a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009710 tx=0x7fdadc009a70 comp rx=0 tx=0).stop 2024-05-21T15:41:57.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdae9cf0640 1 -- 172.21.15.38:0/4070612550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae4103ce0 msgr2=0x7fdae41997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdae9cf0640 1 --2- 172.21.15.38:0/4070612550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae4103ce0 0x7fdae41997d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00e9f0 tx=0x7fdacc00edf0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdae9cf0640 1 -- 172.21.15.38:0/4070612550 shutdown_connections 2024-05-21T15:41:57.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdae9cf0640 1 --2- 172.21.15.38:0/4070612550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdae41089e0 0x7fdae41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdae9cf0640 1 --2- 172.21.15.38:0/4070612550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae4103ce0 0x7fdae41997d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdae9cf0640 1 --2- 172.21.15.38:0/4070612550 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdad00a2b90 0x7fdad00a50a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdae9cf0640 1 --2- 172.21.15.38:0/4070612550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4102aa0 0x7fdae4199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdae9cf0640 1 -- 172.21.15.38:0/4070612550 >> 172.21.15.38:0/4070612550 conn(0x7fdae40fdc70 msgr2=0x7fdae41049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdae9cf0640 1 -- 172.21.15.38:0/4070612550 shutdown_connections 2024-05-21T15:41:57.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.195+0000 7fdae9cf0640 1 -- 172.21.15.38:0/4070612550 wait complete. 2024-05-21T15:41:57.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:56 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4241403138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:41:57.206 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:41:57.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.339+0000 7f28e0a6e640 1 -- 172.21.15.38:0/621141472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc101990 msgr2=0x7f28dc102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.339+0000 7f28e0a6e640 1 --2- 172.21.15.38:0/621141472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc101990 0x7f28dc102690 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009900 tx=0x7f28cc02a410 comp rx=0 tx=0).stop 2024-05-21T15:41:57.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.339+0000 7f28e0a6e640 1 -- 172.21.15.38:0/621141472 shutdown_connections 2024-05-21T15:41:57.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.339+0000 7f28e0a6e640 1 --2- 172.21.15.38:0/621141472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28dc1078d0 0x7f28dc109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.339+0000 7f28e0a6e640 1 --2- 172.21.15.38:0/621141472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28dc102bd0 0x7f28dc102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.339+0000 7f28e0a6e640 1 --2- 172.21.15.38:0/621141472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc101990 0x7f28dc102690 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.339+0000 7f28e0a6e640 1 -- 172.21.15.38:0/621141472 >> 172.21.15.38:0/621141472 conn(0x7f28dc0fcb80 msgr2=0x7f28dc0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.340+0000 7f28e0a6e640 1 -- 172.21.15.38:0/621141472 shutdown_connections 2024-05-21T15:41:57.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.340+0000 7f28e0a6e640 1 -- 172.21.15.38:0/621141472 wait complete. 2024-05-21T15:41:57.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.340+0000 7f28e0a6e640 1 Processor -- start 2024-05-21T15:41:57.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.340+0000 7f28e0a6e640 1 -- start start 2024-05-21T15:41:57.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.340+0000 7f28e0a6e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc101990 0x7f28dc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.340+0000 7f28e0a6e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28dc102bd0 0x7f28dc1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28e0a6e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28dc1078d0 0x7f28dc1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28e0a6e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc10e710 con 0x7f28dc101990 2024-05-21T15:41:57.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28da59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc101990 0x7f28dc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28da59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc101990 0x7f28dc199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40722/0 (socket says 172.21.15.38:40722) 2024-05-21T15:41:57.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28da59c640 1 -- 172.21.15.38:0/1171721364 learned_addr learned my addr 172.21.15.38:0/1171721364 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:57.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28e0a6e640 1 -- 172.21.15.38:0/1171721364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc10e590 con 0x7f28dc1078d0 2024-05-21T15:41:57.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28e0a6e640 1 -- 172.21.15.38:0/1171721364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc10e890 con 0x7f28dc102bd0 2024-05-21T15:41:57.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28dad9d640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28dc1078d0 0x7f28dc1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28d3fff640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28dc102bd0 0x7f28dc1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28da59c640 1 -- 172.21.15.38:0/1171721364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28dc102bd0 msgr2=0x7f28dc1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28da59c640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28dc102bd0 0x7f28dc1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28da59c640 1 -- 172.21.15.38:0/1171721364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28dc1078d0 msgr2=0x7f28dc1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28da59c640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28dc1078d0 0x7f28dc1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28da59c640 1 -- 172.21.15.38:0/1171721364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28dc1a0dd0 con 0x7f28dc101990 2024-05-21T15:41:57.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28dad9d640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28dc1078d0 0x7f28dc1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:57.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.341+0000 7f28da59c640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc101990 0x7f28dc199290 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f28cc006010 tx=0x7f28cc031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.342+0000 7f28d37fe640 1 -- 172.21.15.38:0/1171721364 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc03f030 con 0x7f28dc101990 2024-05-21T15:41:57.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.342+0000 7f28d37fe640 1 -- 172.21.15.38:0/1171721364 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28cc003ad0 con 0x7f28dc101990 2024-05-21T15:41:57.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.342+0000 7f28e0a6e640 1 -- 172.21.15.38:0/1171721364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28dc10a7e0 con 0x7f28dc101990 2024-05-21T15:41:57.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.342+0000 7f28d37fe640 1 -- 172.21.15.38:0/1171721364 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc035070 con 0x7f28dc101990 2024-05-21T15:41:57.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.342+0000 7f28e0a6e640 1 -- 172.21.15.38:0/1171721364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28dc10acd0 con 0x7f28dc101990 2024-05-21T15:41:57.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.345+0000 7f28e79ea740 1 Processor -- start 2024-05-21T15:41:57.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.345+0000 7f28e79ea740 1 -- start start 2024-05-21T15:41:57.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.345+0000 7f28d37fe640 1 -- 172.21.15.38:0/1171721364 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f28cc042030 con 0x7f28dc101990 2024-05-21T15:41:57.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.345+0000 7f28e79ea740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd83390 0x55e5be122b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.345+0000 7f28e79ea740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd5a4d0 0x55e5be123080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.345+0000 7f28e79ea740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd626b0 0x55e5be129f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.345+0000 7f28e79ea740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e5be087c40 con 0x55e5bdd626b0 2024-05-21T15:41:57.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.345+0000 7f28e79ea740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e5be02f580 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.345+0000 7f28e79ea740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e5be083c00 con 0x55e5bdd83390 2024-05-21T15:41:57.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.345+0000 7f28d3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd626b0 0x55e5be129f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.345+0000 7f28da59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd5a4d0 0x55e5be123080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.345+0000 7f28d3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd626b0 0x55e5be129f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40726/0 (socket says 172.21.15.38:40726) 2024-05-21T15:41:57.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.345+0000 7f28d3fff640 1 -- 172.21.15.38:0/1416112608 learned_addr learned my addr 172.21.15.38:0/1416112608 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:57.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28dad9d640 1 --2- 172.21.15.38:0/1416112608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd83390 0x55e5be122b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28d3fff640 1 -- 172.21.15.38:0/1416112608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd83390 msgr2=0x55e5be122b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28d3fff640 1 --2- 172.21.15.38:0/1416112608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd83390 0x55e5be122b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28d3fff640 1 -- 172.21.15.38:0/1416112608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd5a4d0 msgr2=0x55e5be123080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28d3fff640 1 --2- 172.21.15.38:0/1416112608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd5a4d0 0x55e5be123080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28d3fff640 1 -- 172.21.15.38:0/1416112608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5bdd9b3f0 con 0x55e5bdd626b0 2024-05-21T15:41:57.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28dad9d640 1 --2- 172.21.15.38:0/1416112608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd83390 0x55e5be122b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:41:57.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28da59c640 1 --2- 172.21.15.38:0/1416112608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd5a4d0 0x55e5be123080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:57.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28d3fff640 1 --2- 172.21.15.38:0/1416112608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd626b0 0x55e5be129f80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009c90 tx=0x7f28c4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afae9ba0a21edf4a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28d37fe640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28c80a2b90 0x7f28c80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28d0ff9640 1 -- 172.21.15.38:0/1416112608 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4005220 con 0x55e5bdd626b0 2024-05-21T15:41:57.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28dad9d640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28c80a2b90 0x7f28c80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28d0ff9640 1 -- 172.21.15.38:0/1416112608 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c4009e40 con 0x55e5bdd626b0 2024-05-21T15:41:57.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28d0ff9640 1 -- 172.21.15.38:0/1416112608 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4019690 con 0x55e5bdd626b0 2024-05-21T15:41:57.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.346+0000 7f28d37fe640 1 -- 172.21.15.38:0/1171721364 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f28cc0ef590 con 0x7f28dc101990 2024-05-21T15:41:57.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.347+0000 7f28e79ea740 1 -- 172.21.15.38:0/1416112608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd626b0 msgr2=0x55e5be129f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.347+0000 7f28dad9d640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28c80a2b90 0x7f28c80a50a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55e5be123cd0 tx=0x7f28d4016040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.347+0000 7f28e79ea740 1 --2- 172.21.15.38:0/1416112608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd626b0 0x55e5be129f80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009c90 tx=0x7f28c4011810 comp rx=0 tx=0).stop 2024-05-21T15:41:57.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.347+0000 7f28e79ea740 1 -- 172.21.15.38:0/1416112608 shutdown_connections 2024-05-21T15:41:57.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.347+0000 7f28e79ea740 1 --2- 172.21.15.38:0/1416112608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd626b0 0x55e5be129f80 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.347+0000 7f28e79ea740 1 --2- 172.21.15.38:0/1416112608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd5a4d0 0x55e5be123080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.347+0000 7f28e79ea740 1 --2- 172.21.15.38:0/1416112608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd83390 0x55e5be122b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.347+0000 7f28e79ea740 1 -- 172.21.15.38:0/1416112608 >> 172.21.15.38:0/1416112608 conn(0x55e5bdd774e0 msgr2=0x55e5be11e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.347+0000 7f28e79ea740 1 -- 172.21.15.38:0/1416112608 shutdown_connections 2024-05-21T15:41:57.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.347+0000 7f28e79ea740 1 -- 172.21.15.38:0/1416112608 wait complete. 2024-05-21T15:41:57.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28e79ea740 1 -- start start 2024-05-21T15:41:57.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28e79ea740 1 Processor -- start 2024-05-21T15:41:57.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28e79ea740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd5a4d0 0x55e5be12c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28e79ea740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd626b0 0x55e5be12ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28e79ea740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd83390 0x55e5be133dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28e79ea740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e5be12a9d0 con 0x55e5bdd626b0 2024-05-21T15:41:57.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28e79ea740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e5be042c40 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28e79ea740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e5be12ab50 con 0x55e5bdd83390 2024-05-21T15:41:57.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28e79ea740 10 client.0 _ll_register_callbacks cb 0x55e5bdd9abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:57.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28d3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd626b0 0x55e5be12ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28d3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd626b0 0x55e5be12ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40730/0 (socket says 172.21.15.38:40730) 2024-05-21T15:41:57.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28d3fff640 1 -- 172.21.15.38:0/1978000257 learned_addr learned my addr 172.21.15.38:0/1978000257 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:57.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28dad9d640 1 --2- 172.21.15.38:0/1978000257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd83390 0x55e5be133dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28da59c640 1 --2- 172.21.15.38:0/1978000257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd5a4d0 0x55e5be12c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28d3fff640 1 -- 172.21.15.38:0/1978000257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd83390 msgr2=0x55e5be133dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28d3fff640 1 --2- 172.21.15.38:0/1978000257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd83390 0x55e5be133dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28d3fff640 1 -- 172.21.15.38:0/1978000257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd5a4d0 msgr2=0x55e5be12c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28d3fff640 1 --2- 172.21.15.38:0/1978000257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd5a4d0 0x55e5be12c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.348+0000 7f28d3fff640 1 -- 172.21.15.38:0/1978000257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5bde657d0 con 0x55e5bdd626b0 2024-05-21T15:41:57.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28d3fff640 1 --2- 172.21.15.38:0/1978000257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd626b0 0x55e5be12ce60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f28c4000c00 tx=0x7f28c4019c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb464801992ca3de server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:57.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28c1ffb640 1 -- 172.21.15.38:0/1978000257 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4029070 con 0x55e5bdd626b0 2024-05-21T15:41:57.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28c1ffb640 1 -- 172.21.15.38:0/1978000257 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f28c4023840 con 0x55e5bdd626b0 2024-05-21T15:41:57.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28e79ea740 1 -- 172.21.15.38:0/1978000257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e5be095370 con 0x55e5bdd626b0 2024-05-21T15:41:57.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28c1ffb640 1 -- 172.21.15.38:0/1978000257 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f28c4035050 con 0x55e5bdd626b0 2024-05-21T15:41:57.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28e79ea740 10 client.17577 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:57.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28e79ea740 1 -- 172.21.15.38:0/1978000257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e5be1370b0 con 0x55e5bdd626b0 2024-05-21T15:41:57.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28c1ffb640 1 -- 172.21.15.38:0/1978000257 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e5be095370 con 0x55e5bdd626b0 2024-05-21T15:41:57.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28c1ffb640 1 -- 172.21.15.38:0/1978000257 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f28c4022930 con 0x55e5bdd626b0 2024-05-21T15:41:57.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28e79ea740 10 client.17577 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:57.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28e79ea740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:57.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28e79ea740 20 client.17577 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:57.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28e79ea740 1 --2- 172.21.15.38:0/1978000257 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e5bdd41f60 0x55e5be13a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28e79ea740 4 client.17577 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:57.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.349+0000 7f28e79ea740 1 -- 172.21.15.38:0/1978000257 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5be08f6c0 con 0x55e5bdd41f60 2024-05-21T15:41:57.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.350+0000 7f28da59c640 1 --2- 172.21.15.38:0/1978000257 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e5bdd41f60 0x55e5be13a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.350+0000 7f28c1ffb640 10 client.17577 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.350+0000 7f28da59c640 1 --2- 172.21.15.38:0/1978000257 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e5bdd41f60 0x55e5be13a610 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=261f2d1ef91a925d server_cookie=788f6c62bdedc187 in_seq=0 out_seq=0 2024-05-21T15:41:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28c1ffb640 1 -- 172.21.15.38:0/1978000257 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e5be08f6c0 con 0x55e5bdd41f60 2024-05-21T15:41:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28c1ffb640 10 client.17577 handle_command_reply: tid=0 2024-05-21T15:41:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28e79ea740 1 client.17577 shutdown 2024-05-21T15:41:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28e79ea740 1 -- 172.21.15.38:0/1978000257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd626b0 msgr2=0x55e5be12ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28e79ea740 1 --2- 172.21.15.38:0/1978000257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd626b0 0x55e5be12ce60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f28c4000c00 tx=0x7f28c4019c70 comp rx=0 tx=0).stop 2024-05-21T15:41:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28e79ea740 1 -- 172.21.15.38:0/1978000257 shutdown_connections 2024-05-21T15:41:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28e79ea740 1 --2- 172.21.15.38:0/1978000257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd83390 0x55e5be133dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28e79ea740 1 --2- 172.21.15.38:0/1978000257 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e5bdd41f60 0x55e5be13a610 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28e79ea740 1 --2- 172.21.15.38:0/1978000257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd626b0 0x55e5be12ce60 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28e79ea740 1 --2- 172.21.15.38:0/1978000257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd5a4d0 0x55e5be12c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28c1ffb640 0 client.17577 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28e79ea740 1 -- 172.21.15.38:0/1978000257 >> 172.21.15.38:0/1978000257 conn(0x55e5bdd774e0 msgr2=0x55e5be11d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28e79ea740 1 -- 172.21.15.38:0/1978000257 shutdown_connections 2024-05-21T15:41:57.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28e79ea740 1 -- 172.21.15.38:0/1978000257 wait complete. 2024-05-21T15:41:57.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.351+0000 7f28e79ea740 20 client.17577 trim_cache size 0 max 16384 2024-05-21T15:41:57.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.369+0000 7f28e79ea740 1 Processor -- start 2024-05-21T15:41:57.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.369+0000 7f28e79ea740 1 -- start start 2024-05-21T15:41:57.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.369+0000 7f28e79ea740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd5a4d0 0x55e5be12bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.369+0000 7f28e79ea740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd626b0 0x55e5be133ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.369+0000 7f28e79ea740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd83390 0x55e5be13ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28e79ea740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1a8200 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28e79ea740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1a9d40 con 0x55e5bdd83390 2024-05-21T15:41:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28e79ea740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1a88d0 con 0x55e5bdd626b0 2024-05-21T15:41:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28da59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd5a4d0 0x55e5be12bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28dad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd83390 0x55e5be13ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28da59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd5a4d0 0x55e5be12bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40746/0 (socket says 172.21.15.38:40746) 2024-05-21T15:41:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28dad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd83390 0x55e5be13ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:54056/0 (socket says 172.21.15.38:54056) 2024-05-21T15:41:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28da59c640 1 -- 172.21.15.38:0/4206301121 learned_addr learned my addr 172.21.15.38:0/4206301121 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28d3fff640 1 --2- 172.21.15.38:0/4206301121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd626b0 0x55e5be133ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28da59c640 1 -- 172.21.15.38:0/4206301121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd626b0 msgr2=0x55e5be133ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28da59c640 1 --2- 172.21.15.38:0/4206301121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd626b0 0x55e5be133ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28da59c640 1 -- 172.21.15.38:0/4206301121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd83390 msgr2=0x55e5be13ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28da59c640 1 --2- 172.21.15.38:0/4206301121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd83390 0x55e5be13ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28da59c640 1 -- 172.21.15.38:0/4206301121 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5be12d520 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28dad9d640 1 --2- 172.21.15.38:0/4206301121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd83390 0x55e5be13ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:57.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.370+0000 7f28da59c640 1 --2- 172.21.15.38:0/4206301121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd5a4d0 0x55e5be12bf90 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f28cc005ab0 tx=0x7f28cc03b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe15d7881fc6bd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.371+0000 7f28c3fff640 1 -- 172.21.15.38:0/4206301121 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc0bf070 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.371+0000 7f28c3fff640 1 -- 172.21.15.38:0/4206301121 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28cc03b460 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.371+0000 7f28c3fff640 1 -- 172.21.15.38:0/4206301121 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc049040 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 -- 172.21.15.38:0/4206301121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd5a4d0 msgr2=0x55e5be12bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 --2- 172.21.15.38:0/4206301121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd5a4d0 0x55e5be12bf90 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f28cc005ab0 tx=0x7f28cc03b800 comp rx=0 tx=0).stop 2024-05-21T15:41:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 -- 172.21.15.38:0/4206301121 shutdown_connections 2024-05-21T15:41:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 --2- 172.21.15.38:0/4206301121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd83390 0x55e5be13ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 --2- 172.21.15.38:0/4206301121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd626b0 0x55e5be133ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 --2- 172.21.15.38:0/4206301121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd5a4d0 0x55e5be12bf90 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 -- 172.21.15.38:0/4206301121 >> 172.21.15.38:0/4206301121 conn(0x55e5bdd774e0 msgr2=0x55e5be121d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 -- 172.21.15.38:0/4206301121 shutdown_connections 2024-05-21T15:41:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 -- 172.21.15.38:0/4206301121 wait complete. 2024-05-21T15:41:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 -- start start 2024-05-21T15:41:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 Processor -- start 2024-05-21T15:41:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd5a4d0 0x55e5be130ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd626b0 0x55e5be131010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd83390 0x55e5be1a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1a9670 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1a8fa0 con 0x55e5bdd83390 2024-05-21T15:41:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1a61e0 con 0x55e5bdd626b0 2024-05-21T15:41:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28e79ea740 10 client.0 _ll_register_callbacks cb 0x7f28c4009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.372+0000 7f28da59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd5a4d0 0x55e5be130ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28d3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd626b0 0x55e5be131010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28da59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd5a4d0 0x55e5be130ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40762/0 (socket says 172.21.15.38:40762) 2024-05-21T15:41:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28da59c640 1 -- 172.21.15.38:0/3254980650 learned_addr learned my addr 172.21.15.38:0/3254980650 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28dad9d640 1 --2- 172.21.15.38:0/3254980650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd83390 0x55e5be1a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28da59c640 1 -- 172.21.15.38:0/3254980650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd626b0 msgr2=0x55e5be131010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28da59c640 1 --2- 172.21.15.38:0/3254980650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd626b0 0x55e5be131010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28da59c640 1 -- 172.21.15.38:0/3254980650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd83390 msgr2=0x55e5be1a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28da59c640 1 --2- 172.21.15.38:0/3254980650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd83390 0x55e5be1a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28da59c640 1 -- 172.21.15.38:0/3254980650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5be133fd0 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28da59c640 1 --2- 172.21.15.38:0/3254980650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd5a4d0 0x55e5be130ad0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f28cc031440 tx=0x7f28cc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df19b70ca6328a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28c1ffb640 1 -- 172.21.15.38:0/3254980650 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc0bf070 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28c1ffb640 1 -- 172.21.15.38:0/3254980650 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f28cc003c80 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28e79ea740 1 -- 172.21.15.38:0/3254980650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28cc03b460 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28c1ffb640 1 -- 172.21.15.38:0/3254980650 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f28cc049070 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28e79ea740 10 client.17583 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28e79ea740 1 -- 172.21.15.38:0/3254980650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e5be1a1eb0 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.373+0000 7f28c1ffb640 1 -- 172.21.15.38:0/3254980650 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28cc03b460 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.374+0000 7f28c1ffb640 1 -- 172.21.15.38:0/3254980650 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f28cc0bc140 con 0x55e5bdd5a4d0 2024-05-21T15:41:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.374+0000 7f28e79ea740 10 client.17583 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:57.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.374+0000 7f28e79ea740 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:57.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.374+0000 7f28e79ea740 20 client.17583 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:57.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.374+0000 7f28e79ea740 1 --2- 172.21.15.38:0/3254980650 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e5bdd173a0 0x55e5be1a4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:57.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.374+0000 7f28e79ea740 4 client.17583 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:41:57.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.374+0000 7f28e79ea740 1 -- 172.21.15.38:0/3254980650 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5be00c820 con 0x55e5bdd173a0 2024-05-21T15:41:57.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.374+0000 7f28d3fff640 1 --2- 172.21.15.38:0/3254980650 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e5bdd173a0 0x55e5be1a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:57.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.374+0000 7f28c1ffb640 10 client.17583 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:57.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.374+0000 7f28d3fff640 1 --2- 172.21.15.38:0/3254980650 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e5bdd173a0 0x55e5be1a4850 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad44a78c3797d17 server_cookie=360e252edd1e8755 in_seq=0 out_seq=0 2024-05-21T15:41:57.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.374+0000 7f28c1ffb640 1 -- 172.21.15.38:0/3254980650 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5be00c820 con 0x55e5bdd173a0 2024-05-21T15:41:57.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.374+0000 7f28c1ffb640 10 client.17583 handle_command_reply: tid=0 2024-05-21T15:41:57.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e79ea740 1 client.17583 shutdown 2024-05-21T15:41:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e79ea740 1 -- 172.21.15.38:0/3254980650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd5a4d0 msgr2=0x55e5be130ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e79ea740 1 --2- 172.21.15.38:0/3254980650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd5a4d0 0x55e5be130ad0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f28cc031440 tx=0x7f28cc02a410 comp rx=0 tx=0).stop 2024-05-21T15:41:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e79ea740 1 -- 172.21.15.38:0/3254980650 shutdown_connections 2024-05-21T15:41:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e79ea740 1 --2- 172.21.15.38:0/3254980650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bdd83390 0x55e5be1a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e79ea740 1 --2- 172.21.15.38:0/3254980650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5bdd626b0 0x55e5be131010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e79ea740 1 --2- 172.21.15.38:0/3254980650 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55e5bdd173a0 0x55e5be1a4850 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e79ea740 1 --2- 172.21.15.38:0/3254980650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5bdd5a4d0 0x55e5be130ad0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28c1ffb640 0 client.17583 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e79ea740 1 -- 172.21.15.38:0/3254980650 >> 172.21.15.38:0/3254980650 conn(0x55e5bdd774e0 msgr2=0x55e5be121ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e79ea740 1 -- 172.21.15.38:0/3254980650 shutdown_connections 2024-05-21T15:41:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e79ea740 1 -- 172.21.15.38:0/3254980650 wait complete. 2024-05-21T15:41:57.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e79ea740 20 client.17583 trim_cache size 0 max 16384 2024-05-21T15:41:57.380 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:41:57.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e0a6e640 1 -- 172.21.15.38:0/1171721364 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28c80a2b90 msgr2=0x7f28c80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e0a6e640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28c80a2b90 0x7f28c80a50a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55e5be123cd0 tx=0x7f28d4016040 comp rx=0 tx=0).stop 2024-05-21T15:41:57.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e0a6e640 1 -- 172.21.15.38:0/1171721364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc101990 msgr2=0x7f28dc199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:57.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e0a6e640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc101990 0x7f28dc199290 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f28cc006010 tx=0x7f28cc031b60 comp rx=0 tx=0).stop 2024-05-21T15:41:57.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e0a6e640 1 -- 172.21.15.38:0/1171721364 shutdown_connections 2024-05-21T15:41:57.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e0a6e640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28dc1078d0 0x7f28dc1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e0a6e640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28dc102bd0 0x7f28dc1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e0a6e640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f28c80a2b90 0x7f28c80a50a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e0a6e640 1 --2- 172.21.15.38:0/1171721364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc101990 0x7f28dc199290 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:57.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.375+0000 7f28e0a6e640 1 -- 172.21.15.38:0/1171721364 >> 172.21.15.38:0/1171721364 conn(0x7f28dc0fcb80 msgr2=0x7f28dc1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:57.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.376+0000 7f28e0a6e640 1 -- 172.21.15.38:0/1171721364 shutdown_connections 2024-05-21T15:41:57.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:57.376+0000 7f28e0a6e640 1 -- 172.21.15.38:0/1171721364 wait complete. 2024-05-21T15:41:58.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:57 smithi038 ceph-mon[31995]: pgmap v577: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 4.81k op/s 2024-05-21T15:41:58.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:57 smithi053 ceph-mon[36496]: pgmap v577: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 4.81k op/s 2024-05-21T15:41:58.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:57 smithi045 ceph-mon[36409]: pgmap v577: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 4.81k op/s 2024-05-21T15:41:58.388 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:41:58.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:41:58.388 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:41:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.524+0000 7fd1b8db5640 1 -- 172.21.15.38:0/1515735245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1b41078e0 msgr2=0x7fd1b4109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.524+0000 7fd1b8db5640 1 --2- 172.21.15.38:0/1515735245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1b41078e0 0x7fd1b4109d00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009ca0 tx=0x7fd1ac02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.524+0000 7fd1b8db5640 1 -- 172.21.15.38:0/1515735245 shutdown_connections 2024-05-21T15:41:58.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.524+0000 7fd1b8db5640 1 --2- 172.21.15.38:0/1515735245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1b41078e0 0x7fd1b4109d00 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.524+0000 7fd1b8db5640 1 --2- 172.21.15.38:0/1515735245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b4102be0 0x7fd1b4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.524+0000 7fd1b8db5640 1 --2- 172.21.15.38:0/1515735245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1b41019a0 0x7fd1b41026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.524+0000 7fd1b8db5640 1 -- 172.21.15.38:0/1515735245 >> 172.21.15.38:0/1515735245 conn(0x7fd1b40fcb90 msgr2=0x7fd1b40ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:58.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.524+0000 7fd1b8db5640 1 -- 172.21.15.38:0/1515735245 shutdown_connections 2024-05-21T15:41:58.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.524+0000 7fd1b8db5640 1 -- 172.21.15.38:0/1515735245 wait complete. 2024-05-21T15:41:58.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.525+0000 7fd1b8db5640 1 Processor -- start 2024-05-21T15:41:58.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.525+0000 7fd1b8db5640 1 -- start start 2024-05-21T15:41:58.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.525+0000 7fd1b8db5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1b41019a0 0x7fd1b41991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.525+0000 7fd1b8db5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b4102be0 0x7fd1b4199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.525+0000 7fd1b8db5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1b41078e0 0x7fd1b41a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.525+0000 7fd1b8db5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b410d600 con 0x7fd1b41019a0 2024-05-21T15:41:58.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.525+0000 7fd1b8db5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b410d480 con 0x7fd1b41078e0 2024-05-21T15:41:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.525+0000 7fd1b8db5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b410d780 con 0x7fd1b4102be0 2024-05-21T15:41:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.525+0000 7fd1b259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1b41019a0 0x7fd1b41991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.525+0000 7fd1b259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1b41019a0 0x7fd1b41991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40778/0 (socket says 172.21.15.38:40778) 2024-05-21T15:41:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.525+0000 7fd1b259c640 1 -- 172.21.15.38:0/1298238492 learned_addr learned my addr 172.21.15.38:0/1298238492 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.525+0000 7fd1b2d9d640 1 --2- 172.21.15.38:0/1298238492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1b41078e0 0x7fd1b41a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.525+0000 7fd1abfff640 1 --2- 172.21.15.38:0/1298238492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b4102be0 0x7fd1b4199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.526+0000 7fd1b259c640 1 -- 172.21.15.38:0/1298238492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b4102be0 msgr2=0x7fd1b4199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.526+0000 7fd1b259c640 1 --2- 172.21.15.38:0/1298238492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b4102be0 0x7fd1b4199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.526+0000 7fd1b259c640 1 -- 172.21.15.38:0/1298238492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1b41078e0 msgr2=0x7fd1b41a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.526+0000 7fd1b259c640 1 --2- 172.21.15.38:0/1298238492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1b41078e0 0x7fd1b41a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.526+0000 7fd1b259c640 1 -- 172.21.15.38:0/1298238492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b41a0d30 con 0x7fd1b41019a0 2024-05-21T15:41:58.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.526+0000 7fd1b259c640 1 --2- 172.21.15.38:0/1298238492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1b41019a0 0x7fd1b41991f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400e9f0 tx=0x7fd1a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.526+0000 7fd1ab7fe640 1 -- 172.21.15.38:0/1298238492 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a400ccc0 con 0x7fd1b41019a0 2024-05-21T15:41:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.526+0000 7fd1ab7fe640 1 -- 172.21.15.38:0/1298238492 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a40035f0 con 0x7fd1b41019a0 2024-05-21T15:41:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.526+0000 7fd1b8db5640 1 -- 172.21.15.38:0/1298238492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b41a0fc0 con 0x7fd1b41019a0 2024-05-21T15:41:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.526+0000 7fd1ab7fe640 1 -- 172.21.15.38:0/1298238492 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a400ce60 con 0x7fd1b41019a0 2024-05-21T15:41:58.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.526+0000 7fd1b8db5640 1 -- 172.21.15.38:0/1298238492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b41a1480 con 0x7fd1b41019a0 2024-05-21T15:41:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.529+0000 7fd1bfc65740 1 Processor -- start 2024-05-21T15:41:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1bfc65740 1 -- start start 2024-05-21T15:41:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1bfc65740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52d16410 0x560c530b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1bfc65740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52ced550 0x560c530b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1bfc65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52cf5730 0x560c530bd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1bfc65740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560c5301a990 con 0x560c52ced550 2024-05-21T15:41:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1bfc65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560c52fc25b0 con 0x560c52cf5730 2024-05-21T15:41:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1bfc65740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560c53016950 con 0x560c52d16410 2024-05-21T15:41:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1b2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52d16410 0x560c530b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1abfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52cf5730 0x560c530bd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1b2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52d16410 0x560c530b64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:37776/0 (socket says 172.21.15.38:37776) 2024-05-21T15:41:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1abfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52cf5730 0x560c530bd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:54106/0 (socket says 172.21.15.38:54106) 2024-05-21T15:41:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1b2d9d640 1 -- 172.21.15.38:0/2342731788 learned_addr learned my addr 172.21.15.38:0/2342731788 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1b2d9d640 1 -- 172.21.15.38:0/2342731788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52cf5730 msgr2=0x560c530bd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1b2d9d640 1 --2- 172.21.15.38:0/2342731788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52cf5730 0x560c530bd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1ab7fe640 1 -- 172.21.15.38:0/1298238492 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd1a4004020 con 0x7fd1b41019a0 2024-05-21T15:41:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1b2d9d640 1 -- 172.21.15.38:0/2342731788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52ced550 msgr2=0x560c530b6a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:41:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1b2d9d640 1 --2- 172.21.15.38:0/2342731788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52ced550 0x560c530b6a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.530+0000 7fd1b2d9d640 1 -- 172.21.15.38:0/2342731788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c52df67e0 con 0x560c52d16410 2024-05-21T15:41:58.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.531+0000 7fd1b2d9d640 1 --2- 172.21.15.38:0/2342731788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52d16410 0x560c530b64a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac035000 tx=0x7fd1ac004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=239b64124f20cb06 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:58.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.531+0000 7fd1a8ff9640 1 -- 172.21.15.38:0/2342731788 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac049070 con 0x560c52d16410 2024-05-21T15:41:58.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.531+0000 7fd1a8ff9640 1 -- 172.21.15.38:0/2342731788 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ac036040 con 0x560c52d16410 2024-05-21T15:41:58.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.531+0000 7fd1a8ff9640 1 -- 172.21.15.38:0/2342731788 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac02fbe0 con 0x560c52d16410 2024-05-21T15:41:58.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.531+0000 7fd1ab7fe640 1 --2- 172.21.15.38:0/1298238492 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd1a00a2b90 0x7fd1a00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.531+0000 7fd1b259c640 1 --2- 172.21.15.38:0/1298238492 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd1a00a2b90 0x7fd1a00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.532+0000 7fd1ab7fe640 1 -- 172.21.15.38:0/1298238492 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fd1a40cf350 con 0x7fd1b41019a0 2024-05-21T15:41:58.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.532+0000 7fd1bfc65740 1 -- 172.21.15.38:0/2342731788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52d16410 msgr2=0x560c530b64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.532+0000 7fd1b259c640 1 --2- 172.21.15.38:0/1298238492 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd1a00a2b90 0x7fd1a00a50a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400e9c0 tx=0x7fd1a401bb30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:58.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.532+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/2342731788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52d16410 0x560c530b64a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac035000 tx=0x7fd1ac004a40 comp rx=0 tx=0).stop 2024-05-21T15:41:58.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.532+0000 7fd1bfc65740 1 -- 172.21.15.38:0/2342731788 shutdown_connections 2024-05-21T15:41:58.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.532+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/2342731788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52cf5730 0x560c530bd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.532+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/2342731788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52ced550 0x560c530b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.532+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/2342731788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52d16410 0x560c530b64a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.532+0000 7fd1bfc65740 1 -- 172.21.15.38:0/2342731788 >> 172.21.15.38:0/2342731788 conn(0x560c52d0a560 msgr2=0x560c530b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:58.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.532+0000 7fd1bfc65740 1 -- 172.21.15.38:0/2342731788 shutdown_connections 2024-05-21T15:41:58.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.532+0000 7fd1bfc65740 1 -- 172.21.15.38:0/2342731788 wait complete. 2024-05-21T15:41:58.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1bfc65740 1 -- start start 2024-05-21T15:41:58.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1bfc65740 1 Processor -- start 2024-05-21T15:41:58.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1bfc65740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52ced550 0x560c530c0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1bfc65740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 0x560c530c0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1bfc65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52d16410 0x560c530c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1bfc65740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560c530be190 con 0x560c52ced550 2024-05-21T15:41:58.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1bfc65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560c52fd5b40 con 0x560c52d16410 2024-05-21T15:41:58.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1bfc65740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560c530be310 con 0x560c52cf5730 2024-05-21T15:41:58.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1bfc65740 10 client.0 _ll_register_callbacks cb 0x560c52fca0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:58.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1b2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52ced550 0x560c530c0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1b2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52ced550 0x560c530c0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40788/0 (socket says 172.21.15.38:40788) 2024-05-21T15:41:58.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1b2d9d640 1 -- 172.21.15.38:0/331155822 learned_addr learned my addr 172.21.15.38:0/331155822 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:58.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1abfff640 1 --2- 172.21.15.38:0/331155822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 0x560c530c0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1b2d9d640 1 --2- 172.21.15.38:0/331155822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52d16410 0x560c530c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1b2d9d640 1 -- 172.21.15.38:0/331155822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 msgr2=0x560c530c0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1b2d9d640 1 --2- 172.21.15.38:0/331155822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 0x560c530c0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1b2d9d640 1 -- 172.21.15.38:0/331155822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52d16410 msgr2=0x560c530c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1b2d9d640 1 --2- 172.21.15.38:0/331155822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52d16410 0x560c530c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.533+0000 7fd1b2d9d640 1 -- 172.21.15.38:0/331155822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c52e56f70 con 0x560c52ced550 2024-05-21T15:41:58.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.534+0000 7fd1b2d9d640 1 --2- 172.21.15.38:0/331155822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52ced550 0x560c530c0240 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac005e30 tx=0x7fd1ac031f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8dc42da65e49198 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:58.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.534+0000 7fd199ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:41:58.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.534+0000 7fd199ffb640 1 -- 172.21.15.38:0/331155822 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac053060 con 0x560c52ced550 2024-05-21T15:41:58.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.534+0000 7fd199ffb640 1 -- 172.21.15.38:0/331155822 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fd1ac003970 con 0x560c52ced550 2024-05-21T15:41:58.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.534+0000 7fd1bfc65740 1 -- 172.21.15.38:0/331155822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1ac036040 con 0x560c52ced550 2024-05-21T15:41:58.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.534+0000 7fd199ffb640 1 -- 172.21.15.38:0/331155822 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd1ac049040 con 0x560c52ced550 2024-05-21T15:41:58.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.534+0000 7fd1bfc65740 10 client.17592 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:58.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.534+0000 7fd1bfc65740 1 -- 172.21.15.38:0/331155822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560c52cee170 con 0x560c52ced550 2024-05-21T15:41:58.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.534+0000 7fd199ffb640 1 -- 172.21.15.38:0/331155822 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ac036040 con 0x560c52ced550 2024-05-21T15:41:58.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.534+0000 7fd199ffb640 1 -- 172.21.15.38:0/331155822 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fd1ac004040 con 0x560c52ced550 2024-05-21T15:41:58.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.535+0000 7fd1bfc65740 10 client.17592 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:58.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.535+0000 7fd1bfc65740 10 client.17592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:58.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.535+0000 7fd1bfc65740 20 client.17592 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:58.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.535+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/331155822 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c52cd51f0 0x560c530cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.535+0000 7fd1bfc65740 4 client.17592 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:58.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.535+0000 7fd1bfc65740 1 -- 172.21.15.38:0/331155822 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c530280c0 con 0x560c52cd51f0 2024-05-21T15:41:58.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.535+0000 7fd1abfff640 1 --2- 172.21.15.38:0/331155822 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c52cd51f0 0x560c530cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.535+0000 7fd199ffb640 10 client.17592 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:58.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.535+0000 7fd1abfff640 1 --2- 172.21.15.38:0/331155822 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c52cd51f0 0x560c530cd7f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33495a3b5c613f9b server_cookie=ac0b50ab8590137 in_seq=0 out_seq=0 2024-05-21T15:41:58.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd199ffb640 1 -- 172.21.15.38:0/331155822 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560c530280c0 con 0x560c52cd51f0 2024-05-21T15:41:58.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd199ffb640 10 client.17592 handle_command_reply: tid=0 2024-05-21T15:41:58.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd1bfc65740 1 client.17592 shutdown 2024-05-21T15:41:58.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd1bfc65740 1 -- 172.21.15.38:0/331155822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52ced550 msgr2=0x560c530c0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/331155822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52ced550 0x560c530c0240 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac005e30 tx=0x7fd1ac031f40 comp rx=0 tx=0).stop 2024-05-21T15:41:58.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd1bfc65740 1 -- 172.21.15.38:0/331155822 shutdown_connections 2024-05-21T15:41:58.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/331155822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52d16410 0x560c530c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/331155822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 0x560c530c0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/331155822 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c52cd51f0 0x560c530cd7f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/331155822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52ced550 0x560c530c0240 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd199ffb640 0 client.17592 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:58.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd1bfc65740 1 -- 172.21.15.38:0/331155822 >> 172.21.15.38:0/331155822 conn(0x560c52d0a560 msgr2=0x560c530b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:58.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd1bfc65740 1 -- 172.21.15.38:0/331155822 shutdown_connections 2024-05-21T15:41:58.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd1bfc65740 1 -- 172.21.15.38:0/331155822 wait complete. 2024-05-21T15:41:58.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.536+0000 7fd1bfc65740 20 client.17592 trim_cache size 0 max 16384 2024-05-21T15:41:58.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.555+0000 7fd1bfc65740 1 Processor -- start 2024-05-21T15:41:58.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.555+0000 7fd1bfc65740 1 -- start start 2024-05-21T15:41:58.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.555+0000 7fd1bfc65740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52cd51f0 0x560c530b9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.555+0000 7fd1bfc65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52ced550 0x560c530bae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1bfc65740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 0x560c530c0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1bfc65740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41aa370 con 0x560c52cd51f0 2024-05-21T15:41:58.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1bfc65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41abeb0 con 0x560c52ced550 2024-05-21T15:41:58.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1bfc65740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41aaa40 con 0x560c52cf5730 2024-05-21T15:41:58.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1b2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52cd51f0 0x560c530b9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1b2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52cd51f0 0x560c530b9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40794/0 (socket says 172.21.15.38:40794) 2024-05-21T15:41:58.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1b2d9d640 1 -- 172.21.15.38:0/3867713536 learned_addr learned my addr 172.21.15.38:0/3867713536 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:58.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1abfff640 1 --2- 172.21.15.38:0/3867713536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52ced550 0x560c530bae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1b2d9d640 1 --2- 172.21.15.38:0/3867713536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 0x560c530c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1abfff640 1 -- 172.21.15.38:0/3867713536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 msgr2=0x560c530c0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1abfff640 1 --2- 172.21.15.38:0/3867713536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 0x560c530c0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1abfff640 1 -- 172.21.15.38:0/3867713536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52cd51f0 msgr2=0x560c530b9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1abfff640 1 --2- 172.21.15.38:0/3867713536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52cd51f0 0x560c530b9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1abfff640 1 -- 172.21.15.38:0/3867713536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c530bdbc0 con 0x560c52ced550 2024-05-21T15:41:58.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1b2d9d640 1 --2- 172.21.15.38:0/3867713536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52cd51f0 0x560c530b9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:58.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.556+0000 7fd1abfff640 1 --2- 172.21.15.38:0/3867713536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52ced550 0x560c530bae50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x560c530cdf20 tx=0x7fd19c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cecbcf1411e47c64 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:58.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.557+0000 7fd19bfff640 1 -- 172.21.15.38:0/3867713536 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c004050 con 0x560c52ced550 2024-05-21T15:41:58.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.557+0000 7fd19bfff640 1 -- 172.21.15.38:0/3867713536 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd19c005500 con 0x560c52ced550 2024-05-21T15:41:58.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.557+0000 7fd19bfff640 1 -- 172.21.15.38:0/3867713536 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c004ca0 con 0x560c52ced550 2024-05-21T15:41:58.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.557+0000 7fd1bfc65740 1 -- 172.21.15.38:0/3867713536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52ced550 msgr2=0x560c530bae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.557+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/3867713536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52ced550 0x560c530bae50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x560c530cdf20 tx=0x7fd19c005f90 comp rx=0 tx=0).stop 2024-05-21T15:41:58.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 -- 172.21.15.38:0/3867713536 shutdown_connections 2024-05-21T15:41:58.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/3867713536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 0x560c530c0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/3867713536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52ced550 0x560c530bae50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/3867713536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52cd51f0 0x560c530b9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 -- 172.21.15.38:0/3867713536 >> 172.21.15.38:0/3867713536 conn(0x560c52d0a560 msgr2=0x560c530bfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:58.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 -- 172.21.15.38:0/3867713536 shutdown_connections 2024-05-21T15:41:58.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 -- 172.21.15.38:0/3867713536 wait complete. 2024-05-21T15:41:58.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 -- start start 2024-05-21T15:41:58.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 Processor -- start 2024-05-21T15:41:58.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52cd51f0 0x560c530c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52ced550 0x560c530c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 0x560c530c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41ab7e0 con 0x560c52cd51f0 2024-05-21T15:41:58.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41ab110 con 0x560c52ced550 2024-05-21T15:41:58.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41a8350 con 0x560c52cf5730 2024-05-21T15:41:58.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1bfc65740 10 client.0 _ll_register_callbacks cb 0x7fd1ac047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:58.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1b2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52cd51f0 0x560c530c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1b2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52cd51f0 0x560c530c3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40808/0 (socket says 172.21.15.38:40808) 2024-05-21T15:41:58.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.558+0000 7fd1b2d9d640 1 -- 172.21.15.38:0/511388861 learned_addr learned my addr 172.21.15.38:0/511388861 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:58.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.559+0000 7fd1abfff640 1 --2- 172.21.15.38:0/511388861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52ced550 0x560c530c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.559+0000 7fd1b2d9d640 1 --2- 172.21.15.38:0/511388861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 0x560c530c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.559+0000 7fd1abfff640 1 -- 172.21.15.38:0/511388861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 msgr2=0x560c530c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.559+0000 7fd1abfff640 1 --2- 172.21.15.38:0/511388861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 0x560c530c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.559+0000 7fd1abfff640 1 -- 172.21.15.38:0/511388861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52cd51f0 msgr2=0x560c530c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.559+0000 7fd1abfff640 1 --2- 172.21.15.38:0/511388861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52cd51f0 0x560c530c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.559+0000 7fd1abfff640 1 -- 172.21.15.38:0/511388861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c530c6540 con 0x560c52ced550 2024-05-21T15:41:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.559+0000 7fd1b2d9d640 1 --2- 172.21.15.38:0/511388861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52cd51f0 0x560c530c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.559+0000 7fd1abfff640 1 --2- 172.21.15.38:0/511388861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52ced550 0x560c530c5ac0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd19c012fd0 tx=0x7fd19c005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e172bdd2e2514fbc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.559+0000 7fd199ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:41:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.559+0000 7fd199ffb640 1 -- 172.21.15.38:0/511388861 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c02e020 con 0x560c52ced550 2024-05-21T15:41:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.559+0000 7fd199ffb640 1 -- 172.21.15.38:0/511388861 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fd19c004a20 con 0x560c52ced550 2024-05-21T15:41:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.559+0000 7fd1bfc65740 1 -- 172.21.15.38:0/511388861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd19c005500 con 0x560c52ced550 2024-05-21T15:41:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.560+0000 7fd199ffb640 1 -- 172.21.15.38:0/511388861 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd19c039050 con 0x560c52ced550 2024-05-21T15:41:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.560+0000 7fd1bfc65740 10 client.27212 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.560+0000 7fd1bfc65740 1 -- 172.21.15.38:0/511388861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560c5313ec70 con 0x560c52ced550 2024-05-21T15:41:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.560+0000 7fd199ffb640 1 -- 172.21.15.38:0/511388861 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd19c005500 con 0x560c52ced550 2024-05-21T15:41:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.560+0000 7fd199ffb640 1 -- 172.21.15.38:0/511388861 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fd19c003920 con 0x560c52ced550 2024-05-21T15:41:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.560+0000 7fd1bfc65740 10 client.27212 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.560+0000 7fd1bfc65740 10 client.27212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.560+0000 7fd1bfc65740 20 client.27212 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.560+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/511388861 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c52d16410 0x560c531415e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.560+0000 7fd1bfc65740 4 client.27212 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:41:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.560+0000 7fd1bfc65740 1 -- 172.21.15.38:0/511388861 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560c53022410 con 0x560c52d16410 2024-05-21T15:41:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.560+0000 7fd1b2d9d640 1 --2- 172.21.15.38:0/511388861 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c52d16410 0x560c531415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.561+0000 7fd199ffb640 10 client.27212 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.561+0000 7fd1b2d9d640 1 --2- 172.21.15.38:0/511388861 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c52d16410 0x560c531415e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94b4a444ec492f06 server_cookie=69f9d01ff7b83e33 in_seq=0 out_seq=0 2024-05-21T15:41:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.562+0000 7fd199ffb640 1 -- 172.21.15.38:0/511388861 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560c53022410 con 0x560c52d16410 2024-05-21T15:41:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.562+0000 7fd199ffb640 10 client.27212 handle_command_reply: tid=0 2024-05-21T15:41:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.562+0000 7fd1bfc65740 1 client.27212 shutdown 2024-05-21T15:41:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.562+0000 7fd1bfc65740 1 -- 172.21.15.38:0/511388861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52ced550 msgr2=0x560c530c5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.562+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/511388861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52ced550 0x560c530c5ac0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd19c012fd0 tx=0x7fd19c005a90 comp rx=0 tx=0).stop 2024-05-21T15:41:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.562+0000 7fd1bfc65740 1 -- 172.21.15.38:0/511388861 shutdown_connections 2024-05-21T15:41:58.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.562+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/511388861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c52cf5730 0x560c530c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.562+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/511388861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c52ced550 0x560c530c5ac0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.562+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/511388861 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x560c52d16410 0x560c531415e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.562+0000 7fd1bfc65740 1 --2- 172.21.15.38:0/511388861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c52cd51f0 0x560c530c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.562+0000 7fd199ffb640 0 client.27212 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:58.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.562+0000 7fd1bfc65740 1 -- 172.21.15.38:0/511388861 >> 172.21.15.38:0/511388861 conn(0x560c52d0a560 msgr2=0x560c530be860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:58.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.562+0000 7fd1bfc65740 1 -- 172.21.15.38:0/511388861 shutdown_connections 2024-05-21T15:41:58.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.562+0000 7fd1bfc65740 1 -- 172.21.15.38:0/511388861 wait complete. 2024-05-21T15:41:58.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.563+0000 7fd1bfc65740 20 client.27212 trim_cache size 0 max 16384 2024-05-21T15:41:58.565 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:41:58.566 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-05-21T15:41:58.566 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "951108f3-2d15-4bdb-9900-d9fb6ffa3ed5", 2024-05-21T15:41:58.566 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-05-21T15:41:58.566 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:41:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.563+0000 7fd1b8db5640 1 -- 172.21.15.38:0/1298238492 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd1a00a2b90 msgr2=0x7fd1a00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.563+0000 7fd1b8db5640 1 --2- 172.21.15.38:0/1298238492 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd1a00a2b90 0x7fd1a00a50a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400e9c0 tx=0x7fd1a401bb30 comp rx=0 tx=0).stop 2024-05-21T15:41:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.563+0000 7fd1b8db5640 1 -- 172.21.15.38:0/1298238492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1b41019a0 msgr2=0x7fd1b41991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.563+0000 7fd1b8db5640 1 --2- 172.21.15.38:0/1298238492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1b41019a0 0x7fd1b41991f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400e9f0 tx=0x7fd1a400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.563+0000 7fd1b8db5640 1 -- 172.21.15.38:0/1298238492 shutdown_connections 2024-05-21T15:41:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.563+0000 7fd1b8db5640 1 --2- 172.21.15.38:0/1298238492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1b41078e0 0x7fd1b41a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.563+0000 7fd1b8db5640 1 --2- 172.21.15.38:0/1298238492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b4102be0 0x7fd1b4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.563+0000 7fd1b8db5640 1 --2- 172.21.15.38:0/1298238492 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fd1a00a2b90 0x7fd1a00a50a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.563+0000 7fd1b8db5640 1 --2- 172.21.15.38:0/1298238492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1b41019a0 0x7fd1b41991f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.563+0000 7fd1b8db5640 1 -- 172.21.15.38:0/1298238492 >> 172.21.15.38:0/1298238492 conn(0x7fd1b40fcb90 msgr2=0x7fd1b40fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:58.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.563+0000 7fd1b8db5640 1 -- 172.21.15.38:0/1298238492 shutdown_connections 2024-05-21T15:41:58.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.563+0000 7fd1b8db5640 1 -- 172.21.15.38:0/1298238492 wait complete. 2024-05-21T15:41:58.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:41:58.575 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:41:58.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.713+0000 7f824cbc7640 1 -- 172.21.15.38:0/3212561164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82481079a0 msgr2=0x7f8248109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.713+0000 7f824cbc7640 1 --2- 172.21.15.38:0/3212561164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82481079a0 0x7f8248109dc0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8240009ca0 tx=0x7f824002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.713+0000 7f824cbc7640 1 -- 172.21.15.38:0/3212561164 shutdown_connections 2024-05-21T15:41:58.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.713+0000 7f824cbc7640 1 --2- 172.21.15.38:0/3212561164 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82481079a0 0x7f8248109dc0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.713+0000 7f824cbc7640 1 --2- 172.21.15.38:0/3212561164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8248103350 0x7f8248101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.713+0000 7f824cbc7640 1 --2- 172.21.15.38:0/3212561164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248102e70 0x7f8248101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.713+0000 7f824cbc7640 1 -- 172.21.15.38:0/3212561164 >> 172.21.15.38:0/3212561164 conn(0x7f82480fc2f0 msgr2=0x7f82480fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:58.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.713+0000 7f824cbc7640 1 -- 172.21.15.38:0/3212561164 shutdown_connections 2024-05-21T15:41:58.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.713+0000 7f824cbc7640 1 -- 172.21.15.38:0/3212561164 wait complete. 2024-05-21T15:41:58.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824cbc7640 1 Processor -- start 2024-05-21T15:41:58.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824cbc7640 1 -- start start 2024-05-21T15:41:58.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824cbc7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248102e70 0x7f8248198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824cbc7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82481079a0 0x7f8248199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824cbc7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f824819e260 0x7f82481a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824cbc7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f824810d600 con 0x7f82481079a0 2024-05-21T15:41:58.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824cbc7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f824810d480 con 0x7f824819e260 2024-05-21T15:41:58.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824cbc7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f824810d780 con 0x7f8248102e70 2024-05-21T15:41:58.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f8245d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82481079a0 0x7f8248199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248102e70 0x7f8248198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248102e70 0x7f8248198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:37836/0 (socket says 172.21.15.38:37836) 2024-05-21T15:41:58.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824659c640 1 -- 172.21.15.38:0/1639820637 learned_addr learned my addr 172.21.15.38:0/1639820637 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:58.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f8246d9d640 1 --2- 172.21.15.38:0/1639820637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f824819e260 0x7f82481a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824659c640 1 -- 172.21.15.38:0/1639820637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f824819e260 msgr2=0x7f82481a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824659c640 1 --2- 172.21.15.38:0/1639820637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f824819e260 0x7f82481a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824659c640 1 -- 172.21.15.38:0/1639820637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82481079a0 msgr2=0x7f8248199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824659c640 1 --2- 172.21.15.38:0/1639820637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82481079a0 0x7f8248199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.714+0000 7f824659c640 1 -- 172.21.15.38:0/1639820637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82481a0cf0 con 0x7f8248102e70 2024-05-21T15:41:58.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.715+0000 7f824659c640 1 --2- 172.21.15.38:0/1639820637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248102e70 0x7f8248198de0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f823800e9f0 tx=0x7f823800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:58.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.715+0000 7f82377fe640 1 -- 172.21.15.38:0/1639820637 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823800ccc0 con 0x7f8248102e70 2024-05-21T15:41:58.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.715+0000 7f824cbc7640 1 -- 172.21.15.38:0/1639820637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82481a0fe0 con 0x7f8248102e70 2024-05-21T15:41:58.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.715+0000 7f82377fe640 1 -- 172.21.15.38:0/1639820637 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8238005500 con 0x7f8248102e70 2024-05-21T15:41:58.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.715+0000 7f82377fe640 1 -- 172.21.15.38:0/1639820637 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238010690 con 0x7f8248102e70 2024-05-21T15:41:58.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.715+0000 7f824cbc7640 1 -- 172.21.15.38:0/1639820637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82481a1450 con 0x7f8248102e70 2024-05-21T15:41:58.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.718+0000 7f82377fe640 1 -- 172.21.15.38:0/1639820637 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f82380046c0 con 0x7f8248102e70 2024-05-21T15:41:58.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.718+0000 7f8253a74740 1 Processor -- start 2024-05-21T15:41:58.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.718+0000 7f8253a74740 1 -- start start 2024-05-21T15:41:58.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.718+0000 7f8253a74740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5332c390 0x55ba536cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.718+0000 7f8253a74740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba533034d0 0x55ba536cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.718+0000 7f8253a74740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba5330b6b0 0x55ba536d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.718+0000 7f8253a74740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ba53630d00 con 0x55ba5332c390 2024-05-21T15:41:58.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.718+0000 7f8253a74740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ba535d84c0 con 0x55ba533034d0 2024-05-21T15:41:58.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.718+0000 7f8253a74740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ba5362ccc0 con 0x55ba5330b6b0 2024-05-21T15:41:58.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.718+0000 7f8246d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5332c390 0x55ba536cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.718+0000 7f824659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba533034d0 0x55ba536cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.718+0000 7f8245d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba5330b6b0 0x55ba536d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.718+0000 7f8246d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5332c390 0x55ba536cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40836/0 (socket says 172.21.15.38:40836) 2024-05-21T15:41:58.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.718+0000 7f8246d9d640 1 -- 172.21.15.38:0/1800135887 learned_addr learned my addr 172.21.15.38:0/1800135887 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:58.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.719+0000 7f82377fe640 1 --2- 172.21.15.38:0/1639820637 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f82300a2bc0 0x7f82300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.719+0000 7f8246d9d640 1 -- 172.21.15.38:0/1800135887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba5330b6b0 msgr2=0x55ba536d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.719+0000 7f8246d9d640 1 --2- 172.21.15.38:0/1800135887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba5330b6b0 0x55ba536d3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.719+0000 7f8246d9d640 1 -- 172.21.15.38:0/1800135887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba533034d0 msgr2=0x55ba536cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.719+0000 7f8246d9d640 1 --2- 172.21.15.38:0/1800135887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba533034d0 0x55ba536cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.719+0000 7f8246d9d640 1 -- 172.21.15.38:0/1800135887 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba5340e7d0 con 0x55ba5332c390 2024-05-21T15:41:58.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.719+0000 7f8245d9b640 1 --2- 172.21.15.38:0/1800135887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba5330b6b0 0x55ba536d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:58.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.719+0000 7f82377fe640 1 -- 172.21.15.38:0/1639820637 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f82380cf420 con 0x7f8248102e70 2024-05-21T15:41:58.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.719+0000 7f8246d9d640 1 --2- 172.21.15.38:0/1639820637 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f82300a2bc0 0x7f82300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.719+0000 7f8246d9d640 1 --2- 172.21.15.38:0/1800135887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5332c390 0x55ba536cbbb0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f8240035000 tx=0x7f8240009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbde168d73c0c1e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:58.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.719+0000 7f8234ff9640 1 -- 172.21.15.38:0/1800135887 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240042040 con 0x55ba5332c390 2024-05-21T15:41:58.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.719+0000 7f8234ff9640 1 -- 172.21.15.38:0/1800135887 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8240044070 con 0x55ba5332c390 2024-05-21T15:41:58.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.719+0000 7f8234ff9640 1 -- 172.21.15.38:0/1800135887 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240048630 con 0x55ba5332c390 2024-05-21T15:41:58.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.719+0000 7f8246d9d640 1 --2- 172.21.15.38:0/1639820637 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f82300a2bc0 0x7f82300a50d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f824004d000 tx=0x7f824004e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:58.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.720+0000 7f8253a74740 1 -- 172.21.15.38:0/1800135887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5332c390 msgr2=0x55ba536cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.720+0000 7f8253a74740 1 --2- 172.21.15.38:0/1800135887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5332c390 0x55ba536cbbb0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f8240035000 tx=0x7f8240009800 comp rx=0 tx=0).stop 2024-05-21T15:41:58.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 -- 172.21.15.38:0/1800135887 shutdown_connections 2024-05-21T15:41:58.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 --2- 172.21.15.38:0/1800135887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba5330b6b0 0x55ba536d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 --2- 172.21.15.38:0/1800135887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba533034d0 0x55ba536cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 --2- 172.21.15.38:0/1800135887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5332c390 0x55ba536cbbb0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 -- 172.21.15.38:0/1800135887 >> 172.21.15.38:0/1800135887 conn(0x55ba533204e0 msgr2=0x55ba536c7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:58.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 -- 172.21.15.38:0/1800135887 shutdown_connections 2024-05-21T15:41:58.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 -- 172.21.15.38:0/1800135887 wait complete. 2024-05-21T15:41:58.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 -- start start 2024-05-21T15:41:58.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 Processor -- start 2024-05-21T15:41:58.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba533034d0 0x55ba536d5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5330b6b0 0x55ba536d6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba5332c390 0x55ba536dcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ba536d3a80 con 0x55ba5330b6b0 2024-05-21T15:41:58.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ba535ebb90 con 0x55ba5332c390 2024-05-21T15:41:58.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.721+0000 7f8253a74740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ba536d3c00 con 0x55ba533034d0 2024-05-21T15:41:58.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f8253a74740 10 client.0 _ll_register_callbacks cb 0x55ba53627da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:58.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f8245d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5330b6b0 0x55ba536d6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f8245d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5330b6b0 0x55ba536d6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40838/0 (socket says 172.21.15.38:40838) 2024-05-21T15:41:58.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f8245d9b640 1 -- 172.21.15.38:0/768466270 learned_addr learned my addr 172.21.15.38:0/768466270 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:58.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f824659c640 1 --2- 172.21.15.38:0/768466270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba533034d0 0x55ba536d5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f8246d9d640 1 --2- 172.21.15.38:0/768466270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba5332c390 0x55ba536dcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f824659c640 1 -- 172.21.15.38:0/768466270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba5332c390 msgr2=0x55ba536dcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f824659c640 1 --2- 172.21.15.38:0/768466270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba5332c390 0x55ba536dcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f824659c640 1 -- 172.21.15.38:0/768466270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5330b6b0 msgr2=0x55ba536d6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f824659c640 1 --2- 172.21.15.38:0/768466270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5330b6b0 0x55ba536d6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f824659c640 1 -- 172.21.15.38:0/768466270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba533443f0 con 0x55ba533034d0 2024-05-21T15:41:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f8245d9b640 1 --2- 172.21.15.38:0/768466270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5330b6b0 0x55ba536d6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f824659c640 1 --2- 172.21.15.38:0/768466270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba533034d0 0x55ba536d5b10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f823809b8d0 tx=0x7f823809bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4301c80547dfabc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f822dffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:41:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f822dffb640 1 -- 172.21.15.38:0/768466270 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82380056b0 con 0x55ba533034d0 2024-05-21T15:41:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f822dffb640 1 -- 172.21.15.38:0/768466270 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f82380a43b0 con 0x55ba533034d0 2024-05-21T15:41:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.722+0000 7f8253a74740 1 -- 172.21.15.38:0/768466270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8240044070 con 0x55ba533034d0 2024-05-21T15:41:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.723+0000 7f822dffb640 1 -- 172.21.15.38:0/768466270 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f82380b6e60 con 0x55ba533034d0 2024-05-21T15:41:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.723+0000 7f8253a74740 10 client.27304 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.723+0000 7f8253a74740 1 -- 172.21.15.38:0/768466270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ba533040f0 con 0x55ba533034d0 2024-05-21T15:41:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.723+0000 7f822dffb640 1 -- 172.21.15.38:0/768466270 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8240044070 con 0x55ba533034d0 2024-05-21T15:41:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.723+0000 7f822dffb640 1 -- 172.21.15.38:0/768466270 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f823809b020 con 0x55ba533034d0 2024-05-21T15:41:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.723+0000 7f8253a74740 10 client.27304 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.723+0000 7f8253a74740 10 client.27304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.723+0000 7f8253a74740 20 client.27304 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.723+0000 7f8253a74740 1 --2- 172.21.15.38:0/768466270 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ba532eaf60 0x55ba536e3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.723+0000 7f8253a74740 4 client.27304 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:41:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.723+0000 7f8253a74740 1 -- 172.21.15.38:0/768466270 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba5363e430 con 0x55ba532eaf60 2024-05-21T15:41:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.723+0000 7f8245d9b640 1 --2- 172.21.15.38:0/768466270 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ba532eaf60 0x55ba536e3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.724+0000 7f822dffb640 10 client.27304 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.724+0000 7f8245d9b640 1 --2- 172.21.15.38:0/768466270 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ba532eaf60 0x55ba536e3770 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fe245de412ad93d server_cookie=755589983024902f in_seq=0 out_seq=0 2024-05-21T15:41:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f822dffb640 1 -- 172.21.15.38:0/768466270 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ba5363e430 con 0x55ba532eaf60 2024-05-21T15:41:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f822dffb640 10 client.27304 handle_command_reply: tid=0 2024-05-21T15:41:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f8253a74740 1 client.27304 shutdown 2024-05-21T15:41:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f8253a74740 1 -- 172.21.15.38:0/768466270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba533034d0 msgr2=0x55ba536d5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f8253a74740 1 --2- 172.21.15.38:0/768466270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba533034d0 0x55ba536d5b10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f823809b8d0 tx=0x7f823809bac0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f8253a74740 1 -- 172.21.15.38:0/768466270 shutdown_connections 2024-05-21T15:41:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f8253a74740 1 --2- 172.21.15.38:0/768466270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba5332c390 0x55ba536dcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f8253a74740 1 --2- 172.21.15.38:0/768466270 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ba532eaf60 0x55ba536e3770 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f8253a74740 1 --2- 172.21.15.38:0/768466270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5330b6b0 0x55ba536d6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f8253a74740 1 --2- 172.21.15.38:0/768466270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba533034d0 0x55ba536d5b10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f822dffb640 0 client.27304 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f8253a74740 1 -- 172.21.15.38:0/768466270 >> 172.21.15.38:0/768466270 conn(0x55ba533204e0 msgr2=0x55ba536c65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f8253a74740 1 -- 172.21.15.38:0/768466270 shutdown_connections 2024-05-21T15:41:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f8253a74740 1 -- 172.21.15.38:0/768466270 wait complete. 2024-05-21T15:41:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.725+0000 7f8253a74740 20 client.27304 trim_cache size 0 max 16384 2024-05-21T15:41:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8253a74740 1 Processor -- start 2024-05-21T15:41:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8253a74740 1 -- start start 2024-05-21T15:41:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8253a74740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba533034d0 0x55ba536d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8253a74740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5330b6b0 0x55ba536d5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8253a74740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba5332c390 0x55ba536dcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8253a74740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f82481aa2c0 con 0x55ba5330b6b0 2024-05-21T15:41:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8253a74740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82481abe00 con 0x55ba5332c390 2024-05-21T15:41:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8253a74740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f82481aa990 con 0x55ba533034d0 2024-05-21T15:41:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8245d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5330b6b0 0x55ba536d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8245d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5330b6b0 0x55ba536d5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40850/0 (socket says 172.21.15.38:40850) 2024-05-21T15:41:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8245d9b640 1 -- 172.21.15.38:0/385616986 learned_addr learned my addr 172.21.15.38:0/385616986 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8246d9d640 1 --2- 172.21.15.38:0/385616986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba5332c390 0x55ba536dcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f824659c640 1 --2- 172.21.15.38:0/385616986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba533034d0 0x55ba536d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8245d9b640 1 -- 172.21.15.38:0/385616986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba533034d0 msgr2=0x55ba536d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8245d9b640 1 --2- 172.21.15.38:0/385616986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba533034d0 0x55ba536d05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8245d9b640 1 -- 172.21.15.38:0/385616986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba5332c390 msgr2=0x55ba536dcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8245d9b640 1 --2- 172.21.15.38:0/385616986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba5332c390 0x55ba536dcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8245d9b640 1 -- 172.21.15.38:0/385616986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba536ceff0 con 0x55ba5330b6b0 2024-05-21T15:41:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.745+0000 7f8246d9d640 1 --2- 172.21.15.38:0/385616986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba5332c390 0x55ba536dcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:41:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.746+0000 7f8245d9b640 1 --2- 172.21.15.38:0/385616986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5330b6b0 0x55ba536d5ab0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f823c00a520 tx=0x7f823c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738e8a3e57f3b058 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.746+0000 7f822ffff640 1 -- 172.21.15.38:0/385616986 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c004220 con 0x55ba5330b6b0 2024-05-21T15:41:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.746+0000 7f822ffff640 1 -- 172.21.15.38:0/385616986 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f823c0043c0 con 0x55ba5330b6b0 2024-05-21T15:41:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.746+0000 7f822ffff640 1 -- 172.21.15.38:0/385616986 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c003c90 con 0x55ba5330b6b0 2024-05-21T15:41:58.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 -- 172.21.15.38:0/385616986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5330b6b0 msgr2=0x55ba536d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 --2- 172.21.15.38:0/385616986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5330b6b0 0x55ba536d5ab0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f823c00a520 tx=0x7f823c0046c0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 -- 172.21.15.38:0/385616986 shutdown_connections 2024-05-21T15:41:58.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 --2- 172.21.15.38:0/385616986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba5332c390 0x55ba536dcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 --2- 172.21.15.38:0/385616986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5330b6b0 0x55ba536d5ab0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 --2- 172.21.15.38:0/385616986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba533034d0 0x55ba536d05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 -- 172.21.15.38:0/385616986 >> 172.21.15.38:0/385616986 conn(0x55ba533204e0 msgr2=0x55ba536ca340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:58.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 -- 172.21.15.38:0/385616986 shutdown_connections 2024-05-21T15:41:58.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 -- 172.21.15.38:0/385616986 wait complete. 2024-05-21T15:41:58.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 -- start start 2024-05-21T15:41:58.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 Processor -- start 2024-05-21T15:41:58.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba533034d0 0x55ba536dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba5330b6b0 0x55ba536dc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5332c390 0x55ba53749c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f82481ab730 con 0x55ba5332c390 2024-05-21T15:41:58.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.747+0000 7f8253a74740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82481ab060 con 0x55ba533034d0 2024-05-21T15:41:58.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f8253a74740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f82481a82a0 con 0x55ba5330b6b0 2024-05-21T15:41:58.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f8253a74740 10 client.0 _ll_register_callbacks cb 0x7f8238004aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:41:58.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f8246d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5332c390 0x55ba53749c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f824659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba533034d0 0x55ba536dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f824659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba533034d0 0x55ba536dc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:54200/0 (socket says 172.21.15.38:54200) 2024-05-21T15:41:58.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f824659c640 1 -- 172.21.15.38:0/273886692 learned_addr learned my addr 172.21.15.38:0/273886692 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:41:58.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f8246d9d640 1 --2- 172.21.15.38:0/273886692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5332c390 0x55ba53749c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40864/0 (socket says 172.21.15.38:40864) 2024-05-21T15:41:58.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f8245d9b640 1 --2- 172.21.15.38:0/273886692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba5330b6b0 0x55ba536dc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f824659c640 1 -- 172.21.15.38:0/273886692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba5330b6b0 msgr2=0x55ba536dc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f824659c640 1 --2- 172.21.15.38:0/273886692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba5330b6b0 0x55ba536dc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f824659c640 1 -- 172.21.15.38:0/273886692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5332c390 msgr2=0x55ba53749c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:41:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f824659c640 1 --2- 172.21.15.38:0/273886692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5332c390 0x55ba53749c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f824659c640 1 -- 172.21.15.38:0/273886692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba5374a1f0 con 0x55ba533034d0 2024-05-21T15:41:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f824659c640 1 --2- 172.21.15.38:0/273886692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba533034d0 0x55ba536dc1c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8238004be0 tx=0x7f8238005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35a2359182507d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:41:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f822dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:41:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f822dffb640 1 -- 172.21.15.38:0/273886692 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82380a4840 con 0x55ba533034d0 2024-05-21T15:41:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f822dffb640 1 -- 172.21.15.38:0/273886692 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f82380a43b0 con 0x55ba533034d0 2024-05-21T15:41:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f8253a74740 1 -- 172.21.15.38:0/273886692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f823c0043c0 con 0x55ba533034d0 2024-05-21T15:41:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f822dffb640 1 -- 172.21.15.38:0/273886692 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f823809bbd0 con 0x55ba533034d0 2024-05-21T15:41:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f8253a74740 10 client.27227 fetch_fsmap learned FSMap version 35 2024-05-21T15:41:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.748+0000 7f8253a74740 1 -- 172.21.15.38:0/273886692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ba5374b520 con 0x55ba533034d0 2024-05-21T15:41:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.749+0000 7f822dffb640 1 -- 172.21.15.38:0/273886692 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f823c0043c0 con 0x55ba533034d0 2024-05-21T15:41:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.749+0000 7f822dffb640 1 -- 172.21.15.38:0/273886692 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f8238082020 con 0x55ba533034d0 2024-05-21T15:41:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.749+0000 7f8253a74740 10 client.27227 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:41:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.749+0000 7f8253a74740 10 client.27227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:41:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.749+0000 7f8253a74740 20 client.27227 populate_metadata read hostname 'smithi038' 2024-05-21T15:41:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.749+0000 7f8253a74740 1 --2- 172.21.15.38:0/273886692 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ba532c03a0 0x55ba5374df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:41:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.749+0000 7f8253a74740 4 client.27227 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:41:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.749+0000 7f8253a74740 1 -- 172.21.15.38:0/273886692 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ba53638780 con 0x55ba532c03a0 2024-05-21T15:41:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.749+0000 7f8245d9b640 1 --2- 172.21.15.38:0/273886692 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ba532c03a0 0x55ba5374df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:41:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.749+0000 7f8245d9b640 1 --2- 172.21.15.38:0/273886692 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ba532c03a0 0x55ba5374df60 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1beabb372fa7e2d5 server_cookie=846b4e503019e67f in_seq=0 out_seq=0 2024-05-21T15:41:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f822dffb640 10 client.27227 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f822dffb640 1 -- 172.21.15.38:0/273886692 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ba53638780 con 0x55ba532c03a0 2024-05-21T15:41:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f822dffb640 10 client.27227 handle_command_reply: tid=0 2024-05-21T15:41:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f8253a74740 1 client.27227 shutdown 2024-05-21T15:41:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f8253a74740 1 -- 172.21.15.38:0/273886692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba533034d0 msgr2=0x55ba536dc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f8253a74740 1 --2- 172.21.15.38:0/273886692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba533034d0 0x55ba536dc1c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8238004be0 tx=0x7f8238005740 comp rx=0 tx=0).stop 2024-05-21T15:41:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f8253a74740 1 -- 172.21.15.38:0/273886692 shutdown_connections 2024-05-21T15:41:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f8253a74740 1 --2- 172.21.15.38:0/273886692 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55ba532c03a0 0x55ba5374df60 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f8253a74740 1 --2- 172.21.15.38:0/273886692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ba5332c390 0x55ba53749c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f8253a74740 1 --2- 172.21.15.38:0/273886692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ba5330b6b0 0x55ba536dc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f822dffb640 0 client.27227 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:41:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f8253a74740 1 --2- 172.21.15.38:0/273886692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba533034d0 0x55ba536dc1c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f8253a74740 1 -- 172.21.15.38:0/273886692 >> 172.21.15.38:0/273886692 conn(0x55ba533204e0 msgr2=0x55ba536cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f8253a74740 1 -- 172.21.15.38:0/273886692 shutdown_connections 2024-05-21T15:41:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f8253a74740 1 -- 172.21.15.38:0/273886692 wait complete. 2024-05-21T15:41:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.750+0000 7f8253a74740 20 client.27227 trim_cache size 0 max 16384 2024-05-21T15:41:58.755 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:41:58.755 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:41:58.755 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-05-21T15:41:58.755 INFO:teuthology.orchestra.run.smithi038.stdout: "951108f3-2d15-4bdb-9900-d9fb6ffa3ed5": { 2024-05-21T15:41:58.755 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-05-21T15:41:58.755 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "951108f3-2d15-4bdb-9900-d9fb6ffa3ed5", 2024-05-21T15:41:58.755 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-05-21T15:41:58.755 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:41:58.755 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:41:58.755 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:41:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.751+0000 7f824cbc7640 1 -- 172.21.15.38:0/1639820637 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f82300a2bc0 msgr2=0x7f82300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.751+0000 7f824cbc7640 1 --2- 172.21.15.38:0/1639820637 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f82300a2bc0 0x7f82300a50d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f824004d000 tx=0x7f824004e000 comp rx=0 tx=0).stop 2024-05-21T15:41:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.751+0000 7f824cbc7640 1 -- 172.21.15.38:0/1639820637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248102e70 msgr2=0x7f8248198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:41:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.751+0000 7f824cbc7640 1 --2- 172.21.15.38:0/1639820637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248102e70 0x7f8248198de0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f823800e9f0 tx=0x7f823800eeb0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.751+0000 7f824cbc7640 1 -- 172.21.15.38:0/1639820637 shutdown_connections 2024-05-21T15:41:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.751+0000 7f824cbc7640 1 --2- 172.21.15.38:0/1639820637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f824819e260 0x7f82481a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.751+0000 7f824cbc7640 1 --2- 172.21.15.38:0/1639820637 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f82300a2bc0 0x7f82300a50d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.751+0000 7f824cbc7640 1 --2- 172.21.15.38:0/1639820637 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82481079a0 0x7f8248199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.751+0000 7f824cbc7640 1 --2- 172.21.15.38:0/1639820637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248102e70 0x7f8248198de0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:41:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.751+0000 7f824cbc7640 1 -- 172.21.15.38:0/1639820637 >> 172.21.15.38:0/1639820637 conn(0x7f82480fc2f0 msgr2=0x7f8248108c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:41:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.751+0000 7f824cbc7640 1 -- 172.21.15.38:0/1639820637 shutdown_connections 2024-05-21T15:41:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:41:58.751+0000 7f824cbc7640 1 -- 172.21.15.38:0/1639820637 wait complete. 2024-05-21T15:41:58.762 INFO:tasks.cephfs.filesystem:scrub status for tag:951108f3-2d15-4bdb-9900-d9fb6ffa3ed5 - {'path': '/', 'tag': '951108f3-2d15-4bdb-9900-d9fb6ffa3ed5', 'options': 'recursive,force'} 2024-05-21T15:41:59.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:58 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:41:59.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:58 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:41:59.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:58 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:42:00.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:59 smithi038 ceph-mon[31995]: pgmap v578: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 4.57k op/s 2024-05-21T15:42:00.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:59 smithi038 ceph-mon[31995]: scrub queued for path: / 2024-05-21T15:42:00.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:59 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:59 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:42:00.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:59 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:42:00.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:59 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:42:00.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:59 smithi038 ceph-mon[31995]: scrub summary: active paths [/] 2024-05-21T15:42:00.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:59 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:59 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:59 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:59 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:41:59 smithi038 ceph-mon[31995]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:59 smithi053 ceph-mon[36496]: pgmap v578: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 4.57k op/s 2024-05-21T15:42:00.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:59 smithi053 ceph-mon[36496]: scrub queued for path: / 2024-05-21T15:42:00.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:59 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:59 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:42:00.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:59 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:42:00.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:59 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:42:00.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:59 smithi053 ceph-mon[36496]: scrub summary: active paths [/] 2024-05-21T15:42:00.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:59 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:59 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:59 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:59 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:41:59 smithi053 ceph-mon[36496]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:59 smithi045 ceph-mon[36409]: pgmap v578: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 4.57k op/s 2024-05-21T15:42:00.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:59 smithi045 ceph-mon[36409]: scrub queued for path: / 2024-05-21T15:42:00.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:59 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:59 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:42:00.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:59 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:42:00.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:59 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:42:00.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:59 smithi045 ceph-mon[36409]: scrub summary: active paths [/] 2024-05-21T15:42:00.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:59 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:59 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:59 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:59 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:00.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:41:59 smithi045 ceph-mon[36409]: scrub summary: idle+waiting paths [/] 2024-05-21T15:42:02.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:01 smithi038 ceph-mon[31995]: pgmap v579: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 2.63k op/s 2024-05-21T15:42:02.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:01 smithi053 ceph-mon[36496]: pgmap v579: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 2.63k op/s 2024-05-21T15:42:02.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:01 smithi045 ceph-mon[36409]: pgmap v579: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 2.63k op/s 2024-05-21T15:42:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.441+0000 7fcf41e32640 1 -- 172.21.15.38:0/3314890477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf3c1088a0 msgr2=0x7fcf3c10acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.441+0000 7fcf41e32640 1 --2- 172.21.15.38:0/3314890477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf3c1088a0 0x7fcf3c10acc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009900 tx=0x7fcf3402a410 comp rx=0 tx=0).stop 2024-05-21T15:42:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.441+0000 7fcf41e32640 1 -- 172.21.15.38:0/3314890477 shutdown_connections 2024-05-21T15:42:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.441+0000 7fcf41e32640 1 --2- 172.21.15.38:0/3314890477 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf3c1088a0 0x7fcf3c10acc0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.441+0000 7fcf41e32640 1 --2- 172.21.15.38:0/3314890477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf3c105f40 0x7fcf3c108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.441+0000 7fcf41e32640 1 --2- 172.21.15.38:0/3314890477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c1035e0 0x7fcf3c105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.441+0000 7fcf41e32640 1 -- 172.21.15.38:0/3314890477 >> 172.21.15.38:0/3314890477 conn(0x7fcf3c0fcbe0 msgr2=0x7fcf3c0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.441+0000 7fcf41e32640 1 -- 172.21.15.38:0/3314890477 shutdown_connections 2024-05-21T15:42:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.441+0000 7fcf41e32640 1 -- 172.21.15.38:0/3314890477 wait complete. 2024-05-21T15:42:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf41e32640 1 Processor -- start 2024-05-21T15:42:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf41e32640 1 -- start start 2024-05-21T15:42:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf41e32640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf3c1035e0 0x7fcf3c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf41e32640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf3c105f40 0x7fcf3c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf41e32640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c1088a0 0x7fcf3c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf41e32640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c10e770 con 0x7fcf3c105f40 2024-05-21T15:42:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf41e32640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c10e5f0 con 0x7fcf3c1088a0 2024-05-21T15:42:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf41e32640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c10e8f0 con 0x7fcf3c1035e0 2024-05-21T15:42:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf3c105f40 0x7fcf3c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf3b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf3c1035e0 0x7fcf3c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c1088a0 0x7fcf3c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf3b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf3c1035e0 0x7fcf3c199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:37892/0 (socket says 172.21.15.38:37892) 2024-05-21T15:42:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c1088a0 0x7fcf3c1a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:54222/0 (socket says 172.21.15.38:54222) 2024-05-21T15:42:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf3b7fe640 1 -- 172.21.15.38:0/3967623177 learned_addr learned my addr 172.21.15.38:0/3967623177 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.442+0000 7fcf3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf3c105f40 0x7fcf3c1996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40886/0 (socket says 172.21.15.38:40886) 2024-05-21T15:42:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.443+0000 7fcf3affd640 1 -- 172.21.15.38:0/3967623177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf3c1035e0 msgr2=0x7fcf3c199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.443+0000 7fcf3affd640 1 --2- 172.21.15.38:0/3967623177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf3c1035e0 0x7fcf3c199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.443+0000 7fcf3affd640 1 -- 172.21.15.38:0/3967623177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c1088a0 msgr2=0x7fcf3c1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.443+0000 7fcf3affd640 1 --2- 172.21.15.38:0/3967623177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c1088a0 0x7fcf3c1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.443+0000 7fcf3affd640 1 -- 172.21.15.38:0/3967623177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c1a0cc0 con 0x7fcf3c105f40 2024-05-21T15:42:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.443+0000 7fcf3bfff640 1 --2- 172.21.15.38:0/3967623177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c1088a0 0x7fcf3c1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:42:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.443+0000 7fcf3affd640 1 --2- 172.21.15.38:0/3967623177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf3c105f40 0x7fcf3c1996c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400e9f0 tx=0x7fcf2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.443+0000 7fcf38ff9640 1 -- 172.21.15.38:0/3967623177 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2400ccc0 con 0x7fcf3c105f40 2024-05-21T15:42:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.443+0000 7fcf38ff9640 1 -- 172.21.15.38:0/3967623177 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf24005020 con 0x7fcf3c105f40 2024-05-21T15:42:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.443+0000 7fcf41e32640 1 -- 172.21.15.38:0/3967623177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3c1a0fb0 con 0x7fcf3c105f40 2024-05-21T15:42:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.443+0000 7fcf38ff9640 1 -- 172.21.15.38:0/3967623177 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf240049e0 con 0x7fcf3c105f40 2024-05-21T15:42:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.443+0000 7fcf41e32640 1 -- 172.21.15.38:0/3967623177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3c1a1420 con 0x7fcf3c105f40 2024-05-21T15:42:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.446+0000 7fcf38ff9640 1 -- 172.21.15.38:0/3967623177 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcf24005560 con 0x7fcf3c105f40 2024-05-21T15:42:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.446+0000 7fcf48db9740 1 Processor -- start 2024-05-21T15:42:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.446+0000 7fcf48db9740 1 -- start start 2024-05-21T15:42:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.446+0000 7fcf48db9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c652c1b0 0x55c9c68cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.446+0000 7fcf48db9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c65032f0 0x55c9c68cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.446+0000 7fcf48db9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c650b4d0 0x55c9c68d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.446+0000 7fcf48db9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c9c6830340 con 0x55c9c65032f0 2024-05-21T15:42:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.446+0000 7fcf48db9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c9c67d8290 con 0x55c9c652c1b0 2024-05-21T15:42:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.446+0000 7fcf48db9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c9c682c300 con 0x55c9c650b4d0 2024-05-21T15:42:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.446+0000 7fcf3b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c65032f0 0x55c9c68cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.446+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c652c1b0 0x55c9c68cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.446+0000 7fcf3b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c65032f0 0x55c9c68cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40888/0 (socket says 172.21.15.38:40888) 2024-05-21T15:42:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.446+0000 7fcf3b7fe640 1 -- 172.21.15.38:0/110238644 learned_addr learned my addr 172.21.15.38:0/110238644 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.446+0000 7fcf3affd640 1 --2- 172.21.15.38:0/110238644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c650b4d0 0x55c9c68d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.447+0000 7fcf3b7fe640 1 -- 172.21.15.38:0/110238644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c650b4d0 msgr2=0x55c9c68d32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.447+0000 7fcf38ff9640 1 --2- 172.21.15.38:0/3967623177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcf280a2c30 0x7fcf280a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.447+0000 7fcf3b7fe640 1 --2- 172.21.15.38:0/110238644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c650b4d0 0x55c9c68d32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.447+0000 7fcf3b7fe640 1 -- 172.21.15.38:0/110238644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c652c1b0 msgr2=0x55c9c68cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.447+0000 7fcf3b7fe640 1 --2- 172.21.15.38:0/110238644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c652c1b0 0x55c9c68cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.447+0000 7fcf3b7fe640 1 -- 172.21.15.38:0/110238644 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9c67fc8e0 con 0x55c9c65032f0 2024-05-21T15:42:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.447+0000 7fcf3bfff640 1 --2- 172.21.15.38:0/110238644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c652c1b0 0x55c9c68cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:42:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.447+0000 7fcf3bfff640 1 --2- 172.21.15.38:0/3967623177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcf280a2c30 0x7fcf280a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.447+0000 7fcf3b7fe640 1 --2- 172.21.15.38:0/110238644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c65032f0 0x55c9c68cc3e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0115c0 tx=0x7fcf2c011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57428850c4fe3a85 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.447+0000 7fcf38ff9640 1 -- 172.21.15.38:0/3967623177 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fcf240cf010 con 0x7fcf3c105f40 2024-05-21T15:42:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.447+0000 7fcf31ffb640 1 -- 172.21.15.38:0/110238644 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c00f640 con 0x55c9c65032f0 2024-05-21T15:42:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.447+0000 7fcf31ffb640 1 -- 172.21.15.38:0/110238644 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2c00fcc0 con 0x55c9c65032f0 2024-05-21T15:42:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.447+0000 7fcf31ffb640 1 -- 172.21.15.38:0/110238644 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c019de0 con 0x55c9c65032f0 2024-05-21T15:42:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.448+0000 7fcf48db9740 1 -- 172.21.15.38:0/110238644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c65032f0 msgr2=0x55c9c68cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.448+0000 7fcf3bfff640 1 --2- 172.21.15.38:0/3967623177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcf280a2c30 0x7fcf280a5140 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcf34031440 tx=0x7fcf34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.448+0000 7fcf48db9740 1 --2- 172.21.15.38:0/110238644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c65032f0 0x55c9c68cc3e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0115c0 tx=0x7fcf2c011a80 comp rx=0 tx=0).stop 2024-05-21T15:42:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.448+0000 7fcf48db9740 1 -- 172.21.15.38:0/110238644 shutdown_connections 2024-05-21T15:42:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.448+0000 7fcf48db9740 1 --2- 172.21.15.38:0/110238644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c650b4d0 0x55c9c68d32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.448+0000 7fcf48db9740 1 --2- 172.21.15.38:0/110238644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c65032f0 0x55c9c68cc3e0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.448+0000 7fcf48db9740 1 --2- 172.21.15.38:0/110238644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c652c1b0 0x55c9c68cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.448+0000 7fcf48db9740 1 -- 172.21.15.38:0/110238644 >> 172.21.15.38:0/110238644 conn(0x55c9c6520300 msgr2=0x55c9c68c77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.448+0000 7fcf48db9740 1 -- 172.21.15.38:0/110238644 shutdown_connections 2024-05-21T15:42:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.448+0000 7fcf48db9740 1 -- 172.21.15.38:0/110238644 wait complete. 2024-05-21T15:42:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.448+0000 7fcf48db9740 1 -- start start 2024-05-21T15:42:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf48db9740 1 Processor -- start 2024-05-21T15:42:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf48db9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c65032f0 0x55c9c68d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf48db9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c650b4d0 0x55c9c68d6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf48db9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c652c1b0 0x55c9c68dcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf48db9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c9c68d3d30 con 0x55c9c652c1b0 2024-05-21T15:42:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf48db9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c9c67eb790 con 0x55c9c650b4d0 2024-05-21T15:42:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf48db9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c9c68d3eb0 con 0x55c9c65032f0 2024-05-21T15:42:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf48db9740 10 client.0 _ll_register_callbacks cb 0x55c9c6826a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf3b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c65032f0 0x55c9c68d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c652c1b0 0x55c9c68dcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf3affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c650b4d0 0x55c9c68d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf3b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c65032f0 0x55c9c68d5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:37904/0 (socket says 172.21.15.38:37904) 2024-05-21T15:42:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf3b7fe640 1 -- 172.21.15.38:0/1919784931 learned_addr learned my addr 172.21.15.38:0/1919784931 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf3affd640 1 -- 172.21.15.38:0/1919784931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c65032f0 msgr2=0x55c9c68d5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf3affd640 1 --2- 172.21.15.38:0/1919784931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c65032f0 0x55c9c68d5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf3affd640 1 -- 172.21.15.38:0/1919784931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c652c1b0 msgr2=0x55c9c68dcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf3affd640 1 --2- 172.21.15.38:0/1919784931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c652c1b0 0x55c9c68dcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.449+0000 7fcf3affd640 1 -- 172.21.15.38:0/1919784931 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9c6544210 con 0x55c9c650b4d0 2024-05-21T15:42:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf3affd640 1 --2- 172.21.15.38:0/1919784931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c650b4d0 0x55c9c68d6010 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcf240ccb10 tx=0x7fcf240974c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c4dd4af8256ae82 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf1affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:42:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf1affd640 1 -- 172.21.15.38:0/1919784931 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2401c070 con 0x55c9c650b4d0 2024-05-21T15:42:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf48db9740 1 -- 172.21.15.38:0/1919784931 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf2c00fcc0 con 0x55c9c650b4d0 2024-05-21T15:42:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf1affd640 1 -- 172.21.15.38:0/1919784931 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fcf24086070 con 0x55c9c650b4d0 2024-05-21T15:42:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf1affd640 1 -- 172.21.15.38:0/1919784931 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcf2409b180 con 0x55c9c650b4d0 2024-05-21T15:42:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf48db9740 10 client.27239 fetch_fsmap learned FSMap version 35 2024-05-21T15:42:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf48db9740 1 -- 172.21.15.38:0/1919784931 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c9c6610620 con 0x55c9c650b4d0 2024-05-21T15:42:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf1affd640 1 -- 172.21.15.38:0/1919784931 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2c00fcc0 con 0x55c9c650b4d0 2024-05-21T15:42:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf1affd640 1 -- 172.21.15.38:0/1919784931 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fcf240d10d0 con 0x55c9c650b4d0 2024-05-21T15:42:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf48db9740 10 client.27239 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:42:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf48db9740 10 client.27239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:42:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf48db9740 20 client.27239 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf48db9740 1 --2- 172.21.15.38:0/1919784931 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c9c64ead80 0x55c9c68e3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf48db9740 4 client.27239 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:42:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.450+0000 7fcf48db9740 1 -- 172.21.15.38:0/1919784931 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9c683da70 con 0x55c9c64ead80 2024-05-21T15:42:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.451+0000 7fcf3b7fe640 1 --2- 172.21.15.38:0/1919784931 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c9c64ead80 0x55c9c68e3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.451+0000 7fcf1affd640 10 client.27239 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.451+0000 7fcf3b7fe640 1 --2- 172.21.15.38:0/1919784931 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c9c64ead80 0x55c9c68e3500 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed1e68e71a7c4d22 server_cookie=2603676ebaea1bbb in_seq=0 out_seq=0 2024-05-21T15:42:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf1affd640 1 -- 172.21.15.38:0/1919784931 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c9c683da70 con 0x55c9c64ead80 2024-05-21T15:42:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf1affd640 10 client.27239 handle_command_reply: tid=0 2024-05-21T15:42:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf48db9740 1 client.27239 shutdown 2024-05-21T15:42:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf48db9740 1 -- 172.21.15.38:0/1919784931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c650b4d0 msgr2=0x55c9c68d6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf48db9740 1 --2- 172.21.15.38:0/1919784931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c650b4d0 0x55c9c68d6010 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcf240ccb10 tx=0x7fcf240974c0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf48db9740 1 -- 172.21.15.38:0/1919784931 shutdown_connections 2024-05-21T15:42:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf48db9740 1 --2- 172.21.15.38:0/1919784931 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c9c64ead80 0x55c9c68e3500 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf48db9740 1 --2- 172.21.15.38:0/1919784931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c652c1b0 0x55c9c68dcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf48db9740 1 --2- 172.21.15.38:0/1919784931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c650b4d0 0x55c9c68d6010 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf48db9740 1 --2- 172.21.15.38:0/1919784931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c65032f0 0x55c9c68d5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf1affd640 0 client.27239 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf48db9740 1 -- 172.21.15.38:0/1919784931 >> 172.21.15.38:0/1919784931 conn(0x55c9c6520300 msgr2=0x55c9c68ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf48db9740 1 -- 172.21.15.38:0/1919784931 shutdown_connections 2024-05-21T15:42:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf48db9740 1 -- 172.21.15.38:0/1919784931 wait complete. 2024-05-21T15:42:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.452+0000 7fcf48db9740 20 client.27239 trim_cache size 0 max 16384 2024-05-21T15:42:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.470+0000 7fcf48db9740 1 Processor -- start 2024-05-21T15:42:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.470+0000 7fcf48db9740 1 -- start start 2024-05-21T15:42:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.470+0000 7fcf48db9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c65032f0 0x55c9c68cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.470+0000 7fcf48db9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c650b4d0 0x55c9c68ca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.470+0000 7fcf48db9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c652c1b0 0x55c9c68cac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.470+0000 7fcf48db9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c1aa350 con 0x55c9c650b4d0 2024-05-21T15:42:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.470+0000 7fcf48db9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c1abe90 con 0x55c9c65032f0 2024-05-21T15:42:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.470+0000 7fcf48db9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c1aaa20 con 0x55c9c652c1b0 2024-05-21T15:42:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.470+0000 7fcf3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c650b4d0 0x55c9c68ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.470+0000 7fcf3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c650b4d0 0x55c9c68ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40914/0 (socket says 172.21.15.38:40914) 2024-05-21T15:42:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.470+0000 7fcf3affd640 1 -- 172.21.15.38:0/2194225968 learned_addr learned my addr 172.21.15.38:0/2194225968 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.470+0000 7fcf3b7fe640 1 --2- 172.21.15.38:0/2194225968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c65032f0 0x55c9c68cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.470+0000 7fcf3bfff640 1 --2- 172.21.15.38:0/2194225968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c652c1b0 0x55c9c68cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.471+0000 7fcf3affd640 1 -- 172.21.15.38:0/2194225968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c652c1b0 msgr2=0x55c9c68cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.471+0000 7fcf3affd640 1 --2- 172.21.15.38:0/2194225968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c652c1b0 0x55c9c68cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.471+0000 7fcf3affd640 1 -- 172.21.15.38:0/2194225968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c65032f0 msgr2=0x55c9c68cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.471+0000 7fcf3affd640 1 --2- 172.21.15.38:0/2194225968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c65032f0 0x55c9c68cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.471+0000 7fcf3affd640 1 -- 172.21.15.38:0/2194225968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9c68d08f0 con 0x55c9c650b4d0 2024-05-21T15:42:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.471+0000 7fcf3b7fe640 1 --2- 172.21.15.38:0/2194225968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c65032f0 0x55c9c68cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:42:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.471+0000 7fcf3affd640 1 --2- 172.21.15.38:0/2194225968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c650b4d0 0x55c9c68ca710 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fcf240981f0 tx=0x7fcf24005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4243af29ca5e791 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.471+0000 7fcf317fa640 1 -- 172.21.15.38:0/2194225968 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2401c070 con 0x55c9c650b4d0 2024-05-21T15:42:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.471+0000 7fcf317fa640 1 -- 172.21.15.38:0/2194225968 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf240051d0 con 0x55c9c650b4d0 2024-05-21T15:42:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.471+0000 7fcf317fa640 1 -- 172.21.15.38:0/2194225968 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24086270 con 0x55c9c650b4d0 2024-05-21T15:42:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.472+0000 7fcf48db9740 1 -- 172.21.15.38:0/2194225968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c650b4d0 msgr2=0x55c9c68ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.472+0000 7fcf48db9740 1 --2- 172.21.15.38:0/2194225968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c650b4d0 0x55c9c68ca710 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fcf240981f0 tx=0x7fcf24005410 comp rx=0 tx=0).stop 2024-05-21T15:42:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.472+0000 7fcf48db9740 1 -- 172.21.15.38:0/2194225968 shutdown_connections 2024-05-21T15:42:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.472+0000 7fcf48db9740 1 --2- 172.21.15.38:0/2194225968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c652c1b0 0x55c9c68cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.472+0000 7fcf48db9740 1 --2- 172.21.15.38:0/2194225968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c650b4d0 0x55c9c68ca710 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.472+0000 7fcf48db9740 1 --2- 172.21.15.38:0/2194225968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c65032f0 0x55c9c68cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.472+0000 7fcf48db9740 1 -- 172.21.15.38:0/2194225968 >> 172.21.15.38:0/2194225968 conn(0x55c9c6520300 msgr2=0x55c9c68ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.472+0000 7fcf48db9740 1 -- 172.21.15.38:0/2194225968 shutdown_connections 2024-05-21T15:42:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.472+0000 7fcf48db9740 1 -- 172.21.15.38:0/2194225968 wait complete. 2024-05-21T15:42:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.472+0000 7fcf48db9740 1 -- start start 2024-05-21T15:42:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf48db9740 1 Processor -- start 2024-05-21T15:42:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf48db9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c65032f0 0x55c9c68d9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf48db9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c650b4d0 0x55c9c68d9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf48db9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c652c1b0 0x55c9c69498a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf48db9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c1ab7c0 con 0x55c9c652c1b0 2024-05-21T15:42:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf48db9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c1ab0f0 con 0x55c9c65032f0 2024-05-21T15:42:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf48db9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c1a8330 con 0x55c9c650b4d0 2024-05-21T15:42:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf48db9740 10 client.0 _ll_register_callbacks cb 0x7fcf240046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c652c1b0 0x55c9c69498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf3b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c65032f0 0x55c9c68d9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c652c1b0 0x55c9c69498a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40918/0 (socket says 172.21.15.38:40918) 2024-05-21T15:42:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf3bfff640 1 -- 172.21.15.38:0/17476281 learned_addr learned my addr 172.21.15.38:0/17476281 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf3affd640 1 --2- 172.21.15.38:0/17476281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c650b4d0 0x55c9c68d9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf3b7fe640 1 -- 172.21.15.38:0/17476281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c650b4d0 msgr2=0x55c9c68d9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf3b7fe640 1 --2- 172.21.15.38:0/17476281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c650b4d0 0x55c9c68d9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf3b7fe640 1 -- 172.21.15.38:0/17476281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c652c1b0 msgr2=0x55c9c69498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf3b7fe640 1 --2- 172.21.15.38:0/17476281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c652c1b0 0x55c9c69498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf3b7fe640 1 -- 172.21.15.38:0/17476281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9c68dc820 con 0x55c9c65032f0 2024-05-21T15:42:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.473+0000 7fcf3bfff640 1 --2- 172.21.15.38:0/17476281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c652c1b0 0x55c9c69498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:42:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.474+0000 7fcf3b7fe640 1 --2- 172.21.15.38:0/17476281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c65032f0 0x55c9c68d9320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c005c10 tx=0x7fcf2c009db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c1d387b545b4dc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.474+0000 7fcf1affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:42:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.474+0000 7fcf1affd640 1 -- 172.21.15.38:0/17476281 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c004c10 con 0x55c9c65032f0 2024-05-21T15:42:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.474+0000 7fcf1affd640 1 -- 172.21.15.38:0/17476281 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fcf2c016040 con 0x55c9c65032f0 2024-05-21T15:42:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.474+0000 7fcf48db9740 1 -- 172.21.15.38:0/17476281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf240051d0 con 0x55c9c65032f0 2024-05-21T15:42:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.474+0000 7fcf1affd640 1 -- 172.21.15.38:0/17476281 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcf2c004db0 con 0x55c9c65032f0 2024-05-21T15:42:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.474+0000 7fcf48db9740 10 client.27245 fetch_fsmap learned FSMap version 35 2024-05-21T15:42:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.474+0000 7fcf48db9740 1 -- 172.21.15.38:0/17476281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c9c694ae10 con 0x55c9c65032f0 2024-05-21T15:42:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.474+0000 7fcf1affd640 1 -- 172.21.15.38:0/17476281 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf240051d0 con 0x55c9c65032f0 2024-05-21T15:42:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.474+0000 7fcf1affd640 1 -- 172.21.15.38:0/17476281 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fcf2c019b70 con 0x55c9c65032f0 2024-05-21T15:42:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.475+0000 7fcf48db9740 10 client.27245 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:42:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.475+0000 7fcf48db9740 10 client.27245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:42:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.475+0000 7fcf48db9740 20 client.27245 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.475+0000 7fcf48db9740 1 --2- 172.21.15.38:0/17476281 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c9c64c01c0 0x55c9c694d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.475+0000 7fcf48db9740 4 client.27245 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:42:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.475+0000 7fcf48db9740 1 -- 172.21.15.38:0/17476281 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9c6837dc0 con 0x55c9c64c01c0 2024-05-21T15:42:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.475+0000 7fcf3affd640 1 --2- 172.21.15.38:0/17476281 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c9c64c01c0 0x55c9c694d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.475+0000 7fcf1affd640 10 client.27245 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.475+0000 7fcf3affd640 1 --2- 172.21.15.38:0/17476281 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c9c64c01c0 0x55c9c694d5f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15bc8c13442ed016 server_cookie=1fcf3149381d3e07 in_seq=0 out_seq=0 2024-05-21T15:42:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf1affd640 1 -- 172.21.15.38:0/17476281 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9c6837dc0 con 0x55c9c64c01c0 2024-05-21T15:42:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf1affd640 10 client.27245 handle_command_reply: tid=0 2024-05-21T15:42:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf48db9740 1 client.27245 shutdown 2024-05-21T15:42:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf48db9740 1 -- 172.21.15.38:0/17476281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c65032f0 msgr2=0x55c9c68d9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf48db9740 1 --2- 172.21.15.38:0/17476281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c65032f0 0x55c9c68d9320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c005c10 tx=0x7fcf2c009db0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf48db9740 1 -- 172.21.15.38:0/17476281 shutdown_connections 2024-05-21T15:42:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf48db9740 1 --2- 172.21.15.38:0/17476281 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55c9c64c01c0 0x55c9c694d5f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf48db9740 1 --2- 172.21.15.38:0/17476281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c9c652c1b0 0x55c9c69498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf48db9740 1 --2- 172.21.15.38:0/17476281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9c650b4d0 0x55c9c68d9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf48db9740 1 --2- 172.21.15.38:0/17476281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9c65032f0 0x55c9c68d9320 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf1affd640 0 client.27245 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf48db9740 1 -- 172.21.15.38:0/17476281 >> 172.21.15.38:0/17476281 conn(0x55c9c6520300 msgr2=0x55c9c68ca710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf48db9740 1 -- 172.21.15.38:0/17476281 shutdown_connections 2024-05-21T15:42:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf48db9740 1 -- 172.21.15.38:0/17476281 wait complete. 2024-05-21T15:42:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.488+0000 7fcf48db9740 20 client.27245 trim_cache size 0 max 16384 2024-05-21T15:42:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:42:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:42:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:42:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:42:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.489+0000 7fcf41e32640 1 -- 172.21.15.38:0/3967623177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcf280a2c30 msgr2=0x7fcf280a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.489+0000 7fcf41e32640 1 --2- 172.21.15.38:0/3967623177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcf280a2c30 0x7fcf280a5140 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcf34031440 tx=0x7fcf34009260 comp rx=0 tx=0).stop 2024-05-21T15:42:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.489+0000 7fcf41e32640 1 -- 172.21.15.38:0/3967623177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf3c105f40 msgr2=0x7fcf3c1996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.489+0000 7fcf41e32640 1 --2- 172.21.15.38:0/3967623177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf3c105f40 0x7fcf3c1996c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400e9f0 tx=0x7fcf2400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.489+0000 7fcf41e32640 1 -- 172.21.15.38:0/3967623177 shutdown_connections 2024-05-21T15:42:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.489+0000 7fcf41e32640 1 --2- 172.21.15.38:0/3967623177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c1088a0 0x7fcf3c1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.489+0000 7fcf41e32640 1 --2- 172.21.15.38:0/3967623177 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fcf280a2c30 0x7fcf280a5140 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.489+0000 7fcf41e32640 1 --2- 172.21.15.38:0/3967623177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf3c105f40 0x7fcf3c1996c0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.489+0000 7fcf41e32640 1 --2- 172.21.15.38:0/3967623177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf3c1035e0 0x7fcf3c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.489+0000 7fcf41e32640 1 -- 172.21.15.38:0/3967623177 >> 172.21.15.38:0/3967623177 conn(0x7fcf3c0fcbe0 msgr2=0x7fcf3c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.489+0000 7fcf41e32640 1 -- 172.21.15.38:0/3967623177 shutdown_connections 2024-05-21T15:42:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:02.489+0000 7fcf41e32640 1 -- 172.21.15.38:0/3967623177 wait complete. 2024-05-21T15:42:04.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:03 smithi038 ceph-mon[31995]: pgmap v580: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.6 MiB/s wr, 2.12k op/s 2024-05-21T15:42:04.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:03 smithi053 ceph-mon[36496]: pgmap v580: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.6 MiB/s wr, 2.12k op/s 2024-05-21T15:42:04.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:03 smithi045 ceph-mon[36409]: pgmap v580: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.6 MiB/s wr, 2.12k op/s 2024-05-21T15:42:05.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:04 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:42:05.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:04 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:42:05.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:04 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:42:05.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:04 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:42:05.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:04 smithi053 ceph-mon[36496]: scrub summary: idle 2024-05-21T15:42:05.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:04 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:42:05.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:04 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:42:05.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:04 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:42:05.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:04 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:42:05.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:04 smithi045 ceph-mon[36409]: scrub summary: idle 2024-05-21T15:42:05.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:04 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:42:05.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:04 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:42:05.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:04 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:42:05.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:04 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:42:05.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:04 smithi038 ceph-mon[31995]: scrub summary: idle 2024-05-21T15:42:06.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:05 smithi053 ceph-mon[36496]: pgmap v581: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 986 KiB/s wr, 294 op/s 2024-05-21T15:42:06.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:05 smithi045 ceph-mon[36409]: pgmap v581: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 986 KiB/s wr, 294 op/s 2024-05-21T15:42:06.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:05 smithi038 ceph-mon[31995]: pgmap v581: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 986 KiB/s wr, 294 op/s 2024-05-21T15:42:08.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:07 smithi053 ceph-mon[36496]: pgmap v582: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 920 KiB/s wr, 275 op/s 2024-05-21T15:42:08.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:07 smithi045 ceph-mon[36409]: pgmap v582: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 920 KiB/s wr, 275 op/s 2024-05-21T15:42:08.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:07 smithi038 ceph-mon[31995]: pgmap v582: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 920 KiB/s wr, 275 op/s 2024-05-21T15:42:10.060 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:09 smithi045 ceph-mon[36409]: pgmap v583: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.1 MiB/s wr, 327 op/s 2024-05-21T15:42:10.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:09 smithi053 ceph-mon[36496]: pgmap v583: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.1 MiB/s wr, 327 op/s 2024-05-21T15:42:10.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:09 smithi038 ceph-mon[31995]: pgmap v583: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.1 MiB/s wr, 327 op/s 2024-05-21T15:42:12.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:11 smithi053 ceph-mon[36496]: pgmap v584: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 886 KiB/s wr, 270 op/s 2024-05-21T15:42:12.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:11 smithi045 ceph-mon[36409]: pgmap v584: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 886 KiB/s wr, 270 op/s 2024-05-21T15:42:12.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:11 smithi038 ceph-mon[31995]: pgmap v584: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 886 KiB/s wr, 270 op/s 2024-05-21T15:42:14.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:13 smithi053 ceph-mon[36496]: pgmap v585: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.2 MiB/s wr, 342 op/s 2024-05-21T15:42:14.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:13 smithi045 ceph-mon[36409]: pgmap v585: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.2 MiB/s wr, 342 op/s 2024-05-21T15:42:14.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:13 smithi038 ceph-mon[31995]: pgmap v585: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.2 MiB/s wr, 342 op/s 2024-05-21T15:42:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.612+0000 7f2ac6ef7640 1 -- 172.21.15.38:0/767859543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ac0107910 msgr2=0x7f2ac0109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.612+0000 7f2ac6ef7640 1 --2- 172.21.15.38:0/767859543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ac0107910 0x7f2ac0109d30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009ca0 tx=0x7f2ab802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.612+0000 7f2ac6ef7640 1 -- 172.21.15.38:0/767859543 shutdown_connections 2024-05-21T15:42:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.612+0000 7f2ac6ef7640 1 --2- 172.21.15.38:0/767859543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ac0107910 0x7f2ac0109d30 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.612+0000 7f2ac6ef7640 1 --2- 172.21.15.38:0/767859543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ac0102c10 0x7f2ac0103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.612+0000 7f2ac6ef7640 1 --2- 172.21.15.38:0/767859543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ac01019d0 0x7f2ac01026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.612+0000 7f2ac6ef7640 1 -- 172.21.15.38:0/767859543 >> 172.21.15.38:0/767859543 conn(0x7f2ac00fcbc0 msgr2=0x7f2ac00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.612+0000 7f2ac6ef7640 1 -- 172.21.15.38:0/767859543 shutdown_connections 2024-05-21T15:42:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.612+0000 7f2ac6ef7640 1 -- 172.21.15.38:0/767859543 wait complete. 2024-05-21T15:42:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.612+0000 7f2ac6ef7640 1 Processor -- start 2024-05-21T15:42:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.612+0000 7f2ac6ef7640 1 -- start start 2024-05-21T15:42:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac6ef7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ac01019d0 0x7f2ac0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac6ef7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ac0102c10 0x7f2ac01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac6ef7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ac0107910 0x7f2ac019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac6ef7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac010d630 con 0x7f2ac01019d0 2024-05-21T15:42:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac6ef7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac010d4b0 con 0x7f2ac0107910 2024-05-21T15:42:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac6ef7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac010d7b0 con 0x7f2ac0102c10 2024-05-21T15:42:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac4c93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ac01019d0 0x7f2ac0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac4c93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ac01019d0 0x7f2ac0198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35104/0 (socket says 172.21.15.38:35104) 2024-05-21T15:42:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac4c93640 1 -- 172.21.15.38:0/814210458 learned_addr learned my addr 172.21.15.38:0/814210458 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2abffff640 1 --2- 172.21.15.38:0/814210458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ac0102c10 0x7f2ac01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac5494640 1 --2- 172.21.15.38:0/814210458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ac0107910 0x7f2ac019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac4c93640 1 -- 172.21.15.38:0/814210458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ac0102c10 msgr2=0x7f2ac01985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac4c93640 1 --2- 172.21.15.38:0/814210458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ac0102c10 0x7f2ac01985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac4c93640 1 -- 172.21.15.38:0/814210458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ac0107910 msgr2=0x7f2ac019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac4c93640 1 --2- 172.21.15.38:0/814210458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ac0107910 0x7f2ac019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac4c93640 1 -- 172.21.15.38:0/814210458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac019fbd0 con 0x7f2ac01019d0 2024-05-21T15:42:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2ac4c93640 1 --2- 172.21.15.38:0/814210458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ac01019d0 0x7f2ac0198090 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400e9f0 tx=0x7f2ab400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.613+0000 7f2abdffb640 1 -- 172.21.15.38:0/814210458 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab400ccc0 con 0x7f2ac01019d0 2024-05-21T15:42:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.614+0000 7f2abdffb640 1 -- 172.21.15.38:0/814210458 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ab40035f0 con 0x7f2ac01019d0 2024-05-21T15:42:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.614+0000 7f2ac6ef7640 1 -- 172.21.15.38:0/814210458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac019fec0 con 0x7f2ac01019d0 2024-05-21T15:42:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.614+0000 7f2abdffb640 1 -- 172.21.15.38:0/814210458 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab400ce60 con 0x7f2ac01019d0 2024-05-21T15:42:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.614+0000 7f2ac6ef7640 1 -- 172.21.15.38:0/814210458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac01a0380 con 0x7f2ac01019d0 2024-05-21T15:42:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.617+0000 7f2abdffb640 1 -- 172.21.15.38:0/814210458 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ab40037a0 con 0x7f2ac01019d0 2024-05-21T15:42:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.617+0000 7f2acdd9d740 1 Processor -- start 2024-05-21T15:42:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.617+0000 7f2acdd9d740 1 -- start start 2024-05-21T15:42:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.617+0000 7f2acdd9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6541b0 0x55db9d9f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.617+0000 7f2acdd9d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d62b2f0 0x55db9d9f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.617+0000 7f2acdd9d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6334d0 0x55db9d9fb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.617+0000 7f2acdd9d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55db9d958340 con 0x55db9d62b2f0 2024-05-21T15:42:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.617+0000 7f2acdd9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55db9d900290 con 0x55db9d6541b0 2024-05-21T15:42:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.617+0000 7f2acdd9d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55db9d954300 con 0x55db9d6334d0 2024-05-21T15:42:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2ac4c93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d62b2f0 0x55db9d9f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2ac4c93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d62b2f0 0x55db9d9f43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35110/0 (socket says 172.21.15.38:35110) 2024-05-21T15:42:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2ac4c93640 1 -- 172.21.15.38:0/1930930680 learned_addr learned my addr 172.21.15.38:0/1930930680 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2abffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6334d0 0x55db9d9fb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2ac5494640 1 --2- 172.21.15.38:0/1930930680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6541b0 0x55db9d9f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2abdffb640 1 --2- 172.21.15.38:0/814210458 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2ab00a2cb0 0x7f2ab00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2ac4c93640 1 -- 172.21.15.38:0/1930930680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6334d0 msgr2=0x55db9d9fb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2ac4c93640 1 --2- 172.21.15.38:0/1930930680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6334d0 0x55db9d9fb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2ac5494640 1 --2- 172.21.15.38:0/814210458 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2ab00a2cb0 0x7f2ab00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2ac4c93640 1 -- 172.21.15.38:0/1930930680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6541b0 msgr2=0x55db9d9f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2abdffb640 1 -- 172.21.15.38:0/814210458 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2ab40cf310 con 0x7f2ac01019d0 2024-05-21T15:42:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2ac4c93640 1 --2- 172.21.15.38:0/1930930680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6541b0 0x55db9d9f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2ac4c93640 1 -- 172.21.15.38:0/1930930680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db9d9248e0 con 0x55db9d62b2f0 2024-05-21T15:42:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2ac4c93640 1 --2- 172.21.15.38:0/1930930680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d62b2f0 0x55db9d9f43e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2ab409b820 tx=0x7f2ab4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac61d7c1e5b36217 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.618+0000 7f2aaaffd640 1 -- 172.21.15.38:0/1930930680 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab409b480 con 0x55db9d62b2f0 2024-05-21T15:42:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.619+0000 7f2aaaffd640 1 -- 172.21.15.38:0/1930930680 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ab409c220 con 0x55db9d62b2f0 2024-05-21T15:42:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.619+0000 7f2aaaffd640 1 -- 172.21.15.38:0/1930930680 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4003b10 con 0x55db9d62b2f0 2024-05-21T15:42:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.619+0000 7f2acdd9d740 1 -- 172.21.15.38:0/1930930680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d62b2f0 msgr2=0x55db9d9f43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.619+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/1930930680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d62b2f0 0x55db9d9f43e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2ab409b820 tx=0x7f2ab4003f10 comp rx=0 tx=0).stop 2024-05-21T15:42:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.619+0000 7f2ac5494640 1 --2- 172.21.15.38:0/814210458 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2ab00a2cb0 0x7f2ab00a51c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8005cc0 tx=0x7f2ab802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.620+0000 7f2acdd9d740 1 -- 172.21.15.38:0/1930930680 shutdown_connections 2024-05-21T15:42:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.620+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/1930930680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6334d0 0x55db9d9fb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.620+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/1930930680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d62b2f0 0x55db9d9f43e0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.620+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/1930930680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6541b0 0x55db9d9f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.620+0000 7f2acdd9d740 1 -- 172.21.15.38:0/1930930680 >> 172.21.15.38:0/1930930680 conn(0x55db9d648300 msgr2=0x55db9d9ef7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.620+0000 7f2acdd9d740 1 -- 172.21.15.38:0/1930930680 shutdown_connections 2024-05-21T15:42:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.620+0000 7f2acdd9d740 1 -- 172.21.15.38:0/1930930680 wait complete. 2024-05-21T15:42:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.620+0000 7f2acdd9d740 1 -- start start 2024-05-21T15:42:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.620+0000 7f2acdd9d740 1 Processor -- start 2024-05-21T15:42:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.620+0000 7f2acdd9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d62b2f0 0x55db9d9fdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2acdd9d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d6334d0 0x55db9d9fe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2acdd9d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6541b0 0x55db9da04fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2acdd9d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55db9d9fbcd0 con 0x55db9d6334d0 2024-05-21T15:42:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2acdd9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55db9d913790 con 0x55db9d62b2f0 2024-05-21T15:42:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2acdd9d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55db9d9fbe50 con 0x55db9d6541b0 2024-05-21T15:42:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2acdd9d740 10 client.0 _ll_register_callbacks cb 0x55db9d94ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2abffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d6334d0 0x55db9d9fe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2abffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d6334d0 0x55db9d9fe070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35126/0 (socket says 172.21.15.38:35126) 2024-05-21T15:42:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2abffff640 1 -- 172.21.15.38:0/901298129 learned_addr learned my addr 172.21.15.38:0/901298129 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2ac4c93640 1 --2- 172.21.15.38:0/901298129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d62b2f0 0x55db9d9fdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2ac5494640 1 --2- 172.21.15.38:0/901298129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6541b0 0x55db9da04fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2abffff640 1 -- 172.21.15.38:0/901298129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6541b0 msgr2=0x55db9da04fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2abffff640 1 --2- 172.21.15.38:0/901298129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6541b0 0x55db9da04fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2abffff640 1 -- 172.21.15.38:0/901298129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d62b2f0 msgr2=0x55db9d9fdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2abffff640 1 --2- 172.21.15.38:0/901298129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d62b2f0 0x55db9d9fdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2abffff640 1 -- 172.21.15.38:0/901298129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9d66c210 con 0x55db9d6334d0 2024-05-21T15:42:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2ac4c93640 1 --2- 172.21.15.38:0/901298129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d62b2f0 0x55db9d9fdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:42:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2abffff640 1 --2- 172.21.15.38:0/901298129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d6334d0 0x55db9d9fe070 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00bad0 tx=0x7f2aac00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45315b78424e9e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2a9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:42:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2a9ffff640 1 -- 172.21.15.38:0/901298129 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac005020 con 0x55db9d6334d0 2024-05-21T15:42:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2a9ffff640 1 -- 172.21.15.38:0/901298129 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2aac0051c0 con 0x55db9d6334d0 2024-05-21T15:42:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.621+0000 7f2acdd9d740 1 -- 172.21.15.38:0/901298129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ab409c220 con 0x55db9d6334d0 2024-05-21T15:42:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.622+0000 7f2a9ffff640 1 -- 172.21.15.38:0/901298129 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2aac0053c0 con 0x55db9d6334d0 2024-05-21T15:42:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.622+0000 7f2acdd9d740 10 client.17646 fetch_fsmap learned FSMap version 35 2024-05-21T15:42:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.622+0000 7f2acdd9d740 1 -- 172.21.15.38:0/901298129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55db9d738620 con 0x55db9d6334d0 2024-05-21T15:42:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.622+0000 7f2a9ffff640 1 -- 172.21.15.38:0/901298129 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ab409c220 con 0x55db9d6334d0 2024-05-21T15:42:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.622+0000 7f2a9ffff640 1 -- 172.21.15.38:0/901298129 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f2aac01b670 con 0x55db9d6334d0 2024-05-21T15:42:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.622+0000 7f2acdd9d740 10 client.17646 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:42:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.622+0000 7f2acdd9d740 10 client.17646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:42:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.622+0000 7f2acdd9d740 20 client.17646 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.622+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/901298129 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55db9d612d80 0x55db9da0b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.622+0000 7f2acdd9d740 4 client.17646 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:42:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.622+0000 7f2acdd9d740 1 -- 172.21.15.38:0/901298129 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db9d965a70 con 0x55db9d612d80 2024-05-21T15:42:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.622+0000 7f2ac4c93640 1 --2- 172.21.15.38:0/901298129 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55db9d612d80 0x55db9da0b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.623+0000 7f2a9ffff640 10 client.17646 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.623+0000 7f2ac4c93640 1 --2- 172.21.15.38:0/901298129 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55db9d612d80 0x55db9da0b590 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ce3de9614f2ab3 server_cookie=ab8f5aa972d7aeaa in_seq=0 out_seq=0 2024-05-21T15:42:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.623+0000 7f2a9ffff640 1 -- 172.21.15.38:0/901298129 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55db9d965a70 con 0x55db9d612d80 2024-05-21T15:42:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.623+0000 7f2a9ffff640 10 client.17646 handle_command_reply: tid=0 2024-05-21T15:42:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.624+0000 7f2acdd9d740 1 client.17646 shutdown 2024-05-21T15:42:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.624+0000 7f2acdd9d740 1 -- 172.21.15.38:0/901298129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d6334d0 msgr2=0x55db9d9fe070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.624+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/901298129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d6334d0 0x55db9d9fe070 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00bad0 tx=0x7f2aac00bf90 comp rx=0 tx=0).stop 2024-05-21T15:42:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.624+0000 7f2acdd9d740 1 -- 172.21.15.38:0/901298129 shutdown_connections 2024-05-21T15:42:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.624+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/901298129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6541b0 0x55db9da04fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.624+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/901298129 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55db9d612d80 0x55db9da0b590 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.624+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/901298129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d6334d0 0x55db9d9fe070 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.624+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/901298129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d62b2f0 0x55db9d9fdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.624+0000 7f2a9ffff640 0 client.17646 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.624+0000 7f2acdd9d740 1 -- 172.21.15.38:0/901298129 >> 172.21.15.38:0/901298129 conn(0x55db9d648300 msgr2=0x55db9d9f27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.624+0000 7f2acdd9d740 1 -- 172.21.15.38:0/901298129 shutdown_connections 2024-05-21T15:42:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.624+0000 7f2acdd9d740 1 -- 172.21.15.38:0/901298129 wait complete. 2024-05-21T15:42:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.624+0000 7f2acdd9d740 20 client.17646 trim_cache size 0 max 16384 2024-05-21T15:42:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.642+0000 7f2acdd9d740 1 Processor -- start 2024-05-21T15:42:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.642+0000 7f2acdd9d740 1 -- start start 2024-05-21T15:42:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.642+0000 7f2acdd9d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d62b2f0 0x55db9d9ef250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.642+0000 7f2acdd9d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d6334d0 0x55db9d9ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.642+0000 7f2acdd9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6541b0 0x55db9da04ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.642+0000 7f2acdd9d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01a9240 con 0x55db9d6334d0 2024-05-21T15:42:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.642+0000 7f2acdd9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01aad80 con 0x55db9d6541b0 2024-05-21T15:42:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.642+0000 7f2acdd9d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01a9910 con 0x55db9d62b2f0 2024-05-21T15:42:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2abffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d6334d0 0x55db9d9ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2abffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d6334d0 0x55db9d9ef790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35134/0 (socket says 172.21.15.38:35134) 2024-05-21T15:42:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2abffff640 1 -- 172.21.15.38:0/4213490482 learned_addr learned my addr 172.21.15.38:0/4213490482 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2ac4c93640 1 --2- 172.21.15.38:0/4213490482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d62b2f0 0x55db9d9ef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2ac5494640 1 --2- 172.21.15.38:0/4213490482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6541b0 0x55db9da04ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2abffff640 1 -- 172.21.15.38:0/4213490482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d62b2f0 msgr2=0x55db9d9ef250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2abffff640 1 --2- 172.21.15.38:0/4213490482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d62b2f0 0x55db9d9ef250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2abffff640 1 -- 172.21.15.38:0/4213490482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6541b0 msgr2=0x55db9da04ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2abffff640 1 --2- 172.21.15.38:0/4213490482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6541b0 0x55db9da04ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2abffff640 1 -- 172.21.15.38:0/4213490482 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db9d9fe730 con 0x55db9d6334d0 2024-05-21T15:42:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2ac5494640 1 --2- 172.21.15.38:0/4213490482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6541b0 0x55db9da04ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:42:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2abffff640 1 --2- 172.21.15.38:0/4213490482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d6334d0 0x55db9d9ef790 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2aac000c00 tx=0x7f2aac003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d8548abbcea6dc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2aaa7fc640 1 -- 172.21.15.38:0/4213490482 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac005560 con 0x55db9d6334d0 2024-05-21T15:42:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2aaa7fc640 1 -- 172.21.15.38:0/4213490482 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aac02c040 con 0x55db9d6334d0 2024-05-21T15:42:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.643+0000 7f2aaa7fc640 1 -- 172.21.15.38:0/4213490482 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac011de0 con 0x55db9d6334d0 2024-05-21T15:42:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.644+0000 7f2acdd9d740 1 -- 172.21.15.38:0/4213490482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d6334d0 msgr2=0x55db9d9ef790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.644+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/4213490482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d6334d0 0x55db9d9ef790 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2aac000c00 tx=0x7f2aac003e80 comp rx=0 tx=0).stop 2024-05-21T15:42:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.644+0000 7f2acdd9d740 1 -- 172.21.15.38:0/4213490482 shutdown_connections 2024-05-21T15:42:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.644+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/4213490482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6541b0 0x55db9da04ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.644+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/4213490482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d6334d0 0x55db9d9ef790 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.644+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/4213490482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d62b2f0 0x55db9d9ef250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.644+0000 7f2acdd9d740 1 -- 172.21.15.38:0/4213490482 >> 172.21.15.38:0/4213490482 conn(0x55db9d648300 msgr2=0x55db9d9f7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.644+0000 7f2acdd9d740 1 -- 172.21.15.38:0/4213490482 shutdown_connections 2024-05-21T15:42:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.644+0000 7f2acdd9d740 1 -- 172.21.15.38:0/4213490482 wait complete. 2024-05-21T15:42:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2acdd9d740 1 -- start start 2024-05-21T15:42:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2acdd9d740 1 Processor -- start 2024-05-21T15:42:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2acdd9d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d62b2f0 0x55db9da027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2acdd9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6334d0 0x55db9da02cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2acdd9d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6541b0 0x55db9da71930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2acdd9d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01aa6b0 con 0x55db9d62b2f0 2024-05-21T15:42:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2acdd9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01a9fe0 con 0x55db9d6334d0 2024-05-21T15:42:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2acdd9d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01a7220 con 0x55db9d6541b0 2024-05-21T15:42:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2acdd9d740 10 client.0 _ll_register_callbacks cb 0x7f2aac004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2ac4c93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d62b2f0 0x55db9da027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2ac4c93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d62b2f0 0x55db9da027b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35140/0 (socket says 172.21.15.38:35140) 2024-05-21T15:42:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2ac4c93640 1 -- 172.21.15.38:0/322234187 learned_addr learned my addr 172.21.15.38:0/322234187 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2ac5494640 1 --2- 172.21.15.38:0/322234187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6541b0 0x55db9da71930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2abffff640 1 --2- 172.21.15.38:0/322234187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6334d0 0x55db9da02cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2ac4c93640 1 -- 172.21.15.38:0/322234187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6541b0 msgr2=0x55db9da71930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2ac4c93640 1 --2- 172.21.15.38:0/322234187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6541b0 0x55db9da71930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2ac4c93640 1 -- 172.21.15.38:0/322234187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6334d0 msgr2=0x55db9da02cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2ac4c93640 1 --2- 172.21.15.38:0/322234187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6334d0 0x55db9da02cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.645+0000 7f2ac4c93640 1 -- 172.21.15.38:0/322234187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9da71ea0 con 0x55db9d62b2f0 2024-05-21T15:42:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2ac4c93640 1 --2- 172.21.15.38:0/322234187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d62b2f0 0x55db9da027b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40d2e30 tx=0x7f2ab40a47b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7304dd92be398 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2a9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:42:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2a9ffff640 1 -- 172.21.15.38:0/322234187 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab409c220 con 0x55db9d62b2f0 2024-05-21T15:42:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2a9ffff640 1 -- 172.21.15.38:0/322234187 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2ab4099e20 con 0x55db9d62b2f0 2024-05-21T15:42:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2acdd9d740 1 -- 172.21.15.38:0/322234187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aac02c040 con 0x55db9d62b2f0 2024-05-21T15:42:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2a9ffff640 1 -- 172.21.15.38:0/322234187 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2ab409fed0 con 0x55db9d62b2f0 2024-05-21T15:42:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2acdd9d740 10 client.17652 fetch_fsmap learned FSMap version 35 2024-05-21T15:42:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2acdd9d740 1 -- 172.21.15.38:0/322234187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55db9da730d0 con 0x55db9d62b2f0 2024-05-21T15:42:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2a9ffff640 1 -- 172.21.15.38:0/322234187 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aac02c040 con 0x55db9d62b2f0 2024-05-21T15:42:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2a9ffff640 1 -- 172.21.15.38:0/322234187 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f2ab40a4300 con 0x55db9d62b2f0 2024-05-21T15:42:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2acdd9d740 10 client.17652 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T15:42:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2acdd9d740 10 client.17652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:42:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2acdd9d740 20 client.17652 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/322234187 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55db9d5e81c0 0x55db9da755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2acdd9d740 4 client.17652 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:42:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2acdd9d740 1 -- 172.21.15.38:0/322234187 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db9d95fdc0 con 0x55db9d5e81c0 2024-05-21T15:42:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.646+0000 7f2abffff640 1 --2- 172.21.15.38:0/322234187 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55db9d5e81c0 0x55db9da755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.647+0000 7f2a9ffff640 10 client.17652 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.647+0000 7f2abffff640 1 --2- 172.21.15.38:0/322234187 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55db9d5e81c0 0x55db9da755c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f712b16a2558 server_cookie=653a5d9aa0c6ce3d in_seq=0 out_seq=0 2024-05-21T15:42:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2a9ffff640 1 -- 172.21.15.38:0/322234187 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db9d95fdc0 con 0x55db9d5e81c0 2024-05-21T15:42:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2a9ffff640 10 client.17652 handle_command_reply: tid=0 2024-05-21T15:42:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2acdd9d740 1 client.17652 shutdown 2024-05-21T15:42:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2acdd9d740 1 -- 172.21.15.38:0/322234187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d62b2f0 msgr2=0x55db9da027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/322234187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d62b2f0 0x55db9da027b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40d2e30 tx=0x7f2ab40a47b0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2acdd9d740 1 -- 172.21.15.38:0/322234187 shutdown_connections 2024-05-21T15:42:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/322234187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db9d6541b0 0x55db9da71930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/322234187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db9d6334d0 0x55db9da02cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/322234187 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x55db9d5e81c0 0x55db9da755c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2acdd9d740 1 --2- 172.21.15.38:0/322234187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55db9d62b2f0 0x55db9da027b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2a9ffff640 0 client.17652 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2acdd9d740 1 -- 172.21.15.38:0/322234187 >> 172.21.15.38:0/322234187 conn(0x55db9d648300 msgr2=0x55db9d9f2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2acdd9d740 1 -- 172.21.15.38:0/322234187 shutdown_connections 2024-05-21T15:42:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2acdd9d740 1 -- 172.21.15.38:0/322234187 wait complete. 2024-05-21T15:42:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.657+0000 7f2acdd9d740 20 client.17652 trim_cache size 0 max 16384 2024-05-21T15:42:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:{ 2024-05-21T15:42:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "message": "", 2024-05-21T15:42:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout: "return_code": 0 2024-05-21T15:42:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stdout:} 2024-05-21T15:42:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.658+0000 7f2ac6ef7640 1 -- 172.21.15.38:0/814210458 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2ab00a2cb0 msgr2=0x7f2ab00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.658+0000 7f2ac6ef7640 1 --2- 172.21.15.38:0/814210458 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2ab00a2cb0 0x7f2ab00a51c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8005cc0 tx=0x7f2ab802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.658+0000 7f2ac6ef7640 1 -- 172.21.15.38:0/814210458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ac01019d0 msgr2=0x7f2ac0198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.658+0000 7f2ac6ef7640 1 --2- 172.21.15.38:0/814210458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ac01019d0 0x7f2ac0198090 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400e9f0 tx=0x7f2ab400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.658+0000 7f2ac6ef7640 1 -- 172.21.15.38:0/814210458 shutdown_connections 2024-05-21T15:42:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.658+0000 7f2ac6ef7640 1 --2- 172.21.15.38:0/814210458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ac0107910 0x7f2ac019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.658+0000 7f2ac6ef7640 1 --2- 172.21.15.38:0/814210458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ac0102c10 0x7f2ac01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.658+0000 7f2ac6ef7640 1 --2- 172.21.15.38:0/814210458 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2ab00a2cb0 0x7f2ab00a51c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.658+0000 7f2ac6ef7640 1 --2- 172.21.15.38:0/814210458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ac01019d0 0x7f2ac0198090 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.658+0000 7f2ac6ef7640 1 -- 172.21.15.38:0/814210458 >> 172.21.15.38:0/814210458 conn(0x7f2ac00fcbc0 msgr2=0x7f2ac01038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.658+0000 7f2ac6ef7640 1 -- 172.21.15.38:0/814210458 shutdown_connections 2024-05-21T15:42:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi038.front.sepia.ceph.com.smithi038.stderr:2024-05-21T15:42:15.658+0000 7f2ac6ef7640 1 -- 172.21.15.38:0/814210458 wait complete. 2024-05-21T15:42:16.075 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:15 smithi038 ceph-mon[31995]: pgmap v586: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 878 KiB/s wr, 237 op/s 2024-05-21T15:42:16.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:15 smithi053 ceph-mon[36496]: pgmap v586: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 878 KiB/s wr, 237 op/s 2024-05-21T15:42:16.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:15 smithi045 ceph-mon[36409]: pgmap v586: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 878 KiB/s wr, 237 op/s 2024-05-21T15:42:18.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:17 smithi053 ceph-mon[36496]: pgmap v587: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 921 KiB/s wr, 240 op/s 2024-05-21T15:42:18.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:17 smithi045 ceph-mon[36409]: pgmap v587: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 921 KiB/s wr, 240 op/s 2024-05-21T15:42:18.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:17 smithi038 ceph-mon[31995]: pgmap v587: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 921 KiB/s wr, 240 op/s 2024-05-21T15:42:20.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:19 smithi053 ceph-mon[36496]: pgmap v588: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.4 MiB/s wr, 602 op/s 2024-05-21T15:42:20.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:19 smithi045 ceph-mon[36409]: pgmap v588: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.4 MiB/s wr, 602 op/s 2024-05-21T15:42:20.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:19 smithi038 ceph-mon[31995]: pgmap v588: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.4 MiB/s wr, 602 op/s 2024-05-21T15:42:20.733 INFO:tasks.workunit.client.0.smithi038.stdout:Syncing()...1 sec 2024-05-21T15:42:20.733 INFO:tasks.workunit.client.0.smithi038.stdout:FFSB benchmark finished at: Tue May 21 15:42:20 2024 2024-05-21T15:42:20.733 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:42:20.733 INFO:tasks.workunit.client.0.smithi038.stdout:Results: 2024-05-21T15:42:20.733 INFO:tasks.workunit.client.0.smithi038.stdout:Benchmark took 339.34 sec 2024-05-21T15:42:20.733 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:42:20.734 INFO:tasks.workunit.client.0.smithi038.stdout:Total Results 2024-05-21T15:42:20.734 INFO:tasks.workunit.client.0.smithi038.stdout:=============== 2024-05-21T15:42:20.734 INFO:tasks.workunit.client.0.smithi038.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-05-21T15:42:20.734 INFO:tasks.workunit.client.0.smithi038.stdout: ======= ============ ========= ======= =========== ========== 2024-05-21T15:42:20.734 INFO:tasks.workunit.client.0.smithi038.stdout: write : 8861440 26113.57 100.000% 100.000% 102MB/sec 2024-05-21T15:42:20.734 INFO:tasks.workunit.client.0.smithi038.stdout:- 2024-05-21T15:42:20.734 INFO:tasks.workunit.client.0.smithi038.stdout:26113.57 Transactions per Second 2024-05-21T15:42:20.734 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:42:20.734 INFO:tasks.workunit.client.0.smithi038.stdout:Throughput Results 2024-05-21T15:42:20.734 INFO:tasks.workunit.client.0.smithi038.stdout:=================== 2024-05-21T15:42:20.734 INFO:tasks.workunit.client.0.smithi038.stdout:Write Throughput: 102MB/sec 2024-05-21T15:42:20.778 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:42:20.779 INFO:tasks.workunit.client.0.smithi038.stdout:System Call Latency statistics in millisecs 2024-05-21T15:42:20.779 INFO:tasks.workunit.client.0.smithi038.stdout:===== 2024-05-21T15:42:20.779 INFO:tasks.workunit.client.0.smithi038.stdout: Min Avg Max Total Calls 2024-05-21T15:42:20.779 INFO:tasks.workunit.client.0.smithi038.stdout: ======== ======== ======== ============ 2024-05-21T15:42:20.779 INFO:tasks.workunit.client.0.smithi038.stdout:[ open] 0.005000 0.672063 358.799011 6923 2024-05-21T15:42:20.779 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-05-21T15:42:20.779 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-05-21T15:42:20.779 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-05-21T15:42:20.779 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-05-21T15:42:20.779 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[4] 200.000000 - 500.000000 : 4 2024-05-21T15:42:20.779 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[5] 100.000000 - 200.000000 : 4 2024-05-21T15:42:20.779 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[6] 50.000000 - 100.000000 : 9 2024-05-21T15:42:20.780 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[7] 20.000000 - 50.000000 : 37 2024-05-21T15:42:20.780 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[8] 10.000000 - 20.000000 : 31 2024-05-21T15:42:20.780 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[9] 5.000000 - 10.000000 : 24 2024-05-21T15:42:20.780 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[10] 2.000000 - 5.000000 : 54 2024-05-21T15:42:20.780 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[11] 1.000000 - 2.000000 : 69 2024-05-21T15:42:20.780 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[12] 0.500000 - 1.000000 : 158 2024-05-21T15:42:20.780 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[13] 0.200000 - 0.500000 : 7 2024-05-21T15:42:20.780 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[14] 0.100000 - 0.200000 : 31 2024-05-21T15:42:20.780 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[15] 0.050000 - 0.100000 : 308 2024-05-21T15:42:20.780 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[16] 0.020000 - 0.050000 : 2881 2024-05-21T15:42:20.780 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[17] 0.010000 - 0.020000 : 2905 2024-05-21T15:42:20.780 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[18] 0.000000 - 0.010000 : 401 2024-05-21T15:42:20.780 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:42:20.781 INFO:tasks.workunit.client.0.smithi038.stdout:[ write] 0.001000 1.125427 36214.660156 8861440 2024-05-21T15:42:20.781 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[0] 5000.000000 - 10000.000000 : 212 2024-05-21T15:42:20.781 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[1] 2000.000000 - 5000.000000 : 850 2024-05-21T15:42:20.781 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[2] 1000.000000 - 2000.000000 : 390 2024-05-21T15:42:20.781 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[3] 500.000000 - 1000.000000 : 387 2024-05-21T15:42:20.781 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[4] 200.000000 - 500.000000 : 825 2024-05-21T15:42:20.781 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[5] 100.000000 - 200.000000 : 2161 2024-05-21T15:42:20.781 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[6] 50.000000 - 100.000000 : 25086 2024-05-21T15:42:20.781 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[7] 20.000000 - 50.000000 : 20787 2024-05-21T15:42:20.781 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[8] 10.000000 - 20.000000 : 7151 2024-05-21T15:42:20.781 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[9] 5.000000 - 10.000000 : 8326 2024-05-21T15:42:20.781 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[10] 2.000000 - 5.000000 : 14656 2024-05-21T15:42:20.781 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[11] 1.000000 - 2.000000 : 512 2024-05-21T15:42:20.781 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[12] 0.500000 - 1.000000 : 480 2024-05-21T15:42:20.782 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[13] 0.200000 - 0.500000 : 536 2024-05-21T15:42:20.782 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[14] 0.100000 - 0.200000 : 669 2024-05-21T15:42:20.782 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[15] 0.050000 - 0.100000 : 2189 2024-05-21T15:42:20.782 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[16] 0.020000 - 0.050000 : 13091 2024-05-21T15:42:20.782 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[17] 0.010000 - 0.020000 : 42798 2024-05-21T15:42:20.782 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[18] 0.000000 - 0.010000 : 8720202 2024-05-21T15:42:20.782 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:42:20.782 INFO:tasks.workunit.client.0.smithi038.stdout:[ lseek] 0.000000 0.000851 40.185001 8861440 2024-05-21T15:42:20.782 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-05-21T15:42:20.782 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-05-21T15:42:20.782 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-05-21T15:42:20.782 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-05-21T15:42:20.782 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-05-21T15:42:20.782 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-05-21T15:42:20.783 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-05-21T15:42:20.783 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[7] 20.000000 - 50.000000 : 31 2024-05-21T15:42:20.783 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[8] 10.000000 - 20.000000 : 55 2024-05-21T15:42:20.783 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[9] 5.000000 - 10.000000 : 33 2024-05-21T15:42:20.783 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[10] 2.000000 - 5.000000 : 47 2024-05-21T15:42:20.783 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[11] 1.000000 - 2.000000 : 27 2024-05-21T15:42:20.783 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[12] 0.500000 - 1.000000 : 36 2024-05-21T15:42:20.783 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[13] 0.200000 - 0.500000 : 39 2024-05-21T15:42:20.783 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[14] 0.100000 - 0.200000 : 76 2024-05-21T15:42:20.783 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[15] 0.050000 - 0.100000 : 203 2024-05-21T15:42:20.783 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[16] 0.020000 - 0.050000 : 1155 2024-05-21T15:42:20.783 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[17] 0.010000 - 0.020000 : 3642 2024-05-21T15:42:20.783 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[18] 0.000000 - 0.010000 : 8856096 2024-05-21T15:42:20.783 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:42:20.784 INFO:tasks.workunit.client.0.smithi038.stdout:[ close] 0.001000 0.014898 64.014000 6923 2024-05-21T15:42:20.784 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-05-21T15:42:20.784 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-05-21T15:42:20.784 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-05-21T15:42:20.784 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-05-21T15:42:20.784 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-05-21T15:42:20.784 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-05-21T15:42:20.784 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[6] 50.000000 - 100.000000 : 1 2024-05-21T15:42:20.784 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-05-21T15:42:20.784 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-05-21T15:42:20.784 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2024-05-21T15:42:20.785 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[10] 2.000000 - 5.000000 : 0 2024-05-21T15:42:20.785 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[11] 1.000000 - 2.000000 : 0 2024-05-21T15:42:20.785 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[12] 0.500000 - 1.000000 : 0 2024-05-21T15:42:20.785 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[13] 0.200000 - 0.500000 : 0 2024-05-21T15:42:20.785 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-05-21T15:42:20.785 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[15] 0.050000 - 0.100000 : 1 2024-05-21T15:42:20.785 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[16] 0.020000 - 0.050000 : 29 2024-05-21T15:42:20.785 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[17] 0.010000 - 0.020000 : 67 2024-05-21T15:42:20.785 INFO:tasks.workunit.client.0.smithi038.stdout: msec_range[18] 0.000000 - 0.010000 : 6825 2024-05-21T15:42:20.785 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:42:20.785 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:42:20.785 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:42:20.785 INFO:tasks.workunit.client.0.smithi038.stdout: 2024-05-21T15:42:20.786 INFO:tasks.workunit.client.0.smithi038.stdout:2.4% User Time 2024-05-21T15:42:20.786 INFO:tasks.workunit.client.0.smithi038.stdout:8.4% System Time 2024-05-21T15:42:20.786 INFO:tasks.workunit.client.0.smithi038.stdout:10.8% CPU Utilization 2024-05-21T15:42:20.803 INFO:tasks.workunit.client.0.smithi038.stderr:+ cd .. 2024-05-21T15:42:20.804 INFO:tasks.workunit.client.0.smithi038.stderr:+ rm -r tmp ffsb 2024-05-21T15:42:22.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:21 smithi053 ceph-mon[36496]: pgmap v589: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.8 MiB/s wr, 701 op/s 2024-05-21T15:42:22.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:21 smithi045 ceph-mon[36409]: pgmap v589: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.8 MiB/s wr, 701 op/s 2024-05-21T15:42:22.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:21 smithi038 ceph-mon[31995]: pgmap v589: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.8 MiB/s wr, 701 op/s 2024-05-21T15:42:23.506 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-21T15:42:23.507 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-21T15:42:23.589 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-05-21T15:42:23.589 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-21T15:42:24.049 DEBUG:teuthology.parallel:result is None 2024-05-21T15:42:24.050 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-21T15:42:24.079 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:23 smithi038 ceph-mon[31995]: pgmap v590: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 852 op/s 2024-05-21T15:42:24.079 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:23 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:24.110 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-21T15:42:24.111 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-21T15:42:24.121 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi038.front.sepia.ceph.com 2024-05-21T15:42:24.159 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-21T15:42:24.169 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-21T15:42:24.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:23 smithi053 ceph-mon[36496]: pgmap v590: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 852 op/s 2024-05-21T15:42:24.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:23 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:24.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:23 smithi045 ceph-mon[36409]: pgmap v590: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 852 op/s 2024-05-21T15:42:24.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:23 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:25.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:24 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:25.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:24 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:25.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:24 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:26.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:25 smithi053 ceph-mon[36496]: pgmap v591: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 3.2 MiB/s wr, 779 op/s 2024-05-21T15:42:26.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:25 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:26.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:25 smithi045 ceph-mon[36409]: pgmap v591: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 3.2 MiB/s wr, 779 op/s 2024-05-21T15:42:26.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:25 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:26.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:25 smithi038 ceph-mon[31995]: pgmap v591: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 3.2 MiB/s wr, 779 op/s 2024-05-21T15:42:26.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:25 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:27.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:26 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:27.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:26 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:27.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:26 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:28.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:27 smithi053 ceph-mon[36496]: pgmap v592: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.1 MiB/s wr, 778 op/s 2024-05-21T15:42:28.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:27 smithi045 ceph-mon[36409]: pgmap v592: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.1 MiB/s wr, 778 op/s 2024-05-21T15:42:28.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:27 smithi038 ceph-mon[31995]: pgmap v592: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.1 MiB/s wr, 778 op/s 2024-05-21T15:42:28.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:42:28.764 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:42:28.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.954+0000 7fdd11bc3640 1 -- 172.21.15.38:0/3606376153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd0c1093d0 msgr2=0x7fdd0c109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:28.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.954+0000 7fdd11bc3640 1 --2- 172.21.15.38:0/3606376153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd0c1093d0 0x7fdd0c109d80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009ca0 tx=0x7fdcfc02a8f0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.955+0000 7fdd11bc3640 1 -- 172.21.15.38:0/3606376153 shutdown_connections 2024-05-21T15:42:28.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.955+0000 7fdd11bc3640 1 --2- 172.21.15.38:0/3606376153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd0c10efe0 0x7fdd0c10f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.955+0000 7fdd11bc3640 1 --2- 172.21.15.38:0/3606376153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0c10a2c0 0x7fdd0c10a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.955+0000 7fdd11bc3640 1 --2- 172.21.15.38:0/3606376153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd0c1093d0 0x7fdd0c109d80 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.955+0000 7fdd11bc3640 1 -- 172.21.15.38:0/3606376153 >> 172.21.15.38:0/3606376153 conn(0x7fdd0c06f4a0 msgr2=0x7fdd0c071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:28.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.956+0000 7fdd11bc3640 1 -- 172.21.15.38:0/3606376153 shutdown_connections 2024-05-21T15:42:28.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.956+0000 7fdd11bc3640 1 -- 172.21.15.38:0/3606376153 wait complete. 2024-05-21T15:42:28.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.956+0000 7fdd11bc3640 1 Processor -- start 2024-05-21T15:42:28.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.957+0000 7fdd11bc3640 1 -- start start 2024-05-21T15:42:28.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.958+0000 7fdd11bc3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0c1093d0 0x7fdd0c1a8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:28.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.958+0000 7fdd11bc3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd0c10a2c0 0x7fdd0c1a8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:28.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.958+0000 7fdd11bc3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd0c10efe0 0x7fdd0c1af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:28.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.958+0000 7fdd11bc3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c11d690 con 0x7fdd0c10a2c0 2024-05-21T15:42:28.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.958+0000 7fdd11bc3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c11d510 con 0x7fdd0c1093d0 2024-05-21T15:42:28.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.958+0000 7fdd11bc3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c11d810 con 0x7fdd0c10efe0 2024-05-21T15:42:28.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.958+0000 7fdd0bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd0c10efe0 0x7fdd0c1af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:28.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.958+0000 7fdd0bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd0c10efe0 0x7fdd0c1af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39654/0 (socket says 172.21.15.38:39654) 2024-05-21T15:42:28.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.958+0000 7fdd0bfff640 1 -- 172.21.15.38:0/3672131791 learned_addr learned my addr 172.21.15.38:0/3672131791 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:28.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.959+0000 7fdd0b7fe640 1 --2- 172.21.15.38:0/3672131791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0c1093d0 0x7fdd0c1a8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:28.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.959+0000 7fdd0affd640 1 --2- 172.21.15.38:0/3672131791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd0c10a2c0 0x7fdd0c1a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:28.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.960+0000 7fdd0b7fe640 1 -- 172.21.15.38:0/3672131791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd0c10efe0 msgr2=0x7fdd0c1af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:28.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.960+0000 7fdd0b7fe640 1 --2- 172.21.15.38:0/3672131791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd0c10efe0 0x7fdd0c1af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.960+0000 7fdd0b7fe640 1 -- 172.21.15.38:0/3672131791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd0c10a2c0 msgr2=0x7fdd0c1a8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:28.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.960+0000 7fdd0b7fe640 1 --2- 172.21.15.38:0/3672131791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd0c10a2c0 0x7fdd0c1a8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.960+0000 7fdd0b7fe640 1 -- 172.21.15.38:0/3672131791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c1afe40 con 0x7fdd0c1093d0 2024-05-21T15:42:28.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.960+0000 7fdd0b7fe640 1 --2- 172.21.15.38:0/3672131791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0c1093d0 0x7fdd0c1a8300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc02ae10 tx=0x7fdcfc003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:28.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.962+0000 7fdd08ff9640 1 -- 172.21.15.38:0/3672131791 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc004240 con 0x7fdd0c1093d0 2024-05-21T15:42:28.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.962+0000 7fdd08ff9640 1 -- 172.21.15.38:0/3672131791 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcfc02d070 con 0x7fdd0c1093d0 2024-05-21T15:42:28.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.962+0000 7fdd08ff9640 1 -- 172.21.15.38:0/3672131791 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc03b410 con 0x7fdd0c1093d0 2024-05-21T15:42:28.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.963+0000 7fdd11bc3640 1 -- 172.21.15.38:0/3672131791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c1b00d0 con 0x7fdd0c1093d0 2024-05-21T15:42:28.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.963+0000 7fdd11bc3640 1 -- 172.21.15.38:0/3672131791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c1b0650 con 0x7fdd0c1093d0 2024-05-21T15:42:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.969+0000 7fdd08ff9640 1 -- 172.21.15.38:0/3672131791 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdcfc030680 con 0x7fdd0c1093d0 2024-05-21T15:42:28.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.970+0000 7fdd18a6d740 1 Processor -- start 2024-05-21T15:42:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.970+0000 7fdd18a6d740 1 -- start start 2024-05-21T15:42:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.970+0000 7fdd18a6d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 0x56235e6dcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.970+0000 7fdd18a6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e3144d0 0x56235e6dd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:28.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.970+0000 7fdd18a6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e31c6b0 0x56235e6e4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.970+0000 7fdd18a6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56235e641d00 con 0x56235e31c6b0 2024-05-21T15:42:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.970+0000 7fdd18a6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56235e5e94c0 con 0x56235e3144d0 2024-05-21T15:42:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.970+0000 7fdd18a6d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56235e63dcc0 con 0x56235e33d390 2024-05-21T15:42:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.971+0000 7fdd08ff9640 1 --2- 172.21.15.38:0/3672131791 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdcf80a2d60 0x7fdcf80a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.971+0000 7fdd0affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e31c6b0 0x56235e6e4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.971+0000 7fdd0affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e31c6b0 0x56235e6e4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59826/0 (socket says 172.21.15.38:59826) 2024-05-21T15:42:28.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.971+0000 7fdd0affd640 1 -- 172.21.15.38:0/839975333 learned_addr learned my addr 172.21.15.38:0/839975333 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:28.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.971+0000 7fdd08ff9640 1 -- 172.21.15.38:0/3672131791 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdcfc0f0950 con 0x7fdd0c1093d0 2024-05-21T15:42:28.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.971+0000 7fdd0bfff640 1 --2- 172.21.15.38:0/3672131791 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdcf80a2d60 0x7fdcf80a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:28.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.972+0000 7fdd0affd640 1 -- 172.21.15.38:0/839975333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 msgr2=0x56235e6dcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:28.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.972+0000 7fdd0affd640 1 --2- 172.21.15.38:0/839975333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 0x56235e6dcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.972+0000 7fdd0affd640 1 -- 172.21.15.38:0/839975333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e3144d0 msgr2=0x56235e6dd130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:42:28.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.972+0000 7fdd0affd640 1 --2- 172.21.15.38:0/839975333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e3144d0 0x56235e6dd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.972+0000 7fdd0affd640 1 -- 172.21.15.38:0/839975333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235e41f7d0 con 0x56235e31c6b0 2024-05-21T15:42:28.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.973+0000 7fdd0affd640 1 --2- 172.21.15.38:0/839975333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e31c6b0 0x56235e6e4030 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009a10 tx=0x7fdcf4009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80e3ee9fe53be38 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:28.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.973+0000 7fdd0bfff640 1 --2- 172.21.15.38:0/3672131791 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdcf80a2d60 0x7fdcf80a5270 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x56235e6ddd80 tx=0x7fdd0400a0d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:28.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.973+0000 7fdd01ffb640 1 -- 172.21.15.38:0/839975333 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4017070 con 0x56235e31c6b0 2024-05-21T15:42:28.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.973+0000 7fdd01ffb640 1 -- 172.21.15.38:0/839975333 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcf4013470 con 0x56235e31c6b0 2024-05-21T15:42:28.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.973+0000 7fdd01ffb640 1 -- 172.21.15.38:0/839975333 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4012630 con 0x56235e31c6b0 2024-05-21T15:42:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.974+0000 7fdd18a6d740 1 -- 172.21.15.38:0/839975333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e31c6b0 msgr2=0x56235e6e4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.974+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/839975333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e31c6b0 0x56235e6e4030 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009a10 tx=0x7fdcf4009ed0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.975+0000 7fdd18a6d740 1 -- 172.21.15.38:0/839975333 shutdown_connections 2024-05-21T15:42:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.975+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/839975333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e31c6b0 0x56235e6e4030 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.975+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/839975333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e3144d0 0x56235e6dd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.975+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/839975333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 0x56235e6dcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.975+0000 7fdd18a6d740 1 -- 172.21.15.38:0/839975333 >> 172.21.15.38:0/839975333 conn(0x56235e3314e0 msgr2=0x56235e6d8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.975+0000 7fdd18a6d740 1 -- 172.21.15.38:0/839975333 shutdown_connections 2024-05-21T15:42:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.975+0000 7fdd18a6d740 1 -- 172.21.15.38:0/839975333 wait complete. 2024-05-21T15:42:28.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.976+0000 7fdd18a6d740 1 -- start start 2024-05-21T15:42:28.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.976+0000 7fdd18a6d740 1 Processor -- start 2024-05-21T15:42:28.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.976+0000 7fdd18a6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e3144d0 0x56235e6f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:28.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd18a6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e31c6b0 0x56235e6f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:28.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd18a6d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 0x56235e6ff050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:28.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd18a6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56235e6f5a90 con 0x56235e3144d0 2024-05-21T15:42:28.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd18a6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56235e5fcb90 con 0x56235e31c6b0 2024-05-21T15:42:28.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd18a6d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56235e6f5c10 con 0x56235e33d390 2024-05-21T15:42:28.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd18a6d740 10 client.0 _ll_register_callbacks cb 0x56235e638da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:28.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd0b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e3144d0 0x56235e6f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:28.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd0b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e3144d0 0x56235e6f7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59834/0 (socket says 172.21.15.38:59834) 2024-05-21T15:42:28.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd0b7fe640 1 -- 172.21.15.38:0/1274989805 learned_addr learned my addr 172.21.15.38:0/1274989805 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:28.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd0bfff640 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 0x56235e6ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:28.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd0b7fe640 1 -- 172.21.15.38:0/1274989805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 msgr2=0x56235e6ff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:28.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd0b7fe640 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 0x56235e6ff050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd0b7fe640 1 -- 172.21.15.38:0/1274989805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e31c6b0 msgr2=0x56235e6f80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:28.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd0affd640 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e31c6b0 0x56235e6f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:28.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd0b7fe640 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e31c6b0 0x56235e6f80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd0b7fe640 1 -- 172.21.15.38:0/1274989805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56235e3553f0 con 0x56235e3144d0 2024-05-21T15:42:28.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd0bfff640 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 0x56235e6ff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:42:28.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.977+0000 7fdd0affd640 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e31c6b0 0x56235e6f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:42:28.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.978+0000 7fdd0b7fe640 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e3144d0 0x56235e6f7bb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0c5840 tx=0x7fdcfc0bcf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e33d6e781f55885 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:28.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.979+0000 7fdceaffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:42:28.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.979+0000 7fdceaffd640 1 -- 172.21.15.38:0/1274989805 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc0bd100 con 0x56235e3144d0 2024-05-21T15:42:28.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.979+0000 7fdceaffd640 1 -- 172.21.15.38:0/1274989805 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdcfc0c1170 con 0x56235e3144d0 2024-05-21T15:42:28.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.979+0000 7fdd18a6d740 1 -- 172.21.15.38:0/1274989805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcf4013470 con 0x56235e3144d0 2024-05-21T15:42:28.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.980+0000 7fdceaffd640 1 -- 172.21.15.38:0/1274989805 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdcfc0c14b0 con 0x56235e3144d0 2024-05-21T15:42:28.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.981+0000 7fdd18a6d740 10 client.17664 fetch_fsmap learned FSMap version 40 2024-05-21T15:42:28.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.981+0000 7fdd18a6d740 1 -- 172.21.15.38:0/1274989805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56235e3150f0 con 0x56235e3144d0 2024-05-21T15:42:28.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.982+0000 7fdceaffd640 1 -- 172.21.15.38:0/1274989805 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcf4013470 con 0x56235e3144d0 2024-05-21T15:42:28.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.982+0000 7fdceaffd640 1 -- 172.21.15.38:0/1274989805 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fdcfc0c59b0 con 0x56235e3144d0 2024-05-21T15:42:28.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.982+0000 7fdd18a6d740 10 client.17664 fetch_fsmap finished waiting for FSMap version 40 2024-05-21T15:42:28.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.982+0000 7fdd18a6d740 10 client.17664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:42:28.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.982+0000 7fdd18a6d740 20 client.17664 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:28.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.982+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56235e2fbf60 0x56235e705a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:28.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.982+0000 7fdd18a6d740 4 client.17664 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:42:28.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.982+0000 7fdd18a6d740 1 -- 172.21.15.38:0/1274989805 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56235e64f430 con 0x56235e2fbf60 2024-05-21T15:42:28.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.986+0000 7fdd0affd640 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56235e2fbf60 0x56235e705a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:28.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.986+0000 7fdd0affd640 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56235e2fbf60 0x56235e705a60 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=543894a0cc6ab687 server_cookie=9d16de6a1c6f36f7 in_seq=0 out_seq=0 2024-05-21T15:42:28.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.986+0000 7fdceaffd640 10 client.17664 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:28.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.989+0000 7fdceaffd640 1 -- 172.21.15.38:0/1274989805 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56235e64f430 con 0x56235e2fbf60 2024-05-21T15:42:28.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.989+0000 7fdceaffd640 10 client.17664 handle_command_reply: tid=0 2024-05-21T15:42:28.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.989+0000 7fdd18a6d740 1 client.17664 shutdown 2024-05-21T15:42:28.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.989+0000 7fdd18a6d740 1 -- 172.21.15.38:0/1274989805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e3144d0 msgr2=0x56235e6f7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:28.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.989+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e3144d0 0x56235e6f7bb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0c5840 tx=0x7fdcfc0bcf90 comp rx=0 tx=0).stop 2024-05-21T15:42:28.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.991+0000 7fdd18a6d740 1 -- 172.21.15.38:0/1274989805 shutdown_connections 2024-05-21T15:42:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.991+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 0x56235e6ff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.991+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e31c6b0 0x56235e6f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.992+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56235e2fbf60 0x56235e705a60 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.992+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/1274989805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e3144d0 0x56235e6f7bb0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.992+0000 7fdd18a6d740 1 -- 172.21.15.38:0/1274989805 >> 172.21.15.38:0/1274989805 conn(0x56235e3314e0 msgr2=0x56235e6d75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.992+0000 7fdceaffd640 0 client.17664 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.992+0000 7fdd18a6d740 1 -- 172.21.15.38:0/1274989805 shutdown_connections 2024-05-21T15:42:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.992+0000 7fdd18a6d740 1 -- 172.21.15.38:0/1274989805 wait complete. 2024-05-21T15:42:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:28.992+0000 7fdd18a6d740 20 client.17664 trim_cache size 0 max 16384 2024-05-21T15:42:29.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.025+0000 7fdd18a6d740 1 Processor -- start 2024-05-21T15:42:29.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.026+0000 7fdd18a6d740 1 -- start start 2024-05-21T15:42:29.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.026+0000 7fdd18a6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e3144d0 0x56235e6f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.026+0000 7fdd18a6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e31c6b0 0x56235e6fefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.026+0000 7fdd18a6d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 0x56235e6f6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.027+0000 7fdd18a6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1b0bd0 con 0x56235e3144d0 2024-05-21T15:42:29.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.027+0000 7fdd18a6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1b27d0 con 0x56235e31c6b0 2024-05-21T15:42:29.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.027+0000 7fdd18a6d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1b1300 con 0x56235e33d390 2024-05-21T15:42:29.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.027+0000 7fdd0affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e31c6b0 0x56235e6fefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.027+0000 7fdd0affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e31c6b0 0x56235e6fefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:40376/0 (socket says 172.21.15.38:40376) 2024-05-21T15:42:29.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.027+0000 7fdd0affd640 1 -- 172.21.15.38:0/3307615796 learned_addr learned my addr 172.21.15.38:0/3307615796 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:29.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.027+0000 7fdd0b7fe640 1 --2- 172.21.15.38:0/3307615796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e3144d0 0x56235e6f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.027+0000 7fdd0bfff640 1 --2- 172.21.15.38:0/3307615796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 0x56235e6f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.028+0000 7fdd0affd640 1 -- 172.21.15.38:0/3307615796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 msgr2=0x56235e6f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:29.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.028+0000 7fdd0affd640 1 --2- 172.21.15.38:0/3307615796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 0x56235e6f6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.028+0000 7fdd0affd640 1 -- 172.21.15.38:0/3307615796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e3144d0 msgr2=0x56235e6f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:29.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.028+0000 7fdd0affd640 1 --2- 172.21.15.38:0/3307615796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e3144d0 0x56235e6f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.028+0000 7fdd0affd640 1 -- 172.21.15.38:0/3307615796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235e6dd9a0 con 0x56235e31c6b0 2024-05-21T15:42:29.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.028+0000 7fdd0b7fe640 1 --2- 172.21.15.38:0/3307615796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e3144d0 0x56235e6f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:42:29.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.028+0000 7fdd0affd640 1 --2- 172.21.15.38:0/3307615796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e31c6b0 0x56235e6fefe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x56235e6d92d0 tx=0x7fdcf4009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d09d071ea830394 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:29.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.028+0000 7fdd017fa640 1 -- 172.21.15.38:0/3307615796 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf402b070 con 0x56235e31c6b0 2024-05-21T15:42:29.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.029+0000 7fdd017fa640 1 -- 172.21.15.38:0/3307615796 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcf400ed60 con 0x56235e31c6b0 2024-05-21T15:42:29.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.030+0000 7fdd18a6d740 1 -- 172.21.15.38:0/3307615796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e31c6b0 msgr2=0x56235e6fefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.030+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/3307615796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e31c6b0 0x56235e6fefe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x56235e6d92d0 tx=0x7fdcf4009290 comp rx=0 tx=0).stop 2024-05-21T15:42:29.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.030+0000 7fdd18a6d740 1 -- 172.21.15.38:0/3307615796 shutdown_connections 2024-05-21T15:42:29.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.030+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/3307615796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e33d390 0x56235e6f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.030+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/3307615796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e31c6b0 0x56235e6fefe0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.030+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/3307615796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e3144d0 0x56235e6f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.030+0000 7fdd18a6d740 1 -- 172.21.15.38:0/3307615796 >> 172.21.15.38:0/3307615796 conn(0x56235e3314e0 msgr2=0x56235e6e0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:29.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.031+0000 7fdd18a6d740 1 -- 172.21.15.38:0/3307615796 shutdown_connections 2024-05-21T15:42:29.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.031+0000 7fdd18a6d740 1 -- 172.21.15.38:0/3307615796 wait complete. 2024-05-21T15:42:29.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.031+0000 7fdd18a6d740 1 -- start start 2024-05-21T15:42:29.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.031+0000 7fdd18a6d740 1 Processor -- start 2024-05-21T15:42:29.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.032+0000 7fdd18a6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e3144d0 0x56235e6feee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.032+0000 7fdd18a6d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e31c6b0 0x56235e6fd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.032+0000 7fdd18a6d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e33d390 0x56235e6fdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.032+0000 7fdd18a6d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1b2100 con 0x56235e33d390 2024-05-21T15:42:29.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.032+0000 7fdd18a6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1b1a00 con 0x56235e3144d0 2024-05-21T15:42:29.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.032+0000 7fdd18a6d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c117560 con 0x56235e31c6b0 2024-05-21T15:42:29.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.032+0000 7fdd18a6d740 10 client.0 _ll_register_callbacks cb 0x7fdcfc009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:29.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.033+0000 7fdd0affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e31c6b0 0x56235e6fd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.033+0000 7fdd0affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e31c6b0 0x56235e6fd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39694/0 (socket says 172.21.15.38:39694) 2024-05-21T15:42:29.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.033+0000 7fdd0affd640 1 -- 172.21.15.38:0/4176263088 learned_addr learned my addr 172.21.15.38:0/4176263088 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:29.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.033+0000 7fdd0bfff640 1 --2- 172.21.15.38:0/4176263088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e33d390 0x56235e6fdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.033+0000 7fdd0affd640 1 -- 172.21.15.38:0/4176263088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e3144d0 msgr2=0x56235e6feee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:29.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.033+0000 7fdd0affd640 1 --2- 172.21.15.38:0/4176263088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e3144d0 0x56235e6feee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.033+0000 7fdd0affd640 1 -- 172.21.15.38:0/4176263088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e33d390 msgr2=0x56235e6fdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:29.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.033+0000 7fdd0affd640 1 --2- 172.21.15.38:0/4176263088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e33d390 0x56235e6fdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.033+0000 7fdd0affd640 1 -- 172.21.15.38:0/4176263088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56235e6fe250 con 0x56235e31c6b0 2024-05-21T15:42:29.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.033+0000 7fdd0affd640 1 --2- 172.21.15.38:0/4176263088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e31c6b0 0x56235e6fd550 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400ef90 tx=0x7fdcf4009690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c80df5c0f8f005 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:29.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.033+0000 7fdceaffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:42:29.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.034+0000 7fdceaffd640 1 -- 172.21.15.38:0/4176263088 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4034060 con 0x56235e31c6b0 2024-05-21T15:42:29.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.034+0000 7fdceaffd640 1 -- 172.21.15.38:0/4176263088 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fdcf4025650 con 0x56235e31c6b0 2024-05-21T15:42:29.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.034+0000 7fdd18a6d740 1 -- 172.21.15.38:0/4176263088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcf400ed60 con 0x56235e31c6b0 2024-05-21T15:42:29.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.034+0000 7fdceaffd640 1 -- 172.21.15.38:0/4176263088 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdcf402b070 con 0x56235e31c6b0 2024-05-21T15:42:29.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.035+0000 7fdd18a6d740 10 client.27358 fetch_fsmap learned FSMap version 40 2024-05-21T15:42:29.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.035+0000 7fdd18a6d740 1 -- 172.21.15.38:0/4176263088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56235e76ce20 con 0x56235e31c6b0 2024-05-21T15:42:29.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.036+0000 7fdceaffd640 1 -- 172.21.15.38:0/4176263088 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcf400ed60 con 0x56235e31c6b0 2024-05-21T15:42:29.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.036+0000 7fdceaffd640 1 -- 172.21.15.38:0/4176263088 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fdcf4003bb0 con 0x56235e31c6b0 2024-05-21T15:42:29.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.036+0000 7fdd18a6d740 10 client.27358 fetch_fsmap finished waiting for FSMap version 40 2024-05-21T15:42:29.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.036+0000 7fdd18a6d740 10 client.27358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:42:29.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.036+0000 7fdd18a6d740 20 client.27358 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:29.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.036+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/4176263088 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56235e2d13a0 0x56235e76f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.036+0000 7fdd18a6d740 4 client.27358 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:42:29.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.036+0000 7fdd18a6d740 1 -- 172.21.15.38:0/4176263088 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56235e649780 con 0x56235e2d13a0 2024-05-21T15:42:29.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.038+0000 7fdd0b7fe640 1 --2- 172.21.15.38:0/4176263088 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56235e2d13a0 0x56235e76f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.040+0000 7fdd0b7fe640 1 --2- 172.21.15.38:0/4176263088 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56235e2d13a0 0x56235e76f500 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2938e6040859fd3 server_cookie=7d5cbffab30eb864 in_seq=0 out_seq=0 2024-05-21T15:42:29.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.040+0000 7fdceaffd640 10 client.27358 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:29.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.041+0000 7fdceaffd640 1 -- 172.21.15.38:0/4176263088 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56235e649780 con 0x56235e2d13a0 2024-05-21T15:42:29.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.041+0000 7fdceaffd640 10 client.27358 handle_command_reply: tid=0 2024-05-21T15:42:29.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.041+0000 7fdd18a6d740 1 client.27358 shutdown 2024-05-21T15:42:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.042+0000 7fdd18a6d740 1 -- 172.21.15.38:0/4176263088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e31c6b0 msgr2=0x56235e6fd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.042+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/4176263088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e31c6b0 0x56235e6fd550 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400ef90 tx=0x7fdcf4009690 comp rx=0 tx=0).stop 2024-05-21T15:42:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.042+0000 7fdd18a6d740 1 -- 172.21.15.38:0/4176263088 shutdown_connections 2024-05-21T15:42:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.042+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/4176263088 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56235e2d13a0 0x56235e76f500 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.042+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/4176263088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56235e33d390 0x56235e6fdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.042+0000 7fdceaffd640 0 client.27358 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.042+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/4176263088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56235e31c6b0 0x56235e6fd550 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.042+0000 7fdd18a6d740 1 --2- 172.21.15.38:0/4176263088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56235e3144d0 0x56235e6feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.042+0000 7fdd18a6d740 1 -- 172.21.15.38:0/4176263088 >> 172.21.15.38:0/4176263088 conn(0x56235e3314e0 msgr2=0x56235e6f5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:29.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.042+0000 7fdd18a6d740 1 -- 172.21.15.38:0/4176263088 shutdown_connections 2024-05-21T15:42:29.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.042+0000 7fdd18a6d740 1 -- 172.21.15.38:0/4176263088 wait complete. 2024-05-21T15:42:29.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.042+0000 7fdd18a6d740 20 client.27358 trim_cache size 0 max 16384 2024-05-21T15:42:29.042 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:42:29.043 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-05-21T15:42:29.043 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-05-21T15:42:29.043 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:42:29.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.043+0000 7fdd11bc3640 1 -- 172.21.15.38:0/3672131791 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdcf80a2d60 msgr2=0x7fdcf80a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.043+0000 7fdd11bc3640 1 --2- 172.21.15.38:0/3672131791 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdcf80a2d60 0x7fdcf80a5270 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x56235e6ddd80 tx=0x7fdd0400a0d0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.044+0000 7fdd11bc3640 1 -- 172.21.15.38:0/3672131791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0c1093d0 msgr2=0x7fdd0c1a8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.044+0000 7fdd11bc3640 1 --2- 172.21.15.38:0/3672131791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0c1093d0 0x7fdd0c1a8300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc02ae10 tx=0x7fdcfc003f90 comp rx=0 tx=0).stop 2024-05-21T15:42:29.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.044+0000 7fdd11bc3640 1 -- 172.21.15.38:0/3672131791 shutdown_connections 2024-05-21T15:42:29.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.044+0000 7fdd11bc3640 1 --2- 172.21.15.38:0/3672131791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd0c10efe0 0x7fdd0c1af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.044+0000 7fdd11bc3640 1 --2- 172.21.15.38:0/3672131791 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fdcf80a2d60 0x7fdcf80a5270 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.044+0000 7fdd11bc3640 1 --2- 172.21.15.38:0/3672131791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd0c10a2c0 0x7fdd0c1a8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.044+0000 7fdd11bc3640 1 --2- 172.21.15.38:0/3672131791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0c1093d0 0x7fdd0c1a8300 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.044+0000 7fdd11bc3640 1 -- 172.21.15.38:0/3672131791 >> 172.21.15.38:0/3672131791 conn(0x7fdd0c06f4a0 msgr2=0x7fdd0c10afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:29.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.044+0000 7fdd11bc3640 1 -- 172.21.15.38:0/3672131791 shutdown_connections 2024-05-21T15:42:29.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.044+0000 7fdd11bc3640 1 -- 172.21.15.38:0/3672131791 wait complete. 2024-05-21T15:42:29.060 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:42:29.061 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:42:29.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.255+0000 7f12f2502640 1 -- 172.21.15.38:0/65675538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12ec107900 msgr2=0x7f12ec109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.255+0000 7f12f2502640 1 --2- 172.21.15.38:0/65675538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12ec107900 0x7f12ec109d20 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009ca0 tx=0x7f12e402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.255+0000 7f12f2502640 1 -- 172.21.15.38:0/65675538 shutdown_connections 2024-05-21T15:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.255+0000 7f12f2502640 1 --2- 172.21.15.38:0/65675538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12ec107900 0x7f12ec109d20 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.255+0000 7f12f2502640 1 --2- 172.21.15.38:0/65675538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec102c00 0x7f12ec103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.255+0000 7f12f2502640 1 --2- 172.21.15.38:0/65675538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ec1019c0 0x7f12ec1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.255+0000 7f12f2502640 1 -- 172.21.15.38:0/65675538 >> 172.21.15.38:0/65675538 conn(0x7f12ec0fcbb0 msgr2=0x7f12ec0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.255+0000 7f12f2502640 1 -- 172.21.15.38:0/65675538 shutdown_connections 2024-05-21T15:42:29.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.255+0000 7f12f2502640 1 -- 172.21.15.38:0/65675538 wait complete. 2024-05-21T15:42:29.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.256+0000 7f12f2502640 1 Processor -- start 2024-05-21T15:42:29.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.256+0000 7f12f2502640 1 -- start start 2024-05-21T15:42:29.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12f2502640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ec1019c0 0x7f12ec06b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12f2502640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec102c00 0x7f12ec06be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12f2502640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12ec107900 0x7f12ec072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12ebfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ec1019c0 0x7f12ec06b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12f2502640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec10d670 con 0x7f12ec107900 2024-05-21T15:42:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12ebfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ec1019c0 0x7f12ec06b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39718/0 (socket says 172.21.15.38:39718) 2024-05-21T15:42:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12ebfff640 1 -- 172.21.15.38:0/2637008911 learned_addr learned my addr 172.21.15.38:0/2637008911 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12f0a9f640 1 --2- 172.21.15.38:0/2637008911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12ec107900 0x7f12ec072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12f2502640 1 -- 172.21.15.38:0/2637008911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec10d4f0 con 0x7f12ec102c00 2024-05-21T15:42:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12f2502640 1 -- 172.21.15.38:0/2637008911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec10d7f0 con 0x7f12ec1019c0 2024-05-21T15:42:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12eb7fe640 1 --2- 172.21.15.38:0/2637008911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec102c00 0x7f12ec06be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12f0a9f640 1 -- 172.21.15.38:0/2637008911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ec1019c0 msgr2=0x7f12ec06b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12f0a9f640 1 --2- 172.21.15.38:0/2637008911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ec1019c0 0x7f12ec06b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12f0a9f640 1 -- 172.21.15.38:0/2637008911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec102c00 msgr2=0x7f12ec06be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12f0a9f640 1 --2- 172.21.15.38:0/2637008911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec102c00 0x7f12ec06be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.257+0000 7f12f0a9f640 1 -- 172.21.15.38:0/2637008911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ec073450 con 0x7f12ec107900 2024-05-21T15:42:29.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.258+0000 7f12f0a9f640 1 --2- 172.21.15.38:0/2637008911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12ec107900 0x7f12ec072de0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f12e402aeb0 tx=0x7f12e4003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.258+0000 7f12e97fa640 1 -- 172.21.15.38:0/2637008911 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4046030 con 0x7f12ec107900 2024-05-21T15:42:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.258+0000 7f12f2502640 1 -- 172.21.15.38:0/2637008911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ec0736e0 con 0x7f12ec107900 2024-05-21T15:42:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.258+0000 7f12e97fa640 1 -- 172.21.15.38:0/2637008911 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e4031930 con 0x7f12ec107900 2024-05-21T15:42:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.258+0000 7f12e97fa640 1 -- 172.21.15.38:0/2637008911 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e402f870 con 0x7f12ec107900 2024-05-21T15:42:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.258+0000 7f12f2502640 1 -- 172.21.15.38:0/2637008911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ec06a410 con 0x7f12ec107900 2024-05-21T15:42:29.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.259+0000 7f12f2502640 1 -- 172.21.15.38:0/2637008911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12ec068340 con 0x7f12ec107900 2024-05-21T15:42:29.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.262+0000 7f12e97fa640 1 -- 172.21.15.38:0/2637008911 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f12e4049030 con 0x7f12ec107900 2024-05-21T15:42:29.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.263+0000 7f12e97fa640 1 --2- 172.21.15.38:0/2637008911 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12d40a2cb0 0x7f12d40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.263+0000 7f12e97fa640 1 -- 172.21.15.38:0/2637008911 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f12e40f3a50 con 0x7f12ec107900 2024-05-21T15:42:29.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.263+0000 7f12ebfff640 1 --2- 172.21.15.38:0/2637008911 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12d40a2cb0 0x7f12d40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.264+0000 7f12ebfff640 1 --2- 172.21.15.38:0/2637008911 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12d40a2cb0 0x7f12d40a51c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0095f0 tx=0x7f12dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:29.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.267+0000 7f12e97fa640 1 -- 172.21.15.38:0/2637008911 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f12e40c02b0 con 0x7f12ec107900 2024-05-21T15:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.658+0000 7f12f2502640 1 -- 172.21.15.38:0/2637008911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12ec06ada0 con 0x7f12ec107900 2024-05-21T15:42:29.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.659+0000 7f12e97fa640 1 -- 172.21.15.38:0/2637008911 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11920 (secure 0 0 0) 0x7f12e40bae20 con 0x7f12ec107900 2024-05-21T15:42:29.658 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:42:29.658 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:42:28.496301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24515,24550,24602,24551,24523]},"id":1}]} 2024-05-21T15:42:29.658 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 40 2024-05-21T15:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.661+0000 7f12f2502640 1 -- 172.21.15.38:0/2637008911 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12d40a2cb0 msgr2=0x7f12d40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.661+0000 7f12f2502640 1 --2- 172.21.15.38:0/2637008911 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12d40a2cb0 0x7f12d40a51c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0095f0 tx=0x7f12dc005f90 comp rx=0 tx=0).stop 2024-05-21T15:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.661+0000 7f12f2502640 1 -- 172.21.15.38:0/2637008911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12ec107900 msgr2=0x7f12ec072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.661+0000 7f12f2502640 1 --2- 172.21.15.38:0/2637008911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12ec107900 0x7f12ec072de0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f12e402aeb0 tx=0x7f12e4003c70 comp rx=0 tx=0).stop 2024-05-21T15:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.661+0000 7f12f2502640 1 -- 172.21.15.38:0/2637008911 shutdown_connections 2024-05-21T15:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.661+0000 7f12f2502640 1 --2- 172.21.15.38:0/2637008911 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12d40a2cb0 0x7f12d40a51c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.661+0000 7f12f2502640 1 --2- 172.21.15.38:0/2637008911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12ec107900 0x7f12ec072de0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.661+0000 7f12f2502640 1 --2- 172.21.15.38:0/2637008911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec102c00 0x7f12ec06be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.661+0000 7f12f2502640 1 --2- 172.21.15.38:0/2637008911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12ec1019c0 0x7f12ec06b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.661+0000 7f12f2502640 1 -- 172.21.15.38:0/2637008911 >> 172.21.15.38:0/2637008911 conn(0x7f12ec0fcbb0 msgr2=0x7f12ec0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.662+0000 7f12f2502640 1 -- 172.21.15.38:0/2637008911 shutdown_connections 2024-05-21T15:42:29.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.662+0000 7f12f2502640 1 -- 172.21.15.38:0/2637008911 wait complete. 2024-05-21T15:42:29.679 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:42:29.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.845+0000 7fa08872c640 1 -- 172.21.15.38:0/3913844814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa080070190 msgr2=0x7fa0800705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.845+0000 7fa08872c640 1 --2- 172.21.15.38:0/3913844814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa080070190 0x7fa0800705a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009900 tx=0x7fa07c02a440 comp rx=0 tx=0).stop 2024-05-21T15:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.845+0000 7fa08872c640 1 -- 172.21.15.38:0/3913844814 shutdown_connections 2024-05-21T15:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.846+0000 7fa08872c640 1 --2- 172.21.15.38:0/3913844814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa080070190 0x7fa0800705a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.846+0000 7fa08872c640 1 --2- 172.21.15.38:0/3913844814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa08006f490 0x7fa08006f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.846+0000 7fa08872c640 1 --2- 172.21.15.38:0/3913844814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa08006e200 0x7fa08006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.846+0000 7fa08872c640 1 -- 172.21.15.38:0/3913844814 >> 172.21.15.38:0/3913844814 conn(0x7fa08006d3f0 msgr2=0x7fa08010ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:29.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.847+0000 7fa08872c640 1 -- 172.21.15.38:0/3913844814 shutdown_connections 2024-05-21T15:42:29.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.847+0000 7fa08872c640 1 -- 172.21.15.38:0/3913844814 wait complete. 2024-05-21T15:42:29.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.848+0000 7fa08872c640 1 Processor -- start 2024-05-21T15:42:29.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.849+0000 7fa08872c640 1 -- start start 2024-05-21T15:42:29.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.849+0000 7fa08872c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa08006e200 0x7fa0801a7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.849+0000 7fa08872c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa08006f490 0x7fa0801a8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.849+0000 7fa08872c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa080070190 0x7fa0801af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.849+0000 7fa08872c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa08011d4e0 con 0x7fa08006f490 2024-05-21T15:42:29.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.849+0000 7fa08872c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa08011d360 con 0x7fa08006e200 2024-05-21T15:42:29.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.849+0000 7fa08872c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa08011d660 con 0x7fa080070190 2024-05-21T15:42:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.850+0000 7fa085cc7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa08006f490 0x7fa0801a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.850+0000 7fa0864c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa08006e200 0x7fa0801a7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.850+0000 7fa085cc7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa08006f490 0x7fa0801a8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59886/0 (socket says 172.21.15.38:59886) 2024-05-21T15:42:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.850+0000 7fa0864c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa08006e200 0x7fa0801a7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:40428/0 (socket says 172.21.15.38:40428) 2024-05-21T15:42:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.850+0000 7fa0864c8640 1 -- 172.21.15.38:0/842181568 learned_addr learned my addr 172.21.15.38:0/842181568 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:29.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.850+0000 7fa086cc9640 1 --2- 172.21.15.38:0/842181568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa080070190 0x7fa0801af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.851+0000 7fa085cc7640 1 -- 172.21.15.38:0/842181568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa080070190 msgr2=0x7fa0801af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.851+0000 7fa085cc7640 1 --2- 172.21.15.38:0/842181568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa080070190 0x7fa0801af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.851+0000 7fa085cc7640 1 -- 172.21.15.38:0/842181568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa08006e200 msgr2=0x7fa0801a7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.851+0000 7fa085cc7640 1 --2- 172.21.15.38:0/842181568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa08006e200 0x7fa0801a7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.851+0000 7fa085cc7640 1 -- 172.21.15.38:0/842181568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08010cc30 con 0x7fa08006f490 2024-05-21T15:42:29.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.852+0000 7fa085cc7640 1 --2- 172.21.15.38:0/842181568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa08006f490 0x7fa0801a8510 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa07000e9f0 tx=0x7fa07000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.852+0000 7fa0777fe640 1 -- 172.21.15.38:0/842181568 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07000ccc0 con 0x7fa08006f490 2024-05-21T15:42:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.852+0000 7fa08872c640 1 -- 172.21.15.38:0/842181568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08010cf20 con 0x7fa08006f490 2024-05-21T15:42:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.852+0000 7fa08872c640 1 -- 172.21.15.38:0/842181568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08010d4c0 con 0x7fa08006f490 2024-05-21T15:42:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.852+0000 7fa0777fe640 1 -- 172.21.15.38:0/842181568 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa070010910 con 0x7fa08006f490 2024-05-21T15:42:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.854+0000 7fa0777fe640 1 -- 172.21.15.38:0/842181568 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070026060 con 0x7fa08006f490 2024-05-21T15:42:29.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.856+0000 7fa08f5d0740 1 Processor -- start 2024-05-21T15:42:29.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.857+0000 7fa08f5d0740 1 -- start start 2024-05-21T15:42:29.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.857+0000 7fa08f5d0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa82390 0x55d8afe21b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.857+0000 7fa08f5d0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa594d0 0x55d8afe22080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.857+0000 7fa08f5d0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa616b0 0x55d8afe28f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.857+0000 7fa08f5d0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d8afd86c40 con 0x55d8afa616b0 2024-05-21T15:42:29.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.857+0000 7fa08f5d0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d8afd2e580 con 0x55d8afa82390 2024-05-21T15:42:29.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.857+0000 7fa08f5d0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d8afd82c00 con 0x55d8afa594d0 2024-05-21T15:42:29.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.857+0000 7fa0864c8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa594d0 0x55d8afe22080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.858+0000 7fa0864c8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa594d0 0x55d8afe22080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39754/0 (socket says 172.21.15.38:39754) 2024-05-21T15:42:29.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.858+0000 7fa0864c8640 1 -- 172.21.15.38:0/2416396885 learned_addr learned my addr 172.21.15.38:0/2416396885 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:29.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.858+0000 7fa0777fe640 1 -- 172.21.15.38:0/842181568 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa070018880 con 0x7fa08006f490 2024-05-21T15:42:29.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.858+0000 7fa085cc7640 1 --2- 172.21.15.38:0/2416396885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa616b0 0x55d8afe28f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.858+0000 7fa0864c8640 1 -- 172.21.15.38:0/2416396885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa82390 msgr2=0x55d8afe21b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:29.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.858+0000 7fa0864c8640 1 --2- 172.21.15.38:0/2416396885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa82390 0x55d8afe21b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.858+0000 7fa0864c8640 1 -- 172.21.15.38:0/2416396885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa616b0 msgr2=0x55d8afe28f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:29.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.858+0000 7fa0864c8640 1 --2- 172.21.15.38:0/2416396885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa616b0 0x55d8afe28f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.858+0000 7fa0864c8640 1 -- 172.21.15.38:0/2416396885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8afa9a3f0 con 0x55d8afa594d0 2024-05-21T15:42:29.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.858+0000 7fa085cc7640 1 --2- 172.21.15.38:0/2416396885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa616b0 0x55d8afe28f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:42:29.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.859+0000 7fa0864c8640 1 --2- 172.21.15.38:0/2416396885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa594d0 0x55d8afe22080 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa0780113e0 tx=0x7fa0780118a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40fa7242f0e0239 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:29.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.859+0000 7fa074ff9640 1 -- 172.21.15.38:0/2416396885 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078012770 con 0x55d8afa594d0 2024-05-21T15:42:29.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.859+0000 7fa074ff9640 1 -- 172.21.15.38:0/2416396885 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa078012df0 con 0x55d8afa594d0 2024-05-21T15:42:29.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.859+0000 7fa074ff9640 1 -- 172.21.15.38:0/2416396885 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078012770 con 0x55d8afa594d0 2024-05-21T15:42:29.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.860+0000 7fa0777fe640 1 --2- 172.21.15.38:0/842181568 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa06c0a2c60 0x7fa06c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.860+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2416396885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa594d0 msgr2=0x55d8afe22080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.860+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/2416396885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa594d0 0x55d8afe22080 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa0780113e0 tx=0x7fa0780118a0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.860+0000 7fa086cc9640 1 --2- 172.21.15.38:0/842181568 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa06c0a2c60 0x7fa06c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.860+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2416396885 shutdown_connections 2024-05-21T15:42:29.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.860+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/2416396885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa616b0 0x55d8afe28f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.860+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/2416396885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa594d0 0x55d8afe22080 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.860+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/2416396885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa82390 0x55d8afe21b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.860+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2416396885 >> 172.21.15.38:0/2416396885 conn(0x55d8afa764e0 msgr2=0x55d8afe1d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:29.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.860+0000 7fa0777fe640 1 -- 172.21.15.38:0/842181568 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa0700cf570 con 0x7fa08006f490 2024-05-21T15:42:29.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.861+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2416396885 shutdown_connections 2024-05-21T15:42:29.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.861+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2416396885 wait complete. 2024-05-21T15:42:29.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.861+0000 7fa086cc9640 1 --2- 172.21.15.38:0/842181568 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa06c0a2c60 0x7fa06c0a5170 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa07c02aa30 tx=0x7fa07c02a940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:29.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.861+0000 7fa08f5d0740 1 -- start start 2024-05-21T15:42:29.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.861+0000 7fa08f5d0740 1 Processor -- start 2024-05-21T15:42:29.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.861+0000 7fa08f5d0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa594d0 0x55d8afe2b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.861+0000 7fa08f5d0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa616b0 0x55d8afe2bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa08f5d0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa82390 0x55d8afe32d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa08f5d0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d8afe299d0 con 0x55d8afa82390 2024-05-21T15:42:29.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa08f5d0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d8afd41c40 con 0x55d8afa594d0 2024-05-21T15:42:29.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa08f5d0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d8afe29b50 con 0x55d8afa616b0 2024-05-21T15:42:29.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa085cc7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa616b0 0x55d8afe2bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa08f5d0740 10 client.0 _ll_register_callbacks cb 0x55d8afa99be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:29.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa085cc7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa616b0 0x55d8afe2bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39766/0 (socket says 172.21.15.38:39766) 2024-05-21T15:42:29.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa085cc7640 1 -- 172.21.15.38:0/1234123829 learned_addr learned my addr 172.21.15.38:0/1234123829 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:29.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa0864c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa594d0 0x55d8afe2b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa086cc9640 1 --2- 172.21.15.38:0/1234123829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa82390 0x55d8afe32d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa085cc7640 1 -- 172.21.15.38:0/1234123829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa594d0 msgr2=0x55d8afe2b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:29.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa085cc7640 1 --2- 172.21.15.38:0/1234123829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa594d0 0x55d8afe2b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa085cc7640 1 -- 172.21.15.38:0/1234123829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa82390 msgr2=0x55d8afe32d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:29.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa085cc7640 1 --2- 172.21.15.38:0/1234123829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa82390 0x55d8afe32d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa085cc7640 1 -- 172.21.15.38:0/1234123829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8afb647d0 con 0x55d8afa616b0 2024-05-21T15:42:29.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa085cc7640 1 --2- 172.21.15.38:0/1234123829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa616b0 0x55d8afe2bdd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa070098210 tx=0x7fa0700a00d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc0ed91c156777e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:29.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.862+0000 7fa069ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:42:29.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.863+0000 7fa069ffb640 1 -- 172.21.15.38:0/1234123829 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0700cf310 con 0x55d8afa616b0 2024-05-21T15:42:29.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.863+0000 7fa069ffb640 1 -- 172.21.15.38:0/1234123829 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa070018b70 con 0x55d8afa616b0 2024-05-21T15:42:29.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.863+0000 7fa08f5d0740 1 -- 172.21.15.38:0/1234123829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa078012df0 con 0x55d8afa616b0 2024-05-21T15:42:29.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.863+0000 7fa069ffb640 1 -- 172.21.15.38:0/1234123829 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa070018d70 con 0x55d8afa616b0 2024-05-21T15:42:29.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.863+0000 7fa08f5d0740 10 client.27376 fetch_fsmap learned FSMap version 40 2024-05-21T15:42:29.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.863+0000 7fa08f5d0740 1 -- 172.21.15.38:0/1234123829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d8afe36000 con 0x55d8afa616b0 2024-05-21T15:42:29.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.863+0000 7fa069ffb640 1 -- 172.21.15.38:0/1234123829 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa07009b250 con 0x55d8afa616b0 2024-05-21T15:42:29.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.863+0000 7fa069ffb640 1 -- 172.21.15.38:0/1234123829 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fa07009acc0 con 0x55d8afa616b0 2024-05-21T15:42:29.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.863+0000 7fa08f5d0740 10 client.27376 fetch_fsmap finished waiting for FSMap version 40 2024-05-21T15:42:29.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.864+0000 7fa08f5d0740 10 client.27376 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:42:29.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.864+0000 7fa08f5d0740 20 client.27376 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:29.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.864+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/1234123829 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55d8afa40f60 0x55d8afe39600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.864+0000 7fa08f5d0740 4 client.27376 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:42:29.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.864+0000 7fa08f5d0740 1 -- 172.21.15.38:0/1234123829 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa078012df0 con 0x55d8afa40f60 2024-05-21T15:42:29.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.865+0000 7fa0864c8640 1 --2- 172.21.15.38:0/1234123829 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55d8afa40f60 0x55d8afe39600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.867+0000 7fa0864c8640 1 --2- 172.21.15.38:0/1234123829 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55d8afa40f60 0x55d8afe39600 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea4e4db9cc6c2698 server_cookie=f30ffc9a70e5df9e in_seq=0 out_seq=0 2024-05-21T15:42:29.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.867+0000 7fa069ffb640 10 client.27376 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:42:29.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.868+0000 7fa069ffb640 1 -- 172.21.15.38:0/1234123829 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fa078012df0 con 0x55d8afa40f60 2024-05-21T15:42:29.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.868+0000 7fa069ffb640 10 client.27376 handle_command_reply: tid=0 2024-05-21T15:42:29.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.868+0000 7fa08f5d0740 1 client.27376 shutdown 2024-05-21T15:42:29.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.868+0000 7fa08f5d0740 1 -- 172.21.15.38:0/1234123829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa616b0 msgr2=0x55d8afe2bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.868+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/1234123829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa616b0 0x55d8afe2bdd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa070098210 tx=0x7fa0700a00d0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.868+0000 7fa08f5d0740 1 -- 172.21.15.38:0/1234123829 shutdown_connections 2024-05-21T15:42:29.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.868+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/1234123829 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55d8afa40f60 0x55d8afe39600 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.868+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/1234123829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa82390 0x55d8afe32d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.868+0000 7fa069ffb640 0 client.27376 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:42:29.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.868+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/1234123829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa616b0 0x55d8afe2bdd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.868+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/1234123829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa594d0 0x55d8afe2b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.868+0000 7fa08f5d0740 1 -- 172.21.15.38:0/1234123829 >> 172.21.15.38:0/1234123829 conn(0x55d8afa764e0 msgr2=0x55d8afe1c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:29.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.868+0000 7fa08f5d0740 1 -- 172.21.15.38:0/1234123829 shutdown_connections 2024-05-21T15:42:29.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.869+0000 7fa08f5d0740 1 -- 172.21.15.38:0/1234123829 wait complete. 2024-05-21T15:42:29.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.869+0000 7fa08f5d0740 20 client.27376 trim_cache size 0 max 16384 2024-05-21T15:42:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.898+0000 7fa08f5d0740 1 Processor -- start 2024-05-21T15:42:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.898+0000 7fa08f5d0740 1 -- start start 2024-05-21T15:42:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.898+0000 7fa08f5d0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa40f60 0x55d8afe20af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.898+0000 7fa08f5d0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa594d0 0x55d8afe3ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.898+0000 7fa08f5d0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa616b0 0x55d8afe3e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.899+0000 7fa08f5d0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa06400e5b0 con 0x55d8afa40f60 2024-05-21T15:42:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.899+0000 7fa08f5d0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0640101b0 con 0x55d8afa616b0 2024-05-21T15:42:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.899+0000 7fa08f5d0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa06400ece0 con 0x55d8afa594d0 2024-05-21T15:42:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.899+0000 7fa0864c8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa40f60 0x55d8afe20af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.899+0000 7fa0864c8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa40f60 0x55d8afe20af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59912/0 (socket says 172.21.15.38:59912) 2024-05-21T15:42:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.899+0000 7fa0864c8640 1 -- 172.21.15.38:0/275854053 learned_addr learned my addr 172.21.15.38:0/275854053 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.899+0000 7fa085cc7640 1 --2- 172.21.15.38:0/275854053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa594d0 0x55d8afe3ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.899+0000 7fa086cc9640 1 --2- 172.21.15.38:0/275854053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa616b0 0x55d8afe3e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.899+0000 7fa085cc7640 1 -- 172.21.15.38:0/275854053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa616b0 msgr2=0x55d8afe3e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:29.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.899+0000 7fa085cc7640 1 --2- 172.21.15.38:0/275854053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa616b0 0x55d8afe3e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.899+0000 7fa085cc7640 1 -- 172.21.15.38:0/275854053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa40f60 msgr2=0x55d8afe20af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:29.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.899+0000 7fa085cc7640 1 --2- 172.21.15.38:0/275854053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa40f60 0x55d8afe20af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.899+0000 7fa085cc7640 1 -- 172.21.15.38:0/275854053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8afe18d20 con 0x55d8afa594d0 2024-05-21T15:42:29.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.899+0000 7fa0864c8640 1 --2- 172.21.15.38:0/275854053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa40f60 0x55d8afe20af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:42:29.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.900+0000 7fa085cc7640 1 --2- 172.21.15.38:0/275854053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa594d0 0x55d8afe3ddf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa070099b20 tx=0x7fa070010f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8abb1aa7d8bc0669 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:29.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.900+0000 7fa06bfff640 1 -- 172.21.15.38:0/275854053 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07009a3c0 con 0x55d8afa594d0 2024-05-21T15:42:29.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.901+0000 7fa06bfff640 1 -- 172.21.15.38:0/275854053 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa07009ca50 con 0x55d8afa594d0 2024-05-21T15:42:29.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.901+0000 7fa06bfff640 1 -- 172.21.15.38:0/275854053 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07009be00 con 0x55d8afa594d0 2024-05-21T15:42:29.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.902+0000 7fa08f5d0740 1 -- 172.21.15.38:0/275854053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa594d0 msgr2=0x55d8afe3ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.902+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/275854053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa594d0 0x55d8afe3ddf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa070099b20 tx=0x7fa070010f10 comp rx=0 tx=0).stop 2024-05-21T15:42:29.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.902+0000 7fa08f5d0740 1 -- 172.21.15.38:0/275854053 shutdown_connections 2024-05-21T15:42:29.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.902+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/275854053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa616b0 0x55d8afe3e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.902+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/275854053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa594d0 0x55d8afe3ddf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.902+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/275854053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa40f60 0x55d8afe20af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.902+0000 7fa08f5d0740 1 -- 172.21.15.38:0/275854053 >> 172.21.15.38:0/275854053 conn(0x55d8afa764e0 msgr2=0x55d8afe203e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:29.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.902+0000 7fa08f5d0740 1 -- 172.21.15.38:0/275854053 shutdown_connections 2024-05-21T15:42:29.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.902+0000 7fa08f5d0740 1 -- 172.21.15.38:0/275854053 wait complete. 2024-05-21T15:42:29.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.902+0000 7fa08f5d0740 1 -- start start 2024-05-21T15:42:29.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.903+0000 7fa08f5d0740 1 Processor -- start 2024-05-21T15:42:29.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.903+0000 7fa08f5d0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa40f60 0x55d8afe2f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.903+0000 7fa08f5d0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa594d0 0x55d8afe2fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.903+0000 7fa08f5d0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa616b0 0x55d8afe30ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.903+0000 7fa085cc7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa594d0 0x55d8afe2fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.903+0000 7fa085cc7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa594d0 0x55d8afe2fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:40460/0 (socket says 172.21.15.38:40460) 2024-05-21T15:42:29.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.904+0000 7fa085cc7640 1 -- 172.21.15.38:0/2184841845 learned_addr learned my addr 172.21.15.38:0/2184841845 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:29.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.904+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2184841845 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa06400fae0 con 0x55d8afa616b0 2024-05-21T15:42:29.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.904+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2184841845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa06400f3e0 con 0x55d8afa594d0 2024-05-21T15:42:29.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.904+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2184841845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa06400c590 con 0x55d8afa40f60 2024-05-21T15:42:29.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.904+0000 7fa0864c8640 1 --2- 172.21.15.38:0/2184841845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa40f60 0x55d8afe2f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.904+0000 7fa08f5d0740 10 client.0 _ll_register_callbacks cb 0x7fa07009b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:29.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.904+0000 7fa086cc9640 1 --2- 172.21.15.38:0/2184841845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa616b0 0x55d8afe30ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.904+0000 7fa0864c8640 1 -- 172.21.15.38:0/2184841845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa594d0 msgr2=0x55d8afe2fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:29.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.904+0000 7fa0864c8640 1 --2- 172.21.15.38:0/2184841845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa594d0 0x55d8afe2fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.904+0000 7fa0864c8640 1 -- 172.21.15.38:0/2184841845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa616b0 msgr2=0x55d8afe30ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:29.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.904+0000 7fa0864c8640 1 --2- 172.21.15.38:0/2184841845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa616b0 0x55d8afe30ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.904+0000 7fa0864c8640 1 -- 172.21.15.38:0/2184841845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8afe31070 con 0x55d8afa40f60 2024-05-21T15:42:29.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.904+0000 7fa0864c8640 1 --2- 172.21.15.38:0/2184841845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa40f60 0x55d8afe2f6e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa078009740 tx=0x7fa078011fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=483b2d632956184e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:29.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.904+0000 7fa069ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:42:29.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.905+0000 7fa069ffb640 1 -- 172.21.15.38:0/2184841845 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078012770 con 0x55d8afa40f60 2024-05-21T15:42:29.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.905+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2184841845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d8afd94370 con 0x55d8afa40f60 2024-05-21T15:42:29.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.905+0000 7fa069ffb640 1 -- 172.21.15.38:0/2184841845 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fa078012570 con 0x55d8afa40f60 2024-05-21T15:42:29.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.905+0000 7fa069ffb640 1 -- 172.21.15.38:0/2184841845 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa07803a050 con 0x55d8afa40f60 2024-05-21T15:42:29.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.905+0000 7fa08f5d0740 10 client.27382 fetch_fsmap learned FSMap version 40 2024-05-21T15:42:29.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.905+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2184841845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d8afe32360 con 0x55d8afa40f60 2024-05-21T15:42:29.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.905+0000 7fa069ffb640 1 -- 172.21.15.38:0/2184841845 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d8afd94370 con 0x55d8afa40f60 2024-05-21T15:42:29.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.905+0000 7fa069ffb640 1 -- 172.21.15.38:0/2184841845 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fa07802bce0 con 0x55d8afa40f60 2024-05-21T15:42:29.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.905+0000 7fa08f5d0740 10 client.27382 fetch_fsmap finished waiting for FSMap version 40 2024-05-21T15:42:29.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.905+0000 7fa08f5d0740 10 client.27382 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T15:42:29.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.906+0000 7fa08f5d0740 20 client.27382 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.906+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/2184841845 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55d8afa82390 0x55d8afea3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.906+0000 7fa08f5d0740 4 client.27382 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:42:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.906+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2184841845 --> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8afd8e6c0 con 0x55d8afa82390 2024-05-21T15:42:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.906+0000 7fa085cc7640 1 --2- 172.21.15.38:0/2184841845 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55d8afa82390 0x55d8afea3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.906+0000 7fa069ffb640 10 client.27382 ms_handle_connect on v2:172.21.15.45:6834/3817948392 2024-05-21T15:42:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa085cc7640 1 --2- 172.21.15.38:0/2184841845 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55d8afa82390 0x55d8afea3b60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b7483cb0f1f2e75b server_cookie=f7d39004e5de00ca in_seq=0 out_seq=0 2024-05-21T15:42:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa069ffb640 1 -- 172.21.15.38:0/2184841845 <== mds.4 v2:172.21.15.45:6834/3817948392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8afd8e6c0 con 0x55d8afa82390 2024-05-21T15:42:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa069ffb640 10 client.27382 handle_command_reply: tid=0 2024-05-21T15:42:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa08f5d0740 1 client.27382 shutdown 2024-05-21T15:42:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2184841845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa40f60 msgr2=0x55d8afe2f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/2184841845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa40f60 0x55d8afe2f6e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa078009740 tx=0x7fa078011fb0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2184841845 shutdown_connections 2024-05-21T15:42:29.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/2184841845 >> [v2:172.21.15.45:6834/3817948392,v1:172.21.15.45:6835/3817948392] conn(0x55d8afa82390 0x55d8afea3b60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa069ffb640 0 client.27382 ms_handle_reset on v2:172.21.15.45:6834/3817948392 2024-05-21T15:42:29.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/2184841845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d8afa616b0 0x55d8afe30ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/2184841845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8afa594d0 0x55d8afe2fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa08f5d0740 1 --2- 172.21.15.38:0/2184841845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8afa40f60 0x55d8afe2f6e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2184841845 >> 172.21.15.38:0/2184841845 conn(0x55d8afa764e0 msgr2=0x55d8afe29f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:29.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2184841845 shutdown_connections 2024-05-21T15:42:29.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.907+0000 7fa08f5d0740 1 -- 172.21.15.38:0/2184841845 wait complete. 2024-05-21T15:42:29.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.908+0000 7fa08f5d0740 20 client.27382 trim_cache size 0 max 16384 2024-05-21T15:42:29.911 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:42:29.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.909+0000 7fa08872c640 1 -- 172.21.15.38:0/842181568 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa06c0a2c60 msgr2=0x7fa06c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.909+0000 7fa08872c640 1 --2- 172.21.15.38:0/842181568 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa06c0a2c60 0x7fa06c0a5170 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa07c02aa30 tx=0x7fa07c02a940 comp rx=0 tx=0).stop 2024-05-21T15:42:29.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.909+0000 7fa08872c640 1 -- 172.21.15.38:0/842181568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa08006f490 msgr2=0x7fa0801a8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:29.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.909+0000 7fa08872c640 1 --2- 172.21.15.38:0/842181568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa08006f490 0x7fa0801a8510 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa07000e9f0 tx=0x7fa07000eeb0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.909+0000 7fa08872c640 1 -- 172.21.15.38:0/842181568 shutdown_connections 2024-05-21T15:42:29.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.909+0000 7fa08872c640 1 --2- 172.21.15.38:0/842181568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa080070190 0x7fa0801af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.909+0000 7fa08872c640 1 --2- 172.21.15.38:0/842181568 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fa06c0a2c60 0x7fa06c0a5170 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.909+0000 7fa08872c640 1 --2- 172.21.15.38:0/842181568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa08006f490 0x7fa0801a8510 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.909+0000 7fa08872c640 1 --2- 172.21.15.38:0/842181568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa08006e200 0x7fa0801a7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:29.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.909+0000 7fa08872c640 1 -- 172.21.15.38:0/842181568 >> 172.21.15.38:0/842181568 conn(0x7fa08006d3f0 msgr2=0x7fa080117320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:29.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.909+0000 7fa08872c640 1 -- 172.21.15.38:0/842181568 shutdown_connections 2024-05-21T15:42:29.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:29.909+0000 7fa08872c640 1 -- 172.21.15.38:0/842181568 wait complete. 2024-05-21T15:42:29.922 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:42:29.956 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:29 smithi038 ceph-mon[31995]: pgmap v593: 97 pgs: 97 active+clean; 9.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 3.0 MiB/s wr, 820 op/s 2024-05-21T15:42:29.956 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:29 smithi038 ceph-mon[31995]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:29.956 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:29 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2637008911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:42:30.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.114+0000 7f553e7fc640 1 -- 172.21.15.38:0/3119877863 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55380303c0 con 0x7f5540107040 2024-05-21T15:42:30.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.115+0000 7f5545bcc640 1 -- 172.21.15.38:0/3119877863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5540107040 msgr2=0x7f5540109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.115+0000 7f5545bcc640 1 --2- 172.21.15.38:0/3119877863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5540107040 0x7f5540109460 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5538009ca0 tx=0x7f553802a590 comp rx=0 tx=0).stop 2024-05-21T15:42:30.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.115+0000 7f5545bcc640 1 -- 172.21.15.38:0/3119877863 shutdown_connections 2024-05-21T15:42:30.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.115+0000 7f5545bcc640 1 --2- 172.21.15.38:0/3119877863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5540107040 0x7f5540109460 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.115+0000 7f5545bcc640 1 --2- 172.21.15.38:0/3119877863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5540102340 0x7f5540102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.115+0000 7f5545bcc640 1 --2- 172.21.15.38:0/3119877863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5540101100 0x7f5540101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.115+0000 7f5545bcc640 1 -- 172.21.15.38:0/3119877863 >> 172.21.15.38:0/3119877863 conn(0x7f55400fc2f0 msgr2=0x7f55400fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.116+0000 7f5545bcc640 1 -- 172.21.15.38:0/3119877863 shutdown_connections 2024-05-21T15:42:30.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.116+0000 7f5545bcc640 1 -- 172.21.15.38:0/3119877863 wait complete. 2024-05-21T15:42:30.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.117+0000 7f5545bcc640 1 Processor -- start 2024-05-21T15:42:30.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.117+0000 7f5545bcc640 1 -- start start 2024-05-21T15:42:30.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.117+0000 7f5545bcc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5540101100 0x7f5540071560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.117+0000 7f5545bcc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5540102340 0x7f5540071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.117+0000 7f5545bcc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5540107040 0x7f55400729b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.117+0000 7f5545bcc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f554006b6f0 con 0x7f5540102340 2024-05-21T15:42:30.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.117+0000 7f5545bcc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f554006b570 con 0x7f5540107040 2024-05-21T15:42:30.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.117+0000 7f5545bcc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f554006b870 con 0x7f5540101100 2024-05-21T15:42:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.118+0000 7f553effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5540102340 0x7f5540071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.118+0000 7f553effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5540102340 0x7f5540071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59934/0 (socket says 172.21.15.38:59934) 2024-05-21T15:42:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.118+0000 7f553effd640 1 -- 172.21.15.38:0/393881462 learned_addr learned my addr 172.21.15.38:0/393881462 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.118+0000 7f553ffff640 1 --2- 172.21.15.38:0/393881462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5540107040 0x7f55400729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.118+0000 7f553f7fe640 1 --2- 172.21.15.38:0/393881462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5540101100 0x7f5540071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.118+0000 7f553effd640 1 -- 172.21.15.38:0/393881462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5540101100 msgr2=0x7f5540071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.118+0000 7f553effd640 1 --2- 172.21.15.38:0/393881462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5540101100 0x7f5540071560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.118+0000 7f553effd640 1 -- 172.21.15.38:0/393881462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5540107040 msgr2=0x7f55400729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.118+0000 7f553effd640 1 --2- 172.21.15.38:0/393881462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5540107040 0x7f55400729b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.118+0000 7f553effd640 1 -- 172.21.15.38:0/393881462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5540073020 con 0x7f5540102340 2024-05-21T15:42:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.118+0000 7f553effd640 1 --2- 172.21.15.38:0/393881462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5540102340 0x7f5540071aa0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f553400e9f0 tx=0x7f553400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.119+0000 7f553cff9640 1 -- 172.21.15.38:0/393881462 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553400cd80 con 0x7f5540102340 2024-05-21T15:42:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.119+0000 7f553cff9640 1 -- 172.21.15.38:0/393881462 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5534010910 con 0x7f5540102340 2024-05-21T15:42:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.119+0000 7f553cff9640 1 -- 172.21.15.38:0/393881462 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534018990 con 0x7f5540102340 2024-05-21T15:42:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.119+0000 7f5545bcc640 1 -- 172.21.15.38:0/393881462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5540073310 con 0x7f5540102340 2024-05-21T15:42:30.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.119+0000 7f5545bcc640 1 -- 172.21.15.38:0/393881462 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5540073810 con 0x7f5540102340 2024-05-21T15:42:30.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.123+0000 7f554ca70740 1 Processor -- start 2024-05-21T15:42:30.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.123+0000 7f554ca70740 1 -- start start 2024-05-21T15:42:30.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.123+0000 7f554ca70740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 0x559a335a9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.123+0000 7f554ca70740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e14d0 0x559a335aa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.123+0000 7f554ca70740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e96b0 0x559a335b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.123+0000 7f554ca70740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559a3350ec40 con 0x559a331e96b0 2024-05-21T15:42:30.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.123+0000 7f554ca70740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a334b6580 con 0x559a3320a390 2024-05-21T15:42:30.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.123+0000 7f554ca70740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559a3350ac00 con 0x559a331e14d0 2024-05-21T15:42:30.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.124+0000 7f553cff9640 1 -- 172.21.15.38:0/393881462 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5534018b30 con 0x7f5540102340 2024-05-21T15:42:30.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.124+0000 7f553f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e14d0 0x559a335aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.124+0000 7f553effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e96b0 0x559a335b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.124+0000 7f553ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 0x559a335a9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.124+0000 7f553f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e14d0 0x559a335aa080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39798/0 (socket says 172.21.15.38:39798) 2024-05-21T15:42:30.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.124+0000 7f553f7fe640 1 -- 172.21.15.38:0/2584943599 learned_addr learned my addr 172.21.15.38:0/2584943599 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.124+0000 7f553ffff640 1 -- 172.21.15.38:0/2584943599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e14d0 msgr2=0x559a335aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.124+0000 7f553ffff640 1 --2- 172.21.15.38:0/2584943599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e14d0 0x559a335aa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.125+0000 7f553ffff640 1 -- 172.21.15.38:0/2584943599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e96b0 msgr2=0x559a335b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.125+0000 7f553ffff640 1 --2- 172.21.15.38:0/2584943599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e96b0 0x559a335b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.125+0000 7f553ffff640 1 -- 172.21.15.38:0/2584943599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a332223f0 con 0x559a3320a390 2024-05-21T15:42:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.126+0000 7f553cff9640 1 --2- 172.21.15.38:0/393881462 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f55280b3c20 0x7f55280b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.126+0000 7f553f7fe640 1 --2- 172.21.15.38:0/393881462 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f55280b3c20 0x7f55280b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.126+0000 7f553cff9640 1 -- 172.21.15.38:0/393881462 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f55340cf3f0 con 0x7f5540102340 2024-05-21T15:42:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.126+0000 7f553ffff640 1 --2- 172.21.15.38:0/2584943599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 0x559a335a9b00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5538006010 tx=0x7f5538005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acaecb355c85dc49 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.127+0000 7f5525ffb640 1 -- 172.21.15.38:0/2584943599 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538003970 con 0x559a3320a390 2024-05-21T15:42:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.127+0000 7f5525ffb640 1 -- 172.21.15.38:0/2584943599 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5538041070 con 0x559a3320a390 2024-05-21T15:42:30.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.127+0000 7f5525ffb640 1 -- 172.21.15.38:0/2584943599 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538004320 con 0x559a3320a390 2024-05-21T15:42:30.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.129+0000 7f554ca70740 1 -- 172.21.15.38:0/2584943599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 msgr2=0x559a335a9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.129+0000 7f554ca70740 1 --2- 172.21.15.38:0/2584943599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 0x559a335a9b00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5538006010 tx=0x7f5538005c80 comp rx=0 tx=0).stop 2024-05-21T15:42:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.129+0000 7f553f7fe640 1 --2- 172.21.15.38:0/393881462 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f55280b3c20 0x7f55280b6130 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5530014fd0 tx=0x7f553000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.129+0000 7f554ca70740 1 -- 172.21.15.38:0/2584943599 shutdown_connections 2024-05-21T15:42:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.129+0000 7f554ca70740 1 --2- 172.21.15.38:0/2584943599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e96b0 0x559a335b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.129+0000 7f554ca70740 1 --2- 172.21.15.38:0/2584943599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e14d0 0x559a335aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.129+0000 7f554ca70740 1 --2- 172.21.15.38:0/2584943599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 0x559a335a9b00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.129+0000 7f554ca70740 1 -- 172.21.15.38:0/2584943599 >> 172.21.15.38:0/2584943599 conn(0x559a331fe4e0 msgr2=0x559a335a5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.129+0000 7f554ca70740 1 -- 172.21.15.38:0/2584943599 shutdown_connections 2024-05-21T15:42:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.129+0000 7f554ca70740 1 -- 172.21.15.38:0/2584943599 wait complete. 2024-05-21T15:42:30.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.130+0000 7f554ca70740 1 -- start start 2024-05-21T15:42:30.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.130+0000 7f554ca70740 1 Processor -- start 2024-05-21T15:42:30.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.131+0000 7f554ca70740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e14d0 0x559a335c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.131+0000 7f554ca70740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e96b0 0x559a335c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.131+0000 7f554ca70740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 0x559a335cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.131+0000 7f554ca70740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559a335c2980 con 0x559a331e96b0 2024-05-21T15:42:30.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.131+0000 7f554ca70740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a334c9c40 con 0x559a3320a390 2024-05-21T15:42:30.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.131+0000 7f553f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e14d0 0x559a335c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.131+0000 7f553f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e14d0 0x559a335c4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39808/0 (socket says 172.21.15.38:39808) 2024-05-21T15:42:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.131+0000 7f553f7fe640 1 -- 172.21.15.38:0/3579265179 learned_addr learned my addr 172.21.15.38:0/3579265179 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.132+0000 7f553effd640 1 --2- 172.21.15.38:0/3579265179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e96b0 0x559a335c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.132+0000 7f553ffff640 1 --2- 172.21.15.38:0/3579265179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 0x559a335cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.132+0000 7f554ca70740 1 -- 172.21.15.38:0/3579265179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559a335c2b00 con 0x559a331e14d0 2024-05-21T15:42:30.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.132+0000 7f554ca70740 10 client.0 _ll_register_callbacks cb 0x559a33221be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:30.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.132+0000 7f553f7fe640 1 -- 172.21.15.38:0/3579265179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 msgr2=0x559a335cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.132+0000 7f553f7fe640 1 --2- 172.21.15.38:0/3579265179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 0x559a335cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.132+0000 7f553f7fe640 1 -- 172.21.15.38:0/3579265179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e96b0 msgr2=0x559a335c4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.132+0000 7f553f7fe640 1 --2- 172.21.15.38:0/3579265179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e96b0 0x559a335c4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.132+0000 7f553f7fe640 1 -- 172.21.15.38:0/3579265179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a332ec7d0 con 0x559a331e14d0 2024-05-21T15:42:30.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.132+0000 7f553effd640 1 --2- 172.21.15.38:0/3579265179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e96b0 0x559a335c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:42:30.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.132+0000 7f554ca70740 1 -- 172.21.15.38:0/3579265179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5538041070 con 0x559a331e14d0 2024-05-21T15:42:30.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.132+0000 7f553f7fe640 1 --2- 172.21.15.38:0/3579265179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e14d0 0x559a335c4a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5530021640 tx=0x7f5530016040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=182924d889bfca84 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.132+0000 7f551effd640 10 client.27394 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:42:30.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.133+0000 7f551effd640 1 -- 172.21.15.38:0/3579265179 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530022990 con 0x559a331e14d0 2024-05-21T15:42:30.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.133+0000 7f551effd640 1 -- 172.21.15.38:0/3579265179 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f5530022b30 con 0x559a331e14d0 2024-05-21T15:42:30.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.134+0000 7f551effd640 1 -- 172.21.15.38:0/3579265179 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5530022d90 con 0x559a331e14d0 2024-05-21T15:42:30.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.134+0000 7f554ca70740 10 client.27394 fetch_fsmap learned FSMap version 40 2024-05-21T15:42:30.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.134+0000 7f554ca70740 1 -- 172.21.15.38:0/3579265179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=40}) -- 0x559a335cf120 con 0x559a331e14d0 2024-05-21T15:42:30.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.134+0000 7f551effd640 1 -- 172.21.15.38:0/3579265179 <== mon.1 v2:172.21.15.53:3300/0 4 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f553001e030 con 0x559a331e14d0 2024-05-21T15:42:30.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.134+0000 7f554ca70740 10 client.27394 fetch_fsmap finished waiting for FSMap version 40 2024-05-21T15:42:30.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.134+0000 7f554ca70740 10 client.27394 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:42:30.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.134+0000 7f554ca70740 20 client.27394 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:30.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.134+0000 7f554ca70740 1 --2- 172.21.15.38:0/3579265179 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x559a331c8f60 0x559a335d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.134+0000 7f554ca70740 4 client.27394 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:42:30.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.134+0000 7f554ca70740 1 -- 172.21.15.38:0/3579265179 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a3351c370 con 0x559a331c8f60 2024-05-21T15:42:30.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.134+0000 7f553effd640 1 --2- 172.21.15.38:0/3579265179 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x559a331c8f60 0x559a335d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.136+0000 7f551effd640 10 client.27394 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:42:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.136+0000 7f553effd640 1 --2- 172.21.15.38:0/3579265179 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x559a331c8f60 0x559a335d2750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bb86b397a2d97f7c server_cookie=53cd85ad64d1c936 in_seq=0 out_seq=0 2024-05-21T15:42:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.137+0000 7f551effd640 1 -- 172.21.15.38:0/3579265179 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559a3351c370 con 0x559a331c8f60 2024-05-21T15:42:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.137+0000 7f551effd640 10 client.27394 handle_command_reply: tid=0 2024-05-21T15:42:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.137+0000 7f554ca70740 1 client.27394 shutdown 2024-05-21T15:42:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.139+0000 7f554ca70740 1 -- 172.21.15.38:0/3579265179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e14d0 msgr2=0x559a335c4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.139+0000 7f554ca70740 1 --2- 172.21.15.38:0/3579265179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e14d0 0x559a335c4a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5530021640 tx=0x7f5530016040 comp rx=0 tx=0).stop 2024-05-21T15:42:30.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.140+0000 7f554ca70740 1 -- 172.21.15.38:0/3579265179 shutdown_connections 2024-05-21T15:42:30.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.140+0000 7f554ca70740 1 --2- 172.21.15.38:0/3579265179 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x559a331c8f60 0x559a335d2750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.140+0000 7f554ca70740 1 --2- 172.21.15.38:0/3579265179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 0x559a335cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.140+0000 7f551effd640 0 client.27394 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:42:30.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.140+0000 7f554ca70740 1 --2- 172.21.15.38:0/3579265179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e96b0 0x559a335c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.140+0000 7f554ca70740 1 --2- 172.21.15.38:0/3579265179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e14d0 0x559a335c4a10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.140+0000 7f554ca70740 1 -- 172.21.15.38:0/3579265179 >> 172.21.15.38:0/3579265179 conn(0x559a331fe4e0 msgr2=0x559a335a4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.141+0000 7f554ca70740 1 -- 172.21.15.38:0/3579265179 shutdown_connections 2024-05-21T15:42:30.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.141+0000 7f554ca70740 1 -- 172.21.15.38:0/3579265179 wait complete. 2024-05-21T15:42:30.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.141+0000 7f554ca70740 20 client.27394 trim_cache size 0 max 16384 2024-05-21T15:42:30.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.166+0000 7f554ca70740 1 Processor -- start 2024-05-21T15:42:30.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.166+0000 7f554ca70740 1 -- start start 2024-05-21T15:42:30.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.166+0000 7f554ca70740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a331e14d0 0x559a335a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.167+0000 7f554ca70740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e96b0 0x559a335cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.167+0000 7f554ca70740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3320a390 0x559a335a5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.167+0000 7f554ca70740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f554019fa00 con 0x559a3320a390 2024-05-21T15:42:30.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.167+0000 7f554ca70740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f55401a1600 con 0x559a331e14d0 2024-05-21T15:42:30.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.167+0000 7f553f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a331e14d0 0x559a335a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.167+0000 7f553f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a331e14d0 0x559a335a4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:40494/0 (socket says 172.21.15.38:40494) 2024-05-21T15:42:30.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.167+0000 7f553f7fe640 1 -- 172.21.15.38:0/3720503426 learned_addr learned my addr 172.21.15.38:0/3720503426 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.167+0000 7f554ca70740 1 -- 172.21.15.38:0/3720503426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f55401a0130 con 0x559a331e96b0 2024-05-21T15:42:30.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.167+0000 7f553effd640 1 --2- 172.21.15.38:0/3720503426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e96b0 0x559a335cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.167+0000 7f553ffff640 1 --2- 172.21.15.38:0/3720503426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3320a390 0x559a335a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.168+0000 7f553ffff640 1 -- 172.21.15.38:0/3720503426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e96b0 msgr2=0x559a335cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.168+0000 7f553ffff640 1 --2- 172.21.15.38:0/3720503426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e96b0 0x559a335cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.168+0000 7f553ffff640 1 -- 172.21.15.38:0/3720503426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a331e14d0 msgr2=0x559a335a4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.168+0000 7f553ffff640 1 --2- 172.21.15.38:0/3720503426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a331e14d0 0x559a335a4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.168+0000 7f553ffff640 1 -- 172.21.15.38:0/3720503426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a335a0d20 con 0x559a3320a390 2024-05-21T15:42:30.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.168+0000 7f553ffff640 1 --2- 172.21.15.38:0/3720503426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3320a390 0x559a335a5410 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f553802a5d0 tx=0x7f553803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec42febc93bbdc78 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.169+0000 7f55257fa640 1 -- 172.21.15.38:0/3720503426 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538003b70 con 0x559a3320a390 2024-05-21T15:42:30.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.169+0000 7f55257fa640 1 -- 172.21.15.38:0/3720503426 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5538054070 con 0x559a3320a390 2024-05-21T15:42:30.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.169+0000 7f55257fa640 1 -- 172.21.15.38:0/3720503426 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538004310 con 0x559a3320a390 2024-05-21T15:42:30.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.170+0000 7f554ca70740 1 -- 172.21.15.38:0/3720503426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3320a390 msgr2=0x559a335a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.170+0000 7f554ca70740 1 --2- 172.21.15.38:0/3720503426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3320a390 0x559a335a5410 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f553802a5d0 tx=0x7f553803f040 comp rx=0 tx=0).stop 2024-05-21T15:42:30.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.170+0000 7f554ca70740 1 -- 172.21.15.38:0/3720503426 shutdown_connections 2024-05-21T15:42:30.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.170+0000 7f554ca70740 1 --2- 172.21.15.38:0/3720503426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a3320a390 0x559a335a5410 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.170+0000 7f554ca70740 1 --2- 172.21.15.38:0/3720503426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e96b0 0x559a335cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.170+0000 7f554ca70740 1 --2- 172.21.15.38:0/3720503426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a331e14d0 0x559a335a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.170+0000 7f554ca70740 1 -- 172.21.15.38:0/3720503426 >> 172.21.15.38:0/3720503426 conn(0x559a331fe4e0 msgr2=0x559a335c3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.170+0000 7f554ca70740 1 -- 172.21.15.38:0/3720503426 shutdown_connections 2024-05-21T15:42:30.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.170+0000 7f554ca70740 1 -- 172.21.15.38:0/3720503426 wait complete. 2024-05-21T15:42:30.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.171+0000 7f554ca70740 1 -- start start 2024-05-21T15:42:30.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.171+0000 7f554ca70740 1 Processor -- start 2024-05-21T15:42:30.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.171+0000 7f554ca70740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e14d0 0x559a335c9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.172+0000 7f554ca70740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e96b0 0x559a335c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.172+0000 7f553f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e14d0 0x559a335c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.172+0000 7f553f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e14d0 0x559a335c9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59974/0 (socket says 172.21.15.38:59974) 2024-05-21T15:42:30.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.172+0000 7f553f7fe640 1 -- 172.21.15.38:0/3476338845 learned_addr learned my addr 172.21.15.38:0/3476338845 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.172+0000 7f554ca70740 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 0x559a335ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.172+0000 7f554ca70740 1 -- 172.21.15.38:0/3476338845 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f55401a0f30 con 0x559a331e14d0 2024-05-21T15:42:30.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.172+0000 7f554ca70740 1 -- 172.21.15.38:0/3476338845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f55401a0830 con 0x559a3320a390 2024-05-21T15:42:30.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.172+0000 7f553effd640 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e96b0 0x559a335c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.172+0000 7f554ca70740 1 -- 172.21.15.38:0/3476338845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f554019d9e0 con 0x559a331e96b0 2024-05-21T15:42:30.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.173+0000 7f554ca70740 10 client.0 _ll_register_callbacks cb 0x7f5530014840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:30.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.173+0000 7f553f7fe640 1 -- 172.21.15.38:0/3476338845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e96b0 msgr2=0x559a335c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.173+0000 7f553f7fe640 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e96b0 0x559a335c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.173+0000 7f553f7fe640 1 -- 172.21.15.38:0/3476338845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 msgr2=0x559a335ca5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.173+0000 7f553ffff640 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 0x559a335ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.174+0000 7f553f7fe640 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 0x559a335ca5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.174+0000 7f553f7fe640 1 -- 172.21.15.38:0/3476338845 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a335caba0 con 0x559a331e14d0 2024-05-21T15:42:30.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.174+0000 7f553ffff640 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 0x559a335ca5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:42:30.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.175+0000 7f553f7fe640 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e14d0 0x559a335c9170 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5530016010 tx=0x7f553001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ed07922f653fcf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.175+0000 7f551effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:42:30.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.176+0000 7f551effd640 1 -- 172.21.15.38:0/3476338845 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553002a760 con 0x559a331e14d0 2024-05-21T15:42:30.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.176+0000 7f551effd640 1 -- 172.21.15.38:0/3476338845 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f553002a900 con 0x559a331e14d0 2024-05-21T15:42:30.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.177+0000 7f554ca70740 1 -- 172.21.15.38:0/3476338845 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5538054070 con 0x559a331e14d0 2024-05-21T15:42:30.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.179+0000 7f551effd640 1 -- 172.21.15.38:0/3476338845 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f553002ab40 con 0x559a331e14d0 2024-05-21T15:42:30.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.179+0000 7f554ca70740 10 client.17709 fetch_fsmap learned FSMap version 40 2024-05-21T15:42:30.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.179+0000 7f554ca70740 1 -- 172.21.15.38:0/3476338845 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559a335cbe70 con 0x559a331e14d0 2024-05-21T15:42:30.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.179+0000 7f551effd640 1 -- 172.21.15.38:0/3476338845 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5538054070 con 0x559a331e14d0 2024-05-21T15:42:30.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.183+0000 7f551effd640 1 -- 172.21.15.38:0/3476338845 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f553001e040 con 0x559a331e14d0 2024-05-21T15:42:30.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.183+0000 7f554ca70740 10 client.17709 fetch_fsmap finished waiting for FSMap version 40 2024-05-21T15:42:30.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.183+0000 7f554ca70740 10 client.17709 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T15:42:30.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.183+0000 7f554ca70740 20 client.17709 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:30.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.183+0000 7f554ca70740 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x559a3319e3a0 0x559a3363cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.183+0000 7f554ca70740 4 client.17709 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:42:30.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.183+0000 7f554ca70740 1 -- 172.21.15.38:0/3476338845 --> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a335166c0 con 0x559a3319e3a0 2024-05-21T15:42:30.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.188+0000 7f553effd640 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x559a3319e3a0 0x559a3363cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.189+0000 7f553effd640 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x559a3319e3a0 0x559a3363cf50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9dd0bf11f01fdea0 server_cookie=5a2b9373dd0e5e9f in_seq=0 out_seq=0 2024-05-21T15:42:30.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.189+0000 7f551effd640 10 client.17709 ms_handle_connect on v2:172.21.15.53:6836/2684842704 2024-05-21T15:42:30.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.189+0000 7f551effd640 1 -- 172.21.15.38:0/3476338845 <== mds.3 v2:172.21.15.53:6836/2684842704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a335166c0 con 0x559a3319e3a0 2024-05-21T15:42:30.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.190+0000 7f551effd640 10 client.17709 handle_command_reply: tid=0 2024-05-21T15:42:30.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.190+0000 7f554ca70740 1 client.17709 shutdown 2024-05-21T15:42:30.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.196+0000 7f554ca70740 1 -- 172.21.15.38:0/3476338845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e14d0 msgr2=0x559a335c9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.196+0000 7f554ca70740 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e14d0 0x559a335c9170 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5530016010 tx=0x7f553001a000 comp rx=0 tx=0).stop 2024-05-21T15:42:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.197+0000 7f554ca70740 1 -- 172.21.15.38:0/3476338845 shutdown_connections 2024-05-21T15:42:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.197+0000 7f554ca70740 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.53:6836/2684842704,v1:172.21.15.53:6837/2684842704] conn(0x559a3319e3a0 0x559a3363cf50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.197+0000 7f554ca70740 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a3320a390 0x559a335ca5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.197+0000 7f551effd640 0 client.17709 ms_handle_reset on v2:172.21.15.53:6836/2684842704 2024-05-21T15:42:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.197+0000 7f554ca70740 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559a331e96b0 0x559a335c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.197+0000 7f554ca70740 1 --2- 172.21.15.38:0/3476338845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559a331e14d0 0x559a335c9170 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.197+0000 7f554ca70740 1 -- 172.21.15.38:0/3476338845 >> 172.21.15.38:0/3476338845 conn(0x559a331fe4e0 msgr2=0x559a335a4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.198+0000 7f554ca70740 1 -- 172.21.15.38:0/3476338845 shutdown_connections 2024-05-21T15:42:30.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.198+0000 7f554ca70740 1 -- 172.21.15.38:0/3476338845 wait complete. 2024-05-21T15:42:30.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.199+0000 7f554ca70740 20 client.17709 trim_cache size 0 max 16384 2024-05-21T15:42:30.198 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:42:30.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.201+0000 7f5545bcc640 1 -- 172.21.15.38:0/393881462 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f55280b3c20 msgr2=0x7f55280b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.201+0000 7f5545bcc640 1 --2- 172.21.15.38:0/393881462 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f55280b3c20 0x7f55280b6130 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5530014fd0 tx=0x7f553000d040 comp rx=0 tx=0).stop 2024-05-21T15:42:30.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.201+0000 7f5545bcc640 1 -- 172.21.15.38:0/393881462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5540102340 msgr2=0x7f5540071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.201+0000 7f5545bcc640 1 --2- 172.21.15.38:0/393881462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5540102340 0x7f5540071aa0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f553400e9f0 tx=0x7f553400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.201+0000 7f5545bcc640 1 -- 172.21.15.38:0/393881462 shutdown_connections 2024-05-21T15:42:30.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.201+0000 7f5545bcc640 1 --2- 172.21.15.38:0/393881462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5540107040 0x7f55400729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.201+0000 7f5545bcc640 1 --2- 172.21.15.38:0/393881462 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f55280b3c20 0x7f55280b6130 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.201+0000 7f5545bcc640 1 --2- 172.21.15.38:0/393881462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5540102340 0x7f5540071aa0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.201+0000 7f5545bcc640 1 --2- 172.21.15.38:0/393881462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5540101100 0x7f5540071560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.202+0000 7f5545bcc640 1 -- 172.21.15.38:0/393881462 >> 172.21.15.38:0/393881462 conn(0x7f55400fc2f0 msgr2=0x7f5540103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.202+0000 7f5545bcc640 1 -- 172.21.15.38:0/393881462 shutdown_connections 2024-05-21T15:42:30.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.202+0000 7f5545bcc640 1 -- 172.21.15.38:0/393881462 wait complete. 2024-05-21T15:42:30.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:29 smithi053 ceph-mon[36496]: pgmap v593: 97 pgs: 97 active+clean; 9.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 3.0 MiB/s wr, 820 op/s 2024-05-21T15:42:30.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:29 smithi053 ceph-mon[36496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:30.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:29 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2637008911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:42:30.219 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:42:30.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:29 smithi045 ceph-mon[36409]: pgmap v593: 97 pgs: 97 active+clean; 9.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 3.0 MiB/s wr, 820 op/s 2024-05-21T15:42:30.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:29 smithi045 ceph-mon[36409]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:42:30.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:29 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2637008911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:42:30.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.436+0000 7f2c4205a640 1 -- 172.21.15.38:0/3575259663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c1089f0 msgr2=0x7f2c3c10ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.436+0000 7f2c4205a640 1 --2- 172.21.15.38:0/3575259663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c1089f0 0x7f2c3c10ae10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009900 tx=0x7f2c3402a420 comp rx=0 tx=0).stop 2024-05-21T15:42:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.437+0000 7f2c4205a640 1 -- 172.21.15.38:0/3575259663 shutdown_connections 2024-05-21T15:42:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.437+0000 7f2c4205a640 1 --2- 172.21.15.38:0/3575259663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c1089f0 0x7f2c3c10ae10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.437+0000 7f2c4205a640 1 --2- 172.21.15.38:0/3575259663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c3c103cf0 0x7f2c3c104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.437+0000 7f2c4205a640 1 --2- 172.21.15.38:0/3575259663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c3c102ab0 0x7f2c3c1037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.437+0000 7f2c4205a640 1 -- 172.21.15.38:0/3575259663 >> 172.21.15.38:0/3575259663 conn(0x7f2c3c0fdca0 msgr2=0x7f2c3c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.437+0000 7f2c4205a640 1 -- 172.21.15.38:0/3575259663 shutdown_connections 2024-05-21T15:42:30.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.437+0000 7f2c4205a640 1 -- 172.21.15.38:0/3575259663 wait complete. 2024-05-21T15:42:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.438+0000 7f2c4205a640 1 Processor -- start 2024-05-21T15:42:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.438+0000 7f2c4205a640 1 -- start start 2024-05-21T15:42:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.438+0000 7f2c4205a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c102ab0 0x7f2c3c112320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.438+0000 7f2c4205a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c3c103cf0 0x7f2c3c112860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.438+0000 7f2c4205a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c3c1089f0 0x7f2c3c1197f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.438+0000 7f2c4205a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c075040 con 0x7f2c3c103cf0 2024-05-21T15:42:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.438+0000 7f2c4205a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c074ec0 con 0x7f2c3c102ab0 2024-05-21T15:42:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.438+0000 7f2c4205a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c0751c0 con 0x7f2c3c1089f0 2024-05-21T15:42:30.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.439+0000 7f2c3b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c102ab0 0x7f2c3c112320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.439+0000 7f2c3b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c102ab0 0x7f2c3c112320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:40522/0 (socket says 172.21.15.38:40522) 2024-05-21T15:42:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.439+0000 7f2c3b7fe640 1 -- 172.21.15.38:0/602073614 learned_addr learned my addr 172.21.15.38:0/602073614 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.439+0000 7f2c3bfff640 1 --2- 172.21.15.38:0/602073614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c3c1089f0 0x7f2c3c1197f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.439+0000 7f2c3affd640 1 --2- 172.21.15.38:0/602073614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c3c103cf0 0x7f2c3c112860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.439+0000 7f2c3b7fe640 1 -- 172.21.15.38:0/602073614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c3c1089f0 msgr2=0x7f2c3c1197f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.439+0000 7f2c3b7fe640 1 --2- 172.21.15.38:0/602073614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c3c1089f0 0x7f2c3c1197f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.439+0000 7f2c3b7fe640 1 -- 172.21.15.38:0/602073614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c3c103cf0 msgr2=0x7f2c3c112860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.439+0000 7f2c3b7fe640 1 --2- 172.21.15.38:0/602073614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c3c103cf0 0x7f2c3c112860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.439+0000 7f2c3b7fe640 1 -- 172.21.15.38:0/602073614 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c119db0 con 0x7f2c3c102ab0 2024-05-21T15:42:30.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.439+0000 7f2c3b7fe640 1 --2- 172.21.15.38:0/602073614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c102ab0 0x7f2c3c112320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400eb90 tx=0x7f2c2400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.440+0000 7f2c38ff9640 1 -- 172.21.15.38:0/602073614 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24003b70 con 0x7f2c3c102ab0 2024-05-21T15:42:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.440+0000 7f2c38ff9640 1 -- 172.21.15.38:0/602073614 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c2400ee90 con 0x7f2c3c102ab0 2024-05-21T15:42:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.440+0000 7f2c38ff9640 1 -- 172.21.15.38:0/602073614 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24019880 con 0x7f2c3c102ab0 2024-05-21T15:42:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.440+0000 7f2c4205a640 1 -- 172.21.15.38:0/602073614 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c06b570 con 0x7f2c3c102ab0 2024-05-21T15:42:30.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.441+0000 7f2c4205a640 1 -- 172.21.15.38:0/602073614 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c06bac0 con 0x7f2c3c102ab0 2024-05-21T15:42:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.445+0000 7f2c48fd9740 1 Processor -- start 2024-05-21T15:42:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.449+0000 7f2c48fd9740 1 -- start start 2024-05-21T15:42:30.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.449+0000 7f2c48fd9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b19867390 0x558b19c06b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.449+0000 7f2c48fd9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b1983e4d0 0x558b19c07080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.449+0000 7f2c48fd9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c0df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.449+0000 7f2c48fd9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558b19b6bc40 con 0x558b198466b0 2024-05-21T15:42:30.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.449+0000 7f2c48fd9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558b19b13580 con 0x558b1983e4d0 2024-05-21T15:42:30.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.449+0000 7f2c48fd9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558b19b67c00 con 0x558b19867390 2024-05-21T15:42:30.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.450+0000 7f2c3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c0df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.450+0000 7f2c3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c0df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60016/0 (socket says 172.21.15.38:60016) 2024-05-21T15:42:30.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.450+0000 7f2c3affd640 1 -- 172.21.15.38:0/2165059830 learned_addr learned my addr 172.21.15.38:0/2165059830 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.450+0000 7f2c3affd640 1 -- 172.21.15.38:0/2165059830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b19867390 msgr2=0x558b19c06b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.451+0000 7f2c3affd640 1 --2- 172.21.15.38:0/2165059830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b19867390 0x558b19c06b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.451+0000 7f2c3affd640 1 -- 172.21.15.38:0/2165059830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b1983e4d0 msgr2=0x558b19c07080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:42:30.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.451+0000 7f2c3affd640 1 --2- 172.21.15.38:0/2165059830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b1983e4d0 0x558b19c07080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.451+0000 7f2c3affd640 1 -- 172.21.15.38:0/2165059830 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b1987f3f0 con 0x558b198466b0 2024-05-21T15:42:30.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.451+0000 7f2c38ff9640 1 -- 172.21.15.38:0/602073614 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2c24019a20 con 0x7f2c3c102ab0 2024-05-21T15:42:30.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.452+0000 7f2c3affd640 1 --2- 172.21.15.38:0/2165059830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c0df80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00cc20 tx=0x7f2c2c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96f33d001ab3586 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.452+0000 7f2c38ff9640 1 --2- 172.21.15.38:0/602073614 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2c1c0a2d60 0x7f2c1c0a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.452+0000 7f2c38ff9640 1 -- 172.21.15.38:0/602073614 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2c240cf830 con 0x7f2c3c102ab0 2024-05-21T15:42:30.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.452+0000 7f2c31ffb640 1 -- 172.21.15.38:0/2165059830 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c014070 con 0x558b198466b0 2024-05-21T15:42:30.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.452+0000 7f2c31ffb640 1 -- 172.21.15.38:0/2165059830 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c2c0047a0 con 0x558b198466b0 2024-05-21T15:42:30.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.452+0000 7f2c31ffb640 1 -- 172.21.15.38:0/2165059830 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0108d0 con 0x558b198466b0 2024-05-21T15:42:30.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.454+0000 7f2c3bfff640 1 --2- 172.21.15.38:0/602073614 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2c1c0a2d60 0x7f2c1c0a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.454+0000 7f2c48fd9740 1 -- 172.21.15.38:0/2165059830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 msgr2=0x558b19c0df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.454+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/2165059830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c0df80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00cc20 tx=0x7f2c2c00e720 comp rx=0 tx=0).stop 2024-05-21T15:42:30.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.455+0000 7f2c3bfff640 1 --2- 172.21.15.38:0/602073614 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2c1c0a2d60 0x7f2c1c0a5270 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2c3402a940 tx=0x7f2c34009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.456+0000 7f2c48fd9740 1 -- 172.21.15.38:0/2165059830 shutdown_connections 2024-05-21T15:42:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.456+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/2165059830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c0df80 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.456+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/2165059830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b1983e4d0 0x558b19c07080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.456+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/2165059830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b19867390 0x558b19c06b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.456+0000 7f2c48fd9740 1 -- 172.21.15.38:0/2165059830 >> 172.21.15.38:0/2165059830 conn(0x558b1985b4e0 msgr2=0x558b19c02490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.457+0000 7f2c48fd9740 1 -- 172.21.15.38:0/2165059830 shutdown_connections 2024-05-21T15:42:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.458+0000 7f2c48fd9740 1 -- 172.21.15.38:0/2165059830 wait complete. 2024-05-21T15:42:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.459+0000 7f2c48fd9740 1 -- start start 2024-05-21T15:42:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.459+0000 7f2c48fd9740 1 Processor -- start 2024-05-21T15:42:30.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.460+0000 7f2c48fd9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c109c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.460+0000 7f2c48fd9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c10f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.460+0000 7f2c48fd9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19867390 0x558b19c17e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.460+0000 7f2c3b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c109c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.460+0000 7f2c3b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c109c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39836/0 (socket says 172.21.15.38:39836) 2024-05-21T15:42:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.460+0000 7f2c3b7fe640 1 -- 172.21.15.38:0/3999947388 learned_addr learned my addr 172.21.15.38:0/3999947388 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.460+0000 7f2c3bfff640 1 --2- 172.21.15.38:0/3999947388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19867390 0x558b19c17e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.461+0000 7f2c48fd9740 1 -- 172.21.15.38:0/3999947388 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558b19c0e9d0 con 0x558b198466b0 2024-05-21T15:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.461+0000 7f2c48fd9740 1 -- 172.21.15.38:0/3999947388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558b19b26c40 con 0x558b19867390 2024-05-21T15:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.461+0000 7f2c48fd9740 1 -- 172.21.15.38:0/3999947388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558b19c0eb50 con 0x558b1983e4d0 2024-05-21T15:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.461+0000 7f2c48fd9740 10 client.0 _ll_register_callbacks cb 0x558b1987ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.461+0000 7f2c3bfff640 1 -- 172.21.15.38:0/3999947388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 msgr2=0x558b19c109c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.461+0000 7f2c3bfff640 1 --2- 172.21.15.38:0/3999947388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c109c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.461+0000 7f2c3bfff640 1 -- 172.21.15.38:0/3999947388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 msgr2=0x558b19c10f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.461+0000 7f2c3bfff640 1 --2- 172.21.15.38:0/3999947388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c10f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.461+0000 7f2c3bfff640 1 -- 172.21.15.38:0/3999947388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b199497d0 con 0x558b19867390 2024-05-21T15:42:30.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.462+0000 7f2c3bfff640 1 --2- 172.21.15.38:0/3999947388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19867390 0x558b19c17e60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009230 tx=0x7f2c34004c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f7ca2c1c1516890 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.462+0000 7f2c2affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:42:30.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.463+0000 7f2c2affd640 1 -- 172.21.15.38:0/3999947388 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34031910 con 0x558b19867390 2024-05-21T15:42:30.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.463+0000 7f2c2affd640 1 -- 172.21.15.38:0/3999947388 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2c34031ab0 con 0x558b19867390 2024-05-21T15:42:30.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.463+0000 7f2c48fd9740 1 -- 172.21.15.38:0/3999947388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c2c0047a0 con 0x558b19867390 2024-05-21T15:42:30.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.464+0000 7f2c2affd640 1 -- 172.21.15.38:0/3999947388 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2c34031cb0 con 0x558b19867390 2024-05-21T15:42:30.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.465+0000 7f2c48fd9740 10 client.27314 fetch_fsmap learned FSMap version 40 2024-05-21T15:42:30.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.465+0000 7f2c48fd9740 1 -- 172.21.15.38:0/3999947388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558b19c1b170 con 0x558b19867390 2024-05-21T15:42:30.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.467+0000 7f2c2affd640 1 -- 172.21.15.38:0/3999947388 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c34031e50 con 0x558b19867390 2024-05-21T15:42:30.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.467+0000 7f2c2affd640 1 -- 172.21.15.38:0/3999947388 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f2c3405c5f0 con 0x558b19867390 2024-05-21T15:42:30.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.467+0000 7f2c48fd9740 10 client.27314 fetch_fsmap finished waiting for FSMap version 40 2024-05-21T15:42:30.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.467+0000 7f2c48fd9740 10 client.27314 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:42:30.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.467+0000 7f2c48fd9740 20 client.27314 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:30.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.467+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/3999947388 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558b19825f60 0x558b19c1e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.467+0000 7f2c48fd9740 4 client.27314 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:42:30.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.467+0000 7f2c48fd9740 1 -- 172.21.15.38:0/3999947388 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2c2c0047a0 con 0x558b19825f60 2024-05-21T15:42:30.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.468+0000 7f2c3b7fe640 1 --2- 172.21.15.38:0/3999947388 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558b19825f60 0x558b19c1e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.470+0000 7f2c2affd640 10 client.27314 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:42:30.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.470+0000 7f2c3b7fe640 1 --2- 172.21.15.38:0/3999947388 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558b19825f60 0x558b19c1e810 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8dab08b5f8e80378 server_cookie=a220eb4535eb0470 in_seq=0 out_seq=0 2024-05-21T15:42:30.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.472+0000 7f2c2affd640 1 -- 172.21.15.38:0/3999947388 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f2c2c0047a0 con 0x558b19825f60 2024-05-21T15:42:30.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.472+0000 7f2c2affd640 10 client.27314 handle_command_reply: tid=0 2024-05-21T15:42:30.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.472+0000 7f2c48fd9740 1 client.27314 shutdown 2024-05-21T15:42:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.474+0000 7f2c48fd9740 1 -- 172.21.15.38:0/3999947388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19867390 msgr2=0x558b19c17e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.474+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/3999947388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19867390 0x558b19c17e60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009230 tx=0x7f2c34004c30 comp rx=0 tx=0).stop 2024-05-21T15:42:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.474+0000 7f2c48fd9740 1 -- 172.21.15.38:0/3999947388 shutdown_connections 2024-05-21T15:42:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.474+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/3999947388 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558b19825f60 0x558b19c1e810 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.474+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/3999947388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19867390 0x558b19c17e60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.474+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/3999947388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c10f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.474+0000 7f2c2affd640 0 client.27314 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:42:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.474+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/3999947388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.474+0000 7f2c48fd9740 1 -- 172.21.15.38:0/3999947388 >> 172.21.15.38:0/3999947388 conn(0x558b1985b4e0 msgr2=0x558b19c01570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.475+0000 7f2c48fd9740 1 -- 172.21.15.38:0/3999947388 shutdown_connections 2024-05-21T15:42:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.475+0000 7f2c48fd9740 1 -- 172.21.15.38:0/3999947388 wait complete. 2024-05-21T15:42:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.475+0000 7f2c48fd9740 20 client.27314 trim_cache size 0 max 16384 2024-05-21T15:42:30.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.503+0000 7f2c48fd9740 1 Processor -- start 2024-05-21T15:42:30.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.505+0000 7f2c48fd9740 1 -- start start 2024-05-21T15:42:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.506+0000 7f2c48fd9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19825f60 0x558b19c01250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.506+0000 7f2c48fd9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c17d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.506+0000 7f2c48fd9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c0f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.506+0000 7f2c48fd9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c1c1a40 con 0x558b198466b0 2024-05-21T15:42:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.506+0000 7f2c48fd9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c1c3640 con 0x558b19825f60 2024-05-21T15:42:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.506+0000 7f2c48fd9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c1c2170 con 0x558b1983e4d0 2024-05-21T15:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.507+0000 7f2c3affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c17d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.507+0000 7f2c3affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c17d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39838/0 (socket says 172.21.15.38:39838) 2024-05-21T15:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.507+0000 7f2c3affd640 1 -- 172.21.15.38:0/1157153855 learned_addr learned my addr 172.21.15.38:0/1157153855 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.507+0000 7f2c3b7fe640 1 --2- 172.21.15.38:0/1157153855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19825f60 0x558b19c01250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.508+0000 7f2c3bfff640 1 --2- 172.21.15.38:0/1157153855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c0f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.510+0000 7f2c3affd640 1 -- 172.21.15.38:0/1157153855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19825f60 msgr2=0x558b19c01250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.510+0000 7f2c3affd640 1 --2- 172.21.15.38:0/1157153855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19825f60 0x558b19c01250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.510+0000 7f2c3affd640 1 -- 172.21.15.38:0/1157153855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 msgr2=0x558b19c0f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.510+0000 7f2c3affd640 1 --2- 172.21.15.38:0/1157153855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c0f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.510+0000 7f2c3affd640 1 -- 172.21.15.38:0/1157153855 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b19c030c0 con 0x558b1983e4d0 2024-05-21T15:42:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.512+0000 7f2c3affd640 1 --2- 172.21.15.38:0/1157153855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c17d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00cbf0 tx=0x7f2c2c00c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2095471f4e8c10ae server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.514+0000 7f2c317fa640 1 -- 172.21.15.38:0/1157153855 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c014070 con 0x558b1983e4d0 2024-05-21T15:42:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.514+0000 7f2c317fa640 1 -- 172.21.15.38:0/1157153855 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c2c005500 con 0x558b1983e4d0 2024-05-21T15:42:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.514+0000 7f2c317fa640 1 -- 172.21.15.38:0/1157153855 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c010700 con 0x558b1983e4d0 2024-05-21T15:42:30.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.517+0000 7f2c48fd9740 1 -- 172.21.15.38:0/1157153855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 msgr2=0x558b19c17d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.517+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/1157153855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c17d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00cbf0 tx=0x7f2c2c00c370 comp rx=0 tx=0).stop 2024-05-21T15:42:30.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.519+0000 7f2c48fd9740 1 -- 172.21.15.38:0/1157153855 shutdown_connections 2024-05-21T15:42:30.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.519+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/1157153855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c0f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.519+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/1157153855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c17d80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.519+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/1157153855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19825f60 0x558b19c01250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.519+0000 7f2c48fd9740 1 -- 172.21.15.38:0/1157153855 >> 172.21.15.38:0/1157153855 conn(0x558b1985b4e0 msgr2=0x558b19c00b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.521+0000 7f2c48fd9740 1 -- 172.21.15.38:0/1157153855 shutdown_connections 2024-05-21T15:42:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.521+0000 7f2c48fd9740 1 -- 172.21.15.38:0/1157153855 wait complete. 2024-05-21T15:42:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.521+0000 7f2c48fd9740 1 -- start start 2024-05-21T15:42:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.521+0000 7f2c48fd9740 1 Processor -- start 2024-05-21T15:42:30.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.522+0000 7f2c48fd9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19825f60 0x558b19c152c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.522+0000 7f2c48fd9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c15800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.522+0000 7f2c48fd9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c16730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.522+0000 7f2c48fd9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c1c2f70 con 0x558b198466b0 2024-05-21T15:42:30.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.522+0000 7f2c48fd9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c1c2870 con 0x558b19825f60 2024-05-21T15:42:30.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.522+0000 7f2c48fd9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c0729b0 con 0x558b1983e4d0 2024-05-21T15:42:30.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.522+0000 7f2c48fd9740 10 client.0 _ll_register_callbacks cb 0x7f2c34009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:30.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.523+0000 7f2c3affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c15800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.523+0000 7f2c3affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c15800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39846/0 (socket says 172.21.15.38:39846) 2024-05-21T15:42:30.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.523+0000 7f2c3affd640 1 -- 172.21.15.38:0/2463727485 learned_addr learned my addr 172.21.15.38:0/2463727485 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.523+0000 7f2c3b7fe640 1 --2- 172.21.15.38:0/2463727485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19825f60 0x558b19c152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.523+0000 7f2c3bfff640 1 --2- 172.21.15.38:0/2463727485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c16730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.523+0000 7f2c3affd640 1 -- 172.21.15.38:0/2463727485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19825f60 msgr2=0x558b19c152c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.523+0000 7f2c3affd640 1 --2- 172.21.15.38:0/2463727485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19825f60 0x558b19c152c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.523+0000 7f2c3affd640 1 -- 172.21.15.38:0/2463727485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 msgr2=0x558b19c16730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.523+0000 7f2c3affd640 1 --2- 172.21.15.38:0/2463727485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c16730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.523+0000 7f2c3affd640 1 -- 172.21.15.38:0/2463727485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b19c16cf0 con 0x558b1983e4d0 2024-05-21T15:42:30.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.524+0000 7f2c3affd640 1 --2- 172.21.15.38:0/2463727485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c15800 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00c570 tx=0x7f2c2c005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c95b4b1f6bd834a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.524+0000 7f2c2affd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:42:30.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.528+0000 7f2c2affd640 1 -- 172.21.15.38:0/2463727485 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c027070 con 0x558b1983e4d0 2024-05-21T15:42:30.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.528+0000 7f2c48fd9740 1 -- 172.21.15.38:0/2463727485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c2c005500 con 0x558b1983e4d0 2024-05-21T15:42:30.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.528+0000 7f2c2affd640 1 -- 172.21.15.38:0/2463727485 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2c2c010840 con 0x558b1983e4d0 2024-05-21T15:42:30.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.529+0000 7f2c2affd640 1 -- 172.21.15.38:0/2463727485 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2c2c033050 con 0x558b1983e4d0 2024-05-21T15:42:30.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.529+0000 7f2c48fd9740 10 client.27412 fetch_fsmap learned FSMap version 40 2024-05-21T15:42:30.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.529+0000 7f2c48fd9740 1 -- 172.21.15.38:0/2463727485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558b19c17f70 con 0x558b1983e4d0 2024-05-21T15:42:30.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.531+0000 7f2c2affd640 1 -- 172.21.15.38:0/2463727485 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c2c005500 con 0x558b1983e4d0 2024-05-21T15:42:30.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.531+0000 7f2c2affd640 1 -- 172.21.15.38:0/2463727485 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f2c2c02c960 con 0x558b1983e4d0 2024-05-21T15:42:30.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.532+0000 7f2c48fd9740 10 client.27412 fetch_fsmap finished waiting for FSMap version 40 2024-05-21T15:42:30.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.532+0000 7f2c48fd9740 10 client.27412 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-21T15:42:30.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.532+0000 7f2c48fd9740 20 client.27412 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:30.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.532+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/2463727485 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558b19867390 0x558b19c88f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.532+0000 7f2c48fd9740 4 client.27412 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:42:30.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.532+0000 7f2c48fd9740 1 -- 172.21.15.38:0/2463727485 --> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b19b79370 con 0x558b19867390 2024-05-21T15:42:30.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.532+0000 7f2c3b7fe640 1 --2- 172.21.15.38:0/2463727485 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558b19867390 0x558b19c88f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.535+0000 7f2c3b7fe640 1 --2- 172.21.15.38:0/2463727485 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558b19867390 0x558b19c88f10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73ade7e7edf27ff7 server_cookie=d7acd351481b37c7 in_seq=0 out_seq=0 2024-05-21T15:42:30.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.535+0000 7f2c2affd640 10 client.27412 ms_handle_connect on v2:172.21.15.53:6838/3594548281 2024-05-21T15:42:30.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.535+0000 7f2c2affd640 1 -- 172.21.15.38:0/2463727485 <== mds.2 v2:172.21.15.53:6838/3594548281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b19b79370 con 0x558b19867390 2024-05-21T15:42:30.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.535+0000 7f2c2affd640 10 client.27412 handle_command_reply: tid=0 2024-05-21T15:42:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.535+0000 7f2c48fd9740 1 client.27412 shutdown 2024-05-21T15:42:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.535+0000 7f2c48fd9740 1 -- 172.21.15.38:0/2463727485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 msgr2=0x558b19c15800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.535+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/2463727485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c15800 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00c570 tx=0x7f2c2c005230 comp rx=0 tx=0).stop 2024-05-21T15:42:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.535+0000 7f2c48fd9740 1 -- 172.21.15.38:0/2463727485 shutdown_connections 2024-05-21T15:42:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.535+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/2463727485 >> [v2:172.21.15.53:6838/3594548281,v1:172.21.15.53:6839/3594548281] conn(0x558b19867390 0x558b19c88f10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.535+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/2463727485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b198466b0 0x558b19c16730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.535+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/2463727485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558b1983e4d0 0x558b19c15800 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.535+0000 7f2c2affd640 0 client.27412 ms_handle_reset on v2:172.21.15.53:6838/3594548281 2024-05-21T15:42:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.535+0000 7f2c48fd9740 1 --2- 172.21.15.38:0/2463727485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b19825f60 0x558b19c152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.535+0000 7f2c48fd9740 1 -- 172.21.15.38:0/2463727485 >> 172.21.15.38:0/2463727485 conn(0x558b1985b4e0 msgr2=0x558b19c01570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.536+0000 7f2c48fd9740 1 -- 172.21.15.38:0/2463727485 shutdown_connections 2024-05-21T15:42:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.536+0000 7f2c48fd9740 1 -- 172.21.15.38:0/2463727485 wait complete. 2024-05-21T15:42:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.536+0000 7f2c48fd9740 20 client.27412 trim_cache size 0 max 16384 2024-05-21T15:42:30.536 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:42:30.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.537+0000 7f2c4205a640 1 -- 172.21.15.38:0/602073614 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2c1c0a2d60 msgr2=0x7f2c1c0a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.537+0000 7f2c4205a640 1 --2- 172.21.15.38:0/602073614 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2c1c0a2d60 0x7f2c1c0a5270 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2c3402a940 tx=0x7f2c34009560 comp rx=0 tx=0).stop 2024-05-21T15:42:30.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.537+0000 7f2c4205a640 1 -- 172.21.15.38:0/602073614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c102ab0 msgr2=0x7f2c3c112320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.537+0000 7f2c4205a640 1 --2- 172.21.15.38:0/602073614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c102ab0 0x7f2c3c112320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400eb90 tx=0x7f2c2400c6a0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.537+0000 7f2c4205a640 1 -- 172.21.15.38:0/602073614 shutdown_connections 2024-05-21T15:42:30.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.537+0000 7f2c4205a640 1 --2- 172.21.15.38:0/602073614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c3c1089f0 0x7f2c3c1197f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.537+0000 7f2c4205a640 1 --2- 172.21.15.38:0/602073614 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2c1c0a2d60 0x7f2c1c0a5270 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.538+0000 7f2c4205a640 1 --2- 172.21.15.38:0/602073614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c3c103cf0 0x7f2c3c112860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.538+0000 7f2c4205a640 1 --2- 172.21.15.38:0/602073614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c102ab0 0x7f2c3c112320 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.538+0000 7f2c4205a640 1 -- 172.21.15.38:0/602073614 >> 172.21.15.38:0/602073614 conn(0x7f2c3c0fdca0 msgr2=0x7f2c3c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.538+0000 7f2c4205a640 1 -- 172.21.15.38:0/602073614 shutdown_connections 2024-05-21T15:42:30.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.538+0000 7f2c4205a640 1 -- 172.21.15.38:0/602073614 wait complete. 2024-05-21T15:42:30.554 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:42:30.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.751+0000 7f227f7d7640 1 -- 172.21.15.38:0/245677981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f227810a2c0 msgr2=0x7f227810a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.751+0000 7f227f7d7640 1 --2- 172.21.15.38:0/245677981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f227810a2c0 0x7f227810a6d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f227001a910 tx=0x7f227003b450 comp rx=0 tx=0).stop 2024-05-21T15:42:30.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.753+0000 7f227f7d7640 1 -- 172.21.15.38:0/245677981 shutdown_connections 2024-05-21T15:42:30.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.753+0000 7f227f7d7640 1 --2- 172.21.15.38:0/245677981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227810efe0 0x7f227810f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.753+0000 7f227f7d7640 1 --2- 172.21.15.38:0/245677981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f227810a2c0 0x7f227810a6d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.753+0000 7f227f7d7640 1 --2- 172.21.15.38:0/245677981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22781093d0 0x7f2278109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.753+0000 7f227f7d7640 1 -- 172.21.15.38:0/245677981 >> 172.21.15.38:0/245677981 conn(0x7f227806f4a0 msgr2=0x7f2278071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.755+0000 7f227f7d7640 1 -- 172.21.15.38:0/245677981 shutdown_connections 2024-05-21T15:42:30.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.755+0000 7f227f7d7640 1 -- 172.21.15.38:0/245677981 wait complete. 2024-05-21T15:42:30.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.756+0000 7f227f7d7640 1 Processor -- start 2024-05-21T15:42:30.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.756+0000 7f227f7d7640 1 -- start start 2024-05-21T15:42:30.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.757+0000 7f227f7d7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22781093d0 0x7f227819fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.757+0000 7f227f7d7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f227810a2c0 0x7f227819ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.757+0000 7f227f7d7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227810efe0 0x7f22781a6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.757+0000 7f227f7d7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2278112ea0 con 0x7f22781093d0 2024-05-21T15:42:30.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.757+0000 7f227f7d7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2278112d20 con 0x7f227810a2c0 2024-05-21T15:42:30.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.757+0000 7f227f7d7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2278113020 con 0x7f227810efe0 2024-05-21T15:42:30.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.757+0000 7f227dd74640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227810efe0 0x7f22781a6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.757+0000 7f227dd74640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227810efe0 0x7f22781a6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39868/0 (socket says 172.21.15.38:39868) 2024-05-21T15:42:30.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.757+0000 7f227dd74640 1 -- 172.21.15.38:0/3030648628 learned_addr learned my addr 172.21.15.38:0/3030648628 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.758+0000 7f227cd72640 1 --2- 172.21.15.38:0/3030648628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f227810a2c0 0x7f227819ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.759+0000 7f227d573640 1 --2- 172.21.15.38:0/3030648628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22781093d0 0x7f227819fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.760+0000 7f227dd74640 1 -- 172.21.15.38:0/3030648628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f227810a2c0 msgr2=0x7f227819ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.760+0000 7f227dd74640 1 --2- 172.21.15.38:0/3030648628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f227810a2c0 0x7f227819ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.760+0000 7f227dd74640 1 -- 172.21.15.38:0/3030648628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22781093d0 msgr2=0x7f227819fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.760+0000 7f227dd74640 1 --2- 172.21.15.38:0/3030648628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22781093d0 0x7f227819fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.760+0000 7f227dd74640 1 -- 172.21.15.38:0/3030648628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22781a7580 con 0x7f227810efe0 2024-05-21T15:42:30.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.760+0000 7f227dd74640 1 --2- 172.21.15.38:0/3030648628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227810efe0 0x7f22781a6f10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2260009fd0 tx=0x7f226000b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.762+0000 7f22767fc640 1 -- 172.21.15.38:0/3030648628 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260011830 con 0x7f227810efe0 2024-05-21T15:42:30.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.762+0000 7f22767fc640 1 -- 172.21.15.38:0/3030648628 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2260016070 con 0x7f227810efe0 2024-05-21T15:42:30.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.762+0000 7f22767fc640 1 -- 172.21.15.38:0/3030648628 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226000f2e0 con 0x7f227810efe0 2024-05-21T15:42:30.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.762+0000 7f227f7d7640 1 -- 172.21.15.38:0/3030648628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22781a7870 con 0x7f227810efe0 2024-05-21T15:42:30.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.762+0000 7f227f7d7640 1 -- 172.21.15.38:0/3030648628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22781a7df0 con 0x7f227810efe0 2024-05-21T15:42:30.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.766+0000 7f228667d740 1 Processor -- start 2024-05-21T15:42:30.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.767+0000 7f22767fc640 1 -- 172.21.15.38:0/3030648628 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f226000f480 con 0x7f227810efe0 2024-05-21T15:42:30.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.767+0000 7f228667d740 1 -- start start 2024-05-21T15:42:30.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.767+0000 7f228667d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae84782390 0x55ae84b21b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.767+0000 7f228667d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b22080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.767+0000 7f228667d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae847616b0 0x55ae84b28f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.767+0000 7f228667d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ae84a86c40 con 0x55ae84782390 2024-05-21T15:42:30.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.767+0000 7f228667d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ae84a2e580 con 0x55ae847616b0 2024-05-21T15:42:30.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.767+0000 7f228667d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ae84a82c00 con 0x55ae847594d0 2024-05-21T15:42:30.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.768+0000 7f227d573640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b22080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.768+0000 7f227d573640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b22080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39872/0 (socket says 172.21.15.38:39872) 2024-05-21T15:42:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.768+0000 7f227d573640 1 -- 172.21.15.38:0/1647565273 learned_addr learned my addr 172.21.15.38:0/1647565273 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.768+0000 7f227d573640 1 -- 172.21.15.38:0/1647565273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae847616b0 msgr2=0x55ae84b28f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:42:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.768+0000 7f227d573640 1 --2- 172.21.15.38:0/1647565273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae847616b0 0x55ae84b28f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.768+0000 7f227d573640 1 -- 172.21.15.38:0/1647565273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae84782390 msgr2=0x55ae84b21b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:42:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.768+0000 7f227d573640 1 --2- 172.21.15.38:0/1647565273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae84782390 0x55ae84b21b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.768+0000 7f227d573640 1 -- 172.21.15.38:0/1647565273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae8479a3f0 con 0x55ae847594d0 2024-05-21T15:42:30.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.770+0000 7f227d573640 1 --2- 172.21.15.38:0/1647565273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b22080 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2268009fa0 tx=0x7f226800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f58ce8e81600917 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.770+0000 7f22767fc640 1 --2- 172.21.15.38:0/3030648628 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f22640a2d10 0x7f22640a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.770+0000 7f22767fc640 1 -- 172.21.15.38:0/3030648628 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f22600ce670 con 0x7f227810efe0 2024-05-21T15:42:30.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.771+0000 7f226f7fe640 1 -- 172.21.15.38:0/1647565273 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226801a070 con 0x55ae847594d0 2024-05-21T15:42:30.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.771+0000 7f226f7fe640 1 -- 172.21.15.38:0/1647565273 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22680158f0 con 0x55ae847594d0 2024-05-21T15:42:30.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.771+0000 7f227cd72640 1 --2- 172.21.15.38:0/3030648628 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f22640a2d10 0x7f22640a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.772+0000 7f228667d740 1 -- 172.21.15.38:0/1647565273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 msgr2=0x55ae84b22080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.772+0000 7f228667d740 1 --2- 172.21.15.38:0/1647565273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b22080 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2268009fa0 tx=0x7f226800c8b0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.774+0000 7f227cd72640 1 --2- 172.21.15.38:0/3030648628 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f22640a2d10 0x7f22640a5220 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x55ae84b22cd0 tx=0x7f2270000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.775+0000 7f228667d740 1 -- 172.21.15.38:0/1647565273 shutdown_connections 2024-05-21T15:42:30.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.775+0000 7f228667d740 1 --2- 172.21.15.38:0/1647565273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae847616b0 0x55ae84b28f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.775+0000 7f228667d740 1 --2- 172.21.15.38:0/1647565273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b22080 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.775+0000 7f228667d740 1 --2- 172.21.15.38:0/1647565273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae84782390 0x55ae84b21b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.775+0000 7f228667d740 1 -- 172.21.15.38:0/1647565273 >> 172.21.15.38:0/1647565273 conn(0x55ae847764e0 msgr2=0x55ae84b1d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.775+0000 7f228667d740 1 -- 172.21.15.38:0/1647565273 shutdown_connections 2024-05-21T15:42:30.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.776+0000 7f228667d740 1 -- 172.21.15.38:0/1647565273 wait complete. 2024-05-21T15:42:30.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.776+0000 7f228667d740 1 -- start start 2024-05-21T15:42:30.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.776+0000 7f228667d740 1 Processor -- start 2024-05-21T15:42:30.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.777+0000 7f228667d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847594d0 0x55ae84b2b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.777+0000 7f228667d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847616b0 0x55ae84b2bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.777+0000 7f228667d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 0x55ae84b32e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.777+0000 7f228667d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ae84b29a20 con 0x55ae847594d0 2024-05-21T15:42:30.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.777+0000 7f228667d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ae84a41c40 con 0x55ae84782390 2024-05-21T15:42:30.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.777+0000 7f228667d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ae84b29ba0 con 0x55ae847616b0 2024-05-21T15:42:30.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.777+0000 7f227cd72640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847616b0 0x55ae84b2bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.777+0000 7f227cd72640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847616b0 0x55ae84b2bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39880/0 (socket says 172.21.15.38:39880) 2024-05-21T15:42:30.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.777+0000 7f227cd72640 1 -- 172.21.15.38:0/1354749381 learned_addr learned my addr 172.21.15.38:0/1354749381 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.777+0000 7f227dd74640 1 --2- 172.21.15.38:0/1354749381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 0x55ae84b32e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.777+0000 7f228667d740 10 client.0 _ll_register_callbacks cb 0x55ae84799be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:30.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.778+0000 7f227d573640 1 --2- 172.21.15.38:0/1354749381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847594d0 0x55ae84b2b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.778+0000 7f227dd74640 1 -- 172.21.15.38:0/1354749381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847616b0 msgr2=0x55ae84b2bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.778+0000 7f227dd74640 1 --2- 172.21.15.38:0/1354749381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847616b0 0x55ae84b2bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.778+0000 7f227dd74640 1 -- 172.21.15.38:0/1354749381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847594d0 msgr2=0x55ae84b2b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.778+0000 7f227dd74640 1 --2- 172.21.15.38:0/1354749381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847594d0 0x55ae84b2b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.778+0000 7f227dd74640 1 -- 172.21.15.38:0/1354749381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae848647d0 con 0x55ae84782390 2024-05-21T15:42:30.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.779+0000 7f227dd74640 1 --2- 172.21.15.38:0/1354749381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 0x55ae84b32e40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f22600cc170 tx=0x7f226009ae80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c92168603e8aff server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.779+0000 7f226cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-21T15:42:30.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.780+0000 7f226cff9640 1 -- 172.21.15.38:0/1354749381 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260099180 con 0x55ae84782390 2024-05-21T15:42:30.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.780+0000 7f228667d740 1 -- 172.21.15.38:0/1354749381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22680158f0 con 0x55ae84782390 2024-05-21T15:42:30.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.780+0000 7f226cff9640 1 -- 172.21.15.38:0/1354749381 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f22600974f0 con 0x55ae84782390 2024-05-21T15:42:30.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.781+0000 7f226cff9640 1 -- 172.21.15.38:0/1354749381 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f226009b0e0 con 0x55ae84782390 2024-05-21T15:42:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.782+0000 7f228667d740 10 client.27329 fetch_fsmap learned FSMap version 40 2024-05-21T15:42:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.782+0000 7f228667d740 1 -- 172.21.15.38:0/1354749381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ae84b36220 con 0x55ae84782390 2024-05-21T15:42:30.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.782+0000 7f226cff9640 1 -- 172.21.15.38:0/1354749381 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22680158f0 con 0x55ae84782390 2024-05-21T15:42:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.782+0000 7f226cff9640 1 -- 172.21.15.38:0/1354749381 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f226009a2d0 con 0x55ae84782390 2024-05-21T15:42:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.782+0000 7f228667d740 10 client.27329 fetch_fsmap finished waiting for FSMap version 40 2024-05-21T15:42:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.782+0000 7f228667d740 10 client.27329 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:42:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.783+0000 7f228667d740 20 client.27329 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.783+0000 7f228667d740 1 --2- 172.21.15.38:0/1354749381 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55ae84740f60 0x55ae84b39e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.783+0000 7f228667d740 4 client.27329 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:42:30.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.783+0000 7f228667d740 1 -- 172.21.15.38:0/1354749381 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae84a94370 con 0x55ae84740f60 2024-05-21T15:42:30.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.785+0000 7f227d573640 1 --2- 172.21.15.38:0/1354749381 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55ae84740f60 0x55ae84b39e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.787+0000 7f227d573640 1 --2- 172.21.15.38:0/1354749381 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55ae84740f60 0x55ae84b39e30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e671b1c8cb01b69 server_cookie=5f36dfec9a74ed2 in_seq=0 out_seq=0 2024-05-21T15:42:30.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.787+0000 7f226cff9640 10 client.27329 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:42:30.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.788+0000 7f226cff9640 1 -- 172.21.15.38:0/1354749381 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ae84a94370 con 0x55ae84740f60 2024-05-21T15:42:30.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.788+0000 7f226cff9640 10 client.27329 handle_command_reply: tid=0 2024-05-21T15:42:30.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.788+0000 7f228667d740 1 client.27329 shutdown 2024-05-21T15:42:30.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.788+0000 7f228667d740 1 -- 172.21.15.38:0/1354749381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 msgr2=0x55ae84b32e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.788+0000 7f228667d740 1 --2- 172.21.15.38:0/1354749381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 0x55ae84b32e40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f22600cc170 tx=0x7f226009ae80 comp rx=0 tx=0).stop 2024-05-21T15:42:30.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.789+0000 7f228667d740 1 -- 172.21.15.38:0/1354749381 shutdown_connections 2024-05-21T15:42:30.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.789+0000 7f228667d740 1 --2- 172.21.15.38:0/1354749381 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55ae84740f60 0x55ae84b39e30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.789+0000 7f228667d740 1 --2- 172.21.15.38:0/1354749381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 0x55ae84b32e40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.789+0000 7f228667d740 1 --2- 172.21.15.38:0/1354749381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847616b0 0x55ae84b2bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.789+0000 7f228667d740 1 --2- 172.21.15.38:0/1354749381 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847594d0 0x55ae84b2b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.789+0000 7f228667d740 1 -- 172.21.15.38:0/1354749381 >> 172.21.15.38:0/1354749381 conn(0x55ae847764e0 msgr2=0x55ae84b1c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.789+0000 7f226cff9640 0 client.27329 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:42:30.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.790+0000 7f228667d740 1 -- 172.21.15.38:0/1354749381 shutdown_connections 2024-05-21T15:42:30.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.790+0000 7f228667d740 1 -- 172.21.15.38:0/1354749381 wait complete. 2024-05-21T15:42:30.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.790+0000 7f228667d740 20 client.27329 trim_cache size 0 max 16384 2024-05-21T15:42:30.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.816+0000 7f228667d740 1 Processor -- start 2024-05-21T15:42:30.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.816+0000 7f228667d740 1 -- start start 2024-05-21T15:42:30.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.816+0000 7f228667d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b25cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.816+0000 7f228667d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847616b0 0x55ae84b32d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.816+0000 7f228667d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 0x55ae84b26210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.816+0000 7f228667d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f225800e5f0 con 0x55ae847616b0 2024-05-21T15:42:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.816+0000 7f228667d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f22580101f0 con 0x55ae84782390 2024-05-21T15:42:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.816+0000 7f228667d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f225800ed20 con 0x55ae847594d0 2024-05-21T15:42:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.817+0000 7f227d573640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b25cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.817+0000 7f227d573640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b25cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39890/0 (socket says 172.21.15.38:39890) 2024-05-21T15:42:30.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.817+0000 7f227d573640 1 -- 172.21.15.38:0/2383079214 learned_addr learned my addr 172.21.15.38:0/2383079214 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.817+0000 7f227cd72640 1 --2- 172.21.15.38:0/2383079214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847616b0 0x55ae84b32d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.817+0000 7f227d573640 1 -- 172.21.15.38:0/2383079214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 msgr2=0x55ae84b26210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.817+0000 7f227d573640 1 --2- 172.21.15.38:0/2383079214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 0x55ae84b26210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.817+0000 7f227d573640 1 -- 172.21.15.38:0/2383079214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847616b0 msgr2=0x55ae84b32d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.817+0000 7f227d573640 1 --2- 172.21.15.38:0/2383079214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847616b0 0x55ae84b32d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.817+0000 7f227d573640 1 -- 172.21.15.38:0/2383079214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae84b18d20 con 0x55ae847594d0 2024-05-21T15:42:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.818+0000 7f227d573640 1 --2- 172.21.15.38:0/2383079214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b25cd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2268000f80 tx=0x7f226800a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2118e7e6cfdea160 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.818+0000 7f226effd640 1 -- 172.21.15.38:0/2383079214 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268026020 con 0x55ae847594d0 2024-05-21T15:42:30.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.819+0000 7f226effd640 1 -- 172.21.15.38:0/2383079214 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22680261c0 con 0x55ae847594d0 2024-05-21T15:42:30.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.819+0000 7f226effd640 1 -- 172.21.15.38:0/2383079214 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226801a040 con 0x55ae847594d0 2024-05-21T15:42:30.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.820+0000 7f228667d740 1 -- 172.21.15.38:0/2383079214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 msgr2=0x55ae84b25cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.820+0000 7f228667d740 1 --2- 172.21.15.38:0/2383079214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b25cd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2268000f80 tx=0x7f226800a400 comp rx=0 tx=0).stop 2024-05-21T15:42:30.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.822+0000 7f228667d740 1 -- 172.21.15.38:0/2383079214 shutdown_connections 2024-05-21T15:42:30.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.822+0000 7f228667d740 1 --2- 172.21.15.38:0/2383079214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 0x55ae84b26210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.822+0000 7f228667d740 1 --2- 172.21.15.38:0/2383079214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847616b0 0x55ae84b32d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.822+0000 7f228667d740 1 --2- 172.21.15.38:0/2383079214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b25cd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.822+0000 7f228667d740 1 -- 172.21.15.38:0/2383079214 >> 172.21.15.38:0/2383079214 conn(0x55ae847764e0 msgr2=0x55ae84b2a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.822+0000 7f228667d740 1 -- 172.21.15.38:0/2383079214 shutdown_connections 2024-05-21T15:42:30.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.822+0000 7f228667d740 1 -- 172.21.15.38:0/2383079214 wait complete. 2024-05-21T15:42:30.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.822+0000 7f228667d740 1 -- start start 2024-05-21T15:42:30.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.823+0000 7f228667d740 1 Processor -- start 2024-05-21T15:42:30.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.823+0000 7f228667d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b32d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.823+0000 7f228667d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847616b0 0x55ae84b2f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.823+0000 7f228667d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 0x55ae84b2fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.823+0000 7f228667d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f225800fb20 con 0x55ae847616b0 2024-05-21T15:42:30.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.823+0000 7f228667d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f225800f420 con 0x55ae84782390 2024-05-21T15:42:30.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.823+0000 7f228667d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f225800c5d0 con 0x55ae847594d0 2024-05-21T15:42:30.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.824+0000 7f227d573640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b32d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.824+0000 7f227d573640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b32d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39898/0 (socket says 172.21.15.38:39898) 2024-05-21T15:42:30.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.824+0000 7f227d573640 1 -- 172.21.15.38:0/1668862382 learned_addr learned my addr 172.21.15.38:0/1668862382 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:30.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.824+0000 7f227dd74640 1 --2- 172.21.15.38:0/1668862382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 0x55ae84b2fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.824+0000 7f228667d740 10 client.0 _ll_register_callbacks cb 0x7f2260097120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:30.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.824+0000 7f227cd72640 1 --2- 172.21.15.38:0/1668862382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847616b0 0x55ae84b2f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.825+0000 7f227cd72640 1 -- 172.21.15.38:0/1668862382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 msgr2=0x55ae84b32d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.825+0000 7f227cd72640 1 --2- 172.21.15.38:0/1668862382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b32d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.825+0000 7f227cd72640 1 -- 172.21.15.38:0/1668862382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 msgr2=0x55ae84b2fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:30.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.825+0000 7f227cd72640 1 --2- 172.21.15.38:0/1668862382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 0x55ae84b2fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.825+0000 7f227cd72640 1 -- 172.21.15.38:0/1668862382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae84ba0200 con 0x55ae847616b0 2024-05-21T15:42:30.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.825+0000 7f227cd72640 1 --2- 172.21.15.38:0/1668862382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847616b0 0x55ae84b2f890 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f227003ba40 tx=0x7f227001a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c33795dde13d36 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:30.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.825+0000 7f226cff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:42:30.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.826+0000 7f226cff9640 1 -- 172.21.15.38:0/1668862382 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270060070 con 0x55ae847616b0 2024-05-21T15:42:30.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.826+0000 7f226cff9640 1 -- 172.21.15.38:0/1668862382 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2270004300 con 0x55ae847616b0 2024-05-21T15:42:30.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.826+0000 7f228667d740 1 -- 172.21.15.38:0/1668862382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22680261c0 con 0x55ae847616b0 2024-05-21T15:42:30.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.826+0000 7f226cff9640 1 -- 172.21.15.38:0/1668862382 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f227006a050 con 0x55ae847616b0 2024-05-21T15:42:30.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.826+0000 7f228667d740 10 client.17733 fetch_fsmap learned FSMap version 40 2024-05-21T15:42:30.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.826+0000 7f228667d740 1 -- 172.21.15.38:0/1668862382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ae84ba14b0 con 0x55ae847616b0 2024-05-21T15:42:30.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.827+0000 7f226cff9640 1 -- 172.21.15.38:0/1668862382 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2270053070 con 0x55ae847616b0 2024-05-21T15:42:30.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.827+0000 7f226cff9640 1 -- 172.21.15.38:0/1668862382 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f227004b040 con 0x55ae847616b0 2024-05-21T15:42:30.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.827+0000 7f228667d740 10 client.17733 fetch_fsmap finished waiting for FSMap version 40 2024-05-21T15:42:30.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.827+0000 7f228667d740 10 client.17733 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-21T15:42:30.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.827+0000 7f228667d740 20 client.17733 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:30.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.827+0000 7f228667d740 1 --2- 172.21.15.38:0/1668862382 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55ae847163a0 0x55ae84ba3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:30.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.827+0000 7f228667d740 4 client.17733 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:42:30.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.827+0000 7f228667d740 1 -- 172.21.15.38:0/1668862382 --> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f22680261c0 con 0x55ae847163a0 2024-05-21T15:42:30.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.828+0000 7f227d573640 1 --2- 172.21.15.38:0/1668862382 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55ae847163a0 0x55ae84ba3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:30.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.829+0000 7f227d573640 1 --2- 172.21.15.38:0/1668862382 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55ae847163a0 0x55ae84ba3f00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=599c885101023cee server_cookie=9f07adb1e61cba1f in_seq=0 out_seq=0 2024-05-21T15:42:30.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.829+0000 7f226cff9640 10 client.17733 ms_handle_connect on v2:172.21.15.45:6836/977772487 2024-05-21T15:42:30.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.830+0000 7f226cff9640 1 -- 172.21.15.38:0/1668862382 <== mds.1 v2:172.21.15.45:6836/977772487 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f22680261c0 con 0x55ae847163a0 2024-05-21T15:42:30.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.830+0000 7f226cff9640 10 client.17733 handle_command_reply: tid=0 2024-05-21T15:42:30.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.830+0000 7f228667d740 1 client.17733 shutdown 2024-05-21T15:42:30.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.830+0000 7f228667d740 1 -- 172.21.15.38:0/1668862382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847616b0 msgr2=0x55ae84b2f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.830+0000 7f228667d740 1 --2- 172.21.15.38:0/1668862382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847616b0 0x55ae84b2f890 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f227003ba40 tx=0x7f227001a270 comp rx=0 tx=0).stop 2024-05-21T15:42:30.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.830+0000 7f228667d740 1 -- 172.21.15.38:0/1668862382 shutdown_connections 2024-05-21T15:42:30.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.830+0000 7f228667d740 1 --2- 172.21.15.38:0/1668862382 >> [v2:172.21.15.45:6836/977772487,v1:172.21.15.45:6837/977772487] conn(0x55ae847163a0 0x55ae84ba3f00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.830+0000 7f228667d740 1 --2- 172.21.15.38:0/1668862382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae84782390 0x55ae84b2fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.830+0000 7f226cff9640 0 client.17733 ms_handle_reset on v2:172.21.15.45:6836/977772487 2024-05-21T15:42:30.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.831+0000 7f228667d740 1 --2- 172.21.15.38:0/1668862382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae847616b0 0x55ae84b2f890 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.831+0000 7f228667d740 1 --2- 172.21.15.38:0/1668862382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae847594d0 0x55ae84b32d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.831+0000 7f228667d740 1 -- 172.21.15.38:0/1668862382 >> 172.21.15.38:0/1668862382 conn(0x55ae847764e0 msgr2=0x55ae84b25cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.833+0000 7f228667d740 1 -- 172.21.15.38:0/1668862382 shutdown_connections 2024-05-21T15:42:30.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.833+0000 7f228667d740 1 -- 172.21.15.38:0/1668862382 wait complete. 2024-05-21T15:42:30.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.833+0000 7f228667d740 20 client.17733 trim_cache size 0 max 16384 2024-05-21T15:42:30.833 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:42:30.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.840+0000 7f227f7d7640 1 -- 172.21.15.38:0/3030648628 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f22640a2d10 msgr2=0x7f22640a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.840+0000 7f227f7d7640 1 --2- 172.21.15.38:0/3030648628 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f22640a2d10 0x7f22640a5220 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x55ae84b22cd0 tx=0x7f2270000bc0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.840+0000 7f227f7d7640 1 -- 172.21.15.38:0/3030648628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227810efe0 msgr2=0x7f22781a6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:30.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.840+0000 7f227f7d7640 1 --2- 172.21.15.38:0/3030648628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227810efe0 0x7f22781a6f10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2260009fd0 tx=0x7f226000b8e0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.840+0000 7f227f7d7640 1 -- 172.21.15.38:0/3030648628 shutdown_connections 2024-05-21T15:42:30.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.840+0000 7f227f7d7640 1 --2- 172.21.15.38:0/3030648628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227810efe0 0x7f22781a6f10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.840+0000 7f227f7d7640 1 --2- 172.21.15.38:0/3030648628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f227810a2c0 0x7f227819ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.840+0000 7f227f7d7640 1 --2- 172.21.15.38:0/3030648628 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f22640a2d10 0x7f22640a5220 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.840+0000 7f227f7d7640 1 --2- 172.21.15.38:0/3030648628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22781093d0 0x7f227819fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:30.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.840+0000 7f227f7d7640 1 -- 172.21.15.38:0/3030648628 >> 172.21.15.38:0/3030648628 conn(0x7f227806f4a0 msgr2=0x7f227810cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:30.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.840+0000 7f227f7d7640 1 -- 172.21.15.38:0/3030648628 shutdown_connections 2024-05-21T15:42:30.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:30.840+0000 7f227f7d7640 1 -- 172.21.15.38:0/3030648628 wait complete. 2024-05-21T15:42:30.854 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:42:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.015+0000 7f0d8b6fe640 1 -- 172.21.15.38:0/895706458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d84102bc0 msgr2=0x7f0d84102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.015+0000 7f0d8b6fe640 1 --2- 172.21.15.38:0/895706458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d84102bc0 0x7f0d84102fd0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009900 tx=0x7f0d7802a410 comp rx=0 tx=0).stop 2024-05-21T15:42:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.015+0000 7f0d8b6fe640 1 -- 172.21.15.38:0/895706458 shutdown_connections 2024-05-21T15:42:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.015+0000 7f0d8b6fe640 1 --2- 172.21.15.38:0/895706458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d841078c0 0x7f0d84109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.015+0000 7f0d8b6fe640 1 --2- 172.21.15.38:0/895706458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d84102bc0 0x7f0d84102fd0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.015+0000 7f0d8b6fe640 1 --2- 172.21.15.38:0/895706458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d84101980 0x7f0d84102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.015+0000 7f0d8b6fe640 1 -- 172.21.15.38:0/895706458 >> 172.21.15.38:0/895706458 conn(0x7f0d840fcb70 msgr2=0x7f0d840ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.015+0000 7f0d8b6fe640 1 -- 172.21.15.38:0/895706458 shutdown_connections 2024-05-21T15:42:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.015+0000 7f0d8b6fe640 1 -- 172.21.15.38:0/895706458 wait complete. 2024-05-21T15:42:31.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.015+0000 7f0d8b6fe640 1 Processor -- start 2024-05-21T15:42:31.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8b6fe640 1 -- start start 2024-05-21T15:42:31.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8b6fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d84101980 0x7f0d84199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8b6fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d84102bc0 0x7f0d841997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8b6fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d841078c0 0x7f0d841a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8b6fe640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8410e700 con 0x7f0d84101980 2024-05-21T15:42:31.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8b6fe640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8410e580 con 0x7f0d841078c0 2024-05-21T15:42:31.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8b6fe640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8410e880 con 0x7f0d84102bc0 2024-05-21T15:42:31.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8949a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d84101980 0x7f0d84199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8949a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d84101980 0x7f0d84199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60108/0 (socket says 172.21.15.38:60108) 2024-05-21T15:42:31.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8949a640 1 -- 172.21.15.38:0/4222796412 learned_addr learned my addr 172.21.15.38:0/4222796412 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:31.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d88c99640 1 --2- 172.21.15.38:0/4222796412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d84102bc0 0x7f0d841997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/4222796412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d841078c0 0x7f0d841a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8949a640 1 -- 172.21.15.38:0/4222796412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d84102bc0 msgr2=0x7f0d841997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:31.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8949a640 1 --2- 172.21.15.38:0/4222796412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d84102bc0 0x7f0d841997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8949a640 1 -- 172.21.15.38:0/4222796412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d841078c0 msgr2=0x7f0d841a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:31.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8949a640 1 --2- 172.21.15.38:0/4222796412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d841078c0 0x7f0d841a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.016+0000 7f0d8949a640 1 -- 172.21.15.38:0/4222796412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d841a0db0 con 0x7f0d84101980 2024-05-21T15:42:31.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.017+0000 7f0d8949a640 1 --2- 172.21.15.38:0/4222796412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d84101980 0x7f0d84199270 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000e9f0 tx=0x7f0d8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:31.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.017+0000 7f0d767fc640 1 -- 172.21.15.38:0/4222796412 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000ccc0 con 0x7f0d84101980 2024-05-21T15:42:31.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.017+0000 7f0d8b6fe640 1 -- 172.21.15.38:0/4222796412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d841a10a0 con 0x7f0d84101980 2024-05-21T15:42:31.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.017+0000 7f0d767fc640 1 -- 172.21.15.38:0/4222796412 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d800035f0 con 0x7f0d84101980 2024-05-21T15:42:31.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.017+0000 7f0d767fc640 1 -- 172.21.15.38:0/4222796412 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000ce60 con 0x7f0d84101980 2024-05-21T15:42:31.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.017+0000 7f0d8b6fe640 1 -- 172.21.15.38:0/4222796412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d841a15f0 con 0x7f0d84101980 2024-05-21T15:42:31.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.020+0000 7f0d925a2740 1 Processor -- start 2024-05-21T15:42:31.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.020+0000 7f0d925a2740 1 -- start start 2024-05-21T15:42:31.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.020+0000 7f0d767fc640 1 -- 172.21.15.38:0/4222796412 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0d800037a0 con 0x7f0d84101980 2024-05-21T15:42:31.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.020+0000 7f0d925a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e232b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.020+0000 7f0d925a2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de6a4d0 0x56132e233080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.020+0000 7f0d925a2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de726b0 0x56132e239f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.020+0000 7f0d925a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56132e197c40 con 0x56132de93390 2024-05-21T15:42:31.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.020+0000 7f0d925a2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56132e13f580 con 0x56132de726b0 2024-05-21T15:42:31.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.020+0000 7f0d925a2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56132e193c00 con 0x56132de6a4d0 2024-05-21T15:42:31.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.020+0000 7f0d89c9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e232b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.020+0000 7f0d8949a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de6a4d0 0x56132e233080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.020+0000 7f0d89c9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e232b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60122/0 (socket says 172.21.15.38:60122) 2024-05-21T15:42:31.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.020+0000 7f0d89c9b640 1 -- 172.21.15.38:0/395976227 learned_addr learned my addr 172.21.15.38:0/395976227 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:31.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.020+0000 7f0d88c99640 1 --2- 172.21.15.38:0/395976227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de726b0 0x56132e239f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.021+0000 7f0d89c9b640 1 -- 172.21.15.38:0/395976227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de6a4d0 msgr2=0x56132e233080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:31.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.021+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/395976227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de6a4d0 0x56132e233080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.021+0000 7f0d89c9b640 1 -- 172.21.15.38:0/395976227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de726b0 msgr2=0x56132e239f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:31.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.021+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/395976227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de726b0 0x56132e239f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.021+0000 7f0d89c9b640 1 -- 172.21.15.38:0/395976227 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56132deab3f0 con 0x56132de93390 2024-05-21T15:42:31.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.021+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/395976227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e232b00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c010370 tx=0x7f0d7c010770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d08416a9ddf168d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:31.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.021+0000 7f0d767fc640 1 --2- 172.21.15.38:0/4222796412 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0d700a2b90 0x7f0d700a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.021+0000 7f0d6f7fe640 1 -- 172.21.15.38:0/395976227 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c016070 con 0x56132de93390 2024-05-21T15:42:31.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.021+0000 7f0d767fc640 1 -- 172.21.15.38:0/4222796412 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f0d800cf590 con 0x7f0d84101980 2024-05-21T15:42:31.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.021+0000 7f0d8949a640 1 --2- 172.21.15.38:0/4222796412 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0d700a2b90 0x7f0d700a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.021+0000 7f0d6f7fe640 1 -- 172.21.15.38:0/395976227 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d7c004d20 con 0x56132de93390 2024-05-21T15:42:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.022+0000 7f0d6f7fe640 1 -- 172.21.15.38:0/395976227 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0127f0 con 0x56132de93390 2024-05-21T15:42:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.023+0000 7f0d925a2740 1 -- 172.21.15.38:0/395976227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 msgr2=0x56132e232b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.023+0000 7f0d8949a640 1 --2- 172.21.15.38:0/4222796412 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0d700a2b90 0x7f0d700a50a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x56132e233ee0 tx=0x7f0d800d50e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.023+0000 7f0d925a2740 1 --2- 172.21.15.38:0/395976227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e232b00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c010370 tx=0x7f0d7c010770 comp rx=0 tx=0).stop 2024-05-21T15:42:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.023+0000 7f0d925a2740 1 -- 172.21.15.38:0/395976227 shutdown_connections 2024-05-21T15:42:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.023+0000 7f0d925a2740 1 --2- 172.21.15.38:0/395976227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de726b0 0x56132e239f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.023+0000 7f0d925a2740 1 --2- 172.21.15.38:0/395976227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de6a4d0 0x56132e233080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.023+0000 7f0d925a2740 1 --2- 172.21.15.38:0/395976227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e232b00 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.023+0000 7f0d925a2740 1 -- 172.21.15.38:0/395976227 >> 172.21.15.38:0/395976227 conn(0x56132de874e0 msgr2=0x56132e22e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.023+0000 7f0d925a2740 1 -- 172.21.15.38:0/395976227 shutdown_connections 2024-05-21T15:42:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.023+0000 7f0d925a2740 1 -- 172.21.15.38:0/395976227 wait complete. 2024-05-21T15:42:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.023+0000 7f0d925a2740 1 -- start start 2024-05-21T15:42:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.023+0000 7f0d925a2740 1 Processor -- start 2024-05-21T15:42:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d925a2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de6a4d0 0x56132e23c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d925a2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de726b0 0x56132e23ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d925a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e243df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d925a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56132e23aa70 con 0x56132de93390 2024-05-21T15:42:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d925a2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56132e152c40 con 0x56132de6a4d0 2024-05-21T15:42:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d925a2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56132e23abf0 con 0x56132de726b0 2024-05-21T15:42:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d925a2740 10 client.0 _ll_register_callbacks cb 0x56132deaabe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d89c9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e243df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d88c99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de726b0 0x56132e23ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d89c9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e243df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60136/0 (socket says 172.21.15.38:60136) 2024-05-21T15:42:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d89c9b640 1 -- 172.21.15.38:0/3165712026 learned_addr learned my addr 172.21.15.38:0/3165712026 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/3165712026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de6a4d0 0x56132e23c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d88c99640 1 -- 172.21.15.38:0/3165712026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de6a4d0 msgr2=0x56132e23c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d88c99640 1 --2- 172.21.15.38:0/3165712026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de6a4d0 0x56132e23c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d88c99640 1 -- 172.21.15.38:0/3165712026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 msgr2=0x56132e243df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d88c99640 1 --2- 172.21.15.38:0/3165712026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e243df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d88c99640 1 -- 172.21.15.38:0/3165712026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56132df757d0 con 0x56132de726b0 2024-05-21T15:42:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.024+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/3165712026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e243df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:42:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.025+0000 7f0d88c99640 1 --2- 172.21.15.38:0/3165712026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de726b0 0x56132e23ce90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0d78005ab0 tx=0x7f0d78032e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dadc9dcc2893373 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.025+0000 7f0d6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-21T15:42:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.025+0000 7f0d6cff9640 1 -- 172.21.15.38:0/3165712026 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78035040 con 0x56132de726b0 2024-05-21T15:42:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.025+0000 7f0d6cff9640 1 -- 172.21.15.38:0/3165712026 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f0d78031980 con 0x56132de726b0 2024-05-21T15:42:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.025+0000 7f0d925a2740 1 -- 172.21.15.38:0/3165712026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d7c004d20 con 0x56132de726b0 2024-05-21T15:42:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.025+0000 7f0d6cff9640 1 -- 172.21.15.38:0/3165712026 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0d78004040 con 0x56132de726b0 2024-05-21T15:42:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.025+0000 7f0d925a2740 10 client.27439 fetch_fsmap learned FSMap version 40 2024-05-21T15:42:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.025+0000 7f0d925a2740 1 -- 172.21.15.38:0/3165712026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56132e247100 con 0x56132de726b0 2024-05-21T15:42:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.025+0000 7f0d6cff9640 1 -- 172.21.15.38:0/3165712026 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d7c004d20 con 0x56132de726b0 2024-05-21T15:42:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.026+0000 7f0d6cff9640 1 -- 172.21.15.38:0/3165712026 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f0d78004860 con 0x56132de726b0 2024-05-21T15:42:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.026+0000 7f0d925a2740 10 client.27439 fetch_fsmap finished waiting for FSMap version 40 2024-05-21T15:42:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.026+0000 7f0d925a2740 10 client.27439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:42:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.026+0000 7f0d925a2740 20 client.27439 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.026+0000 7f0d925a2740 1 --2- 172.21.15.38:0/3165712026 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56132de51f60 0x56132e24a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.026+0000 7f0d925a2740 4 client.27439 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:42:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.026+0000 7f0d925a2740 1 -- 172.21.15.38:0/3165712026 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56132e1a5370 con 0x56132de51f60 2024-05-21T15:42:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.026+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/3165712026 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56132de51f60 0x56132e24a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.027+0000 7f0d6cff9640 10 client.27439 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.027+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/3165712026 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56132de51f60 0x56132e24a800 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77246e6b950ab007 server_cookie=c18d03bd9bb0644d in_seq=0 out_seq=0 2024-05-21T15:42:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d6cff9640 1 -- 172.21.15.38:0/3165712026 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56132e1a5370 con 0x56132de51f60 2024-05-21T15:42:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d6cff9640 10 client.27439 handle_command_reply: tid=0 2024-05-21T15:42:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d925a2740 1 client.27439 shutdown 2024-05-21T15:42:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d925a2740 1 -- 172.21.15.38:0/3165712026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de726b0 msgr2=0x56132e23ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d925a2740 1 --2- 172.21.15.38:0/3165712026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de726b0 0x56132e23ce90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0d78005ab0 tx=0x7f0d78032e40 comp rx=0 tx=0).stop 2024-05-21T15:42:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d925a2740 1 -- 172.21.15.38:0/3165712026 shutdown_connections 2024-05-21T15:42:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d925a2740 1 --2- 172.21.15.38:0/3165712026 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56132de51f60 0x56132e24a800 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d925a2740 1 --2- 172.21.15.38:0/3165712026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e243df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d6cff9640 0 client.27439 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d925a2740 1 --2- 172.21.15.38:0/3165712026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de726b0 0x56132e23ce90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d925a2740 1 --2- 172.21.15.38:0/3165712026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de6a4d0 0x56132e23c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d925a2740 1 -- 172.21.15.38:0/3165712026 >> 172.21.15.38:0/3165712026 conn(0x56132de874e0 msgr2=0x56132e22d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d925a2740 1 -- 172.21.15.38:0/3165712026 shutdown_connections 2024-05-21T15:42:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d925a2740 1 -- 172.21.15.38:0/3165712026 wait complete. 2024-05-21T15:42:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.029+0000 7f0d925a2740 20 client.27439 trim_cache size 0 max 16384 2024-05-21T15:42:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.045+0000 7f0d925a2740 1 Processor -- start 2024-05-21T15:42:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.045+0000 7f0d925a2740 1 -- start start 2024-05-21T15:42:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.045+0000 7f0d925a2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de6a4d0 0x56132e24eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.045+0000 7f0d925a2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de726b0 0x56132e243d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.045+0000 7f0d925a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e24f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.045+0000 7f0d925a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841a8160 con 0x56132de93390 2024-05-21T15:42:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.045+0000 7f0d925a2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841a9d60 con 0x56132de726b0 2024-05-21T15:42:31.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.045+0000 7f0d925a2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841a8890 con 0x56132de6a4d0 2024-05-21T15:42:31.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.045+0000 7f0d89c9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de6a4d0 0x56132e24eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.045+0000 7f0d89c9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de6a4d0 0x56132e24eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:39932/0 (socket says 172.21.15.38:39932) 2024-05-21T15:42:31.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.045+0000 7f0d89c9b640 1 -- 172.21.15.38:0/4018394514 learned_addr learned my addr 172.21.15.38:0/4018394514 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:31.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.045+0000 7f0d88c99640 1 --2- 172.21.15.38:0/4018394514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de726b0 0x56132e243d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.045+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/4018394514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e24f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.046+0000 7f0d89c9b640 1 -- 172.21.15.38:0/4018394514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de726b0 msgr2=0x56132e243d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:31.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.046+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/4018394514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de726b0 0x56132e243d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.046+0000 7f0d89c9b640 1 -- 172.21.15.38:0/4018394514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 msgr2=0x56132e24f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:31.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.046+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/4018394514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e24f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.046+0000 7f0d89c9b640 1 -- 172.21.15.38:0/4018394514 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56132e244250 con 0x56132de6a4d0 2024-05-21T15:42:31.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.046+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/4018394514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de6a4d0 0x56132e24eff0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c010370 tx=0x7f0d7c009600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3627ee2296673852 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:31.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.046+0000 7f0d6effd640 1 -- 172.21.15.38:0/4018394514 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c016040 con 0x56132de6a4d0 2024-05-21T15:42:31.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.046+0000 7f0d6effd640 1 -- 172.21.15.38:0/4018394514 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d7c003e50 con 0x56132de6a4d0 2024-05-21T15:42:31.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.046+0000 7f0d6effd640 1 -- 172.21.15.38:0/4018394514 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c004020 con 0x56132de6a4d0 2024-05-21T15:42:31.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.047+0000 7f0d925a2740 1 -- 172.21.15.38:0/4018394514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de6a4d0 msgr2=0x56132e24eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:31.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.047+0000 7f0d925a2740 1 --2- 172.21.15.38:0/4018394514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de6a4d0 0x56132e24eff0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c010370 tx=0x7f0d7c009600 comp rx=0 tx=0).stop 2024-05-21T15:42:31.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.047+0000 7f0d925a2740 1 -- 172.21.15.38:0/4018394514 shutdown_connections 2024-05-21T15:42:31.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.047+0000 7f0d925a2740 1 --2- 172.21.15.38:0/4018394514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de93390 0x56132e24f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.047+0000 7f0d925a2740 1 --2- 172.21.15.38:0/4018394514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de726b0 0x56132e243d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.047+0000 7f0d925a2740 1 --2- 172.21.15.38:0/4018394514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de6a4d0 0x56132e24eff0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.047+0000 7f0d925a2740 1 -- 172.21.15.38:0/4018394514 >> 172.21.15.38:0/4018394514 conn(0x56132de874e0 msgr2=0x56132e236c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:31.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.047+0000 7f0d925a2740 1 -- 172.21.15.38:0/4018394514 shutdown_connections 2024-05-21T15:42:31.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.047+0000 7f0d925a2740 1 -- 172.21.15.38:0/4018394514 wait complete. 2024-05-21T15:42:31.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.048+0000 7f0d925a2740 1 -- start start 2024-05-21T15:42:31.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.048+0000 7f0d925a2740 1 Processor -- start 2024-05-21T15:42:31.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.048+0000 7f0d925a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de6a4d0 0x56132e240e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.048+0000 7f0d925a2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de726b0 0x56132e2413d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.048+0000 7f0d925a2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de93390 0x56132e2422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.048+0000 7f0d925a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841a9690 con 0x56132de6a4d0 2024-05-21T15:42:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.048+0000 7f0d925a2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841a8f90 con 0x56132de93390 2024-05-21T15:42:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.048+0000 7f0d925a2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841a6170 con 0x56132de726b0 2024-05-21T15:42:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.048+0000 7f0d925a2740 10 client.0 _ll_register_callbacks cb 0x7f0d78009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:42:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.048+0000 7f0d89c9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de6a4d0 0x56132e240e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.048+0000 7f0d89c9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de6a4d0 0x56132e240e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60164/0 (socket says 172.21.15.38:60164) 2024-05-21T15:42:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.048+0000 7f0d89c9b640 1 -- 172.21.15.38:0/1984472515 learned_addr learned my addr 172.21.15.38:0/1984472515 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.048+0000 7f0d88c99640 1 --2- 172.21.15.38:0/1984472515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de726b0 0x56132e2413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/1984472515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de93390 0x56132e2422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d89c9b640 1 -- 172.21.15.38:0/1984472515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de726b0 msgr2=0x56132e2413d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/1984472515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de726b0 0x56132e2413d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d89c9b640 1 -- 172.21.15.38:0/1984472515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de93390 msgr2=0x56132e2422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:42:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/1984472515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de93390 0x56132e2422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d89c9b640 1 -- 172.21.15.38:0/1984472515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56132e242870 con 0x56132de6a4d0 2024-05-21T15:42:31.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d88c99640 1 --2- 172.21.15.38:0/1984472515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de726b0 0x56132e2413d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:42:31.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d89c9b640 1 --2- 172.21.15.38:0/1984472515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de6a4d0 0x56132e240e90 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c005cc0 tx=0x7f0d7c0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ace93ce1590b39 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:31.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-05-21T15:42:31.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d6cff9640 1 -- 172.21.15.38:0/1984472515 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c016030 con 0x56132de6a4d0 2024-05-21T15:42:31.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d925a2740 1 -- 172.21.15.38:0/1984472515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d7c003e50 con 0x56132de6a4d0 2024-05-21T15:42:31.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d6cff9640 1 -- 172.21.15.38:0/1984472515 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f0d7c005960 con 0x56132de6a4d0 2024-05-21T15:42:31.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d6cff9640 1 -- 172.21.15.38:0/1984472515 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0d7c003970 con 0x56132de6a4d0 2024-05-21T15:42:31.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d925a2740 10 client.17751 fetch_fsmap learned FSMap version 40 2024-05-21T15:42:31.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.049+0000 7f0d925a2740 1 -- 172.21.15.38:0/1984472515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56132e243af0 con 0x56132de6a4d0 2024-05-21T15:42:31.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.050+0000 7f0d6cff9640 1 -- 172.21.15.38:0/1984472515 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d7c003e50 con 0x56132de6a4d0 2024-05-21T15:42:31.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.050+0000 7f0d6cff9640 1 -- 172.21.15.38:0/1984472515 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f0d7c028040 con 0x56132de6a4d0 2024-05-21T15:42:31.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.050+0000 7f0d925a2740 10 client.17751 fetch_fsmap finished waiting for FSMap version 40 2024-05-21T15:42:31.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.050+0000 7f0d925a2740 10 client.17751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:42:31.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.050+0000 7f0d925a2740 20 client.17751 populate_metadata read hostname 'smithi038' 2024-05-21T15:42:31.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.050+0000 7f0d925a2740 1 --2- 172.21.15.38:0/1984472515 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56132de273a0 0x56132e2b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:31.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.050+0000 7f0d925a2740 4 client.17751 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:42:31.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.050+0000 7f0d925a2740 1 -- 172.21.15.38:0/1984472515 --> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56132e19f6c0 con 0x56132de273a0 2024-05-21T15:42:31.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.050+0000 7f0d88c99640 1 --2- 172.21.15.38:0/1984472515 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56132de273a0 0x56132e2b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:31.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.051+0000 7f0d6cff9640 10 client.17751 ms_handle_connect on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:31.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.051+0000 7f0d88c99640 1 --2- 172.21.15.38:0/1984472515 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56132de273a0 0x56132e2b4c60 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a82b896963e96eb0 server_cookie=4157dda859d4542f in_seq=0 out_seq=0 2024-05-21T15:42:31.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.051+0000 7f0d6cff9640 1 -- 172.21.15.38:0/1984472515 <== mds.0 v2:172.21.15.45:6838/2790647513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56132e19f6c0 con 0x56132de273a0 2024-05-21T15:42:31.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.051+0000 7f0d6cff9640 10 client.17751 handle_command_reply: tid=0 2024-05-21T15:42:31.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.051+0000 7f0d925a2740 1 client.17751 shutdown 2024-05-21T15:42:31.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.051+0000 7f0d925a2740 1 -- 172.21.15.38:0/1984472515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de6a4d0 msgr2=0x56132e240e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:31.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.051+0000 7f0d925a2740 1 --2- 172.21.15.38:0/1984472515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de6a4d0 0x56132e240e90 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c005cc0 tx=0x7f0d7c0097a0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.051+0000 7f0d925a2740 1 -- 172.21.15.38:0/1984472515 shutdown_connections 2024-05-21T15:42:31.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.051+0000 7f0d925a2740 1 --2- 172.21.15.38:0/1984472515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56132de93390 0x56132e2422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.051+0000 7f0d925a2740 1 --2- 172.21.15.38:0/1984472515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56132de726b0 0x56132e2413d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.051+0000 7f0d925a2740 1 --2- 172.21.15.38:0/1984472515 >> [v2:172.21.15.45:6838/2790647513,v1:172.21.15.45:6839/2790647513] conn(0x56132de273a0 0x56132e2b4c60 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.051+0000 7f0d925a2740 1 --2- 172.21.15.38:0/1984472515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56132de6a4d0 0x56132e240e90 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.051+0000 7f0d6cff9640 0 client.17751 ms_handle_reset on v2:172.21.15.45:6838/2790647513 2024-05-21T15:42:31.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.051+0000 7f0d925a2740 1 -- 172.21.15.38:0/1984472515 >> 172.21.15.38:0/1984472515 conn(0x56132de874e0 msgr2=0x56132e231330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:31.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.052+0000 7f0d925a2740 1 -- 172.21.15.38:0/1984472515 shutdown_connections 2024-05-21T15:42:31.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.052+0000 7f0d925a2740 1 -- 172.21.15.38:0/1984472515 wait complete. 2024-05-21T15:42:31.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.052+0000 7f0d925a2740 20 client.17751 trim_cache size 0 max 16384 2024-05-21T15:42:31.056 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-05-21T15:42:31.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.052+0000 7f0d8b6fe640 1 -- 172.21.15.38:0/4222796412 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0d700a2b90 msgr2=0x7f0d700a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:31.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.052+0000 7f0d8b6fe640 1 --2- 172.21.15.38:0/4222796412 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0d700a2b90 0x7f0d700a50a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x56132e233ee0 tx=0x7f0d800d50e0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.052+0000 7f0d8b6fe640 1 -- 172.21.15.38:0/4222796412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d84101980 msgr2=0x7f0d84199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:31.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.052+0000 7f0d8b6fe640 1 --2- 172.21.15.38:0/4222796412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d84101980 0x7f0d84199270 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000e9f0 tx=0x7f0d8000eeb0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.052+0000 7f0d8b6fe640 1 -- 172.21.15.38:0/4222796412 shutdown_connections 2024-05-21T15:42:31.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.052+0000 7f0d8b6fe640 1 --2- 172.21.15.38:0/4222796412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d841078c0 0x7f0d841a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.052+0000 7f0d8b6fe640 1 --2- 172.21.15.38:0/4222796412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d84102bc0 0x7f0d841997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.052+0000 7f0d8b6fe640 1 --2- 172.21.15.38:0/4222796412 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f0d700a2b90 0x7f0d700a50a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.052+0000 7f0d8b6fe640 1 --2- 172.21.15.38:0/4222796412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d84101980 0x7f0d84199270 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:31.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.052+0000 7f0d8b6fe640 1 -- 172.21.15.38:0/4222796412 >> 172.21.15.38:0/4222796412 conn(0x7f0d840fcb70 msgr2=0x7f0d841038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:31.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.053+0000 7f0d8b6fe640 1 -- 172.21.15.38:0/4222796412 shutdown_connections 2024-05-21T15:42:31.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:31.053+0000 7f0d8b6fe640 1 -- 172.21.15.38:0/4222796412 wait complete. 2024-05-21T15:42:32.063 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-05-21T15:42:32.063 INFO:tasks.fwd_scrub:done joining 2024-05-21T15:42:32.064 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-21T15:42:32.075 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:42:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.203+0000 7f60bfd02640 1 -- 172.21.15.38:0/3352030639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60b8101130 msgr2=0x7f60b8101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.203+0000 7f60bfd02640 1 --2- 172.21.15.38:0/3352030639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60b8101130 0x7f60b8101e30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009900 tx=0x7f60b402a410 comp rx=0 tx=0).stop 2024-05-21T15:42:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.203+0000 7f60bfd02640 1 -- 172.21.15.38:0/3352030639 shutdown_connections 2024-05-21T15:42:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.203+0000 7f60bfd02640 1 --2- 172.21.15.38:0/3352030639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b8107070 0x7f60b8109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.203+0000 7f60bfd02640 1 --2- 172.21.15.38:0/3352030639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b8102370 0x7f60b8102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.203+0000 7f60bfd02640 1 --2- 172.21.15.38:0/3352030639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60b8101130 0x7f60b8101e30 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.203+0000 7f60bfd02640 1 -- 172.21.15.38:0/3352030639 >> 172.21.15.38:0/3352030639 conn(0x7f60b80fc340 msgr2=0x7f60b80fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.204+0000 7f60bfd02640 1 -- 172.21.15.38:0/3352030639 shutdown_connections 2024-05-21T15:42:32.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.204+0000 7f60bfd02640 1 -- 172.21.15.38:0/3352030639 wait complete. 2024-05-21T15:42:32.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.205+0000 7f60bfd02640 1 Processor -- start 2024-05-21T15:42:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.205+0000 7f60bfd02640 1 -- start start 2024-05-21T15:42:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.205+0000 7f60bfd02640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b8101130 0x7f60b8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.205+0000 7f60bfd02640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b8102370 0x7f60b8199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.205+0000 7f60bfd02640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60b8107070 0x7f60b81a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.205+0000 7f60bfd02640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810c560 con 0x7f60b8107070 2024-05-21T15:42:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.205+0000 7f60bfd02640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810c3e0 con 0x7f60b8102370 2024-05-21T15:42:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.205+0000 7f60bfd02640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810c6e0 con 0x7f60b8101130 2024-05-21T15:42:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.205+0000 7f60be29f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60b8107070 0x7f60b81a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.205+0000 7f60bd29d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b8102370 0x7f60b8199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.205+0000 7f60be29f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60b8107070 0x7f60b81a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60182/0 (socket says 172.21.15.38:60182) 2024-05-21T15:42:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.205+0000 7f60be29f640 1 -- 172.21.15.38:0/3650681091 learned_addr learned my addr 172.21.15.38:0/3650681091 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.205+0000 7f60bda9e640 1 --2- 172.21.15.38:0/3650681091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b8101130 0x7f60b8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:32.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.206+0000 7f60bd29d640 1 -- 172.21.15.38:0/3650681091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b8101130 msgr2=0x7f60b8199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:32.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.206+0000 7f60bd29d640 1 --2- 172.21.15.38:0/3650681091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b8101130 0x7f60b8199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.206+0000 7f60bd29d640 1 -- 172.21.15.38:0/3650681091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60b8107070 msgr2=0x7f60b81a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:32.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.206+0000 7f60bd29d640 1 --2- 172.21.15.38:0/3650681091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60b8107070 0x7f60b81a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.206+0000 7f60bd29d640 1 -- 172.21.15.38:0/3650681091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b81a0d80 con 0x7f60b8102370 2024-05-21T15:42:32.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.206+0000 7f60be29f640 1 --2- 172.21.15.38:0/3650681091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60b8107070 0x7f60b81a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:42:32.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.206+0000 7f60bd29d640 1 --2- 172.21.15.38:0/3650681091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b8102370 0x7f60b8199780 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d720 tx=0x7f60a800dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:32.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.206+0000 7f60aeffd640 1 -- 172.21.15.38:0/3650681091 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8017070 con 0x7f60b8102370 2024-05-21T15:42:32.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.206+0000 7f60bfd02640 1 -- 172.21.15.38:0/3650681091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b81a1070 con 0x7f60b8102370 2024-05-21T15:42:32.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.206+0000 7f60bfd02640 1 -- 172.21.15.38:0/3650681091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b81a15c0 con 0x7f60b8102370 2024-05-21T15:42:32.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.207+0000 7f60aeffd640 1 -- 172.21.15.38:0/3650681091 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a800de30 con 0x7f60b8102370 2024-05-21T15:42:32.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.207+0000 7f60aeffd640 1 -- 172.21.15.38:0/3650681091 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a80047a0 con 0x7f60b8102370 2024-05-21T15:42:32.207 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:31 smithi053 ceph-mon[36496]: pgmap v594: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 1.5 MiB/s wr, 459 op/s 2024-05-21T15:42:32.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.208+0000 7f60bfd02640 1 -- 172.21.15.38:0/3650681091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f609c0052c0 con 0x7f60b8102370 2024-05-21T15:42:32.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.209+0000 7f60aeffd640 1 -- 172.21.15.38:0/3650681091 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f60a8004220 con 0x7f60b8102370 2024-05-21T15:42:32.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.210+0000 7f60aeffd640 1 --2- 172.21.15.38:0/3650681091 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f60a40a2c30 0x7f60a40a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:32.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.210+0000 7f60aeffd640 1 -- 172.21.15.38:0/3650681091 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f60a80cef90 con 0x7f60b8102370 2024-05-21T15:42:32.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.211+0000 7f60bda9e640 1 --2- 172.21.15.38:0/3650681091 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f60a40a2c30 0x7f60a40a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:32.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.212+0000 7f60bda9e640 1 --2- 172.21.15.38:0/3650681091 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f60a40a2c30 0x7f60a40a5140 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f60b402aa00 tx=0x7f60b40315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:32.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.214+0000 7f60aeffd640 1 -- 172.21.15.38:0/3650681091 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f60a8020020 con 0x7f60b8102370 2024-05-21T15:42:32.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:31 smithi045 ceph-mon[36409]: pgmap v594: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 1.5 MiB/s wr, 459 op/s 2024-05-21T15:42:32.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:31 smithi038 ceph-mon[31995]: pgmap v594: 97 pgs: 97 active+clean; 8.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 1.5 MiB/s wr, 459 op/s 2024-05-21T15:42:32.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.551+0000 7f60bfd02640 1 -- 172.21.15.38:0/3650681091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f609c005880 con 0x7f60b8102370 2024-05-21T15:42:32.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.555+0000 7f60aeffd640 1 -- 172.21.15.38:0/3650681091 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22704 (secure 0 0 0) 0x7f60a809be80 con 0x7f60b8102370 2024-05-21T15:42:32.553 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:42:32.553 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":90,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:34:23.996759+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:42:32.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.560+0000 7f60bfd02640 1 -- 172.21.15.38:0/3650681091 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f60a40a2c30 msgr2=0x7f60a40a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:32.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.560+0000 7f60bfd02640 1 --2- 172.21.15.38:0/3650681091 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f60a40a2c30 0x7f60a40a5140 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f60b402aa00 tx=0x7f60b40315b0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.560+0000 7f60bfd02640 1 -- 172.21.15.38:0/3650681091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b8102370 msgr2=0x7f60b8199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.560+0000 7f60bfd02640 1 --2- 172.21.15.38:0/3650681091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b8102370 0x7f60b8199780 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d720 tx=0x7f60a800dbe0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.560+0000 7f60bfd02640 1 -- 172.21.15.38:0/3650681091 shutdown_connections 2024-05-21T15:42:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.560+0000 7f60bfd02640 1 --2- 172.21.15.38:0/3650681091 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f60a40a2c30 0x7f60a40a5140 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.560+0000 7f60bfd02640 1 --2- 172.21.15.38:0/3650681091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60b8107070 0x7f60b81a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.560+0000 7f60bfd02640 1 --2- 172.21.15.38:0/3650681091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b8102370 0x7f60b8199780 secure :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d720 tx=0x7f60a800dbe0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.560+0000 7f60bfd02640 1 --2- 172.21.15.38:0/3650681091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60b8101130 0x7f60b8199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.560+0000 7f60bfd02640 1 -- 172.21.15.38:0/3650681091 >> 172.21.15.38:0/3650681091 conn(0x7f60b80fc340 msgr2=0x7f60b8103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.561+0000 7f60bfd02640 1 -- 172.21.15.38:0/3650681091 shutdown_connections 2024-05-21T15:42:32.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.561+0000 7f60bfd02640 1 -- 172.21.15.38:0/3650681091 wait complete. 2024-05-21T15:42:32.589 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:27:50.659367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:32:31.645647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:32:32.955451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:42:32.589 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:42:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.803+0000 7f3b09db5640 1 -- 172.21.15.38:0/693894454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b04109340 msgr2=0x7f3b04109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.803+0000 7f3b09db5640 1 --2- 172.21.15.38:0/693894454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b04109340 0x7f3b04109c00 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009860 tx=0x7f3af402a380 comp rx=0 tx=0).stop 2024-05-21T15:42:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.803+0000 7f3b09db5640 1 -- 172.21.15.38:0/693894454 shutdown_connections 2024-05-21T15:42:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.803+0000 7f3b09db5640 1 --2- 172.21.15.38:0/693894454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b0410ee60 0x7f3b0410f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.803+0000 7f3b09db5640 1 --2- 172.21.15.38:0/693894454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b0410a140 0x7f3b0410a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.803+0000 7f3b09db5640 1 --2- 172.21.15.38:0/693894454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b04109340 0x7f3b04109c00 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.803+0000 7f3b09db5640 1 -- 172.21.15.38:0/693894454 >> 172.21.15.38:0/693894454 conn(0x7f3b0406f3e0 msgr2=0x7f3b04071800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:32.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.804+0000 7f3b09db5640 1 -- 172.21.15.38:0/693894454 shutdown_connections 2024-05-21T15:42:32.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.804+0000 7f3b09db5640 1 -- 172.21.15.38:0/693894454 wait complete. 2024-05-21T15:42:32.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.805+0000 7f3b09db5640 1 Processor -- start 2024-05-21T15:42:32.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.805+0000 7f3b09db5640 1 -- start start 2024-05-21T15:42:32.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.806+0000 7f3b09db5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b04109340 0x7f3b0419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:32.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.806+0000 7f3b09db5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b0410a140 0x7f3b0419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:32.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.806+0000 7f3b09db5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b0410ee60 0x7f3b041a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:32.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.806+0000 7f3b09db5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3b04112b90 con 0x7f3b0410ee60 2024-05-21T15:42:32.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.806+0000 7f3b09db5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b04112a10 con 0x7f3b0410a140 2024-05-21T15:42:32.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.806+0000 7f3b09db5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3b04112d10 con 0x7f3b04109340 2024-05-21T15:42:32.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.806+0000 7f3b03fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b0410ee60 0x7f3b041a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:32.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.806+0000 7f3b03fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b0410ee60 0x7f3b041a6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60196/0 (socket says 172.21.15.38:60196) 2024-05-21T15:42:32.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.806+0000 7f3b03fff640 1 -- 172.21.15.38:0/3711735103 learned_addr learned my addr 172.21.15.38:0/3711735103 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:32.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.807+0000 7f3b02ffd640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b0410a140 0x7f3b0419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:32.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.807+0000 7f3b037fe640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b04109340 0x7f3b0419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:32.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.808+0000 7f3b03fff640 1 -- 172.21.15.38:0/3711735103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b04109340 msgr2=0x7f3b0419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:32.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.808+0000 7f3b03fff640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b04109340 0x7f3b0419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.808+0000 7f3b03fff640 1 -- 172.21.15.38:0/3711735103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b0410a140 msgr2=0x7f3b0419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:32.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.808+0000 7f3b03fff640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b0410a140 0x7f3b0419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:32.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.808+0000 7f3b03fff640 1 -- 172.21.15.38:0/3711735103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b041a7280 con 0x7f3b0410ee60 2024-05-21T15:42:32.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.808+0000 7f3b037fe640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b04109340 0x7f3b0419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:42:32.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.808+0000 7f3b02ffd640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b0410a140 0x7f3b0419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:42:32.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.809+0000 7f3b03fff640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b0410ee60 0x7f3b041a6c10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009fd0 tx=0x7f3afc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:32.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.809+0000 7f3b00ff9640 1 -- 172.21.15.38:0/3711735103 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc017070 con 0x7f3b0410ee60 2024-05-21T15:42:32.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.810+0000 7f3b00ff9640 1 -- 172.21.15.38:0/3711735103 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3afc003e50 con 0x7f3b0410ee60 2024-05-21T15:42:32.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.810+0000 7f3b09db5640 1 -- 172.21.15.38:0/3711735103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b041a7570 con 0x7f3b0410ee60 2024-05-21T15:42:32.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.810+0000 7f3b00ff9640 1 -- 172.21.15.38:0/3711735103 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0106c0 con 0x7f3b0410ee60 2024-05-21T15:42:32.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.810+0000 7f3b09db5640 1 -- 172.21.15.38:0/3711735103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b041a7a70 con 0x7f3b0410ee60 2024-05-21T15:42:32.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.812+0000 7f3b09db5640 1 -- 172.21.15.38:0/3711735103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b041a7cd0 con 0x7f3b0410ee60 2024-05-21T15:42:32.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.816+0000 7f3b00ff9640 1 -- 172.21.15.38:0/3711735103 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3afc003970 con 0x7f3b0410ee60 2024-05-21T15:42:32.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.818+0000 7f3b00ff9640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3aec0a2c60 0x7f3aec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:32.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.818+0000 7f3b00ff9640 1 -- 172.21.15.38:0/3711735103 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f3afc0cef60 con 0x7f3b0410ee60 2024-05-21T15:42:32.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.821+0000 7f3b037fe640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3aec0a2c60 0x7f3aec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:32.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.823+0000 7f3b037fe640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3aec0a2c60 0x7f3aec0a5170 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009830 tx=0x7f3af4030d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:32.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:32.828+0000 7f3b00ff9640 1 -- 172.21.15.38:0/3711735103 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3afc09be10 con 0x7f3b0410ee60 2024-05-21T15:42:33.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.191+0000 7f3b09db5640 1 -- 172.21.15.38:0/3711735103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3b041a7ea0 con 0x7f3b0410ee60 2024-05-21T15:42:33.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.192+0000 7f3b00ff9640 1 -- 172.21.15.38:0/3711735103 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f3afc09b710 con 0x7f3b0410ee60 2024-05-21T15:42:33.190 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-05-21T15:42:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.195+0000 7f3b09db5640 1 -- 172.21.15.38:0/3711735103 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3aec0a2c60 msgr2=0x7f3aec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.195+0000 7f3b09db5640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3aec0a2c60 0x7f3aec0a5170 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009830 tx=0x7f3af4030d00 comp rx=0 tx=0).stop 2024-05-21T15:42:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.195+0000 7f3b09db5640 1 -- 172.21.15.38:0/3711735103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b0410ee60 msgr2=0x7f3b041a6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.195+0000 7f3b09db5640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b0410ee60 0x7f3b041a6c10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009fd0 tx=0x7f3afc00b260 comp rx=0 tx=0).stop 2024-05-21T15:42:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.195+0000 7f3b09db5640 1 -- 172.21.15.38:0/3711735103 shutdown_connections 2024-05-21T15:42:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.195+0000 7f3b09db5640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f3aec0a2c60 0x7f3aec0a5170 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.195+0000 7f3b09db5640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b0410ee60 0x7f3b041a6c10 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.195+0000 7f3b09db5640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b0410a140 0x7f3b0419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.195+0000 7f3b09db5640 1 --2- 172.21.15.38:0/3711735103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b04109340 0x7f3b0419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.195+0000 7f3b09db5640 1 -- 172.21.15.38:0/3711735103 >> 172.21.15.38:0/3711735103 conn(0x7f3b0406f3e0 msgr2=0x7f3b0410ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.195+0000 7f3b09db5640 1 -- 172.21.15.38:0/3711735103 shutdown_connections 2024-05-21T15:42:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.195+0000 7f3b09db5640 1 -- 172.21.15.38:0/3711735103 wait complete. 2024-05-21T15:42:33.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:32 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:42:33.203 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:32 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3650681091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:42:33.205 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:32 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:42:33.205 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:32 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3650681091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:42:33.207 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:42:33.258 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:32 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:42:33.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:32 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3650681091' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:42:33.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.408+0000 7f68af9e3640 1 -- 172.21.15.38:0/1955182935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68a8102f50 msgr2=0x7f68a8103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:33.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.408+0000 7f68af9e3640 1 --2- 172.21.15.38:0/1955182935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68a8102f50 0x7f68a8103360 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f68a0013490 tx=0x7f68a0032b10 comp rx=0 tx=0).stop 2024-05-21T15:42:33.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.410+0000 7f68af9e3640 1 -- 172.21.15.38:0/1955182935 shutdown_connections 2024-05-21T15:42:33.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.410+0000 7f68af9e3640 1 --2- 172.21.15.38:0/1955182935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68a810a930 0x7f68a810ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:33.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.410+0000 7f68af9e3640 1 --2- 172.21.15.38:0/1955182935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68a8107e00 0x7f68a810a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:33.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.410+0000 7f68af9e3640 1 --2- 172.21.15.38:0/1955182935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68a8102f50 0x7f68a8103360 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:33.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.410+0000 7f68af9e3640 1 -- 172.21.15.38:0/1955182935 >> 172.21.15.38:0/1955182935 conn(0x7f68a80fc2f0 msgr2=0x7f68a80fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:33.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.412+0000 7f68af9e3640 1 -- 172.21.15.38:0/1955182935 shutdown_connections 2024-05-21T15:42:33.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.412+0000 7f68af9e3640 1 -- 172.21.15.38:0/1955182935 wait complete. 2024-05-21T15:42:33.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.413+0000 7f68af9e3640 1 Processor -- start 2024-05-21T15:42:33.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.413+0000 7f68af9e3640 1 -- start start 2024-05-21T15:42:33.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.413+0000 7f68af9e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68a8102f50 0x7f68a806fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:33.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.413+0000 7f68af9e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68a8107e00 0x7f68a8070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:33.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.413+0000 7f68af9e3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68a810a930 0x7f68a8072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:33.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.413+0000 7f68af9e3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8106580 con 0x7f68a8107e00 2024-05-21T15:42:33.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.413+0000 7f68af9e3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8106400 con 0x7f68a8102f50 2024-05-21T15:42:33.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.413+0000 7f68af9e3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8106700 con 0x7f68a810a930 2024-05-21T15:42:33.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.414+0000 7f68ad77f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68a8102f50 0x7f68a806fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:33.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.414+0000 7f68ad77f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68a8102f50 0x7f68a806fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.38:40692/0 (socket says 172.21.15.38:40692) 2024-05-21T15:42:33.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.414+0000 7f68ad77f640 1 -- 172.21.15.38:0/1543532643 learned_addr learned my addr 172.21.15.38:0/1543532643 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:33.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.414+0000 7f68acf7e640 1 --2- 172.21.15.38:0/1543532643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68a8107e00 0x7f68a8070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:33.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.414+0000 7f68adf80640 1 --2- 172.21.15.38:0/1543532643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68a810a930 0x7f68a8072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:33.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.414+0000 7f68ad77f640 1 -- 172.21.15.38:0/1543532643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68a810a930 msgr2=0x7f68a8072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:33.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.414+0000 7f68ad77f640 1 --2- 172.21.15.38:0/1543532643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68a810a930 0x7f68a8072fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:33.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.414+0000 7f68ad77f640 1 -- 172.21.15.38:0/1543532643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68a8107e00 msgr2=0x7f68a8070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:33.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.414+0000 7f68ad77f640 1 --2- 172.21.15.38:0/1543532643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68a8107e00 0x7f68a8070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:33.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.414+0000 7f68ad77f640 1 -- 172.21.15.38:0/1543532643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a8073630 con 0x7f68a8102f50 2024-05-21T15:42:33.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.415+0000 7f68ad77f640 1 --2- 172.21.15.38:0/1543532643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68a8102f50 0x7f68a806fb10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68a0013460 tx=0x7f68a0005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:33.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.415+0000 7f689e7fc640 1 -- 172.21.15.38:0/1543532643 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a003a040 con 0x7f68a8102f50 2024-05-21T15:42:33.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.417+0000 7f68af9e3640 1 -- 172.21.15.38:0/1543532643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a80738c0 con 0x7f68a8102f50 2024-05-21T15:42:33.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.417+0000 7f68af9e3640 1 -- 172.21.15.38:0/1543532643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a81b8ca0 con 0x7f68a8102f50 2024-05-21T15:42:33.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.417+0000 7f689e7fc640 1 -- 172.21.15.38:0/1543532643 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a00348f0 con 0x7f68a8102f50 2024-05-21T15:42:33.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.419+0000 7f68af9e3640 1 -- 172.21.15.38:0/1543532643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f688c0052c0 con 0x7f68a8102f50 2024-05-21T15:42:33.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.419+0000 7f689e7fc640 1 -- 172.21.15.38:0/1543532643 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a000c990 con 0x7f68a8102f50 2024-05-21T15:42:33.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.421+0000 7f689e7fc640 1 -- 172.21.15.38:0/1543532643 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68a003d070 con 0x7f68a8102f50 2024-05-21T15:42:33.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.422+0000 7f689e7fc640 1 --2- 172.21.15.38:0/1543532643 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f68940a2aa0 0x7f68940a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:33.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.422+0000 7f689e7fc640 1 -- 172.21.15.38:0/1543532643 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f68a00ef2d0 con 0x7f68a8102f50 2024-05-21T15:42:33.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.423+0000 7f68acf7e640 1 --2- 172.21.15.38:0/1543532643 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f68940a2aa0 0x7f68940a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:33.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.424+0000 7f68acf7e640 1 --2- 172.21.15.38:0/1543532643 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f68940a2aa0 0x7f68940a4fb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f68a8070bb0 tx=0x7f68a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:33.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.429+0000 7f689e7fc640 1 -- 172.21.15.38:0/1543532643 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f68a00bc180 con 0x7f68a8102f50 2024-05-21T15:42:33.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.788+0000 7f68af9e3640 1 -- 172.21.15.38:0/1543532643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f688c005880 con 0x7f68a8102f50 2024-05-21T15:42:33.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.789+0000 7f689e7fc640 1 -- 172.21.15.38:0/1543532643 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f68a00bba80 con 0x7f68a8102f50 2024-05-21T15:42:33.787 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-05-21T15:42:33.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.792+0000 7f68af9e3640 1 -- 172.21.15.38:0/1543532643 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f68940a2aa0 msgr2=0x7f68940a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:33.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.792+0000 7f68af9e3640 1 --2- 172.21.15.38:0/1543532643 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f68940a2aa0 0x7f68940a4fb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f68a8070bb0 tx=0x7f68a4005f90 comp rx=0 tx=0).stop 2024-05-21T15:42:33.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.792+0000 7f68af9e3640 1 -- 172.21.15.38:0/1543532643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68a8102f50 msgr2=0x7f68a806fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:33.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.792+0000 7f68af9e3640 1 --2- 172.21.15.38:0/1543532643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68a8102f50 0x7f68a806fb10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68a0013460 tx=0x7f68a0005d80 comp rx=0 tx=0).stop 2024-05-21T15:42:33.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.792+0000 7f68af9e3640 1 -- 172.21.15.38:0/1543532643 shutdown_connections 2024-05-21T15:42:33.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.792+0000 7f68af9e3640 1 --2- 172.21.15.38:0/1543532643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68a810a930 0x7f68a8072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:33.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.792+0000 7f68af9e3640 1 --2- 172.21.15.38:0/1543532643 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f68940a2aa0 0x7f68940a4fb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:33.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.792+0000 7f68af9e3640 1 --2- 172.21.15.38:0/1543532643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68a8107e00 0x7f68a8070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:33.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.792+0000 7f68af9e3640 1 --2- 172.21.15.38:0/1543532643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68a8102f50 0x7f68a806fb10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:33.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.792+0000 7f68af9e3640 1 -- 172.21.15.38:0/1543532643 >> 172.21.15.38:0/1543532643 conn(0x7f68a80fc2f0 msgr2=0x7f68a810add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:33.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.794+0000 7f68af9e3640 1 -- 172.21.15.38:0/1543532643 shutdown_connections 2024-05-21T15:42:33.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:33.796+0000 7f68af9e3640 1 -- 172.21.15.38:0/1543532643 wait complete. 2024-05-21T15:42:33.812 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:42:34.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.018+0000 7ff04272a640 1 -- 172.21.15.38:0/1190273039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff03c10a2c0 msgr2=0x7ff03c10a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:34.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.018+0000 7ff04272a640 1 --2- 172.21.15.38:0/1190273039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff03c10a2c0 0x7ff03c10a6d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009ca0 tx=0x7ff02c02a8e0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.018+0000 7ff04272a640 1 -- 172.21.15.38:0/1190273039 shutdown_connections 2024-05-21T15:42:34.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.018+0000 7ff04272a640 1 --2- 172.21.15.38:0/1190273039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff03c10efe0 0x7ff03c10f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.018+0000 7ff04272a640 1 --2- 172.21.15.38:0/1190273039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff03c10a2c0 0x7ff03c10a6d0 secure :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009ca0 tx=0x7ff02c02a8e0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.018+0000 7ff04272a640 1 --2- 172.21.15.38:0/1190273039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff03c1093d0 0x7ff03c109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.018+0000 7ff04272a640 1 -- 172.21.15.38:0/1190273039 >> 172.21.15.38:0/1190273039 conn(0x7ff03c06f4a0 msgr2=0x7ff03c071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:34.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.018+0000 7ff04272a640 1 -- 172.21.15.38:0/1190273039 shutdown_connections 2024-05-21T15:42:34.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.019+0000 7ff04272a640 1 -- 172.21.15.38:0/1190273039 wait complete. 2024-05-21T15:42:34.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.020+0000 7ff04272a640 1 Processor -- start 2024-05-21T15:42:34.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.020+0000 7ff04272a640 1 -- start start 2024-05-21T15:42:34.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.020+0000 7ff04272a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff03c1093d0 0x7ff03c1a8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:34.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.020+0000 7ff04272a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff03c10efe0 0x7ff03c1a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:34.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.020+0000 7ff04272a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff03c1ad6e0 0x7ff03c1afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:34.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.020+0000 7ff04272a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c11d610 con 0x7ff03c1093d0 2024-05-21T15:42:34.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.020+0000 7ff04272a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c11d490 con 0x7ff03c10efe0 2024-05-21T15:42:34.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.020+0000 7ff04272a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c11d790 con 0x7ff03c1ad6e0 2024-05-21T15:42:34.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.021+0000 7ff040cc7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff03c1ad6e0 0x7ff03c1afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:34.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.021+0000 7ff040cc7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff03c1ad6e0 0x7ff03c1afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:40032/0 (socket says 172.21.15.38:40032) 2024-05-21T15:42:34.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.021+0000 7ff040cc7640 1 -- 172.21.15.38:0/475560084 learned_addr learned my addr 172.21.15.38:0/475560084 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:34.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.021+0000 7ff03b7fe640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff03c10efe0 0x7ff03c1a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:34.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.022+0000 7ff03b7fe640 1 -- 172.21.15.38:0/475560084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff03c1ad6e0 msgr2=0x7ff03c1afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:34.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.022+0000 7ff03b7fe640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff03c1ad6e0 0x7ff03c1afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.022+0000 7ff03b7fe640 1 -- 172.21.15.38:0/475560084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff03c1093d0 msgr2=0x7ff03c1a8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:34.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.021+0000 7ff03bfff640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff03c1093d0 0x7ff03c1a8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:34.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.022+0000 7ff03b7fe640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff03c1093d0 0x7ff03c1a8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.022+0000 7ff03b7fe640 1 -- 172.21.15.38:0/475560084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c110c40 con 0x7ff03c10efe0 2024-05-21T15:42:34.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.022+0000 7ff03bfff640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff03c1093d0 0x7ff03c1a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:42:34.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.022+0000 7ff03b7fe640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff03c10efe0 0x7ff03c1a87e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff02c008000 tx=0x7ff02c003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:34.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.022+0000 7ff0397fa640 1 -- 172.21.15.38:0/475560084 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c030880 con 0x7ff03c10efe0 2024-05-21T15:42:34.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.023+0000 7ff04272a640 1 -- 172.21.15.38:0/475560084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03c110ed0 con 0x7ff03c10efe0 2024-05-21T15:42:34.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.023+0000 7ff04272a640 1 -- 172.21.15.38:0/475560084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03c1114a0 con 0x7ff03c10efe0 2024-05-21T15:42:34.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.024+0000 7ff0397fa640 1 -- 172.21.15.38:0/475560084 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff02c004040 con 0x7ff03c10efe0 2024-05-21T15:42:34.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.024+0000 7ff0397fa640 1 -- 172.21.15.38:0/475560084 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c03b660 con 0x7ff03c10efe0 2024-05-21T15:42:34.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.025+0000 7ff04272a640 1 -- 172.21.15.38:0/475560084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff01c0052c0 con 0x7ff03c10efe0 2024-05-21T15:42:34.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.027+0000 7ff0397fa640 1 -- 172.21.15.38:0/475560084 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff02c030a20 con 0x7ff03c10efe0 2024-05-21T15:42:34.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.029+0000 7ff0397fa640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff0280a2c90 0x7ff0280a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:34.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.029+0000 7ff0397fa640 1 -- 172.21.15.38:0/475560084 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7ff02c0f0740 con 0x7ff03c10efe0 2024-05-21T15:42:34.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.029+0000 7ff03bfff640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff0280a2c90 0x7ff0280a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:34.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.030+0000 7ff03bfff640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff0280a2c90 0x7ff0280a51a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff0240058c0 tx=0x7ff024005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:34.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.038+0000 7ff0397fa640 1 -- 172.21.15.38:0/475560084 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff02c0bd5f0 con 0x7ff03c10efe0 2024-05-21T15:42:34.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:33 smithi053 ceph-mon[36496]: pgmap v595: 97 pgs: 97 active+clean; 5.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 828 KiB/s wr, 348 op/s 2024-05-21T15:42:34.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:42:34.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:42:34.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:42:34.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:42:34.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:33 smithi053 ceph-mon[36496]: Adjusting osd_memory_target on smithi053 to 177.2M 2024-05-21T15:42:34.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:33 smithi053 ceph-mon[36496]: Unable to set osd_memory_target on smithi053 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-21T15:42:34.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:42:34.204 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:42:34.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:42:34.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:33 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3711735103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:42:34.205 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:33 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1543532643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:42:34.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:33 smithi045 ceph-mon[36409]: pgmap v595: 97 pgs: 97 active+clean; 5.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 828 KiB/s wr, 348 op/s 2024-05-21T15:42:34.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:42:34.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:42:34.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:42:34.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:42:34.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:33 smithi045 ceph-mon[36409]: Adjusting osd_memory_target on smithi053 to 177.2M 2024-05-21T15:42:34.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:33 smithi045 ceph-mon[36409]: Unable to set osd_memory_target on smithi053 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-21T15:42:34.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:42:34.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:42:34.259 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:42:34.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:33 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3711735103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:42:34.260 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:33 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1543532643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:42:34.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:33 smithi038 ceph-mon[31995]: pgmap v595: 97 pgs: 97 active+clean; 5.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 828 KiB/s wr, 348 op/s 2024-05-21T15:42:34.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:42:34.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:42:34.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:42:34.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:42:34.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:33 smithi038 ceph-mon[31995]: Adjusting osd_memory_target on smithi053 to 177.2M 2024-05-21T15:42:34.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:33 smithi038 ceph-mon[31995]: Unable to set osd_memory_target on smithi053 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-21T15:42:34.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:42:34.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:42:34.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:42:34.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:33 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3711735103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:42:34.274 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:33 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1543532643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:42:34.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.359+0000 7ff04272a640 1 -- 172.21.15.38:0/475560084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff01c005dc0 con 0x7ff03c10efe0 2024-05-21T15:42:34.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.361+0000 7ff0397fa640 1 -- 172.21.15.38:0/475560084 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7ff02c0bcef0 con 0x7ff03c10efe0 2024-05-21T15:42:34.358 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-05-21T15:42:34.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.365+0000 7ff04272a640 1 -- 172.21.15.38:0/475560084 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff0280a2c90 msgr2=0x7ff0280a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:34.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.365+0000 7ff04272a640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff0280a2c90 0x7ff0280a51a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff0240058c0 tx=0x7ff024005d50 comp rx=0 tx=0).stop 2024-05-21T15:42:34.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.365+0000 7ff04272a640 1 -- 172.21.15.38:0/475560084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff03c10efe0 msgr2=0x7ff03c1a87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:34.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.365+0000 7ff04272a640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff03c10efe0 0x7ff03c1a87e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff02c008000 tx=0x7ff02c003ba0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.365+0000 7ff04272a640 1 -- 172.21.15.38:0/475560084 shutdown_connections 2024-05-21T15:42:34.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.365+0000 7ff04272a640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff03c1ad6e0 0x7ff03c1afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.365+0000 7ff04272a640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff03c10efe0 0x7ff03c1a87e0 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff02c008000 tx=0x7ff02c003ba0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.365+0000 7ff04272a640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7ff0280a2c90 0x7ff0280a51a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.365+0000 7ff04272a640 1 --2- 172.21.15.38:0/475560084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff03c1093d0 0x7ff03c1a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.365+0000 7ff04272a640 1 -- 172.21.15.38:0/475560084 >> 172.21.15.38:0/475560084 conn(0x7ff03c06f4a0 msgr2=0x7ff03c10afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:34.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.366+0000 7ff04272a640 1 -- 172.21.15.38:0/475560084 shutdown_connections 2024-05-21T15:42:34.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.366+0000 7ff04272a640 1 -- 172.21.15.38:0/475560084 wait complete. 2024-05-21T15:42:34.381 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-21T15:42:34.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.577+0000 7f9056b8b640 1 -- 172.21.15.38:0/868002377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f905010a300 msgr2=0x7f905010a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:34.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.577+0000 7f9056b8b640 1 --2- 172.21.15.38:0/868002377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f905010a300 0x7f905010a710 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f903c009900 tx=0x7f903c02a400 comp rx=0 tx=0).stop 2024-05-21T15:42:34.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.582+0000 7f9056b8b640 1 -- 172.21.15.38:0/868002377 shutdown_connections 2024-05-21T15:42:34.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.582+0000 7f9056b8b640 1 --2- 172.21.15.38:0/868002377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f905010f020 0x7f905010f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.582+0000 7f9056b8b640 1 --2- 172.21.15.38:0/868002377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f905010a300 0x7f905010a710 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.582+0000 7f9056b8b640 1 --2- 172.21.15.38:0/868002377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90501093d0 0x7f9050109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.582+0000 7f9056b8b640 1 -- 172.21.15.38:0/868002377 >> 172.21.15.38:0/868002377 conn(0x7f905006f4e0 msgr2=0x7f9050071970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:34.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.583+0000 7f9056b8b640 1 -- 172.21.15.38:0/868002377 shutdown_connections 2024-05-21T15:42:34.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.587+0000 7f9056b8b640 1 -- 172.21.15.38:0/868002377 wait complete. 2024-05-21T15:42:34.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.588+0000 7f9056b8b640 1 Processor -- start 2024-05-21T15:42:34.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.588+0000 7f9056b8b640 1 -- start start 2024-05-21T15:42:34.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.589+0000 7f9056b8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90501093d0 0x7f9050117ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:34.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.589+0000 7f9056b8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f905010a300 0x7f9050118400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:34.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.589+0000 7f9056b8b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f905010f020 0x7f9050119360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:34.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.589+0000 7f9056b8b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f905011d620 con 0x7f90501093d0 2024-05-21T15:42:34.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.589+0000 7f9056b8b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f905011d4a0 con 0x7f905010a300 2024-05-21T15:42:34.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.589+0000 7f9056b8b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f905011d7a0 con 0x7f905010f020 2024-05-21T15:42:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.589+0000 7f9055128640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f905010f020 0x7f9050119360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.589+0000 7f9055128640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f905010f020 0x7f9050119360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.38:40036/0 (socket says 172.21.15.38:40036) 2024-05-21T15:42:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.589+0000 7f9055128640 1 -- 172.21.15.38:0/1967675948 learned_addr learned my addr 172.21.15.38:0/1967675948 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:42:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.590+0000 7f904ffff640 1 --2- 172.21.15.38:0/1967675948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f905010a300 0x7f9050118400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.590+0000 7f904ffff640 1 -- 172.21.15.38:0/1967675948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f905010f020 msgr2=0x7f9050119360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.590+0000 7f904ffff640 1 --2- 172.21.15.38:0/1967675948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f905010f020 0x7f9050119360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.590+0000 7f904ffff640 1 -- 172.21.15.38:0/1967675948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90501093d0 msgr2=0x7f9050117ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:42:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.590+0000 7f904ffff640 1 --2- 172.21.15.38:0/1967675948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90501093d0 0x7f9050117ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.590+0000 7f904ffff640 1 -- 172.21.15.38:0/1967675948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90501199d0 con 0x7f905010a300 2024-05-21T15:42:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.591+0000 7f904ffff640 1 --2- 172.21.15.38:0/1967675948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f905010a300 0x7f9050118400 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f903c02a9f0 tx=0x7f903c032c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.591+0000 7f904dffb640 1 -- 172.21.15.38:0/1967675948 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c03f030 con 0x7f905010a300 2024-05-21T15:42:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.591+0000 7f904dffb640 1 -- 172.21.15.38:0/1967675948 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f903c003ad0 con 0x7f905010a300 2024-05-21T15:42:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.591+0000 7f904dffb640 1 -- 172.21.15.38:0/1967675948 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c035070 con 0x7f905010a300 2024-05-21T15:42:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.591+0000 7f9056b8b640 1 -- 172.21.15.38:0/1967675948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9050119c60 con 0x7f905010a300 2024-05-21T15:42:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.592+0000 7f9056b8b640 1 -- 172.21.15.38:0/1967675948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905011a010 con 0x7f905010a300 2024-05-21T15:42:34.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.595+0000 7f904dffb640 1 -- 172.21.15.38:0/1967675948 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f903c042030 con 0x7f905010a300 2024-05-21T15:42:34.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.596+0000 7f904dffb640 1 --2- 172.21.15.38:0/1967675948 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f90400a2e70 0x7f90400a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:42:34.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.596+0000 7f904dffb640 1 -- 172.21.15.38:0/1967675948 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f903c0ef500 con 0x7f905010a300 2024-05-21T15:42:34.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.596+0000 7f9054927640 1 --2- 172.21.15.38:0/1967675948 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f90400a2e70 0x7f90400a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:42:34.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.596+0000 7f9056b8b640 1 -- 172.21.15.38:0/1967675948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90340052c0 con 0x7f905010a300 2024-05-21T15:42:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.599+0000 7f9054927640 1 --2- 172.21.15.38:0/1967675948 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f90400a2e70 0x7f90400a5380 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9050108c10 tx=0x7f9044005b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:42:34.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:34.603+0000 7f904dffb640 1 -- 172.21.15.38:0/1967675948 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f903c0bd460 con 0x7f905010a300 2024-05-21T15:42:35.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:35.021+0000 7f9056b8b640 1 -- 172.21.15.38:0/1967675948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f90340055b0 con 0x7f905010a300 2024-05-21T15:42:35.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:35.031+0000 7f904dffb640 1 -- 172.21.15.38:0/1967675948 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740371 (secure 0 0 0) 0x7f903c0bcd60 con 0x7f905010a300 2024-05-21T15:42:35.039 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:42:35.039 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:42:35.040 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 18, 2024-05-21T15:42:35.040 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.040 INFO:teuthology.orchestra.run.smithi038.stdout: "active_gid": 14152, 2024-05-21T15:42:35.040 INFO:teuthology.orchestra.run.smithi038.stdout: "active_name": "x", 2024-05-21T15:42:35.040 INFO:teuthology.orchestra.run.smithi038.stdout: "active_addrs": { 2024-05-21T15:42:35.040 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-05-21T15:42:35.040 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.040 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-05-21T15:42:35.040 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:6800", 2024-05-21T15:42:35.040 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1295226559 2024-05-21T15:42:35.040 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.040 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.041 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-05-21T15:42:35.041 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:6801", 2024-05-21T15:42:35.041 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1295226559 2024-05-21T15:42:35.041 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.041 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-05-21T15:42:35.041 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.041 INFO:teuthology.orchestra.run.smithi038.stdout: "active_addr": "172.21.15.38:6801/1295226559", 2024-05-21T15:42:35.041 INFO:teuthology.orchestra.run.smithi038.stdout: "active_change": "2024-05-21T15:25:22.118108+0000", 2024-05-21T15:42:35.041 INFO:teuthology.orchestra.run.smithi038.stdout: "active_mgr_features": 4540701547738038271, 2024-05-21T15:42:35.041 INFO:teuthology.orchestra.run.smithi038.stdout: "available": true, 2024-05-21T15:42:35.041 INFO:teuthology.orchestra.run.smithi038.stdout: "standbys": [ 2024-05-21T15:42:35.041 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.041 INFO:teuthology.orchestra.run.smithi038.stdout: "gid": 24118, 2024-05-21T15:42:35.042 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "z", 2024-05-21T15:42:35.042 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr_features": 4540701547738038271, 2024-05-21T15:42:35.042 INFO:teuthology.orchestra.run.smithi038.stdout: "available_modules": [ 2024-05-21T15:42:35.042 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.042 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "alerts", 2024-05-21T15:42:35.042 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.042 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.042 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.042 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:35.042 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:35.042 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.042 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.043 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.043 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:35.043 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.043 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.043 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.043 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How frequently to reexamine health status", 2024-05-21T15:42:35.043 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.043 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.043 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.043 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.043 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.043 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.043 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.044 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.044 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.044 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.044 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.044 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.044 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.044 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.044 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.044 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.044 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.044 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.044 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.044 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.044 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.045 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.045 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.045 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.045 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.045 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.045 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.045 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.045 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.045 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.045 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.045 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.045 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.045 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.046 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.046 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.046 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.046 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.046 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.046 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.046 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.046 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.046 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.046 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.046 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.046 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.047 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.047 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.047 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.047 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.047 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.047 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.047 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.047 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.047 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.047 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.047 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.047 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.047 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.048 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.048 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.048 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.048 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.048 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.048 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.048 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.048 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.048 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.048 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.048 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.048 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.048 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.048 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.049 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.049 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_destination": { 2024-05-21T15:42:35.049 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_destination", 2024-05-21T15:42:35.049 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.049 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.049 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.049 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.049 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.049 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.049 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.049 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email address to send alerts to", 2024-05-21T15:42:35.049 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.049 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.050 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.050 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.050 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_from_name": { 2024-05-21T15:42:35.050 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_from_name", 2024-05-21T15:42:35.050 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.050 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.050 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.050 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "Ceph", 2024-05-21T15:42:35.050 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.050 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.050 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.050 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email From: name", 2024-05-21T15:42:35.051 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.051 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.051 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.051 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.051 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_host": { 2024-05-21T15:42:35.051 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_host", 2024-05-21T15:42:35.051 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.051 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.051 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.051 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.051 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.051 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.051 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.051 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP server", 2024-05-21T15:42:35.052 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.052 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.052 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.052 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.052 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_password": { 2024-05-21T15:42:35.052 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_password", 2024-05-21T15:42:35.052 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.052 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.052 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.052 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.052 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.052 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.052 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.053 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Password to authenticate with", 2024-05-21T15:42:35.053 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.053 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.053 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.053 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.053 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_port": { 2024-05-21T15:42:35.053 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_port", 2024-05-21T15:42:35.053 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.053 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.053 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.053 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "465", 2024-05-21T15:42:35.054 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.054 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.054 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.054 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP port", 2024-05-21T15:42:35.054 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.054 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.054 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.054 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.054 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_sender": { 2024-05-21T15:42:35.054 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_sender", 2024-05-21T15:42:35.054 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.054 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.054 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.055 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.055 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.055 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.055 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.055 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP envelope sender", 2024-05-21T15:42:35.055 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.055 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.055 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.055 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.055 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_ssl": { 2024-05-21T15:42:35.055 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_ssl", 2024-05-21T15:42:35.055 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.056 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.056 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.056 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.056 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.056 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.056 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.056 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-21T15:42:35.056 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.056 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.056 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.056 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.056 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_user": { 2024-05-21T15:42:35.056 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_user", 2024-05-21T15:42:35.057 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.057 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.057 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.057 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.057 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.057 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.057 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.057 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "User to authenticate as", 2024-05-21T15:42:35.057 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.057 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.057 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.057 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.057 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.058 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.058 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.058 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "balancer", 2024-05-21T15:42:35.058 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.058 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.058 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.058 INFO:teuthology.orchestra.run.smithi038.stdout: "active": { 2024-05-21T15:42:35.058 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "active", 2024-05-21T15:42:35.058 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.058 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.058 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.058 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.058 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.058 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.059 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.059 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically balance PGs across cluster", 2024-05-21T15:42:35.059 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.059 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.059 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.059 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.059 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_time": { 2024-05-21T15:42:35.059 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_time", 2024-05-21T15:42:35.059 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.059 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.059 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.059 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0000", 2024-05-21T15:42:35.059 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.060 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.060 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.060 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "beginning time of day to automatically balance", 2024-05-21T15:42:35.060 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T15:42:35.060 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.060 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.060 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.060 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_weekday": { 2024-05-21T15:42:35.060 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_weekday", 2024-05-21T15:42:35.060 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:35.060 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.060 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.060 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:35.060 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-05-21T15:42:35.061 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-05-21T15:42:35.061 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.061 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-21T15:42:35.061 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T15:42:35.061 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.061 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.061 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.061 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_max_iterations": { 2024-05-21T15:42:35.061 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_max_iterations", 2024-05-21T15:42:35.061 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:35.061 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.061 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.061 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "25", 2024-05-21T15:42:35.062 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-05-21T15:42:35.062 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "250", 2024-05-21T15:42:35.062 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.062 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-21T15:42:35.062 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.062 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.062 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.062 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.062 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_metrics": { 2024-05-21T15:42:35.062 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_metrics", 2024-05-21T15:42:35.062 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.062 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.062 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.063 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "pgs,objects,bytes", 2024-05-21T15:42:35.063 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.063 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.063 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.063 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-21T15:42:35.063 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-21T15:42:35.063 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.063 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.063 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.063 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_step": { 2024-05-21T15:42:35.063 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_step", 2024-05-21T15:42:35.063 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.063 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.063 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.064 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.5", 2024-05-21T15:42:35.064 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0.001", 2024-05-21T15:42:35.064 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "0.999", 2024-05-21T15:42:35.064 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.064 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "aggressiveness of optimization", 2024-05-21T15:42:35.064 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-21T15:42:35.064 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.064 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.064 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.064 INFO:teuthology.orchestra.run.smithi038.stdout: "end_time": { 2024-05-21T15:42:35.064 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_time", 2024-05-21T15:42:35.064 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.064 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2359", 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ending time of day to automatically balance", 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: "end_weekday": { 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_weekday", 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.065 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.066 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:35.066 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-05-21T15:42:35.066 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-05-21T15:42:35.066 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.066 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-21T15:42:35.066 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T15:42:35.066 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.066 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.066 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.066 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.066 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.066 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.066 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.067 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.068 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.068 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.068 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.068 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.068 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.068 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.068 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.068 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.068 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.068 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.068 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.068 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.070 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.070 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.070 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.070 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.070 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.070 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.070 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.070 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.070 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.070 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.070 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.070 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.070 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.071 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.071 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.071 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.071 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.071 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.071 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.071 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.071 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.071 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.071 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.071 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.071 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.071 INFO:teuthology.orchestra.run.smithi038.stdout: "min_score": { 2024-05-21T15:42:35.071 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_score", 2024-05-21T15:42:35.072 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.072 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.072 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.072 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:35.072 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.072 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.072 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.072 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-21T15:42:35.072 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.072 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.072 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.072 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.072 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-05-21T15:42:35.072 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-05-21T15:42:35.073 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.073 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.073 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.073 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "upmap", 2024-05-21T15:42:35.073 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.073 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.073 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.073 INFO:teuthology.orchestra.run.smithi038.stdout: "crush-compat", 2024-05-21T15:42:35.073 INFO:teuthology.orchestra.run.smithi038.stdout: "none", 2024-05-21T15:42:35.073 INFO:teuthology.orchestra.run.smithi038.stdout: "read", 2024-05-21T15:42:35.073 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap", 2024-05-21T15:42:35.073 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap-read" 2024-05-21T15:42:35.073 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.073 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Balancer mode", 2024-05-21T15:42:35.074 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.074 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.074 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.074 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.074 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_ids": { 2024-05-21T15:42:35.074 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_ids", 2024-05-21T15:42:35.074 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.074 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.074 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.074 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.074 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.074 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.074 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-21T15:42:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.075 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.076 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-21T15:42:35.076 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.076 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.076 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.076 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.076 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_deviation": { 2024-05-21T15:42:35.076 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_deviation", 2024-05-21T15:42:35.076 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.076 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.076 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.076 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-05-21T15:42:35.076 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-05-21T15:42:35.076 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.076 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.077 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-21T15:42:35.077 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-21T15:42:35.077 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.077 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.077 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.077 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_optimizations": { 2024-05-21T15:42:35.077 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_optimizations", 2024-05-21T15:42:35.077 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:35.077 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.077 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.077 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.077 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.077 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.077 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.078 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-21T15:42:35.078 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.078 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.078 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.078 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.078 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.078 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.078 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.078 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm", 2024-05-21T15:42:35.078 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.078 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.078 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.079 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_down_multiplier": { 2024-05-21T15:42:35.079 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_down_multiplier", 2024-05-21T15:42:35.079 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.079 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.079 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.079 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-05-21T15:42:35.079 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.079 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.079 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.079 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-21T15:42:35.079 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.079 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.079 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.079 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.080 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_refresh_rate": { 2024-05-21T15:42:35.080 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_refresh_rate", 2024-05-21T15:42:35.080 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.080 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.080 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.080 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "20", 2024-05-21T15:42:35.080 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.080 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.080 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.080 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-21T15:42:35.080 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.080 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.080 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.080 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.081 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_starting_port": { 2024-05-21T15:42:35.081 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_starting_port", 2024-05-21T15:42:35.081 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.081 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.081 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.081 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4721", 2024-05-21T15:42:35.081 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.081 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.081 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.081 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-21T15:42:35.081 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.081 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.081 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.082 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.082 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_ptrace": { 2024-05-21T15:42:35.082 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_ptrace", 2024-05-21T15:42:35.082 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.082 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.082 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.082 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.082 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.082 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.082 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.082 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-21T15:42:35.082 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_interval": { 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_interval", 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.083 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.084 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.084 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.084 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_memory_target_ratio": { 2024-05-21T15:42:35.084 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_memory_target_ratio", 2024-05-21T15:42:35.084 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.084 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.084 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.084 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.7", 2024-05-21T15:42:35.084 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.084 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.084 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.084 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-21T15:42:35.084 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.085 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.085 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.085 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.085 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm_log_destination": { 2024-05-21T15:42:35.085 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm_log_destination", 2024-05-21T15:42:35.085 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.085 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.085 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.085 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.085 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.085 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.085 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.085 INFO:teuthology.orchestra.run.smithi038.stdout: "file", 2024-05-21T15:42:35.086 INFO:teuthology.orchestra.run.smithi038.stdout: "file,syslog", 2024-05-21T15:42:35.086 INFO:teuthology.orchestra.run.smithi038.stdout: "syslog" 2024-05-21T15:42:35.086 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.086 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-21T15:42:35.086 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.086 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.086 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.086 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.086 INFO:teuthology.orchestra.run.smithi038.stdout: "cgroups_split": { 2024-05-21T15:42:35.086 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cgroups_split", 2024-05-21T15:42:35.086 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.086 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.086 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.087 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.087 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.087 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.087 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.087 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-21T15:42:35.087 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.087 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.087 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.087 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.087 INFO:teuthology.orchestra.run.smithi038.stdout: "config_checks_enabled": { 2024-05-21T15:42:35.087 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_checks_enabled", 2024-05-21T15:42:35.087 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.087 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.087 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.088 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.088 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.088 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.088 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.088 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-21T15:42:35.088 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.088 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.088 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.088 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.088 INFO:teuthology.orchestra.run.smithi038.stdout: "config_dashboard": { 2024-05-21T15:42:35.088 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_dashboard", 2024-05-21T15:42:35.088 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.088 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.089 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.089 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.089 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.089 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.089 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.089 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-21T15:42:35.089 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.089 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.089 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.089 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.089 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_alertmanager": { 2024-05-21T15:42:35.089 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_alertmanager", 2024-05-21T15:42:35.089 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.089 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.090 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.090 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-21T15:42:35.090 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.090 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.090 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.090 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-05-21T15:42:35.090 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.090 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.090 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.090 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.090 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_base": { 2024-05-21T15:42:35.090 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_base", 2024-05-21T15:42:35.090 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.091 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.091 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.091 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-21T15:42:35.091 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.091 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.091 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.091 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Container image name, without the tag", 2024-05-21T15:42:35.091 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.091 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.091 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.091 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.091 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_elasticsearch": { 2024-05-21T15:42:35.091 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_elasticsearch", 2024-05-21T15:42:35.092 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.092 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.092 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.092 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-21T15:42:35.092 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.092 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.092 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.092 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "elasticsearch container image", 2024-05-21T15:42:35.092 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.092 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.092 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.092 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.092 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_grafana": { 2024-05-21T15:42:35.092 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_grafana", 2024-05-21T15:42:35.093 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.093 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.093 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.093 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-21T15:42:35.093 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.093 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.093 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.093 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-05-21T15:42:35.093 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.093 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.093 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.093 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.093 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_haproxy": { 2024-05-21T15:42:35.093 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_haproxy", 2024-05-21T15:42:35.094 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.094 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.094 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.094 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-21T15:42:35.094 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.094 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.094 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.094 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "HAproxy container image", 2024-05-21T15:42:35.094 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.094 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.094 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.094 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.095 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_agent": { 2024-05-21T15:42:35.095 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_agent", 2024-05-21T15:42:35.095 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.095 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.095 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.095 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-21T15:42:35.095 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.095 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.095 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.095 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger agent container image", 2024-05-21T15:42:35.095 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.095 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.095 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.096 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.096 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_collector": { 2024-05-21T15:42:35.096 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_collector", 2024-05-21T15:42:35.096 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.096 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.096 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.096 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-21T15:42:35.096 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.096 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.096 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.096 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger collector container image", 2024-05-21T15:42:35.096 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.096 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.096 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.097 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.097 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_query": { 2024-05-21T15:42:35.097 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_query", 2024-05-21T15:42:35.097 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.097 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.097 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.097 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-21T15:42:35.097 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.097 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.097 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.097 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger query container image", 2024-05-21T15:42:35.097 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.097 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.098 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.098 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.098 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_keepalived": { 2024-05-21T15:42:35.098 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_keepalived", 2024-05-21T15:42:35.098 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.098 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.098 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.098 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-21T15:42:35.098 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.098 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.098 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.098 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Keepalived container image", 2024-05-21T15:42:35.098 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.099 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.099 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.099 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.099 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_loki": { 2024-05-21T15:42:35.099 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_loki", 2024-05-21T15:42:35.099 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.099 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.099 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.099 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-21T15:42:35.099 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.099 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.099 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.099 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Loki container image", 2024-05-21T15:42:35.100 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.100 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.100 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.100 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.100 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_node_exporter": { 2024-05-21T15:42:35.100 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_node_exporter", 2024-05-21T15:42:35.100 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.100 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.100 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.100 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-21T15:42:35.100 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.100 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.100 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.101 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-05-21T15:42:35.101 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.101 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.101 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.101 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.101 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_nvmeof": { 2024-05-21T15:42:35.101 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_nvmeof", 2024-05-21T15:42:35.101 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.101 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.101 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.101 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-21T15:42:35.101 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.101 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.102 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.102 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Nvme-of container image", 2024-05-21T15:42:35.102 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.102 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.102 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.102 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.102 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_prometheus": { 2024-05-21T15:42:35.102 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_prometheus", 2024-05-21T15:42:35.102 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.102 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.102 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.102 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-21T15:42:35.102 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.103 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.103 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.103 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-05-21T15:42:35.103 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.103 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.103 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.103 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.103 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_promtail": { 2024-05-21T15:42:35.103 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_promtail", 2024-05-21T15:42:35.103 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.103 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.103 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.103 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-21T15:42:35.103 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.104 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.104 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.104 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Promtail container image", 2024-05-21T15:42:35.104 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.104 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.104 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.104 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.104 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_samba": { 2024-05-21T15:42:35.104 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_samba", 2024-05-21T15:42:35.104 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.104 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.104 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.104 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-21T15:42:35.105 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.105 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.105 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.105 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Samba/SMB container image", 2024-05-21T15:42:35.105 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.105 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.105 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.105 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.105 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_snmp_gateway": { 2024-05-21T15:42:35.105 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_snmp_gateway", 2024-05-21T15:42:35.105 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.105 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.105 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.105 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-21T15:42:35.106 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.106 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.106 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.106 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SNMP Gateway container image", 2024-05-21T15:42:35.106 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.106 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.106 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.106 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.106 INFO:teuthology.orchestra.run.smithi038.stdout: "container_init": { 2024-05-21T15:42:35.106 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_init", 2024-05-21T15:42:35.106 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.106 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.106 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.106 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.107 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.110 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.110 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.110 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Run podman/docker with `--init`", 2024-05-21T15:42:35.110 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.110 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.110 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.110 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.110 INFO:teuthology.orchestra.run.smithi038.stdout: "daemon_cache_timeout": { 2024-05-21T15:42:35.110 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "daemon_cache_timeout", 2024-05-21T15:42:35.110 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.110 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.110 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.110 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:35.111 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.111 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.111 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.111 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-21T15:42:35.111 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.111 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.111 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.111 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.111 INFO:teuthology.orchestra.run.smithi038.stdout: "default_cephadm_command_timeout": { 2024-05-21T15:42:35.111 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_cephadm_command_timeout", 2024-05-21T15:42:35.111 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.111 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.112 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.112 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "900", 2024-05-21T15:42:35.112 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.112 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.112 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.112 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-21T15:42:35.112 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.112 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.112 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.112 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.112 INFO:teuthology.orchestra.run.smithi038.stdout: "default_registry": { 2024-05-21T15:42:35.112 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_registry", 2024-05-21T15:42:35.112 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.112 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.113 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.113 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io", 2024-05-21T15:42:35.113 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.113 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.113 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.113 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-21T15:42:35.113 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.113 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.113 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.113 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.113 INFO:teuthology.orchestra.run.smithi038.stdout: "device_cache_timeout": { 2024-05-21T15:42:35.113 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_cache_timeout", 2024-05-21T15:42:35.113 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.114 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.114 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.114 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1800", 2024-05-21T15:42:35.114 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.114 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.114 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.114 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache device inventory", 2024-05-21T15:42:35.114 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.114 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.114 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.114 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.114 INFO:teuthology.orchestra.run.smithi038.stdout: "device_enhanced_scan": { 2024-05-21T15:42:35.114 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_enhanced_scan", 2024-05-21T15:42:35.115 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.115 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.115 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.115 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.115 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.115 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.115 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.115 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-21T15:42:35.115 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.115 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.115 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.115 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.115 INFO:teuthology.orchestra.run.smithi038.stdout: "facts_cache_timeout": { 2024-05-21T15:42:35.116 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "facts_cache_timeout", 2024-05-21T15:42:35.116 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.116 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.116 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.116 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:35.116 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.116 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.116 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.116 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache host facts data", 2024-05-21T15:42:35.116 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.116 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.116 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.116 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.117 INFO:teuthology.orchestra.run.smithi038.stdout: "grafana_dashboards_path": { 2024-05-21T15:42:35.117 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "grafana_dashboards_path", 2024-05-21T15:42:35.117 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.117 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.117 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.117 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-21T15:42:35.117 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.117 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.117 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.117 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-21T15:42:35.117 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.117 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.117 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.117 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.118 INFO:teuthology.orchestra.run.smithi038.stdout: "host_check_interval": { 2024-05-21T15:42:35.118 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "host_check_interval", 2024-05-21T15:42:35.118 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.118 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.118 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.118 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:35.118 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.118 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.118 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.118 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to perform a host check", 2024-05-21T15:42:35.118 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.118 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.118 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.119 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.119 INFO:teuthology.orchestra.run.smithi038.stdout: "hw_monitoring": { 2024-05-21T15:42:35.119 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hw_monitoring", 2024-05-21T15:42:35.119 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.119 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.119 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.119 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.119 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.119 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.119 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.119 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-21T15:42:35.119 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.119 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.119 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.120 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.120 INFO:teuthology.orchestra.run.smithi038.stdout: "inventory_list_all": { 2024-05-21T15:42:35.120 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "inventory_list_all", 2024-05-21T15:42:35.120 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.120 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.120 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.120 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.120 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.120 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.120 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.120 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-21T15:42:35.120 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.120 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.121 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.121 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.121 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.121 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.121 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.121 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.121 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.121 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.121 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.121 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.121 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.121 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.121 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.122 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.122 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.122 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.122 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.122 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.122 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.122 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.122 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.122 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.122 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.122 INFO:teuthology.orchestra.run.smithi038.stdout: "log_refresh_metadata": { 2024-05-21T15:42:35.122 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_refresh_metadata", 2024-05-21T15:42:35.122 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.122 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.123 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.123 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.123 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.123 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.123 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.123 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-21T15:42:35.123 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.123 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.123 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.123 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.123 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.123 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.124 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.124 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.124 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.124 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.124 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.124 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.124 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.124 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-21T15:42:35.124 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.124 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.124 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.124 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.124 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.125 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.125 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.125 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.125 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.125 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.125 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.125 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.125 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.125 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.125 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.125 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.125 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.125 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.125 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.126 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.126 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.126 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.126 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.126 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.126 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.126 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.126 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.126 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.126 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.126 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.126 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.127 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.127 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.127 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.127 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.127 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.127 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.127 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.127 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.127 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-21T15:42:35.127 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-21T15:42:35.127 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.127 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.127 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.127 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.128 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.128 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.128 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.128 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-21T15:42:35.128 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.128 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.128 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.128 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.128 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-21T15:42:35.128 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-21T15:42:35.128 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.128 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.128 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.129 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "*", 2024-05-21T15:42:35.129 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.129 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.129 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.129 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-21T15:42:35.129 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.129 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.129 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.129 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.129 INFO:teuthology.orchestra.run.smithi038.stdout: "max_count_per_host": { 2024-05-21T15:42:35.129 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_count_per_host", 2024-05-21T15:42:35.129 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.129 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.129 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.130 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.130 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.130 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.130 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.130 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of daemons per service per host", 2024-05-21T15:42:35.130 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.130 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.130 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.130 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.130 INFO:teuthology.orchestra.run.smithi038.stdout: "max_osd_draining_count": { 2024-05-21T15:42:35.130 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_osd_draining_count", 2024-05-21T15:42:35.130 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.130 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.130 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.131 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.131 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.131 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.131 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.131 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-21T15:42:35.131 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.131 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.131 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.131 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.131 INFO:teuthology.orchestra.run.smithi038.stdout: "migration_current": { 2024-05-21T15:42:35.131 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "migration_current", 2024-05-21T15:42:35.131 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.131 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.131 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.132 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.132 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.132 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.132 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.132 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "internal - do not modify", 2024-05-21T15:42:35.132 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.132 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.132 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.132 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.132 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-05-21T15:42:35.132 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-05-21T15:42:35.132 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.132 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.132 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.133 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "root", 2024-05-21T15:42:35.133 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.133 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.133 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.133 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm-package", 2024-05-21T15:42:35.133 INFO:teuthology.orchestra.run.smithi038.stdout: "root" 2024-05-21T15:42:35.133 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.133 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "mode for remote execution of cephadm", 2024-05-21T15:42:35.133 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.133 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.133 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.133 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.133 INFO:teuthology.orchestra.run.smithi038.stdout: "oob_default_addr": { 2024-05-21T15:42:35.133 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "oob_default_addr", 2024-05-21T15:42:35.134 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.134 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.134 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.134 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "169.254.1.1", 2024-05-21T15:42:35.134 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.134 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.134 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.134 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-21T15:42:35.134 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.134 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.134 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.134 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.134 INFO:teuthology.orchestra.run.smithi038.stdout: "prometheus_alerts_path": { 2024-05-21T15:42:35.134 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus_alerts_path", 2024-05-21T15:42:35.135 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.135 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.135 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.135 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-21T15:42:35.135 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.135 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.135 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.135 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-21T15:42:35.135 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.135 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.135 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.135 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.135 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_insecure": { 2024-05-21T15:42:35.136 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_insecure", 2024-05-21T15:42:35.136 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.136 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.136 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.136 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.136 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.136 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.136 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.136 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-21T15:42:35.136 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.136 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.136 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.136 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.137 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_password": { 2024-05-21T15:42:35.137 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_password", 2024-05-21T15:42:35.137 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.137 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.137 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.137 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.137 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.137 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.137 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.137 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-21T15:42:35.137 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.137 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.137 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.138 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.138 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_url": { 2024-05-21T15:42:35.138 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_url", 2024-05-21T15:42:35.138 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.138 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.138 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.138 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.138 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.138 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.139 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.139 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-21T15:42:35.139 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.139 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.139 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.139 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.139 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_username": { 2024-05-21T15:42:35.139 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_username", 2024-05-21T15:42:35.139 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.140 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.140 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.140 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.140 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.140 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.140 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.140 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-21T15:42:35.140 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.140 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.140 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.140 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.140 INFO:teuthology.orchestra.run.smithi038.stdout: "secure_monitoring_stack": { 2024-05-21T15:42:35.140 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secure_monitoring_stack", 2024-05-21T15:42:35.140 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.141 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.141 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.141 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.141 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.141 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.141 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.141 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-21T15:42:35.141 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.141 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.141 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.141 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.141 INFO:teuthology.orchestra.run.smithi038.stdout: "service_discovery_port": { 2024-05-21T15:42:35.141 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "service_discovery_port", 2024-05-21T15:42:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8765", 2024-05-21T15:42:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "cephadm service discovery port", 2024-05-21T15:42:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: "ssh_config_file": { 2024-05-21T15:42:35.143 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssh_config_file", 2024-05-21T15:42:35.143 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.143 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.143 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.143 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.143 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.143 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.143 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.143 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-21T15:42:35.143 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.143 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.143 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.143 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: "use_agent": { 2024-05-21T15:42:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_agent", 2024-05-21T15:42:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-21T15:42:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.145 INFO:teuthology.orchestra.run.smithi038.stdout: "use_repo_digest": { 2024-05-21T15:42:35.145 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_repo_digest", 2024-05-21T15:42:35.145 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.145 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.145 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.145 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.145 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.145 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.145 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.145 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-21T15:42:35.145 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.145 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.145 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.146 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.146 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_failed_host_check": { 2024-05-21T15:42:35.146 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_failed_host_check", 2024-05-21T15:42:35.146 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.146 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.146 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.146 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.146 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.146 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.146 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.146 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if the host check fails", 2024-05-21T15:42:35.146 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.146 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.147 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.147 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.147 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_daemons": { 2024-05-21T15:42:35.147 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_daemons", 2024-05-21T15:42:35.147 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.147 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.147 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.147 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.147 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.147 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.147 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.147 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-21T15:42:35.147 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.147 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_hosts": { 2024-05-21T15:42:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_hosts", 2024-05-21T15:42:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-21T15:42:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crash", 2024-05-21T15:42:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.151 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.151 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.151 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.151 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.151 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.151 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.151 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.151 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.151 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.151 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.151 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.151 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.151 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.151 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.152 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.152 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.152 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.152 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.152 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.152 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.152 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.152 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.152 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.152 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.152 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.152 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.152 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.153 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.153 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.153 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.153 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.153 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.153 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.153 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.153 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.153 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.153 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.153 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.153 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.153 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.154 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.154 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.154 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.154 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.154 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.154 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.154 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.154 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.154 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.154 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.154 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.154 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.154 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: "retain_interval": { 2024-05-21T15:42:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retain_interval", 2024-05-21T15:42:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "31536000", 2024-05-21T15:42:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-21T15:42:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.156 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.156 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_recent_interval": { 2024-05-21T15:42:35.156 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_recent_interval", 2024-05-21T15:42:35.156 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.156 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.156 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.156 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1209600", 2024-05-21T15:42:35.156 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.156 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.156 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.156 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-21T15:42:35.156 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.156 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dashboard", 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.157 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.158 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.158 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.158 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.158 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.158 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.158 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.158 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.158 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.158 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_HOST": { 2024-05-21T15:42:35.158 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-21T15:42:35.158 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.158 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.158 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.159 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.159 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.159 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.159 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.159 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.159 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.159 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.159 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.159 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.159 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-21T15:42:35.159 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-21T15:42:35.159 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.159 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.159 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.160 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.160 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.160 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.160 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.160 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.160 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.160 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.160 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.160 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.160 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_ENABLED": { 2024-05-21T15:42:35.160 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_ENABLED", 2024-05-21T15:42:35.160 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.160 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.160 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.161 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.161 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.161 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.161 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.161 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.161 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.161 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.161 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.161 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.161 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-21T15:42:35.161 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-21T15:42:35.161 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.161 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.161 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.162 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.162 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.162 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.162 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.162 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.162 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.162 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.162 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.162 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.162 INFO:teuthology.orchestra.run.smithi038.stdout: "ENABLE_BROWSABLE_API": { 2024-05-21T15:42:35.162 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-21T15:42:35.162 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.162 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.163 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.163 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.163 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.163 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.163 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.163 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.163 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.163 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.163 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.163 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.163 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-21T15:42:35.163 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-21T15:42:35.163 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.164 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.164 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.164 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.164 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.164 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.164 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.164 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.164 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.164 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.164 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.164 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.164 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-21T15:42:35.164 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-21T15:42:35.165 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.165 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.165 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.165 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.165 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.165 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.165 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.165 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.165 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.165 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.165 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.165 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.165 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-21T15:42:35.165 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-21T15:42:35.166 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.166 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.166 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.166 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.166 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.166 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.166 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.166 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.166 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.166 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.166 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.166 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.166 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-21T15:42:35.167 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-21T15:42:35.167 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.167 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.167 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.167 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.167 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.167 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.167 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.167 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.167 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.167 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.167 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.167 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.167 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-21T15:42:35.168 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-21T15:42:35.169 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.169 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.169 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.169 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.169 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.169 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.169 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.169 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.169 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.169 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.169 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.169 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.169 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-21T15:42:35.169 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-21T15:42:35.170 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.170 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.170 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.170 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.170 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.170 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.170 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.170 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.170 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.170 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.170 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.170 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.170 INFO:teuthology.orchestra.run.smithi038.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-21T15:42:35.171 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-21T15:42:35.171 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.171 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.171 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.171 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.171 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.171 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.171 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.171 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.171 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.171 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.171 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.171 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.171 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_PASSWORD": { 2024-05-21T15:42:35.172 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-21T15:42:35.172 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.172 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.172 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.172 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-05-21T15:42:35.172 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.172 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.172 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.172 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.172 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.172 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.172 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.172 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.172 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-21T15:42:35.173 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-21T15:42:35.173 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.173 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.173 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.173 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.173 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.173 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.173 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.173 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.173 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.173 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.173 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.173 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.173 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_URL": { 2024-05-21T15:42:35.174 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_URL", 2024-05-21T15:42:35.174 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.174 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.174 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.174 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.174 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.174 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.174 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.174 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.174 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.174 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.174 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.174 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_USERNAME": { 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.175 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-21T15:42:35.176 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-21T15:42:35.176 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.176 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.176 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.176 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.176 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.176 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.176 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.176 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.176 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.176 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.176 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.176 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.177 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-21T15:42:35.177 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-21T15:42:35.177 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.177 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.177 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.177 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.177 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.177 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.177 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.177 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.177 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.177 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.178 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.178 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.178 INFO:teuthology.orchestra.run.smithi038.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-21T15:42:35.178 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-21T15:42:35.178 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.178 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.178 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.178 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.178 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.178 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.178 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.178 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.178 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.179 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.179 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.179 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.179 INFO:teuthology.orchestra.run.smithi038.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-21T15:42:35.179 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-21T15:42:35.179 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.179 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.179 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.179 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.179 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.179 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.179 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.179 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.180 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.180 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.180 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.180 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.180 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_HOST": { 2024-05-21T15:42:35.180 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-21T15:42:35.180 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.180 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.180 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.180 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.180 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.180 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.180 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.181 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.182 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.182 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.182 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.182 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.182 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-21T15:42:35.182 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-21T15:42:35.182 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.182 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.182 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.182 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.182 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.182 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.183 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.183 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.183 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.183 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.183 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.183 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.183 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-21T15:42:35.183 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-21T15:42:35.183 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.183 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.183 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.183 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.183 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.183 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.184 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.184 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.184 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.184 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.184 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.184 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.184 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-21T15:42:35.184 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-21T15:42:35.184 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.184 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.184 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.184 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.184 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.184 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-21T15:42:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-21T15:42:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.185 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.186 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.186 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.186 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.186 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.186 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.186 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.186 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-21T15:42:35.186 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-21T15:42:35.186 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.186 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.186 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.186 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.186 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.187 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.187 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.187 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.187 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.187 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.187 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.187 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.187 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-21T15:42:35.187 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-21T15:42:35.187 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.187 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.187 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.187 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.187 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.188 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_ENABLED": { 2024-05-21T15:42:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-21T15:42:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.190 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.190 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.190 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.190 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.190 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.190 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.190 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-21T15:42:35.190 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-21T15:42:35.190 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.190 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.190 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-21T15:42:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-21T15:42:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-21T15:42:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.192 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.192 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.192 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.192 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.192 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.192 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.192 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.192 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.192 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.192 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-21T15:42:35.192 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-21T15:42:35.192 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.192 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.192 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.193 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8", 2024-05-21T15:42:35.193 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.193 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.193 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.193 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.193 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.193 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.193 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.193 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.193 INFO:teuthology.orchestra.run.smithi038.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-21T15:42:35.193 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-21T15:42:35.193 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.193 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.193 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.194 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "45", 2024-05-21T15:42:35.194 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.194 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.194 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.194 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.194 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.194 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.194 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.194 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.194 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ACCESS_KEY": { 2024-05-21T15:42:35.194 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-21T15:42:35.194 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.194 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.195 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.195 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.195 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.195 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.195 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.195 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.195 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.195 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.195 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.195 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.195 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-21T15:42:35.195 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-21T15:42:35.195 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.195 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.196 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.196 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-05-21T15:42:35.196 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.196 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.196 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.196 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.196 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.196 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.196 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.196 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.196 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SECRET_KEY": { 2024-05-21T15:42:35.196 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-21T15:42:35.196 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.197 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.197 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.197 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.197 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.197 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.197 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.197 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.197 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.197 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.197 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.197 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.197 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SSL_VERIFY": { 2024-05-21T15:42:35.197 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-21T15:42:35.198 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.198 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.198 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.198 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.198 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.198 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.198 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.198 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.198 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.198 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.198 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.199 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.199 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-21T15:42:35.199 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-21T15:42:35.199 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.199 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.199 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.199 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:35.199 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.199 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.199 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.199 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.199 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.199 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.200 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.200 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.200 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-21T15:42:35.200 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-21T15:42:35.200 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.200 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.200 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.200 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.200 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.200 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.200 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.200 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.200 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.200 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.201 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.201 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.201 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-21T15:42:35.201 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-21T15:42:35.201 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.201 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.201 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.201 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-05-21T15:42:35.201 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.201 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.201 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.201 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.201 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.201 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.202 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.202 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.202 INFO:teuthology.orchestra.run.smithi038.stdout: "cross_origin_url": { 2024-05-21T15:42:35.202 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cross_origin_url", 2024-05-21T15:42:35.202 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.202 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.202 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.202 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.202 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.202 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.202 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.202 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.202 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.203 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.203 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.203 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.203 INFO:teuthology.orchestra.run.smithi038.stdout: "crt_file": { 2024-05-21T15:42:35.203 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crt_file", 2024-05-21T15:42:35.203 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.203 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.203 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.203 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.203 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.203 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.203 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.204 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.204 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.204 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.204 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.204 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.204 INFO:teuthology.orchestra.run.smithi038.stdout: "debug": { 2024-05-21T15:42:35.204 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "debug", 2024-05-21T15:42:35.204 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.204 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.204 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.204 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.204 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.204 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.205 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.205 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable/disable debug options", 2024-05-21T15:42:35.205 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.205 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.205 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.205 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.205 INFO:teuthology.orchestra.run.smithi038.stdout: "jwt_token_ttl": { 2024-05-21T15:42:35.205 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "jwt_token_ttl", 2024-05-21T15:42:35.205 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.205 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.205 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.205 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "28800", 2024-05-21T15:42:35.205 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.206 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.206 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.206 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.206 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.206 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.206 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.206 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.206 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-05-21T15:42:35.206 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-05-21T15:42:35.206 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.206 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.206 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.207 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.207 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.207 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.207 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.207 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.207 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.207 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.207 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.207 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.207 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.207 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.207 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.207 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.208 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.208 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.208 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.208 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.208 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.208 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.208 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.208 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.208 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.208 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.208 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.208 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.208 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.209 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.209 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.209 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.209 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.209 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.209 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.209 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.209 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.209 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.209 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.209 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.209 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.209 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.209 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.210 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.210 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.210 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.210 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.210 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.210 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.210 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.210 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.210 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.210 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.210 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.210 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.211 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.211 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.211 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.211 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.211 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.211 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.211 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.211 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.211 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.211 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.211 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.211 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.211 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.211 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.212 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.212 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.212 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.212 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.212 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.212 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.212 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.212 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.212 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.212 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.212 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.212 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.212 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.212 INFO:teuthology.orchestra.run.smithi038.stdout: "motd": { 2024-05-21T15:42:35.213 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "motd", 2024-05-21T15:42:35.213 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.213 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.213 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.213 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.213 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.213 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.213 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.213 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "The message of the day", 2024-05-21T15:42:35.213 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.213 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.213 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.213 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.214 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect_resolve_ip_addr": { 2024-05-21T15:42:35.214 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "redirect_resolve_ip_addr", 2024-05-21T15:42:35.214 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.214 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.214 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.214 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.214 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.214 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.214 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.214 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.214 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.214 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.215 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.215 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.215 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-05-21T15:42:35.215 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-05-21T15:42:35.215 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.215 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.215 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.215 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-05-21T15:42:35.215 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.215 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.215 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.215 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.215 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.215 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.216 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.216 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.216 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-05-21T15:42:35.216 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-05-21T15:42:35.216 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.216 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.216 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.216 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8080", 2024-05-21T15:42:35.216 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.216 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.216 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.216 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.216 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.217 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.217 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.217 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.217 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-05-21T15:42:35.217 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-05-21T15:42:35.217 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.217 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.217 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.217 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.217 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.217 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.217 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.217 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.217 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.218 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.218 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.218 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.218 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl_server_port": { 2024-05-21T15:42:35.218 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl_server_port", 2024-05-21T15:42:35.218 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.218 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.218 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.218 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8443", 2024-05-21T15:42:35.218 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.218 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.218 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.219 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.219 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.219 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.219 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.219 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.219 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-05-21T15:42:35.219 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-05-21T15:42:35.219 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.219 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.219 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.219 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "redirect", 2024-05-21T15:42:35.219 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.220 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.220 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.220 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.220 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect" 2024-05-21T15:42:35.220 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.220 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.220 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.220 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.220 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.220 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.220 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-05-21T15:42:35.220 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-05-21T15:42:35.221 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.221 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.221 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.221 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-05-21T15:42:35.221 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-05-21T15:42:35.221 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-05-21T15:42:35.221 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.221 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.221 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.221 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.221 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.221 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.221 INFO:teuthology.orchestra.run.smithi038.stdout: "url_prefix": { 2024-05-21T15:42:35.221 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url_prefix", 2024-05-21T15:42:35.222 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.222 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.222 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.222 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.222 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.222 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.222 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.222 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.222 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.222 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.222 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.222 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.222 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.223 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.223 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.223 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-05-21T15:42:35.223 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.223 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.223 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.223 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_monitoring": { 2024-05-21T15:42:35.223 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_monitoring", 2024-05-21T15:42:35.223 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.223 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.223 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.223 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.223 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.224 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.224 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.224 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "monitor device health metrics", 2024-05-21T15:42:35.224 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.224 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.224 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.224 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.224 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.224 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.224 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.224 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.224 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.224 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.224 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.225 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.225 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.225 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.225 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.225 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.225 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.225 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.225 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.225 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.225 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.225 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.225 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.225 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.225 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.226 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.226 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.226 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.226 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.226 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.226 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.226 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.226 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.226 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.226 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.226 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.226 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.226 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.227 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.227 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.227 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.227 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.227 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.227 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.227 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.227 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.227 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.227 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.227 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.227 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.227 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.228 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.228 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.228 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.228 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.228 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.228 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.228 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.228 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.228 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.228 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.228 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.228 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.228 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.228 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.229 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.229 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.229 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.229 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.229 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.229 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.229 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.229 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.229 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.229 INFO:teuthology.orchestra.run.smithi038.stdout: "mark_out_threshold": { 2024-05-21T15:42:35.229 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mark_out_threshold", 2024-05-21T15:42:35.229 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.229 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.229 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.230 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2419200", 2024-05-21T15:42:35.230 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.230 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.230 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.230 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-21T15:42:35.230 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.230 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.230 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.230 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.230 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_name": { 2024-05-21T15:42:35.230 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_name", 2024-05-21T15:42:35.230 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.230 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.231 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.231 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "device_health_metrics", 2024-05-21T15:42:35.231 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.231 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.231 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.231 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-21T15:42:35.231 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.231 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.231 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.231 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.231 INFO:teuthology.orchestra.run.smithi038.stdout: "retention_period": { 2024-05-21T15:42:35.231 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retention_period", 2024-05-21T15:42:35.231 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.231 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.232 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.232 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15552000", 2024-05-21T15:42:35.232 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.232 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.232 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.232 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain device health metrics", 2024-05-21T15:42:35.232 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.232 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.232 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.232 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.232 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_frequency": { 2024-05-21T15:42:35.232 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_frequency", 2024-05-21T15:42:35.232 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.232 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.233 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.233 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-05-21T15:42:35.233 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.233 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.233 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.233 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-21T15:42:35.233 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.233 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.233 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.233 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.233 INFO:teuthology.orchestra.run.smithi038.stdout: "self_heal": { 2024-05-21T15:42:35.233 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "self_heal", 2024-05-21T15:42:35.234 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.234 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.234 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.234 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.234 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.234 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.234 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.234 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-21T15:42:35.234 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.234 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.234 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.234 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.234 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:35.235 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:35.235 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.235 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.235 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.235 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:35.235 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.235 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.235 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.235 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and check device health", 2024-05-21T15:42:35.235 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.235 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.235 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.235 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.235 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_threshold": { 2024-05-21T15:42:35.236 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_threshold", 2024-05-21T15:42:35.236 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.236 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.236 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.236 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7257600", 2024-05-21T15:42:35.236 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.236 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.236 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.236 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-21T15:42:35.236 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.236 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.236 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.237 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.237 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.237 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.237 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.237 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "diskprediction_local", 2024-05-21T15:42:35.237 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.237 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.237 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.237 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.237 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.237 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.237 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.237 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.237 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.238 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.238 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.238 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.238 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.238 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.238 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.238 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.238 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.238 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.238 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.238 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.238 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.238 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.239 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.240 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.240 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.240 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.240 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.240 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.240 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.240 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.240 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.240 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.240 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.240 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.240 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.240 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.240 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.241 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.241 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.241 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.241 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.241 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.241 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.241 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.241 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.241 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.241 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.241 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.241 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.241 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.242 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.242 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.242 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.242 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.242 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.242 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.242 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.242 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.242 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.242 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.242 INFO:teuthology.orchestra.run.smithi038.stdout: "predict_interval": { 2024-05-21T15:42:35.242 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predict_interval", 2024-05-21T15:42:35.242 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.242 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.243 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.243 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-05-21T15:42:35.243 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.243 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.243 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.243 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.243 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.243 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.243 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.243 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.243 INFO:teuthology.orchestra.run.smithi038.stdout: "predictor_model": { 2024-05-21T15:42:35.243 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predictor_model", 2024-05-21T15:42:35.243 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.243 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.244 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.244 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "prophetstor", 2024-05-21T15:42:35.244 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.244 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.244 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.244 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.244 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.244 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.244 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.244 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.244 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:35.244 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:35.244 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.245 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.245 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.245 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:35.245 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.245 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.245 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.245 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.245 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.245 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.245 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.245 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.245 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.245 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.245 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.246 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "influx", 2024-05-21T15:42:35.246 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": false, 2024-05-21T15:42:35.246 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "influxdb python module not found", 2024-05-21T15:42:35.246 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.246 INFO:teuthology.orchestra.run.smithi038.stdout: "batch_size": { 2024-05-21T15:42:35.246 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "batch_size", 2024-05-21T15:42:35.246 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.246 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.246 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.246 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5000", 2024-05-21T15:42:35.246 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.246 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.246 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.246 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-21T15:42:35.247 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.247 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.247 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.247 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.247 INFO:teuthology.orchestra.run.smithi038.stdout: "database": { 2024-05-21T15:42:35.247 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "database", 2024-05-21T15:42:35.247 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.247 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.247 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.247 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "ceph", 2024-05-21T15:42:35.247 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.247 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.248 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.248 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-21T15:42:35.248 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.248 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.248 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.248 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.248 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": { 2024-05-21T15:42:35.248 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hostname", 2024-05-21T15:42:35.248 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.248 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.248 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.248 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.248 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.248 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.249 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.249 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server hostname", 2024-05-21T15:42:35.249 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.249 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.249 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.249 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.249 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:35.249 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:35.249 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.249 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.249 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.249 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "30", 2024-05-21T15:42:35.249 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "5", 2024-05-21T15:42:35.250 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.250 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.250 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-21T15:42:35.250 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.250 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.250 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.250 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.250 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.250 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.250 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.253 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.253 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.253 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.254 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.254 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.254 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.254 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.254 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.254 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.254 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.254 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.254 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.254 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.254 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.254 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.254 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.255 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.255 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.255 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.255 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.255 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.255 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.255 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.255 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.255 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.255 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.255 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.255 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.256 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.256 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.256 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.256 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.256 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.256 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.256 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.256 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.256 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.256 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.256 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.256 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.256 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.256 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.257 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.257 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.257 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.257 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.257 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.257 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.257 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.257 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.257 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.257 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.257 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.257 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.257 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.258 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.258 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.258 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.258 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.258 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.258 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.258 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.258 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.258 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.258 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.258 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.258 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.258 INFO:teuthology.orchestra.run.smithi038.stdout: "password": { 2024-05-21T15:42:35.259 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "password", 2024-05-21T15:42:35.259 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.259 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.259 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.259 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.259 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.259 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.259 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.259 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "password of InfluxDB server user", 2024-05-21T15:42:35.259 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.259 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.259 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.259 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.259 INFO:teuthology.orchestra.run.smithi038.stdout: "port": { 2024-05-21T15:42:35.260 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "port", 2024-05-21T15:42:35.260 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.260 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.260 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.260 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8086", 2024-05-21T15:42:35.260 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.260 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.260 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.260 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server port", 2024-05-21T15:42:35.260 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.260 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.260 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.260 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.261 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-05-21T15:42:35.261 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-05-21T15:42:35.261 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.261 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.261 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.261 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "false", 2024-05-21T15:42:35.261 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.261 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.261 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.261 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T15:42:35.261 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.261 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.261 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.261 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.262 INFO:teuthology.orchestra.run.smithi038.stdout: "threads": { 2024-05-21T15:42:35.262 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threads", 2024-05-21T15:42:35.262 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.262 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.262 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.262 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-05-21T15:42:35.262 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-05-21T15:42:35.262 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "32", 2024-05-21T15:42:35.262 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.262 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-21T15:42:35.262 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.262 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.262 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.262 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.263 INFO:teuthology.orchestra.run.smithi038.stdout: "username": { 2024-05-21T15:42:35.263 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "username", 2024-05-21T15:42:35.263 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.263 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.263 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.263 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.263 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.263 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.263 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.263 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "username of InfluxDB server user", 2024-05-21T15:42:35.263 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.263 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.263 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.263 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: "verify_ssl": { 2024-05-21T15:42:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "verify_ssl", 2024-05-21T15:42:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "true", 2024-05-21T15:42:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T15:42:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.264 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.265 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.265 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.265 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.265 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.265 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "insights", 2024-05-21T15:42:35.265 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.265 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.265 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.265 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.265 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.265 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.265 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.265 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.266 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.266 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.266 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.266 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.266 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.266 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.266 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.266 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.266 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.266 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.266 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.266 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.266 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.266 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.267 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.267 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.267 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.267 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.267 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.267 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.267 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.267 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.267 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.267 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.267 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.267 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.267 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.268 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.268 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.268 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.268 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.268 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.268 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.268 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.268 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.268 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.268 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.268 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.268 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.268 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.268 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.269 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.269 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.269 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.269 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.269 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.269 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.269 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.269 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.269 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.269 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.269 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.269 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.269 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.270 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.270 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.270 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.270 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.270 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.270 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.270 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.270 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.270 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.270 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.270 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.270 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.270 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.271 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.271 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "iostat", 2024-05-21T15:42:35.271 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.271 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.271 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.271 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.271 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.271 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.271 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.271 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.271 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.271 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.271 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.272 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.273 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.273 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.273 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.273 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.273 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.273 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.273 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.273 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.273 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.273 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.273 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.273 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.273 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.273 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.274 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.274 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.274 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.274 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.274 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.274 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.274 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.274 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.274 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.274 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.274 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.274 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.274 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.275 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.275 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.275 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.275 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.275 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.275 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.275 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.275 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.275 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.275 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.275 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.275 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.275 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.275 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.276 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.276 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.276 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.276 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.276 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.276 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.276 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.276 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.276 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.276 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.276 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "k8sevents", 2024-05-21T15:42:35.276 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.276 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.277 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.277 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_event_retention_days": { 2024-05-21T15:42:35.277 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ceph_event_retention_days", 2024-05-21T15:42:35.277 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.277 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.277 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.277 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7", 2024-05-21T15:42:35.277 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.277 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.277 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.277 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-21T15:42:35.277 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.277 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.277 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.278 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.278 INFO:teuthology.orchestra.run.smithi038.stdout: "config_check_secs": { 2024-05-21T15:42:35.278 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_check_secs", 2024-05-21T15:42:35.278 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.278 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.278 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.278 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.278 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "10", 2024-05-21T15:42:35.278 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.278 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.278 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-21T15:42:35.278 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.279 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.279 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.279 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.279 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.279 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.279 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.279 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.279 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.279 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.279 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.279 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.279 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.279 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.280 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.280 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.280 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.280 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.280 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.280 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.280 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.280 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.280 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.280 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.280 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.280 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.280 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.281 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.281 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.281 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.281 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.281 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.281 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.281 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.281 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.281 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.281 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.281 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.281 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.281 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.282 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.282 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.282 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.282 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.282 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.282 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.282 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.282 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.282 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.282 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.282 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.282 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.282 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.283 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.283 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.283 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.283 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.283 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.283 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.283 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.283 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.283 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.283 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.283 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.283 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.283 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.283 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.284 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.284 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.284 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.284 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.284 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.284 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.284 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.284 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.284 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.284 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.284 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "localpool", 2024-05-21T15:42:35.284 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.284 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.285 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.285 INFO:teuthology.orchestra.run.smithi038.stdout: "failure_domain": { 2024-05-21T15:42:35.285 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "failure_domain", 2024-05-21T15:42:35.285 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.285 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.285 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.285 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "host", 2024-05-21T15:42:35.285 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.285 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.285 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.285 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "failure domain for any created local pool", 2024-05-21T15:42:35.285 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-21T15:42:35.285 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.286 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.286 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.286 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.286 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.286 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.286 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.286 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.286 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.286 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.286 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.286 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.286 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.286 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.287 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.287 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.287 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.287 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.287 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.287 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.287 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.287 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.287 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.287 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.287 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.287 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.287 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.287 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.288 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.288 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.288 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.288 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.288 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.288 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.288 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.288 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.288 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.288 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.288 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.288 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.288 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.289 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.289 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.289 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.289 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.289 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.289 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.289 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.289 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.289 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.289 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.289 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.289 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.289 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.289 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.290 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.290 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.290 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.290 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.290 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.290 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.290 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.290 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.290 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.290 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.290 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.290 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.290 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.291 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.291 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.291 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.291 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.291 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.291 INFO:teuthology.orchestra.run.smithi038.stdout: "min_size": { 2024-05-21T15:42:35.291 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_size", 2024-05-21T15:42:35.291 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.291 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.291 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.291 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.291 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.291 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default min_size for any created local pool", 2024-05-21T15:42:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-21T15:42:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: "num_rep": { 2024-05-21T15:42:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "num_rep", 2024-05-21T15:42:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3", 2024-05-21T15:42:35.292 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default replica count for any created local pool", 2024-05-21T15:42:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_num": { 2024-05-21T15:42:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_num", 2024-05-21T15:42:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.293 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "128", 2024-05-21T15:42:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default pg_num for any created local pool", 2024-05-21T15:42:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: "prefix": { 2024-05-21T15:42:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prefix", 2024-05-21T15:42:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.294 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.295 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.295 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.295 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.295 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name prefix for any created local pool", 2024-05-21T15:42:35.295 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.295 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.295 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.295 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.295 INFO:teuthology.orchestra.run.smithi038.stdout: "subtree": { 2024-05-21T15:42:35.295 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "subtree", 2024-05-21T15:42:35.295 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.295 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.295 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.295 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "rack", 2024-05-21T15:42:35.296 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.296 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.296 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.296 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-21T15:42:35.296 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-21T15:42:35.296 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.296 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.296 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.296 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.296 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.296 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.296 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mds_autoscaler", 2024-05-21T15:42:35.296 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.297 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.297 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.297 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.297 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.297 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.297 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.297 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.297 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.297 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.297 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.297 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.297 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.297 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.297 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.298 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.298 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.298 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.298 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.298 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.298 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.298 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.298 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.298 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.298 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.298 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.298 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.298 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.298 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.299 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.299 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.299 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.299 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.299 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.299 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.299 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.299 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.299 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.299 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.299 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.299 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.299 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.299 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.300 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.300 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.300 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.300 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.300 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.300 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.300 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.300 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.300 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.300 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.300 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.300 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.300 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.301 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.302 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.302 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.302 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.302 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.302 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.302 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.302 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirroring", 2024-05-21T15:42:35.302 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.302 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.302 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.302 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.302 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.302 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.303 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.303 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.303 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.303 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.303 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.303 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.303 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.303 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.303 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.303 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.303 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.303 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.303 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.304 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.305 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.305 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.305 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.305 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.305 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.305 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.305 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.305 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.305 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.305 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.305 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.305 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.305 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.306 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.306 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.306 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.306 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.306 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.306 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.306 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.306 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.306 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.306 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.306 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.306 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.306 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.307 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.307 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.307 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.307 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.307 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.307 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.307 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.307 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.307 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.307 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.307 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.307 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "nfs", 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.308 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.309 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.309 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.309 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.309 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.309 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.309 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.309 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.309 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.309 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.309 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.309 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.309 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.309 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.310 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.310 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.310 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.310 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.310 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.310 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.310 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.310 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.310 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.310 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.310 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.310 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.311 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.311 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.311 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.311 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.311 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.311 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.311 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.311 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.311 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.311 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.311 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.311 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.311 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.311 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.312 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.312 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.312 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.312 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.312 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.312 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.312 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.312 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.312 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.312 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.312 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.312 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.312 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.313 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.313 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.313 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.313 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.313 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.313 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.313 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.313 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.313 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.313 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.313 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.313 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.313 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.313 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.314 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.314 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-05-21T15:42:35.314 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.314 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.314 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.314 INFO:teuthology.orchestra.run.smithi038.stdout: "fail_fs": { 2024-05-21T15:42:35.314 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "fail_fs", 2024-05-21T15:42:35.314 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.314 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.314 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.314 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.314 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.314 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.315 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.315 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-21T15:42:35.315 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.315 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.315 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.315 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.315 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.315 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.315 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.315 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.315 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.315 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.315 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.315 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.316 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.316 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.316 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.316 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.316 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.316 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.316 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.316 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.316 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.316 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.316 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.316 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.316 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.316 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.317 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.317 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.317 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.317 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.317 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.317 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.317 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.317 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.317 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.317 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.317 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.317 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.317 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.318 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.318 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.318 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.318 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.318 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.318 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.318 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.318 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.318 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.318 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.318 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.318 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.318 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.318 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.319 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.319 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.319 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.319 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.319 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.319 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.319 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.319 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.319 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.319 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.319 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.319 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.319 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.319 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.320 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.320 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.320 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.320 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.320 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.320 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.320 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.320 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator": { 2024-05-21T15:42:35.320 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-05-21T15:42:35.320 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.320 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.320 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.320 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.320 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.321 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.321 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.321 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-05-21T15:42:35.321 INFO:teuthology.orchestra.run.smithi038.stdout: "rook", 2024-05-21T15:42:35.321 INFO:teuthology.orchestra.run.smithi038.stdout: "test_orchestrator" 2024-05-21T15:42:35.321 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.321 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Orchestrator backend", 2024-05-21T15:42:35.321 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.321 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.321 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.321 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.321 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.321 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.322 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.322 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_perf_query", 2024-05-21T15:42:35.322 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.322 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.322 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.322 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.322 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.322 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.322 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.322 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.322 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.322 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.322 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.323 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.323 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.323 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.323 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.323 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.323 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.323 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.323 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.323 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.323 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.323 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.323 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.323 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.324 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.324 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.324 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.324 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.324 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.324 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.324 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.324 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.324 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.324 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.324 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.324 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.324 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.325 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.325 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.325 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.325 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.325 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.325 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.325 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.325 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.325 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.325 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.325 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.325 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.325 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.325 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.326 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.326 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.326 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.326 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.326 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.326 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.326 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.326 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.326 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.326 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.326 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.326 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.326 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.327 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.327 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.327 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.327 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.327 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.327 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.327 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.327 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.327 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.327 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.327 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.327 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.327 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_support", 2024-05-21T15:42:35.328 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.328 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.328 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.328 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.328 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.328 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.328 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.328 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.328 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.328 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.328 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.328 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.328 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.328 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.329 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.329 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.329 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.329 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.329 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.329 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.329 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.329 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.329 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.329 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.329 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.329 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.329 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.330 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.330 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.330 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.330 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.330 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.330 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.330 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.330 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.330 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.330 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.330 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.330 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.331 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.331 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.331 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.331 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.331 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.331 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.331 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.331 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.331 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.331 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.331 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.331 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.331 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.331 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.332 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.332 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.332 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.332 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.332 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.332 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.332 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.332 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.332 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.332 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.332 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.332 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.332 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.333 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.333 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.333 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.333 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.333 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.333 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.333 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.333 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.333 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.333 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.333 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_autoscaler", 2024-05-21T15:42:35.333 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.333 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.333 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.334 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.334 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.334 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.334 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.334 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.334 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.334 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.334 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.334 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.334 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.334 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.334 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.334 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.335 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.335 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.335 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.335 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.335 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.335 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.335 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.335 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.335 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.335 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.335 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.335 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.335 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.336 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.336 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.336 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.336 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.336 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.336 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.336 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.336 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.336 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.336 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.336 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.336 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.337 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.337 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.337 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.337 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.337 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.337 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.337 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.337 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.337 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.337 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.337 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.337 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.337 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.337 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.338 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.338 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.338 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.338 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.338 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.338 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.338 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.338 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.338 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.338 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.338 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.338 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.338 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.339 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.339 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.339 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.339 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.339 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.339 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:35.339 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:35.339 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.339 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.339 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.339 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:35.339 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.339 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.340 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.340 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.340 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.340 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.340 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.340 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.340 INFO:teuthology.orchestra.run.smithi038.stdout: "threshold": { 2024-05-21T15:42:35.340 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threshold", 2024-05-21T15:42:35.340 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.340 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.340 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.340 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-05-21T15:42:35.340 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1.0", 2024-05-21T15:42:35.340 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.341 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.341 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "scaling threshold", 2024-05-21T15:42:35.341 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-21T15:42:35.341 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.341 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.341 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.341 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.341 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.341 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.341 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "progress", 2024-05-21T15:42:35.341 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.341 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.341 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.342 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_pg_recovery_event": { 2024-05-21T15:42:35.342 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_pg_recovery_event", 2024-05-21T15:42:35.342 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.342 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.342 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.342 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.342 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.342 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.342 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.342 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-21T15:42:35.342 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.342 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.343 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.343 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.343 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-05-21T15:42:35.343 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-05-21T15:42:35.343 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.343 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.343 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.343 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.343 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.343 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.343 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.343 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.343 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.343 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.344 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.344 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.344 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.344 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.344 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.344 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.344 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.344 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.344 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.344 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.344 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.344 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.344 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.345 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.345 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.345 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.345 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.345 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.345 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.345 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.345 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.345 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.345 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.345 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.345 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.345 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.345 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.346 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.346 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.346 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.346 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.346 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.346 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.346 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.346 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.346 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.346 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.346 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.346 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.346 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.347 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.347 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.347 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.347 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.347 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.347 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.347 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.347 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.347 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.347 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.347 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.347 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.347 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.348 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.348 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.348 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.348 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.348 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.348 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.348 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.348 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.348 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.348 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.348 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.348 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.348 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.348 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.349 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.349 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.349 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.349 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.349 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.349 INFO:teuthology.orchestra.run.smithi038.stdout: "max_completed_events": { 2024-05-21T15:42:35.349 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_completed_events", 2024-05-21T15:42:35.349 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.349 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.349 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.349 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "50", 2024-05-21T15:42:35.349 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.349 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.350 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.350 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "number of past completed events to remember", 2024-05-21T15:42:35.350 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.350 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.350 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.350 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.350 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:35.350 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:35.350 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.350 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.350 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.350 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-05-21T15:42:35.350 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.351 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.351 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.351 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long the module is going to sleep", 2024-05-21T15:42:35.351 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.351 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.351 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.351 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.351 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.351 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.351 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.351 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus", 2024-05-21T15:42:35.351 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.351 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.352 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.352 INFO:teuthology.orchestra.run.smithi038.stdout: "cache": { 2024-05-21T15:42:35.352 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cache", 2024-05-21T15:42:35.352 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.352 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.352 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.352 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.352 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.352 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.352 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.352 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.352 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.352 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.353 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.353 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.353 INFO:teuthology.orchestra.run.smithi038.stdout: "exclude_perf_counters": { 2024-05-21T15:42:35.353 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "exclude_perf_counters", 2024-05-21T15:42:35.353 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.353 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.353 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.353 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.353 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.353 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.353 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.354 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-21T15:42:35.354 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-21T15:42:35.354 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.354 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.354 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.354 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.354 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.354 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.354 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.354 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.354 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.354 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.354 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.355 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.355 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.355 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.355 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.355 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.355 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.355 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.355 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.355 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.355 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.355 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.355 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.355 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.356 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.356 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.356 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.356 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.356 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.356 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.356 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.356 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.356 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.356 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.356 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.356 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.356 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.357 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.357 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.357 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.357 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.357 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.357 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.357 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.357 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.357 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.357 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.357 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.357 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.357 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.357 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.358 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.358 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.358 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.358 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.358 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.358 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.358 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.358 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.358 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.358 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.358 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.358 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.358 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.359 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.359 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.359 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.359 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.359 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.359 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.359 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.359 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.359 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.359 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools": { 2024-05-21T15:42:35.359 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools", 2024-05-21T15:42:35.359 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.359 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.359 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.360 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.360 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.360 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.360 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.360 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.360 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.360 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.360 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.360 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.360 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-21T15:42:35.360 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-21T15:42:35.360 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.360 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.360 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.361 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "300", 2024-05-21T15:42:35.361 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.361 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.361 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.361 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.361 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.361 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.361 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.361 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.361 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_interval": { 2024-05-21T15:42:35.361 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_interval", 2024-05-21T15:42:35.361 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.361 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.361 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.362 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15.0", 2024-05-21T15:42:35.362 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.362 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.362 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.362 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.362 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.362 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.362 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.362 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.362 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-05-21T15:42:35.362 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-05-21T15:42:35.362 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.362 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.363 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.363 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-05-21T15:42:35.363 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.363 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.363 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.363 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-21T15:42:35.363 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.363 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.363 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.363 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.363 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-05-21T15:42:35.363 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-05-21T15:42:35.363 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.363 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.364 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.364 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "9283", 2024-05-21T15:42:35.364 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.364 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.364 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.364 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-21T15:42:35.364 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.364 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.364 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.364 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.364 INFO:teuthology.orchestra.run.smithi038.stdout: "stale_cache_strategy": { 2024-05-21T15:42:35.364 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stale_cache_strategy", 2024-05-21T15:42:35.364 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "log", 2024-05-21T15:42:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-05-21T15:42:35.365 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-05-21T15:42:35.366 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.366 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.366 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.366 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "default", 2024-05-21T15:42:35.366 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.366 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.366 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.366 INFO:teuthology.orchestra.run.smithi038.stdout: "default", 2024-05-21T15:42:35.366 INFO:teuthology.orchestra.run.smithi038.stdout: "error" 2024-05-21T15:42:35.366 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.366 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.366 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.366 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.367 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.367 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.367 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-05-21T15:42:35.367 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-05-21T15:42:35.367 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.367 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.367 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.367 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-05-21T15:42:35.367 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-05-21T15:42:35.367 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-05-21T15:42:35.367 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.367 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.367 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.367 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-05-21T15:42:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.369 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.369 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.369 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.369 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.369 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.369 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.369 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.369 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.369 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.369 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.369 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.369 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.369 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.370 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.370 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.370 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.370 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.370 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.370 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.370 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.370 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.370 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.370 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.370 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.370 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.370 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.371 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.371 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.371 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.371 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.371 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.371 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.371 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.371 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.371 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.371 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.371 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.371 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.371 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.371 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.372 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.372 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.372 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.372 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.372 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.372 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.372 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.372 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.372 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.372 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.372 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.372 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.372 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.372 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.373 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.373 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.373 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.373 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.373 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.373 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.373 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.373 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.373 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.373 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.373 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.373 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.373 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_snap_create": { 2024-05-21T15:42:35.373 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_snap_create", 2024-05-21T15:42:35.374 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.374 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.374 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.374 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.374 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.374 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.374 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.374 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.374 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.374 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.374 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.374 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.374 INFO:teuthology.orchestra.run.smithi038.stdout: "mirror_snapshot_schedule": { 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirror_snapshot_schedule", 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: "trash_purge_schedule": { 2024-05-21T15:42:35.375 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "trash_purge_schedule", 2024-05-21T15:42:35.376 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.376 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.376 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.376 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.376 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.376 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.376 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.376 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.376 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.376 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.376 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.376 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.376 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "restful", 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_auth": { 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_auth", 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.377 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.378 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.378 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.378 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.378 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.378 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.378 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-05-21T15:42:35.378 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-05-21T15:42:35.378 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.378 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.378 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.378 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.378 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.378 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.379 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.379 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.379 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.379 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.379 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.379 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.379 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.379 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.379 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.380 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.380 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.380 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.380 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.380 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.380 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.380 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.380 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.380 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.380 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.380 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.380 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.380 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.383 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.383 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.383 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.383 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.383 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.383 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.383 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.383 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.383 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.383 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.383 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.383 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.383 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.383 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.384 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.384 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.384 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.384 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.384 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.384 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.384 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.384 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.384 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.384 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.384 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.384 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.384 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.385 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.385 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.385 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-05-21T15:42:35.385 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-05-21T15:42:35.385 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.385 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.385 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.385 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.385 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.385 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.385 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.385 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.385 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.386 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.386 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.386 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.386 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-05-21T15:42:35.386 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-05-21T15:42:35.386 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.386 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.386 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.386 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.386 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.386 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.386 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.387 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.387 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.387 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.387 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.387 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.387 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.387 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.387 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.387 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rgw", 2024-05-21T15:42:35.387 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.387 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.387 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.387 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.387 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.388 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.389 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.389 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.389 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.389 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.389 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.389 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.389 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.389 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.389 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.389 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.389 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.389 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.389 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.389 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.390 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.390 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.390 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.390 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.390 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.390 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.390 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.390 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.390 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.390 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.390 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.390 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.390 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.391 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.391 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.391 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.391 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.391 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.391 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.391 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.391 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.391 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.391 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.391 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.391 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.391 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.392 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.393 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.393 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.393 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.393 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rook", 2024-05-21T15:42:35.393 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.393 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.393 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.393 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.393 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.393 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.393 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.393 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.393 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.393 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.394 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.394 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.394 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.394 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.394 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.394 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.394 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.394 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.394 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.394 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.394 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.394 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.395 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.395 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.395 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.395 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.395 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.395 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.395 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.395 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.395 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.395 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.395 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.395 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.395 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.396 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.396 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.396 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.396 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.396 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.396 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.396 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.396 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.396 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.396 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.396 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.396 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.396 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.397 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.397 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.397 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.397 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.397 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.397 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.397 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.397 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.397 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.397 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.397 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.397 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.397 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.397 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.398 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.398 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.398 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.398 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.398 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.398 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.398 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.398 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.398 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.398 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.398 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.398 INFO:teuthology.orchestra.run.smithi038.stdout: "storage_class": { 2024-05-21T15:42:35.399 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "storage_class", 2024-05-21T15:42:35.399 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.399 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.399 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.399 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "local", 2024-05-21T15:42:35.399 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.399 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.399 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.399 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-21T15:42:35.399 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.399 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.399 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.399 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.400 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.400 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.400 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.400 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "selftest", 2024-05-21T15:42:35.400 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.400 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.400 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.400 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.400 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.400 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.400 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.400 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.400 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.400 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.401 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.401 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.401 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.401 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.401 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.401 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.401 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.401 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.401 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.401 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.401 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.401 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.402 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.402 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.402 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.402 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.402 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.402 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.402 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.402 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.402 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.402 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.402 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.402 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.402 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.402 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.403 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.403 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.403 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.403 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.403 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.403 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.403 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.403 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.403 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.403 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.403 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.403 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.403 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.403 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.404 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.404 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.404 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.404 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.404 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.404 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.404 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.404 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.404 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.404 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.404 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.404 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.404 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.405 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.405 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.405 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.405 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.405 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.405 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.405 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.405 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.405 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.405 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.405 INFO:teuthology.orchestra.run.smithi038.stdout: "roption1": { 2024-05-21T15:42:35.405 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption1", 2024-05-21T15:42:35.405 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.405 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.406 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.406 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.406 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.406 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.406 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.406 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.406 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.406 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.406 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.406 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.406 INFO:teuthology.orchestra.run.smithi038.stdout: "roption2": { 2024-05-21T15:42:35.406 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption2", 2024-05-21T15:42:35.406 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "xyz", 2024-05-21T15:42:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption1": { 2024-05-21T15:42:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption1", 2024-05-21T15:42:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption2": { 2024-05-21T15:42:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption2", 2024-05-21T15:42:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.409 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.409 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.409 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.409 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.409 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.409 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.409 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.409 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.409 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.409 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.409 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.409 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption3": { 2024-05-21T15:42:35.410 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption3", 2024-05-21T15:42:35.410 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.410 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.410 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.410 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.410 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.410 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.410 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.410 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.410 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.410 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.410 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.410 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.411 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption4": { 2024-05-21T15:42:35.411 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption4", 2024-05-21T15:42:35.411 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.411 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.411 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.411 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.411 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.411 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.411 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.411 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.411 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.411 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.411 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.412 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.412 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption5": { 2024-05-21T15:42:35.412 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption5", 2024-05-21T15:42:35.412 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.412 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.412 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.412 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.412 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.412 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.412 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.412 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.412 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.412 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.413 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.413 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.413 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption6": { 2024-05-21T15:42:35.413 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption6", 2024-05-21T15:42:35.413 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.413 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.413 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.413 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.413 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.413 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.413 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.413 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.413 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.414 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.414 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.414 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.414 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption7": { 2024-05-21T15:42:35.414 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption7", 2024-05-21T15:42:35.414 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.414 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.414 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.414 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.414 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-05-21T15:42:35.414 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "42", 2024-05-21T15:42:35.414 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.415 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.415 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.415 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.415 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.415 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.415 INFO:teuthology.orchestra.run.smithi038.stdout: "testkey": { 2024-05-21T15:42:35.415 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testkey", 2024-05-21T15:42:35.415 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.415 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.415 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.415 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.415 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.415 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.416 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.416 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.416 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.416 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.416 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.416 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.416 INFO:teuthology.orchestra.run.smithi038.stdout: "testlkey": { 2024-05-21T15:42:35.416 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testlkey", 2024-05-21T15:42:35.416 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.416 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.416 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.416 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.416 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.416 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.417 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.417 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.417 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.417 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.417 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.417 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.417 INFO:teuthology.orchestra.run.smithi038.stdout: "testnewline": { 2024-05-21T15:42:35.417 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testnewline", 2024-05-21T15:42:35.417 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.417 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.417 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.417 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.418 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.418 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.418 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.418 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.418 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.418 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.418 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.418 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.418 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.418 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.418 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.418 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snap_schedule", 2024-05-21T15:42:35.418 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.419 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.419 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.419 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_m_granularity": { 2024-05-21T15:42:35.419 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_m_granularity", 2024-05-21T15:42:35.419 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.419 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.419 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.419 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.419 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.419 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.419 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.419 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow minute scheduled snapshots", 2024-05-21T15:42:35.419 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.420 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.420 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.420 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.420 INFO:teuthology.orchestra.run.smithi038.stdout: "dump_on_update": { 2024-05-21T15:42:35.420 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dump_on_update", 2024-05-21T15:42:35.420 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.420 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.420 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.420 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.420 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.420 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.420 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.420 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "dump database to debug log on update", 2024-05-21T15:42:35.420 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.421 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.421 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.421 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.421 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.421 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.421 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.421 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.421 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.421 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.421 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.421 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.421 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.421 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.422 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.422 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.422 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.422 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.422 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.422 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.422 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.422 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.422 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.422 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.422 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.422 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.422 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.423 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.423 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.423 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.423 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.423 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.423 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.423 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.423 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.423 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.423 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.423 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.423 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.423 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.423 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.424 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.424 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.424 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.424 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.424 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.424 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.424 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.424 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.424 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.424 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.424 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.425 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.425 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.425 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.425 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.425 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.425 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.425 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.425 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.425 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.425 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.425 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.425 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.425 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.426 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.426 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.426 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.426 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.426 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.426 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.426 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.426 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.426 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.426 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.426 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.426 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.426 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stats", 2024-05-21T15:42:35.427 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.427 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.427 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.427 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.427 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.427 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.427 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.427 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.427 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.427 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.427 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.427 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.427 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.428 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.428 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.428 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.428 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.428 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.428 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.428 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.428 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.428 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.428 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.428 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.428 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.428 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.428 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.429 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.429 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.429 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.429 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.429 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.429 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.429 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.429 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.429 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.429 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.429 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.429 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.429 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.429 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.430 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.430 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.430 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.430 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.430 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.430 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.430 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.430 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.430 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.430 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.430 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.430 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.430 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.431 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.431 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.431 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.431 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.431 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.431 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.431 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.431 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.431 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.431 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.431 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.431 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.431 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.432 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.432 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.432 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.432 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.432 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.432 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.432 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.432 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.432 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.432 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "status", 2024-05-21T15:42:35.432 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.432 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.432 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.433 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.433 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.433 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.433 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.433 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.433 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.433 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.433 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.433 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.433 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.433 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.433 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.433 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.433 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.434 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.434 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.434 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.434 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.434 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.434 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.434 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.434 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.434 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.434 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.434 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.434 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.434 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.435 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.435 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.435 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.435 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.435 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.435 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.435 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.435 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.435 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.435 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.435 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.435 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.435 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.435 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.436 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.436 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.436 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.436 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.436 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.436 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.436 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.436 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.436 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.436 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.436 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.436 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.436 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.437 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.437 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.437 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.437 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.437 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.437 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.437 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.437 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.437 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.437 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.437 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.437 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.437 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.437 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.438 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.438 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.438 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.438 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.438 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.438 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telegraf", 2024-05-21T15:42:35.438 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.438 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.438 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.438 INFO:teuthology.orchestra.run.smithi038.stdout: "address": { 2024-05-21T15:42:35.438 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "address", 2024-05-21T15:42:35.438 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.438 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.439 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.439 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-21T15:42:35.439 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.439 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.439 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.439 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.439 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.439 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.439 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.439 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.439 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:35.439 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:35.440 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.440 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.440 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.440 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15", 2024-05-21T15:42:35.440 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.440 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.440 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.440 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.440 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.440 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.440 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.440 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.440 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.441 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.441 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.441 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.441 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.441 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.441 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.441 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.441 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.441 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.441 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.441 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.441 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.441 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.442 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.442 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.442 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.442 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.442 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.442 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.442 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.442 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.442 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.442 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.442 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.442 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.442 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.443 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.443 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.443 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.443 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.443 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.443 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.443 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.443 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.443 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.443 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.443 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.443 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.443 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.444 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.444 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.444 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.444 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.444 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.444 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.444 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.444 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.444 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.444 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.444 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.444 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.444 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.444 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.445 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.445 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.445 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.445 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.445 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.445 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.445 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.445 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.445 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.445 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.445 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.445 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.445 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.446 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.446 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.446 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.446 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.446 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.446 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.446 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telemetry", 2024-05-21T15:42:35.446 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.446 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.446 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.446 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_basic": { 2024-05-21T15:42:35.446 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_basic", 2024-05-21T15:42:35.446 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-21T15:42:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_crash": { 2024-05-21T15:42:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_crash", 2024-05-21T15:42:35.447 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.448 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.448 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.448 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.448 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.448 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.448 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.448 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-21T15:42:35.448 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.448 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.448 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.448 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.448 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_device": { 2024-05-21T15:42:35.448 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_device", 2024-05-21T15:42:35.449 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.449 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.449 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.449 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.449 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.449 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.449 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.449 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-21T15:42:35.449 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.449 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.449 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.449 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.449 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_ident": { 2024-05-21T15:42:35.450 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_ident", 2024-05-21T15:42:35.450 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.450 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.450 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.450 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.450 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.450 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.450 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.450 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-21T15:42:35.450 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.450 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.450 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.450 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.451 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_perf": { 2024-05-21T15:42:35.451 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_perf", 2024-05-21T15:42:35.451 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.451 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.451 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.451 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.451 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.451 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.451 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.451 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-21T15:42:35.451 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.451 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.451 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.452 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.452 INFO:teuthology.orchestra.run.smithi038.stdout: "contact": { 2024-05-21T15:42:35.452 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "contact", 2024-05-21T15:42:35.452 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.452 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.452 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.452 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.452 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.452 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.452 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.452 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.452 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.452 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.452 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.453 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.453 INFO:teuthology.orchestra.run.smithi038.stdout: "description": { 2024-05-21T15:42:35.453 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "description", 2024-05-21T15:42:35.453 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.453 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.453 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.453 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.453 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.453 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.453 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.453 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.453 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.454 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.454 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.454 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.454 INFO:teuthology.orchestra.run.smithi038.stdout: "device_url": { 2024-05-21T15:42:35.454 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_url", 2024-05-21T15:42:35.454 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.454 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.454 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.454 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-21T15:42:35.454 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.454 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.455 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.455 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.455 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.455 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.455 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.455 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.455 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-05-21T15:42:35.455 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-05-21T15:42:35.455 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.455 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.455 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.455 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.455 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.456 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.456 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.456 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.456 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.456 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.456 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.456 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.456 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:35.456 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:35.456 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.456 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.456 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.456 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "24", 2024-05-21T15:42:35.457 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "8", 2024-05-21T15:42:35.457 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.457 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.457 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.457 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.457 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.457 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.457 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.457 INFO:teuthology.orchestra.run.smithi038.stdout: "last_opt_revision": { 2024-05-21T15:42:35.457 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "last_opt_revision", 2024-05-21T15:42:35.457 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.457 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.457 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.458 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1", 2024-05-21T15:42:35.458 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.458 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.458 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.458 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.458 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.458 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.458 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.458 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.458 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard": { 2024-05-21T15:42:35.458 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard", 2024-05-21T15:42:35.458 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.459 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.459 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.459 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.459 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.459 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.459 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.459 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.459 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.459 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.459 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.459 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.459 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard_description": { 2024-05-21T15:42:35.459 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard_description", 2024-05-21T15:42:35.460 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.460 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.461 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.461 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.461 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.461 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.461 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.461 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.462 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.462 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.462 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.462 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.462 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.462 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.462 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.462 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.462 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.462 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.462 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.462 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.462 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.463 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.463 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.463 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.463 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.463 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.463 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.463 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.463 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.463 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.463 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.463 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.463 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.463 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.464 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.464 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.464 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.464 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.464 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.464 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.464 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.464 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.464 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.464 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.464 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.464 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.464 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.464 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.465 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.465 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.465 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.465 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.465 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.465 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.465 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.465 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.465 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.465 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.465 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.465 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.465 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.466 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.466 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.466 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.466 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.466 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.466 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.466 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.466 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.466 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.466 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.466 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.466 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.467 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.467 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.467 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.467 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.467 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.467 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.467 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.467 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.467 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.467 INFO:teuthology.orchestra.run.smithi038.stdout: "organization": { 2024-05-21T15:42:35.467 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "organization", 2024-05-21T15:42:35.467 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.467 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.467 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.468 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.468 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.468 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.468 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.468 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.468 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.468 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.468 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.468 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.468 INFO:teuthology.orchestra.run.smithi038.stdout: "proxy": { 2024-05-21T15:42:35.468 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "proxy", 2024-05-21T15:42:35.468 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.468 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.469 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.469 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.469 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.469 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.469 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.469 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.469 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.469 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.469 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.469 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.469 INFO:teuthology.orchestra.run.smithi038.stdout: "url": { 2024-05-21T15:42:35.469 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url", 2024-05-21T15:42:35.470 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.470 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.470 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.470 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-21T15:42:35.470 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.470 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.470 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.470 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.470 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.470 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.470 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.470 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.470 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.471 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.471 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.471 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "test_orchestrator", 2024-05-21T15:42:35.471 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.471 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.471 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.471 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.471 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.471 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.471 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.471 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.471 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.471 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.471 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.472 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.472 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.472 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.472 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.472 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.472 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.472 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.472 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.472 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.472 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.472 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.472 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.472 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.476 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.476 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.476 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.476 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.476 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.476 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.476 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.476 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.476 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.476 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.476 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.476 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.476 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-05-21T15:42:35.477 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.477 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.477 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.477 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.477 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.477 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.477 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.477 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.477 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.477 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.477 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.477 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.477 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.477 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.478 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.478 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.478 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.478 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.478 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.478 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.478 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.478 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.478 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.478 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.478 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.478 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.478 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.479 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.479 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.479 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.479 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.479 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.479 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.479 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.479 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.479 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.479 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.479 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.479 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.479 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.480 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.480 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.480 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.480 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.480 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.480 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.480 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.480 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.480 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.480 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.480 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.480 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.480 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.481 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.481 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.481 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.481 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.481 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.481 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.481 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.481 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.481 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.481 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.481 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.481 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.481 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.482 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.482 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.482 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.482 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.482 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.482 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.482 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.482 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_clones": { 2024-05-21T15:42:35.482 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_clones", 2024-05-21T15:42:35.482 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.482 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.482 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.482 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4", 2024-05-21T15:42:35.483 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.483 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.483 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.483 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-21T15:42:35.483 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.483 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.483 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.483 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.483 INFO:teuthology.orchestra.run.smithi038.stdout: "periodic_async_work": { 2024-05-21T15:42:35.483 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "periodic_async_work", 2024-05-21T15:42:35.483 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.483 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.483 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.484 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.484 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.484 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.484 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.484 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Periodically check for async work", 2024-05-21T15:42:35.484 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.484 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.484 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.484 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.484 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_delay": { 2024-05-21T15:42:35.484 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_delay", 2024-05-21T15:42:35.484 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.485 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.485 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.485 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:35.485 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.485 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.485 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.485 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-21T15:42:35.485 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.485 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.485 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.485 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.485 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_no_wait": { 2024-05-21T15:42:35.485 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_no_wait", 2024-05-21T15:42:35.486 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.486 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.486 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.486 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.486 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.486 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.486 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.486 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-21T15:42:35.486 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.486 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.486 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.486 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.487 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.487 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.487 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.487 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix", 2024-05-21T15:42:35.487 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.487 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.487 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.487 INFO:teuthology.orchestra.run.smithi038.stdout: "discovery_interval": { 2024-05-21T15:42:35.487 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "discovery_interval", 2024-05-21T15:42:35.487 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:35.487 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.487 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.487 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "100", 2024-05-21T15:42:35.487 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.488 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.488 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.488 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.488 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.488 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.488 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.488 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.488 INFO:teuthology.orchestra.run.smithi038.stdout: "identifier": { 2024-05-21T15:42:35.488 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "identifier", 2024-05-21T15:42:35.488 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.488 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.488 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.488 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.488 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.489 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.489 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.489 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.489 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.489 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.489 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.489 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.489 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:35.489 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:35.489 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.489 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.489 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.489 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:35.490 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.490 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.490 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.490 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.490 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.490 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.490 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.490 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.490 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.490 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.490 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.490 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.490 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.491 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.491 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.491 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.491 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.491 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.491 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.491 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.491 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.491 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.491 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.491 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.491 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.491 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.492 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.492 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.492 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.492 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.492 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.492 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.492 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.492 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.492 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.492 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.492 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.492 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.492 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.493 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.493 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.493 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.493 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.493 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.493 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.493 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.493 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.493 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.493 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.493 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.493 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.493 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.493 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.494 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.494 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.494 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.494 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.494 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.494 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.494 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.494 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.494 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.494 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.494 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.494 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.495 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.495 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.495 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.495 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.495 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.495 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.495 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.495 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.495 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.495 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.495 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.495 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.495 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.495 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_host": { 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_host", 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_port": { 2024-05-21T15:42:35.496 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_port", 2024-05-21T15:42:35.497 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.497 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.497 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.497 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10051", 2024-05-21T15:42:35.497 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.497 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.497 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.497 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.497 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.497 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.497 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.497 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.497 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_sender": { 2024-05-21T15:42:35.497 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_sender", 2024-05-21T15:42:35.498 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.498 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.498 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.498 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-21T15:42:35.498 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.498 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.498 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.498 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.498 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.498 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.498 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.498 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.499 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.499 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.499 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-05-21T15:42:35.499 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.499 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.499 INFO:teuthology.orchestra.run.smithi038.stdout: "gid": 24124, 2024-05-21T15:42:35.499 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "y", 2024-05-21T15:42:35.499 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr_features": 4540701547738038271, 2024-05-21T15:42:35.499 INFO:teuthology.orchestra.run.smithi038.stdout: "available_modules": [ 2024-05-21T15:42:35.499 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.499 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "alerts", 2024-05-21T15:42:35.499 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.499 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.500 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.500 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:35.500 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:35.500 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.500 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.500 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.500 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:35.500 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.500 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.500 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.500 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How frequently to reexamine health status", 2024-05-21T15:42:35.500 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.500 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.500 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.501 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.501 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.501 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.501 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.501 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.501 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.501 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.501 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.501 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.501 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.501 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.501 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.501 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.501 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.502 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.502 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.502 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.502 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.502 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.502 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.502 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.502 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.502 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.502 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.502 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.502 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.503 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.503 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.503 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.503 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.503 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.503 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.503 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.503 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.503 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.503 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.503 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.503 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.503 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.504 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.504 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.504 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.504 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.504 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.504 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.504 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.504 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.504 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.504 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.504 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.504 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.504 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.505 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.505 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.505 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.505 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.505 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.505 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.505 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.505 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.505 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.505 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.505 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.505 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.505 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.506 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.506 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.506 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.506 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.506 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.506 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.506 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_destination": { 2024-05-21T15:42:35.506 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_destination", 2024-05-21T15:42:35.506 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.506 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.506 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.506 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.506 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.507 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.507 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.507 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email address to send alerts to", 2024-05-21T15:42:35.507 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.507 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.507 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.507 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.507 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_from_name": { 2024-05-21T15:42:35.507 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_from_name", 2024-05-21T15:42:35.507 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.507 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.507 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.507 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "Ceph", 2024-05-21T15:42:35.507 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.508 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.508 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.508 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email From: name", 2024-05-21T15:42:35.508 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.508 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.508 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.508 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.508 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_host": { 2024-05-21T15:42:35.508 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_host", 2024-05-21T15:42:35.508 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.508 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.508 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.508 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.508 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP server", 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_password": { 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_password", 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.509 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.510 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.510 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Password to authenticate with", 2024-05-21T15:42:35.510 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.510 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.510 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.510 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.510 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_port": { 2024-05-21T15:42:35.510 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_port", 2024-05-21T15:42:35.510 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.510 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.510 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.510 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "465", 2024-05-21T15:42:35.511 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.511 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.511 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.511 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP port", 2024-05-21T15:42:35.511 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.511 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.511 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.511 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.511 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_sender": { 2024-05-21T15:42:35.511 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_sender", 2024-05-21T15:42:35.511 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.511 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.511 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.511 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.512 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.512 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.512 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.512 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP envelope sender", 2024-05-21T15:42:35.512 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.512 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.512 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.512 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.512 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_ssl": { 2024-05-21T15:42:35.512 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_ssl", 2024-05-21T15:42:35.512 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.512 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.512 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.512 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.513 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.513 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.513 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.513 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-21T15:42:35.513 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.513 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.513 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.513 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.513 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_user": { 2024-05-21T15:42:35.513 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_user", 2024-05-21T15:42:35.513 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.513 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.514 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.514 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.514 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.514 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.514 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.514 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "User to authenticate as", 2024-05-21T15:42:35.514 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.514 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.514 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.514 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.514 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.514 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.514 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.515 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "balancer", 2024-05-21T15:42:35.515 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.515 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.515 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.515 INFO:teuthology.orchestra.run.smithi038.stdout: "active": { 2024-05-21T15:42:35.515 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "active", 2024-05-21T15:42:35.515 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.515 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.515 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.515 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.515 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.515 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.515 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.516 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically balance PGs across cluster", 2024-05-21T15:42:35.516 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.516 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.516 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.516 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.516 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_time": { 2024-05-21T15:42:35.516 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_time", 2024-05-21T15:42:35.516 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.516 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.516 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0000", 2024-05-21T15:42:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "beginning time of day to automatically balance", 2024-05-21T15:42:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T15:42:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_weekday": { 2024-05-21T15:42:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_weekday", 2024-05-21T15:42:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.518 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.518 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:35.518 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-05-21T15:42:35.518 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-05-21T15:42:35.518 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.518 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-21T15:42:35.518 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T15:42:35.518 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.518 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.518 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.518 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_max_iterations": { 2024-05-21T15:42:35.518 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_max_iterations", 2024-05-21T15:42:35.519 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:35.519 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.519 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.519 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "25", 2024-05-21T15:42:35.519 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-05-21T15:42:35.519 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "250", 2024-05-21T15:42:35.519 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.519 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-21T15:42:35.519 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.519 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.519 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.519 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.519 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_metrics": { 2024-05-21T15:42:35.519 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_metrics", 2024-05-21T15:42:35.520 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.520 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.520 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.520 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "pgs,objects,bytes", 2024-05-21T15:42:35.520 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.520 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.520 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.520 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-21T15:42:35.520 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-21T15:42:35.520 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.520 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.520 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.520 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_step": { 2024-05-21T15:42:35.521 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_step", 2024-05-21T15:42:35.521 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.521 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.521 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.521 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.5", 2024-05-21T15:42:35.521 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0.001", 2024-05-21T15:42:35.521 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "0.999", 2024-05-21T15:42:35.521 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.521 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "aggressiveness of optimization", 2024-05-21T15:42:35.521 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-21T15:42:35.521 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.521 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.521 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.521 INFO:teuthology.orchestra.run.smithi038.stdout: "end_time": { 2024-05-21T15:42:35.522 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_time", 2024-05-21T15:42:35.522 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.522 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.522 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.522 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2359", 2024-05-21T15:42:35.522 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.522 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.522 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.522 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ending time of day to automatically balance", 2024-05-21T15:42:35.522 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T15:42:35.522 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.522 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.523 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.523 INFO:teuthology.orchestra.run.smithi038.stdout: "end_weekday": { 2024-05-21T15:42:35.523 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_weekday", 2024-05-21T15:42:35.523 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:35.523 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.523 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.523 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:35.523 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-05-21T15:42:35.523 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-05-21T15:42:35.523 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.523 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-21T15:42:35.523 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T15:42:35.524 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.524 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.524 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.524 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.524 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.524 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.524 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.524 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.524 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.524 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.524 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.524 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.524 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.524 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.525 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.525 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.525 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.525 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.525 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.525 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.525 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.525 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.525 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.525 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.525 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.525 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.525 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.525 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.526 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.526 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.526 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.526 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.526 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.526 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.526 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.526 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.526 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.526 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.526 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.526 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.526 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.527 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.527 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.527 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.527 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.527 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.527 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.527 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.527 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.527 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.527 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.527 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.527 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.528 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.528 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.528 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.528 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.528 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.528 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.528 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.528 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.528 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.528 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.528 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.528 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.528 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.528 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.529 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.529 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.529 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.529 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.529 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.529 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.529 INFO:teuthology.orchestra.run.smithi038.stdout: "min_score": { 2024-05-21T15:42:35.529 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_score", 2024-05-21T15:42:35.529 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.529 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.529 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.529 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:35.529 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.530 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.530 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.530 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-21T15:42:35.530 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.530 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.530 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.530 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.530 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-05-21T15:42:35.530 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-05-21T15:42:35.530 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.530 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.530 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.531 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "upmap", 2024-05-21T15:42:35.531 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.531 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.531 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.531 INFO:teuthology.orchestra.run.smithi038.stdout: "crush-compat", 2024-05-21T15:42:35.531 INFO:teuthology.orchestra.run.smithi038.stdout: "none", 2024-05-21T15:42:35.531 INFO:teuthology.orchestra.run.smithi038.stdout: "read", 2024-05-21T15:42:35.531 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap", 2024-05-21T15:42:35.531 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap-read" 2024-05-21T15:42:35.531 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.531 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Balancer mode", 2024-05-21T15:42:35.531 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.531 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.532 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.532 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.532 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_ids": { 2024-05-21T15:42:35.532 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_ids", 2024-05-21T15:42:35.532 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.532 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.532 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.532 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.532 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.532 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.532 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.532 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-21T15:42:35.532 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.532 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.533 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.533 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.533 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:35.533 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:35.533 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.533 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.533 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.533 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:35.533 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.533 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.533 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.533 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-21T15:42:35.533 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.534 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.534 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.534 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.534 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_deviation": { 2024-05-21T15:42:35.534 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_deviation", 2024-05-21T15:42:35.534 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.534 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.534 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.534 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-05-21T15:42:35.534 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-05-21T15:42:35.534 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.534 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.535 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-21T15:42:35.535 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-21T15:42:35.535 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.535 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.535 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.535 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_optimizations": { 2024-05-21T15:42:35.535 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_optimizations", 2024-05-21T15:42:35.535 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:35.535 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.535 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.535 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.535 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.535 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm", 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_down_multiplier": { 2024-05-21T15:42:35.536 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_down_multiplier", 2024-05-21T15:42:35.537 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.537 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.537 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.537 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-05-21T15:42:35.537 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.537 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.537 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.537 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-21T15:42:35.537 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.537 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.537 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.537 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.537 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_refresh_rate": { 2024-05-21T15:42:35.538 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_refresh_rate", 2024-05-21T15:42:35.538 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.538 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.538 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.538 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "20", 2024-05-21T15:42:35.538 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.538 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.538 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.538 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-21T15:42:35.538 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.538 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.538 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.538 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.539 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_starting_port": { 2024-05-21T15:42:35.539 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_starting_port", 2024-05-21T15:42:35.539 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.539 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.539 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.539 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4721", 2024-05-21T15:42:35.539 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.539 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.539 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.539 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-21T15:42:35.539 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.539 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.539 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.539 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.540 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_ptrace": { 2024-05-21T15:42:35.540 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_ptrace", 2024-05-21T15:42:35.540 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.540 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.540 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.540 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.540 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.540 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.540 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.540 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-21T15:42:35.540 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-21T15:42:35.540 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.541 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.541 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.541 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_interval": { 2024-05-21T15:42:35.541 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_interval", 2024-05-21T15:42:35.541 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.541 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.541 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.541 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:35.541 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.541 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.541 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.541 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-21T15:42:35.541 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.542 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.542 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.542 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.542 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_memory_target_ratio": { 2024-05-21T15:42:35.542 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_memory_target_ratio", 2024-05-21T15:42:35.542 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.542 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.542 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.542 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.7", 2024-05-21T15:42:35.542 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.542 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.542 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.543 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-21T15:42:35.543 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.543 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.543 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.543 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.543 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm_log_destination": { 2024-05-21T15:42:35.543 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm_log_destination", 2024-05-21T15:42:35.543 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.543 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.543 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.543 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.543 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.543 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.543 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.544 INFO:teuthology.orchestra.run.smithi038.stdout: "file", 2024-05-21T15:42:35.544 INFO:teuthology.orchestra.run.smithi038.stdout: "file,syslog", 2024-05-21T15:42:35.544 INFO:teuthology.orchestra.run.smithi038.stdout: "syslog" 2024-05-21T15:42:35.544 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.544 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-21T15:42:35.544 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.544 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.544 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.544 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.544 INFO:teuthology.orchestra.run.smithi038.stdout: "cgroups_split": { 2024-05-21T15:42:35.544 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cgroups_split", 2024-05-21T15:42:35.544 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.544 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.545 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.545 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.545 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.545 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.545 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.545 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-21T15:42:35.545 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.545 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.545 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.545 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.545 INFO:teuthology.orchestra.run.smithi038.stdout: "config_checks_enabled": { 2024-05-21T15:42:35.545 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_checks_enabled", 2024-05-21T15:42:35.545 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.545 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.546 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.546 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.546 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.546 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.546 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.546 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-21T15:42:35.546 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.546 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.546 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.546 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.546 INFO:teuthology.orchestra.run.smithi038.stdout: "config_dashboard": { 2024-05-21T15:42:35.546 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_dashboard", 2024-05-21T15:42:35.546 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.547 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.547 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.547 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.547 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.547 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.547 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.547 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-21T15:42:35.547 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.547 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.547 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.547 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.547 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_alertmanager": { 2024-05-21T15:42:35.547 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_alertmanager", 2024-05-21T15:42:35.548 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.548 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.548 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.548 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-21T15:42:35.548 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.548 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.548 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.548 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-05-21T15:42:35.548 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.548 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.548 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.548 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.548 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_base": { 2024-05-21T15:42:35.549 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_base", 2024-05-21T15:42:35.549 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.549 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.549 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.549 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-21T15:42:35.549 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.549 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.549 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.549 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Container image name, without the tag", 2024-05-21T15:42:35.549 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.549 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.549 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.549 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.549 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_elasticsearch": { 2024-05-21T15:42:35.550 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_elasticsearch", 2024-05-21T15:42:35.550 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.550 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.550 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.550 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-21T15:42:35.550 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.550 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.550 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.550 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "elasticsearch container image", 2024-05-21T15:42:35.550 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.550 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.550 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.550 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.551 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_grafana": { 2024-05-21T15:42:35.551 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_grafana", 2024-05-21T15:42:35.551 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.551 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.551 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.551 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-21T15:42:35.551 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.551 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.551 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.551 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-05-21T15:42:35.551 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.551 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.552 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.552 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.552 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_haproxy": { 2024-05-21T15:42:35.552 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_haproxy", 2024-05-21T15:42:35.552 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.552 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.552 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.552 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-21T15:42:35.552 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.552 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.552 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.552 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "HAproxy container image", 2024-05-21T15:42:35.552 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.553 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.553 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.553 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.553 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_agent": { 2024-05-21T15:42:35.553 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_agent", 2024-05-21T15:42:35.553 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.553 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.553 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.553 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-21T15:42:35.553 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.553 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.553 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.554 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger agent container image", 2024-05-21T15:42:35.554 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.554 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.554 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.554 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.554 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_collector": { 2024-05-21T15:42:35.554 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_collector", 2024-05-21T15:42:35.554 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.554 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.554 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.555 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-21T15:42:35.555 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.555 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.555 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.555 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger collector container image", 2024-05-21T15:42:35.555 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.555 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.555 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.555 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.555 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_query": { 2024-05-21T15:42:35.555 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_query", 2024-05-21T15:42:35.555 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.555 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.556 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.556 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-21T15:42:35.556 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.556 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.556 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.556 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger query container image", 2024-05-21T15:42:35.556 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.556 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.556 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.556 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.556 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_keepalived": { 2024-05-21T15:42:35.556 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_keepalived", 2024-05-21T15:42:35.556 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.557 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.557 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.557 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-21T15:42:35.557 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.557 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.557 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.557 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Keepalived container image", 2024-05-21T15:42:35.557 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.557 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.557 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.557 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.557 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_loki": { 2024-05-21T15:42:35.557 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_loki", 2024-05-21T15:42:35.558 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.558 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.558 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.558 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-21T15:42:35.558 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.558 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.558 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.558 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Loki container image", 2024-05-21T15:42:35.558 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.558 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.558 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.558 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.559 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_node_exporter": { 2024-05-21T15:42:35.559 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_node_exporter", 2024-05-21T15:42:35.559 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.559 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.559 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.559 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-21T15:42:35.559 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.559 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.559 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.559 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-05-21T15:42:35.559 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.559 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.559 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.560 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.560 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_nvmeof": { 2024-05-21T15:42:35.560 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_nvmeof", 2024-05-21T15:42:35.560 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.560 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.560 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.560 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-21T15:42:35.560 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.560 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.560 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.560 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Nvme-of container image", 2024-05-21T15:42:35.560 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.560 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.560 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_prometheus": { 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_prometheus", 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.561 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.562 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_promtail": { 2024-05-21T15:42:35.562 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_promtail", 2024-05-21T15:42:35.562 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.562 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.562 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.562 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-21T15:42:35.562 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.562 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.562 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.562 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Promtail container image", 2024-05-21T15:42:35.562 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.562 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.562 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.563 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.563 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_samba": { 2024-05-21T15:42:35.563 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_samba", 2024-05-21T15:42:35.563 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.563 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.563 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.563 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-21T15:42:35.563 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.563 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.563 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.563 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Samba/SMB container image", 2024-05-21T15:42:35.563 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.564 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.564 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.564 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.564 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_snmp_gateway": { 2024-05-21T15:42:35.564 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_snmp_gateway", 2024-05-21T15:42:35.564 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.564 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.564 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.564 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-21T15:42:35.564 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.564 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.564 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.564 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SNMP Gateway container image", 2024-05-21T15:42:35.564 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.565 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.565 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.565 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.565 INFO:teuthology.orchestra.run.smithi038.stdout: "container_init": { 2024-05-21T15:42:35.565 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_init", 2024-05-21T15:42:35.565 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.565 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.565 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.565 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.565 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.565 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.565 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.565 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Run podman/docker with `--init`", 2024-05-21T15:42:35.566 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.566 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.566 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.566 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.566 INFO:teuthology.orchestra.run.smithi038.stdout: "daemon_cache_timeout": { 2024-05-21T15:42:35.566 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "daemon_cache_timeout", 2024-05-21T15:42:35.566 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.566 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.566 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.566 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:35.566 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.566 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.566 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.567 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-21T15:42:35.567 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.567 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.567 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.567 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.567 INFO:teuthology.orchestra.run.smithi038.stdout: "default_cephadm_command_timeout": { 2024-05-21T15:42:35.567 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_cephadm_command_timeout", 2024-05-21T15:42:35.567 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.567 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.567 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.567 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "900", 2024-05-21T15:42:35.567 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.567 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-21T15:42:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: "default_registry": { 2024-05-21T15:42:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_registry", 2024-05-21T15:42:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io", 2024-05-21T15:42:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.569 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.569 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-21T15:42:35.569 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.569 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.569 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.569 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.569 INFO:teuthology.orchestra.run.smithi038.stdout: "device_cache_timeout": { 2024-05-21T15:42:35.569 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_cache_timeout", 2024-05-21T15:42:35.569 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.569 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.569 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.569 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1800", 2024-05-21T15:42:35.569 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.569 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.570 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.570 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache device inventory", 2024-05-21T15:42:35.570 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.570 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.570 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.570 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.570 INFO:teuthology.orchestra.run.smithi038.stdout: "device_enhanced_scan": { 2024-05-21T15:42:35.570 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_enhanced_scan", 2024-05-21T15:42:35.570 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.570 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.570 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.570 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.570 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.571 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.571 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.571 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-21T15:42:35.571 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.571 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.571 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.571 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.571 INFO:teuthology.orchestra.run.smithi038.stdout: "facts_cache_timeout": { 2024-05-21T15:42:35.571 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "facts_cache_timeout", 2024-05-21T15:42:35.571 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.571 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.571 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.571 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:35.572 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.572 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.572 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.572 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache host facts data", 2024-05-21T15:42:35.572 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.572 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.572 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.572 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.572 INFO:teuthology.orchestra.run.smithi038.stdout: "grafana_dashboards_path": { 2024-05-21T15:42:35.572 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "grafana_dashboards_path", 2024-05-21T15:42:35.572 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.572 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.572 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.572 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-21T15:42:35.573 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.573 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.573 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.573 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-21T15:42:35.573 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.573 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.573 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.573 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.573 INFO:teuthology.orchestra.run.smithi038.stdout: "host_check_interval": { 2024-05-21T15:42:35.573 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "host_check_interval", 2024-05-21T15:42:35.573 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.573 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.573 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.573 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:35.574 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.574 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.574 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.574 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to perform a host check", 2024-05-21T15:42:35.574 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.574 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.574 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.574 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.574 INFO:teuthology.orchestra.run.smithi038.stdout: "hw_monitoring": { 2024-05-21T15:42:35.574 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hw_monitoring", 2024-05-21T15:42:35.574 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.574 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.574 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.575 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.575 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.575 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.575 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.575 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-21T15:42:35.575 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.575 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.575 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.575 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.575 INFO:teuthology.orchestra.run.smithi038.stdout: "inventory_list_all": { 2024-05-21T15:42:35.575 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "inventory_list_all", 2024-05-21T15:42:35.575 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.575 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.575 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.576 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.576 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.576 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.576 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.576 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-21T15:42:35.576 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.576 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.576 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.576 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.576 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.576 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.576 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.576 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.577 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.577 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.577 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.577 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.577 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.577 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.577 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.577 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.577 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.577 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.577 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.577 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.577 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.578 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.578 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.578 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.578 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.578 INFO:teuthology.orchestra.run.smithi038.stdout: "log_refresh_metadata": { 2024-05-21T15:42:35.578 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_refresh_metadata", 2024-05-21T15:42:35.578 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.578 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.578 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.578 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.578 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.578 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.578 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.579 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-21T15:42:35.579 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.579 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.579 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.579 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.579 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.579 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.579 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.579 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.579 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.579 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.579 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.579 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.580 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.580 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-21T15:42:35.580 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.580 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.580 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.580 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.580 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.580 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.580 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.580 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.580 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.580 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.580 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.581 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.581 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.581 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.581 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.581 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.581 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.581 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.581 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.581 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.581 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.581 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.581 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.582 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.582 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.582 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.582 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.582 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.582 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.582 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.582 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.582 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.582 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.582 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.582 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.582 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.583 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.583 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.583 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.583 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-21T15:42:35.583 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-21T15:42:35.583 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.583 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.583 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.583 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.583 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.583 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.583 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.583 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-21T15:42:35.584 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.584 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.584 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.584 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.584 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-21T15:42:35.584 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-21T15:42:35.584 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.584 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.584 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.584 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "*", 2024-05-21T15:42:35.584 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.584 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.584 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.584 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-21T15:42:35.585 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.585 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.585 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.585 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.585 INFO:teuthology.orchestra.run.smithi038.stdout: "max_count_per_host": { 2024-05-21T15:42:35.585 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_count_per_host", 2024-05-21T15:42:35.585 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.585 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.585 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.585 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.585 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.585 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.585 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.586 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of daemons per service per host", 2024-05-21T15:42:35.586 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.586 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.586 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.586 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.586 INFO:teuthology.orchestra.run.smithi038.stdout: "max_osd_draining_count": { 2024-05-21T15:42:35.586 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_osd_draining_count", 2024-05-21T15:42:35.586 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.586 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.586 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.586 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.586 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.586 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.587 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.587 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-21T15:42:35.587 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.587 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.587 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.587 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.587 INFO:teuthology.orchestra.run.smithi038.stdout: "migration_current": { 2024-05-21T15:42:35.587 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "migration_current", 2024-05-21T15:42:35.587 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.587 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.587 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.587 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.587 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.588 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.588 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.588 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "internal - do not modify", 2024-05-21T15:42:35.588 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.588 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.588 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.588 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.588 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-05-21T15:42:35.588 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-05-21T15:42:35.588 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.588 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.588 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.588 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "root", 2024-05-21T15:42:35.588 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.589 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.589 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.589 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm-package", 2024-05-21T15:42:35.589 INFO:teuthology.orchestra.run.smithi038.stdout: "root" 2024-05-21T15:42:35.589 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.589 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "mode for remote execution of cephadm", 2024-05-21T15:42:35.589 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.589 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.589 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.589 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.589 INFO:teuthology.orchestra.run.smithi038.stdout: "oob_default_addr": { 2024-05-21T15:42:35.589 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "oob_default_addr", 2024-05-21T15:42:35.590 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.590 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.590 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.590 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "169.254.1.1", 2024-05-21T15:42:35.590 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.590 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.590 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.590 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-21T15:42:35.590 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.590 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.590 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.590 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.590 INFO:teuthology.orchestra.run.smithi038.stdout: "prometheus_alerts_path": { 2024-05-21T15:42:35.590 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus_alerts_path", 2024-05-21T15:42:35.591 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.591 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.591 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.591 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-21T15:42:35.591 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.591 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.591 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.591 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-21T15:42:35.591 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.591 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.591 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.591 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.591 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_insecure": { 2024-05-21T15:42:35.592 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_insecure", 2024-05-21T15:42:35.592 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.592 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.592 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.592 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.592 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.592 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.592 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.592 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-21T15:42:35.592 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.592 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.592 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.593 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.593 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_password": { 2024-05-21T15:42:35.593 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_password", 2024-05-21T15:42:35.593 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.593 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.593 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.593 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.593 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.593 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.593 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.593 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-21T15:42:35.593 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.593 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.594 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.594 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.594 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_url": { 2024-05-21T15:42:35.594 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_url", 2024-05-21T15:42:35.594 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.594 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.594 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.594 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.594 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.594 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.594 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.594 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-21T15:42:35.595 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.595 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.595 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.595 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.595 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_username": { 2024-05-21T15:42:35.595 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_username", 2024-05-21T15:42:35.595 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.595 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.595 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.595 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.595 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.595 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.595 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.595 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-21T15:42:35.596 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.596 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.596 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.596 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.596 INFO:teuthology.orchestra.run.smithi038.stdout: "secure_monitoring_stack": { 2024-05-21T15:42:35.596 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secure_monitoring_stack", 2024-05-21T15:42:35.596 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.596 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.596 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.596 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.596 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.596 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.596 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.597 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-21T15:42:35.597 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.597 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.597 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.597 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.597 INFO:teuthology.orchestra.run.smithi038.stdout: "service_discovery_port": { 2024-05-21T15:42:35.597 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "service_discovery_port", 2024-05-21T15:42:35.597 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.597 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.597 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.597 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8765", 2024-05-21T15:42:35.597 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.597 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.598 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.598 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "cephadm service discovery port", 2024-05-21T15:42:35.598 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.598 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.598 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.598 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.598 INFO:teuthology.orchestra.run.smithi038.stdout: "ssh_config_file": { 2024-05-21T15:42:35.598 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssh_config_file", 2024-05-21T15:42:35.598 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.598 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.598 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.598 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.599 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.599 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.599 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.599 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-21T15:42:35.599 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.599 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.599 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.599 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.599 INFO:teuthology.orchestra.run.smithi038.stdout: "use_agent": { 2024-05-21T15:42:35.599 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_agent", 2024-05-21T15:42:35.599 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.599 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.599 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.600 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.600 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.600 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.600 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.600 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-21T15:42:35.600 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.600 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.600 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.600 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.600 INFO:teuthology.orchestra.run.smithi038.stdout: "use_repo_digest": { 2024-05-21T15:42:35.600 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_repo_digest", 2024-05-21T15:42:35.600 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.600 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.600 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.601 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.601 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.601 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.601 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.601 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-21T15:42:35.601 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.601 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.601 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.601 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.601 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_failed_host_check": { 2024-05-21T15:42:35.601 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_failed_host_check", 2024-05-21T15:42:35.601 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.601 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.601 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.602 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.602 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.602 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.602 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.602 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if the host check fails", 2024-05-21T15:42:35.602 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.602 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.602 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.602 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.602 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_daemons": { 2024-05-21T15:42:35.602 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_daemons", 2024-05-21T15:42:35.602 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_hosts": { 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_hosts", 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.603 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.604 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.604 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.604 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.604 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.604 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.604 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-21T15:42:35.604 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.604 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.604 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.604 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.604 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.604 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.604 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.605 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crash", 2024-05-21T15:42:35.605 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.605 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.605 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.605 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.605 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.605 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.605 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.605 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.605 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.605 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.605 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.605 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.605 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.606 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.606 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.606 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.606 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.606 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.606 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.606 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.606 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.606 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.606 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.606 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.606 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.607 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.607 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.607 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.607 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.607 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.607 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.607 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.607 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.607 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.607 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.607 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.607 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.607 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.608 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.608 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.608 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.608 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.608 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.608 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.608 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.608 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.608 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.608 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.608 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.608 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.608 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.609 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.609 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.609 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.609 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.609 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.609 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.609 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.609 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.609 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.609 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.609 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.609 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.609 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.610 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.610 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.610 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.610 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.610 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.610 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.610 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.610 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.610 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.610 INFO:teuthology.orchestra.run.smithi038.stdout: "retain_interval": { 2024-05-21T15:42:35.610 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retain_interval", 2024-05-21T15:42:35.610 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.610 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.611 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.611 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "31536000", 2024-05-21T15:42:35.611 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.611 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.611 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.611 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-21T15:42:35.611 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.611 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.611 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.611 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.611 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_recent_interval": { 2024-05-21T15:42:35.611 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_recent_interval", 2024-05-21T15:42:35.611 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.612 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.612 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.612 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1209600", 2024-05-21T15:42:35.612 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.612 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.612 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.612 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-21T15:42:35.612 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.612 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.612 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.612 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.612 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.612 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.613 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.613 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dashboard", 2024-05-21T15:42:35.613 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.613 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.613 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.613 INFO:teuthology.orchestra.run.smithi038.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-21T15:42:35.613 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-21T15:42:35.613 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.613 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.613 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.613 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.613 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.613 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.613 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.614 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.614 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.614 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.614 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.614 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.614 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_HOST": { 2024-05-21T15:42:35.614 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-21T15:42:35.614 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.614 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.614 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.614 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.614 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.614 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.615 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.615 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.615 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.615 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.615 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.615 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.615 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-21T15:42:35.615 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-21T15:42:35.615 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.615 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.615 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.615 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.615 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.615 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.616 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.616 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.616 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.616 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.616 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.616 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.616 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_ENABLED": { 2024-05-21T15:42:35.616 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_ENABLED", 2024-05-21T15:42:35.616 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.616 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.616 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.616 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.616 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.617 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.617 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.617 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.617 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.617 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.617 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.617 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.617 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-21T15:42:35.617 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-21T15:42:35.617 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.617 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.617 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.617 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.618 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.618 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.618 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.618 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.618 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.618 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.618 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.618 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.618 INFO:teuthology.orchestra.run.smithi038.stdout: "ENABLE_BROWSABLE_API": { 2024-05-21T15:42:35.618 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-21T15:42:35.618 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.618 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.618 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.619 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.619 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.619 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.619 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.619 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.619 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.619 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.619 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.619 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.619 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-21T15:42:35.619 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-21T15:42:35.619 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.619 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.620 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.620 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.620 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.620 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.620 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.620 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.620 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.620 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.620 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.620 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.620 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-21T15:42:35.620 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-21T15:42:35.621 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.621 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.621 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.621 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.621 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.621 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.621 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.621 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.621 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.621 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.621 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.621 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.621 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-21T15:42:35.622 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-21T15:42:35.622 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.622 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.622 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.622 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.622 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.622 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.622 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.622 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.622 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.622 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.622 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.622 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.623 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-21T15:42:35.623 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-21T15:42:35.623 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.623 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.623 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.623 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.623 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.623 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.623 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.623 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.623 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.623 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.623 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.624 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.624 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-21T15:42:35.624 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-21T15:42:35.624 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.624 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.624 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.624 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.624 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.624 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.624 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.624 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.624 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.624 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.624 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.625 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.625 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-21T15:42:35.625 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-21T15:42:35.625 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.625 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.625 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.625 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.625 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.625 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.625 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.625 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.625 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.625 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.626 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.626 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.626 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-21T15:42:35.626 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-21T15:42:35.626 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.626 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.626 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.626 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.626 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.626 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.626 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.626 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.626 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.626 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.627 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.627 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.627 INFO:teuthology.orchestra.run.smithi038.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-21T15:42:35.627 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-21T15:42:35.627 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.627 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.627 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.627 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.627 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.627 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.627 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.627 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.627 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.628 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.628 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.628 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.628 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_PASSWORD": { 2024-05-21T15:42:35.628 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-21T15:42:35.628 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.628 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.628 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.628 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-05-21T15:42:35.628 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.628 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.628 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.628 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.629 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.629 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.629 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.629 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.629 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-21T15:42:35.629 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-21T15:42:35.629 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.629 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.629 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.629 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.629 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.629 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.629 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.629 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.630 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.630 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.630 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.630 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.630 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_URL": { 2024-05-21T15:42:35.630 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_URL", 2024-05-21T15:42:35.630 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.630 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.630 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.630 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.630 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.630 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.630 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.631 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.631 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.631 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.631 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.631 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.631 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_USERNAME": { 2024-05-21T15:42:35.631 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-21T15:42:35.631 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.631 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.631 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.631 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-05-21T15:42:35.631 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.631 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.631 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-21T15:42:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-21T15:42:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-21T15:42:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-21T15:42:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.634 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.634 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.634 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.634 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.634 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.634 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.634 INFO:teuthology.orchestra.run.smithi038.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-21T15:42:35.634 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-21T15:42:35.634 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.634 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.634 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.634 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.635 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.635 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.635 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.635 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.635 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.635 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.635 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.635 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.635 INFO:teuthology.orchestra.run.smithi038.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-21T15:42:35.635 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-21T15:42:35.635 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.635 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.635 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.635 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.636 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.636 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.636 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.636 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.636 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.636 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.636 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.636 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.636 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_HOST": { 2024-05-21T15:42:35.636 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-21T15:42:35.636 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.636 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.636 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.637 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.637 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.637 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.637 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.637 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.637 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.637 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.637 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.637 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.637 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-21T15:42:35.637 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-21T15:42:35.637 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.638 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.638 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.638 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.638 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.638 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.638 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.638 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.638 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.638 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.638 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.638 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.638 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-21T15:42:35.638 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-21T15:42:35.639 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.639 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.639 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.639 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.639 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.639 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.639 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.639 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.639 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.639 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.639 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.639 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.639 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-21T15:42:35.640 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-21T15:42:35.640 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.640 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.640 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.640 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.640 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.640 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.640 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.640 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.640 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.640 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.640 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.640 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.641 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-21T15:42:35.641 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-21T15:42:35.641 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.641 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.641 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.641 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.641 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.641 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.641 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.641 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.641 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.641 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.641 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.641 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.642 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-21T15:42:35.642 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-21T15:42:35.642 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.642 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.642 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.642 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.642 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.642 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.642 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.642 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.642 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.642 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.642 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.643 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.643 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-21T15:42:35.643 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-21T15:42:35.643 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.643 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.643 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.643 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.643 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.643 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.643 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.643 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.643 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.643 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.644 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.644 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.644 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-21T15:42:35.644 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-21T15:42:35.644 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.644 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.644 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.644 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.644 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.644 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.644 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.644 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.644 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.644 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.645 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.645 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.645 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-21T15:42:35.645 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-21T15:42:35.645 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.645 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.645 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.645 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.645 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.645 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.645 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.645 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.645 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.645 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.646 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.646 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.646 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_ENABLED": { 2024-05-21T15:42:35.646 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-21T15:42:35.646 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.646 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.646 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.646 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.646 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.646 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.646 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.646 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.646 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.647 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.647 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.647 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.647 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-21T15:42:35.647 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-21T15:42:35.647 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.647 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.647 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.647 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-21T15:42:35.647 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.647 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.647 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.647 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.648 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.648 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.648 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.648 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.648 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-21T15:42:35.648 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-21T15:42:35.648 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.648 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.648 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.648 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.648 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.648 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.648 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.649 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.649 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.649 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.649 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.649 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.649 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-21T15:42:35.649 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-21T15:42:35.649 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.649 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.649 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.649 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8", 2024-05-21T15:42:35.649 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.649 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.649 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.650 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.650 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.650 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.650 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.650 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.650 INFO:teuthology.orchestra.run.smithi038.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-21T15:42:35.650 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-21T15:42:35.650 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.650 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.650 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.650 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "45", 2024-05-21T15:42:35.650 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.650 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.651 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.651 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.651 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.651 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.651 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.651 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.651 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ACCESS_KEY": { 2024-05-21T15:42:35.651 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-21T15:42:35.651 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.651 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.651 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.651 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.651 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.652 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.652 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.652 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.652 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.652 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.652 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.652 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.652 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-21T15:42:35.652 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-21T15:42:35.652 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.652 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.652 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.652 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-05-21T15:42:35.652 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.653 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.653 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.653 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.653 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.653 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.653 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.653 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.653 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SECRET_KEY": { 2024-05-21T15:42:35.653 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-21T15:42:35.653 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.653 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.653 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.653 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.654 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.654 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.654 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.654 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.654 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.654 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.654 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.654 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.654 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SSL_VERIFY": { 2024-05-21T15:42:35.654 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-21T15:42:35.654 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.654 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.654 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.655 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.655 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.655 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.655 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.655 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.655 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.655 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.655 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.655 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.655 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-21T15:42:35.655 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-21T15:42:35.655 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.655 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.656 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.656 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:35.656 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.656 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.656 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.656 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.656 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.656 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.656 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.656 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.656 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-21T15:42:35.656 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-21T15:42:35.656 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.656 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.657 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.657 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.657 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.657 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.657 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.657 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.657 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.657 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.657 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.657 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.657 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-21T15:42:35.657 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-21T15:42:35.657 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.657 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.658 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.658 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-05-21T15:42:35.658 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.658 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.658 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.658 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.658 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.658 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.658 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.658 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.658 INFO:teuthology.orchestra.run.smithi038.stdout: "cross_origin_url": { 2024-05-21T15:42:35.658 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cross_origin_url", 2024-05-21T15:42:35.658 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.659 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.659 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.659 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.659 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.659 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.659 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.659 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.659 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.659 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.659 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.659 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.659 INFO:teuthology.orchestra.run.smithi038.stdout: "crt_file": { 2024-05-21T15:42:35.659 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crt_file", 2024-05-21T15:42:35.659 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.660 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.660 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.660 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.660 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.660 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.660 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.660 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.660 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.660 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.660 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.660 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.660 INFO:teuthology.orchestra.run.smithi038.stdout: "debug": { 2024-05-21T15:42:35.660 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "debug", 2024-05-21T15:42:35.660 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.661 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.661 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.661 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.661 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.661 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.661 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.661 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable/disable debug options", 2024-05-21T15:42:35.661 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.661 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.661 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.661 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.661 INFO:teuthology.orchestra.run.smithi038.stdout: "jwt_token_ttl": { 2024-05-21T15:42:35.662 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "jwt_token_ttl", 2024-05-21T15:42:35.662 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.662 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.662 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.662 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "28800", 2024-05-21T15:42:35.662 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.662 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.662 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.662 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.662 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.662 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.662 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.663 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.663 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-05-21T15:42:35.663 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-05-21T15:42:35.663 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.663 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.663 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.663 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.663 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.663 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.663 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.663 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.663 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.663 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.663 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.664 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.665 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.665 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.665 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.665 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.665 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.665 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.666 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.667 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.667 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.667 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.667 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.667 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.667 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.667 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.667 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.667 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.667 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.667 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.667 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.667 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.667 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.668 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.668 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.668 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.668 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.668 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.668 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.668 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.668 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.668 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.668 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.668 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.668 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.668 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.669 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.669 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.669 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.669 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.669 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.669 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.669 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.669 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.669 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.669 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.669 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.669 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.669 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.670 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.670 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.670 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.670 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.670 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.670 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.670 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.670 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.670 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.670 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.670 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.670 INFO:teuthology.orchestra.run.smithi038.stdout: "motd": { 2024-05-21T15:42:35.670 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "motd", 2024-05-21T15:42:35.670 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.671 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.671 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.671 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.671 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.671 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.671 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.671 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "The message of the day", 2024-05-21T15:42:35.671 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.671 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.671 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.671 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.671 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect_resolve_ip_addr": { 2024-05-21T15:42:35.671 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "redirect_resolve_ip_addr", 2024-05-21T15:42:35.672 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.672 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.672 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.672 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.672 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.672 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.672 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.672 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.672 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.672 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.672 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.672 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.672 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-05-21T15:42:35.672 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-05-21T15:42:35.673 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.673 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.673 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.673 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-05-21T15:42:35.673 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.673 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.673 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.673 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.673 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.673 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.673 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.673 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.673 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-05-21T15:42:35.673 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-05-21T15:42:35.674 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.674 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.674 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.674 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8080", 2024-05-21T15:42:35.674 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.674 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.674 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.674 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.674 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.674 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.674 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.674 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.674 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-05-21T15:42:35.675 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-05-21T15:42:35.675 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.675 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.675 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.675 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.675 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.675 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.675 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.675 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.675 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.675 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.675 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.675 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.675 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl_server_port": { 2024-05-21T15:42:35.676 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl_server_port", 2024-05-21T15:42:35.676 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.676 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.676 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.676 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8443", 2024-05-21T15:42:35.676 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.676 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.676 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.676 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.676 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.676 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.676 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.676 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.676 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-05-21T15:42:35.677 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-05-21T15:42:35.677 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.677 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.677 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.677 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "redirect", 2024-05-21T15:42:35.677 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.677 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.677 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.677 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.677 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect" 2024-05-21T15:42:35.677 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.677 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.678 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.678 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.678 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.678 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.678 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-05-21T15:42:35.678 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-05-21T15:42:35.678 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.678 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.678 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.678 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-05-21T15:42:35.678 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-05-21T15:42:35.678 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-05-21T15:42:35.678 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.678 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.679 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.679 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.679 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.679 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.679 INFO:teuthology.orchestra.run.smithi038.stdout: "url_prefix": { 2024-05-21T15:42:35.679 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url_prefix", 2024-05-21T15:42:35.679 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.679 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.679 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.679 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.679 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.679 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.679 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.679 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-05-21T15:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_monitoring": { 2024-05-21T15:42:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_monitoring", 2024-05-21T15:42:35.681 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.681 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.681 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.681 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.681 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.681 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.681 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.681 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "monitor device health metrics", 2024-05-21T15:42:35.681 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.681 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.681 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.681 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.681 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.681 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.682 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.682 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.682 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.682 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.682 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.682 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.682 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.682 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.682 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.682 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.682 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.682 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.683 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.683 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.683 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.683 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.683 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.683 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.683 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.683 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.683 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.683 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.683 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.683 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.684 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.684 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.684 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.684 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.684 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.684 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.684 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.684 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.684 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.684 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.684 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.684 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.684 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.685 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.685 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.685 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.685 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.685 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.685 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.685 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.685 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.685 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.685 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.685 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.685 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.685 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.686 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.686 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.686 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.686 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.686 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.686 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.686 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.686 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.686 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.686 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.686 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.686 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.686 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.686 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.687 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.687 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.687 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.687 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.687 INFO:teuthology.orchestra.run.smithi038.stdout: "mark_out_threshold": { 2024-05-21T15:42:35.687 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mark_out_threshold", 2024-05-21T15:42:35.687 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.687 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.687 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.687 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2419200", 2024-05-21T15:42:35.687 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.687 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.687 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.688 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-21T15:42:35.688 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.688 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.688 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.688 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.688 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_name": { 2024-05-21T15:42:35.688 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_name", 2024-05-21T15:42:35.688 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.688 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.688 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.688 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "device_health_metrics", 2024-05-21T15:42:35.688 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.688 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.688 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.689 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-21T15:42:35.689 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.689 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.689 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.689 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.689 INFO:teuthology.orchestra.run.smithi038.stdout: "retention_period": { 2024-05-21T15:42:35.689 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retention_period", 2024-05-21T15:42:35.689 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.689 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.689 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.689 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15552000", 2024-05-21T15:42:35.689 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.689 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.689 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.690 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain device health metrics", 2024-05-21T15:42:35.690 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.690 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.690 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.690 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.690 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_frequency": { 2024-05-21T15:42:35.690 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_frequency", 2024-05-21T15:42:35.690 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.690 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.690 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.690 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-05-21T15:42:35.690 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.690 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-21T15:42:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: "self_heal": { 2024-05-21T15:42:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "self_heal", 2024-05-21T15:42:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-21T15:42:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and check device health", 2024-05-21T15:42:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_threshold": { 2024-05-21T15:42:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_threshold", 2024-05-21T15:42:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7257600", 2024-05-21T15:42:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.694 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.694 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.694 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-21T15:42:35.694 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.694 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.694 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.694 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.694 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.694 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.694 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.694 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "diskprediction_local", 2024-05-21T15:42:35.694 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.694 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.695 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.695 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.695 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.695 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.695 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.695 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.695 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.695 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.695 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.695 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.695 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.695 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.695 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.695 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.696 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.696 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.696 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.696 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.696 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.696 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.696 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.696 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.696 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.696 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.696 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.696 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.696 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.697 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.697 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.697 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.697 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.697 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.697 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.697 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.697 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.697 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.697 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.697 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.697 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.697 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.697 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.698 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.698 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.698 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.698 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.698 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.698 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.698 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.698 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.698 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.698 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.698 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.698 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.699 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.699 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.699 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.699 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.699 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.699 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.699 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.699 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.699 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.699 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.699 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.699 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.699 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.699 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.700 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.700 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.700 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.700 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.700 INFO:teuthology.orchestra.run.smithi038.stdout: "predict_interval": { 2024-05-21T15:42:35.700 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predict_interval", 2024-05-21T15:42:35.700 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.700 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.700 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.700 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-05-21T15:42:35.700 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.700 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.700 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.701 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.701 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.701 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.701 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.701 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.701 INFO:teuthology.orchestra.run.smithi038.stdout: "predictor_model": { 2024-05-21T15:42:35.701 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predictor_model", 2024-05-21T15:42:35.701 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.701 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.701 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.701 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "prophetstor", 2024-05-21T15:42:35.701 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.701 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.702 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.702 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.702 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.702 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.702 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.702 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.702 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:35.702 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:35.702 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.702 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.702 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.702 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:35.702 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.703 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.703 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.703 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.703 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.703 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.703 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.703 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.703 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.703 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.703 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.703 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "influx", 2024-05-21T15:42:35.703 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": false, 2024-05-21T15:42:35.703 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "influxdb python module not found", 2024-05-21T15:42:35.704 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.704 INFO:teuthology.orchestra.run.smithi038.stdout: "batch_size": { 2024-05-21T15:42:35.704 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "batch_size", 2024-05-21T15:42:35.704 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.704 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.704 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.704 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5000", 2024-05-21T15:42:35.704 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.704 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.704 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.704 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-21T15:42:35.704 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.704 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.705 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.705 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.705 INFO:teuthology.orchestra.run.smithi038.stdout: "database": { 2024-05-21T15:42:35.705 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "database", 2024-05-21T15:42:35.705 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.705 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.705 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.705 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "ceph", 2024-05-21T15:42:35.705 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.705 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.705 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.705 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-21T15:42:35.705 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.705 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.706 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.706 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.706 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": { 2024-05-21T15:42:35.706 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hostname", 2024-05-21T15:42:35.706 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.706 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.706 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.706 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.706 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.706 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.706 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.706 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server hostname", 2024-05-21T15:42:35.707 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.707 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.707 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.707 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.707 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:35.707 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:35.707 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.707 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.707 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.707 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "30", 2024-05-21T15:42:35.707 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "5", 2024-05-21T15:42:35.707 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.707 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.708 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-21T15:42:35.708 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.708 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.708 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.708 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.708 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.708 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.708 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.708 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.708 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.708 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.708 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.708 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.709 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.709 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.709 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.709 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.709 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.709 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.709 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.709 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.709 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.709 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.709 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.709 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.709 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.710 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.710 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.710 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.710 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.710 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.710 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.710 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.710 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.710 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.710 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.710 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.710 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.711 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.711 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.711 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.711 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.711 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.711 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.711 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.711 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.711 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.711 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.711 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.711 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.711 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.712 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.712 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.712 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.712 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.712 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.712 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.712 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.712 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.712 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.712 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.712 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.712 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.712 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.712 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.713 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.713 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.713 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.713 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.713 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.713 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.713 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.713 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.713 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.713 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.713 INFO:teuthology.orchestra.run.smithi038.stdout: "password": { 2024-05-21T15:42:35.713 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "password", 2024-05-21T15:42:35.713 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.714 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.714 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.714 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.714 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.714 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.714 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.714 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "password of InfluxDB server user", 2024-05-21T15:42:35.714 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.714 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.714 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.714 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.714 INFO:teuthology.orchestra.run.smithi038.stdout: "port": { 2024-05-21T15:42:35.715 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "port", 2024-05-21T15:42:35.715 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.715 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.715 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.715 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8086", 2024-05-21T15:42:35.715 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.715 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.715 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.715 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server port", 2024-05-21T15:42:35.715 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.715 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.715 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.715 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.716 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-05-21T15:42:35.716 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-05-21T15:42:35.716 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.716 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.716 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.716 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "false", 2024-05-21T15:42:35.716 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.716 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.716 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.716 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T15:42:35.716 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.716 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.716 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.716 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.717 INFO:teuthology.orchestra.run.smithi038.stdout: "threads": { 2024-05-21T15:42:35.717 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threads", 2024-05-21T15:42:35.717 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.717 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.717 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.717 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-05-21T15:42:35.717 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-05-21T15:42:35.717 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "32", 2024-05-21T15:42:35.717 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.717 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-21T15:42:35.717 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.717 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.717 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.717 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.718 INFO:teuthology.orchestra.run.smithi038.stdout: "username": { 2024-05-21T15:42:35.718 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "username", 2024-05-21T15:42:35.718 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.718 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.718 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.718 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.718 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.718 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.718 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.718 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "username of InfluxDB server user", 2024-05-21T15:42:35.718 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.718 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.718 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: "verify_ssl": { 2024-05-21T15:42:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "verify_ssl", 2024-05-21T15:42:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "true", 2024-05-21T15:42:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T15:42:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.719 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.720 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.720 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.720 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.720 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.720 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "insights", 2024-05-21T15:42:35.720 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.720 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.720 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.720 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.720 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.720 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.720 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.720 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.721 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.721 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.721 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.721 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.721 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.721 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.721 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.721 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.721 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.721 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.721 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.721 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.721 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.721 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.722 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.722 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.722 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.722 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.722 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.722 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.722 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.722 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.722 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.722 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.722 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.722 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.723 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.723 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.723 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.723 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.723 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.723 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.723 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.723 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.723 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.723 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.723 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.723 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.723 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.723 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.724 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.724 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.724 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.724 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.724 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.724 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.724 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.724 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.724 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.724 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.724 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.724 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.724 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.724 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.725 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.725 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.725 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.725 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.725 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.725 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.725 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.725 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.725 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.725 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.725 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.725 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.725 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.726 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.726 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "iostat", 2024-05-21T15:42:35.726 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.726 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.726 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.726 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.726 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.726 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.726 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.726 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.726 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.726 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.727 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.727 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.727 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.727 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.727 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.727 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.727 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.727 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.727 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.727 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.727 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.727 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.727 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.728 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.728 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.728 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.728 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.728 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.728 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.728 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.728 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.728 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.728 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.728 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.728 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.728 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.728 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.729 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.729 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.729 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.729 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.729 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.729 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.729 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.729 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.729 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.729 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.729 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.729 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.729 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.729 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.730 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.730 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.730 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.730 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.730 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.730 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.730 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.730 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.730 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.730 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.730 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.730 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.730 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.731 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.731 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.731 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.731 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.731 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.731 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.731 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.731 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.731 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.731 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.731 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.731 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.731 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "k8sevents", 2024-05-21T15:42:35.732 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.732 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.732 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.732 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_event_retention_days": { 2024-05-21T15:42:35.732 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ceph_event_retention_days", 2024-05-21T15:42:35.732 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.732 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.732 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.732 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7", 2024-05-21T15:42:35.732 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.732 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.732 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.732 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-21T15:42:35.732 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.733 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.733 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.733 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.733 INFO:teuthology.orchestra.run.smithi038.stdout: "config_check_secs": { 2024-05-21T15:42:35.733 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_check_secs", 2024-05-21T15:42:35.733 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.733 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.733 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.733 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.733 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "10", 2024-05-21T15:42:35.733 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.733 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.733 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-21T15:42:35.733 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.734 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.734 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.734 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.734 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.734 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.734 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.734 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.734 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.734 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.734 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.734 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.734 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.734 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.735 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.735 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.735 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.735 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.735 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.735 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.735 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.735 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.735 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.735 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.735 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.735 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.735 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.736 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.736 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.736 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.736 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.736 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.736 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.736 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.736 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.736 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.736 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.736 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.736 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.736 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.736 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.737 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.737 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.737 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.737 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.737 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.737 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.737 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.737 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.737 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.737 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.737 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.737 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.737 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.738 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.738 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.738 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.738 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.738 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.738 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.738 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.738 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.738 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.738 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.738 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.738 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.739 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.739 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.739 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.739 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.739 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.739 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.739 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.739 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.739 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.739 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.739 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.739 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "localpool", 2024-05-21T15:42:35.739 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.740 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.740 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.740 INFO:teuthology.orchestra.run.smithi038.stdout: "failure_domain": { 2024-05-21T15:42:35.740 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "failure_domain", 2024-05-21T15:42:35.740 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.740 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.740 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.740 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "host", 2024-05-21T15:42:35.740 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.740 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.740 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.740 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "failure domain for any created local pool", 2024-05-21T15:42:35.740 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-21T15:42:35.741 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.741 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.741 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.741 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.741 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.741 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.741 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.741 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.741 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.741 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.741 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.741 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.741 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.741 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.742 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.742 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.742 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.742 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.742 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.742 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.742 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.742 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.742 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.742 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.742 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.742 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.743 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.744 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.744 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.744 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.744 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.744 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.744 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.744 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.744 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.744 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.744 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.744 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.744 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.744 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.745 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.745 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.745 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.745 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.745 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.745 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.745 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.745 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.745 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.745 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.745 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.745 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.745 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.745 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.746 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.746 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.746 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.746 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.746 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.746 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.746 INFO:teuthology.orchestra.run.smithi038.stdout: "min_size": { 2024-05-21T15:42:35.746 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_size", 2024-05-21T15:42:35.746 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.746 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.746 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.746 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.746 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.747 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.747 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.747 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default min_size for any created local pool", 2024-05-21T15:42:35.747 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-21T15:42:35.747 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.747 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.747 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.747 INFO:teuthology.orchestra.run.smithi038.stdout: "num_rep": { 2024-05-21T15:42:35.747 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "num_rep", 2024-05-21T15:42:35.747 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.747 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.747 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.747 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3", 2024-05-21T15:42:35.748 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.748 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.748 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.748 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default replica count for any created local pool", 2024-05-21T15:42:35.748 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.748 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.748 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.748 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.748 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_num": { 2024-05-21T15:42:35.748 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_num", 2024-05-21T15:42:35.748 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.748 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.748 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.749 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "128", 2024-05-21T15:42:35.749 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.749 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.749 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.749 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default pg_num for any created local pool", 2024-05-21T15:42:35.749 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.749 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.749 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.749 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.749 INFO:teuthology.orchestra.run.smithi038.stdout: "prefix": { 2024-05-21T15:42:35.750 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prefix", 2024-05-21T15:42:35.750 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.750 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.750 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.750 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.750 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.750 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.750 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.750 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name prefix for any created local pool", 2024-05-21T15:42:35.750 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.750 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.750 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.750 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.751 INFO:teuthology.orchestra.run.smithi038.stdout: "subtree": { 2024-05-21T15:42:35.751 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "subtree", 2024-05-21T15:42:35.751 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.751 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.751 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.751 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "rack", 2024-05-21T15:42:35.751 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.751 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.751 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.751 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-21T15:42:35.751 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-21T15:42:35.751 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.751 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.752 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.752 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.752 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.752 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.752 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mds_autoscaler", 2024-05-21T15:42:35.752 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.752 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.752 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.752 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.752 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.752 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.752 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.752 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.753 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.753 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.753 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.753 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.753 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.753 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.753 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.753 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.753 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.753 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.753 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.753 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.753 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.753 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.754 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.754 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.754 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.754 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.754 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.754 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.754 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.754 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.754 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.754 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.754 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.754 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.754 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.755 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.755 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.755 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.755 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.755 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.755 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.755 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.755 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.755 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.755 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.755 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.755 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.755 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.756 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.756 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.756 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.756 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.756 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.756 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.756 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.756 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.756 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.756 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.756 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.756 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.756 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.757 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.757 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.757 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.757 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.757 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.757 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.757 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.757 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.757 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.757 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.757 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.757 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.757 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.757 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.758 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.758 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirroring", 2024-05-21T15:42:35.758 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.758 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.758 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.758 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.758 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.758 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.758 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.758 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.758 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.758 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.758 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.759 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.759 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.759 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.759 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.759 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.759 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.759 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.759 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.759 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.759 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.759 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.759 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.759 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.759 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.760 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.760 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.760 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.760 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.760 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.760 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.760 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.760 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.760 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.760 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.760 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.760 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.760 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.761 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.761 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.761 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.761 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.761 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.761 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.761 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.761 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.761 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.761 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.761 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.761 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.761 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.761 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.762 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.762 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.762 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.762 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.762 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.762 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.762 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.762 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.762 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.762 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.762 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.762 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.762 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.763 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.763 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.763 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.763 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.763 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.763 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.763 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.763 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.763 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.763 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.763 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.763 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "nfs", 2024-05-21T15:42:35.764 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.764 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.764 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.764 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.764 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.764 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.764 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.764 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.764 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.764 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.764 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.764 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.764 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.764 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.765 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.765 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.765 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.765 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.765 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.765 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.765 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.765 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.765 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.765 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.765 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.765 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.765 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.766 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.766 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.766 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.766 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.766 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.766 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.766 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.766 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.766 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.766 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.766 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.766 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.766 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.766 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.767 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.767 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.767 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.767 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.767 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.767 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.767 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.767 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.767 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.767 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.767 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.767 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.767 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.767 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.768 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.768 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.768 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.768 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.768 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.768 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.768 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.768 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.768 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.768 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.768 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.768 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.768 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.769 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.769 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.769 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.769 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.769 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.769 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.769 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.769 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.769 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-05-21T15:42:35.769 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.769 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.769 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.769 INFO:teuthology.orchestra.run.smithi038.stdout: "fail_fs": { 2024-05-21T15:42:35.769 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "fail_fs", 2024-05-21T15:42:35.770 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.770 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.770 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.770 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.770 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.770 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.770 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.770 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-21T15:42:35.770 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.770 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.770 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.770 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.770 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.771 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.771 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.771 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.771 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.771 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.771 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.771 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.771 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.771 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.771 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.771 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.771 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.771 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.771 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.772 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.772 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.772 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.772 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.772 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.772 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.772 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.772 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.772 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.772 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.772 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.772 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.772 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.773 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.773 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.773 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.773 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.773 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.773 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.773 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.773 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.773 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.773 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.773 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.773 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.773 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.773 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.774 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.774 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.774 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.774 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.774 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.774 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.774 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.774 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.774 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.774 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.774 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.774 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.774 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.775 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.775 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.775 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.775 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.775 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.775 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.775 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.775 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.775 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.775 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.775 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.775 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.775 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.776 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.776 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.776 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator": { 2024-05-21T15:42:35.776 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-05-21T15:42:35.776 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.776 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.776 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.776 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.776 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.776 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.776 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.776 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-05-21T15:42:35.776 INFO:teuthology.orchestra.run.smithi038.stdout: "rook", 2024-05-21T15:42:35.777 INFO:teuthology.orchestra.run.smithi038.stdout: "test_orchestrator" 2024-05-21T15:42:35.777 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.777 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Orchestrator backend", 2024-05-21T15:42:35.777 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.777 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.777 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.777 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.777 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.777 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.777 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.777 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_perf_query", 2024-05-21T15:42:35.777 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.777 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.778 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.778 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.778 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.778 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.778 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.778 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.778 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.778 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.778 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.778 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.778 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.778 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.779 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.779 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.779 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.779 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.779 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.779 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.779 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.779 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.779 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.779 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.779 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.779 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.779 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.780 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.780 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.780 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.780 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.780 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.780 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.780 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.780 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.780 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.780 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.780 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.780 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.780 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.781 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.781 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.781 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.781 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.781 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.781 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.781 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.781 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.781 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.781 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.781 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.781 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.781 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.781 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.782 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.782 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.782 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.782 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.782 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.782 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.782 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.782 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.782 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.782 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.782 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.782 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.782 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.783 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.783 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.783 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.783 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.783 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.783 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.783 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.783 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.783 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.783 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_support", 2024-05-21T15:42:35.783 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.783 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.783 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.784 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.784 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.784 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.784 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.784 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.784 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.784 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.784 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.784 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.784 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.784 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.784 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.784 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.784 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.785 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.785 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.785 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.785 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.785 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.785 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.785 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.785 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.785 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.785 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.785 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.785 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.785 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.786 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.786 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.786 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.786 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.786 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.786 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.786 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.786 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.786 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.786 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.786 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.786 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.786 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.787 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.787 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.787 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.787 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.787 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.787 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.787 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.787 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.787 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.787 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.787 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.787 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.787 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.787 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.788 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.788 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.788 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.788 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.788 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.788 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.788 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.788 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.788 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.788 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.788 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.788 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.788 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.789 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.789 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.789 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.789 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.789 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.789 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.789 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_autoscaler", 2024-05-21T15:42:35.789 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.789 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.789 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.789 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.789 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.789 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.789 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.790 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.790 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.790 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.790 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.790 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.790 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.790 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.790 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.790 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.790 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.790 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.790 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.790 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.791 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.791 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.791 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.791 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.791 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.791 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.791 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.791 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.791 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.791 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.791 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.791 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.791 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.791 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.792 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.792 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.792 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.792 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.792 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.792 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.792 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.792 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.792 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.792 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.792 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.792 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.792 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.792 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.793 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.793 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.793 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.793 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.793 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.793 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.793 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.793 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.793 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.793 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.793 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.793 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.793 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.794 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.794 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.794 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.794 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.794 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.794 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.794 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.794 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.794 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.794 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.794 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.794 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.794 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:35.795 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:35.795 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.795 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.795 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.795 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:35.795 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.795 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.795 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.795 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.795 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.795 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.795 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.795 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.795 INFO:teuthology.orchestra.run.smithi038.stdout: "threshold": { 2024-05-21T15:42:35.796 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threshold", 2024-05-21T15:42:35.796 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.796 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.796 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.796 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-05-21T15:42:35.796 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1.0", 2024-05-21T15:42:35.796 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.796 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.796 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "scaling threshold", 2024-05-21T15:42:35.796 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-21T15:42:35.796 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.796 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.796 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.796 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.797 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.797 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.797 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "progress", 2024-05-21T15:42:35.797 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.797 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.797 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.797 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_pg_recovery_event": { 2024-05-21T15:42:35.797 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_pg_recovery_event", 2024-05-21T15:42:35.797 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.797 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.797 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.797 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.797 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.797 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.798 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.798 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-21T15:42:35.798 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.798 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.798 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.798 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.798 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-05-21T15:42:35.798 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-05-21T15:42:35.798 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.798 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.798 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.798 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.798 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.799 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.799 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.799 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.799 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.799 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.799 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.799 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.799 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.799 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.799 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.799 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.799 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.799 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.800 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.800 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.800 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.800 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.800 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.800 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.800 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.800 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.800 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.800 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.800 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.800 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.800 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.800 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.801 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.801 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.801 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.801 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.801 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.801 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.801 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.801 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.801 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.801 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.801 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.801 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.801 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.801 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.802 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.802 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.802 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.802 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.802 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.802 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.802 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.802 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.802 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.802 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.802 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.802 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.802 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.803 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.803 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.803 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.803 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.803 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.803 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.803 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.803 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.803 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.803 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.803 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.803 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.803 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.804 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.804 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.804 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.804 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.804 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.804 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.804 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.804 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.804 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.804 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.804 INFO:teuthology.orchestra.run.smithi038.stdout: "max_completed_events": { 2024-05-21T15:42:35.804 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_completed_events", 2024-05-21T15:42:35.804 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.804 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.805 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.805 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "50", 2024-05-21T15:42:35.805 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.805 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.805 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.805 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "number of past completed events to remember", 2024-05-21T15:42:35.805 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.805 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.805 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.805 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.805 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:35.805 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:35.805 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.805 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.806 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.806 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-05-21T15:42:35.806 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.806 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.806 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.806 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long the module is going to sleep", 2024-05-21T15:42:35.806 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.806 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.806 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.806 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.806 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.806 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.806 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.807 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus", 2024-05-21T15:42:35.807 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.807 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.807 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.807 INFO:teuthology.orchestra.run.smithi038.stdout: "cache": { 2024-05-21T15:42:35.807 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cache", 2024-05-21T15:42:35.807 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.807 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.807 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.807 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.807 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.807 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.808 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.808 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.808 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.808 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.808 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.808 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.808 INFO:teuthology.orchestra.run.smithi038.stdout: "exclude_perf_counters": { 2024-05-21T15:42:35.808 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "exclude_perf_counters", 2024-05-21T15:42:35.808 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.808 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.808 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.808 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.808 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.809 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.809 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.809 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-21T15:42:35.809 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-21T15:42:35.809 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.809 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.809 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.809 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.809 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.809 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.809 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.809 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.809 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.810 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.810 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.810 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.810 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.810 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.810 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.810 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.810 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.810 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.810 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.810 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.810 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.810 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.810 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.811 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.811 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.811 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.811 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.811 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.811 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.811 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.811 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.811 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.811 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.811 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.811 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.811 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.812 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.812 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.812 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.812 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.812 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.812 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.812 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.812 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.812 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.812 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.812 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.812 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.812 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.813 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.813 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.813 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.813 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.813 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.813 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.813 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.813 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.813 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.813 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.813 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.813 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.813 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.814 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.814 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.814 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.814 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.814 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.814 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.814 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.814 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.814 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.814 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.814 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.814 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools": { 2024-05-21T15:42:35.814 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools", 2024-05-21T15:42:35.815 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.815 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.815 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.815 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.815 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.815 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.815 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.815 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.815 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.815 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.815 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.815 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.815 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-21T15:42:35.816 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-21T15:42:35.816 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.816 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.816 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.816 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "300", 2024-05-21T15:42:35.816 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.816 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.816 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.816 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.816 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.816 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.816 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.816 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.816 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_interval": { 2024-05-21T15:42:35.817 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_interval", 2024-05-21T15:42:35.817 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.817 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.817 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.817 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15.0", 2024-05-21T15:42:35.817 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.817 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.817 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.817 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.817 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.817 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.817 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.817 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.817 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-05-21T15:42:35.818 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-05-21T15:42:35.818 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.818 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.818 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.818 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-05-21T15:42:35.818 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.818 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.818 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.818 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-21T15:42:35.818 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.818 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.818 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.819 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.819 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-05-21T15:42:35.819 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-05-21T15:42:35.819 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.819 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.819 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.819 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "9283", 2024-05-21T15:42:35.819 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.819 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.819 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.819 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-21T15:42:35.819 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.819 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.819 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.820 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.820 INFO:teuthology.orchestra.run.smithi038.stdout: "stale_cache_strategy": { 2024-05-21T15:42:35.820 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stale_cache_strategy", 2024-05-21T15:42:35.820 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.820 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.820 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.820 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "log", 2024-05-21T15:42:35.820 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.820 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.820 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.820 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.820 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.820 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.821 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.821 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.821 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-05-21T15:42:35.821 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-05-21T15:42:35.821 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.821 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.821 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.821 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "default", 2024-05-21T15:42:35.821 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.821 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.821 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.821 INFO:teuthology.orchestra.run.smithi038.stdout: "default", 2024-05-21T15:42:35.821 INFO:teuthology.orchestra.run.smithi038.stdout: "error" 2024-05-21T15:42:35.822 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.822 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.822 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.822 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.822 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.822 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.822 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-05-21T15:42:35.822 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-05-21T15:42:35.822 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.822 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.822 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.822 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-05-21T15:42:35.823 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-05-21T15:42:35.823 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-05-21T15:42:35.823 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.823 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.823 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.823 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.823 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.823 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.823 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.823 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.823 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.823 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-05-21T15:42:35.823 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.824 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.824 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.824 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.824 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.824 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.824 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.824 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.824 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.824 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.824 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.824 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.824 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.824 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.825 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.825 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.825 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.825 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.825 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.825 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.825 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.825 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.825 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.825 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.825 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.825 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.825 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.825 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.826 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.826 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.826 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.826 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.826 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.826 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.826 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.826 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.826 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.826 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.826 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.826 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.826 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.827 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.827 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.827 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.827 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.827 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.827 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.827 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.827 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.827 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.827 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.827 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.827 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.827 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.827 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.828 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.828 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.828 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.828 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.828 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.828 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.828 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.828 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.828 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.828 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.828 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.828 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.828 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.829 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.829 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.829 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.829 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.829 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.829 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_snap_create": { 2024-05-21T15:42:35.829 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_snap_create", 2024-05-21T15:42:35.829 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.829 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.829 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.829 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.829 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.829 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.829 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.830 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.830 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.830 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.830 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.830 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.830 INFO:teuthology.orchestra.run.smithi038.stdout: "mirror_snapshot_schedule": { 2024-05-21T15:42:35.830 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirror_snapshot_schedule", 2024-05-21T15:42:35.830 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.830 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.830 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.830 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.830 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.830 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.830 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.831 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.831 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.831 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.831 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.831 INFO:teuthology.orchestra.run.smithi038.stdout: "trash_purge_schedule": { 2024-05-21T15:42:35.831 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "trash_purge_schedule", 2024-05-21T15:42:35.831 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.831 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.831 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.831 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.831 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.831 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.832 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.832 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.832 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.832 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.832 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.832 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.832 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.832 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.832 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.832 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "restful", 2024-05-21T15:42:35.832 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.832 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.832 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.832 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_auth": { 2024-05-21T15:42:35.833 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_auth", 2024-05-21T15:42:35.833 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.833 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.833 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.833 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.833 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.833 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.833 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.833 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.833 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.833 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.833 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.833 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.833 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-05-21T15:42:35.834 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-05-21T15:42:35.834 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.834 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.834 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.834 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.834 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.834 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.834 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.834 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.834 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.834 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.834 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.834 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.835 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.835 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.835 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.835 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.835 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.835 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.835 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.835 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.835 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.835 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.835 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.835 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.835 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.837 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.837 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.837 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.837 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.837 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.837 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.837 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.837 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.837 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.837 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.837 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.837 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.837 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.837 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.838 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.838 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.838 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.838 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.838 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.838 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.838 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.838 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.838 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.838 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.838 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.838 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.838 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.839 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.839 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.839 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.839 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.839 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.839 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.839 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.839 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.839 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.839 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.839 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.839 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.839 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.840 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.840 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.840 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.840 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-05-21T15:42:35.840 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-05-21T15:42:35.840 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.840 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.840 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.840 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.840 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.840 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.840 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.840 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.840 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.841 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.841 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.841 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.841 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-05-21T15:42:35.841 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-05-21T15:42:35.841 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.841 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.841 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.841 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.841 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.841 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.841 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.841 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.842 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.842 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.842 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.842 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.842 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.842 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.842 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.842 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rgw", 2024-05-21T15:42:35.842 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.842 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.842 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.842 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.842 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.843 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.843 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.843 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.843 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.843 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.843 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.843 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.843 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.843 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.843 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.843 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.843 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.843 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.843 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.844 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.844 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.844 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.844 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.844 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.844 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.844 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.844 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.844 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.844 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.844 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.844 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.844 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.844 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.845 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.845 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.845 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.845 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.845 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.845 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.845 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.845 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.845 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.845 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.845 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.845 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.845 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.846 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.846 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.846 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.846 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.846 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.846 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.846 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.846 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.846 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.846 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.846 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.846 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.846 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.846 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.847 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.847 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.847 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.847 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.847 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.847 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.847 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.847 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.847 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.847 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.847 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.847 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.847 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.848 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.848 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.848 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.848 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rook", 2024-05-21T15:42:35.848 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.848 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.848 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.848 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.848 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.848 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.848 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.848 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.848 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.849 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.849 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.849 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.849 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.849 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.849 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.849 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.849 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.849 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.849 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.849 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.849 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.849 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.849 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.850 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.850 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.850 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.850 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.850 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.850 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.850 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.850 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.850 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.850 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.850 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.850 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.851 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.851 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.851 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.851 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.851 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.851 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.851 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.851 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.851 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.851 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.851 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.851 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.851 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.852 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.852 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.852 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.852 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.852 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.852 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.852 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.852 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.852 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.852 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.852 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.852 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.852 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.853 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.853 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.853 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.853 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.853 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.853 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.853 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.853 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.853 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.853 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.853 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.853 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.853 INFO:teuthology.orchestra.run.smithi038.stdout: "storage_class": { 2024-05-21T15:42:35.854 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "storage_class", 2024-05-21T15:42:35.854 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.854 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.854 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.854 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "local", 2024-05-21T15:42:35.854 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.854 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.854 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.854 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-21T15:42:35.854 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.854 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.854 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.854 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.854 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.855 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.855 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.855 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "selftest", 2024-05-21T15:42:35.855 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.855 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.855 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.855 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.855 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.855 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.855 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.855 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.855 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.855 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.855 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.856 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.856 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.856 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.856 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.856 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.856 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.856 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.856 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.856 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.856 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.856 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.856 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.857 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.857 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.857 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.857 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.857 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.857 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.857 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.857 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.857 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.857 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.857 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.857 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.857 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.858 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.858 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.858 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.858 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.858 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.858 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.858 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.858 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.858 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.858 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.858 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.858 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.858 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.858 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.859 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.859 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.859 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.859 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.859 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.859 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.859 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.859 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.859 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.859 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.859 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.859 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.859 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.860 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.860 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.860 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.860 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.860 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.860 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.860 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.860 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.860 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.860 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.860 INFO:teuthology.orchestra.run.smithi038.stdout: "roption1": { 2024-05-21T15:42:35.860 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption1", 2024-05-21T15:42:35.860 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.861 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.861 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.861 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.861 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.861 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.861 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.861 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.861 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.861 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.861 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.861 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.861 INFO:teuthology.orchestra.run.smithi038.stdout: "roption2": { 2024-05-21T15:42:35.861 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption2", 2024-05-21T15:42:35.862 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.862 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.862 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.862 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "xyz", 2024-05-21T15:42:35.862 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.862 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.862 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.862 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.862 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.862 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.862 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.862 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.862 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption1": { 2024-05-21T15:42:35.863 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption1", 2024-05-21T15:42:35.863 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.863 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.863 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.863 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.863 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.863 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.863 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.863 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.863 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.863 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.863 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.863 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.863 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption2": { 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption2", 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption3": { 2024-05-21T15:42:35.864 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption3", 2024-05-21T15:42:35.865 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.865 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.865 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.865 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.865 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.865 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.865 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.865 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.865 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.865 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.865 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.865 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.865 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption4": { 2024-05-21T15:42:35.866 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption4", 2024-05-21T15:42:35.866 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.866 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.866 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.866 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.866 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.866 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.866 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.866 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.866 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.866 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.866 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.866 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.867 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption5": { 2024-05-21T15:42:35.867 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption5", 2024-05-21T15:42:35.867 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.867 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.867 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.867 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.867 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.867 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.867 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.867 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.867 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.867 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.867 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.867 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.868 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption6": { 2024-05-21T15:42:35.868 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption6", 2024-05-21T15:42:35.868 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.868 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.868 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.868 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.868 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.868 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.868 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.868 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.868 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.868 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.868 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.868 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.869 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption7": { 2024-05-21T15:42:35.869 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption7", 2024-05-21T15:42:35.869 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.869 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.869 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.869 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.869 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-05-21T15:42:35.869 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "42", 2024-05-21T15:42:35.869 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.869 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.869 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.869 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.869 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: "testkey": { 2024-05-21T15:42:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testkey", 2024-05-21T15:42:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.870 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.871 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.871 INFO:teuthology.orchestra.run.smithi038.stdout: "testlkey": { 2024-05-21T15:42:35.871 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testlkey", 2024-05-21T15:42:35.871 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.871 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.871 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.871 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.871 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.871 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.871 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.871 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.871 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.871 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.872 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.873 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.873 INFO:teuthology.orchestra.run.smithi038.stdout: "testnewline": { 2024-05-21T15:42:35.873 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testnewline", 2024-05-21T15:42:35.873 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.873 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.873 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.873 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.874 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.874 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.874 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.874 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.874 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.874 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.874 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.874 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.874 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.874 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.874 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.874 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snap_schedule", 2024-05-21T15:42:35.874 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.875 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.875 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.875 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_m_granularity": { 2024-05-21T15:42:35.875 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_m_granularity", 2024-05-21T15:42:35.875 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.875 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.875 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.875 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.875 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.875 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.875 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.875 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow minute scheduled snapshots", 2024-05-21T15:42:35.875 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.875 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.876 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.876 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.876 INFO:teuthology.orchestra.run.smithi038.stdout: "dump_on_update": { 2024-05-21T15:42:35.876 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dump_on_update", 2024-05-21T15:42:35.876 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.876 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.876 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.876 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.876 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.876 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.876 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.876 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "dump database to debug log on update", 2024-05-21T15:42:35.876 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.876 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.877 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.877 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.877 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.877 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.877 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.877 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.877 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.877 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.877 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.877 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.877 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.877 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.877 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.878 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.878 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.878 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.878 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.878 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.878 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.878 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.878 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.878 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.878 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.878 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.878 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.878 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.879 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.879 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.879 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.879 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.879 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.879 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.879 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.879 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.879 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.879 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.879 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.879 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.879 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.879 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.880 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.880 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.880 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.880 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.880 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.880 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.880 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.880 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.880 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.880 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.880 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.880 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.880 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.881 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.881 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.881 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.881 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.881 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.881 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.881 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.881 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.881 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.881 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.881 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.881 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.881 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.881 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.882 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.882 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.882 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.882 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.882 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.882 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.882 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.882 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.882 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stats", 2024-05-21T15:42:35.882 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.882 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.882 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.882 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.883 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.883 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.883 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.883 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.883 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.883 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.883 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.883 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.883 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.883 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.883 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.883 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.883 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.883 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.884 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.884 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.884 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.884 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.884 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.884 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.884 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.884 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.884 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.884 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.884 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.884 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.884 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.885 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.885 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.885 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.885 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.885 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.885 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.885 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.885 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.885 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.885 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.885 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.885 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.885 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.885 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.886 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.886 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.886 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.886 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.886 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.886 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.886 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.886 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.886 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.886 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.886 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.886 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.886 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.887 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.887 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.887 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.887 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.887 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.887 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.887 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.887 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.887 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.887 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.887 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.887 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.887 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.887 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.888 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.888 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.888 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.888 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.888 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "status", 2024-05-21T15:42:35.888 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.888 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.888 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.888 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.888 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.888 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.888 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.889 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.889 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.889 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.889 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.889 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.889 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.889 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.889 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.889 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.889 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.889 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.889 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.889 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.890 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.890 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.890 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.890 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.890 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.890 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.890 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.890 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.890 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.890 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.890 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.890 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.890 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.891 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.891 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.891 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.891 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.891 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.891 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.891 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.891 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.891 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.891 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.891 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.891 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.891 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.892 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.892 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.892 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.892 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.892 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.892 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.892 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.892 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.892 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.892 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.892 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.892 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.892 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.892 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.893 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.893 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.893 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.893 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.893 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.893 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.893 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.893 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.893 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.893 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.893 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.893 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.893 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.893 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.894 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.894 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.894 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telegraf", 2024-05-21T15:42:35.894 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.894 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.894 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.894 INFO:teuthology.orchestra.run.smithi038.stdout: "address": { 2024-05-21T15:42:35.894 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "address", 2024-05-21T15:42:35.894 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.894 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.894 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.894 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-21T15:42:35.894 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.895 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.895 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.895 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.895 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.895 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.895 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.895 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.895 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:35.895 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:35.895 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.895 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.895 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.896 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15", 2024-05-21T15:42:35.896 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.896 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.896 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.896 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.896 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.896 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.896 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.896 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.896 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.896 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.896 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.896 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.897 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.897 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.897 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.897 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.897 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.897 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.897 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.897 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.897 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.897 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.897 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.897 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.897 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.898 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.898 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.898 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.898 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.898 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.898 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.898 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.898 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.898 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.898 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.898 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.898 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.899 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.899 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.899 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.899 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.899 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.899 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.899 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.899 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.899 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.899 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.899 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.899 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.899 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.900 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.900 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.900 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.900 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.900 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.900 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.900 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.900 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.900 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.900 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.900 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.900 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.900 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.901 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.901 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.901 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.901 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.901 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.901 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.901 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.901 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.901 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.901 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.901 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.901 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.901 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.901 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telemetry", 2024-05-21T15:42:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_basic": { 2024-05-21T15:42:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_basic", 2024-05-21T15:42:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.903 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.903 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.903 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.903 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.903 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.903 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-21T15:42:35.903 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.903 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.903 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.903 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.903 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_crash": { 2024-05-21T15:42:35.903 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_crash", 2024-05-21T15:42:35.903 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.904 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.904 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.904 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.904 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.904 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.904 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.904 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-21T15:42:35.904 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.904 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.904 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.904 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.904 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_device": { 2024-05-21T15:42:35.904 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_device", 2024-05-21T15:42:35.905 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.905 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.905 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.905 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.905 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.905 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.905 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.905 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-21T15:42:35.905 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.905 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.905 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.905 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.905 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_ident": { 2024-05-21T15:42:35.905 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_ident", 2024-05-21T15:42:35.906 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.906 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.906 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.906 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.906 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.906 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.906 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.906 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-21T15:42:35.906 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.906 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.906 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.906 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.906 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_perf": { 2024-05-21T15:42:35.907 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_perf", 2024-05-21T15:42:35.907 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.907 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.907 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.907 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.907 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.907 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.907 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.907 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-21T15:42:35.907 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.907 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.907 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.907 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.908 INFO:teuthology.orchestra.run.smithi038.stdout: "contact": { 2024-05-21T15:42:35.908 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "contact", 2024-05-21T15:42:35.908 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.908 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.908 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.908 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.908 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.908 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.908 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.908 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.908 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.908 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.908 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.909 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.909 INFO:teuthology.orchestra.run.smithi038.stdout: "description": { 2024-05-21T15:42:35.909 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "description", 2024-05-21T15:42:35.909 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.909 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.909 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.909 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.909 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.909 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.909 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.909 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.909 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.909 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.909 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: "device_url": { 2024-05-21T15:42:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_url", 2024-05-21T15:42:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-21T15:42:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.910 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-05-21T15:42:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-05-21T15:42:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.911 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "24", 2024-05-21T15:42:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "8", 2024-05-21T15:42:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.912 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: "last_opt_revision": { 2024-05-21T15:42:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "last_opt_revision", 2024-05-21T15:42:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1", 2024-05-21T15:42:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.913 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard": { 2024-05-21T15:42:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard", 2024-05-21T15:42:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.914 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard_description": { 2024-05-21T15:42:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard_description", 2024-05-21T15:42:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.915 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.916 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.917 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.918 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.919 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.920 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: "organization": { 2024-05-21T15:42:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "organization", 2024-05-21T15:42:35.921 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: "proxy": { 2024-05-21T15:42:35.922 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "proxy", 2024-05-21T15:42:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: "url": { 2024-05-21T15:42:35.923 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url", 2024-05-21T15:42:35.924 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.924 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.924 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.924 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-21T15:42:35.924 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.924 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.924 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.924 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.924 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.924 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.924 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.924 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.924 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.925 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.925 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.925 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "test_orchestrator", 2024-05-21T15:42:35.925 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.925 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.925 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.925 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.925 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.925 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.925 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.925 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.925 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.925 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.926 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.926 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.926 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.926 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.926 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.926 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.926 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.926 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.926 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.926 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.926 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.926 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.926 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.927 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.927 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.927 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.927 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.927 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.927 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.927 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.927 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.927 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.927 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.927 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.927 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.927 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.928 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.928 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.928 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.928 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.928 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.928 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.928 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.928 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.928 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.928 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.928 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.928 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.928 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.929 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.929 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.929 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.929 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.929 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.929 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.929 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.929 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.929 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.929 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.929 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.929 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.929 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.930 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.930 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.930 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.930 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.930 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.930 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.930 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.930 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.930 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.930 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.930 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.930 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.930 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.931 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.931 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-05-21T15:42:35.931 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.931 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.931 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.931 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.931 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.931 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.931 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.931 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.931 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.931 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.931 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.932 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.932 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.932 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.932 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.932 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.932 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.932 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.932 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.932 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.932 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.932 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.932 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.932 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.933 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.933 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.933 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.933 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.933 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.933 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.933 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.933 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.933 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.933 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.933 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.933 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.933 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.934 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.934 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.934 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.934 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.934 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.934 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.934 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.934 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.934 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.934 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.934 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.934 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.934 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.935 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.935 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.935 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.935 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.935 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.935 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.935 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.935 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.935 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.935 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.935 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.935 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.935 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.936 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.936 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.936 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.936 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.936 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.936 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.936 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.936 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.936 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.936 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.936 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_clones": { 2024-05-21T15:42:35.936 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_clones", 2024-05-21T15:42:35.936 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.936 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.937 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.937 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4", 2024-05-21T15:42:35.937 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.937 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.937 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.937 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-21T15:42:35.937 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.937 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.937 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.937 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.937 INFO:teuthology.orchestra.run.smithi038.stdout: "periodic_async_work": { 2024-05-21T15:42:35.937 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "periodic_async_work", 2024-05-21T15:42:35.937 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.938 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.938 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.938 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.938 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.938 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.938 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.938 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Periodically check for async work", 2024-05-21T15:42:35.938 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.938 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.938 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.938 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.938 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_delay": { 2024-05-21T15:42:35.938 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_delay", 2024-05-21T15:42:35.939 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.939 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.939 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.939 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:35.939 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.939 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.939 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.939 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-21T15:42:35.939 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.939 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.939 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.939 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.939 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_no_wait": { 2024-05-21T15:42:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_no_wait", 2024-05-21T15:42:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-21T15:42:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.941 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.941 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.941 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix", 2024-05-21T15:42:35.941 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.941 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.941 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.941 INFO:teuthology.orchestra.run.smithi038.stdout: "discovery_interval": { 2024-05-21T15:42:35.941 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "discovery_interval", 2024-05-21T15:42:35.941 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:35.941 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.941 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.941 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "100", 2024-05-21T15:42:35.941 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.942 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.942 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.942 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.942 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.942 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.942 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.942 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.942 INFO:teuthology.orchestra.run.smithi038.stdout: "identifier": { 2024-05-21T15:42:35.942 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "identifier", 2024-05-21T15:42:35.942 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.942 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.942 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.942 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:35.944 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.944 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.944 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.944 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.944 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.944 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.944 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.944 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.944 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.944 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.944 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.944 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.944 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.944 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.945 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.945 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.945 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.945 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.945 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.945 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.945 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.945 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.945 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.945 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.945 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.945 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.945 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.945 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.946 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.946 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.946 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.946 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.946 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.946 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.946 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.946 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.946 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.946 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.946 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.946 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.946 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.947 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.947 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.947 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.947 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.947 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.947 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.947 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.947 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.947 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.947 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.947 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.947 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.947 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.947 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.948 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.948 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.948 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.948 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.948 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.948 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.948 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.948 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.948 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.948 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.948 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.948 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.948 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.948 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.949 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.949 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.949 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.949 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.949 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.949 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.949 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.949 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.949 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.949 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_host": { 2024-05-21T15:42:35.949 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_host", 2024-05-21T15:42:35.949 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.949 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.950 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.950 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.950 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.950 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.950 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.950 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.950 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.950 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.950 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.950 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.950 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_port": { 2024-05-21T15:42:35.950 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_port", 2024-05-21T15:42:35.950 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.950 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.951 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.951 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10051", 2024-05-21T15:42:35.951 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.951 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.951 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.951 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.951 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.951 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.951 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.951 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.951 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_sender": { 2024-05-21T15:42:35.951 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_sender", 2024-05-21T15:42:35.951 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.952 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.952 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.952 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-21T15:42:35.952 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.952 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.952 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.952 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.952 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.952 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.952 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.952 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.952 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.952 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.952 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-05-21T15:42:35.953 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.953 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.953 INFO:teuthology.orchestra.run.smithi038.stdout: "modules": [ 2024-05-21T15:42:35.953 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-05-21T15:42:35.953 INFO:teuthology.orchestra.run.smithi038.stdout: "dashboard", 2024-05-21T15:42:35.953 INFO:teuthology.orchestra.run.smithi038.stdout: "iostat", 2024-05-21T15:42:35.953 INFO:teuthology.orchestra.run.smithi038.stdout: "nfs", 2024-05-21T15:42:35.953 INFO:teuthology.orchestra.run.smithi038.stdout: "restful" 2024-05-21T15:42:35.953 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.953 INFO:teuthology.orchestra.run.smithi038.stdout: "available_modules": [ 2024-05-21T15:42:35.953 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.953 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "alerts", 2024-05-21T15:42:35.953 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.954 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.954 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.954 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:35.954 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:35.954 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.954 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.954 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.954 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:35.954 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.954 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.954 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.954 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How frequently to reexamine health status", 2024-05-21T15:42:35.955 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.955 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.955 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.955 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.955 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.955 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.955 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.955 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.955 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.955 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.955 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.955 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.955 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.956 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.956 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.956 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.956 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.956 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.956 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.956 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.956 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.956 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.956 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.956 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.956 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.956 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.957 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.957 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.957 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.957 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.957 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.957 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.957 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.957 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.957 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.957 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.957 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.957 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.957 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.957 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.958 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.958 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.958 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.958 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.958 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.958 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.958 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.958 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.958 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.958 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.958 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.958 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.959 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.959 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.959 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.959 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.959 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.959 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.959 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.959 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.959 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.959 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.959 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.959 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.959 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.959 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.960 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.960 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.960 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.960 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.960 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.960 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.960 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.960 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.960 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_destination": { 2024-05-21T15:42:35.960 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_destination", 2024-05-21T15:42:35.960 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.960 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.960 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.961 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.961 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.961 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.961 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.961 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email address to send alerts to", 2024-05-21T15:42:35.961 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.961 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.961 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.961 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.961 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_from_name": { 2024-05-21T15:42:35.961 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_from_name", 2024-05-21T15:42:35.961 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.961 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.962 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.962 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "Ceph", 2024-05-21T15:42:35.962 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.962 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.962 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.962 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email From: name", 2024-05-21T15:42:35.962 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.962 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.962 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.962 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.962 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_host": { 2024-05-21T15:42:35.962 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_host", 2024-05-21T15:42:35.963 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.963 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.963 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.963 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.963 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.963 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.963 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.963 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP server", 2024-05-21T15:42:35.963 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.963 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.963 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.963 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.963 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_password": { 2024-05-21T15:42:35.963 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_password", 2024-05-21T15:42:35.964 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.964 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.964 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.964 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.964 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.964 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.964 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.964 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Password to authenticate with", 2024-05-21T15:42:35.964 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.964 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.964 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.964 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.964 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_port": { 2024-05-21T15:42:35.965 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_port", 2024-05-21T15:42:35.965 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.965 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.965 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.965 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "465", 2024-05-21T15:42:35.965 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.965 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.965 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.965 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP port", 2024-05-21T15:42:35.965 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.965 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.965 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.965 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.966 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_sender": { 2024-05-21T15:42:35.966 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_sender", 2024-05-21T15:42:35.966 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.966 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.966 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.966 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.966 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.966 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.966 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.966 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP envelope sender", 2024-05-21T15:42:35.966 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.966 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.966 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.967 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.967 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_ssl": { 2024-05-21T15:42:35.967 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_ssl", 2024-05-21T15:42:35.967 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.967 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.967 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.967 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.967 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.967 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.967 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.967 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-21T15:42:35.967 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.967 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.968 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.968 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.968 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_user": { 2024-05-21T15:42:35.968 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_user", 2024-05-21T15:42:35.968 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.968 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.968 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.968 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.968 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.968 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.968 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.968 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "User to authenticate as", 2024-05-21T15:42:35.968 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.969 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.969 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.969 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.969 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.969 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.969 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.969 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "balancer", 2024-05-21T15:42:35.969 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.969 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.969 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.969 INFO:teuthology.orchestra.run.smithi038.stdout: "active": { 2024-05-21T15:42:35.969 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "active", 2024-05-21T15:42:35.969 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.969 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.970 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.970 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.970 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.970 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.970 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.970 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically balance PGs across cluster", 2024-05-21T15:42:35.970 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.970 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.970 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.970 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.970 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_time": { 2024-05-21T15:42:35.970 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_time", 2024-05-21T15:42:35.971 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.971 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.971 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.971 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0000", 2024-05-21T15:42:35.971 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.971 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.971 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.971 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "beginning time of day to automatically balance", 2024-05-21T15:42:35.971 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T15:42:35.971 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.971 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.971 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.971 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_weekday": { 2024-05-21T15:42:35.972 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_weekday", 2024-05-21T15:42:35.972 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:35.972 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.972 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.972 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:35.972 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-05-21T15:42:35.972 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-05-21T15:42:35.972 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.972 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-21T15:42:35.972 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T15:42:35.972 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.972 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.972 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.973 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_max_iterations": { 2024-05-21T15:42:35.973 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_max_iterations", 2024-05-21T15:42:35.973 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:35.973 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.973 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.973 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "25", 2024-05-21T15:42:35.973 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-05-21T15:42:35.973 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "250", 2024-05-21T15:42:35.973 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.973 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-21T15:42:35.973 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.973 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.973 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.974 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.974 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_metrics": { 2024-05-21T15:42:35.974 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_metrics", 2024-05-21T15:42:35.974 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.974 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.974 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.974 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "pgs,objects,bytes", 2024-05-21T15:42:35.974 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.974 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.974 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.974 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-21T15:42:35.974 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-21T15:42:35.975 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.975 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.975 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.975 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_step": { 2024-05-21T15:42:35.975 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_step", 2024-05-21T15:42:35.975 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.975 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.975 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.975 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.5", 2024-05-21T15:42:35.975 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0.001", 2024-05-21T15:42:35.975 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "0.999", 2024-05-21T15:42:35.975 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.975 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "aggressiveness of optimization", 2024-05-21T15:42:35.976 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-21T15:42:35.976 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.976 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.976 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.976 INFO:teuthology.orchestra.run.smithi038.stdout: "end_time": { 2024-05-21T15:42:35.976 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_time", 2024-05-21T15:42:35.976 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.976 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.976 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.976 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2359", 2024-05-21T15:42:35.976 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.976 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.976 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.976 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ending time of day to automatically balance", 2024-05-21T15:42:35.977 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T15:42:35.977 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.977 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.977 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.977 INFO:teuthology.orchestra.run.smithi038.stdout: "end_weekday": { 2024-05-21T15:42:35.977 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_weekday", 2024-05-21T15:42:35.977 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:35.977 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.977 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.977 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:35.977 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-05-21T15:42:35.977 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-05-21T15:42:35.977 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.977 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-21T15:42:35.978 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T15:42:35.978 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.978 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.978 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.978 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:35.978 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:35.978 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.978 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.978 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.978 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.978 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.978 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.979 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.979 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.979 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.979 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.979 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.979 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.979 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.979 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.979 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.979 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.979 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.979 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.979 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.979 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:35.980 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:35.980 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.980 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.980 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.980 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.980 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.980 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.980 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.980 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.980 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.980 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.980 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.980 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.981 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:35.981 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:35.981 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.981 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.981 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.981 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:35.981 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.981 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.981 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.981 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:35.981 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:35.981 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:35.981 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:35.981 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:35.982 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:35.982 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.982 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.982 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.982 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.982 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.982 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.982 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:35.982 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:35.982 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.982 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.982 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.982 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.982 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.983 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.983 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.983 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:35.983 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.983 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.983 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.983 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.983 INFO:teuthology.orchestra.run.smithi038.stdout: "min_score": { 2024-05-21T15:42:35.983 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_score", 2024-05-21T15:42:35.983 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.983 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.983 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.983 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:35.983 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.984 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.984 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.984 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-21T15:42:35.984 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.984 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.984 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.984 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.984 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-05-21T15:42:35.984 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-05-21T15:42:35.984 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.984 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.984 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.984 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "upmap", 2024-05-21T15:42:35.984 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.985 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.985 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.985 INFO:teuthology.orchestra.run.smithi038.stdout: "crush-compat", 2024-05-21T15:42:35.985 INFO:teuthology.orchestra.run.smithi038.stdout: "none", 2024-05-21T15:42:35.985 INFO:teuthology.orchestra.run.smithi038.stdout: "read", 2024-05-21T15:42:35.985 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap", 2024-05-21T15:42:35.985 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap-read" 2024-05-21T15:42:35.985 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.985 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Balancer mode", 2024-05-21T15:42:35.985 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.985 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.985 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.985 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.986 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_ids": { 2024-05-21T15:42:35.986 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_ids", 2024-05-21T15:42:35.986 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.986 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.986 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.986 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.986 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.986 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.986 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.986 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-21T15:42:35.986 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.986 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.986 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.986 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.987 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:35.987 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:35.987 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.987 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.987 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.987 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:35.987 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.987 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.987 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.987 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-21T15:42:35.987 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.987 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.987 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.987 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.988 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_deviation": { 2024-05-21T15:42:35.988 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_deviation", 2024-05-21T15:42:35.988 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.988 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.988 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.988 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-05-21T15:42:35.988 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-05-21T15:42:35.988 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.988 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.988 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-21T15:42:35.988 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-21T15:42:35.988 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.988 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.989 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.989 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_optimizations": { 2024-05-21T15:42:35.989 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_optimizations", 2024-05-21T15:42:35.989 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:35.989 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.989 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:35.989 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:35.989 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.989 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.989 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.989 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-21T15:42:35.989 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.989 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.990 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.990 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.990 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:35.990 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.990 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:35.990 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm", 2024-05-21T15:42:35.990 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:35.990 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:35.990 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:35.990 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_down_multiplier": { 2024-05-21T15:42:35.990 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_down_multiplier", 2024-05-21T15:42:35.990 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.990 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.990 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.991 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-05-21T15:42:35.991 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.991 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.991 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.991 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-21T15:42:35.991 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.991 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.991 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.991 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.991 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_refresh_rate": { 2024-05-21T15:42:35.991 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_refresh_rate", 2024-05-21T15:42:35.991 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.991 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.991 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.992 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "20", 2024-05-21T15:42:35.992 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.992 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.992 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.992 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-21T15:42:35.992 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.992 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.992 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.992 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.992 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_starting_port": { 2024-05-21T15:42:35.992 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_starting_port", 2024-05-21T15:42:35.992 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:35.992 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.993 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.993 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4721", 2024-05-21T15:42:35.993 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.993 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.993 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.993 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-21T15:42:35.993 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.993 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.993 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.993 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.993 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_ptrace": { 2024-05-21T15:42:35.993 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_ptrace", 2024-05-21T15:42:35.993 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.993 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.994 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.994 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.994 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.994 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.994 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.994 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-21T15:42:35.994 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-21T15:42:35.994 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.994 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.994 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.994 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_interval": { 2024-05-21T15:42:35.994 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_interval", 2024-05-21T15:42:35.994 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-21T15:42:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_memory_target_ratio": { 2024-05-21T15:42:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_memory_target_ratio", 2024-05-21T15:42:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:35.996 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.996 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.996 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.7", 2024-05-21T15:42:35.996 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.996 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.996 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.996 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-21T15:42:35.996 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.996 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.996 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.996 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.996 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm_log_destination": { 2024-05-21T15:42:35.996 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm_log_destination", 2024-05-21T15:42:35.996 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: "file", 2024-05-21T15:42:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: "file,syslog", 2024-05-21T15:42:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: "syslog" 2024-05-21T15:42:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-21T15:42:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.997 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.998 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.998 INFO:teuthology.orchestra.run.smithi038.stdout: "cgroups_split": { 2024-05-21T15:42:35.998 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cgroups_split", 2024-05-21T15:42:35.998 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.998 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.998 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.998 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:35.998 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.998 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.998 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.998 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-21T15:42:35.998 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.998 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.998 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:35.999 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:35.999 INFO:teuthology.orchestra.run.smithi038.stdout: "config_checks_enabled": { 2024-05-21T15:42:35.999 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_checks_enabled", 2024-05-21T15:42:35.999 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:35.999 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:35.999 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:35.999 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:35.999 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:35.999 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:35.999 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:35.999 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-21T15:42:35.999 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:35.999 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:35.999 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.000 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.000 INFO:teuthology.orchestra.run.smithi038.stdout: "config_dashboard": { 2024-05-21T15:42:36.000 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_dashboard", 2024-05-21T15:42:36.000 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.000 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.000 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.000 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.000 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.000 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.000 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.000 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-21T15:42:36.000 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.000 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.000 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_alertmanager": { 2024-05-21T15:42:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_alertmanager", 2024-05-21T15:42:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-21T15:42:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-05-21T15:42:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.002 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.002 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_base": { 2024-05-21T15:42:36.002 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_base", 2024-05-21T15:42:36.002 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.002 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.002 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.002 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-21T15:42:36.002 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.002 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.002 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.002 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Container image name, without the tag", 2024-05-21T15:42:36.002 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.002 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.003 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.003 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.003 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_elasticsearch": { 2024-05-21T15:42:36.003 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_elasticsearch", 2024-05-21T15:42:36.003 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.003 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.003 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.003 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-21T15:42:36.003 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.003 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.003 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.003 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "elasticsearch container image", 2024-05-21T15:42:36.003 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.003 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.004 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.004 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.004 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_grafana": { 2024-05-21T15:42:36.004 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_grafana", 2024-05-21T15:42:36.004 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.004 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.004 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.004 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-21T15:42:36.004 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.004 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.004 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.004 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-05-21T15:42:36.004 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.004 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.005 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.005 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.005 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_haproxy": { 2024-05-21T15:42:36.005 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_haproxy", 2024-05-21T15:42:36.005 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.005 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.005 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.005 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-21T15:42:36.005 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.005 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.005 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.005 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "HAproxy container image", 2024-05-21T15:42:36.005 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.006 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.006 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.006 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.006 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_agent": { 2024-05-21T15:42:36.006 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_agent", 2024-05-21T15:42:36.006 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.006 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.006 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.006 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-21T15:42:36.006 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.006 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.006 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.006 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger agent container image", 2024-05-21T15:42:36.006 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.007 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.007 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.007 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.007 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_collector": { 2024-05-21T15:42:36.007 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_collector", 2024-05-21T15:42:36.007 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.007 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.007 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.007 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-21T15:42:36.007 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.007 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.007 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.007 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger collector container image", 2024-05-21T15:42:36.007 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.008 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.008 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.008 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.008 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_query": { 2024-05-21T15:42:36.008 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_query", 2024-05-21T15:42:36.008 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.008 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.008 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.008 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-21T15:42:36.008 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.008 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.008 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.008 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger query container image", 2024-05-21T15:42:36.009 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.009 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.009 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.009 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.009 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_keepalived": { 2024-05-21T15:42:36.009 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_keepalived", 2024-05-21T15:42:36.009 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.009 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.009 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.009 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-21T15:42:36.009 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.009 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.009 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.009 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Keepalived container image", 2024-05-21T15:42:36.010 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.010 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.010 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.010 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.010 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_loki": { 2024-05-21T15:42:36.010 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_loki", 2024-05-21T15:42:36.010 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.010 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.010 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.010 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-21T15:42:36.010 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.010 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.010 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.011 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Loki container image", 2024-05-21T15:42:36.011 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.011 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.011 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.011 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.011 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_node_exporter": { 2024-05-21T15:42:36.011 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_node_exporter", 2024-05-21T15:42:36.011 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.011 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.011 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.011 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-21T15:42:36.011 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.011 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.011 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.012 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-05-21T15:42:36.012 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.012 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.012 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.012 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.012 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_nvmeof": { 2024-05-21T15:42:36.012 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_nvmeof", 2024-05-21T15:42:36.012 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.012 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.012 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.012 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-21T15:42:36.012 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.012 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.012 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.013 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Nvme-of container image", 2024-05-21T15:42:36.013 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.013 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.013 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.013 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.013 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_prometheus": { 2024-05-21T15:42:36.013 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_prometheus", 2024-05-21T15:42:36.013 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.013 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.013 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.013 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-21T15:42:36.013 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.013 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.013 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.014 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-05-21T15:42:36.014 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.014 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.014 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.014 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.014 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_promtail": { 2024-05-21T15:42:36.014 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_promtail", 2024-05-21T15:42:36.014 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.014 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.014 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.014 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-21T15:42:36.014 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.014 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Promtail container image", 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_samba": { 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_samba", 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.015 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.016 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Samba/SMB container image", 2024-05-21T15:42:36.016 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.016 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.016 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.016 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.016 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_snmp_gateway": { 2024-05-21T15:42:36.016 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_snmp_gateway", 2024-05-21T15:42:36.016 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.016 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.016 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.016 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-21T15:42:36.016 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.016 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.017 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.017 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SNMP Gateway container image", 2024-05-21T15:42:36.017 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.017 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.017 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.017 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.017 INFO:teuthology.orchestra.run.smithi038.stdout: "container_init": { 2024-05-21T15:42:36.017 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_init", 2024-05-21T15:42:36.017 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.017 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.017 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.017 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.017 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.018 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.018 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.018 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Run podman/docker with `--init`", 2024-05-21T15:42:36.018 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.018 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.018 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.018 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.018 INFO:teuthology.orchestra.run.smithi038.stdout: "daemon_cache_timeout": { 2024-05-21T15:42:36.018 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "daemon_cache_timeout", 2024-05-21T15:42:36.018 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.018 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.018 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.018 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:36.019 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.019 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.019 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.019 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-21T15:42:36.019 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.019 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.019 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.019 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.019 INFO:teuthology.orchestra.run.smithi038.stdout: "default_cephadm_command_timeout": { 2024-05-21T15:42:36.019 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_cephadm_command_timeout", 2024-05-21T15:42:36.019 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.019 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.020 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.020 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "900", 2024-05-21T15:42:36.020 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.020 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.020 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.020 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-21T15:42:36.020 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.020 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.020 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.020 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.020 INFO:teuthology.orchestra.run.smithi038.stdout: "default_registry": { 2024-05-21T15:42:36.020 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_registry", 2024-05-21T15:42:36.020 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.020 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.021 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.021 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io", 2024-05-21T15:42:36.021 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.021 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.021 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.021 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-21T15:42:36.021 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.021 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.021 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.021 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.021 INFO:teuthology.orchestra.run.smithi038.stdout: "device_cache_timeout": { 2024-05-21T15:42:36.021 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_cache_timeout", 2024-05-21T15:42:36.021 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.022 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.022 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.022 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1800", 2024-05-21T15:42:36.022 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.022 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.022 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.022 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache device inventory", 2024-05-21T15:42:36.022 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.022 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.022 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.022 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.022 INFO:teuthology.orchestra.run.smithi038.stdout: "device_enhanced_scan": { 2024-05-21T15:42:36.023 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_enhanced_scan", 2024-05-21T15:42:36.023 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.023 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.023 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.023 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.023 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.023 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.023 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.023 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-21T15:42:36.023 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.023 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.023 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.023 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.024 INFO:teuthology.orchestra.run.smithi038.stdout: "facts_cache_timeout": { 2024-05-21T15:42:36.024 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "facts_cache_timeout", 2024-05-21T15:42:36.024 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.024 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.024 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.024 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:36.024 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.024 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.024 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.024 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache host facts data", 2024-05-21T15:42:36.024 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.024 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.024 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.024 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.025 INFO:teuthology.orchestra.run.smithi038.stdout: "grafana_dashboards_path": { 2024-05-21T15:42:36.025 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "grafana_dashboards_path", 2024-05-21T15:42:36.025 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.025 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.025 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.025 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-21T15:42:36.025 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.025 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.025 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.025 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-21T15:42:36.025 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.025 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.025 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.025 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.026 INFO:teuthology.orchestra.run.smithi038.stdout: "host_check_interval": { 2024-05-21T15:42:36.026 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "host_check_interval", 2024-05-21T15:42:36.026 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.026 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.026 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.026 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:36.026 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.026 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.026 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.026 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to perform a host check", 2024-05-21T15:42:36.026 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.026 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.026 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.027 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.027 INFO:teuthology.orchestra.run.smithi038.stdout: "hw_monitoring": { 2024-05-21T15:42:36.027 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hw_monitoring", 2024-05-21T15:42:36.027 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.027 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.027 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.027 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.027 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.027 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.027 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.027 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-21T15:42:36.027 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.027 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.027 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.028 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.028 INFO:teuthology.orchestra.run.smithi038.stdout: "inventory_list_all": { 2024-05-21T15:42:36.028 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "inventory_list_all", 2024-05-21T15:42:36.028 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.028 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.028 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.028 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.028 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.028 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.028 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.028 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-21T15:42:36.028 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.028 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.028 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.029 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.029 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.029 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.029 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.029 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.029 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.029 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.029 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.029 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.029 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.029 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.029 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.029 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.029 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.030 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.030 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.030 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.030 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.030 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.030 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.030 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.030 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.030 INFO:teuthology.orchestra.run.smithi038.stdout: "log_refresh_metadata": { 2024-05-21T15:42:36.030 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_refresh_metadata", 2024-05-21T15:42:36.030 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.030 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.030 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.031 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.031 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.031 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.031 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.031 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-21T15:42:36.031 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.031 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.031 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.031 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.031 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.031 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.031 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.031 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.032 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.032 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.032 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.032 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.032 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.032 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-21T15:42:36.032 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.032 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.032 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.032 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.032 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.032 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.032 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.032 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.033 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.033 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.033 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.033 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.033 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.033 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.033 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.033 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.033 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.033 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.033 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.033 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.034 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.034 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.034 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.034 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.034 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.034 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.034 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.034 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.034 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.034 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.034 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.034 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.034 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.034 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.035 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.035 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.035 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.035 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.035 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.035 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-21T15:42:36.035 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-21T15:42:36.035 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.035 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.035 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.035 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.035 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.035 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.036 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.036 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-21T15:42:36.036 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.036 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.036 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.036 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.036 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-21T15:42:36.036 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-21T15:42:36.036 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.036 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.036 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.036 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "*", 2024-05-21T15:42:36.036 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.037 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.037 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.037 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-21T15:42:36.037 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.037 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.037 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.037 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.037 INFO:teuthology.orchestra.run.smithi038.stdout: "max_count_per_host": { 2024-05-21T15:42:36.037 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_count_per_host", 2024-05-21T15:42:36.037 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.037 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.037 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.037 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:36.037 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.038 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.038 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.038 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of daemons per service per host", 2024-05-21T15:42:36.038 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.038 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.038 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.038 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.038 INFO:teuthology.orchestra.run.smithi038.stdout: "max_osd_draining_count": { 2024-05-21T15:42:36.038 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_osd_draining_count", 2024-05-21T15:42:36.038 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.038 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.038 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.038 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:36.038 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.039 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.039 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.039 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-21T15:42:36.039 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.039 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.039 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.039 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.039 INFO:teuthology.orchestra.run.smithi038.stdout: "migration_current": { 2024-05-21T15:42:36.039 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "migration_current", 2024-05-21T15:42:36.039 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.039 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.039 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.039 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.039 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.040 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.040 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.040 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "internal - do not modify", 2024-05-21T15:42:36.040 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.040 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.040 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.040 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.040 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-05-21T15:42:36.040 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-05-21T15:42:36.040 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.040 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.040 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.040 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "root", 2024-05-21T15:42:36.041 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.041 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.041 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.041 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm-package", 2024-05-21T15:42:36.041 INFO:teuthology.orchestra.run.smithi038.stdout: "root" 2024-05-21T15:42:36.041 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.041 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "mode for remote execution of cephadm", 2024-05-21T15:42:36.041 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.041 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.041 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.041 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.041 INFO:teuthology.orchestra.run.smithi038.stdout: "oob_default_addr": { 2024-05-21T15:42:36.041 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "oob_default_addr", 2024-05-21T15:42:36.041 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.042 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.042 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.042 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "169.254.1.1", 2024-05-21T15:42:36.042 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.042 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.042 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.042 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-21T15:42:36.042 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.042 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.042 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.042 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.042 INFO:teuthology.orchestra.run.smithi038.stdout: "prometheus_alerts_path": { 2024-05-21T15:42:36.042 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus_alerts_path", 2024-05-21T15:42:36.042 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.043 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.043 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.043 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-21T15:42:36.043 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.043 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.043 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.043 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-21T15:42:36.043 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.043 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.043 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.043 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.043 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_insecure": { 2024-05-21T15:42:36.043 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_insecure", 2024-05-21T15:42:36.044 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.044 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.044 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.044 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.044 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.044 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.044 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.044 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-21T15:42:36.044 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.044 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.044 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.044 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.044 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_password": { 2024-05-21T15:42:36.044 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_password", 2024-05-21T15:42:36.045 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.045 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.045 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.045 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.045 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.045 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.045 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.045 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-21T15:42:36.045 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.045 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.045 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.045 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.045 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_url": { 2024-05-21T15:42:36.046 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_url", 2024-05-21T15:42:36.046 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.046 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.046 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.046 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.046 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.046 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.046 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.046 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-21T15:42:36.046 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.046 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.046 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.046 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.046 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_username": { 2024-05-21T15:42:36.047 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_username", 2024-05-21T15:42:36.047 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.047 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.047 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.047 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.047 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.047 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.047 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.047 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-21T15:42:36.047 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.047 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.047 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.047 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.047 INFO:teuthology.orchestra.run.smithi038.stdout: "secure_monitoring_stack": { 2024-05-21T15:42:36.048 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secure_monitoring_stack", 2024-05-21T15:42:36.048 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.048 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.048 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.048 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.048 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.048 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.048 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.048 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-21T15:42:36.048 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.048 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.048 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.048 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: "service_discovery_port": { 2024-05-21T15:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "service_discovery_port", 2024-05-21T15:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8765", 2024-05-21T15:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "cephadm service discovery port", 2024-05-21T15:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.049 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.050 INFO:teuthology.orchestra.run.smithi038.stdout: "ssh_config_file": { 2024-05-21T15:42:36.050 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssh_config_file", 2024-05-21T15:42:36.050 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.050 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.050 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.050 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.050 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.050 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.050 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.050 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-21T15:42:36.050 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.050 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.050 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.051 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.051 INFO:teuthology.orchestra.run.smithi038.stdout: "use_agent": { 2024-05-21T15:42:36.051 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_agent", 2024-05-21T15:42:36.051 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.051 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.051 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.051 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.051 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.051 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.051 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.051 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-21T15:42:36.051 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.051 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.051 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.052 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.052 INFO:teuthology.orchestra.run.smithi038.stdout: "use_repo_digest": { 2024-05-21T15:42:36.052 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_repo_digest", 2024-05-21T15:42:36.052 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.052 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.052 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.052 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.052 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.052 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.052 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.052 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-21T15:42:36.052 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.052 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_failed_host_check": { 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_failed_host_check", 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if the host check fails", 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.053 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.054 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.054 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_daemons": { 2024-05-21T15:42:36.054 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_daemons", 2024-05-21T15:42:36.054 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.054 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.054 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.054 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.054 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.054 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.054 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.054 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-21T15:42:36.054 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.054 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.055 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.055 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.055 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_hosts": { 2024-05-21T15:42:36.055 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_hosts", 2024-05-21T15:42:36.055 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.055 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.055 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.055 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.055 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.055 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.055 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.055 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-21T15:42:36.055 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.056 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.056 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.056 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.056 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.056 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.056 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.056 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crash", 2024-05-21T15:42:36.056 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.056 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.056 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.056 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.056 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.056 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.056 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.057 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.057 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.057 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.057 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.057 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.057 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.057 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.057 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.057 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.057 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.057 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.057 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.057 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.057 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.058 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.058 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.058 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.058 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.058 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.058 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.058 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.058 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.058 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.058 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.058 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.058 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.058 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.059 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.059 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.059 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.059 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.059 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.059 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.059 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.059 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.059 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.059 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.059 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.059 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.059 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.059 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.061 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.061 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.061 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.061 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.061 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.061 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.061 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.061 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.061 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.061 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.061 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.061 INFO:teuthology.orchestra.run.smithi038.stdout: "retain_interval": { 2024-05-21T15:42:36.061 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retain_interval", 2024-05-21T15:42:36.061 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "31536000", 2024-05-21T15:42:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-21T15:42:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_recent_interval": { 2024-05-21T15:42:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_recent_interval", 2024-05-21T15:42:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.063 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.063 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.063 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1209600", 2024-05-21T15:42:36.063 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.063 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.063 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.063 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-21T15:42:36.063 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.063 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.063 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.063 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.063 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.063 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.064 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.064 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dashboard", 2024-05-21T15:42:36.064 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.064 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.064 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.064 INFO:teuthology.orchestra.run.smithi038.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-21T15:42:36.064 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-21T15:42:36.064 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.064 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.064 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.064 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:36.064 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.065 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.065 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.065 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.065 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.065 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.065 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.065 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.065 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_HOST": { 2024-05-21T15:42:36.065 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-21T15:42:36.065 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.065 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.065 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.065 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.066 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.066 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.066 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.066 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.066 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.066 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.066 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.066 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.066 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-21T15:42:36.066 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-21T15:42:36.066 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.066 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.066 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.067 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.067 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.067 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.067 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.067 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.067 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.067 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.067 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.067 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.067 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_ENABLED": { 2024-05-21T15:42:36.067 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_ENABLED", 2024-05-21T15:42:36.067 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.067 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.068 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.068 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.068 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.068 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.068 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.068 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.068 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.068 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.068 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.068 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.068 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-21T15:42:36.068 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-21T15:42:36.068 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.068 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.069 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.069 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.069 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.069 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.069 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.069 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.069 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.069 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.069 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.069 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.069 INFO:teuthology.orchestra.run.smithi038.stdout: "ENABLE_BROWSABLE_API": { 2024-05-21T15:42:36.069 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-21T15:42:36.069 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.070 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.070 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.070 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.070 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.070 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.070 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.070 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.070 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.070 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.070 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.070 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.070 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-21T15:42:36.070 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-21T15:42:36.070 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.071 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.071 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.071 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.071 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.071 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.071 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.071 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.071 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.071 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.071 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.071 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.071 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-21T15:42:36.071 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-21T15:42:36.071 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.072 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.072 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.072 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.072 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.072 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.072 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.072 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.072 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.072 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.072 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.072 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.072 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-21T15:42:36.072 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-21T15:42:36.073 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.073 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.073 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.073 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.073 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.073 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.073 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.073 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.073 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.073 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.073 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.073 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.073 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-21T15:42:36.073 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-21T15:42:36.074 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.074 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.074 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.074 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.074 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.074 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.074 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.074 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.074 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.074 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.074 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.074 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.074 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-21T15:42:36.074 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-21T15:42:36.075 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.075 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.075 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.075 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.075 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.075 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.075 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.075 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.075 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.075 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.075 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.075 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.075 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-21T15:42:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-21T15:42:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-21T15:42:36.077 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-21T15:42:36.077 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.077 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.077 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.077 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.077 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.077 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.077 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.077 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.077 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.077 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.077 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.077 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.077 INFO:teuthology.orchestra.run.smithi038.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-21T15:42:36.078 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-21T15:42:36.078 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.078 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.078 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.078 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.078 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.078 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.078 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.078 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.078 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.078 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.078 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_PASSWORD": { 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.079 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.080 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-21T15:42:36.080 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-21T15:42:36.080 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.080 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.080 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.080 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.080 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.080 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.080 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.080 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.080 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.080 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.080 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.080 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.081 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_URL": { 2024-05-21T15:42:36.081 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_URL", 2024-05-21T15:42:36.081 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.081 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.081 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.081 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.081 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.081 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.081 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.081 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.081 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.081 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.081 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.082 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.082 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_USERNAME": { 2024-05-21T15:42:36.082 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-21T15:42:36.082 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.082 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.082 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.082 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-05-21T15:42:36.082 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.082 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.082 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.082 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.082 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.082 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.083 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.083 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.083 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-21T15:42:36.083 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-21T15:42:36.083 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.083 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.083 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.083 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.083 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.083 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.083 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.083 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.083 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.083 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.084 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.084 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.084 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-21T15:42:36.084 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-21T15:42:36.084 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.084 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.084 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.084 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.084 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.084 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.084 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.084 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.084 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.084 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.085 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.085 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.085 INFO:teuthology.orchestra.run.smithi038.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-21T15:42:36.085 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-21T15:42:36.085 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.085 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.085 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.085 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.085 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.085 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.085 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.085 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.085 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.085 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.086 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.086 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.086 INFO:teuthology.orchestra.run.smithi038.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-21T15:42:36.086 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-21T15:42:36.086 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.086 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.086 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.086 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.086 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.086 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.086 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.086 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.086 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.087 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.087 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.087 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.087 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_HOST": { 2024-05-21T15:42:36.087 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-21T15:42:36.087 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.087 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.087 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.087 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.087 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.087 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.087 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.087 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.088 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.088 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.088 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.088 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.088 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-21T15:42:36.088 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-21T15:42:36.088 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.088 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.088 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.088 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.088 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.088 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.088 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.089 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.089 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.089 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.089 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.089 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.089 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-21T15:42:36.089 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-21T15:42:36.089 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.089 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.089 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.089 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.089 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.089 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.090 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.090 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.090 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.090 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.090 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.090 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.090 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-21T15:42:36.090 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-21T15:42:36.090 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.090 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.090 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.090 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.091 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.091 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.091 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.091 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.091 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.091 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.091 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.091 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.091 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-21T15:42:36.091 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-21T15:42:36.091 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.091 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.091 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.092 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.092 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.092 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.092 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.092 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.092 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.092 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.092 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.092 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.092 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-21T15:42:36.092 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-21T15:42:36.092 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.092 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.093 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.093 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.093 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.093 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.093 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.093 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.093 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.093 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.093 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.093 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.093 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-21T15:42:36.093 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-21T15:42:36.093 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.094 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.094 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.094 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.094 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.094 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.094 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.094 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.094 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.094 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.094 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.094 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.094 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-21T15:42:36.094 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-21T15:42:36.095 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.095 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.095 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.095 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.095 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.095 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.095 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.095 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.095 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.095 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.095 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.095 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.095 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-21T15:42:36.096 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-21T15:42:36.096 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.096 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.096 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.096 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.096 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.096 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.096 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.096 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.096 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.096 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.096 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.096 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.096 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_ENABLED": { 2024-05-21T15:42:36.097 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-21T15:42:36.097 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.097 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.097 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.097 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.097 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.097 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.097 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.097 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.097 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.097 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.097 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.097 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.098 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-21T15:42:36.098 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-21T15:42:36.098 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.098 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.098 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.098 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-21T15:42:36.098 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.098 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.098 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.098 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.098 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.098 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.099 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.099 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.099 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-21T15:42:36.099 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-21T15:42:36.099 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.099 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.099 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.099 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:36.099 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.099 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.099 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.099 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.099 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.099 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.100 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.100 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.100 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-21T15:42:36.100 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-21T15:42:36.100 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.100 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.100 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.100 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8", 2024-05-21T15:42:36.100 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.100 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.100 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.100 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.100 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.101 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.101 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.101 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.101 INFO:teuthology.orchestra.run.smithi038.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-21T15:42:36.101 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-21T15:42:36.101 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.101 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.101 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.101 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "45", 2024-05-21T15:42:36.101 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.101 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.101 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.101 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.102 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.102 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.102 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.102 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.102 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ACCESS_KEY": { 2024-05-21T15:42:36.102 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-21T15:42:36.102 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.102 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.102 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.102 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.102 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.102 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.102 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.103 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.103 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.103 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.103 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.103 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.103 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-21T15:42:36.103 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-21T15:42:36.103 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.103 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.103 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.103 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-05-21T15:42:36.103 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.103 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SECRET_KEY": { 2024-05-21T15:42:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-21T15:42:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.104 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.105 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.105 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.105 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.105 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.105 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.105 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.105 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SSL_VERIFY": { 2024-05-21T15:42:36.105 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-21T15:42:36.105 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.105 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.105 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.105 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.105 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.106 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.106 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.106 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.106 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.106 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.106 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.106 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.106 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-21T15:42:36.106 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-21T15:42:36.106 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.106 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.106 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.106 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:36.107 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.107 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.107 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.107 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.107 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.107 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.107 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.107 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.107 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-21T15:42:36.107 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-21T15:42:36.107 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.107 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.107 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.107 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:36.108 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.108 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.108 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.108 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.108 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.108 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.108 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.108 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.108 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-21T15:42:36.108 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-21T15:42:36.108 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.108 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.108 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-05-21T15:42:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: "cross_origin_url": { 2024-05-21T15:42:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cross_origin_url", 2024-05-21T15:42:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: "crt_file": { 2024-05-21T15:42:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crt_file", 2024-05-21T15:42:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: "debug": { 2024-05-21T15:42:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "debug", 2024-05-21T15:42:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable/disable debug options", 2024-05-21T15:42:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: "jwt_token_ttl": { 2024-05-21T15:42:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "jwt_token_ttl", 2024-05-21T15:42:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "28800", 2024-05-21T15:42:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-05-21T15:42:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-05-21T15:42:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: "motd": { 2024-05-21T15:42:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "motd", 2024-05-21T15:42:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "The message of the day", 2024-05-21T15:42:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect_resolve_ip_addr": { 2024-05-21T15:42:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "redirect_resolve_ip_addr", 2024-05-21T15:42:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-05-21T15:42:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-05-21T15:42:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-05-21T15:42:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-05-21T15:42:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-05-21T15:42:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8080", 2024-05-21T15:42:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-05-21T15:42:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-05-21T15:42:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl_server_port": { 2024-05-21T15:42:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl_server_port", 2024-05-21T15:42:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8443", 2024-05-21T15:42:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-05-21T15:42:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-05-21T15:42:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "redirect", 2024-05-21T15:42:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect" 2024-05-21T15:42:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-05-21T15:42:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-05-21T15:42:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-05-21T15:42:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-05-21T15:42:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-05-21T15:42:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: "url_prefix": { 2024-05-21T15:42:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url_prefix", 2024-05-21T15:42:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-05-21T15:42:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_monitoring": { 2024-05-21T15:42:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_monitoring", 2024-05-21T15:42:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.131 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "monitor device health metrics", 2024-05-21T15:42:36.131 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.131 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.131 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.131 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.131 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.131 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.131 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.131 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.131 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.131 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.131 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.131 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.131 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.132 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.132 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.132 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.132 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.132 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.132 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.132 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.132 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.132 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.132 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.132 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.132 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.132 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.133 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.133 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.133 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.133 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.133 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.133 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.133 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.133 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.133 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.133 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.133 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.133 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.133 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.134 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.134 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.134 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.134 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.134 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.134 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.134 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.134 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.134 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.134 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.134 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.134 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.135 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.135 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.135 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.135 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.135 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.135 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.135 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.135 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.135 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.135 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.135 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.135 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.135 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.135 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.136 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.136 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.136 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.136 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.136 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.136 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.136 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.136 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.136 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.136 INFO:teuthology.orchestra.run.smithi038.stdout: "mark_out_threshold": { 2024-05-21T15:42:36.136 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mark_out_threshold", 2024-05-21T15:42:36.136 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.136 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.137 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.137 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2419200", 2024-05-21T15:42:36.137 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.137 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.137 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.137 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-21T15:42:36.137 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.137 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.137 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.137 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.137 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_name": { 2024-05-21T15:42:36.137 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_name", 2024-05-21T15:42:36.137 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.138 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.138 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.138 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "device_health_metrics", 2024-05-21T15:42:36.138 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.138 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.138 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.138 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-21T15:42:36.138 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.138 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.138 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.138 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.138 INFO:teuthology.orchestra.run.smithi038.stdout: "retention_period": { 2024-05-21T15:42:36.138 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retention_period", 2024-05-21T15:42:36.139 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.139 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.139 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.139 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15552000", 2024-05-21T15:42:36.139 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.139 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.139 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.139 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain device health metrics", 2024-05-21T15:42:36.139 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.139 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.139 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.139 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.139 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_frequency": { 2024-05-21T15:42:36.140 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_frequency", 2024-05-21T15:42:36.140 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.140 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.140 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.140 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-05-21T15:42:36.140 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.140 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.140 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.140 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-21T15:42:36.140 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.140 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.140 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.140 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.140 INFO:teuthology.orchestra.run.smithi038.stdout: "self_heal": { 2024-05-21T15:42:36.141 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "self_heal", 2024-05-21T15:42:36.141 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.141 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.141 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.141 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.141 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.141 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.141 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.141 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-21T15:42:36.141 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.141 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.141 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.141 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.142 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:36.142 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:36.142 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.142 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.142 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.142 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:36.142 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.142 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.142 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.142 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and check device health", 2024-05-21T15:42:36.142 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.142 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.143 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.143 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.143 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_threshold": { 2024-05-21T15:42:36.143 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_threshold", 2024-05-21T15:42:36.143 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.143 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.143 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.143 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7257600", 2024-05-21T15:42:36.143 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.143 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.143 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.143 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-21T15:42:36.143 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.143 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.144 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.144 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.144 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.144 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.144 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.144 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "diskprediction_local", 2024-05-21T15:42:36.144 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.144 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.144 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.144 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.144 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.144 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.144 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.145 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.145 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.145 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.145 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.145 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.145 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.145 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.145 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.145 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.145 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.145 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.145 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.145 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.146 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.146 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.146 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.146 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.146 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.146 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.146 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.146 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.146 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.146 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.146 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.146 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.146 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.147 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.147 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.147 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.147 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.147 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.147 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.147 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.147 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.147 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.147 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.147 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.147 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.147 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.148 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.148 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.148 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.148 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.148 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.148 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.148 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.148 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.148 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.148 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.148 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.148 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.148 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.149 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.149 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.149 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.149 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.149 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.149 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.149 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.149 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.149 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.149 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.149 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.149 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.149 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.149 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.150 INFO:teuthology.orchestra.run.smithi038.stdout: "predict_interval": { 2024-05-21T15:42:36.150 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predict_interval", 2024-05-21T15:42:36.150 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.150 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.150 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.150 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-05-21T15:42:36.150 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.150 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.150 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.151 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.151 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.151 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.151 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.151 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.151 INFO:teuthology.orchestra.run.smithi038.stdout: "predictor_model": { 2024-05-21T15:42:36.151 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predictor_model", 2024-05-21T15:42:36.151 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.151 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.151 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.151 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "prophetstor", 2024-05-21T15:42:36.151 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.151 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.152 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.152 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.152 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.152 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.152 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.152 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.152 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:36.152 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:36.152 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.152 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.152 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.152 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-05-21T15:42:36.152 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.152 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.153 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.153 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.153 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.153 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.153 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.153 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.153 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.153 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.153 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.153 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "influx", 2024-05-21T15:42:36.153 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": false, 2024-05-21T15:42:36.153 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "influxdb python module not found", 2024-05-21T15:42:36.153 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.154 INFO:teuthology.orchestra.run.smithi038.stdout: "batch_size": { 2024-05-21T15:42:36.154 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "batch_size", 2024-05-21T15:42:36.154 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.154 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.154 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.154 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5000", 2024-05-21T15:42:36.154 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.154 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.154 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.154 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-21T15:42:36.154 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.154 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.154 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: "database": { 2024-05-21T15:42:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "database", 2024-05-21T15:42:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "ceph", 2024-05-21T15:42:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-21T15:42:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.156 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.156 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": { 2024-05-21T15:42:36.156 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hostname", 2024-05-21T15:42:36.156 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.156 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.156 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.156 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.156 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.156 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.156 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.156 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server hostname", 2024-05-21T15:42:36.156 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.156 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.156 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.157 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.157 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:36.157 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:36.157 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.157 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.157 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.157 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "30", 2024-05-21T15:42:36.157 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "5", 2024-05-21T15:42:36.157 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.157 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.157 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-21T15:42:36.157 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.157 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.158 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.158 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.158 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.158 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.158 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.158 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.158 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.158 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.158 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.158 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.158 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.158 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.158 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.158 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.159 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.159 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.159 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.159 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.159 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.159 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.159 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.159 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.159 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.159 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.159 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.159 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.159 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.159 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.160 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.160 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.160 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.160 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.160 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.160 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.160 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.160 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.160 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.160 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.160 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.160 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.160 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.161 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.161 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.161 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.161 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.161 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.161 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.161 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.161 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.161 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.161 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.161 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.161 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.161 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.161 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.162 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.162 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.162 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.162 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.162 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.162 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.162 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.162 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.162 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.162 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.162 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.162 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.162 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: "password": { 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "password", 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "password of InfluxDB server user", 2024-05-21T15:42:36.163 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.164 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.164 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.164 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.164 INFO:teuthology.orchestra.run.smithi038.stdout: "port": { 2024-05-21T15:42:36.164 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "port", 2024-05-21T15:42:36.164 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.164 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.164 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.164 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8086", 2024-05-21T15:42:36.164 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.164 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.164 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.164 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server port", 2024-05-21T15:42:36.165 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.165 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.165 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.165 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.165 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-05-21T15:42:36.165 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-05-21T15:42:36.165 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.165 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.165 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.165 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "false", 2024-05-21T15:42:36.165 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.165 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.165 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.165 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T15:42:36.166 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.166 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.166 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.166 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.166 INFO:teuthology.orchestra.run.smithi038.stdout: "threads": { 2024-05-21T15:42:36.166 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threads", 2024-05-21T15:42:36.166 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.166 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.166 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.166 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-05-21T15:42:36.166 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-05-21T15:42:36.166 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "32", 2024-05-21T15:42:36.166 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.166 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-21T15:42:36.167 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.167 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.167 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.167 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.167 INFO:teuthology.orchestra.run.smithi038.stdout: "username": { 2024-05-21T15:42:36.167 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "username", 2024-05-21T15:42:36.167 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.167 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.167 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.167 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.167 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.167 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.168 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.168 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "username of InfluxDB server user", 2024-05-21T15:42:36.168 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.168 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.168 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.168 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.168 INFO:teuthology.orchestra.run.smithi038.stdout: "verify_ssl": { 2024-05-21T15:42:36.168 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "verify_ssl", 2024-05-21T15:42:36.168 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.168 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.168 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.168 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "true", 2024-05-21T15:42:36.168 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.168 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.169 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.169 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T15:42:36.169 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.169 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.169 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.169 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.169 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.169 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.169 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.169 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "insights", 2024-05-21T15:42:36.169 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.169 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.169 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.170 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.170 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.170 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.170 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.170 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.170 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.170 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.170 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.170 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.170 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.170 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.170 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.170 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.171 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.171 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.171 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.171 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.171 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.171 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.171 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.171 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.171 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.171 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.171 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.171 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.171 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.171 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.172 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.172 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.172 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.172 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.172 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.172 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.172 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.172 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.172 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.172 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.172 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.172 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.172 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.173 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.173 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.173 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.173 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.173 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.173 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.173 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.173 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.173 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.173 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.173 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.173 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.173 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.174 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.174 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.174 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.174 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.174 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.174 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.174 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.174 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.174 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.174 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.174 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.174 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.175 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.175 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.175 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.175 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.175 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.175 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.175 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.175 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.175 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "iostat", 2024-05-21T15:42:36.175 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.175 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.175 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.175 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.175 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.176 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.177 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.177 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.177 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.177 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.177 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.177 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.177 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.177 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.177 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.177 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.177 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.177 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.177 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.178 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.178 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.178 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.178 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.178 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.178 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.178 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.178 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.178 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.178 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.178 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.178 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.178 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.178 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.179 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.179 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.179 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.179 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.179 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.179 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.179 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.179 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.179 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.179 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.179 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.179 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.179 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.179 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.180 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.180 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.180 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.180 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.180 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.180 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.180 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.180 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.180 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.180 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.180 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.180 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.180 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.180 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.181 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.181 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.181 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "k8sevents", 2024-05-21T15:42:36.181 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.181 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.181 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.181 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_event_retention_days": { 2024-05-21T15:42:36.181 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ceph_event_retention_days", 2024-05-21T15:42:36.181 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.181 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.181 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.181 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7", 2024-05-21T15:42:36.181 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.181 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.182 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.182 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-21T15:42:36.182 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.182 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.182 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.182 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.182 INFO:teuthology.orchestra.run.smithi038.stdout: "config_check_secs": { 2024-05-21T15:42:36.182 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_check_secs", 2024-05-21T15:42:36.182 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.182 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.182 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.182 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:36.182 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "10", 2024-05-21T15:42:36.182 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.183 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.183 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-21T15:42:36.183 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.183 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.183 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.183 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.183 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.183 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.183 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.183 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.183 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.183 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.183 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.183 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.184 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.184 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.184 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.184 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.184 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.184 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.184 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.184 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.184 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.184 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.184 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.184 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.184 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.184 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.185 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.185 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.185 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.185 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.185 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.185 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.185 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.185 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.185 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.185 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.185 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.185 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.185 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.186 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.186 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.186 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.186 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.186 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.186 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.186 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.186 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.186 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.186 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.186 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.186 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.186 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.186 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.187 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.187 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.187 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.187 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.187 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.187 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.187 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.187 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.187 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.187 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.187 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.187 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.187 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.188 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.188 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.188 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.188 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.188 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.188 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.188 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.188 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.188 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.188 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.188 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.188 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "localpool", 2024-05-21T15:42:36.188 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.188 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: "failure_domain": { 2024-05-21T15:42:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "failure_domain", 2024-05-21T15:42:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "host", 2024-05-21T15:42:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "failure domain for any created local pool", 2024-05-21T15:42:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-21T15:42:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.190 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.190 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.190 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.190 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.190 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.190 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.190 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.190 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.190 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.190 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.190 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.190 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.190 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.191 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.191 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.191 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.191 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.191 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.191 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.191 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.191 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.191 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.191 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.191 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.191 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.191 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.191 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.192 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.192 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.192 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.192 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.192 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.192 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.192 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.192 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.192 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.192 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.192 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.192 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.192 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.192 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.193 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.193 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.193 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.193 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.193 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.193 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.193 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.193 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.193 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.193 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.193 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.193 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.193 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.193 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.194 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.194 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.194 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.194 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.194 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.194 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.194 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.194 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.194 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.194 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.194 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.194 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.194 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.195 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.195 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.195 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.195 INFO:teuthology.orchestra.run.smithi038.stdout: "min_size": { 2024-05-21T15:42:36.195 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_size", 2024-05-21T15:42:36.195 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.195 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.195 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.195 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.195 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.195 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.195 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.195 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default min_size for any created local pool", 2024-05-21T15:42:36.195 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-21T15:42:36.196 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.196 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.196 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.196 INFO:teuthology.orchestra.run.smithi038.stdout: "num_rep": { 2024-05-21T15:42:36.196 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "num_rep", 2024-05-21T15:42:36.196 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.196 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.196 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.196 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3", 2024-05-21T15:42:36.196 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.196 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.196 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.196 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default replica count for any created local pool", 2024-05-21T15:42:36.196 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_num": { 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_num", 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "128", 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default pg_num for any created local pool", 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.197 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.198 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.198 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.198 INFO:teuthology.orchestra.run.smithi038.stdout: "prefix": { 2024-05-21T15:42:36.198 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prefix", 2024-05-21T15:42:36.198 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.198 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.198 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.198 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.198 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.198 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.198 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.198 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name prefix for any created local pool", 2024-05-21T15:42:36.198 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: "subtree": { 2024-05-21T15:42:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "subtree", 2024-05-21T15:42:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "rack", 2024-05-21T15:42:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-21T15:42:36.199 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-21T15:42:36.200 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.200 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.200 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.200 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.200 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.200 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.200 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mds_autoscaler", 2024-05-21T15:42:36.200 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.200 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.200 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.200 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.200 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.200 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.200 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.201 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.201 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.201 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.201 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.201 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.201 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.201 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.201 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.201 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.201 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.201 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.201 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.201 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.201 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.202 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.202 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.202 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.202 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.202 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.202 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.202 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.202 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.202 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.202 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.202 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.202 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.202 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.203 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.203 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.203 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.203 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.203 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.203 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.203 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.203 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.203 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.203 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.203 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.203 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.203 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.204 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.204 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.204 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.204 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.204 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.204 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.204 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.204 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.204 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.204 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.204 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.204 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.204 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.205 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.205 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.205 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.205 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.205 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.205 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.205 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.205 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.205 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.205 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.205 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.205 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.205 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.206 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.206 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.206 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.206 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirroring", 2024-05-21T15:42:36.206 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.206 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.206 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.206 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.206 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.206 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.206 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.206 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.206 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.207 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.207 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.207 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.207 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.207 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.207 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.207 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.207 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.207 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.207 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.207 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.207 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.207 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.208 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.208 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.208 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.208 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.208 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.208 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.208 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.208 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.208 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.208 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.208 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.208 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.208 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.208 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.209 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.209 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.209 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.209 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.209 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.209 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.209 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.209 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.209 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.209 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.209 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.209 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.209 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.210 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.210 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.210 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.210 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.210 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.210 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.210 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.210 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.210 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.210 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.210 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.210 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.210 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.211 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.211 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.211 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.211 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.211 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.211 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.211 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.211 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.211 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.211 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.211 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.211 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.211 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.211 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.212 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "nfs", 2024-05-21T15:42:36.212 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.212 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.212 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.212 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.212 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.212 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.212 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.212 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.212 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.212 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.212 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.212 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.213 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.213 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.213 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.213 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.213 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.213 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.213 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.213 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.213 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.213 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.213 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.213 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.213 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.213 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.214 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.214 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.214 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.214 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.214 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.214 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.214 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.214 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.214 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.214 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.214 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.214 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.215 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.215 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.215 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.215 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.215 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.215 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.215 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.215 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.215 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.215 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.215 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.215 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.215 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.216 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.216 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.216 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.216 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.216 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.216 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.216 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.216 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.216 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.216 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.216 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.216 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.216 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.216 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.217 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.217 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.217 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.217 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.217 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.217 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.217 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.217 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.217 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.217 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.217 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.217 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-05-21T15:42:36.217 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.217 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.218 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.218 INFO:teuthology.orchestra.run.smithi038.stdout: "fail_fs": { 2024-05-21T15:42:36.218 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "fail_fs", 2024-05-21T15:42:36.218 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.218 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.218 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.218 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.218 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.218 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.218 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.218 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-21T15:42:36.218 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.218 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.219 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.219 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.219 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.219 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.219 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.219 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.219 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.219 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.219 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.219 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.219 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.219 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.219 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.219 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.220 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.220 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.220 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.220 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.220 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.220 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.220 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.220 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.220 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.220 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.220 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.220 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.220 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.221 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.222 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.222 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.222 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.222 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.222 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.222 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.222 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.222 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.222 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.222 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.222 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.222 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.222 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.223 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.223 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.223 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.223 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.223 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.223 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.223 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.223 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.223 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.223 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.223 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.223 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.223 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.223 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator": { 2024-05-21T15:42:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-05-21T15:42:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.224 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-05-21T15:42:36.225 INFO:teuthology.orchestra.run.smithi038.stdout: "rook", 2024-05-21T15:42:36.225 INFO:teuthology.orchestra.run.smithi038.stdout: "test_orchestrator" 2024-05-21T15:42:36.225 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.225 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Orchestrator backend", 2024-05-21T15:42:36.225 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.225 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.225 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.225 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.225 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.225 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.225 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.225 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_perf_query", 2024-05-21T15:42:36.225 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.226 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.226 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.226 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.226 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.226 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.226 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.226 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.226 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.226 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.226 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.226 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.226 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.226 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.226 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.227 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.227 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.227 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.227 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.227 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.227 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.227 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.227 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.227 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.227 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.227 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.227 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.227 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.227 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.228 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.228 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.228 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.228 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.228 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.228 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.228 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.228 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.228 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.228 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.228 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.228 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.228 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.229 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.230 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.230 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.230 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.230 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.230 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.230 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.230 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.230 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.230 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.230 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.230 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.230 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.230 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.231 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.231 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.231 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.231 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.231 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.231 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.231 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_support", 2024-05-21T15:42:36.231 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.231 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.231 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.231 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.231 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.231 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.231 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.233 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.233 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.233 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.233 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.233 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.233 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.233 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.233 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.233 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.233 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.233 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.233 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.233 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.233 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.234 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.234 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.234 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.234 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.234 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.234 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.234 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.234 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.234 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.234 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.234 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.234 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.234 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.234 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.235 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.235 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.235 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.235 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.235 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.235 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.235 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.235 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.235 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.235 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.235 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.235 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.235 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.235 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.236 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.236 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.236 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.236 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.236 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.236 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.236 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.236 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.236 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.236 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.236 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.236 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.236 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.236 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_autoscaler", 2024-05-21T15:42:36.237 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.237 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.237 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.237 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.237 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.237 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.237 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.237 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.237 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.237 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.237 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.237 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.237 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.237 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.238 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.238 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.238 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.238 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.238 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.238 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.238 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.238 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.238 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.238 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.238 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.238 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.238 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.239 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.239 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.239 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.239 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.239 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.239 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.239 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.239 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.239 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.239 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.239 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.239 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.239 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.239 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.240 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.240 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.240 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.240 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.240 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.240 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.240 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.240 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.240 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.240 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.240 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.240 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.240 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.240 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.241 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.241 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.241 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.241 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.241 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.241 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.241 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.241 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.241 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.241 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.241 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.241 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.241 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.242 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.242 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.242 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.242 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.242 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.242 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:36.242 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:36.242 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.242 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.242 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.242 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:36.242 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.242 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.243 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.243 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.243 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.243 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.243 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.243 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.243 INFO:teuthology.orchestra.run.smithi038.stdout: "threshold": { 2024-05-21T15:42:36.243 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threshold", 2024-05-21T15:42:36.243 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:36.243 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.243 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.243 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-05-21T15:42:36.243 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1.0", 2024-05-21T15:42:36.244 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.244 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.244 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "scaling threshold", 2024-05-21T15:42:36.244 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-21T15:42:36.244 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.244 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.244 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.244 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.244 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.244 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.244 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "progress", 2024-05-21T15:42:36.244 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.244 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.245 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.245 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_pg_recovery_event": { 2024-05-21T15:42:36.245 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_pg_recovery_event", 2024-05-21T15:42:36.245 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.245 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.245 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.245 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.245 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.245 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.245 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.245 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-21T15:42:36.245 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.245 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.246 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.246 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.246 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-05-21T15:42:36.246 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-05-21T15:42:36.246 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.246 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.246 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.246 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.246 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.246 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.246 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.246 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.246 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.247 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.247 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.247 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.247 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.247 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.247 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.247 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.247 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.247 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.247 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.247 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.247 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.248 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.248 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.248 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.248 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.248 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.248 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.248 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.248 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.248 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.248 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.248 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.248 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.248 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.248 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.249 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.249 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.249 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.249 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.249 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.249 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.249 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.249 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.249 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.249 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.249 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.249 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.249 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.250 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.250 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.250 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.250 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.250 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.250 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.250 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.250 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.250 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.250 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.250 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.250 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.250 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.251 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.251 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.251 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.251 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.251 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.251 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.251 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.251 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.251 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.251 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.251 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.251 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.251 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.252 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.252 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.252 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.252 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.252 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.252 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.252 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.252 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.252 INFO:teuthology.orchestra.run.smithi038.stdout: "max_completed_events": { 2024-05-21T15:42:36.252 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_completed_events", 2024-05-21T15:42:36.252 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.252 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.252 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.252 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "50", 2024-05-21T15:42:36.253 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.253 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.253 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.253 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "number of past completed events to remember", 2024-05-21T15:42:36.253 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.253 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.253 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.253 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.253 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-05-21T15:42:36.253 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-05-21T15:42:36.253 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.253 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.253 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.253 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-05-21T15:42:36.254 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.254 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.254 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.254 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long the module is going to sleep", 2024-05-21T15:42:36.254 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.254 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.254 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.254 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.254 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.254 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.254 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.254 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus", 2024-05-21T15:42:36.254 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.255 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.255 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.255 INFO:teuthology.orchestra.run.smithi038.stdout: "cache": { 2024-05-21T15:42:36.255 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cache", 2024-05-21T15:42:36.255 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.255 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.255 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.255 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.255 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.255 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.255 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.255 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.255 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.256 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.256 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.256 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.256 INFO:teuthology.orchestra.run.smithi038.stdout: "exclude_perf_counters": { 2024-05-21T15:42:36.256 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "exclude_perf_counters", 2024-05-21T15:42:36.256 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.256 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.256 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.256 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.256 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.256 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.256 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.256 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-21T15:42:36.256 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-21T15:42:36.257 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.257 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.257 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.257 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.257 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.257 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.257 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.257 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.257 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.257 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.257 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.257 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.257 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.257 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.258 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.258 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.258 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.258 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.258 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.258 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.258 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.258 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.258 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.258 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.258 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.258 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.258 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.258 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.259 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.259 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.259 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.259 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.259 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.259 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.259 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.259 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.259 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.259 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.259 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.259 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.259 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.260 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.260 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.260 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.260 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.260 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.260 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.260 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.260 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.260 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.260 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.260 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.260 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.260 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.260 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.261 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.261 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.261 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.261 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.261 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.261 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.261 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.261 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.261 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.261 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.261 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.261 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.261 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.262 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.262 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.262 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.262 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.262 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.262 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools": { 2024-05-21T15:42:36.262 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools", 2024-05-21T15:42:36.262 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.262 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.262 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.262 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.262 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.262 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.262 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.263 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.263 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.263 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.263 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.263 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.263 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-21T15:42:36.263 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-21T15:42:36.263 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.263 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.263 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.263 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "300", 2024-05-21T15:42:36.263 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.263 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.263 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.264 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.264 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.264 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.264 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.264 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.264 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_interval": { 2024-05-21T15:42:36.264 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_interval", 2024-05-21T15:42:36.264 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:36.264 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.264 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.264 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15.0", 2024-05-21T15:42:36.264 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.264 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.265 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.265 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.265 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.265 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.265 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.265 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.265 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-05-21T15:42:36.265 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-05-21T15:42:36.265 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.265 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.265 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.265 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-05-21T15:42:36.265 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.265 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.266 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.266 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-21T15:42:36.266 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.266 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.266 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.266 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.266 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-05-21T15:42:36.266 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-05-21T15:42:36.266 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.266 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.266 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.266 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "9283", 2024-05-21T15:42:36.266 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-21T15:42:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: "stale_cache_strategy": { 2024-05-21T15:42:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stale_cache_strategy", 2024-05-21T15:42:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "log", 2024-05-21T15:42:36.267 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.268 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.268 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.268 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.268 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.268 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.268 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.268 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.268 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-05-21T15:42:36.268 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-05-21T15:42:36.268 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.268 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.268 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.268 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "default", 2024-05-21T15:42:36.268 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.269 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.269 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.269 INFO:teuthology.orchestra.run.smithi038.stdout: "default", 2024-05-21T15:42:36.269 INFO:teuthology.orchestra.run.smithi038.stdout: "error" 2024-05-21T15:42:36.269 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.269 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.269 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.269 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.269 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.269 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.269 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-05-21T15:42:36.269 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-05-21T15:42:36.269 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.270 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.270 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.270 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-05-21T15:42:36.270 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-05-21T15:42:36.270 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-05-21T15:42:36.270 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.270 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.270 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.270 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.270 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.270 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.270 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.270 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.271 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.271 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-05-21T15:42:36.271 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.271 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.271 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.271 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.271 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.271 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.271 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.271 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.271 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.271 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.271 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.272 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.272 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.272 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.272 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.272 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.272 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.272 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.272 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.272 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.272 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.272 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.272 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.272 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.273 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.273 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.273 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.273 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.273 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.273 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.273 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.273 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.273 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.273 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.273 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.273 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.273 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.274 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.274 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.274 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.274 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.274 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.274 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.274 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.274 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.274 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.274 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.274 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.274 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.274 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.274 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.275 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.275 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.275 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.275 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.275 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.275 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.275 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.275 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.275 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.275 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.275 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.275 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.275 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.275 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.277 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.277 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.278 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.278 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.278 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.278 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.278 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.278 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.278 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_snap_create": { 2024-05-21T15:42:36.278 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_snap_create", 2024-05-21T15:42:36.278 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.278 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.278 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.278 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-05-21T15:42:36.279 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.279 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.279 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.279 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.279 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.279 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.279 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.279 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.279 INFO:teuthology.orchestra.run.smithi038.stdout: "mirror_snapshot_schedule": { 2024-05-21T15:42:36.279 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirror_snapshot_schedule", 2024-05-21T15:42:36.279 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.279 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.279 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.280 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.280 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.280 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.280 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.280 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.280 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.280 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.280 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.280 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.280 INFO:teuthology.orchestra.run.smithi038.stdout: "trash_purge_schedule": { 2024-05-21T15:42:36.280 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "trash_purge_schedule", 2024-05-21T15:42:36.280 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.280 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.280 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.281 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.281 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.281 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.281 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.281 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.281 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.281 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.281 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.281 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.281 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.281 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.281 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.281 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "restful", 2024-05-21T15:42:36.282 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.282 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.282 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.282 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_auth": { 2024-05-21T15:42:36.282 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_auth", 2024-05-21T15:42:36.282 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.282 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.282 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.282 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.282 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.282 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.282 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.282 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.282 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.283 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.283 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.283 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.283 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-05-21T15:42:36.283 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-05-21T15:42:36.283 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.283 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.283 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.283 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.283 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.283 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.283 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.283 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.284 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.284 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.284 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.284 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.284 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.284 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.284 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.284 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.284 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.284 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.284 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.284 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.284 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.285 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.285 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.285 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.285 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.285 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.285 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.285 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.285 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.285 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.285 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.285 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.285 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.285 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.285 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.286 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.286 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.286 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.286 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.286 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.286 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.286 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.286 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.286 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.286 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.286 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.286 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.286 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.286 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.287 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.287 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.287 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.287 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.287 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.287 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.287 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.287 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.287 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.287 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.287 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.287 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.287 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.288 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.288 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.288 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.288 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.288 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.288 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.288 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.288 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.288 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.288 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.288 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.288 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.288 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.288 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.289 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.289 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.289 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.289 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.289 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.289 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.289 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-05-21T15:42:36.289 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-05-21T15:42:36.289 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.289 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.289 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.289 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.289 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.290 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.290 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.290 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.290 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.290 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.290 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.290 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.290 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-05-21T15:42:36.290 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-05-21T15:42:36.290 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.290 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.291 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.291 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.291 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.291 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.291 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.291 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.291 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.291 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.291 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.291 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.291 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.291 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.291 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.292 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rgw", 2024-05-21T15:42:36.292 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.292 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.292 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.292 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.292 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.292 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.292 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.292 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.292 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.292 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.292 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.292 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.293 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.293 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.293 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.293 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.293 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.293 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.293 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.293 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.293 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.293 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.293 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.293 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.293 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.294 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.294 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.294 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.294 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.294 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.294 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.294 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.294 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.294 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.294 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.294 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.294 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.294 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.295 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.295 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.295 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.295 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.295 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.295 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.295 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.295 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.295 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.295 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.295 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.295 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.295 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.296 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.296 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.296 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.296 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.296 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.296 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.296 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.296 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.296 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.296 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.296 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.296 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.296 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.297 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.297 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.297 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.297 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.297 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.297 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.297 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.297 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.297 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.297 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.297 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.297 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.297 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rook", 2024-05-21T15:42:36.297 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.298 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.298 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.298 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.298 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.298 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.298 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.298 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.298 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.298 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.298 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.299 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.299 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.299 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.299 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.299 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.299 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.299 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.299 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.299 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.299 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.299 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.299 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.299 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.299 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.300 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.300 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.300 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.300 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.300 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.300 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.300 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.300 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.300 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.300 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.300 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.300 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.300 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.300 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.301 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.301 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.301 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.301 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.301 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.301 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.301 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.301 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.301 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.301 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.301 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.301 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.301 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.302 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.302 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.302 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.302 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.302 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.302 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.302 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.302 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.302 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.302 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.302 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.302 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.303 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.303 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.303 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.303 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.303 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.303 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.303 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.303 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.303 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.303 INFO:teuthology.orchestra.run.smithi038.stdout: "storage_class": { 2024-05-21T15:42:36.303 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "storage_class", 2024-05-21T15:42:36.303 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.303 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.303 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.304 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "local", 2024-05-21T15:42:36.304 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.304 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.304 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.304 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-21T15:42:36.304 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.304 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.304 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.304 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.304 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.304 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.304 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.304 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "selftest", 2024-05-21T15:42:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.305 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.306 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.306 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.306 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.306 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.306 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.306 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.306 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.306 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.306 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.306 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.306 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.306 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.306 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.307 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.307 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.307 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.307 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.307 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.307 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.307 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.307 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.307 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.307 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.307 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.307 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.307 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.307 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.308 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.308 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.308 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.308 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.308 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.308 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.308 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.308 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.308 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.308 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.308 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.308 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.308 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.309 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.309 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.309 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.309 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.309 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.309 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.309 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.309 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.309 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.309 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.309 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.309 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.309 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.309 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.310 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.310 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.310 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.310 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.310 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.310 INFO:teuthology.orchestra.run.smithi038.stdout: "roption1": { 2024-05-21T15:42:36.310 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption1", 2024-05-21T15:42:36.310 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.310 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.310 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.310 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.310 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.311 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.311 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.311 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.311 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.311 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.311 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.311 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.311 INFO:teuthology.orchestra.run.smithi038.stdout: "roption2": { 2024-05-21T15:42:36.311 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption2", 2024-05-21T15:42:36.311 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.311 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.311 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.311 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "xyz", 2024-05-21T15:42:36.311 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.312 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.312 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.312 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.312 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.312 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.312 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.312 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.312 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption1": { 2024-05-21T15:42:36.312 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption1", 2024-05-21T15:42:36.312 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.312 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.312 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.312 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.312 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.313 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.313 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.313 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.313 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.313 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.313 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.313 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.313 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption2": { 2024-05-21T15:42:36.313 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption2", 2024-05-21T15:42:36.313 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.313 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.313 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.314 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.314 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.314 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.314 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.314 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.314 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.314 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.314 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.314 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.314 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption3": { 2024-05-21T15:42:36.314 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption3", 2024-05-21T15:42:36.314 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-05-21T15:42:36.314 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.315 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.315 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.315 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.315 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.315 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.315 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.315 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.315 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.315 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.315 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.315 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption4": { 2024-05-21T15:42:36.315 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption4", 2024-05-21T15:42:36.315 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.316 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.316 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.316 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.316 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.316 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.316 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.316 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.316 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.316 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.316 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.316 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.316 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption5": { 2024-05-21T15:42:36.316 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption5", 2024-05-21T15:42:36.317 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.317 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.317 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.317 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.317 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.317 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.317 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.317 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.317 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.317 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.317 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.317 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.317 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption6": { 2024-05-21T15:42:36.317 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption6", 2024-05-21T15:42:36.318 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.318 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.318 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.318 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.318 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.318 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.318 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.318 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.318 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.318 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.318 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.318 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.318 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption7": { 2024-05-21T15:42:36.319 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption7", 2024-05-21T15:42:36.319 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.319 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.319 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.319 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.319 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-05-21T15:42:36.319 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "42", 2024-05-21T15:42:36.319 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.319 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.319 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.319 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.319 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.319 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.320 INFO:teuthology.orchestra.run.smithi038.stdout: "testkey": { 2024-05-21T15:42:36.320 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testkey", 2024-05-21T15:42:36.320 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.320 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.320 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.320 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.320 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.320 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.320 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.320 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.320 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.320 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.320 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.320 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.321 INFO:teuthology.orchestra.run.smithi038.stdout: "testlkey": { 2024-05-21T15:42:36.321 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testlkey", 2024-05-21T15:42:36.321 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.321 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.321 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.321 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.321 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.321 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.321 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.321 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.321 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.321 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.321 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.322 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.322 INFO:teuthology.orchestra.run.smithi038.stdout: "testnewline": { 2024-05-21T15:42:36.322 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testnewline", 2024-05-21T15:42:36.322 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.322 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.322 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.322 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.322 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.322 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.322 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.322 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.322 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.322 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.323 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.323 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.323 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.323 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.323 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.323 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snap_schedule", 2024-05-21T15:42:36.323 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.323 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.323 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.323 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_m_granularity": { 2024-05-21T15:42:36.323 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_m_granularity", 2024-05-21T15:42:36.323 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.323 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.324 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.324 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.324 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.324 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.324 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.324 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow minute scheduled snapshots", 2024-05-21T15:42:36.324 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.324 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.324 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.324 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.324 INFO:teuthology.orchestra.run.smithi038.stdout: "dump_on_update": { 2024-05-21T15:42:36.324 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dump_on_update", 2024-05-21T15:42:36.324 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.324 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.325 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.325 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.325 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.325 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.325 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.325 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "dump database to debug log on update", 2024-05-21T15:42:36.325 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.325 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.325 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.325 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.325 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.325 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.325 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.326 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.326 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.326 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.326 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.326 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.326 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.326 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.326 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.326 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.326 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.326 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.326 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.327 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.327 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.327 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.327 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.327 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.327 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.327 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.327 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.327 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.327 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.327 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.327 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.327 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.328 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.328 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.328 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.328 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.328 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.328 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.328 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.328 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.328 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.328 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.328 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.328 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.328 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.329 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.329 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.329 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.329 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.329 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.329 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.329 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.329 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.329 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.329 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.329 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.329 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.330 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.330 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.330 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.330 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.330 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.330 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.330 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.330 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.330 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.330 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.330 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.330 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.330 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.331 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.331 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.331 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.331 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.331 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.331 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.331 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.331 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stats", 2024-05-21T15:42:36.331 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.331 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.331 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.331 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.331 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.332 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.332 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.332 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.332 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.332 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.332 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.332 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.332 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.332 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.332 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.332 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.332 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.332 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.332 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.333 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.333 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.333 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.333 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.333 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.333 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.333 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.333 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.333 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.333 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.333 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.333 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.333 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.334 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.334 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.334 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.334 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.334 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.334 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.334 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.334 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.334 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.334 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.334 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.334 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.334 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.335 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.335 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.335 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.335 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.335 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.335 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.335 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.335 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.335 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.335 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.335 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.335 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.335 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.335 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.336 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.336 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.336 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.336 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.336 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.336 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.336 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.336 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.336 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.336 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.336 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.336 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.336 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.337 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.337 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.337 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.337 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.337 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "status", 2024-05-21T15:42:36.337 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.337 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.337 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.337 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.337 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.337 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.337 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.338 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.338 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.338 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.338 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.338 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.338 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.338 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.338 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.338 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.338 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.338 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.338 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.338 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.338 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.339 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.339 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.339 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.339 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.339 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.339 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.339 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.339 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.339 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.339 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.339 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.339 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.339 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.340 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.340 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.340 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.340 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.340 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.340 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.340 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.340 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.340 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.340 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.340 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.340 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.340 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.341 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.341 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.341 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.341 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.341 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.341 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.341 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.341 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.341 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.341 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.341 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.341 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.341 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.341 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.342 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.342 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.342 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.342 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.342 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.342 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.342 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.342 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.342 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.342 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.342 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.342 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.342 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.343 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.343 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.343 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telegraf", 2024-05-21T15:42:36.343 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.343 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.343 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.343 INFO:teuthology.orchestra.run.smithi038.stdout: "address": { 2024-05-21T15:42:36.343 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "address", 2024-05-21T15:42:36.343 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.343 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.343 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.343 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-21T15:42:36.343 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.343 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.344 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.344 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.344 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.344 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.344 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.344 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.344 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:36.344 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:36.344 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.344 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.344 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.344 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15", 2024-05-21T15:42:36.344 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.344 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.345 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.345 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.345 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.345 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.345 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.345 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.345 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.345 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.345 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.345 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.345 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.345 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.345 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.345 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.346 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.346 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.346 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.346 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.346 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.346 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.346 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.346 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.346 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.346 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.346 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.346 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.346 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.347 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.347 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.347 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.347 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.347 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.347 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.347 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.347 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.347 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.347 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.347 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.347 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.347 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.348 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.348 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.348 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.348 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.348 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.348 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.348 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.348 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.348 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.348 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.348 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.348 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.348 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.348 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.349 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.349 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.349 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.349 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.349 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.349 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.349 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.349 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.349 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.349 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.349 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.349 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.349 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.350 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.350 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.350 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.350 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.350 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.350 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.350 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.350 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.350 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.350 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.350 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.350 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.351 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telemetry", 2024-05-21T15:42:36.351 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.351 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.351 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.351 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_basic": { 2024-05-21T15:42:36.351 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_basic", 2024-05-21T15:42:36.351 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.351 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.351 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.351 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.351 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.351 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.351 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.352 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-21T15:42:36.352 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.352 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.352 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.352 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.352 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_crash": { 2024-05-21T15:42:36.352 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_crash", 2024-05-21T15:42:36.352 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.352 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.352 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.352 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.352 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.352 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.353 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.353 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-21T15:42:36.353 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.353 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.353 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.353 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.353 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_device": { 2024-05-21T15:42:36.353 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_device", 2024-05-21T15:42:36.353 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.353 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.353 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.353 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.353 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.354 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.354 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.354 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-21T15:42:36.354 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.354 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.354 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.354 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.354 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_ident": { 2024-05-21T15:42:36.354 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_ident", 2024-05-21T15:42:36.354 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.354 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.354 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.355 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.355 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.355 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.355 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.355 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-21T15:42:36.355 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.355 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.355 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.355 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.355 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_perf": { 2024-05-21T15:42:36.355 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_perf", 2024-05-21T15:42:36.355 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.355 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.355 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.356 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.356 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.356 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.356 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.356 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-21T15:42:36.356 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.356 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.356 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.356 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.356 INFO:teuthology.orchestra.run.smithi038.stdout: "contact": { 2024-05-21T15:42:36.356 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "contact", 2024-05-21T15:42:36.356 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.356 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.357 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.357 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.357 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.357 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.357 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.357 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.357 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.357 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.357 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.357 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.357 INFO:teuthology.orchestra.run.smithi038.stdout: "description": { 2024-05-21T15:42:36.357 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "description", 2024-05-21T15:42:36.357 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.357 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.358 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.358 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.358 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.358 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.358 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.358 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.358 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.358 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.358 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.358 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.358 INFO:teuthology.orchestra.run.smithi038.stdout: "device_url": { 2024-05-21T15:42:36.358 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_url", 2024-05-21T15:42:36.358 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.359 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.360 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.360 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.360 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.360 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.360 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.360 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.360 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.360 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.360 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.360 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.360 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:36.360 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:36.360 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.361 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.361 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.361 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "24", 2024-05-21T15:42:36.361 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "8", 2024-05-21T15:42:36.361 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.361 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.361 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.361 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.361 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.361 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.361 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.361 INFO:teuthology.orchestra.run.smithi038.stdout: "last_opt_revision": { 2024-05-21T15:42:36.361 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "last_opt_revision", 2024-05-21T15:42:36.361 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.362 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.362 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.362 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1", 2024-05-21T15:42:36.362 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.362 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.362 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.362 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.362 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.362 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.362 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.362 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.362 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard": { 2024-05-21T15:42:36.362 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard", 2024-05-21T15:42:36.362 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.363 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.363 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.363 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.363 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.363 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.363 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.363 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.363 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.363 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.363 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.363 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.363 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard_description": { 2024-05-21T15:42:36.363 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard_description", 2024-05-21T15:42:36.364 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.364 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.364 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.364 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.364 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.364 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.364 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.364 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.364 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.364 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.364 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.364 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.364 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.365 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.365 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.365 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.365 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.365 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.365 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.365 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.365 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.365 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.365 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.365 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.365 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.365 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.365 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.366 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.366 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.366 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.366 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.366 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.366 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.366 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.366 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.366 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.366 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.366 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.366 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.367 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.367 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.367 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.367 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.367 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.367 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.367 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.367 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.367 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.367 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.367 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.367 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.367 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.368 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.368 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.368 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.368 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.368 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.368 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.368 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.368 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.368 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.368 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.368 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.368 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.368 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.368 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.369 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.369 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.369 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.369 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.369 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.369 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.369 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.369 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.369 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.369 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.369 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.369 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.369 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.370 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.370 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.370 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.370 INFO:teuthology.orchestra.run.smithi038.stdout: "organization": { 2024-05-21T15:42:36.370 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "organization", 2024-05-21T15:42:36.370 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.370 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.370 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.370 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.370 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.370 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.370 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.370 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.371 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.371 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.371 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.371 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.371 INFO:teuthology.orchestra.run.smithi038.stdout: "proxy": { 2024-05-21T15:42:36.371 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "proxy", 2024-05-21T15:42:36.371 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.371 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.371 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.371 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.371 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.371 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.371 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.371 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.372 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.372 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.372 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.372 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.372 INFO:teuthology.orchestra.run.smithi038.stdout: "url": { 2024-05-21T15:42:36.372 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url", 2024-05-21T15:42:36.372 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.372 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.372 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.372 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-21T15:42:36.372 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.372 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.372 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.373 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.373 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.373 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.373 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.373 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.373 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.373 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.373 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.373 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "test_orchestrator", 2024-05-21T15:42:36.373 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.373 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.373 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.373 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.373 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.374 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.374 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.374 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.374 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.374 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.374 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.374 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.374 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.374 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.374 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.374 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.374 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.374 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.374 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.375 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.375 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.375 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.375 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.375 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.375 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.375 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.375 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.375 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.375 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.375 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.375 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.375 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.376 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.377 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.377 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.377 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.377 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.377 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.377 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.377 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.377 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.377 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.377 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.377 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.377 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.377 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.378 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.378 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.378 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.378 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.378 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.378 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.378 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.378 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.378 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.378 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.378 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.378 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.378 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.379 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.379 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.379 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.379 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-05-21T15:42:36.379 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.379 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.379 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.379 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.379 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.379 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.379 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.379 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.379 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.379 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.380 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.380 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.380 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.380 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.380 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.380 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.380 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.380 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.380 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.380 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.380 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.380 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.380 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.380 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.381 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.381 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.381 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.381 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.381 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.381 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.381 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.381 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.381 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.381 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.381 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.381 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.381 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.381 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.382 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.382 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.382 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.382 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.382 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.382 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.382 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.382 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.382 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.382 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.382 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.382 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.382 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.383 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.383 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.383 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.383 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.383 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.383 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.383 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.383 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.383 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.383 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.383 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.383 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.383 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.383 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.384 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.384 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.384 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.384 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.384 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.384 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.384 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.384 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.384 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_clones": { 2024-05-21T15:42:36.384 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_clones", 2024-05-21T15:42:36.384 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.384 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.384 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.384 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4", 2024-05-21T15:42:36.385 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.385 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.385 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.385 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-21T15:42:36.385 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.385 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.385 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.385 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.385 INFO:teuthology.orchestra.run.smithi038.stdout: "periodic_async_work": { 2024-05-21T15:42:36.385 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "periodic_async_work", 2024-05-21T15:42:36.385 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.385 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.385 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.385 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.386 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.386 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.386 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.386 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Periodically check for async work", 2024-05-21T15:42:36.386 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.386 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.386 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.386 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.386 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_delay": { 2024-05-21T15:42:36.386 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_delay", 2024-05-21T15:42:36.386 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.386 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.387 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.387 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-05-21T15:42:36.387 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.387 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.387 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.387 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-21T15:42:36.387 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.387 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.387 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.387 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.387 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_no_wait": { 2024-05-21T15:42:36.387 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_no_wait", 2024-05-21T15:42:36.387 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.387 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.388 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.388 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-05-21T15:42:36.388 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.388 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.388 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.388 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-21T15:42:36.388 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.388 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.388 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.388 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.388 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.388 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.388 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.389 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix", 2024-05-21T15:42:36.389 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-05-21T15:42:36.389 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-05-21T15:42:36.389 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-05-21T15:42:36.389 INFO:teuthology.orchestra.run.smithi038.stdout: "discovery_interval": { 2024-05-21T15:42:36.389 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "discovery_interval", 2024-05-21T15:42:36.389 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-05-21T15:42:36.389 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.389 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.389 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "100", 2024-05-21T15:42:36.389 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.389 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.389 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.390 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.390 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.390 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.390 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.390 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.390 INFO:teuthology.orchestra.run.smithi038.stdout: "identifier": { 2024-05-21T15:42:36.390 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "identifier", 2024-05-21T15:42:36.390 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.390 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.390 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.390 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.390 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.390 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.390 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.391 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.391 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.391 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.391 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.391 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.391 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-05-21T15:42:36.391 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-05-21T15:42:36.391 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-05-21T15:42:36.391 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.391 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.391 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-05-21T15:42:36.391 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.391 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.392 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.392 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.392 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.392 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.392 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.392 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.392 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-05-21T15:42:36.392 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-05-21T15:42:36.392 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.392 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.392 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.392 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.392 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.393 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.393 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.393 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.393 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.393 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.393 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.393 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.393 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.393 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.393 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.393 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.393 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.393 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.393 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.394 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-05-21T15:42:36.394 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-05-21T15:42:36.394 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.394 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.394 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.394 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.394 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.394 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.394 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.394 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.394 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.394 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.394 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.395 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.395 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-05-21T15:42:36.395 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-05-21T15:42:36.395 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.395 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.395 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.395 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-05-21T15:42:36.395 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.395 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.395 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-05-21T15:42:36.395 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-05-21T15:42:36.395 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-05-21T15:42:36.395 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-05-21T15:42:36.396 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-05-21T15:42:36.396 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-05-21T15:42:36.396 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-05-21T15:42:36.396 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.396 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.396 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.396 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.396 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.396 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.396 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-05-21T15:42:36.396 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-05-21T15:42:36.396 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-05-21T15:42:36.396 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_host": { 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_host", 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.397 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.398 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-05-21T15:42:36.398 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.398 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.398 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.398 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.398 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.398 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.398 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.398 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.398 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_port": { 2024-05-21T15:42:36.398 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_port", 2024-05-21T15:42:36.398 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-05-21T15:42:36.398 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10051", 2024-05-21T15:42:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_sender": { 2024-05-21T15:42:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_sender", 2024-05-21T15:42:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-05-21T15:42:36.399 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-05-21T15:42:36.400 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-05-21T15:42:36.400 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-21T15:42:36.400 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-05-21T15:42:36.400 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-05-21T15:42:36.400 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-05-21T15:42:36.400 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-05-21T15:42:36.400 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-05-21T15:42:36.400 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-05-21T15:42:36.400 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-05-21T15:42:36.400 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.400 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.400 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.400 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.401 INFO:teuthology.orchestra.run.smithi038.stdout: "services": { 2024-05-21T15:42:36.401 INFO:teuthology.orchestra.run.smithi038.stdout: "dashboard": "https://172.21.15.38:8443/" 2024-05-21T15:42:36.401 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.401 INFO:teuthology.orchestra.run.smithi038.stdout: "always_on_modules": { 2024-05-21T15:42:36.401 INFO:teuthology.orchestra.run.smithi038.stdout: "octopus": [ 2024-05-21T15:42:36.401 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-05-21T15:42:36.401 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-05-21T15:42:36.401 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-05-21T15:42:36.401 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-05-21T15:42:36.401 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-05-21T15:42:36.401 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-05-21T15:42:36.401 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-05-21T15:42:36.401 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-05-21T15:42:36.401 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-05-21T15:42:36.402 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-05-21T15:42:36.402 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.402 INFO:teuthology.orchestra.run.smithi038.stdout: "pacific": [ 2024-05-21T15:42:36.402 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-05-21T15:42:36.402 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-05-21T15:42:36.402 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-05-21T15:42:36.402 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-05-21T15:42:36.402 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-05-21T15:42:36.402 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-05-21T15:42:36.402 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-05-21T15:42:36.402 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-05-21T15:42:36.402 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-05-21T15:42:36.402 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-05-21T15:42:36.402 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.403 INFO:teuthology.orchestra.run.smithi038.stdout: "quincy": [ 2024-05-21T15:42:36.403 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-05-21T15:42:36.403 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-05-21T15:42:36.403 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-05-21T15:42:36.403 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-05-21T15:42:36.403 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-05-21T15:42:36.403 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-05-21T15:42:36.403 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-05-21T15:42:36.403 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-05-21T15:42:36.403 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-05-21T15:42:36.403 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-05-21T15:42:36.403 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.403 INFO:teuthology.orchestra.run.smithi038.stdout: "reef": [ 2024-05-21T15:42:36.403 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-05-21T15:42:36.404 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-05-21T15:42:36.404 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-05-21T15:42:36.404 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-05-21T15:42:36.404 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-05-21T15:42:36.404 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-05-21T15:42:36.404 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-05-21T15:42:36.404 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-05-21T15:42:36.404 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-05-21T15:42:36.404 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-05-21T15:42:36.404 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-05-21T15:42:36.404 INFO:teuthology.orchestra.run.smithi038.stdout: "squid": [ 2024-05-21T15:42:36.404 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-05-21T15:42:36.404 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-05-21T15:42:36.404 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-05-21T15:42:36.405 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-05-21T15:42:36.405 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-05-21T15:42:36.405 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-05-21T15:42:36.405 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-05-21T15:42:36.405 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-05-21T15:42:36.405 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-05-21T15:42:36.405 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-05-21T15:42:36.405 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-05-21T15:42:36.405 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.405 INFO:teuthology.orchestra.run.smithi038.stdout: "last_failure_osd_epoch": 3, 2024-05-21T15:42:36.405 INFO:teuthology.orchestra.run.smithi038.stdout: "active_clients": [ 2024-05-21T15:42:36.405 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.405 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-05-21T15:42:36.405 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-05-21T15:42:36.406 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.406 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-05-21T15:42:36.406 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-05-21T15:42:36.406 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 4233783744 2024-05-21T15:42:36.406 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.406 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-05-21T15:42:36.406 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.406 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.406 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "libcephsqlite", 2024-05-21T15:42:36.406 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-05-21T15:42:36.406 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.406 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-05-21T15:42:36.406 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-05-21T15:42:36.406 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 2951450801 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 3399447322 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.407 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-05-21T15:42:36.408 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-05-21T15:42:36.408 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-05-21T15:42:36.408 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-05-21T15:42:36.408 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-05-21T15:42:36.408 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 4215114230 2024-05-21T15:42:36.408 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.408 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-05-21T15:42:36.408 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:42:36.408 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-05-21T15:42:36.408 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:42:36.410 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:34 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/475560084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:42:36.410 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:35 smithi045 ceph-mon[36409]: pgmap v596: 97 pgs: 97 active+clean; 4.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 139 KiB/s wr, 214 op/s 2024-05-21T15:42:36.410 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:35 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1967675948' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-21T15:42:36.410 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:34 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/475560084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:42:36.411 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:35 smithi053 ceph-mon[36496]: pgmap v596: 97 pgs: 97 active+clean; 4.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 139 KiB/s wr, 214 op/s 2024-05-21T15:42:36.411 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:35 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1967675948' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-21T15:42:36.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:35.039+0000 7f9056b8b640 1 -- 172.21.15.38:0/1967675948 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f90400a2e70 msgr2=0x7f90400a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:36.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:35.039+0000 7f9056b8b640 1 --2- 172.21.15.38:0/1967675948 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f90400a2e70 0x7f90400a5380 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9050108c10 tx=0x7f9044005b80 comp rx=0 tx=0).stop 2024-05-21T15:42:36.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:35.039+0000 7f9056b8b640 1 -- 172.21.15.38:0/1967675948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f905010a300 msgr2=0x7f9050118400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:42:36.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:35.039+0000 7f9056b8b640 1 --2- 172.21.15.38:0/1967675948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f905010a300 0x7f9050118400 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f903c02a9f0 tx=0x7f903c032c10 comp rx=0 tx=0).stop 2024-05-21T15:42:36.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:35.039+0000 7f9056b8b640 1 -- 172.21.15.38:0/1967675948 shutdown_connections 2024-05-21T15:42:36.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:35.039+0000 7f9056b8b640 1 --2- 172.21.15.38:0/1967675948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f905010f020 0x7f9050119360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:36.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:35.039+0000 7f9056b8b640 1 --2- 172.21.15.38:0/1967675948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f905010a300 0x7f9050118400 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:36.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:35.039+0000 7f9056b8b640 1 --2- 172.21.15.38:0/1967675948 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f90400a2e70 0x7f90400a5380 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:36.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:35.039+0000 7f9056b8b640 1 --2- 172.21.15.38:0/1967675948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90501093d0 0x7f9050117ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:42:36.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:35.039+0000 7f9056b8b640 1 -- 172.21.15.38:0/1967675948 >> 172.21.15.38:0/1967675948 conn(0x7f905006f4e0 msgr2=0x7f90500718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:42:36.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:35.040+0000 7f9056b8b640 1 -- 172.21.15.38:0/1967675948 shutdown_connections 2024-05-21T15:42:36.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:42:35.040+0000 7f9056b8b640 1 -- 172.21.15.38:0/1967675948 wait complete. 2024-05-21T15:42:36.417 DEBUG:tasks.check_counter:Getting stats from a 2024-05-21T15:42:36.418 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-21T15:42:36.418 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:34 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/475560084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:42:36.418 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:35 smithi038 ceph-mon[31995]: pgmap v596: 97 pgs: 97 active+clean; 4.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 139 KiB/s wr, 214 op/s 2024-05-21T15:42:36.418 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:35 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1967675948' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-21T15:42:36.832 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:42:38.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:38 smithi053 ceph-mon[36496]: pgmap v597: 97 pgs: 97 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 KiB/s wr, 204 op/s 2024-05-21T15:42:38.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:38 smithi045 ceph-mon[36409]: pgmap v597: 97 pgs: 97 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 KiB/s wr, 204 op/s 2024-05-21T15:42:38.522 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:38 smithi038 ceph-mon[31995]: pgmap v597: 97 pgs: 97 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 KiB/s wr, 204 op/s 2024-05-21T15:42:40.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:40 smithi053 ceph-mon[36496]: pgmap v598: 97 pgs: 97 active+clean; 750 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 KiB/s wr, 266 op/s 2024-05-21T15:42:40.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:40 smithi045 ceph-mon[36409]: pgmap v598: 97 pgs: 97 active+clean; 750 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 KiB/s wr, 266 op/s 2024-05-21T15:42:40.522 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:40 smithi038 ceph-mon[31995]: pgmap v598: 97 pgs: 97 active+clean; 750 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 KiB/s wr, 266 op/s 2024-05-21T15:42:42.453 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:42 smithi053 ceph-mon[36496]: pgmap v599: 97 pgs: 97 active+clean; 213 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 KiB/s wr, 216 op/s 2024-05-21T15:42:42.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:42 smithi045 ceph-mon[36409]: pgmap v599: 97 pgs: 97 active+clean; 213 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 KiB/s wr, 216 op/s 2024-05-21T15:42:42.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:42 smithi038 ceph-mon[31995]: pgmap v599: 97 pgs: 97 active+clean; 213 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 KiB/s wr, 216 op/s 2024-05-21T15:42:44.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:44 smithi053 ceph-mon[36496]: pgmap v600: 97 pgs: 97 active+clean; 33 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.0 KiB/s wr, 207 op/s 2024-05-21T15:42:44.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:44 smithi045 ceph-mon[36409]: pgmap v600: 97 pgs: 97 active+clean; 33 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.0 KiB/s wr, 207 op/s 2024-05-21T15:42:44.523 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:44 smithi038 ceph-mon[31995]: pgmap v600: 97 pgs: 97 active+clean; 33 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.0 KiB/s wr, 207 op/s 2024-05-21T15:42:45.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:45 smithi053 ceph-mon[36496]: pgmap v601: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 853 B/s wr, 150 op/s 2024-05-21T15:42:45.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:45 smithi045 ceph-mon[36409]: pgmap v601: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 853 B/s wr, 150 op/s 2024-05-21T15:42:45.522 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:45 smithi038 ceph-mon[31995]: pgmap v601: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 853 B/s wr, 150 op/s 2024-05-21T15:42:47.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:47 smithi045 ceph-mon[36409]: pgmap v602: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 117 op/s 2024-05-21T15:42:47.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:47 smithi038 ceph-mon[31995]: pgmap v602: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 117 op/s 2024-05-21T15:42:47.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:47 smithi053 ceph-mon[36496]: pgmap v602: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 117 op/s 2024-05-21T15:42:49.758 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:49 smithi045 ceph-mon[36409]: pgmap v603: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 107 op/s 2024-05-21T15:42:49.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:49 smithi038 ceph-mon[31995]: pgmap v603: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 107 op/s 2024-05-21T15:42:49.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:49 smithi053 ceph-mon[36496]: pgmap v603: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 107 op/s 2024-05-21T15:42:51.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:51 smithi045 ceph-mon[36409]: pgmap v604: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 37 op/s 2024-05-21T15:42:51.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:51 smithi038 ceph-mon[31995]: pgmap v604: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 37 op/s 2024-05-21T15:42:51.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:51 smithi053 ceph-mon[36496]: pgmap v604: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 37 op/s 2024-05-21T15:42:53.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:53 smithi045 ceph-mon[36409]: pgmap v605: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 20 op/s 2024-05-21T15:42:53.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:53 smithi038 ceph-mon[31995]: pgmap v605: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 20 op/s 2024-05-21T15:42:53.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:53 smithi053 ceph-mon[36496]: pgmap v605: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 20 op/s 2024-05-21T15:42:55.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:55 smithi045 ceph-mon[36409]: pgmap v606: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-21T15:42:55.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:55 smithi038 ceph-mon[31995]: pgmap v606: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-21T15:42:55.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:55 smithi053 ceph-mon[36496]: pgmap v606: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-21T15:42:57.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:57 smithi045 ceph-mon[36409]: pgmap v607: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:42:57.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:57 smithi038 ceph-mon[31995]: pgmap v607: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:42:57.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:57 smithi053 ceph-mon[36496]: pgmap v607: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:42:59.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:42:59 smithi038 ceph-mon[31995]: pgmap v608: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:42:59.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:42:59 smithi053 ceph-mon[36496]: pgmap v608: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:43:00.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:42:59 smithi045 ceph-mon[36409]: pgmap v608: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:43:00.812 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:43:00.812 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:43:00.812 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1021, 2024-05-21T15:43:00.812 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1023, 2024-05-21T15:43:00.812 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 4509371, 2024-05-21T15:43:00.812 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 324679, 2024-05-21T15:43:00.812 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 6, 2024-05-21T15:43:00.813 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-05-21T15:43:00.813 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.109470859, 2024-05-21T15:43:00.813 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.029968317, 2024-05-21T15:43:00.813 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.034263002, 2024-05-21T15:43:00.813 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.035890208, 2024-05-21T15:43:00.813 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:00.813 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1021, 2024-05-21T15:43:00.813 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.060472829, 2024-05-21T15:43:00.813 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000059229 2024-05-21T15:43:00.813 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.813 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:00.813 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.814 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.814 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.814 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.814 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T15:43:00.814 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T15:43:00.814 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.814 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:43:00.814 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 816, 2024-05-21T15:43:00.814 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 797, 2024-05-21T15:43:00.814 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 1563751, 2024-05-21T15:43:00.814 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 302472, 2024-05-21T15:43:00.814 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-05-21T15:43:00.814 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 3, 2024-05-21T15:43:00.815 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.086771082, 2024-05-21T15:43:00.815 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.025421723, 2024-05-21T15:43:00.815 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.032075222, 2024-05-21T15:43:00.815 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.026266986, 2024-05-21T15:43:00.815 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:00.815 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 797, 2024-05-21T15:43:00.815 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.044770998, 2024-05-21T15:43:00.815 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000056174 2024-05-21T15:43:00.815 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.815 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:00.815 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.815 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.815 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.816 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.816 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 580368, 2024-05-21T15:43:00.816 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 107152 2024-05-21T15:43:00.816 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.816 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:43:00.816 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 255, 2024-05-21T15:43:00.816 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 384, 2024-05-21T15:43:00.816 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 62666, 2024-05-21T15:43:00.816 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 390964, 2024-05-21T15:43:00.816 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 5, 2024-05-21T15:43:00.816 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-05-21T15:43:00.816 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.036771983, 2024-05-21T15:43:00.816 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.015588699, 2024-05-21T15:43:00.817 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.007741784, 2024-05-21T15:43:00.817 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.010591482, 2024-05-21T15:43:00.817 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:00.817 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 382, 2024-05-21T15:43:00.817 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.084101798, 2024-05-21T15:43:00.817 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000220161 2024-05-21T15:43:00.817 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.817 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:00.817 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.817 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.817 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.817 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.817 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-21T15:43:00.818 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 310192 2024-05-21T15:43:00.818 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.818 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-05-21T15:43:00.818 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-05-21T15:43:00.818 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-05-21T15:43:00.818 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.818 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-05-21T15:43:00.818 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-05-21T15:43:00.818 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-05-21T15:43:00.818 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3170, 2024-05-21T15:43:00.818 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.091150251, 2024-05-21T15:43:00.818 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000028754 2024-05-21T15:43:00.819 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:00.819 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.819 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-05-21T15:43:00.819 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-05-21T15:43:00.819 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-05-21T15:43:00.819 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.819 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.819 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.819 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:00.819 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.819 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-05-21T15:43:00.819 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 0, 2024-05-21T15:43:00.819 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 0, 2024-05-21T15:43:00.820 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-05-21T15:43:00.820 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.820 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.820 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.820 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.820 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-05-21T15:43:00.820 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-05-21T15:43:00.820 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-05-21T15:43:00.820 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-05-21T15:43:00.820 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 0, 2024-05-21T15:43:00.821 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-05-21T15:43:00.821 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-05-21T15:43:00.821 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 32, 2024-05-21T15:43:00.821 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 0, 2024-05-21T15:43:00.821 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 32, 2024-05-21T15:43:00.821 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-05-21T15:43:00.821 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 32, 2024-05-21T15:43:00.821 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-05-21T15:43:00.821 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 0, 2024-05-21T15:43:00.821 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 0, 2024-05-21T15:43:00.821 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 1, 2024-05-21T15:43:00.821 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 0, 2024-05-21T15:43:00.821 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 0, 2024-05-21T15:43:00.822 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-05-21T15:43:00.822 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-05-21T15:43:00.822 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:43:00.822 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-05-21T15:43:00.822 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 0, 2024-05-21T15:43:00.822 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-05-21T15:43:00.822 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-05-21T15:43:00.822 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-05-21T15:43:00.822 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-05-21T15:43:00.822 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-05-21T15:43:00.822 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-05-21T15:43:00.822 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-05-21T15:43:00.822 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 0, 2024-05-21T15:43:00.823 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-05-21T15:43:00.823 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-05-21T15:43:00.823 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-05-21T15:43:00.823 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-05-21T15:43:00.823 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T15:43:00.823 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-05-21T15:43:00.823 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:43:00.823 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:43:00.823 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:43:00.823 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:43:00.823 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:43:00.823 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 0, 2024-05-21T15:43:00.823 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-05-21T15:43:00.823 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T15:43:00.824 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 0, 2024-05-21T15:43:00.824 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:43:00.824 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:43:00.824 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:43:00.824 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 0, 2024-05-21T15:43:00.824 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 0, 2024-05-21T15:43:00.824 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 0, 2024-05-21T15:43:00.824 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 0 2024-05-21T15:43:00.824 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.824 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-05-21T15:43:00.824 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-05-21T15:43:00.824 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-05-21T15:43:00.824 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-05-21T15:43:00.825 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-05-21T15:43:00.825 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-05-21T15:43:00.825 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-05-21T15:43:00.825 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-05-21T15:43:00.825 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:43:00.825 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 0, 2024-05-21T15:43:00.825 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 0, 2024-05-21T15:43:00.825 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-05-21T15:43:00.825 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-05-21T15:43:00.825 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-05-21T15:43:00.825 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:43:00.825 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:43:00.825 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-05-21T15:43:00.825 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-05-21T15:43:00.826 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:43:00.826 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:43:00.826 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:43:00.826 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-05-21T15:43:00.826 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-05-21T15:43:00.826 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:43:00.826 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-05-21T15:43:00.826 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-05-21T15:43:00.826 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.826 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-05-21T15:43:00.826 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-05-21T15:43:00.826 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 0, 2024-05-21T15:43:00.826 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-05-21T15:43:00.827 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-05-21T15:43:00.827 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 4162, 2024-05-21T15:43:00.827 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-05-21T15:43:00.827 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-05-21T15:43:00.827 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 0, 2024-05-21T15:43:00.827 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-05-21T15:43:00.827 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-05-21T15:43:00.827 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 6, 2024-05-21T15:43:00.827 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 2, 2024-05-21T15:43:00.827 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-05-21T15:43:00.827 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-05-21T15:43:00.827 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-05-21T15:43:00.827 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 9216231, 2024-05-21T15:43:00.828 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 9214580, 2024-05-21T15:43:00.828 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-05-21T15:43:00.828 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.828 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.828 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.828 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.828 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 4162 2024-05-21T15:43:00.828 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.828 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-05-21T15:43:00.828 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 4, 2024-05-21T15:43:00.828 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 45, 2024-05-21T15:43:00.828 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 41, 2024-05-21T15:43:00.828 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 2, 2024-05-21T15:43:00.828 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 14, 2024-05-21T15:43:00.829 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 12, 2024-05-21T15:43:00.829 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 32, 2024-05-21T15:43:00.829 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 73, 2024-05-21T15:43:00.829 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 41, 2024-05-21T15:43:00.829 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 0, 2024-05-21T15:43:00.829 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 0, 2024-05-21T15:43:00.829 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 0, 2024-05-21T15:43:00.829 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 121804, 2024-05-21T15:43:00.829 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 215444 2024-05-21T15:43:00.829 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.829 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-05-21T15:43:00.829 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 0, 2024-05-21T15:43:00.829 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-05-21T15:43:00.829 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 0, 2024-05-21T15:43:00.830 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 0, 2024-05-21T15:43:00.830 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-05-21T15:43:00.830 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-05-21T15:43:00.830 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.830 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.830 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.830 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.830 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-05-21T15:43:00.830 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.830 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.830 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.830 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.830 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-05-21T15:43:00.831 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.831 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.831 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.831 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-05-21T15:43:00.831 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.831 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.831 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.832 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-05-21T15:43:00.832 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.832 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.832 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.832 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.832 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-05-21T15:43:00.832 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.832 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.832 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.832 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.832 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-05-21T15:43:00.832 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.832 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.833 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.833 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.833 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-05-21T15:43:00.833 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.833 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.833 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.833 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.833 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-05-21T15:43:00.833 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.833 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.833 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.833 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.833 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-05-21T15:43:00.833 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.834 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.834 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.834 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.834 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-05-21T15:43:00.834 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.834 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.834 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.834 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.834 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-05-21T15:43:00.834 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.838 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.838 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.838 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.838 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-05-21T15:43:00.838 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.838 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.838 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.838 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.838 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-05-21T15:43:00.838 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-05-21T15:43:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-05-21T15:43:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-05-21T15:43:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-05-21T15:43:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-05-21T15:43:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-05-21T15:43:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-05-21T15:43:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-05-21T15:43:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-05-21T15:43:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-05-21T15:43:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-05-21T15:43:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-05-21T15:43:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-05-21T15:43:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-05-21T15:43:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-05-21T15:43:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:43:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:43:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-05-21T15:43:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-05-21T15:43:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-05-21T15:43:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 2, 2024-05-21T15:43:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 1, 2024-05-21T15:43:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-05-21T15:43:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-05-21T15:43:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 0, 2024-05-21T15:43:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 0, 2024-05-21T15:43:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 1604, 2024-05-21T15:43:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-05-21T15:43:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-05-21T15:43:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:43:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-05-21T15:43:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:43:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:43:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:43:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:43:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:43:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:43:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:43:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:43:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:43:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:43:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:43:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:43:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:43:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-05-21T15:43:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:43:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-05-21T15:43:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:43:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:43:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:43:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:43:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:43:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:43:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:43:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-05-21T15:43:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:43:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:43:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:43:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-05-21T15:43:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-05-21T15:43:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-05-21T15:43:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:43:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:43:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:43:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:43:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 198247, 2024-05-21T15:43:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 56, 2024-05-21T15:43:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:43:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-05-21T15:43:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-05-21T15:43:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-05-21T15:43:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:43:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-05-21T15:43:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:43:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-05-21T15:43:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 7824, 2024-05-21T15:43:00.850 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 165, 2024-05-21T15:43:00.850 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:43:00.850 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:43:00.850 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-05-21T15:43:00.850 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-05-21T15:43:00.850 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 34512, 2024-05-21T15:43:00.850 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 156, 2024-05-21T15:43:00.850 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-05-21T15:43:00.850 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-05-21T15:43:00.850 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-05-21T15:43:00.850 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-05-21T15:43:00.850 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.850 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-05-21T15:43:00.851 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-05-21T15:43:00.851 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-05-21T15:43:00.851 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 1889, 2024-05-21T15:43:00.851 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 21, 2024-05-21T15:43:00.851 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 3, 2024-05-21T15:43:00.851 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 1889, 2024-05-21T15:43:00.851 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-05-21T15:43:00.851 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1889, 2024-05-21T15:43:00.851 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 8.252857692, 2024-05-21T15:43:00.851 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.004368902 2024-05-21T15:43:00.852 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.852 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-05-21T15:43:00.852 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-05-21T15:43:00.852 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1889, 2024-05-21T15:43:00.852 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1890 2024-05-21T15:43:00.852 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.852 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 1889, 2024-05-21T15:43:00.852 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 1889, 2024-05-21T15:43:00.852 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 0, 2024-05-21T15:43:00.852 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-05-21T15:43:00.852 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-05-21T15:43:00.852 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 1152, 2024-05-21T15:43:00.852 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 0, 2024-05-21T15:43:00.852 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 735, 2024-05-21T15:43:00.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 0, 2024-05-21T15:43:00.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 0, 2024-05-21T15:43:00.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-05-21T15:43:00.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-05-21T15:43:00.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-05-21T15:43:00.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-05-21T15:43:00.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 0, 2024-05-21T15:43:00.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-05-21T15:43:00.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-05-21T15:43:00.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-05-21T15:43:00.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 1, 2024-05-21T15:43:00.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 0, 2024-05-21T15:43:00.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:43:00.854 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-05-21T15:43:00.854 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:43:00.854 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-05-21T15:43:00.854 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-05-21T15:43:00.854 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-05-21T15:43:00.854 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:43:00.854 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-05-21T15:43:00.854 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:43:00.854 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-05-21T15:43:00.854 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-05-21T15:43:00.854 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-05-21T15:43:00.854 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-05-21T15:43:00.854 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-05-21T15:43:00.854 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-05-21T15:43:00.855 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-05-21T15:43:00.855 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-05-21T15:43:00.855 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-05-21T15:43:00.855 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-05-21T15:43:00.855 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-05-21T15:43:00.855 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-05-21T15:43:00.855 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-05-21T15:43:00.855 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-05-21T15:43:00.855 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-05-21T15:43:00.855 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-05-21T15:43:00.855 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-05-21T15:43:00.855 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 90, 2024-05-21T15:43:00.855 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-05-21T15:43:00.855 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 4, 2024-05-21T15:43:00.856 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 6, 2024-05-21T15:43:00.856 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 6, 2024-05-21T15:43:00.856 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-05-21T15:43:00.856 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-05-21T15:43:00.856 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 0, 2024-05-21T15:43:00.856 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 2, 2024-05-21T15:43:00.856 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 0 2024-05-21T15:43:00.856 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.856 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-05-21T15:43:00.856 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 0, 2024-05-21T15:43:00.856 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:43:00.856 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-05-21T15:43:00.857 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-05-21T15:43:00.857 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.857 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-05-21T15:43:00.857 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-05-21T15:43:00.857 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.857 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-05-21T15:43:00.857 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-05-21T15:43:00.857 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T15:43:00.857 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-05-21T15:43:00.857 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 0, 2024-05-21T15:43:00.857 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 0, 2024-05-21T15:43:00.857 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 0, 2024-05-21T15:43:00.858 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-05-21T15:43:00.858 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.858 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:43:00.858 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-05-21T15:43:00.858 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-05-21T15:43:00.858 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-05-21T15:43:00.858 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 2088, 2024-05-21T15:43:00.858 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 5934798, 2024-05-21T15:43:00.858 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:00.858 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 2088, 2024-05-21T15:43:00.858 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-05-21T15:43:00.858 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-05-21T15:43:00.858 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 2088, 2024-05-21T15:43:00.859 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 5934798, 2024-05-21T15:43:00.859 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-05-21T15:43:00.859 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.859 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.859 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.859 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:00.859 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.859 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-05-21T15:43:00.859 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-05-21T15:43:00.859 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-05-21T15:43:00.859 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-05-21T15:43:00.859 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-05-21T15:43:00.859 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-05-21T15:43:00.860 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:00.860 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:00.860 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 1889, 2024-05-21T15:43:00.860 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 5024974, 2024-05-21T15:43:00.860 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 1889, 2024-05-21T15:43:00.860 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 5024974, 2024-05-21T15:43:00.860 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-05-21T15:43:00.860 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.860 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.860 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.860 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:00.860 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.860 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-05-21T15:43:00.861 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-05-21T15:43:00.861 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-05-21T15:43:00.861 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-05-21T15:43:00.861 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-05-21T15:43:00.861 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-05-21T15:43:00.861 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:00.861 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:00.861 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 1889, 2024-05-21T15:43:00.861 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 1889, 2024-05-21T15:43:00.861 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 1889, 2024-05-21T15:43:00.861 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1889, 2024-05-21T15:43:00.861 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-05-21T15:43:00.861 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.861 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.862 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.862 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:00.862 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.862 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:43:00.862 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-05-21T15:43:00.862 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-05-21T15:43:00.862 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-05-21T15:43:00.862 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-05-21T15:43:00.862 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-05-21T15:43:00.862 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:00.862 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:00.862 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-05-21T15:43:00.862 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-05-21T15:43:00.863 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-05-21T15:43:00.863 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-05-21T15:43:00.863 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-05-21T15:43:00.863 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.863 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.863 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.863 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:00.863 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:00.863 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x55c8ee2a4a90": { 2024-05-21T15:43:00.863 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-05-21T15:43:00.863 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-05-21T15:43:00.863 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-05-21T15:43:00.863 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-05-21T15:43:00.863 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-05-21T15:43:00.864 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:00.864 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:00.864 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-05-21T15:43:00.864 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-05-21T15:43:00.864 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-05-21T15:43:00.864 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-05-21T15:43:00.864 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-05-21T15:43:00.864 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:00.864 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:00.864 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:00.864 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:00.864 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:00.864 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:43:01.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:01 smithi038 ceph-mon[31995]: pgmap v609: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:01.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:01 smithi053 ceph-mon[36496]: pgmap v609: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:02.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:01 smithi045 ceph-mon[36409]: pgmap v609: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:03.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:03 smithi053 ceph-mon[36496]: pgmap v610: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:04.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:03 smithi045 ceph-mon[36409]: pgmap v610: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:04.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:03 smithi038 ceph-mon[31995]: pgmap v610: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:04.693 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-21T15:43:04.693 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-21T15:43:04.694 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-05-21T15:43:04.694 DEBUG:tasks.check_counter:Getting stats from d 2024-05-21T15:43:04.695 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-21T15:43:04.955 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:43:05.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:05 smithi038 ceph-mon[31995]: pgmap v611: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:05.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:05 smithi053 ceph-mon[36496]: pgmap v611: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:06.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:05 smithi045 ceph-mon[36409]: pgmap v611: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:06.791 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:43:06.791 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:43:06.791 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 186, 2024-05-21T15:43:06.792 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 181, 2024-05-21T15:43:06.792 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 470480, 2024-05-21T15:43:06.792 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 110096, 2024-05-21T15:43:06.792 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-05-21T15:43:06.792 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 0, 2024-05-21T15:43:06.792 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.029913306, 2024-05-21T15:43:06.792 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.010383713, 2024-05-21T15:43:06.792 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.016011937, 2024-05-21T15:43:06.792 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.004031837, 2024-05-21T15:43:06.792 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:06.793 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 181, 2024-05-21T15:43:06.793 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.009442570, 2024-05-21T15:43:06.793 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000052168 2024-05-21T15:43:06.793 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:06.793 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:06.793 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:06.793 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:06.793 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:06.793 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:06.793 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 470480, 2024-05-21T15:43:06.793 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 110096 2024-05-21T15:43:06.793 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:06.794 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:43:06.794 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1, 2024-05-21T15:43:06.794 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 136, 2024-05-21T15:43:06.794 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 160, 2024-05-21T15:43:06.794 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 101584, 2024-05-21T15:43:06.794 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 3, 2024-05-21T15:43:06.794 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-05-21T15:43:06.794 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.010880327, 2024-05-21T15:43:06.794 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.007772314, 2024-05-21T15:43:06.794 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.001631043, 2024-05-21T15:43:06.794 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T15:43:06.795 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:06.795 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 134, 2024-05-21T15:43:06.795 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.006412073, 2024-05-21T15:43:06.795 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000047851 2024-05-21T15:43:06.795 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:06.795 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:06.795 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:06.795 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:06.795 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:06.795 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:06.795 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-21T15:43:06.796 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 101584 2024-05-21T15:43:06.796 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:06.796 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:43:06.796 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 0, 2024-05-21T15:43:06.796 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 2, 2024-05-21T15:43:06.796 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 0, 2024-05-21T15:43:06.796 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 0, 2024-05-21T15:43:06.796 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 2, 2024-05-21T15:43:06.796 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-05-21T15:43:06.796 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.000534097, 2024-05-21T15:43:06.796 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.000000000, 2024-05-21T15:43:06.796 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.000447812, 2024-05-21T15:43:06.797 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T15:43:06.797 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:06.797 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:06.797 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:06.797 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:06.797 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:06.797 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:06.797 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:06.797 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:06.797 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:06.797 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:06.798 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T15:43:06.798 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T15:43:06.798 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:06.798 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-05-21T15:43:06.798 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 0, 2024-05-21T15:43:06.798 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-05-21T15:43:06.798 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:06.798 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-05-21T15:43:06.798 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:43:06.798 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-05-21T15:43:06.798 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:43:06.799 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:43:06.799 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:43:06.799 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:43:06.799 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:43:06.799 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:43:06.799 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:43:06.799 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:43:06.799 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:43:06.799 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:43:06.799 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:43:06.799 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:43:06.799 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:43:06.800 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-05-21T15:43:06.800 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:43:06.800 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-05-21T15:43:06.800 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:43:06.800 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:43:06.800 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:43:06.800 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:43:06.800 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:43:06.800 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:43:06.800 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:43:06.800 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-05-21T15:43:06.801 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:43:06.801 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:43:06.801 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:43:06.801 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-05-21T15:43:06.801 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-05-21T15:43:06.801 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-05-21T15:43:06.801 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:43:06.801 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:43:06.801 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:43:06.801 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:43:06.801 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 148967, 2024-05-21T15:43:06.802 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 29, 2024-05-21T15:43:06.802 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:43:06.802 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-05-21T15:43:06.802 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-05-21T15:43:06.802 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-05-21T15:43:06.802 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:43:06.802 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-05-21T15:43:06.802 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:43:06.802 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-05-21T15:43:06.802 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 0, 2024-05-21T15:43:06.803 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 0, 2024-05-21T15:43:06.803 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:43:06.803 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:43:06.803 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-05-21T15:43:06.803 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-05-21T15:43:06.803 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 928, 2024-05-21T15:43:06.803 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 2, 2024-05-21T15:43:06.803 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-05-21T15:43:06.803 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-05-21T15:43:06.803 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-05-21T15:43:06.803 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-05-21T15:43:06.804 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:06.804 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:43:06.804 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-05-21T15:43:06.804 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-05-21T15:43:06.805 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-05-21T15:43:06.805 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 183, 2024-05-21T15:43:06.805 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 441777, 2024-05-21T15:43:06.805 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:06.805 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 183, 2024-05-21T15:43:06.805 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-05-21T15:43:06.805 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-05-21T15:43:06.805 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 183, 2024-05-21T15:43:06.805 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 441777, 2024-05-21T15:43:06.805 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-05-21T15:43:06.805 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:06.806 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:06.806 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:06.806 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:06.806 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:06.806 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:43:07.526 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-21T15:43:07.526 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-21T15:43:07.526 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-05-21T15:43:07.527 DEBUG:tasks.check_counter:Getting stats from g 2024-05-21T15:43:07.527 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-21T15:43:07.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:07 smithi038 ceph-mon[31995]: pgmap v612: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:07.787 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:43:07.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:07 smithi053 ceph-mon[36496]: pgmap v612: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:08.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:07 smithi045 ceph-mon[36409]: pgmap v612: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:09.526 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:43:09.527 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:43:09.527 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 535, 2024-05-21T15:43:09.527 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 537, 2024-05-21T15:43:09.527 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 179705, 2024-05-21T15:43:09.527 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 170131, 2024-05-21T15:43:09.527 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 6, 2024-05-21T15:43:09.527 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-05-21T15:43:09.527 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.069240770, 2024-05-21T15:43:09.527 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.025649177, 2024-05-21T15:43:09.527 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.015435876, 2024-05-21T15:43:09.527 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.021705378, 2024-05-21T15:43:09.527 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:09.527 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 535, 2024-05-21T15:43:09.528 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.038808056, 2024-05-21T15:43:09.528 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000072538 2024-05-21T15:43:09.528 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.528 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:09.528 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.528 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.528 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.528 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.528 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T15:43:09.528 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T15:43:09.528 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.528 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:43:09.529 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 924, 2024-05-21T15:43:09.529 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1056, 2024-05-21T15:43:09.529 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 6285656, 2024-05-21T15:43:09.529 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 607523, 2024-05-21T15:43:09.529 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 5, 2024-05-21T15:43:09.529 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-05-21T15:43:09.529 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.096158880, 2024-05-21T15:43:09.529 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.024961592, 2024-05-21T15:43:09.529 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.031548476, 2024-05-21T15:43:09.529 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.033595625, 2024-05-21T15:43:09.529 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:09.529 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1054, 2024-05-21T15:43:09.529 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.043657424, 2024-05-21T15:43:09.529 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000041420 2024-05-21T15:43:09.530 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.530 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:09.530 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.530 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.530 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.530 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.530 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-21T15:43:09.530 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 313952 2024-05-21T15:43:09.530 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.530 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:43:09.530 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 208, 2024-05-21T15:43:09.530 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 189, 2024-05-21T15:43:09.530 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 582532, 2024-05-21T15:43:09.531 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 110037, 2024-05-21T15:43:09.531 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-05-21T15:43:09.531 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 3, 2024-05-21T15:43:09.531 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.029489666, 2024-05-21T15:43:09.531 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.010434043, 2024-05-21T15:43:09.531 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.016954259, 2024-05-21T15:43:09.531 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.003945469, 2024-05-21T15:43:09.531 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:09.531 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 189, 2024-05-21T15:43:09.531 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.010657146, 2024-05-21T15:43:09.531 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000056387 2024-05-21T15:43:09.531 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.532 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:09.532 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.532 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.532 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.532 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.532 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 581184, 2024-05-21T15:43:09.532 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 109072 2024-05-21T15:43:09.532 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.532 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-05-21T15:43:09.532 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-05-21T15:43:09.534 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-05-21T15:43:09.534 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.534 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-05-21T15:43:09.534 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-05-21T15:43:09.534 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-05-21T15:43:09.534 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2603, 2024-05-21T15:43:09.534 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.071088932, 2024-05-21T15:43:09.534 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000027310 2024-05-21T15:43:09.534 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:09.534 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.534 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-05-21T15:43:09.535 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-05-21T15:43:09.535 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-05-21T15:43:09.535 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.535 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.535 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.535 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:09.535 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.535 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-05-21T15:43:09.535 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 0, 2024-05-21T15:43:09.535 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 0, 2024-05-21T15:43:09.536 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-05-21T15:43:09.536 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.536 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.536 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.536 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.536 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-05-21T15:43:09.536 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-05-21T15:43:09.536 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-05-21T15:43:09.536 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-05-21T15:43:09.536 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 0, 2024-05-21T15:43:09.536 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-05-21T15:43:09.536 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-05-21T15:43:09.536 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 303, 2024-05-21T15:43:09.537 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 0, 2024-05-21T15:43:09.537 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 303, 2024-05-21T15:43:09.537 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-05-21T15:43:09.537 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 303, 2024-05-21T15:43:09.537 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-05-21T15:43:09.537 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 0, 2024-05-21T15:43:09.537 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 0, 2024-05-21T15:43:09.537 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 1, 2024-05-21T15:43:09.537 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 0, 2024-05-21T15:43:09.537 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 0, 2024-05-21T15:43:09.537 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-05-21T15:43:09.537 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-05-21T15:43:09.537 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:43:09.538 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-05-21T15:43:09.538 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 0, 2024-05-21T15:43:09.538 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-05-21T15:43:09.538 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-05-21T15:43:09.538 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-05-21T15:43:09.538 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-05-21T15:43:09.538 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-05-21T15:43:09.538 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-05-21T15:43:09.538 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-05-21T15:43:09.538 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 0, 2024-05-21T15:43:09.538 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-05-21T15:43:09.538 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-05-21T15:43:09.538 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-05-21T15:43:09.538 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-05-21T15:43:09.539 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T15:43:09.539 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-05-21T15:43:09.539 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:43:09.539 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:43:09.539 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:43:09.539 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:43:09.539 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:43:09.539 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 0, 2024-05-21T15:43:09.539 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-05-21T15:43:09.539 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T15:43:09.539 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 0, 2024-05-21T15:43:09.539 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:43:09.539 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:43:09.540 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:43:09.540 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 0, 2024-05-21T15:43:09.540 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 0, 2024-05-21T15:43:09.540 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 0, 2024-05-21T15:43:09.540 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 0 2024-05-21T15:43:09.540 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.540 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-05-21T15:43:09.540 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-05-21T15:43:09.540 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-05-21T15:43:09.540 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-05-21T15:43:09.540 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-05-21T15:43:09.540 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-05-21T15:43:09.540 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-05-21T15:43:09.541 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-05-21T15:43:09.541 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:43:09.541 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 0, 2024-05-21T15:43:09.541 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 0, 2024-05-21T15:43:09.541 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-05-21T15:43:09.541 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-05-21T15:43:09.541 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-05-21T15:43:09.541 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:43:09.541 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:43:09.541 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-05-21T15:43:09.541 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-05-21T15:43:09.541 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:43:09.541 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:43:09.541 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:43:09.542 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-05-21T15:43:09.542 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-05-21T15:43:09.542 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:43:09.542 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-05-21T15:43:09.542 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-05-21T15:43:09.542 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.542 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-05-21T15:43:09.542 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-05-21T15:43:09.542 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 0, 2024-05-21T15:43:09.542 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-05-21T15:43:09.542 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-05-21T15:43:09.542 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 4564, 2024-05-21T15:43:09.542 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-05-21T15:43:09.543 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-05-21T15:43:09.543 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 0, 2024-05-21T15:43:09.543 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-05-21T15:43:09.543 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-05-21T15:43:09.543 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 7, 2024-05-21T15:43:09.543 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 2, 2024-05-21T15:43:09.543 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-05-21T15:43:09.543 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-05-21T15:43:09.543 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-05-21T15:43:09.543 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 10243958, 2024-05-21T15:43:09.543 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 10241794, 2024-05-21T15:43:09.543 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-05-21T15:43:09.543 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.543 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.544 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.544 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.544 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 4564 2024-05-21T15:43:09.544 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.544 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-05-21T15:43:09.544 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 5, 2024-05-21T15:43:09.544 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 317, 2024-05-21T15:43:09.544 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 312, 2024-05-21T15:43:09.544 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 3, 2024-05-21T15:43:09.544 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 93, 2024-05-21T15:43:09.544 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 90, 2024-05-21T15:43:09.545 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 303, 2024-05-21T15:43:09.545 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 637, 2024-05-21T15:43:09.545 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 334, 2024-05-21T15:43:09.545 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 0, 2024-05-21T15:43:09.545 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 0, 2024-05-21T15:43:09.545 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 0, 2024-05-21T15:43:09.545 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 122116, 2024-05-21T15:43:09.545 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 215444 2024-05-21T15:43:09.545 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.545 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-05-21T15:43:09.546 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 0, 2024-05-21T15:43:09.546 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-05-21T15:43:09.546 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 0, 2024-05-21T15:43:09.546 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 0, 2024-05-21T15:43:09.546 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-05-21T15:43:09.546 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-05-21T15:43:09.546 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.546 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.546 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.546 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.546 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-05-21T15:43:09.546 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.546 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.547 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.547 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.547 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-05-21T15:43:09.547 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.547 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.547 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.547 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.547 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-05-21T15:43:09.547 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.547 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.547 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.547 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.547 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-05-21T15:43:09.547 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.548 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.548 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.548 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.548 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-05-21T15:43:09.548 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.548 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.548 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.548 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.548 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-05-21T15:43:09.548 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.548 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.548 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.548 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.548 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-05-21T15:43:09.549 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.549 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.549 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.549 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.549 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-05-21T15:43:09.549 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.549 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.549 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.549 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.549 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-05-21T15:43:09.549 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.549 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.549 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.550 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.550 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-05-21T15:43:09.550 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.550 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.550 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.550 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.550 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-05-21T15:43:09.551 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.551 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.552 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.552 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.552 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-05-21T15:43:09.552 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.552 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.552 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.553 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.553 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-05-21T15:43:09.553 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.553 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.553 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.553 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.553 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-05-21T15:43:09.553 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.553 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.553 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.553 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.553 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-05-21T15:43:09.554 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.554 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.554 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.554 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.554 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-05-21T15:43:09.554 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.554 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.555 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.555 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.555 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-05-21T15:43:09.555 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.555 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.555 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.555 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.555 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-05-21T15:43:09.555 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.555 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.555 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.555 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.555 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-05-21T15:43:09.556 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.556 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.556 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.556 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.556 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-05-21T15:43:09.556 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.556 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.556 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.556 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.556 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-05-21T15:43:09.556 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.556 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.556 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.556 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.557 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-05-21T15:43:09.557 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.557 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.557 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.557 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.557 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-05-21T15:43:09.557 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.557 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.557 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.557 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.557 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-05-21T15:43:09.557 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.557 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.558 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.558 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.558 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-05-21T15:43:09.558 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.558 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.558 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.558 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.558 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-05-21T15:43:09.558 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.558 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.558 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.558 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.558 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-05-21T15:43:09.559 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.559 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.559 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.559 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.559 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-05-21T15:43:09.559 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.559 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.559 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.559 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.559 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:43:09.559 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:43:09.559 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-05-21T15:43:09.559 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.560 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.560 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.560 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:09.560 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.560 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-05-21T15:43:09.560 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-05-21T15:43:09.560 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 1, 2024-05-21T15:43:09.560 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 0, 2024-05-21T15:43:09.560 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-05-21T15:43:09.560 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-05-21T15:43:09.560 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 0, 2024-05-21T15:43:09.560 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 0, 2024-05-21T15:43:09.560 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 1616, 2024-05-21T15:43:09.561 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-05-21T15:43:09.561 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.561 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-05-21T15:43:09.561 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:43:09.561 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-05-21T15:43:09.561 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:43:09.561 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:43:09.561 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:43:09.561 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:43:09.561 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:43:09.561 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:43:09.561 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:43:09.561 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:43:09.562 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:43:09.562 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:43:09.562 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:43:09.562 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:43:09.562 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:43:09.562 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-05-21T15:43:09.562 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:43:09.562 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-05-21T15:43:09.562 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:43:09.562 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:43:09.562 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:43:09.562 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:43:09.563 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:43:09.563 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:43:09.563 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:43:09.563 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-05-21T15:43:09.563 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:43:09.563 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:43:09.563 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:43:09.563 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-05-21T15:43:09.563 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-05-21T15:43:09.563 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-05-21T15:43:09.563 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:43:09.563 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:43:09.563 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:43:09.563 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:43:09.564 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 197480, 2024-05-21T15:43:09.564 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 55, 2024-05-21T15:43:09.564 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:43:09.564 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-05-21T15:43:09.564 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-05-21T15:43:09.564 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-05-21T15:43:09.564 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:43:09.564 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-05-21T15:43:09.564 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:43:09.564 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-05-21T15:43:09.565 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 7824, 2024-05-21T15:43:09.565 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 165, 2024-05-21T15:43:09.565 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:43:09.565 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:43:09.565 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-05-21T15:43:09.565 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-05-21T15:43:09.565 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 228784, 2024-05-21T15:43:09.565 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 1251, 2024-05-21T15:43:09.565 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-05-21T15:43:09.565 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-05-21T15:43:09.565 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-05-21T15:43:09.565 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-05-21T15:43:09.565 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.566 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-05-21T15:43:09.566 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-05-21T15:43:09.566 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-05-21T15:43:09.566 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 1461, 2024-05-21T15:43:09.566 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 21, 2024-05-21T15:43:09.566 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 3, 2024-05-21T15:43:09.566 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 1461, 2024-05-21T15:43:09.566 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-05-21T15:43:09.566 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1461, 2024-05-21T15:43:09.566 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 111.605073754, 2024-05-21T15:43:09.566 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.076389509 2024-05-21T15:43:09.566 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.566 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-05-21T15:43:09.567 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-05-21T15:43:09.567 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1461, 2024-05-21T15:43:09.567 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1462 2024-05-21T15:43:09.567 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.567 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 1461, 2024-05-21T15:43:09.567 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 1461, 2024-05-21T15:43:09.567 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 0, 2024-05-21T15:43:09.567 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-05-21T15:43:09.567 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-05-21T15:43:09.567 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 812, 2024-05-21T15:43:09.568 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 0, 2024-05-21T15:43:09.568 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 647, 2024-05-21T15:43:09.568 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 0, 2024-05-21T15:43:09.568 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 0, 2024-05-21T15:43:09.568 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-05-21T15:43:09.568 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-05-21T15:43:09.568 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-05-21T15:43:09.568 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-05-21T15:43:09.568 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 0, 2024-05-21T15:43:09.568 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-05-21T15:43:09.568 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-05-21T15:43:09.568 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-05-21T15:43:09.568 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 1, 2024-05-21T15:43:09.569 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 0, 2024-05-21T15:43:09.569 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:43:09.569 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-05-21T15:43:09.569 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:43:09.569 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-05-21T15:43:09.569 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-05-21T15:43:09.569 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-05-21T15:43:09.569 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:43:09.569 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-05-21T15:43:09.569 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:43:09.569 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-05-21T15:43:09.569 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-05-21T15:43:09.569 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-05-21T15:43:09.570 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-05-21T15:43:09.570 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-05-21T15:43:09.570 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-05-21T15:43:09.570 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-05-21T15:43:09.570 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-05-21T15:43:09.570 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-05-21T15:43:09.570 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-05-21T15:43:09.570 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-05-21T15:43:09.570 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-05-21T15:43:09.570 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-05-21T15:43:09.570 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-05-21T15:43:09.570 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-05-21T15:43:09.570 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-05-21T15:43:09.570 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-05-21T15:43:09.571 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 90, 2024-05-21T15:43:09.571 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-05-21T15:43:09.571 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 4, 2024-05-21T15:43:09.571 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 6, 2024-05-21T15:43:09.571 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 6, 2024-05-21T15:43:09.571 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-05-21T15:43:09.571 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-05-21T15:43:09.571 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 0, 2024-05-21T15:43:09.571 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 2, 2024-05-21T15:43:09.571 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 0 2024-05-21T15:43:09.571 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.571 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-05-21T15:43:09.571 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 0, 2024-05-21T15:43:09.572 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:43:09.572 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-05-21T15:43:09.572 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-05-21T15:43:09.572 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.572 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-05-21T15:43:09.572 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-05-21T15:43:09.572 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.572 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-05-21T15:43:09.572 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-05-21T15:43:09.572 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T15:43:09.572 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-05-21T15:43:09.573 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 0, 2024-05-21T15:43:09.573 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 0, 2024-05-21T15:43:09.573 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 0, 2024-05-21T15:43:09.573 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-05-21T15:43:09.573 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.573 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:43:09.573 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-05-21T15:43:09.573 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-05-21T15:43:09.573 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-05-21T15:43:09.573 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 1663, 2024-05-21T15:43:09.573 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 6884997, 2024-05-21T15:43:09.573 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:09.573 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 1663, 2024-05-21T15:43:09.574 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-05-21T15:43:09.574 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-05-21T15:43:09.574 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 1663, 2024-05-21T15:43:09.574 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 6884997, 2024-05-21T15:43:09.574 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-05-21T15:43:09.574 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.574 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.574 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.574 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:09.574 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.575 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-05-21T15:43:09.575 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-05-21T15:43:09.575 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-05-21T15:43:09.575 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-05-21T15:43:09.575 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-05-21T15:43:09.575 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-05-21T15:43:09.575 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:09.575 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:09.575 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 1461, 2024-05-21T15:43:09.575 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 6049659, 2024-05-21T15:43:09.575 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 1461, 2024-05-21T15:43:09.575 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 6049659, 2024-05-21T15:43:09.575 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-05-21T15:43:09.576 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.576 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.576 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.576 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:09.576 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.576 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-05-21T15:43:09.576 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-05-21T15:43:09.576 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-05-21T15:43:09.576 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-05-21T15:43:09.576 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-05-21T15:43:09.576 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-05-21T15:43:09.576 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:09.576 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:09.577 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 1461, 2024-05-21T15:43:09.577 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 1461, 2024-05-21T15:43:09.577 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 1461, 2024-05-21T15:43:09.577 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1461, 2024-05-21T15:43:09.577 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-05-21T15:43:09.577 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.577 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.577 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.577 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:09.577 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.577 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:43:09.577 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-05-21T15:43:09.577 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-05-21T15:43:09.577 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-05-21T15:43:09.578 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-05-21T15:43:09.578 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-05-21T15:43:09.578 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:09.578 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:09.578 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-05-21T15:43:09.578 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-05-21T15:43:09.578 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-05-21T15:43:09.578 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-05-21T15:43:09.578 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-05-21T15:43:09.578 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.578 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.578 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.579 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:09.579 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:09.579 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x55b848ac8a90": { 2024-05-21T15:43:09.579 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-05-21T15:43:09.579 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-05-21T15:43:09.579 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-05-21T15:43:09.579 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-05-21T15:43:09.579 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-05-21T15:43:09.579 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:09.579 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:09.580 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-05-21T15:43:09.580 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-05-21T15:43:09.580 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-05-21T15:43:09.580 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-05-21T15:43:09.580 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-05-21T15:43:09.580 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:09.580 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:09.580 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:09.580 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:09.580 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:09.580 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:43:09.853 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:09 smithi038 ceph-mon[31995]: pgmap v613: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:43:09.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:09 smithi053 ceph-mon[36496]: pgmap v613: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:43:10.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:09 smithi045 ceph-mon[36409]: pgmap v613: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:43:10.320 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-21T15:43:10.320 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-21T15:43:10.320 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-05-21T15:43:10.320 DEBUG:tasks.check_counter:Getting stats from j 2024-05-21T15:43:10.321 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-21T15:43:10.574 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:43:11.700 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:11 smithi038 ceph-mon[31995]: pgmap v614: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:11.811 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:11 smithi045 ceph-mon[36409]: pgmap v614: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:11.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:11 smithi053 ceph-mon[36496]: pgmap v614: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:12.301 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-05-21T15:43:12.302 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:43:12.302 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 4, 2024-05-21T15:43:12.302 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 4, 2024-05-21T15:43:12.302 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 2992, 2024-05-21T15:43:12.302 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 448, 2024-05-21T15:43:12.302 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-05-21T15:43:12.302 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 0, 2024-05-21T15:43:12.302 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.001840127, 2024-05-21T15:43:12.303 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.000037484, 2024-05-21T15:43:12.303 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.000808192, 2024-05-21T15:43:12.303 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T15:43:12.303 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:12.303 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3, 2024-05-21T15:43:12.303 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000883031, 2024-05-21T15:43:12.303 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000294343 2024-05-21T15:43:12.303 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:12.303 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:12.304 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:12.304 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:12.304 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:12.304 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:12.304 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-21T15:43:12.304 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 448 2024-05-21T15:43:12.304 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:12.304 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:43:12.304 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1, 2024-05-21T15:43:12.304 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 137, 2024-05-21T15:43:12.304 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 160, 2024-05-21T15:43:12.305 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 101760, 2024-05-21T15:43:12.305 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 3, 2024-05-21T15:43:12.305 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-05-21T15:43:12.305 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.010204843, 2024-05-21T15:43:12.305 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.007351845, 2024-05-21T15:43:12.305 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.001518090, 2024-05-21T15:43:12.305 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T15:43:12.305 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:12.305 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 135, 2024-05-21T15:43:12.305 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.007387806, 2024-05-21T15:43:12.305 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000054724 2024-05-21T15:43:12.306 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:12.306 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:12.306 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:12.306 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:12.306 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:12.306 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:12.306 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-21T15:43:12.306 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 101760 2024-05-21T15:43:12.306 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:12.306 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:43:12.307 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 184, 2024-05-21T15:43:12.307 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 181, 2024-05-21T15:43:12.307 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 468032, 2024-05-21T15:43:12.307 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 110928, 2024-05-21T15:43:12.307 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 2, 2024-05-21T15:43:12.307 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-05-21T15:43:12.307 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.028392419, 2024-05-21T15:43:12.307 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.010887806, 2024-05-21T15:43:12.307 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.014565931, 2024-05-21T15:43:12.307 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.004132711, 2024-05-21T15:43:12.307 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:12.308 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 180, 2024-05-21T15:43:12.308 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.008017831, 2024-05-21T15:43:12.308 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000044543 2024-05-21T15:43:12.308 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:12.308 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:12.308 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:12.308 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:12.308 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:12.308 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:12.309 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 468032, 2024-05-21T15:43:12.309 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 110928 2024-05-21T15:43:12.309 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:12.309 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-05-21T15:43:12.309 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 0, 2024-05-21T15:43:12.309 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-05-21T15:43:12.309 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:12.309 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-05-21T15:43:12.309 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:43:12.309 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-05-21T15:43:12.310 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:43:12.310 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:43:12.310 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:43:12.310 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:43:12.310 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:43:12.310 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:43:12.310 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:43:12.310 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:43:12.310 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:43:12.310 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:43:12.310 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:43:12.310 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:43:12.310 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:43:12.311 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-05-21T15:43:12.311 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:43:12.311 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-05-21T15:43:12.311 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:43:12.311 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:43:12.311 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:43:12.311 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:43:12.311 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:43:12.311 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:43:12.311 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:43:12.311 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-05-21T15:43:12.311 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:43:12.311 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:43:12.311 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:43:12.312 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-05-21T15:43:12.312 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-05-21T15:43:12.312 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-05-21T15:43:12.312 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:43:12.312 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:43:12.312 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:43:12.312 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:43:12.312 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 149998, 2024-05-21T15:43:12.312 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 30, 2024-05-21T15:43:12.312 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:43:12.312 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-05-21T15:43:12.312 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-05-21T15:43:12.313 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-05-21T15:43:12.313 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:43:12.313 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-05-21T15:43:12.313 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:43:12.313 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-05-21T15:43:12.313 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 0, 2024-05-21T15:43:12.313 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 0, 2024-05-21T15:43:12.313 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:43:12.313 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:43:12.313 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-05-21T15:43:12.313 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-05-21T15:43:12.313 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 928, 2024-05-21T15:43:12.314 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 2, 2024-05-21T15:43:12.314 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-05-21T15:43:12.314 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-05-21T15:43:12.314 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-05-21T15:43:12.314 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-05-21T15:43:12.314 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-05-21T15:43:12.314 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:43:12.314 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-05-21T15:43:12.314 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-05-21T15:43:12.314 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-05-21T15:43:12.314 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 185, 2024-05-21T15:43:12.314 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 442037, 2024-05-21T15:43:12.314 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:12.315 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 185, 2024-05-21T15:43:12.315 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-05-21T15:43:12.315 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-05-21T15:43:12.315 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 185, 2024-05-21T15:43:12.315 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 442037, 2024-05-21T15:43:12.315 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-05-21T15:43:12.315 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-05-21T15:43:12.315 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-05-21T15:43:12.315 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-05-21T15:43:12.315 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:12.315 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-05-21T15:43:12.315 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-05-21T15:43:12.957 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-21T15:43:12.957 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-21T15:43:12.957 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-05-21T15:43:12.957 DEBUG:tasks.check_counter:Getting stats from b 2024-05-21T15:43:12.957 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-21T15:43:13.315 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:43:13.758 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:13 smithi045 ceph-mon[36409]: pgmap v615: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:13.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:13 smithi053 ceph-mon[36496]: pgmap v615: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:14.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:13 smithi038 ceph-mon[31995]: pgmap v615: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:15.864 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:15 smithi045 ceph-mon[36409]: pgmap v616: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:15.865 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-21T15:43:15.865 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:43:15.865 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 6080, 2024-05-21T15:43:15.865 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 6318, 2024-05-21T15:43:15.865 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2674676, 2024-05-21T15:43:15.865 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 6954399, 2024-05-21T15:43:15.865 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 9, 2024-05-21T15:43:15.866 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 4, 2024-05-21T15:43:15.866 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.624182774, 2024-05-21T15:43:15.866 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.271517553, 2024-05-21T15:43:15.866 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.256362398, 2024-05-21T15:43:15.866 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.090328393, 2024-05-21T15:43:15.866 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:15.866 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 6320, 2024-05-21T15:43:15.866 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.043009667, 2024-05-21T15:43:15.866 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000165033 2024-05-21T15:43:15.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.867 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:15.867 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 7302, 2024-05-21T15:43:15.867 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.002562905, 2024-05-21T15:43:15.867 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000350 2024-05-21T15:43:15.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.867 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 616688, 2024-05-21T15:43:15.867 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 131520 2024-05-21T15:43:15.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.867 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:43:15.867 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4285, 2024-05-21T15:43:15.867 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 4637, 2024-05-21T15:43:15.867 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1447002, 2024-05-21T15:43:15.867 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 12696971, 2024-05-21T15:43:15.867 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 10, 2024-05-21T15:43:15.868 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 5, 2024-05-21T15:43:15.868 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.420325329, 2024-05-21T15:43:15.868 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.145505450, 2024-05-21T15:43:15.868 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.146539022, 2024-05-21T15:43:15.868 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.111525487, 2024-05-21T15:43:15.868 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:15.868 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 4638, 2024-05-21T15:43:15.868 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.160118088, 2024-05-21T15:43:15.868 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000250133 2024-05-21T15:43:15.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.868 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:15.868 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3236, 2024-05-21T15:43:15.868 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.001528890, 2024-05-21T15:43:15.869 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000472 2024-05-21T15:43:15.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-21T15:43:15.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 320832 2024-05-21T15:43:15.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.869 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:43:15.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 18611, 2024-05-21T15:43:15.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 22830, 2024-05-21T15:43:15.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 7169122, 2024-05-21T15:43:15.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 23472381, 2024-05-21T15:43:15.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 186, 2024-05-21T15:43:15.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 9, 2024-05-21T15:43:15.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 1.637066904, 2024-05-21T15:43:15.870 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.621434408, 2024-05-21T15:43:15.870 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 1.510145941, 2024-05-21T15:43:15.870 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.107006128, 2024-05-21T15:43:15.870 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:15.870 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 22823, 2024-05-21T15:43:15.870 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 6.345913439, 2024-05-21T15:43:15.870 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000278049 2024-05-21T15:43:15.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.871 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:15.871 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 31187, 2024-05-21T15:43:15.871 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.010952647, 2024-05-21T15:43:15.871 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000351 2024-05-21T15:43:15.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.871 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 5946928, 2024-05-21T15:43:15.871 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 10109840 2024-05-21T15:43:15.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.871 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-05-21T15:43:15.871 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2024-05-21T15:43:15.871 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-05-21T15:43:15.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.871 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2024-05-21T15:43:15.871 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-05-21T15:43:15.872 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-05-21T15:43:15.873 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 5418, 2024-05-21T15:43:15.873 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 8.400490061, 2024-05-21T15:43:15.873 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001550478 2024-05-21T15:43:15.873 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:15.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.873 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2024-05-21T15:43:15.874 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-05-21T15:43:15.874 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-05-21T15:43:15.874 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1222, 2024-05-21T15:43:15.874 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.070249261, 2024-05-21T15:43:15.874 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000057487 2024-05-21T15:43:15.874 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:15.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.875 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2024-05-21T15:43:15.875 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 2121, 2024-05-21T15:43:15.875 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 2118, 2024-05-21T15:43:15.875 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2024-05-21T15:43:15.875 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2118, 2024-05-21T15:43:15.875 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 18.793164450, 2024-05-21T15:43:15.875 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.008873071 2024-05-21T15:43:15.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.876 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2024-05-21T15:43:15.876 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 3, 2024-05-21T15:43:15.876 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 11, 2024-05-21T15:43:15.876 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 4, 2024-05-21T15:43:15.876 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 448, 2024-05-21T15:43:15.876 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 1, 2024-05-21T15:43:15.876 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2024-05-21T15:43:15.876 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 27, 2024-05-21T15:43:15.876 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 7, 2024-05-21T15:43:15.876 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 6, 2024-05-21T15:43:15.876 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 14, 2024-05-21T15:43:15.876 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 17, 2024-05-21T15:43:15.877 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 757, 2024-05-21T15:43:15.877 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 2, 2024-05-21T15:43:15.877 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 2, 2024-05-21T15:43:15.877 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 6, 2024-05-21T15:43:15.877 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 2873, 2024-05-21T15:43:15.877 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 2711, 2024-05-21T15:43:15.877 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 2, 2024-05-21T15:43:15.877 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 0, 2024-05-21T15:43:15.877 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:43:15.877 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2024-05-21T15:43:15.877 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 51, 2024-05-21T15:43:15.878 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2024-05-21T15:43:15.878 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2024-05-21T15:43:15.878 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 13, 2024-05-21T15:43:15.878 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 612, 2024-05-21T15:43:15.878 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 13, 2024-05-21T15:43:15.878 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 203, 2024-05-21T15:43:15.878 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2024-05-21T15:43:15.878 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 1026, 2024-05-21T15:43:15.878 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 0, 2024-05-21T15:43:15.878 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 4, 2024-05-21T15:43:15.878 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 234, 2024-05-21T15:43:15.878 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 0, 2024-05-21T15:43:15.879 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 1026, 2024-05-21T15:43:15.879 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2024-05-21T15:43:15.879 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:43:15.879 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:43:15.879 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 313, 2024-05-21T15:43:15.879 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 13, 2024-05-21T15:43:15.879 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 300, 2024-05-21T15:43:15.879 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 713, 2024-05-21T15:43:15.879 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 10, 2024-05-21T15:43:15.879 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 206, 2024-05-21T15:43:15.880 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 3337, 2024-05-21T15:43:15.880 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 6, 2024-05-21T15:43:15.880 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:43:15.880 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:43:15.880 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 11678, 2024-05-21T15:43:15.880 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 11002, 2024-05-21T15:43:15.880 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 78, 2024-05-21T15:43:15.880 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 960 2024-05-21T15:43:15.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.880 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2024-05-21T15:43:15.880 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 4, 2024-05-21T15:43:15.880 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 0, 2024-05-21T15:43:15.881 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 0, 2024-05-21T15:43:15.881 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 0, 2024-05-21T15:43:15.881 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 21, 2024-05-21T15:43:15.881 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 1, 2024-05-21T15:43:15.881 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 0, 2024-05-21T15:43:15.881 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:43:15.881 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 548, 2024-05-21T15:43:15.881 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 486, 2024-05-21T15:43:15.881 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 61, 2024-05-21T15:43:15.881 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2024-05-21T15:43:15.881 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2024-05-21T15:43:15.881 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:43:15.881 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:43:15.881 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2024-05-21T15:43:15.882 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2024-05-21T15:43:15.882 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:43:15.882 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:43:15.882 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 13, 2024-05-21T15:43:15.882 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 13, 2024-05-21T15:43:15.882 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2024-05-21T15:43:15.882 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 1, 2024-05-21T15:43:15.882 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2024-05-21T15:43:15.882 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2024-05-21T15:43:15.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.882 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2024-05-21T15:43:15.882 INFO:teuthology.orchestra.run.smithi045.stdout: "evlrg": 0, 2024-05-21T15:43:15.883 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 13548, 2024-05-21T15:43:15.883 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 13475, 2024-05-21T15:43:15.883 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 13475, 2024-05-21T15:43:15.883 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 73, 2024-05-21T15:43:15.883 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2024-05-21T15:43:15.883 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 7, 2024-05-21T15:43:15.883 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 46, 2024-05-21T15:43:15.883 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 45, 2024-05-21T15:43:15.883 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 45, 2024-05-21T15:43:15.883 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 1, 2024-05-21T15:43:15.883 INFO:teuthology.orchestra.run.smithi045.stdout: "segmjr": 1, 2024-05-21T15:43:15.883 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2024-05-21T15:43:15.883 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 1, 2024-05-21T15:43:15.884 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 33922637, 2024-05-21T15:43:15.884 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 34057902, 2024-05-21T15:43:15.884 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 0, 2024-05-21T15:43:15.884 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2024-05-21T15:43:15.884 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2367, 2024-05-21T15:43:15.884 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 107.598992461, 2024-05-21T15:43:15.884 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.045457960 2024-05-21T15:43:15.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.884 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 0 2024-05-21T15:43:15.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.884 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2024-05-21T15:43:15.884 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 32, 2024-05-21T15:43:15.884 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 1111, 2024-05-21T15:43:15.884 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 1079, 2024-05-21T15:43:15.885 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 27, 2024-05-21T15:43:15.885 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 210, 2024-05-21T15:43:15.885 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 183, 2024-05-21T15:43:15.885 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 27, 2024-05-21T15:43:15.885 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 1504, 2024-05-21T15:43:15.885 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 1477, 2024-05-21T15:43:15.885 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 2, 2024-05-21T15:43:15.885 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 1045, 2024-05-21T15:43:15.885 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 1043, 2024-05-21T15:43:15.885 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 139512, 2024-05-21T15:43:15.885 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 231828 2024-05-21T15:43:15.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.886 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2024-05-21T15:43:15.886 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 2514, 2024-05-21T15:43:15.886 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 217, 2024-05-21T15:43:15.886 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 2121, 2024-05-21T15:43:15.886 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 25, 2024-05-21T15:43:15.886 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 455, 2024-05-21T15:43:15.886 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2024-05-21T15:43:15.886 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 765, 2024-05-21T15:43:15.886 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.025981688, 2024-05-21T15:43:15.886 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001341152 2024-05-21T15:43:15.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.886 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2024-05-21T15:43:15.886 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 79, 2024-05-21T15:43:15.887 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.023681878, 2024-05-21T15:43:15.887 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000299770 2024-05-21T15:43:15.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.887 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2024-05-21T15:43:15.887 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.887 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.887 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.887 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2024-05-21T15:43:15.887 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 61, 2024-05-21T15:43:15.887 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.051579573, 2024-05-21T15:43:15.887 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000845566 2024-05-21T15:43:15.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.888 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2024-05-21T15:43:15.888 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 230, 2024-05-21T15:43:15.888 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 2.853169462, 2024-05-21T15:43:15.888 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.012405084 2024-05-21T15:43:15.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.888 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2024-05-21T15:43:15.888 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.888 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.888 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.888 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2024-05-21T15:43:15.888 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-05-21T15:43:15.888 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000142773, 2024-05-21T15:43:15.888 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000142773 2024-05-21T15:43:15.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.889 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2024-05-21T15:43:15.889 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.889 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.889 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.889 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2024-05-21T15:43:15.889 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.889 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.889 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.889 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2024-05-21T15:43:15.889 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.890 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.890 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.890 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2024-05-21T15:43:15.891 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.891 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.891 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.891 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2024-05-21T15:43:15.891 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 105, 2024-05-21T15:43:15.891 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.113455911, 2024-05-21T15:43:15.891 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001080532 2024-05-21T15:43:15.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.891 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2024-05-21T15:43:15.891 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.891 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.892 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.892 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2024-05-21T15:43:15.892 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.892 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.892 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.892 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2024-05-21T15:43:15.892 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 52, 2024-05-21T15:43:15.892 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.054099375, 2024-05-21T15:43:15.892 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001040372 2024-05-21T15:43:15.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.892 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2024-05-21T15:43:15.892 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 166, 2024-05-21T15:43:15.893 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.977032308, 2024-05-21T15:43:15.893 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.005885736 2024-05-21T15:43:15.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.893 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2024-05-21T15:43:15.893 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 113, 2024-05-21T15:43:15.893 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 12.106236387, 2024-05-21T15:43:15.893 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.107134835 2024-05-21T15:43:15.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.893 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2024-05-21T15:43:15.893 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.893 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.893 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.894 INFO:teuthology.orchestra.run.smithi045.stdout: "req_snapdiff_latency": { 2024-05-21T15:43:15.894 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.894 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.894 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.894 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2024-05-21T15:43:15.894 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 22, 2024-05-21T15:43:15.894 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.757803557, 2024-05-21T15:43:15.894 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.034445616 2024-05-21T15:43:15.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.894 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2024-05-21T15:43:15.894 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.894 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.895 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.895 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2024-05-21T15:43:15.895 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.895 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.895 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.895 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2024-05-21T15:43:15.895 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 7, 2024-05-21T15:43:15.895 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.032809125, 2024-05-21T15:43:15.895 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.004687017 2024-05-21T15:43:15.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.895 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2024-05-21T15:43:15.896 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.896 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.896 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.896 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2024-05-21T15:43:15.896 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 4, 2024-05-21T15:43:15.896 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.001128751, 2024-05-21T15:43:15.896 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000282187 2024-05-21T15:43:15.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.896 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2024-05-21T15:43:15.896 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.896 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.896 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.897 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2024-05-21T15:43:15.897 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 7, 2024-05-21T15:43:15.897 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.019459970, 2024-05-21T15:43:15.897 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.002779995 2024-05-21T15:43:15.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.897 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2024-05-21T15:43:15.897 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 13, 2024-05-21T15:43:15.897 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.009736340, 2024-05-21T15:43:15.897 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000748949 2024-05-21T15:43:15.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.897 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2024-05-21T15:43:15.897 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 493, 2024-05-21T15:43:15.897 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.766847352, 2024-05-21T15:43:15.898 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001555471 2024-05-21T15:43:15.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.898 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:43:15.898 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:43:15.898 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2024-05-21T15:43:15.898 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.898 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.898 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.898 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:15.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.898 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2024-05-21T15:43:15.898 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 1, 2024-05-21T15:43:15.898 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 2, 2024-05-21T15:43:15.899 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 1, 2024-05-21T15:43:15.899 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 1, 2024-05-21T15:43:15.899 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2024-05-21T15:43:15.899 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 75, 2024-05-21T15:43:15.899 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 75, 2024-05-21T15:43:15.899 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 518, 2024-05-21T15:43:15.899 INFO:teuthology.orchestra.run.smithi045.stdout: "mdthresh_evicted": 0 2024-05-21T15:43:15.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.899 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-05-21T15:43:15.899 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:43:15.899 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-05-21T15:43:15.899 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:43:15.900 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:43:15.900 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:43:15.900 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:43:15.900 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:43:15.900 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:43:15.900 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:43:15.900 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:43:15.900 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:43:15.900 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:43:15.900 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:43:15.900 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:43:15.900 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:43:15.900 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-05-21T15:43:15.900 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:43:15.901 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-05-21T15:43:15.901 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:43:15.901 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:43:15.901 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:43:15.901 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:43:15.901 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:43:15.901 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:43:15.901 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:43:15.901 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-05-21T15:43:15.901 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:43:15.901 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:43:15.901 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:43:15.901 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-05-21T15:43:15.902 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-05-21T15:43:15.902 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-05-21T15:43:15.902 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:43:15.902 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:43:15.902 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:43:15.902 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:43:15.902 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 298604, 2024-05-21T15:43:15.902 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 125, 2024-05-21T15:43:15.902 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 704, 2024-05-21T15:43:15.902 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 8, 2024-05-21T15:43:15.902 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-05-21T15:43:15.902 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-05-21T15:43:15.902 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:43:15.903 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-05-21T15:43:15.903 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:43:15.903 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-05-21T15:43:15.903 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 7824, 2024-05-21T15:43:15.903 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 165, 2024-05-21T15:43:15.903 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:43:15.903 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:43:15.903 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-05-21T15:43:15.903 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-05-21T15:43:15.903 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 129766, 2024-05-21T15:43:15.903 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 811, 2024-05-21T15:43:15.903 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-05-21T15:43:15.903 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-05-21T15:43:15.903 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-05-21T15:43:15.904 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-05-21T15:43:15.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.904 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2024-05-21T15:43:15.904 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2024-05-21T15:43:15.904 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2024-05-21T15:43:15.904 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 6081, 2024-05-21T15:43:15.904 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 31276926, 2024-05-21T15:43:15.904 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 37, 2024-05-21T15:43:15.904 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 6081, 2024-05-21T15:43:15.904 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2024-05-21T15:43:15.904 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 6081, 2024-05-21T15:43:15.904 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 538.621766372, 2024-05-21T15:43:15.904 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.088574538 2024-05-21T15:43:15.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.905 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2024-05-21T15:43:15.905 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2024-05-21T15:43:15.905 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 6081, 2024-05-21T15:43:15.905 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 8143 2024-05-21T15:43:15.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.905 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 6081, 2024-05-21T15:43:15.905 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 1082, 2024-05-21T15:43:15.905 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 4999, 2024-05-21T15:43:15.905 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2024-05-21T15:43:15.905 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2024-05-21T15:43:15.905 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 339, 2024-05-21T15:43:15.905 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 521, 2024-05-21T15:43:15.905 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 28, 2024-05-21T15:43:15.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 2382, 2024-05-21T15:43:15.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 122, 2024-05-21T15:43:15.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2024-05-21T15:43:15.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2024-05-21T15:43:15.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 0, 2024-05-21T15:43:15.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 1, 2024-05-21T15:43:15.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 1334, 2024-05-21T15:43:15.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2024-05-21T15:43:15.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2024-05-21T15:43:15.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2024-05-21T15:43:15.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 1054, 2024-05-21T15:43:15.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 1042, 2024-05-21T15:43:15.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:43:15.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2024-05-21T15:43:15.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:43:15.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2024-05-21T15:43:15.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2024-05-21T15:43:15.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2024-05-21T15:43:15.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:43:15.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2024-05-21T15:43:15.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:43:15.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 6, 2024-05-21T15:43:15.908 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2024-05-21T15:43:15.908 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2024-05-21T15:43:15.908 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2024-05-21T15:43:15.908 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2024-05-21T15:43:15.908 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2024-05-21T15:43:15.908 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2024-05-21T15:43:15.908 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2024-05-21T15:43:15.908 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2024-05-21T15:43:15.908 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2024-05-21T15:43:15.908 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2024-05-21T15:43:15.908 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2024-05-21T15:43:15.908 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2024-05-21T15:43:15.908 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2024-05-21T15:43:15.908 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2024-05-21T15:43:15.909 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2024-05-21T15:43:15.909 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2024-05-21T15:43:15.909 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 90, 2024-05-21T15:43:15.909 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2024-05-21T15:43:15.909 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 5, 2024-05-21T15:43:15.909 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 12, 2024-05-21T15:43:15.909 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 12, 2024-05-21T15:43:15.909 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 0, 2024-05-21T15:43:15.909 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2024-05-21T15:43:15.909 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 1136, 2024-05-21T15:43:15.909 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 30, 2024-05-21T15:43:15.909 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 148 2024-05-21T15:43:15.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.910 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2024-05-21T15:43:15.910 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 1, 2024-05-21T15:43:15.910 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:43:15.910 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2024-05-21T15:43:15.910 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2024-05-21T15:43:15.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.910 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2024-05-21T15:43:15.910 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 0 2024-05-21T15:43:15.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.910 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2024-05-21T15:43:15.910 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2024-05-21T15:43:15.910 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 51, 2024-05-21T15:43:15.910 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2024-05-21T15:43:15.910 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 6, 2024-05-21T15:43:15.911 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed_ops": 808, 2024-05-21T15:43:15.911 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 486, 2024-05-21T15:43:15.911 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2024-05-21T15:43:15.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.911 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:43:15.911 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:15.911 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-05-21T15:43:15.911 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:15.911 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 28972, 2024-05-21T15:43:15.911 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 7930935, 2024-05-21T15:43:15.911 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:15.911 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 28972, 2024-05-21T15:43:15.911 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-05-21T15:43:15.912 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-05-21T15:43:15.912 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 28972, 2024-05-21T15:43:15.912 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 7930935, 2024-05-21T15:43:15.912 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:15.912 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.912 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.912 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.912 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:15.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.912 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2024-05-21T15:43:15.912 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:15.912 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-05-21T15:43:15.912 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:15.913 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-05-21T15:43:15.913 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-05-21T15:43:15.913 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:15.913 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:15.913 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 6081, 2024-05-21T15:43:15.913 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 31325828, 2024-05-21T15:43:15.913 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 6081, 2024-05-21T15:43:15.913 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 31325828, 2024-05-21T15:43:15.913 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:15.913 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.913 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.913 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.913 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:15.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.914 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2024-05-21T15:43:15.914 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:15.914 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2024-05-21T15:43:15.914 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:15.914 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-05-21T15:43:15.914 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-05-21T15:43:15.914 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:15.914 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:15.914 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 6081, 2024-05-21T15:43:15.914 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 6081, 2024-05-21T15:43:15.914 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 6081, 2024-05-21T15:43:15.914 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 6081, 2024-05-21T15:43:15.914 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:15.915 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.915 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.915 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.915 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:15.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.915 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:43:15.915 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:15.915 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-05-21T15:43:15.915 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:15.915 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 486, 2024-05-21T15:43:15.915 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 49094, 2024-05-21T15:43:15.915 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:15.915 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 486, 2024-05-21T15:43:15.916 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-05-21T15:43:15.916 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-05-21T15:43:15.916 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 15, 2024-05-21T15:43:15.916 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 49094, 2024-05-21T15:43:15.916 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:15.916 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.916 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.916 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.916 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:15.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:15.916 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x556948894b60": { 2024-05-21T15:43:15.916 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:15.916 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-05-21T15:43:15.917 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:15.917 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 13548, 2024-05-21T15:43:15.917 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 29863598, 2024-05-21T15:43:15.917 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:15.917 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 13548, 2024-05-21T15:43:15.917 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-05-21T15:43:15.917 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-05-21T15:43:15.917 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 2367, 2024-05-21T15:43:15.917 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 29863598, 2024-05-21T15:43:15.917 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:15.917 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:15.917 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:15.917 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:15.918 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:15.918 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:15.918 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-21T15:43:15.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:15 smithi053 ceph-mon[36496]: pgmap v616: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:16.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:15 smithi038 ceph-mon[31995]: pgmap v616: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:16.882 INFO:tasks.check_counter:Daemon mds.b mds.exported=13 2024-05-21T15:43:16.882 INFO:tasks.check_counter:Daemon mds.b mds.imported=13 2024-05-21T15:43:16.882 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=1 2024-05-21T15:43:16.882 DEBUG:tasks.check_counter:Getting stats from e 2024-05-21T15:43:16.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-21T15:43:17.159 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:43:17.826 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:17 smithi038 ceph-mon[31995]: pgmap v617: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:19.196 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:17 smithi045 ceph-mon[36409]: pgmap v617: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:19.196 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:17 smithi053 ceph-mon[36496]: pgmap v617: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:19.197 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-21T15:43:19.197 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:43:19.198 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 5923, 2024-05-21T15:43:19.198 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 7028, 2024-05-21T15:43:19.198 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2522089, 2024-05-21T15:43:19.198 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 7429384, 2024-05-21T15:43:19.198 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 25, 2024-05-21T15:43:19.198 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 7, 2024-05-21T15:43:19.198 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.556621225, 2024-05-21T15:43:19.198 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.188992267, 2024-05-21T15:43:19.198 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.454063623, 2024-05-21T15:43:19.198 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.039027096, 2024-05-21T15:43:19.198 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:19.198 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 7021, 2024-05-21T15:43:19.199 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 2.134768088, 2024-05-21T15:43:19.199 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000304054 2024-05-21T15:43:19.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.199 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:19.199 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 8366, 2024-05-21T15:43:19.199 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.002215451, 2024-05-21T15:43:19.199 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000264 2024-05-21T15:43:19.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.199 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1856416, 2024-05-21T15:43:19.199 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 2591664 2024-05-21T15:43:19.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.199 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:43:19.199 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 3132, 2024-05-21T15:43:19.199 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 3517, 2024-05-21T15:43:19.200 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1485167, 2024-05-21T15:43:19.200 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 1450492, 2024-05-21T15:43:19.200 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 9, 2024-05-21T15:43:19.200 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 5, 2024-05-21T15:43:19.200 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.327512031, 2024-05-21T15:43:19.200 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.149823352, 2024-05-21T15:43:19.200 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.150264354, 2024-05-21T15:43:19.200 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.028465199, 2024-05-21T15:43:19.200 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:19.200 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3521, 2024-05-21T15:43:19.200 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.246101389, 2024-05-21T15:43:19.200 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000069895 2024-05-21T15:43:19.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.200 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:19.201 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 4270, 2024-05-21T15:43:19.201 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.001589848, 2024-05-21T15:43:19.201 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000372 2024-05-21T15:43:19.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.201 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-21T15:43:19.201 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 318832 2024-05-21T15:43:19.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.201 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:43:19.201 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 1756, 2024-05-21T15:43:19.201 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 1843, 2024-05-21T15:43:19.201 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1279150, 2024-05-21T15:43:19.201 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 2814619, 2024-05-21T15:43:19.201 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 26, 2024-05-21T15:43:19.201 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 7, 2024-05-21T15:43:19.202 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.184156634, 2024-05-21T15:43:19.202 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.056380583, 2024-05-21T15:43:19.202 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.070437616, 2024-05-21T15:43:19.202 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.045344025, 2024-05-21T15:43:19.202 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:19.202 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1842, 2024-05-21T15:43:19.202 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.224673315, 2024-05-21T15:43:19.202 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000121972 2024-05-21T15:43:19.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.202 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:19.202 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 407, 2024-05-21T15:43:19.202 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000095787, 2024-05-21T15:43:19.202 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000235 2024-05-21T15:43:19.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.203 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 596272, 2024-05-21T15:43:19.203 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 126224 2024-05-21T15:43:19.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.203 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-05-21T15:43:19.203 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2024-05-21T15:43:19.203 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-05-21T15:43:19.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.203 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2024-05-21T15:43:19.204 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-05-21T15:43:19.204 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-05-21T15:43:19.204 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2200, 2024-05-21T15:43:19.204 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.735183711, 2024-05-21T15:43:19.204 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000788719 2024-05-21T15:43:19.204 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:19.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.204 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2024-05-21T15:43:19.204 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-05-21T15:43:19.204 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-05-21T15:43:19.204 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1387, 2024-05-21T15:43:19.205 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.085483228, 2024-05-21T15:43:19.205 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000061631 2024-05-21T15:43:19.205 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:19.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.205 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2024-05-21T15:43:19.205 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 516, 2024-05-21T15:43:19.205 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 516, 2024-05-21T15:43:19.205 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2024-05-21T15:43:19.205 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 516, 2024-05-21T15:43:19.205 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.627328238, 2024-05-21T15:43:19.205 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.003153736 2024-05-21T15:43:19.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.205 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2024-05-21T15:43:19.206 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 0, 2024-05-21T15:43:19.206 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 0, 2024-05-21T15:43:19.206 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 0, 2024-05-21T15:43:19.206 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 11, 2024-05-21T15:43:19.206 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 0, 2024-05-21T15:43:19.206 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2024-05-21T15:43:19.206 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 15, 2024-05-21T15:43:19.206 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 1, 2024-05-21T15:43:19.206 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 1, 2024-05-21T15:43:19.206 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 13, 2024-05-21T15:43:19.206 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 13, 2024-05-21T15:43:19.206 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 10, 2024-05-21T15:43:19.206 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 0, 2024-05-21T15:43:19.207 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 0, 2024-05-21T15:43:19.207 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 3, 2024-05-21T15:43:19.207 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 874, 2024-05-21T15:43:19.207 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 719, 2024-05-21T15:43:19.207 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 0, 2024-05-21T15:43:19.207 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 3, 2024-05-21T15:43:19.207 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:43:19.207 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2024-05-21T15:43:19.207 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 0, 2024-05-21T15:43:19.207 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2024-05-21T15:43:19.207 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2024-05-21T15:43:19.207 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 3, 2024-05-21T15:43:19.207 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 1, 2024-05-21T15:43:19.207 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 3, 2024-05-21T15:43:19.208 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 322, 2024-05-21T15:43:19.208 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2024-05-21T15:43:19.208 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 1899, 2024-05-21T15:43:19.208 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 0, 2024-05-21T15:43:19.208 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 4, 2024-05-21T15:43:19.208 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 234, 2024-05-21T15:43:19.208 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 0, 2024-05-21T15:43:19.208 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 1899, 2024-05-21T15:43:19.208 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2024-05-21T15:43:19.208 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:43:19.208 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:43:19.208 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 462, 2024-05-21T15:43:19.208 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:43:19.208 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 462, 2024-05-21T15:43:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 1437, 2024-05-21T15:43:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 0, 2024-05-21T15:43:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 160, 2024-05-21T15:43:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 866, 2024-05-21T15:43:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:43:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:43:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:43:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 4091, 2024-05-21T15:43:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 3598, 2024-05-21T15:43:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 7, 2024-05-21T15:43:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 433 2024-05-21T15:43:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.209 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2024-05-21T15:43:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 0, 2024-05-21T15:43:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 3, 2024-05-21T15:43:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 2, 2024-05-21T15:43:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 4, 2024-05-21T15:43:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 0, 2024-05-21T15:43:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 0, 2024-05-21T15:43:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 0, 2024-05-21T15:43:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:43:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 301, 2024-05-21T15:43:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 301, 2024-05-21T15:43:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 0, 2024-05-21T15:43:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2024-05-21T15:43:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2024-05-21T15:43:19.210 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:43:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:43:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2024-05-21T15:43:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2024-05-21T15:43:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:43:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:43:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:43:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 3, 2024-05-21T15:43:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2024-05-21T15:43:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:43:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2024-05-21T15:43:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2024-05-21T15:43:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2024-05-21T15:43:19.211 INFO:teuthology.orchestra.run.smithi045.stdout: "evlrg": 0, 2024-05-21T15:43:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 4564, 2024-05-21T15:43:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 0, 2024-05-21T15:43:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 0, 2024-05-21T15:43:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 4564, 2024-05-21T15:43:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2024-05-21T15:43:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 0, 2024-05-21T15:43:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 7, 2024-05-21T15:43:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 0, 2024-05-21T15:43:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 0, 2024-05-21T15:43:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 7, 2024-05-21T15:43:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "segmjr": 2, 2024-05-21T15:43:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2024-05-21T15:43:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 0, 2024-05-21T15:43:19.212 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 4194304, 2024-05-21T15:43:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 10243958, 2024-05-21T15:43:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 0, 2024-05-21T15:43:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2024-05-21T15:43:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 279, 2024-05-21T15:43:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 44.090541581, 2024-05-21T15:43:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.158030614 2024-05-21T15:43:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 0 2024-05-21T15:43:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2024-05-21T15:43:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 19, 2024-05-21T15:43:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 325, 2024-05-21T15:43:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 306, 2024-05-21T15:43:19.213 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 18, 2024-05-21T15:43:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 102, 2024-05-21T15:43:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 84, 2024-05-21T15:43:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 15, 2024-05-21T15:43:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 647, 2024-05-21T15:43:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 632, 2024-05-21T15:43:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 0, 2024-05-21T15:43:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 393, 2024-05-21T15:43:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 393, 2024-05-21T15:43:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 131928, 2024-05-21T15:43:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 223636 2024-05-21T15:43:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2024-05-21T15:43:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 1022, 2024-05-21T15:43:19.214 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 2, 2024-05-21T15:43:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 516, 2024-05-21T15:43:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 6, 2024-05-21T15:43:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 126, 2024-05-21T15:43:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2024-05-21T15:43:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.215 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2024-05-21T15:43:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2024-05-21T15:43:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2024-05-21T15:43:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2024-05-21T15:43:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 200, 2024-05-21T15:43:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.087864706, 2024-05-21T15:43:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000439323 2024-05-21T15:43:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2024-05-21T15:43:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2024-05-21T15:43:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.217 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2024-05-21T15:43:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2024-05-21T15:43:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.218 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2024-05-21T15:43:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2024-05-21T15:43:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2024-05-21T15:43:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2024-05-21T15:43:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2024-05-21T15:43:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2024-05-21T15:43:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 5, 2024-05-21T15:43:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.003388439, 2024-05-21T15:43:19.220 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000677687 2024-05-21T15:43:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2024-05-21T15:43:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 10, 2024-05-21T15:43:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.013024607, 2024-05-21T15:43:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001302460 2024-05-21T15:43:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2024-05-21T15:43:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2024-05-21T15:43:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.221 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "req_snapdiff_latency": { 2024-05-21T15:43:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2024-05-21T15:43:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 77, 2024-05-21T15:43:19.222 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.089677211, 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.014151652 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.223 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2024-05-21T15:43:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.224 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.225 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2024-05-21T15:43:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2024-05-21T15:43:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2024-05-21T15:43:19.226 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2024-05-21T15:43:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2024-05-21T15:43:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 224, 2024-05-21T15:43:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.433373275, 2024-05-21T15:43:19.227 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001934702 2024-05-21T15:43:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:43:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:43:19.228 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2024-05-21T15:43:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.229 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2024-05-21T15:43:19.230 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 1, 2024-05-21T15:43:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 1, 2024-05-21T15:43:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 0, 2024-05-21T15:43:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 1, 2024-05-21T15:43:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2024-05-21T15:43:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 462, 2024-05-21T15:43:19.231 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 462, 2024-05-21T15:43:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 1845, 2024-05-21T15:43:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: "mdthresh_evicted": 0 2024-05-21T15:43:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-05-21T15:43:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:43:19.232 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-05-21T15:43:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:43:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:43:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:43:19.233 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:43:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:43:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:43:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:43:19.234 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:43:19.236 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:43:19.236 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:43:19.236 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:43:19.236 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:43:19.236 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:43:19.236 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-05-21T15:43:19.236 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:43:19.236 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-05-21T15:43:19.236 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:43:19.236 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:43:19.237 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:43:19.237 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:43:19.237 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:43:19.237 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:43:19.237 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:43:19.237 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-05-21T15:43:19.237 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:43:19.237 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:43:19.237 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:43:19.237 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-05-21T15:43:19.237 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-05-21T15:43:19.237 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-05-21T15:43:19.237 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:43:19.237 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:43:19.238 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:43:19.238 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:43:19.238 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 284471, 2024-05-21T15:43:19.238 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 114, 2024-05-21T15:43:19.238 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:43:19.238 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-05-21T15:43:19.238 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-05-21T15:43:19.238 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-05-21T15:43:19.238 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:43:19.238 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-05-21T15:43:19.238 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:43:19.238 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-05-21T15:43:19.239 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 7824, 2024-05-21T15:43:19.239 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 165, 2024-05-21T15:43:19.239 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:43:19.239 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:43:19.239 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-05-21T15:43:19.239 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-05-21T15:43:19.239 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 77013, 2024-05-21T15:43:19.239 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 342, 2024-05-21T15:43:19.239 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-05-21T15:43:19.239 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-05-21T15:43:19.239 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-05-21T15:43:19.239 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-05-21T15:43:19.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.240 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2024-05-21T15:43:19.240 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2024-05-21T15:43:19.240 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2024-05-21T15:43:19.240 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 3424, 2024-05-21T15:43:19.240 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 6130078, 2024-05-21T15:43:19.240 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 33, 2024-05-21T15:43:19.240 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 3424, 2024-05-21T15:43:19.240 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2024-05-21T15:43:19.240 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3424, 2024-05-21T15:43:19.240 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 222.741619136, 2024-05-21T15:43:19.240 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.065053042 2024-05-21T15:43:19.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.241 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2024-05-21T15:43:19.241 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2024-05-21T15:43:19.241 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3424, 2024-05-21T15:43:19.241 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 3564 2024-05-21T15:43:19.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.241 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 3424, 2024-05-21T15:43:19.241 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 1903, 2024-05-21T15:43:19.241 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 1521, 2024-05-21T15:43:19.241 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2024-05-21T15:43:19.241 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2024-05-21T15:43:19.241 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 0, 2024-05-21T15:43:19.241 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 10, 2024-05-21T15:43:19.241 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 4, 2024-05-21T15:43:19.242 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 283, 2024-05-21T15:43:19.242 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 29, 2024-05-21T15:43:19.242 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2024-05-21T15:43:19.242 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2024-05-21T15:43:19.242 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 0, 2024-05-21T15:43:19.242 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 0, 2024-05-21T15:43:19.242 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 1084, 2024-05-21T15:43:19.242 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2024-05-21T15:43:19.242 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2024-05-21T15:43:19.242 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2024-05-21T15:43:19.242 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 1899, 2024-05-21T15:43:19.242 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 20, 2024-05-21T15:43:19.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:43:19.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2024-05-21T15:43:19.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:43:19.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2024-05-21T15:43:19.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2024-05-21T15:43:19.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2024-05-21T15:43:19.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:43:19.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2024-05-21T15:43:19.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:43:19.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 0, 2024-05-21T15:43:19.243 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2024-05-21T15:43:19.243 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2024-05-21T15:43:19.243 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2024-05-21T15:43:19.244 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2024-05-21T15:43:19.244 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2024-05-21T15:43:19.244 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2024-05-21T15:43:19.244 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2024-05-21T15:43:19.244 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2024-05-21T15:43:19.244 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2024-05-21T15:43:19.244 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2024-05-21T15:43:19.244 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2024-05-21T15:43:19.244 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2024-05-21T15:43:19.244 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2024-05-21T15:43:19.244 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2024-05-21T15:43:19.244 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2024-05-21T15:43:19.244 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2024-05-21T15:43:19.245 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 90, 2024-05-21T15:43:19.245 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2024-05-21T15:43:19.245 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 4, 2024-05-21T15:43:19.245 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 12, 2024-05-21T15:43:19.245 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 12, 2024-05-21T15:43:19.245 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 0, 2024-05-21T15:43:19.245 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2024-05-21T15:43:19.245 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 162, 2024-05-21T15:43:19.245 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 0, 2024-05-21T15:43:19.245 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 73 2024-05-21T15:43:19.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.245 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2024-05-21T15:43:19.245 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 1, 2024-05-21T15:43:19.246 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:43:19.246 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2024-05-21T15:43:19.246 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2024-05-21T15:43:19.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.246 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2024-05-21T15:43:19.246 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 0 2024-05-21T15:43:19.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.247 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2024-05-21T15:43:19.247 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2024-05-21T15:43:19.247 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 51, 2024-05-21T15:43:19.247 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2024-05-21T15:43:19.247 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 6, 2024-05-21T15:43:19.247 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed_ops": 1071, 2024-05-21T15:43:19.247 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 301, 2024-05-21T15:43:19.247 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2024-05-21T15:43:19.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.247 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:43:19.247 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:19.247 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-05-21T15:43:19.247 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:19.248 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 10807, 2024-05-21T15:43:19.248 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 4067760, 2024-05-21T15:43:19.248 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:19.248 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 10807, 2024-05-21T15:43:19.248 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-05-21T15:43:19.248 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-05-21T15:43:19.248 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 10807, 2024-05-21T15:43:19.248 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 4067760, 2024-05-21T15:43:19.248 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:19.248 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.248 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.248 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.248 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:19.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.249 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2024-05-21T15:43:19.249 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:19.249 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-05-21T15:43:19.249 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:19.249 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-05-21T15:43:19.249 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-05-21T15:43:19.249 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:19.249 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:19.249 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 3424, 2024-05-21T15:43:19.249 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 6160479, 2024-05-21T15:43:19.249 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 3424, 2024-05-21T15:43:19.249 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 6160479, 2024-05-21T15:43:19.249 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:19.250 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.250 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.250 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.250 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:19.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.250 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2024-05-21T15:43:19.250 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:19.250 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2024-05-21T15:43:19.251 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:19.251 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-05-21T15:43:19.251 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-05-21T15:43:19.251 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:19.251 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:19.251 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 3424, 2024-05-21T15:43:19.251 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 3424, 2024-05-21T15:43:19.251 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 3424, 2024-05-21T15:43:19.251 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 3424, 2024-05-21T15:43:19.251 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:19.251 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.251 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.251 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:19.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.252 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:43:19.252 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:19.252 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-05-21T15:43:19.252 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:19.252 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 301, 2024-05-21T15:43:19.252 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 30401, 2024-05-21T15:43:19.252 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:19.252 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 301, 2024-05-21T15:43:19.252 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-05-21T15:43:19.252 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-05-21T15:43:19.252 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 4, 2024-05-21T15:43:19.252 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 30401, 2024-05-21T15:43:19.253 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:19.253 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.253 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.253 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.253 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:19.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:19.253 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x555722b7eb60": { 2024-05-21T15:43:19.253 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:19.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-05-21T15:43:19.253 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:19.253 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 4564, 2024-05-21T15:43:19.253 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 6049654, 2024-05-21T15:43:19.253 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:19.254 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 4564, 2024-05-21T15:43:19.254 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-05-21T15:43:19.254 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-05-21T15:43:19.254 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 279, 2024-05-21T15:43:19.254 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 6049654, 2024-05-21T15:43:19.254 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:19.254 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:19.254 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:19.254 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:19.254 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:19.254 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:19.254 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-21T15:43:19.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:19 smithi053 ceph-mon[36496]: pgmap v618: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:43:20.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:19 smithi045 ceph-mon[36409]: pgmap v618: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:43:20.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:19 smithi038 ceph-mon[31995]: pgmap v618: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:43:20.328 INFO:tasks.check_counter:Daemon mds.e mds.exported=3 2024-05-21T15:43:20.328 INFO:tasks.check_counter:Daemon mds.e mds.imported=3 2024-05-21T15:43:20.329 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-05-21T15:43:20.329 DEBUG:tasks.check_counter:Getting stats from h 2024-05-21T15:43:20.329 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-21T15:43:20.593 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:43:21.847 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:21 smithi045 ceph-mon[36409]: pgmap v619: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:21.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:21 smithi053 ceph-mon[36496]: pgmap v619: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:22.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:21 smithi038 ceph-mon[31995]: pgmap v619: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:22.492 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-21T15:43:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:43:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 3212, 2024-05-21T15:43:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2579, 2024-05-21T15:43:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1914734, 2024-05-21T15:43:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 6965136, 2024-05-21T15:43:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 10, 2024-05-21T15:43:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 4, 2024-05-21T15:43:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.324783429, 2024-05-21T15:43:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.127335364, 2024-05-21T15:43:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.179150300, 2024-05-21T15:43:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.047448984, 2024-05-21T15:43:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2578, 2024-05-21T15:43:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.201781710, 2024-05-21T15:43:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000078270 2024-05-21T15:43:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3621, 2024-05-21T15:43:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000875143, 2024-05-21T15:43:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000241 2024-05-21T15:43:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 611264, 2024-05-21T15:43:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 125424 2024-05-21T15:43:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:43:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 5513, 2024-05-21T15:43:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 6008, 2024-05-21T15:43:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2267842, 2024-05-21T15:43:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 2826602, 2024-05-21T15:43:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 15, 2024-05-21T15:43:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 6, 2024-05-21T15:43:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.524092073, 2024-05-21T15:43:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.179825832, 2024-05-21T15:43:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.490361518, 2024-05-21T15:43:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.017709163, 2024-05-21T15:43:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 6007, 2024-05-21T15:43:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.258060678, 2024-05-21T15:43:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000209432 2024-05-21T15:43:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 8602, 2024-05-21T15:43:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.004475809, 2024-05-21T15:43:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000520 2024-05-21T15:43:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1980096, 2024-05-21T15:43:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 2543280 2024-05-21T15:43:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:43:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 721, 2024-05-21T15:43:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 728, 2024-05-21T15:43:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 325998, 2024-05-21T15:43:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 639233, 2024-05-21T15:43:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 35, 2024-05-21T15:43:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 6, 2024-05-21T15:43:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.085528886, 2024-05-21T15:43:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.019081888, 2024-05-21T15:43:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.034494935, 2024-05-21T15:43:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.025247682, 2024-05-21T15:43:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 726, 2024-05-21T15:43:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.098586974, 2024-05-21T15:43:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000135794 2024-05-21T15:43:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 112, 2024-05-21T15:43:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000036459, 2024-05-21T15:43:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000325 2024-05-21T15:43:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T15:43:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T15:43:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-05-21T15:43:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2024-05-21T15:43:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-05-21T15:43:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2024-05-21T15:43:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-05-21T15:43:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-05-21T15:43:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1275, 2024-05-21T15:43:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.673439899, 2024-05-21T15:43:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001312501 2024-05-21T15:43:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2024-05-21T15:43:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-05-21T15:43:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-05-21T15:43:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 813, 2024-05-21T15:43:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.030601269, 2024-05-21T15:43:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000037639 2024-05-21T15:43:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2024-05-21T15:43:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 171, 2024-05-21T15:43:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 163, 2024-05-21T15:43:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2024-05-21T15:43:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 163, 2024-05-21T15:43:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 49.546460323, 2024-05-21T15:43:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.303966014 2024-05-21T15:43:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2024-05-21T15:43:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 8, 2024-05-21T15:43:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 0, 2024-05-21T15:43:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 0, 2024-05-21T15:43:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 11, 2024-05-21T15:43:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 0, 2024-05-21T15:43:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2024-05-21T15:43:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 16, 2024-05-21T15:43:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 2, 2024-05-21T15:43:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 1, 2024-05-21T15:43:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 13, 2024-05-21T15:43:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 13, 2024-05-21T15:43:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 223, 2024-05-21T15:43:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 1, 2024-05-21T15:43:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 1, 2024-05-21T15:43:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 4, 2024-05-21T15:43:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 332, 2024-05-21T15:43:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 285, 2024-05-21T15:43:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 7, 2024-05-21T15:43:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 7, 2024-05-21T15:43:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:43:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2024-05-21T15:43:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 0, 2024-05-21T15:43:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2024-05-21T15:43:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2024-05-21T15:43:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 6, 2024-05-21T15:43:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 202, 2024-05-21T15:43:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 6, 2024-05-21T15:43:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 233, 2024-05-21T15:43:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2024-05-21T15:43:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 616, 2024-05-21T15:43:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 1, 2024-05-21T15:43:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 4, 2024-05-21T15:43:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 234, 2024-05-21T15:43:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 0, 2024-05-21T15:43:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 616, 2024-05-21T15:43:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2024-05-21T15:43:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:43:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:43:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 146, 2024-05-21T15:43:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:43:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 146, 2024-05-21T15:43:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 470, 2024-05-21T15:43:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 12, 2024-05-21T15:43:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 118, 2024-05-21T15:43:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 277, 2024-05-21T15:43:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:43:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:43:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:43:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 4623, 2024-05-21T15:43:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 4215, 2024-05-21T15:43:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 10, 2024-05-21T15:43:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 52 2024-05-21T15:43:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2024-05-21T15:43:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 0, 2024-05-21T15:43:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 3, 2024-05-21T15:43:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 2, 2024-05-21T15:43:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 9, 2024-05-21T15:43:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 0, 2024-05-21T15:43:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 0, 2024-05-21T15:43:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 0, 2024-05-21T15:43:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:43:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 28, 2024-05-21T15:43:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 28, 2024-05-21T15:43:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 0, 2024-05-21T15:43:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2024-05-21T15:43:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2024-05-21T15:43:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:43:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:43:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2024-05-21T15:43:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2024-05-21T15:43:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:43:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:43:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:43:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 6, 2024-05-21T15:43:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2024-05-21T15:43:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:43:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2024-05-21T15:43:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2024-05-21T15:43:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2024-05-21T15:43:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "evlrg": 0, 2024-05-21T15:43:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 4645, 2024-05-21T15:43:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 0, 2024-05-21T15:43:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 0, 2024-05-21T15:43:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 4645, 2024-05-21T15:43:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2024-05-21T15:43:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 0, 2024-05-21T15:43:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 7, 2024-05-21T15:43:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 0, 2024-05-21T15:43:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 0, 2024-05-21T15:43:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 7, 2024-05-21T15:43:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "segmjr": 2, 2024-05-21T15:43:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2024-05-21T15:43:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 0, 2024-05-21T15:43:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 4194304, 2024-05-21T15:43:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 10043782, 2024-05-21T15:43:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 0, 2024-05-21T15:43:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2024-05-21T15:43:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 276, 2024-05-21T15:43:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 15.330610287, 2024-05-21T15:43:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.055545689 2024-05-21T15:43:22.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.508 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 0 2024-05-21T15:43:22.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.508 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2024-05-21T15:43:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 21, 2024-05-21T15:43:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 241, 2024-05-21T15:43:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 220, 2024-05-21T15:43:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 20, 2024-05-21T15:43:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 104, 2024-05-21T15:43:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 84, 2024-05-21T15:43:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 16, 2024-05-21T15:43:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 295, 2024-05-21T15:43:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 279, 2024-05-21T15:43:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 1, 2024-05-21T15:43:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 208, 2024-05-21T15:43:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 207, 2024-05-21T15:43:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 128144, 2024-05-21T15:43:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 223636 2024-05-21T15:43:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2024-05-21T15:43:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 256, 2024-05-21T15:43:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 7, 2024-05-21T15:43:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 171, 2024-05-21T15:43:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 8, 2024-05-21T15:43:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 48, 2024-05-21T15:43:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2024-05-21T15:43:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2024-05-21T15:43:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 47, 2024-05-21T15:43:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 49.392277186, 2024-05-21T15:43:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 1.050899514 2024-05-21T15:43:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2024-05-21T15:43:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2024-05-21T15:43:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2024-05-21T15:43:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 79, 2024-05-21T15:43:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.085140059, 2024-05-21T15:43:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001077722 2024-05-21T15:43:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2024-05-21T15:43:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2024-05-21T15:43:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-05-21T15:43:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000126565, 2024-05-21T15:43:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000126565 2024-05-21T15:43:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2024-05-21T15:43:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2024-05-21T15:43:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2024-05-21T15:43:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2024-05-21T15:43:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2024-05-21T15:43:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2024-05-21T15:43:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2024-05-21T15:43:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2024-05-21T15:43:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 6, 2024-05-21T15:43:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.008362907, 2024-05-21T15:43:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001393817 2024-05-21T15:43:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2024-05-21T15:43:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2, 2024-05-21T15:43:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.001549848, 2024-05-21T15:43:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000774924 2024-05-21T15:43:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2024-05-21T15:43:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2024-05-21T15:43:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "req_snapdiff_latency": { 2024-05-21T15:43:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2024-05-21T15:43:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2024-05-21T15:43:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2024-05-21T15:43:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2024-05-21T15:43:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2024-05-21T15:43:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2024-05-21T15:43:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2024-05-21T15:43:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2024-05-21T15:43:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2024-05-21T15:43:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2024-05-21T15:43:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 28, 2024-05-21T15:43:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.059003758, 2024-05-21T15:43:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.002107277 2024-05-21T15:43:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:43:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:43:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2024-05-21T15:43:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2024-05-21T15:43:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 1, 2024-05-21T15:43:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 1, 2024-05-21T15:43:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 0, 2024-05-21T15:43:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 1, 2024-05-21T15:43:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2024-05-21T15:43:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 70, 2024-05-21T15:43:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 70, 2024-05-21T15:43:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 518, 2024-05-21T15:43:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "mdthresh_evicted": 0 2024-05-21T15:43:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-05-21T15:43:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:43:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-05-21T15:43:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:43:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:43:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:43:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:43:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:43:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:43:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:43:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:43:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:43:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:43:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:43:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:43:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:43:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-05-21T15:43:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:43:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-05-21T15:43:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:43:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:43:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:43:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:43:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:43:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:43:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:43:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-05-21T15:43:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:43:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:43:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:43:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-05-21T15:43:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-05-21T15:43:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-05-21T15:43:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:43:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:43:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:43:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:43:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 306855, 2024-05-21T15:43:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 124, 2024-05-21T15:43:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:43:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-05-21T15:43:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-05-21T15:43:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-05-21T15:43:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:43:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-05-21T15:43:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:43:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-05-21T15:43:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 7824, 2024-05-21T15:43:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 165, 2024-05-21T15:43:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:43:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:43:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-05-21T15:43:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-05-21T15:43:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 84741, 2024-05-21T15:43:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 350, 2024-05-21T15:43:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-05-21T15:43:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-05-21T15:43:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-05-21T15:43:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-05-21T15:43:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2024-05-21T15:43:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2024-05-21T15:43:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2024-05-21T15:43:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 1826, 2024-05-21T15:43:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 5897241, 2024-05-21T15:43:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 33, 2024-05-21T15:43:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 1826, 2024-05-21T15:43:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2024-05-21T15:43:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1826, 2024-05-21T15:43:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 432.042418055, 2024-05-21T15:43:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.236605924 2024-05-21T15:43:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2024-05-21T15:43:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2024-05-21T15:43:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1826, 2024-05-21T15:43:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1965 2024-05-21T15:43:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 1826, 2024-05-21T15:43:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 618, 2024-05-21T15:43:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 1208, 2024-05-21T15:43:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2024-05-21T15:43:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2024-05-21T15:43:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 0, 2024-05-21T15:43:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 10, 2024-05-21T15:43:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 2, 2024-05-21T15:43:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 278, 2024-05-21T15:43:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 31, 2024-05-21T15:43:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2024-05-21T15:43:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2024-05-21T15:43:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 0, 2024-05-21T15:43:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 0, 2024-05-21T15:43:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 765, 2024-05-21T15:43:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2024-05-21T15:43:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2024-05-21T15:43:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2024-05-21T15:43:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 616, 2024-05-21T15:43:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 20, 2024-05-21T15:43:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:43:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2024-05-21T15:43:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:43:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2024-05-21T15:43:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2024-05-21T15:43:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2024-05-21T15:43:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:43:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2024-05-21T15:43:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:43:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 0, 2024-05-21T15:43:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2024-05-21T15:43:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2024-05-21T15:43:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2024-05-21T15:43:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2024-05-21T15:43:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2024-05-21T15:43:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2024-05-21T15:43:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2024-05-21T15:43:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2024-05-21T15:43:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2024-05-21T15:43:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2024-05-21T15:43:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2024-05-21T15:43:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2024-05-21T15:43:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2024-05-21T15:43:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2024-05-21T15:43:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2024-05-21T15:43:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2024-05-21T15:43:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 90, 2024-05-21T15:43:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2024-05-21T15:43:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 4, 2024-05-21T15:43:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 12, 2024-05-21T15:43:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 12, 2024-05-21T15:43:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 0, 2024-05-21T15:43:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2024-05-21T15:43:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 168, 2024-05-21T15:43:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 0, 2024-05-21T15:43:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 75 2024-05-21T15:43:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2024-05-21T15:43:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 1, 2024-05-21T15:43:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:43:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2024-05-21T15:43:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2024-05-21T15:43:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2024-05-21T15:43:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 1 2024-05-21T15:43:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2024-05-21T15:43:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2024-05-21T15:43:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 51, 2024-05-21T15:43:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2024-05-21T15:43:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 1, 2024-05-21T15:43:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed_ops": 752, 2024-05-21T15:43:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 28, 2024-05-21T15:43:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2024-05-21T15:43:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:43:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-05-21T15:43:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 9442, 2024-05-21T15:43:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 3395968, 2024-05-21T15:43:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 9442, 2024-05-21T15:43:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-05-21T15:43:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-05-21T15:43:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 9442, 2024-05-21T15:43:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 3395968, 2024-05-21T15:43:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2024-05-21T15:43:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-05-21T15:43:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-05-21T15:43:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-05-21T15:43:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 1826, 2024-05-21T15:43:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 5900069, 2024-05-21T15:43:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 1826, 2024-05-21T15:43:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 5900069, 2024-05-21T15:43:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2024-05-21T15:43:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2024-05-21T15:43:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-05-21T15:43:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-05-21T15:43:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 1826, 2024-05-21T15:43:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 1826, 2024-05-21T15:43:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 1826, 2024-05-21T15:43:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1826, 2024-05-21T15:43:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:43:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-05-21T15:43:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 28, 2024-05-21T15:43:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 2828, 2024-05-21T15:43:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 28, 2024-05-21T15:43:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-05-21T15:43:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-05-21T15:43:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 2, 2024-05-21T15:43:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 2828, 2024-05-21T15:43:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x5582c6fb6b60": { 2024-05-21T15:43:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-05-21T15:43:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 4645, 2024-05-21T15:43:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 5849478, 2024-05-21T15:43:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 4645, 2024-05-21T15:43:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-05-21T15:43:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-05-21T15:43:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 276, 2024-05-21T15:43:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 5849478, 2024-05-21T15:43:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:22.543 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-21T15:43:23.298 INFO:tasks.check_counter:Daemon mds.h mds.exported=6 2024-05-21T15:43:23.299 INFO:tasks.check_counter:Daemon mds.h mds.imported=6 2024-05-21T15:43:23.299 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-05-21T15:43:23.299 DEBUG:tasks.check_counter:Getting stats from k 2024-05-21T15:43:23.299 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-21T15:43:23.550 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:43:23.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:23 smithi053 ceph-mon[36496]: pgmap v620: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:23.972 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:23 smithi045 ceph-mon[36409]: pgmap v620: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:24.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:23 smithi038 ceph-mon[31995]: pgmap v620: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-21T15:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:43:25.414 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 1593, 2024-05-21T15:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 1595, 2024-05-21T15:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 6308277, 2024-05-21T15:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 506581, 2024-05-21T15:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2024-05-21T15:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2024-05-21T15:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.161191661, 2024-05-21T15:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.042943809, 2024-05-21T15:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.045629022, 2024-05-21T15:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.060025596, 2024-05-21T15:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1593, 2024-05-21T15:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.086060019, 2024-05-21T15:43:25.415 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000054023 2024-05-21T15:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T15:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T15:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 309, 2024-05-21T15:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 447, 2024-05-21T15:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 78000, 2024-05-21T15:43:25.416 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 417390, 2024-05-21T15:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-05-21T15:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 4, 2024-05-21T15:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.037096523, 2024-05-21T15:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.011978928, 2024-05-21T15:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.009005355, 2024-05-21T15:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.012678729, 2024-05-21T15:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 446, 2024-05-21T15:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.025758700, 2024-05-21T15:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000057754 2024-05-21T15:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:25.417 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-21T15:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 320656 2024-05-21T15:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 212, 2024-05-21T15:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 195, 2024-05-21T15:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 581180, 2024-05-21T15:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 115342, 2024-05-21T15:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 3, 2024-05-21T15:43:25.418 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2024-05-21T15:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.033041764, 2024-05-21T15:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.011143008, 2024-05-21T15:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.018301579, 2024-05-21T15:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.005168850, 2024-05-21T15:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 194, 2024-05-21T15:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.009157642, 2024-05-21T15:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000047204 2024-05-21T15:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 580912, 2024-05-21T15:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 115024 2024-05-21T15:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-05-21T15:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2024-05-21T15:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-05-21T15:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2024-05-21T15:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-05-21T15:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-05-21T15:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3221, 2024-05-21T15:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.088558592, 2024-05-21T15:43:25.420 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000027494 2024-05-21T15:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2024-05-21T15:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-05-21T15:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-05-21T15:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2024-05-21T15:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 0, 2024-05-21T15:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 0, 2024-05-21T15:43:25.421 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2024-05-21T15:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2024-05-21T15:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 0, 2024-05-21T15:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 0, 2024-05-21T15:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 0, 2024-05-21T15:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 0, 2024-05-21T15:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 0, 2024-05-21T15:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2024-05-21T15:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 29, 2024-05-21T15:43:25.422 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 0, 2024-05-21T15:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 29, 2024-05-21T15:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 0, 2024-05-21T15:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 29, 2024-05-21T15:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 0, 2024-05-21T15:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 0, 2024-05-21T15:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 0, 2024-05-21T15:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 1, 2024-05-21T15:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 0, 2024-05-21T15:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 0, 2024-05-21T15:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 0, 2024-05-21T15:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 0, 2024-05-21T15:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:43:25.423 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2024-05-21T15:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 0, 2024-05-21T15:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2024-05-21T15:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2024-05-21T15:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 0, 2024-05-21T15:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 0, 2024-05-21T15:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 0, 2024-05-21T15:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 0, 2024-05-21T15:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2024-05-21T15:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 0, 2024-05-21T15:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 0, 2024-05-21T15:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 0, 2024-05-21T15:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 0, 2024-05-21T15:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 0, 2024-05-21T15:43:25.424 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T15:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2024-05-21T15:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 0, 2024-05-21T15:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 0, 2024-05-21T15:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T15:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 0, 2024-05-21T15:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:43:25.425 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 0, 2024-05-21T15:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 0, 2024-05-21T15:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 0, 2024-05-21T15:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 0 2024-05-21T15:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2024-05-21T15:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 0, 2024-05-21T15:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 0, 2024-05-21T15:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 0, 2024-05-21T15:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 0, 2024-05-21T15:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 0, 2024-05-21T15:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 0, 2024-05-21T15:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 0, 2024-05-21T15:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:43:25.426 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 0, 2024-05-21T15:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 0, 2024-05-21T15:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 0, 2024-05-21T15:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2024-05-21T15:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2024-05-21T15:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2024-05-21T15:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2024-05-21T15:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 0, 2024-05-21T15:43:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "evlrg": 0, 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 0, 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 0, 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 0, 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 4645, 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 0, 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 0, 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 0, 2024-05-21T15:43:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 0, 2024-05-21T15:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 7, 2024-05-21T15:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "segmjr": 2, 2024-05-21T15:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2024-05-21T15:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 0, 2024-05-21T15:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 4194304, 2024-05-21T15:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 10043782, 2024-05-21T15:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 10042131, 2024-05-21T15:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2024-05-21T15:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.429 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 4645 2024-05-21T15:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2024-05-21T15:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 4, 2024-05-21T15:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 231, 2024-05-21T15:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 227, 2024-05-21T15:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 2, 2024-05-21T15:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 93, 2024-05-21T15:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 91, 2024-05-21T15:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 29, 2024-05-21T15:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 279, 2024-05-21T15:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 250, 2024-05-21T15:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 0, 2024-05-21T15:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 0, 2024-05-21T15:43:25.430 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 0, 2024-05-21T15:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 121964, 2024-05-21T15:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 215444 2024-05-21T15:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2024-05-21T15:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 0, 2024-05-21T15:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 0, 2024-05-21T15:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 0, 2024-05-21T15:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 0, 2024-05-21T15:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 0, 2024-05-21T15:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2024-05-21T15:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.431 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2024-05-21T15:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2024-05-21T15:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2024-05-21T15:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.432 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2024-05-21T15:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2024-05-21T15:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.433 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2024-05-21T15:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2024-05-21T15:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2024-05-21T15:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2024-05-21T15:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2024-05-21T15:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.435 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2024-05-21T15:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2024-05-21T15:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.436 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2024-05-21T15:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2024-05-21T15:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.437 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2024-05-21T15:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2024-05-21T15:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2024-05-21T15:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.438 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "req_snapdiff_latency": { 2024-05-21T15:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2024-05-21T15:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2024-05-21T15:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.439 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2024-05-21T15:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2024-05-21T15:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.440 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2024-05-21T15:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2024-05-21T15:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2024-05-21T15:43:25.441 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2024-05-21T15:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2024-05-21T15:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2024-05-21T15:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2024-05-21T15:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2024-05-21T15:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 1, 2024-05-21T15:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 1, 2024-05-21T15:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 0, 2024-05-21T15:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 1, 2024-05-21T15:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2024-05-21T15:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 0, 2024-05-21T15:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 0, 2024-05-21T15:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 1851, 2024-05-21T15:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mdthresh_evicted": 0 2024-05-21T15:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-05-21T15:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-05-21T15:43:25.444 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:43:25.445 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-05-21T15:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-05-21T15:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-05-21T15:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:43:25.446 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-05-21T15:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-05-21T15:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-05-21T15:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 190184, 2024-05-21T15:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 53, 2024-05-21T15:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-05-21T15:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-05-21T15:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-05-21T15:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:43:25.447 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-05-21T15:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-05-21T15:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 7824, 2024-05-21T15:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 165, 2024-05-21T15:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-05-21T15:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-05-21T15:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 32400, 2024-05-21T15:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 144, 2024-05-21T15:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-05-21T15:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-05-21T15:43:25.448 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-05-21T15:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-05-21T15:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2024-05-21T15:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2024-05-21T15:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2024-05-21T15:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 1898, 2024-05-21T15:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 21, 2024-05-21T15:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 3, 2024-05-21T15:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 1898, 2024-05-21T15:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2024-05-21T15:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1898, 2024-05-21T15:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 22.208658787, 2024-05-21T15:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.011701084 2024-05-21T15:43:25.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2024-05-21T15:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2024-05-21T15:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1898, 2024-05-21T15:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1899 2024-05-21T15:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 1898, 2024-05-21T15:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 1898, 2024-05-21T15:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 0, 2024-05-21T15:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2024-05-21T15:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2024-05-21T15:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 1127, 2024-05-21T15:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 0, 2024-05-21T15:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 769, 2024-05-21T15:43:25.450 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 0, 2024-05-21T15:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 0, 2024-05-21T15:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2024-05-21T15:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2024-05-21T15:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 0, 2024-05-21T15:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 0, 2024-05-21T15:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 0, 2024-05-21T15:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2024-05-21T15:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2024-05-21T15:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2024-05-21T15:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 1, 2024-05-21T15:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 0, 2024-05-21T15:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2024-05-21T15:43:25.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2024-05-21T15:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2024-05-21T15:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2024-05-21T15:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2024-05-21T15:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 0, 2024-05-21T15:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2024-05-21T15:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2024-05-21T15:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2024-05-21T15:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2024-05-21T15:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2024-05-21T15:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2024-05-21T15:43:25.452 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2024-05-21T15:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2024-05-21T15:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2024-05-21T15:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2024-05-21T15:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2024-05-21T15:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2024-05-21T15:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2024-05-21T15:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2024-05-21T15:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2024-05-21T15:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2024-05-21T15:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 90, 2024-05-21T15:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2024-05-21T15:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 4, 2024-05-21T15:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 5, 2024-05-21T15:43:25.453 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 5, 2024-05-21T15:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 0, 2024-05-21T15:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2024-05-21T15:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 0, 2024-05-21T15:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 2, 2024-05-21T15:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 0 2024-05-21T15:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2024-05-21T15:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 0, 2024-05-21T15:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2024-05-21T15:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2024-05-21T15:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.454 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2024-05-21T15:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 0 2024-05-21T15:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2024-05-21T15:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2024-05-21T15:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T15:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2024-05-21T15:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 0, 2024-05-21T15:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed_ops": 0, 2024-05-21T15:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 0, 2024-05-21T15:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2024-05-21T15:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:25.455 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 2110, 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 6763811, 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 2110, 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 2110, 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 6763811, 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:25.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2024-05-21T15:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-05-21T15:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-05-21T15:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-05-21T15:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 1898, 2024-05-21T15:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 5849483, 2024-05-21T15:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 1898, 2024-05-21T15:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 5849483, 2024-05-21T15:43:25.457 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2024-05-21T15:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2024-05-21T15:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-05-21T15:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-05-21T15:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:25.458 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 1898, 2024-05-21T15:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 1898, 2024-05-21T15:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 1898, 2024-05-21T15:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1898, 2024-05-21T15:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-05-21T15:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:25.459 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-05-21T15:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-05-21T15:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-05-21T15:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-05-21T15:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 0, 2024-05-21T15:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 0, 2024-05-21T15:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-21T15:43:25.460 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x556d7d2bcb60": { 2024-05-21T15:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-05-21T15:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-05-21T15:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-05-21T15:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-05-21T15:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-05-21T15:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-05-21T15:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-05-21T15:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 0, 2024-05-21T15:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 0, 2024-05-21T15:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-05-21T15:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-05-21T15:43:25.461 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-05-21T15:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-05-21T15:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-21T15:43:25.462 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-21T15:43:25.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:25 smithi053 ceph-mon[36496]: pgmap v621: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:26.006 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:25 smithi045 ceph-mon[36409]: pgmap v621: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:26.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:25 smithi038 ceph-mon[31995]: pgmap v621: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T15:43:26.218 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-21T15:43:26.218 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-21T15:43:26.218 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-05-21T15:43:26.218 DEBUG:tasks.check_counter:Getting stats from c 2024-05-21T15:43:26.218 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-21T15:43:26.518 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:43:27.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:27 smithi053 ceph-mon[36496]: pgmap v622: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:28.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:27 smithi045 ceph-mon[36409]: pgmap v622: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:28.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:27 smithi038 ceph-mon[31995]: pgmap v622: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:43:29.082 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-21T15:43:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:43:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 4884, 2024-05-21T15:43:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 5134, 2024-05-21T15:43:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1950342, 2024-05-21T15:43:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 6453647, 2024-05-21T15:43:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 20, 2024-05-21T15:43:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-05-21T15:43:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.485660661, 2024-05-21T15:43:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.160998234, 2024-05-21T15:43:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.437323258, 2024-05-21T15:43:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.024755568, 2024-05-21T15:43:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5133, 2024-05-21T15:43:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.963448725, 2024-05-21T15:43:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000382514 2024-05-21T15:43:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 7458, 2024-05-21T15:43:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.006045471, 2024-05-21T15:43:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000810 2024-05-21T15:43:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1706864, 2024-05-21T15:43:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 1885008 2024-05-21T15:43:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:43:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 921, 2024-05-21T15:43:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 899, 2024-05-21T15:43:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 864071, 2024-05-21T15:43:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 1271595, 2024-05-21T15:43:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 13, 2024-05-21T15:43:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2024-05-21T15:43:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.109109013, 2024-05-21T15:43:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.032630019, 2024-05-21T15:43:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.046059849, 2024-05-21T15:43:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.028244281, 2024-05-21T15:43:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 900, 2024-05-21T15:43:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.108419293, 2024-05-21T15:43:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000120465 2024-05-21T15:43:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 87, 2024-05-21T15:43:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000031837, 2024-05-21T15:43:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000365 2024-05-21T15:43:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 603232, 2024-05-21T15:43:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 127328 2024-05-21T15:43:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:43:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2408, 2024-05-21T15:43:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 2556, 2024-05-21T15:43:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 844864, 2024-05-21T15:43:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 1385672, 2024-05-21T15:43:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 28, 2024-05-21T15:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 9, 2024-05-21T15:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.213193803, 2024-05-21T15:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.072863913, 2024-05-21T15:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.132649872, 2024-05-21T15:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.017604776, 2024-05-21T15:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2552, 2024-05-21T15:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.186921214, 2024-05-21T15:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000073244 2024-05-21T15:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3832, 2024-05-21T15:43:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.001194930, 2024-05-21T15:43:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000311 2024-05-21T15:43:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-05-21T15:43:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 323024 2024-05-21T15:43:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-05-21T15:43:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-05-21T15:43:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-05-21T15:43:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-05-21T15:43:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-21T15:43:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-21T15:43:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 819, 2024-05-21T15:43:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.179233348, 2024-05-21T15:43:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001439845 2024-05-21T15:43:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-05-21T15:43:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-21T15:43:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-21T15:43:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 858, 2024-05-21T15:43:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.033093361, 2024-05-21T15:43:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000038570 2024-05-21T15:43:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-05-21T15:43:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 130, 2024-05-21T15:43:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 123, 2024-05-21T15:43:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-05-21T15:43:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 123, 2024-05-21T15:43:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.130663017, 2024-05-21T15:43:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001062300 2024-05-21T15:43:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-05-21T15:43:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 7, 2024-05-21T15:43:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-05-21T15:43:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-05-21T15:43:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 11, 2024-05-21T15:43:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-05-21T15:43:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-05-21T15:43:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 15, 2024-05-21T15:43:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 0, 2024-05-21T15:43:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 15, 2024-05-21T15:43:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-05-21T15:43:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 13, 2024-05-21T15:43:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 10, 2024-05-21T15:43:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-05-21T15:43:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-05-21T15:43:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 3, 2024-05-21T15:43:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 275, 2024-05-21T15:43:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 211, 2024-05-21T15:43:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 7, 2024-05-21T15:43:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 2, 2024-05-21T15:43:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:43:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-05-21T15:43:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-05-21T15:43:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-05-21T15:43:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-05-21T15:43:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 2, 2024-05-21T15:43:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-05-21T15:43:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 2, 2024-05-21T15:43:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 31, 2024-05-21T15:43:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-05-21T15:43:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 279, 2024-05-21T15:43:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-05-21T15:43:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 4, 2024-05-21T15:43:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 234, 2024-05-21T15:43:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-05-21T15:43:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 279, 2024-05-21T15:43:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-05-21T15:43:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:43:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:43:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:43:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:43:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:43:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 279, 2024-05-21T15:43:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-05-21T15:43:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 5, 2024-05-21T15:43:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 539, 2024-05-21T15:43:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:43:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:43:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:43:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 4094, 2024-05-21T15:43:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 3737, 2024-05-21T15:43:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 5, 2024-05-21T15:43:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 66 2024-05-21T15:43:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-05-21T15:43:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-05-21T15:43:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 3, 2024-05-21T15:43:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 2, 2024-05-21T15:43:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 4, 2024-05-21T15:43:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-05-21T15:43:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-05-21T15:43:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-05-21T15:43:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:43:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 31, 2024-05-21T15:43:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 31, 2024-05-21T15:43:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-05-21T15:43:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-05-21T15:43:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-05-21T15:43:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:43:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:43:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-05-21T15:43:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-05-21T15:43:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:43:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:43:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:43:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 2, 2024-05-21T15:43:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-05-21T15:43:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:43:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-05-21T15:43:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-05-21T15:43:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-05-21T15:43:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 0, 2024-05-21T15:43:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 4162, 2024-05-21T15:43:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-05-21T15:43:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-05-21T15:43:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 4162, 2024-05-21T15:43:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-05-21T15:43:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-05-21T15:43:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 6, 2024-05-21T15:43:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-05-21T15:43:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-05-21T15:43:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 6, 2024-05-21T15:43:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 2, 2024-05-21T15:43:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-05-21T15:43:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-05-21T15:43:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-05-21T15:43:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 9216231, 2024-05-21T15:43:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 0, 2024-05-21T15:43:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-05-21T15:43:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 202, 2024-05-21T15:43:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 7.288365915, 2024-05-21T15:43:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.036081019 2024-05-21T15:43:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 0 2024-05-21T15:43:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-05-21T15:43:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 19, 2024-05-21T15:43:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 55, 2024-05-21T15:43:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 36, 2024-05-21T15:43:29.098 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 18, 2024-05-21T15:43:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 25, 2024-05-21T15:43:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 7, 2024-05-21T15:43:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 15, 2024-05-21T15:43:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 87, 2024-05-21T15:43:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 72, 2024-05-21T15:43:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-05-21T15:43:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 37, 2024-05-21T15:43:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 37, 2024-05-21T15:43:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 127412, 2024-05-21T15:43:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 223636 2024-05-21T15:43:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-05-21T15:43:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 233, 2024-05-21T15:43:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 2, 2024-05-21T15:43:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 130, 2024-05-21T15:43:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 10, 2024-05-21T15:43:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 37, 2024-05-21T15:43:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-05-21T15:43:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-05-21T15:43:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-05-21T15:43:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-05-21T15:43:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-05-21T15:43:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 85, 2024-05-21T15:43:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.045262581, 2024-05-21T15:43:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000532500 2024-05-21T15:43:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-05-21T15:43:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-05-21T15:43:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-05-21T15:43:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-05-21T15:43:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-05-21T15:43:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-05-21T15:43:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-05-21T15:43:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-05-21T15:43:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-05-21T15:43:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-05-21T15:43:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3, 2024-05-21T15:43:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.004069767, 2024-05-21T15:43:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001356589 2024-05-21T15:43:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-05-21T15:43:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4, 2024-05-21T15:43:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002638424, 2024-05-21T15:43:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000659606 2024-05-21T15:43:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-05-21T15:43:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-05-21T15:43:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-05-21T15:43:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-05-21T15:43:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-05-21T15:43:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-05-21T15:43:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-05-21T15:43:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-05-21T15:43:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-05-21T15:43:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-05-21T15:43:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-05-21T15:43:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-05-21T15:43:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-05-21T15:43:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 31, 2024-05-21T15:43:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.078692245, 2024-05-21T15:43:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002538459 2024-05-21T15:43:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:43:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:43:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-05-21T15:43:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-05-21T15:43:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2024-05-21T15:43:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 2, 2024-05-21T15:43:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 1, 2024-05-21T15:43:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 1, 2024-05-21T15:43:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-05-21T15:43:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 76, 2024-05-21T15:43:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 76, 2024-05-21T15:43:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 424, 2024-05-21T15:43:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-05-21T15:43:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-05-21T15:43:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:43:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-05-21T15:43:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:43:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:43:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:43:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:43:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:43:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:43:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:43:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:43:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:43:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:43:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:43:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:43:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:43:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-05-21T15:43:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:43:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-05-21T15:43:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:43:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:43:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:43:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:43:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:43:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:43:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:43:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-05-21T15:43:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:43:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:43:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:43:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-05-21T15:43:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-05-21T15:43:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-05-21T15:43:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:43:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:43:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:43:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:43:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 300743, 2024-05-21T15:43:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 123, 2024-05-21T15:43:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:43:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-05-21T15:43:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-05-21T15:43:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-05-21T15:43:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:43:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-05-21T15:43:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:43:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-05-21T15:43:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 7824, 2024-05-21T15:43:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 165, 2024-05-21T15:43:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:43:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:43:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-05-21T15:43:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-05-21T15:43:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 76741, 2024-05-21T15:43:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 328, 2024-05-21T15:43:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-05-21T15:43:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-05-21T15:43:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-05-21T15:43:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-05-21T15:43:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-05-21T15:43:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-05-21T15:43:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-05-21T15:43:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 1454, 2024-05-21T15:43:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 5055903, 2024-05-21T15:43:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 33, 2024-05-21T15:43:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 1454, 2024-05-21T15:43:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-05-21T15:43:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1454, 2024-05-21T15:43:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 291.214491835, 2024-05-21T15:43:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.200285070 2024-05-21T15:43:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-05-21T15:43:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-05-21T15:43:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1454, 2024-05-21T15:43:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1594 2024-05-21T15:43:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 1454, 2024-05-21T15:43:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 280, 2024-05-21T15:43:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 1174, 2024-05-21T15:43:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-05-21T15:43:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-05-21T15:43:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 0, 2024-05-21T15:43:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 10, 2024-05-21T15:43:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 1, 2024-05-21T15:43:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 203, 2024-05-21T15:43:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 29, 2024-05-21T15:43:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-05-21T15:43:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-05-21T15:43:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-05-21T15:43:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-05-21T15:43:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 814, 2024-05-21T15:43:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-05-21T15:43:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-05-21T15:43:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-05-21T15:43:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 279, 2024-05-21T15:43:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 20, 2024-05-21T15:43:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:43:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-05-21T15:43:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:43:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-05-21T15:43:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-05-21T15:43:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-05-21T15:43:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:43:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-05-21T15:43:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:43:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-05-21T15:43:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-05-21T15:43:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-05-21T15:43:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-05-21T15:43:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-05-21T15:43:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-05-21T15:43:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-05-21T15:43:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-05-21T15:43:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-05-21T15:43:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-05-21T15:43:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-05-21T15:43:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-05-21T15:43:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-05-21T15:43:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-05-21T15:43:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-05-21T15:43:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-05-21T15:43:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-05-21T15:43:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 90, 2024-05-21T15:43:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-05-21T15:43:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 4, 2024-05-21T15:43:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 12, 2024-05-21T15:43:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 12, 2024-05-21T15:43:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-05-21T15:43:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-05-21T15:43:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 167, 2024-05-21T15:43:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 0, 2024-05-21T15:43:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 71 2024-05-21T15:43:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-05-21T15:43:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 1, 2024-05-21T15:43:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:43:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-05-21T15:43:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-05-21T15:43:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-05-21T15:43:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-05-21T15:43:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-05-21T15:43:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-05-21T15:43:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 51, 2024-05-21T15:43:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-05-21T15:43:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 1, 2024-05-21T15:43:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 801, 2024-05-21T15:43:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 31, 2024-05-21T15:43:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-05-21T15:43:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:43:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-21T15:43:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 8209, 2024-05-21T15:43:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 2691542, 2024-05-21T15:43:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 8209, 2024-05-21T15:43:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-21T15:43:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-21T15:43:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 8209, 2024-05-21T15:43:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2691542, 2024-05-21T15:43:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-05-21T15:43:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-21T15:43:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-21T15:43:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-21T15:43:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 1454, 2024-05-21T15:43:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 5059034, 2024-05-21T15:43:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1454, 2024-05-21T15:43:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 5059034, 2024-05-21T15:43:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-05-21T15:43:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-05-21T15:43:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-21T15:43:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-21T15:43:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 1454, 2024-05-21T15:43:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 1454, 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1454, 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1454, 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 31, 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 3131, 2024-05-21T15:43:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 31, 2024-05-21T15:43:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-21T15:43:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-21T15:43:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1, 2024-05-21T15:43:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 3131, 2024-05-21T15:43:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x5601df2b6b60": { 2024-05-21T15:43:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-21T15:43:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 4162, 2024-05-21T15:43:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 5021927, 2024-05-21T15:43:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 4162, 2024-05-21T15:43:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-21T15:43:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-21T15:43:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 202, 2024-05-21T15:43:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 5021927, 2024-05-21T15:43:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:29.130 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:29.130 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-21T15:43:29.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:29 smithi053 ceph-mon[36496]: pgmap v623: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:43:30.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:29 smithi045 ceph-mon[36409]: pgmap v623: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:43:30.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:29 smithi038 ceph-mon[31995]: pgmap v623: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T15:43:30.059 INFO:tasks.check_counter:Daemon mds.c mds.exported=2 2024-05-21T15:43:30.059 INFO:tasks.check_counter:Daemon mds.c mds.imported=2 2024-05-21T15:43:30.059 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-05-21T15:43:30.059 DEBUG:tasks.check_counter:Getting stats from f 2024-05-21T15:43:30.059 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-21T15:43:30.318 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:43:31.782 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:31 smithi053 ceph-mon[36496]: pgmap v624: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-21T15:43:32.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:31 smithi045 ceph-mon[36409]: pgmap v624: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-21T15:43:32.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:31 smithi038 ceph-mon[31995]: pgmap v624: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-21T15:43:32.261 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-21T15:43:32.261 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:43:32.261 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 901, 2024-05-21T15:43:32.262 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 903, 2024-05-21T15:43:32.262 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 12864460, 2024-05-21T15:43:32.262 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 286528, 2024-05-21T15:43:32.262 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 7, 2024-05-21T15:43:32.262 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2024-05-21T15:43:32.262 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.113330613, 2024-05-21T15:43:32.262 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.030647361, 2024-05-21T15:43:32.262 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.039066657, 2024-05-21T15:43:32.262 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.033134955, 2024-05-21T15:43:32.262 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:32.262 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 901, 2024-05-21T15:43:32.262 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.054361139, 2024-05-21T15:43:32.262 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000060334 2024-05-21T15:43:32.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.263 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:32.263 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.263 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.263 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.263 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T15:43:32.263 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T15:43:32.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.263 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:43:32.263 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1026, 2024-05-21T15:43:32.263 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1007, 2024-05-21T15:43:32.263 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 9681560, 2024-05-21T15:43:32.263 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 372956, 2024-05-21T15:43:32.264 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2024-05-21T15:43:32.264 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-05-21T15:43:32.264 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.105585433, 2024-05-21T15:43:32.264 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.029185723, 2024-05-21T15:43:32.264 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.043476034, 2024-05-21T15:43:32.264 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.030595574, 2024-05-21T15:43:32.264 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:32.264 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1007, 2024-05-21T15:43:32.264 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.068630414, 2024-05-21T15:43:32.264 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000068153 2024-05-21T15:43:32.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.264 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:32.264 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.265 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.265 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.265 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 583792, 2024-05-21T15:43:32.265 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 114992 2024-05-21T15:43:32.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.265 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:43:32.265 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 267, 2024-05-21T15:43:32.265 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 406, 2024-05-21T15:43:32.265 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 8455989, 2024-05-21T15:43:32.265 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 407853, 2024-05-21T15:43:32.265 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2024-05-21T15:43:32.265 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-05-21T15:43:32.265 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.040972435, 2024-05-21T15:43:32.266 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.015073631, 2024-05-21T15:43:32.266 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.012199033, 2024-05-21T15:43:32.266 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.009337019, 2024-05-21T15:43:32.266 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:32.266 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 404, 2024-05-21T15:43:32.266 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.020861504, 2024-05-21T15:43:32.266 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000051637 2024-05-21T15:43:32.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.266 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:32.266 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.266 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.266 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.266 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-21T15:43:32.267 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 323264 2024-05-21T15:43:32.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.267 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-05-21T15:43:32.267 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-05-21T15:43:32.267 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-05-21T15:43:32.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.267 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-05-21T15:43:32.267 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-21T15:43:32.267 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-21T15:43:32.267 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3271, 2024-05-21T15:43:32.267 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.111404747, 2024-05-21T15:43:32.267 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000034058 2024-05-21T15:43:32.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:32.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.268 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-05-21T15:43:32.268 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-21T15:43:32.268 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-21T15:43:32.268 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.268 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.268 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.268 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:32.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.268 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-05-21T15:43:32.268 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2024-05-21T15:43:32.268 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2024-05-21T15:43:32.268 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-05-21T15:43:32.268 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.269 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.269 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.269 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-05-21T15:43:32.269 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2024-05-21T15:43:32.269 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-05-21T15:43:32.269 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-05-21T15:43:32.269 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2024-05-21T15:43:32.269 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-05-21T15:43:32.269 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-05-21T15:43:32.269 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 567, 2024-05-21T15:43:32.269 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 375, 2024-05-21T15:43:32.269 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 192, 2024-05-21T15:43:32.269 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-05-21T15:43:32.270 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 30, 2024-05-21T15:43:32.270 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2024-05-21T15:43:32.270 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-05-21T15:43:32.270 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-05-21T15:43:32.270 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 2, 2024-05-21T15:43:32.270 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2024-05-21T15:43:32.270 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2024-05-21T15:43:32.270 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2024-05-21T15:43:32.270 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2024-05-21T15:43:32.270 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:43:32.270 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-05-21T15:43:32.270 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-05-21T15:43:32.270 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 4, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 234, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:43:32.271 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:43:32.272 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:43:32.272 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:43:32.272 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:43:32.272 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-05-21T15:43:32.272 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-05-21T15:43:32.272 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T15:43:32.272 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2024-05-21T15:43:32.272 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:43:32.272 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:43:32.272 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:43:32.272 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2024-05-21T15:43:32.272 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2024-05-21T15:43:32.272 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2024-05-21T15:43:32.272 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2024-05-21T15:43:32.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.273 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-05-21T15:43:32.273 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-05-21T15:43:32.273 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-05-21T15:43:32.273 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-05-21T15:43:32.273 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2024-05-21T15:43:32.273 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-05-21T15:43:32.273 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-05-21T15:43:32.273 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-05-21T15:43:32.273 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:43:32.273 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2024-05-21T15:43:32.273 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2024-05-21T15:43:32.273 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-05-21T15:43:32.273 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-05-21T15:43:32.274 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-05-21T15:43:32.274 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:43:32.274 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:43:32.274 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-05-21T15:43:32.274 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-05-21T15:43:32.274 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:43:32.274 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:43:32.274 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:43:32.274 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-05-21T15:43:32.274 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-05-21T15:43:32.274 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:43:32.274 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-05-21T15:43:32.274 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-05-21T15:43:32.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.275 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-05-21T15:43:32.275 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 0, 2024-05-21T15:43:32.275 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2024-05-21T15:43:32.275 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-05-21T15:43:32.275 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-05-21T15:43:32.275 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 13548, 2024-05-21T15:43:32.275 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-05-21T15:43:32.275 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-05-21T15:43:32.275 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2024-05-21T15:43:32.275 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-05-21T15:43:32.275 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-05-21T15:43:32.275 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 2, 2024-05-21T15:43:32.275 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 34, 2024-05-21T15:43:32.275 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 33922637, 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 34057902, 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 34056764, 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 13548 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 18, 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 1071, 2024-05-21T15:43:32.276 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 1053, 2024-05-21T15:43:32.277 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 13, 2024-05-21T15:43:32.277 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 192, 2024-05-21T15:43:32.277 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 179, 2024-05-21T15:43:32.277 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 567, 2024-05-21T15:43:32.277 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 1354, 2024-05-21T15:43:32.277 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 787, 2024-05-21T15:43:32.277 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-05-21T15:43:32.277 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2024-05-21T15:43:32.277 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2024-05-21T15:43:32.277 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 122120, 2024-05-21T15:43:32.277 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 215444 2024-05-21T15:43:32.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.277 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-05-21T15:43:32.277 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2024-05-21T15:43:32.278 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2024-05-21T15:43:32.278 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2024-05-21T15:43:32.278 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2024-05-21T15:43:32.278 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2024-05-21T15:43:32.278 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-05-21T15:43:32.278 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.278 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.278 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.278 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-05-21T15:43:32.278 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.278 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.278 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.279 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-05-21T15:43:32.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.279 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.279 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-05-21T15:43:32.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.279 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.279 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-05-21T15:43:32.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.279 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.280 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-05-21T15:43:32.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.280 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.280 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-05-21T15:43:32.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.280 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.280 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-05-21T15:43:32.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.280 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.281 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-05-21T15:43:32.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.281 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.281 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-05-21T15:43:32.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.281 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.281 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-05-21T15:43:32.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.282 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.282 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-05-21T15:43:32.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.282 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.282 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-05-21T15:43:32.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.282 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.283 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-05-21T15:43:32.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.283 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.283 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-05-21T15:43:32.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.283 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.283 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-05-21T15:43:32.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.283 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.284 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-05-21T15:43:32.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.284 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.284 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-05-21T15:43:32.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.284 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.284 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-05-21T15:43:32.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.284 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.285 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.285 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-05-21T15:43:32.285 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.285 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.285 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.285 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-05-21T15:43:32.285 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.285 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.285 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.286 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-05-21T15:43:32.286 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.286 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.286 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.286 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-05-21T15:43:32.286 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.286 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.286 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.286 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-05-21T15:43:32.286 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.286 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.287 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.287 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-05-21T15:43:32.287 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.287 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.287 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.287 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-05-21T15:43:32.287 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.287 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.287 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.287 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-05-21T15:43:32.287 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.288 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.288 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.288 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-05-21T15:43:32.288 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.288 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.288 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.288 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-05-21T15:43:32.288 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.288 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.288 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.288 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:43:32.289 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:43:32.289 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-05-21T15:43:32.289 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.289 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.289 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.289 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:32.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.289 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-05-21T15:43:32.289 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2024-05-21T15:43:32.289 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 2, 2024-05-21T15:43:32.289 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 1, 2024-05-21T15:43:32.289 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 1, 2024-05-21T15:43:32.289 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-05-21T15:43:32.289 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2024-05-21T15:43:32.290 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2024-05-21T15:43:32.290 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 1870, 2024-05-21T15:43:32.290 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-05-21T15:43:32.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.290 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-05-21T15:43:32.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:43:32.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-05-21T15:43:32.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:43:32.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:43:32.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:43:32.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:43:32.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:43:32.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:43:32.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:43:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:43:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:43:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:43:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:43:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:43:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:43:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-05-21T15:43:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:43:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-05-21T15:43:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:43:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:43:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:43:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:43:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:43:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:43:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:43:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-05-21T15:43:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:43:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:43:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:43:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-05-21T15:43:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-05-21T15:43:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-05-21T15:43:32.293 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:43:32.293 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:43:32.293 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:43:32.293 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:43:32.293 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 215228, 2024-05-21T15:43:32.293 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 67, 2024-05-21T15:43:32.293 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:43:32.293 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-05-21T15:43:32.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-05-21T15:43:32.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-05-21T15:43:32.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:43:32.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-05-21T15:43:32.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:43:32.293 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-05-21T15:43:32.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 7824, 2024-05-21T15:43:32.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 165, 2024-05-21T15:43:32.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:43:32.294 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:43:32.294 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-05-21T15:43:32.294 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-05-21T15:43:32.294 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 456030, 2024-05-21T15:43:32.294 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 2588, 2024-05-21T15:43:32.294 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-05-21T15:43:32.294 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-05-21T15:43:32.294 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-05-21T15:43:32.294 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-05-21T15:43:32.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.295 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-05-21T15:43:32.295 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-05-21T15:43:32.295 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-05-21T15:43:32.295 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 1978, 2024-05-21T15:43:32.295 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 26, 2024-05-21T15:43:32.295 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 4, 2024-05-21T15:43:32.295 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 1978, 2024-05-21T15:43:32.295 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-05-21T15:43:32.295 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1978, 2024-05-21T15:43:32.295 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 95.935153129, 2024-05-21T15:43:32.295 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.048501088 2024-05-21T15:43:32.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.295 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-05-21T15:43:32.296 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-05-21T15:43:32.296 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1978, 2024-05-21T15:43:32.296 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1979 2024-05-21T15:43:32.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.296 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 1978, 2024-05-21T15:43:32.296 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 1978, 2024-05-21T15:43:32.296 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2024-05-21T15:43:32.296 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-05-21T15:43:32.296 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-05-21T15:43:32.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 1181, 2024-05-21T15:43:32.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2024-05-21T15:43:32.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 794, 2024-05-21T15:43:32.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2024-05-21T15:43:32.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2024-05-21T15:43:32.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-05-21T15:43:32.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-05-21T15:43:32.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-05-21T15:43:32.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-05-21T15:43:32.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2024-05-21T15:43:32.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-05-21T15:43:32.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-05-21T15:43:32.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-05-21T15:43:32.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 2, 2024-05-21T15:43:32.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2024-05-21T15:43:32.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:43:32.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-05-21T15:43:32.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:43:32.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-05-21T15:43:32.298 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-05-21T15:43:32.298 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-05-21T15:43:32.298 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:43:32.298 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-05-21T15:43:32.298 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:43:32.298 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-05-21T15:43:32.298 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-05-21T15:43:32.298 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-05-21T15:43:32.298 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-05-21T15:43:32.298 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-05-21T15:43:32.298 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-05-21T15:43:32.298 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-05-21T15:43:32.298 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-05-21T15:43:32.298 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-05-21T15:43:32.299 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-05-21T15:43:32.299 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-05-21T15:43:32.299 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-05-21T15:43:32.299 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-05-21T15:43:32.299 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-05-21T15:43:32.299 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-05-21T15:43:32.299 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-05-21T15:43:32.299 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-05-21T15:43:32.299 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 90, 2024-05-21T15:43:32.299 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-05-21T15:43:32.299 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 4, 2024-05-21T15:43:32.299 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 8, 2024-05-21T15:43:32.299 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 8, 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-05-21T15:43:32.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.301 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-05-21T15:43:32.301 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-05-21T15:43:32.301 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T15:43:32.301 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-05-21T15:43:32.301 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2024-05-21T15:43:32.301 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 0, 2024-05-21T15:43:32.301 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2024-05-21T15:43:32.301 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-05-21T15:43:32.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.301 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:43:32.301 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:32.301 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-21T15:43:32.301 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:32.301 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 2190, 2024-05-21T15:43:32.302 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 30791169, 2024-05-21T15:43:32.302 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:32.302 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 2190, 2024-05-21T15:43:32.302 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-21T15:43:32.302 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-21T15:43:32.302 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2190, 2024-05-21T15:43:32.302 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 30791169, 2024-05-21T15:43:32.302 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:32.302 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.302 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.302 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.302 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:32.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.303 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-05-21T15:43:32.303 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:32.303 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-21T15:43:32.303 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:32.303 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-21T15:43:32.303 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-21T15:43:32.303 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:32.303 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:32.303 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 1978, 2024-05-21T15:43:32.303 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 29866431, 2024-05-21T15:43:32.303 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1978, 2024-05-21T15:43:32.303 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 29866431, 2024-05-21T15:43:32.304 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:32.304 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.304 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.304 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:32.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.304 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-05-21T15:43:32.304 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:32.304 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-05-21T15:43:32.304 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:32.305 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-21T15:43:32.305 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-21T15:43:32.305 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:32.305 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:32.305 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 1978, 2024-05-21T15:43:32.305 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 1978, 2024-05-21T15:43:32.305 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1978, 2024-05-21T15:43:32.306 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1978, 2024-05-21T15:43:32.306 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:32.306 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.306 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.306 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.306 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:32.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.306 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:43:32.306 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:32.306 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-21T15:43:32.306 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:32.306 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-21T15:43:32.306 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-21T15:43:32.307 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:32.307 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:32.307 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-21T15:43:32.307 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-21T15:43:32.307 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-05-21T15:43:32.307 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-05-21T15:43:32.307 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:32.307 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.307 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.307 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.307 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:32.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:32.307 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x56272a46ea90": { 2024-05-21T15:43:32.307 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:32.308 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-21T15:43:32.308 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:32.308 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-21T15:43:32.308 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-21T15:43:32.308 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:32.308 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:32.308 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-21T15:43:32.308 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-21T15:43:32.308 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-05-21T15:43:32.308 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-05-21T15:43:32.308 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:32.308 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:32.308 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:32.308 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:32.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:32.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:32.309 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-21T15:43:32.992 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-21T15:43:32.992 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-21T15:43:32.992 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-05-21T15:43:32.992 DEBUG:tasks.check_counter:Getting stats from i 2024-05-21T15:43:32.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-21T15:43:33.245 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:43:33.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:33 smithi053 ceph-mon[36496]: pgmap v625: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-21T15:43:33.953 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:33 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:43:34.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:33 smithi045 ceph-mon[36409]: pgmap v625: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-21T15:43:34.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:33 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:43:34.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:33 smithi038 ceph-mon[31995]: pgmap v625: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-21T15:43:34.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:33 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:43:35.056 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-21T15:43:35.057 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:43:35.057 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 3646, 2024-05-21T15:43:35.057 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 3800, 2024-05-21T15:43:35.057 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1451305, 2024-05-21T15:43:35.057 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 5232968, 2024-05-21T15:43:35.057 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 20, 2024-05-21T15:43:35.057 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2024-05-21T15:43:35.057 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.374794498, 2024-05-21T15:43:35.057 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.118472050, 2024-05-21T15:43:35.057 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.328431951, 2024-05-21T15:43:35.057 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.023301139, 2024-05-21T15:43:35.058 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:35.058 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3798, 2024-05-21T15:43:35.058 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.051821165, 2024-05-21T15:43:35.058 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000276940 2024-05-21T15:43:35.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.058 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:35.058 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5628, 2024-05-21T15:43:35.058 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.001643731, 2024-05-21T15:43:35.058 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000292 2024-05-21T15:43:35.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.058 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1250656, 2024-05-21T15:43:35.058 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 1386096 2024-05-21T15:43:35.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.059 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:43:35.059 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 859, 2024-05-21T15:43:35.059 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 837, 2024-05-21T15:43:35.059 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 861627, 2024-05-21T15:43:35.059 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 448261, 2024-05-21T15:43:35.059 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 13, 2024-05-21T15:43:35.059 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 6, 2024-05-21T15:43:35.059 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.102339902, 2024-05-21T15:43:35.059 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.029479110, 2024-05-21T15:43:35.059 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.043708435, 2024-05-21T15:43:35.059 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.025802735, 2024-05-21T15:43:35.059 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:35.059 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 836, 2024-05-21T15:43:35.060 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.075278822, 2024-05-21T15:43:35.060 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000090046 2024-05-21T15:43:35.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.060 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:35.060 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 87, 2024-05-21T15:43:35.060 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000047973, 2024-05-21T15:43:35.060 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000551 2024-05-21T15:43:35.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.060 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 608112, 2024-05-21T15:43:35.060 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 127008 2024-05-21T15:43:35.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.060 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:43:35.060 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2170, 2024-05-21T15:43:35.061 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 2090, 2024-05-21T15:43:35.061 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 795254, 2024-05-21T15:43:35.061 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 1320245, 2024-05-21T15:43:35.061 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 28, 2024-05-21T15:43:35.061 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 6, 2024-05-21T15:43:35.061 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.196523597, 2024-05-21T15:43:35.061 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.065492703, 2024-05-21T15:43:35.061 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.127355607, 2024-05-21T15:43:35.061 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.016206733, 2024-05-21T15:43:35.061 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:35.061 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2089, 2024-05-21T15:43:35.061 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.134791080, 2024-05-21T15:43:35.061 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000064524 2024-05-21T15:43:35.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.062 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:35.062 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3339, 2024-05-21T15:43:35.062 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.001294430, 2024-05-21T15:43:35.062 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000387 2024-05-21T15:43:35.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.062 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-21T15:43:35.062 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 402336 2024-05-21T15:43:35.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.062 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-05-21T15:43:35.062 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-05-21T15:43:35.062 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-05-21T15:43:35.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.062 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-05-21T15:43:35.063 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-21T15:43:35.063 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-21T15:43:35.063 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 724, 2024-05-21T15:43:35.063 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.869443964, 2024-05-21T15:43:35.063 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001200889 2024-05-21T15:43:35.063 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:35.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.063 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-05-21T15:43:35.063 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-21T15:43:35.063 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-21T15:43:35.063 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 703, 2024-05-21T15:43:35.063 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.026159413, 2024-05-21T15:43:35.063 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000037211 2024-05-21T15:43:35.064 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:35.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.064 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-05-21T15:43:35.064 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 121, 2024-05-21T15:43:35.064 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 107, 2024-05-21T15:43:35.064 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-05-21T15:43:35.064 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 107, 2024-05-21T15:43:35.064 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.107121227, 2024-05-21T15:43:35.064 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001001132 2024-05-21T15:43:35.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.064 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-05-21T15:43:35.064 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 14, 2024-05-21T15:43:35.064 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-05-21T15:43:35.064 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-05-21T15:43:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 11, 2024-05-21T15:43:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-05-21T15:43:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-05-21T15:43:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 15, 2024-05-21T15:43:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 0, 2024-05-21T15:43:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 15, 2024-05-21T15:43:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-05-21T15:43:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 13, 2024-05-21T15:43:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 10, 2024-05-21T15:43:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-05-21T15:43:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-05-21T15:43:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 3, 2024-05-21T15:43:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 241, 2024-05-21T15:43:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 185, 2024-05-21T15:43:35.066 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 14, 2024-05-21T15:43:35.066 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 2, 2024-05-21T15:43:35.066 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:43:35.066 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-05-21T15:43:35.066 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-05-21T15:43:35.066 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-05-21T15:43:35.066 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-05-21T15:43:35.066 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 2, 2024-05-21T15:43:35.066 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-05-21T15:43:35.066 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 2, 2024-05-21T15:43:35.066 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 26, 2024-05-21T15:43:35.066 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-05-21T15:43:35.066 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 216, 2024-05-21T15:43:35.067 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-05-21T15:43:35.067 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 4, 2024-05-21T15:43:35.067 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 234, 2024-05-21T15:43:35.067 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-05-21T15:43:35.067 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 216, 2024-05-21T15:43:35.067 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-05-21T15:43:35.067 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:43:35.067 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:43:35.067 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:43:35.067 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:43:35.067 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:43:35.067 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 216, 2024-05-21T15:43:35.067 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-05-21T15:43:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 9, 2024-05-21T15:43:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 382, 2024-05-21T15:43:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:43:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:43:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:43:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 2995, 2024-05-21T15:43:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 2708, 2024-05-21T15:43:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 4, 2024-05-21T15:43:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 56 2024-05-21T15:43:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-05-21T15:43:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-05-21T15:43:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 3, 2024-05-21T15:43:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 2, 2024-05-21T15:43:35.069 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 4, 2024-05-21T15:43:35.069 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-05-21T15:43:35.069 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-05-21T15:43:35.069 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-05-21T15:43:35.069 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:43:35.069 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 26, 2024-05-21T15:43:35.069 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 26, 2024-05-21T15:43:35.069 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-05-21T15:43:35.069 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-05-21T15:43:35.069 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-05-21T15:43:35.069 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:43:35.069 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:43:35.069 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-05-21T15:43:35.069 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-05-21T15:43:35.070 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:43:35.070 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:43:35.070 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:43:35.070 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 2, 2024-05-21T15:43:35.070 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-05-21T15:43:35.070 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:43:35.070 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-05-21T15:43:35.070 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-05-21T15:43:35.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.070 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-05-21T15:43:35.070 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 0, 2024-05-21T15:43:35.070 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 3043, 2024-05-21T15:43:35.070 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-05-21T15:43:35.071 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-05-21T15:43:35.071 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 3043, 2024-05-21T15:43:35.071 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-05-21T15:43:35.071 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-05-21T15:43:35.071 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 4, 2024-05-21T15:43:35.071 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-05-21T15:43:35.071 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-05-21T15:43:35.071 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 4, 2024-05-21T15:43:35.071 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 2, 2024-05-21T15:43:35.071 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-05-21T15:43:35.071 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-05-21T15:43:35.071 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-05-21T15:43:35.071 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 7857807, 2024-05-21T15:43:35.071 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 0, 2024-05-21T15:43:35.072 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-05-21T15:43:35.072 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 186, 2024-05-21T15:43:35.072 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 27.660278343, 2024-05-21T15:43:35.072 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.148711173 2024-05-21T15:43:35.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.072 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 0 2024-05-21T15:43:35.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.072 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-05-21T15:43:35.072 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 19, 2024-05-21T15:43:35.072 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 50, 2024-05-21T15:43:35.072 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 31, 2024-05-21T15:43:35.072 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 18, 2024-05-21T15:43:35.072 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 25, 2024-05-21T15:43:35.072 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 7, 2024-05-21T15:43:35.073 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 15, 2024-05-21T15:43:35.073 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 77, 2024-05-21T15:43:35.073 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 62, 2024-05-21T15:43:35.073 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-05-21T15:43:35.073 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 30, 2024-05-21T15:43:35.073 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 30, 2024-05-21T15:43:35.073 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 127824, 2024-05-21T15:43:35.073 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 223636 2024-05-21T15:43:35.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.073 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-05-21T15:43:35.073 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 189, 2024-05-21T15:43:35.073 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 2, 2024-05-21T15:43:35.073 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 121, 2024-05-21T15:43:35.073 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 6, 2024-05-21T15:43:35.074 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 32, 2024-05-21T15:43:35.074 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-05-21T15:43:35.074 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.074 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.074 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.074 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-05-21T15:43:35.074 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.074 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.074 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.074 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-05-21T15:43:35.074 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-05-21T15:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-05-21T15:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 75, 2024-05-21T15:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.039366353, 2024-05-21T15:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000524884 2024-05-21T15:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-05-21T15:43:35.076 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.076 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.076 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.076 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-05-21T15:43:35.076 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.076 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.076 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.076 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-05-21T15:43:35.076 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.076 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.076 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.077 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-05-21T15:43:35.077 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.077 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.077 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.077 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-05-21T15:43:35.077 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.077 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.077 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.077 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-05-21T15:43:35.077 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.077 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.078 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.078 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-05-21T15:43:35.078 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.079 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.079 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.079 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-05-21T15:43:35.079 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.079 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.079 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.079 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-05-21T15:43:35.079 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.079 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.079 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.079 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-05-21T15:43:35.080 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2, 2024-05-21T15:43:35.080 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002260113, 2024-05-21T15:43:35.080 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001130056 2024-05-21T15:43:35.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.080 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-05-21T15:43:35.080 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4, 2024-05-21T15:43:35.080 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002299746, 2024-05-21T15:43:35.080 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000574936 2024-05-21T15:43:35.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.080 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-05-21T15:43:35.080 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.080 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.080 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.081 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-05-21T15:43:35.081 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.081 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.081 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.081 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-05-21T15:43:35.081 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.081 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.081 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.081 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-05-21T15:43:35.081 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.081 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.082 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.082 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-05-21T15:43:35.082 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.082 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.082 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.082 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-05-21T15:43:35.082 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.082 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.082 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.082 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-05-21T15:43:35.083 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.083 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.083 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.083 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-05-21T15:43:35.083 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.083 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.083 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.083 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-05-21T15:43:35.083 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.083 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.083 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.084 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-05-21T15:43:35.084 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.084 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.084 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.084 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-05-21T15:43:35.084 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.084 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.084 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.084 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-05-21T15:43:35.084 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.084 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.085 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.085 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-05-21T15:43:35.085 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 26, 2024-05-21T15:43:35.085 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.063195015, 2024-05-21T15:43:35.085 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002430577 2024-05-21T15:43:35.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.085 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:43:35.085 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:43:35.085 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-05-21T15:43:35.085 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.085 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.085 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:35.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-05-21T15:43:35.086 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2024-05-21T15:43:35.086 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 1, 2024-05-21T15:43:35.086 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 0, 2024-05-21T15:43:35.086 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 1, 2024-05-21T15:43:35.086 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-05-21T15:43:35.086 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 65, 2024-05-21T15:43:35.086 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 65, 2024-05-21T15:43:35.086 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 430, 2024-05-21T15:43:35.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-05-21T15:43:35.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-05-21T15:43:35.087 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:43:35.087 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-05-21T15:43:35.087 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:43:35.087 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:43:35.087 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:43:35.087 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:43:35.087 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:43:35.087 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:43:35.087 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:43:35.087 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:43:35.087 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:43:35.087 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:43:35.088 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:43:35.088 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:43:35.088 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:43:35.088 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-05-21T15:43:35.088 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:43:35.088 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-05-21T15:43:35.088 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:43:35.088 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:43:35.088 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:43:35.088 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:43:35.088 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:43:35.088 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:43:35.088 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:43:35.088 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-05-21T15:43:35.089 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:43:35.089 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:43:35.089 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:43:35.089 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-05-21T15:43:35.089 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-05-21T15:43:35.089 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-05-21T15:43:35.089 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:43:35.089 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:43:35.089 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:43:35.089 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:43:35.089 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 316647, 2024-05-21T15:43:35.089 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 126, 2024-05-21T15:43:35.089 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:43:35.089 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-05-21T15:43:35.090 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-05-21T15:43:35.090 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-05-21T15:43:35.090 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:43:35.090 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-05-21T15:43:35.090 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:43:35.090 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-05-21T15:43:35.090 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 7824, 2024-05-21T15:43:35.090 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 165, 2024-05-21T15:43:35.090 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:43:35.090 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:43:35.090 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-05-21T15:43:35.090 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-05-21T15:43:35.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 76741, 2024-05-21T15:43:35.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 328, 2024-05-21T15:43:35.091 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-05-21T15:43:35.091 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-05-21T15:43:35.091 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-05-21T15:43:35.091 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-05-21T15:43:35.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.091 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-05-21T15:43:35.091 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-05-21T15:43:35.091 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-05-21T15:43:35.091 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 1218, 2024-05-21T15:43:35.091 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 3693759, 2024-05-21T15:43:35.091 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 33, 2024-05-21T15:43:35.091 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 1218, 2024-05-21T15:43:35.091 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-05-21T15:43:35.092 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1218, 2024-05-21T15:43:35.092 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 152.835145016, 2024-05-21T15:43:35.092 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.125480414 2024-05-21T15:43:35.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.092 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-05-21T15:43:35.092 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-05-21T15:43:35.092 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1218, 2024-05-21T15:43:35.092 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1349 2024-05-21T15:43:35.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.092 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 1218, 2024-05-21T15:43:35.092 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 217, 2024-05-21T15:43:35.092 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 1001, 2024-05-21T15:43:35.092 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-05-21T15:43:35.092 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-05-21T15:43:35.093 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 0, 2024-05-21T15:43:35.093 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 10, 2024-05-21T15:43:35.093 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 1, 2024-05-21T15:43:35.093 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 187, 2024-05-21T15:43:35.093 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 29, 2024-05-21T15:43:35.093 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-05-21T15:43:35.093 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-05-21T15:43:35.093 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-05-21T15:43:35.093 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-05-21T15:43:35.093 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 662, 2024-05-21T15:43:35.093 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-05-21T15:43:35.093 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-05-21T15:43:35.093 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-05-21T15:43:35.094 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 216, 2024-05-21T15:43:35.094 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 20, 2024-05-21T15:43:35.094 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:43:35.094 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-05-21T15:43:35.094 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:43:35.094 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-05-21T15:43:35.094 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-05-21T15:43:35.094 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-05-21T15:43:35.094 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:43:35.094 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-05-21T15:43:35.094 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:43:35.094 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-05-21T15:43:35.094 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-05-21T15:43:35.095 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-05-21T15:43:35.095 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-05-21T15:43:35.095 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-05-21T15:43:35.095 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-05-21T15:43:35.095 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-05-21T15:43:35.095 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-05-21T15:43:35.095 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-05-21T15:43:35.095 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-05-21T15:43:35.095 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-05-21T15:43:35.095 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-05-21T15:43:35.095 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-05-21T15:43:35.095 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-05-21T15:43:35.095 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-05-21T15:43:35.095 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-05-21T15:43:35.096 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-05-21T15:43:35.096 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 90, 2024-05-21T15:43:35.096 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-05-21T15:43:35.096 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 4, 2024-05-21T15:43:35.096 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 12, 2024-05-21T15:43:35.096 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 12, 2024-05-21T15:43:35.096 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-05-21T15:43:35.096 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-05-21T15:43:35.096 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 157, 2024-05-21T15:43:35.096 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 0, 2024-05-21T15:43:35.096 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 67 2024-05-21T15:43:35.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.096 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-05-21T15:43:35.096 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 1, 2024-05-21T15:43:35.097 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:43:35.097 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-05-21T15:43:35.097 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-05-21T15:43:35.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.097 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-05-21T15:43:35.097 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-05-21T15:43:35.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.097 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-05-21T15:43:35.097 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-05-21T15:43:35.097 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 25, 2024-05-21T15:43:35.097 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-05-21T15:43:35.097 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 1, 2024-05-21T15:43:35.097 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 650, 2024-05-21T15:43:35.098 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 26, 2024-05-21T15:43:35.098 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-05-21T15:43:35.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.098 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:43:35.098 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:35.098 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-21T15:43:35.098 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:35.098 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 6671, 2024-05-21T15:43:35.098 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 2335439, 2024-05-21T15:43:35.098 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:35.098 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 6671, 2024-05-21T15:43:35.098 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-21T15:43:35.098 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-21T15:43:35.098 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 6671, 2024-05-21T15:43:35.099 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2335439, 2024-05-21T15:43:35.099 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:35.099 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.099 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.099 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:35.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.099 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-05-21T15:43:35.099 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:35.099 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-21T15:43:35.099 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:35.099 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-21T15:43:35.099 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-21T15:43:35.099 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:35.100 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:35.100 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 1218, 2024-05-21T15:43:35.100 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 3696385, 2024-05-21T15:43:35.100 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1218, 2024-05-21T15:43:35.100 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 3696385, 2024-05-21T15:43:35.100 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:35.100 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.100 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.100 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.100 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:35.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.100 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-05-21T15:43:35.100 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:35.101 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-05-21T15:43:35.101 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:35.101 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-21T15:43:35.101 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-21T15:43:35.101 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:35.101 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:35.101 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 1218, 2024-05-21T15:43:35.101 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 1218, 2024-05-21T15:43:35.101 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1218, 2024-05-21T15:43:35.101 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1218, 2024-05-21T15:43:35.101 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:35.101 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.101 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.102 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.102 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:35.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.102 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:43:35.102 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:35.102 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-21T15:43:35.102 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:35.102 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 26, 2024-05-21T15:43:35.102 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 2626, 2024-05-21T15:43:35.102 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:35.102 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 26, 2024-05-21T15:43:35.102 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-21T15:43:35.102 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-21T15:43:35.102 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1, 2024-05-21T15:43:35.103 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2626, 2024-05-21T15:43:35.103 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:35.103 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.103 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.103 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:35.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:35.103 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x55be6b16ec30": { 2024-05-21T15:43:35.103 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:35.103 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-21T15:43:35.103 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:35.103 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 3043, 2024-05-21T15:43:35.103 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 3663503, 2024-05-21T15:43:35.104 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:35.104 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 3043, 2024-05-21T15:43:35.104 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-21T15:43:35.104 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-21T15:43:35.104 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 186, 2024-05-21T15:43:35.104 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 3663503, 2024-05-21T15:43:35.104 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:35.104 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:35.104 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:35.104 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:35.104 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:35.104 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:35.104 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-21T15:43:35.845 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:35 smithi053 ceph-mon[36496]: pgmap v626: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-21T15:43:35.845 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:35 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:43:35.845 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:35 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:43:35.845 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:35 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:43:35.873 INFO:tasks.check_counter:Daemon mds.i mds.exported=2 2024-05-21T15:43:35.874 INFO:tasks.check_counter:Daemon mds.i mds.imported=2 2024-05-21T15:43:35.874 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-05-21T15:43:35.874 DEBUG:tasks.check_counter:Getting stats from l 2024-05-21T15:43:35.874 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-21T15:43:36.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:35 smithi045 ceph-mon[36409]: pgmap v626: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-21T15:43:36.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:35 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:43:36.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:35 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:43:36.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:35 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:43:36.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:35 smithi038 ceph-mon[31995]: pgmap v626: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-21T15:43:36.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:35 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:43:36.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:35 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:43:36.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:35 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' 2024-05-21T15:43:36.121 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:43:37.940 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:37 smithi053 ceph-mon[36496]: pgmap v627: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-21T15:43:38.008 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-21T15:43:38.008 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:43:38.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 717, 2024-05-21T15:43:38.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 719, 2024-05-21T15:43:38.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 3850060, 2024-05-21T15:43:38.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 228070, 2024-05-21T15:43:38.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 6, 2024-05-21T15:43:38.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 2, 2024-05-21T15:43:38.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.072039060, 2024-05-21T15:43:38.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.018477734, 2024-05-21T15:43:38.008 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.022708537, 2024-05-21T15:43:38.009 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.023578017, 2024-05-21T15:43:38.009 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:38.009 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 717, 2024-05-21T15:43:38.009 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.043707026, 2024-05-21T15:43:38.009 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000060958 2024-05-21T15:43:38.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.009 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:38.009 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.009 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.009 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.009 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T15:43:38.009 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T15:43:38.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.010 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:43:38.010 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 377, 2024-05-21T15:43:38.010 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 520, 2024-05-21T15:43:38.010 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 92678, 2024-05-21T15:43:38.010 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 445157, 2024-05-21T15:43:38.010 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2024-05-21T15:43:38.010 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-05-21T15:43:38.010 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.047389016, 2024-05-21T15:43:38.010 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.014506740, 2024-05-21T15:43:38.010 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.011379597, 2024-05-21T15:43:38.010 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.016901754, 2024-05-21T15:43:38.010 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:38.011 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 518, 2024-05-21T15:43:38.011 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.021666393, 2024-05-21T15:43:38.011 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000041827 2024-05-21T15:43:38.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.011 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:38.011 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.011 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.011 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.011 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-21T15:43:38.011 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 325584 2024-05-21T15:43:38.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.011 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:43:38.012 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 812, 2024-05-21T15:43:38.012 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 793, 2024-05-21T15:43:38.012 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 784240, 2024-05-21T15:43:38.012 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 306606, 2024-05-21T15:43:38.012 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 3, 2024-05-21T15:43:38.012 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 2, 2024-05-21T15:43:38.012 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.106349869, 2024-05-21T15:43:38.012 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.033455380, 2024-05-21T15:43:38.012 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.038356322, 2024-05-21T15:43:38.012 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.029885213, 2024-05-21T15:43:38.012 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:43:38.012 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 793, 2024-05-21T15:43:38.012 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.044674308, 2024-05-21T15:43:38.013 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000056335 2024-05-21T15:43:38.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.013 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:43:38.013 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.013 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.013 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.013 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 584992, 2024-05-21T15:43:38.013 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 118032 2024-05-21T15:43:38.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.013 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-05-21T15:43:38.013 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-05-21T15:43:38.013 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-05-21T15:43:38.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.014 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-05-21T15:43:38.014 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-21T15:43:38.014 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-21T15:43:38.014 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2968, 2024-05-21T15:43:38.014 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.092894740, 2024-05-21T15:43:38.014 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000031298 2024-05-21T15:43:38.014 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:38.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.014 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-05-21T15:43:38.014 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-21T15:43:38.014 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-21T15:43:38.014 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.014 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.015 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.015 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:38.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.015 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-05-21T15:43:38.015 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2024-05-21T15:43:38.015 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2024-05-21T15:43:38.015 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-05-21T15:43:38.015 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.015 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.015 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.015 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-05-21T15:43:38.015 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2024-05-21T15:43:38.016 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-05-21T15:43:38.016 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-05-21T15:43:38.016 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2024-05-21T15:43:38.016 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-05-21T15:43:38.016 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-05-21T15:43:38.016 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 27, 2024-05-21T15:43:38.016 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 0, 2024-05-21T15:43:38.016 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 27, 2024-05-21T15:43:38.016 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-05-21T15:43:38.016 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 27, 2024-05-21T15:43:38.016 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2024-05-21T15:43:38.016 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-05-21T15:43:38.016 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-05-21T15:43:38.017 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 1, 2024-05-21T15:43:38.017 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2024-05-21T15:43:38.017 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2024-05-21T15:43:38.017 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2024-05-21T15:43:38.017 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2024-05-21T15:43:38.017 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:43:38.017 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-05-21T15:43:38.017 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-05-21T15:43:38.017 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-05-21T15:43:38.017 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-05-21T15:43:38.017 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-05-21T15:43:38.017 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-05-21T15:43:38.017 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-05-21T15:43:38.017 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-05-21T15:43:38.018 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-05-21T15:43:38.018 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-05-21T15:43:38.018 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-05-21T15:43:38.018 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 0, 2024-05-21T15:43:38.018 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 0, 2024-05-21T15:43:38.018 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-05-21T15:43:38.018 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T15:43:38.018 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-05-21T15:43:38.018 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:43:38.018 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:43:38.018 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:43:38.018 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:43:38.018 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:43:38.019 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-05-21T15:43:38.019 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-05-21T15:43:38.019 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T15:43:38.019 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2024-05-21T15:43:38.019 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:43:38.019 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:43:38.019 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:43:38.019 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2024-05-21T15:43:38.019 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2024-05-21T15:43:38.019 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2024-05-21T15:43:38.019 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2024-05-21T15:43:38.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.019 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-05-21T15:43:38.019 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-05-21T15:43:38.020 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-05-21T15:43:38.020 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-05-21T15:43:38.020 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2024-05-21T15:43:38.020 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-05-21T15:43:38.020 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-05-21T15:43:38.020 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-05-21T15:43:38.020 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:43:38.020 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2024-05-21T15:43:38.020 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2024-05-21T15:43:38.020 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-05-21T15:43:38.020 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-05-21T15:43:38.020 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-05-21T15:43:38.020 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:43:38.021 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:43:38.021 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-05-21T15:43:38.021 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-05-21T15:43:38.021 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:43:38.021 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:43:38.021 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:43:38.021 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-05-21T15:43:38.021 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-05-21T15:43:38.021 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:43:38.021 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-05-21T15:43:38.021 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-05-21T15:43:38.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.021 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-05-21T15:43:38.021 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 0, 2024-05-21T15:43:38.022 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2024-05-21T15:43:38.022 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-05-21T15:43:38.022 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-05-21T15:43:38.022 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 3043, 2024-05-21T15:43:38.022 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-05-21T15:43:38.022 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-05-21T15:43:38.022 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2024-05-21T15:43:38.022 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-05-21T15:43:38.022 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-05-21T15:43:38.022 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 4, 2024-05-21T15:43:38.022 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 2, 2024-05-21T15:43:38.022 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-05-21T15:43:38.022 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-05-21T15:43:38.023 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-05-21T15:43:38.023 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 7857807, 2024-05-21T15:43:38.023 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 7856156, 2024-05-21T15:43:38.023 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-05-21T15:43:38.023 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.023 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.023 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.023 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 3043 2024-05-21T15:43:38.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.023 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-05-21T15:43:38.023 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 4, 2024-05-21T15:43:38.023 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 40, 2024-05-21T15:43:38.024 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 36, 2024-05-21T15:43:38.024 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 2, 2024-05-21T15:43:38.024 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 14, 2024-05-21T15:43:38.024 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 12, 2024-05-21T15:43:38.024 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 27, 2024-05-21T15:43:38.024 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 63, 2024-05-21T15:43:38.024 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 36, 2024-05-21T15:43:38.024 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-05-21T15:43:38.024 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2024-05-21T15:43:38.024 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2024-05-21T15:43:38.024 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 120888, 2024-05-21T15:43:38.024 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 215444 2024-05-21T15:43:38.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.025 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-05-21T15:43:38.025 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2024-05-21T15:43:38.025 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2024-05-21T15:43:38.025 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2024-05-21T15:43:38.025 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2024-05-21T15:43:38.025 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2024-05-21T15:43:38.025 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-05-21T15:43:38.025 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.025 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.025 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.025 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-05-21T15:43:38.025 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.025 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.026 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.026 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-05-21T15:43:38.026 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.026 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.026 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.026 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-05-21T15:43:38.026 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.026 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.026 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.027 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-05-21T15:43:38.027 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.027 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.027 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.027 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-05-21T15:43:38.027 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.027 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.027 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.027 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-05-21T15:43:38.027 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.027 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.027 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.028 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-05-21T15:43:38.028 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.028 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.028 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.028 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-05-21T15:43:38.028 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.028 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.028 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.028 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-05-21T15:43:38.028 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.028 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.029 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.029 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-05-21T15:43:38.029 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.029 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.029 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.029 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-05-21T15:43:38.029 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.030 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:37 smithi038 ceph-mon[31995]: pgmap v627: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-21T15:43:38.030 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:37 smithi045 ceph-mon[36409]: pgmap v627: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-21T15:43:38.030 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.031 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.031 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-05-21T15:43:38.031 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.031 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.031 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.031 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-05-21T15:43:38.031 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.031 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.031 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.032 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-05-21T15:43:38.032 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.032 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.032 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.032 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-05-21T15:43:38.032 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.032 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.032 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.032 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-05-21T15:43:38.032 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.032 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.032 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-05-21T15:43:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-05-21T15:43:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-05-21T15:43:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.034 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.034 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-05-21T15:43:38.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.034 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.034 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-05-21T15:43:38.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.034 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.035 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-05-21T15:43:38.035 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.035 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.035 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.035 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-05-21T15:43:38.035 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.035 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.035 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.035 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-05-21T15:43:38.035 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.035 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.036 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.036 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-05-21T15:43:38.036 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.036 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.036 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.036 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-05-21T15:43:38.036 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.036 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.036 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.036 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-05-21T15:43:38.036 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.037 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.037 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.037 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-05-21T15:43:38.037 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.037 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.037 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.037 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:43:38.037 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:43:38.037 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-05-21T15:43:38.037 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.037 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.038 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:38.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.038 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-05-21T15:43:38.038 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2024-05-21T15:43:38.038 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 1, 2024-05-21T15:43:38.038 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 0, 2024-05-21T15:43:38.038 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 1, 2024-05-21T15:43:38.038 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-05-21T15:43:38.038 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2024-05-21T15:43:38.038 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2024-05-21T15:43:38.038 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 1875, 2024-05-21T15:43:38.038 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-05-21T15:43:38.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.039 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-05-21T15:43:38.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:43:38.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-05-21T15:43:38.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:43:38.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:43:38.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:43:38.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:43:38.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:43:38.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:43:38.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:43:38.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:43:38.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:43:38.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:43:38.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:43:38.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:43:38.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:43:38.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-05-21T15:43:38.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:43:38.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-05-21T15:43:38.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:43:38.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:43:38.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:43:38.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:43:38.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:43:38.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:43:38.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:43:38.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-05-21T15:43:38.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:43:38.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:43:38.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:43:38.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-05-21T15:43:38.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-05-21T15:43:38.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-05-21T15:43:38.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:43:38.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:43:38.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:43:38.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:43:38.041 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 190167, 2024-05-21T15:43:38.041 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 52, 2024-05-21T15:43:38.042 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:43:38.042 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-05-21T15:43:38.042 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-05-21T15:43:38.042 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-05-21T15:43:38.042 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:43:38.042 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-05-21T15:43:38.042 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:43:38.042 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-05-21T15:43:38.042 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 7824, 2024-05-21T15:43:38.042 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 165, 2024-05-21T15:43:38.042 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:43:38.042 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:43:38.042 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-05-21T15:43:38.042 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-05-21T15:43:38.043 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 30992, 2024-05-21T15:43:38.043 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 136, 2024-05-21T15:43:38.043 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-05-21T15:43:38.043 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-05-21T15:43:38.043 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-05-21T15:43:38.043 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-05-21T15:43:38.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.043 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-05-21T15:43:38.043 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-05-21T15:43:38.043 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-05-21T15:43:38.043 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 1686, 2024-05-21T15:43:38.043 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 21, 2024-05-21T15:43:38.043 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 3, 2024-05-21T15:43:38.044 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 1686, 2024-05-21T15:43:38.044 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-05-21T15:43:38.044 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1686, 2024-05-21T15:43:38.044 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 71.727617181, 2024-05-21T15:43:38.044 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.042543070 2024-05-21T15:43:38.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.044 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-05-21T15:43:38.044 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-05-21T15:43:38.044 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1686, 2024-05-21T15:43:38.044 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1687 2024-05-21T15:43:38.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.044 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 1686, 2024-05-21T15:43:38.044 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 1686, 2024-05-21T15:43:38.044 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2024-05-21T15:43:38.045 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-05-21T15:43:38.045 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-05-21T15:43:38.045 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 967, 2024-05-21T15:43:38.045 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2024-05-21T15:43:38.045 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 717, 2024-05-21T15:43:38.045 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2024-05-21T15:43:38.045 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2024-05-21T15:43:38.045 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-05-21T15:43:38.045 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-05-21T15:43:38.045 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-05-21T15:43:38.045 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-05-21T15:43:38.045 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2024-05-21T15:43:38.045 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-05-21T15:43:38.045 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-05-21T15:43:38.046 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-05-21T15:43:38.046 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 1, 2024-05-21T15:43:38.046 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2024-05-21T15:43:38.046 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:43:38.046 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-05-21T15:43:38.046 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:43:38.046 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-05-21T15:43:38.046 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-05-21T15:43:38.046 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-05-21T15:43:38.046 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:43:38.046 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-05-21T15:43:38.046 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:43:38.046 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-05-21T15:43:38.047 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-05-21T15:43:38.047 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-05-21T15:43:38.047 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-05-21T15:43:38.047 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-05-21T15:43:38.047 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-05-21T15:43:38.047 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-05-21T15:43:38.047 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-05-21T15:43:38.047 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-05-21T15:43:38.047 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-05-21T15:43:38.047 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-05-21T15:43:38.047 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-05-21T15:43:38.047 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-05-21T15:43:38.047 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-05-21T15:43:38.048 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-05-21T15:43:38.048 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-05-21T15:43:38.048 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-05-21T15:43:38.048 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 90, 2024-05-21T15:43:38.048 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-05-21T15:43:38.048 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 4, 2024-05-21T15:43:38.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 5, 2024-05-21T15:43:38.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 5, 2024-05-21T15:43:38.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-05-21T15:43:38.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-05-21T15:43:38.048 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2024-05-21T15:43:38.048 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2024-05-21T15:43:38.048 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2024-05-21T15:43:38.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.049 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-05-21T15:43:38.049 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2024-05-21T15:43:38.049 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:43:38.049 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-05-21T15:43:38.049 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-05-21T15:43:38.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.049 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-05-21T15:43:38.049 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-05-21T15:43:38.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.049 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-05-21T15:43:38.049 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-05-21T15:43:38.049 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T15:43:38.049 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-05-21T15:43:38.049 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2024-05-21T15:43:38.050 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 0, 2024-05-21T15:43:38.050 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2024-05-21T15:43:38.050 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-05-21T15:43:38.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.050 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:43:38.050 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:38.050 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-21T15:43:38.050 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:38.050 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 1902, 2024-05-21T15:43:38.050 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 4541844, 2024-05-21T15:43:38.050 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:38.050 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 1902, 2024-05-21T15:43:38.050 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-21T15:43:38.051 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-21T15:43:38.051 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1902, 2024-05-21T15:43:38.051 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 4541844, 2024-05-21T15:43:38.051 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:38.051 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.051 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.051 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.051 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:38.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.051 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-05-21T15:43:38.051 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:38.051 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-21T15:43:38.051 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:38.052 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-21T15:43:38.052 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-21T15:43:38.052 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:38.052 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:38.052 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 1686, 2024-05-21T15:43:38.052 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 3663508, 2024-05-21T15:43:38.052 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1686, 2024-05-21T15:43:38.052 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 3663508, 2024-05-21T15:43:38.052 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:38.052 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.052 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.052 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:38.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.053 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-05-21T15:43:38.053 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:38.053 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-05-21T15:43:38.053 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:38.053 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-21T15:43:38.053 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-21T15:43:38.053 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:38.053 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:38.053 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 1686, 2024-05-21T15:43:38.053 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 1686, 2024-05-21T15:43:38.053 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1686, 2024-05-21T15:43:38.053 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1686, 2024-05-21T15:43:38.053 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:38.053 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.054 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.054 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.054 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:38.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.054 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:43:38.054 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:38.054 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-21T15:43:38.054 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:38.054 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-21T15:43:38.054 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-21T15:43:38.054 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:38.054 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:38.055 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-21T15:43:38.055 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-21T15:43:38.055 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-05-21T15:43:38.055 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-05-21T15:43:38.055 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:38.055 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.055 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.055 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.055 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:38.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-21T15:43:38.055 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x55ba0d8629c0": { 2024-05-21T15:43:38.055 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-21T15:43:38.055 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-21T15:43:38.055 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-21T15:43:38.056 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-21T15:43:38.056 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-21T15:43:38.056 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-21T15:43:38.056 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-21T15:43:38.056 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-21T15:43:38.056 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-21T15:43:38.056 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-05-21T15:43:38.056 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-05-21T15:43:38.056 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-21T15:43:38.056 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-21T15:43:38.056 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-21T15:43:38.056 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-21T15:43:38.056 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:38.056 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-21T15:43:38.057 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-21T15:43:38.724 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-21T15:43:38.724 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-21T15:43:38.724 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-05-21T15:43:38.725 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-21T15:43:38.735 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-21T15:43:38.735 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:43:38.735 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-21T15:43:38.764 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-05-21T15:43:38.764 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-21T15:43:38.824 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:43:38.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.996+0000 7f6eb26f0640 1 -- 172.21.15.38:0/3383787878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eac1035b0 msgr2=0x7f6eac1059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:38.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.996+0000 7f6eb26f0640 1 --2- 172.21.15.38:0/3383787878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eac1035b0 0x7f6eac1059d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009900 tx=0x7f6e9c02a410 comp rx=0 tx=0).stop 2024-05-21T15:43:38.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.996+0000 7f6eb26f0640 1 -- 172.21.15.38:0/3383787878 shutdown_connections 2024-05-21T15:43:38.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.996+0000 7f6eb26f0640 1 --2- 172.21.15.38:0/3383787878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6eac108870 0x7f6eac10ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:38.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.996+0000 7f6eb26f0640 1 --2- 172.21.15.38:0/3383787878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eac105f10 0x7f6eac108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:38.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.996+0000 7f6eb26f0640 1 --2- 172.21.15.38:0/3383787878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eac1035b0 0x7f6eac1059d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:38.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.996+0000 7f6eb26f0640 1 -- 172.21.15.38:0/3383787878 >> 172.21.15.38:0/3383787878 conn(0x7f6eac0fcbb0 msgr2=0x7f6eac0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:38.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.996+0000 7f6eb26f0640 1 -- 172.21.15.38:0/3383787878 shutdown_connections 2024-05-21T15:43:38.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.996+0000 7f6eb26f0640 1 -- 172.21.15.38:0/3383787878 wait complete. 2024-05-21T15:43:38.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.997+0000 7f6eb26f0640 1 Processor -- start 2024-05-21T15:43:38.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.997+0000 7f6eb26f0640 1 -- start start 2024-05-21T15:43:38.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.997+0000 7f6eb26f0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eac1035b0 0x7f6eac1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:38.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.997+0000 7f6eb26f0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eac105f10 0x7f6eac199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:38.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.997+0000 7f6eb26f0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6eac108870 0x7f6eac1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:38.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.997+0000 7f6eb26f0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac10e710 con 0x7f6eac1035b0 2024-05-21T15:43:38.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.997+0000 7f6eb26f0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac10e590 con 0x7f6eac108870 2024-05-21T15:43:38.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.997+0000 7f6eb26f0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac10e890 con 0x7f6eac105f10 2024-05-21T15:43:38.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.997+0000 7f6eabfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eac1035b0 0x7f6eac1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:38.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.997+0000 7f6eabfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eac1035b0 0x7f6eac1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50528/0 (socket says 172.21.15.38:50528) 2024-05-21T15:43:38.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.997+0000 7f6eabfff640 1 -- 172.21.15.38:0/1660899295 learned_addr learned my addr 172.21.15.38:0/1660899295 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:43:38.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.997+0000 7f6eab7fe640 1 --2- 172.21.15.38:0/1660899295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eac105f10 0x7f6eac199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:38.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.997+0000 7f6eb0c8d640 1 --2- 172.21.15.38:0/1660899295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6eac108870 0x7f6eac1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:38.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.998+0000 7f6eabfff640 1 -- 172.21.15.38:0/1660899295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eac105f10 msgr2=0x7f6eac199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:38.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.998+0000 7f6eabfff640 1 --2- 172.21.15.38:0/1660899295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eac105f10 0x7f6eac199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:38.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.998+0000 7f6eabfff640 1 -- 172.21.15.38:0/1660899295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6eac108870 msgr2=0x7f6eac1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:38.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.998+0000 7f6eabfff640 1 --2- 172.21.15.38:0/1660899295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6eac108870 0x7f6eac1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:38.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.998+0000 7f6eabfff640 1 -- 172.21.15.38:0/1660899295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac1a0d30 con 0x7f6eac1035b0 2024-05-21T15:43:38.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.998+0000 7f6eabfff640 1 --2- 172.21.15.38:0/1660899295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eac1035b0 0x7f6eac1991f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c006010 tx=0x7f6e9c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:38.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.998+0000 7f6ea97fa640 1 -- 172.21.15.38:0/1660899295 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c03f030 con 0x7f6eac1035b0 2024-05-21T15:43:38.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.998+0000 7f6ea97fa640 1 -- 172.21.15.38:0/1660899295 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e9c003ad0 con 0x7f6eac1035b0 2024-05-21T15:43:38.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.998+0000 7f6ea97fa640 1 -- 172.21.15.38:0/1660899295 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c035070 con 0x7f6eac1035b0 2024-05-21T15:43:38.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.998+0000 7f6eb26f0640 1 -- 172.21.15.38:0/1660899295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eac1a0fc0 con 0x7f6eac1035b0 2024-05-21T15:43:38.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.998+0000 7f6eb26f0640 1 -- 172.21.15.38:0/1660899295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eac1a13d0 con 0x7f6eac1035b0 2024-05-21T15:43:38.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:38.999+0000 7f6eb26f0640 1 -- 172.21.15.38:0/1660899295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eac0fe6d0 con 0x7f6eac1035b0 2024-05-21T15:43:38.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.000+0000 7f6ea97fa640 1 -- 172.21.15.38:0/1660899295 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6e9c042030 con 0x7f6eac1035b0 2024-05-21T15:43:38.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.001+0000 7f6ea97fa640 1 --2- 172.21.15.38:0/1660899295 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e940a2c30 0x7f6e940a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:38.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.001+0000 7f6ea97fa640 1 -- 172.21.15.38:0/1660899295 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f6e9c0ef830 con 0x7f6eac1035b0 2024-05-21T15:43:38.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.001+0000 7f6eab7fe640 1 --2- 172.21.15.38:0/1660899295 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e940a2c30 0x7f6e940a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:39.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.002+0000 7f6eab7fe640 1 --2- 172.21.15.38:0/1660899295 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e940a2c30 0x7f6e940a5140 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6eac19a290 tx=0x7f6ea0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:39.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.005+0000 7f6ea97fa640 1 -- 172.21.15.38:0/1660899295 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6e9c0bc0f0 con 0x7f6eac1035b0 2024-05-21T15:43:39.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.192+0000 7f6eb26f0640 1 -- 172.21.15.38:0/1660899295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6eac02d4f0 con 0x7f6eac1035b0 2024-05-21T15:43:39.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.193+0000 7f6ea97fa640 1 -- 172.21.15.38:0/1660899295 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22704 (secure 0 0 0) 0x7f6e9c0c03f0 con 0x7f6eac1035b0 2024-05-21T15:43:39.191 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:43:39.191 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":90,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:34:23.996759+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:43:39.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.195+0000 7f6eb26f0640 1 -- 172.21.15.38:0/1660899295 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e940a2c30 msgr2=0x7f6e940a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.195+0000 7f6eb26f0640 1 --2- 172.21.15.38:0/1660899295 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e940a2c30 0x7f6e940a5140 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6eac19a290 tx=0x7f6ea0005f90 comp rx=0 tx=0).stop 2024-05-21T15:43:39.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.195+0000 7f6eb26f0640 1 -- 172.21.15.38:0/1660899295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eac1035b0 msgr2=0x7f6eac1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.195+0000 7f6eb26f0640 1 --2- 172.21.15.38:0/1660899295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eac1035b0 0x7f6eac1991f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c006010 tx=0x7f6e9c031b60 comp rx=0 tx=0).stop 2024-05-21T15:43:39.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.195+0000 7f6eb26f0640 1 -- 172.21.15.38:0/1660899295 shutdown_connections 2024-05-21T15:43:39.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.195+0000 7f6eb26f0640 1 --2- 172.21.15.38:0/1660899295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6eac108870 0x7f6eac1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.195+0000 7f6eb26f0640 1 --2- 172.21.15.38:0/1660899295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eac105f10 0x7f6eac199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.195+0000 7f6eb26f0640 1 --2- 172.21.15.38:0/1660899295 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f6e940a2c30 0x7f6e940a5140 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.195+0000 7f6eb26f0640 1 --2- 172.21.15.38:0/1660899295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eac1035b0 0x7f6eac1991f0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.195+0000 7f6eb26f0640 1 -- 172.21.15.38:0/1660899295 >> 172.21.15.38:0/1660899295 conn(0x7f6eac0fcbb0 msgr2=0x7f6eac104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:39.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.195+0000 7f6eb26f0640 1 -- 172.21.15.38:0/1660899295 shutdown_connections 2024-05-21T15:43:39.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.195+0000 7f6eb26f0640 1 -- 172.21.15.38:0/1660899295 wait complete. 2024-05-21T15:43:39.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:27:50.659367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:32:31.645647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:32:32.955451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:43:39.206 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:43:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.326+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3526686371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a141078c0 msgr2=0x7f1a14109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.326+0000 7f1a1ad65640 1 --2- 172.21.15.38:0/3526686371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a141078c0 0x7f1a14109ce0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009ca0 tx=0x7f1a0c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.326+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3526686371 shutdown_connections 2024-05-21T15:43:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.326+0000 7f1a1ad65640 1 --2- 172.21.15.38:0/3526686371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a141078c0 0x7f1a14109ce0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.326+0000 7f1a1ad65640 1 --2- 172.21.15.38:0/3526686371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a14102bc0 0x7f1a14102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.326+0000 7f1a1ad65640 1 --2- 172.21.15.38:0/3526686371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a14101980 0x7f1a14102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.326+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3526686371 >> 172.21.15.38:0/3526686371 conn(0x7f1a140fcb70 msgr2=0x7f1a140ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.326+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3526686371 shutdown_connections 2024-05-21T15:43:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.326+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3526686371 wait complete. 2024-05-21T15:43:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a1ad65640 1 Processor -- start 2024-05-21T15:43:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a1ad65640 1 -- start start 2024-05-21T15:43:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a1ad65640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a14101980 0x7f1a14199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a1ad65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a14102bc0 0x7f1a141997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a1ad65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a141078c0 0x7f1a141a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a1ad65640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1410e700 con 0x7f1a14101980 2024-05-21T15:43:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a1ad65640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1410e580 con 0x7f1a14102bc0 2024-05-21T15:43:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a1ad65640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1410e880 con 0x7f1a141078c0 2024-05-21T15:43:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a18b01640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a14101980 0x7f1a14199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a18b01640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a14101980 0x7f1a14199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50546/0 (socket says 172.21.15.38:50546) 2024-05-21T15:43:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a18b01640 1 -- 172.21.15.38:0/3426191858 learned_addr learned my addr 172.21.15.38:0/3426191858 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:43:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a13fff640 1 --2- 172.21.15.38:0/3426191858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a14102bc0 0x7f1a141997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:39.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a19302640 1 --2- 172.21.15.38:0/3426191858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a141078c0 0x7f1a141a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a18b01640 1 -- 172.21.15.38:0/3426191858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a141078c0 msgr2=0x7f1a141a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a18b01640 1 --2- 172.21.15.38:0/3426191858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a141078c0 0x7f1a141a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a18b01640 1 -- 172.21.15.38:0/3426191858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a14102bc0 msgr2=0x7f1a141997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a18b01640 1 --2- 172.21.15.38:0/3426191858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a14102bc0 0x7f1a141997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.327+0000 7f1a18b01640 1 -- 172.21.15.38:0/3426191858 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a141a0d50 con 0x7f1a14101980 2024-05-21T15:43:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.328+0000 7f1a18b01640 1 --2- 172.21.15.38:0/3426191858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a14101980 0x7f1a14199280 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800e9f0 tx=0x7f1a0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.328+0000 7f1a11ffb640 1 -- 172.21.15.38:0/3426191858 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800ccc0 con 0x7f1a14101980 2024-05-21T15:43:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.328+0000 7f1a11ffb640 1 -- 172.21.15.38:0/3426191858 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a080035f0 con 0x7f1a14101980 2024-05-21T15:43:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.328+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3426191858 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a141a0fe0 con 0x7f1a14101980 2024-05-21T15:43:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.328+0000 7f1a11ffb640 1 -- 172.21.15.38:0/3426191858 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800ce60 con 0x7f1a14101980 2024-05-21T15:43:39.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.328+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3426191858 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a141a1450 con 0x7f1a14101980 2024-05-21T15:43:39.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.329+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3426191858 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a14068d70 con 0x7f1a14101980 2024-05-21T15:43:39.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.330+0000 7f1a11ffb640 1 -- 172.21.15.38:0/3426191858 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1a080037a0 con 0x7f1a14101980 2024-05-21T15:43:39.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.331+0000 7f1a11ffb640 1 --2- 172.21.15.38:0/3426191858 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1a040a2b90 0x7f1a040a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:39.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.331+0000 7f1a13fff640 1 --2- 172.21.15.38:0/3426191858 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1a040a2b90 0x7f1a040a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:39.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.331+0000 7f1a11ffb640 1 -- 172.21.15.38:0/3426191858 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f1a080cf5e0 con 0x7f1a14101980 2024-05-21T15:43:39.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.332+0000 7f1a13fff640 1 --2- 172.21.15.38:0/3426191858 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1a040a2b90 0x7f1a040a50a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1a1419a320 tx=0x7f1a0000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:39.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.336+0000 7f1a11ffb640 1 -- 172.21.15.38:0/3426191858 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1a0809c490 con 0x7f1a14101980 2024-05-21T15:43:39.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.520+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3426191858 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a14062750 con 0x7f1a14101980 2024-05-21T15:43:39.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.521+0000 7f1a11ffb640 1 -- 172.21.15.38:0/3426191858 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f1a0809bd90 con 0x7f1a14101980 2024-05-21T15:43:39.518 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-05-21T15:43:39.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.522+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3426191858 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1a040a2b90 msgr2=0x7f1a040a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.522+0000 7f1a1ad65640 1 --2- 172.21.15.38:0/3426191858 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1a040a2b90 0x7f1a040a50a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1a1419a320 tx=0x7f1a0000b390 comp rx=0 tx=0).stop 2024-05-21T15:43:39.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.522+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3426191858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a14101980 msgr2=0x7f1a14199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.522+0000 7f1a1ad65640 1 --2- 172.21.15.38:0/3426191858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a14101980 0x7f1a14199280 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800e9f0 tx=0x7f1a0800eeb0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.523+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3426191858 shutdown_connections 2024-05-21T15:43:39.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.523+0000 7f1a1ad65640 1 --2- 172.21.15.38:0/3426191858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a141078c0 0x7f1a141a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.523+0000 7f1a1ad65640 1 --2- 172.21.15.38:0/3426191858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a14102bc0 0x7f1a141997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.523+0000 7f1a1ad65640 1 --2- 172.21.15.38:0/3426191858 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f1a040a2b90 0x7f1a040a50a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.523+0000 7f1a1ad65640 1 --2- 172.21.15.38:0/3426191858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a14101980 0x7f1a14199280 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.523+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3426191858 >> 172.21.15.38:0/3426191858 conn(0x7f1a140fcb70 msgr2=0x7f1a141038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:39.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.523+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3426191858 shutdown_connections 2024-05-21T15:43:39.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.523+0000 7f1a1ad65640 1 -- 172.21.15.38:0/3426191858 wait complete. 2024-05-21T15:43:39.532 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:43:39.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.653+0000 7f2b06b37640 1 -- 172.21.15.38:0/4286456947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b00107910 msgr2=0x7f2b00109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.653+0000 7f2b06b37640 1 --2- 172.21.15.38:0/4286456947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b00107910 0x7f2b00109d30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009ca0 tx=0x7f2af802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.653+0000 7f2b06b37640 1 -- 172.21.15.38:0/4286456947 shutdown_connections 2024-05-21T15:43:39.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.653+0000 7f2b06b37640 1 --2- 172.21.15.38:0/4286456947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b00107910 0x7f2b00109d30 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.653+0000 7f2b06b37640 1 --2- 172.21.15.38:0/4286456947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b00102c10 0x7f2b00103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.653+0000 7f2b06b37640 1 --2- 172.21.15.38:0/4286456947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b001019d0 0x7f2b001026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.653+0000 7f2b06b37640 1 -- 172.21.15.38:0/4286456947 >> 172.21.15.38:0/4286456947 conn(0x7f2b000fcbc0 msgr2=0x7f2b000ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:39.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.653+0000 7f2b06b37640 1 -- 172.21.15.38:0/4286456947 shutdown_connections 2024-05-21T15:43:39.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.653+0000 7f2b06b37640 1 -- 172.21.15.38:0/4286456947 wait complete. 2024-05-21T15:43:39.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2b06b37640 1 Processor -- start 2024-05-21T15:43:39.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2b06b37640 1 -- start start 2024-05-21T15:43:39.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2b06b37640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b001019d0 0x7f2b00199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:39.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2b06b37640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b00102c10 0x7f2b00199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:39.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2b06b37640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b00107910 0x7f2b001a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:39.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2b06b37640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0010e750 con 0x7f2b00102c10 2024-05-21T15:43:39.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2b06b37640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0010e5d0 con 0x7f2b001019d0 2024-05-21T15:43:39.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2b06b37640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0010e8d0 con 0x7f2b00107910 2024-05-21T15:43:39.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2afffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b00102c10 0x7f2b00199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:39.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2afffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b00102c10 0x7f2b00199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50574/0 (socket says 172.21.15.38:50574) 2024-05-21T15:43:39.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2afffff640 1 -- 172.21.15.38:0/2531869558 learned_addr learned my addr 172.21.15.38:0/2531869558 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:43:39.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2b048d3640 1 --2- 172.21.15.38:0/2531869558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b001019d0 0x7f2b00199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:39.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2afffff640 1 -- 172.21.15.38:0/2531869558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b00107910 msgr2=0x7f2b001a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2afffff640 1 --2- 172.21.15.38:0/2531869558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b00107910 0x7f2b001a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2afffff640 1 -- 172.21.15.38:0/2531869558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b001019d0 msgr2=0x7f2b00199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2afffff640 1 --2- 172.21.15.38:0/2531869558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b001019d0 0x7f2b00199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2afffff640 1 -- 172.21.15.38:0/2531869558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b001a10d0 con 0x7f2b00102c10 2024-05-21T15:43:39.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.654+0000 7f2b048d3640 1 --2- 172.21.15.38:0/2531869558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b001019d0 0x7f2b00199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:43:39.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.655+0000 7f2afffff640 1 --2- 172.21.15.38:0/2531869558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b00102c10 0x7f2b00199ad0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00e9f0 tx=0x7f2aec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:39.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.655+0000 7f2afdffb640 1 -- 172.21.15.38:0/2531869558 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00cd80 con 0x7f2b00102c10 2024-05-21T15:43:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.655+0000 7f2b06b37640 1 -- 172.21.15.38:0/2531869558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b001a13c0 con 0x7f2b00102c10 2024-05-21T15:43:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.655+0000 7f2afdffb640 1 -- 172.21.15.38:0/2531869558 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aec010910 con 0x7f2b00102c10 2024-05-21T15:43:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.655+0000 7f2afdffb640 1 -- 172.21.15.38:0/2531869558 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0188f0 con 0x7f2b00102c10 2024-05-21T15:43:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.655+0000 7f2b06b37640 1 -- 172.21.15.38:0/2531869558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b001a1960 con 0x7f2b00102c10 2024-05-21T15:43:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.656+0000 7f2b06b37640 1 -- 172.21.15.38:0/2531869558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b00106720 con 0x7f2b00102c10 2024-05-21T15:43:39.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.658+0000 7f2afdffb640 1 -- 172.21.15.38:0/2531869558 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2aec010ac0 con 0x7f2b00102c10 2024-05-21T15:43:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.659+0000 7f2afdffb640 1 --2- 172.21.15.38:0/2531869558 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2af00a2b90 0x7f2af00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.659+0000 7f2b048d3640 1 --2- 172.21.15.38:0/2531869558 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2af00a2b90 0x7f2af00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.659+0000 7f2afdffb640 1 -- 172.21.15.38:0/2531869558 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f2aec0cf980 con 0x7f2b00102c10 2024-05-21T15:43:39.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.659+0000 7f2b048d3640 1 --2- 172.21.15.38:0/2531869558 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2af00a2b90 0x7f2af00a50a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2b001025d0 tx=0x7f2af4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:39.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.662+0000 7f2afdffb640 1 -- 172.21.15.38:0/2531869558 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2aec09c890 con 0x7f2b00102c10 2024-05-21T15:43:39.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.846+0000 7f2b06b37640 1 -- 172.21.15.38:0/2531869558 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2b0002d3c0 con 0x7f2b00102c10 2024-05-21T15:43:39.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.846+0000 7f2afdffb640 1 -- 172.21.15.38:0/2531869558 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f2aec09c190 con 0x7f2b00102c10 2024-05-21T15:43:39.844 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-05-21T15:43:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.848+0000 7f2b06b37640 1 -- 172.21.15.38:0/2531869558 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2af00a2b90 msgr2=0x7f2af00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.848+0000 7f2b06b37640 1 --2- 172.21.15.38:0/2531869558 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2af00a2b90 0x7f2af00a50a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2b001025d0 tx=0x7f2af4005f90 comp rx=0 tx=0).stop 2024-05-21T15:43:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.848+0000 7f2b06b37640 1 -- 172.21.15.38:0/2531869558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b00102c10 msgr2=0x7f2b00199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.848+0000 7f2b06b37640 1 --2- 172.21.15.38:0/2531869558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b00102c10 0x7f2b00199ad0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00e9f0 tx=0x7f2aec00eeb0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.848+0000 7f2b06b37640 1 -- 172.21.15.38:0/2531869558 shutdown_connections 2024-05-21T15:43:39.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.848+0000 7f2b06b37640 1 --2- 172.21.15.38:0/2531869558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b00107910 0x7f2b001a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.848+0000 7f2b06b37640 1 --2- 172.21.15.38:0/2531869558 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f2af00a2b90 0x7f2af00a50a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.848+0000 7f2b06b37640 1 --2- 172.21.15.38:0/2531869558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b00102c10 0x7f2b00199ad0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.848+0000 7f2b06b37640 1 --2- 172.21.15.38:0/2531869558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b001019d0 0x7f2b00199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.848+0000 7f2b06b37640 1 -- 172.21.15.38:0/2531869558 >> 172.21.15.38:0/2531869558 conn(0x7f2b000fcbc0 msgr2=0x7f2b001038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:39.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.848+0000 7f2b06b37640 1 -- 172.21.15.38:0/2531869558 shutdown_connections 2024-05-21T15:43:39.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.848+0000 7f2b06b37640 1 -- 172.21.15.38:0/2531869558 wait complete. 2024-05-21T15:43:39.857 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:39 smithi038 ceph-mon[31995]: pgmap v628: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-21T15:43:39.857 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:39 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1660899295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:43:39.857 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:39 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3426191858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:43:39.858 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:43:39.918 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:39 smithi053 ceph-mon[36496]: pgmap v628: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-21T15:43:39.918 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:39 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1660899295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:43:39.918 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:39 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3426191858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:43:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.978+0000 7f129e4ca640 1 -- 172.21.15.38:0/2178384795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12981078d0 msgr2=0x7f1298109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.978+0000 7f129e4ca640 1 --2- 172.21.15.38:0/2178384795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12981078d0 0x7f1298109cf0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1290009ca0 tx=0x7f129002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.978+0000 7f129e4ca640 1 -- 172.21.15.38:0/2178384795 shutdown_connections 2024-05-21T15:43:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.978+0000 7f129e4ca640 1 --2- 172.21.15.38:0/2178384795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12981078d0 0x7f1298109cf0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.978+0000 7f129e4ca640 1 --2- 172.21.15.38:0/2178384795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1298102bd0 0x7f1298102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.978+0000 7f129e4ca640 1 --2- 172.21.15.38:0/2178384795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1298101990 0x7f1298102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.978+0000 7f129e4ca640 1 -- 172.21.15.38:0/2178384795 >> 172.21.15.38:0/2178384795 conn(0x7f12980fcb80 msgr2=0x7f12980ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.979+0000 7f129e4ca640 1 -- 172.21.15.38:0/2178384795 shutdown_connections 2024-05-21T15:43:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.979+0000 7f129e4ca640 1 -- 172.21.15.38:0/2178384795 wait complete. 2024-05-21T15:43:39.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.979+0000 7f129e4ca640 1 Processor -- start 2024-05-21T15:43:39.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.979+0000 7f129e4ca640 1 -- start start 2024-05-21T15:43:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129e4ca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1298101990 0x7f1298198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129e4ca640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1298102bd0 0x7f1298198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129e4ca640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12981078d0 0x7f129819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129e4ca640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f129810d640 con 0x7f12981078d0 2024-05-21T15:43:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129e4ca640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f129810d4c0 con 0x7f1298102bd0 2024-05-21T15:43:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129e4ca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f129810d7c0 con 0x7f1298101990 2024-05-21T15:43:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129ca67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12981078d0 0x7f129819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129ca67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12981078d0 0x7f129819f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50598/0 (socket says 172.21.15.38:50598) 2024-05-21T15:43:39.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129ca67640 1 -- 172.21.15.38:0/1339858949 learned_addr learned my addr 172.21.15.38:0/1339858949 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:43:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f12977fe640 1 --2- 172.21.15.38:0/1339858949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1298102bd0 0x7f1298198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f1297fff640 1 --2- 172.21.15.38:0/1339858949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1298101990 0x7f1298198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129ca67640 1 -- 172.21.15.38:0/1339858949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1298101990 msgr2=0x7f1298198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129ca67640 1 --2- 172.21.15.38:0/1339858949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1298101990 0x7f1298198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129ca67640 1 -- 172.21.15.38:0/1339858949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1298102bd0 msgr2=0x7f1298198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129ca67640 1 --2- 172.21.15.38:0/1339858949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1298102bd0 0x7f1298198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129ca67640 1 -- 172.21.15.38:0/1339858949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129819fc70 con 0x7f12981078d0 2024-05-21T15:43:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f129ca67640 1 --2- 172.21.15.38:0/1339858949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12981078d0 0x7f129819f600 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f12900319f0 tx=0x7f129002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.980+0000 7f12957fa640 1 -- 172.21.15.38:0/1339858949 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290003bf0 con 0x7f12981078d0 2024-05-21T15:43:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.981+0000 7f129e4ca640 1 -- 172.21.15.38:0/1339858949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129819ff00 con 0x7f12981078d0 2024-05-21T15:43:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.981+0000 7f12957fa640 1 -- 172.21.15.38:0/1339858949 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1290003d90 con 0x7f12981078d0 2024-05-21T15:43:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.981+0000 7f12957fa640 1 -- 172.21.15.38:0/1339858949 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129002fd00 con 0x7f12981078d0 2024-05-21T15:43:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.981+0000 7f129e4ca640 1 -- 172.21.15.38:0/1339858949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12981a03c0 con 0x7f12981078d0 2024-05-21T15:43:39.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.982+0000 7f129e4ca640 1 -- 172.21.15.38:0/1339858949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12981066e0 con 0x7f12981078d0 2024-05-21T15:43:39.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.983+0000 7f12957fa640 1 -- 172.21.15.38:0/1339858949 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1290004040 con 0x7f12981078d0 2024-05-21T15:43:39.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.984+0000 7f12957fa640 1 --2- 172.21.15.38:0/1339858949 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12800a2b90 0x7f12800a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:39.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.984+0000 7f1297fff640 1 --2- 172.21.15.38:0/1339858949 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12800a2b90 0x7f12800a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:39.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.984+0000 7f12957fa640 1 -- 172.21.15.38:0/1339858949 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f12900f0970 con 0x7f12981078d0 2024-05-21T15:43:39.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.984+0000 7f1297fff640 1 --2- 172.21.15.38:0/1339858949 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12800a2b90 0x7f12800a50a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1288005fd0 tx=0x7f1288005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:39.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:39.987+0000 7f12957fa640 1 -- 172.21.15.38:0/1339858949 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f12900bd880 con 0x7f12981078d0 2024-05-21T15:43:40.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:39 smithi045 ceph-mon[36409]: pgmap v628: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-21T15:43:40.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:39 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1660899295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:43:40.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:39 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3426191858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:43:40.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.172+0000 7f129e4ca640 1 -- 172.21.15.38:0/1339858949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f129802d4a0 con 0x7f12981078d0 2024-05-21T15:43:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.173+0000 7f12957fa640 1 -- 172.21.15.38:0/1339858949 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f12900bd180 con 0x7f12981078d0 2024-05-21T15:43:40.171 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-05-21T15:43:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.175+0000 7f129e4ca640 1 -- 172.21.15.38:0/1339858949 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12800a2b90 msgr2=0x7f12800a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.175+0000 7f129e4ca640 1 --2- 172.21.15.38:0/1339858949 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12800a2b90 0x7f12800a50a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1288005fd0 tx=0x7f1288005850 comp rx=0 tx=0).stop 2024-05-21T15:43:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.175+0000 7f129e4ca640 1 -- 172.21.15.38:0/1339858949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12981078d0 msgr2=0x7f129819f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.175+0000 7f129e4ca640 1 --2- 172.21.15.38:0/1339858949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12981078d0 0x7f129819f600 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f12900319f0 tx=0x7f129002f380 comp rx=0 tx=0).stop 2024-05-21T15:43:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.175+0000 7f129e4ca640 1 -- 172.21.15.38:0/1339858949 shutdown_connections 2024-05-21T15:43:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.175+0000 7f129e4ca640 1 --2- 172.21.15.38:0/1339858949 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f12800a2b90 0x7f12800a50a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.175+0000 7f129e4ca640 1 --2- 172.21.15.38:0/1339858949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12981078d0 0x7f129819f600 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.175+0000 7f129e4ca640 1 --2- 172.21.15.38:0/1339858949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1298102bd0 0x7f1298198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.175+0000 7f129e4ca640 1 --2- 172.21.15.38:0/1339858949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1298101990 0x7f1298198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.175+0000 7f129e4ca640 1 -- 172.21.15.38:0/1339858949 >> 172.21.15.38:0/1339858949 conn(0x7f12980fcb80 msgr2=0x7f12981038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.175+0000 7f129e4ca640 1 -- 172.21.15.38:0/1339858949 shutdown_connections 2024-05-21T15:43:40.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.175+0000 7f129e4ca640 1 -- 172.21.15.38:0/1339858949 wait complete. 2024-05-21T15:43:40.185 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-21T15:43:40.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.303+0000 7fe62d75e640 1 -- 172.21.15.38:0/151087272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe628103d20 msgr2=0x7fe628104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:40.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.303+0000 7fe62d75e640 1 --2- 172.21.15.38:0/151087272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe628103d20 0x7fe628104130 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe610009900 tx=0x7fe61002a410 comp rx=0 tx=0).stop 2024-05-21T15:43:40.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.304+0000 7fe62d75e640 1 -- 172.21.15.38:0/151087272 shutdown_connections 2024-05-21T15:43:40.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.304+0000 7fe62d75e640 1 --2- 172.21.15.38:0/151087272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe628108a20 0x7fe62810ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.304+0000 7fe62d75e640 1 --2- 172.21.15.38:0/151087272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe628103d20 0x7fe628104130 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.304+0000 7fe62d75e640 1 --2- 172.21.15.38:0/151087272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe628102ae0 0x7fe6281037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.304+0000 7fe62d75e640 1 -- 172.21.15.38:0/151087272 >> 172.21.15.38:0/151087272 conn(0x7fe6280fdcf0 msgr2=0x7fe628100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:40.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.304+0000 7fe62d75e640 1 -- 172.21.15.38:0/151087272 shutdown_connections 2024-05-21T15:43:40.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.304+0000 7fe62d75e640 1 -- 172.21.15.38:0/151087272 wait complete. 2024-05-21T15:43:40.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.304+0000 7fe62d75e640 1 Processor -- start 2024-05-21T15:43:40.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.304+0000 7fe62d75e640 1 -- start start 2024-05-21T15:43:40.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.304+0000 7fe62d75e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe628102ae0 0x7fe628199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:40.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.304+0000 7fe62d75e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe628103d20 0x7fe6281997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:40.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.304+0000 7fe62d75e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe628108a20 0x7fe6281a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:40.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe62d75e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe62810e730 con 0x7fe628103d20 2024-05-21T15:43:40.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe62d75e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe62810e5b0 con 0x7fe628102ae0 2024-05-21T15:43:40.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe62d75e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe62810e8b0 con 0x7fe628108a20 2024-05-21T15:43:40.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe61ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe628103d20 0x7fe6281997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:40.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe61ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe628103d20 0x7fe6281997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50616/0 (socket says 172.21.15.38:50616) 2024-05-21T15:43:40.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe61ffff640 1 -- 172.21.15.38:0/607899799 learned_addr learned my addr 172.21.15.38:0/607899799 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:43:40.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe626ffd640 1 --2- 172.21.15.38:0/607899799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe628102ae0 0x7fe628199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:40.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe6277fe640 1 --2- 172.21.15.38:0/607899799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe628108a20 0x7fe6281a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:40.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe61ffff640 1 -- 172.21.15.38:0/607899799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe628108a20 msgr2=0x7fe6281a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:40.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe61ffff640 1 --2- 172.21.15.38:0/607899799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe628108a20 0x7fe6281a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe61ffff640 1 -- 172.21.15.38:0/607899799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe628102ae0 msgr2=0x7fe628199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:40.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe61ffff640 1 --2- 172.21.15.38:0/607899799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe628102ae0 0x7fe628199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe61ffff640 1 -- 172.21.15.38:0/607899799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6281a0db0 con 0x7fe628103d20 2024-05-21T15:43:40.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe61ffff640 1 --2- 172.21.15.38:0/607899799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe628103d20 0x7fe6281997b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe610006010 tx=0x7fe6100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:40.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe624ff9640 1 -- 172.21.15.38:0/607899799 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61003f030 con 0x7fe628103d20 2024-05-21T15:43:40.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe62d75e640 1 -- 172.21.15.38:0/607899799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6281a1040 con 0x7fe628103d20 2024-05-21T15:43:40.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.305+0000 7fe624ff9640 1 -- 172.21.15.38:0/607899799 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe610003ce0 con 0x7fe628103d20 2024-05-21T15:43:40.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.306+0000 7fe624ff9640 1 -- 172.21.15.38:0/607899799 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610035070 con 0x7fe628103d20 2024-05-21T15:43:40.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.306+0000 7fe62d75e640 1 -- 172.21.15.38:0/607899799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6281a1530 con 0x7fe628103d20 2024-05-21T15:43:40.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.306+0000 7fe62d75e640 1 -- 172.21.15.38:0/607899799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe628107830 con 0x7fe628103d20 2024-05-21T15:43:40.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.308+0000 7fe624ff9640 1 -- 172.21.15.38:0/607899799 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe610042030 con 0x7fe628103d20 2024-05-21T15:43:40.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.309+0000 7fe624ff9640 1 --2- 172.21.15.38:0/607899799 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe6140a2b90 0x7fe6140a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:40.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.309+0000 7fe624ff9640 1 -- 172.21.15.38:0/607899799 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe6100efc10 con 0x7fe628103d20 2024-05-21T15:43:40.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.309+0000 7fe626ffd640 1 --2- 172.21.15.38:0/607899799 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe6140a2b90 0x7fe6140a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:40.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.309+0000 7fe626ffd640 1 --2- 172.21.15.38:0/607899799 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe6140a2b90 0x7fe6140a50a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe6281036e0 tx=0x7fe618005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:40.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.312+0000 7fe624ff9640 1 -- 172.21.15.38:0/607899799 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe6100bcac0 con 0x7fe628103d20 2024-05-21T15:43:40.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.513+0000 7fe62d75e640 1 -- 172.21.15.38:0/607899799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe62802d2a0 con 0x7fe628103d20 2024-05-21T15:43:40.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.514+0000 7fe624ff9640 1 -- 172.21.15.38:0/607899799 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+591 (secure 0 0 0) 0x7fe6100bc3c0 con 0x7fe628103d20 2024-05-21T15:43:40.512 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/592277081 2024-05-22T15:25:22.117984+0000 2024-05-21T15:43:40.512 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1186842456 2024-05-22T15:25:22.117984+0000 2024-05-21T15:43:40.512 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/1481949523 2024-05-22T15:25:22.117984+0000 2024-05-21T15:43:40.512 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/1481949523 2024-05-22T15:25:22.117984+0000 2024-05-21T15:43:40.512 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1771432089 2024-05-22T15:24:54.404192+0000 2024-05-21T15:43:40.512 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/3523378851 2024-05-22T15:24:54.404192+0000 2024-05-21T15:43:40.512 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2903150778 2024-05-22T15:24:54.404192+0000 2024-05-21T15:43:40.512 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/3523378851 2024-05-22T15:24:54.404192+0000 2024-05-21T15:43:40.512 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2666943685 2024-05-22T15:24:54.404192+0000 2024-05-21T15:43:40.512 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/3586196771 2024-05-22T15:25:22.117984+0000 2024-05-21T15:43:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:listed 10 entries 2024-05-21T15:43:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.516+0000 7fe62d75e640 1 -- 172.21.15.38:0/607899799 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe6140a2b90 msgr2=0x7fe6140a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.516+0000 7fe62d75e640 1 --2- 172.21.15.38:0/607899799 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe6140a2b90 0x7fe6140a50a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe6281036e0 tx=0x7fe618005f90 comp rx=0 tx=0).stop 2024-05-21T15:43:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.516+0000 7fe62d75e640 1 -- 172.21.15.38:0/607899799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe628103d20 msgr2=0x7fe6281997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.516+0000 7fe62d75e640 1 --2- 172.21.15.38:0/607899799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe628103d20 0x7fe6281997b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe610006010 tx=0x7fe6100042f0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.516+0000 7fe62d75e640 1 -- 172.21.15.38:0/607899799 shutdown_connections 2024-05-21T15:43:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.516+0000 7fe62d75e640 1 --2- 172.21.15.38:0/607899799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe628108a20 0x7fe6281a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.517+0000 7fe62d75e640 1 --2- 172.21.15.38:0/607899799 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe6140a2b90 0x7fe6140a50a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.517+0000 7fe62d75e640 1 --2- 172.21.15.38:0/607899799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe628103d20 0x7fe6281997b0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.517+0000 7fe62d75e640 1 --2- 172.21.15.38:0/607899799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe628102ae0 0x7fe628199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.517+0000 7fe62d75e640 1 -- 172.21.15.38:0/607899799 >> 172.21.15.38:0/607899799 conn(0x7fe6280fdcf0 msgr2=0x7fe628104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.517+0000 7fe62d75e640 1 -- 172.21.15.38:0/607899799 shutdown_connections 2024-05-21T15:43:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.517+0000 7fe62d75e640 1 -- 172.21.15.38:0/607899799 wait complete. 2024-05-21T15:43:40.526 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-21T15:43:40.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:43:40.527 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-21T15:43:40.575 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-05-21T15:43:40.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:43:40.575 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-21T15:43:40.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:43:40.624 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-05-21T15:43:40.710 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-21T15:43:40.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:43:40.711 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-21T15:43:40.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:43:40.759 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-05-21T15:43:40.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:40 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2531869558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:43:40.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:40 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/1339858949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:43:40.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:40 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/607899799' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-21T15:43:40.836 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-21T15:43:40.846 INFO:tasks.cephadm:Teardown begin 2024-05-21T15:43:40.846 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-21T15:43:40.846 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:43:40.913 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:40 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2531869558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:43:40.913 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:40 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/1339858949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:43:40.913 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:40 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/607899799' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-21T15:43:40.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.996+0000 7fe0eea37640 1 -- 172.21.15.38:0/1895610823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0e8102ae0 msgr2=0x7fe0e81037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:40.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.996+0000 7fe0eea37640 1 --2- 172.21.15.38:0/1895610823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0e8102ae0 0x7fe0e81037e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009900 tx=0x7fe0d802a410 comp rx=0 tx=0).stop 2024-05-21T15:43:40.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.996+0000 7fe0eea37640 1 -- 172.21.15.38:0/1895610823 shutdown_connections 2024-05-21T15:43:40.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.996+0000 7fe0eea37640 1 --2- 172.21.15.38:0/1895610823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0e8108a20 0x7fe0e810ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.996+0000 7fe0eea37640 1 --2- 172.21.15.38:0/1895610823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e8103d20 0x7fe0e8104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.996+0000 7fe0eea37640 1 --2- 172.21.15.38:0/1895610823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0e8102ae0 0x7fe0e81037e0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.996+0000 7fe0eea37640 1 -- 172.21.15.38:0/1895610823 >> 172.21.15.38:0/1895610823 conn(0x7fe0e80fdcf0 msgr2=0x7fe0e8100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:40.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.996+0000 7fe0eea37640 1 -- 172.21.15.38:0/1895610823 shutdown_connections 2024-05-21T15:43:40.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.996+0000 7fe0eea37640 1 -- 172.21.15.38:0/1895610823 wait complete. 2024-05-21T15:43:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.997+0000 7fe0eea37640 1 Processor -- start 2024-05-21T15:43:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.997+0000 7fe0eea37640 1 -- start start 2024-05-21T15:43:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.997+0000 7fe0eea37640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0e8102ae0 0x7fe0e8076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.997+0000 7fe0eea37640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0e8103d20 0x7fe0e8074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.997+0000 7fe0eea37640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e8108a20 0x7fe0e80754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.997+0000 7fe0eea37640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e810e730 con 0x7fe0e8102ae0 2024-05-21T15:43:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.997+0000 7fe0eea37640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e810e5b0 con 0x7fe0e8108a20 2024-05-21T15:43:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.997+0000 7fe0eea37640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e810e8b0 con 0x7fe0e8103d20 2024-05-21T15:43:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.997+0000 7fe0e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0e8102ae0 0x7fe0e8076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.997+0000 7fe0e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0e8102ae0 0x7fe0e8076910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50634/0 (socket says 172.21.15.38:50634) 2024-05-21T15:43:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.997+0000 7fe0e7fff640 1 -- 172.21.15.38:0/3011826794 learned_addr learned my addr 172.21.15.38:0/3011826794 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:43:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.997+0000 7fe0e77fe640 1 --2- 172.21.15.38:0/3011826794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0e8103d20 0x7fe0e8074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.998+0000 7fe0ecfd4640 1 --2- 172.21.15.38:0/3011826794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e8108a20 0x7fe0e80754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.998+0000 7fe0e7fff640 1 -- 172.21.15.38:0/3011826794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0e8103d20 msgr2=0x7fe0e8074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.998+0000 7fe0e7fff640 1 --2- 172.21.15.38:0/3011826794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0e8103d20 0x7fe0e8074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.998+0000 7fe0e7fff640 1 -- 172.21.15.38:0/3011826794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e8108a20 msgr2=0x7fe0e80754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.998+0000 7fe0e7fff640 1 --2- 172.21.15.38:0/3011826794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e8108a20 0x7fe0e80754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.998+0000 7fe0e7fff640 1 -- 172.21.15.38:0/3011826794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e8075c20 con 0x7fe0e8102ae0 2024-05-21T15:43:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.998+0000 7fe0e7fff640 1 --2- 172.21.15.38:0/3011826794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0e8102ae0 0x7fe0e8076910 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8006010 tx=0x7fe0d8031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.998+0000 7fe0e57fa640 1 -- 172.21.15.38:0/3011826794 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d803f030 con 0x7fe0e8102ae0 2024-05-21T15:43:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.998+0000 7fe0eea37640 1 -- 172.21.15.38:0/3011826794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e81a3300 con 0x7fe0e8102ae0 2024-05-21T15:43:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.998+0000 7fe0e57fa640 1 -- 172.21.15.38:0/3011826794 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0d8003ad0 con 0x7fe0e8102ae0 2024-05-21T15:43:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.998+0000 7fe0e57fa640 1 -- 172.21.15.38:0/3011826794 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8035070 con 0x7fe0e8102ae0 2024-05-21T15:43:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.998+0000 7fe0eea37640 1 -- 172.21.15.38:0/3011826794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e81a3760 con 0x7fe0e8102ae0 2024-05-21T15:43:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:40.999+0000 7fe0eea37640 1 -- 172.21.15.38:0/3011826794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e8107830 con 0x7fe0e8102ae0 2024-05-21T15:43:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.000+0000 7fe0e57fa640 1 -- 172.21.15.38:0/3011826794 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe0d8042030 con 0x7fe0e8102ae0 2024-05-21T15:43:41.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.001+0000 7fe0e57fa640 1 --2- 172.21.15.38:0/3011826794 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe0d00a2c30 0x7fe0d00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:41.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.001+0000 7fe0e57fa640 1 -- 172.21.15.38:0/3011826794 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe0d80ef8e0 con 0x7fe0e8102ae0 2024-05-21T15:43:41.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.001+0000 7fe0e77fe640 1 --2- 172.21.15.38:0/3011826794 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe0d00a2c30 0x7fe0d00a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:41.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.002+0000 7fe0e77fe640 1 --2- 172.21.15.38:0/3011826794 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe0d00a2c30 0x7fe0d00a5140 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80762b0 tx=0x7fe0dc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:41.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.005+0000 7fe0e57fa640 1 -- 172.21.15.38:0/3011826794 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe0d80bc7f0 con 0x7fe0e8102ae0 2024-05-21T15:43:41.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:40 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2531869558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:43:41.008 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:40 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/1339858949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:43:41.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:40 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/607899799' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-21T15:43:41.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.229+0000 7fe0eea37640 1 -- 172.21.15.38:0/3011826794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0e802d2a0 con 0x7fe0e8102ae0 2024-05-21T15:43:41.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.230+0000 7fe0e57fa640 1 -- 172.21.15.38:0/3011826794 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11920 (secure 0 0 0) 0x7fe0d80bc0f0 con 0x7fe0e8102ae0 2024-05-21T15:43:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:43:41.228 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:42:28.496301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24515,24550,24602,24551,24523]},"id":1}]} 2024-05-21T15:43:41.228 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 40 2024-05-21T15:43:41.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.232+0000 7fe0eea37640 1 -- 172.21.15.38:0/3011826794 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe0d00a2c30 msgr2=0x7fe0d00a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:41.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.232+0000 7fe0eea37640 1 --2- 172.21.15.38:0/3011826794 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe0d00a2c30 0x7fe0d00a5140 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80762b0 tx=0x7fe0dc00b390 comp rx=0 tx=0).stop 2024-05-21T15:43:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.232+0000 7fe0eea37640 1 -- 172.21.15.38:0/3011826794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0e8102ae0 msgr2=0x7fe0e8076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.232+0000 7fe0eea37640 1 --2- 172.21.15.38:0/3011826794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0e8102ae0 0x7fe0e8076910 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8006010 tx=0x7fe0d8031b60 comp rx=0 tx=0).stop 2024-05-21T15:43:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.232+0000 7fe0eea37640 1 -- 172.21.15.38:0/3011826794 shutdown_connections 2024-05-21T15:43:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.232+0000 7fe0eea37640 1 --2- 172.21.15.38:0/3011826794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e8108a20 0x7fe0e80754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.232+0000 7fe0eea37640 1 --2- 172.21.15.38:0/3011826794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0e8103d20 0x7fe0e8074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.232+0000 7fe0eea37640 1 --2- 172.21.15.38:0/3011826794 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe0d00a2c30 0x7fe0d00a5140 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.232+0000 7fe0eea37640 1 --2- 172.21.15.38:0/3011826794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0e8102ae0 0x7fe0e8076910 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.232+0000 7fe0eea37640 1 -- 172.21.15.38:0/3011826794 >> 172.21.15.38:0/3011826794 conn(0x7fe0e80fdcf0 msgr2=0x7fe0e8104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.232+0000 7fe0eea37640 1 -- 172.21.15.38:0/3011826794 shutdown_connections 2024-05-21T15:43:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.232+0000 7fe0eea37640 1 -- 172.21.15.38:0/3011826794 wait complete. 2024-05-21T15:43:41.241 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:43:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.360+0000 7fe389d38640 1 -- 172.21.15.38:0/3638287109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3841078f0 msgr2=0x7fe384109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.360+0000 7fe389d38640 1 --2- 172.21.15.38:0/3638287109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3841078f0 0x7fe384109d10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009ca0 tx=0x7fe37c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.360+0000 7fe389d38640 1 -- 172.21.15.38:0/3638287109 shutdown_connections 2024-05-21T15:43:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.360+0000 7fe389d38640 1 --2- 172.21.15.38:0/3638287109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3841078f0 0x7fe384109d10 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.360+0000 7fe389d38640 1 --2- 172.21.15.38:0/3638287109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe384102bf0 0x7fe384103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.360+0000 7fe389d38640 1 --2- 172.21.15.38:0/3638287109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3841019b0 0x7fe3841026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.360+0000 7fe389d38640 1 -- 172.21.15.38:0/3638287109 >> 172.21.15.38:0/3638287109 conn(0x7fe3840fcba0 msgr2=0x7fe3840ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.360+0000 7fe389d38640 1 -- 172.21.15.38:0/3638287109 shutdown_connections 2024-05-21T15:43:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.360+0000 7fe389d38640 1 -- 172.21.15.38:0/3638287109 wait complete. 2024-05-21T15:43:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.360+0000 7fe389d38640 1 Processor -- start 2024-05-21T15:43:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe389d38640 1 -- start start 2024-05-21T15:43:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe389d38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3841019b0 0x7fe3841991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe389d38640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe384102bf0 0x7fe384199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe389d38640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3841078f0 0x7fe3841a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe389d38640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe38410e730 con 0x7fe3841019b0 2024-05-21T15:43:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe389d38640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe38410e5b0 con 0x7fe384102bf0 2024-05-21T15:43:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe389d38640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe38410e8b0 con 0x7fe3841078f0 2024-05-21T15:43:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe3837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3841019b0 0x7fe3841991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe3837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3841019b0 0x7fe3841991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50664/0 (socket says 172.21.15.38:50664) 2024-05-21T15:43:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe3837fe640 1 -- 172.21.15.38:0/2028821689 learned_addr learned my addr 172.21.15.38:0/2028821689 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:43:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe382ffd640 1 --2- 172.21.15.38:0/2028821689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe384102bf0 0x7fe384199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe383fff640 1 --2- 172.21.15.38:0/2028821689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3841078f0 0x7fe3841a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe3837fe640 1 -- 172.21.15.38:0/2028821689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3841078f0 msgr2=0x7fe3841a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe3837fe640 1 --2- 172.21.15.38:0/2028821689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3841078f0 0x7fe3841a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe3837fe640 1 -- 172.21.15.38:0/2028821689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe384102bf0 msgr2=0x7fe384199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe3837fe640 1 --2- 172.21.15.38:0/2028821689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe384102bf0 0x7fe384199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.361+0000 7fe3837fe640 1 -- 172.21.15.38:0/2028821689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3841a0d20 con 0x7fe3841019b0 2024-05-21T15:43:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.362+0000 7fe3837fe640 1 --2- 172.21.15.38:0/2028821689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3841019b0 0x7fe3841991e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe37400e9f0 tx=0x7fe37400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.362+0000 7fe380ff9640 1 -- 172.21.15.38:0/2028821689 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37400ccc0 con 0x7fe3841019b0 2024-05-21T15:43:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.362+0000 7fe389d38640 1 -- 172.21.15.38:0/2028821689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3841a0fb0 con 0x7fe3841019b0 2024-05-21T15:43:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.362+0000 7fe380ff9640 1 -- 172.21.15.38:0/2028821689 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3740035f0 con 0x7fe3841019b0 2024-05-21T15:43:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.362+0000 7fe380ff9640 1 -- 172.21.15.38:0/2028821689 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37400ce60 con 0x7fe3841019b0 2024-05-21T15:43:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.362+0000 7fe389d38640 1 -- 172.21.15.38:0/2028821689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3841a1470 con 0x7fe3841019b0 2024-05-21T15:43:41.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.363+0000 7fe389d38640 1 -- 172.21.15.38:0/2028821689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe384105ff0 con 0x7fe3841019b0 2024-05-21T15:43:41.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.364+0000 7fe380ff9640 1 -- 172.21.15.38:0/2028821689 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe3740037a0 con 0x7fe3841019b0 2024-05-21T15:43:41.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.365+0000 7fe380ff9640 1 --2- 172.21.15.38:0/2028821689 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe36c0a2c30 0x7fe36c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:41.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.365+0000 7fe380ff9640 1 -- 172.21.15.38:0/2028821689 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7fe3740cf6f0 con 0x7fe3841019b0 2024-05-21T15:43:41.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.365+0000 7fe382ffd640 1 --2- 172.21.15.38:0/2028821689 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe36c0a2c30 0x7fe36c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:41.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.366+0000 7fe382ffd640 1 --2- 172.21.15.38:0/2028821689 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe36c0a2c30 0x7fe36c0a5140 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe38419a280 tx=0x7fe37800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:41.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.369+0000 7fe380ff9640 1 -- 172.21.15.38:0/2028821689 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe37409c5a0 con 0x7fe3841019b0 2024-05-21T15:43:41.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.596+0000 7fe389d38640 1 -- 172.21.15.38:0/2028821689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe38402d2a0 con 0x7fe3841019b0 2024-05-21T15:43:41.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.597+0000 7fe380ff9640 1 -- 172.21.15.38:0/2028821689 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11920 (secure 0 0 0) 0x7fe37409bea0 con 0x7fe3841019b0 2024-05-21T15:43:41.594 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:43:41.595 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3555626860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3555626860},{"type":"v1","addr":"172.21.15.38:6835","nonce":3555626860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/2818680145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":2818680145},{"type":"v1","addr":"172.21.15.38:6837","nonce":2818680145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:32:33.564616+0000","modified":"2024-05-21T15:42:28.496301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24602,"mds_1":24551,"mds_2":24550,"mds_3":24523,"mds_4":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14655":{"gid":14655,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2313367718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2313367718},{"type":"v1","addr":"172.21.15.38:6839","nonce":2313367718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14667":{"gid":14667,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/106491696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":106491696},{"type":"v1","addr":"172.21.15.38:6841","nonce":106491696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1390207425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1390207425},{"type":"v1","addr":"172.21.15.53:6833","nonce":1390207425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6833/2551213098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2551213098},{"type":"v1","addr":"172.21.15.45:6833","nonce":2551213098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/1330413091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1330413091},{"type":"v1","addr":"172.21.15.53:6835","nonce":1330413091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.45:6835/3817948392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3817948392},{"type":"v1","addr":"172.21.15.45:6835","nonce":3817948392}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6837/2684842704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2684842704},{"type":"v1","addr":"172.21.15.53:6837","nonce":2684842704}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6839/3594548281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3594548281},{"type":"v1","addr":"172.21.15.53:6839","nonce":3594548281}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6837/977772487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":977772487},{"type":"v1","addr":"172.21.15.45:6837","nonce":977772487}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6839/2790647513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2790647513},{"type":"v1","addr":"172.21.15.45:6839","nonce":2790647513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24602,"qdb_cluster":[24515,24550,24602,24551,24523]},"id":1}]} 2024-05-21T15:43:41.595 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 40 2024-05-21T15:43:41.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.599+0000 7fe389d38640 1 -- 172.21.15.38:0/2028821689 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe36c0a2c30 msgr2=0x7fe36c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:41.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.599+0000 7fe389d38640 1 --2- 172.21.15.38:0/2028821689 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe36c0a2c30 0x7fe36c0a5140 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe38419a280 tx=0x7fe37800b390 comp rx=0 tx=0).stop 2024-05-21T15:43:41.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.599+0000 7fe389d38640 1 -- 172.21.15.38:0/2028821689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3841019b0 msgr2=0x7fe3841991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:41.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.599+0000 7fe389d38640 1 --2- 172.21.15.38:0/2028821689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3841019b0 0x7fe3841991e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe37400e9f0 tx=0x7fe37400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.599+0000 7fe389d38640 1 -- 172.21.15.38:0/2028821689 shutdown_connections 2024-05-21T15:43:41.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.599+0000 7fe389d38640 1 --2- 172.21.15.38:0/2028821689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3841078f0 0x7fe3841a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.599+0000 7fe389d38640 1 --2- 172.21.15.38:0/2028821689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe384102bf0 0x7fe384199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.599+0000 7fe389d38640 1 --2- 172.21.15.38:0/2028821689 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fe36c0a2c30 0x7fe36c0a5140 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.599+0000 7fe389d38640 1 --2- 172.21.15.38:0/2028821689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3841019b0 0x7fe3841991e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.599+0000 7fe389d38640 1 -- 172.21.15.38:0/2028821689 >> 172.21.15.38:0/2028821689 conn(0x7fe3840fcba0 msgr2=0x7fe3840fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:41.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.599+0000 7fe389d38640 1 -- 172.21.15.38:0/2028821689 shutdown_connections 2024-05-21T15:43:41.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.599+0000 7fe389d38640 1 -- 172.21.15.38:0/2028821689 wait complete. 2024-05-21T15:43:41.609 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:43:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.727+0000 7f9d4ede3640 1 -- 172.21.15.38:0/1555590736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d481078f0 msgr2=0x7f9d48109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.727+0000 7f9d4ede3640 1 --2- 172.21.15.38:0/1555590736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d481078f0 0x7f9d48109d10 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009ca0 tx=0x7f9d4002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.727+0000 7f9d4ede3640 1 -- 172.21.15.38:0/1555590736 shutdown_connections 2024-05-21T15:43:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.727+0000 7f9d4ede3640 1 --2- 172.21.15.38:0/1555590736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d481078f0 0x7f9d48109d10 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.727+0000 7f9d4ede3640 1 --2- 172.21.15.38:0/1555590736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d48102bf0 0x7f9d48103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.727+0000 7f9d4ede3640 1 --2- 172.21.15.38:0/1555590736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d481019b0 0x7f9d481026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.727+0000 7f9d4ede3640 1 -- 172.21.15.38:0/1555590736 >> 172.21.15.38:0/1555590736 conn(0x7f9d480fcba0 msgr2=0x7f9d480ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.728+0000 7f9d4ede3640 1 -- 172.21.15.38:0/1555590736 shutdown_connections 2024-05-21T15:43:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.728+0000 7f9d4ede3640 1 -- 172.21.15.38:0/1555590736 wait complete. 2024-05-21T15:43:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.728+0000 7f9d4ede3640 1 Processor -- start 2024-05-21T15:43:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.728+0000 7f9d4ede3640 1 -- start start 2024-05-21T15:43:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.728+0000 7f9d4ede3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d481019b0 0x7f9d48198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.728+0000 7f9d4ede3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d48102bf0 0x7f9d481985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.728+0000 7f9d4ede3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d481078f0 0x7f9d4819f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.728+0000 7f9d4ede3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4810d610 con 0x7f9d481019b0 2024-05-21T15:43:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.728+0000 7f9d4ede3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4810d490 con 0x7f9d481078f0 2024-05-21T15:43:41.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.728+0000 7f9d4ede3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4810d790 con 0x7f9d48102bf0 2024-05-21T15:43:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d4cb7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d481019b0 0x7f9d48198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d47fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d48102bf0 0x7f9d481985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d4cb7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d481019b0 0x7f9d48198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50676/0 (socket says 172.21.15.38:50676) 2024-05-21T15:43:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d4cb7f640 1 -- 172.21.15.38:0/3747910754 learned_addr learned my addr 172.21.15.38:0/3747910754 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:43:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d4d380640 1 --2- 172.21.15.38:0/3747910754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d481078f0 0x7f9d4819f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d4cb7f640 1 -- 172.21.15.38:0/3747910754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d48102bf0 msgr2=0x7f9d481985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d4cb7f640 1 --2- 172.21.15.38:0/3747910754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d48102bf0 0x7f9d481985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d4cb7f640 1 -- 172.21.15.38:0/3747910754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d481078f0 msgr2=0x7f9d4819f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d4cb7f640 1 --2- 172.21.15.38:0/3747910754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d481078f0 0x7f9d4819f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d4cb7f640 1 -- 172.21.15.38:0/3747910754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4819fba0 con 0x7f9d481019b0 2024-05-21T15:43:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d4cb7f640 1 --2- 172.21.15.38:0/3747910754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d481019b0 0x7f9d48198060 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00e9f0 tx=0x7f9d3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:41.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d45ffb640 1 -- 172.21.15.38:0/3747910754 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00ccc0 con 0x7f9d481019b0 2024-05-21T15:43:41.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d4ede3640 1 -- 172.21.15.38:0/3747910754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4819fe90 con 0x7f9d481019b0 2024-05-21T15:43:41.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d45ffb640 1 -- 172.21.15.38:0/3747910754 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d3c0035f0 con 0x7f9d481019b0 2024-05-21T15:43:41.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.729+0000 7f9d45ffb640 1 -- 172.21.15.38:0/3747910754 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00ce60 con 0x7f9d481019b0 2024-05-21T15:43:41.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.730+0000 7f9d4ede3640 1 -- 172.21.15.38:0/3747910754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d481a0350 con 0x7f9d481019b0 2024-05-21T15:43:41.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.730+0000 7f9d4ede3640 1 -- 172.21.15.38:0/3747910754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d48105ff0 con 0x7f9d481019b0 2024-05-21T15:43:41.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.732+0000 7f9d45ffb640 1 -- 172.21.15.38:0/3747910754 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9d3c0037a0 con 0x7f9d481019b0 2024-05-21T15:43:41.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.732+0000 7f9d45ffb640 1 --2- 172.21.15.38:0/3747910754 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9d380a2c30 0x7f9d380a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:41.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.733+0000 7f9d45ffb640 1 -- 172.21.15.38:0/3747910754 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f9d3c0cf570 con 0x7f9d481019b0 2024-05-21T15:43:41.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.733+0000 7f9d47fff640 1 --2- 172.21.15.38:0/3747910754 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9d380a2c30 0x7f9d380a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:41.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.733+0000 7f9d47fff640 1 --2- 172.21.15.38:0/3747910754 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9d380a2c30 0x7f9d380a5140 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9d48199100 tx=0x7f9d3400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:41.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.736+0000 7f9d45ffb640 1 -- 172.21.15.38:0/3747910754 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9d3c09c420 con 0x7f9d481019b0 2024-05-21T15:43:41.910 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:41 smithi045 ceph-mon[36409]: pgmap v629: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-21T15:43:41.910 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:41 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3011826794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:43:41.910 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:41 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2028821689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:43:41.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.926+0000 7f9d4ede3640 1 -- 172.21.15.38:0/3747910754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d4802d420 con 0x7f9d481019b0 2024-05-21T15:43:41.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.927+0000 7f9d45ffb640 1 -- 172.21.15.38:0/3747910754 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22704 (secure 0 0 0) 0x7f9d3c09bd20 con 0x7f9d481019b0 2024-05-21T15:43:41.925 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-05-21T15:43:41.925 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":90,"fsid":"102ec4fa-1786-11ef-bc9a-c7b262605968","created":"2024-05-21T15:24:24.082082+0000","modified":"2024-05-21T15:34:23.996759+0000","last_up_change":"2024-05-21T15:31:44.805508+0000","last_in_change":"2024-05-21T15:31:23.773877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:27:50.659367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:32:31.645647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:32:32.955451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f6fb6df5-af66-434f-9f35-f571243a0e39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6803","nonce":2546787282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6805","nonce":2546787282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6809","nonce":2546787282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":2546787282},{"type":"v1","addr":"172.21.15.38:6807","nonce":2546787282}]},"public_addr":"172.21.15.38:6803/2546787282","cluster_addr":"172.21.15.38:6805/2546787282","heartbeat_back_addr":"172.21.15.38:6809/2546787282","heartbeat_front_addr":"172.21.15.38:6807/2546787282","state":["exists","up"]},{"osd":1,"uuid":"61df021e-6d15-4875-baab-e7ff8c388a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6801","nonce":3379390940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6803","nonce":3379390940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6807","nonce":3379390940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3379390940},{"type":"v1","addr":"172.21.15.45:6805","nonce":3379390940}]},"public_addr":"172.21.15.45:6801/3379390940","cluster_addr":"172.21.15.45:6803/3379390940","heartbeat_back_addr":"172.21.15.45:6807/3379390940","heartbeat_front_addr":"172.21.15.45:6805/3379390940","state":["exists","up"]},{"osd":2,"uuid":"7dba2ab3-c388-47ea-b756-68611a94f77b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6801","nonce":2547348925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6803","nonce":2547348925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6807","nonce":2547348925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2547348925},{"type":"v1","addr":"172.21.15.53:6805","nonce":2547348925}]},"public_addr":"172.21.15.53:6801/2547348925","cluster_addr":"172.21.15.53:6803/2547348925","heartbeat_back_addr":"172.21.15.53:6807/2547348925","heartbeat_front_addr":"172.21.15.53:6805/2547348925","state":["exists","up"]},{"osd":3,"uuid":"a3de4eed-679b-4828-ae1e-468585b7c1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6811","nonce":1590427033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6813","nonce":1590427033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6817","nonce":1590427033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1590427033},{"type":"v1","addr":"172.21.15.38:6815","nonce":1590427033}]},"public_addr":"172.21.15.38:6811/1590427033","cluster_addr":"172.21.15.38:6813/1590427033","heartbeat_back_addr":"172.21.15.38:6817/1590427033","heartbeat_front_addr":"172.21.15.38:6815/1590427033","state":["exists","up"]},{"osd":4,"uuid":"b728a85c-8f55-4a40-a5e5-9ce853372338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6809","nonce":283670193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6811","nonce":283670193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6815","nonce":283670193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":283670193},{"type":"v1","addr":"172.21.15.45:6813","nonce":283670193}]},"public_addr":"172.21.15.45:6809/283670193","cluster_addr":"172.21.15.45:6811/283670193","heartbeat_back_addr":"172.21.15.45:6815/283670193","heartbeat_front_addr":"172.21.15.45:6813/283670193","state":["exists","up"]},{"osd":5,"uuid":"b94853cc-2ef8-410f-b900-023c6d74ea56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6809","nonce":1942044902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6811","nonce":1942044902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6815","nonce":1942044902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1942044902},{"type":"v1","addr":"172.21.15.53:6813","nonce":1942044902}]},"public_addr":"172.21.15.53:6809/1942044902","cluster_addr":"172.21.15.53:6811/1942044902","heartbeat_back_addr":"172.21.15.53:6815/1942044902","heartbeat_front_addr":"172.21.15.53:6813/1942044902","state":["exists","up"]},{"osd":6,"uuid":"d609b60e-d3c8-482f-850d-f313e97c8c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6819","nonce":1680263500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6821","nonce":1680263500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6825","nonce":1680263500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1680263500},{"type":"v1","addr":"172.21.15.38:6823","nonce":1680263500}]},"public_addr":"172.21.15.38:6819/1680263500","cluster_addr":"172.21.15.38:6821/1680263500","heartbeat_back_addr":"172.21.15.38:6825/1680263500","heartbeat_front_addr":"172.21.15.38:6823/1680263500","state":["exists","up"]},{"osd":7,"uuid":"6d28ddc5-15d6-4475-8ec8-1c2fd82a48d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6817","nonce":424307746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6819","nonce":424307746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6823","nonce":424307746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":424307746},{"type":"v1","addr":"172.21.15.45:6821","nonce":424307746}]},"public_addr":"172.21.15.45:6817/424307746","cluster_addr":"172.21.15.45:6819/424307746","heartbeat_back_addr":"172.21.15.45:6823/424307746","heartbeat_front_addr":"172.21.15.45:6821/424307746","state":["exists","up"]},{"osd":8,"uuid":"bc9538ac-44ea-4518-86b7-1a4dc9f10d57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6817","nonce":53196173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6819","nonce":53196173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6823","nonce":53196173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":53196173},{"type":"v1","addr":"172.21.15.53:6821","nonce":53196173}]},"public_addr":"172.21.15.53:6817/53196173","cluster_addr":"172.21.15.53:6819/53196173","heartbeat_back_addr":"172.21.15.53:6823/53196173","heartbeat_front_addr":"172.21.15.53:6821/53196173","state":["exists","up"]},{"osd":9,"uuid":"e8885020-dc81-4f22-8592-45c5179c24ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6827","nonce":2989402649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6829","nonce":2989402649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6833","nonce":2989402649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2989402649},{"type":"v1","addr":"172.21.15.38:6831","nonce":2989402649}]},"public_addr":"172.21.15.38:6827/2989402649","cluster_addr":"172.21.15.38:6829/2989402649","heartbeat_back_addr":"172.21.15.38:6833/2989402649","heartbeat_front_addr":"172.21.15.38:6831/2989402649","state":["exists","up"]},{"osd":10,"uuid":"56587d67-8492-49d1-8e4f-1ee050b7f9cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6825","nonce":3908617766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6827","nonce":3908617766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6831","nonce":3908617766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3908617766},{"type":"v1","addr":"172.21.15.45:6829","nonce":3908617766}]},"public_addr":"172.21.15.45:6825/3908617766","cluster_addr":"172.21.15.45:6827/3908617766","heartbeat_back_addr":"172.21.15.45:6831/3908617766","heartbeat_front_addr":"172.21.15.45:6829/3908617766","state":["exists","up"]},{"osd":11,"uuid":"b27f85e0-a264-4ee8-9e24-251bc01305c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6825","nonce":2775995423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6827","nonce":2775995423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6831","nonce":2775995423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2775995423},{"type":"v1","addr":"172.21.15.53:6829","nonce":2775995423}]},"public_addr":"172.21.15.53:6825/2775995423","cluster_addr":"172.21.15.53:6827/2775995423","heartbeat_back_addr":"172.21.15.53:6831/2775995423","heartbeat_front_addr":"172.21.15.53:6829/2775995423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:26:58.546389+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:23.071554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:27:46.948496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:13.514025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:28:38.986724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:04.884631+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:31.330442+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:29:57.099679+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:23.066224+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:30:49.343015+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:16.158082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:31:42.756984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6},{"from":3,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/592277081":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1186842456":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6801/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:6800/1481949523":"2024-05-22T15:25:22.117984+0000","172.21.15.38:0/1771432089":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6801/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2903150778":"2024-05-22T15:24:54.404192+0000","172.21.15.38:6800/3523378851":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/2666943685":"2024-05-22T15:24:54.404192+0000","172.21.15.38:0/3586196771":"2024-05-22T15:25:22.117984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:43:41.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.930+0000 7f9d4ede3640 1 -- 172.21.15.38:0/3747910754 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9d380a2c30 msgr2=0x7f9d380a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:41.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.930+0000 7f9d4ede3640 1 --2- 172.21.15.38:0/3747910754 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9d380a2c30 0x7f9d380a5140 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9d48199100 tx=0x7f9d3400b390 comp rx=0 tx=0).stop 2024-05-21T15:43:41.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.930+0000 7f9d4ede3640 1 -- 172.21.15.38:0/3747910754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d481019b0 msgr2=0x7f9d48198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:41.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.930+0000 7f9d4ede3640 1 --2- 172.21.15.38:0/3747910754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d481019b0 0x7f9d48198060 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00e9f0 tx=0x7f9d3c00eeb0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.930+0000 7f9d4ede3640 1 -- 172.21.15.38:0/3747910754 shutdown_connections 2024-05-21T15:43:41.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.930+0000 7f9d4ede3640 1 --2- 172.21.15.38:0/3747910754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d481078f0 0x7f9d4819f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.930+0000 7f9d4ede3640 1 --2- 172.21.15.38:0/3747910754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d48102bf0 0x7f9d481985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.930+0000 7f9d4ede3640 1 --2- 172.21.15.38:0/3747910754 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f9d380a2c30 0x7f9d380a5140 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.930+0000 7f9d4ede3640 1 --2- 172.21.15.38:0/3747910754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d481019b0 0x7f9d48198060 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:41.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.930+0000 7f9d4ede3640 1 -- 172.21.15.38:0/3747910754 >> 172.21.15.38:0/3747910754 conn(0x7f9d480fcba0 msgr2=0x7f9d480ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:41.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.930+0000 7f9d4ede3640 1 -- 172.21.15.38:0/3747910754 shutdown_connections 2024-05-21T15:43:41.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:41.930+0000 7f9d4ede3640 1 -- 172.21.15.38:0/3747910754 wait complete. 2024-05-21T15:43:41.938 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:41 smithi038 ceph-mon[31995]: pgmap v629: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-21T15:43:41.938 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:41 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3011826794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:43:41.938 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:41 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2028821689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:43:41.941 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-21T15:43:41.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:41 smithi053 ceph-mon[36496]: pgmap v629: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-21T15:43:41.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:41 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3011826794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:43:41.954 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:41 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2028821689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:43:42.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.060+0000 7f364425a640 1 -- 172.21.15.38:0/537791732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c103d20 msgr2=0x7f363c104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:42.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.060+0000 7f364425a640 1 --2- 172.21.15.38:0/537791732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c103d20 0x7f363c104130 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3630009900 tx=0x7f363002a410 comp rx=0 tx=0).stop 2024-05-21T15:43:42.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.060+0000 7f364425a640 1 -- 172.21.15.38:0/537791732 shutdown_connections 2024-05-21T15:43:42.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.060+0000 7f364425a640 1 --2- 172.21.15.38:0/537791732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f363c108a20 0x7f363c10ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.060+0000 7f364425a640 1 --2- 172.21.15.38:0/537791732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c103d20 0x7f363c104130 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.060+0000 7f364425a640 1 --2- 172.21.15.38:0/537791732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c102ae0 0x7f363c1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.060+0000 7f364425a640 1 -- 172.21.15.38:0/537791732 >> 172.21.15.38:0/537791732 conn(0x7f363c0fdcf0 msgr2=0x7f363c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:42.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.061+0000 7f364425a640 1 -- 172.21.15.38:0/537791732 shutdown_connections 2024-05-21T15:43:42.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.061+0000 7f364425a640 1 -- 172.21.15.38:0/537791732 wait complete. 2024-05-21T15:43:42.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.061+0000 7f364425a640 1 Processor -- start 2024-05-21T15:43:42.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.061+0000 7f364425a640 1 -- start start 2024-05-21T15:43:42.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.061+0000 7f364425a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c102ae0 0x7f363c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:42.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.061+0000 7f364425a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f363c103d20 0x7f363c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:42.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.061+0000 7f364425a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c108a20 0x7f363c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:42.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.061+0000 7f364425a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f363c10e760 con 0x7f363c102ae0 2024-05-21T15:43:42.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.061+0000 7f364425a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f363c10e5e0 con 0x7f363c108a20 2024-05-21T15:43:42.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.061+0000 7f364425a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f363c10e8e0 con 0x7f363c103d20 2024-05-21T15:43:42.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f3641ff6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c102ae0 0x7f363c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:42.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f3641ff6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c102ae0 0x7f363c1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50702/0 (socket says 172.21.15.38:50702) 2024-05-21T15:43:42.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f3641ff6640 1 -- 172.21.15.38:0/233680098 learned_addr learned my addr 172.21.15.38:0/233680098 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:43:42.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f36427f7640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c108a20 0x7f363c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:42.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f3641ff6640 1 -- 172.21.15.38:0/233680098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f363c103d20 msgr2=0x7f363c199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:42.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f36417f5640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f363c103d20 0x7f363c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:42.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f3641ff6640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f363c103d20 0x7f363c199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f3641ff6640 1 -- 172.21.15.38:0/233680098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c108a20 msgr2=0x7f363c1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:42.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f3641ff6640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c108a20 0x7f363c1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f3641ff6640 1 -- 172.21.15.38:0/233680098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c1a0dc0 con 0x7f363c102ae0 2024-05-21T15:43:42.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f36427f7640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c108a20 0x7f363c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:43:42.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f3641ff6640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c102ae0 0x7f363c1992d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f362c00eb90 tx=0x7f362c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:42.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f363affd640 1 -- 172.21.15.38:0/233680098 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c003b70 con 0x7f363c102ae0 2024-05-21T15:43:42.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f363affd640 1 -- 172.21.15.38:0/233680098 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f362c00ee90 con 0x7f363c102ae0 2024-05-21T15:43:42.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f364425a640 1 -- 172.21.15.38:0/233680098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363c1a10b0 con 0x7f363c102ae0 2024-05-21T15:43:42.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f363affd640 1 -- 172.21.15.38:0/233680098 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c019820 con 0x7f363c102ae0 2024-05-21T15:43:42.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.062+0000 7f364425a640 1 -- 172.21.15.38:0/233680098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363c1a1650 con 0x7f363c102ae0 2024-05-21T15:43:42.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.063+0000 7f364425a640 1 -- 172.21.15.38:0/233680098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f363c107830 con 0x7f363c102ae0 2024-05-21T15:43:42.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.065+0000 7f363affd640 1 -- 172.21.15.38:0/233680098 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f362c0199c0 con 0x7f363c102ae0 2024-05-21T15:43:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.065+0000 7f363affd640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f36240a2c10 0x7f36240a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.066+0000 7f36417f5640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f36240a2c10 0x7f36240a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.066+0000 7f363affd640 1 -- 172.21.15.38:0/233680098 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8306+0+0 (secure 0 0 0) 0x7f362c0cfa50 con 0x7f363c102ae0 2024-05-21T15:43:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.066+0000 7f36417f5640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f36240a2c10 0x7f36240a5120 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3630005f70 tx=0x7f3630000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:42.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.069+0000 7f363affd640 1 -- 172.21.15.38:0/233680098 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f362c09c9f0 con 0x7f363c102ae0 2024-05-21T15:43:42.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.293+0000 7f364425a640 1 -- 172.21.15.38:0/233680098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f363c062750 con 0x7f363c102ae0 2024-05-21T15:43:42.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.295+0000 7f363affd640 1 -- 172.21.15.38:0/233680098 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) ==== 161+0+0 (secure 0 0 0) 0x7f362c003d10 con 0x7f363c102ae0 2024-05-21T15:43:42.292 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-21T15:43:42.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.297+0000 7f364425a640 1 -- 172.21.15.38:0/233680098 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f36240a2c10 msgr2=0x7f36240a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:42.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.297+0000 7f364425a640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f36240a2c10 0x7f36240a5120 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3630005f70 tx=0x7f3630000bc0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.297+0000 7f364425a640 1 -- 172.21.15.38:0/233680098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c102ae0 msgr2=0x7f363c1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:42.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.297+0000 7f364425a640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c102ae0 0x7f363c1992d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f362c00eb90 tx=0x7f362c00c6a0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.297+0000 7f364425a640 1 -- 172.21.15.38:0/233680098 shutdown_connections 2024-05-21T15:43:42.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.297+0000 7f364425a640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c108a20 0x7f363c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.297+0000 7f364425a640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f363c103d20 0x7f363c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.297+0000 7f364425a640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f36240a2c10 0x7f36240a5120 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.297+0000 7f364425a640 1 --2- 172.21.15.38:0/233680098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f363c102ae0 0x7f363c1992d0 secure :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f362c00eb90 tx=0x7f362c00c6a0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.297+0000 7f364425a640 1 -- 172.21.15.38:0/233680098 >> 172.21.15.38:0/233680098 conn(0x7f363c0fdcf0 msgr2=0x7f363c104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:42.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.298+0000 7f364425a640 1 -- 172.21.15.38:0/233680098 shutdown_connections 2024-05-21T15:43:42.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.298+0000 7f364425a640 1 -- 172.21.15.38:0/233680098 wait complete. 2024-05-21T15:43:42.311 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-21T15:43:42.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.441+0000 7fea99a3d640 1 -- 172.21.15.38:0/992073810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea94102bd0 msgr2=0x7fea94102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.441+0000 7fea99a3d640 1 --2- 172.21.15.38:0/992073810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea94102bd0 0x7fea94102fe0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fea88009900 tx=0x7fea8802a410 comp rx=0 tx=0).stop 2024-05-21T15:43:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.441+0000 7fea99a3d640 1 -- 172.21.15.38:0/992073810 shutdown_connections 2024-05-21T15:43:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.441+0000 7fea99a3d640 1 --2- 172.21.15.38:0/992073810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea941078d0 0x7fea94109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.441+0000 7fea99a3d640 1 --2- 172.21.15.38:0/992073810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea94102bd0 0x7fea94102fe0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.441+0000 7fea99a3d640 1 --2- 172.21.15.38:0/992073810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea94101990 0x7fea94102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.441+0000 7fea99a3d640 1 -- 172.21.15.38:0/992073810 >> 172.21.15.38:0/992073810 conn(0x7fea940fcb80 msgr2=0x7fea940ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.441+0000 7fea99a3d640 1 -- 172.21.15.38:0/992073810 shutdown_connections 2024-05-21T15:43:42.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.441+0000 7fea99a3d640 1 -- 172.21.15.38:0/992073810 wait complete. 2024-05-21T15:43:42.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.442+0000 7fea99a3d640 1 Processor -- start 2024-05-21T15:43:42.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.442+0000 7fea99a3d640 1 -- start start 2024-05-21T15:43:42.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.442+0000 7fea99a3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea94101990 0x7fea940747f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:42.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.442+0000 7fea99a3d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea94102bd0 0x7fea9419dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:42.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.442+0000 7fea99a3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea941078d0 0x7fea941a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:42.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.442+0000 7fea99a3d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fea9410d5e0 con 0x7fea94101990 2024-05-21T15:43:42.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.442+0000 7fea99a3d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea9410d460 con 0x7fea941078d0 2024-05-21T15:43:42.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.442+0000 7fea92ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea94101990 0x7fea940747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:42.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.442+0000 7fea99a3d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fea9410d760 con 0x7fea94102bd0 2024-05-21T15:43:42.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.442+0000 7fea92ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea94101990 0x7fea940747f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50762/0 (socket says 172.21.15.38:50762) 2024-05-21T15:43:42.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.442+0000 7fea92ffd640 1 -- 172.21.15.38:0/2342257960 learned_addr learned my addr 172.21.15.38:0/2342257960 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:43:42.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.443+0000 7fea927fc640 1 --2- 172.21.15.38:0/2342257960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea94102bd0 0x7fea9419dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:42.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.443+0000 7fea937fe640 1 --2- 172.21.15.38:0/2342257960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea941078d0 0x7fea941a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:42.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.443+0000 7fea92ffd640 1 -- 172.21.15.38:0/2342257960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea94102bd0 msgr2=0x7fea9419dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:42.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.443+0000 7fea92ffd640 1 --2- 172.21.15.38:0/2342257960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea94102bd0 0x7fea9419dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.443+0000 7fea92ffd640 1 -- 172.21.15.38:0/2342257960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea941078d0 msgr2=0x7fea941a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:42.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.443+0000 7fea92ffd640 1 --2- 172.21.15.38:0/2342257960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea941078d0 0x7fea941a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:42.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.443+0000 7fea92ffd640 1 -- 172.21.15.38:0/2342257960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea941a0d40 con 0x7fea94101990 2024-05-21T15:43:42.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.443+0000 7fea92ffd640 1 --2- 172.21.15.38:0/2342257960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea94101990 0x7fea940747f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fea8400e9f0 tx=0x7fea8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:42.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.443+0000 7fea98a3b640 1 -- 172.21.15.38:0/2342257960 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8400ccc0 con 0x7fea94101990 2024-05-21T15:43:42.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.443+0000 7fea98a3b640 1 -- 172.21.15.38:0/2342257960 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea840035f0 con 0x7fea94101990 2024-05-21T15:43:42.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.443+0000 7fea99a3d640 1 -- 172.21.15.38:0/2342257960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea941a0fd0 con 0x7fea94101990 2024-05-21T15:43:42.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.443+0000 7fea98a3b640 1 -- 172.21.15.38:0/2342257960 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8400ce60 con 0x7fea94101990 2024-05-21T15:43:42.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.443+0000 7fea99a3d640 1 -- 172.21.15.38:0/2342257960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea941a1490 con 0x7fea94101990 2024-05-21T15:43:42.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.445+0000 7fea99a3d640 1 -- 172.21.15.38:0/2342257960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea941066e0 con 0x7fea94101990 2024-05-21T15:43:42.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.446+0000 7fea98a3b640 1 -- 172.21.15.38:0/2342257960 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fea840037a0 con 0x7fea94101990 2024-05-21T15:43:42.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.447+0000 7fea98a3b640 1 --2- 172.21.15.38:0/2342257960 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fea7c0a2b90 0x7fea7c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:42.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.447+0000 7fea98a3b640 1 -- 172.21.15.38:0/2342257960 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8736+0+0 (secure 0 0 0) 0x7fea840cf5b0 con 0x7fea94101990 2024-05-21T15:43:42.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.447+0000 7fea927fc640 1 --2- 172.21.15.38:0/2342257960 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fea7c0a2b90 0x7fea7c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:42.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.448+0000 7fea927fc640 1 --2- 172.21.15.38:0/2342257960 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fea7c0a2b90 0x7fea7c0a50a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fea88009230 tx=0x7fea88031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:42.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.450+0000 7fea98a3b640 1 -- 172.21.15.38:0/2342257960 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fea8409c320 con 0x7fea94101990 2024-05-21T15:43:42.508 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 21 15:43:42 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-b[66443]: ignoring --setuser ceph since I am not root 2024-05-21T15:43:42.509 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 21 15:43:42 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-b[66443]: ignoring --setgroup ceph since I am not root 2024-05-21T15:43:42.509 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 21 15:43:42 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-b[66443]: starting mds.b at 2024-05-21T15:43:42.509 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:43:42 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-k[65513]: ignoring --setuser ceph since I am not root 2024-05-21T15:43:42.509 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:43:42 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-k[65513]: ignoring --setgroup ceph since I am not root 2024-05-21T15:43:42.509 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:43:42 smithi045 ceph-mds[65517]: -- 172.21.15.45:0/2506893898 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6bae49a40 con 0x55b6baec1180 2024-05-21T15:43:42.509 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:43:42 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-k[65513]: starting mds.k at 2024-05-21T15:43:42.510 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:43:42 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-e[65828]: ignoring --setuser ceph since I am not root 2024-05-21T15:43:42.510 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:43:42 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-e[65828]: ignoring --setgroup ceph since I am not root 2024-05-21T15:43:42.510 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:43:42 smithi045 ceph-mds[65832]: -- 172.21.15.45:0/3479090540 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bf2d5aba40 con 0x55bf2d623600 2024-05-21T15:43:42.510 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:43:42 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-e[65828]: starting mds.e at 2024-05-21T15:43:42.510 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:43:42 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-h[66137]: ignoring --setuser ceph since I am not root 2024-05-21T15:43:42.510 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:43:42 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-h[66137]: ignoring --setgroup ceph since I am not root 2024-05-21T15:43:42.510 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:43:42 smithi045 ceph-mds[66141]: -- 172.21.15.45:0/2301911276 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b5b49bba40 con 0x55b5b4a33180 2024-05-21T15:43:42.510 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:43:42 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-h[66137]: starting mds.h at 2024-05-21T15:43:42.600 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:42 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a[31991]: 2024-05-21T15:43:42.293+0000 7f95d4c9d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T15:43:42.601 INFO:journalctl@ceph.mds.g.smithi038.stdout:May 21 15:43:42 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-g[72540]: ignoring --setuser ceph since I am not root 2024-05-21T15:43:42.601 INFO:journalctl@ceph.mds.g.smithi038.stdout:May 21 15:43:42 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-g[72540]: ignoring --setgroup ceph since I am not root 2024-05-21T15:43:42.601 INFO:journalctl@ceph.mds.g.smithi038.stdout:May 21 15:43:42 smithi038 ceph-mds[72544]: -- 172.21.15.38:0/1166384622 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c27a363a40 con 0x55c27a3db180 2024-05-21T15:43:42.601 INFO:journalctl@ceph.mds.g.smithi038.stdout:May 21 15:43:42 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-g[72540]: starting mds.g at 2024-05-21T15:43:42.601 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:43:42 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-a[72894]: ignoring --setuser ceph since I am not root 2024-05-21T15:43:42.601 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:43:42 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-a[72894]: ignoring --setgroup ceph since I am not root 2024-05-21T15:43:42.601 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:43:42 smithi038 ceph-mds[72898]: -- 172.21.15.38:0/2738286844 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557da0deba40 con 0x557da0e63180 2024-05-21T15:43:42.602 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:43:42 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-a[72894]: starting mds.a at 2024-05-21T15:43:42.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:42.681+0000 7fea99a3d640 1 -- 172.21.15.38:0/2342257960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fea9402d430 con 0x7fea94101990 2024-05-21T15:43:42.703 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:42 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3747910754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:43:42.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:42 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/233680098' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-21T15:43:42.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:42 smithi053 ceph-mon[36496]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-21T15:43:42.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:42 smithi053 ceph-mon[36496]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T15:43:42.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:42 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/233680098' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-21T15:43:42.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:42 smithi053 ceph-mon[36496]: osdmap e91: 12 total, 12 up, 12 in 2024-05-21T15:43:42.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:42 smithi053 ceph-mon[36496]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-21T15:43:42.704 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:43:42 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-c[66608]: ignoring --setuser ceph since I am not root 2024-05-21T15:43:42.704 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:43:42 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-c[66608]: ignoring --setgroup ceph since I am not root 2024-05-21T15:43:42.704 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:43:42 smithi053 ceph-mds[66612]: -- 172.21.15.53:0/4234021388 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5558b912fa40 con 0x5558b91a7600 2024-05-21T15:43:42.705 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:43:42 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-c[66608]: starting mds.c at 2024-05-21T15:43:42.705 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:43:42 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-l[66003]: ignoring --setuser ceph since I am not root 2024-05-21T15:43:42.705 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:43:42 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-l[66003]: ignoring --setgroup ceph since I am not root 2024-05-21T15:43:42.705 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:43:42 smithi053 ceph-mds[66007]: -- 172.21.15.53:0/3297079484 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55791d1f3a40 con 0x55791d26b600 2024-05-21T15:43:42.705 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:43:42 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-l[66003]: starting mds.l at 2024-05-21T15:43:42.705 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:43:42 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-i[66908]: ignoring --setuser ceph since I am not root 2024-05-21T15:43:42.706 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:43:42 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-i[66908]: ignoring --setgroup ceph since I am not root 2024-05-21T15:43:42.706 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:43:42 smithi053 ceph-mds[66912]: -- 172.21.15.53:0/1770764727 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5568891a3a40 con 0x55688921b180 2024-05-21T15:43:42.706 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:43:42 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-i[66908]: starting mds.i at 2024-05-21T15:43:42.706 INFO:journalctl@ceph.mds.f.smithi053.stdout:May 21 15:43:42 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-f[66305]: ignoring --setuser ceph since I am not root 2024-05-21T15:43:42.706 INFO:journalctl@ceph.mds.f.smithi053.stdout:May 21 15:43:42 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-f[66305]: ignoring --setgroup ceph since I am not root 2024-05-21T15:43:42.706 INFO:journalctl@ceph.mds.f.smithi053.stdout:May 21 15:43:42 smithi053 ceph-mds[66309]: -- 172.21.15.53:0/2139946699 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5645fc0c1a40 con 0x5645fc139180 2024-05-21T15:43:42.706 INFO:journalctl@ceph.mds.f.smithi053.stdout:May 21 15:43:42 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-f[66305]: starting mds.f at 2024-05-21T15:43:43.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:42 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3747910754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:43:43.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:42 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/233680098' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-21T15:43:43.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:42 smithi045 ceph-mon[36409]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-21T15:43:43.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:42 smithi045 ceph-mon[36409]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T15:43:43.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:42 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/233680098' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-21T15:43:43.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:42 smithi045 ceph-mon[36409]: osdmap e91: 12 total, 12 up, 12 in 2024-05-21T15:43:43.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:42 smithi045 ceph-mon[36409]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-21T15:43:43.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:42 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3747910754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:43:43.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:42 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/233680098' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-21T15:43:43.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:42 smithi038 ceph-mon[31995]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-21T15:43:43.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:42 smithi038 ceph-mon[31995]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T15:43:43.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:42 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/233680098' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-21T15:43:43.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:42 smithi038 ceph-mon[31995]: osdmap e91: 12 total, 12 up, 12 in 2024-05-21T15:43:43.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:42 smithi038 ceph-mon[31995]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-21T15:43:43.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.301+0000 7fea98a3b640 1 -- 172.21.15.38:0/2342257960 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) ==== 104+0+0 (secure 0 0 0) 0x7fea840860a0 con 0x7fea94101990 2024-05-21T15:43:43.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.305+0000 7fea99a3d640 1 -- 172.21.15.38:0/2342257960 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fea7c0a2b90 msgr2=0x7fea7c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:43.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.305+0000 7fea99a3d640 1 --2- 172.21.15.38:0/2342257960 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fea7c0a2b90 0x7fea7c0a50a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fea88009230 tx=0x7fea88031400 comp rx=0 tx=0).stop 2024-05-21T15:43:43.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.305+0000 7fea99a3d640 1 -- 172.21.15.38:0/2342257960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea94101990 msgr2=0x7fea940747f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:43.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.305+0000 7fea99a3d640 1 --2- 172.21.15.38:0/2342257960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea94101990 0x7fea940747f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fea8400e9f0 tx=0x7fea8400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:43:43.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.305+0000 7fea99a3d640 1 -- 172.21.15.38:0/2342257960 shutdown_connections 2024-05-21T15:43:43.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.305+0000 7fea99a3d640 1 --2- 172.21.15.38:0/2342257960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea941078d0 0x7fea941a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:43.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.305+0000 7fea99a3d640 1 --2- 172.21.15.38:0/2342257960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea94102bd0 0x7fea9419dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:43.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.305+0000 7fea99a3d640 1 --2- 172.21.15.38:0/2342257960 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7fea7c0a2b90 0x7fea7c0a50a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:43.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.305+0000 7fea99a3d640 1 --2- 172.21.15.38:0/2342257960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea94101990 0x7fea940747f0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:43.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.305+0000 7fea99a3d640 1 -- 172.21.15.38:0/2342257960 >> 172.21.15.38:0/2342257960 conn(0x7fea940fcb80 msgr2=0x7fea941038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:43.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.305+0000 7fea99a3d640 1 -- 172.21.15.38:0/2342257960 shutdown_connections 2024-05-21T15:43:43.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.305+0000 7fea99a3d640 1 -- 172.21.15.38:0/2342257960 wait complete. 2024-05-21T15:43:43.318 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-21T15:43:43.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.441+0000 7f8fee40d640 1 -- 172.21.15.38:0/613183649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe8102ab0 msgr2=0x7f8fe81037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.441+0000 7f8fee40d640 1 --2- 172.21.15.38:0/613183649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe8102ab0 0x7f8fe81037b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009900 tx=0x7f8fd802a410 comp rx=0 tx=0).stop 2024-05-21T15:43:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.442+0000 7f8fee40d640 1 -- 172.21.15.38:0/613183649 shutdown_connections 2024-05-21T15:43:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.442+0000 7f8fee40d640 1 --2- 172.21.15.38:0/613183649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe81089f0 0x7f8fe810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.442+0000 7f8fee40d640 1 --2- 172.21.15.38:0/613183649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fe8103cf0 0x7f8fe8104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.442+0000 7f8fee40d640 1 --2- 172.21.15.38:0/613183649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe8102ab0 0x7f8fe81037b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.442+0000 7f8fee40d640 1 -- 172.21.15.38:0/613183649 >> 172.21.15.38:0/613183649 conn(0x7f8fe80fdca0 msgr2=0x7f8fe8100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.442+0000 7f8fee40d640 1 -- 172.21.15.38:0/613183649 shutdown_connections 2024-05-21T15:43:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.442+0000 7f8fee40d640 1 -- 172.21.15.38:0/613183649 wait complete. 2024-05-21T15:43:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.442+0000 7f8fee40d640 1 Processor -- start 2024-05-21T15:43:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.442+0000 7f8fee40d640 1 -- start start 2024-05-21T15:43:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.442+0000 7f8fee40d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fe8102ab0 0x7f8fe8199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:43.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.442+0000 7f8fee40d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe8103cf0 0x7f8fe81997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.442+0000 7f8fee40d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe81089f0 0x7f8fe81a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.442+0000 7f8fee40d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe810e6e0 con 0x7f8fe81089f0 2024-05-21T15:43:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fee40d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe810e560 con 0x7f8fe8103cf0 2024-05-21T15:43:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fee40d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe810e860 con 0x7f8fe8102ab0 2024-05-21T15:43:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fec9aa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe81089f0 0x7f8fe81a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fec9aa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe81089f0 0x7f8fe81a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50776/0 (socket says 172.21.15.38:50776) 2024-05-21T15:43:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fec9aa640 1 -- 172.21.15.38:0/4105485889 learned_addr learned my addr 172.21.15.38:0/4105485889 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:43:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fe7fff640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fe8102ab0 0x7f8fe8199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fdffff640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe8103cf0 0x7f8fe81997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fec9aa640 1 -- 172.21.15.38:0/4105485889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fe8102ab0 msgr2=0x7f8fe8199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fec9aa640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fe8102ab0 0x7f8fe8199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fec9aa640 1 -- 172.21.15.38:0/4105485889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe8103cf0 msgr2=0x7f8fe81997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:43.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fec9aa640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe8103cf0 0x7f8fe81997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:43.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fec9aa640 1 -- 172.21.15.38:0/4105485889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe81a0dd0 con 0x7f8fe81089f0 2024-05-21T15:43:43.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fe7fff640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fe8102ab0 0x7f8fe8199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:43:43.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fec9aa640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe81089f0 0x7f8fe81a0760 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000eaa0 tx=0x7f8fe000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:43.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fe5ffb640 1 -- 172.21.15.38:0/4105485889 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe000cdb0 con 0x7f8fe81089f0 2024-05-21T15:43:43.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fe5ffb640 1 -- 172.21.15.38:0/4105485889 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe0005020 con 0x7f8fe81089f0 2024-05-21T15:43:43.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fee40d640 1 -- 172.21.15.38:0/4105485889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe81a10c0 con 0x7f8fe81089f0 2024-05-21T15:43:43.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fe5ffb640 1 -- 172.21.15.38:0/4105485889 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00116b0 con 0x7f8fe81089f0 2024-05-21T15:43:43.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.443+0000 7f8fee40d640 1 -- 172.21.15.38:0/4105485889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe81a15c0 con 0x7f8fe81089f0 2024-05-21T15:43:43.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.445+0000 7f8fee40d640 1 -- 172.21.15.38:0/4105485889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe8107800 con 0x7f8fe81089f0 2024-05-21T15:43:43.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.447+0000 7f8fe5ffb640 1 -- 172.21.15.38:0/4105485889 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8fe0005560 con 0x7f8fe81089f0 2024-05-21T15:43:43.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.447+0000 7f8fe5ffb640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8fd40a2b90 0x7f8fd40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:43.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.448+0000 7f8fe7fff640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8fd40a2b90 0x7f8fd40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:43.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.448+0000 7f8fe5ffb640 1 -- 172.21.15.38:0/4105485889 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8736+0+0 (secure 0 0 0) 0x7f8fe00cf8a0 con 0x7f8fe81089f0 2024-05-21T15:43:43.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.448+0000 7f8fe7fff640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8fd40a2b90 0x7f8fd40a50a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009230 tx=0x7f8fd8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:43.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.451+0000 7f8fe5ffb640 1 -- 172.21.15.38:0/4105485889 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8fe0016030 con 0x7f8fe81089f0 2024-05-21T15:43:43.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:43.635+0000 7f8fee40d640 1 -- 172.21.15.38:0/4105485889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8fe802d2a0 con 0x7f8fe81089f0 2024-05-21T15:43:43.956 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: pgmap v631: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-21T15:43:43.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2342257960' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:43:43.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-21T15:43:43.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-21T15:43:43.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: Cluster is now healthy 2024-05-21T15:43:43.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/2342257960' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-21T15:43:43.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.45:6832/69321258,v1:172.21.15.45:6833/69321258] up:boot 2024-05-21T15:43:43.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.45:6834/237798583,v1:172.21.15.45:6835/237798583] up:boot 2024-05-21T15:43:43.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.45:6836/3551390708,v1:172.21.15.45:6838/3551390708] up:boot 2024-05-21T15:43:43.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.45:6837/2042255970,v1:172.21.15.45:6839/2042255970] up:boot 2024-05-21T15:43:43.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.38:6838/3675165176,v1:172.21.15.38:6840/3675165176] up:boot 2024-05-21T15:43:43.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.38:6839/1236972289,v1:172.21.15.38:6841/1236972289] up:boot 2024-05-21T15:43:43.957 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.53:6832/1023005529,v1:172.21.15.53:6834/1023005529] up:boot 2024-05-21T15:43:43.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.53:6833/3216845783,v1:172.21.15.53:6835/3216845783] up:boot 2024-05-21T15:43:43.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.53:6837/1704212595,v1:172.21.15.53:6839/1704212595] up:boot 2024-05-21T15:43:43.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: mds.? [v2:172.21.15.53:6836/3150105428,v1:172.21.15.53:6838/3150105428] up:boot 2024-05-21T15:43:43.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: fsmap 12 up:standby 2024-05-21T15:43:43.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:43:43.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:43:43.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:43:43.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:43:43.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:43:43.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:43:43.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:43:43.958 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:43:43.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:43:43.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:43:43.959 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:43 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4105485889' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T15:43:44.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: pgmap v631: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-21T15:43:44.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2342257960' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:43:44.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-21T15:43:44.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-21T15:43:44.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: Cluster is now healthy 2024-05-21T15:43:44.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/2342257960' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-21T15:43:44.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.45:6832/69321258,v1:172.21.15.45:6833/69321258] up:boot 2024-05-21T15:43:44.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.45:6834/237798583,v1:172.21.15.45:6835/237798583] up:boot 2024-05-21T15:43:44.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.45:6836/3551390708,v1:172.21.15.45:6838/3551390708] up:boot 2024-05-21T15:43:44.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.45:6837/2042255970,v1:172.21.15.45:6839/2042255970] up:boot 2024-05-21T15:43:44.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.38:6838/3675165176,v1:172.21.15.38:6840/3675165176] up:boot 2024-05-21T15:43:44.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.38:6839/1236972289,v1:172.21.15.38:6841/1236972289] up:boot 2024-05-21T15:43:44.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.53:6832/1023005529,v1:172.21.15.53:6834/1023005529] up:boot 2024-05-21T15:43:44.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.53:6833/3216845783,v1:172.21.15.53:6835/3216845783] up:boot 2024-05-21T15:43:44.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.53:6837/1704212595,v1:172.21.15.53:6839/1704212595] up:boot 2024-05-21T15:43:44.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: mds.? [v2:172.21.15.53:6836/3150105428,v1:172.21.15.53:6838/3150105428] up:boot 2024-05-21T15:43:44.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: fsmap 12 up:standby 2024-05-21T15:43:44.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:43:44.010 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:43:44.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:43:44.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:43:44.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:43:44.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:43:44.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:43:44.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:43:44.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:43:44.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:43:44.011 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:43 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4105485889' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T15:43:44.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: pgmap v631: 97 pgs: 97 active+clean; 21 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-21T15:43:44.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2342257960' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:43:44.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-21T15:43:44.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-21T15:43:44.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: Cluster is now healthy 2024-05-21T15:43:44.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/2342257960' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-21T15:43:44.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.45:6832/69321258,v1:172.21.15.45:6833/69321258] up:boot 2024-05-21T15:43:44.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.45:6834/237798583,v1:172.21.15.45:6835/237798583] up:boot 2024-05-21T15:43:44.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.45:6836/3551390708,v1:172.21.15.45:6838/3551390708] up:boot 2024-05-21T15:43:44.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.45:6837/2042255970,v1:172.21.15.45:6839/2042255970] up:boot 2024-05-21T15:43:44.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.38:6838/3675165176,v1:172.21.15.38:6840/3675165176] up:boot 2024-05-21T15:43:44.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.38:6839/1236972289,v1:172.21.15.38:6841/1236972289] up:boot 2024-05-21T15:43:44.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.53:6832/1023005529,v1:172.21.15.53:6834/1023005529] up:boot 2024-05-21T15:43:44.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.53:6833/3216845783,v1:172.21.15.53:6835/3216845783] up:boot 2024-05-21T15:43:44.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.53:6837/1704212595,v1:172.21.15.53:6839/1704212595] up:boot 2024-05-21T15:43:44.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: mds.? [v2:172.21.15.53:6836/3150105428,v1:172.21.15.53:6838/3150105428] up:boot 2024-05-21T15:43:44.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: fsmap 12 up:standby 2024-05-21T15:43:44.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:43:44.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:43:44.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:43:44.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:43:44.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:43:44.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:43:44.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:43:44.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:43:44.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:43:44.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: from='mgr.14152 172.21.15.38:0/2770153605' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:43:44.025 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:43 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4105485889' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T15:43:44.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.304+0000 7f8fe5ffb640 1 -- 172.21.15.38:0/4105485889 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) ==== 181+0+0 (secure 0 0 0) 0x7f8fe009c610 con 0x7f8fe81089f0 2024-05-21T15:43:44.302 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-05-21T15:43:44.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.307+0000 7f8fee40d640 1 -- 172.21.15.38:0/4105485889 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8fd40a2b90 msgr2=0x7f8fd40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:44.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.307+0000 7f8fee40d640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8fd40a2b90 0x7f8fd40a50a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009230 tx=0x7f8fd8000bc0 comp rx=0 tx=0).stop 2024-05-21T15:43:44.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.307+0000 7f8fee40d640 1 -- 172.21.15.38:0/4105485889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe81089f0 msgr2=0x7f8fe81a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:44.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.307+0000 7f8fee40d640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe81089f0 0x7f8fe81a0760 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000eaa0 tx=0x7f8fe000ef60 comp rx=0 tx=0).stop 2024-05-21T15:43:44.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.307+0000 7f8fee40d640 1 -- 172.21.15.38:0/4105485889 shutdown_connections 2024-05-21T15:43:44.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.307+0000 7f8fee40d640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f8fd40a2b90 0x7f8fd40a50a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:44.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.307+0000 7f8fee40d640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fe81089f0 0x7f8fe81a0760 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:44.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.307+0000 7f8fee40d640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe8103cf0 0x7f8fe81997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:44.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.307+0000 7f8fee40d640 1 --2- 172.21.15.38:0/4105485889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fe8102ab0 0x7f8fe8199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:44.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.307+0000 7f8fee40d640 1 -- 172.21.15.38:0/4105485889 >> 172.21.15.38:0/4105485889 conn(0x7f8fe80fdca0 msgr2=0x7f8fe81049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:44.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.307+0000 7f8fee40d640 1 -- 172.21.15.38:0/4105485889 shutdown_connections 2024-05-21T15:43:44.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.307+0000 7f8fee40d640 1 -- 172.21.15.38:0/4105485889 wait complete. 2024-05-21T15:43:44.322 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-21T15:43:44.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.452+0000 7f63b947a640 1 -- 172.21.15.38:0/2503844968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63b4105ed0 msgr2=0x7f63b41082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:44.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.452+0000 7f63b947a640 1 --2- 172.21.15.38:0/2503844968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63b4105ed0 0x7f63b41082f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009900 tx=0x7f63a802a410 comp rx=0 tx=0).stop 2024-05-21T15:43:44.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.452+0000 7f63b947a640 1 -- 172.21.15.38:0/2503844968 shutdown_connections 2024-05-21T15:43:44.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.452+0000 7f63b947a640 1 --2- 172.21.15.38:0/2503844968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63b4108830 0x7f63b410ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:44.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.452+0000 7f63b947a640 1 --2- 172.21.15.38:0/2503844968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63b4105ed0 0x7f63b41082f0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:44.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.452+0000 7f63b947a640 1 --2- 172.21.15.38:0/2503844968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b4103570 0x7f63b4105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:44.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.452+0000 7f63b947a640 1 -- 172.21.15.38:0/2503844968 >> 172.21.15.38:0/2503844968 conn(0x7f63b40fcb70 msgr2=0x7f63b40ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:44.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.452+0000 7f63b947a640 1 -- 172.21.15.38:0/2503844968 shutdown_connections 2024-05-21T15:43:44.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.452+0000 7f63b947a640 1 -- 172.21.15.38:0/2503844968 wait complete. 2024-05-21T15:43:44.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.453+0000 7f63b947a640 1 Processor -- start 2024-05-21T15:43:44.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.453+0000 7f63b947a640 1 -- start start 2024-05-21T15:43:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.453+0000 7f63b947a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b4103570 0x7f63b4199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.453+0000 7f63b947a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63b4105ed0 0x7f63b41997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.453+0000 7f63b947a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63b4108830 0x7f63b41a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.453+0000 7f63b947a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f63b410e6e0 con 0x7f63b4105ed0 2024-05-21T15:43:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.453+0000 7f63b947a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f63b410e560 con 0x7f63b4108830 2024-05-21T15:43:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.453+0000 7f63b947a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f63b410e860 con 0x7f63b4103570 2024-05-21T15:43:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f63b27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63b4105ed0 0x7f63b41997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f63b27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63b4105ed0 0x7f63b41997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50784/0 (socket says 172.21.15.38:50784) 2024-05-21T15:43:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f63b27fc640 1 -- 172.21.15.38:0/3184321841 learned_addr learned my addr 172.21.15.38:0/3184321841 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-05-21T15:43:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f63b2ffd640 1 --2- 172.21.15.38:0/3184321841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b4103570 0x7f63b4199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f63b37fe640 1 --2- 172.21.15.38:0/3184321841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63b4108830 0x7f63b41a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f63b27fc640 1 -- 172.21.15.38:0/3184321841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b4103570 msgr2=0x7f63b4199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f63b27fc640 1 --2- 172.21.15.38:0/3184321841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b4103570 0x7f63b4199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f63b27fc640 1 -- 172.21.15.38:0/3184321841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63b4108830 msgr2=0x7f63b41a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f63b27fc640 1 --2- 172.21.15.38:0/3184321841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63b4108830 0x7f63b41a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:44.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f63b27fc640 1 -- 172.21.15.38:0/3184321841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b41a0dd0 con 0x7f63b4105ed0 2024-05-21T15:43:44.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f63b27fc640 1 --2- 172.21.15.38:0/3184321841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63b4105ed0 0x7f63b41997d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f63a8006010 tx=0x7f63a8031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:44.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f639bfff640 1 -- 172.21.15.38:0/3184321841 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a803f030 con 0x7f63b4105ed0 2024-05-21T15:43:44.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f63b947a640 1 -- 172.21.15.38:0/3184321841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b41a1060 con 0x7f63b4105ed0 2024-05-21T15:43:44.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f639bfff640 1 -- 172.21.15.38:0/3184321841 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a8003ad0 con 0x7f63b4105ed0 2024-05-21T15:43:44.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f639bfff640 1 -- 172.21.15.38:0/3184321841 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8035070 con 0x7f63b4105ed0 2024-05-21T15:43:44.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.454+0000 7f63b947a640 1 -- 172.21.15.38:0/3184321841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b41a15a0 con 0x7f63b4105ed0 2024-05-21T15:43:44.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.455+0000 7f63b947a640 1 -- 172.21.15.38:0/3184321841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b40fe690 con 0x7f63b4105ed0 2024-05-21T15:43:44.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.457+0000 7f639bfff640 1 -- 172.21.15.38:0/3184321841 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f63a8042030 con 0x7f63b4105ed0 2024-05-21T15:43:44.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.457+0000 7f639bfff640 1 --2- 172.21.15.38:0/3184321841 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f639c0a2b90 0x7f639c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:43:44.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.458+0000 7f63b2ffd640 1 --2- 172.21.15.38:0/3184321841 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f639c0a2b90 0x7f639c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:43:44.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.458+0000 7f639bfff640 1 -- 172.21.15.38:0/3184321841 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f63a80efad0 con 0x7f63b4105ed0 2024-05-21T15:43:44.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.458+0000 7f63b2ffd640 1 --2- 172.21.15.38:0/3184321841 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f639c0a2b90 0x7f639c0a50a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f63a40099a0 tx=0x7f63a4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:43:44.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.461+0000 7f639bfff640 1 -- 172.21.15.38:0/3184321841 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f63a8003c80 con 0x7f63b4105ed0 2024-05-21T15:43:44.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:44.648+0000 7f63b947a640 1 -- 172.21.15.38:0/3184321841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f63b402d2a0 con 0x7f63b4105ed0 2024-05-21T15:43:45.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:45.310+0000 7f639bfff640 1 -- 172.21.15.38:0/3184321841 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) ==== 169+0+0 (secure 0 0 0) 0x7f63a80bc480 con 0x7f63b4105ed0 2024-05-21T15:43:45.308 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-05-21T15:43:45.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:45.313+0000 7f63b947a640 1 -- 172.21.15.38:0/3184321841 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f639c0a2b90 msgr2=0x7f639c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:45.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:45.313+0000 7f63b947a640 1 --2- 172.21.15.38:0/3184321841 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f639c0a2b90 0x7f639c0a50a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f63a40099a0 tx=0x7f63a4009500 comp rx=0 tx=0).stop 2024-05-21T15:43:45.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:45.313+0000 7f63b947a640 1 -- 172.21.15.38:0/3184321841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63b4105ed0 msgr2=0x7f63b41997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:43:45.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:45.313+0000 7f63b947a640 1 --2- 172.21.15.38:0/3184321841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63b4105ed0 0x7f63b41997d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f63a8006010 tx=0x7f63a8031b60 comp rx=0 tx=0).stop 2024-05-21T15:43:45.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:45.313+0000 7f63b947a640 1 -- 172.21.15.38:0/3184321841 shutdown_connections 2024-05-21T15:43:45.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:45.313+0000 7f63b947a640 1 --2- 172.21.15.38:0/3184321841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63b4108830 0x7f63b41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:45.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:45.313+0000 7f63b947a640 1 --2- 172.21.15.38:0/3184321841 >> [v2:172.21.15.38:6800/1295226559,v1:172.21.15.38:6801/1295226559] conn(0x7f639c0a2b90 0x7f639c0a50a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:45.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:45.313+0000 7f63b947a640 1 --2- 172.21.15.38:0/3184321841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63b4105ed0 0x7f63b41997d0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:45.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:45.313+0000 7f63b947a640 1 --2- 172.21.15.38:0/3184321841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b4103570 0x7f63b4199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:43:45.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:45.313+0000 7f63b947a640 1 -- 172.21.15.38:0/3184321841 >> 172.21.15.38:0/3184321841 conn(0x7f63b40fcb70 msgr2=0x7f63b41041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:43:45.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:45.314+0000 7f63b947a640 1 -- 172.21.15.38:0/3184321841 shutdown_connections 2024-05-21T15:43:45.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-05-21T15:43:45.315+0000 7f63b947a640 1 -- 172.21.15.38:0/3184321841 wait complete. 2024-05-21T15:43:45.330 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:43:45.371 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:43:45.424 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:43:45.466 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-21T15:43:45.466 DEBUG:teuthology.orchestra.run.smithi038:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T15:43:45.507 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T15:43:45.528 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T15:43:45.549 INFO:tasks.cephadm:Stopping all daemons... 2024-05-21T15:43:45.550 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-21T15:43:45.550 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.a 2024-05-21T15:43:45.584 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:45 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/4105485889' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T15:43:45.584 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:45 smithi038 ceph-mon[31995]: osdmap e92: 12 total, 12 up, 12 in 2024-05-21T15:43:45.584 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:45 smithi038 ceph-mon[31995]: pgmap v633: 65 pgs: 65 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:43:45.584 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:45 smithi038 ceph-mon[31995]: from='client.? 172.21.15.38:0/3184321841' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T15:43:45.703 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:45 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/4105485889' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T15:43:45.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:45 smithi053 ceph-mon[36496]: osdmap e92: 12 total, 12 up, 12 in 2024-05-21T15:43:45.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:45 smithi053 ceph-mon[36496]: pgmap v633: 65 pgs: 65 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:43:45.704 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:45 smithi053 ceph-mon[36496]: from='client.? 172.21.15.38:0/3184321841' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T15:43:45.758 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:45 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/4105485889' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T15:43:45.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:45 smithi045 ceph-mon[36409]: osdmap e92: 12 total, 12 up, 12 in 2024-05-21T15:43:45.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:45 smithi045 ceph-mon[36409]: pgmap v633: 65 pgs: 65 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:43:45.759 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:45 smithi045 ceph-mon[36409]: from='client.? 172.21.15.38:0/3184321841' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T15:43:46.017 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:45 smithi038 systemd[1]: Stopping Ceph mon.a for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:43:46.272 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:46 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a[31991]: 2024-05-21T15:43:46.021+0000 7f95da4a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:43:46.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:46 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a[31991]: 2024-05-21T15:43:46.021+0000 7f95da4a8700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-21T15:43:46.273 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:46 smithi038 podman[99549]: 2024-05-21 15:43:46.200896931 +0000 UTC m=+0.347891637 container died df34e728985d64b8cb94871454f459374e35a1ecad3d31ae2cfdcc627ae65131 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-21T15:43:46.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:46 smithi038 podman[99549]: 2024-05-21 15:43:46.604810574 +0000 UTC m=+0.751805271 container cleanup df34e728985d64b8cb94871454f459374e35a1ecad3d31ae2cfdcc627ae65131 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-21T15:43:46.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:46 smithi038 bash[99549]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a 2024-05-21T15:43:47.165 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.a.service' 2024-05-21T15:43:47.230 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:46 smithi038 podman[99559]: 2024-05-21 15:43:46.957255997 +0000 UTC m=+0.754906782 container remove df34e728985d64b8cb94871454f459374e35a1ecad3d31ae2cfdcc627ae65131 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-05-21T15:43:47.230 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:47 smithi038 systemd[1]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-21T15:43:47.230 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:47 smithi038 systemd[1]: Stopped Ceph mon.a for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:43:47.230 INFO:journalctl@ceph.mon.a.smithi038.stdout:May 21 15:43:47 smithi038 systemd[1]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.a.service: Consumed 22.700s CPU time. 2024-05-21T15:43:47.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:43:47.266 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-21T15:43:47.266 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-21T15:43:47.266 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.b 2024-05-21T15:43:47.677 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:47 smithi045 systemd[1]: Stopping Ceph mon.b for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:43:48.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:47 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-b[36405]: 2024-05-21T15:43:47.678+0000 7f2cffb5a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:43:48.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:47 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-b[36405]: 2024-05-21T15:43:47.678+0000 7f2cffb5a700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-21T15:43:48.009 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:47 smithi045 podman[70265]: 2024-05-21 15:43:47.874119928 +0000 UTC m=+0.342195255 container died fb1f297bbcde079c548f455709483ddafb5dae0586f44ee3f2aef418eedea204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:43:48.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:48 smithi045 podman[70265]: 2024-05-21 15:43:48.268592588 +0000 UTC m=+0.736667908 container cleanup fb1f297bbcde079c548f455709483ddafb5dae0586f44ee3f2aef418eedea204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-21T15:43:48.509 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 21 15:43:48 smithi045 bash[70265]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-b 2024-05-21T15:43:48.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.b.service' 2024-05-21T15:43:48.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:43:48.749 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-21T15:43:48.749 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-21T15:43:48.749 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.c 2024-05-21T15:43:49.116 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:48 smithi053 systemd[1]: Stopping Ceph mon.c for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:43:49.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:49 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-c[36478]: 2024-05-21T15:43:49.117+0000 7fc50a6da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:43:49.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:49 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-c[36478]: 2024-05-21T15:43:49.117+0000 7fc50a6da700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-21T15:43:49.454 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:49 smithi053 podman[70895]: 2024-05-21 15:43:49.281938984 +0000 UTC m=+0.297434791 container died a6c1844a3a251836c80eb534c8d379d1d2d73567e0f37828e99f46e9f90a58ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:43:49.893 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:49 smithi053 podman[70895]: 2024-05-21 15:43:49.661164499 +0000 UTC m=+0.676660298 container cleanup a6c1844a3a251836c80eb534c8d379d1d2d73567e0f37828e99f46e9f90a58ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-21T15:43:49.893 INFO:journalctl@ceph.mon.c.smithi053.stdout:May 21 15:43:49 smithi053 bash[70895]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mon-c 2024-05-21T15:43:50.028 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mon.c.service' 2024-05-21T15:43:50.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:43:50.080 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-21T15:43:50.080 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-21T15:43:50.080 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.x 2024-05-21T15:43:50.514 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:43:50 smithi038 systemd[1]: Stopping Ceph mgr.x for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:43:50.772 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:43:50 smithi038 podman[99658]: 2024-05-21 15:43:50.51697518 +0000 UTC m=+0.227048574 container died f79809031e3a1490d0609b03ea3f23db78d1d82421e4d8a88070f25a7564a839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True) 2024-05-21T15:43:51.362 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:43:51 smithi038 podman[99658]: 2024-05-21 15:43:51.112237548 +0000 UTC m=+0.822310939 container cleanup f79809031e3a1490d0609b03ea3f23db78d1d82421e4d8a88070f25a7564a839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-21T15:43:51.362 INFO:journalctl@ceph.mgr.x.smithi038.stdout:May 21 15:43:51 smithi038 bash[99658]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-x 2024-05-21T15:43:51.491 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.x.service' 2024-05-21T15:43:51.530 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:43:51.530 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-21T15:43:51.530 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-21T15:43:51.530 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.y 2024-05-21T15:43:51.854 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:43:51 smithi045 systemd[1]: Stopping Ceph mgr.y for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:43:52.168 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:43:51 smithi045 podman[70388]: 2024-05-21 15:43:51.856916584 +0000 UTC m=+0.120917287 container died 65f2b4e1696a869ba2ff4e8d0f47cc0b184e4f5d22f38158974c5bc048041316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:43:52.508 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:43:52 smithi045 podman[70388]: 2024-05-21 15:43:52.27030734 +0000 UTC m=+0.534308030 container cleanup 65f2b4e1696a869ba2ff4e8d0f47cc0b184e4f5d22f38158974c5bc048041316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-21T15:43:52.508 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 21 15:43:52 smithi045 bash[70388]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-y 2024-05-21T15:43:52.707 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.y.service' 2024-05-21T15:43:52.741 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:43:52.741 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-21T15:43:52.741 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-21T15:43:52.741 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.z 2024-05-21T15:43:53.092 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:43:52 smithi053 systemd[1]: Stopping Ceph mgr.z for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:43:53.453 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:43:53 smithi053 podman[71019]: 2024-05-21 15:43:53.095111043 +0000 UTC m=+0.153473811 container died 75e9738b9c6d42b7585a5490bf14a295007d5ee13c00a4e84ce2d9684f02884a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:43:53.953 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:43:53 smithi053 podman[71019]: 2024-05-21 15:43:53.716504129 +0000 UTC m=+0.774866890 container cleanup 75e9738b9c6d42b7585a5490bf14a295007d5ee13c00a4e84ce2d9684f02884a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:43:53.954 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:43:53 smithi053 bash[71019]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z 2024-05-21T15:43:54.206 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.z.service' 2024-05-21T15:43:54.228 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:43:53 smithi053 podman[71032]: 2024-05-21 15:43:53.959995064 +0000 UTC m=+0.849462430 container remove 75e9738b9c6d42b7585a5490bf14a295007d5ee13c00a4e84ce2d9684f02884a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:43:54.228 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:43:53 smithi053 systemd[1]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-21T15:43:54.229 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:43:54 smithi053 systemd[1]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-05-21T15:43:54.229 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:43:54 smithi053 systemd[1]: Stopped Ceph mgr.z for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:43:54.229 INFO:journalctl@ceph.mgr.z.smithi053.stdout:May 21 15:43:54 smithi053 systemd[1]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mgr.z.service: Consumed 13.504s CPU time. 2024-05-21T15:43:54.244 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:43:54.244 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-21T15:43:54.244 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-21T15:43:54.244 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.0 2024-05-21T15:43:54.581 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:43:54 smithi038 systemd[1]: Stopping Ceph osd.0 for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:43:55.022 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:43:54 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-0[46982]: 2024-05-21T15:43:54.583+0000 7fe72d327700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:43:55.023 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:43:54 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-0[46982]: 2024-05-21T15:43:54.583+0000 7fe72d327700 -1 osd.0 93 *** Got signal Terminated *** 2024-05-21T15:43:55.023 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:43:54 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-0[46982]: 2024-05-21T15:43:54.583+0000 7fe72d327700 -1 osd.0 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:44:00.468 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:44:00 smithi038 podman[99782]: 2024-05-21 15:44:00.170533969 +0000 UTC m=+5.716786226 container died 3c2b383373b255db8a4ea68a1723ef4f0bdd53bd07558d4bb83045c25d506090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-21T15:44:00.772 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:44:00 smithi038 podman[99782]: 2024-05-21 15:44:00.578927254 +0000 UTC m=+6.125179510 container cleanup 3c2b383373b255db8a4ea68a1723ef4f0bdd53bd07558d4bb83045c25d506090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-21T15:44:00.773 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:44:00 smithi038 bash[99782]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-0 2024-05-21T15:44:01.250 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:44:00 smithi038 podman[99800]: 2024-05-21 15:44:00.905847697 +0000 UTC m=+0.736468632 container remove 3c2b383373b255db8a4ea68a1723ef4f0bdd53bd07558d4bb83045c25d506090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:01.523 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:44:01 smithi038 podman[99894]: 2024-05-21 15:44:01.156590931 +0000 UTC m=+0.017856392 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:44:01.523 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:44:01 smithi038 podman[99894]: 2024-05-21 15:44:01.278697637 +0000 UTC m=+0.139963084 container create e7d25de9eaf2869a9ef699ace013a6f5aa163965ef4dc3cec33f2de4223497bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-21T15:44:01.523 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:44:01 smithi038 podman[99894]: 2024-05-21 15:44:01.492918071 +0000 UTC m=+0.354183523 container init e7d25de9eaf2869a9ef699ace013a6f5aa163965ef4dc3cec33f2de4223497bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-21T15:44:01.523 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:44:01 smithi038 podman[99894]: 2024-05-21 15:44:01.496441254 +0000 UTC m=+0.357706705 container start e7d25de9eaf2869a9ef699ace013a6f5aa163965ef4dc3cec33f2de4223497bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:02.022 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:44:01 smithi038 podman[99894]: 2024-05-21 15:44:01.609766557 +0000 UTC m=+0.471032006 container attach e7d25de9eaf2869a9ef699ace013a6f5aa163965ef4dc3cec33f2de4223497bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-05-21T15:44:02.385 INFO:journalctl@ceph.osd.0.smithi038.stdout:May 21 15:44:02 smithi038 podman[99894]: 2024-05-21 15:44:02.037593616 +0000 UTC m=+0.898859062 container died e7d25de9eaf2869a9ef699ace013a6f5aa163965ef4dc3cec33f2de4223497bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:44:02.720 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.0.service' 2024-05-21T15:44:02.768 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:44:02.768 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-21T15:44:02.768 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-21T15:44:02.769 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.1 2024-05-21T15:44:03.106 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:44:02 smithi045 systemd[1]: Stopping Ceph osd.1 for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:44:03.509 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:44:03 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-1[41015]: 2024-05-21T15:44:03.107+0000 7f5a0ab6c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:44:03.509 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:44:03 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-1[41015]: 2024-05-21T15:44:03.107+0000 7f5a0ab6c700 -1 osd.1 93 *** Got signal Terminated *** 2024-05-21T15:44:03.509 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:44:03 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-1[41015]: 2024-05-21T15:44:03.107+0000 7f5a0ab6c700 -1 osd.1 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:44:09.009 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:44:08 smithi045 podman[70509]: 2024-05-21 15:44:08.72162208 +0000 UTC m=+5.746567331 container died 173c9125b57ecaafbd2bce6c27fa33ea837697478d6028686a257ecea4f29a51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-1, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514) 2024-05-21T15:44:09.508 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:44:09 smithi045 podman[70509]: 2024-05-21 15:44:09.17633091 +0000 UTC m=+6.201276183 container cleanup 173c9125b57ecaafbd2bce6c27fa33ea837697478d6028686a257ecea4f29a51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-1, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:09.509 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:44:09 smithi045 bash[70509]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-1 2024-05-21T15:44:09.850 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:44:09 smithi045 podman[70525]: 2024-05-21 15:44:09.520044849 +0000 UTC m=+0.803094789 container remove 173c9125b57ecaafbd2bce6c27fa33ea837697478d6028686a257ecea4f29a51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:44:10.163 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:44:09 smithi045 podman[70613]: 2024-05-21 15:44:09.755749424 +0000 UTC m=+0.017640950 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:44:10.163 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:44:09 smithi045 podman[70613]: 2024-05-21 15:44:09.887805945 +0000 UTC m=+0.149697450 container create 4f5adb52dd1c08c507ff304f0f762c3823532ef7dc804b1d37bd9cb6843c26c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-21T15:44:10.163 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:44:10 smithi045 podman[70613]: 2024-05-21 15:44:10.166296049 +0000 UTC m=+0.428187554 container init 4f5adb52dd1c08c507ff304f0f762c3823532ef7dc804b1d37bd9cb6843c26c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-21T15:44:10.509 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:44:10 smithi045 podman[70613]: 2024-05-21 15:44:10.169952434 +0000 UTC m=+0.431843948 container start 4f5adb52dd1c08c507ff304f0f762c3823532ef7dc804b1d37bd9cb6843c26c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:10.509 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:44:10 smithi045 podman[70613]: 2024-05-21 15:44:10.290995025 +0000 UTC m=+0.552886539 container attach 4f5adb52dd1c08c507ff304f0f762c3823532ef7dc804b1d37bd9cb6843c26c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-21T15:44:10.966 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 15:44:10 smithi045 podman[70613]: 2024-05-21 15:44:10.684890785 +0000 UTC m=+0.946782290 container died 4f5adb52dd1c08c507ff304f0f762c3823532ef7dc804b1d37bd9cb6843c26c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:44:11.343 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.1.service' 2024-05-21T15:44:11.382 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:44:11.382 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-21T15:44:11.382 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-21T15:44:11.382 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.2 2024-05-21T15:44:11.701 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:44:11 smithi053 systemd[1]: Stopping Ceph osd.2 for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:44:11.954 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:44:11 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-2[41627]: 2024-05-21T15:44:11.703+0000 7fc709f73700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:44:11.954 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:44:11 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-2[41627]: 2024-05-21T15:44:11.703+0000 7fc709f73700 -1 osd.2 93 *** Got signal Terminated *** 2024-05-21T15:44:11.954 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:44:11 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-2[41627]: 2024-05-21T15:44:11.703+0000 7fc709f73700 -1 osd.2 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:44:17.502 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:44:17 smithi053 podman[71143]: 2024-05-21 15:44:17.207091507 +0000 UTC m=+5.618531212 container died ba771b3286d0eb54bef52dc954dd2c6f204b9c23dd835f052199717c0414c543 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:17.886 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:44:17 smithi053 podman[71143]: 2024-05-21 15:44:17.596027762 +0000 UTC m=+6.007467509 container cleanup ba771b3286d0eb54bef52dc954dd2c6f204b9c23dd835f052199717c0414c543 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-21T15:44:17.886 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:44:17 smithi053 bash[71143]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-2 2024-05-21T15:44:18.203 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:44:17 smithi053 podman[71170]: 2024-05-21 15:44:17.88902331 +0000 UTC m=+0.675927671 container remove ba771b3286d0eb54bef52dc954dd2c6f204b9c23dd835f052199717c0414c543 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:18.556 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:44:18 smithi053 podman[71256]: 2024-05-21 15:44:18.137889432 +0000 UTC m=+0.017975134 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:44:18.557 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:44:18 smithi053 podman[71256]: 2024-05-21 15:44:18.290470304 +0000 UTC m=+0.170556013 container create 2347fbebb5bb842b303a9c2ad8695017e6f02944fba9d27c71d6345009d2ab47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:44:18.557 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:44:18 smithi053 podman[71256]: 2024-05-21 15:44:18.559804051 +0000 UTC m=+0.439889735 container init 2347fbebb5bb842b303a9c2ad8695017e6f02944fba9d27c71d6345009d2ab47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-21T15:44:18.954 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:44:18 smithi053 podman[71256]: 2024-05-21 15:44:18.563381853 +0000 UTC m=+0.443467554 container start 2347fbebb5bb842b303a9c2ad8695017e6f02944fba9d27c71d6345009d2ab47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-21T15:44:18.954 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:44:18 smithi053 podman[71256]: 2024-05-21 15:44:18.684839111 +0000 UTC m=+0.564924816 container attach 2347fbebb5bb842b303a9c2ad8695017e6f02944fba9d27c71d6345009d2ab47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:44:19.410 INFO:journalctl@ceph.osd.2.smithi053.stdout:May 21 15:44:19 smithi053 podman[71256]: 2024-05-21 15:44:19.110971015 +0000 UTC m=+0.991056711 container died 2347fbebb5bb842b303a9c2ad8695017e6f02944fba9d27c71d6345009d2ab47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-2-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:44:19.726 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.2.service' 2024-05-21T15:44:19.768 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:44:19.768 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-21T15:44:19.768 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-21T15:44:19.768 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.3 2024-05-21T15:44:20.072 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:44:19 smithi038 systemd[1]: Stopping Ceph osd.3 for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:44:20.349 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:44:20 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-3[53448]: 2024-05-21T15:44:20.074+0000 7ff92cb33700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:44:20.350 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:44:20 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-3[53448]: 2024-05-21T15:44:20.074+0000 7ff92cb33700 -1 osd.3 93 *** Got signal Terminated *** 2024-05-21T15:44:20.350 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:44:20 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-3[53448]: 2024-05-21T15:44:20.074+0000 7ff92cb33700 -1 osd.3 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:44:25.791 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:44:25 smithi038 podman[99999]: 2024-05-21 15:44:25.541298089 +0000 UTC m=+5.567276190 container died 6f372c2fb2d10cbae5535c4c3ba24ebf9e6ce4df40c981f5216d1fcce3aca98c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-05-21T15:44:26.091 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:44:25 smithi038 podman[99999]: 2024-05-21 15:44:25.884320494 +0000 UTC m=+5.910298637 container cleanup 6f372c2fb2d10cbae5535c4c3ba24ebf9e6ce4df40c981f5216d1fcce3aca98c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:44:26.091 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:44:25 smithi038 bash[99999]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-3 2024-05-21T15:44:26.509 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:44:26 smithi038 podman[100026]: 2024-05-21 15:44:26.094083271 +0000 UTC m=+0.550415499 container remove 6f372c2fb2d10cbae5535c4c3ba24ebf9e6ce4df40c981f5216d1fcce3aca98c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-3, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS) 2024-05-21T15:44:26.772 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:44:26 smithi038 podman[100117]: 2024-05-21 15:44:26.414848879 +0000 UTC m=+0.017630753 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:44:26.772 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:44:26 smithi038 podman[100117]: 2024-05-21 15:44:26.53681273 +0000 UTC m=+0.139594586 container create 618d65e669d71dd12d2614d19d898c32d640969f72ebd1140bdb6c3b3d54864f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-21T15:44:27.147 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:44:26 smithi038 podman[100117]: 2024-05-21 15:44:26.83640478 +0000 UTC m=+0.439186661 container init 618d65e669d71dd12d2614d19d898c32d640969f72ebd1140bdb6c3b3d54864f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-21T15:44:27.147 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:44:26 smithi038 podman[100117]: 2024-05-21 15:44:26.840084484 +0000 UTC m=+0.442866356 container start 618d65e669d71dd12d2614d19d898c32d640969f72ebd1140bdb6c3b3d54864f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:27.148 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:44:26 smithi038 podman[100117]: 2024-05-21 15:44:26.931395648 +0000 UTC m=+0.534177522 container attach 618d65e669d71dd12d2614d19d898c32d640969f72ebd1140bdb6c3b3d54864f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0) 2024-05-21T15:44:27.449 INFO:journalctl@ceph.osd.3.smithi038.stdout:May 21 15:44:27 smithi038 podman[100117]: 2024-05-21 15:44:27.15128063 +0000 UTC m=+0.754062492 container died 618d65e669d71dd12d2614d19d898c32d640969f72ebd1140bdb6c3b3d54864f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:27.774 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.3.service' 2024-05-21T15:44:27.817 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:44:27.817 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-21T15:44:27.817 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-21T15:44:27.817 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.4 2024-05-21T15:44:28.155 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:44:27 smithi045 systemd[1]: Stopping Ceph osd.4 for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:44:28.508 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:44:28 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-4[47644]: 2024-05-21T15:44:28.156+0000 7fd498f7c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:44:28.508 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:44:28 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-4[47644]: 2024-05-21T15:44:28.156+0000 7fd498f7c700 -1 osd.4 93 *** Got signal Terminated *** 2024-05-21T15:44:28.508 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:44:28 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-4[47644]: 2024-05-21T15:44:28.156+0000 7fd498f7c700 -1 osd.4 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:44:33.856 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:44:33 smithi045 podman[70730]: 2024-05-21 15:44:33.605563823 +0000 UTC m=+5.573436876 container died 6f86f02149c388d89cb1ee8ef1daca8ad165a8d4a05ffe25e7e5e230e0a85190 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-4, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:44:34.258 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:44:33 smithi045 podman[70730]: 2024-05-21 15:44:33.966162666 +0000 UTC m=+5.934035718 container cleanup 6f86f02149c388d89cb1ee8ef1daca8ad165a8d4a05ffe25e7e5e230e0a85190 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:34.258 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:44:33 smithi045 bash[70730]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-4 2024-05-21T15:44:34.652 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:44:34 smithi045 podman[70744]: 2024-05-21 15:44:34.28452567 +0000 UTC m=+0.676965801 container remove 6f86f02149c388d89cb1ee8ef1daca8ad165a8d4a05ffe25e7e5e230e0a85190 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-21T15:44:34.943 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:44:34 smithi045 podman[70838]: 2024-05-21 15:44:34.558474521 +0000 UTC m=+0.017543450 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:44:34.943 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:44:34 smithi045 podman[70838]: 2024-05-21 15:44:34.693998182 +0000 UTC m=+0.153067116 container create b874713f09e66ceb5b755b1806e8636ac996cede71067be51a3ed7d22cd14af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:44:34.943 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:44:34 smithi045 podman[70838]: 2024-05-21 15:44:34.946606199 +0000 UTC m=+0.405675115 container init b874713f09e66ceb5b755b1806e8636ac996cede71067be51a3ed7d22cd14af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:35.247 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:44:34 smithi045 podman[70838]: 2024-05-21 15:44:34.949772822 +0000 UTC m=+0.408841739 container start b874713f09e66ceb5b755b1806e8636ac996cede71067be51a3ed7d22cd14af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-21T15:44:35.247 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:44:35 smithi045 podman[70838]: 2024-05-21 15:44:35.038500392 +0000 UTC m=+0.497569322 container attach b874713f09e66ceb5b755b1806e8636ac996cede71067be51a3ed7d22cd14af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:35.509 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 21 15:44:35 smithi045 podman[70838]: 2024-05-21 15:44:35.250911048 +0000 UTC m=+0.709979964 container died b874713f09e66ceb5b755b1806e8636ac996cede71067be51a3ed7d22cd14af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:35.931 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.4.service' 2024-05-21T15:44:35.974 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:44:35.975 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-21T15:44:35.975 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-21T15:44:35.975 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.5 2024-05-21T15:44:36.289 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:44:36 smithi053 systemd[1]: Stopping Ceph osd.5 for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:44:36.703 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:44:36 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-5[48261]: 2024-05-21T15:44:36.291+0000 7f0d12426700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:44:36.703 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:44:36 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-5[48261]: 2024-05-21T15:44:36.291+0000 7f0d12426700 -1 osd.5 93 *** Got signal Terminated *** 2024-05-21T15:44:36.703 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:44:36 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-5[48261]: 2024-05-21T15:44:36.291+0000 7f0d12426700 -1 osd.5 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:44:42.203 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:44:41 smithi053 podman[71362]: 2024-05-21 15:44:41.896574999 +0000 UTC m=+5.719534980 container died 54053abdd0d3e71880f64d35b5fc2d85b1bcbbf822a2a38cdd23cb342fd370ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:42.540 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:44:42 smithi053 podman[71362]: 2024-05-21 15:44:42.325027831 +0000 UTC m=+6.147987807 container cleanup 54053abdd0d3e71880f64d35b5fc2d85b1bcbbf822a2a38cdd23cb342fd370ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:42.540 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:44:42 smithi053 bash[71362]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-5 2024-05-21T15:44:42.953 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:44:42 smithi053 podman[71377]: 2024-05-21 15:44:42.543106049 +0000 UTC m=+0.638500662 container remove 54053abdd0d3e71880f64d35b5fc2d85b1bcbbf822a2a38cdd23cb342fd370ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-21T15:44:43.243 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:44:42 smithi053 podman[71469]: 2024-05-21 15:44:42.898797468 +0000 UTC m=+0.017970773 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:44:43.243 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:44:43 smithi053 podman[71469]: 2024-05-21 15:44:43.018831928 +0000 UTC m=+0.138005226 container create 109bd69681f614555baf5fce7a66e45825fb12d1ae0402489358ab4c401f3d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-21T15:44:43.243 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:44:43 smithi053 podman[71469]: 2024-05-21 15:44:43.246405289 +0000 UTC m=+0.365578586 container init 109bd69681f614555baf5fce7a66e45825fb12d1ae0402489358ab4c401f3d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-21T15:44:43.548 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:44:43 smithi053 podman[71469]: 2024-05-21 15:44:43.249548638 +0000 UTC m=+0.368721937 container start 109bd69681f614555baf5fce7a66e45825fb12d1ae0402489358ab4c401f3d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-21T15:44:43.548 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:44:43 smithi053 podman[71469]: 2024-05-21 15:44:43.338152188 +0000 UTC m=+0.457325486 container attach 109bd69681f614555baf5fce7a66e45825fb12d1ae0402489358ab4c401f3d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:44:43.847 INFO:journalctl@ceph.osd.5.smithi053.stdout:May 21 15:44:43 smithi053 podman[71469]: 2024-05-21 15:44:43.551421173 +0000 UTC m=+0.670594479 container died 109bd69681f614555baf5fce7a66e45825fb12d1ae0402489358ab4c401f3d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-21T15:44:44.157 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.5.service' 2024-05-21T15:44:44.197 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:44:44.197 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-21T15:44:44.197 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-21T15:44:44.197 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.6 2024-05-21T15:44:44.522 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:44:44 smithi038 systemd[1]: Stopping Ceph osd.6 for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:44:45.022 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:44:44 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-6[59961]: 2024-05-21T15:44:44.529+0000 7f6058c65700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:44:45.023 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:44:44 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-6[59961]: 2024-05-21T15:44:44.529+0000 7f6058c65700 -1 osd.6 93 *** Got signal Terminated *** 2024-05-21T15:44:45.023 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:44:44 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-6[59961]: 2024-05-21T15:44:44.529+0000 7f6058c65700 -1 osd.6 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:44:50.253 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:44:49 smithi038 podman[100225]: 2024-05-21 15:44:49.992227559 +0000 UTC m=+5.587680278 container died 6a097f6fd6fe93de7749958f05f0acd5a87b2a27d008324d551302ae6a716e99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-21T15:44:50.522 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:44:50 smithi038 podman[100225]: 2024-05-21 15:44:50.346470692 +0000 UTC m=+5.941923398 container cleanup 6a097f6fd6fe93de7749958f05f0acd5a87b2a27d008324d551302ae6a716e99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-21T15:44:50.522 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:44:50 smithi038 bash[100225]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-6 2024-05-21T15:44:51.022 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:44:50 smithi038 podman[100242]: 2024-05-21 15:44:50.681692897 +0000 UTC m=+0.685626899 container remove 6a097f6fd6fe93de7749958f05f0acd5a87b2a27d008324d551302ae6a716e99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-6, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-21T15:44:51.307 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:44:51 smithi038 podman[100331]: 2024-05-21 15:44:50.945233769 +0000 UTC m=+0.017957806 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:44:51.308 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:44:51 smithi038 podman[100331]: 2024-05-21 15:44:51.065974348 +0000 UTC m=+0.138698373 container create 5e622ebb1f04293a4244f9f798a176da71e0296470dafa59f59e125f50296e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-21T15:44:51.308 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:44:51 smithi038 podman[100331]: 2024-05-21 15:44:51.310432663 +0000 UTC m=+0.383156708 container init 5e622ebb1f04293a4244f9f798a176da71e0296470dafa59f59e125f50296e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0) 2024-05-21T15:44:51.618 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:44:51 smithi038 podman[100331]: 2024-05-21 15:44:51.314338525 +0000 UTC m=+0.387062551 container start 5e622ebb1f04293a4244f9f798a176da71e0296470dafa59f59e125f50296e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:51.619 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:44:51 smithi038 podman[100331]: 2024-05-21 15:44:51.418529543 +0000 UTC m=+0.491253591 container attach 5e622ebb1f04293a4244f9f798a176da71e0296470dafa59f59e125f50296e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-21T15:44:51.902 INFO:journalctl@ceph.osd.6.smithi038.stdout:May 21 15:44:51 smithi038 podman[100331]: 2024-05-21 15:44:51.622178525 +0000 UTC m=+0.694902559 container died 5e622ebb1f04293a4244f9f798a176da71e0296470dafa59f59e125f50296e96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-21T15:44:52.197 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.6.service' 2024-05-21T15:44:52.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:44:52.243 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-21T15:44:52.243 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-21T15:44:52.243 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.7 2024-05-21T15:44:52.559 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:44:52 smithi045 systemd[1]: Stopping Ceph osd.7 for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:44:52.934 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:44:52 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-7[54512]: 2024-05-21T15:44:52.561+0000 7fedba907700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:44:52.935 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:44:52 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-7[54512]: 2024-05-21T15:44:52.561+0000 7fedba907700 -1 osd.7 93 *** Got signal Terminated *** 2024-05-21T15:44:52.935 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:44:52 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-7[54512]: 2024-05-21T15:44:52.561+0000 7fedba907700 -1 osd.7 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:44:58.551 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:44:58 smithi045 podman[70943]: 2024-05-21 15:44:58.265868527 +0000 UTC m=+5.817418960 container died 469d0cab7f742f2f54f46062f1a5cba97894dc9426953e117596f6ced6d20cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:58.936 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:44:58 smithi045 podman[70943]: 2024-05-21 15:44:58.653637472 +0000 UTC m=+6.205187899 container cleanup 469d0cab7f742f2f54f46062f1a5cba97894dc9426953e117596f6ced6d20cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-7, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-21T15:44:58.936 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:44:58 smithi045 bash[70943]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-7 2024-05-21T15:44:59.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:44:58 smithi045 podman[70970]: 2024-05-21 15:44:58.938923236 +0000 UTC m=+0.668065308 container remove 469d0cab7f742f2f54f46062f1a5cba97894dc9426953e117596f6ced6d20cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-7, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-21T15:44:59.508 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:44:59 smithi045 podman[71061]: 2024-05-21 15:44:59.161982578 +0000 UTC m=+0.017475066 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:44:59.509 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:44:59 smithi045 podman[71061]: 2024-05-21 15:44:59.29005392 +0000 UTC m=+0.145546406 container create ed7a6d6e99aba50a5368ae4219ca71671b72e5bd5cfa83062065d2861a9af673 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-21T15:44:59.509 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:44:59 smithi045 podman[71061]: 2024-05-21 15:44:59.492986959 +0000 UTC m=+0.348479433 container init ed7a6d6e99aba50a5368ae4219ca71671b72e5bd5cfa83062065d2861a9af673 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-21T15:44:59.509 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:44:59 smithi045 podman[71061]: 2024-05-21 15:44:59.496510184 +0000 UTC m=+0.352002665 container start ed7a6d6e99aba50a5368ae4219ca71671b72e5bd5cfa83062065d2861a9af673 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:44:59.798 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:44:59 smithi045 podman[71061]: 2024-05-21 15:44:59.63465557 +0000 UTC m=+0.490148038 container attach ed7a6d6e99aba50a5368ae4219ca71671b72e5bd5cfa83062065d2861a9af673 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-21T15:45:00.110 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 21 15:44:59 smithi045 podman[71061]: 2024-05-21 15:44:59.802069226 +0000 UTC m=+0.657561693 container died ed7a6d6e99aba50a5368ae4219ca71671b72e5bd5cfa83062065d2861a9af673 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:45:00.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.7.service' 2024-05-21T15:45:00.523 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:45:00.523 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-21T15:45:00.524 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-21T15:45:00.524 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.8 2024-05-21T15:45:00.836 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:45:00 smithi053 systemd[1]: Stopping Ceph osd.8 for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:45:01.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:00 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:00.542+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:58.513926+0000 front 2024-05-21T15:43:58.513859+0000 (oldest deadline 2024-05-21T15:45:00.214092+0000) 2024-05-21T15:45:01.203 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:00 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:00.838+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:01.204 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:45:00 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-8[54931]: 2024-05-21T15:45:00.859+0000 7fb0042d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:45:01.204 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:45:00 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-8[54931]: 2024-05-21T15:45:00.859+0000 7fb0042d0700 -1 osd.8 93 *** Got signal Terminated *** 2024-05-21T15:45:01.204 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:45:00 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-8[54931]: 2024-05-21T15:45:00.859+0000 7fb0042d0700 -1 osd.8 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:45:02.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:01 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:01.588+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:58.513926+0000 front 2024-05-21T15:43:58.513859+0000 (oldest deadline 2024-05-21T15:45:00.214092+0000) 2024-05-21T15:45:02.203 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:01 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:01.863+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:02.933 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:02 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:02.576+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:58.513926+0000 front 2024-05-21T15:43:58.513859+0000 (oldest deadline 2024-05-21T15:45:00.214092+0000) 2024-05-21T15:45:03.203 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:02 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:02.821+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:04.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:03 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:03.613+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:58.513926+0000 front 2024-05-21T15:43:58.513859+0000 (oldest deadline 2024-05-21T15:45:00.214092+0000) 2024-05-21T15:45:04.203 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:03 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:03.796+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:05.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:04 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:04.646+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:58.513926+0000 front 2024-05-21T15:43:58.513859+0000 (oldest deadline 2024-05-21T15:45:00.214092+0000) 2024-05-21T15:45:05.204 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:04 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:04.799+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:06.009 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:05 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:05.562+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:06.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:05 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:05.639+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:58.513926+0000 front 2024-05-21T15:43:58.513859+0000 (oldest deadline 2024-05-21T15:45:00.214092+0000) 2024-05-21T15:45:06.203 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:05 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:05.756+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:06.833 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:06 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:06.708+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:06.833 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:45:06 smithi053 podman[71574]: 2024-05-21 15:45:06.552831142 +0000 UTC m=+5.825471119 container died d2c2e989f77e4a28a1b6de038f80ab46660f004c540dcde8c7972043afb50995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-8, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-05-21T15:45:07.008 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:06 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:06.548+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:07.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:06 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:06.688+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:58.513926+0000 front 2024-05-21T15:43:58.513859+0000 (oldest deadline 2024-05-21T15:45:00.214092+0000) 2024-05-21T15:45:07.203 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:45:06 smithi053 podman[71574]: 2024-05-21 15:45:06.927302317 +0000 UTC m=+6.199942293 container cleanup d2c2e989f77e4a28a1b6de038f80ab46660f004c540dcde8c7972043afb50995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-8, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-05-21T15:45:07.203 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:45:06 smithi053 bash[71574]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-8 2024-05-21T15:45:07.618 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:45:07 smithi053 podman[71592]: 2024-05-21 15:45:07.236840423 +0000 UTC m=+0.681315474 container remove d2c2e989f77e4a28a1b6de038f80ab46660f004c540dcde8c7972043afb50995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:45:07.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:07 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:07.507+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:07.888 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:07 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:07.666+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:07.888 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:45:07 smithi053 podman[71682]: 2024-05-21 15:45:07.513371805 +0000 UTC m=+0.017694796 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:45:07.888 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:45:07 smithi053 podman[71682]: 2024-05-21 15:45:07.654208115 +0000 UTC m=+0.158531095 container create 09be504b6f35d6f4fbd5f98cfc8a069ad588c0437f17a47859171e73499d7235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-21T15:45:07.888 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:45:07 smithi053 podman[71682]: 2024-05-21 15:45:07.89107083 +0000 UTC m=+0.395393818 container init 09be504b6f35d6f4fbd5f98cfc8a069ad588c0437f17a47859171e73499d7235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-21T15:45:08.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:07 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:07.639+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:58.513926+0000 front 2024-05-21T15:43:58.513859+0000 (oldest deadline 2024-05-21T15:45:00.214092+0000) 2024-05-21T15:45:08.202 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:45:07 smithi053 podman[71682]: 2024-05-21 15:45:07.894806113 +0000 UTC m=+0.399129093 container start 09be504b6f35d6f4fbd5f98cfc8a069ad588c0437f17a47859171e73499d7235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True) 2024-05-21T15:45:08.202 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:45:08 smithi053 podman[71682]: 2024-05-21 15:45:08.024023193 +0000 UTC m=+0.528346185 container attach 09be504b6f35d6f4fbd5f98cfc8a069ad588c0437f17a47859171e73499d7235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-21T15:45:08.453 INFO:journalctl@ceph.osd.8.smithi053.stdout:May 21 15:45:08 smithi053 podman[71682]: 2024-05-21 15:45:08.205755328 +0000 UTC m=+0.710078320 container died 09be504b6f35d6f4fbd5f98cfc8a069ad588c0437f17a47859171e73499d7235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:45:08.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:08 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:08.485+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:08.933 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:08 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:08.627+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:08.948 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.8.service' 2024-05-21T15:45:08.987 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:45:08.987 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-21T15:45:08.987 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-21T15:45:08.987 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.9 2024-05-21T15:45:09.015 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:08 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:08.602+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:58.513926+0000 front 2024-05-21T15:43:58.513859+0000 (oldest deadline 2024-05-21T15:45:00.214092+0000) 2024-05-21T15:45:09.272 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:09 smithi038 systemd[1]: Stopping Ceph osd.9 for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:45:09.604 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:09 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:09.299+0000 7fd964bb0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:45:09.604 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:09 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:09.299+0000 7fd964bb0700 -1 osd.9 93 *** Got signal Terminated *** 2024-05-21T15:45:09.604 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:09 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:09.299+0000 7fd964bb0700 -1 osd.9 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:45:09.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:09 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:09.628+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:10.008 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:09 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:09.521+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:10.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:09 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:09.606+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:58.513926+0000 front 2024-05-21T15:43:58.513859+0000 (oldest deadline 2024-05-21T15:45:00.214092+0000) 2024-05-21T15:45:10.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:09 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:09.606+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.216568+0000 front 2024-05-21T15:44:07.216211+0000 (oldest deadline 2024-05-21T15:45:08.916312+0000) 2024-05-21T15:45:10.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:10 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:10.669+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:11.008 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:10 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:10.547+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:11.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:10 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:10.609+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:58.513926+0000 front 2024-05-21T15:43:58.513859+0000 (oldest deadline 2024-05-21T15:45:00.214092+0000) 2024-05-21T15:45:11.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:10 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:10.609+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.216568+0000 front 2024-05-21T15:44:07.216211+0000 (oldest deadline 2024-05-21T15:45:08.916312+0000) 2024-05-21T15:45:11.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:11 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:11.504+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:11.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:11 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:11.504+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:05.098939+0000 front 2024-05-21T15:44:05.098903+0000 (oldest deadline 2024-05-21T15:45:10.999062+0000) 2024-05-21T15:45:11.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:11 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:11.627+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:12.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:11 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:11.586+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:58.513926+0000 front 2024-05-21T15:43:58.513859+0000 (oldest deadline 2024-05-21T15:45:00.214092+0000) 2024-05-21T15:45:12.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:11 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:11.586+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.216568+0000 front 2024-05-21T15:44:07.216211+0000 (oldest deadline 2024-05-21T15:45:08.916312+0000) 2024-05-21T15:45:12.759 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:12 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:12.467+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:12.759 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:12 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:12.467+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:05.098939+0000 front 2024-05-21T15:44:05.098903+0000 (oldest deadline 2024-05-21T15:45:10.999062+0000) 2024-05-21T15:45:12.855 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:12 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:12.583+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:13.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:12 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:12.541+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:58.513926+0000 front 2024-05-21T15:43:58.513859+0000 (oldest deadline 2024-05-21T15:45:00.214092+0000) 2024-05-21T15:45:13.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:12 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:12.541+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.216568+0000 front 2024-05-21T15:44:07.216211+0000 (oldest deadline 2024-05-21T15:45:08.916312+0000) 2024-05-21T15:45:13.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:13 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:13.498+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:13.759 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:13 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:13.498+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:05.098939+0000 front 2024-05-21T15:44:05.098903+0000 (oldest deadline 2024-05-21T15:45:10.999062+0000) 2024-05-21T15:45:13.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:13 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:13.622+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:13.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:13 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:13.622+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:14.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:13 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:13.544+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:58.513926+0000 front 2024-05-21T15:43:58.513859+0000 (oldest deadline 2024-05-21T15:45:00.214092+0000) 2024-05-21T15:45:14.022 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:13 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9[66520]: 2024-05-21T15:45:13.544+0000 7fd95d15e700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.216568+0000 front 2024-05-21T15:44:07.216211+0000 (oldest deadline 2024-05-21T15:45:08.916312+0000) 2024-05-21T15:45:14.759 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:14 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:14.456+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:14.759 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:14 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:14.456+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:05.098939+0000 front 2024-05-21T15:44:05.098903+0000 (oldest deadline 2024-05-21T15:45:10.999062+0000) 2024-05-21T15:45:14.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:14 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:14.619+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:14.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:14 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:14.619+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:15.131 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:14 smithi038 podman[100449]: 2024-05-21 15:45:14.840051223 +0000 UTC m=+5.649770066 container died 6be7680a5c9b554a21d0681512ac0c9a1e1334d7ca85dab7d884bb9d9a2dc38a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:45:15.522 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:15 smithi038 podman[100449]: 2024-05-21 15:45:15.208296817 +0000 UTC m=+6.018015650 container cleanup 6be7680a5c9b554a21d0681512ac0c9a1e1334d7ca85dab7d884bb9d9a2dc38a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-21T15:45:15.522 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:15 smithi038 bash[100449]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9 2024-05-21T15:45:15.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:15 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:15.476+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:15.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:15 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:15.476+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:05.098939+0000 front 2024-05-21T15:44:05.098903+0000 (oldest deadline 2024-05-21T15:45:10.999062+0000) 2024-05-21T15:45:15.894 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:15 smithi038 podman[100466]: 2024-05-21 15:45:15.56775863 +0000 UTC m=+0.724150768 container remove 6be7680a5c9b554a21d0681512ac0c9a1e1334d7ca85dab7d884bb9d9a2dc38a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-21T15:45:15.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:15 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:15.625+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:15.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:15 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:15.625+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:16.226 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:15 smithi038 podman[100556]: 2024-05-21 15:45:15.80047137 +0000 UTC m=+0.017572772 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:45:16.226 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:15 smithi038 podman[100556]: 2024-05-21 15:45:15.902118142 +0000 UTC m=+0.119219525 container create 750c3bfc92e3e00ff82ea589840a3d5d908b63458bb86843ea7ce8b88cb65936 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514) 2024-05-21T15:45:16.226 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:16 smithi038 podman[100556]: 2024-05-21 15:45:16.129646231 +0000 UTC m=+0.346747633 container init 750c3bfc92e3e00ff82ea589840a3d5d908b63458bb86843ea7ce8b88cb65936 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-21T15:45:16.226 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:16 smithi038 podman[100556]: 2024-05-21 15:45:16.132884754 +0000 UTC m=+0.349986146 container start 750c3bfc92e3e00ff82ea589840a3d5d908b63458bb86843ea7ce8b88cb65936 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-05-21T15:45:16.227 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:16 smithi038 podman[100556]: 2024-05-21 15:45:16.229620158 +0000 UTC m=+0.446721563 container attach 750c3bfc92e3e00ff82ea589840a3d5d908b63458bb86843ea7ce8b88cb65936 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:45:16.522 INFO:journalctl@ceph.osd.9.smithi038.stdout:May 21 15:45:16 smithi038 podman[100556]: 2024-05-21 15:45:16.440419633 +0000 UTC m=+0.657521040 container died 750c3bfc92e3e00ff82ea589840a3d5d908b63458bb86843ea7ce8b88cb65936 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True) 2024-05-21T15:45:16.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:16 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:16.610+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:16.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:16 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:16.610+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:17.008 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:16 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:16.517+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:17.008 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:16 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:16.517+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:05.098939+0000 front 2024-05-21T15:44:05.098903+0000 (oldest deadline 2024-05-21T15:45:10.999062+0000) 2024-05-21T15:45:17.122 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.9.service' 2024-05-21T15:45:17.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:45:17.205 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-21T15:45:17.205 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-21T15:45:17.205 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.10 2024-05-21T15:45:17.508 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:17 smithi045 systemd[1]: Stopping Ceph osd.10 for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:45:17.509 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:17 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:17.482+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:17.509 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:17 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:17.482+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:05.098939+0000 front 2024-05-21T15:44:05.098903+0000 (oldest deadline 2024-05-21T15:45:10.999062+0000) 2024-05-21T15:45:17.509 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:17 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:17.482+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:16.200574+0000 front 2024-05-21T15:44:16.200663+0000 (oldest deadline 2024-05-21T15:45:17.300714+0000) 2024-05-21T15:45:17.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:17 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:17.630+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:17.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:17 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:17.630+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:18.008 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:17 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:17.551+0000 7f533afdf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:45:18.009 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:17 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:17.551+0000 7f533afdf700 -1 osd.10 93 *** Got signal Terminated *** 2024-05-21T15:45:18.009 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:17 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:17.551+0000 7f533afdf700 -1 osd.10 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:45:18.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:18 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:18.443+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:18.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:18 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:18.443+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:05.098939+0000 front 2024-05-21T15:44:05.098903+0000 (oldest deadline 2024-05-21T15:45:10.999062+0000) 2024-05-21T15:45:18.759 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:18 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:18.443+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:16.200574+0000 front 2024-05-21T15:44:16.200663+0000 (oldest deadline 2024-05-21T15:45:17.300714+0000) 2024-05-21T15:45:18.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:18 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:18.652+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:18.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:18 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:18.652+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:19.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:19 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:19.418+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:19.759 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:19 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:19.418+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:05.098939+0000 front 2024-05-21T15:44:05.098903+0000 (oldest deadline 2024-05-21T15:45:10.999062+0000) 2024-05-21T15:45:19.759 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:19 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:19.418+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:16.200574+0000 front 2024-05-21T15:44:16.200663+0000 (oldest deadline 2024-05-21T15:45:17.300714+0000) 2024-05-21T15:45:19.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:19 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:19.682+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:19.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:19 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:19.682+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:19.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:19 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:19.682+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:15.541903+0000 front 2024-05-21T15:44:15.541950+0000 (oldest deadline 2024-05-21T15:45:19.042033+0000) 2024-05-21T15:45:20.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:20 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:20.467+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:20.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:20 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:20.467+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:05.098939+0000 front 2024-05-21T15:44:05.098903+0000 (oldest deadline 2024-05-21T15:45:10.999062+0000) 2024-05-21T15:45:20.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:20 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:20.467+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:16.200574+0000 front 2024-05-21T15:44:16.200663+0000 (oldest deadline 2024-05-21T15:45:17.300714+0000) 2024-05-21T15:45:20.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:20 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:20.640+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:20.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:20 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:20.640+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:20.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:20 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:20.640+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:15.541903+0000 front 2024-05-21T15:44:15.541950+0000 (oldest deadline 2024-05-21T15:45:19.042033+0000) 2024-05-21T15:45:21.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:21 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:21.465+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:21.759 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:21 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:21.465+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:05.098939+0000 front 2024-05-21T15:44:05.098903+0000 (oldest deadline 2024-05-21T15:45:10.999062+0000) 2024-05-21T15:45:21.759 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:21 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:21.465+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:16.200574+0000 front 2024-05-21T15:44:16.200663+0000 (oldest deadline 2024-05-21T15:45:17.300714+0000) 2024-05-21T15:45:21.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:21 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:21.678+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:21.954 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:21 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:21.678+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:21.954 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:21 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:21.678+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:15.541903+0000 front 2024-05-21T15:44:15.541950+0000 (oldest deadline 2024-05-21T15:45:19.042033+0000) 2024-05-21T15:45:22.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:22 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:22.442+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:59.297921+0000 front 2024-05-21T15:43:59.297840+0000 (oldest deadline 2024-05-21T15:45:05.098348+0000) 2024-05-21T15:45:22.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:22 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:22.442+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:05.098939+0000 front 2024-05-21T15:44:05.098903+0000 (oldest deadline 2024-05-21T15:45:10.999062+0000) 2024-05-21T15:45:22.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:22 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10[61102]: 2024-05-21T15:45:22.442+0000 7f5337595700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:16.200574+0000 front 2024-05-21T15:44:16.200663+0000 (oldest deadline 2024-05-21T15:45:17.300714+0000) 2024-05-21T15:45:22.941 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:22 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:22.662+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:22.941 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:22 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:22.662+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:22.941 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:22 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:22.662+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:15.541903+0000 front 2024-05-21T15:44:15.541950+0000 (oldest deadline 2024-05-21T15:45:19.042033+0000) 2024-05-21T15:45:23.472 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:23 smithi045 podman[71168]: 2024-05-21 15:45:23.2115279 +0000 UTC m=+5.793176935 container died f54f4ab9db6e583002a0c1f76826de6d0a48cdae944698c67dae7b9f47716074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:45:23.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:23 smithi045 podman[71168]: 2024-05-21 15:45:23.566108194 +0000 UTC m=+6.147757226 container cleanup f54f4ab9db6e583002a0c1f76826de6d0a48cdae944698c67dae7b9f47716074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-21T15:45:23.758 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:23 smithi045 bash[71168]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10 2024-05-21T15:45:23.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:23 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:23.630+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:23.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:23 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:23.630+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:23.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:23 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:23.630+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:15.541903+0000 front 2024-05-21T15:44:15.541950+0000 (oldest deadline 2024-05-21T15:45:19.042033+0000) 2024-05-21T15:45:24.183 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:23 smithi045 podman[71195]: 2024-05-21 15:45:23.76774237 +0000 UTC m=+0.547933020 container remove f54f4ab9db6e583002a0c1f76826de6d0a48cdae944698c67dae7b9f47716074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:45:24.443 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:24 smithi045 podman[71284]: 2024-05-21 15:45:24.089913645 +0000 UTC m=+0.017233040 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:45:24.444 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:24 smithi045 podman[71284]: 2024-05-21 15:45:24.223865474 +0000 UTC m=+0.151184845 container create 58bbad6f24c3ba924800efbe648df11d748cc4e8522ae7a93a9afc82fe88e35b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-05-21T15:45:24.444 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:24 smithi045 podman[71284]: 2024-05-21 15:45:24.447103786 +0000 UTC m=+0.374423165 container init 58bbad6f24c3ba924800efbe648df11d748cc4e8522ae7a93a9afc82fe88e35b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:45:24.751 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:24 smithi045 podman[71284]: 2024-05-21 15:45:24.450439958 +0000 UTC m=+0.377759338 container start 58bbad6f24c3ba924800efbe648df11d748cc4e8522ae7a93a9afc82fe88e35b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-21T15:45:24.751 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:24 smithi045 podman[71284]: 2024-05-21 15:45:24.52960982 +0000 UTC m=+0.456929230 container attach 58bbad6f24c3ba924800efbe648df11d748cc4e8522ae7a93a9afc82fe88e35b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-05-21T15:45:24.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:24 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:24.644+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:24.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:24 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:24.644+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:24.954 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:24 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:24.644+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:15.541903+0000 front 2024-05-21T15:44:15.541950+0000 (oldest deadline 2024-05-21T15:45:19.042033+0000) 2024-05-21T15:45:25.008 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 21 15:45:24 smithi045 podman[71284]: 2024-05-21 15:45:24.754902631 +0000 UTC m=+0.682222019 container died 58bbad6f24c3ba924800efbe648df11d748cc4e8522ae7a93a9afc82fe88e35b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-21T15:45:25.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.10.service' 2024-05-21T15:45:25.367 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:45:25.367 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-21T15:45:25.367 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-21T15:45:25.368 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.11 2024-05-21T15:45:25.702 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:25 smithi053 systemd[1]: Stopping Ceph osd.11 for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:45:25.703 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:25 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:25.623+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:25.703 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:25 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:25.623+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:25.703 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:25 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:25.623+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:15.541903+0000 front 2024-05-21T15:44:15.541950+0000 (oldest deadline 2024-05-21T15:45:19.042033+0000) 2024-05-21T15:45:25.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:25 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:25.705+0000 7ff24e688700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:45:25.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:25 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:25.705+0000 7ff24e688700 -1 osd.11 93 *** Got signal Terminated *** 2024-05-21T15:45:25.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:25 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:25.705+0000 7ff24e688700 -1 osd.11 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:45:26.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:26 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:26.619+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:26.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:26 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:26.619+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:26.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:26 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:26.619+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:15.541903+0000 front 2024-05-21T15:44:15.541950+0000 (oldest deadline 2024-05-21T15:45:19.042033+0000) 2024-05-21T15:45:27.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:27 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:27.578+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:27.954 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:27 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:27.578+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:27.954 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:27 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:27.578+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:15.541903+0000 front 2024-05-21T15:44:15.541950+0000 (oldest deadline 2024-05-21T15:45:19.042033+0000) 2024-05-21T15:45:27.954 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:27 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:27.578+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-05-21T15:44:23.043737+0000 front 2024-05-21T15:44:23.043761+0000 (oldest deadline 2024-05-21T15:45:27.143739+0000) 2024-05-21T15:45:28.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:28 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:28.620+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:28.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:28 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:28.620+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:28.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:28 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:28.620+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:15.541903+0000 front 2024-05-21T15:44:15.541950+0000 (oldest deadline 2024-05-21T15:45:19.042033+0000) 2024-05-21T15:45:28.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:28 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:28.620+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-05-21T15:44:23.043737+0000 front 2024-05-21T15:44:23.043761+0000 (oldest deadline 2024-05-21T15:45:27.143739+0000) 2024-05-21T15:45:29.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:29 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:29.594+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:29.954 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:29 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:29.594+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:29.954 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:29 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:29.594+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:15.541903+0000 front 2024-05-21T15:44:15.541950+0000 (oldest deadline 2024-05-21T15:45:19.042033+0000) 2024-05-21T15:45:29.954 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:29 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:29.594+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-05-21T15:44:23.043737+0000 front 2024-05-21T15:44:23.043761+0000 (oldest deadline 2024-05-21T15:45:27.143739+0000) 2024-05-21T15:45:30.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:30 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:30.598+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-05-21T15:43:55.137306+0000 front 2024-05-21T15:43:55.137456+0000 (oldest deadline 2024-05-21T15:45:00.437523+0000) 2024-05-21T15:45:30.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:30 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:30.598+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-21T15:44:07.439997+0000 front 2024-05-21T15:44:07.439958+0000 (oldest deadline 2024-05-21T15:45:13.240746+0000) 2024-05-21T15:45:30.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:30 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:30.598+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2024-05-21T15:44:15.541903+0000 front 2024-05-21T15:44:15.541950+0000 (oldest deadline 2024-05-21T15:45:19.042033+0000) 2024-05-21T15:45:30.954 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:30 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11[61659]: 2024-05-21T15:45:30.598+0000 7ff24ac3e700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-05-21T15:44:23.043737+0000 front 2024-05-21T15:44:23.043761+0000 (oldest deadline 2024-05-21T15:45:27.143739+0000) 2024-05-21T15:45:31.495 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:31 smithi053 podman[71799]: 2024-05-21 15:45:31.227835858 +0000 UTC m=+5.650877832 container died 8067a1c59cc5f8fa6d9131d2ead256ad159ba0d174a5695acd6c6a7bffbb71d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-21T15:45:31.836 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:31 smithi053 podman[71799]: 2024-05-21 15:45:31.579907801 +0000 UTC m=+6.002949774 container cleanup 8067a1c59cc5f8fa6d9131d2ead256ad159ba0d174a5695acd6c6a7bffbb71d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:45:31.836 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:31 smithi053 bash[71799]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11 2024-05-21T15:45:32.203 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:31 smithi053 podman[71814]: 2024-05-21 15:45:31.839483512 +0000 UTC m=+0.611158527 container remove 8067a1c59cc5f8fa6d9131d2ead256ad159ba0d174a5695acd6c6a7bffbb71d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:45:32.480 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:32 smithi053 podman[71903]: 2024-05-21 15:45:32.115136458 +0000 UTC m=+0.018349410 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:45:32.481 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:32 smithi053 podman[71903]: 2024-05-21 15:45:32.222713987 +0000 UTC m=+0.125926925 container create 810d5ddd567dea034d4ddc502c3adef1ede2b946c84697b8f2ed11bd67a24483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:45:32.481 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:32 smithi053 podman[71903]: 2024-05-21 15:45:32.417274803 +0000 UTC m=+0.320487759 container init 810d5ddd567dea034d4ddc502c3adef1ede2b946c84697b8f2ed11bd67a24483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:45:32.481 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:32 smithi053 podman[71903]: 2024-05-21 15:45:32.420528568 +0000 UTC m=+0.323741509 container start 810d5ddd567dea034d4ddc502c3adef1ede2b946c84697b8f2ed11bd67a24483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:45:32.481 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:32 smithi053 podman[71903]: 2024-05-21 15:45:32.48334622 +0000 UTC m=+0.386559159 container attach 810d5ddd567dea034d4ddc502c3adef1ede2b946c84697b8f2ed11bd67a24483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0) 2024-05-21T15:45:32.953 INFO:journalctl@ceph.osd.11.smithi053.stdout:May 21 15:45:32 smithi053 podman[71903]: 2024-05-21 15:45:32.724183683 +0000 UTC m=+0.627396621 container died 810d5ddd567dea034d4ddc502c3adef1ede2b946c84697b8f2ed11bd67a24483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:45:33.337 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@osd.11.service' 2024-05-21T15:45:33.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:45:33.380 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-21T15:45:33.380 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-21T15:45:33.380 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.a 2024-05-21T15:45:33.675 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:45:33 smithi038 systemd[1]: Stopping Ceph mds.a for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:45:34.022 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:45:33 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-a[72894]: 2024-05-21T15:45:33.677+0000 7fce3e806700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:45:34.022 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:45:33 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-a[72894]: 2024-05-21T15:45:33.677+0000 7fce3e806700 -1 mds.a *** got signal Terminated *** 2024-05-21T15:45:37.941 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:45:37 smithi038 podman[100687]: 2024-05-21 15:45:37.691124948 +0000 UTC m=+4.105623990 container died 0ddaa288b363cd79565a5a425fa0ca50890e2427213e605aae5247ee41170274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-a, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-21T15:45:38.200 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:45:38 smithi038 podman[100687]: 2024-05-21 15:45:38.043610403 +0000 UTC m=+4.458109400 container cleanup 0ddaa288b363cd79565a5a425fa0ca50890e2427213e605aae5247ee41170274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:45:38.200 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:45:38 smithi038 bash[100687]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-a 2024-05-21T15:45:38.444 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.a.service' 2024-05-21T15:45:38.467 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:45:38 smithi038 podman[100701]: 2024-05-21 15:45:38.203405989 +0000 UTC m=+0.505091839 container remove 0ddaa288b363cd79565a5a425fa0ca50890e2427213e605aae5247ee41170274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=) 2024-05-21T15:45:38.467 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:45:38 smithi038 systemd[1]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.a.service: Deactivated successfully. 2024-05-21T15:45:38.467 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:45:38 smithi038 systemd[1]: Stopped Ceph mds.a for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:45:38.467 INFO:journalctl@ceph.mds.a.smithi038.stdout:May 21 15:45:38 smithi038 systemd[1]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.a.service: Consumed 2.655s CPU time. 2024-05-21T15:45:38.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:45:38.481 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-21T15:45:38.481 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-21T15:45:38.481 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.d 2024-05-21T15:45:38.772 INFO:journalctl@ceph.mds.d.smithi038.stdout:May 21 15:45:38 smithi038 systemd[1]: Stopping Ceph mds.d for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:45:39.180 INFO:journalctl@ceph.mds.d.smithi038.stdout:May 21 15:45:38 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-d[71896]: 2024-05-21T15:45:38.822+0000 7fb544e58700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:45:39.180 INFO:journalctl@ceph.mds.d.smithi038.stdout:May 21 15:45:38 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-d[71896]: 2024-05-21T15:45:38.822+0000 7fb544e58700 -1 mds.d *** got signal Terminated *** 2024-05-21T15:45:41.911 INFO:journalctl@ceph.mds.d.smithi038.stdout:May 21 15:45:41 smithi038 podman[100807]: 2024-05-21 15:45:41.597347017 +0000 UTC m=+2.888323106 container died a77c7155721b199c11597165167f6d6f90be7677d41a30018b6d314ad13bb9a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-05-21T15:45:41.911 INFO:journalctl@ceph.mds.d.smithi038.stdout:May 21 15:45:41 smithi038 podman[100807]: 2024-05-21 15:45:41.914271367 +0000 UTC m=+3.205247456 container cleanup a77c7155721b199c11597165167f6d6f90be7677d41a30018b6d314ad13bb9a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-21T15:45:42.272 INFO:journalctl@ceph.mds.d.smithi038.stdout:May 21 15:45:41 smithi038 bash[100807]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-d 2024-05-21T15:45:42.272 INFO:journalctl@ceph.mds.d.smithi038.stdout:May 21 15:45:42 smithi038 podman[100818]: 2024-05-21 15:45:42.098873735 +0000 UTC m=+0.498958061 container remove a77c7155721b199c11597165167f6d6f90be7677d41a30018b6d314ad13bb9a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-21T15:45:42.350 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.d.service' 2024-05-21T15:45:42.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:45:42.384 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-21T15:45:42.385 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-21T15:45:42.385 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.g 2024-05-21T15:45:42.715 INFO:journalctl@ceph.mds.g.smithi038.stdout:May 21 15:45:42 smithi038 systemd[1]: Stopping Ceph mds.g for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:45:43.022 INFO:journalctl@ceph.mds.g.smithi038.stdout:May 21 15:45:42 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-g[72540]: 2024-05-21T15:45:42.718+0000 7f8e0a5f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:45:43.022 INFO:journalctl@ceph.mds.g.smithi038.stdout:May 21 15:45:42 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-g[72540]: 2024-05-21T15:45:42.718+0000 7f8e0a5f5700 -1 mds.g *** got signal Terminated *** 2024-05-21T15:45:46.724 INFO:journalctl@ceph.mds.g.smithi038.stdout:May 21 15:45:46 smithi038 podman[100929]: 2024-05-21 15:45:46.433460047 +0000 UTC m=+3.815892797 container died 24307d2a9a6a946b21e65091f4bef5bcdea7192f2604ca2dc4c40d76a0e650c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-g, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:45:47.022 INFO:journalctl@ceph.mds.g.smithi038.stdout:May 21 15:45:46 smithi038 podman[100929]: 2024-05-21 15:45:46.810201971 +0000 UTC m=+4.192634712 container cleanup 24307d2a9a6a946b21e65091f4bef5bcdea7192f2604ca2dc4c40d76a0e650c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-g, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:45:47.022 INFO:journalctl@ceph.mds.g.smithi038.stdout:May 21 15:45:46 smithi038 bash[100929]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-g 2024-05-21T15:45:47.209 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.g.service' 2024-05-21T15:45:47.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:45:47.243 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-21T15:45:47.243 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-21T15:45:47.243 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.j 2024-05-21T15:45:47.686 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:45:47 smithi038 systemd[1]: Stopping Ceph mds.j for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:45:48.022 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:45:47 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-j[71551]: 2024-05-21T15:45:47.688+0000 7f8b57eff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:45:48.022 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:45:47 smithi038 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-j[71551]: 2024-05-21T15:45:47.688+0000 7f8b57eff700 -1 mds.j *** got signal Terminated *** 2024-05-21T15:45:51.127 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:45:50 smithi038 podman[101050]: 2024-05-21 15:45:50.809528553 +0000 UTC m=+3.338997907 container died e998518fd25d28828fe2bfc5d64c192f6f6fd8e9762176e9b18688b39014dfaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-j, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:45:51.127 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:45:51 smithi038 podman[101050]: 2024-05-21 15:45:51.13041241 +0000 UTC m=+3.659881755 container cleanup e998518fd25d28828fe2bfc5d64c192f6f6fd8e9762176e9b18688b39014dfaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-j, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:45:51.523 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:45:51 smithi038 bash[101050]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-j 2024-05-21T15:45:51.523 INFO:journalctl@ceph.mds.j.smithi038.stdout:May 21 15:45:51 smithi038 podman[101063]: 2024-05-21 15:45:51.306813529 +0000 UTC m=+0.495162044 container remove e998518fd25d28828fe2bfc5d64c192f6f6fd8e9762176e9b18688b39014dfaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-j, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-21T15:45:51.563 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.j.service' 2024-05-21T15:45:51.595 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:45:51.596 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-21T15:45:51.596 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-21T15:45:51.596 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.b 2024-05-21T15:45:51.906 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 21 15:45:51 smithi045 systemd[1]: Stopping Ceph mds.b for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:45:52.258 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 21 15:45:51 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-b[66443]: 2024-05-21T15:45:51.908+0000 7fc8fd271700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:45:52.258 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 21 15:45:51 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-b[66443]: 2024-05-21T15:45:51.908+0000 7fc8fd271700 -1 mds.b *** got signal Terminated *** 2024-05-21T15:45:54.706 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 21 15:45:54 smithi045 podman[71388]: 2024-05-21 15:45:54.426561366 +0000 UTC m=+2.629198899 container died 4d4648d5453d3b3536d29f788e45eec8dbbd2e295257bb83ee62d009b37b2e41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-21T15:45:55.008 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 21 15:45:54 smithi045 podman[71388]: 2024-05-21 15:45:54.775714723 +0000 UTC m=+2.978352234 container cleanup 4d4648d5453d3b3536d29f788e45eec8dbbd2e295257bb83ee62d009b37b2e41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:45:55.008 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 21 15:45:54 smithi045 bash[71388]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-b 2024-05-21T15:45:55.225 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.b.service' 2024-05-21T15:45:55.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:45:55.261 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-21T15:45:55.261 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-21T15:45:55.261 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.e 2024-05-21T15:45:55.585 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:45:55 smithi045 systemd[1]: Stopping Ceph mds.e for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:45:56.008 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:45:55 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-e[65828]: 2024-05-21T15:45:55.587+0000 7f9e0ca92700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:45:56.008 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:45:55 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-e[65828]: 2024-05-21T15:45:55.587+0000 7f9e0ca92700 -1 mds.e *** got signal Terminated *** 2024-05-21T15:45:58.694 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:45:58 smithi045 podman[71522]: 2024-05-21 15:45:58.419591285 +0000 UTC m=+2.937587754 container died d12932c6e8e9f7c44604d22e80e721922690038fba14e212d0079c2a3d8c00e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:45:59.008 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:45:58 smithi045 podman[71522]: 2024-05-21 15:45:58.805261843 +0000 UTC m=+3.323258311 container cleanup d12932c6e8e9f7c44604d22e80e721922690038fba14e212d0079c2a3d8c00e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-21T15:45:59.008 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 21 15:45:58 smithi045 bash[71522]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-e 2024-05-21T15:45:59.231 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.e.service' 2024-05-21T15:45:59.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:45:59.265 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-21T15:45:59.265 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-21T15:45:59.265 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.h 2024-05-21T15:45:59.758 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:45:59 smithi045 systemd[1]: Stopping Ceph mds.h for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:45:59.759 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:45:59 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-h[66137]: 2024-05-21T15:45:59.600+0000 7fd39b1b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:45:59.759 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:45:59 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-h[66137]: 2024-05-21T15:45:59.600+0000 7fd39b1b0700 -1 mds.h *** got signal Terminated *** 2024-05-21T15:46:02.674 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:46:02 smithi045 podman[71644]: 2024-05-21 15:46:02.408414304 +0000 UTC m=+2.910406306 container died d464e15f31c2e862e20ad622f6f453aea6893fb62cab5d1646e2cac7850e9cf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-21T15:46:02.957 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:46:02 smithi045 podman[71644]: 2024-05-21 15:46:02.751175833 +0000 UTC m=+3.253167826 container cleanup d464e15f31c2e862e20ad622f6f453aea6893fb62cab5d1646e2cac7850e9cf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, io.buildah.version=1.33.5) 2024-05-21T15:46:02.957 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:46:02 smithi045 bash[71644]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-h 2024-05-21T15:46:03.196 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.h.service' 2024-05-21T15:46:03.217 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:46:02 smithi045 podman[71656]: 2024-05-21 15:46:02.960821922 +0000 UTC m=+0.545428097 container remove d464e15f31c2e862e20ad622f6f453aea6893fb62cab5d1646e2cac7850e9cf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-h, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-21T15:46:03.217 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:46:03 smithi045 systemd[1]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.h.service: Deactivated successfully. 2024-05-21T15:46:03.217 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:46:03 smithi045 systemd[1]: Stopped Ceph mds.h for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:46:03.217 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 21 15:46:03 smithi045 systemd[1]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.h.service: Consumed 9.672s CPU time. 2024-05-21T15:46:03.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:46:03.229 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-21T15:46:03.229 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-21T15:46:03.229 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.k 2024-05-21T15:46:03.508 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:46:03 smithi045 systemd[1]: Stopping Ceph mds.k for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:46:04.008 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:46:03 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-k[65513]: 2024-05-21T15:46:03.570+0000 7fb9bfecb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:46:04.008 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:46:03 smithi045 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-k[65513]: 2024-05-21T15:46:03.570+0000 7fb9bfecb700 -1 mds.k *** got signal Terminated *** 2024-05-21T15:46:06.727 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:46:06 smithi045 podman[71765]: 2024-05-21 15:46:06.430044903 +0000 UTC m=+2.974197218 container died cf789efac309d22cc12bc417bf204dfa7edcf129f7f4cf98d7c0c878460963dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-21T15:46:07.008 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:46:06 smithi045 podman[71765]: 2024-05-21 15:46:06.822146853 +0000 UTC m=+3.366299160 container cleanup cf789efac309d22cc12bc417bf204dfa7edcf129f7f4cf98d7c0c878460963dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-k, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-21T15:46:07.008 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 21 15:46:06 smithi045 bash[71765]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-k 2024-05-21T15:46:07.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.k.service' 2024-05-21T15:46:07.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:46:07.288 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-21T15:46:07.288 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-21T15:46:07.288 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.c 2024-05-21T15:46:07.592 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:46:07 smithi053 systemd[1]: Stopping Ceph mds.c for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:46:07.868 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:46:07 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-c[66608]: 2024-05-21T15:46:07.594+0000 7f47195fc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:46:07.868 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:46:07 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-c[66608]: 2024-05-21T15:46:07.594+0000 7f47195fc700 -1 mds.c *** got signal Terminated *** 2024-05-21T15:46:10.743 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:46:10 smithi053 podman[72011]: 2024-05-21 15:46:10.483147796 +0000 UTC m=+2.986454215 container died d992e7fc917d7d4065e0e5860333cd05393d63a04404507c36832d8a9a9cf084 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:46:11.109 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:46:10 smithi053 podman[72011]: 2024-05-21 15:46:10.844267166 +0000 UTC m=+3.347573576 container cleanup d992e7fc917d7d4065e0e5860333cd05393d63a04404507c36832d8a9a9cf084 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-c, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:46:11.109 INFO:journalctl@ceph.mds.c.smithi053.stdout:May 21 15:46:10 smithi053 bash[72011]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-c 2024-05-21T15:46:11.278 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.c.service' 2024-05-21T15:46:11.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:46:11.314 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-21T15:46:11.314 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-21T15:46:11.314 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.f 2024-05-21T15:46:11.645 INFO:journalctl@ceph.mds.f.smithi053.stdout:May 21 15:46:11 smithi053 systemd[1]: Stopping Ceph mds.f for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:46:11.953 INFO:journalctl@ceph.mds.f.smithi053.stdout:May 21 15:46:11 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-f[66305]: 2024-05-21T15:46:11.647+0000 7fe1d1e0e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:46:11.953 INFO:journalctl@ceph.mds.f.smithi053.stdout:May 21 15:46:11 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-f[66305]: 2024-05-21T15:46:11.647+0000 7fe1d1e0e700 -1 mds.f *** got signal Terminated *** 2024-05-21T15:46:14.747 INFO:journalctl@ceph.mds.f.smithi053.stdout:May 21 15:46:14 smithi053 podman[72144]: 2024-05-21 15:46:14.479018617 +0000 UTC m=+2.934494670 container died 22b42be6955b8779bd653ee904679c6f8e4b7edc71147bac9fe8d7c8bc6413c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-f, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-21T15:46:15.104 INFO:journalctl@ceph.mds.f.smithi053.stdout:May 21 15:46:14 smithi053 podman[72144]: 2024-05-21 15:46:14.848023982 +0000 UTC m=+3.303500027 container cleanup 22b42be6955b8779bd653ee904679c6f8e4b7edc71147bac9fe8d7c8bc6413c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-21T15:46:15.104 INFO:journalctl@ceph.mds.f.smithi053.stdout:May 21 15:46:14 smithi053 bash[72144]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-f 2024-05-21T15:46:15.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.f.service' 2024-05-21T15:46:15.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:46:15.314 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-21T15:46:15.314 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-21T15:46:15.314 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.i 2024-05-21T15:46:15.640 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:46:15 smithi053 systemd[1]: Stopping Ceph mds.i for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:46:15.953 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:46:15 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-i[66908]: 2024-05-21T15:46:15.642+0000 7f875cbf6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:46:15.953 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:46:15 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-i[66908]: 2024-05-21T15:46:15.642+0000 7f875cbf6700 -1 mds.i *** got signal Terminated *** 2024-05-21T15:46:18.703 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:46:18 smithi053 podman[72265]: 2024-05-21 15:46:18.462550177 +0000 UTC m=+2.915914291 container died 80998c82a6867763fcbc759064d96f920d2f6d4f9843065e5049910690fc5da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:46:18.985 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:46:18 smithi053 podman[72265]: 2024-05-21 15:46:18.793338349 +0000 UTC m=+3.246702455 container cleanup 80998c82a6867763fcbc759064d96f920d2f6d4f9843065e5049910690fc5da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-i, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-21T15:46:18.985 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:46:18 smithi053 bash[72265]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-i 2024-05-21T15:46:19.234 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.i.service' 2024-05-21T15:46:19.256 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:46:18 smithi053 podman[72277]: 2024-05-21 15:46:18.988302664 +0000 UTC m=+0.528033063 container remove 80998c82a6867763fcbc759064d96f920d2f6d4f9843065e5049910690fc5da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-21T15:46:19.256 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:46:19 smithi053 systemd[1]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.i.service: Deactivated successfully. 2024-05-21T15:46:19.256 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:46:19 smithi053 systemd[1]: Stopped Ceph mds.i for 102ec4fa-1786-11ef-bc9a-c7b262605968. 2024-05-21T15:46:19.256 INFO:journalctl@ceph.mds.i.smithi053.stdout:May 21 15:46:19 smithi053 systemd[1]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.i.service: Consumed 6.564s CPU time. 2024-05-21T15:46:19.268 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:46:19.268 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-21T15:46:19.269 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-21T15:46:19.269 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.l 2024-05-21T15:46:19.596 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:46:19 smithi053 systemd[1]: Stopping Ceph mds.l for 102ec4fa-1786-11ef-bc9a-c7b262605968... 2024-05-21T15:46:19.953 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:46:19 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-l[66003]: 2024-05-21T15:46:19.599+0000 7f71e0f7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:46:19.953 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:46:19 smithi053 ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-l[66003]: 2024-05-21T15:46:19.599+0000 7f71e0f7b700 -1 mds.l *** got signal Terminated *** 2024-05-21T15:46:22.738 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:46:22 smithi053 podman[72386]: 2024-05-21 15:46:22.460163149 +0000 UTC m=+2.968130818 container died ea5eb1669df59a23d73271dbae0febc4d2d363775ce71ecf32bdd0893998337a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=) 2024-05-21T15:46:23.158 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:46:22 smithi053 podman[72386]: 2024-05-21 15:46:22.824010721 +0000 UTC m=+3.331978378 container cleanup ea5eb1669df59a23d73271dbae0febc4d2d363775ce71ecf32bdd0893998337a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-21T15:46:23.158 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:46:22 smithi053 bash[72386]: ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-l 2024-05-21T15:46:23.158 INFO:journalctl@ceph.mds.l.smithi053.stdout:May 21 15:46:22 smithi053 podman[72398]: 2024-05-21 15:46:22.966706338 +0000 UTC m=+0.479491615 container remove ea5eb1669df59a23d73271dbae0febc4d2d363775ce71ecf32bdd0893998337a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-102ec4fa-1786-11ef-bc9a-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:46:23.182 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-102ec4fa-1786-11ef-bc9a-c7b262605968@mds.l.service' 2024-05-21T15:46:23.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:46:23.214 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-21T15:46:23.214 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T15:46:23.416 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:46:26.096 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T15:46:26.296 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:46:28.963 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T15:46:29.162 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T15:46:31.729 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:46:31.764 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:46:31.797 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:46:31.823 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-21T15:46:31.825 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589/remote/smithi038/crash 2024-05-21T15:46:31.825 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T15:46:31.868 INFO:teuthology.orchestra.run.smithi038.stderr:tar: /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T15:46:31.868 INFO:teuthology.orchestra.run.smithi038.stderr:tar: Error is not recoverable: exiting now 2024-05-21T15:46:31.870 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589/remote/smithi045/crash 2024-05-21T15:46:31.871 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T15:46:31.915 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T15:46:31.915 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2024-05-21T15:46:31.917 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589/remote/smithi053/crash 2024-05-21T15:46:31.918 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T15:46:31.956 INFO:teuthology.orchestra.run.smithi053.stderr:tar: /var/lib/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T15:46:31.957 INFO:teuthology.orchestra.run.smithi053.stderr:tar: Error is not recoverable: exiting now 2024-05-21T15:46:31.958 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-21T15:46:31.959 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-21T15:46:32.061 INFO:tasks.cephadm:Compressing logs... 2024-05-21T15:46:32.061 DEBUG:teuthology.orchestra.run.smithi038:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T15:46:32.064 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T15:46:32.067 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T15:46:32.095 INFO:teuthology.orchestra.run.smithi038.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-21T15:46:32.096 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T15:46:32.096 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.log 2024-05-21T15:46:32.096 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T15:46:32.101 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.g.log 2024-05-21T15:46:32.101 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.j.log 2024-05-21T15:46:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.3.log 2024-05-21T15:46:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.0.log 2024-05-21T15:46:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mgr.x.log 2024-05-21T15:46:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mon.a.log 2024-05-21T15:46:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.6.log 2024-05-21T15:46:32.102 INFO:teuthology.orchestra.run.smithi038.stderr: 86.6% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-21T15:46:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T15:46:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.a.log 2024-05-21T15:46:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T15:46:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.d.log 2024-05-21T15:46:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.9.log 2024-05-21T15:46:32.103 INFO:teuthology.orchestra.run.smithi038.stderr: 94.8% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.j.log.gz 2024-05-21T15:46:32.107 INFO:teuthology.orchestra.run.smithi053.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-21T15:46:32.107 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T15:46:32.107 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.log 2024-05-21T15:46:32.107 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.2.log 2024-05-21T15:46:32.107 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.f.log 2024-05-21T15:46:32.107 INFO:teuthology.orchestra.run.smithi053.stderr:gzip/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.2.log: -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T15:46:32.108 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.5.log 2024-05-21T15:46:32.108 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.c.log 2024-05-21T15:46:32.108 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.audit.log: /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mon.c.log 2024-05-21T15:46:32.108 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mgr.z.log 2024-05-21T15:46:32.108 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.i.log 2024-05-21T15:46:32.108 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mon.c.log: /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T15:46:32.108 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.11.log 2024-05-21T15:46:32.108 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.8.log 2024-05-21T15:46:32.109 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.cephadm.log: /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T15:46:32.109 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.l.log 2024-05-21T15:46:32.113 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-21T15:46:32.113 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T15:46:32.113 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.log 2024-05-21T15:46:32.113 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose --/var/log/ceph/cephadm.log: /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T15:46:32.113 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.7.log 2024-05-21T15:46:32.113 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.b.log 2024-05-21T15:46:32.113 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.4.log 2024-05-21T15:46:32.113 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.1.log 2024-05-21T15:46:32.113 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mgr.y.log 2024-05-21T15:46:32.113 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.h.log 2024-05-21T15:46:32.114 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.10.log 2024-05-21T15:46:32.114 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T15:46:32.114 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.e.log 2024-05-21T15:46:32.114 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T15:46:32.114 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mon.b.log 2024-05-21T15:46:32.114 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.k.log 2024-05-21T15:46:32.128 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mon.b.log: /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.k.log: 82.6% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T15:46:32.159 INFO:teuthology.orchestra.run.smithi045.stderr: 92.7% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mgr.y.log.gz 2024-05-21T15:46:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.d.log: /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.9.log: 82.6% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T15:46:32.186 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-volume.log: /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.l.log: 82.5% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T15:46:32.219 INFO:teuthology.orchestra.run.smithi053.stderr: 95.4% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.l.log.gz 2024-05-21T15:46:32.229 INFO:teuthology.orchestra.run.smithi038.stderr: 90.1% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T15:46:32.252 INFO:teuthology.orchestra.run.smithi053.stderr: 90.2% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T15:46:32.291 INFO:teuthology.orchestra.run.smithi053.stderr: 86.6% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-21T15:46:32.293 INFO:teuthology.orchestra.run.smithi053.stderr: 92.8% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mgr.z.log.gz 2024-05-21T15:46:32.303 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T15:46:32.322 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T15:46:32.330 INFO:teuthology.orchestra.run.smithi038.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T15:46:32.397 INFO:teuthology.orchestra.run.smithi045.stderr: 90.2% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T15:46:32.430 INFO:teuthology.orchestra.run.smithi038.stderr: 91.2% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T15:46:32.457 INFO:teuthology.orchestra.run.smithi053.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T15:46:32.521 INFO:teuthology.orchestra.run.smithi038.stderr: 94.6% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.d.log.gz 2024-05-21T15:46:32.766 INFO:teuthology.orchestra.run.smithi045.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T15:46:32.768 INFO:teuthology.orchestra.run.smithi045.stderr: 86.6% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-21T15:46:33.022 INFO:teuthology.orchestra.run.smithi053.stderr: 95.6% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.i.log.gz 2024-05-21T15:46:33.431 INFO:teuthology.orchestra.run.smithi038.stderr: 90.3% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mgr.x.log.gz 2024-05-21T15:46:33.961 INFO:teuthology.orchestra.run.smithi053.stderr: 91.9% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mon.c.log.gz 2024-05-21T15:46:34.693 INFO:teuthology.orchestra.run.smithi038.stderr: 95.6% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.a.log.gz 2024-05-21T15:46:35.267 INFO:teuthology.orchestra.run.smithi045.stderr: 95.6% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.k.log.gz 2024-05-21T15:46:35.331 INFO:teuthology.orchestra.run.smithi038.stderr: 95.5% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.g.log.gz 2024-05-21T15:46:35.490 INFO:teuthology.orchestra.run.smithi045.stderr: 92.1% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mon.b.log.gz 2024-05-21T15:46:38.753 INFO:teuthology.orchestra.run.smithi045.stderr: 94.4% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.b.log.gz 2024-05-21T15:46:39.405 INFO:teuthology.orchestra.run.smithi038.stderr: 90.4% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mon.a.log.gz 2024-05-21T15:46:41.724 INFO:teuthology.orchestra.run.smithi053.stderr: 95.8% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.f.log.gz 2024-05-21T15:46:44.255 INFO:teuthology.orchestra.run.smithi053.stderr: 95.6% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.c.log.gz 2024-05-21T15:46:46.769 INFO:teuthology.orchestra.run.smithi045.stderr: 95.0% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.e.log.gz 2024-05-21T15:46:57.254 INFO:teuthology.orchestra.run.smithi045.stderr: 95.6% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-mds.h.log.gz 2024-05-21T15:48:32.586 INFO:teuthology.orchestra.run.smithi038.stderr: 91.7% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.0.log.gz 2024-05-21T15:52:34.134 INFO:teuthology.orchestra.run.smithi053.stderr: 91.8% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.2.log.gz 2024-05-21T15:53:30.239 INFO:teuthology.orchestra.run.smithi038.stderr: 91.9% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.9.log.gz 2024-05-21T15:54:01.852 INFO:teuthology.orchestra.run.smithi038.stderr: 91.8% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.6.log.gz 2024-05-21T15:54:28.868 INFO:teuthology.orchestra.run.smithi045.stderr: 91.6% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.1.log.gz 2024-05-21T15:54:38.801 INFO:teuthology.orchestra.run.smithi038.stderr: 91.7% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.3.log.gz 2024-05-21T15:54:38.804 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-05-21T15:54:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:real 8m6.726s 2024-05-21T15:54:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:user 6m41.238s 2024-05-21T15:54:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:sys 0m14.372s 2024-05-21T15:58:18.203 INFO:teuthology.orchestra.run.smithi045.stderr: 91.7% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.7.log.gz 2024-05-21T15:59:02.178 INFO:teuthology.orchestra.run.smithi045.stderr: 91.9% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.10.log.gz 2024-05-21T15:59:30.928 INFO:teuthology.orchestra.run.smithi045.stderr: 91.9% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.4.log.gz 2024-05-21T15:59:30.930 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-21T15:59:30.930 INFO:teuthology.orchestra.run.smithi045.stderr:real 12m58.849s 2024-05-21T15:59:30.930 INFO:teuthology.orchestra.run.smithi045.stderr:user 7m27.300s 2024-05-21T15:59:30.930 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m17.621s 2024-05-21T16:00:06.201 INFO:teuthology.orchestra.run.smithi053.stderr: 91.8% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.8.log.gz 2024-05-21T16:01:02.012 INFO:teuthology.orchestra.run.smithi053.stderr: 91.8% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.11.log.gz 2024-05-21T16:01:29.796 INFO:teuthology.orchestra.run.smithi053.stderr: 91.8% -- replaced with /var/log/ceph/102ec4fa-1786-11ef-bc9a-c7b262605968/ceph-osd.5.log.gz 2024-05-21T16:01:29.800 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-05-21T16:01:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:real 14m57.718s 2024-05-21T16:01:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:user 7m17.203s 2024-05-21T16:01:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m18.055s 2024-05-21T16:01:29.801 INFO:tasks.cephadm:Archiving logs... 2024-05-21T16:01:29.801 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589/remote/smithi038/log 2024-05-21T16:01:29.802 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T16:02:33.740 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589/remote/smithi045/log 2024-05-21T16:02:33.742 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T16:03:44.567 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589/remote/smithi053/log 2024-05-21T16:03:44.568 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T16:04:52.723 INFO:tasks.cephadm:Removing cluster... 2024-05-21T16:04:52.727 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 --force 2024-05-21T16:04:52.930 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T16:04:53.656 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 --force 2024-05-21T16:04:53.909 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T16:04:54.666 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 102ec4fa-1786-11ef-bc9a-c7b262605968 --force 2024-05-21T16:04:54.906 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: 102ec4fa-1786-11ef-bc9a-c7b262605968 2024-05-21T16:04:55.669 INFO:tasks.cephadm:Removing cephadm ... 2024-05-21T16:04:55.669 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T16:04:55.690 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T16:04:55.721 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T16:04:55.737 INFO:tasks.cephadm:Teardown complete 2024-05-21T16:04:55.738 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-21T16:04:55.748 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-21T16:04:55.748 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T16:04:55.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T16:04:55.765 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T16:04:55.890 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-21T16:04:55.902 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-21T16:04:55.902 DEBUG:teuthology.orchestra.run.smithi038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T16:04:55.905 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T16:04:55.908 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T16:04:55.921 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-05-21T16:04:55.923 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-05-21T16:04:55.923 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-05-21T16:04:55.926 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T16:04:55.926 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-05-21T16:04:55.926 INFO:teuthology.orchestra.run.smithi038.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 +408us[ +399us] +/- 48ms 2024-05-21T16:04:55.926 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 -1533us[-1533us] +/- 66ms 2024-05-21T16:04:55.926 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 79 +1493us[+1492us] +/- 98ms 2024-05-21T16:04:55.926 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T16:04:55.935 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T16:04:55.935 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-05-21T16:04:55.935 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 +427us[ +445us] +/- 48ms 2024-05-21T16:04:55.935 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -1514us[-1514us] +/- 66ms 2024-05-21T16:04:55.936 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +1522us[+1522us] +/- 68ms 2024-05-21T16:04:55.936 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T16:04:55.938 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T16:04:55.939 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-05-21T16:04:55.939 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 +387us[ +418us] +/- 48ms 2024-05-21T16:04:55.939 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 -1556us[-1525us] +/- 66ms 2024-05-21T16:04:55.939 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +1490us[+1490us] +/- 68ms 2024-05-21T16:04:55.939 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T16:04:55.940 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-21T16:04:55.950 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-21T16:04:55.951 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-21T16:04:55.982 DEBUG:teuthology.orchestra.run.smithi038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T16:04:56.139 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T16:04:56.227 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T16:04:56.310 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T16:04:56.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T16:04:56.343 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T16:04:56.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T16:04:56.375 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T16:04:56.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T16:04:56.403 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-21T16:04:56.414 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-21T16:04:56.437 INFO:teuthology.task.internal:Duration was 2898.123317 seconds 2024-05-21T16:04:56.438 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-21T16:04:56.449 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-21T16:04:56.449 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T16:04:56.452 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T16:04:56.455 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T16:04:56.510 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T16:04:56.519 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T16:04:56.525 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T16:04:56.827 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-21T16:04:56.828 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi038.front.sepia.ceph.com 2024-05-21T16:04:56.828 DEBUG:teuthology.orchestra.run.smithi038:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T16:04:56.852 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-05-21T16:04:56.852 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T16:04:56.882 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-05-21T16:04:56.882 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T16:04:56.911 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-21T16:04:56.911 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T16:04:56.913 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T16:04:56.925 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T16:04:56.980 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-21T16:04:56.980 DEBUG:teuthology.orchestra.run.smithi038:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T16:04:56.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T16:04:57.004 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T16:04:57.390 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-21T16:04:57.401 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-21T16:04:57.401 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T16:04:57.437 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T16:04:57.469 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T16:04:57.504 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-21T16:04:57.516 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T16:04:57.518 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T16:04:57.520 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T16:04:57.543 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = core 2024-05-21T16:04:57.548 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-05-21T16:04:57.575 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-05-21T16:04:57.615 DEBUG:teuthology.orchestra.run.smithi038:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T16:04:57.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T16:04:57.631 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T16:04:57.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T16:04:57.646 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T16:04:57.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T16:04:57.671 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-21T16:04:57.682 INFO:teuthology.task.internal:Transferring archived files... 2024-05-21T16:04:57.684 DEBUG:teuthology.misc:Transferring archived files from smithi038:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589/remote/smithi038 2024-05-21T16:04:57.684 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T16:04:57.728 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589/remote/smithi045 2024-05-21T16:04:57.729 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T16:04:57.768 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718589/remote/smithi053 2024-05-21T16:04:57.768 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T16:04:57.805 INFO:teuthology.task.internal:Removing archive directory... 2024-05-21T16:04:57.806 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T16:04:57.808 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T16:04:57.810 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T16:04:57.856 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-21T16:04:57.869 INFO:teuthology.task.internal:Not uploading archives. 2024-05-21T16:04:57.869 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-21T16:04:57.880 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-21T16:04:57.881 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T16:04:57.883 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T16:04:57.885 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T16:04:57.897 INFO:teuthology.orchestra.run.smithi038.stdout: 265501 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 16:04 /home/ubuntu/cephtest 2024-05-21T16:04:57.898 INFO:teuthology.orchestra.run.smithi045.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 16:04 /home/ubuntu/cephtest 2024-05-21T16:04:57.911 INFO:teuthology.orchestra.run.smithi053.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 16:04 /home/ubuntu/cephtest 2024-05-21T16:04:57.924 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-21T16:04:57.936 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-21T16:04:57.963 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} duration: 2898.1233174800873 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-21T16:04:57.963 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-21T16:04:58.058 INFO:teuthology.run:pass